DeepSeek V3.1 Terminus

Exact model and requirements

Artifact:

Unsloth (named GGUF derivative); DeepSeek AI (native Terminus checkpoint) / DeepSeek-V3.1-Terminus Q4_K_M, complete nine-file main-model composition · checkpoint unsloth/DeepSeek-V3.1-Terminus-GGUF

Publisher revision:

fe48342e95b8b3ca863189919605651fc2e88f4e

Artifact digest:

No published whole-composition digest; all nine individual primary LFS SHA 256 values are verified.

Download size:

377.56 GiB

Memory requirement (unknown):

Unknown — No measured full CPU process peak for all quantized weights, MLA cache, scratch, loading, runtime and OS.

System RAM / disk:

Unknown / Unknown

Placement:

single device. Nine disk splits form one complete model on one CPU host; no distributed/tensor/pipeline sharding. One CPU host, zero GPU layers and no accelerator offload.

KV cache:

Explicit F 16 K/V cache flags; native MLA graph determines compressed or expanded layout. Exact cache/state bytes and process peak remain unmeasured.

Workload:

8,192 token context; Unknown; Unknown; concurrency 1; batch 1; cache unknown; repetitions Unknown.

Tokenizer:

Exact Terminus GGUF tokenizer and original embedded native Terminus Jinja template · fe48342e95b8b3ca863189919605651fc2e88f4e; native template 19510d6dc61f79dbd925bd51ee8a9081c509a4b6

Sampling / reasoning:

Exact Terminus generation configuration: temperature 0.6 and top_p 0.95. Top_k 0, min_p 0, repeat_penalty 1 and presence/frequency penalties 0 are neutral reader choices. True native EOS, not a closing-think stop word or exhausted cap, is required. Exact native thinking boolean selects both modes through the native template. First thinking answer must close reasoning and produce final text. A non-thinking follow-up reuses history; native template removal of prior reasoning is checked by identical rendering with final-only history. No V3.2 encoder or DSA workflow is substituted.

Runtime prerequisites:

Ubuntu24.04 x86_64; See prerequisites above.

Official artifact · License

The exact Terminus model card explicitly licenses repository and weights under MIT; the full native LICENSE requires retaining copyright and permission notices in copies or substantial portions and disclaims warranties. The named Unsloth derivative declares MIT and links this exact Terminus base repository. Native and derivative pins are independent; no exact quantizer input base commit or legal clearance is inferred. No terms are accepted for the reader.

• DeepSeek-V3.1-Terminus-Q4_K_M-00001-of-00009.gguf · 45.14 GiB · revision fe48342e95b8b3ca863189919605651fc2e88f4e
• DeepSeek-V3.1-Terminus-Q4_K_M-00002-of-00009.gguf · 46.15 GiB · revision fe48342e95b8b3ca863189919605651fc2e88f4e
• DeepSeek-V3.1-Terminus-Q4_K_M-00003-of-00009.gguf · 46.25 GiB · revision fe48342e95b8b3ca863189919605651fc2e88f4e
• DeepSeek-V3.1-Terminus-Q4_K_M-00004-of-00009.gguf · 45.06 GiB · revision fe48342e95b8b3ca863189919605651fc2e88f4e
• DeepSeek-V3.1-Terminus-Q4_K_M-00005-of-00009.gguf · 46.14 GiB · revision fe48342e95b8b3ca863189919605651fc2e88f4e
• DeepSeek-V3.1-Terminus-Q4_K_M-00006-of-00009.gguf · 45.06 GiB · revision fe48342e95b8b3ca863189919605651fc2e88f4e
• DeepSeek-V3.1-Terminus-Q4_K_M-00007-of-00009.gguf · 46.15 GiB · revision fe48342e95b8b3ca863189919605651fc2e88f4e
• DeepSeek-V3.1-Terminus-Q4_K_M-00008-of-00009.gguf · 43.81 GiB · revision fe48342e95b8b3ca863189919605651fc2e88f4e
• DeepSeek-V3.1-Terminus-Q4_K_M-00009-of-00009.gguf · 13.8 GiB · revision fe48342e95b8b3ca863189919605651fc2e88f4e
• tokenizer_config.json (exact embedded native template source) · 0 GiB · revision 19510d6dc61f79dbd925bd51ee8a9081c509a4b6

• All nine exact language files total 405,398,012,864 bytes, already exceeding 256 GiB. Active 37B computation is not a resident-weight allowance.
• Native family branding is 671B main / 37B active. Exact native API metadata totals 684,531,386,000 parameters across mixed types and includes one additional NextN layer; this is distinct from main-model branding. Derivative optional MTP composition is uninspected; the full selected main composition is required.
• Native DEEPSEEK2 MLA/MoE has 61 main layers, three leading dense layers, 256 routed experts/eight selected and one shared expert. No dense-only KV formula or DSA indexer estimate is applied.
• The 512 GiB/600 GiB allocation must be actually available with compatible DIMMs/BIOS/OS. Loading, scratch, cache, page cache and OS add to serialized bytes. Peak, fit, speed and quality remain unknown.

Install the runtime

1. 1. Build the exact full CPU runtime

   Build exact pinned official source on this CPU. The native DeepseekV3ForCausalLM conversion maps to DEEPSEEK2; its 61-layer graph preserves MLA, dense-leading/routed/shared experts and router normalization/scaling. Actual build, fit and throughput remain unexecuted.

   CommandCopy command

   set -e
   sudo apt-get update
   sudo apt-get install -y build-essential cmake git curl ca-certificates openssl python3
   mkdir -p "$HOME/deepseek-v31-terminus-hosting/models"
   cd "$HOME/deepseek-v31-terminus-hosting"
   git clone https://github.com/ggml-org/llama.cpp.git
   git -C llama.cpp checkout --detach 620d2225cc436ab0b001e3200e590e3b286ef9ab
   cmake -S llama.cpp -B llama.cpp/build -DCMAKE_BUILD_TYPE=Release -DGGML_NATIVE=ON -DGGML_METAL=OFF
   cmake --build llama.cpp/build --config Release -j 64
   ./llama.cpp/build/bin/llama-server --version

   Expected result: The exact source builds and version is recorded. Stop on any failure. No author build, model load or inference result is preclaimed.

Download the model

1. 1. Download and verify all nine Terminus files and native template

   Keep exact split filenames together; the first shard discovers the set. Check every file SHA before use. Extract the complete original native template verbatim from the full hash-verified tokenizer configuration, then check its derived hash. No conversion or selective expert download is performed.

   CommandCopy command

   set -e
   cd "$HOME/deepseek-v31-terminus-hosting"
   curl --fail --location --output models/DeepSeek-V3.1-Terminus-Q4_K_M-00001-of-00009.gguf 'https://huggingface.co/unsloth/DeepSeek-V3.1-Terminus-GGUF/resolve/fe48342e95b8b3ca863189919605651fc2e88f4e/Q4_K_M/DeepSeek-V3.1-Terminus-Q4_K_M-00001-of-00009.gguf'
   curl --fail --location --output models/DeepSeek-V3.1-Terminus-Q4_K_M-00002-of-00009.gguf 'https://huggingface.co/unsloth/DeepSeek-V3.1-Terminus-GGUF/resolve/fe48342e95b8b3ca863189919605651fc2e88f4e/Q4_K_M/DeepSeek-V3.1-Terminus-Q4_K_M-00002-of-00009.gguf'
   curl --fail --location --output models/DeepSeek-V3.1-Terminus-Q4_K_M-00003-of-00009.gguf 'https://huggingface.co/unsloth/DeepSeek-V3.1-Terminus-GGUF/resolve/fe48342e95b8b3ca863189919605651fc2e88f4e/Q4_K_M/DeepSeek-V3.1-Terminus-Q4_K_M-00003-of-00009.gguf'
   curl --fail --location --output models/DeepSeek-V3.1-Terminus-Q4_K_M-00004-of-00009.gguf 'https://huggingface.co/unsloth/DeepSeek-V3.1-Terminus-GGUF/resolve/fe48342e95b8b3ca863189919605651fc2e88f4e/Q4_K_M/DeepSeek-V3.1-Terminus-Q4_K_M-00004-of-00009.gguf'
   curl --fail --location --output models/DeepSeek-V3.1-Terminus-Q4_K_M-00005-of-00009.gguf 'https://huggingface.co/unsloth/DeepSeek-V3.1-Terminus-GGUF/resolve/fe48342e95b8b3ca863189919605651fc2e88f4e/Q4_K_M/DeepSeek-V3.1-Terminus-Q4_K_M-00005-of-00009.gguf'
   curl --fail --location --output models/DeepSeek-V3.1-Terminus-Q4_K_M-00006-of-00009.gguf 'https://huggingface.co/unsloth/DeepSeek-V3.1-Terminus-GGUF/resolve/fe48342e95b8b3ca863189919605651fc2e88f4e/Q4_K_M/DeepSeek-V3.1-Terminus-Q4_K_M-00006-of-00009.gguf'
   curl --fail --location --output models/DeepSeek-V3.1-Terminus-Q4_K_M-00007-of-00009.gguf 'https://huggingface.co/unsloth/DeepSeek-V3.1-Terminus-GGUF/resolve/fe48342e95b8b3ca863189919605651fc2e88f4e/Q4_K_M/DeepSeek-V3.1-Terminus-Q4_K_M-00007-of-00009.gguf'
   curl --fail --location --output models/DeepSeek-V3.1-Terminus-Q4_K_M-00008-of-00009.gguf 'https://huggingface.co/unsloth/DeepSeek-V3.1-Terminus-GGUF/resolve/fe48342e95b8b3ca863189919605651fc2e88f4e/Q4_K_M/DeepSeek-V3.1-Terminus-Q4_K_M-00008-of-00009.gguf'
   curl --fail --location --output models/DeepSeek-V3.1-Terminus-Q4_K_M-00009-of-00009.gguf 'https://huggingface.co/unsloth/DeepSeek-V3.1-Terminus-GGUF/resolve/fe48342e95b8b3ca863189919605651fc2e88f4e/Q4_K_M/DeepSeek-V3.1-Terminus-Q4_K_M-00009-of-00009.gguf'
   curl --fail --location --output models/tokenizer_config.json 'https://huggingface.co/deepseek-ai/DeepSeek-V3.1-Terminus/resolve/19510d6dc61f79dbd925bd51ee8a9081c509a4b6/tokenizer_config.json'
   sha256sum --check <<'SHA256'
   501db013c7e0eac73ea339825970a058faf1311e92f30e402cae4b3f4e8bf930  models/DeepSeek-V3.1-Terminus-Q4_K_M-00001-of-00009.gguf
   5e037a00cfb5ae7885a0b78ec602e08914d1a7c18a3f15cf1bdaa3dd1b3ddedc  models/DeepSeek-V3.1-Terminus-Q4_K_M-00002-of-00009.gguf
   d51ea62e07e7d3b789b4cfb262d0bc06aea364b29afa987ce10011de2dffaa14  models/DeepSeek-V3.1-Terminus-Q4_K_M-00003-of-00009.gguf
   381be0f5c52cdaaddd117bab701741aa93d9172bafca8295dbc02d4d3ff503f5  models/DeepSeek-V3.1-Terminus-Q4_K_M-00004-of-00009.gguf
   6902679264fe29639a1e4e3b830919984d92e69883d496e497ec638c280e2887  models/DeepSeek-V3.1-Terminus-Q4_K_M-00005-of-00009.gguf
   daf8e418c4339961946dfa31b73dc335f420ab07c42bbb0d869287347dc45925  models/DeepSeek-V3.1-Terminus-Q4_K_M-00006-of-00009.gguf
   871d002c96e1c66c4606171497c1de1faca7f1cc725092d765b60f5723d64212  models/DeepSeek-V3.1-Terminus-Q4_K_M-00007-of-00009.gguf
   c557feca48296bd57b059ee132bd2f22f8b987e3b39eb225c62fc50f819b4da2  models/DeepSeek-V3.1-Terminus-Q4_K_M-00008-of-00009.gguf
   9cf3ac7231b2f71ebf04315fbfb0c2f677c2c644c9e4629c965f1c5bd4d18430  models/DeepSeek-V3.1-Terminus-Q4_K_M-00009-of-00009.gguf
   f3d66f405ce0315c754a3cc041f13be863fefcd62c240068098efdbd1924f8b5  models/tokenizer_config.json
   SHA256
   python3 - <<'PYTEMPLATE'
   import pathlib,json,hashlib
   b=json.loads(pathlib.Path('models/tokenizer_config.json').read_text())['chat_template'].encode('utf-8')
   assert hashlib.sha256(b).hexdigest()=='45690185067ea25c073bff2d3e425a45ee2564d2bbf6cdf7d885e139d460b8e0'
   pathlib.Path('models/native-chat-template.jinja').write_bytes(b)
   PYTEMPLATE

   Expected result: All ten downloads report OK and the exact derived template hash matches. No first shard or active-only substitute is accepted.

Configure

1. 1. Create a private service credential

   Generate a reader-owned local service password, preserving an existing key on repeat setup. Never print or share it; this is not a provider API credential.

   CommandCopy command

   set -e
   cd "$HOME/deepseek-v31-terminus-hosting"
   umask 077
   if [ ! -e .api-key ]; then openssl rand -hex 32 > .api-key; fi
   test -s .api-key
   chmod 600 .api-key

   Expected result: A nonempty mode-600 private .api-key exists; existing key preserved.

2. 2. Write the native hybrid diagnostic client

   This stdlib client asks the native server to render the exact pinned template, then tokenizes raw complete prompts with added special tokens disabled. It requires one BOS, correct native mode prefixes, full context budget, actual EOS and strict closure/final text; native history reasoning removal is checked separately.

   CommandCopy command

   set -e
   cd "$HOME/deepseek-v31-terminus-hosting"
   cat > hybrid-client.py <<'PYCLIENT'
   """Fixed native Terminus template/raw completion diagnostic; stdlib only."""
   import copy,json,pathlib,urllib.request,urllib.error
   BASE='http://127.0.0.1:8080';CONTEXT=8192;OUTPUT=2048
   BOS='<｜begin▁of▁sentence｜>';EOS='<｜end▁of▁sentence｜>'
   def positive(x):return type(x) is int and x>0
   def messages_ok(messages):
    if not isinstance(messages,list) or not 1<=len(messages)<=8:raise ValueError('Bounded text history required')
    for i,m in enumerate(messages):
     if not isinstance(m,dict) or set(m)!={'role','content'}:raise ValueError('Only role/content accepted')
     role,text=m['role'],m['content']
     if role not in ('system','user','assistant') or (role=='system' and i!=0):raise ValueError('Role order')
     if not isinstance(text,str) or not text.strip() or len(text.encode())>12000:raise ValueError('Invalid text')
     reserved=(BOS,EOS,'<｜User｜>','<｜Assistant｜>','<｜tool')
     if any(x in text for x in reserved) or (role!='assistant' and any(x in text for x in ('<think>','</think>'))):raise ValueError('Reserved native markers')
    roles=[m['role'] for m in messages if m['role']!='system']
    if not roles or roles!=['user' if i%2==0 else 'assistant' for i in range(len(roles))] or roles[-1]!='user':raise ValueError('Alternating history ending in user required')
   def render(call,messages,thinking):
    if type(thinking) is not bool:raise ValueError('Native thinking boolean required')
    messages_ok(messages)
    p=call('/apply-template',{'messages':messages,'add_generation_prompt':True,'chat_template_kwargs':{'thinking':thinking}}).get('prompt')
    end='<think>' if thinking else '</think>'
    if not isinstance(p,str) or not p.startswith(BOS) or p.count(BOS)!=1 or not p.endswith(end):raise ValueError('Native BOS or mode prefix mismatch')
    return p
   def parse_output(result,count,thinking):
    if result.get('stop_type')!='eos' or result.get('truncated') is not False:raise ValueError('Actual EOS without truncation required')
    n=result.get('tokens_predicted');s=result.get('content')
    if not positive(n) or n>OUTPUT or result.get('tokens_evaluated')!=count:raise ValueError('Usage mismatch or cap')
    if not isinstance(s,str) or not s.strip():raise ValueError('Empty output')
    if EOS in s:
     if s.count(EOS)!=1 or not s.endswith(EOS):raise ValueError('Internal or repeated EOS')
     s=s[:-len(EOS)]
    if any(t in s for t in (BOS,'<｜User｜>','<｜Assistant｜>','<｜tool','<think>')):raise ValueError('Unexpected native control or tool output')
    if thinking:
     if s.count('</think>')!=1:raise ValueError('Exactly one thinking closure required')
     reasoning,final=s.split('</think>')
     if not reasoning.strip() or not final.strip():raise ValueError('Nonempty reasoning and final required')
     history='<think>'+s
    else:
     if '</think>' in s:raise ValueError('Unexpected extra thinking closure')
     reasoning,final,history='',s,s
    if not final.strip():raise ValueError('Empty final')
    return {'reasoning_content':reasoning,'content':final,'historyContent':history}
   def complete(call,messages,thinking):
    p=render(call,messages,thinking)
    ids=call('/tokenize',{'content':p,'add_special':False,'parse_special':True}).get('tokens')
    if not isinstance(ids,list) or not ids or any(type(x) is not int or x<0 for x in ids) or ids[0]!=0 or ids.count(0)!=1:raise ValueError('Invalid IDs or duplicate BOS')
    if len(ids)+OUTPUT>CONTEXT:raise ValueError('Full input plus reasoning/final reserve exceeds context')
    body={'prompt':ids,'n_predict':OUTPUT,'temperature':0.6,'top_p':0.95,'top_k':0,'min_p':0,'repeat_penalty':1,'presence_penalty':0,'frequency_penalty':0,'cache_prompt':False,'stream':False,'return_tokens':True,'stop':[]}
    result=call('/completion',body);parsed=parse_output(result,len(ids),thinking)
    return {'thinking':thinking,'inputTokens':len(ids),'outputCap':OUTPUT,'parsed':parsed,'nativeResponse':result}
   def http(key):
    def call(route,body=None):
     data=None if body is None else json.dumps(body).encode()
     req=urllib.request.Request(BASE+route,data=data,headers={'Authorization':'Bearer '+key,'Content-Type':'application/json'})
     with urllib.request.urlopen(req,timeout=7200) as r:
      if r.status!=200:raise ValueError('HTTP failure')
      raw=r.read(2000001)
     if len(raw)>2000000:raise ValueError('Response bound')
     d=json.loads(raw)
     if not isinstance(d,dict):raise ValueError('Object response required')
     return d
    return call
   def main():
    key=pathlib.Path('.api-key').read_text().strip()
    if len(key)!=64 or any(x not in '0123456789abcdef' for x in key):raise ValueError('Restore service key')
    call=http(key);call('/health')
    req=urllib.request.Request(BASE+'/completion',data=b'{}',headers={'Content-Type':'application/json'})
    try:
     with urllib.request.urlopen(req,timeout=30):raise ValueError('Unauthorized completion accepted')
    except urllib.error.HTTPError as e:
     if e.code!=401:raise
    history=[{'role':'user','content':'The community library opens weekdays 9am to 6pm, Saturday 10am to 4pm and is closed Sunday. Explain briefly how many hours it is open on Saturday.'}]
    first=complete(call,history,True);pathlib.Path('thinking-response.json').write_text(json.dumps(first,indent=2)+'\n')
    history += [{'role':'assistant','content':first['parsed']['historyContent']},{'role':'user','content':'From the same supplied schedule, state the weekday opening time and whether it is open Sunday. Reply briefly.'}]
    stripped=copy.deepcopy(history);stripped[1]['content']=first['parsed']['content']
    if render(call,history,False)!=render(call,stripped,False):raise ValueError('Native template did not discard past reasoning as prescribed')
    second=complete(call,history,False);pathlib.Path('chat-response.json').write_text(json.dumps(second,indent=2)+'\n')
    print(json.dumps({'thinking':first,'chat':second},ensure_ascii=False))
    print('Manually require Saturday six hours, weekday9am and Sundayclosed; protocol shape is not a quality or speed measurement.')
   if __name__=='__main__':main()
   PYCLIENT

   Expected result: Client saved; no model code or native encoder is imported by this client.

Start the model

1. 1. Start the complete Terminus CPU hybrid service

   Run all main model parts on one CPU host, 16 inference threads, one 8192-position sequence and 2048 response cap. The native server renders the exact complete Jinja template; each request sets its real thinking boolean. Native BOS/EOS framing and MLA are preserved. No speculative MTP is requested; flash-attention fusion is off for this CPU plan.

   CommandCopy command

   set -e
   cd "$HOME/deepseek-v31-terminus-hosting"
   export TERMINUS_SERVICE_KEY="$(cat .api-key)"
   ./llama.cpp/build/bin/llama-server -m models/DeepSeek-V3.1-Terminus-Q4_K_M-00001-of-00009.gguf --alias deepseek-v31-terminus --host 127.0.0.1 --port 8080 --api-key "$TERMINUS_SERVICE_KEY" --device none -ngl 0 -t 16 -c 8192 -np 1 -n 2048 --no-context-shift --flash-attn off --cache-type-k f16 --cache-type-v f16 -b 256 -ub 256 --jinja --chat-template-file models/native-chat-template.jinja --skip-chat-parsing --reasoning-format none --special --temp 0.6 --top-p 0.95 --top-k 0 --min-p 0 --repeat-penalty 1 --presence-penalty 0 --frequency-penalty 0

   Expected result: Every split loads and loopback readiness succeeds. Stop on missing tensors, architecture, cache, template, allocation or tokenizer failures. No successful startup or speed is preclaimed.

Verify the setup

1. 1. Check native thinking, ordinary follow-up and history

   Use a second shell inside the host. The fixed client checks health/401, uses exact native template rendering and raw token counts before generation, requires real EOS/no truncation/valid usage and checks native reasoning removal from history. Manually require Saturday six hours, weekday opening 9am and Sunday closure; protocol shape does not prove accuracy.

   CommandCopy command

   set -e
   cd "$HOME/deepseek-v31-terminus-hosting"
   python3 hybrid-client.py

   Expected result: Both modes return complete nonempty final text; thinking has exactly one closing marker and nonempty reasoning. Source-prescribed history removal renders identically to final-only history. Wrong facts, length exhaustion, malformed reasoning or any budget failure is a failed check.

2. 2. Reach the private service through SSH

   From a trusted client terminal use an existing SSH key and the exact VM host. Keep the tunnel open; the supplied diagnostic runs inside the VM. A separate local client must securely use the matching service key. Do not expose the native inference port publicly.

   CommandCopy command

   ssh -i /path/to/existing-key.pem -N -L 8080:127.0.0.1:8080 ubuntu@YOUR_EXISTING_INSTANCE_HOST

   Expected result: Client loopback forwards to this VM loopback. Closing the tunnel does not stop server compute or storage billing.

Performance evidence

No compatible measurement available. Documented support and estimated memory do not establish response speed.

Limits and troubleshooting

• Source-qualified documentation only. No weights/model binaries, native model/template execution, build, installation, inference, fit, speed or quality measurement was performed.
• All nine Terminus files total 405,398,012,864 bytes. All individual hashes differ from the separate V3.2 files; similar part sizes do not imply model identity or byte equivalence.
• The 61-main-layer native model has MLA and routed/shared experts. V3.2 DSA/indexer architecture and its Python encoder are not transferred.
• Native config stores 163840 maximum positions and tokenizer config 131072; publisher-linked family documentation describes 128K. This bounded guide selects 8192 total positions and 2048 for complete reasoning/final output, not full-capacity or measured reasoning quality.
• The native embedded Jinja uses thinking true/false and defaults false; Native BOS 0 and EOS 1 are verified here; native prompt formats require their own checkpoint-specific qualification. Native generation configuration selects temperature 0.6/top_p 0.95. No borrowed enable_thinking toggle or generic template is used.
• Terminus explicitly retains a known native FP8 o_proj scale-format issue; no source proves this derivative fixes it. The exact named Q4 artifact is independently pinned and output must pass the factual checks. Native router correction bias must remain F32; the pinned converter preserves 1D tensors as F32, but derivative body tensor types were not inspected and no native byte-equivalence is claimed.
• Optional NextN/MTP tensor inclusion is uninspected in derivative binary bodies; all selected published parts/full main model remain required. No speculative speedup is claimed.
• The complete template drops past reasoning before a later turn. The client checks full-history rendering equals final-only rendering, then validates both modes and true EOS. It does not stop at the closing-think token or treat cap exhaustion as success.
• Tools/search-agent workflows, native FP8/BF 16, GPUs/distribution, larger contexts and other quantizations remain open. Updated Terminus search-agent template is not substituted with ordinary V3.1 search prompting. Family required totals remain null.
• Trusted loopback and a private key are not a public TLS/multiuser service. Actual cache/scratch/load peaks, fit, latency, throughput, quality and task spend remain unknown.

Shard/hash or architecture/template failure

Stop. Check all exact Terminus files, native template derivation and pinned DEEPSEEK2 runtime. Do not substitute later-family encoders, different checkpoints or partial expert sets.

Out of memory or budget failure

Stop and retain actual logs/counts. Verify 512 GiB actually available or separately review a changed configuration. Active parameters do not establish fit; no silent prompt/reasoning truncation.

EOS, reasoning or history check fails

Treat verification as failed. Inspect true native stop type, raw usage, exact native template/thinking boolean and prefix. Require complete final text; do not add a closing-think stopword to conceal truncation.

HTTP401 or connection refused

Inspect readiness, intended VM/tunnel and exact local service key. Keep public inference port closed.

Stop the model

1. 1. Stop the native service

   Press Ctrl+C in the foreground server shell; close any SSH tunnel with Ctrl+C and unset the key in every shell. Confirm the intended listener is gone. Cloud billing requires the separate resource lifecycle step.

   CommandCopy command

   unset TERMINUS_SERVICE_KEY
   if curl --silent --max-time 3 http://127.0.0.1:8080/health >/dev/null; then echo "Endpoint still responds: stop intended process"; exit 1; else echo "Endpoint no longer responds"; fi

   Expected result: The intended loopback service no longer responds; cloud compute/storage may still bill.

Uninstall

1. 1. Remove dedicated walkthrough files

   Back up wanted results and logs and confirm the dedicated path before removal. Shared system build packages remain installed.

   CommandCopy command

   rm -r -- "$HOME/deepseek-v31-terminus-hosting"

   Expected result: Dedicated model splits, native template, client, key, results and build files are removed; shared system packages remain.

Shut down cloud resources

1. 1. End cloud resource billing

   In EC2 us-east-1 stop this exact VM for later reuse or terminate it. Review root-volume Delete on termination and remove unwanted detached volumes, snapshots and reserved/Elastic/public IP resources. Stopped-instance storage and retained resources may still bill. Check Bills/Cost Explorer for final usage.

   Expected result: The exact instance is stopped or terminated as intended, and unwanted retained billed resources are removed. Stopping the native service alone does not end compute billing.

Related guides

• https://opentools.ai/llms/deepseek-v31-terminus

Exact model and requirements

Artifact:

Unsloth (named GGUF derivative); DeepSeek AI (native Terminus checkpoint) / DeepSeek-V3.1-Terminus Q4_K_M, complete nine-file main-model composition · checkpoint unsloth/DeepSeek-V3.1-Terminus-GGUF

Publisher revision:

fe48342e95b8b3ca863189919605651fc2e88f4e

Artifact digest:

No published whole-composition digest; all nine individual primary LFS SHA 256 values are verified.

Download size:

377.56 GiB

Memory requirement (unknown):

Unknown — No measured full CPU process peak for all quantized weights, MLA cache, scratch, loading, runtime and OS.

System RAM / disk:

Unknown / Unknown

Placement:

single device. Nine disk splits form one complete model on one CPU host; no distributed/tensor/pipeline sharding. One CPU host, zero GPU layers and no accelerator offload.

KV cache:

Explicit F 16 K/V cache flags; native MLA graph determines compressed or expanded layout. Exact cache/state bytes and process peak remain unmeasured.

Workload:

8,192 token context; Unknown; Unknown; concurrency 1; batch 1; cache unknown; repetitions Unknown.

Tokenizer:

Exact Terminus GGUF tokenizer and original embedded native Terminus Jinja template · fe48342e95b8b3ca863189919605651fc2e88f4e; native template 19510d6dc61f79dbd925bd51ee8a9081c509a4b6

Sampling / reasoning:

Exact Terminus generation configuration: temperature 0.6 and top_p 0.95. Top_k 0, min_p 0, repeat_penalty 1 and presence/frequency penalties 0 are neutral reader choices. True native EOS, not a closing-think stop word or exhausted cap, is required. Exact native thinking boolean selects both modes through the native template. First thinking answer must close reasoning and produce final text. A non-thinking follow-up reuses history; native template removal of prior reasoning is checked by identical rendering with final-only history. No V3.2 encoder or DSA workflow is substituted.

Runtime prerequisites:

Ubuntu24.04 x86_64; See prerequisites above.

Official artifact · License

The exact Terminus model card explicitly licenses repository and weights under MIT; the full native LICENSE requires retaining copyright and permission notices in copies or substantial portions and disclaims warranties. The named Unsloth derivative declares MIT and links this exact Terminus base repository. Native and derivative pins are independent; no exact quantizer input base commit or legal clearance is inferred. No terms are accepted for the reader.

• DeepSeek-V3.1-Terminus-Q4_K_M-00001-of-00009.gguf · 45.14 GiB · revision fe48342e95b8b3ca863189919605651fc2e88f4e
• DeepSeek-V3.1-Terminus-Q4_K_M-00002-of-00009.gguf · 46.15 GiB · revision fe48342e95b8b3ca863189919605651fc2e88f4e
• DeepSeek-V3.1-Terminus-Q4_K_M-00003-of-00009.gguf · 46.25 GiB · revision fe48342e95b8b3ca863189919605651fc2e88f4e
• DeepSeek-V3.1-Terminus-Q4_K_M-00004-of-00009.gguf · 45.06 GiB · revision fe48342e95b8b3ca863189919605651fc2e88f4e
• DeepSeek-V3.1-Terminus-Q4_K_M-00005-of-00009.gguf · 46.14 GiB · revision fe48342e95b8b3ca863189919605651fc2e88f4e
• DeepSeek-V3.1-Terminus-Q4_K_M-00006-of-00009.gguf · 45.06 GiB · revision fe48342e95b8b3ca863189919605651fc2e88f4e
• DeepSeek-V3.1-Terminus-Q4_K_M-00007-of-00009.gguf · 46.15 GiB · revision fe48342e95b8b3ca863189919605651fc2e88f4e
• DeepSeek-V3.1-Terminus-Q4_K_M-00008-of-00009.gguf · 43.81 GiB · revision fe48342e95b8b3ca863189919605651fc2e88f4e
• DeepSeek-V3.1-Terminus-Q4_K_M-00009-of-00009.gguf · 13.8 GiB · revision fe48342e95b8b3ca863189919605651fc2e88f4e
• tokenizer_config.json (exact embedded native template source) · 0 GiB · revision 19510d6dc61f79dbd925bd51ee8a9081c509a4b6

• All nine exact language files total 405,398,012,864 bytes, already exceeding 256 GiB. Active 37B computation is not a resident-weight allowance.
• Native family branding is 671B main / 37B active. Exact native API metadata totals 684,531,386,000 parameters across mixed types and includes one additional NextN layer; this is distinct from main-model branding. Derivative optional MTP composition is uninspected; the full selected main composition is required.
• Native DEEPSEEK2 MLA/MoE has 61 main layers, three leading dense layers, 256 routed experts/eight selected and one shared expert. No dense-only KV formula or DSA indexer estimate is applied.
• The 512 GiB/600 GiB allocation must be actually available with compatible DIMMs/BIOS/OS. Loading, scratch, cache, page cache and OS add to serialized bytes. Peak, fit, speed and quality remain unknown.

Install the runtime

1. 1. Build the exact full CPU runtime

   Build exact pinned official source on this CPU. The native DeepseekV3ForCausalLM conversion maps to DEEPSEEK2; its 61-layer graph preserves MLA, dense-leading/routed/shared experts and router normalization/scaling. Actual build, fit and throughput remain unexecuted.

   CommandCopy command

   set -e
   sudo apt-get update
   sudo apt-get install -y build-essential cmake git curl ca-certificates openssl python3
   mkdir -p "$HOME/deepseek-v31-terminus-hosting/models"
   cd "$HOME/deepseek-v31-terminus-hosting"
   git clone https://github.com/ggml-org/llama.cpp.git
   git -C llama.cpp checkout --detach 620d2225cc436ab0b001e3200e590e3b286ef9ab
   cmake -S llama.cpp -B llama.cpp/build -DCMAKE_BUILD_TYPE=Release -DGGML_NATIVE=ON -DGGML_METAL=OFF
   cmake --build llama.cpp/build --config Release -j 16
   ./llama.cpp/build/bin/llama-server --version

   Expected result: The exact source builds and version is recorded. Stop on any failure. No author build, model load or inference result is preclaimed.

Download the model

1. 1. Download and verify all nine Terminus files and native template

   Keep exact split filenames together; the first shard discovers the set. Check every file SHA before use. Extract the complete original native template verbatim from the full hash-verified tokenizer configuration, then check its derived hash. No conversion or selective expert download is performed.

   CommandCopy command

   set -e
   cd "$HOME/deepseek-v31-terminus-hosting"
   curl --fail --location --output models/DeepSeek-V3.1-Terminus-Q4_K_M-00001-of-00009.gguf 'https://huggingface.co/unsloth/DeepSeek-V3.1-Terminus-GGUF/resolve/fe48342e95b8b3ca863189919605651fc2e88f4e/Q4_K_M/DeepSeek-V3.1-Terminus-Q4_K_M-00001-of-00009.gguf'
   curl --fail --location --output models/DeepSeek-V3.1-Terminus-Q4_K_M-00002-of-00009.gguf 'https://huggingface.co/unsloth/DeepSeek-V3.1-Terminus-GGUF/resolve/fe48342e95b8b3ca863189919605651fc2e88f4e/Q4_K_M/DeepSeek-V3.1-Terminus-Q4_K_M-00002-of-00009.gguf'
   curl --fail --location --output models/DeepSeek-V3.1-Terminus-Q4_K_M-00003-of-00009.gguf 'https://huggingface.co/unsloth/DeepSeek-V3.1-Terminus-GGUF/resolve/fe48342e95b8b3ca863189919605651fc2e88f4e/Q4_K_M/DeepSeek-V3.1-Terminus-Q4_K_M-00003-of-00009.gguf'
   curl --fail --location --output models/DeepSeek-V3.1-Terminus-Q4_K_M-00004-of-00009.gguf 'https://huggingface.co/unsloth/DeepSeek-V3.1-Terminus-GGUF/resolve/fe48342e95b8b3ca863189919605651fc2e88f4e/Q4_K_M/DeepSeek-V3.1-Terminus-Q4_K_M-00004-of-00009.gguf'
   curl --fail --location --output models/DeepSeek-V3.1-Terminus-Q4_K_M-00005-of-00009.gguf 'https://huggingface.co/unsloth/DeepSeek-V3.1-Terminus-GGUF/resolve/fe48342e95b8b3ca863189919605651fc2e88f4e/Q4_K_M/DeepSeek-V3.1-Terminus-Q4_K_M-00005-of-00009.gguf'
   curl --fail --location --output models/DeepSeek-V3.1-Terminus-Q4_K_M-00006-of-00009.gguf 'https://huggingface.co/unsloth/DeepSeek-V3.1-Terminus-GGUF/resolve/fe48342e95b8b3ca863189919605651fc2e88f4e/Q4_K_M/DeepSeek-V3.1-Terminus-Q4_K_M-00006-of-00009.gguf'
   curl --fail --location --output models/DeepSeek-V3.1-Terminus-Q4_K_M-00007-of-00009.gguf 'https://huggingface.co/unsloth/DeepSeek-V3.1-Terminus-GGUF/resolve/fe48342e95b8b3ca863189919605651fc2e88f4e/Q4_K_M/DeepSeek-V3.1-Terminus-Q4_K_M-00007-of-00009.gguf'
   curl --fail --location --output models/DeepSeek-V3.1-Terminus-Q4_K_M-00008-of-00009.gguf 'https://huggingface.co/unsloth/DeepSeek-V3.1-Terminus-GGUF/resolve/fe48342e95b8b3ca863189919605651fc2e88f4e/Q4_K_M/DeepSeek-V3.1-Terminus-Q4_K_M-00008-of-00009.gguf'
   curl --fail --location --output models/DeepSeek-V3.1-Terminus-Q4_K_M-00009-of-00009.gguf 'https://huggingface.co/unsloth/DeepSeek-V3.1-Terminus-GGUF/resolve/fe48342e95b8b3ca863189919605651fc2e88f4e/Q4_K_M/DeepSeek-V3.1-Terminus-Q4_K_M-00009-of-00009.gguf'
   curl --fail --location --output models/tokenizer_config.json 'https://huggingface.co/deepseek-ai/DeepSeek-V3.1-Terminus/resolve/19510d6dc61f79dbd925bd51ee8a9081c509a4b6/tokenizer_config.json'
   sha256sum --check <<'SHA256'
   501db013c7e0eac73ea339825970a058faf1311e92f30e402cae4b3f4e8bf930  models/DeepSeek-V3.1-Terminus-Q4_K_M-00001-of-00009.gguf
   5e037a00cfb5ae7885a0b78ec602e08914d1a7c18a3f15cf1bdaa3dd1b3ddedc  models/DeepSeek-V3.1-Terminus-Q4_K_M-00002-of-00009.gguf
   d51ea62e07e7d3b789b4cfb262d0bc06aea364b29afa987ce10011de2dffaa14  models/DeepSeek-V3.1-Terminus-Q4_K_M-00003-of-00009.gguf
   381be0f5c52cdaaddd117bab701741aa93d9172bafca8295dbc02d4d3ff503f5  models/DeepSeek-V3.1-Terminus-Q4_K_M-00004-of-00009.gguf
   6902679264fe29639a1e4e3b830919984d92e69883d496e497ec638c280e2887  models/DeepSeek-V3.1-Terminus-Q4_K_M-00005-of-00009.gguf
   daf8e418c4339961946dfa31b73dc335f420ab07c42bbb0d869287347dc45925  models/DeepSeek-V3.1-Terminus-Q4_K_M-00006-of-00009.gguf
   871d002c96e1c66c4606171497c1de1faca7f1cc725092d765b60f5723d64212  models/DeepSeek-V3.1-Terminus-Q4_K_M-00007-of-00009.gguf
   c557feca48296bd57b059ee132bd2f22f8b987e3b39eb225c62fc50f819b4da2  models/DeepSeek-V3.1-Terminus-Q4_K_M-00008-of-00009.gguf
   9cf3ac7231b2f71ebf04315fbfb0c2f677c2c644c9e4629c965f1c5bd4d18430  models/DeepSeek-V3.1-Terminus-Q4_K_M-00009-of-00009.gguf
   f3d66f405ce0315c754a3cc041f13be863fefcd62c240068098efdbd1924f8b5  models/tokenizer_config.json
   SHA256
   python3 - <<'PYTEMPLATE'
   import pathlib,json,hashlib
   b=json.loads(pathlib.Path('models/tokenizer_config.json').read_text())['chat_template'].encode('utf-8')
   assert hashlib.sha256(b).hexdigest()=='45690185067ea25c073bff2d3e425a45ee2564d2bbf6cdf7d885e139d460b8e0'
   pathlib.Path('models/native-chat-template.jinja').write_bytes(b)
   PYTEMPLATE

   Expected result: All ten downloads report OK and the exact derived template hash matches. No first shard or active-only substitute is accepted.

Configure

1. 1. Create a private service credential

   Generate a reader-owned local service password, preserving an existing key on repeat setup. Never print or share it; this is not a provider API credential.

   CommandCopy command

   set -e
   cd "$HOME/deepseek-v31-terminus-hosting"
   umask 077
   if [ ! -e .api-key ]; then openssl rand -hex 32 > .api-key; fi
   test -s .api-key
   chmod 600 .api-key

   Expected result: A nonempty mode-600 private .api-key exists; existing key preserved.

2. 2. Write the native hybrid diagnostic client

   This stdlib client asks the native server to render the exact pinned template, then tokenizes raw complete prompts with added special tokens disabled. It requires one BOS, correct native mode prefixes, full context budget, actual EOS and strict closure/final text; native history reasoning removal is checked separately.

   CommandCopy command

   set -e
   cd "$HOME/deepseek-v31-terminus-hosting"
   cat > hybrid-client.py <<'PYCLIENT'
   """Fixed native Terminus template/raw completion diagnostic; stdlib only."""
   import copy,json,pathlib,urllib.request,urllib.error
   BASE='http://127.0.0.1:8080';CONTEXT=8192;OUTPUT=2048
   BOS='<｜begin▁of▁sentence｜>';EOS='<｜end▁of▁sentence｜>'
   def positive(x):return type(x) is int and x>0
   def messages_ok(messages):
    if not isinstance(messages,list) or not 1<=len(messages)<=8:raise ValueError('Bounded text history required')
    for i,m in enumerate(messages):
     if not isinstance(m,dict) or set(m)!={'role','content'}:raise ValueError('Only role/content accepted')
     role,text=m['role'],m['content']
     if role not in ('system','user','assistant') or (role=='system' and i!=0):raise ValueError('Role order')
     if not isinstance(text,str) or not text.strip() or len(text.encode())>12000:raise ValueError('Invalid text')
     reserved=(BOS,EOS,'<｜User｜>','<｜Assistant｜>','<｜tool')
     if any(x in text for x in reserved) or (role!='assistant' and any(x in text for x in ('<think>','</think>'))):raise ValueError('Reserved native markers')
    roles=[m['role'] for m in messages if m['role']!='system']
    if not roles or roles!=['user' if i%2==0 else 'assistant' for i in range(len(roles))] or roles[-1]!='user':raise ValueError('Alternating history ending in user required')
   def render(call,messages,thinking):
    if type(thinking) is not bool:raise ValueError('Native thinking boolean required')
    messages_ok(messages)
    p=call('/apply-template',{'messages':messages,'add_generation_prompt':True,'chat_template_kwargs':{'thinking':thinking}}).get('prompt')
    end='<think>' if thinking else '</think>'
    if not isinstance(p,str) or not p.startswith(BOS) or p.count(BOS)!=1 or not p.endswith(end):raise ValueError('Native BOS or mode prefix mismatch')
    return p
   def parse_output(result,count,thinking):
    if result.get('stop_type')!='eos' or result.get('truncated') is not False:raise ValueError('Actual EOS without truncation required')
    n=result.get('tokens_predicted');s=result.get('content')
    if not positive(n) or n>OUTPUT or result.get('tokens_evaluated')!=count:raise ValueError('Usage mismatch or cap')
    if not isinstance(s,str) or not s.strip():raise ValueError('Empty output')
    if EOS in s:
     if s.count(EOS)!=1 or not s.endswith(EOS):raise ValueError('Internal or repeated EOS')
     s=s[:-len(EOS)]
    if any(t in s for t in (BOS,'<｜User｜>','<｜Assistant｜>','<｜tool','<think>')):raise ValueError('Unexpected native control or tool output')
    if thinking:
     if s.count('</think>')!=1:raise ValueError('Exactly one thinking closure required')
     reasoning,final=s.split('</think>')
     if not reasoning.strip() or not final.strip():raise ValueError('Nonempty reasoning and final required')
     history='<think>'+s
    else:
     if '</think>' in s:raise ValueError('Unexpected extra thinking closure')
     reasoning,final,history='',s,s
    if not final.strip():raise ValueError('Empty final')
    return {'reasoning_content':reasoning,'content':final,'historyContent':history}
   def complete(call,messages,thinking):
    p=render(call,messages,thinking)
    ids=call('/tokenize',{'content':p,'add_special':False,'parse_special':True}).get('tokens')
    if not isinstance(ids,list) or not ids or any(type(x) is not int or x<0 for x in ids) or ids[0]!=0 or ids.count(0)!=1:raise ValueError('Invalid IDs or duplicate BOS')
    if len(ids)+OUTPUT>CONTEXT:raise ValueError('Full input plus reasoning/final reserve exceeds context')
    body={'prompt':ids,'n_predict':OUTPUT,'temperature':0.6,'top_p':0.95,'top_k':0,'min_p':0,'repeat_penalty':1,'presence_penalty':0,'frequency_penalty':0,'cache_prompt':False,'stream':False,'return_tokens':True,'stop':[]}
    result=call('/completion',body);parsed=parse_output(result,len(ids),thinking)
    return {'thinking':thinking,'inputTokens':len(ids),'outputCap':OUTPUT,'parsed':parsed,'nativeResponse':result}
   def http(key):
    def call(route,body=None):
     data=None if body is None else json.dumps(body).encode()
     req=urllib.request.Request(BASE+route,data=data,headers={'Authorization':'Bearer '+key,'Content-Type':'application/json'})
     with urllib.request.urlopen(req,timeout=7200) as r:
      if r.status!=200:raise ValueError('HTTP failure')
      raw=r.read(2000001)
     if len(raw)>2000000:raise ValueError('Response bound')
     d=json.loads(raw)
     if not isinstance(d,dict):raise ValueError('Object response required')
     return d
    return call
   def main():
    key=pathlib.Path('.api-key').read_text().strip()
    if len(key)!=64 or any(x not in '0123456789abcdef' for x in key):raise ValueError('Restore service key')
    call=http(key);call('/health')
    req=urllib.request.Request(BASE+'/completion',data=b'{}',headers={'Content-Type':'application/json'})
    try:
     with urllib.request.urlopen(req,timeout=30):raise ValueError('Unauthorized completion accepted')
    except urllib.error.HTTPError as e:
     if e.code!=401:raise
    history=[{'role':'user','content':'The community library opens weekdays 9am to 6pm, Saturday 10am to 4pm and is closed Sunday. Explain briefly how many hours it is open on Saturday.'}]
    first=complete(call,history,True);pathlib.Path('thinking-response.json').write_text(json.dumps(first,indent=2)+'\n')
    history += [{'role':'assistant','content':first['parsed']['historyContent']},{'role':'user','content':'From the same supplied schedule, state the weekday opening time and whether it is open Sunday. Reply briefly.'}]
    stripped=copy.deepcopy(history);stripped[1]['content']=first['parsed']['content']
    if render(call,history,False)!=render(call,stripped,False):raise ValueError('Native template did not discard past reasoning as prescribed')
    second=complete(call,history,False);pathlib.Path('chat-response.json').write_text(json.dumps(second,indent=2)+'\n')
    print(json.dumps({'thinking':first,'chat':second},ensure_ascii=False))
    print('Manually require Saturday six hours, weekday9am and Sundayclosed; protocol shape is not a quality or speed measurement.')
   if __name__=='__main__':main()
   PYCLIENT

   Expected result: Client saved; no model code or native encoder is imported by this client.

Start the model

1. 1. Start the complete Terminus CPU hybrid service

   Run all main model parts on one CPU host, 16 inference threads, one 8192-position sequence and 2048 response cap. The native server renders the exact complete Jinja template; each request sets its real thinking boolean. Native BOS/EOS framing and MLA are preserved. No speculative MTP is requested; flash-attention fusion is off for this CPU plan.

   CommandCopy command

   set -e
   cd "$HOME/deepseek-v31-terminus-hosting"
   export TERMINUS_SERVICE_KEY="$(cat .api-key)"
   ./llama.cpp/build/bin/llama-server -m models/DeepSeek-V3.1-Terminus-Q4_K_M-00001-of-00009.gguf --alias deepseek-v31-terminus --host 127.0.0.1 --port 8080 --api-key-file .api-key --device none -ngl 0 -t 16 -c 8192 -np 1 -n 2048 --no-context-shift --flash-attn off --cache-type-k f16 --cache-type-v f16 -b 256 -ub 256 --jinja --chat-template-file models/native-chat-template.jinja --skip-chat-parsing --reasoning-format none --special --temp 0.6 --top-p 0.95 --top-k 0 --min-p 0 --repeat-penalty 1 --presence-penalty 0 --frequency-penalty 0

   Expected result: Every split loads and loopback readiness succeeds. Stop on missing tensors, architecture, cache, template, allocation or tokenizer failures. No successful startup or speed is preclaimed.

Verify the setup

1. 1. Check native thinking, ordinary follow-up and history

   Use a second shell inside the host. The fixed client checks health/401, uses exact native template rendering and raw token counts before generation, requires real EOS/no truncation/valid usage and checks native reasoning removal from history. Manually require Saturday six hours, weekday opening 9am and Sunday closure; protocol shape does not prove accuracy.

   CommandCopy command

   set -e
   cd "$HOME/deepseek-v31-terminus-hosting"
   python3 hybrid-client.py

   Expected result: Both modes return complete nonempty final text; thinking has exactly one closing marker and nonempty reasoning. Source-prescribed history removal renders identically to final-only history. Wrong facts, length exhaustion, malformed reasoning or any budget failure is a failed check.

Performance evidence

No compatible measurement available. Documented support and estimated memory do not establish response speed.

Limits and troubleshooting

• Source-qualified documentation only. No weights/model binaries, native model/template execution, build, installation, inference, fit, speed or quality measurement was performed.
• All nine Terminus files total 405,398,012,864 bytes. All individual hashes differ from the separate V3.2 files; similar part sizes do not imply model identity or byte equivalence.
• The 61-main-layer native model has MLA and routed/shared experts. V3.2 DSA/indexer architecture and its Python encoder are not transferred.
• Native config stores 163840 maximum positions and tokenizer config 131072; publisher-linked family documentation describes 128K. This bounded guide selects 8192 total positions and 2048 for complete reasoning/final output, not full-capacity or measured reasoning quality.
• The native embedded Jinja uses thinking true/false and defaults false; Native BOS 0 and EOS 1 are verified here; native prompt formats require their own checkpoint-specific qualification. Native generation configuration selects temperature 0.6/top_p 0.95. No borrowed enable_thinking toggle or generic template is used.
• Terminus explicitly retains a known native FP8 o_proj scale-format issue; no source proves this derivative fixes it. The exact named Q4 artifact is independently pinned and output must pass the factual checks. Native router correction bias must remain F32; the pinned converter preserves 1D tensors as F32, but derivative body tensor types were not inspected and no native byte-equivalence is claimed.
• Optional NextN/MTP tensor inclusion is uninspected in derivative binary bodies; all selected published parts/full main model remain required. No speculative speedup is claimed.
• The complete template drops past reasoning before a later turn. The client checks full-history rendering equals final-only rendering, then validates both modes and true EOS. It does not stop at the closing-think token or treat cap exhaustion as success.
• Tools/search-agent workflows, native FP8/BF 16, GPUs/distribution, larger contexts and other quantizations remain open. Updated Terminus search-agent template is not substituted with ordinary V3.1 search prompting. Family required totals remain null.
• Trusted loopback and a private key are not a public TLS/multiuser service. Actual cache/scratch/load peaks, fit, latency, throughput, quality and task spend remain unknown.

Shard/hash or architecture/template failure

Stop. Check all exact Terminus files, native template derivation and pinned DEEPSEEK2 runtime. Do not substitute later-family encoders, different checkpoints or partial expert sets.

Out of memory or budget failure

Stop and retain actual logs/counts. Verify 512 GiB actually available or separately review a changed configuration. Active parameters do not establish fit; no silent prompt/reasoning truncation.

EOS, reasoning or history check fails

Treat verification as failed. Inspect true native stop type, raw usage, exact native template/thinking boolean and prefix. Require complete final text; do not add a closing-think stopword to conceal truncation.

HTTP401 or connection refused

Inspect readiness, workstation loopback port and exact local service key.

Stop the model

1. 1. Stop the native service

   Press Ctrl+C in the foreground workstation server shell, then unset the key in every shell. Confirm the intended loopback listener is gone.

   CommandCopy command

   unset TERMINUS_SERVICE_KEY
   if curl --silent --max-time 3 http://127.0.0.1:8080/health >/dev/null; then echo "Endpoint still responds: stop intended process"; exit 1; else echo "Endpoint no longer responds"; fi

   Expected result: The workstation loopback service no longer responds.

Uninstall

1. 1. Remove dedicated walkthrough files

   Back up wanted results and logs and confirm the dedicated path before removal. Shared system build packages remain installed.

   CommandCopy command

   rm -r -- "$HOME/deepseek-v31-terminus-hosting"

   Expected result: Dedicated model splits, native template, client, key, results and build files are removed; shared system packages remain.

Related guides

• https://opentools.ai/llms/deepseek-v31-terminus