DeepSeek V3.2 Exp

Exact model and requirements

Artifact:

DeepSeek AI (native Exp checkpoint); reader-built GGUF / Complete DeepSeek V3.2 Exp, reader-generated Q4_K_M with DSA and NextN · checkpoint deepseek-ai/DeepSeek-V3.2-Exp

Publisher revision:

194c67e12b1b0d6df0ef373ddcf215bc84027409

Artifact digest:

Final generated composition and per-file hashes must be recorded by the reader after successful conversion; no prebuilt derivative digest is invented.

Download size:

Unknown

Memory requirement (unknown):

Unknown — No measured full CPU process peak for all quantized weights, DSA/MLA cache, scratch, loading, runtime and OS.

System RAM / disk:

Unknown / Unknown

Placement:

single device. One CPU host. Full native163-shard input and generated split GGUF composition are disk splits, not device sharding. All language layers, routed/shared experts and DSA/MLA computation on one CPU host; no GPU offload or selective expert subset.

KV cache:

F16 cache flags are explicit; native DSA maintains separate MLA and lightning-indexer state. Exact state layout and working-memory peak remain unmeasured.

Workload:

8,192 token context; Unknown; Unknown; concurrency 1; batch 1; cache unknown; repetitions Unknown.

Tokenizer:

Native Exp tokenizer and exact Jinja; derived conversion-only BOS flag, serving raw token IDs · 194c67e12b1b0d6df0ef373ddcf215bc84027409; converter620d2225

Sampling / reasoning:

Native generation config temperature 0.6 and top_p0.95. Top_k0, min_p0 and repeat_penalty 1 are author choices. True native EOS required; length stops fail verification. Exact native Exp Jinja thinking=false supplies a closed-think prefix for ordinary chat; thinking=true supplies an open-think prefix. Full alternating history is rendered by the pinned template. Reader verifies one BOS, full input reserve, actual EOS, closed nonempty reasoning and final text. This reduced walkthrough budget is an author choice; native longer contexts, tools and GPU workflows remain open.

Runtime prerequisites:

Ubuntu 24.04 x86_64; See prerequisites above.

Official artifact · License

Exact native README explicitly grants MIT for code and model weights. The pinned MIT LICENSE requires the copyright and permission notice in copies or substantial portions and disclaims warranties. Read and retain actual terms. No terms acceptance or legal clearance is performed. Public source metadata is ungated.

• assets/chat_template.jinja · 0 GiB · revision 194c67e12b1b0d6df0ef373ddcf215bc84027409
• config.json · 0 GiB · revision 194c67e12b1b0d6df0ef373ddcf215bc84027409
• tokenizer_config.json · 0 GiB · revision 194c67e12b1b0d6df0ef373ddcf215bc84027409
• tokenizer.json · 0.01 GiB · revision 194c67e12b1b0d6df0ef373ddcf215bc84027409
• model.safetensors.index.json · 0.01 GiB · revision 194c67e12b1b0d6df0ef373ddcf215bc84027409
• generation_config.json · 0 GiB · revision 194c67e12b1b0d6df0ef373ddcf215bc84027409
• LICENSE · 0 GiB · revision 194c67e12b1b0d6df0ef373ddcf215bc84027409

• Full native 163-shard serialized input is 689,483,049,129 bytes, not the 37B-active routing label. Main-model family branding is 671B; native parameter metadata 685,355,329,792 includes appended NextN/MTP tensors.
• Native index metadata total_size 1,370,793,842,752 is not the stored FP8 shard total. A F16 intermediate is roughly 1.371 TB by parameter-width planning; actual generated bytes and Q4 output sizes must be recorded.
• Plan 4 TiB free for native assets, F16 intermediate, final Q4 outputs, dependencies, logs and buffer. Conversion hardlinks reuse source weights on one filesystem. No proof that this allocation bounds the peak; stop on resource failures.
• All main 61 layers plus one appended NextN layer and DSA indexer/MLA state are retained. DSA64 heads,128 index dimension and top-k2048 do not justify an ordinary dense KV formula.
• 512 GiB RAM is a host allocation, not a measured conversion or serving requirement. CPU/board maximums do not prove actual DIMM/BIOS/OS configuration. Speed, fit and quantization quality are unknown.

Install the runtime

1. 1. Build the exact full CPU runtime

   Build exact official CPU runtime and install the official hash-pinned Torch 2.11 CPU wheel for Ubuntu 24.04 Python 3.12. Exact converter requirements additionally pin Transformers 4.57.6 and compatible conversion libraries; record every resolved dependency and log. Lazy FP8 dequantization is source-qualified, not executed here.

   CommandCopy command

   set -e
   sudo apt-get update
   sudo apt-get install -y build-essential cmake git curl ca-certificates openssl python3 python3-venv
   mkdir -p "$HOME/deepseek-v32-exp-hosting/models"
   cd "$HOME/deepseek-v32-exp-hosting"
   git clone https://github.com/ggml-org/llama.cpp.git
   git -C llama.cpp checkout --detach 620d2225cc436ab0b001e3200e590e3b286ef9ab
   cmake -S llama.cpp -B llama.cpp/build -DCMAKE_BUILD_TYPE=Release -DGGML_NATIVE=ON -DGGML_METAL=OFF
   cmake --build llama.cpp/build --config Release -j 16
   python3 -m venv .venv
   .venv/bin/python -m pip install 'https://download-r2.pytorch.org/whl/cpu/torch-2.11.0%2Bcpu-cp312-cp312-manylinux_2_28_x86_64.whl#sha256=f82e2ae20c1545bb03997d1cc3143d94e14b800038669ee1aca45808a9acc338'
   .venv/bin/python -m pip install -r llama.cpp/requirements/requirements-convert_hf_to_gguf.txt
   .venv/bin/python -m pip freeze > resolved-python-packages.txt
   ./llama.cpp/build/bin/llama-server --version

   Expected result: The exact source builds and version is recorded. Stop on any failure. No author build, model load or inference result is preclaimed.

Download the model

1. 1. Save complete native asset verifier

   Save this bounded stdlib reader helper. It validates stable native identity/revision and the canonical 163-file manifest, streams all downloads/checksums, verifies exact auxiliary assets and preserves originals. Whole mutable API-response bytes are not used as a validity gate.

   CommandCopy command

   set -e
   cd "$HOME/deepseek-v32-exp-hosting"
   cat > reader_assets.py <<'PYASSETS'
   """Reader-run complete native asset verification; stdlib only. Does not import model code."""
   import hashlib,json,pathlib,urllib.request,os,shutil
   PIN='194c67e12b1b0d6df0ef373ddcf215bc84027409'
   REPO='deepseek-ai/DeepSeek-V3.2-Exp'
   MANIFEST_SHA='5acc12f424e193d1495f90e5a8edb1529f6729958a8df77aecfa1c3b341a2b47'
   TOTAL=689483049129
   AUX={
    'config.json':(1552,'c7fa8b191e9936d8e6a57d864baab82b792fae16a116416cdd3a75ba76bc5af1'),
    'tokenizer_config.json':(4062,'2689fc149928cb32970c7e1d592e845f56edcf0b9c04ae4386e387efed992da1'),
    'tokenizer.json':(7847578,'32b34a41212e92f62e859cbbea121ae705a1fabbf157d9acf22d134ecd8dcf70'),
    'assets/chat_template.jinja':(3698,'a15911cb55988f908122b60ec6d4591b6207a0784fc294376827a155fe196155'),
    'generation_config.json':(171,'0ef9febae6b6087f4822b02bc9a1c03a83263dabaa931fb9155d061c8951ca07'),
    'LICENSE':(1084,'f2c6c602815669d292889e5be8c802f2ed950653b77999b1584e8e6aed25d040')}
   
   def digest(path):
    h=hashlib.sha256();n=0
    with open(path,'rb') as f:
     for b in iter(lambda:f.read(8*1024*1024),b''):h.update(b);n+=len(b)
    return n,h.hexdigest()
   
   def check_manifest(obj):
    if obj.get('id')!=REPO or obj.get('sha')!=PIN:raise ValueError('Wrong native identity or revision')
    out=[]
    for f in obj.get('siblings',[]):
     name=f.get('rfilename','')
     if name.endswith('.safetensors'):
      out.append({'name':name,'bytes':f.get('size'),'sha256':f.get('lfs',{}).get('sha256')})
    out.sort(key=lambda f:f['name'])
    if len(out)!=163 or sum(f['bytes'] for f in out)!=TOTAL:raise ValueError('Incomplete checkpoint')
    if hashlib.sha256(json.dumps(out,sort_keys=True,separators=(',',':')).encode()).hexdigest()!=MANIFEST_SHA:raise ValueError('Exact all-shard manifest mismatch')
    return out
   
   def fetch(name,n,h):
    path=pathlib.Path('native')/name
    path.parent.mkdir(parents=True,exist_ok=True)
    if path.exists():
     if digest(path)!=(n,h):raise ValueError('Existing asset mismatch '+name)
     return
    tmp=path.with_suffix(path.suffix+'.partial')
    with urllib.request.urlopen('https://huggingface.co/'+REPO+'/resolve/'+PIN+'/'+name,timeout=120) as r,open(tmp,'wb') as f:
     for b in iter(lambda:r.read(8*1024*1024),b''):f.write(b)
    if digest(tmp)!=(n,h):raise ValueError('Downloaded asset mismatch '+name)
    tmp.replace(path)
   
   def main():
    # Four TiB is prescribed working storage, not a proven conversion peak.
    existing=sum(p.stat().st_size for p in pathlib.Path('native').glob('*.safetensors'))
    if shutil.disk_usage('.').free+existing<3848290697216:raise ValueError('Prescribe at least3.5TiB free at initial download (existing native bytes credited only pending checksum); conversion may still need more')
    with urllib.request.urlopen('https://huggingface.co/api/models/'+REPO+'/revision/'+PIN+'?blobs=true',timeout=120) as r:raw=r.read(2000001)
    if len(raw)>2000000:raise ValueError('Metadata bound exceeded')
    files=check_manifest(json.loads(raw));pathlib.Path('native').mkdir(exist_ok=True)
    for f in files:fetch(f['name'],f['bytes'],f['sha256'])
    for name,(n,h) in AUX.items():fetch(name,n,h)
    # Index is native source text, exact digest derived privately from complete archived bytes.
    index_n,index_h=(8938926, '2a150b2af4aba7b037edc9fdba70f3b41abf758ee926279e7701d954298f9884')
    fetch('model.safetensors.index.json',index_n,index_h)
    idx=json.loads(pathlib.Path('native/model.safetensors.index.json').read_text())
    if set(idx['weight_map'].values())!={f['name'] for f in files}:raise ValueError('Index/shard composition mismatch')
    out=pathlib.Path('conversion-input');out.mkdir(exist_ok=True)
    for path in pathlib.Path('native').iterdir():
     if not path.is_file():continue
     dest=out/path.name
     if path.name=='tokenizer_config.json':
      obj=json.loads(path.read_text());assert obj['add_bos_token'] is False;obj['add_bos_token']=True
      data=(json.dumps(obj,ensure_ascii=False,indent=2)+'\n').encode()
      if dest.exists() and dest.read_bytes()!=data:raise ValueError('Unexpected derived tokenizer config')
      dest.write_bytes(data)
     elif dest.exists():
      if not os.path.samefile(path,dest):raise ValueError('Expected same-filesystem hardlink '+str(dest))
     else:os.link(path,dest)
    pathlib.Path('native-template.jinja').write_bytes(pathlib.Path('native/assets/chat_template.jinja').read_bytes())
    pathlib.Path('native-asset-manifest.json').write_text(json.dumps(files,indent=2)+'\n')
    pathlib.Path('derived-tokenizer-config-audit.json').write_text(json.dumps({'nativeSha256':AUX['tokenizer_config.json'][1],'derivedSha256':digest(out/'tokenizer_config.json')[1],'onlySemanticChange':{'add_bos_token':[False,True]},'reason':'Pinned converter assertion; serving overrides native template and uses raw IDs with add_special false.'},indent=2)+'\n')
    print('All163 native shards and exact auxiliary files pass. Native files unchanged; conversion-only tokenizer flag recorded.')
   if __name__=='__main__':main()
   PYASSETS

   Expected result: Verifier saved. No model files are downloaded until the next reader-run step.

2. 2. Download all native shards and prepare a separate conversion input

   Run only after checking at least 3.5 TiB initially free on the 4 TiB working disk. Existing files must match exact metadata. Hardlinks avoid a second full native copy on this filesystem; the derived tokenizer configuration changes only the required BOS flag. Any failure stops setup.

   CommandCopy command

   set -e
   cd "$HOME/deepseek-v32-exp-hosting"
   python3 reader_assets.py

   Expected result: All 163 native shards, index and auxiliary checks pass. Native bytes remain unchanged. The derived tokenizer configuration audit records its exact hash and sole flag change; original weight total is 689,483,049,129 bytes.

3. 3. Convert the complete original checkpoint and quantize every split

   Reader operation only. Default lazy conversion preserves all main and NextN tensors; never pass no-mtp or mtp-only options. F16 output and final Q4 files coexist with native input, so monitor disk and memory and stop on exhaustion. No completed conversion or fit is preclaimed.

   CommandCopy command

   set -e
   cd "$HOME/deepseek-v32-exp-hosting"
   .venv/bin/python llama.cpp/convert_hf_to_gguf.py conversion-input --outfile models/DeepSeek-V3.2-Exp-F16.gguf --outtype f16 --split-max-size 40G
   FIRST_F16="$(find models -maxdepth 1 -name 'DeepSeek-V3.2-Exp-F16-00001-of-*.gguf' -print)"
   test -n "$FIRST_F16"
   ./llama.cpp/build/bin/llama-quantize --keep-split "$FIRST_F16" models/DeepSeek-V3.2-Exp-Q4_K_M.gguf Q4_K_M 16

   Expected result: Full F16 and Q4 split files are generated if reader conversion succeeds. Actual split counts, bytes and digests are unknown until execution; a partial file set is not a usable model.

4. 4. Audit complete generated architecture and record output hashes

   Save and run this metadata-only reader audit with the pinned local GGUF library. Full architecture/DSA metadata is required in the first split; exact writer/quantizer places only split metadata in later parts. Every split must be ordered and present, and the combined tensor set must retain all main-layer DSA indexer tensors and appended NextN. Streaming checksums record actual generated output.

   CommandCopy command

   set -e
   cd "$HOME/deepseek-v32-exp-hosting"
   cat > reader-check-gguf.py <<'PYGGUF'
   """Reader-only generated GGUF metadata and streaming-digest audit; no model inference."""
   import glob,json,pathlib,sys
   from reader_assets import digest
   
   def inspect(readers):
    if not readers:raise ValueError('No generated split files')
    names=set()
    expected={'general.architecture':'deepseek32','deepseek32.block_count':62,'deepseek32.nextn_predict_layers':1,'deepseek32.expert_count':256,'deepseek32.attention.indexer.head_count':64,'deepseek32.attention.indexer.key_length':128,'deepseek32.attention.indexer.top_k':2048}
    for i,r in enumerate(readers):
     for key,value in expected.items():
      f=r.get_field(key)
      # Exact writer/quantizer stores full model KV only in the first split.
      if (i==0 and f is None) or (f is not None and f.contents()!=value):raise ValueError('Wrong full architecture metadata '+key)
     f=r.get_field('split.count');n=r.get_field('split.no')
     if len(readers)>1 and (f is None or f.contents()!=len(readers) or n is None or n.contents()!=i):raise ValueError('Missing or misordered split')
     for t in r.tensors:
      if t.name in names:raise ValueError('Duplicate generated tensor')
      names.add(t.name)
    for layer in range(61):
     for suffix in ('indexer.k_norm.weight','indexer.k_norm.bias','indexer.proj.weight','indexer.attn_k.weight','indexer.attn_q_b.weight'):
      if 'blk.'+str(layer)+'.'+suffix not in names:raise ValueError('Missing native DSA indexer tensor')
    if not any(n.startswith('blk.61.') for n in names):raise ValueError('Missing appended NextN tensors')
    return len(names)
   
   def main():
    # Import only when reader runs after conversion. Author never imports gguf/native dependencies.
    import gguf
    files=sorted(glob.glob('models/DeepSeek-V3.2-Exp-Q4_K_M*.gguf'))
    readers=[gguf.GGUFReader(f,'r') for f in files]
    count=inspect(readers)
    manifest=[]
    for f in files:
     n,h=digest(f);manifest.append({'name':f,'bytes':n,'sha256':h})
    pathlib.Path('generated-gguf-manifest.json').write_text(json.dumps({'nativeRevision':'194c67e12b1b0d6df0ef373ddcf215bc84027409','tensorCount':count,'files':manifest},indent=2)+'\n')
    pathlib.Path('first-model-path').write_text(files[0]+'\n')
    print('All generated splits retain deepseek32, DSA indexer fields/tensors and appended NextN; actual output hashes recorded. This is not fit, speed or quantization-quality proof.')
   if __name__=='__main__':main()
   PYGGUF
   PYTHONPATH=llama.cpp/gguf-py .venv/bin/python reader-check-gguf.py

   Expected result: Generated manifest and first-model-path are recorded after all architecture/composition checks. No inference or quantization-quality conclusion follows.

Configure

1. 1. Create a private service credential

   Generate a reader-owned local service password, preserving an existing key on repeat setup. Never print or share it; this is not a provider API credential.

   CommandCopy command

   set -e
   cd "$HOME/deepseek-v32-exp-hosting"
   umask 077
   if [ ! -e .api-key ]; then openssl rand -hex 32 > .api-key; fi
   test -s .api-key
   chmod 600 .api-key

   Expected result: A nonempty mode-600 private .api-key exists; existing key preserved.

2. 2. Save bounded native Exp conversation diagnostic

   The stdlib client asks the pinned runtime to apply the exact native Jinja. It verifies ordinary and reasoning prefixes, one BOS, full input budget, native EOS, no truncation and closed nonempty thinking/final text. It imports no model or native encoder.

   CommandCopy command

   set -e
   cd "$HOME/deepseek-v32-exp-hosting"
   cat > reasoning-client.py <<'PYCLIENT'
   """Private native DeepSeek V3.2 Exp raw-completion diagnostic; stdlib only."""
   import hashlib, json, pathlib, urllib.request, urllib.error
   BASE='http://127.0.0.1:8080'
   CONTEXT, OUTPUT = 8192, 2048
   TEMPLATE_SHA='a15911cb55988f908122b60ec6d4591b6207a0784fc294376827a155fe196155'
   BOS='<｜begin▁of▁sentence｜>'
   EOS='<｜end▁of▁sentence｜>'
   
   def positive(v): return type(v) is int and v>0
   
   def checked_messages(messages):
       if not isinstance(messages,list) or not 1<=len(messages)<=12: raise ValueError('Bounded conversation required')
       for i,m in enumerate(messages):
           if not isinstance(m,dict) or set(m)-{'role','content'}: raise ValueError('Only ordinary text conversation is accepted by this diagnostic')
           role,text=m.get('role'),m.get('content')
           if role not in ('system','user','assistant') or (role=='system' and i!=0): raise ValueError('Invalid role order')
           if not isinstance(text,str) or not text.strip() or len(text.encode())>12000: raise ValueError('Invalid text')
           if any(t in text for t in (BOS,EOS,'<｜User｜>','<｜Assistant｜>','<think>','</think>','｜DSML｜')): raise ValueError('Reserved native control marker in user content')
       roles=[m['role'] for m in messages if m['role']!='system']
       if not roles or roles!=['user' if i%2==0 else 'assistant' for i in range(len(roles))] or roles[-1]!='user': raise ValueError('Alternating user/assistant history ending in user required')
       return messages
   
   def complete(call,messages,mode):
       if mode not in ('thinking','chat'): raise ValueError('Invalid native mode')
       checked_messages(messages)
       prompt=call('/apply-template',{'messages':messages,'add_generation_prompt':True,'chat_template_kwargs':{'thinking':mode=='thinking'}}).get('prompt')
       if not isinstance(prompt,str) or not prompt.startswith(BOS) or prompt.count(BOS)!=1: raise ValueError('Exactly one native BOS required')
       expected='<think>' if mode=='thinking' else '</think>'
       if not prompt.endswith(expected): raise ValueError('Native mode prefix mismatch')
       ids=call('/tokenize',{'content':prompt,'add_special':False,'parse_special':True}).get('tokens')
       if not isinstance(ids,list) or not ids or any(type(x)is not int or x<0 for x in ids) or ids[0]!=0 or ids.count(0)!=1: raise ValueError('Invalid token IDs or duplicate BOS')
       if len(ids)+OUTPUT>CONTEXT: raise ValueError('Input plus complete reasoning/output reserve exceeds context')
       body={'prompt':ids,'n_predict':OUTPUT,'temperature':0.6,'top_p':0.95,'top_k':0,'min_p':0,'repeat_penalty':1,'stream':False,'cache_prompt':False,'return_tokens':True,'stop':[]}
       result=call('/completion',body)
       if result.get('stop_type')!='eos' or result.get('truncated') is not False: raise ValueError('Require actual native EOS without truncation; limit/word stop does not prove completion')
       n=result.get('tokens_predicted')
       if not positive(n) or n>OUTPUT or result.get('tokens_evaluated')!=len(ids): raise ValueError('Usage mismatch')
       text=result.get('content')
       if not isinstance(text,str) or not text.strip(): raise ValueError('Empty completion')
       if EOS in text:
           if not text.endswith(EOS) or text.count(EOS)!=1: raise ValueError('Unexpected EOS text')
           text=text[:-len(EOS)]
       if mode=='thinking':
           if text.count('</think>')!=1: raise ValueError('Require one closed reasoning block')
           reasoning,final=text.split('</think>',1)
           if not reasoning.strip() or not final.strip(): raise ValueError('Empty reasoning/final body')
           parsed={'reasoning_content':reasoning,'content':final}
       else:
           parsed={'content':text}
       if any(marker in parsed['content'] for marker in (BOS,EOS,'<｜User｜>','<｜Assistant｜>','<think>','</think>','<｜tool')): raise ValueError('Unexpected native control marker')
       return {'mode':mode,'inputTokens':len(ids),'outputCap':OUTPUT,'parsed':parsed,'nativeResponse':result}
   
   def http_call(key):
       def call(path,body=None):
           data=None if body is None else json.dumps(body).encode()
           req=urllib.request.Request(BASE+path,data=data,headers={'Authorization':'Bearer '+key,'Content-Type':'application/json'})
           with urllib.request.urlopen(req,timeout=7200) as r:
               if r.status!=200: raise ValueError('HTTP failure')
               raw=r.read(2000001)
           if len(raw)>2000000: raise ValueError('Response limit')
           obj=json.loads(raw)
           if not isinstance(obj,dict): raise ValueError('JSON object required')
           return obj
       return call
   
   def main():
       key=pathlib.Path('.api-key').read_text().strip()
       if len(key)!=64 or any(c not in '0123456789abcdef' for c in key): raise ValueError('Restore private service key')
       if hashlib.sha256(pathlib.Path('native-template.jinja').read_bytes()).hexdigest()!=TEMPLATE_SHA: raise ValueError('Native template checksum mismatch')
       call=http_call(key);call('/health')
       req=urllib.request.Request(BASE+'/completion',data=b'{}',headers={'Content-Type':'application/json'})
       try:
           with urllib.request.urlopen(req,timeout=30): raise ValueError('Unauthorized completion accepted')
       except urllib.error.HTTPError as e:
           if e.code!=401: raise
       history=[{'role':'user','content':'The community library is open weekdays 9am to 6pm, Saturday 10am to 4pm and closed Sunday. State the weekday opening time only.'}]
       first=complete(call,history,'chat')
       pathlib.Path('chat-response.json').write_text(json.dumps(first,indent=2)+'\n')
       history += [{'role':'assistant','content':first['parsed']['content']},{'role':'user','content':'Using the same supplied schedule, how many hours is the library open on Saturday, and is it open on Sunday? Explain briefly.'}]
       second=complete(call,history,'thinking')
       pathlib.Path('thinking-response.json').write_text(json.dumps(second,indent=2)+'\n')
       print(json.dumps({'chat':first,'thinking':second},ensure_ascii=False))
       print('Manually inspect supplied schedule facts and reasoning. Native EOS/closed thinking proves completion shape, not answer quality or speed.')
   if __name__=='__main__': main()
   PYCLIENT

   Expected result: Client saved without model execution. Native template bytes remain verbatim.

Start the model

1. 1. Serve the complete converted Exp model with native DSA

   Keep the process foreground and logs. The generated first split loads the full audited composition. Native Jinja override is explicit; raw token IDs avoid duplicate BOS. DSA indexer and main MLA remain active; flash fusion off is not DSA removal. No speculative MTP request is made, though tensors remain present.

   CommandCopy command

   set -e
   cd "$HOME/deepseek-v32-exp-hosting"
   export DEEPSEEK_EXP_SERVICE_KEY="$(cat .api-key)"
   ./llama.cpp/build/bin/llama-server -m "$(cat first-model-path)" --alias deepseek-v32-exp --jinja --chat-template-file native-template.jinja --host 127.0.0.1 --port 8080 --api-key-file .api-key --device none -ngl 0 -t 16 -c 8192 -np 1 -n 2048 --no-context-shift --flash-attn off --cache-type-k f16 --cache-type-v f16 -b 256 -ub 256 --special --temp 0.6 --top-p 0.95 --top-k 0 --min-p 0 --repeat-penalty 1

   Expected result: If successful, complete DEEPSEEK32 DSA initializes and authenticated loopback becomes ready. Architecture, split, tokenizer or resource errors require stopping; no author load or fit result exists.

Verify the setup

1. 1. Verify native ordinary chat, reasoning, history and actual completion

   Use a second shell on the same host. Cloud diagnostics run inside the VM with its local files/key. Health and unauthorized 401 are checked first. Both full conversations are rendered by native Exp Jinja, exact token IDs and budgets checked before generation. Require actual EOS, closed reasoning and a nonempty final body; human inspection assesses supplied schedule facts separately.

   CommandCopy command

   set -e
   cd "$HOME/deepseek-v32-exp-hosting"
   python3 reasoning-client.py

   Expected result: Both native modes complete with actual EOS, matching usage and no truncation. Inspect weekday/Saturday/Sunday facts in saved outputs; protocol shape is not semantic correctness, speed or general reasoning quality.

Performance evidence

No compatible measurement available. Documented support and estimated memory do not establish response speed.

Limits and troubleshooting

• Documented source qualification only: no native dependencies imported, weights downloaded, conversion/build/model load/inference performed or fit/speed/quality measured.
• Exact Exp native checkpoint and 3698-byte Jinja are independently pinned. Ordinary V3.2 uses a different encoder; Speciale is a distinct checkpoint. Neither substitutes here.
• All 163 native shards total 689,483,049,129 bytes. Reader-produced F16/Q4 outputs have unknown actual sizes/digests until successful execution; all expert and DSA tensors must remain.
• The native 61 main layers plus appended NextN layer are retained by default conversion. Main branding 671B/37B-active differs from 685,355,329,792 total parameter metadata; routing activity is not resident weight size.
• The exact runtime includes Exp conversion and independent indexer RoPE handling. Native README records a past indexer RoPE bug; use the pinned corrected runtime rather than an older demo.
• Native config maximum positions 163,840 and tokenizer maximum 131,072 are source metadata, not tested serving limits. Author selects 8,192 combined tokens and 2,048 generated reasoning/final tokens; this is not the publisher full-reasoning recommendation.
• DSA indexer selects up to 2,048 prior positions with 64 heads of 128 dimensions. MLA/indexer cache and scratch require separate memory; no dense ordinary KV estimate or sparse speed claim is made.
• Only derived conversion tokenizer config changes add_bos_token to true for the converter assertion; native original and serving Jinja are unchanged. Raw serving tokenization add_special false and one BOS are explicitly checked.
• 512 GiB RAM and 4 TiB working disk are planning allocations, not proven peak bounds. Source weights, F16 intermediate, Q4 outputs and dependencies coexist. Stop on allocation/disk errors and preserve logs.
• Torch 2.11 official CPU wheel supports exact dtype conversion; resolved remaining conversion dependencies are reader-installed and logged, not a fully agent-executed environment or compatibility result.
• Private loopback plus Bearer key requires a trusted host; privileged local users remain outside this boundary. Public TLS, multiuser isolation, tools, native GPU/distribution, full context and speculative MTP remain separate open scopes; required guide total is null.

Native checksum, missing shard or generated architecture mismatch

Stop and preserve the failed check. Require all exact native assets and generated deepseek32 DSA metadata/tensors. Do not disable sparse attention or substitute an unrelated checkpoint.

Converter BOS assertion

Use the separate conversion input and recorded single flag change; never edit the pinned native tokenizer or template. Keep serving raw token IDs and verify exactly one BOS.

Memory, disk or native runtime failure

Stop. Planning capacity does not prove conversion or serving fit. Preserve logs and actual resource usage for a separately reviewed plan.

Length stop, open thinking or empty final body

Treat verification as failed. Require actual EOS and closed nonempty reasoning/final text; no silent truncation or fabricated answer-quality claim.

Unauthorized or unreachable service

Check exact foreground process, workstation loopback address and matching reader key. Keep the service bound to this trusted workstation loopback.

Stop the model

1. 1. Stop the native service

   Press Ctrl+C in the foreground server shell, unset the key in every shell and confirm the intended local listener is gone.

   CommandCopy command

   unset DEEPSEEK_EXP_SERVICE_KEY
   if curl --silent --max-time 3 http://127.0.0.1:8080/health >/dev/null; then echo "Endpoint still responds: stop intended process"; exit 1; else echo "Endpoint no longer responds"; fi

   Expected result: The intended loopback service no longer responds; cloud compute/storage may still bill.

Uninstall

1. 1. Remove dedicated walkthrough files

   Back up wanted results and logs and confirm the dedicated path before removal. Shared system build packages remain installed.

   CommandCopy command

   rm -r -- "$HOME/deepseek-v32-exp-hosting"

   Expected result: Dedicated native shards, derived conversion inputs, generated model files, dependency environment, client, key, results and build files are removed; shared system packages remain.

Exact model and requirements

Artifact:

DeepSeek AI (native Exp checkpoint); reader-built GGUF / Complete DeepSeek V3.2 Exp, reader-generated Q4_K_M with DSA and NextN · checkpoint deepseek-ai/DeepSeek-V3.2-Exp

Publisher revision:

194c67e12b1b0d6df0ef373ddcf215bc84027409

Artifact digest:

Final generated composition and per-file hashes must be recorded by the reader after successful conversion; no prebuilt derivative digest is invented.

Download size:

Unknown

Memory requirement (unknown):

Unknown — No measured full CPU process peak for all quantized weights, DSA/MLA cache, scratch, loading, runtime and OS.

System RAM / disk:

Unknown / Unknown

Placement:

single device. One CPU host. Full native163-shard input and generated split GGUF composition are disk splits, not device sharding. All language layers, routed/shared experts and DSA/MLA computation on one CPU host; no GPU offload or selective expert subset.

KV cache:

F16 cache flags are explicit; native DSA maintains separate MLA and lightning-indexer state. Exact state layout and working-memory peak remain unmeasured.

Workload:

8,192 token context; Unknown; Unknown; concurrency 1; batch 1; cache unknown; repetitions Unknown.

Tokenizer:

Native Exp tokenizer and exact Jinja; derived conversion-only BOS flag, serving raw token IDs · 194c67e12b1b0d6df0ef373ddcf215bc84027409; converter620d2225

Sampling / reasoning:

Native generation config temperature 0.6 and top_p0.95. Top_k0, min_p0 and repeat_penalty 1 are author choices. True native EOS required; length stops fail verification. Exact native Exp Jinja thinking=false supplies a closed-think prefix for ordinary chat; thinking=true supplies an open-think prefix. Full alternating history is rendered by the pinned template. Reader verifies one BOS, full input reserve, actual EOS, closed nonempty reasoning and final text. This reduced walkthrough budget is an author choice; native longer contexts, tools and GPU workflows remain open.

Runtime prerequisites:

Ubuntu 24.04 x86_64; See prerequisites above.

Official artifact · License

Exact native README explicitly grants MIT for code and model weights. The pinned MIT LICENSE requires the copyright and permission notice in copies or substantial portions and disclaims warranties. Read and retain actual terms. No terms acceptance or legal clearance is performed. Public source metadata is ungated.

• assets/chat_template.jinja · 0 GiB · revision 194c67e12b1b0d6df0ef373ddcf215bc84027409
• config.json · 0 GiB · revision 194c67e12b1b0d6df0ef373ddcf215bc84027409
• tokenizer_config.json · 0 GiB · revision 194c67e12b1b0d6df0ef373ddcf215bc84027409
• tokenizer.json · 0.01 GiB · revision 194c67e12b1b0d6df0ef373ddcf215bc84027409
• model.safetensors.index.json · 0.01 GiB · revision 194c67e12b1b0d6df0ef373ddcf215bc84027409
• generation_config.json · 0 GiB · revision 194c67e12b1b0d6df0ef373ddcf215bc84027409
• LICENSE · 0 GiB · revision 194c67e12b1b0d6df0ef373ddcf215bc84027409

• Full native 163-shard serialized input is 689,483,049,129 bytes, not the 37B-active routing label. Main-model family branding is 671B; native parameter metadata 685,355,329,792 includes appended NextN/MTP tensors.
• Native index metadata total_size 1,370,793,842,752 is not the stored FP8 shard total. A F16 intermediate is roughly 1.371 TB by parameter-width planning; actual generated bytes and Q4 output sizes must be recorded.
• Plan 4 TiB free for native assets, F16 intermediate, final Q4 outputs, dependencies, logs and buffer. Conversion hardlinks reuse source weights on one filesystem. No proof that this allocation bounds the peak; stop on resource failures.
• All main 61 layers plus one appended NextN layer and DSA indexer/MLA state are retained. DSA64 heads,128 index dimension and top-k2048 do not justify an ordinary dense KV formula.
• 512 GiB RAM is a host allocation, not a measured conversion or serving requirement. CPU/board maximums do not prove actual DIMM/BIOS/OS configuration. Speed, fit and quantization quality are unknown.

Install the runtime

1. 1. Build the exact full CPU runtime

   Build exact official CPU runtime and install the official hash-pinned Torch 2.11 CPU wheel for Ubuntu 24.04 Python 3.12. Exact converter requirements additionally pin Transformers 4.57.6 and compatible conversion libraries; record every resolved dependency and log. Lazy FP8 dequantization is source-qualified, not executed here.

   CommandCopy command

   set -e
   sudo apt-get update
   sudo apt-get install -y build-essential cmake git curl ca-certificates openssl python3 python3-venv
   mkdir -p "$HOME/deepseek-v32-exp-hosting/models"
   cd "$HOME/deepseek-v32-exp-hosting"
   git clone https://github.com/ggml-org/llama.cpp.git
   git -C llama.cpp checkout --detach 620d2225cc436ab0b001e3200e590e3b286ef9ab
   cmake -S llama.cpp -B llama.cpp/build -DCMAKE_BUILD_TYPE=Release -DGGML_NATIVE=ON -DGGML_METAL=OFF
   cmake --build llama.cpp/build --config Release -j 64
   python3 -m venv .venv
   .venv/bin/python -m pip install 'https://download-r2.pytorch.org/whl/cpu/torch-2.11.0%2Bcpu-cp312-cp312-manylinux_2_28_x86_64.whl#sha256=f82e2ae20c1545bb03997d1cc3143d94e14b800038669ee1aca45808a9acc338'
   .venv/bin/python -m pip install -r llama.cpp/requirements/requirements-convert_hf_to_gguf.txt
   .venv/bin/python -m pip freeze > resolved-python-packages.txt
   ./llama.cpp/build/bin/llama-server --version

   Expected result: The exact source builds and version is recorded. Stop on any failure. No author build, model load or inference result is preclaimed.

Download the model

1. 1. Save complete native asset verifier

   Save this bounded stdlib reader helper. It validates stable native identity/revision and the canonical 163-file manifest, streams all downloads/checksums, verifies exact auxiliary assets and preserves originals. Whole mutable API-response bytes are not used as a validity gate.

   CommandCopy command

   set -e
   cd "$HOME/deepseek-v32-exp-hosting"
   cat > reader_assets.py <<'PYASSETS'
   """Reader-run complete native asset verification; stdlib only. Does not import model code."""
   import hashlib,json,pathlib,urllib.request,os,shutil
   PIN='194c67e12b1b0d6df0ef373ddcf215bc84027409'
   REPO='deepseek-ai/DeepSeek-V3.2-Exp'
   MANIFEST_SHA='5acc12f424e193d1495f90e5a8edb1529f6729958a8df77aecfa1c3b341a2b47'
   TOTAL=689483049129
   AUX={
    'config.json':(1552,'c7fa8b191e9936d8e6a57d864baab82b792fae16a116416cdd3a75ba76bc5af1'),
    'tokenizer_config.json':(4062,'2689fc149928cb32970c7e1d592e845f56edcf0b9c04ae4386e387efed992da1'),
    'tokenizer.json':(7847578,'32b34a41212e92f62e859cbbea121ae705a1fabbf157d9acf22d134ecd8dcf70'),
    'assets/chat_template.jinja':(3698,'a15911cb55988f908122b60ec6d4591b6207a0784fc294376827a155fe196155'),
    'generation_config.json':(171,'0ef9febae6b6087f4822b02bc9a1c03a83263dabaa931fb9155d061c8951ca07'),
    'LICENSE':(1084,'f2c6c602815669d292889e5be8c802f2ed950653b77999b1584e8e6aed25d040')}
   
   def digest(path):
    h=hashlib.sha256();n=0
    with open(path,'rb') as f:
     for b in iter(lambda:f.read(8*1024*1024),b''):h.update(b);n+=len(b)
    return n,h.hexdigest()
   
   def check_manifest(obj):
    if obj.get('id')!=REPO or obj.get('sha')!=PIN:raise ValueError('Wrong native identity or revision')
    out=[]
    for f in obj.get('siblings',[]):
     name=f.get('rfilename','')
     if name.endswith('.safetensors'):
      out.append({'name':name,'bytes':f.get('size'),'sha256':f.get('lfs',{}).get('sha256')})
    out.sort(key=lambda f:f['name'])
    if len(out)!=163 or sum(f['bytes'] for f in out)!=TOTAL:raise ValueError('Incomplete checkpoint')
    if hashlib.sha256(json.dumps(out,sort_keys=True,separators=(',',':')).encode()).hexdigest()!=MANIFEST_SHA:raise ValueError('Exact all-shard manifest mismatch')
    return out
   
   def fetch(name,n,h):
    path=pathlib.Path('native')/name
    path.parent.mkdir(parents=True,exist_ok=True)
    if path.exists():
     if digest(path)!=(n,h):raise ValueError('Existing asset mismatch '+name)
     return
    tmp=path.with_suffix(path.suffix+'.partial')
    with urllib.request.urlopen('https://huggingface.co/'+REPO+'/resolve/'+PIN+'/'+name,timeout=120) as r,open(tmp,'wb') as f:
     for b in iter(lambda:r.read(8*1024*1024),b''):f.write(b)
    if digest(tmp)!=(n,h):raise ValueError('Downloaded asset mismatch '+name)
    tmp.replace(path)
   
   def main():
    # Four TiB is prescribed working storage, not a proven conversion peak.
    existing=sum(p.stat().st_size for p in pathlib.Path('native').glob('*.safetensors'))
    if shutil.disk_usage('.').free+existing<3848290697216:raise ValueError('Prescribe at least3.5TiB free at initial download (existing native bytes credited only pending checksum); conversion may still need more')
    with urllib.request.urlopen('https://huggingface.co/api/models/'+REPO+'/revision/'+PIN+'?blobs=true',timeout=120) as r:raw=r.read(2000001)
    if len(raw)>2000000:raise ValueError('Metadata bound exceeded')
    files=check_manifest(json.loads(raw));pathlib.Path('native').mkdir(exist_ok=True)
    for f in files:fetch(f['name'],f['bytes'],f['sha256'])
    for name,(n,h) in AUX.items():fetch(name,n,h)
    # Index is native source text, exact digest derived privately from complete archived bytes.
    index_n,index_h=(8938926, '2a150b2af4aba7b037edc9fdba70f3b41abf758ee926279e7701d954298f9884')
    fetch('model.safetensors.index.json',index_n,index_h)
    idx=json.loads(pathlib.Path('native/model.safetensors.index.json').read_text())
    if set(idx['weight_map'].values())!={f['name'] for f in files}:raise ValueError('Index/shard composition mismatch')
    out=pathlib.Path('conversion-input');out.mkdir(exist_ok=True)
    for path in pathlib.Path('native').iterdir():
     if not path.is_file():continue
     dest=out/path.name
     if path.name=='tokenizer_config.json':
      obj=json.loads(path.read_text());assert obj['add_bos_token'] is False;obj['add_bos_token']=True
      data=(json.dumps(obj,ensure_ascii=False,indent=2)+'\n').encode()
      if dest.exists() and dest.read_bytes()!=data:raise ValueError('Unexpected derived tokenizer config')
      dest.write_bytes(data)
     elif dest.exists():
      if not os.path.samefile(path,dest):raise ValueError('Expected same-filesystem hardlink '+str(dest))
     else:os.link(path,dest)
    pathlib.Path('native-template.jinja').write_bytes(pathlib.Path('native/assets/chat_template.jinja').read_bytes())
    pathlib.Path('native-asset-manifest.json').write_text(json.dumps(files,indent=2)+'\n')
    pathlib.Path('derived-tokenizer-config-audit.json').write_text(json.dumps({'nativeSha256':AUX['tokenizer_config.json'][1],'derivedSha256':digest(out/'tokenizer_config.json')[1],'onlySemanticChange':{'add_bos_token':[False,True]},'reason':'Pinned converter assertion; serving overrides native template and uses raw IDs with add_special false.'},indent=2)+'\n')
    print('All163 native shards and exact auxiliary files pass. Native files unchanged; conversion-only tokenizer flag recorded.')
   if __name__=='__main__':main()
   PYASSETS

   Expected result: Verifier saved. No model files are downloaded until the next reader-run step.

2. 2. Download all native shards and prepare a separate conversion input

   Run only after checking at least 3.5 TiB initially free on the 4 TiB working disk. Existing files must match exact metadata. Hardlinks avoid a second full native copy on this filesystem; the derived tokenizer configuration changes only the required BOS flag. Any failure stops setup.

   CommandCopy command

   set -e
   cd "$HOME/deepseek-v32-exp-hosting"
   python3 reader_assets.py

   Expected result: All 163 native shards, index and auxiliary checks pass. Native bytes remain unchanged. The derived tokenizer configuration audit records its exact hash and sole flag change; original weight total is 689,483,049,129 bytes.

3. 3. Convert the complete original checkpoint and quantize every split

   Reader operation only. Default lazy conversion preserves all main and NextN tensors; never pass no-mtp or mtp-only options. F16 output and final Q4 files coexist with native input, so monitor disk and memory and stop on exhaustion. No completed conversion or fit is preclaimed.

   CommandCopy command

   set -e
   cd "$HOME/deepseek-v32-exp-hosting"
   .venv/bin/python llama.cpp/convert_hf_to_gguf.py conversion-input --outfile models/DeepSeek-V3.2-Exp-F16.gguf --outtype f16 --split-max-size 40G
   FIRST_F16="$(find models -maxdepth 1 -name 'DeepSeek-V3.2-Exp-F16-00001-of-*.gguf' -print)"
   test -n "$FIRST_F16"
   ./llama.cpp/build/bin/llama-quantize --keep-split "$FIRST_F16" models/DeepSeek-V3.2-Exp-Q4_K_M.gguf Q4_K_M 64

   Expected result: Full F16 and Q4 split files are generated if reader conversion succeeds. Actual split counts, bytes and digests are unknown until execution; a partial file set is not a usable model.

4. 4. Audit complete generated architecture and record output hashes

   Save and run this metadata-only reader audit with the pinned local GGUF library. Full architecture/DSA metadata is required in the first split; exact writer/quantizer places only split metadata in later parts. Every split must be ordered and present, and the combined tensor set must retain all main-layer DSA indexer tensors and appended NextN. Streaming checksums record actual generated output.

   CommandCopy command

   set -e
   cd "$HOME/deepseek-v32-exp-hosting"
   cat > reader-check-gguf.py <<'PYGGUF'
   """Reader-only generated GGUF metadata and streaming-digest audit; no model inference."""
   import glob,json,pathlib,sys
   from reader_assets import digest
   
   def inspect(readers):
    if not readers:raise ValueError('No generated split files')
    names=set()
    expected={'general.architecture':'deepseek32','deepseek32.block_count':62,'deepseek32.nextn_predict_layers':1,'deepseek32.expert_count':256,'deepseek32.attention.indexer.head_count':64,'deepseek32.attention.indexer.key_length':128,'deepseek32.attention.indexer.top_k':2048}
    for i,r in enumerate(readers):
     for key,value in expected.items():
      f=r.get_field(key)
      # Exact writer/quantizer stores full model KV only in the first split.
      if (i==0 and f is None) or (f is not None and f.contents()!=value):raise ValueError('Wrong full architecture metadata '+key)
     f=r.get_field('split.count');n=r.get_field('split.no')
     if len(readers)>1 and (f is None or f.contents()!=len(readers) or n is None or n.contents()!=i):raise ValueError('Missing or misordered split')
     for t in r.tensors:
      if t.name in names:raise ValueError('Duplicate generated tensor')
      names.add(t.name)
    for layer in range(61):
     for suffix in ('indexer.k_norm.weight','indexer.k_norm.bias','indexer.proj.weight','indexer.attn_k.weight','indexer.attn_q_b.weight'):
      if 'blk.'+str(layer)+'.'+suffix not in names:raise ValueError('Missing native DSA indexer tensor')
    if not any(n.startswith('blk.61.') for n in names):raise ValueError('Missing appended NextN tensors')
    return len(names)
   
   def main():
    # Import only when reader runs after conversion. Author never imports gguf/native dependencies.
    import gguf
    files=sorted(glob.glob('models/DeepSeek-V3.2-Exp-Q4_K_M*.gguf'))
    readers=[gguf.GGUFReader(f,'r') for f in files]
    count=inspect(readers)
    manifest=[]
    for f in files:
     n,h=digest(f);manifest.append({'name':f,'bytes':n,'sha256':h})
    pathlib.Path('generated-gguf-manifest.json').write_text(json.dumps({'nativeRevision':'194c67e12b1b0d6df0ef373ddcf215bc84027409','tensorCount':count,'files':manifest},indent=2)+'\n')
    pathlib.Path('first-model-path').write_text(files[0]+'\n')
    print('All generated splits retain deepseek32, DSA indexer fields/tensors and appended NextN; actual output hashes recorded. This is not fit, speed or quantization-quality proof.')
   if __name__=='__main__':main()
   PYGGUF
   PYTHONPATH=llama.cpp/gguf-py .venv/bin/python reader-check-gguf.py

   Expected result: Generated manifest and first-model-path are recorded after all architecture/composition checks. No inference or quantization-quality conclusion follows.

Configure

1. 1. Create a private service credential

   Generate a reader-owned local service password, preserving an existing key on repeat setup. Never print or share it; this is not a provider API credential.

   CommandCopy command

   set -e
   cd "$HOME/deepseek-v32-exp-hosting"
   umask 077
   if [ ! -e .api-key ]; then openssl rand -hex 32 > .api-key; fi
   test -s .api-key
   chmod 600 .api-key

   Expected result: A nonempty mode-600 private .api-key exists; existing key preserved.

2. 2. Save bounded native Exp conversation diagnostic

   The stdlib client asks the pinned runtime to apply the exact native Jinja. It verifies ordinary and reasoning prefixes, one BOS, full input budget, native EOS, no truncation and closed nonempty thinking/final text. It imports no model or native encoder.

   CommandCopy command

   set -e
   cd "$HOME/deepseek-v32-exp-hosting"
   cat > reasoning-client.py <<'PYCLIENT'
   """Private native DeepSeek V3.2 Exp raw-completion diagnostic; stdlib only."""
   import hashlib, json, pathlib, urllib.request, urllib.error
   BASE='http://127.0.0.1:8080'
   CONTEXT, OUTPUT = 8192, 2048
   TEMPLATE_SHA='a15911cb55988f908122b60ec6d4591b6207a0784fc294376827a155fe196155'
   BOS='<｜begin▁of▁sentence｜>'
   EOS='<｜end▁of▁sentence｜>'
   
   def positive(v): return type(v) is int and v>0
   
   def checked_messages(messages):
       if not isinstance(messages,list) or not 1<=len(messages)<=12: raise ValueError('Bounded conversation required')
       for i,m in enumerate(messages):
           if not isinstance(m,dict) or set(m)-{'role','content'}: raise ValueError('Only ordinary text conversation is accepted by this diagnostic')
           role,text=m.get('role'),m.get('content')
           if role not in ('system','user','assistant') or (role=='system' and i!=0): raise ValueError('Invalid role order')
           if not isinstance(text,str) or not text.strip() or len(text.encode())>12000: raise ValueError('Invalid text')
           if any(t in text for t in (BOS,EOS,'<｜User｜>','<｜Assistant｜>','<think>','</think>','｜DSML｜')): raise ValueError('Reserved native control marker in user content')
       roles=[m['role'] for m in messages if m['role']!='system']
       if not roles or roles!=['user' if i%2==0 else 'assistant' for i in range(len(roles))] or roles[-1]!='user': raise ValueError('Alternating user/assistant history ending in user required')
       return messages
   
   def complete(call,messages,mode):
       if mode not in ('thinking','chat'): raise ValueError('Invalid native mode')
       checked_messages(messages)
       prompt=call('/apply-template',{'messages':messages,'add_generation_prompt':True,'chat_template_kwargs':{'thinking':mode=='thinking'}}).get('prompt')
       if not isinstance(prompt,str) or not prompt.startswith(BOS) or prompt.count(BOS)!=1: raise ValueError('Exactly one native BOS required')
       expected='<think>' if mode=='thinking' else '</think>'
       if not prompt.endswith(expected): raise ValueError('Native mode prefix mismatch')
       ids=call('/tokenize',{'content':prompt,'add_special':False,'parse_special':True}).get('tokens')
       if not isinstance(ids,list) or not ids or any(type(x)is not int or x<0 for x in ids) or ids[0]!=0 or ids.count(0)!=1: raise ValueError('Invalid token IDs or duplicate BOS')
       if len(ids)+OUTPUT>CONTEXT: raise ValueError('Input plus complete reasoning/output reserve exceeds context')
       body={'prompt':ids,'n_predict':OUTPUT,'temperature':0.6,'top_p':0.95,'top_k':0,'min_p':0,'repeat_penalty':1,'stream':False,'cache_prompt':False,'return_tokens':True,'stop':[]}
       result=call('/completion',body)
       if result.get('stop_type')!='eos' or result.get('truncated') is not False: raise ValueError('Require actual native EOS without truncation; limit/word stop does not prove completion')
       n=result.get('tokens_predicted')
       if not positive(n) or n>OUTPUT or result.get('tokens_evaluated')!=len(ids): raise ValueError('Usage mismatch')
       text=result.get('content')
       if not isinstance(text,str) or not text.strip(): raise ValueError('Empty completion')
       if EOS in text:
           if not text.endswith(EOS) or text.count(EOS)!=1: raise ValueError('Unexpected EOS text')
           text=text[:-len(EOS)]
       if mode=='thinking':
           if text.count('</think>')!=1: raise ValueError('Require one closed reasoning block')
           reasoning,final=text.split('</think>',1)
           if not reasoning.strip() or not final.strip(): raise ValueError('Empty reasoning/final body')
           parsed={'reasoning_content':reasoning,'content':final}
       else:
           parsed={'content':text}
       if any(marker in parsed['content'] for marker in (BOS,EOS,'<｜User｜>','<｜Assistant｜>','<think>','</think>','<｜tool')): raise ValueError('Unexpected native control marker')
       return {'mode':mode,'inputTokens':len(ids),'outputCap':OUTPUT,'parsed':parsed,'nativeResponse':result}
   
   def http_call(key):
       def call(path,body=None):
           data=None if body is None else json.dumps(body).encode()
           req=urllib.request.Request(BASE+path,data=data,headers={'Authorization':'Bearer '+key,'Content-Type':'application/json'})
           with urllib.request.urlopen(req,timeout=7200) as r:
               if r.status!=200: raise ValueError('HTTP failure')
               raw=r.read(2000001)
           if len(raw)>2000000: raise ValueError('Response limit')
           obj=json.loads(raw)
           if not isinstance(obj,dict): raise ValueError('JSON object required')
           return obj
       return call
   
   def main():
       key=pathlib.Path('.api-key').read_text().strip()
       if len(key)!=64 or any(c not in '0123456789abcdef' for c in key): raise ValueError('Restore private service key')
       if hashlib.sha256(pathlib.Path('native-template.jinja').read_bytes()).hexdigest()!=TEMPLATE_SHA: raise ValueError('Native template checksum mismatch')
       call=http_call(key);call('/health')
       req=urllib.request.Request(BASE+'/completion',data=b'{}',headers={'Content-Type':'application/json'})
       try:
           with urllib.request.urlopen(req,timeout=30): raise ValueError('Unauthorized completion accepted')
       except urllib.error.HTTPError as e:
           if e.code!=401: raise
       history=[{'role':'user','content':'The community library is open weekdays 9am to 6pm, Saturday 10am to 4pm and closed Sunday. State the weekday opening time only.'}]
       first=complete(call,history,'chat')
       pathlib.Path('chat-response.json').write_text(json.dumps(first,indent=2)+'\n')
       history += [{'role':'assistant','content':first['parsed']['content']},{'role':'user','content':'Using the same supplied schedule, how many hours is the library open on Saturday, and is it open on Sunday? Explain briefly.'}]
       second=complete(call,history,'thinking')
       pathlib.Path('thinking-response.json').write_text(json.dumps(second,indent=2)+'\n')
       print(json.dumps({'chat':first,'thinking':second},ensure_ascii=False))
       print('Manually inspect supplied schedule facts and reasoning. Native EOS/closed thinking proves completion shape, not answer quality or speed.')
   if __name__=='__main__': main()
   PYCLIENT

   Expected result: Client saved without model execution. Native template bytes remain verbatim.

Start the model

1. 1. Serve the complete converted Exp model with native DSA

   Keep the process foreground and logs. The generated first split loads the full audited composition. Native Jinja override is explicit; raw token IDs avoid duplicate BOS. DSA indexer and main MLA remain active; flash fusion off is not DSA removal. No speculative MTP request is made, though tensors remain present.

   CommandCopy command

   set -e
   cd "$HOME/deepseek-v32-exp-hosting"
   export DEEPSEEK_EXP_SERVICE_KEY="$(cat .api-key)"
   ./llama.cpp/build/bin/llama-server -m "$(cat first-model-path)" --alias deepseek-v32-exp --jinja --chat-template-file native-template.jinja --host 127.0.0.1 --port 8080 --api-key-file .api-key --device none -ngl 0 -t 64 -c 8192 -np 1 -n 2048 --no-context-shift --flash-attn off --cache-type-k f16 --cache-type-v f16 -b 256 -ub 256 --special --temp 0.6 --top-p 0.95 --top-k 0 --min-p 0 --repeat-penalty 1

   Expected result: If successful, complete DEEPSEEK32 DSA initializes and authenticated loopback becomes ready. Architecture, split, tokenizer or resource errors require stopping; no author load or fit result exists.

Verify the setup

1. 1. Verify native ordinary chat, reasoning, history and actual completion

   Use a second shell on the same host. Cloud diagnostics run inside the VM with its local files/key. Health and unauthorized 401 are checked first. Both full conversations are rendered by native Exp Jinja, exact token IDs and budgets checked before generation. Require actual EOS, closed reasoning and a nonempty final body; human inspection assesses supplied schedule facts separately.

   CommandCopy command

   set -e
   cd "$HOME/deepseek-v32-exp-hosting"
   python3 reasoning-client.py

   Expected result: Both native modes complete with actual EOS, matching usage and no truncation. Inspect weekday/Saturday/Sunday facts in saved outputs; protocol shape is not semantic correctness, speed or general reasoning quality.

2. 2. Reach the private service through SSH

   From a trusted client terminal use an existing SSH key and the exact VM host. Keep the tunnel open; the supplied diagnostic runs inside the VM. A separate local client must securely use the matching service key. Do not expose the native inference port publicly.

   CommandCopy command

   ssh -i /path/to/existing-key.pem -N -L 8080:127.0.0.1:8080 ubuntu@YOUR_EXISTING_INSTANCE_HOST

   Expected result: Client loopback forwards to this VM loopback. Closing the tunnel does not stop server compute or storage billing.

Performance evidence

No compatible measurement available. Documented support and estimated memory do not establish response speed.

Limits and troubleshooting

• Documented source qualification only: no native dependencies imported, weights downloaded, conversion/build/model load/inference performed or fit/speed/quality measured.
• Exact Exp native checkpoint and 3698-byte Jinja are independently pinned. Ordinary V3.2 uses a different encoder; Speciale is a distinct checkpoint. Neither substitutes here.
• All 163 native shards total 689,483,049,129 bytes. Reader-produced F16/Q4 outputs have unknown actual sizes/digests until successful execution; all expert and DSA tensors must remain.
• The native 61 main layers plus appended NextN layer are retained by default conversion. Main branding 671B/37B-active differs from 685,355,329,792 total parameter metadata; routing activity is not resident weight size.
• The exact runtime includes Exp conversion and independent indexer RoPE handling. Native README records a past indexer RoPE bug; use the pinned corrected runtime rather than an older demo.
• Native config maximum positions 163,840 and tokenizer maximum 131,072 are source metadata, not tested serving limits. Author selects 8,192 combined tokens and 2,048 generated reasoning/final tokens; this is not the publisher full-reasoning recommendation.
• DSA indexer selects up to 2,048 prior positions with 64 heads of 128 dimensions. MLA/indexer cache and scratch require separate memory; no dense ordinary KV estimate or sparse speed claim is made.
• Only derived conversion tokenizer config changes add_bos_token to true for the converter assertion; native original and serving Jinja are unchanged. Raw serving tokenization add_special false and one BOS are explicitly checked.
• 512 GiB RAM and 4 TiB working disk are planning allocations, not proven peak bounds. Source weights, F16 intermediate, Q4 outputs and dependencies coexist. Stop on allocation/disk errors and preserve logs.
• Torch 2.11 official CPU wheel supports exact dtype conversion; resolved remaining conversion dependencies are reader-installed and logged, not a fully agent-executed environment or compatibility result.
• Private loopback plus Bearer key requires a trusted host; privileged local users remain outside this boundary. Public TLS, multiuser isolation, tools, native GPU/distribution, full context and speculative MTP remain separate open scopes; required guide total is null.

Native checksum, missing shard or generated architecture mismatch

Stop and preserve the failed check. Require all exact native assets and generated deepseek32 DSA metadata/tensors. Do not disable sparse attention or substitute an unrelated checkpoint.

Converter BOS assertion

Use the separate conversion input and recorded single flag change; never edit the pinned native tokenizer or template. Keep serving raw token IDs and verify exactly one BOS.

Memory, disk or native runtime failure

Stop. Planning capacity does not prove conversion or serving fit. Preserve logs and actual resource usage for a separately reviewed plan.

Length stop, open thinking or empty final body

Treat verification as failed. Require actual EOS and closed nonempty reasoning/final text; no silent truncation or fabricated answer-quality claim.

Unauthorized or unreachable service

Check exact foreground process, loopback address and matching reader key. Keep public inference ingress closed; cloud access uses SSH.

Stop the model

1. 1. Stop the native service

   Press Ctrl+C in the foreground server shell; close any SSH tunnel with Ctrl+C and unset the key in every shell. Confirm the intended listener is gone. Cloud billing requires the separate resource lifecycle step.

   CommandCopy command

   unset DEEPSEEK_EXP_SERVICE_KEY
   if curl --silent --max-time 3 http://127.0.0.1:8080/health >/dev/null; then echo "Endpoint still responds: stop intended process"; exit 1; else echo "Endpoint no longer responds"; fi

   Expected result: The intended loopback service no longer responds; cloud compute/storage may still bill.

Uninstall

1. 1. Remove dedicated walkthrough files

   Back up wanted results and logs and confirm the dedicated path before removal. Shared system build packages remain installed.

   CommandCopy command

   rm -r -- "$HOME/deepseek-v32-exp-hosting"

   Expected result: Dedicated native shards, derived conversion inputs, generated model files, dependency environment, client, key, results and build files are removed; shared system packages remain.

Shut down cloud resources

1. 1. End cloud resource billing

   In EC2 us-east-1 stop this exact VM for later reuse or terminate it. Review root-volume Delete on termination and remove unwanted detached volumes, snapshots and reserved/Elastic/public IP resources. Stopped-instance storage and retained resources may still bill. Check Bills/Cost Explorer for final usage.

   Expected result: The exact instance is stopped or terminated as intended, and unwanted retained billed resources are removed. Stopping the native service alone does not end compute billing.