Devstral 2 2512

Exact model and requirements

Artifact:

Bartowski (named derivative); Mistral AI (native model) / Devstral 2 123B Instruct 2512 complete Q4_K_M, both shards · checkpoint bartowski/mistralai_Devstral-2-123B-Instruct-2512-GGUF

Publisher revision:

19239829ba3b52ef2f05c99db3cc73486fda768b

Artifact digest:

No publisher whole-composition digest exists in the archived metadata; both complete shards have exact individual SHA256 checks.

Download size:

69.75 GiB

Memory requirement (unknown):

Unknown — No full peak measured for all 74,897,653,312 GGUF bytes, attention cache, loading/scratch/runtime and OS.

System RAM / disk:

Unknown / Unknown

Placement:

single device. One CPU host with one conversation slot. All model layers run on CPU; no GPU or disk-offload path is configured.

KV cache:

F16 K and V attention cache; actual process/cache allocation unmeasured.

Workload:

4,096 token context; Unknown; Unknown; concurrency 1; batch 1; cache unknown; repetitions Unknown.

Tokenizer:

Exact Bartowski derivative tokenizer with separately pinned native Devstral 2 Jinja template · 19239829ba3b52ef2f05c99db3cc73486fda768b

Sampling / reasoning:

Temperature 0.15 follows the exact native generation_config. Author neutralizes top-k/top-p/min-p/repetition filters with 0/1/0/1, selects 512 output tokens and EOS </s> stop; no tool execution or benchmark claim. Full coding conversation with explicit author system message replaces the template default Vibe prompt; user/assistant history uses exact native template structure and one BOS. The native 262,144-position capacity is reduced to 4,096 total including full system/history and up to 512 generated tokens; parsed special IDs prevent duplicate BOS. No no-thinking guarantee, tool protocol or generated code execution is claimed.

Runtime prerequisites:

Ubuntu 24.04 x86_64; See prerequisites above.

Official artifact · License

Modified MIT, not standard MIT or Apache. Preserve the Mistral AI attribution and permission notice in all copies/substantial portions. You are not authorized to exercise rights if your company or employer global consolidated monthly revenue exceeded USD 20 million (or equivalent) in the preceding month; this applies to model derivatives, modifications and combined works, including third-party derivatives. Obtain a separate written commercial grant from Mistral if required before use. The AS IS disclaimer applies. Private use and a paid cloud VM do not waive the condition. No terms acceptance or legal clearance is supplied.

• mistralai_Devstral-2-123B-Instruct-2512-Q4_K_M-00001-of-00002.gguf · 37.18 GiB · revision 19239829ba3b52ef2f05c99db3cc73486fda768b
• mistralai_Devstral-2-123B-Instruct-2512-Q4_K_M-00002-of-00002.gguf · 32.57 GiB · revision 19239829ba3b52ef2f05c99db3cc73486fda768b

• Both complete GGUF shards total 74,897,653,312 serialized bytes (more than 64 GiB). A first-shard-only load is not this model. This is a dense 123B model-card identity; native metadata includes FP8 weights and BF16 tensors, not MoE active-parameter storage.
• 128 GiB RAM and 160 GiB free disk are planning allocations, not measured minima or fit. F16 attention cache, full runtime/scratch/loading state and OS add unknown overhead. Local board/DIMM capacity must be checked; native FP8/fullprecision needs a separately qualified path.

Install the runtime

1. 1. Build the exact CPU runtime

   Build pinned llama.cpp v0.7.0 source. Its ancestor includes the publisher-required PR17945 attention-factor fix. Native Mistral3 architecture/CPU dispatch and split loading are source-qualified, but no build, inference or fit is preclaimed. Stop on any failure.

   CommandCopy command

   set -e
   sudo apt-get update
   sudo apt-get install -y build-essential cmake git curl ca-certificates openssl python3
   mkdir -p "$HOME/devstral2-hosting/models"
   cd "$HOME/devstral2-hosting"
   git clone https://github.com/ggml-org/llama.cpp.git
   git -C llama.cpp checkout --detach 620d2225cc436ab0b001e3200e590e3b286ef9ab
   cmake -S llama.cpp -B llama.cpp/build -DCMAKE_BUILD_TYPE=Release -DGGML_METAL=OFF -DGGML_NATIVE=ON
   cmake --build llama.cpp/build --config Release -j 4
   ./llama.cpp/build/bin/llama-server --version

   Expected result: Checkout and build succeed; server version identifies the pinned source. Stop on failure. No inference has been executed by the author.

Download the model

1. 1. Download both exact shards, native template and full terms

   Review the conditional license before downloading. Every exact shard must pass its byte count and SHA256. Keep both native split names together; no conversion, partial model or merge is used. Separately verify the exact original template and license. Stop on access or checksum failure.

   CommandCopy command

   set -e
   cd "$HOME/devstral2-hosting"
   curl --fail --location --retry 0 'https://huggingface.co/bartowski/mistralai_Devstral-2-123B-Instruct-2512-GGUF/resolve/19239829ba3b52ef2f05c99db3cc73486fda768b/mistralai_Devstral-2-123B-Instruct-2512-Q4_K_M/mistralai_Devstral-2-123B-Instruct-2512-Q4_K_M-00001-of-00002.gguf' -o 'models/mistralai_Devstral-2-123B-Instruct-2512-Q4_K_M-00001-of-00002.gguf'
   echo 'e42aeda9772d7b779e1b675d7d16de42af169593f2deccdc542424816a2f421d  models/mistralai_Devstral-2-123B-Instruct-2512-Q4_K_M-00001-of-00002.gguf' | sha256sum --check
   test "$(stat -c %s 'models/mistralai_Devstral-2-123B-Instruct-2512-Q4_K_M-00001-of-00002.gguf')" = 39925785920
   curl --fail --location --retry 0 'https://huggingface.co/bartowski/mistralai_Devstral-2-123B-Instruct-2512-GGUF/resolve/19239829ba3b52ef2f05c99db3cc73486fda768b/mistralai_Devstral-2-123B-Instruct-2512-Q4_K_M/mistralai_Devstral-2-123B-Instruct-2512-Q4_K_M-00002-of-00002.gguf' -o 'models/mistralai_Devstral-2-123B-Instruct-2512-Q4_K_M-00002-of-00002.gguf'
   echo 'e3a596050861f0683c29a8aff4f458339dd3e8c641eb99fea9efc534bad72f94  models/mistralai_Devstral-2-123B-Instruct-2512-Q4_K_M-00002-of-00002.gguf' | sha256sum --check
   test "$(stat -c %s 'models/mistralai_Devstral-2-123B-Instruct-2512-Q4_K_M-00002-of-00002.gguf')" = 34971867392
   curl --fail --location --retry 0 'https://huggingface.co/mistralai/Devstral-2-123B-Instruct-2512/resolve/1613bf01adb5e1c6fdc196b46e6b173eae75eb4a/chat_template.jinja' -o models/devstral-native.jinja
   echo '28cac7f6cb901a70a216cc4e1d13dd7e6f105ccfe031fd7cbfbdfe907d91dc32  models/devstral-native.jinja' | sha256sum --check
   curl --fail --location --retry 0 'https://huggingface.co/mistralai/Devstral-2-123B-Instruct-2512/resolve/1613bf01adb5e1c6fdc196b46e6b173eae75eb4a/LICENSE' -o models/LICENSE
   echo '867daee3b68e4e9392c9767fec4f099096fa6ad67adfc9c24020494e97766778  models/LICENSE' | sha256sum --check

   Expected result: Both full shard hashes/sizes, exact native Jinja and license hash pass.

Configure

1. 1. Create a private service credential

   Generate a local server password on your own host when following this guide. Preserve an existing key on repeat setup and never print or share it. This is not an provider API key.

   CommandCopy command

   set -e
   cd "$HOME/devstral2-hosting"
   umask 077
   if [ ! -e .api-key ]; then
     openssl rand -hex 32 > .api-key
   fi
   test -s .api-key || { echo 'Restore the empty service-key file before continuing.'; exit 1; }
   chmod 600 .api-key

   Expected result: A nonempty private .api-key exists with mode 600. Existing key preserved; no secret printed.

2. 2. Open an SSH tunnel from your client

   On your client replace the placeholders with an existing SSH key and this VM address. Keep the server bound to VM loopback; run verification in the SSH shell or forward localhost 8080 for your local client.

   CommandCopy command

   set -e
   ssh -i /path/to/existing-key.pem -N -L 8080:127.0.0.1:8080 ubuntu@YOUR_INSTANCE_ADDRESS

   Expected result: Tunnel remains open while the SSH session runs. Client localhost:8080 reaches only the selected VM loopback service.

Start the model

1. 1. Start the full private Devstral 2 CPU model

   Run foreground. The first exact split filename discovers and requires the second. Native Jinja is supplied verbatim; the client supplies a truthful coding-assistant system message instead of the default Vibe identity. One slot has 4,096 total context tokens; no tools, public binding or generated-code executor is exposed.

   CommandCopy command

   set -e
   cd "$HOME/devstral2-hosting"
   ./llama.cpp/build/bin/llama-server -m 'models/mistralai_Devstral-2-123B-Instruct-2512-Q4_K_M-00001-of-00002.gguf' --alias devstral2 --host 127.0.0.1 --port 8080 --api-key-file .api-key --jinja --chat-template-file models/devstral-native.jinja -c 4096 -np 1 -t 16 -ngl 0 --cache-type-k f16 --cache-type-v f16

   Expected result: Both complete shards load and private loopback becomes ready, or an explicit architecture/shard/template/resource failure stops the setup. No successful load or fit is asserted.

Verify the setup

1. 1. Check readiness, authentication and the complete coding conversation

   Run on the server in a second terminal. The original client applies the native template, counts all parsed prompt IDs with no added BOS, enforces 3,584 input / 512 output within 4,096 total, and rejects truncation or abnormal stops. It asks for an empty-list-safe Python function and a follow-up; manually inspect behavior for [] and [7]. The reader does not execute any generated code, shell command or tool call. Correct shape does not prove code correctness.

   CommandCopy command

   set -e
   cd "$HOME/devstral2-hosting"
   python3 - <<'PYHEALTH'
   import pathlib,urllib.request,urllib.error,json
   key=pathlib.Path('.api-key').read_text().strip()
   with urllib.request.urlopen(urllib.request.Request('http://127.0.0.1:8080/health',headers={'Authorization':'Bearer '+key}),timeout=30) as response: assert json.load(response)['status']=='ok'
   try: urllib.request.urlopen('http://127.0.0.1:8080/apply-template',timeout=30)
   except urllib.error.HTTPError as e: assert e.code==401
   else: raise RuntimeError('Authentication was not enforced')
   PYHEALTH
   cat > coding-client.py <<'PYCLIENT'
   """Reader-run Devstral 2 coding conversation; never executes generated code check."""
   import json,pathlib,urllib.request
   
   def check_messages(messages):
       if not isinstance(messages,list) or not messages or len(messages)%2!=1: raise ValueError('user_ending_conversation_required')
       for i,m in enumerate(messages):
           if not isinstance(m,dict) or set(m)!={'role','content'} or m['role']!=('user' if i%2==0 else 'assistant') or not isinstance(m['content'],str) or not m['content'].strip() or len(m['content'].encode())>8192 or any(t in m['content'] for t in ['<s>','</s>','[INST]','[/INST]','[SYSTEM_PROMPT]','[/SYSTEM_PROMPT]','[TOOL_CALLS]','[AVAILABLE_TOOLS]']):
               raise ValueError('invalid_plain_text_conversation')
   
   def run_case(messages,post):
       check_messages(messages)
       system={'role':'system','content':'You are a coding assistant. Explain and propose code in plain text. No tools or execution are available. Do not claim that code was run.'}
       prompt=post('/apply-template',{'messages':[system]+messages})['prompt']
       if not isinstance(prompt,str) or not prompt.startswith('<s>[SYSTEM_PROMPT]') or prompt.count('<s>')!=1 or not prompt.endswith('[/INST]'):
           raise ValueError('native_template_mismatch')
       ids=post('/tokenize',{'content':prompt,'add_special':False,'parse_special':True})['tokens']
       if not isinstance(ids,list) or not ids or any(type(i)!=int or i<0 for i in ids) or len(ids)>3584: raise ValueError('prompt_budget_or_token_shape')
       result=post('/completion',{'prompt':ids,'n_predict':512,'temperature':0.15,'top_k':0,'top_p':1.0,'min_p':0.0,'repeat_penalty':1.0,'stop':['</s>'],'return_tokens':True,'cache_prompt':False})
       if result.get('stop_type') not in ('eos','word') or result.get('truncated') is not False: raise ValueError('unfinished_or_truncated_response')
       tokens=result.get('tokens');text=result.get('content')
       if not isinstance(tokens,list) or not 1<=len(tokens)<=512 or any(type(i)!=int or i<0 for i in tokens) or not isinstance(text,str) or not text.strip() or any(t in text for t in ['<s>','</s>','[INST]','[/INST]','[TOOL_CALLS]']): raise ValueError('response_shape')
       return {'prompt_tokens':len(ids),'generated_tokens':len(tokens),'response':text}
   
   def main():
       key=pathlib.Path('.api-key').read_text().strip()
       def post(path,data):
           req=urllib.request.Request('http://127.0.0.1:8080'+path,json.dumps(data).encode(),headers={'Authorization':'Bearer '+key,'Content-Type':'application/json'})
           with urllib.request.urlopen(req,timeout=600) as response:return json.load(response)
       first=[{'role':'user','content':'Review this Python function: def first(xs): return xs[0]. Explain its empty-list bug and propose a version returning None for an empty list. Do not execute any code.'}]
       answer=run_case(first,post);print(json.dumps(answer,ensure_ascii=False))
       follow=first+[{'role':'assistant','content':answer['response']},{'role':'user','content':'Now explain how your proposed function behaves for [7] and for an empty list, and identify one test case. Do not run code.'}]
       print(json.dumps(run_case(follow,post),ensure_ascii=False))
       print('Manually inspect both answers: empty list must return None, [7] must return 7, and the follow-up should use its proposed function. Never execute generated code. Shape and stop checks do not establish coding correctness, benchmark performance or agentic tool quality.')
   if __name__=='__main__':main()
   PYCLIENT
   echo '8794c4ed60f8b206a1fc9aed28ed0b41551359266fd4e44bbef607dc50352391  coding-client.py' | sha256sum --check
   python3 coding-client.py

   Expected result: Authenticated readiness passes, unauthenticated protected route rejects, and two complete nontruncated responses can be manually reviewed. Any control, budget, access or response error fails; no coding-quality or tool-use result is preclaimed.

Performance evidence

No compatible measurement available. Documented support and estimated memory do not establish response speed.

Limits and troubleshooting

• Native Devstral 2 card identifies a dense 123B FP8 instruction model with 256K context; exact config has 88 layers, hidden size 12,288, eight KV heads and 262,144 positions. This guide uses a separately published mixed Q4_K_M derivative, not native weight-byte equivalence.
• The complete named derivative has two shards totaling 74,897,653,312 bytes with separate LFS SHA256 identities. Full process peak and 128 GiB host fit are unknown. Do not substitute one shard or a 64 GiB allocation.
• Exact native Jinja includes tools and a default Mistral Vibe system message. The original file remains unchanged; an explicit author system message accurately describes this no-tools coding-conversation setup. No shell executor, Vibe integration or agentic benchmark result is claimed.
• Publisher explicitly names the community derivative and requires llama.cpp PR17945. Pinned runtime is a descendant and has Mistral3 native CPU graph/conversion mapping; architecture support is source evidence, not an executed compatibility test.
• Quantizer card explicitly warns Vibe/chained tool calling may fail and identifies FP8 conversion work and its quantization release. The exact input base checkpoint commit and native byte-equivalence are not asserted. Native FP8, other backends, full context and actual tool workflows remain open.
• The reduced 4,096 context / 512 output cap and neutralized secondary sampler filters are author choices. All system/history/control-token IDs count toward the bound; no hidden no-thinking or measured speed/fit assumption.
• Modified MIT revenue condition applies to company/employer and model derivatives, modifications and combined works. Private use and cloud rental do not waive it. Preserve the grant and obtain any required separate written permission before exercising rights.
• Local Intel eight-thread and cloud R7i.4xlarge sixteen-vCPU / 128 GiB / 160 GiB allocations are planning configurations. Verify actual local motherboard/DIMM/OS capacity. No memory, speed, quality, SWE-bench or total task-cost measurement exists.

Architecture, split-file, template or resource error

Stop, retain the error and verify the full pinned runtime, both names/hashes/sizes and original template. Do not merge/substitute shards or relax checks. A valid source graph does not prove actual fit.

Unexpected markers, abnormal stop, truncation or incoherent follow-up

Reject the result. Verify native template and complete prompt budget; inspect the proposed function manually. Never execute generated code to imply a quality result, and do not enable tools or claim Vibe compatibility as a workaround.

Connection/authentication failure

Use the private server terminal or documented existing SSH tunnel; confirm the reader key and loopback port. Keep public 8080 closed and check process logs.

Stop the model

1. 1. Stop the CPU backend

   Press Ctrl+C in the foreground server terminal. Unset the private key in each shell and close any SSH tunnel. This does not stop compute or storage billing.

   CommandCopy command

   unset COMMAND_SERVICE_KEY

   Expected result: The backend stops serving loopback port 8080.

Uninstall

1. 1. Remove the dedicated walkthrough files

   Back up wanted outputs, confirm the exact path and remove only this dedicated directory. Shared system packages remain.

   CommandCopy command

   rm -r -- "$HOME/devstral2-hosting"

   Expected result: Dedicated complete weights, build, native metadata/template, client, private key and outputs removed.

Shut down cloud resources

1. 1. End cloud billing resources

   In EC2 us-east-1, stop for later reuse or terminate this exact VM. Review Delete on termination for its root volume. Delete unwanted detached EBS volumes, snapshots and reserved/Elastic/public IP resources. Stopped VM storage and retained resources may still bill. Check Bills/Cost Explorer later for final usage.

   Expected result: Exact instance is stopped/terminated as intended; no unwanted attached/retained billed resources remain. Instance compute stops only after its lifecycle transition, not when llama-server stops.

Exact model and requirements

Artifact:

Bartowski (named derivative); Mistral AI (native model) / Devstral 2 123B Instruct 2512 complete Q4_K_M, both shards · checkpoint bartowski/mistralai_Devstral-2-123B-Instruct-2512-GGUF

Publisher revision:

19239829ba3b52ef2f05c99db3cc73486fda768b

Artifact digest:

No publisher whole-composition digest exists in the archived metadata; both complete shards have exact individual SHA256 checks.

Download size:

69.75 GiB

Memory requirement (unknown):

Unknown — No full peak measured for all 74,897,653,312 GGUF bytes, attention cache, loading/scratch/runtime and OS.

System RAM / disk:

Unknown / Unknown

Placement:

single device. One CPU host with one conversation slot. All model layers run on CPU; no GPU or disk-offload path is configured.

KV cache:

F16 K and V attention cache; actual process/cache allocation unmeasured.

Workload:

4,096 token context; Unknown; Unknown; concurrency 1; batch 1; cache unknown; repetitions Unknown.

Tokenizer:

Exact Bartowski derivative tokenizer with separately pinned native Devstral 2 Jinja template · 19239829ba3b52ef2f05c99db3cc73486fda768b

Sampling / reasoning:

Temperature 0.15 follows the exact native generation_config. Author neutralizes top-k/top-p/min-p/repetition filters with 0/1/0/1, selects 512 output tokens and EOS </s> stop; no tool execution or benchmark claim. Full coding conversation with explicit author system message replaces the template default Vibe prompt; user/assistant history uses exact native template structure and one BOS. The native 262,144-position capacity is reduced to 4,096 total including full system/history and up to 512 generated tokens; parsed special IDs prevent duplicate BOS. No no-thinking guarantee, tool protocol or generated code execution is claimed.

Runtime prerequisites:

Ubuntu 24.04 x86_64; See prerequisites above.

Official artifact · License

Modified MIT, not standard MIT or Apache. Preserve the Mistral AI attribution and permission notice in all copies/substantial portions. You are not authorized to exercise rights if your company or employer global consolidated monthly revenue exceeded USD 20 million (or equivalent) in the preceding month; this applies to model derivatives, modifications and combined works, including third-party derivatives. Obtain a separate written commercial grant from Mistral if required before use. The AS IS disclaimer applies. Private use and a paid cloud VM do not waive the condition. No terms acceptance or legal clearance is supplied.

• mistralai_Devstral-2-123B-Instruct-2512-Q4_K_M-00001-of-00002.gguf · 37.18 GiB · revision 19239829ba3b52ef2f05c99db3cc73486fda768b
• mistralai_Devstral-2-123B-Instruct-2512-Q4_K_M-00002-of-00002.gguf · 32.57 GiB · revision 19239829ba3b52ef2f05c99db3cc73486fda768b

• Both complete GGUF shards total 74,897,653,312 serialized bytes (more than 64 GiB). A first-shard-only load is not this model. This is a dense 123B model-card identity; native metadata includes FP8 weights and BF16 tensors, not MoE active-parameter storage.
• 128 GiB RAM and 160 GiB free disk are planning allocations, not measured minima or fit. F16 attention cache, full runtime/scratch/loading state and OS add unknown overhead. Local board/DIMM capacity must be checked; native FP8/fullprecision needs a separately qualified path.

Install the runtime

1. 1. Build the exact CPU runtime

   Build pinned llama.cpp v0.7.0 source. Its ancestor includes the publisher-required PR17945 attention-factor fix. Native Mistral3 architecture/CPU dispatch and split loading are source-qualified, but no build, inference or fit is preclaimed. Stop on any failure.

   CommandCopy command

   set -e
   sudo apt-get update
   sudo apt-get install -y build-essential cmake git curl ca-certificates openssl python3
   mkdir -p "$HOME/devstral2-hosting/models"
   cd "$HOME/devstral2-hosting"
   git clone https://github.com/ggml-org/llama.cpp.git
   git -C llama.cpp checkout --detach 620d2225cc436ab0b001e3200e590e3b286ef9ab
   cmake -S llama.cpp -B llama.cpp/build -DCMAKE_BUILD_TYPE=Release -DGGML_METAL=OFF -DGGML_NATIVE=ON
   cmake --build llama.cpp/build --config Release -j 4
   ./llama.cpp/build/bin/llama-server --version

   Expected result: Checkout and build succeed; server version identifies the pinned source. Stop on failure. No inference has been executed by the author.

Download the model

1. 1. Download both exact shards, native template and full terms

   Review the conditional license before downloading. Every exact shard must pass its byte count and SHA256. Keep both native split names together; no conversion, partial model or merge is used. Separately verify the exact original template and license. Stop on access or checksum failure.

   CommandCopy command

   set -e
   cd "$HOME/devstral2-hosting"
   curl --fail --location --retry 0 'https://huggingface.co/bartowski/mistralai_Devstral-2-123B-Instruct-2512-GGUF/resolve/19239829ba3b52ef2f05c99db3cc73486fda768b/mistralai_Devstral-2-123B-Instruct-2512-Q4_K_M/mistralai_Devstral-2-123B-Instruct-2512-Q4_K_M-00001-of-00002.gguf' -o 'models/mistralai_Devstral-2-123B-Instruct-2512-Q4_K_M-00001-of-00002.gguf'
   echo 'e42aeda9772d7b779e1b675d7d16de42af169593f2deccdc542424816a2f421d  models/mistralai_Devstral-2-123B-Instruct-2512-Q4_K_M-00001-of-00002.gguf' | sha256sum --check
   test "$(stat -c %s 'models/mistralai_Devstral-2-123B-Instruct-2512-Q4_K_M-00001-of-00002.gguf')" = 39925785920
   curl --fail --location --retry 0 'https://huggingface.co/bartowski/mistralai_Devstral-2-123B-Instruct-2512-GGUF/resolve/19239829ba3b52ef2f05c99db3cc73486fda768b/mistralai_Devstral-2-123B-Instruct-2512-Q4_K_M/mistralai_Devstral-2-123B-Instruct-2512-Q4_K_M-00002-of-00002.gguf' -o 'models/mistralai_Devstral-2-123B-Instruct-2512-Q4_K_M-00002-of-00002.gguf'
   echo 'e3a596050861f0683c29a8aff4f458339dd3e8c641eb99fea9efc534bad72f94  models/mistralai_Devstral-2-123B-Instruct-2512-Q4_K_M-00002-of-00002.gguf' | sha256sum --check
   test "$(stat -c %s 'models/mistralai_Devstral-2-123B-Instruct-2512-Q4_K_M-00002-of-00002.gguf')" = 34971867392
   curl --fail --location --retry 0 'https://huggingface.co/mistralai/Devstral-2-123B-Instruct-2512/resolve/1613bf01adb5e1c6fdc196b46e6b173eae75eb4a/chat_template.jinja' -o models/devstral-native.jinja
   echo '28cac7f6cb901a70a216cc4e1d13dd7e6f105ccfe031fd7cbfbdfe907d91dc32  models/devstral-native.jinja' | sha256sum --check
   curl --fail --location --retry 0 'https://huggingface.co/mistralai/Devstral-2-123B-Instruct-2512/resolve/1613bf01adb5e1c6fdc196b46e6b173eae75eb4a/LICENSE' -o models/LICENSE
   echo '867daee3b68e4e9392c9767fec4f099096fa6ad67adfc9c24020494e97766778  models/LICENSE' | sha256sum --check

   Expected result: Both full shard hashes/sizes, exact native Jinja and license hash pass.

Configure

1. 1. Create a private service credential

   Generate a local server password on your own host when following this guide. Preserve an existing key on repeat setup and never print or share it. This is not an provider API key.

   CommandCopy command

   set -e
   cd "$HOME/devstral2-hosting"
   umask 077
   if [ ! -e .api-key ]; then
     openssl rand -hex 32 > .api-key
   fi
   test -s .api-key || { echo 'Restore the empty service-key file before continuing.'; exit 1; }
   chmod 600 .api-key

   Expected result: A nonempty private .api-key exists with mode 600. Existing key preserved; no secret printed.

Start the model

1. 1. Start the full private Devstral 2 CPU model

   Run foreground. The first exact split filename discovers and requires the second. Native Jinja is supplied verbatim; the client supplies a truthful coding-assistant system message instead of the default Vibe identity. One slot has 4,096 total context tokens; no tools, public binding or generated-code executor is exposed.

   CommandCopy command

   set -e
   cd "$HOME/devstral2-hosting"
   ./llama.cpp/build/bin/llama-server -m 'models/mistralai_Devstral-2-123B-Instruct-2512-Q4_K_M-00001-of-00002.gguf' --alias devstral2 --host 127.0.0.1 --port 8080 --api-key-file .api-key --jinja --chat-template-file models/devstral-native.jinja -c 4096 -np 1 -t 8 -ngl 0 --cache-type-k f16 --cache-type-v f16

   Expected result: Both complete shards load and private loopback becomes ready, or an explicit architecture/shard/template/resource failure stops the setup. No successful load or fit is asserted.

Verify the setup

1. 1. Check readiness, authentication and the complete coding conversation

   Run on the server in a second terminal. The original client applies the native template, counts all parsed prompt IDs with no added BOS, enforces 3,584 input / 512 output within 4,096 total, and rejects truncation or abnormal stops. It asks for an empty-list-safe Python function and a follow-up; manually inspect behavior for [] and [7]. The reader does not execute any generated code, shell command or tool call. Correct shape does not prove code correctness.

   CommandCopy command

   set -e
   cd "$HOME/devstral2-hosting"
   python3 - <<'PYHEALTH'
   import pathlib,urllib.request,urllib.error,json
   key=pathlib.Path('.api-key').read_text().strip()
   with urllib.request.urlopen(urllib.request.Request('http://127.0.0.1:8080/health',headers={'Authorization':'Bearer '+key}),timeout=30) as response: assert json.load(response)['status']=='ok'
   try: urllib.request.urlopen('http://127.0.0.1:8080/apply-template',timeout=30)
   except urllib.error.HTTPError as e: assert e.code==401
   else: raise RuntimeError('Authentication was not enforced')
   PYHEALTH
   cat > coding-client.py <<'PYCLIENT'
   """Reader-run Devstral 2 coding conversation; never executes generated code check."""
   import json,pathlib,urllib.request
   
   def check_messages(messages):
       if not isinstance(messages,list) or not messages or len(messages)%2!=1: raise ValueError('user_ending_conversation_required')
       for i,m in enumerate(messages):
           if not isinstance(m,dict) or set(m)!={'role','content'} or m['role']!=('user' if i%2==0 else 'assistant') or not isinstance(m['content'],str) or not m['content'].strip() or len(m['content'].encode())>8192 or any(t in m['content'] for t in ['<s>','</s>','[INST]','[/INST]','[SYSTEM_PROMPT]','[/SYSTEM_PROMPT]','[TOOL_CALLS]','[AVAILABLE_TOOLS]']):
               raise ValueError('invalid_plain_text_conversation')
   
   def run_case(messages,post):
       check_messages(messages)
       system={'role':'system','content':'You are a coding assistant. Explain and propose code in plain text. No tools or execution are available. Do not claim that code was run.'}
       prompt=post('/apply-template',{'messages':[system]+messages})['prompt']
       if not isinstance(prompt,str) or not prompt.startswith('<s>[SYSTEM_PROMPT]') or prompt.count('<s>')!=1 or not prompt.endswith('[/INST]'):
           raise ValueError('native_template_mismatch')
       ids=post('/tokenize',{'content':prompt,'add_special':False,'parse_special':True})['tokens']
       if not isinstance(ids,list) or not ids or any(type(i)!=int or i<0 for i in ids) or len(ids)>3584: raise ValueError('prompt_budget_or_token_shape')
       result=post('/completion',{'prompt':ids,'n_predict':512,'temperature':0.15,'top_k':0,'top_p':1.0,'min_p':0.0,'repeat_penalty':1.0,'stop':['</s>'],'return_tokens':True,'cache_prompt':False})
       if result.get('stop_type') not in ('eos','word') or result.get('truncated') is not False: raise ValueError('unfinished_or_truncated_response')
       tokens=result.get('tokens');text=result.get('content')
       if not isinstance(tokens,list) or not 1<=len(tokens)<=512 or any(type(i)!=int or i<0 for i in tokens) or not isinstance(text,str) or not text.strip() or any(t in text for t in ['<s>','</s>','[INST]','[/INST]','[TOOL_CALLS]']): raise ValueError('response_shape')
       return {'prompt_tokens':len(ids),'generated_tokens':len(tokens),'response':text}
   
   def main():
       key=pathlib.Path('.api-key').read_text().strip()
       def post(path,data):
           req=urllib.request.Request('http://127.0.0.1:8080'+path,json.dumps(data).encode(),headers={'Authorization':'Bearer '+key,'Content-Type':'application/json'})
           with urllib.request.urlopen(req,timeout=600) as response:return json.load(response)
       first=[{'role':'user','content':'Review this Python function: def first(xs): return xs[0]. Explain its empty-list bug and propose a version returning None for an empty list. Do not execute any code.'}]
       answer=run_case(first,post);print(json.dumps(answer,ensure_ascii=False))
       follow=first+[{'role':'assistant','content':answer['response']},{'role':'user','content':'Now explain how your proposed function behaves for [7] and for an empty list, and identify one test case. Do not run code.'}]
       print(json.dumps(run_case(follow,post),ensure_ascii=False))
       print('Manually inspect both answers: empty list must return None, [7] must return 7, and the follow-up should use its proposed function. Never execute generated code. Shape and stop checks do not establish coding correctness, benchmark performance or agentic tool quality.')
   if __name__=='__main__':main()
   PYCLIENT
   echo '8794c4ed60f8b206a1fc9aed28ed0b41551359266fd4e44bbef607dc50352391  coding-client.py' | sha256sum --check
   python3 coding-client.py

   Expected result: Authenticated readiness passes, unauthenticated protected route rejects, and two complete nontruncated responses can be manually reviewed. Any control, budget, access or response error fails; no coding-quality or tool-use result is preclaimed.

Performance evidence

No compatible measurement available. Documented support and estimated memory do not establish response speed.

Limits and troubleshooting

• Native Devstral 2 card identifies a dense 123B FP8 instruction model with 256K context; exact config has 88 layers, hidden size 12,288, eight KV heads and 262,144 positions. This guide uses a separately published mixed Q4_K_M derivative, not native weight-byte equivalence.
• The complete named derivative has two shards totaling 74,897,653,312 bytes with separate LFS SHA256 identities. Full process peak and 128 GiB host fit are unknown. Do not substitute one shard or a 64 GiB allocation.
• Exact native Jinja includes tools and a default Mistral Vibe system message. The original file remains unchanged; an explicit author system message accurately describes this no-tools coding-conversation setup. No shell executor, Vibe integration or agentic benchmark result is claimed.
• Publisher explicitly names the community derivative and requires llama.cpp PR17945. Pinned runtime is a descendant and has Mistral3 native CPU graph/conversion mapping; architecture support is source evidence, not an executed compatibility test.
• Quantizer card explicitly warns Vibe/chained tool calling may fail and identifies FP8 conversion work and its quantization release. The exact input base checkpoint commit and native byte-equivalence are not asserted. Native FP8, other backends, full context and actual tool workflows remain open.
• The reduced 4,096 context / 512 output cap and neutralized secondary sampler filters are author choices. All system/history/control-token IDs count toward the bound; no hidden no-thinking or measured speed/fit assumption.
• Modified MIT revenue condition applies to company/employer and model derivatives, modifications and combined works. Private use and cloud rental do not waive it. Preserve the grant and obtain any required separate written permission before exercising rights.
• Local Intel eight-thread and cloud R7i.4xlarge sixteen-vCPU / 128 GiB / 160 GiB allocations are planning configurations. Verify actual local motherboard/DIMM/OS capacity. No memory, speed, quality, SWE-bench or total task-cost measurement exists.

Architecture, split-file, template or resource error

Stop, retain the error and verify the full pinned runtime, both names/hashes/sizes and original template. Do not merge/substitute shards or relax checks. A valid source graph does not prove actual fit.

Unexpected markers, abnormal stop, truncation or incoherent follow-up

Reject the result. Verify native template and complete prompt budget; inspect the proposed function manually. Never execute generated code to imply a quality result, and do not enable tools or claim Vibe compatibility as a workaround.

Connection/authentication failure

Confirm the foreground process, loopback address and reader key on this trusted host.

Stop the model

1. 1. Stop the CPU backend

   Press Ctrl+C in the foreground server terminal. Unset the private key in each shell and close any SSH tunnel. Verify loopback port 8080 closes.

   CommandCopy command

   unset COMMAND_SERVICE_KEY

   Expected result: The backend stops serving loopback port 8080.

Uninstall

1. 1. Remove the dedicated walkthrough files

   Back up wanted outputs, confirm the exact path and remove only this dedicated directory. Shared system packages remain.

   CommandCopy command

   rm -r -- "$HOME/devstral2-hosting"

   Expected result: Dedicated complete weights, build, native metadata/template, client, private key and outputs removed.