Mistral Small 4

Exact model and requirements

Artifact:

Unsloth (named GGUF derivative); Mistral AI (native multimodal model) / Complete Mistral Small 4 119B UD-Q4_K_M, three split files plus F16 Pixtral projector · checkpoint unsloth/Mistral-Small-4-119B-2603-GGUF

Publisher revision:

bd93c721735aa32c035c0f19e738cb3371fd56ff

Artifact digest:

No publisher single digest for the complete three-part model composition. Every individual part and the projector has its own checked SHA256.

Download size:

68.7 GiB

Memory requirement (unknown):

Unknown — No complete CPU peak measurement for the complete language splits, MLA cache, Pixtral tensors, scratch, loading and OS.

System RAM / disk:

Unknown / Unknown

Placement:

single device. One CPU host; all three GGUF disk splits form one complete model, not tensor/pipeline/device sharding. CPU language graph and full Pixtral image encoder/projector, with both GPU layer and projector offload disabled.

KV cache:

F16 cache flags are explicit; native MLA implementation determines actual cache layout. Cache bytes and process peak remain unknown.

Workload:

4,096 token context; Unknown; Unknown; concurrency 1; batch 1; cache unknown; repetitions Unknown.

Tokenizer:

Exact complete Unsloth GGUF tokenizer and independently pinned native Mistral Small 4 Jinja template · bd93c721735aa32c035c0f19e738cb3371fd56ff; native template a11f36bebf709121056b1dbcc943d1c6afbe494d

Sampling / reasoning:

Temperature zero is within the publisher none-mode range 0.0 to 0.7. Top_k 0, top_p 1, min_p 0, presence_penalty 0 and repeat_penalty 1 are author choices. The client requires a nonempty stop-finished answer and valid actual usage, not a prescribed generated result. Native reasoning_effort none is passed as a Jinja template keyword and produces the native MODEL_SETTINGS none prefix. The exact template emits native BOS and image boundaries. This is complete text plus image input with text output; reasoning high, tool calls, speculative Eagle, native FP8/NVFP4, GPU and long-context scopes remain open.

Runtime prerequisites:

Ubuntu 24.04 x86_64; See prerequisites above.

Official artifact · License

Mistral AI explicitly grants Apache License 2.0 in its pinned native model card and metadata; Unsloth declares Apache-2.0 and names this exact native base repository. Read the linked full Apache text and retain required notices, license, attribution and modification notices for covered redistribution. The native root LICENSE URL returns 404; the explicit model-card grant is preserved rather than inventing that file. Public metadata indicates ungated access; no terms acceptance or legal clearance is performed for the reader.

• Mistral-Small-4-119B-2603-UD-Q4_K_M-00001-of-00003.gguf · 0.01 GiB · revision bd93c721735aa32c035c0f19e738cb3371fd56ff
• Mistral-Small-4-119B-2603-UD-Q4_K_M-00002-of-00003.gguf · 45.91 GiB · revision bd93c721735aa32c035c0f19e738cb3371fd56ff
• Mistral-Small-4-119B-2603-UD-Q4_K_M-00003-of-00003.gguf · 22.78 GiB · revision bd93c721735aa32c035c0f19e738cb3371fd56ff
• mmproj-F16.gguf · 0.8 GiB · revision bd93c721735aa32c035c0f19e738cb3371fd56ff
• mistral-small4-chat-template.jinja · 0 GiB · revision a11f36bebf709121056b1dbcc943d1c6afbe494d

• All three language parts total 73,763,180,544 bytes; the F16 projector adds 857,078,496 bytes, for 74,620,259,040 artifact bytes before template, fixture and build files. The tiny first split alone is not the complete model.
• The publisher labels 119B total parameters and 6.5B active per token. Native metadata reports 119,401,317,952 parameters across F8 and BF16 tensors. Active compute does not imply only active weights are resident; the complete model must load.
• The 36-layer native MISTRAL4 model has MLA, 128 routed experts, four selected experts per token and one shared expert. This is not a dense-model KV memory estimate. Actual MLA cache, quantized weight placement, vision preprocessing, 2,048-token batch/ubatch buffers and warmup peaks remain unknown.
• Local eight threads / cloud sixteen vCPUs, 128 GiB RAM and 160 GiB disk are author allocations. Verify actual local DIMM/motherboard/OS capacity. No wall power, fit, latency, throughput or quality result is claimed.

Install the runtime

1. 1. Build the exact CPU runtime

   Build exact official llama.cpp source on this host. Native CPU architecture mapping includes MISTRAL4 MLA/MoE and full Pixtral processing. Build dependencies resolve at execution time; record resolved package/tool versions and logs. No build or model inference was run by the author.

   CommandCopy command

   set -e
   sudo apt-get update
   sudo apt-get install -y build-essential cmake git curl ca-certificates openssl python3
   mkdir -p "$HOME/mistral-small4-vision-hosting/models"
   cd "$HOME/mistral-small4-vision-hosting"
   git clone https://github.com/ggml-org/llama.cpp.git
   git -C llama.cpp checkout --detach 620d2225cc436ab0b001e3200e590e3b286ef9ab
   cmake -S llama.cpp -B llama.cpp/build -DCMAKE_BUILD_TYPE=Release -DGGML_METAL=OFF -DGGML_NATIVE=ON
   cmake --build llama.cpp/build --config Release -j 8
   ./llama.cpp/build/bin/llama-server --version

   Expected result: Checkout and build succeed; server version identifies the pinned source. Stop on failure. No inference has been executed by the author.

Download the model

1. 1. Download and verify the complete model, projector, template and fixture

   All three split files and the exact F16 projector are mandatory. Stop on any HTTP, byte-count or checksum failure. SHA tools stream the files; no whole-weight Python read or conversion is used. Preserve filenames together. Template and quantizer are independent pins; Unsloth advertises embedded template fixes, while this guide explicitly overrides with the exact native template and requires functional checks.

   CommandCopy command

   set -e
   cd "$HOME/mistral-small4-vision-hosting"
   curl --fail --location --retry 0 'https://huggingface.co/unsloth/Mistral-Small-4-119B-2603-GGUF/resolve/bd93c721735aa32c035c0f19e738cb3371fd56ff/UD-Q4_K_M/Mistral-Small-4-119B-2603-UD-Q4_K_M-00001-of-00003.gguf' -o 'models/Mistral-Small-4-119B-2603-UD-Q4_K_M-00001-of-00003.gguf'
   echo 'fd3cc46082e4e64eb623eea4611b02583d1de4c20a59411bf1820925d5117dfe  models/Mistral-Small-4-119B-2603-UD-Q4_K_M-00001-of-00003.gguf' | sha256sum --check
   test "$(stat -c %s 'models/Mistral-Small-4-119B-2603-UD-Q4_K_M-00001-of-00003.gguf')" = 7875616
   curl --fail --location --retry 0 'https://huggingface.co/unsloth/Mistral-Small-4-119B-2603-GGUF/resolve/bd93c721735aa32c035c0f19e738cb3371fd56ff/UD-Q4_K_M/Mistral-Small-4-119B-2603-UD-Q4_K_M-00002-of-00003.gguf' -o 'models/Mistral-Small-4-119B-2603-UD-Q4_K_M-00002-of-00003.gguf'
   echo 'f357d8dc029824fa4515ae60d7a8b7e108546763614f16c00a1b4ea0cd94145c  models/Mistral-Small-4-119B-2603-UD-Q4_K_M-00002-of-00003.gguf' | sha256sum --check
   test "$(stat -c %s 'models/Mistral-Small-4-119B-2603-UD-Q4_K_M-00002-of-00003.gguf')" = 49299298464
   curl --fail --location --retry 0 'https://huggingface.co/unsloth/Mistral-Small-4-119B-2603-GGUF/resolve/bd93c721735aa32c035c0f19e738cb3371fd56ff/UD-Q4_K_M/Mistral-Small-4-119B-2603-UD-Q4_K_M-00003-of-00003.gguf' -o 'models/Mistral-Small-4-119B-2603-UD-Q4_K_M-00003-of-00003.gguf'
   echo '9dc960d67fb1ef23029d24878b4cf6c63b743ab26cf9137402549591bd770c50  models/Mistral-Small-4-119B-2603-UD-Q4_K_M-00003-of-00003.gguf' | sha256sum --check
   test "$(stat -c %s 'models/Mistral-Small-4-119B-2603-UD-Q4_K_M-00003-of-00003.gguf')" = 24456006464
   curl --fail --location --retry 0 'https://huggingface.co/unsloth/Mistral-Small-4-119B-2603-GGUF/resolve/bd93c721735aa32c035c0f19e738cb3371fd56ff/mmproj-F16.gguf' -o 'models/mmproj-F16.gguf'
   echo '8afb53096537664e248a0b4a9240c25677bb0f8a8cb5a5aad3cd1d608776a4e7  models/mmproj-F16.gguf' | sha256sum --check
   test "$(stat -c %s 'models/mmproj-F16.gguf')" = 857078496
   curl --fail --location --retry 0 'https://huggingface.co/mistralai/Mistral-Small-4-119B-2603/raw/a11f36bebf709121056b1dbcc943d1c6afbe494d/chat_template.jinja' -o models/mistral-small4-chat-template.jinja
   echo 'de56319deee1deb2b88cf576cf20bdc57a8789723f7ff75f9c54acd740d15ecb  models/mistral-small4-chat-template.jinja' | sha256sum --check
   test "$(stat -c %s models/mistral-small4-chat-template.jinja)" = 15168
   curl --fail --location --retry 0 'https://raw.githubusercontent.com/ggml-org/llama.cpp/620d2225cc436ab0b001e3200e590e3b286ef9ab/tools/mtmd/test-1.jpeg' -o fixture.jpg
   echo '2dff664c0c8aaea18aff8cbe7e868845b775e90cdd7a0bac98df709b131deaa3  fixture.jpg' | sha256sum --check
   test "$(stat -c %s fixture.jpg)" = 124071

   Expected result: All three model parts, projector, native template and JPEG pass their exact byte and checksum checks. Template is 15,168 bytes; fixture is 124,071 bytes and 640 by 488 pixels. Stop on any mismatch.

Configure

1. 1. Create a private service credential

   Generate a local server password on your own host when following this guide. Preserve an existing key on repeat setup and never print or share it. This is not an OpenAI or Mistral API key.

   CommandCopy command

   cd "$HOME/mistral-small4-vision-hosting"
   umask 077
   if [ ! -e .api-key ]; then
     openssl rand -hex 32 > .api-key
   fi
   test -s .api-key || { echo 'Restore the empty service-key file before continuing.'; exit 1; }
   chmod 600 .api-key

   Expected result: A nonempty private .api-key exists with mode 600. Existing key preserved; no secret printed.

2. 2. Write the bounded native diagnostic client

   Save the exact stdlib-only client. It checks the fixed JPEG bytes and geometry, requires vision capability and unauthorized rejection, then uses the native multimodal input-count endpoint before each generation. It never fetches a remote image and rejects exhausted output caps or inconsistent usage.

   CommandCopy command

   cd "$HOME/mistral-small4-vision-hosting"
   cat > vision-client.py <<'PYCLIENT'
   """Fixed private text+image diagnostic; no arbitrary remote images or URLs."""
   import base64, hashlib, json, pathlib, struct, urllib.request, urllib.error
   BASE = 'http://127.0.0.1:8080'
   MODEL = 'mistral-small4'
   CONTEXT, OUTPUT = 4096, 512
   IMAGE_SHA = '2dff664c0c8aaea18aff8cbe7e868845b775e90cdd7a0bac98df709b131deaa3'
   
   def positive_int(v):
       return type(v) is int and v > 0
   
   def fixed_image(data):
       if len(data) != 124071 or hashlib.sha256(data).hexdigest() != IMAGE_SHA:
           raise ValueError('Exact diagnostic JPEG bytes required')
       if data[:2] != b'\xff\xd8' or data[-2:] != b'\xff\xd9':
           raise ValueError('Invalid JPEG envelope')
       i, dimensions = 2, None
       while i < len(data):
           if data[i] != 255: raise ValueError('Invalid JPEG segment')
           while data[i] == 255: i += 1
           marker = data[i]; i += 1
           if marker == 218: break
           size = int.from_bytes(data[i:i+2], 'big')
           if size < 2 or i+size > len(data): raise ValueError('Invalid JPEG length')
           if marker in (192,193,194):
               h,w = struct.unpack('>HH', data[i+3:i+7]); dimensions = (w,h)
           i += size
       if dimensions != (640,488): raise ValueError('Unexpected JPEG geometry')
       return 'data:image/jpeg;base64,'+base64.b64encode(data).decode('ascii')
   
   def request(messages):
       return {'model':MODEL,'messages':messages,'max_tokens':OUTPUT,'stream':False,
               'temperature':0,'top_k':0,'top_p':1,'min_p':0,
               'presence_penalty':0,'repeat_penalty':1,
               'chat_template_kwargs':{'reasoning_effort':'none'}}
   
   def checked_completion(call, body):
       count = call('/v1/chat/completions/input_tokens', body).get('input_tokens')
       if not positive_int(count) or count+OUTPUT > CONTEXT:
           raise ValueError('Full image/text/template count plus output cap exceeds context or count invalid')
       # The same body is sent only after native multimodal token counting succeeds.
       result = call('/v1/chat/completions', body)
       choices = result.get('choices')
       if not isinstance(choices,list) or len(choices)!=1: raise ValueError('One completion required')
       choice = choices[0]; message = choice.get('message',{})
       text = message.get('content')
       if choice.get('finish_reason') != 'stop' or not isinstance(text,str) or not text.strip():
           raise ValueError('Incomplete or empty answer; no success from an exhausted cap')
       if message.get('tool_calls') or message.get('reasoning_content') or '[THINK]' in text:
           raise ValueError('Expected non-reasoning supplied-evidence answer')
       usage = result.get('usage',{})
       n = usage.get('completion_tokens')
       if usage.get('prompt_tokens') != count or not positive_int(n) or n > OUTPUT or count+n > CONTEXT:
           raise ValueError('Actual usage disagrees with preflight or exceeds budget')
       return {'preflightInputTokens':count,'response':result}
   
   def http_call(key):
       def call(path, body=None):
           data = None if body is None else json.dumps(body).encode('utf-8')
           req=urllib.request.Request(BASE+path,data=data,headers={'Authorization':'Bearer '+key,'Content-Type':'application/json'})
           with urllib.request.urlopen(req, timeout=1800) as response:
               if response.status!=200: raise ValueError('HTTP failure')
               raw=response.read(2_000_001)
           if len(raw)>2_000_000: raise ValueError('Response too large')
           obj=json.loads(raw)
           if not isinstance(obj,dict): raise ValueError('JSON object required')
           return obj
       return call
   
   def main():
       key=pathlib.Path('.api-key').read_text().strip()
       if len(key)!=64 or any(c not in '0123456789abcdef' for c in key): raise ValueError('Restore private service key')
       call=http_call(key)
       call('/health')
       props=call('/props')
       if props.get('modalities',{}).get('vision') is not True: raise ValueError('Matching vision projector not active')
       # Access check never sends a generation request without credentials successfully.
       req=urllib.request.Request(BASE+'/v1/chat/completions',data=b'{}',headers={'Content-Type':'application/json'})
       try:
           with urllib.request.urlopen(req,timeout=30) as r: raise ValueError('Unauthorized request was accepted')
       except urllib.error.HTTPError as e:
           if e.code!=401: raise
       text=request([{'role':'system','content':'Answer only from the supplied evidence; do not use tools.'},
                     {'role':'user','content':'In one sentence, state this library schedule: weekdays 9am to 6pm; Saturday 10am to 4pm; Sunday closed.'}])
       image=request([{'role':'system','content':'Answer only from the supplied image; do not use tools.'},
                      {'role':'user','content':[{'type':'text','text':'Identify the newspaper and transcribe its largest headline. Briefly describe the front page.'},
                        {'type':'image_url','image_url':{'url':fixed_image(pathlib.Path('fixture.jpg').read_bytes())}}]}])
       for name,body in [('text',text),('image',image)]:
           checked=checked_completion(call,body)
           pathlib.Path(name+'-response.json').write_text(json.dumps(checked,indent=2)+'\n')
           print(name, json.dumps(checked,ensure_ascii=False))
       print('Manually inspect the library facts and image newspaper/headline; protocol success alone is not semantic accuracy.')
   
   if __name__=='__main__': main()
   PYCLIENT

   Expected result: The diagnostic client is saved. Saving or parsing code does not execute a model or prove fit.

Start the model

1. 1. Start the complete fast-mode CPU image and text service

   Run in the foreground on the trusted host. Native loader discovers all three splits from the first filename. Missing or incompatible parts must fail. Native Jinja settings select none reasoning; image, row-break and template tokens share one 4,096-token context. Nominal image bound 1,024 does not independently bound every overhead token; preflight counts the complete request. No context shifting is allowed.

   CommandCopy command

   set -e
   cd "$HOME/mistral-small4-vision-hosting"
   export MISTRAL_SMALL4_SERVICE_KEY="$(cat .api-key)"
   ./llama.cpp/build/bin/llama-server -m 'models/Mistral-Small-4-119B-2603-UD-Q4_K_M-00001-of-00003.gguf' --alias mistral-small4 --host 127.0.0.1 --port 8080 --api-key "$MISTRAL_SMALL4_SERVICE_KEY" --device none -ngl 0 -t 8 -c 4096 -np 1 --no-context-shift --cache-type-k f16 --cache-type-v f16 --jinja --chat-template-file models/mistral-small4-chat-template.jinja --chat-template-kwargs '{"reasoning_effort":"none"}' --skip-chat-parsing --reasoning-format none --temp 0 --top-k 0 --top-p 1 --min-p 0 --presence-penalty 0 --repeat-penalty 1 --mmproj models/mmproj-F16.gguf --no-mmproj-offload --mmproj-device none --image-max-tokens 1024 --mtmd-batch-max-tokens 2048 -b 2048 -ub 2048

   Expected result: All three language splits and the matching Pixtral encoder/projector load, then the authenticated loopback listener starts. Stop on architecture, template, projector, allocation or shard errors. No successful startup or performance is preclaimed.

Verify the setup

1. 1. Check native text and image input budgets, completion and grounding

   Use a second terminal on the same host; for cloud use a shell inside the VM. The native input-count route parses the same full request and counts multimodal tokens before generation. Each count plus the 512-token reserve must fit 4,096. Inspect both saved responses: correct supplied library hours and the visibly grounded newspaper/headline. Protocol completion alone is not semantic quality.

   CommandCopy command

   set -e
   cd "$HOME/mistral-small4-vision-hosting"
   python3 vision-client.py

   Expected result: Health is ready, vision capability is true and unauthenticated chat returns HTTP 401. Native counts pass before either generation. Both responses have nonempty stop-finished content and matching actual usage. Manually require correct library facts, The New York Times and MEN WALK ON MOON; wrong facts, truncation or image rejection fail the smoke test.

Performance evidence

No compatible measurement available. Documented support and estimated memory do not establish response speed.

Limits and troubleshooting

• Documented source-qualified configuration only. No build, weight or model import, inference, memory fit, startup time, speed or quality measurement was executed by the author.
• The frozen API shorthand is retained. The official March 2026 release and native Mistral Small 4 119B 2603 card establish the selected exact multimodal family identity; the literal shorthand repository probe returned an access error and is not treated as an absence or exclusion proof.
• The native card advertises 256k context. Config stores 1,048,576 maximum positions and YaRN factor 128 / original 8,192; those metadata differ in scope and do not establish tested million-token serving. This guide selects only 4,096 combined tokens, including image/template overhead and output.
• The 119B model has 6.5B active parameters per token; all three quantized language parts and the full vision encoder/projector are still required. The first split is only 7,875,616 bytes and cannot stand alone.
• Unsloth names the native base repository but its exact input base commit is not established by the archived quantizer metadata. Native template and derivative are independent pins, not a byte-equivalence or quantization-quality claim.
• Pixtral patch size 14, spatial merge 2, image row-break embeddings and GELU projection are source-qualified. Native preprocessing handles the JPEG; the client checks exact fixture bytes/geometry. A nominal 1,024 image-token bound excludes some overhead; actual combined count is obtained through the native multimodal counter before generation.
• Native MLA uses a compressed/cache implementation selected by runtime source. F16 cache flags are explicit, but no ordinary dense KV formula, measured memory peak or full host fit is inferred.
• The exact source fixture visibly shows The New York Times and MEN WALK ON MOON. This narrow supplied-evidence test does not establish general OCR accuracy. Photograph redistribution rights remain unknown; do not infer them from the runtime code license.
• The complete endpoint is intended for a trusted private host. The fixed client is not an arbitrary upload sanitizer, public TLS service or multiuser operations design. Local privileged users and native decoder maintenance remain operational considerations.
• Reasoning high, tools, native FP8/NVFP4, Eagle speculation, GPU, larger-image workloads and long context remain separate open scopes. Required total guide jobs remains null; this pair does not close the model family.

Checksum, template, split or projector mismatch

Stop. Verify exact download commands, all three preserved split filenames, native template and matching same-revision projector. Do not substitute a sibling quantization, projector or older generic runtime.

Input count fails or context is too large

Do not send the generation request. Preserve the native count/error and use a separately reviewed smaller workload or different configuration; do not truncate images, remove vision or suppress the budget failure.

Out of memory or allocation failure

Stop the foreground service and free or provide more resources. Record a separately reviewed configuration; 128 GiB RAM and 160 GiB disk are planning allocations, not guaranteed fit.

Empty, reasoning-trace, truncated or inaccurate response

Treat the smoke test as failed. Inspect the exact template settings, full input count, actual usage, finish reason and supplied evidence. No exact generated paragraph or quality score is promised.

Connection refused or HTTP 401

Inspect foreground logs, matching private key and loopback port. For cloud confirm the tunnel targets the intended VM; never open the inference port publicly to bypass authentication.

Stop the model

1. 1. Stop the service and close its tunnel

   Press Ctrl+C in the foreground server terminal, then close any SSH tunnel with Ctrl+C. Unset the key in every shell. Confirm the intended listener is gone; stopping the process does not end cloud resource billing.

   CommandCopy command

   unset MISTRAL_SMALL4_SERVICE_KEY
   if curl --silent --max-time 3 http://127.0.0.1:8080/health >/dev/null; then echo "Endpoint still responds: stop the intended process"; exit 1; else echo "Endpoint no longer responds"; fi

   Expected result: The intended loopback endpoint no longer responds and client tunnels are closed. Cloud compute/storage still require lifecycle shutdown.

Uninstall

1. 1. Remove dedicated walkthrough files

   Back up wanted requests/results and logs. Confirm this path before removing only its dedicated model/build/key folder; shared apt packages remain.

   CommandCopy command

   rm -r -- "$HOME/mistral-small4-vision-hosting"

   Expected result: Dedicated model parts, projector, native template, fixture, client, results, build and service key files are removed. Shared system packages remain.

Exact model and requirements

Artifact:

Unsloth (named GGUF derivative); Mistral AI (native multimodal model) / Complete Mistral Small 4 119B UD-Q4_K_M, three split files plus F16 Pixtral projector · checkpoint unsloth/Mistral-Small-4-119B-2603-GGUF

Publisher revision:

bd93c721735aa32c035c0f19e738cb3371fd56ff

Artifact digest:

No publisher single digest for the complete three-part model composition. Every individual part and the projector has its own checked SHA256.

Download size:

68.7 GiB

Memory requirement (unknown):

Unknown — No complete CPU peak measurement for the complete language splits, MLA cache, Pixtral tensors, scratch, loading and OS.

System RAM / disk:

Unknown / Unknown

Placement:

single device. One CPU host; all three GGUF disk splits form one complete model, not tensor/pipeline/device sharding. CPU language graph and full Pixtral image encoder/projector, with both GPU layer and projector offload disabled.

KV cache:

F16 cache flags are explicit; native MLA implementation determines actual cache layout. Cache bytes and process peak remain unknown.

Workload:

4,096 token context; Unknown; Unknown; concurrency 1; batch 1; cache unknown; repetitions Unknown.

Tokenizer:

Exact complete Unsloth GGUF tokenizer and independently pinned native Mistral Small 4 Jinja template · bd93c721735aa32c035c0f19e738cb3371fd56ff; native template a11f36bebf709121056b1dbcc943d1c6afbe494d

Sampling / reasoning:

Temperature zero is within the publisher none-mode range 0.0 to 0.7. Top_k 0, top_p 1, min_p 0, presence_penalty 0 and repeat_penalty 1 are author choices. The client requires a nonempty stop-finished answer and valid actual usage, not a prescribed generated result. Native reasoning_effort none is passed as a Jinja template keyword and produces the native MODEL_SETTINGS none prefix. The exact template emits native BOS and image boundaries. This is complete text plus image input with text output; reasoning high, tool calls, speculative Eagle, native FP8/NVFP4, GPU and long-context scopes remain open.

Runtime prerequisites:

Ubuntu 24.04 x86_64; See prerequisites above.

Official artifact · License

Mistral AI explicitly grants Apache License 2.0 in its pinned native model card and metadata; Unsloth declares Apache-2.0 and names this exact native base repository. Read the linked full Apache text and retain required notices, license, attribution and modification notices for covered redistribution. The native root LICENSE URL returns 404; the explicit model-card grant is preserved rather than inventing that file. Public metadata indicates ungated access; no terms acceptance or legal clearance is performed for the reader.

• Mistral-Small-4-119B-2603-UD-Q4_K_M-00001-of-00003.gguf · 0.01 GiB · revision bd93c721735aa32c035c0f19e738cb3371fd56ff
• Mistral-Small-4-119B-2603-UD-Q4_K_M-00002-of-00003.gguf · 45.91 GiB · revision bd93c721735aa32c035c0f19e738cb3371fd56ff
• Mistral-Small-4-119B-2603-UD-Q4_K_M-00003-of-00003.gguf · 22.78 GiB · revision bd93c721735aa32c035c0f19e738cb3371fd56ff
• mmproj-F16.gguf · 0.8 GiB · revision bd93c721735aa32c035c0f19e738cb3371fd56ff
• mistral-small4-chat-template.jinja · 0 GiB · revision a11f36bebf709121056b1dbcc943d1c6afbe494d

• All three language parts total 73,763,180,544 bytes; the F16 projector adds 857,078,496 bytes, for 74,620,259,040 artifact bytes before template, fixture and build files. The tiny first split alone is not the complete model.
• The publisher labels 119B total parameters and 6.5B active per token. Native metadata reports 119,401,317,952 parameters across F8 and BF16 tensors. Active compute does not imply only active weights are resident; the complete model must load.
• The 36-layer native MISTRAL4 model has MLA, 128 routed experts, four selected experts per token and one shared expert. This is not a dense-model KV memory estimate. Actual MLA cache, quantized weight placement, vision preprocessing, 2,048-token batch/ubatch buffers and warmup peaks remain unknown.
• Local eight threads / cloud sixteen vCPUs, 128 GiB RAM and 160 GiB disk are author allocations. Verify actual local DIMM/motherboard/OS capacity. No wall power, fit, latency, throughput or quality result is claimed.

Install the runtime

1. 1. Build the exact CPU runtime

   Build exact official llama.cpp source on this host. Native CPU architecture mapping includes MISTRAL4 MLA/MoE and full Pixtral processing. Build dependencies resolve at execution time; record resolved package/tool versions and logs. No build or model inference was run by the author.

   CommandCopy command

   set -e
   sudo apt-get update
   sudo apt-get install -y build-essential cmake git curl ca-certificates openssl python3
   mkdir -p "$HOME/mistral-small4-vision-hosting/models"
   cd "$HOME/mistral-small4-vision-hosting"
   git clone https://github.com/ggml-org/llama.cpp.git
   git -C llama.cpp checkout --detach 620d2225cc436ab0b001e3200e590e3b286ef9ab
   cmake -S llama.cpp -B llama.cpp/build -DCMAKE_BUILD_TYPE=Release -DGGML_METAL=OFF -DGGML_NATIVE=ON
   cmake --build llama.cpp/build --config Release -j 16
   ./llama.cpp/build/bin/llama-server --version

   Expected result: Checkout and build succeed; server version identifies the pinned source. Stop on failure. No inference has been executed by the author.

Download the model

1. 1. Download and verify the complete model, projector, template and fixture

   All three split files and the exact F16 projector are mandatory. Stop on any HTTP, byte-count or checksum failure. SHA tools stream the files; no whole-weight Python read or conversion is used. Preserve filenames together. Template and quantizer are independent pins; Unsloth advertises embedded template fixes, while this guide explicitly overrides with the exact native template and requires functional checks.

   CommandCopy command

   set -e
   cd "$HOME/mistral-small4-vision-hosting"
   curl --fail --location --retry 0 'https://huggingface.co/unsloth/Mistral-Small-4-119B-2603-GGUF/resolve/bd93c721735aa32c035c0f19e738cb3371fd56ff/UD-Q4_K_M/Mistral-Small-4-119B-2603-UD-Q4_K_M-00001-of-00003.gguf' -o 'models/Mistral-Small-4-119B-2603-UD-Q4_K_M-00001-of-00003.gguf'
   echo 'fd3cc46082e4e64eb623eea4611b02583d1de4c20a59411bf1820925d5117dfe  models/Mistral-Small-4-119B-2603-UD-Q4_K_M-00001-of-00003.gguf' | sha256sum --check
   test "$(stat -c %s 'models/Mistral-Small-4-119B-2603-UD-Q4_K_M-00001-of-00003.gguf')" = 7875616
   curl --fail --location --retry 0 'https://huggingface.co/unsloth/Mistral-Small-4-119B-2603-GGUF/resolve/bd93c721735aa32c035c0f19e738cb3371fd56ff/UD-Q4_K_M/Mistral-Small-4-119B-2603-UD-Q4_K_M-00002-of-00003.gguf' -o 'models/Mistral-Small-4-119B-2603-UD-Q4_K_M-00002-of-00003.gguf'
   echo 'f357d8dc029824fa4515ae60d7a8b7e108546763614f16c00a1b4ea0cd94145c  models/Mistral-Small-4-119B-2603-UD-Q4_K_M-00002-of-00003.gguf' | sha256sum --check
   test "$(stat -c %s 'models/Mistral-Small-4-119B-2603-UD-Q4_K_M-00002-of-00003.gguf')" = 49299298464
   curl --fail --location --retry 0 'https://huggingface.co/unsloth/Mistral-Small-4-119B-2603-GGUF/resolve/bd93c721735aa32c035c0f19e738cb3371fd56ff/UD-Q4_K_M/Mistral-Small-4-119B-2603-UD-Q4_K_M-00003-of-00003.gguf' -o 'models/Mistral-Small-4-119B-2603-UD-Q4_K_M-00003-of-00003.gguf'
   echo '9dc960d67fb1ef23029d24878b4cf6c63b743ab26cf9137402549591bd770c50  models/Mistral-Small-4-119B-2603-UD-Q4_K_M-00003-of-00003.gguf' | sha256sum --check
   test "$(stat -c %s 'models/Mistral-Small-4-119B-2603-UD-Q4_K_M-00003-of-00003.gguf')" = 24456006464
   curl --fail --location --retry 0 'https://huggingface.co/unsloth/Mistral-Small-4-119B-2603-GGUF/resolve/bd93c721735aa32c035c0f19e738cb3371fd56ff/mmproj-F16.gguf' -o 'models/mmproj-F16.gguf'
   echo '8afb53096537664e248a0b4a9240c25677bb0f8a8cb5a5aad3cd1d608776a4e7  models/mmproj-F16.gguf' | sha256sum --check
   test "$(stat -c %s 'models/mmproj-F16.gguf')" = 857078496
   curl --fail --location --retry 0 'https://huggingface.co/mistralai/Mistral-Small-4-119B-2603/raw/a11f36bebf709121056b1dbcc943d1c6afbe494d/chat_template.jinja' -o models/mistral-small4-chat-template.jinja
   echo 'de56319deee1deb2b88cf576cf20bdc57a8789723f7ff75f9c54acd740d15ecb  models/mistral-small4-chat-template.jinja' | sha256sum --check
   test "$(stat -c %s models/mistral-small4-chat-template.jinja)" = 15168
   curl --fail --location --retry 0 'https://raw.githubusercontent.com/ggml-org/llama.cpp/620d2225cc436ab0b001e3200e590e3b286ef9ab/tools/mtmd/test-1.jpeg' -o fixture.jpg
   echo '2dff664c0c8aaea18aff8cbe7e868845b775e90cdd7a0bac98df709b131deaa3  fixture.jpg' | sha256sum --check
   test "$(stat -c %s fixture.jpg)" = 124071

   Expected result: All three model parts, projector, native template and JPEG pass their exact byte and checksum checks. Template is 15,168 bytes; fixture is 124,071 bytes and 640 by 488 pixels. Stop on any mismatch.

Configure

1. 1. Create a private service credential

   Generate a local server password on your own host when following this guide. Preserve an existing key on repeat setup and never print or share it. This is not an OpenAI or Mistral API key.

   CommandCopy command

   cd "$HOME/mistral-small4-vision-hosting"
   umask 077
   if [ ! -e .api-key ]; then
     openssl rand -hex 32 > .api-key
   fi
   test -s .api-key || { echo 'Restore the empty service-key file before continuing.'; exit 1; }
   chmod 600 .api-key

   Expected result: A nonempty private .api-key exists with mode 600. Existing key preserved; no secret printed.

2. 2. Write the bounded native diagnostic client

   Save the exact stdlib-only client. It checks the fixed JPEG bytes and geometry, requires vision capability and unauthorized rejection, then uses the native multimodal input-count endpoint before each generation. It never fetches a remote image and rejects exhausted output caps or inconsistent usage.

   CommandCopy command

   cd "$HOME/mistral-small4-vision-hosting"
   cat > vision-client.py <<'PYCLIENT'
   """Fixed private text+image diagnostic; no arbitrary remote images or URLs."""
   import base64, hashlib, json, pathlib, struct, urllib.request, urllib.error
   BASE = 'http://127.0.0.1:8080'
   MODEL = 'mistral-small4'
   CONTEXT, OUTPUT = 4096, 512
   IMAGE_SHA = '2dff664c0c8aaea18aff8cbe7e868845b775e90cdd7a0bac98df709b131deaa3'
   
   def positive_int(v):
       return type(v) is int and v > 0
   
   def fixed_image(data):
       if len(data) != 124071 or hashlib.sha256(data).hexdigest() != IMAGE_SHA:
           raise ValueError('Exact diagnostic JPEG bytes required')
       if data[:2] != b'\xff\xd8' or data[-2:] != b'\xff\xd9':
           raise ValueError('Invalid JPEG envelope')
       i, dimensions = 2, None
       while i < len(data):
           if data[i] != 255: raise ValueError('Invalid JPEG segment')
           while data[i] == 255: i += 1
           marker = data[i]; i += 1
           if marker == 218: break
           size = int.from_bytes(data[i:i+2], 'big')
           if size < 2 or i+size > len(data): raise ValueError('Invalid JPEG length')
           if marker in (192,193,194):
               h,w = struct.unpack('>HH', data[i+3:i+7]); dimensions = (w,h)
           i += size
       if dimensions != (640,488): raise ValueError('Unexpected JPEG geometry')
       return 'data:image/jpeg;base64,'+base64.b64encode(data).decode('ascii')
   
   def request(messages):
       return {'model':MODEL,'messages':messages,'max_tokens':OUTPUT,'stream':False,
               'temperature':0,'top_k':0,'top_p':1,'min_p':0,
               'presence_penalty':0,'repeat_penalty':1,
               'chat_template_kwargs':{'reasoning_effort':'none'}}
   
   def checked_completion(call, body):
       count = call('/v1/chat/completions/input_tokens', body).get('input_tokens')
       if not positive_int(count) or count+OUTPUT > CONTEXT:
           raise ValueError('Full image/text/template count plus output cap exceeds context or count invalid')
       # The same body is sent only after native multimodal token counting succeeds.
       result = call('/v1/chat/completions', body)
       choices = result.get('choices')
       if not isinstance(choices,list) or len(choices)!=1: raise ValueError('One completion required')
       choice = choices[0]; message = choice.get('message',{})
       text = message.get('content')
       if choice.get('finish_reason') != 'stop' or not isinstance(text,str) or not text.strip():
           raise ValueError('Incomplete or empty answer; no success from an exhausted cap')
       if message.get('tool_calls') or message.get('reasoning_content') or '[THINK]' in text:
           raise ValueError('Expected non-reasoning supplied-evidence answer')
       usage = result.get('usage',{})
       n = usage.get('completion_tokens')
       if usage.get('prompt_tokens') != count or not positive_int(n) or n > OUTPUT or count+n > CONTEXT:
           raise ValueError('Actual usage disagrees with preflight or exceeds budget')
       return {'preflightInputTokens':count,'response':result}
   
   def http_call(key):
       def call(path, body=None):
           data = None if body is None else json.dumps(body).encode('utf-8')
           req=urllib.request.Request(BASE+path,data=data,headers={'Authorization':'Bearer '+key,'Content-Type':'application/json'})
           with urllib.request.urlopen(req, timeout=1800) as response:
               if response.status!=200: raise ValueError('HTTP failure')
               raw=response.read(2_000_001)
           if len(raw)>2_000_000: raise ValueError('Response too large')
           obj=json.loads(raw)
           if not isinstance(obj,dict): raise ValueError('JSON object required')
           return obj
       return call
   
   def main():
       key=pathlib.Path('.api-key').read_text().strip()
       if len(key)!=64 or any(c not in '0123456789abcdef' for c in key): raise ValueError('Restore private service key')
       call=http_call(key)
       call('/health')
       props=call('/props')
       if props.get('modalities',{}).get('vision') is not True: raise ValueError('Matching vision projector not active')
       # Access check never sends a generation request without credentials successfully.
       req=urllib.request.Request(BASE+'/v1/chat/completions',data=b'{}',headers={'Content-Type':'application/json'})
       try:
           with urllib.request.urlopen(req,timeout=30) as r: raise ValueError('Unauthorized request was accepted')
       except urllib.error.HTTPError as e:
           if e.code!=401: raise
       text=request([{'role':'system','content':'Answer only from the supplied evidence; do not use tools.'},
                     {'role':'user','content':'In one sentence, state this library schedule: weekdays 9am to 6pm; Saturday 10am to 4pm; Sunday closed.'}])
       image=request([{'role':'system','content':'Answer only from the supplied image; do not use tools.'},
                      {'role':'user','content':[{'type':'text','text':'Identify the newspaper and transcribe its largest headline. Briefly describe the front page.'},
                        {'type':'image_url','image_url':{'url':fixed_image(pathlib.Path('fixture.jpg').read_bytes())}}]}])
       for name,body in [('text',text),('image',image)]:
           checked=checked_completion(call,body)
           pathlib.Path(name+'-response.json').write_text(json.dumps(checked,indent=2)+'\n')
           print(name, json.dumps(checked,ensure_ascii=False))
       print('Manually inspect the library facts and image newspaper/headline; protocol success alone is not semantic accuracy.')
   
   if __name__=='__main__': main()
   PYCLIENT

   Expected result: The diagnostic client is saved. Saving or parsing code does not execute a model or prove fit.

Start the model

1. 1. Start the complete fast-mode CPU image and text service

   Run in the foreground on the trusted host. Native loader discovers all three splits from the first filename. Missing or incompatible parts must fail. Native Jinja settings select none reasoning; image, row-break and template tokens share one 4,096-token context. Nominal image bound 1,024 does not independently bound every overhead token; preflight counts the complete request. No context shifting is allowed.

   CommandCopy command

   set -e
   cd "$HOME/mistral-small4-vision-hosting"
   export MISTRAL_SMALL4_SERVICE_KEY="$(cat .api-key)"
   ./llama.cpp/build/bin/llama-server -m 'models/Mistral-Small-4-119B-2603-UD-Q4_K_M-00001-of-00003.gguf' --alias mistral-small4 --host 127.0.0.1 --port 8080 --api-key "$MISTRAL_SMALL4_SERVICE_KEY" --device none -ngl 0 -t 16 -c 4096 -np 1 --no-context-shift --cache-type-k f16 --cache-type-v f16 --jinja --chat-template-file models/mistral-small4-chat-template.jinja --chat-template-kwargs '{"reasoning_effort":"none"}' --skip-chat-parsing --reasoning-format none --temp 0 --top-k 0 --top-p 1 --min-p 0 --presence-penalty 0 --repeat-penalty 1 --mmproj models/mmproj-F16.gguf --no-mmproj-offload --mmproj-device none --image-max-tokens 1024 --mtmd-batch-max-tokens 2048 -b 2048 -ub 2048

   Expected result: All three language splits and the matching Pixtral encoder/projector load, then the authenticated loopback listener starts. Stop on architecture, template, projector, allocation or shard errors. No successful startup or performance is preclaimed.

Verify the setup

1. 1. Check native text and image input budgets, completion and grounding

   Use a second terminal on the same host; for cloud use a shell inside the VM. The native input-count route parses the same full request and counts multimodal tokens before generation. Each count plus the 512-token reserve must fit 4,096. Inspect both saved responses: correct supplied library hours and the visibly grounded newspaper/headline. Protocol completion alone is not semantic quality.

   CommandCopy command

   set -e
   cd "$HOME/mistral-small4-vision-hosting"
   python3 vision-client.py

   Expected result: Health is ready, vision capability is true and unauthenticated chat returns HTTP 401. Native counts pass before either generation. Both responses have nonempty stop-finished content and matching actual usage. Manually require correct library facts, The New York Times and MEN WALK ON MOON; wrong facts, truncation or image rejection fail the smoke test.

2. 2. Reach the private service through SSH

   From your trusted client terminal use an existing SSH key and replace the host placeholder with this exact instance host. Keep the tunnel open; run the supplied diagnostic inside the VM so its key, fixture and files stay there. A separate local API client must use the same private service key securely; never print it or expose the native port publicly.

   CommandCopy command

   ssh -N -L 8080:127.0.0.1:8080 ubuntu@YOUR_EXISTING_INSTANCE_HOST

   Expected result: An authenticated SSH tunnel maps client loopback to VM loopback. Public inference port 8080 remains closed; closing the tunnel does not stop the VM or compute billing.

Performance evidence

No compatible measurement available. Documented support and estimated memory do not establish response speed.

Limits and troubleshooting

• Documented source-qualified configuration only. No build, weight or model import, inference, memory fit, startup time, speed or quality measurement was executed by the author.
• The frozen API shorthand is retained. The official March 2026 release and native Mistral Small 4 119B 2603 card establish the selected exact multimodal family identity; the literal shorthand repository probe returned an access error and is not treated as an absence or exclusion proof.
• The native card advertises 256k context. Config stores 1,048,576 maximum positions and YaRN factor 128 / original 8,192; those metadata differ in scope and do not establish tested million-token serving. This guide selects only 4,096 combined tokens, including image/template overhead and output.
• The 119B model has 6.5B active parameters per token; all three quantized language parts and the full vision encoder/projector are still required. The first split is only 7,875,616 bytes and cannot stand alone.
• Unsloth names the native base repository but its exact input base commit is not established by the archived quantizer metadata. Native template and derivative are independent pins, not a byte-equivalence or quantization-quality claim.
• Pixtral patch size 14, spatial merge 2, image row-break embeddings and GELU projection are source-qualified. Native preprocessing handles the JPEG; the client checks exact fixture bytes/geometry. A nominal 1,024 image-token bound excludes some overhead; actual combined count is obtained through the native multimodal counter before generation.
• Native MLA uses a compressed/cache implementation selected by runtime source. F16 cache flags are explicit, but no ordinary dense KV formula, measured memory peak or full host fit is inferred.
• The exact source fixture visibly shows The New York Times and MEN WALK ON MOON. This narrow supplied-evidence test does not establish general OCR accuracy. Photograph redistribution rights remain unknown; do not infer them from the runtime code license.
• The complete endpoint is intended for a trusted private host. The fixed client is not an arbitrary upload sanitizer, public TLS service or multiuser operations design. Local privileged users and native decoder maintenance remain operational considerations.
• Reasoning high, tools, native FP8/NVFP4, Eagle speculation, GPU, larger-image workloads and long context remain separate open scopes. Required total guide jobs remains null; this pair does not close the model family.

Checksum, template, split or projector mismatch

Stop. Verify exact download commands, all three preserved split filenames, native template and matching same-revision projector. Do not substitute a sibling quantization, projector or older generic runtime.

Input count fails or context is too large

Do not send the generation request. Preserve the native count/error and use a separately reviewed smaller workload or different configuration; do not truncate images, remove vision or suppress the budget failure.

Out of memory or allocation failure

Stop the foreground service and free or provide more resources. Record a separately reviewed configuration; 128 GiB RAM and 160 GiB disk are planning allocations, not guaranteed fit.

Empty, reasoning-trace, truncated or inaccurate response

Treat the smoke test as failed. Inspect the exact template settings, full input count, actual usage, finish reason and supplied evidence. No exact generated paragraph or quality score is promised.

Connection refused or HTTP 401

Inspect foreground logs, matching private key and loopback port. For cloud confirm the tunnel targets the intended VM; never open the inference port publicly to bypass authentication.

Stop the model

1. 1. Stop the service and close its tunnel

   Press Ctrl+C in the foreground server terminal, then close any SSH tunnel with Ctrl+C. Unset the key in every shell. Confirm the intended listener is gone; stopping the process does not end cloud resource billing.

   CommandCopy command

   unset MISTRAL_SMALL4_SERVICE_KEY
   if curl --silent --max-time 3 http://127.0.0.1:8080/health >/dev/null; then echo "Endpoint still responds: stop the intended process"; exit 1; else echo "Endpoint no longer responds"; fi

   Expected result: The intended loopback endpoint no longer responds and client tunnels are closed. Cloud compute/storage still require lifecycle shutdown.

Uninstall

1. 1. Remove dedicated walkthrough files

   Back up wanted requests/results and logs. Confirm this path before removing only its dedicated model/build/key folder; shared apt packages remain.

   CommandCopy command

   rm -r -- "$HOME/mistral-small4-vision-hosting"

   Expected result: Dedicated model parts, projector, native template, fixture, client, results, build and service key files are removed. Shared system packages remain.

Shut down cloud resources

1. 1. End cloud billing resources

   In EC2 us-east-1, stop for later reuse or terminate this exact VM. Review Delete on termination for its root volume. Delete unwanted detached EBS volumes, snapshots and reserved/Elastic/public IP resources. Stopped VM storage and retained resources may still bill. Check Bills/Cost Explorer later for final usage.

   Expected result: Exact instance is stopped/terminated as intended; no unwanted attached/retained billed resources remain. Instance compute stops only after its lifecycle transition, not when llama-server stops.