OpenBMB logo

MiniCPM

MiniCPMv4 / 4.1Current
byOpenBMBOpenBMB(open source)
Context128K tokens
Training CutoffNot publicly specified in queued source
CategoryLarge Language Model
Max Output32.8K tokens

About MiniCPM

MiniCPM is OpenBMB’s ultra-efficient open language-model family for edge and end-device deployment. The MiniCPM4 and MiniCPM4.1 lines focus on fast local reasoning, while MiniCPM-SALA extends the family toward sparse/linear attention and million-token context research.

Capabilities

textcodereasoninglocal inference

Input Modalities

text

Output Modalities

text

Technical Details

API Identifier
OpenBMB/MiniCPM
Category
Large Language Model
Context Window
128,000 tokens
Max Output Tokens
32,768 tokens

Tags

open-weightedge-ailocal-llmreasoningopenbmb

Run this model yourself

Local and cloud setup for MiniCPM, with exact configurations, sources and clear evidence limits.

Find a model for your hardware
Local hardwareDocumented setupSources checked Oct 16, 2026

Serve MiniCPM4-0.5B text on a local CPU

Run the exact native MiniCPM4-0.5B checkpoint with its own chat template and pinned custom Python. This private serial text endpoint uses four CPU threads, full FP32 eager execution, a 4,096-token walkthrough budget and a 512-token output cap. Cache is disabled. Actual fit, speed and answer quality remain unmeasured.

Hardware
Intel Core i5-12400 desktop, 4 inference threads, 32 GiB RAM
Runtime
Native MiniCPM4-0.5B private eager CPU text service Transformers 4.57.6; torch 2.6.0 CPU; Python 3.12 · CPU
Model format
Safetensors with pinned native MiniCPM Python · Full native MiniCPM CPU FP32; publisher serialized checkpoint BF16; no quantization · Native BF16 serialized checkpoint; reader full FP32 CPU, no quantization
Workload
4,096 token context · 1 concurrent

Based on source documentation; this setup has not been executed by OpenTools. Memory: unknown. No compatible measurement available.

Read the complete setup guide

Before you start

  • Prepare Ubuntu 24.04 x86_64 on the referenced Intel CPU, with four threads selected, 32 GiB RAM and 40 GiB free disk. Capacity is planning only; unknown fit and CPU speed remain.
  • Read the exact native model card and actual Apache 2.0 license. This is the publisher-listed MiniCPM4-0.5B variant within frozen MiniCPM 4/4.1, not a substitute MiniCPM5 or a completion claim for every family member.
  • Review both hash-pinned custom Python files before enabling local trusted code. The wrapper loads only the verified local directory with offline flags and safe weight loading. No unreviewed remote code fallback is allowed.
  • Use Python 3.12 and a dedicated directory. This loopback serial demonstration has no inference deadline or public TLS/multiuser protection. Do not expose it publicly or use untrusted multi-tenant inputs.

Exact model and requirements

Artifact:
OpenBMB / Native MiniCPM4-0.5B · checkpoint openbmb/MiniCPM4-0.5B
Publisher revision:
5253c7fcc5e29e1cf3eacb59a58adf1ba4df8630
Artifact digest:
sha256:ef74f4de1d1c580dd64e425d11f29881b9fd39f9be16a485150f727d4583181c
Download size:
0.81 GiB
Memory requirement (unknown):
Unknown — Native full FP32 process/loading/activations/OS peak is unmeasured.
System RAM / disk:
Unknown / Unknown
Placement:
single device. One CPU host and one sequence. All native language model computation is on CPU; no GPU or disk offload.
KV cache:
Not allocated by this recipe: native use_cache=False; eager recomputation on CPU.
Workload:
4,096 token context; Unknown; Unknown; concurrency 1; batch 1; cache cold; repetitions Unknown.
Tokenizer:
Native BPE tokenizer JSON and exact MiniCPM4 chat template · Native checkpoint pin; tokenizers 0.22.2
Sampling / reasoning:
Publisher card uses temperature 0.7 and top_p 0.7. This walkthrough uses those values, top_k 0 (author choice), repetition penalty 1.0 and one beam, with native EOS IDs 2 and 73440 and pad ID 2. Output sampling is nondeterministic; no exact answer is promised. This exact small MiniCPM4 text checkpoint uses its native ordinary user/assistant template. No hybrid reasoning toggle is introduced. The native configuration allows 32,768 positions; 4,096 is an author-selected bounded walkthrough, not a claim of full-family or long-context support. Render the native template, then tokenize once with its BOS postprocessor. Disable native cache explicitly.
Runtime prerequisites:
Ubuntu 24.04 x86_64; See prerequisites above.

Official artifact · License

The exact native model card declares Apache 2.0 and links the publisher repository license. Preserve the license, applicable notices, modification notices and patent conditions. No inference quality, suitability, endorsement or legal clearance is claimed. Review the archived actual publisher license before use.

  • The exact serialized native weight file is 867,772,648 bytes. FP32 expansion, loading transients, tokenizer/runtime and OS buffers require additional memory; file size is not RAM fit.
  • 32 GiB RAM, 40 GiB disk and four CPU threads/vCPUs are planning allocations. Eager recomputation with cache disabled may be slow; no CPU timing or complete-process fit was measured.

One-time setup and hardware

hardware
Unknown — Existing desktop purchase price and incremental hardware cost depend on the reader; unknown.
Existing desktop purchase price and incremental hardware cost depend on the reader; unknown. Tax: unknown.
setup
Unknown — Initial provisioning/build time and labor cost are unknown.
Initial provisioning/build time and labor cost are unknown. Tax: unknown.

Recurring costs

electricity
Unknown — Actual wall power, electricity rate and powered hours are unknown; no TDP-to-wall-power conversion.
Actual wall power, electricity rate and powered hours are unknown; no TDP-to-wall-power conversion. Tax: unknown.
storage
Unknown — Incremental cost of local disk capacity is unknown. The 40 GiB free-disk walkthrough allocation is an author choice, not a measured requirement.
Incremental cost of local disk capacity is unknown. The 40 GiB free-disk walkthrough allocation is an author choice, not a measured requirement. Tax: unknown.
egress
Unknown — Data transfer price, allowances and traffic volume are unknown.
Data transfer price, allowances and traffic volume are unknown. Tax: unknown.
administration
Unknown — Maintenance labor rate and hours are unknown.
Maintenance labor rate and hours are unknown. Tax: unknown.

Install the runtime

  1. 1. Prepare the native CPU environment

    Install the exact CPU dependency set in the dedicated virtual environment. Package compatibility is source-qualified; installation was not executed by the author.

    Command
    set -e
    sudo apt-get update
    sudo apt-get install -y python3.12 python3.12-venv curl ca-certificates openssl
    mkdir -p "$HOME/minicpm4-05b/model"
    cd "$HOME/minicpm4-05b"
    python3.12 -m venv .venv
    .venv/bin/python -m pip install 'https://download.pytorch.org/whl/cpu/torch-2.6.0%2Bcpu-cp312-cp312-linux_x86_64.whl#sha256=59e78aa0c690f70734e42670036d6b541930b8eabbaa18d94e090abf14cc4d91'
    .venv/bin/python -m pip install 'transformers==4.57.6' 'accelerate==1.10.1' 'safetensors==0.5.3' 'tokenizers==0.22.2' 'huggingface-hub==0.36.0' 'numpy==2.2.3'
    .venv/bin/python -m pip check
    .venv/bin/python -m pip freeze > resolved-environment.txt

    Expected result: Dedicated virtual environment, exact CPU packages and a passing dependency check; stop on any package or version error.

Download the model

  1. 1. Download the exact native checkpoint and source files

    Download all nine pinned model, tokenizer, configuration and custom-code assets. Streaming size/checksum checks fail fast. The native tokenizer JSON is used directly; no tokenizer conversion or extra vocabulary is created. Read the custom Python before continuing.

    Command
    set -eu
    cd "$HOME/minicpm4-05b"
    mkdir -p model
    curl -fL --retry 0 'https://huggingface.co/openbmb/MiniCPM4-0.5B/resolve/5253c7fcc5e29e1cf3eacb59a58adf1ba4df8630/config.json' -o 'model/config.json'
    curl -fL --retry 0 'https://huggingface.co/openbmb/MiniCPM4-0.5B/resolve/5253c7fcc5e29e1cf3eacb59a58adf1ba4df8630/generation_config.json' -o 'model/generation_config.json'
    curl -fL --retry 0 'https://huggingface.co/openbmb/MiniCPM4-0.5B/resolve/5253c7fcc5e29e1cf3eacb59a58adf1ba4df8630/configuration_minicpm.py' -o 'model/configuration_minicpm.py'
    curl -fL --retry 0 'https://huggingface.co/openbmb/MiniCPM4-0.5B/resolve/5253c7fcc5e29e1cf3eacb59a58adf1ba4df8630/modeling_minicpm.py' -o 'model/modeling_minicpm.py'
    curl -fL --retry 0 'https://huggingface.co/openbmb/MiniCPM4-0.5B/resolve/5253c7fcc5e29e1cf3eacb59a58adf1ba4df8630/tokenizer.json' -o 'model/tokenizer.json'
    curl -fL --retry 0 'https://huggingface.co/openbmb/MiniCPM4-0.5B/resolve/5253c7fcc5e29e1cf3eacb59a58adf1ba4df8630/tokenizer_config.json' -o 'model/tokenizer_config.json'
    curl -fL --retry 0 'https://huggingface.co/openbmb/MiniCPM4-0.5B/resolve/5253c7fcc5e29e1cf3eacb59a58adf1ba4df8630/special_tokens_map.json' -o 'model/special_tokens_map.json'
    curl -fL --retry 0 'https://huggingface.co/openbmb/MiniCPM4-0.5B/resolve/5253c7fcc5e29e1cf3eacb59a58adf1ba4df8630/added_tokens.json' -o 'model/added_tokens.json'
    curl -fL --retry 0 'https://huggingface.co/openbmb/MiniCPM4-0.5B/resolve/5253c7fcc5e29e1cf3eacb59a58adf1ba4df8630/model.safetensors' -o 'model/model.safetensors'
    cat > asset-manifest.json <<'JSONMANIFEST'
    {
      "model/config.json": {
        "bytes": 2403,
        "sha256": "610c5b519420a3633ec4950c15a26d83ad0e9be35ef9d00a9e3d2b621b325f72",
        "url": "https://huggingface.co/openbmb/MiniCPM4-0.5B/resolve/5253c7fcc5e29e1cf3eacb59a58adf1ba4df8630/config.json"
      },
      "model/generation_config.json": {
        "bytes": 182,
        "sha256": "83595cf3b045d7ed16593ae399b0e3c13fe45640a4bec3a1cd371b057a6a98b7",
        "url": "https://huggingface.co/openbmb/MiniCPM4-0.5B/resolve/5253c7fcc5e29e1cf3eacb59a58adf1ba4df8630/generation_config.json"
      },
      "model/configuration_minicpm.py": {
        "bytes": 9660,
        "sha256": "1e289912fdc0608ccad3983f4ff8b381ec8f161f45c85be4d694f92f17596b0a",
        "url": "https://huggingface.co/openbmb/MiniCPM4-0.5B/resolve/5253c7fcc5e29e1cf3eacb59a58adf1ba4df8630/configuration_minicpm.py"
      },
      "model/modeling_minicpm.py": {
        "bytes": 69994,
        "sha256": "c646022c44d0e1cc1d149fb80422a5d5f1ba6de66dd0f525f2be1e18d99994bf",
        "url": "https://huggingface.co/openbmb/MiniCPM4-0.5B/resolve/5253c7fcc5e29e1cf3eacb59a58adf1ba4df8630/modeling_minicpm.py"
      },
      "model/tokenizer.json": {
        "bytes": 6701371,
        "sha256": "adf7208af154a5ca065d2eda4e5419e02aac58c2c00627874748b75ec6769094",
        "url": "https://huggingface.co/openbmb/MiniCPM4-0.5B/resolve/5253c7fcc5e29e1cf3eacb59a58adf1ba4df8630/tokenizer.json"
      },
      "model/tokenizer_config.json": {
        "bytes": 2835,
        "sha256": "8bef75d59004eb0e62db89e97090d4736b14fd6c4a0282370c28833be1e83051",
        "url": "https://huggingface.co/openbmb/MiniCPM4-0.5B/resolve/5253c7fcc5e29e1cf3eacb59a58adf1ba4df8630/tokenizer_config.json"
      },
      "model/special_tokens_map.json": {
        "bytes": 630,
        "sha256": "9ec9d6f07cb0e436598bf29b4d97c6355434c08fdbe9c264e6926907f3751c7e",
        "url": "https://huggingface.co/openbmb/MiniCPM4-0.5B/resolve/5253c7fcc5e29e1cf3eacb59a58adf1ba4df8630/special_tokens_map.json"
      },
      "model/added_tokens.json": {
        "bytes": 216,
        "sha256": "4760fcbf90bc193f33827ffe02f2e7ba1af1ec43644cc02ac22fdd611f6cca15",
        "url": "https://huggingface.co/openbmb/MiniCPM4-0.5B/resolve/5253c7fcc5e29e1cf3eacb59a58adf1ba4df8630/added_tokens.json"
      },
      "model/model.safetensors": {
        "bytes": 867772648,
        "sha256": "ef74f4de1d1c580dd64e425d11f29881b9fd39f9be16a485150f727d4583181c",
        "url": "https://huggingface.co/openbmb/MiniCPM4-0.5B/resolve/5253c7fcc5e29e1cf3eacb59a58adf1ba4df8630/model.safetensors"
      }
    }
    JSONMANIFEST
    .venv/bin/python - <<'PYHASH'
    import pathlib,json,hashlib
    for name,info in json.load(open('asset-manifest.json')).items():
     p=pathlib.Path(name);digest=hashlib.sha256()
     with p.open('rb') as stream:
      for chunk in iter(lambda:stream.read(1024*1024),b''):digest.update(chunk)
     assert p.stat().st_size==info['bytes'] and digest.hexdigest()==info['sha256'],name
    print('All nine native asset checks passed.')
    PYHASH

    Expected result: All nine exact asset checks pass. Stop on an access error, byte-size mismatch or checksum mismatch.

Configure

  1. 1. Create a private key

    Keep the key in this private directory. Never print it or share it. Requests and health checks require the same key.

    Command
    set -eu
    cd "$HOME/minicpm4-05b"
    umask 077
    .venv/bin/python - <<'PYKEY'
    import pathlib,secrets
    p=pathlib.Path('.api-key')
    with p.open('x') as f:f.write(secrets.token_hex(32)+'\n')
    PYKEY

    Expected result: Private key file created with restrictive permissions; no key printed.

  2. 2. Write the checked native CPU loader

    The loader streams all asset checksums, rejects all native loading diagnostics and uses full FP32 eager CPU execution. Trust is limited to the exact reviewed local MiniCPM source. Cache is explicitly disabled at generation.

    Command
    cd "$HOME/minicpm4-05b"
    cat > reader-chat-server.py <<'PYSERVER1'
    import pathlib,json,hashlib,hmac
    from http.server import BaseHTTPRequestHandler,HTTPServer
    ROOT=pathlib.Path(__file__).resolve().parent
    PORT=8994
    
    def valid_prompt(prompt):
        return type(prompt) is str and bool(prompt.strip()) and len(prompt.encode('utf-8'))<=4096 and not any(x in prompt for x in ['<|','<s>','</s>'])
    
    def validate_completion(result):
        n,ids,answer=result
        if type(n) is not int or not 1<=n<=3584:raise ValueError('prompt_budget')
        if not isinstance(ids,list) or not 2<=len(ids)<512 or any(type(x) is not int or x<0 or x>=73448 for x in ids):raise ValueError('output_budget_or_ids')
        if ids[-1] not in (2,73440) or any(x in (2,73440) for x in ids[:-1]):raise ValueError('native_eos_required')
        if type(answer) is not str or not answer.strip() or len(answer.encode('utf-8'))>16384:raise ValueError('empty_or_excessive_answer')
        return {'answer':answer,'prompt_tokens':n,'generated_tokens_including_eos':len(ids),'complete_by_native_eos':True,'quality_measured':False}
    
    def validate_loading_info(info):
        fields={'missing_keys','unexpected_keys','mismatched_keys','error_msgs'}
        if type(info) is not dict or set(info)!=fields or any(type(info[k]) is not list or info[k] for k in fields):raise RuntimeError('incomplete_native_weight_load')
    
    def make_encoder():
        import os
        os.environ['HF_HUB_OFFLINE']='1';os.environ['TRANSFORMERS_OFFLINE']='1'
        import torch
        from transformers import AutoModelForCausalLM,PreTrainedTokenizerFast
        torch.set_num_threads(4)
        manifest=json.loads((ROOT/'asset-manifest.json').read_text())
        for name,info in manifest.items():
            file=ROOT/name;digest=hashlib.sha256()
            with file.open('rb') as stream:
                for chunk in iter(lambda:stream.read(1024*1024),b''):digest.update(chunk)
            if file.stat().st_size!=info['bytes'] or digest.hexdigest()!=info['sha256']:raise RuntimeError('asset_mismatch:'+name)
        tokenizer=PreTrainedTokenizerFast.from_pretrained(ROOT/'model',local_files_only=True)
        if tokenizer.bos_token_id!=1 or tokenizer.eos_token_id!=73440 or len(tokenizer)!=73448:raise RuntimeError('native_tokenizer_configuration')
        model,info=AutoModelForCausalLM.from_pretrained(ROOT/'model',local_files_only=True,trust_remote_code=True,
            dtype=torch.float32,attn_implementation='eager',use_safetensors=True,output_loading_info=True,low_cpu_mem_usage=True)
        validate_loading_info(info)
        cfg=model.config
        if not (cfg.model_type=='minicpm' and cfg.hidden_size==1024 and cfg.num_hidden_layers==24 and cfg.num_attention_heads==16 and cfg.num_key_value_heads==2 and cfg.max_position_embeddings==32768):raise RuntimeError('native_architecture')
        model=model.to('cpu').float().eval()
        for param in model.parameters():param.requires_grad_(False)
        template=json.loads((ROOT/'model/tokenizer_config.json').read_text())['chat_template']
        def encode(prompt):
            text=tokenizer.apply_chat_template([{'role':'user','content':prompt}],chat_template=template,tokenize=False,add_generation_prompt=True)
            inputs=tokenizer(text,add_special_tokens=True,return_tensors='pt',truncation=False)
            n=inputs['input_ids'].shape[1]
            if n<1 or n>3584 or inputs['input_ids'][0,0].item()!=1 or (inputs['input_ids']==1).sum().item()!=1:raise ValueError('prompt_budget_or_bos')
            with torch.inference_mode():
                full=model.generate(**inputs,max_new_tokens=512,do_sample=True,temperature=0.7,top_p=0.7,top_k=0,
                    repetition_penalty=1.0,num_beams=1,eos_token_id=[2,73440],pad_token_id=2,use_cache=False)
            if full.shape[0]!=1 or not torch.equal(full[0,:n],inputs['input_ids'][0]):raise ValueError('prefix_changed')
            ids=full[0,n:].tolist()
            return n,ids,tokenizer.decode(ids,skip_special_tokens=True)
        return encode
    
    PYSERVER1

    Expected result: Native loader and protocol validation written; no model loaded yet.

  3. 3. Write the private serial HTTP boundary

    Complete the server. It accepts one plain prompt with strict framing, authentication and byte/token limits; template control strings are rejected. It performs no tool action, URL fetch or file upload.

    Command
    cd "$HOME/minicpm4-05b"
    cat >> reader-chat-server.py <<'PYSERVER2'
    def create_server(encoder, key, port=PORT):
        if not isinstance(key, str) or len(key) != 64 or any(c not in '0123456789abcdef' for c in key):
            raise ValueError('key_format')
    
        class Handler(BaseHTTPRequestHandler):
            def setup(self):
                super().setup()
                self.connection.settimeout(5)
    
            def log_message(self, *_):
                pass  # Do not log Bearer keys or request bodies.
    
            def reply(self, status, obj):
                data = json.dumps(obj, allow_nan=False, separators=(',', ':')).encode()
                self.send_response(status)
                self.send_header('Content-Type', 'application/json')
                self.send_header('Content-Length', str(len(data)))
                self.send_header('Connection', 'close')
                self.end_headers()
                self.wfile.write(data)
                self.close_connection = True
    
            def authorized(self):
                values = self.headers.get_all('Authorization', [])
                return len(values) == 1 and hmac.compare_digest(values[0].encode(), ('Bearer ' + key).encode())
    
            def do_GET(self):
                if not self.authorized():
                    return self.reply(401, {'error': 'unauthorized'})
                if self.path != '/health':
                    return self.reply(404, {'error': 'not_found'})
                return self.reply(200, {'ready': True, 'context_tokens': 4096,
                                        'output_cap': 512})
    
            def do_POST(self):
                if not self.authorized():
                    return self.reply(401, {'error': 'unauthorized'})
                if self.path != '/v1/chat':
                    return self.reply(404, {'error': 'not_found'})
                if self.headers.get('Transfer-Encoding') is not None:
                    return self.reply(400, {'error': 'transfer_encoding_unsupported'})
                lengths = self.headers.get_all('Content-Length', [])
                if len(lengths) != 1:
                    return self.reply(411, {'error': 'content_length_required'})
                raw = lengths[0]
                if not raw or len(raw) > 4 or any(c not in '0123456789' for c in raw):
                    return self.reply(413, {'error': 'invalid_or_excessive_length'})
                n = int(raw)
                if n < 1 or n > 8192:
                    return self.reply(413, {'error': 'body_limit'})
                if self.headers.get_content_type() != 'application/json':
                    return self.reply(415, {'error': 'json_required'})
                try:
                    body = self.rfile.read(n)
                except (TimeoutError, OSError):
                    return self.reply(408, {'error': 'body_timeout'})
                if len(body) != n:
                    return self.reply(400, {'error': 'incomplete_body'})
                try:
                    def unique(pairs):
                        obj = {}
                        for k, v in pairs:
                            if k in obj:
                                raise ValueError('duplicate_key')
                            obj[k] = v
                        return obj
                    request = json.loads(body.decode('utf-8'), object_pairs_hook=unique,
                                         parse_constant=lambda _: (_ for _ in ()).throw(ValueError('constant')))
                except (ValueError, UnicodeError, RecursionError):
                    return self.reply(400, {'error': 'invalid_json'})
                if type(request) is not dict or set(request)!={'prompt'} or not valid_prompt(request['prompt']):
                    return self.reply(400, {'error': 'bounded_plain_prompt_required'})
                try:
                    result = validate_completion(encoder(request['prompt']))
                except Exception:
                    return self.reply(500, {'error': 'generation_or_protocol_failure'})
                return self.reply(200, result)
    
        return HTTPServer(('127.0.0.1', port), Handler)
    
    
    if __name__=='__main__':
        encoder=make_encoder()
        with create_server(encoder,(ROOT/'.api-key').read_text().strip()) as server:server.serve_forever()
    PYSERVER2

    Expected result: Complete private serial server written; model work starts only in the next step.

Start the model

  1. 1. Start the native model process

    Keep this foreground terminal open. Startup first validates all files and native loading diagnostics. Readiness occurs only after the actual model finishes loading; stop on any failure rather than relaxing checks.

    Command
    cd "$HOME/minicpm4-05b"
    .venv/bin/python reader-chat-server.py

    Expected result: Foreground process remains running after a successful native load. Memory fit and startup time must be observed by the reader; no result was measured here.

Connect to your endpoint

http://127.0.0.1:8994/v1/chat

Host loopback only; shown checks run in a second local terminal.

Access: loopback. Private reader-generated Bearer key for health and POST. Strict JSON contains only prompt; at most 4,096 UTF-8 prompt bytes and 8,192 body bytes. Response contains actual answer and observed counts. No provider API equivalence or external action is claimed.

Verify the setup

  1. 1. Check readiness, actual text completion and authentication

    Use a second terminal on the same host. Commands use the virtual environment by absolute relative path and require no prior activation. Inspect the actual answer and record semantic errors. EOS, shape and token-budget validity are separate from quality. A client timeout does not cancel native inference.

    Command
    set -eu
    cd "$HOME/minicpm4-05b"
    MINICPM_KEY=$(cat .api-key)
    export MINICPM_KEY
    curl -fsS --max-time 30 -H "Authorization: Bearer $MINICPM_KEY" http://127.0.0.1:8994/health -o health.json
    curl -fsS --max-time 1800 -H "Authorization: Bearer $MINICPM_KEY" -H 'Content-Type: application/json' --data '{"prompt":"Describe a quiet garden in two short sentences."}' http://127.0.0.1:8994/v1/chat -o answer.json
    .venv/bin/python - <<'PYCHECK'
    import json
    h=json.load(open('health.json'));r=json.load(open('answer.json'))
    assert h=={'ready':True,'context_tokens':4096,'output_cap':512}
    assert r['complete_by_native_eos'] is True and r['quality_measured'] is False
    assert 1<=r['prompt_tokens']<=3584 and 2<=r['generated_tokens_including_eos']<512
    assert isinstance(r['answer'],str) and r['answer'].strip()
    print(r['answer'])
    print('Protocol complete. Record whether this actual answer describes a garden and follows the requested length; this is human assessment, not a forced model result.')
    PYCHECK
    code=$(curl -sS --max-time 30 -o unauthorized.json -w '%{http_code}' http://127.0.0.1:8994/health)
    test "$code" = 401
    unset MINICPM_KEY

    Expected result: Authenticated health succeeds, an actual nonempty answer terminates at a native EOS below the cap, and unauthenticated health returns 401. No exact wording or quality is promised.

Performance evidence

No compatible measurement available. Documented support and estimated memory do not establish response speed.

Limits and troubleshooting

  • Only this exact native MiniCPM4-0.5B text checkpoint and bounded CPU path are source-qualified. Other MiniCPM4/4.1 sizes, sparse/hybrid reasoning, quantization, GPU, long-context, tool and newer-family workflows remain open.
  • The native configuration has 32,768 positions; this author selects a 4,096-token budget with 512 reserved output tokens. Tokenizer metadata contains a sentinel maximum rather than a usable model limit; it is not used as context evidence. Actual prompt/output counts remain unmeasured until execution.
  • The native template has no BOS; its pinned tokenizer postprocessor adds one BOS during the single tokenization pass. The wrapper rejects duplicate BOS, template-control input, truncated prompts, partial cap-length completions and every native loading diagnostic.
  • Custom native Python executes only after reader review of the exact pinned files. Source compatibility is not an executed installation or load. All model work uses CPU FP32 eager operations with cache disabled, recomputing the sequence rather than making a cache-performance claim.
  • The serial HTTP boundary has authentication, loopback, body and socket limits, but no generation deadline, public TLS or multiuser resilience. Socket/client timeouts do not cancel generation. Actual answer quality, fit and speed are unknown.
Asset, custom code or load diagnostic mismatch
Stop. Restore the exact pinned download and native runtime. Do not ignore missing, unexpected or mismatched keys, substitute a different model, enable network code fallback or relax checks.
Empty answer, no EOS, output cap or prompt overflow
Record the actual failure. Do not silently truncate inputs or accept partial answers. Keep the selected total budget and requalify any changed configuration.
Incorrect or unhelpful answer
Record the actual answer and human assessment. Protocol validity does not establish task quality or a benchmark score.
OOM, slow CPU or dependency error
Stop and record the actual environment and error. Planning capacity is not proven fit. Eager cache-disabled inference may be slow; any changed runtime or precision requires requalification.
401 or connection refused
Check foreground readiness, private key and loopback or SSH boundary. Keep the service port closed publicly and never print the key.

Stop the model

  1. 1. Stop the private model process

    Press Ctrl+C in the foreground terminal and unset the key in each client terminal. Confirm the loopback endpoint no longer responds.

    Command
    unset MINICPM_KEY

    Expected result: Private Python service stops and the model process exits.

Uninstall

  1. 1. Remove the dedicated workspace

    Back up wanted results and license records, verify the exact directory and remove only this workspace. Shared Ubuntu packages remain installed.

    Command
    rm -r -- "$HOME/minicpm4-05b"

    Expected result: Dedicated model, custom code, tokenizer, virtual environment, reader wrapper, outputs and private key removed.

Sources and verification

Sources checked Oct 16, 2026. Verification source

Compare this configuration
Cloud serverDocumented setupSources checked Oct 16, 2026

Serve MiniCPM4-0.5B text on a private CPU VM

Run the exact native MiniCPM4-0.5B checkpoint with its own chat template and pinned custom Python. This private serial text endpoint uses four CPU threads, full FP32 eager execution, a 4,096-token walkthrough budget and a 512-token output cap. Cache is disabled. Actual fit, speed and answer quality remain unmeasured.

Hardware
AWS r7i.xlarge, 32 GiB RAM, Ubuntu x86_64
Runtime
Native MiniCPM4-0.5B private eager CPU text service Transformers 4.57.6; torch 2.6.0 CPU; Python 3.12 · CPU
Model format
Safetensors with pinned native MiniCPM Python · Full native MiniCPM CPU FP32; publisher serialized checkpoint BF16; no quantization · Native BF16 serialized checkpoint; reader full FP32 CPU, no quantization
Workload
4,096 token context · 1 concurrent

Based on source documentation; this setup has not been executed by OpenTools. Memory: unknown. No compatible measurement available.

Read the complete setup guide

Before you start

  • Prepare an Ubuntu 24.04 x86_64 AWS r7i.xlarge in us-east-1 with 32 GiB RAM, four vCPUs and a 40 GiB gp3 root disk. Restrict SSH to your source IP and keep port 8994 closed publicly. Provisioning is a reader action and incurs the dated compute/storage charges below.
  • Read the exact native model card and actual Apache 2.0 license. This is the publisher-listed MiniCPM4-0.5B variant within frozen MiniCPM 4/4.1, not a substitute MiniCPM5 or a completion claim for every family member.
  • Review both hash-pinned custom Python files before enabling local trusted code. The wrapper loads only the verified local directory with offline flags and safe weight loading. No unreviewed remote code fallback is allowed.
  • Use Python 3.12 and a dedicated directory. This loopback serial demonstration has no inference deadline or public TLS/multiuser protection. Do not expose it publicly or use untrusted multi-tenant inputs.

Exact model and requirements

Artifact:
OpenBMB / Native MiniCPM4-0.5B · checkpoint openbmb/MiniCPM4-0.5B
Publisher revision:
5253c7fcc5e29e1cf3eacb59a58adf1ba4df8630
Artifact digest:
sha256:ef74f4de1d1c580dd64e425d11f29881b9fd39f9be16a485150f727d4583181c
Download size:
0.81 GiB
Memory requirement (unknown):
Unknown — Native full FP32 process/loading/activations/OS peak is unmeasured.
System RAM / disk:
Unknown / Unknown
Placement:
single device. One CPU host and one sequence. All native language model computation is on CPU; no GPU or disk offload.
KV cache:
Not allocated by this recipe: native use_cache=False; eager recomputation on CPU.
Workload:
4,096 token context; Unknown; Unknown; concurrency 1; batch 1; cache cold; repetitions Unknown.
Tokenizer:
Native BPE tokenizer JSON and exact MiniCPM4 chat template · Native checkpoint pin; tokenizers 0.22.2
Sampling / reasoning:
Publisher card uses temperature 0.7 and top_p 0.7. This walkthrough uses those values, top_k 0 (author choice), repetition penalty 1.0 and one beam, with native EOS IDs 2 and 73440 and pad ID 2. Output sampling is nondeterministic; no exact answer is promised. This exact small MiniCPM4 text checkpoint uses its native ordinary user/assistant template. No hybrid reasoning toggle is introduced. The native configuration allows 32,768 positions; 4,096 is an author-selected bounded walkthrough, not a claim of full-family or long-context support. Render the native template, then tokenize once with its BOS postprocessor. Disable native cache explicitly.
Runtime prerequisites:
Ubuntu 24.04 x86_64; See prerequisites above.

Official artifact · License

The exact native model card declares Apache 2.0 and links the publisher repository license. Preserve the license, applicable notices, modification notices and patent conditions. No inference quality, suitability, endorsement or legal clearance is claimed. Review the archived actual publisher license before use.

  • The exact serialized native weight file is 867,772,648 bytes. FP32 expansion, loading transients, tokenizer/runtime and OS buffers require additional memory; file size is not RAM fit.
  • 32 GiB RAM, 40 GiB disk and four CPU threads/vCPUs are planning allocations. Eager recomputation with cache disabled may be slow; no CPU timing or complete-process fit was measured.

One-time setup and hardware

hardware
Not applicable — AWS hardware acquisition is included in compute rental; no purchased GPU required. Any separate hardware cost is not applicable.
AWS hardware acquisition is included in compute rental; no purchased GPU required. Any separate hardware cost is not applicable. Tax: unknown.
setup
Unknown — Initial provisioning/build time and labor cost are unknown.
Initial provisioning/build time and labor cost are unknown. Tax: unknown.

Recurring costs

compute
$0.26 / hour
AWS public us-east-1 Linux On-Demand rate USD 0.2646 per billed instance-hour, no paid preinstalled software. Eight hours USD 2.1168; 24 hours USD 6.3504 compute. Idle running hours bill. Storage, egress, tax and administration separate; no Spot/reservation/discount assumed. Capacity/quota unverified. Region: us-east-1. SKU: r7i.xlarge Linux On-Demand. Tax: excluded. Price dated Oct 15, 2026. Price source
storage
$0.08 / GB month
US East (N. Virginia) gp3 USD 0.08 per GB-month; AWS defines GB as 1024^3 bytes. Selected 40 GiB root volume is USD 3.20 per full month, prorated by provisioned duration. Baseline 3000 IOPS / 125 MB/s; no extras assumed. Retained snapshots/resources separate; storage persists after instance stop. Tax excluded. Region: us-east-1. SKU: EBS gp3 baseline 3000 IOPS / 125 MB/s. Tax: excluded. Price dated Oct 16, 2026. Price source
egress
Unknown — Data transfer price, allowances and traffic volume are unknown.
Data transfer price, allowances and traffic volume are unknown. Tax: unknown.
administration
Unknown — Maintenance labor rate and hours are unknown.
Maintenance labor rate and hours are unknown. Tax: unknown.

Install the runtime

  1. 1. Prepare the native CPU environment

    Install the exact CPU dependency set in the dedicated virtual environment. Package compatibility is source-qualified; installation was not executed by the author.

    Command
    set -e
    sudo apt-get update
    sudo apt-get install -y python3.12 python3.12-venv curl ca-certificates openssl
    mkdir -p "$HOME/minicpm4-05b/model"
    cd "$HOME/minicpm4-05b"
    python3.12 -m venv .venv
    .venv/bin/python -m pip install 'https://download.pytorch.org/whl/cpu/torch-2.6.0%2Bcpu-cp312-cp312-linux_x86_64.whl#sha256=59e78aa0c690f70734e42670036d6b541930b8eabbaa18d94e090abf14cc4d91'
    .venv/bin/python -m pip install 'transformers==4.57.6' 'accelerate==1.10.1' 'safetensors==0.5.3' 'tokenizers==0.22.2' 'huggingface-hub==0.36.0' 'numpy==2.2.3'
    .venv/bin/python -m pip check
    .venv/bin/python -m pip freeze > resolved-environment.txt

    Expected result: Dedicated virtual environment, exact CPU packages and a passing dependency check; stop on any package or version error.

Download the model

  1. 1. Download the exact native checkpoint and source files

    Download all nine pinned model, tokenizer, configuration and custom-code assets. Streaming size/checksum checks fail fast. The native tokenizer JSON is used directly; no tokenizer conversion or extra vocabulary is created. Read the custom Python before continuing.

    Command
    set -eu
    cd "$HOME/minicpm4-05b"
    mkdir -p model
    curl -fL --retry 0 'https://huggingface.co/openbmb/MiniCPM4-0.5B/resolve/5253c7fcc5e29e1cf3eacb59a58adf1ba4df8630/config.json' -o 'model/config.json'
    curl -fL --retry 0 'https://huggingface.co/openbmb/MiniCPM4-0.5B/resolve/5253c7fcc5e29e1cf3eacb59a58adf1ba4df8630/generation_config.json' -o 'model/generation_config.json'
    curl -fL --retry 0 'https://huggingface.co/openbmb/MiniCPM4-0.5B/resolve/5253c7fcc5e29e1cf3eacb59a58adf1ba4df8630/configuration_minicpm.py' -o 'model/configuration_minicpm.py'
    curl -fL --retry 0 'https://huggingface.co/openbmb/MiniCPM4-0.5B/resolve/5253c7fcc5e29e1cf3eacb59a58adf1ba4df8630/modeling_minicpm.py' -o 'model/modeling_minicpm.py'
    curl -fL --retry 0 'https://huggingface.co/openbmb/MiniCPM4-0.5B/resolve/5253c7fcc5e29e1cf3eacb59a58adf1ba4df8630/tokenizer.json' -o 'model/tokenizer.json'
    curl -fL --retry 0 'https://huggingface.co/openbmb/MiniCPM4-0.5B/resolve/5253c7fcc5e29e1cf3eacb59a58adf1ba4df8630/tokenizer_config.json' -o 'model/tokenizer_config.json'
    curl -fL --retry 0 'https://huggingface.co/openbmb/MiniCPM4-0.5B/resolve/5253c7fcc5e29e1cf3eacb59a58adf1ba4df8630/special_tokens_map.json' -o 'model/special_tokens_map.json'
    curl -fL --retry 0 'https://huggingface.co/openbmb/MiniCPM4-0.5B/resolve/5253c7fcc5e29e1cf3eacb59a58adf1ba4df8630/added_tokens.json' -o 'model/added_tokens.json'
    curl -fL --retry 0 'https://huggingface.co/openbmb/MiniCPM4-0.5B/resolve/5253c7fcc5e29e1cf3eacb59a58adf1ba4df8630/model.safetensors' -o 'model/model.safetensors'
    cat > asset-manifest.json <<'JSONMANIFEST'
    {
      "model/config.json": {
        "bytes": 2403,
        "sha256": "610c5b519420a3633ec4950c15a26d83ad0e9be35ef9d00a9e3d2b621b325f72",
        "url": "https://huggingface.co/openbmb/MiniCPM4-0.5B/resolve/5253c7fcc5e29e1cf3eacb59a58adf1ba4df8630/config.json"
      },
      "model/generation_config.json": {
        "bytes": 182,
        "sha256": "83595cf3b045d7ed16593ae399b0e3c13fe45640a4bec3a1cd371b057a6a98b7",
        "url": "https://huggingface.co/openbmb/MiniCPM4-0.5B/resolve/5253c7fcc5e29e1cf3eacb59a58adf1ba4df8630/generation_config.json"
      },
      "model/configuration_minicpm.py": {
        "bytes": 9660,
        "sha256": "1e289912fdc0608ccad3983f4ff8b381ec8f161f45c85be4d694f92f17596b0a",
        "url": "https://huggingface.co/openbmb/MiniCPM4-0.5B/resolve/5253c7fcc5e29e1cf3eacb59a58adf1ba4df8630/configuration_minicpm.py"
      },
      "model/modeling_minicpm.py": {
        "bytes": 69994,
        "sha256": "c646022c44d0e1cc1d149fb80422a5d5f1ba6de66dd0f525f2be1e18d99994bf",
        "url": "https://huggingface.co/openbmb/MiniCPM4-0.5B/resolve/5253c7fcc5e29e1cf3eacb59a58adf1ba4df8630/modeling_minicpm.py"
      },
      "model/tokenizer.json": {
        "bytes": 6701371,
        "sha256": "adf7208af154a5ca065d2eda4e5419e02aac58c2c00627874748b75ec6769094",
        "url": "https://huggingface.co/openbmb/MiniCPM4-0.5B/resolve/5253c7fcc5e29e1cf3eacb59a58adf1ba4df8630/tokenizer.json"
      },
      "model/tokenizer_config.json": {
        "bytes": 2835,
        "sha256": "8bef75d59004eb0e62db89e97090d4736b14fd6c4a0282370c28833be1e83051",
        "url": "https://huggingface.co/openbmb/MiniCPM4-0.5B/resolve/5253c7fcc5e29e1cf3eacb59a58adf1ba4df8630/tokenizer_config.json"
      },
      "model/special_tokens_map.json": {
        "bytes": 630,
        "sha256": "9ec9d6f07cb0e436598bf29b4d97c6355434c08fdbe9c264e6926907f3751c7e",
        "url": "https://huggingface.co/openbmb/MiniCPM4-0.5B/resolve/5253c7fcc5e29e1cf3eacb59a58adf1ba4df8630/special_tokens_map.json"
      },
      "model/added_tokens.json": {
        "bytes": 216,
        "sha256": "4760fcbf90bc193f33827ffe02f2e7ba1af1ec43644cc02ac22fdd611f6cca15",
        "url": "https://huggingface.co/openbmb/MiniCPM4-0.5B/resolve/5253c7fcc5e29e1cf3eacb59a58adf1ba4df8630/added_tokens.json"
      },
      "model/model.safetensors": {
        "bytes": 867772648,
        "sha256": "ef74f4de1d1c580dd64e425d11f29881b9fd39f9be16a485150f727d4583181c",
        "url": "https://huggingface.co/openbmb/MiniCPM4-0.5B/resolve/5253c7fcc5e29e1cf3eacb59a58adf1ba4df8630/model.safetensors"
      }
    }
    JSONMANIFEST
    .venv/bin/python - <<'PYHASH'
    import pathlib,json,hashlib
    for name,info in json.load(open('asset-manifest.json')).items():
     p=pathlib.Path(name);digest=hashlib.sha256()
     with p.open('rb') as stream:
      for chunk in iter(lambda:stream.read(1024*1024),b''):digest.update(chunk)
     assert p.stat().st_size==info['bytes'] and digest.hexdigest()==info['sha256'],name
    print('All nine native asset checks passed.')
    PYHASH

    Expected result: All nine exact asset checks pass. Stop on an access error, byte-size mismatch or checksum mismatch.

Configure

  1. 1. Create a private key

    Keep the key in this private directory. Never print it or share it. Requests and health checks require the same key.

    Command
    set -eu
    cd "$HOME/minicpm4-05b"
    umask 077
    .venv/bin/python - <<'PYKEY'
    import pathlib,secrets
    p=pathlib.Path('.api-key')
    with p.open('x') as f:f.write(secrets.token_hex(32)+'\n')
    PYKEY

    Expected result: Private key file created with restrictive permissions; no key printed.

  2. 2. Write the checked native CPU loader

    The loader streams all asset checksums, rejects all native loading diagnostics and uses full FP32 eager CPU execution. Trust is limited to the exact reviewed local MiniCPM source. Cache is explicitly disabled at generation.

    Command
    cd "$HOME/minicpm4-05b"
    cat > reader-chat-server.py <<'PYSERVER1'
    import pathlib,json,hashlib,hmac
    from http.server import BaseHTTPRequestHandler,HTTPServer
    ROOT=pathlib.Path(__file__).resolve().parent
    PORT=8994
    
    def valid_prompt(prompt):
        return type(prompt) is str and bool(prompt.strip()) and len(prompt.encode('utf-8'))<=4096 and not any(x in prompt for x in ['<|','<s>','</s>'])
    
    def validate_completion(result):
        n,ids,answer=result
        if type(n) is not int or not 1<=n<=3584:raise ValueError('prompt_budget')
        if not isinstance(ids,list) or not 2<=len(ids)<512 or any(type(x) is not int or x<0 or x>=73448 for x in ids):raise ValueError('output_budget_or_ids')
        if ids[-1] not in (2,73440) or any(x in (2,73440) for x in ids[:-1]):raise ValueError('native_eos_required')
        if type(answer) is not str or not answer.strip() or len(answer.encode('utf-8'))>16384:raise ValueError('empty_or_excessive_answer')
        return {'answer':answer,'prompt_tokens':n,'generated_tokens_including_eos':len(ids),'complete_by_native_eos':True,'quality_measured':False}
    
    def validate_loading_info(info):
        fields={'missing_keys','unexpected_keys','mismatched_keys','error_msgs'}
        if type(info) is not dict or set(info)!=fields or any(type(info[k]) is not list or info[k] for k in fields):raise RuntimeError('incomplete_native_weight_load')
    
    def make_encoder():
        import os
        os.environ['HF_HUB_OFFLINE']='1';os.environ['TRANSFORMERS_OFFLINE']='1'
        import torch
        from transformers import AutoModelForCausalLM,PreTrainedTokenizerFast
        torch.set_num_threads(4)
        manifest=json.loads((ROOT/'asset-manifest.json').read_text())
        for name,info in manifest.items():
            file=ROOT/name;digest=hashlib.sha256()
            with file.open('rb') as stream:
                for chunk in iter(lambda:stream.read(1024*1024),b''):digest.update(chunk)
            if file.stat().st_size!=info['bytes'] or digest.hexdigest()!=info['sha256']:raise RuntimeError('asset_mismatch:'+name)
        tokenizer=PreTrainedTokenizerFast.from_pretrained(ROOT/'model',local_files_only=True)
        if tokenizer.bos_token_id!=1 or tokenizer.eos_token_id!=73440 or len(tokenizer)!=73448:raise RuntimeError('native_tokenizer_configuration')
        model,info=AutoModelForCausalLM.from_pretrained(ROOT/'model',local_files_only=True,trust_remote_code=True,
            dtype=torch.float32,attn_implementation='eager',use_safetensors=True,output_loading_info=True,low_cpu_mem_usage=True)
        validate_loading_info(info)
        cfg=model.config
        if not (cfg.model_type=='minicpm' and cfg.hidden_size==1024 and cfg.num_hidden_layers==24 and cfg.num_attention_heads==16 and cfg.num_key_value_heads==2 and cfg.max_position_embeddings==32768):raise RuntimeError('native_architecture')
        model=model.to('cpu').float().eval()
        for param in model.parameters():param.requires_grad_(False)
        template=json.loads((ROOT/'model/tokenizer_config.json').read_text())['chat_template']
        def encode(prompt):
            text=tokenizer.apply_chat_template([{'role':'user','content':prompt}],chat_template=template,tokenize=False,add_generation_prompt=True)
            inputs=tokenizer(text,add_special_tokens=True,return_tensors='pt',truncation=False)
            n=inputs['input_ids'].shape[1]
            if n<1 or n>3584 or inputs['input_ids'][0,0].item()!=1 or (inputs['input_ids']==1).sum().item()!=1:raise ValueError('prompt_budget_or_bos')
            with torch.inference_mode():
                full=model.generate(**inputs,max_new_tokens=512,do_sample=True,temperature=0.7,top_p=0.7,top_k=0,
                    repetition_penalty=1.0,num_beams=1,eos_token_id=[2,73440],pad_token_id=2,use_cache=False)
            if full.shape[0]!=1 or not torch.equal(full[0,:n],inputs['input_ids'][0]):raise ValueError('prefix_changed')
            ids=full[0,n:].tolist()
            return n,ids,tokenizer.decode(ids,skip_special_tokens=True)
        return encode
    
    PYSERVER1

    Expected result: Native loader and protocol validation written; no model loaded yet.

  3. 3. Write the private serial HTTP boundary

    Complete the server. It accepts one plain prompt with strict framing, authentication and byte/token limits; template control strings are rejected. It performs no tool action, URL fetch or file upload.

    Command
    cd "$HOME/minicpm4-05b"
    cat >> reader-chat-server.py <<'PYSERVER2'
    def create_server(encoder, key, port=PORT):
        if not isinstance(key, str) or len(key) != 64 or any(c not in '0123456789abcdef' for c in key):
            raise ValueError('key_format')
    
        class Handler(BaseHTTPRequestHandler):
            def setup(self):
                super().setup()
                self.connection.settimeout(5)
    
            def log_message(self, *_):
                pass  # Do not log Bearer keys or request bodies.
    
            def reply(self, status, obj):
                data = json.dumps(obj, allow_nan=False, separators=(',', ':')).encode()
                self.send_response(status)
                self.send_header('Content-Type', 'application/json')
                self.send_header('Content-Length', str(len(data)))
                self.send_header('Connection', 'close')
                self.end_headers()
                self.wfile.write(data)
                self.close_connection = True
    
            def authorized(self):
                values = self.headers.get_all('Authorization', [])
                return len(values) == 1 and hmac.compare_digest(values[0].encode(), ('Bearer ' + key).encode())
    
            def do_GET(self):
                if not self.authorized():
                    return self.reply(401, {'error': 'unauthorized'})
                if self.path != '/health':
                    return self.reply(404, {'error': 'not_found'})
                return self.reply(200, {'ready': True, 'context_tokens': 4096,
                                        'output_cap': 512})
    
            def do_POST(self):
                if not self.authorized():
                    return self.reply(401, {'error': 'unauthorized'})
                if self.path != '/v1/chat':
                    return self.reply(404, {'error': 'not_found'})
                if self.headers.get('Transfer-Encoding') is not None:
                    return self.reply(400, {'error': 'transfer_encoding_unsupported'})
                lengths = self.headers.get_all('Content-Length', [])
                if len(lengths) != 1:
                    return self.reply(411, {'error': 'content_length_required'})
                raw = lengths[0]
                if not raw or len(raw) > 4 or any(c not in '0123456789' for c in raw):
                    return self.reply(413, {'error': 'invalid_or_excessive_length'})
                n = int(raw)
                if n < 1 or n > 8192:
                    return self.reply(413, {'error': 'body_limit'})
                if self.headers.get_content_type() != 'application/json':
                    return self.reply(415, {'error': 'json_required'})
                try:
                    body = self.rfile.read(n)
                except (TimeoutError, OSError):
                    return self.reply(408, {'error': 'body_timeout'})
                if len(body) != n:
                    return self.reply(400, {'error': 'incomplete_body'})
                try:
                    def unique(pairs):
                        obj = {}
                        for k, v in pairs:
                            if k in obj:
                                raise ValueError('duplicate_key')
                            obj[k] = v
                        return obj
                    request = json.loads(body.decode('utf-8'), object_pairs_hook=unique,
                                         parse_constant=lambda _: (_ for _ in ()).throw(ValueError('constant')))
                except (ValueError, UnicodeError, RecursionError):
                    return self.reply(400, {'error': 'invalid_json'})
                if type(request) is not dict or set(request)!={'prompt'} or not valid_prompt(request['prompt']):
                    return self.reply(400, {'error': 'bounded_plain_prompt_required'})
                try:
                    result = validate_completion(encoder(request['prompt']))
                except Exception:
                    return self.reply(500, {'error': 'generation_or_protocol_failure'})
                return self.reply(200, result)
    
        return HTTPServer(('127.0.0.1', port), Handler)
    
    
    if __name__=='__main__':
        encoder=make_encoder()
        with create_server(encoder,(ROOT/'.api-key').read_text().strip()) as server:server.serve_forever()
    PYSERVER2

    Expected result: Complete private serial server written; model work starts only in the next step.

Start the model

  1. 1. Start the native model process

    Keep this foreground terminal open. Startup first validates all files and native loading diagnostics. Readiness occurs only after the actual model finishes loading; stop on any failure rather than relaxing checks.

    Command
    cd "$HOME/minicpm4-05b"
    .venv/bin/python reader-chat-server.py

    Expected result: Foreground process remains running after a successful native load. Memory fit and startup time must be observed by the reader; no result was measured here.

Connect to your endpoint

http://127.0.0.1:8994/v1/chat

VM loopback only. Optional reader SSH tunnel: ssh -N -L 8994:127.0.0.1:8994 USER@VM_IP. Keep public port 8994 closed. Shown checks run on the VM and read its private local key.

Access: loopback. Private reader-generated Bearer key for health and POST. Strict JSON contains only prompt; at most 4,096 UTF-8 prompt bytes and 8,192 body bytes. Response contains actual answer and observed counts. No provider API equivalence or external action is claimed.

Verify the setup

  1. 1. Check readiness, actual text completion and authentication

    Use a second terminal on the same host. Commands use the virtual environment by absolute relative path and require no prior activation. Inspect the actual answer and record semantic errors. EOS, shape and token-budget validity are separate from quality. A client timeout does not cancel native inference.

    Command
    set -eu
    cd "$HOME/minicpm4-05b"
    MINICPM_KEY=$(cat .api-key)
    export MINICPM_KEY
    curl -fsS --max-time 30 -H "Authorization: Bearer $MINICPM_KEY" http://127.0.0.1:8994/health -o health.json
    curl -fsS --max-time 1800 -H "Authorization: Bearer $MINICPM_KEY" -H 'Content-Type: application/json' --data '{"prompt":"Describe a quiet garden in two short sentences."}' http://127.0.0.1:8994/v1/chat -o answer.json
    .venv/bin/python - <<'PYCHECK'
    import json
    h=json.load(open('health.json'));r=json.load(open('answer.json'))
    assert h=={'ready':True,'context_tokens':4096,'output_cap':512}
    assert r['complete_by_native_eos'] is True and r['quality_measured'] is False
    assert 1<=r['prompt_tokens']<=3584 and 2<=r['generated_tokens_including_eos']<512
    assert isinstance(r['answer'],str) and r['answer'].strip()
    print(r['answer'])
    print('Protocol complete. Record whether this actual answer describes a garden and follows the requested length; this is human assessment, not a forced model result.')
    PYCHECK
    code=$(curl -sS --max-time 30 -o unauthorized.json -w '%{http_code}' http://127.0.0.1:8994/health)
    test "$code" = 401
    unset MINICPM_KEY

    Expected result: Authenticated health succeeds, an actual nonempty answer terminates at a native EOS below the cap, and unauthenticated health returns 401. No exact wording or quality is promised.

Performance evidence

No compatible measurement available. Documented support and estimated memory do not establish response speed.

Limits and troubleshooting

  • Only this exact native MiniCPM4-0.5B text checkpoint and bounded CPU path are source-qualified. Other MiniCPM4/4.1 sizes, sparse/hybrid reasoning, quantization, GPU, long-context, tool and newer-family workflows remain open.
  • The native configuration has 32,768 positions; this author selects a 4,096-token budget with 512 reserved output tokens. Tokenizer metadata contains a sentinel maximum rather than a usable model limit; it is not used as context evidence. Actual prompt/output counts remain unmeasured until execution.
  • The native template has no BOS; its pinned tokenizer postprocessor adds one BOS during the single tokenization pass. The wrapper rejects duplicate BOS, template-control input, truncated prompts, partial cap-length completions and every native loading diagnostic.
  • Custom native Python executes only after reader review of the exact pinned files. Source compatibility is not an executed installation or load. All model work uses CPU FP32 eager operations with cache disabled, recomputing the sequence rather than making a cache-performance claim.
  • The serial HTTP boundary has authentication, loopback, body and socket limits, but no generation deadline, public TLS or multiuser resilience. Socket/client timeouts do not cancel generation. Actual answer quality, fit and speed are unknown.
Asset, custom code or load diagnostic mismatch
Stop. Restore the exact pinned download and native runtime. Do not ignore missing, unexpected or mismatched keys, substitute a different model, enable network code fallback or relax checks.
Empty answer, no EOS, output cap or prompt overflow
Record the actual failure. Do not silently truncate inputs or accept partial answers. Keep the selected total budget and requalify any changed configuration.
Incorrect or unhelpful answer
Record the actual answer and human assessment. Protocol validity does not establish task quality or a benchmark score.
OOM, slow CPU or dependency error
Stop and record the actual environment and error. Planning capacity is not proven fit. Eager cache-disabled inference may be slow; any changed runtime or precision requires requalification.
401 or connection refused
Check foreground readiness, private key and loopback or SSH boundary. Keep the service port closed publicly and never print the key.

Stop the model

  1. 1. Stop the private model process

    Press Ctrl+C in the foreground terminal and unset the key in each client terminal. Close the SSH tunnel; stopping Python does not stop AWS billing.

    Command
    unset MINICPM_KEY

    Expected result: Private Python service stops and the model process exits.

Uninstall

  1. 1. Remove the dedicated workspace

    Back up wanted results and license records, verify the exact directory and remove only this workspace. Shared Ubuntu packages remain installed.

    Command
    rm -r -- "$HOME/minicpm4-05b"

    Expected result: Dedicated model, custom code, tokenizer, virtual environment, reader wrapper, outputs and private key removed.

Shut down cloud resources

  1. 1. End cloud billing resources

    In EC2 us-east-1, stop for later reuse or terminate this exact VM. Review Delete on termination for its root volume. Delete unwanted detached EBS volumes, snapshots and reserved/Elastic/public IP resources. Stopped VM storage and retained resources may still bill. Check Bills/Cost Explorer later for final usage.

    Expected result: Exact instance is stopped/terminated as intended; no unwanted attached/retained billed resources remain. Instance compute stops only after its lifecycle transition, not when the Python service stops.

Sources and verification

Sources checked Oct 16, 2026. Verification source

Compare this configuration

Benchmarks

Performance scores for MiniCPM across standard benchmarks.

MiniCPM-SALA standard benchmark averageofficial-github-readme · May 2026
76.5
MiniCPM-SALA long-context averageofficial-github-readme · May 2026
39
MiniCPM-SALA 2048K extrapolation scoreofficial-github-readme · May 2026
81.6
MiniCPM4.1 reasoning decoding speedupofficial-github-readme · May 2026
3
MiniCPM4 Jetson AGX Orin decoding speedup vs Qwen3-8Bofficial-github-readme · May 2026
7

Pricing and access

Open-weight GitHub and Hugging Face model family. There is no fixed vendor API price; runtime cost depends on the host, hardware, or inference provider.

OpenBMB logo
OpenBMBopen source

Open-source efficient AI models and agent infrastructure

1 ModelsFounded 2022China
View full profile

AI Models by OpenBMB

AI models from the same organization.

ModelContext WindowPrice (In / Out per M)
VoxCPMCurrent4K-- / --