Hermes 4 405B

Exact model and requirements

Artifact:

Nous Research / Hermes 4 405B full native BF16 dense Llama checkpoint · checkpoint NousResearch/Hermes-4-405B

Publisher revision:

88e3dce03c4a5535e2f4a2bcc08e939a2b302f82

Artifact digest:

No published whole-checkpoint SHA; the complete fixed reader manifest pins all 191 weight parts and eight required auxiliary files separately.

Download size:

755.96 GiB

Memory requirement (unknown):

Unknown — Full native BF16 target, TP8 partitions, KV, collective/kernel workspaces, runtime and peak per-GPU memory have not been measured.

System RAM / disk:

Unknown / Unknown

Placement:

supported sharding. Native vLLM Llama tensor parallel eight; 128 query heads and eight KV heads are divisible by eight. Per-device weight balance and peak usage are not measured. Full target on eight GPUs; no CPU/disk weight offload. Host staging and download buffers remain additional.

KV cache:

BF16 attention K/V via explicit model dtype and auto cache dtype

Workload:

8,192 token context; Unknown; Unknown; concurrency 1; batch 1; cache cold; repetitions Unknown.

Tokenizer:

Exact native Hermes 4 tokenizer and 4,121-byte Llama-header Jinja · 88e3dce03c4a5535e2f4a2bcc08e939a2b302f82

Sampling / reasoning:

Publisher card recommendation temperature 0.6, top_p 0.95, top_k 20. Native generation_config instead has top_p 0.9; this selects the documented card recipe deliberately. No sampler inherited from Hermes 3 or 70B. Native thinking=True and keep_cots=True. Count one BOS including full tools and unmodified raw reasoning/tool history. Own strict client reads raw completions, requires complete think/tool boundaries and stop rather than length, then appends only a fixed synthetic tool response without executing any tool. Native generation EOS 128001 differs from tokenizer EOT 128009; both stop IDs explicitly sent. All native text capabilities retained; no vision architecture is asserted.

Runtime prerequisites:

Ubuntu 24.04 x86_64 host; pinned Ubuntu 22.04 / Python 3.12 source defaults container; NVIDIA Linux driver >=580.126.20, Working Docker GPU support with NVIDIA Container Toolkit; actual visibility preflight required.

Official artifact · License

Publisher card metadata declares llama3 while naming the Llama 3.1 405B base. The native LICENSE and USE_POLICY probes returned scoped 404; this does not erase its explicit metadata-linked grant. Full official Meta Llama 3 and stated-base Llama 3.1 agreements/AUP are archived separately, with no equivalence or publisher-intent claim. Retain both applicable agreement copies, notices and attribution (Built with Meta Llama 3 / Built with Llama); comply with redistribution and derivative naming requirements and both AUPs. Each agreement has its own release-date 700-million-MAU prior-month commercial license trigger. Preserve Llama 3 restrictions on using materials/outputs to improve other large language models, subject to its stated derivative exception. Patent/IP litigation, termination, warranty and liability conditions apply. This inert own-use walkthrough provides no legal clearance and accepts no terms on the reader behalf.

• README.md · 0 GiB · revision 88e3dce03c4a5535e2f4a2bcc08e939a2b302f82
• chat_template.jinja · 0 GiB · revision 88e3dce03c4a5535e2f4a2bcc08e939a2b302f82
• config.json · 0 GiB · revision 88e3dce03c4a5535e2f4a2bcc08e939a2b302f82
• generation_config.json · 0 GiB · revision 88e3dce03c4a5535e2f4a2bcc08e939a2b302f82
• model.safetensors.index.json · 0 GiB · revision 88e3dce03c4a5535e2f4a2bcc08e939a2b302f82
• special_tokens_map.json · 0 GiB · revision 88e3dce03c4a5535e2f4a2bcc08e939a2b302f82
• tokenizer.json · 0.02 GiB · revision 88e3dce03c4a5535e2f4a2bcc08e939a2b302f82
• tokenizer_config.json · 0 GiB · revision 88e3dce03c4a5535e2f4a2bcc08e939a2b302f82

• 811,706,916,800 serialized weight bytes are a full-model storage lower bound, not total memory or peak. Tensor-index metadata and file sizes are separate accounting.
• Eight advertised 141 GB H200 devices are not silently converted into GiB. Usable per-device bytes and runtime partition balance are unknown; no aggregate-fit claim.
• Native Llama has 126 dense layers, width 16,384, 128 query/eight KV heads and no MoE active-parameter shortcut. Host staging, Docker image and tokenizer add overhead.

Install the runtime

1. 1. Prepare an already owned eight-H200 instance

   Use an existing Ubuntu 24.04 x86_64 p5en.48xlarge with eight H200 devices and working driver >=580.126.20. These commands install host utilities only; provisioning, quota, price and all spending are reader responsibilities.

   CommandCopy command

   set -eu
   sudo apt-get update
   sudo apt-get install -y docker.io python3 curl ca-certificates openssl
   sudo systemctl enable --now docker
   mkdir -p "$HOME/hermes4-405b"
   cd "$HOME/hermes4-405b"
   nvidia-smi --query-gpu=name,memory.total,driver_version --format=csv,noheader
   python3 - <<'PYPREFLIGHT'
   import csv,subprocess,shutil
   r=list(csv.reader(subprocess.check_output(["nvidia-smi","--query-gpu=name,memory.total,driver_version","--format=csv,noheader,nounits"],text=True).splitlines()))
   assert len(r)==8 and all("H200" in x[0] for x in r)
   assert all(tuple(map(int,x[2].strip().split("."))) >= (580,126,20) for x in r)
   assert shutil.disk_usage(".").free >=1800*1024**3
   print("Eight H200 devices and prescribed disk/driver checks passed; no model fit measured.")
   PYPREFLIGHT
   sudo docker version

   Expected result: Record eight actual GPU identities, usable memory and drivers. Stop on missing devices, driver/GPU Docker failure or inadequate prescribed disk. No native fit claim.

2. 2. Configure NVIDIA GPU containers on this dedicated host

   Use the official production repository and exact documented Container Toolkit 1.20.1-1 packages. This modifies Docker daemon configuration and restarts Docker on this dedicated host; preserve its original configuration and do not apply on a shared production daemon without its owner review. Stop on repository/signature/package errors.

   CommandCopy command

   set -euo pipefail
   sudo apt-get install -y gnupg2
   curl --fail --silent --show-error --location https://nvidia.github.io/libnvidia-container/gpgkey | sudo gpg --batch --yes --dearmor -o /usr/share/keyrings/nvidia-container-toolkit-keyring.gpg
   curl --fail --silent --show-error --location https://nvidia.github.io/libnvidia-container/stable/deb/nvidia-container-toolkit.list | sed 's#deb https://#deb [signed-by=/usr/share/keyrings/nvidia-container-toolkit-keyring.gpg] https://#g' | sudo tee /etc/apt/sources.list.d/nvidia-container-toolkit.list >/dev/null
   sudo apt-get update
   sudo apt-get install -y nvidia-container-toolkit=1.20.1-1 nvidia-container-toolkit-base=1.20.1-1 libnvidia-container-tools=1.20.1-1 libnvidia-container1=1.20.1-1
   sudo nvidia-ctk runtime configure --runtime=docker
   sudo systemctl restart docker

   Expected result: The exact GPU container packages install and Docker is configured. The subsequent container nvidia-smi check must pass; this is not model inference or fit proof.

Download the model

1. 1. Create stable private service credentials and asset checker

   Only create the dedicated key when absent; existing empty or malformed credentials fail. The fixed checker verifies its own complete manifest before any download or model import.

   CommandCopy command

   set -eu
   cd "$HOME/hermes4-405b"
   umask 077
   python3 - <<'PYPREWRITE'
   import os,pathlib,stat
   paths=[pathlib.Path(n)for n in [".api-key",".api-env"]]
   for p in paths:
    try:s=p.lstat()
    except FileNotFoundError:continue
    if not stat.S_ISREG(s.st_mode):raise ValueError('existing secret target must be a regular file; no links')
   for p in paths:
    if p.exists():os.chmod(p,0o600)
   PYPREWRITE
   if [ ! -e .api-key ]; then openssl rand -hex 32 > .api-key; fi
   python3 - <<'PYKEY'
   from pathlib import Path
   k=Path(".api-key").read_text().strip()
   assert len(k)==64 and all(c in "0123456789abcdef" for c in k)
   Path(".api-env").write_text("VLLM_API_KEY="+k+"\nHF_HUB_OFFLINE=1\nTRANSFORMERS_OFFLINE=1\n")
   PYKEY
   chmod 600 .api-key .api-env
   cat > reader_assets.py <<'PYCHECKER'
   """Reader-run full checkpoint integrity; no model imports."""
   import hashlib
   import json
   import pathlib
   import sys
   import urllib.request
   ROOT = pathlib.Path(__file__).resolve().parent
   MANIFEST_SHA = '9257bca5b7f7e450f8a7e6c77b1ebede425c68677dfc6d1d658600ada734c28c'
   
   
   def verified_manifest():
       raw = (ROOT / 'assets.json').read_bytes()
       if hashlib.sha256(raw).hexdigest() != MANIFEST_SHA:
           raise ValueError('manifest_hash')
       d = json.loads(raw)
       if d['revision'] != '88e3dce03c4a5535e2f4a2bcc08e939a2b302f82' or d['repository'] != 'NousResearch/Hermes-4-405B' or len(d['files']) != 199:
           raise ValueError('manifest_identity')
       return d['files']
   
   
   def check(path, entry):
       digest = hashlib.sha256()
       count = 0
       with path.open('rb') as f:
           while block := f.read(1024 * 1024):
               count += len(block)
               digest.update(block)
       if count != entry['bytes'] or digest.hexdigest() != entry['sha256']:
           raise ValueError('asset_integrity_' + entry['path'])
   
   
   def main(download):
       files = verified_manifest()
       out = ROOT / 'model'
       out.mkdir(exist_ok=True)
       for entry in files:
           dest = out / entry['path']
           if pathlib.PurePosixPath(entry['path']).name != entry['path']:
               raise ValueError('asset_path')
           if download and not dest.exists():
               temp = dest.with_suffix(dest.suffix + '.partial')
               with urllib.request.urlopen('https://huggingface.co/NousResearch/Hermes-4-405B/resolve/88e3dce03c4a5535e2f4a2bcc08e939a2b302f82/'+entry['path'], timeout=60) as r, temp.open('wb') as f:
                   total = 0
                   while block := r.read(1024 * 1024):
                       total += len(block)
                       if total > entry['bytes']:
                           raise ValueError('download_size')
                       f.write(block)
               check(temp, entry)
               temp.replace(dest)
           check(dest, entry)
       cfg = json.loads((out / 'config.json').read_text())
       assert cfg['architectures']==['LlamaForCausalLM'] and cfg['num_hidden_layers']==126
       assert cfg['hidden_size']==16384 and cfg['num_attention_heads']==128 and cfg['num_key_value_heads']==8
       assert cfg['torch_dtype']=='bfloat16' and cfg['bos_token_id']==128000 and cfg['eos_token_id']==128001
       assert cfg['max_position_embeddings']==131072
       tok=json.loads((out/'tokenizer_config.json').read_text())
       assert 'chat_template' not in tok  # Native standalone Jinja is loaded and byte-checked by client.
       print('All complete native weight and auxiliary file sizes/SHA values pass, including all 191 BF16 parts.')
   
   
   
   if __name__ == '__main__':
       if sys.argv[1:] not in (['download'], ['verify']):
           raise SystemExit('Use download or verify')
       main(sys.argv[1] == 'download')
   PYCHECKER

   Expected result: Private files and stdlib checker written; no model load or fetch occurs yet.

2. 2. Write exact complete manifest segment 1

   The compressed manifest is an original reader packaging of exact publisher metadata, not a model binary. Preserve every segment and fail if final checksum differs.

   CommandCopy command

   set -eu
   cd "$HOME/hermes4-405b"
   cat > assets.b64 <<'MANIFESTPART'
   eNqdnUuvZclRhef8jB7bTb4j0zMQlpjAwFOErHzaZdxdra4CYSz/d751kKDK7KNupR9dt/qee3aczIgVa0VG5P3zNz/uHz5++vD5449/+uZX3/zzx3//9Jv9afcf5+//9h/3j9/tT79Mv0wu//03v+CV//Hh04eP3/O6Wndcc7s4U8855h1O6mHM6epusfFldOHUwE+dD3/cn7751b/8+Zsf+uff87O/+fXf/cM//frb7xbfHH/6rG+2kOwX33z6fQ+58Iq107LsUzgrpHzm2rNancncOat730uKfvUS0xqh5uCsxzz6WSMH39L85i+/+N+nzd/3z7/9vL/74Y/98/72Dx++/0P/v+cmH/wXz63N+zJ9nyHw2VzmHY83C3MkV1KaPofWT+nNR+fmKTnnUpZ3wdJK3vJXz/34/fnwu2//8In1+t/n1RS/eFy25Hx2OYS+qrGeLR2zvreNOtZchU+3NqvqLBZfRtsxupxy7ekE6199zN/t7/eP/TO789vHJ/vqvniyVc/qphVYz1VjKlbKmH705kpPs7Uxzp6nn9ZaT2nn5HZ3bvTJD7AVXz75u49r//GXjv/4X348fOGb//ZTP/vz/v7Txx8/fbHa1ZVioX5lSq2njTZXPNlyj+xhdSuP5tPstWQ/DpvNR+9t8M29Xehj26mt+Dzq7s+mhJ8yxYWSYwghfWHKXKsc00+vtvoZNhy7G8Yx3/OKedbtY6zbWUitnVpbPnGGWFbEXUJ5NiX+lCmFTdWGhC9M2dPPNMfuvveaVy2xB+9djL6mPNg6a9FOKp2QWH3luNckEuLeZ+djb1Yl3Zgih0yJ1S6HP/NKJ83du48JV8ylbrCgsxQpxR7TxHnyDGdjzJjNoj2bkn/ClJjY4eJ9+9IUD8oQ6DGxKqu03ppjado5IfW6T9u+9biMpSg+sjIF/FiLf9ncOCO0Z1PKzaqc7qqdAiqFVks9wM5JI25Lbc/OU7GlxH3Yu8OunT6CTWCSrXNhtje+YjemdCB41F6jHzyi5M5LWsm2XAwx1FQJIu9Pcym6RTwX18II7FfpY5m9MaXebNBKgS/8LHvzxynTzVHYMLbD9Trn8bX4aiX4OscRji/CKcbZdsJC/2xKu9qgHVveRNBehTDfPpW2yR9+LUzvuWWh97ISly9gT+w9ulY6LyGT9PNoindXpqx8ghs4qY8gK1iS+Jv3O1RcNJfBSvkwDUdO2RHapfeTGgsccgvhGW29v9mg1lZq7riCPZYnH8gSSSZ0smZOfa6hbOYX1tXSSTSgG/md1+bp2ddnU8LNqqwWK2i74yGjeguEdBsEtfdjZfym40XxRDYjOvJi9rX2NbR0s/b9boOu0LYVl9vcoErZxXj/mNqqrW/cssbSMknJWu0D8O1sVgfYbK5mq+3tl3s2Jd1sUJq18Xl3YUWIW7dPATEIolNnBehwluUaoHZm6YvwWSuTrncHjceob1Yl36yKA8/3LqeSDSEAimC8xwxiN/JpICCom+BBdY2ES+HeCyw6K2Vo1ZrPpvxctA1fkYQxLJ5ezql5QCvd8jnX0IPboBn7xFJOPj/x5UdLCwBcfvC9FNOo8zkHebvZoJLaCd2Zt2H4CuG8HZuxB8ja4aFp4yaED/zCA7Q9eldiZclWqTG4+GxKvVkVcsyJJZ1Vdhq7QUgTxNDV2CfrAsTxfgeOAwTijS6za34lCNeyGkd+Toe+3ZhCFiV4a94ZvziE7iQTkwBr8nC7Djma0INQyciezBNw2s1brDAMhuXGoynB3WzQrrFC20431qXGzJeg3YLBuB02XNMPqABUsiRLpG6hXJTaGKFDq599JfibVZmg/YwICJfZptMz4kLCAj8A9hvM7nhILdCX44xukAgD4HKKNNRxz+kwhBtTkA5oG/Oh+wnRLlkWDBIjTpyTQWBKIKIhU3A4HyG60KacgL4dh39Ds0O82iCgC1vaziLzIAuMH7LbB7QRYzCLxTBnuAjyC9+ObkomnWkrEWfPpqQrtwW+IhozxmMzEMpEy4pVngm9rwXdOfZuMCr9tTc2DURu60zoS2hvfCXfmHJEB+ax1optFBnqAkZPcmykwebYEQgD6q+vbTMF9Hdfvg9HenJ19fxsSrlicQtpX0ovjaUx31bpoSzEIlAXwzwNMus6kEdoHYwlY8eZmqUyyUXhjSl2syq2NgjhbIH7ZDdXMMwdqXMwDUaHjB0nrQ7JJBU3FMfuJfQJY9l5hOd0GK7QtngALnTzPKYHe1EmpHECXcC5gRrlkSIOnbxZE9AP4+Nb0Fyf53wmCaFdbVCHqcBSFDsQ59bh/U7PNbS56hV9oxNnj7muMlDwbCCJ/AygWNnz0ZToriButR7SPMh0shj4ttyx2irZ+GT4wkg8E+3sLZGoXbeEJCI95RQgM/7ZV+Id2hKizs4uPECqB/dFqW40B1nPueHM9h4ELuG7wu5xEk6pbkMlxXSecSWGK5o9I/BmzfHfmYx0E81PNkaSiE9eAFo7LEonqAzButLgXVwNLaIh36xKvMKVMxJia09fohdBMDe6yheHhA2j26oDDUSRt3n28qm6mg0aR66Gv6RnU67QNiOGMywfhdNqbAjj4hw0AMQktHsLr/VAmxC6KKHDm0CfNm5Emh75jSlXlQTC1FDtpB3PCrhlEkGInQrQDV/zfIloaKW1XHvBkFnX7liY8p7xOTPHK24bPREJS4XMjRUArQFfQpK6I3QB+nbKIC6JePHjI5dgU5IeRrxBuGeIi3aXDj3srBs0qJW0I7RRnMhDeRElHjwBeBKAU5Lfa0BoZo58n/0ppPA3G3RVSTgo5AbFdkrNySMVS8i7kbBLx5nLSp4fgcdEFEl/FSrNo97RJCH2/WZVrrjtqCVlyLQi15uEmbExQzA3ClJVhq02laNZMDJ2Ep+aO5c9+IlnU9IV2pJKIAAJPJtwSIRpXqft5nOHkYDCVlZlSaJ1slU3GPhc5OQ5yJuSz8+mXFUS0KHA+SI7nwiMNNeh/03eiVj1p1UYBIShYBIMjtDp3k3lCUf+JEc9m3LFbc/025xH/C074OdBuqumzLNOQMe3WZH2RDIKhFTdMS/yJgl5O/w8b0y5QltrwxEUNZbOkpWDf4CrKqIHMXqDtwR9+t1U6AgjEEcsrLiNT9s9R1C6qiSMlVIaszTSHPl4h5adudlXWa72CcnjBYiRw1rZwtx+BhIpjrKXrfhMElK+00HTWTlQpWQoeBg2yDpRyspMftU9KvTNzPUD2WPLTl0us0dnW2/1zapcoW2IeGvupGXSIGodhh3bLFmnHYQWXHIeR5JyUzUEsngjhhFL8HHp/GeSkK4qCbNOCRvWwC0kXzlKzzCVM1usPqGL5+4DvolEORlhgpcgBopHmyDR3gTzFbc1PrmrgFJikzaeAGs5Fkp1B/fF0Ew8qQgzIS0ZA85oSBW/ygkJXfRsyhXaNtWFIQcjEhwQBFi3VyWlh9FiSBvGsucYbu8YOgiTQ9QJHHh8qn+3KvmqkiDIEBsh06ikw+MgCxtuHyFr5knGc6SYGxq1kRtn5pUzzIAOkkM/l3ryFbddq0ssg/Y+hoPXwqB5ooNiDo/DyK3TBl2jg0oCMyenOEs0rVZsz4QyX6Fts2WEisUwBqQAYpARpnvCGSbpcWzXEnrQhUnU7OBU2ydT9tV1ivamqJGvKgm1GGAChyQD46qWJsTAvNNGeEg3jw8CFcd31ul9FjvQA1avhbL3s3zPV9wWajh0PN1rNVdy3vHAGw40P8/gycmTJAX6BbgLBA9ARiGRq0qLOqR4s0FXaFtIJivn5BFjfRr0pOc8ERc47dRx7xFd8yKUqGo/Y8Zn3aq8U869vHHbq0pC6BGNXsWc+Bn+xtID7tJmKLSzNkwT0ThGIIKLTlih/wRawnERrs9lwXzFbUs+hx0Yi3QaDYzrjkQ3XKjmR9w1hKIzgAzfdhVWIP7C+qQVY4ATP5985Cu0rbDDiFIdMbnwqsWpbD5UQnH4aIxujjyb991BOBE+sF9kPcrt5AKjezblqpIQF3gmIQybdXuUMFx3aYUFnbPsdsefI4JgJNXbx2KJILgLXhnXhIs+mlLcXQl5KnjmZEXgCYuIhqtN3FYnPsVB8WdmvWJGwKvCMrWBIewTB8poP5tyh7ZlI/pmP1kw64+vqNMdXUcxj9qHN4x1HmP9YoMG9BPKFCLZseZszycf5aqSsDKaGAE0IhIQfj0h9TOHYm5LLae64LHFozTYrgzn6zw/7Z3DiYGgfzblitsGWGEENgwumzqEsmdX+ch7uOj92bFknVeCdcPpKC1CKzOKBa2Wcepnty13dVvLG70xPckP5Q63FIjEysahW88KFVWCpiZZWiBlepK4ayyPdSL7vDHlqpKA2grbAgTRNdUMFvQSibNLLmskhBdaHh4L5CPd1bDBohWRYaTBwI2eTSl3bgspq3AAWNzKKeMxZOcJo8KaHX0ZyW2UQNpDxzQid8R1BKCxjbT5bIrdCVXwgk9vHRJgBMjJOaRUwBrUaY3jJBsQ/4XN1Q3paDJhjEpRSJRnklCuKgm5qXWmAQXWass9tSqS3eOr+2oH3zr4T75mY3wpHakKFqsRa65I2D2bcndK1o6PAzarAldgR6aOn6yzXRvnCFmHZWRulqrykHVm2T6Ujj8lqPAzX7ErtIU5T8Q5BAGuUiEEHiFfXPL9kJMXmKfSSoYt9IYWii2gai1G9D2JIDyvivm7rh5UzvRFyNWb8t5WBaEksOWV95zW5SzYDIR7ZmC5BZYOFEy1jGeSYOGOZueVxPhJhaSAMGb3OpQ7LAZSecIaU2mdmMKp4ZITXuk6ntKnKt9vViXepcNdF+SMDSp4o8f7cwoTR7GTwkizq8Ac2YyFp3Qdu/d4MEMdLO1NJcGuKgnrdcAQ3MY1XJlqfIRxh+QL3HpCINxQvUTbkU7ttklC7N503eNj47kAZlfcNiNSwfG02qi1LuPDj3BC1WFmz634aTsklaLcTmqhiwWEIxsGRP3wz7hi5a6wTv4311qc6gIg9SBIUazwqAhZ8Px1+4rn7hSAoFlhlFiC5hdxys98xa4qCd7CXu2M3VrzQZmftFjA28B+wBKa267vxRYuPiw8LveOj29Drg5e/GzKFbc9VSXQyAcfOWyVYgmgU2Ijsg6h28acKPZmEe4NPYBP9BOhdgRX2+HNqlyi7bQEcFZ/2sJ7AyJ6bJstt2TOH15lwXzuSBHrO84Nspw8ibGinstHU+pVJaFt34lclDAUEhBl+WtNe4H/Cd1cVcxoiu16lKtx8rPxEoPmIN/eNI3UK25b+YA+wj4AlYMkt+AEYgeBVpKDa7IhwBvixPeKQJ0+6AgPzsBPgS3PplyhrfeJj7ty9cJQmHxAP5d12DLQrY8GzCMzKpoE4jIPXkyayEj6A49Yz3XbelVJiDptKeAYrNbjBUTIqtYIYjQqsgwtCvuHdK6zCPETy2Jl1iErRnsnyeoVt4Vf9xrKKDo/DJB5EiJsBMw/eztoLOxjqkdg6ICqeQLqnOzHzvuoRfzZlCu0hcxXYpcYnWFVElGI7FHXqdixBfRuaUW0GRwT7wij25gYKfoA4XrOQfWqksB2nNEJaBgK8UvIInn2cXWot7QcdBI0HwxGiamtBn9FvyHzgbjq35R6qt3poNFOCdM770nQEqepmEccDZjJdIBbAHFYj8MqqehRdaQHHfYJnf9MEupdTwKIVkaT7MFHukFlSTYByqQOweHgVCOeVqWQJ2psp4UKYBVDys3e9DrVu54EgiagKSaAGkUi4fkjAfuQkRE2DBZvWgTuOm6SqGAuBJUOoM9x0T2XkNtdT0KqY5zKJ09NCQi6+uqNL/souDcQh11TZ9w4BxxblIYkM8jTrb45UW1XaAsnc7n0qukJHWj7WvHQIFduYio4hFPuhnOSLAn4QxCRE+BSy5Z7ToftqpLAlkDgNwx6OTLw8kFlSD02xCMJNt1xJ9tmD/kDC3g9VGI3nKX3Z7Rt8a5xkdDgSUOg5dKuheBRjYNgKvAT
   MANIFESTPART

   Expected result: Manifest segment written; complete SHA is checked before use.

3. 3. Write exact complete manifest segment 2

   The compressed manifest is an original reader packaging of exact publisher metadata, not a model binary. Preserve every segment and fail if final checksum differs.

   CommandCopy command

   set -eu
   cd "$HOME/hermes4-405b"
   cat >> assets.b64 <<'MANIFESTPART'
   S5ClGQ0kcWerNwCCDWUKrB8891kzt3TXdtXVWaSOvDi9J/F6uAGIQkjhtKU5tSg3FqKEsZHWNcABHAKll7je1FfaVSVhkw0hlCDXTlHHP5KELoeeTsuvA46BxyCO0I8a0QFz04owYBJRjm/64tpdvy3mdTyjFWIa8LcQcwHXUucfO1XCyM3lfR1K3pE8aR76UiH/a7wjCe0ObfOxqaZ1AlOo0g5bkSvJMSMxKtg+1LRXIpaSrXHctm0v+E3MEK03G3RVSajoCUt+mYehNBJM0cALSgdy3dWEcGqCUfnRPcimBrDWc2hZvLL59pwO2xW35YVbDQDwj+wX6LJQHHDargPEuMTYetxId+8yUdU37JKPw6JsQwk9kgTvrtAW7kzoEqNRLSJlxsSaTFIvhGEOTdHNOFVeSIgALUrt8P5dLWiKp89nU64qCSpJzorg47Orx4qcd8C2siyiNPwKpQ/wdSXI3IRQADDVypnALqLoudfJu7tKwhBqwhzZd+A2oshrHyxLXpC0nIZ6NzxZexqIMhRoc9uqvNUK/nnmw7u7noTY22YvVFqDpcGN3AFmIwpPVNvSbLHoxNcjGYv6R2CXbqhl7wC549mUq0pCQbsPtTundIrfLUY1HzgXj+aWbMCqq/lTm9apIIAyUYbT6ryT7L2fTbnitqBts2zguDrQSf7LSQpG4mg0PwOZSSeaQH48lkUjS8XVQSJAd63zbMoV2rYeDgmOvXBACb4JHThgHgamkAXDBLsIZE4IoYRKYdG2LIH3gjPPplxVEhAcZ9tQJ3qLe2tAKKA9ADmeCGkiwHBoMo8yM7SyTBSq2WoIp2PxTTBfcduNXK9Qo5xgh9I5mDoJ4zIADzJ0PvX47KMZsseqDZvlAH8NEa25qmdTrtC2gibJt+Mr6Tf5eAAvaaAMU7AI3gxUNb60AT0Vf8wGFFNjDmuwas+m+KtKQg2v/v0QkyatF9CRJde7RnklwCCQEJRMHLc0NT1KCuxxnkBMA73PuOKvuC2BMFwg+5NUwvK1WNdRT0WXdkO/DpWxIY4J7hAQZNanBEE0jW/284wr/q4DrKKYQ1XZYg596gitKw0ykpVtcKOSWJSTo1o9q0OIsT2qTIWGPc8R5K8qCW73vQnLYb0uxOlZE+uSje0bzLap6dnNtGAx4B3h5IpLgO1Se5h7t0HpTr6TAlHFLo5NNG0otNBd3ddLFa+1EV9E8OgJl9bsftJsQ3UbxLPnGr/3V2ir6iQvbjZxla6qvRfLjTp0b6+S8QLH60A9x5YbwmCpcgmpw4vzm2D2V5UEdY0ekONgZpxDs+7wJDHIwytIStiEfkQo+p56tU2qPKrO+Ymi9m985YrbNpJa3J54CToGqqIMaB5oCng7QFadlQH9gD7sCmAGjxGMGWUNIW/t2ZQrtI1a7SqBkXd2peokLq0wYQ0uHNk0SUVeXyxEAJQb9hlnGsXNMWw9m9LuunpUSMdXvbrEUyYZrxB0HlciVp8gL0Yk75caKSnPidUsHvYan+vRlHDFbUXSEpuUCziLVi62JDHGkmRGv9bXDMY6IxtpCB+GuMyo6ceSemhvTLlCWwRg37V5yKvvJ23+pxZTFS5tALxCWtWT96p44+xJjAVmS+irwvwcQeGqkrB168ZC9JWmFqYKnXTQZ8QHH73qFLdBG1acHaU0A4sEkYBQJeSQCuzPplxxWxRoB2cXqL9QqA4uO6c49RoTRdrE1GotQclynKbR1ZHVqHzEceozobybJYsDodyGrrfwZFx1y+gYBJQbqzVtlCeeA8DnyNKZNQOURyaOQcO83qzKVSXB5WmtVK380aR9IKR47Gl1qi2jdenS0pPaZxLkypcOBS5Hc5Pq13s2pdy1/i4HtqhnRZ1xcFhEK4pHh8XqSVBbeLKSw3aF+JkO1zmhz0lEb/L1syl3aNs1SxZVQPfAhkY54GdwN/zYvDr1cMK6nIgKMuB073c0NnAF+MN5E8xXlQQJwhnVqx9C7ONMnTn4hFFo1e5HxGVbcSjoobNCEpFhaF+rwm1TKc+mXHHbUypULUBvVw3o9RKihxUUh2xUwwSLpp4JqFsCYUcbZSedUoUSwb3xTBLuZsn2Im480jTPRSZml7wutFgxgfFIDLUUB5WayAtewy912lptISDxr/0cQfGqkrB0OkjYgGjxFTYavsRhXR0aCz1qCY+arZiqjyIEiga8lhpy0Sr2jLbxrt+WjYc9Q8zW6NG2X6OBIMgOVIYftc2yQ85ZRdKQT8CdUYgp1doWhPg5M9/Nktmcsa/4mlNNQeP/XbWVank2lKoF2zEfOyceRIBttUdr5RLKY8z4HMzxqpJgBtFW86qaJAcwX1RfspAhtVmpiKxQ1D+ZyIUw27EtekKLj7XIFM+lnpjvTlTVn79xzkEIdd1MEHcZqUJZVFPw3W8Ua/asDIptwn5hDrqoJmLt8yGMv5slC6ohFKRQ0SQXbMFcXGqjIUnCqWfUSIEbw5GHNJyUNsuiqntzhNJzK42Pdz0JmiGzjjSfbI8OvBfgzz90M0E9Qy1xeHA/fVuARGp6swZcZpeMQnuGuHjFbdETWCCd4Sw4XWJUCyGUB/p1e8n00PbKfHPr5NkmUByB3iDwKW8KYHezZKbhCWKmacZvD7/4q8ZEY1OH4GwDo3CSVPzc/riii1fM4DNn90SaeDQlXVUSYGJkHjUtDkMEZl3NxUrV5M9oyDID0SANucud04RvrpageE0DXATYsyn+rhGArJPjKMjOI/KoBox4JoGSZ8kGXQNigWLdjrNzS8o+SVcVsF0pPUfQ3SzZFNg73Y00cYipYQuoJPkR8pq2J9IBN1JAqmn613USBwQqS00jup3u2ZR417iow59lYAEkMVsySL7r5BtER4C7DwdZwSsUU9Z1T4vhWM13+O08z2ib7rgtJK0cNbAgzL1Y49FtEoYihBzhu7Um9GBxiDWonIe2kDY3CJeWRr2eTblCW2C/1myl2iLVqV+xq2h6ku5nODspHMg2GEgCSBFUcxp7gNXwY3U8y/d0N92gea3SO4piq3lJ4uNoxHnLacFVpOJBFHidBdReoby6PcaK7kl802/r0909CXCVUSJb4UQSXxMuO8SRg6lxk+AOYaMZQ4WxwG95tcY/4HWH/7c3EXQ3S7YitKNu3LKN1A0hjHatxwF1E7IQpq/IZ6IJZmNYE0lCiKC1gZv55uQjXVUS5tTg2EyEBs/foVssJxw1S0DeRliw/pbJCzrThcv68LoNEVq32MI3bpuvuO1pmgVSZ2IofDGzrioqcb46Xif+u9aGmujYIclVdceULtHjAyBv43Op5+fPktUvD2F4Swc508CHQ4is5Xew5E7pYboRdAsLf4yi3ix1vyoLzNJP2FWVsmdTfmYlAeX3VTpMOjVeub/Gt3RUp8NMT/5hd/DLPtZrNnMlq03XpQG25CeYRIaLPkdQjjer0oYA/wAoLQw7saug4IbXLSsh6DI9lEdDIqPa5zCsQ6PAV9KKmqN6Y0q6MSURedDlkaBwB5Kg0hLOYi2GAIQV9c9rmG1C6tImdTt4/zELusRmP9/V43O+2aCyXk22QBpvEAqSgigVR4kVb17yZqfT+K1hmKitwleceqKBgWFvIqjcrIo/8ziotMakZoMeZGibhmeBM0REK6Si1YniMXc5WZ01CEkITj1T9+89m2JXG5TB9w6crqBLx7LunSlOs0DYNB0O42uGOaLOhCauVy+ixRsl9vbdBtWbDeo5+Km7Q0osHtpMGkzLti7TM7debSLxVUwJbbpty49lhe05bmzdTvZsSrtZle7gy6x16CAtIgy4V92kDs9uodPTkXBduhcgjg2bSzOp+pKSronMz7jy82fJ6lfVbBeKQbAHvAz6JZxD9fipuwWH7uzReOwiPeiymHKOZv0G+RIdf+J+ZvzF32yQJyVHjddARFKLS93gcYvJtqpJP83aG+lZBzCkxMq39tKddsRzQIw8mxJuVmXpLjoxDHTyduWoN4G8h54f+M008mM40fU0lmo95J481Zqr+0dI3c+Mv1yhrXvdyTmCayXAa1skH+2p0WJEzlI/noG5OssVEMM6KxLaVfBm6pa9N6akK4jLQdeQrVHVmN5VRhmrib6wKy3nHm14ciBeks8ajRRu9XWBj9fYxRtT8lUOIukWlQ3w0vLakzN17TLJuZ+8eyNkq06iQp0SqK9++v4qTjYA+hniyhXans0jX7OfFnzT/Theh+9ZE6EqNS7APUG3YdvQzlQmAgR6oMLP68alZ1PsaoOGRtS27vILHWBFKcNvR8iZSJ31YBq7RwLUaNV4tfENtbSQsJvOsp5NqVfAn8LSjQDAqB/b/FFHoK52FadGGa7KLkAYfKsa1O+6IQWKyfedrhZ64ytXaGsE8tH0y54xOfYH8ptA2TBI0up4JvOQdiAPg01Es1eDc/tTlq7xfB6X9eauIC60EFY7ycEIDpgbNFW2SIAIgLozqtTmSMPppAOlVHVqpKvbdC+zPd8m7u2K2x592JVOaERSP7q2FqSDw3ZRaaJ7rwTNU4+rvgNXzEWjDWTt8G7CztsV2g5ziWeoGw6pvhuOuU8DURGwCHpdsxUUzebaOrAVwT07lPVGW1fnP5oSryIorhh8zPiMbkPV6YMh/ybMdcVZFsJj+ubiAvDmKBBP1AkC1bkzWM43G3TFbVv0AMQoayxdlqGBC3h/1ySfmzrp1U3esyWda0IcULE2ROgiwHjSebMqd2jrSDA4IroGO3DeWXT5CBhcm+65qrZWPt6f01kZ73Ckklbnbbpaop9xxcrNBrUzdN18N19wzV67RvFTXGEcl3Up5Rni+ZUNMRRBbRmKF5vupXEOwvtsyhW3hT6ir9ZQS9GLh0ikbtthVlM65GtgRLOHKappW5e0oXNgdOSk/qYWZ1do+5ob14F2y68Je98sD6+D1bZ3Wq+uY93P2eEOZwV4lZocw2n4N3D7xpR2s0HweFP/rLitruTxR3Mmuqh7sQwZ69gnVJoLSePEoUIngTAdrQ6HJHk0pV5x2xMOYKtJ3KDb+C2oQZKN4ZOzLaQfxDQrhTvvfOLZ+mUGQLJBMnmz+Xw0Va/Q1mvUR+y6b10xSC5U84juuwVtLUKOIoGe2an0Pzfvqmtuq69Qv/NivzHlqpIAq154Ko/QrXkBZMsE8Laa1DXu9bbhdSd+wYcgwiD/SgQSeqVpmOvZlCtuaz2wD3FqxVV14+m9k++ketTc7yx3MmGNpEKXYwGIu+hLYMHw72d1WK/QtpaDQIfUo0abWlKKRrRqOKY+K1fG0QQeUVPAOA2RIUaq5iBrQMqXZxZX850O0o0emrAhlFJrdS7d4eCQ8QggMl8l4+jaxSJUAWZzUkEmuWjHtn9G23rFbTfaigzr0muUPTgr8/Vv1hhEbyFeGr6hozIIw6sIoyv5w9DsuW72ezblrpKgTog5Dh8XxWEzI380d2msjmOlkBlwya4wDwn+4hXgulfP6WL28wxx9aqSEHRlhKYu3ekANgBWgoonx0OkdHFTCzhNUYmwsHmoITYTUpkOMEQafzblitvCaSU9t6VYIYlgKNyx8TciVclGtzeBwa8Lu8vZrNM8OmlwHuGKcHo0pf0cbmu5VV++9BXdeBNejYt5BVd8XyjFVNdLlBbSD+sI8Q9tl+R50cbQGDUws2Jfzxv007NkSNLSgsUvTTH9FpE2Mac68EXVJPU6RRKyrgLV7WTWYHroZ90qu0/QdDz/Jhf13Pz/K6a+fPa3H75f+z//6pcGaXThy9/LtAkYUAM/1G1xRVeGOw0MjKqLoBde2iGPmoacbCF6o4D9ESaBCWSJryz49MOeH/off/v5479hwW+/6z/81cNTSl+Balm8IYLUqf1bF57FqV5e9fWC5iQ8/QYAdsgANrWIvG4kYG+qs/o1sX898sN/7R//+nckmRz+q08c20763SC5zN5saJJjaRhW3aIlpr73jnz8BMKGFXQnTu+6A9ugImj39fjY59/QlJELXz56bLQd1GIPHdaH/Jo0
   MANIFESTPART

   Expected result: Manifest segment written; complete SHA is checked before use.

4. 4. Write exact complete manifest segment 3

   The compressed manifest is an original reader packaging of exact publisher metadata, not a model binary. Preserve every segment and fail if final checksum differs.

   CommandCopy command

   set -eu
   cd "$HOME/hermes4-405b"
   cat >> assets.b64 <<'MANIFESTPART'
   nTYbXlX0m03UJKkSwZEPuKJr48vKOeAapDXSyL/+5W/+G5hMUnY=
   MANIFESTPART

   Expected result: Manifest segment written; complete SHA is checked before use.

5. 5. Decode, bind and fetch the complete native checkpoint

   The reader alone downloads all 191 weight parts and eight auxiliary assets. Existing corrupt files fail; complete streaming size/SHA checks precede native loading. No partial/alternate checkpoint is used.

   CommandCopy command

   set -eu
   cd "$HOME/hermes4-405b"
   python3 - <<'PYMANIFEST'
   import base64,zlib,hashlib,pathlib
   b=zlib.decompress(base64.b64decode(pathlib.Path("assets.b64").read_bytes()))
   assert hashlib.sha256(b).hexdigest()=="9257bca5b7f7e450f8a7e6c77b1ebede425c68677dfc6d1d658600ada734c28c"
   pathlib.Path("assets.json").write_bytes(b)
   PYMANIFEST
   python3 reader_assets.py download

   Expected result: All 199 exact asset size and SHA checks pass. Integrity is not model fit or answer quality.

Configure

1. 1. Write the exact native raw protocol client

   Native tokenizer/template is used only during reader execution inside the pinned container. Full raw assistant thinking/tool history is retained; no tool executor exists. All special-token IDs and completion budgets are model-specific. The fixed fixture reserves angle brackets for complete native thought/tool boundaries; unexpected, repeated or truncated role/control markers reject the response. Raw IDs must end in the genuine native EOS, while explicit vLLM output flags omit EOS from decoded text. The reader native tokenizer must reproduce the entire returned body exactly; no role/control or terminal-token text is silently stripped.

   CommandCopy command

   set -eu
   cd "$HOME/hermes4-405b"
   cat > reader_client.py <<'PYCLIENT'
   """Reader-owned native Hermes 4 405B fixed protocol check; no tool executor."""
   import http.client,json,pathlib,re
   ROOT=pathlib.Path('/work');PORT=8000;MODEL='hermes4-405b-native';CONTEXT=8192;OUTPUT=4096
   TOOLS=[{'type':'function','function':{'name':'lookup_demo_constant','description':'Return the inert synthetic constant; no external actions.','parameters':{'type':'object','properties':{'key':{'type':'string','enum':['alpha']}},'required':['key'],'additionalProperties':False}}}]
   def unique(pairs):
    d={}
    for k,v in pairs:
     if k in d:raise ValueError('duplicate_key')
     d[k]=v
    return d
   def parse(raw):
    if type(raw)is not bytes or len(raw)>2097152:raise ValueError('response_bound')
    return json.loads(raw.decode(),object_pairs_hook=unique,parse_constant=lambda _:(_ for _ in()).throw(ValueError('nonfinite')))
   def budget(ids):
    if type(ids)is not list or not 1<=len(ids)<=CONTEXT-OUTPUT or ids[0]!=128000 or ids.count(128000)!=1 or any(type(i)is not int or not 0<=i<128256 for i in ids):raise ValueError('native_prompt_budget_or_BOS')
    return len(ids)
   def checked(d,count,tool,decode=None):
    if type(count)is not int or not 1<=count<=CONTEXT-OUTPUT or type(tool)is not bool:raise ValueError('request_contract')
    if type(d)is not dict or d.get('model')!=MODEL or type(d.get('choices'))is not list or len(d['choices'])!=1:raise ValueError('identity')
    c=d['choices'][0]
    if type(c)is not dict:raise ValueError('choice')
    ids=c.get('token_ids');text=c.get('text');stop=c.get('stop_reason')
    if c.get('finish_reason')!='stop' or (stop is not None and type(stop)is not int):raise ValueError('incomplete')
    if type(ids)is not list or not 1<=len(ids)<OUTPUT or any(type(i)is not int or not 0<=i<128256 for i in ids):raise ValueError('raw_token_ids')
    # Released vLLM keeps the sampled terminal token in raw token_ids even when decoded output excludes it.
    if ids[-1]not in (128001,128009) or (stop is not None and stop!=ids[-1]):raise ValueError('terminal_eos')
    expected=[128011,128012]+([128002,128013]if tool else [])+[ids[-1]]
    if [i for i in ids if i>=128000]!=expected:raise ValueError('native_control_order_count_or_role')
    positions=[i for i,t in enumerate(ids)if t>=128000]
    if positions[1]<=positions[0]+1 or (not tool and positions[-1]<=positions[-2]+1):raise ValueError('empty_native_span')
    if tool and positions[3]<=positions[2]+1:raise ValueError('empty_tool_span')
    u=d.get('usage')
    if type(u)is not dict or any(type(u.get(k))is not int for k in ('prompt_tokens','completion_tokens','total_tokens')):raise ValueError('usage_integer')
    if u['prompt_tokens']!=count or u['completion_tokens']!=len(ids)or u['total_tokens']!=count+len(ids)or count+len(ids)>CONTEXT:raise ValueError('usage')
    if type(text)is not str:raise ValueError('text')
    # Fixed fixture grammar reserves angle brackets for complete native boundaries only.
    # This conservative check rejects truncated/unknown markers instead of sanitizing them.
    match=re.fullmatch(r'\s*<think>([^<>]+)</think>(.*)',text,re.S)
    if not match or not match[1].strip():raise ValueError('reasoning_boundary')
    tail=match[2].strip()
    if tool:
     call=re.fullmatch(r'<tool_call>\s*([^<>]+)\s*</tool_call>',tail,re.S)
     if not call:raise ValueError('tool_boundary')
     obj=parse(call[1].encode())
     if type(obj)is not dict or set(obj)!={'name','arguments'}or obj['name']!='lookup_demo_constant'or type(obj['arguments'])is not dict or obj['arguments']!={'key':'alpha'}:raise ValueError('inert_tool_required')
    elif not tail or '<'in tail or '>'in tail:raise ValueError('final_required')
    if decode is not None:
     # Native reader tokenizer closes text/raw-ID identity; terminal EOS is deliberately absent in text.
     if decode(ids[:-1])!=text:raise ValueError('native_decode_identity')
    return text
   
   def run(encode,request,decode=None):
    messages=[{'role':'user','content':'Use lookup_demo_constant with key alpha, then explain in one sentence why its synthetic value plus one equals eight. This is only an inert protocol experiment.'}];results=[]
    for tool in [True,False]:
     ids=encode(messages,TOOLS);n=budget(ids)
     d=request({'model':MODEL,'prompt':ids,'max_tokens':OUTPUT,'temperature':.6,'top_p':.95,'top_k':20,'n':1,'stream':False,'add_special_tokens':False,'skip_special_tokens':False,'spaces_between_special_tokens':False,'include_stop_str_in_output':False,'return_token_ids':True,'stop_token_ids':[128001,128009]})
     text=checked(d,n,tool,decode);results.append(d)
     if tool:
      # Entire raw native assistant body retained. Native Jinja handles tool-response serialization.
      history=text
      messages.append({'role':'assistant','content':history})
      messages.append({'role':'tool','content':'{"key":"alpha","value":7,"synthetic":true}'})
    return results
   
   def main():
    # Reader execution only: pinned local native tokenizer, not model loading or a replacement template.
    from transformers import AutoTokenizer
    tokenizer=AutoTokenizer.from_pretrained('/model',local_files_only=True)
    exact=(ROOT/'model/chat_template.jinja').read_text()
    if tokenizer.chat_template!=exact:raise ValueError('native_template_equality')
    def encode(messages,tools):
     return tokenizer.apply_chat_template(messages,tools=tools,thinking=True,keep_cots=True,add_generation_prompt=True,tokenize=True,truncation=False)
    key=(ROOT/'.api-key').read_text().strip()
    if len(key)!=64 or any(c not in '0123456789abcdef'for c in key):raise ValueError('key')
    def request(body):
     payload=json.dumps(body,allow_nan=False).encode()
     if len(payload)>65536:raise ValueError('request_bound')
     c=http.client.HTTPConnection('127.0.0.1',PORT,timeout=180)
     try:
      c.request('POST','/v1/completions',payload,{'Authorization':'Bearer '+key,'Content-Type':'application/json'});r=c.getresponse();raw=r.read(2097153)
      if r.status!=200:raise ValueError('endpoint_status_'+str(r.status))
      return parse(raw)
     finally:c.close()
    results=run(encode,request,lambda ids:tokenizer.decode(ids,skip_special_tokens=False,clean_up_tokenization_spaces=False));(ROOT/'result.json').write_text(json.dumps(results,indent=2,allow_nan=False)+'\n')
    print('Closed native reasoning, inert tool and final response protocol passed. Inspect the arithmetic manually; no measured quality or speed claim.')
   if __name__=='__main__':main()
   PYCLIENT
   python3 -m py_compile reader_assets.py reader_client.py
   sha256sum --check <<'SHA256'
   cc4476b0831d18ff727987b667745df0b11817e1065128e48f4b76c2d1a4526f  reader_client.py
   729d2ca30a30f86a75209db2952805a4d1d1a0218687133a6079035db4ee3ad9  reader_assets.py
   SHA256

   Expected result: Own stdlib client/checker syntax and exact checksums pass without native execution.

2. 2. Build the pinned native runtime layer

   Official vLLM amd64 image is immutable; the same source-qualified exact Transformers replacement used in prior native guides is explicit. This image is unbuilt here. Stop on pip check, version, CUDA, tensor-parallel or loader errors; record realized versions.

   CommandCopy command

   set -eu
   cd "$HOME/hermes4-405b"
   cat > Dockerfile <<'DOCKERFILE'
   FROM vllm/vllm-openai@sha256:c2f3b1b964e47809b722b5e75b61b1e7b39a50f70388cf2bf2418f16a9f31da2
   RUN python3 -m pip install --no-deps 'https://files.pythonhosted.org/packages/fc/b1/8be7e7ef0b5200491312201918b6125ef9c9df9dd0f0240ccef9ac824e6b/transformers-5.8.1-py3-none-any.whl#sha256=5340fb95962162cdfdae5cc91d7f8fedd92ed75216c1154c5e1f590fcf56dd0e' && python3 -m pip check
   RUN python3 -c "import vllm,transformers; assert vllm.__version__=='0.27.1'; assert transformers.__version__=='5.8.1'"
   ENTRYPOINT ["python3", "-m", "vllm.entrypoints.openai.api_server"]
   DOCKERFILE
   sudo docker build --pull -t hermes4-405b:reader .
   sudo docker run --rm --gpus all --entrypoint nvidia-smi hermes4-405b:reader
   sudo docker run --rm --entrypoint python3 hermes4-405b:reader -m pip freeze > resolved-container-environment.txt
   sudo docker network inspect hermes405-private >/dev/null 2>&1 || sudo docker network create --internal hermes405-private

   Expected result: Only after actual reader execution: package/version guards and all eight GPU visibility pass; no model load implied.

Start the model

1. 1. Start complete native BF16 with tensor parallel eight

   Foreground lifetime is 1,800 seconds including loading; it is not an inference deadline/SLA. Native full loader must account for all model parameters; do not bypass diagnostics, precision or architecture failure.

   CommandCopy command

   set -eu
   cd "$HOME/hermes4-405b"
   python3 reader_assets.py verify
   timeout --signal=TERM --kill-after=10s 1800s sudo docker run --rm --name hermes4-405b --gpus all --network hermes405-private -p 127.0.0.1:8994:8000 --shm-size 64g --env-file .api-env -v "$PWD/model:/model:ro" -v "$PWD:/work" hermes4-405b:reader --model /model --served-model-name hermes4-405b-native --host 0.0.0.0 --port 8000 --dtype bfloat16 --kv-cache-dtype auto --tensor-parallel-size 8 --max-model-len 8192 --max-num-seqs 1 --max-num-batched-tokens 8192 --gpu-memory-utilization 0.95 --no-enable-prefix-caching --enforce-eager --chat-template /model/chat_template.jinja

   Expected result: Only after actual reader initialization: complete native model starts without missing-parameter/collective/CUDA errors and private health becomes ready. Unknown fit may prevent startup.

Verify the setup

1. 1. Check private routes and the full two-turn native protocol

   Run on the host from a second terminal. Native health is unauthenticated but loopback-only; v1 requires the key. Execute the pinned reader client within the running container so it uses its exact local tokenizer. Inspect actual final arithmetic and record errors; protocol closure is not accuracy or calibrated quality.

   CommandCopy command

   set -eu
   cd "$HOME/hermes4-405b"
   curl --fail --max-time 5 http://127.0.0.1:8994/health
   test "$(curl -s --max-time 5 -o /dev/null -w '%{http_code}' http://127.0.0.1:8994/v1/models)" = 401
   sudo docker exec hermes4-405b python3 /work/reader_client.py

   Expected result: After actual reader inference only: one exact inert lookup call followed by a closed-think nonempty final response, with full raw IDs/usage recorded in result.json. Manually check seven plus one; no tool was executed.

Performance evidence

No compatible measurement available. Documented support and estimated memory do not establish response speed.

Limits and troubleshooting

• No install, build, tokenizer execution, weight download, native inference, provisioning or spending occurred. Source compatibility is not measured runtime success.
• Native config has 131,072 positions. Reduced walkthrough budgets do not establish full native window fit, quality or speed. Input count includes one BOS, tools and full retained native history.
• Native generation EOS is 128001, tokenizer EOT is 128009; both stops are sent. Complete raw think/tool closure and finish_reason stop required; length termination fails. Reasoning does not establish correctness.
• Publisher card uses a different illustrative repository spelling and recommends top_p 0.95, while selected generation metadata says 0.9. Exact selected repo/revision and card sampler are explicit, with no provider-alias byte equality.
• Native card llama3 metadata and Llama 3.1 base identify distinct agreements; both actual term/AUP sets retained. No later-base license silently replaces the linked license or grants legal clearance.
• Eight H200 per-device usable capacity, actual TP balance, kernel/collective buffers and peak are unknown. All weights remain; no smaller model or offload workaround is silently substituted.
• Client 180-second socket timeout does not cancel GPU work. Foreground 1,800-second lifetime includes initialization; stop/kill the named experiment and cloud billing separately.
• Docker workspace mount contains the private key; the container and Docker administrator are trusted. No arbitrary tools, uploads, URL fetch or remote model choices are provided.

Manifest, template, native loading, CUDA or TP failure

Stop and preserve diagnostics without keys. Restore exact full pinned assets/image and all eight H200 devices. Do not omit tensors, reduce precision, substitute 70B or relax strict load checks.

Length termination or native reasoning/tool mismatch

Treat as incomplete. Preserve actual results where available; do not silently truncate history, strip reasoning or execute returned tools. Larger budgets need separate qualification.

OOM or timeout

Stop the exact named container. Allocation and source route are not measured fit. Confirm endpoint closure and manage EC2/EBS lifecycle; do not leave billed resources running.

Stop the model

1. 1. Stop the named native experiment

   Use a second server terminal; foreground timeout is not proof the container exited. Kill only this dedicated experiment if it remains running.

   CommandCopy command

   sudo docker stop --time 10 hermes4-405b || true
   if [ "$(sudo docker inspect -f '{{.State.Running}}' hermes4-405b 2>/dev/null || true)" = true ]; then sudo docker kill hermes4-405b; fi
   curl --max-time 3 http://127.0.0.1:8994/health || true

   Expected result: The exact container and endpoint stop; instance/storage billing continues until separately managed.

Uninstall

1. 1. Remove dedicated checkpoint, runtime tag and private files

   Preserve wanted results and legal/source records first. Confirm dedicated paths. Shared drivers/Docker and retained image layers remain; no broad prune.

   CommandCopy command

   sudo docker rm -f hermes4-405b 2>/dev/null || true
   sudo docker network rm hermes405-private
   sudo docker image rm hermes4-405b:reader
   rm -r -- "$HOME/hermes4-405b"

   Expected result: Dedicated files, results/key and image tag/network removed; shared packages/layers remain.

Shut down cloud resources

1. 1. End instance and retained-resource billing

   In EC2 us-east-1 stop for reuse or terminate this exact instance. Review Delete on termination for the 2 TiB root volume; delete unwanted detached EBS volumes, snapshots and retained public/Elastic IP resources. Included local NVMe is ephemeral. Check Bills/Cost Explorer for actual final usage.

   Expected result: Compute ends only after the actual instance lifecycle transition. Retained volumes/snapshots/IPs may still bill; stopping the model container is insufficient.

Exact model and requirements

Artifact:

Nous Research / Hermes 4 405B full native BF16 dense Llama checkpoint · checkpoint NousResearch/Hermes-4-405B

Publisher revision:

88e3dce03c4a5535e2f4a2bcc08e939a2b302f82

Artifact digest:

No published whole-checkpoint SHA; the complete fixed reader manifest pins all 191 weight parts and eight required auxiliary files separately.

Download size:

755.96 GiB

Memory requirement (unknown):

Unknown — Full native BF16 target, TP8 partitions, KV, collective/kernel workspaces, runtime and peak per-GPU memory have not been measured.

System RAM / disk:

Unknown / Unknown

Placement:

supported sharding. Native vLLM Llama tensor parallel eight; 128 query heads and eight KV heads are divisible by eight. Per-device weight balance and peak usage are not measured. Full target on eight GPUs; no CPU/disk weight offload. Host staging and download buffers remain additional.

KV cache:

BF16 attention K/V via explicit model dtype and auto cache dtype

Workload:

8,192 token context; Unknown; Unknown; concurrency 1; batch 1; cache cold; repetitions Unknown.

Tokenizer:

Exact native Hermes 4 tokenizer and 4,121-byte Llama-header Jinja · 88e3dce03c4a5535e2f4a2bcc08e939a2b302f82

Sampling / reasoning:

Publisher card recommendation temperature 0.6, top_p 0.95, top_k 20. Native generation_config instead has top_p 0.9; this selects the documented card recipe deliberately. No sampler inherited from Hermes 3 or 70B. Native thinking=True and keep_cots=True. Count one BOS including full tools and unmodified raw reasoning/tool history. Own strict client reads raw completions, requires complete think/tool boundaries and stop rather than length, then appends only a fixed synthetic tool response without executing any tool. Native generation EOS 128001 differs from tokenizer EOT 128009; both stop IDs explicitly sent. All native text capabilities retained; no vision architecture is asserted.

Runtime prerequisites:

Ubuntu 24.04 x86_64 host; pinned Ubuntu 22.04 / Python 3.12 source defaults container; NVIDIA Linux driver >=580.126.20, Working Docker GPU support with NVIDIA Container Toolkit; actual visibility preflight required.

Official artifact · License

Publisher card metadata declares llama3 while naming the Llama 3.1 405B base. The native LICENSE and USE_POLICY probes returned scoped 404; this does not erase its explicit metadata-linked grant. Full official Meta Llama 3 and stated-base Llama 3.1 agreements/AUP are archived separately, with no equivalence or publisher-intent claim. Retain both applicable agreement copies, notices and attribution (Built with Meta Llama 3 / Built with Llama); comply with redistribution and derivative naming requirements and both AUPs. Each agreement has its own release-date 700-million-MAU prior-month commercial license trigger. Preserve Llama 3 restrictions on using materials/outputs to improve other large language models, subject to its stated derivative exception. Patent/IP litigation, termination, warranty and liability conditions apply. This inert own-use walkthrough provides no legal clearance and accepts no terms on the reader behalf.

• README.md · 0 GiB · revision 88e3dce03c4a5535e2f4a2bcc08e939a2b302f82
• chat_template.jinja · 0 GiB · revision 88e3dce03c4a5535e2f4a2bcc08e939a2b302f82
• config.json · 0 GiB · revision 88e3dce03c4a5535e2f4a2bcc08e939a2b302f82
• generation_config.json · 0 GiB · revision 88e3dce03c4a5535e2f4a2bcc08e939a2b302f82
• model.safetensors.index.json · 0 GiB · revision 88e3dce03c4a5535e2f4a2bcc08e939a2b302f82
• special_tokens_map.json · 0 GiB · revision 88e3dce03c4a5535e2f4a2bcc08e939a2b302f82
• tokenizer.json · 0.02 GiB · revision 88e3dce03c4a5535e2f4a2bcc08e939a2b302f82
• tokenizer_config.json · 0 GiB · revision 88e3dce03c4a5535e2f4a2bcc08e939a2b302f82

• 811,706,916,800 serialized weight bytes are a full-model storage lower bound, not total memory or peak. Tensor-index metadata and file sizes are separate accounting.
• Eight advertised 141 GB H200 devices are not silently converted into GiB. Usable per-device bytes and runtime partition balance are unknown; no aggregate-fit claim.
• Native Llama has 126 dense layers, width 16,384, 128 query/eight KV heads and no MoE active-parameter shortcut. Host staging, Docker image and tokenizer add overhead.

Install the runtime

1. 1. Prepare an already owned eight-H200 instance

   Install host utilities on the existing dedicated eight-H200 host with driver >=580.126.20. Inventory actual device memory, compute capability and peer topology before native loading; no acquisition/provisioning is performed. The nominal 2 TiB host may report lower usable Linux memory;1900GiB MemTotal and1800GiB free disk are conservative reader allocation guards, not measured minima or fit. Inspect the actual eight-GPU topology output for intended peer/collective connectivity before loading; stop on unsupported connectivity. Compute capability9.0 and eight uniqueGPU UUIDs are explicitly checked.

   CommandCopy command

   set -eu
   sudo apt-get update
   sudo apt-get install -y docker.io python3 curl ca-certificates openssl
   sudo systemctl enable --now docker
   mkdir -p "$HOME/hermes4-405b"
   cd "$HOME/hermes4-405b"
   nvidia-smi --query-gpu=name,memory.total,driver_version --format=csv,noheader
   python3 - <<'PYPREFLIGHT'
   import csv,subprocess,shutil,os
   assert os.cpu_count()>=64
   mem={k:int(v.strip().split()[0])*1024 for k,v in (line.split(":",1)for line in open("/proc/meminfo"))if k in ("MemTotal","MemAvailable")}
   print("Actual Linux memory bytes",mem)
   assert mem["MemTotal"]>=1900*1024**3
   r=list(csv.reader(subprocess.check_output(["nvidia-smi","--query-gpu=name,memory.total,driver_version","--format=csv,noheader,nounits"],text=True).splitlines()))
   assert len(r)==8 and all("H200" in x[0] for x in r)
   assert all(tuple(map(int,x[2].strip().split("."))) >= (580,126,20) for x in r)
   assert shutil.disk_usage(".").free >=1800*1024**3
   print("Eight H200 devices and prescribed disk/driver checks passed; no model fit measured.")
   PYPREFLIGHT
   sudo docker version
   python3 - <<'PYCAP'
   import csv,subprocess
   r=list(csv.reader(subprocess.check_output(["nvidia-smi","--query-gpu=uuid,compute_cap","--format=csv,noheader,nounits"],text=True).splitlines()))
   assert len(r)==8 and len({x[0]for x in r})==8 and all(x[1].strip()=="9.0"for x in r)
   PYCAP
   nvidia-smi topo -m

   Expected result: Record eight actual GPU identities, usable memory and drivers. Stop on missing devices, driver/GPU Docker failure or inadequate prescribed disk. No native fit claim.

2. 2. Configure NVIDIA GPU containers on this dedicated host

   Use the official production repository and exact documented Container Toolkit 1.20.1-1 packages. This modifies Docker daemon configuration and restarts Docker on this dedicated host; preserve its original configuration and do not apply on a shared production daemon without its owner review. Stop on repository/signature/package errors.

   CommandCopy command

   set -euo pipefail
   sudo apt-get install -y gnupg2
   curl --fail --silent --show-error --location https://nvidia.github.io/libnvidia-container/gpgkey | sudo gpg --batch --yes --dearmor -o /usr/share/keyrings/nvidia-container-toolkit-keyring.gpg
   curl --fail --silent --show-error --location https://nvidia.github.io/libnvidia-container/stable/deb/nvidia-container-toolkit.list | sed 's#deb https://#deb [signed-by=/usr/share/keyrings/nvidia-container-toolkit-keyring.gpg] https://#g' | sudo tee /etc/apt/sources.list.d/nvidia-container-toolkit.list >/dev/null
   sudo apt-get update
   sudo apt-get install -y nvidia-container-toolkit=1.20.1-1 nvidia-container-toolkit-base=1.20.1-1 libnvidia-container-tools=1.20.1-1 libnvidia-container1=1.20.1-1
   sudo nvidia-ctk runtime configure --runtime=docker
   sudo systemctl restart docker

   Expected result: The exact GPU container packages install and Docker is configured. The subsequent container nvidia-smi check must pass; this is not model inference or fit proof.

Download the model

1. 1. Bind existing private service key and exact asset checker

   Use an existing authorized 64-hex key; no key is created. Both reader/server files are forced 0600 before any key write, including pre-existing files. The exact unchanged native full-asset verifier follows.

   CommandCopy command

   set -eu
   cd "$HOME/hermes4-405b"
   test -n "${HERMES4_READER_API_KEY:-}"
   python3 - <<'PYKEY'
   import os,pathlib
   k=os.environ['HERMES4_READER_API_KEY'];assert len(k)==64 and all(c in '0123456789abcdef'for c in k)
   for name,value in [('.api-key',k+'\n'),('.api-env','VLLM_API_KEY='+k+'\nHF_HUB_OFFLINE=1\nTRANSFORMERS_OFFLINE=1\n')]:
    p=pathlib.Path(name);assert not p.is_symlink()
    fd=os.open(p,os.O_WRONLY|os.O_CREAT|os.O_TRUNC,0o600);os.fchmod(fd,0o600)
    with os.fdopen(fd,'w')as f:f.write(value)
   PYKEY
   cat > reader_assets.py <<'PYCHECKER'
   """Reader-run full checkpoint integrity; no model imports."""
   import hashlib
   import json
   import pathlib
   import sys
   import urllib.request
   ROOT = pathlib.Path(__file__).resolve().parent
   MANIFEST_SHA = '9257bca5b7f7e450f8a7e6c77b1ebede425c68677dfc6d1d658600ada734c28c'
   
   
   def verified_manifest():
       raw = (ROOT / 'assets.json').read_bytes()
       if hashlib.sha256(raw).hexdigest() != MANIFEST_SHA:
           raise ValueError('manifest_hash')
       d = json.loads(raw)
       if d['revision'] != '88e3dce03c4a5535e2f4a2bcc08e939a2b302f82' or d['repository'] != 'NousResearch/Hermes-4-405B' or len(d['files']) != 199:
           raise ValueError('manifest_identity')
       return d['files']
   
   
   def check(path, entry):
       digest = hashlib.sha256()
       count = 0
       with path.open('rb') as f:
           while block := f.read(1024 * 1024):
               count += len(block)
               digest.update(block)
       if count != entry['bytes'] or digest.hexdigest() != entry['sha256']:
           raise ValueError('asset_integrity_' + entry['path'])
   
   
   def main(download):
       files = verified_manifest()
       out = ROOT / 'model'
       out.mkdir(exist_ok=True)
       for entry in files:
           dest = out / entry['path']
           if pathlib.PurePosixPath(entry['path']).name != entry['path']:
               raise ValueError('asset_path')
           if download and not dest.exists():
               temp = dest.with_suffix(dest.suffix + '.partial')
               with urllib.request.urlopen('https://huggingface.co/NousResearch/Hermes-4-405B/resolve/88e3dce03c4a5535e2f4a2bcc08e939a2b302f82/'+entry['path'], timeout=60) as r, temp.open('wb') as f:
                   total = 0
                   while block := r.read(1024 * 1024):
                       total += len(block)
                       if total > entry['bytes']:
                           raise ValueError('download_size')
                       f.write(block)
               check(temp, entry)
               temp.replace(dest)
           check(dest, entry)
       cfg = json.loads((out / 'config.json').read_text())
       assert cfg['architectures']==['LlamaForCausalLM'] and cfg['num_hidden_layers']==126
       assert cfg['hidden_size']==16384 and cfg['num_attention_heads']==128 and cfg['num_key_value_heads']==8
       assert cfg['torch_dtype']=='bfloat16' and cfg['bos_token_id']==128000 and cfg['eos_token_id']==128001
       assert cfg['max_position_embeddings']==131072
       tok=json.loads((out/'tokenizer_config.json').read_text())
       assert 'chat_template' not in tok  # Native standalone Jinja is loaded and byte-checked by client.
       print('All complete native weight and auxiliary file sizes/SHA values pass, including all 191 BF16 parts.')
   
   
   
   if __name__ == '__main__':
       if sys.argv[1:] not in (['download'], ['verify']):
           raise SystemExit('Use download or verify')
       main(sys.argv[1] == 'download')
   PYCHECKER

   Expected result: Private files and stdlib checker written; no model load or fetch occurs yet.

2. 2. Write exact complete manifest segment 1

   The compressed manifest is an original reader packaging of exact publisher metadata, not a model binary. Preserve every segment and fail if final checksum differs.

   CommandCopy command

   set -eu
   cd "$HOME/hermes4-405b"
   cat > assets.b64 <<'MANIFESTPART'
   eNqdnUuvZclRhef8jB7bTb4j0zMQlpjAwFOErHzaZdxdra4CYSz/d751kKDK7KNupR9dt/qee3aczIgVa0VG5P3zNz/uHz5++vD5449/+uZX3/zzx3//9Jv9afcf5+//9h/3j9/tT79Mv0wu//03v+CV//Hh04eP3/O6Wndcc7s4U8855h1O6mHM6epusfFldOHUwE+dD3/cn7751b/8+Zsf+uff87O/+fXf/cM//frb7xbfHH/6rG+2kOwX33z6fQ+58Iq107LsUzgrpHzm2rNancncOat730uKfvUS0xqh5uCsxzz6WSMH39L85i+/+N+nzd/3z7/9vL/74Y/98/72Dx++/0P/v+cmH/wXz63N+zJ9nyHw2VzmHY83C3MkV1KaPofWT+nNR+fmKTnnUpZ3wdJK3vJXz/34/fnwu2//8In1+t/n1RS/eFy25Hx2OYS+qrGeLR2zvreNOtZchU+3NqvqLBZfRtsxupxy7ekE6199zN/t7/eP/TO789vHJ/vqvniyVc/qphVYz1VjKlbKmH705kpPs7Uxzp6nn9ZaT2nn5HZ3bvTJD7AVXz75u49r//GXjv/4X348fOGb//ZTP/vz/v7Txx8/fbHa1ZVioX5lSq2njTZXPNlyj+xhdSuP5tPstWQ/DpvNR+9t8M29Xehj26mt+Dzq7s+mhJ8yxYWSYwghfWHKXKsc00+vtvoZNhy7G8Yx3/OKedbtY6zbWUitnVpbPnGGWFbEXUJ5NiX+lCmFTdWGhC9M2dPPNMfuvveaVy2xB+9djL6mPNg6a9FOKp2QWH3luNckEuLeZ+djb1Yl3Zgih0yJ1S6HP/NKJ83du48JV8ylbrCgsxQpxR7TxHnyDGdjzJjNoj2bkn/ClJjY4eJ9+9IUD8oQ6DGxKqu03ppjado5IfW6T9u+9biMpSg+sjIF/FiLf9ncOCO0Z1PKzaqc7qqdAiqFVks9wM5JI25Lbc/OU7GlxH3Yu8OunT6CTWCSrXNhtje+YjemdCB41F6jHzyi5M5LWsm2XAwx1FQJIu9Pcym6RTwX18II7FfpY5m9MaXebNBKgS/8LHvzxynTzVHYMLbD9Trn8bX4aiX4OscRji/CKcbZdsJC/2xKu9qgHVveRNBehTDfPpW2yR9+LUzvuWWh97ISly9gT+w9ulY6LyGT9PNoindXpqx8ghs4qY8gK1iS+Jv3O1RcNJfBSvkwDUdO2RHapfeTGgsccgvhGW29v9mg1lZq7riCPZYnH8gSSSZ0smZOfa6hbOYX1tXSSTSgG/md1+bp2ddnU8LNqqwWK2i74yGjeguEdBsEtfdjZfym40XxRDYjOvJi9rX2NbR0s/b9boOu0LYVl9vcoErZxXj/mNqqrW/cssbSMknJWu0D8O1sVgfYbK5mq+3tl3s2Jd1sUJq18Xl3YUWIW7dPATEIolNnBehwluUaoHZm6YvwWSuTrncHjceob1Yl36yKA8/3LqeSDSEAimC8xwxiN/JpICCom+BBdY2ES+HeCyw6K2Vo1ZrPpvxctA1fkYQxLJ5ezql5QCvd8jnX0IPboBn7xFJOPj/x5UdLCwBcfvC9FNOo8zkHebvZoJLaCd2Zt2H4CuG8HZuxB8ja4aFp4yaED/zCA7Q9eldiZclWqTG4+GxKvVkVcsyJJZ1Vdhq7QUgTxNDV2CfrAsTxfgeOAwTijS6za34lCNeyGkd+Toe+3ZhCFiV4a94ZvziE7iQTkwBr8nC7Djma0INQyciezBNw2s1brDAMhuXGoynB3WzQrrFC20431qXGzJeg3YLBuB02XNMPqABUsiRLpG6hXJTaGKFDq599JfibVZmg/YwICJfZptMz4kLCAj8A9hvM7nhILdCX44xukAgD4HKKNNRxz+kwhBtTkA5oG/Oh+wnRLlkWDBIjTpyTQWBKIKIhU3A4HyG60KacgL4dh39Ds0O82iCgC1vaziLzIAuMH7LbB7QRYzCLxTBnuAjyC9+ObkomnWkrEWfPpqQrtwW+IhozxmMzEMpEy4pVngm9rwXdOfZuMCr9tTc2DURu60zoS2hvfCXfmHJEB+ax1optFBnqAkZPcmykwebYEQgD6q+vbTMF9Hdfvg9HenJ19fxsSrlicQtpX0ovjaUx31bpoSzEIlAXwzwNMus6kEdoHYwlY8eZmqUyyUXhjSl2syq2NgjhbIH7ZDdXMMwdqXMwDUaHjB0nrQ7JJBU3FMfuJfQJY9l5hOd0GK7QtngALnTzPKYHe1EmpHECXcC5gRrlkSIOnbxZE9AP4+Nb0Fyf53wmCaFdbVCHqcBSFDsQ59bh/U7PNbS56hV9oxNnj7muMlDwbCCJ/AygWNnz0ZToriButR7SPMh0shj4ttyx2irZ+GT4wkg8E+3sLZGoXbeEJCI95RQgM/7ZV+Id2hKizs4uPECqB/dFqW40B1nPueHM9h4ELuG7wu5xEk6pbkMlxXSecSWGK5o9I/BmzfHfmYx0E81PNkaSiE9eAFo7LEonqAzButLgXVwNLaIh36xKvMKVMxJia09fohdBMDe6yheHhA2j26oDDUSRt3n28qm6mg0aR66Gv6RnU67QNiOGMywfhdNqbAjj4hw0AMQktHsLr/VAmxC6KKHDm0CfNm5Emh75jSlXlQTC1FDtpB3PCrhlEkGInQrQDV/zfIloaKW1XHvBkFnX7liY8p7xOTPHK24bPREJS4XMjRUArQFfQpK6I3QB+nbKIC6JePHjI5dgU5IeRrxBuGeIi3aXDj3srBs0qJW0I7RRnMhDeRElHjwBeBKAU5Lfa0BoZo58n/0ppPA3G3RVSTgo5AbFdkrNySMVS8i7kbBLx5nLSp4fgcdEFEl/FSrNo97RJCH2/WZVrrjtqCVlyLQi15uEmbExQzA3ClJVhq02laNZMDJ2Ep+aO5c9+IlnU9IV2pJKIAAJPJtwSIRpXqft5nOHkYDCVlZlSaJ1slU3GPhc5OQ5yJuSz8+mXFUS0KHA+SI7nwiMNNeh/03eiVj1p1UYBIShYBIMjtDp3k3lCUf+JEc9m3LFbc/025xH/C074OdBuqumzLNOQMe3WZH2RDIKhFTdMS/yJgl5O/w8b0y5QltrwxEUNZbOkpWDf4CrKqIHMXqDtwR9+t1U6AgjEEcsrLiNT9s9R1C6qiSMlVIaszTSHPl4h5adudlXWa72CcnjBYiRw1rZwtx+BhIpjrKXrfhMElK+00HTWTlQpWQoeBg2yDpRyspMftU9KvTNzPUD2WPLTl0us0dnW2/1zapcoW2IeGvupGXSIGodhh3bLFmnHYQWXHIeR5JyUzUEsngjhhFL8HHp/GeSkK4qCbNOCRvWwC0kXzlKzzCVM1usPqGL5+4DvolEORlhgpcgBopHmyDR3gTzFbc1PrmrgFJikzaeAGs5Fkp1B/fF0Ew8qQgzIS0ZA85oSBW/ygkJXfRsyhXaNtWFIQcjEhwQBFi3VyWlh9FiSBvGsucYbu8YOgiTQ9QJHHh8qn+3KvmqkiDIEBsh06ikw+MgCxtuHyFr5knGc6SYGxq1kRtn5pUzzIAOkkM/l3ryFbddq0ssg/Y+hoPXwqB5ooNiDo/DyK3TBl2jg0oCMyenOEs0rVZsz4QyX6Fts2WEisUwBqQAYpARpnvCGSbpcWzXEnrQhUnU7OBU2ydT9tV1ivamqJGvKgm1GGAChyQD46qWJsTAvNNGeEg3jw8CFcd31ul9FjvQA1avhbL3s3zPV9wWajh0PN1rNVdy3vHAGw40P8/gycmTJAX6BbgLBA9ARiGRq0qLOqR4s0FXaFtIJivn5BFjfRr0pOc8ERc47dRx7xFd8yKUqGo/Y8Zn3aq8U869vHHbq0pC6BGNXsWc+Bn+xtID7tJmKLSzNkwT0ThGIIKLTlih/wRawnERrs9lwXzFbUs+hx0Yi3QaDYzrjkQ3XKjmR9w1hKIzgAzfdhVWIP7C+qQVY4ATP5985Cu0rbDDiFIdMbnwqsWpbD5UQnH4aIxujjyb991BOBE+sF9kPcrt5AKjezblqpIQF3gmIQybdXuUMFx3aYUFnbPsdsefI4JgJNXbx2KJILgLXhnXhIs+mlLcXQl5KnjmZEXgCYuIhqtN3FYnPsVB8WdmvWJGwKvCMrWBIewTB8poP5tyh7ZlI/pmP1kw64+vqNMdXUcxj9qHN4x1HmP9YoMG9BPKFCLZseZszycf5aqSsDKaGAE0IhIQfj0h9TOHYm5LLae64LHFozTYrgzn6zw/7Z3DiYGgfzblitsGWGEENgwumzqEsmdX+ch7uOj92bFknVeCdcPpKC1CKzOKBa2Wcepnty13dVvLG70xPckP5Q63FIjEysahW88KFVWCpiZZWiBlepK4ayyPdSL7vDHlqpKA2grbAgTRNdUMFvQSibNLLmskhBdaHh4L5CPd1bDBohWRYaTBwI2eTSl3bgspq3AAWNzKKeMxZOcJo8KaHX0ZyW2UQNpDxzQid8R1BKCxjbT5bIrdCVXwgk9vHRJgBMjJOaRUwBrUaY3jJBsQ/4XN1Q3paDJhjEpRSJRnklCuKgm5qXWmAQXWass9tSqS3eOr+2oH3zr4T75mY3wpHakKFqsRa65I2D2bcndK1o6PAzarAldgR6aOn6yzXRvnCFmHZWRulqrykHVm2T6Ujj8lqPAzX7ErtIU5T8Q5BAGuUiEEHiFfXPL9kJMXmKfSSoYt9IYWii2gai1G9D2JIDyvivm7rh5UzvRFyNWb8t5WBaEksOWV95zW5SzYDIR7ZmC5BZYOFEy1jGeSYOGOZueVxPhJhaSAMGb3OpQ7LAZSecIaU2mdmMKp4ZITXuk6ntKnKt9vViXepcNdF+SMDSp4o8f7cwoTR7GTwkizq8Ac2YyFp3Qdu/d4MEMdLO1NJcGuKgnrdcAQ3MY1XJlqfIRxh+QL3HpCINxQvUTbkU7ttklC7N503eNj47kAZlfcNiNSwfG02qi1LuPDj3BC1WFmz634aTsklaLcTmqhiwWEIxsGRP3wz7hi5a6wTv4311qc6gIg9SBIUazwqAhZ8Px1+4rn7hSAoFlhlFiC5hdxys98xa4qCd7CXu2M3VrzQZmftFjA28B+wBKa267vxRYuPiw8LveOj29Drg5e/GzKFbc9VSXQyAcfOWyVYgmgU2Ijsg6h28acKPZmEe4NPYBP9BOhdgRX2+HNqlyi7bQEcFZ/2sJ7AyJ6bJstt2TOH15lwXzuSBHrO84Nspw8ibGinstHU+pVJaFt34lclDAUEhBl+WtNe4H/Cd1cVcxoiu16lKtx8rPxEoPmIN/eNI3UK25b+YA+wj4AlYMkt+AEYgeBVpKDa7IhwBvixPeKQJ0+6AgPzsBPgS3PplyhrfeJj7ty9cJQmHxAP5d12DLQrY8GzCMzKpoE4jIPXkyayEj6A49Yz3XbelVJiDptKeAYrNbjBUTIqtYIYjQqsgwtCvuHdK6zCPETy2Jl1iErRnsnyeoVt4Vf9xrKKDo/DJB5EiJsBMw/eztoLOxjqkdg6ICqeQLqnOzHzvuoRfzZlCu0hcxXYpcYnWFVElGI7FHXqdixBfRuaUW0GRwT7wij25gYKfoA4XrOQfWqksB2nNEJaBgK8UvIInn2cXWot7QcdBI0HwxGiamtBn9FvyHzgbjq35R6qt3poNFOCdM770nQEqepmEccDZjJdIBbAHFYj8MqqehRdaQHHfYJnf9MEupdTwKIVkaT7MFHukFlSTYByqQOweHgVCOeVqWQJ2psp4UKYBVDys3e9DrVu54EgiagKSaAGkUi4fkjAfuQkRE2DBZvWgTuOm6SqGAuBJUOoM9x0T2XkNtdT0KqY5zKJ09NCQi6+uqNL/souDcQh11TZ9w4BxxblIYkM8jTrb45UW1XaAsnc7n0qukJHWj7WvHQIFduYio4hFPuhnOSLAn4QxCRE+BSy5Z7ToftqpLAlkDgNwx6OTLw8kFlSD02xCMJNt1xJ9tmD/kDC3g9VGI3nKX3Z7Rt8a5xkdDgSUOg5dKuheBRjYNgKvAT
   MANIFESTPART

   Expected result: Manifest segment written; complete SHA is checked before use.

3. 3. Write exact complete manifest segment 2

   The compressed manifest is an original reader packaging of exact publisher metadata, not a model binary. Preserve every segment and fail if final checksum differs.

   CommandCopy command

   set -eu
   cd "$HOME/hermes4-405b"
   cat >> assets.b64 <<'MANIFESTPART'
   S5ClGQ0kcWerNwCCDWUKrB8891kzt3TXdtXVWaSOvDi9J/F6uAGIQkjhtKU5tSg3FqKEsZHWNcABHAKll7je1FfaVSVhkw0hlCDXTlHHP5KELoeeTsuvA46BxyCO0I8a0QFz04owYBJRjm/64tpdvy3mdTyjFWIa8LcQcwHXUucfO1XCyM3lfR1K3pE8aR76UiH/a7wjCe0ObfOxqaZ1AlOo0g5bkSvJMSMxKtg+1LRXIpaSrXHctm0v+E3MEK03G3RVSajoCUt+mYehNBJM0cALSgdy3dWEcGqCUfnRPcimBrDWc2hZvLL59pwO2xW35YVbDQDwj+wX6LJQHHDargPEuMTYetxId+8yUdU37JKPw6JsQwk9kgTvrtAW7kzoEqNRLSJlxsSaTFIvhGEOTdHNOFVeSIgALUrt8P5dLWiKp89nU64qCSpJzorg47Orx4qcd8C2siyiNPwKpQ/wdSXI3IRQADDVypnALqLoudfJu7tKwhBqwhzZd+A2oshrHyxLXpC0nIZ6NzxZexqIMhRoc9uqvNUK/nnmw7u7noTY22YvVFqDpcGN3AFmIwpPVNvSbLHoxNcjGYv6R2CXbqhl7wC549mUq0pCQbsPtTundIrfLUY1HzgXj+aWbMCqq/lTm9apIIAyUYbT6ryT7L2fTbnitqBts2zguDrQSf7LSQpG4mg0PwOZSSeaQH48lkUjS8XVQSJAd63zbMoV2rYeDgmOvXBACb4JHThgHgamkAXDBLsIZE4IoYRKYdG2LIH3gjPPplxVEhAcZ9tQJ3qLe2tAKKA9ADmeCGkiwHBoMo8yM7SyTBSq2WoIp2PxTTBfcduNXK9Qo5xgh9I5mDoJ4zIADzJ0PvX47KMZsseqDZvlAH8NEa25qmdTrtC2gibJt+Mr6Tf5eAAvaaAMU7AI3gxUNb60AT0Vf8wGFFNjDmuwas+m+KtKQg2v/v0QkyatF9CRJde7RnklwCCQEJRMHLc0NT1KCuxxnkBMA73PuOKvuC2BMFwg+5NUwvK1WNdRT0WXdkO/DpWxIY4J7hAQZNanBEE0jW/284wr/q4DrKKYQ1XZYg596gitKw0ykpVtcKOSWJSTo1o9q0OIsT2qTIWGPc8R5K8qCW73vQnLYb0uxOlZE+uSje0bzLap6dnNtGAx4B3h5IpLgO1Se5h7t0HpTr6TAlHFLo5NNG0otNBd3ddLFa+1EV9E8OgJl9bsftJsQ3UbxLPnGr/3V2ir6iQvbjZxla6qvRfLjTp0b6+S8QLH60A9x5YbwmCpcgmpw4vzm2D2V5UEdY0ekONgZpxDs+7wJDHIwytIStiEfkQo+p56tU2qPKrO+Ymi9m985YrbNpJa3J54CToGqqIMaB5oCng7QFadlQH9gD7sCmAGjxGMGWUNIW/t2ZQrtI1a7SqBkXd2peokLq0wYQ0uHNk0SUVeXyxEAJQb9hlnGsXNMWw9m9LuunpUSMdXvbrEUyYZrxB0HlciVp8gL0Yk75caKSnPidUsHvYan+vRlHDFbUXSEpuUCziLVi62JDHGkmRGv9bXDMY6IxtpCB+GuMyo6ceSemhvTLlCWwRg37V5yKvvJ23+pxZTFS5tALxCWtWT96p44+xJjAVmS+irwvwcQeGqkrB168ZC9JWmFqYKnXTQZ8QHH73qFLdBG1acHaU0A4sEkYBQJeSQCuzPplxxWxRoB2cXqL9QqA4uO6c49RoTRdrE1GotQclynKbR1ZHVqHzEceozobybJYsDodyGrrfwZFx1y+gYBJQbqzVtlCeeA8DnyNKZNQOURyaOQcO83qzKVSXB5WmtVK380aR9IKR47Gl1qi2jdenS0pPaZxLkypcOBS5Hc5Pq13s2pdy1/i4HtqhnRZ1xcFhEK4pHh8XqSVBbeLKSw3aF+JkO1zmhz0lEb/L1syl3aNs1SxZVQPfAhkY54GdwN/zYvDr1cMK6nIgKMuB073c0NnAF+MN5E8xXlQQJwhnVqx9C7ONMnTn4hFFo1e5HxGVbcSjoobNCEpFhaF+rwm1TKc+mXHHbUypULUBvVw3o9RKihxUUh2xUwwSLpp4JqFsCYUcbZSedUoUSwb3xTBLuZsn2Im480jTPRSZml7wutFgxgfFIDLUUB5WayAtewy912lptISDxr/0cQfGqkrB0OkjYgGjxFTYavsRhXR0aCz1qCY+arZiqjyIEiga8lhpy0Sr2jLbxrt+WjYc9Q8zW6NG2X6OBIMgOVIYftc2yQ85ZRdKQT8CdUYgp1doWhPg5M9/Nktmcsa/4mlNNQeP/XbWVank2lKoF2zEfOyceRIBttUdr5RLKY8z4HMzxqpJgBtFW86qaJAcwX1RfspAhtVmpiKxQ1D+ZyIUw27EtekKLj7XIFM+lnpjvTlTVn79xzkEIdd1MEHcZqUJZVFPw3W8Ua/asDIptwn5hDrqoJmLt8yGMv5slC6ohFKRQ0SQXbMFcXGqjIUnCqWfUSIEbw5GHNJyUNsuiqntzhNJzK42Pdz0JmiGzjjSfbI8OvBfgzz90M0E9Qy1xeHA/fVuARGp6swZcZpeMQnuGuHjFbdETWCCd4Sw4XWJUCyGUB/p1e8n00PbKfHPr5NkmUByB3iDwKW8KYHezZKbhCWKmacZvD7/4q8ZEY1OH4GwDo3CSVPzc/riii1fM4DNn90SaeDQlXVUSYGJkHjUtDkMEZl3NxUrV5M9oyDID0SANucud04RvrpageE0DXATYsyn+rhGArJPjKMjOI/KoBox4JoGSZ8kGXQNigWLdjrNzS8o+SVcVsF0pPUfQ3SzZFNg73Y00cYipYQuoJPkR8pq2J9IBN1JAqmn613USBwQqS00jup3u2ZR417iow59lYAEkMVsySL7r5BtER4C7DwdZwSsUU9Z1T4vhWM13+O08z2ib7rgtJK0cNbAgzL1Y49FtEoYihBzhu7Um9GBxiDWonIe2kDY3CJeWRr2eTblCW2C/1myl2iLVqV+xq2h6ku5nODspHMg2GEgCSBFUcxp7gNXwY3U8y/d0N92gea3SO4piq3lJ4uNoxHnLacFVpOJBFHidBdReoby6PcaK7kl802/r0909CXCVUSJb4UQSXxMuO8SRg6lxk+AOYaMZQ4WxwG95tcY/4HWH/7c3EXQ3S7YitKNu3LKN1A0hjHatxwF1E7IQpq/IZ6IJZmNYE0lCiKC1gZv55uQjXVUS5tTg2EyEBs/foVssJxw1S0DeRliw/pbJCzrThcv68LoNEVq32MI3bpuvuO1pmgVSZ2IofDGzrioqcb46Xif+u9aGmujYIclVdceULtHjAyBv43Op5+fPktUvD2F4Swc508CHQ4is5Xew5E7pYboRdAsLf4yi3ix1vyoLzNJP2FWVsmdTfmYlAeX3VTpMOjVeub/Gt3RUp8NMT/5hd/DLPtZrNnMlq03XpQG25CeYRIaLPkdQjjer0oYA/wAoLQw7saug4IbXLSsh6DI9lEdDIqPa5zCsQ6PAV9KKmqN6Y0q6MSURedDlkaBwB5Kg0hLOYi2GAIQV9c9rmG1C6tImdTt4/zELusRmP9/V43O+2aCyXk22QBpvEAqSgigVR4kVb17yZqfT+K1hmKitwleceqKBgWFvIqjcrIo/8ziotMakZoMeZGibhmeBM0REK6Si1YniMXc5WZ01CEkITj1T9+89m2JXG5TB9w6crqBLx7LunSlOs0DYNB0O42uGOaLOhCauVy+ixRsl9vbdBtWbDeo5+Km7Q0osHtpMGkzLti7TM7debSLxVUwJbbpty49lhe05bmzdTvZsSrtZle7gy6x16CAtIgy4V92kDs9uodPTkXBduhcgjg2bSzOp+pKSronMz7jy82fJ6lfVbBeKQbAHvAz6JZxD9fipuwWH7uzReOwiPeiymHKOZv0G+RIdf+J+ZvzF32yQJyVHjddARFKLS93gcYvJtqpJP83aG+lZBzCkxMq39tKddsRzQIw8mxJuVmXpLjoxDHTyduWoN4G8h54f+M008mM40fU0lmo95J481Zqr+0dI3c+Mv1yhrXvdyTmCayXAa1skH+2p0WJEzlI/noG5OssVEMM6KxLaVfBm6pa9N6akK4jLQdeQrVHVmN5VRhmrib6wKy3nHm14ciBeks8ajRRu9XWBj9fYxRtT8lUOIukWlQ3w0vLakzN17TLJuZ+8eyNkq06iQp0SqK9++v4qTjYA+hniyhXans0jX7OfFnzT/Theh+9ZE6EqNS7APUG3YdvQzlQmAgR6oMLP68alZ1PsaoOGRtS27vILHWBFKcNvR8iZSJ31YBq7RwLUaNV4tfENtbSQsJvOsp5NqVfAn8LSjQDAqB/b/FFHoK52FadGGa7KLkAYfKsa1O+6IQWKyfedrhZ64ytXaGsE8tH0y54xOfYH8ptA2TBI0up4JvOQdiAPg01Es1eDc/tTlq7xfB6X9eauIC60EFY7ycEIDpgbNFW2SIAIgLozqtTmSMPppAOlVHVqpKvbdC+zPd8m7u2K2x592JVOaERSP7q2FqSDw3ZRaaJ7rwTNU4+rvgNXzEWjDWTt8G7CztsV2g5ziWeoGw6pvhuOuU8DURGwCHpdsxUUzebaOrAVwT07lPVGW1fnP5oSryIorhh8zPiMbkPV6YMh/ybMdcVZFsJj+ubiAvDmKBBP1AkC1bkzWM43G3TFbVv0AMQoayxdlqGBC3h/1ySfmzrp1U3esyWda0IcULE2ROgiwHjSebMqd2jrSDA4IroGO3DeWXT5CBhcm+65qrZWPt6f01kZ73Ckklbnbbpaop9xxcrNBrUzdN18N19wzV67RvFTXGEcl3Up5Rni+ZUNMRRBbRmKF5vupXEOwvtsyhW3hT6ir9ZQS9GLh0ikbtthVlM65GtgRLOHKappW5e0oXNgdOSk/qYWZ1do+5ob14F2y68Je98sD6+D1bZ3Wq+uY93P2eEOZwV4lZocw2n4N3D7xpR2s0HweFP/rLitruTxR3Mmuqh7sQwZ69gnVJoLSePEoUIngTAdrQ6HJHk0pV5x2xMOYKtJ3KDb+C2oQZKN4ZOzLaQfxDQrhTvvfOLZ+mUGQLJBMnmz+Xw0Va/Q1mvUR+y6b10xSC5U84juuwVtLUKOIoGe2an0Pzfvqmtuq69Qv/NivzHlqpIAq154Ko/QrXkBZMsE8Laa1DXu9bbhdSd+wYcgwiD/SgQSeqVpmOvZlCtuaz2wD3FqxVV14+m9k++ketTc7yx3MmGNpEKXYwGIu+hLYMHw72d1WK/QtpaDQIfUo0abWlKKRrRqOKY+K1fG0QQeUVPAOA2RIUaq5iBrQMqXZxZX850O0o0emrAhlFJrdS7d4eCQ8QggMl8l4+jaxSJUAWZzUkEmuWjHtn9G23rFbTfaigzr0muUPTgr8/Vv1hhEbyFeGr6hozIIw6sIoyv5w9DsuW72ezblrpKgTog5Dh8XxWEzI380d2msjmOlkBlwya4wDwn+4hXgulfP6WL28wxx9aqSEHRlhKYu3ekANgBWgoonx0OkdHFTCzhNUYmwsHmoITYTUpkOMEQafzblitvCaSU9t6VYIYlgKNyx8TciVclGtzeBwa8Lu8vZrNM8OmlwHuGKcHo0pf0cbmu5VV++9BXdeBNejYt5BVd8XyjFVNdLlBbSD+sI8Q9tl+R50cbQGDUws2Jfzxv007NkSNLSgsUvTTH9FpE2Mac68EXVJPU6RRKyrgLV7WTWYHroZ90qu0/QdDz/Jhf13Pz/K6a+fPa3H75f+z//6pcGaXThy9/LtAkYUAM/1G1xRVeGOw0MjKqLoBde2iGPmoacbCF6o4D9ESaBCWSJryz49MOeH/off/v5479hwW+/6z/81cNTSl+Balm8IYLUqf1bF57FqV5e9fWC5iQ8/QYAdsgANrWIvG4kYG+qs/o1sX898sN/7R//+nckmRz+q08c20763SC5zN5saJJjaRhW3aIlpr73jnz8BMKGFXQnTu+6A9ugImj39fjY59/QlJELXz56bLQd1GIPHdaH/Jo0
   MANIFESTPART

   Expected result: Manifest segment written; complete SHA is checked before use.

4. 4. Write exact complete manifest segment 3

   The compressed manifest is an original reader packaging of exact publisher metadata, not a model binary. Preserve every segment and fail if final checksum differs.

   CommandCopy command

   set -eu
   cd "$HOME/hermes4-405b"
   cat >> assets.b64 <<'MANIFESTPART'
   nTYbXlX0m03UJKkSwZEPuKJr48vKOeAapDXSyL/+5W/+G5hMUnY=
   MANIFESTPART

   Expected result: Manifest segment written; complete SHA is checked before use.

5. 5. Decode, bind and fetch the complete native checkpoint

   The reader alone downloads all 191 weight parts and eight auxiliary assets. Existing corrupt files fail; complete streaming size/SHA checks precede native loading. No partial/alternate checkpoint is used.

   CommandCopy command

   set -eu
   cd "$HOME/hermes4-405b"
   python3 - <<'PYMANIFEST'
   import base64,zlib,hashlib,pathlib
   b=zlib.decompress(base64.b64decode(pathlib.Path("assets.b64").read_bytes()))
   assert hashlib.sha256(b).hexdigest()=="9257bca5b7f7e450f8a7e6c77b1ebede425c68677dfc6d1d658600ada734c28c"
   pathlib.Path("assets.json").write_bytes(b)
   PYMANIFEST
   python3 reader_assets.py download

   Expected result: All 199 exact asset size and SHA checks pass. Integrity is not model fit or answer quality.

Configure

1. 1. Write the exact native raw protocol client

   Native tokenizer/template is used only during reader execution inside the pinned container. Full raw assistant thinking/tool history is retained; no tool executor exists. All special-token IDs and completion budgets are model-specific. The fixed fixture reserves angle brackets for complete native thought/tool boundaries; unexpected, repeated or truncated role/control markers reject the response. Raw IDs must end in the genuine native EOS, while explicit vLLM output flags omit EOS from decoded text. The reader native tokenizer must reproduce the entire returned body exactly; no role/control or terminal-token text is silently stripped.

   CommandCopy command

   set -eu
   cd "$HOME/hermes4-405b"
   cat > reader_client.py <<'PYCLIENT'
   """Reader-owned native Hermes 4 405B fixed protocol check; no tool executor."""
   import http.client,json,pathlib,re
   ROOT=pathlib.Path('/work');PORT=8000;MODEL='hermes4-405b-native';CONTEXT=8192;OUTPUT=4096
   TOOLS=[{'type':'function','function':{'name':'lookup_demo_constant','description':'Return the inert synthetic constant; no external actions.','parameters':{'type':'object','properties':{'key':{'type':'string','enum':['alpha']}},'required':['key'],'additionalProperties':False}}}]
   def unique(pairs):
    d={}
    for k,v in pairs:
     if k in d:raise ValueError('duplicate_key')
     d[k]=v
    return d
   def parse(raw):
    if type(raw)is not bytes or len(raw)>2097152:raise ValueError('response_bound')
    return json.loads(raw.decode(),object_pairs_hook=unique,parse_constant=lambda _:(_ for _ in()).throw(ValueError('nonfinite')))
   def budget(ids):
    if type(ids)is not list or not 1<=len(ids)<=CONTEXT-OUTPUT or ids[0]!=128000 or ids.count(128000)!=1 or any(type(i)is not int or not 0<=i<128256 for i in ids):raise ValueError('native_prompt_budget_or_BOS')
    return len(ids)
   def checked(d,count,tool,decode=None):
    if type(count)is not int or not 1<=count<=CONTEXT-OUTPUT or type(tool)is not bool:raise ValueError('request_contract')
    if type(d)is not dict or d.get('model')!=MODEL or type(d.get('choices'))is not list or len(d['choices'])!=1:raise ValueError('identity')
    c=d['choices'][0]
    if type(c)is not dict:raise ValueError('choice')
    ids=c.get('token_ids');text=c.get('text');stop=c.get('stop_reason')
    if c.get('finish_reason')!='stop' or (stop is not None and type(stop)is not int):raise ValueError('incomplete')
    if type(ids)is not list or not 1<=len(ids)<OUTPUT or any(type(i)is not int or not 0<=i<128256 for i in ids):raise ValueError('raw_token_ids')
    # Released vLLM keeps the sampled terminal token in raw token_ids even when decoded output excludes it.
    if ids[-1]not in (128001,128009) or (stop is not None and stop!=ids[-1]):raise ValueError('terminal_eos')
    expected=[128011,128012]+([128002,128013]if tool else [])+[ids[-1]]
    if [i for i in ids if i>=128000]!=expected:raise ValueError('native_control_order_count_or_role')
    positions=[i for i,t in enumerate(ids)if t>=128000]
    if positions[1]<=positions[0]+1 or (not tool and positions[-1]<=positions[-2]+1):raise ValueError('empty_native_span')
    if tool and positions[3]<=positions[2]+1:raise ValueError('empty_tool_span')
    u=d.get('usage')
    if type(u)is not dict or any(type(u.get(k))is not int for k in ('prompt_tokens','completion_tokens','total_tokens')):raise ValueError('usage_integer')
    if u['prompt_tokens']!=count or u['completion_tokens']!=len(ids)or u['total_tokens']!=count+len(ids)or count+len(ids)>CONTEXT:raise ValueError('usage')
    if type(text)is not str:raise ValueError('text')
    # Fixed fixture grammar reserves angle brackets for complete native boundaries only.
    # This conservative check rejects truncated/unknown markers instead of sanitizing them.
    match=re.fullmatch(r'\s*<think>([^<>]+)</think>(.*)',text,re.S)
    if not match or not match[1].strip():raise ValueError('reasoning_boundary')
    tail=match[2].strip()
    if tool:
     call=re.fullmatch(r'<tool_call>\s*([^<>]+)\s*</tool_call>',tail,re.S)
     if not call:raise ValueError('tool_boundary')
     obj=parse(call[1].encode())
     if type(obj)is not dict or set(obj)!={'name','arguments'}or obj['name']!='lookup_demo_constant'or type(obj['arguments'])is not dict or obj['arguments']!={'key':'alpha'}:raise ValueError('inert_tool_required')
    elif not tail or '<'in tail or '>'in tail:raise ValueError('final_required')
    if decode is not None:
     # Native reader tokenizer closes text/raw-ID identity; terminal EOS is deliberately absent in text.
     if decode(ids[:-1])!=text:raise ValueError('native_decode_identity')
    return text
   
   def run(encode,request,decode=None):
    messages=[{'role':'user','content':'Use lookup_demo_constant with key alpha, then explain in one sentence why its synthetic value plus one equals eight. This is only an inert protocol experiment.'}];results=[]
    for tool in [True,False]:
     ids=encode(messages,TOOLS);n=budget(ids)
     d=request({'model':MODEL,'prompt':ids,'max_tokens':OUTPUT,'temperature':.6,'top_p':.95,'top_k':20,'n':1,'stream':False,'add_special_tokens':False,'skip_special_tokens':False,'spaces_between_special_tokens':False,'include_stop_str_in_output':False,'return_token_ids':True,'stop_token_ids':[128001,128009]})
     text=checked(d,n,tool,decode);results.append(d)
     if tool:
      # Entire raw native assistant body retained. Native Jinja handles tool-response serialization.
      history=text
      messages.append({'role':'assistant','content':history})
      messages.append({'role':'tool','content':'{"key":"alpha","value":7,"synthetic":true}'})
    return results
   
   def main():
    # Reader execution only: pinned local native tokenizer, not model loading or a replacement template.
    from transformers import AutoTokenizer
    tokenizer=AutoTokenizer.from_pretrained('/model',local_files_only=True)
    exact=(ROOT/'model/chat_template.jinja').read_text()
    if tokenizer.chat_template!=exact:raise ValueError('native_template_equality')
    def encode(messages,tools):
     return tokenizer.apply_chat_template(messages,tools=tools,thinking=True,keep_cots=True,add_generation_prompt=True,tokenize=True,truncation=False)
    key=(ROOT/'.api-key').read_text().strip()
    if len(key)!=64 or any(c not in '0123456789abcdef'for c in key):raise ValueError('key')
    def request(body):
     payload=json.dumps(body,allow_nan=False).encode()
     if len(payload)>65536:raise ValueError('request_bound')
     c=http.client.HTTPConnection('127.0.0.1',PORT,timeout=180)
     try:
      c.request('POST','/v1/completions',payload,{'Authorization':'Bearer '+key,'Content-Type':'application/json'});r=c.getresponse();raw=r.read(2097153)
      if r.status!=200:raise ValueError('endpoint_status_'+str(r.status))
      return parse(raw)
     finally:c.close()
    results=run(encode,request,lambda ids:tokenizer.decode(ids,skip_special_tokens=False,clean_up_tokenization_spaces=False));(ROOT/'result.json').write_text(json.dumps(results,indent=2,allow_nan=False)+'\n')
    print('Closed native reasoning, inert tool and final response protocol passed. Inspect the arithmetic manually; no measured quality or speed claim.')
   if __name__=='__main__':main()
   PYCLIENT
   python3 -m py_compile reader_assets.py reader_client.py
   sha256sum --check <<'SHA256'
   cc4476b0831d18ff727987b667745df0b11817e1065128e48f4b76c2d1a4526f  reader_client.py
   729d2ca30a30f86a75209db2952805a4d1d1a0218687133a6079035db4ee3ad9  reader_assets.py
   SHA256

   Expected result: Own stdlib client/checker syntax and exact checksums pass without native execution.

2. 2. Build the pinned native runtime layer

   Official vLLM amd64 image is immutable; the same source-qualified exact Transformers replacement used in prior native guides is explicit. This image is unbuilt here. Stop on pip check, version, CUDA, tensor-parallel or loader errors; record realized versions.

   CommandCopy command

   set -eu
   cd "$HOME/hermes4-405b"
   cat > Dockerfile <<'DOCKERFILE'
   FROM vllm/vllm-openai@sha256:c2f3b1b964e47809b722b5e75b61b1e7b39a50f70388cf2bf2418f16a9f31da2
   RUN python3 -m pip install --no-deps 'https://files.pythonhosted.org/packages/fc/b1/8be7e7ef0b5200491312201918b6125ef9c9df9dd0f0240ccef9ac824e6b/transformers-5.8.1-py3-none-any.whl#sha256=5340fb95962162cdfdae5cc91d7f8fedd92ed75216c1154c5e1f590fcf56dd0e' && python3 -m pip check
   RUN python3 -c "import vllm,transformers; assert vllm.__version__=='0.27.1'; assert transformers.__version__=='5.8.1'"
   ENTRYPOINT ["python3", "-m", "vllm.entrypoints.openai.api_server"]
   DOCKERFILE
   sudo docker build --pull -t hermes4-405b:reader .
   sudo docker run --rm --gpus all --entrypoint nvidia-smi hermes4-405b:reader
   sudo docker run --rm --entrypoint python3 hermes4-405b:reader -m pip freeze > resolved-container-environment.txt
   sudo docker network inspect hermes405-private >/dev/null 2>&1 || sudo docker network create --internal hermes405-private

   Expected result: Only after actual reader execution: package/version guards and all eight GPU visibility pass; no model load implied.

Start the model

1. 1. Start complete native BF16 with tensor parallel eight

   Foreground lifetime is 1,800 seconds including loading; it is not an inference deadline/SLA. Native full loader must account for all model parameters; do not bypass diagnostics, precision or architecture failure.

   CommandCopy command

   set -eu
   cd "$HOME/hermes4-405b"
   python3 reader_assets.py verify
   timeout --signal=TERM --kill-after=10s 1800s sudo docker run --rm --name hermes4-405b --gpus all --network hermes405-private -p 127.0.0.1:8994:8000 --shm-size 64g --env-file .api-env -v "$PWD/model:/model:ro" -v "$PWD:/work" hermes4-405b:reader --model /model --served-model-name hermes4-405b-native --host 0.0.0.0 --port 8000 --dtype bfloat16 --kv-cache-dtype auto --tensor-parallel-size 8 --max-model-len 8192 --max-num-seqs 1 --max-num-batched-tokens 8192 --gpu-memory-utilization 0.95 --no-enable-prefix-caching --enforce-eager --chat-template /model/chat_template.jinja

   Expected result: Only after actual reader initialization: complete native model starts without missing-parameter/collective/CUDA errors and private health becomes ready. Unknown fit may prevent startup.

Verify the setup

1. 1. Check private routes and the full two-turn native protocol

   Run on the host from a second terminal. Native health is unauthenticated but loopback-only; v1 requires the key. Execute the pinned reader client within the running container so it uses its exact local tokenizer. Inspect actual final arithmetic and record errors; protocol closure is not accuracy or calibrated quality.

   CommandCopy command

   set -eu
   cd "$HOME/hermes4-405b"
   curl --fail --max-time 5 http://127.0.0.1:8994/health
   test "$(curl -s --max-time 5 -o /dev/null -w '%{http_code}' http://127.0.0.1:8994/v1/models)" = 401
   sudo docker exec hermes4-405b python3 /work/reader_client.py

   Expected result: After actual reader inference only: one exact inert lookup call followed by a closed-think nonempty final response, with full raw IDs/usage recorded in result.json. Manually check seven plus one; no tool was executed.

Performance evidence

No compatible measurement available. Documented support and estimated memory do not establish response speed.

Limits and troubleshooting

• No install, build, tokenizer execution, weight download, native inference, provisioning or spending occurred. Source compatibility is not measured runtime success.
• Native config has 131,072 positions. Reduced walkthrough budgets do not establish full native window fit, quality or speed. Input count includes one BOS, tools and full retained native history.
• Native generation EOS is 128001, tokenizer EOT is 128009; both stops are sent. Complete raw think/tool closure and finish_reason stop required; length termination fails. Reasoning does not establish correctness.
• Publisher card uses a different illustrative repository spelling and recommends top_p 0.95, while selected generation metadata says 0.9. Exact selected repo/revision and card sampler are explicit, with no provider-alias byte equality.
• Native card llama3 metadata and Llama 3.1 base identify distinct agreements; both actual term/AUP sets retained. No later-base license silently replaces the linked license or grants legal clearance.
• Eight H200 per-device usable capacity, actual TP balance, kernel/collective buffers and peak are unknown. All weights remain; no smaller model or offload workaround is silently substituted.
• Docker workspace mount contains the private key; the container and Docker administrator are trusted. No arbitrary tools, uploads, URL fetch or remote model choices are provided.
• This is a distinct owned platform configuration. Nominal2TiB RAM/disk and 64 threads are planning allocations, not measured requirements. Read actual memory, eight unique SM90 devices, drivers and peer topology before loading; stop on unsupported connectivity.
• The source-evidenced cloud companion remains unchanged. No accepted cloud guide is reapplied or counted as this new private job.

Manifest, template, native loading, CUDA or TP failure

Stop and preserve diagnostics without keys. Restore exact full pinned assets/image and all eight H200 devices. Do not omit tensors, reduce precision, substitute 70B or relax strict load checks.

Length termination or native reasoning/tool mismatch

Treat as incomplete. Preserve actual results where available; do not silently truncate history, strip reasoning or execute returned tools. Larger budgets need separate qualification.

OOM or timeout

Stop the named dedicated container and verify its endpoint has stopped. Preserve wanted results and source/legal records, then remove only the documented dedicated files, image tag and network. Manage the existing owned host, retained local storage and electricity separately; no EC2, EBS or cloud billing lifecycle is assumed.

Stop the model

1. 1. Stop the named native experiment

   Use a second server terminal; foreground timeout is not proof the container exited. Kill only this dedicated experiment if it remains running.

   CommandCopy command

   sudo docker stop --time 10 hermes4-405b || true
   if [ "$(sudo docker inspect -f '{{.State.Running}}' hermes4-405b 2>/dev/null || true)" = true ]; then sudo docker kill hermes4-405b; fi
   curl --max-time 3 http://127.0.0.1:8994/health || true

   Expected result: The exact dedicated container and endpoint stop. Owned host/storage/electricity remain reader-managed; no cloud billing applies.

Uninstall

1. 1. Remove dedicated checkpoint, runtime tag and private files

   Preserve wanted results and legal/source records first. Confirm dedicated paths. Shared drivers/Docker and retained image layers remain; no broad prune.

   CommandCopy command

   sudo docker rm -f hermes4-405b 2>/dev/null || true
   sudo docker network rm hermes405-private
   sudo docker image rm hermes4-405b:reader
   rm -r -- "$HOME/hermes4-405b"

   Expected result: Dedicated files, results/key and image tag/network removed; shared packages/layers remain.