Local hardwareDocumented setupSources checked Oct 17, 2026
Host full native Nemotron 3.5 Lightning reasoning and tools on an owned H100
Serve the exact full native BF16 Lightning checkpoint with all experts and embedded MTP, own publisher template and preserved reasoning/tool history. A bounded 8,192-position/4,096 output two-turn inert-tool check verifies native raw completion boundaries, not quality or speed. One H100 80 GB and 256 GiB host RAM/disk are source-qualified planning, not measured fit; no model execution occurred.
- Hardware
- Owned single NVIDIA H100 80 GB GPU host
- Runtime
- Native Lightning BF16 + embedded MTP1 reader-built vLLM GPU layer vLLM 0.27.1 commit 6e448d0; pinned official amd64 image plus SHA-pinned Transformers 5.8.1 replacement; CUDA 13.0.3 source defaults, reader preflight required · CUDA
- Model format
- Native HF sharded safetensors · Native BF16 complete publisher weights, including all experts and embedded MTP; no quantization · Unquantized BF16 publisher checkpoint; full experts and embedded MTP retained
- Workload
- 8,192 token context · 1 concurrent
Based on source documentation; this setup has not been executed by OpenTools. Memory: unknown. No compatible measurement available.
Read the complete setup guide
Before you start
- Use one owned H100 80 GB on Ubuntu 24.04 x86_64, driver >=580.126.20, Docker and NVIDIA Container Toolkit already working. Use the official installation guide for prerequisite setup; a GPU Docker visibility failure stops this recipe. No card purchase, provisioning or terms acceptance is performed by the author.
- Choose 256 GiB host RAM, 256 GiB disk allocation and 16 CPU threads, with at least 160 GiB free before native downloads. These are planning choices, not measured minima. The GPU advertises 80 GB, with exact usable bytes unresolved. All 14 weight parts, every expert and embedded MTP are retained.
- Review the exact native OpenMDW1.1 terms, third-party-rights and notice obligations. This private research walkthrough accepts nothing on your behalf. Native API/provider byte equivalence and frozen August 11 release versus selected later repository revision are not assumed.
- 8,192 total context and 4,096 output per turn are bounded author choices. Complete history must fit without truncation. Actual fit, speed, task quality, output length and spend remain unknown. Do not interpret 3B active as resident memory.
Exact model and requirements
- Artifact:
- NVIDIA / Nemotron 3.5 Lightning 30B A3B full native BF16 including embedded MTP · checkpoint nvidia/NVIDIA-Nemotron-3.5-Lightning-30B-A3B-BF16
- Publisher revision:
- a9904d24bcc1d289a1950fa9d2b978c47cf903b9
- Artifact digest:
- Composite whole-checkpoint digest not published; exact 14 weight-part SHA256 values and all 22 asset sizes/hashes are pinned in embedded reader manifest.
- Download size:
- 61.31 GiB
- Memory requirement (unknown):
- Unknown — No measured GPU/full-process peak for full native BF16/MTP1 / FP32 SSM at 8,192 context.
- System RAM / disk:
- Unknown / Unknown
- Placement:
- single device. One GPU; no tensor/expert/pipeline parallel decomposition. Full target and embedded MTP on one H100, no CPU/disk weight offload. Host download/loading/cache buffers are additional and unmeasured.
- KV cache:
- Attention KV auto/BF16; recurrent Mamba SSM state explicitly FP32; these are distinct cache types
- Workload:
- 8,192 token context; Unknown; Unknown; concurrency 1; batch 1; cache cold; repetitions Unknown.
- Tokenizer:
- Exact native Lightning tokenizer and independently pinned 9,867-byte publisher Jinja · a9904d24bcc1d289a1950fa9d2b978c47cf903b9
- Sampling / reasoning:
- Native publisher temperature 1.0 and top_p 0.95; n=1, nonstreaming, explicit stop IDs 2/11 and return raw generated IDs. No generic Qwen template or task sampling transferred. Native thinking enabled, force_nonempty_content false and truncate_history_thinking false. Require raw closing-think token=13, complete native tool tokens 14/15 on first turn, EOS/stop completion and nonempty final content. Preserve full reasoning and tool history.8,192 context/4,096 output is author-selected bounded walkthrough, not publisher full-reasoning recommendation.
- Runtime prerequisites:
- Ubuntu 24.04 x86_64 host; pinned Ubuntu 22.04 / Python 3.12 source defaults container; NVIDIA Linux driver >=580.126.20, Working Docker GPU support with NVIDIA Container Toolkit; actual visibility preflight required.
Exact publisher OpenMDW1.1 grant and full terms are archived. Exercising rights accepts its conditions; no acceptance is performed here. Retain agreement and relevant origin/copyright notices on distribution; specified patent/copyright litigation terminates grants. Outputs have no additional license restrictions, but user remains responsible for third-party rights, consents and due diligence. Warranty/liability exclusions apply. This guide grants no legal clearance.
- config.json · 0 GiB · revision a9904d24bcc1d289a1950fa9d2b978c47cf903b9
- tokenizer.json · 0.02 GiB · revision a9904d24bcc1d289a1950fa9d2b978c47cf903b9
- tokenizer_config.json · 0 GiB · revision a9904d24bcc1d289a1950fa9d2b978c47cf903b9
- special_tokens_map.json · 0 GiB · revision a9904d24bcc1d289a1950fa9d2b978c47cf903b9
- chat_template.jinja · 0 GiB · revision a9904d24bcc1d289a1950fa9d2b978c47cf903b9
- generation_config.json · 0 GiB · revision a9904d24bcc1d289a1950fa9d2b978c47cf903b9
- model.safetensors.index.json · 0 GiB · revision a9904d24bcc1d289a1950fa9d2b978c47cf903b9
- 65,827,374,264 native serialized weight bytes include full experts and MTP, not complete runtime memory.
- GPU advertised 80 GB usable byte capacity unresolved. All weights, target/draft activations, attention KV, recurrent states, workspace, container/compiler, host buffers and OS add unmeasured overhead.
One-time setup and hardware
- hardware
- Unknown — Existing owned GPU/host purchase and incremental cost are unknown.
- Existing owned GPU/host purchase and incremental cost are unknown. Tax: unknown.
- setup
- Unknown — Reader provisioning, image build and labor time/cost are unknown.
- Reader provisioning, image build and labor time/cost are unknown. Tax: unknown.
Recurring costs
- electricity
- Unknown — Actual wall power/rate/poweredhours unknown; no GPU TDP conversion.
- Actual wall power/rate/poweredhours unknown; no GPU TDP conversion. Tax: unknown.
- storage
- Unknown — Incremental local 256 GiB storage cost is unknown.
- Incremental local 256 GiB storage cost is unknown. Tax: unknown.
- egress
- Unknown — Actual transfer/allowance/traffic cost unknown.
- Actual transfer/allowance/traffic cost unknown. Tax: unknown.
- administration
- Unknown — Maintenance time/rate unknown.
- Maintenance time/rate unknown. Tax: unknown.
Install the runtime
1. Prepare an existing GPU host and private workspace
Use an already provisioned Ubuntu 24.04 x86_64 H100 80 GB host with working NVIDIA driver >=580.126.20. Follow the linked official NVIDIA Container Toolkit installation guide if GPU Docker is absent; do not bypass GPU/driver failure or substitute another architecture. The commands below install only host utilities/Docker; no model executes during source authoring.
Commandset -eu sudo apt-get update sudo apt-get install -y docker.io python3 curl ca-certificates openssl sudo systemctl enable --now docker mkdir -p "$HOME/nemotron35-lightning" cd "$HOME/nemotron35-lightning" nvidia-smi --query-gpu=name,memory.total,driver_version --format=csv,noheader python3 - <<'PYPREFLIGHT' import csv,subprocess,shutil r=list(csv.reader(subprocess.check_output(["nvidia-smi","--query-gpu=name,memory.total,driver_version","--format=csv,noheader,nounits"],text=True).splitlines())) assert len(r)==1 and "H100" in r[0][0] and "80GB" in r[0][0].replace(" ","") assert tuple(map(int,r[0][2].strip().split("."))) >= (580,126,20) assert shutil.disk_usage(".").free >=160*1024**3 print("Reader GPU/driver and prescribed160GiB free-disk preflight passed; no model fit measured.") PYPREFLIGHT sudo docker versionExpected result: Record actual GPU/driver and available host RAM/disk. Stop if the exact H100 80 GB/driver prerequisite, Docker GPU support or prescribed free space fails. Host capacity is planning, not measured native fit.
2. Configure NVIDIA GPU containers on this dedicated host
Use the official production repository and exact documented Container Toolkit 1.20.1-1 packages. This modifies Docker daemon configuration and restarts Docker on this dedicated host; preserve its original configuration and do not apply on a shared production daemon without its owner review. Stop on repository/signature/package errors.
Commandset -euo pipefail sudo apt-get install -y gnupg2 curl --fail --silent --show-error --location https://nvidia.github.io/libnvidia-container/gpgkey | sudo gpg --batch --yes --dearmor -o /usr/share/keyrings/nvidia-container-toolkit-keyring.gpg curl --fail --silent --show-error --location https://nvidia.github.io/libnvidia-container/stable/deb/nvidia-container-toolkit.list | sed 's#deb https://#deb [signed-by=/usr/share/keyrings/nvidia-container-toolkit-keyring.gpg] https://#g' | sudo tee /etc/apt/sources.list.d/nvidia-container-toolkit.list >/dev/null sudo apt-get update sudo apt-get install -y nvidia-container-toolkit=1.20.1-1 nvidia-container-toolkit-base=1.20.1-1 libnvidia-container-tools=1.20.1-1 libnvidia-container1=1.20.1-1 sudo nvidia-ctk runtime configure --runtime=docker sudo systemctl restart dockerExpected result: The exact GPU container packages install and Docker is configured. The subsequent container nvidia-smi check must pass; this is not model inference or fit proof.
Download the model
1. Create a stable private key and complete asset checker
Generate a key only when absent. Keep key files private; environment-file access is equivalent to this service key. The checker uses only stdlib and streaming hashes before native loading.
Commandset -eu cd "$HOME/nemotron35-lightning" umask 077 python3 - <<'PYPREWRITE' import os,pathlib,stat paths=[pathlib.Path(n)for n in [".api-key",".api-env"]] for p in paths: try:s=p.lstat() except FileNotFoundError:continue if not stat.S_ISREG(s.st_mode):raise ValueError('existing secret target must be a regular file; no links') for p in paths: if p.exists():os.chmod(p,0o600) PYPREWRITE if [ ! -e .api-key ]; then openssl rand -hex 32 > .api-key; fi python3 - <<'PYKEY' from pathlib import Path k=Path(".api-key").read_text().strip() assert len(k)==64 and all(c in "0123456789abcdef" for c in k) Path(".api-env").write_text("VLLM_API_KEY="+k+"\nHF_HUB_OFFLINE=1\nTRANSFORMERS_OFFLINE=1\n") PYKEY chmod 600 .api-key .api-env cat > reader_assets.py <<'PYCHECKER' """Reader-run full checkpoint integrity; no model imports.""" import hashlib import json import pathlib import sys import urllib.request ROOT = pathlib.Path(__file__).resolve().parent MANIFEST_SHA = '5d7d233cad9eb1113534957de3315e1845bbc49de985b165a6230841194d7b47' def verified_manifest(): raw = (ROOT / 'assets.json').read_bytes() if hashlib.sha256(raw).hexdigest() != MANIFEST_SHA: raise ValueError('manifest_hash') d = json.loads(raw) if d['revision'] != 'a9904d24bcc1d289a1950fa9d2b978c47cf903b9' or d['repository'] != 'nvidia/NVIDIA-Nemotron-3.5-Lightning-30B-A3B-BF16' or len(d['files']) != 22: raise ValueError('manifest_identity') return d['files'] def check(path, entry): digest = hashlib.sha256() count = 0 with path.open('rb') as f: while block := f.read(1024 * 1024): count += len(block) digest.update(block) if count != entry['bytes'] or digest.hexdigest() != entry['sha256']: raise ValueError('asset_integrity_' + entry['path']) def main(download): files = verified_manifest() out = ROOT / 'model' out.mkdir(exist_ok=True) for entry in files: dest = out / entry['path'] if pathlib.PurePosixPath(entry['path']).name != entry['path']: raise ValueError('asset_path') if download and not dest.exists(): temp = dest.with_suffix(dest.suffix + '.partial') with urllib.request.urlopen(entry['url'], timeout=60) as r, temp.open('wb') as f: total = 0 while block := r.read(1024 * 1024): total += len(block) if total > entry['bytes']: raise ValueError('download_size') f.write(block) check(temp, entry) temp.replace(dest) check(dest, entry) cfg = json.loads((out / 'config.json').read_text()) assert cfg['hidden_size'] == 2688 and cfg['num_hidden_layers'] == 52 assert cfg['n_routed_experts'] == 128 and cfg['num_experts_per_tok'] == 6 assert cfg['num_nextn_predict_layers'] == 1 and cfg['mtp_layers_block_type'] == ['attention','moe'] print('All 22 exact assets passed size and streaming checksum checks, including every expert and embedded MTP weight.') if __name__ == '__main__': if sys.argv[1:] not in (['download'], ['verify']): raise SystemExit('Use download or verify') main(sys.argv[1] == 'download') PYCHECKER sha256sum --check <<'SHA256' e6d2cc59dfd775641cb56ef8939279f21b93606bd018f210339b3610cb2fae28 reader_assets.py SHA256Expected result: Existing access is preserved, key format is checked and exact checker bytes verify. No key is printed and no native model operation occurs.
2. Write the complete exact native manifest
All 14 weight parts and eight native auxiliary files are required, including embedded MTP and tokenizer vocabulary. This manifest has no binary contents. The author fetched only official source/metadata, not these reader weight/tokenizer bodies.
Commandset -eu cd "$HOME/nemotron35-lightning" cat > assets.json <<'JSONASSETS' {"repository":"nvidia/NVIDIA-Nemotron-3.5-Lightning-30B-A3B-BF16","revision":"a9904d24bcc1d289a1950fa9d2b978c47cf903b9","files":[{"path":"LICENSE","bytes":2693,"sha256":"d7c8a9e5d1896d0a9588319cc7b1433e64645ad6d9e55632c30b78d8c038c23b","url":"https://huggingface.co/nvidia/NVIDIA-Nemotron-3.5-Lightning-30B-A3B-BF16/resolve/a9904d24bcc1d289a1950fa9d2b978c47cf903b9/LICENSE"},{"path":"chat_template.jinja","bytes":9867,"sha256":"58933db77d3099b4f78c55a38347a72e1ea05b97d6bd8f38775303dc0194e0a9","url":"https://huggingface.co/nvidia/NVIDIA-Nemotron-3.5-Lightning-30B-A3B-BF16/resolve/a9904d24bcc1d289a1950fa9d2b978c47cf903b9/chat_template.jinja"},{"path":"config.json","bytes":2352,"sha256":"a3827a0f5e311547b40943dc081e3ff2f8a277466e8c1a3df2291e8db8a7617c","url":"https://huggingface.co/nvidia/NVIDIA-Nemotron-3.5-Lightning-30B-A3B-BF16/resolve/a9904d24bcc1d289a1950fa9d2b978c47cf903b9/config.json"},{"path":"generation_config.json","bytes":210,"sha256":"0ed94823ae737b507aef6af01e837041b7aa2fb0306793a310cd201d89a31ff4","url":"https://huggingface.co/nvidia/NVIDIA-Nemotron-3.5-Lightning-30B-A3B-BF16/resolve/a9904d24bcc1d289a1950fa9d2b978c47cf903b9/generation_config.json"},{"path":"model-00001-of-00014.safetensors","bytes":4991210024,"sha256":"7a3e74d24969eac6657cb9b6422091fd062b7d30fd4bc90ecee6c3d7f692626a","url":"https://huggingface.co/nvidia/NVIDIA-Nemotron-3.5-Lightning-30B-A3B-BF16/resolve/a9904d24bcc1d289a1950fa9d2b978c47cf903b9/model-00001-of-00014.safetensors"},{"path":"model-00002-of-00014.safetensors","bytes":4992601016,"sha256":"a80c3846e959f6dda34aa031cbb668b766643691ac7e5c9f43f62e6fb1221059","url":"https://huggingface.co/nvidia/NVIDIA-Nemotron-3.5-Lightning-30B-A3B-BF16/resolve/a9904d24bcc1d289a1950fa9d2b978c47cf903b9/model-00002-of-00014.safetensors"},{"path":"model-00003-of-00014.safetensors","bytes":4992601368,"sha256":"996bf3a46668e14075cb82259390267c2b1f71446caeb3a5f1f9e0bd7a079ae7","url":"https://huggingface.co/nvidia/NVIDIA-Nemotron-3.5-Lightning-30B-A3B-BF16/resolve/a9904d24bcc1d289a1950fa9d2b978c47cf903b9/model-00003-of-00014.safetensors"},{"path":"model-00004-of-00014.safetensors","bytes":4995692800,"sha256":"cce1d79e7ff2e2b746b9d97166156aa1dcec796116675cc7fd1aa70a8d73ccb2","url":"https://huggingface.co/nvidia/NVIDIA-Nemotron-3.5-Lightning-30B-A3B-BF16/resolve/a9904d24bcc1d289a1950fa9d2b978c47cf903b9/model-00004-of-00014.safetensors"},{"path":"model-00005-of-00014.safetensors","bytes":4980553832,"sha256":"9338f59d3fd5f074288c71a9559452d5b5c581da00e9ccb9fe7aa2ce9b04d2ad","url":"https://huggingface.co/nvidia/NVIDIA-Nemotron-3.5-Lightning-30B-A3B-BF16/resolve/a9904d24bcc1d289a1950fa9d2b978c47cf903b9/model-00005-of-00014.safetensors"},{"path":"model-00006-of-00014.safetensors","bytes":4989423536,"sha256":"e947fa4097c68263ba3d097fceb32a300a22cebd9d909fb5198af7d18a4022fe","url":"https://huggingface.co/nvidia/NVIDIA-Nemotron-3.5-Lightning-30B-A3B-BF16/resolve/a9904d24bcc1d289a1950fa9d2b978c47cf903b9/model-00006-of-00014.safetensors"},{"path":"model-00007-of-00014.safetensors","bytes":4992601488,"sha256":"c7362630bf4256275ca58354ed8f31506cec67ffab30573e753703fd5ae48795","url":"https://huggingface.co/nvidia/NVIDIA-Nemotron-3.5-Lightning-30B-A3B-BF16/resolve/a9904d24bcc1d289a1950fa9d2b978c47cf903b9/model-00007-of-00014.safetensors"},{"path":"model-00008-of-00014.safetensors","bytes":4992601520,"sha256":"1b33a5a6b5505b25b57c90ddda19e9cd1598176468917df0c56116fa6f6072cb","url":"https://huggingface.co/nvidia/NVIDIA-Nemotron-3.5-Lightning-30B-A3B-BF16/resolve/a9904d24bcc1d289a1950fa9d2b978c47cf903b9/model-00008-of-00014.safetensors"},{"path":"model-00009-of-00014.safetensors","bytes":4995692800,"sha256":"fcc9b48caff427beaf1f87b78cf4ab42bc2a3c43b3c487bccb013ba22d7468b2","url":"https://huggingface.co/nvidia/NVIDIA-Nemotron-3.5-Lightning-30B-A3B-BF16/resolve/a9904d24bcc1d289a1950fa9d2b978c47cf903b9/model-00009-of-00014.safetensors"},{"path":"model-00010-of-00014.safetensors","bytes":4992601520,"sha256":"d1924fd54d1b77a15aff8df591a317e8a82a58e46d6f5742fa21e90e0be3cfa6","url":"https://huggingface.co/nvidia/NVIDIA-Nemotron-3.5-Lightning-30B-A3B-BF16/resolve/a9904d24bcc1d289a1950fa9d2b978c47cf903b9/model-00010-of-00014.safetensors"},{"path":"model-00011-of-00014.safetensors","bytes":4995692800,"sha256":"e9d0b83bcf04bd4d5758b94e6495a3f1595d121d82faffaa27ce6a884e2c8757","url":"https://huggingface.co/nvidia/NVIDIA-Nemotron-3.5-Lightning-30B-A3B-BF16/resolve/a9904d24bcc1d289a1950fa9d2b978c47cf903b9/model-00011-of-00014.safetensors"},{"path":"model-00012-of-00014.safetensors","bytes":4995692816,"sha256":"fce184f112cb0b97024cadc60f993f6f660e365d04e21c240fb3c75747bd3a01","url":"https://huggingface.co/nvidia/NVIDIA-Nemotron-3.5-Lightning-30B-A3B-BF16/resolve/a9904d24bcc1d289a1950fa9d2b978c47cf903b9/model-00012-of-00014.safetensors"},{"path":"model-00013-of-00014.safetensors","bytes":3249723504,"sha256":"f427d037d2a3e4d44323325aa38571e3551e8ad702f469c3dacc4e59ea2e50d1","url":"https://huggingface.co/nvidia/NVIDIA-Nemotron-3.5-Lightning-30B-A3B-BF16/resolve/a9904d24bcc1d289a1950fa9d2b978c47cf903b9/model-00013-of-00014.safetensors"},{"path":"model-00014-of-00014.safetensors","bytes":2670685240,"sha256":"64577b275ca4e7e5266eae0903674f7f46ec2a8cbf4f4f1a3207f80d503cd1d0","url":"https://huggingface.co/nvidia/NVIDIA-Nemotron-3.5-Lightning-30B-A3B-BF16/resolve/a9904d24bcc1d289a1950fa9d2b978c47cf903b9/model-00014-of-00014.safetensors"},{"path":"model.safetensors.index.json","bytes":612207,"sha256":"67f21da80ce245a3e24967f54eef3fa10e67a63eba16ef19a2d0569f06103f50","url":"https://huggingface.co/nvidia/NVIDIA-Nemotron-3.5-Lightning-30B-A3B-BF16/resolve/a9904d24bcc1d289a1950fa9d2b978c47cf903b9/model.safetensors.index.json"},{"path":"special_tokens_map.json","bytes":563,"sha256":"e9435fefd6d838fd9fcbbc44b97a8e3ff322be7f6dfb7e4fd2468586574bb52b","url":"https://huggingface.co/nvidia/NVIDIA-Nemotron-3.5-Lightning-30B-A3B-BF16/resolve/a9904d24bcc1d289a1950fa9d2b978c47cf903b9/special_tokens_map.json"},{"path":"tokenizer.json","bytes":17077484,"sha256":"623c34567aebb18582765289fbe23d901c62704d6518d71866e0e58db892b5b7","url":"https://huggingface.co/nvidia/NVIDIA-Nemotron-3.5-Lightning-30B-A3B-BF16/resolve/a9904d24bcc1d289a1950fa9d2b978c47cf903b9/tokenizer.json"},{"path":"tokenizer_config.json","bytes":177209,"sha256":"10f93eabcb9b1602fbb991d6308e787ce1df28ee9cd7a1c6d1e8c3f338b957bc","url":"https://huggingface.co/nvidia/NVIDIA-Nemotron-3.5-Lightning-30B-A3B-BF16/resolve/a9904d24bcc1d289a1950fa9d2b978c47cf903b9/tokenizer_config.json"}]} JSONASSETS sha256sum --check <<'SHA256' 5d7d233cad9eb1113534957de3315e1845bbc49de985b165a6230841194d7b47 assets.json SHA256Expected result: The complete 22-entry manifest verifies; omitting a part or changing any expected digest fails its binding.
3. Download and verify every native asset
After creating the exact checker in Configure, run this reader download. Existing mismatched files fail closed. Partial downloads remain separate and are never used as model files. No remote code or alternative quantization is substituted.
Commandset -eu cd "$HOME/nemotron35-lightning" python3 reader_assets.py downloadExpected result: All 22 exact sizes and streaming SHA256 checks pass before startup. Integrity establishes exact downloaded bytes, not numerical quality or GPU fit.
Configure
1. Write the native two-turn protocol client
The fixed client performs no tool execution or external lookup. It retains native reasoning/history and fails over-budget, partial raw-think/tool tokens, length termination and invalid strict JSON arguments.
Commandset -eu cd "$HOME/nemotron35-lightning" cat > verify_native.py <<'PYCLIENT' """Fixed inert tool/reasoning check for exact native Lightning; stdlib only.""" import http.client import json import pathlib MODEL = 'nemotron35-lightning-native' CONTEXT = 8192 OUTPUT = 4096 KWARGS = {'enable_thinking': True, 'force_nonempty_content': False, 'truncate_history_thinking': False} TOOLS = [{'type': 'function', 'function': {'name': 'lookup_demo_constant', 'description': 'Look up the inert demonstration constant; no external action.', 'parameters': {'type': 'object', 'properties': {'key': {'type': 'string', 'enum': ['alpha']}}, 'required': ['key'], 'additionalProperties': False}}}] def unique(pairs): out = {} for k, v in pairs: if k in out: raise ValueError('duplicate_json_key') out[k] = v return out def parse(raw): if type(raw) is not bytes or len(raw) > 2 * 1024 * 1024: raise ValueError('response_bytes') return json.loads(raw.decode('utf-8'), object_pairs_hook=unique, parse_constant=lambda _: (_ for _ in ()).throw(ValueError('nonfinite_json'))) def budget(d): if type(d) is not dict or type(d.get('count')) is not int or not 1 <= d['count'] <= CONTEXT - OUTPUT: raise ValueError('prompt_budget') if d.get('max_model_len') != CONTEXT: raise ValueError('wrong_server_context') return d['count'] def checked(d, prompt, tool_turn): if type(d) is not dict or d.get('model') != MODEL or type(d.get('choices')) is not list or len(d['choices']) != 1: raise ValueError('response_identity') choice = d['choices'][0] expected = 'tool_calls' if tool_turn else 'stop' if choice.get('finish_reason') != expected or choice.get('stop_reason') not in (None, 2, 11): raise ValueError('not_native_complete') ids = choice.get('token_ids') if type(ids) is not list or not 1 <= len(ids) < OUTPUT or any(type(i) is not int or i < 0 for i in ids): raise ValueError('raw_output_tokens') # Native added-token13 is </think>. Require it even when the parser supplies content. if 13 not in ids: raise ValueError('unclosed_native_thinking') usage = d.get('usage', {}) if usage.get('prompt_tokens') != prompt or usage.get('completion_tokens') != len(ids) or usage.get('total_tokens') != prompt + len(ids): raise ValueError('usage_mismatch') if prompt + len(ids) > CONTEXT: raise ValueError('total_budget') message = choice.get('message') if type(message) is not dict or message.get('role') != 'assistant' or type(message.get('reasoning')) is not str or not message['reasoning'].strip(): raise ValueError('native_reasoning_required') if tool_turn: calls = message.get('tool_calls') if type(calls) is not list or len(calls) != 1 or 14 not in ids or 15 not in ids: raise ValueError('complete_one_native_tool') call = calls[0] if call.get('type') != 'function' or type(call.get('id')) is not str or not 1 <= len(call['id']) <= 128: raise ValueError('tool_identity') fn = call.get('function', {}) if fn.get('name') != 'lookup_demo_constant' or type(fn.get('arguments')) is not str: raise ValueError('tool_name') args = parse(fn['arguments'].encode()) if type(args) is not dict or args != {'key': 'alpha'}: raise ValueError('fixed_tool_arguments') elif message.get('tool_calls') or type(message.get('content')) is not str or not message['content'].strip(): raise ValueError('final_answer_required') return message def call(path, body, key): payload = json.dumps(body, allow_nan=False).encode() if len(payload) > 65536: raise ValueError('fixed_client_body_bound') c = http.client.HTTPConnection('127.0.0.1', 8995, timeout=120) try: c.request('POST', path, payload, {'Authorization': 'Bearer ' + key, 'Content-Type': 'application/json'}) r = c.getresponse() if r.status != 200: raise RuntimeError('endpoint_status_' + str(r.status)) return parse(r.read(2 * 1024 * 1024 + 1)) finally: c.close() def run(request): messages = [{'role': 'user', 'content': 'Use lookup_demo_constant for key alpha, then explain in one sentence why its value plus one is eight. Treat this only as synthetic protocol testing.'}] outputs = [] for tool_turn in (True, False): shared = {'model': MODEL, 'messages': messages, 'tools': TOOLS, 'chat_template_kwargs': KWARGS, 'add_special_tokens': False, 'add_generation_prompt': True} count = budget(request('/tokenize', shared)) response = request('/v1/chat/completions', {**shared, 'tool_choice': 'auto' if tool_turn else 'none', 'temperature': 1.0, 'top_p': 0.95, 'max_tokens': OUTPUT, 'n': 1, 'stream': False, 'return_token_ids': True, 'stop_token_ids': [2, 11], 'include_reasoning': True}) msg = checked(response, count, tool_turn) outputs.append(response) if tool_turn: # Preserve complete native reasoning, final body and tool call history. No executor. messages.append({'role': 'assistant', 'content': msg.get('content') or '', 'reasoning_content': msg['reasoning'], 'tool_calls': msg['tool_calls']}) messages.append({'role': 'tool', 'tool_call_id': msg['tool_calls'][0]['id'], 'content': '{"key":"alpha","value":7,"synthetic":true}'}) return outputs if __name__ == '__main__': key = (pathlib.Path(__file__).resolve().parent / '.api-key').read_text().strip() if len(key) != 64 or any(c not in '0123456789abcdef' for c in key): raise SystemExit('private_key_format') result = run(lambda path, body: call(path, body, key)) pathlib.Path('protocol-result.json').write_text(json.dumps(result, indent=2, allow_nan=False) + '\n') print('Native raw closing-think/tool boundaries and token budgets passed. Inspect final answer and record semantic failures; this is not a quality or performance benchmark.') PYCLIENT python3 -m py_compile reader_assets.py verify_native.py sha256sum --check <<'SHA256' 3307e24fecd4092f1fdd8038fc28fb7ab4d42a247d34e44b48e35cfe3258ed85 verify_native.py SHA256Expected result: The complete stdlib client syntax and checksum pass; fake tests and syntax are not native compatibility or model-quality evidence.
2. Build the reader-owned immutable-base runtime
This custom layer is source-qualified and unbuilt by the author. The exact official amd64 image closes its existing packages; the exact SHA-pinned Transformers 5.8.1 wheel is installed without dependency resolution. pip check and exact version guards must pass. No floating fallback is allowed.
Commandset -eu cd "$HOME/nemotron35-lightning" cat > Dockerfile <<'DOCKERFILE' FROM vllm/vllm-openai@sha256:c2f3b1b964e47809b722b5e75b61b1e7b39a50f70388cf2bf2418f16a9f31da2 RUN python3 -m pip install --no-deps 'https://files.pythonhosted.org/packages/fc/b1/8be7e7ef0b5200491312201918b6125ef9c9df9dd0f0240ccef9ac824e6b/transformers-5.8.1-py3-none-any.whl#sha256=5340fb95962162cdfdae5cc91d7f8fedd92ed75216c1154c5e1f590fcf56dd0e' && python3 -m pip check RUN python3 -c "import vllm,transformers; assert vllm.__version__=='0.27.1'; assert transformers.__version__=='5.8.1'" ENTRYPOINT ["python3", "-m", "vllm.entrypoints.openai.api_server"] DOCKERFILE sudo docker build --pull -t nemotron35-lightning:reader . sudo docker run --rm --gpus device=0 --entrypoint nvidia-smi nemotron35-lightning:reader sudo docker run --rm --entrypoint python3 nemotron35-lightning:reader -m pip freeze > resolved-container-environment.txt sudo docker network inspect nemotron35-private >/dev/null 2>&1 || sudo docker network create --internal nemotron35-privateExpected result: Only after actual reader execution: pinned layer builds, dependency/version guards and GPU visibility pass. Save realized versions. Stop on CUDA, package, kernel or image errors; this is not a validated image or fit guarantee.
Start the model
1. Start complete native reasoning and embedded MTP
Run in a foreground terminal; all source-based flags are explicit. The 900-second process deadline is a demo bound, not an inference SLA. MTP uses the same full checkpoint with one speculative token. This small-context FP32 SSM/MTP variant differs from the publisher high-throughput FP16 SSM baseline; no performance transfer is claimed.
Commandset -eu cd "$HOME/nemotron35-lightning" python3 reader_assets.py verify timeout --signal=TERM --kill-after=10s 900s sudo docker run --rm --name nemotron35-lightning --gpus device=0 --network nemotron35-private -p 127.0.0.1:8995:8000 --shm-size 16g --env-file .api-env -v "$PWD/model:/model:ro" nemotron35-lightning:reader --model /model --served-model-name nemotron35-lightning-native --host 0.0.0.0 --port 8000 --dtype bfloat16 --kv-cache-dtype auto --mamba-ssm-cache-dtype float32 --mamba-cache-mode none --mamba-backend flashinfer --max-model-len 8192 --max-num-seqs 1 --max-num-batched-tokens 8192 --gpu-memory-utilization 0.95 --no-enable-prefix-caching --enforce-eager --speculative-config '{"method":"mtp","num_speculative_tokens":1}' --chat-template /model/chat_template.jinja --reasoning-parser nemotron_v3 --tool-call-parser qwen3_coder --enable-auto-tool-choiceExpected result: Only after reader initialization: exact native model and MTP load, no missing-parameter or CUDA errors occur, and private mapped service is ready. Failure or timeout is not a successful model run.
Connect to your endpoint
http://127.0.0.1:8995/v1/chat/completions
Same owned host loopback only; container is on an internal Docker network and port mapping binds 127.0.0.1.
Access: loopback. Private 64-character hexadecimal Bearer key protects native /v1 routes. Source exempts /health and /tokenize; those stay loopback-only. The standard native API is trusted single-user research, not a public/multiuser hardened service. max-num-seqs=1 limits scheduling, not admission or arbitrary queued clients. Only the fixed client is used here.
Verify the setup
1. Check route boundaries and real native completion
Run on the server from a second terminal. Native /health and /tokenize are not protected by vLLM Bearer middleware; keep them loopback-only. /v1/models without a key must return 401. The fixed client counts the full template before each generation, retains native reasoning/tool history and requires actual raw closing delimiters and native stop completion. Inspect final semantics; failure is recorded rather than changing the fixture or budget silently.
Commandset -eu cd "$HOME/nemotron35-lightning" curl --fail --max-time 5 http://127.0.0.1:8995/health test "$(curl -s --max-time 5 -o /dev/null -w '%{http_code}' http://127.0.0.1:8995/v1/models)" = 401 python3 verify_native.pyExpected result: After actual reader inference only: two native turns produce complete reasoning/tool/final protocol within their actual budgets. Save protocol-result.json and inspect the arithmetic sentence. Closed tokens are not a quality, latency or fit benchmark.
Performance evidence
No compatible measurement available. Documented support and estimated memory do not establish response speed.
Limits and troubleshooting
- Exact full publisher checkpoint, own native template, all MoE experts and embedded MTP are retained. The local artifact is not proven byte-equivalent to a managed provider API.
- Native BF16 config 262,144 and frozen catalog 1,048,576 are distinct; publisher documents single H100 256K and other 1M routes. This chosen 8,192-position walkthrough does not establish full advertised context fit.
- MTP is explicitly enabled with one speculative token; DSpark, DFlash and NVFP4 alternatives are separate open scopes. No speculation acceptance or throughput is measured.
- Attention KV follows BF16 auto; Mamba SSM state is explicitly FP32. Prefix reuse is disabled and eager execution selected. GPU workspace, draft/target buffers and retained state add unmeasured memory beyond 65.827 GB weights.
- The custom runtime layer is reader-built and unexecuted here, with exact immutable amd64 base and exact Transformers wheel; stop on pip check, native architecture, loader or kernel failure. Source compatibility is not execution proof.
- Client 120-second timeout does not cancel server work. Foreground 900-second lifetime can expire during loading or inference; stop and kill the exact container if needed. No latency SLA or quality guarantee.
- Native unguarded health/tokenize and all authorized routes remain host loopback only. No arbitrary tool execution, public deployment or multiuser resilience is qualified.
- The fixed two-turn client preserves full reasoning and tool history, requires raw closingthink/tool IDs and stop completion, and records actual usage. Token-bounded protocol success is not semantic accuracy.
- 256 GiB host RAM/256 GiB disk/16 threads/16 GiB shared memory are prescribed planning allocations, not hardware minima or proven fit. Advertised 80 GB GPU memory units are not silently converted into 80 GiB.
- All relevant costs are separated; no cloud VM, GPU, paid service, build, install, model download or inference was performed by the author. Family job totals and final scope remain open.
- Missing weights, MTP, architecture, package or CUDA failure
- Stop and preserve exact selected checkpoint, image and reader layer. Do not drop MTP/experts, switch to Nano/Super, change precision or bypass loader/pip errors. Record diagnostics without key values. A smaller or different variant requires its own configuration qualification.
- Length termination, missing closing-think/tool IDs or bad arguments
- Treat the run as incomplete. Preserve the raw result and actual token usage; do not force reasoning into content, strip history or execute returned tools. Larger budgets/resource allocations are a different workload requiring review.
- Out of memory or client deadline
- No fit or latency was measured. Stop the exact container, verify endpoint closure and record actual failure. Client timeout does not stop native computation; do not leave GPU or cloud billing running.
Stop the model
1. Stop the exact private container
Use another server terminal. Stop gracefully, then kill only this named experiment if it remains running. Confirm the endpoint no longer serves; process exit is not completed inference.
Commandsudo docker stop --time 10 nemotron35-lightning || true if [ "$(sudo docker inspect -f '{{.State.Running}}' nemotron35-lightning 2>/dev/null || true)" = true ]; then sudo docker kill nemotron35-lightning; fi curl --max-time 3 http://127.0.0.1:8995/v1/models || trueExpected result: The named native container and endpoint stop. A foreground timeout still requires this check; no completion or refund is inferred.
Uninstall
1. Remove dedicated reader assets and network
Preserve desired results, source/license records and logs first. Confirm the dedicated directory and image tag before deletion. Shared Docker/NVIDIA drivers and other images remain.
Commandsudo docker rm -f nemotron35-lightning 2>/dev/null || true sudo docker network rm nemotron35-private sudo docker image rm nemotron35-lightning:reader rm -r -- "$HOME/nemotron35-lightning"Expected result: Dedicated checkpoint, reader image tag, network, client results and private key files are removed. Shared image layers may remain until separately reviewed cleanup; no broad Docker prune is prescribed.
Sources and verification
Sources checked Oct 17, 2026. Verification source
- https://huggingface.co/api/models/nvidia/NVIDIA-Nemotron-3.5-Lightning-30B-A3B-BF16?blobs=truepublisher · retrieved Oct 17, 2026 · revision a9904d24bcc1d289a1950fa9d2b978c47cf903b9
- https://huggingface.co/nvidia/NVIDIA-Nemotron-3.5-Lightning-30B-A3B-BF16/raw/a9904d24bcc1d289a1950fa9d2b978c47cf903b9/README.mdpublisher · retrieved Oct 17, 2026 · revision a9904d24bcc1d289a1950fa9d2b978c47cf903b9
- https://huggingface.co/nvidia/NVIDIA-Nemotron-3.5-Lightning-30B-A3B-BF16/raw/a9904d24bcc1d289a1950fa9d2b978c47cf903b9/LICENSEpublisher · retrieved Oct 17, 2026 · revision a9904d24bcc1d289a1950fa9d2b978c47cf903b9
- https://huggingface.co/nvidia/NVIDIA-Nemotron-3.5-Lightning-30B-A3B-BF16/raw/a9904d24bcc1d289a1950fa9d2b978c47cf903b9/config.jsonpublisher · retrieved Oct 17, 2026 · revision a9904d24bcc1d289a1950fa9d2b978c47cf903b9
- https://huggingface.co/nvidia/NVIDIA-Nemotron-3.5-Lightning-30B-A3B-BF16/raw/a9904d24bcc1d289a1950fa9d2b978c47cf903b9/generation_config.jsonpublisher · retrieved Oct 17, 2026 · revision a9904d24bcc1d289a1950fa9d2b978c47cf903b9
- https://huggingface.co/nvidia/NVIDIA-Nemotron-3.5-Lightning-30B-A3B-BF16/raw/a9904d24bcc1d289a1950fa9d2b978c47cf903b9/tokenizer_config.jsonpublisher · retrieved Oct 17, 2026 · revision a9904d24bcc1d289a1950fa9d2b978c47cf903b9
- https://huggingface.co/nvidia/NVIDIA-Nemotron-3.5-Lightning-30B-A3B-BF16/raw/a9904d24bcc1d289a1950fa9d2b978c47cf903b9/chat_template.jinjapublisher · retrieved Oct 17, 2026 · revision a9904d24bcc1d289a1950fa9d2b978c47cf903b9
- https://huggingface.co/nvidia/NVIDIA-Nemotron-3.5-Lightning-30B-A3B-BF16/raw/a9904d24bcc1d289a1950fa9d2b978c47cf903b9/model.safetensors.index.jsonpublisher · retrieved Oct 17, 2026 · revision a9904d24bcc1d289a1950fa9d2b978c47cf903b9
- https://api.github.com/repos/vllm-project/vllm/git/ref/tags/v0.27.1runtime maintainer · retrieved Oct 17, 2026 · revision 6e448d0ea9bf3d88d898b65449ca6dc2aec170ac
- https://aws.amazon.com/ec2/instance-types/p5/cloud provider · retrieved Oct 17, 2026 · revision retrieved
- https://hub.docker.com/v2/repositories/vllm/vllm-openai/tags/v0.27.1runtime maintainer · retrieved Oct 17, 2026 · revision retrieved
- https://raw.githubusercontent.com/vllm-project/vllm/6e448d0ea9bf3d88d898b65449ca6dc2aec170ac/vllm/model_executor/models/nemotron_h.pyruntime maintainer · retrieved Oct 17, 2026 · revision 6e448d0ea9bf3d88d898b65449ca6dc2aec170ac
- https://raw.githubusercontent.com/vllm-project/vllm/6e448d0ea9bf3d88d898b65449ca6dc2aec170ac/vllm/model_executor/models/nemotron_h_mtp.pyruntime maintainer · retrieved Oct 17, 2026 · revision 6e448d0ea9bf3d88d898b65449ca6dc2aec170ac
- https://raw.githubusercontent.com/vllm-project/vllm/6e448d0ea9bf3d88d898b65449ca6dc2aec170ac/docker/Dockerfileruntime maintainer · retrieved Oct 17, 2026 · revision 6e448d0ea9bf3d88d898b65449ca6dc2aec170ac
- https://raw.githubusercontent.com/vllm-project/vllm/6e448d0ea9bf3d88d898b65449ca6dc2aec170ac/vllm/model_executor/models/registry.pyruntime maintainer · retrieved Oct 17, 2026 · revision 6e448d0ea9bf3d88d898b65449ca6dc2aec170ac
- https://api.github.com/repos/vllm-project/vllm/git/trees/6e448d0ea9bf3d88d898b65449ca6dc2aec170ac?recursive=1runtime maintainer · retrieved Oct 17, 2026 · revision 6e448d0ea9bf3d88d898b65449ca6dc2aec170ac
- https://docs.nvidia.com/cuda/cuda-toolkit-release-notes/index.htmlruntime maintainer · retrieved Oct 17, 2026 · revision retrieved
- https://raw.githubusercontent.com/vllm-project/vllm/6e448d0ea9bf3d88d898b65449ca6dc2aec170ac/vllm/transformers_utils/configs/nemotron_h.pyruntime maintainer · retrieved Oct 17, 2026 · revision 6e448d0ea9bf3d88d898b65449ca6dc2aec170ac
- https://raw.githubusercontent.com/vllm-project/vllm/6e448d0ea9bf3d88d898b65449ca6dc2aec170ac/vllm/transformers_utils/config.pyruntime maintainer · retrieved Oct 17, 2026 · revision 6e448d0ea9bf3d88d898b65449ca6dc2aec170ac
- https://raw.githubusercontent.com/vllm-project/vllm/6e448d0ea9bf3d88d898b65449ca6dc2aec170ac/vllm/config/speculative.pyruntime maintainer · retrieved Oct 17, 2026 · revision 6e448d0ea9bf3d88d898b65449ca6dc2aec170ac
- https://raw.githubusercontent.com/vllm-project/vllm/6e448d0ea9bf3d88d898b65449ca6dc2aec170ac/vllm/parser/nemotron_v3.pyruntime maintainer · retrieved Oct 17, 2026 · revision 6e448d0ea9bf3d88d898b65449ca6dc2aec170ac
- https://raw.githubusercontent.com/vllm-project/vllm/6e448d0ea9bf3d88d898b65449ca6dc2aec170ac/vllm/reasoning/nemotron_v3_engine_reasoning_parser.pyruntime maintainer · retrieved Oct 17, 2026 · revision 6e448d0ea9bf3d88d898b65449ca6dc2aec170ac
- https://raw.githubusercontent.com/vllm-project/vllm/6e448d0ea9bf3d88d898b65449ca6dc2aec170ac/requirements/common.txtruntime maintainer · retrieved Oct 17, 2026 · revision 6e448d0ea9bf3d88d898b65449ca6dc2aec170ac
- https://raw.githubusercontent.com/vllm-project/vllm/6e448d0ea9bf3d88d898b65449ca6dc2aec170ac/docker/versions.jsonruntime maintainer · retrieved Oct 17, 2026 · revision 6e448d0ea9bf3d88d898b65449ca6dc2aec170ac
- https://raw.githubusercontent.com/vllm-project/vllm/6e448d0ea9bf3d88d898b65449ca6dc2aec170ac/vllm/transformers_utils/configs/__init__.pyruntime maintainer · retrieved Oct 17, 2026 · revision 6e448d0ea9bf3d88d898b65449ca6dc2aec170ac
- https://api.github.com/repos/huggingface/transformers/git/ref/tags/v5.8.1runtime maintainer · retrieved Oct 17, 2026 · revision retrieved
- https://raw.githubusercontent.com/huggingface/transformers/v5.8.1/src/transformers/models/nemotron_h/configuration_nemotron_h.pyruntime maintainer · retrieved Oct 17, 2026 · revision retrieved
- https://b0.p.awsstatic.com/pricing/2.0/meteredUnitMaps/ec2/USD/current/ec2-ondemand-without-sec-sel/US%20East%20(N.%20Virginia)/Linux/index.jsoncloud provider · retrieved Oct 17, 2026 · revision retrieved
- https://pypi.org/pypi/transformers/5.8.1/jsonruntime maintainer · retrieved Oct 17, 2026 · revision retrieved
- https://aws.amazon.com/ebs/pricing/cloud provider · retrieved Oct 17, 2026 · revision retrieved
- https://docs.nvidia.com/datacenter/cloud-native/container-toolkit/latest/install-guide.htmlruntime maintainer · retrieved Oct 17, 2026 · revision retrieved
- https://raw.githubusercontent.com/vllm-project/vllm/6e448d0ea9bf3d88d898b65449ca6dc2aec170ac/vllm/entrypoints/openai/api_server.pyruntime maintainer · retrieved Oct 17, 2026 · revision 6e448d0ea9bf3d88d898b65449ca6dc2aec170ac
- https://raw.githubusercontent.com/vllm-project/vllm/6e448d0ea9bf3d88d898b65449ca6dc2aec170ac/vllm/entrypoints/openai/chat_completion/protocol.pyruntime maintainer · retrieved Oct 17, 2026 · revision 6e448d0ea9bf3d88d898b65449ca6dc2aec170ac
- https://raw.githubusercontent.com/vllm-project/vllm/6e448d0ea9bf3d88d898b65449ca6dc2aec170ac/vllm/entrypoints/serve/tokenize/protocol.pyruntime maintainer · retrieved Oct 17, 2026 · revision 6e448d0ea9bf3d88d898b65449ca6dc2aec170ac
- https://raw.githubusercontent.com/vllm-project/vllm/6e448d0ea9bf3d88d898b65449ca6dc2aec170ac/vllm/entrypoints/serve/tokenize/serving.pyruntime maintainer · retrieved Oct 17, 2026 · revision 6e448d0ea9bf3d88d898b65449ca6dc2aec170ac
- https://raw.githubusercontent.com/vllm-project/vllm/6e448d0ea9bf3d88d898b65449ca6dc2aec170ac/vllm/parser/qwen3.pyruntime maintainer · retrieved Oct 17, 2026 · revision 6e448d0ea9bf3d88d898b65449ca6dc2aec170ac
- https://raw.githubusercontent.com/vllm-project/vllm/6e448d0ea9bf3d88d898b65449ca6dc2aec170ac/vllm/tool_parsers/qwen3_engine_tool_parser.pyruntime maintainer · retrieved Oct 17, 2026 · revision 6e448d0ea9bf3d88d898b65449ca6dc2aec170ac
- https://docs.nvidia.com/cuda/archive/13.0.3/cuda-toolkit-release-notes/index.htmlruntime maintainer · retrieved Oct 17, 2026 · revision retrieved
- https://raw.githubusercontent.com/vllm-project/vllm/6e448d0ea9bf3d88d898b65449ca6dc2aec170ac/vllm/entrypoints/serve/utils/server_utils.pyruntime maintainer · retrieved Oct 17, 2026 · revision 6e448d0ea9bf3d88d898b65449ca6dc2aec170ac
- https://raw.githubusercontent.com/vllm-project/vllm/6e448d0ea9bf3d88d898b65449ca6dc2aec170ac/vllm/entrypoints/openai/chat_completion/serving.pyruntime maintainer · retrieved Oct 17, 2026 · revision 6e448d0ea9bf3d88d898b65449ca6dc2aec170ac
- https://huggingface.co/nvidia/NVIDIA-Nemotron-3.5-Lightning-30B-A3B-BF16/raw/a9904d24bcc1d289a1950fa9d2b978c47cf903b9/special_tokens_map.jsonpublisher · retrieved Oct 17, 2026 · revision retrieved
- https://raw.githubusercontent.com/vllm-project/vllm/6e448d0ea9bf3d88d898b65449ca6dc2aec170ac/vllm/config/cache.pyruntime maintainer · retrieved Oct 17, 2026 · revision 6e448d0ea9bf3d88d898b65449ca6dc2aec170ac
- https://raw.githubusercontent.com/vllm-project/vllm/6e448d0ea9bf3d88d898b65449ca6dc2aec170ac/vllm/model_executor/model_loader/default_loader.pyruntime maintainer · retrieved Oct 17, 2026 · revision 6e448d0ea9bf3d88d898b65449ca6dc2aec170ac
- https://opentools.ai/llms/nemotron-35-lightning-30b-a3bopentools · retrieved Oct 17, 2026 · revision reader-author-v1