Trinity Large Preview

Exact model and requirements

Artifact:

Arcee AI (native checkpoint and publisher-owned GGUF) / Trinity-Large-Preview Q4_K_M, complete seven-file publisher composition · checkpoint arcee-ai/Trinity-Large-Preview-GGUF

Publisher revision:

4f7d8c1427170f603c9a0f34aed936aebeb90615

Artifact digest:

The seven separately hashed split files have no published whole-composition digest.

Download size:

225.04 GiB

Memory requirement (unknown):

Unknown — No full CPU peak measurement for this complete model, hybrid cache, buffers, loading and OS.

System RAM / disk:

Unknown / Unknown

Placement:

single device. All seven disk splits are one complete model on one CPU host; no tensor/pipeline sharding. CPU only on one host, no GPU layers or accelerator offload.

KV cache:

Explicit F16 attention K/V cache flags; actual interleaved sliding/global allocation and other working tensors are unmeasured.

Workload:

4,096 token context; Unknown; Unknown; concurrency 1; batch 1; cache unknown; repetitions Unknown.

Tokenizer:

Publisher Trinity-Large-Preview GGUF tokenizer with independently pinned exact native template · 4f7d8c1427170f603c9a0f34aed936aebeb90615; template d8e126719c8bf2b3ac5a3551bd1f07fcee767a90

Sampling / reasoning:

Native example temperature0.8, top-k50 and top-p0.8. Explicit min-p0, presence/frequency penalties0 and repeat penalty1 are neutral reader choices. No unrelated family preset is inherited. Trinity Large Preview is the lightly post-trained chat-ready native checkpoint, distinct from Trinity Large Thinking. The selected template has no enable_thinking toggle. No fabricated thinking disable flag or alternate checkpoint is used; tools and separate reasoning-optimized workflows remain open.

Runtime prerequisites:

Ubuntu 24.04 x86_64; See prerequisites above.

Official artifact · License

The selected native checkpoint and publisher GGUF both carry the actual OpenMDW License Agreement version 1.1. Subject to compliance it grants use of Model Materials free of charge under its included rights. Distribution must retain the agreement and applicable copyright/origin notices. Filing, maintaining or voluntarily participating in covered patent/copyright litigation terminates grants, except the specified responsive-lawsuit case. Outputs have no restrictions or obligations under this agreement. Materials are provided as is; clearing other rights and necessary consents remains the user responsibility. Exercising rights accepts its terms; the documentation author has not exercised model rights or accepted terms for the reader. Read the complete agreement. OpenRouter free routing is separate from independent selfhosting costs and rights.

• Trinity-Large-Preview-Q4_K_M-00001-of-00007.gguf · 35.86 GiB · revision 4f7d8c1427170f603c9a0f34aed936aebeb90615
• Trinity-Large-Preview-Q4_K_M-00002-of-00007.gguf · 36.48 GiB · revision 4f7d8c1427170f603c9a0f34aed936aebeb90615
• Trinity-Large-Preview-Q4_K_M-00003-of-00007.gguf · 36.48 GiB · revision 4f7d8c1427170f603c9a0f34aed936aebeb90615
• Trinity-Large-Preview-Q4_K_M-00004-of-00007.gguf · 36.48 GiB · revision 4f7d8c1427170f603c9a0f34aed936aebeb90615
• Trinity-Large-Preview-Q4_K_M-00005-of-00007.gguf · 36.48 GiB · revision 4f7d8c1427170f603c9a0f34aed936aebeb90615
• Trinity-Large-Preview-Q4_K_M-00006-of-00007.gguf · 36.24 GiB · revision 4f7d8c1427170f603c9a0f34aed936aebeb90615
• Trinity-Large-Preview-Q4_K_M-00007-of-00007.gguf · 7 GiB · revision 4f7d8c1427170f603c9a0f34aed936aebeb90615
• tokenizer_config.json (exact embedded native template source) · 0 GiB · revision d8e126719c8bf2b3ac5a3551bd1f07fcee767a90

• The complete seven model parts total 241,632,210,848 bytes. The native card describes 398B total and approximately 13B active parameters; all dense/routed/shared weights remain required.
• The selected serialized model is about 225 GiB before KV cache, runtime buffers, OS and build files. Remaining capacity does not establish actual startup or working-memory fit.
• Native AFMoE has 60 layers, six leading dense layers,256 routed experts/four selected and one shared expert, dual normalization, gated attention and three sliding layers per global layer attention. No active-parameter or dense-only cache formula is used.

Install the runtime

1. 1. Build the exact CPU runtime

   Build the pinned official runtime on this same CPU. Its AFMoE converter and graph handle dense-leading layers, routed/shared experts, MuP, dual normalization, attention gates and interleaved sliding/global attention. Support is source qualified, not executed fit.

   CommandCopy command

   set -e
   sudo apt-get update
   sudo apt-get install -y build-essential cmake git curl ca-certificates openssl python3
   mkdir -p "$HOME/trinity-large-preview-hosting/models"
   cd "$HOME/trinity-large-preview-hosting"
   git clone https://github.com/ggml-org/llama.cpp.git
   cd llama.cpp
   git checkout --detach 620d2225cc436ab0b001e3200e590e3b286ef9ab
   test "$(git rev-parse HEAD)" = 620d2225cc436ab0b001e3200e590e3b286ef9ab
   cmake -S . -B build -DCMAKE_BUILD_TYPE=Release -DGGML_METAL=OFF -DGGML_NATIVE=ON
   cmake --build build --config Release -j 16
   ./build/bin/llama-server --version

   Expected result: Checkout and build succeed. Stop on any failure; save version/build logs for actual execution.

Download the model

1. 1. Download every publisher shard and the exact native template

   Keep all seven exact split names together. Opening the first file discovers the split set. Verify every SHA before starting; the publisher GGUF and native template are separate immutable revisions. No conversion or partial expert subset is used. Extract the exact native embedded template from the full hash-verified tokenizer configuration. The standalone native Jinja is a separate file, not assumed identical.

   CommandCopy command

   set -e
   cd "$HOME/trinity-large-preview-hosting"
   curl --fail --location --output models/Trinity-Large-Preview-Q4_K_M-00001-of-00007.gguf 'https://huggingface.co/arcee-ai/Trinity-Large-Preview-GGUF/resolve/4f7d8c1427170f603c9a0f34aed936aebeb90615/Trinity-Large-Preview-Q4_K_M/Trinity-Large-Preview-Q4_K_M-00001-of-00007.gguf'
   curl --fail --location --output models/Trinity-Large-Preview-Q4_K_M-00002-of-00007.gguf 'https://huggingface.co/arcee-ai/Trinity-Large-Preview-GGUF/resolve/4f7d8c1427170f603c9a0f34aed936aebeb90615/Trinity-Large-Preview-Q4_K_M/Trinity-Large-Preview-Q4_K_M-00002-of-00007.gguf'
   curl --fail --location --output models/Trinity-Large-Preview-Q4_K_M-00003-of-00007.gguf 'https://huggingface.co/arcee-ai/Trinity-Large-Preview-GGUF/resolve/4f7d8c1427170f603c9a0f34aed936aebeb90615/Trinity-Large-Preview-Q4_K_M/Trinity-Large-Preview-Q4_K_M-00003-of-00007.gguf'
   curl --fail --location --output models/Trinity-Large-Preview-Q4_K_M-00004-of-00007.gguf 'https://huggingface.co/arcee-ai/Trinity-Large-Preview-GGUF/resolve/4f7d8c1427170f603c9a0f34aed936aebeb90615/Trinity-Large-Preview-Q4_K_M/Trinity-Large-Preview-Q4_K_M-00004-of-00007.gguf'
   curl --fail --location --output models/Trinity-Large-Preview-Q4_K_M-00005-of-00007.gguf 'https://huggingface.co/arcee-ai/Trinity-Large-Preview-GGUF/resolve/4f7d8c1427170f603c9a0f34aed936aebeb90615/Trinity-Large-Preview-Q4_K_M/Trinity-Large-Preview-Q4_K_M-00005-of-00007.gguf'
   curl --fail --location --output models/Trinity-Large-Preview-Q4_K_M-00006-of-00007.gguf 'https://huggingface.co/arcee-ai/Trinity-Large-Preview-GGUF/resolve/4f7d8c1427170f603c9a0f34aed936aebeb90615/Trinity-Large-Preview-Q4_K_M/Trinity-Large-Preview-Q4_K_M-00006-of-00007.gguf'
   curl --fail --location --output models/Trinity-Large-Preview-Q4_K_M-00007-of-00007.gguf 'https://huggingface.co/arcee-ai/Trinity-Large-Preview-GGUF/resolve/4f7d8c1427170f603c9a0f34aed936aebeb90615/Trinity-Large-Preview-Q4_K_M/Trinity-Large-Preview-Q4_K_M-00007-of-00007.gguf'
   curl --fail --location --output models/tokenizer_config.json 'https://huggingface.co/arcee-ai/Trinity-Large-Preview/resolve/d8e126719c8bf2b3ac5a3551bd1f07fcee767a90/tokenizer_config.json'
   sha256sum --check <<'SHA256'
   ced79cf97045db8717d4b89957325aa4252e10704d607373ce6c2667f8e387ef  models/Trinity-Large-Preview-Q4_K_M-00001-of-00007.gguf
   ada93ed215e3c6c942e488150d49e39b11d233b9c0739705135cb4ef61c22c57  models/Trinity-Large-Preview-Q4_K_M-00002-of-00007.gguf
   022400c38de5ac52b975a723fe971332ffdfd8c911c495a26f16683276be64b1  models/Trinity-Large-Preview-Q4_K_M-00003-of-00007.gguf
   67ba91191c53a1ce9dbc2a0548980f85a000f0157c1db93a1bc479275b827546  models/Trinity-Large-Preview-Q4_K_M-00004-of-00007.gguf
   0d7bd7177321b31ba5547f1ca6ae8cbc87b0672ea751b7f8fabc264dba3e4a45  models/Trinity-Large-Preview-Q4_K_M-00005-of-00007.gguf
   aa22b2e05fb06887078ee92e1239399832f16703c8d4d059a94264c0f020ba97  models/Trinity-Large-Preview-Q4_K_M-00006-of-00007.gguf
   139e9b1c86ed21e570a3ee45244db0b34cc06ca4327aa3740a08b3997ec076d1  models/Trinity-Large-Preview-Q4_K_M-00007-of-00007.gguf
   2f0dbb87667b9c94b3f205ea61fca0752fd02c0eefa20153faea6627aef2c67c  models/tokenizer_config.json
   SHA256
   python3 - <<'PYTEMPLATE'
   import pathlib,json,hashlib
   b=json.loads(pathlib.Path('models/tokenizer_config.json').read_text())['chat_template'].encode('utf-8')
   assert hashlib.sha256(b).hexdigest()=='5aaedc16d1ba0398644bf7caf1226aedebcaf17facf987f7c0336ca24795839d'
   pathlib.Path('models/native-chat-template.jinja').write_bytes(b)
   PYTEMPLATE

   Expected result: All eight files report OK. Complete seven-shard composition required; a first shard alone cannot run this model.

Configure

1. 1. Preserve a private service key

   Create only when absent; an existing empty key fails rather than silently rotating access. This is your local service credential, not a paid provider key.

   CommandCopy command

   set -e
   cd "$HOME/trinity-large-preview-hosting"
   umask 077
   if [ ! -e .api-key ]; then openssl rand -hex 32 > .api-key; fi
   test -s .api-key
   chmod 600 .api-key

   Expected result: Private nonempty key exists without printing it.

2. 2. Write the fixed bounded native diagnostic client

   The client counts the exact same full native-templated request before generation, reserves 512 positions and rejects length truncation, tools or inconsistent usage. It accepts only the fixed original supplied-evidence prompt; factual accuracy still needs human inspection.

   CommandCopy command

   set -e
   cd "$HOME/trinity-large-preview-hosting"
   cat > text-client.py <<'PYCLIENT'
   """Fixed original native text diagnostic, not a public proxy or tool runner."""
   import json,pathlib,urllib.request,urllib.error
   BASE='http://127.0.0.1:8080';CONTEXT=4096;OUTPUT=512
   BODY={'model':'trinity-large-preview','messages':[{'role':'system','content':'Answer only from the supplied passage; do not use tools.'},{'role':'user','content':'Summarize this passage in one sentence without adding facts: The community library opens at 9am on weekdays and closes at 6pm. On Saturday it opens at 10am and closes at 4pm. It is closed on Sunday.'}],'max_tokens':OUTPUT,'temperature':0.8,'top_k':50,'top_p':0.8,'min_p':0,'presence_penalty':0,'frequency_penalty':0,'repeat_penalty':1,'stream':False}
   def positive(v):return type(v) is int and v>0
   def checked(call,body):
    count=call('/v1/chat/completions/input_tokens',body).get('input_tokens')
    if not positive(count) or count+OUTPUT>CONTEXT:raise ValueError('Invalid full input count or context budget')
    result=call('/v1/chat/completions',body);choices=result.get('choices')
    if not isinstance(choices,list) or len(choices)!=1:raise ValueError('One complete response required')
    c=choices[0];message=c.get('message',{});s=message.get('content')
    if c.get('finish_reason')!='stop' or not isinstance(s,str) or not s.strip():raise ValueError('Empty, incomplete or exhausted-cap response')
    if message.get('tool_calls') or '<tool_call>' in s:raise ValueError('No tool result accepted')
    usage=result.get('usage',{});n=usage.get('completion_tokens')
    if usage.get('prompt_tokens')!=count or not positive(n) or n>OUTPUT or count+n>CONTEXT:raise ValueError('Usage mismatch or cap exceeded')
    return {'preflightInputTokens':count,'response':result}
   def http(key):
    def call(route,body=None):
     data=None if body is None else json.dumps(body).encode()
     req=urllib.request.Request(BASE+route,data=data,headers={'Authorization':'Bearer '+key,'Content-Type':'application/json'})
     with urllib.request.urlopen(req,timeout=1800) as r:
      if r.status!=200:raise ValueError('HTTP failure')
      raw=r.read(2000001)
     if len(raw)>2000000:raise ValueError('Oversized response')
     d=json.loads(raw)
     if not isinstance(d,dict):raise ValueError('Object response required')
     return d
    return call
   def main():
    key=pathlib.Path('.api-key').read_text().strip()
    if len(key)!=64 or any(c not in '0123456789abcdef' for c in key):raise ValueError('Restore private service key')
    call=http(key);call('/health')
    req=urllib.request.Request(BASE+'/v1/chat/completions',data=b'{}',headers={'Content-Type':'application/json'})
    try:
     with urllib.request.urlopen(req,timeout=30):raise ValueError('Unauthenticated request accepted')
    except urllib.error.HTTPError as e:
     if e.code!=401:raise
    result=checked(call,BODY);pathlib.Path('response.json').write_text(json.dumps(result,indent=2)+'\n');print(json.dumps(result,ensure_ascii=False))
    print('Manually require the supplied weekday/Saturday hours and Sunday closure; structural success does not prove factual correctness.')
   if __name__=='__main__':main()
   PYCLIENT

   Expected result: Client saved; no inference is performed by writing it.

Start the model

1. 1. Start the full Trinity CPU model

   Run one sequence with 4096 total positions and 16 CPU threads. Exact native BOS/chat boundaries come from the complete native template. Native EOS ends ordinary text; no invented thinking toggle or MTP draft is installed. Context shifting is disabled; the client checks input plus response reserve.

   CommandCopy command

   set -e
   cd "$HOME/trinity-large-preview-hosting"
   export TRINITY_SERVICE_KEY="$(cat .api-key)"
   ./llama.cpp/build/bin/llama-server -m models/Trinity-Large-Preview-Q4_K_M-00001-of-00007.gguf --alias trinity-large-preview --host 127.0.0.1 --port 8080 --api-key "$TRINITY_SERVICE_KEY" --device none -ngl 0 -t 16 -c 4096 -np 1 --no-context-shift --cache-type-k f16 --cache-type-v f16 --jinja --chat-template-file models/native-chat-template.jinja --reasoning-format none --skip-chat-parsing --temp 0.8 --top-k 50 --top-p 0.8 --min-p 0 --presence-penalty 0 --frequency-penalty 0 --repeat-penalty 1

   Expected result: All seven splits load before readiness. Stop on missing tensors, architecture/template, allocation or context errors. A ?B size display is not evidence of successful compatibility; health and real checks remain required.

Verify the setup

1. 1. Check readiness, authentication, budget and supplied evidence

   Use a second terminal inside the host. Read both the native preflight count and saved final response. Require the correct weekday/Saturday hours and Sunday closure from the supplied passage. A stop-finished response or protocol test alone does not prove accuracy.

   CommandCopy command

   set -e
   cd "$HOME/trinity-large-preview-hosting"
   python3 text-client.py

   Expected result: Health ready; unauthenticated 401; positive full input count plus 512 no greater than 4096; nonempty stop-finished text with matching actual usage. Manually reject invented hours or omitted closure.

Performance evidence

No compatible measurement available. Documented support and estimated memory do not establish response speed.

Limits and troubleshooting

• Source-qualified configuration only; no weights/model imports, build, installation, startup, inference, fit, speed or quality was executed.
• All seven publisher Q4_K_M files total 241,632,210,848 bytes. Approximate 13B active computation does not shrink 398B total stored parameters into a 13B memory allowance.
• The native card advertises 512k context, while the pinned config stores 262144 maximum positions; neither is an executed long-context result. Configured sliding window is 4096 with every fourth layer global. This guide allocates 4096 total input/output positions for one sequence, reserving 512 output. Full input/output counts remain unknown until actual reader execution.
• Pinned AFMoE model size labeling lists smaller known sizes and labels 60 layers unknown/?B. Actual tensor creation and graph iteration use loaded dimensions and leading-dense/expert configuration, not that display label. This source argument is not an executed compatibility result; stop on actual loading errors.
• The publisher-owned GGUF explicitly names the native Preview base; its exact quantization input base commit is not established. Native template and GGUF pins are independent, not a weight byte-equivalence claim.
• The native OpenMDW 1.1 grant covers selected artifacts; third-party rights/consents remain reader responsibility. No terms acceptance or legal clearance is performed by the author.
• The OpenRouter free alias is independently documented identity/routing context only, with no assumed endpoint availability. Your owned CPU electricity and cloud rental/storage costs remain separate.
• Preview is a chat-ready lightly post-trained checkpoint. Thinking, tools, other quantizations, native BF16/FP8/GPU, concurrency and long context remain separate open scopes. Required family totals stay null.
• A private loopback service is not a public TLS or multi-user isolation design. Retain logs and stop on unexpected output, decoder/template errors or resource failures.
• The exact selected template is the native tokenizer configuration embedded field. Its separately published standalone Jinja adds generation-mask tags and a trailing newline; these files are not byte-identical. The guide uses the complete embedded native template with a checked derivation, not an edited or sibling template.

Shard/checksum or architecture failure

Stop. Check all seven exact publisher files and the native template hashes and pinned runtime. Do not substitute Mini, Base, Thinking or a partial expert subset.

Out of memory or slow response

Stop the foreground server and inspect actual OS-visible RAM/logs. Capacity beyond serialized bytes is not measured working fit. A changed allocation or context is a different configuration.

Input budget, response cap or usage fails

Do not bypass the native count or silently truncate. Preserve errors; review a changed workload/configuration separately. Wrong factual output fails even when protocol fields pass.

Connection refused or401

Check successful initialization, exact local key/Bearer header and intended workstation loopback host. Keep the public inference port closed.

Stop the model

1. 1. Stop the foreground server

   Press Ctrl+C in the server terminal, then unset the key in each shell. Verify localhost no longer serves.

   CommandCopy command

   unset TRINITY_SERVICE_KEY
   if curl --silent --max-time 2 http://127.0.0.1:8080/health >/dev/null; then echo 'Still serving; stop the foreground process.'; exit 1; fi

   Expected result: No service remains on loopback 8080.

Uninstall

1. 1. Remove the dedicated walkthrough files

   After stopping, back up wanted logs, confirm the dedicated directory and remove only it. Shared apt packages can remain.

   CommandCopy command

   rm -r -- "$HOME/trinity-large-preview-hosting"

   Expected result: Dedicated model, build, responses and key removed; shared packages remain.

Related guides

• https://opentools.ai/llms/trinity-large-preview

Exact model and requirements

Artifact:

Arcee AI (native checkpoint and publisher-owned GGUF) / Trinity-Large-Preview Q4_K_M, complete seven-file publisher composition · checkpoint arcee-ai/Trinity-Large-Preview-GGUF

Publisher revision:

4f7d8c1427170f603c9a0f34aed936aebeb90615

Artifact digest:

The seven separately hashed split files have no published whole-composition digest.

Download size:

225.04 GiB

Memory requirement (unknown):

Unknown — No full CPU peak measurement for this complete model, hybrid cache, buffers, loading and OS.

System RAM / disk:

Unknown / Unknown

Placement:

single device. All seven disk splits are one complete model on one CPU host; no tensor/pipeline sharding. CPU only on one host, no GPU layers or accelerator offload.

KV cache:

Explicit F16 attention K/V cache flags; actual interleaved sliding/global allocation and other working tensors are unmeasured.

Workload:

4,096 token context; Unknown; Unknown; concurrency 1; batch 1; cache unknown; repetitions Unknown.

Tokenizer:

Publisher Trinity-Large-Preview GGUF tokenizer with independently pinned exact native template · 4f7d8c1427170f603c9a0f34aed936aebeb90615; template d8e126719c8bf2b3ac5a3551bd1f07fcee767a90

Sampling / reasoning:

Native example temperature0.8, top-k50 and top-p0.8. Explicit min-p0, presence/frequency penalties0 and repeat penalty1 are neutral reader choices. No unrelated family preset is inherited. Trinity Large Preview is the lightly post-trained chat-ready native checkpoint, distinct from Trinity Large Thinking. The selected template has no enable_thinking toggle. No fabricated thinking disable flag or alternate checkpoint is used; tools and separate reasoning-optimized workflows remain open.

Runtime prerequisites:

Ubuntu 24.04 x86_64; See prerequisites above.

Official artifact · License

The selected native checkpoint and publisher GGUF both carry the actual OpenMDW License Agreement version 1.1. Subject to compliance it grants use of Model Materials free of charge under its included rights. Distribution must retain the agreement and applicable copyright/origin notices. Filing, maintaining or voluntarily participating in covered patent/copyright litigation terminates grants, except the specified responsive-lawsuit case. Outputs have no restrictions or obligations under this agreement. Materials are provided as is; clearing other rights and necessary consents remains the user responsibility. Exercising rights accepts its terms; the documentation author has not exercised model rights or accepted terms for the reader. Read the complete agreement. OpenRouter free routing is separate from independent selfhosting costs and rights.

• Trinity-Large-Preview-Q4_K_M-00001-of-00007.gguf · 35.86 GiB · revision 4f7d8c1427170f603c9a0f34aed936aebeb90615
• Trinity-Large-Preview-Q4_K_M-00002-of-00007.gguf · 36.48 GiB · revision 4f7d8c1427170f603c9a0f34aed936aebeb90615
• Trinity-Large-Preview-Q4_K_M-00003-of-00007.gguf · 36.48 GiB · revision 4f7d8c1427170f603c9a0f34aed936aebeb90615
• Trinity-Large-Preview-Q4_K_M-00004-of-00007.gguf · 36.48 GiB · revision 4f7d8c1427170f603c9a0f34aed936aebeb90615
• Trinity-Large-Preview-Q4_K_M-00005-of-00007.gguf · 36.48 GiB · revision 4f7d8c1427170f603c9a0f34aed936aebeb90615
• Trinity-Large-Preview-Q4_K_M-00006-of-00007.gguf · 36.24 GiB · revision 4f7d8c1427170f603c9a0f34aed936aebeb90615
• Trinity-Large-Preview-Q4_K_M-00007-of-00007.gguf · 7 GiB · revision 4f7d8c1427170f603c9a0f34aed936aebeb90615
• tokenizer_config.json (exact embedded native template source) · 0 GiB · revision d8e126719c8bf2b3ac5a3551bd1f07fcee767a90

• The complete seven model parts total 241,632,210,848 bytes. The native card describes 398B total and approximately 13B active parameters; all dense/routed/shared weights remain required.
• The selected serialized model is about 225 GiB before KV cache, runtime buffers, OS and build files. Remaining capacity does not establish actual startup or working-memory fit.
• Native AFMoE has 60 layers, six leading dense layers,256 routed experts/four selected and one shared expert, dual normalization, gated attention and three sliding layers per global layer attention. No active-parameter or dense-only cache formula is used.

Install the runtime

1. 1. Build the exact CPU runtime

   Build the pinned official runtime on this same CPU. Its AFMoE converter and graph handle dense-leading layers, routed/shared experts, MuP, dual normalization, attention gates and interleaved sliding/global attention. Support is source qualified, not executed fit.

   CommandCopy command

   set -e
   sudo apt-get update
   sudo apt-get install -y build-essential cmake git curl ca-certificates openssl python3
   mkdir -p "$HOME/trinity-large-preview-hosting/models"
   cd "$HOME/trinity-large-preview-hosting"
   git clone https://github.com/ggml-org/llama.cpp.git
   cd llama.cpp
   git checkout --detach 620d2225cc436ab0b001e3200e590e3b286ef9ab
   test "$(git rev-parse HEAD)" = 620d2225cc436ab0b001e3200e590e3b286ef9ab
   cmake -S . -B build -DCMAKE_BUILD_TYPE=Release -DGGML_METAL=OFF -DGGML_NATIVE=ON
   cmake --build build --config Release -j 16
   ./build/bin/llama-server --version

   Expected result: Checkout and build succeed. Stop on any failure; save version/build logs for actual execution.

Download the model

1. 1. Download every publisher shard and the exact native template

   Keep all seven exact split names together. Opening the first file discovers the split set. Verify every SHA before starting; the publisher GGUF and native template are separate immutable revisions. No conversion or partial expert subset is used. Extract the exact native embedded template from the full hash-verified tokenizer configuration. The standalone native Jinja is a separate file, not assumed identical.

   CommandCopy command

   set -e
   cd "$HOME/trinity-large-preview-hosting"
   curl --fail --location --output models/Trinity-Large-Preview-Q4_K_M-00001-of-00007.gguf 'https://huggingface.co/arcee-ai/Trinity-Large-Preview-GGUF/resolve/4f7d8c1427170f603c9a0f34aed936aebeb90615/Trinity-Large-Preview-Q4_K_M/Trinity-Large-Preview-Q4_K_M-00001-of-00007.gguf'
   curl --fail --location --output models/Trinity-Large-Preview-Q4_K_M-00002-of-00007.gguf 'https://huggingface.co/arcee-ai/Trinity-Large-Preview-GGUF/resolve/4f7d8c1427170f603c9a0f34aed936aebeb90615/Trinity-Large-Preview-Q4_K_M/Trinity-Large-Preview-Q4_K_M-00002-of-00007.gguf'
   curl --fail --location --output models/Trinity-Large-Preview-Q4_K_M-00003-of-00007.gguf 'https://huggingface.co/arcee-ai/Trinity-Large-Preview-GGUF/resolve/4f7d8c1427170f603c9a0f34aed936aebeb90615/Trinity-Large-Preview-Q4_K_M/Trinity-Large-Preview-Q4_K_M-00003-of-00007.gguf'
   curl --fail --location --output models/Trinity-Large-Preview-Q4_K_M-00004-of-00007.gguf 'https://huggingface.co/arcee-ai/Trinity-Large-Preview-GGUF/resolve/4f7d8c1427170f603c9a0f34aed936aebeb90615/Trinity-Large-Preview-Q4_K_M/Trinity-Large-Preview-Q4_K_M-00004-of-00007.gguf'
   curl --fail --location --output models/Trinity-Large-Preview-Q4_K_M-00005-of-00007.gguf 'https://huggingface.co/arcee-ai/Trinity-Large-Preview-GGUF/resolve/4f7d8c1427170f603c9a0f34aed936aebeb90615/Trinity-Large-Preview-Q4_K_M/Trinity-Large-Preview-Q4_K_M-00005-of-00007.gguf'
   curl --fail --location --output models/Trinity-Large-Preview-Q4_K_M-00006-of-00007.gguf 'https://huggingface.co/arcee-ai/Trinity-Large-Preview-GGUF/resolve/4f7d8c1427170f603c9a0f34aed936aebeb90615/Trinity-Large-Preview-Q4_K_M/Trinity-Large-Preview-Q4_K_M-00006-of-00007.gguf'
   curl --fail --location --output models/Trinity-Large-Preview-Q4_K_M-00007-of-00007.gguf 'https://huggingface.co/arcee-ai/Trinity-Large-Preview-GGUF/resolve/4f7d8c1427170f603c9a0f34aed936aebeb90615/Trinity-Large-Preview-Q4_K_M/Trinity-Large-Preview-Q4_K_M-00007-of-00007.gguf'
   curl --fail --location --output models/tokenizer_config.json 'https://huggingface.co/arcee-ai/Trinity-Large-Preview/resolve/d8e126719c8bf2b3ac5a3551bd1f07fcee767a90/tokenizer_config.json'
   sha256sum --check <<'SHA256'
   ced79cf97045db8717d4b89957325aa4252e10704d607373ce6c2667f8e387ef  models/Trinity-Large-Preview-Q4_K_M-00001-of-00007.gguf
   ada93ed215e3c6c942e488150d49e39b11d233b9c0739705135cb4ef61c22c57  models/Trinity-Large-Preview-Q4_K_M-00002-of-00007.gguf
   022400c38de5ac52b975a723fe971332ffdfd8c911c495a26f16683276be64b1  models/Trinity-Large-Preview-Q4_K_M-00003-of-00007.gguf
   67ba91191c53a1ce9dbc2a0548980f85a000f0157c1db93a1bc479275b827546  models/Trinity-Large-Preview-Q4_K_M-00004-of-00007.gguf
   0d7bd7177321b31ba5547f1ca6ae8cbc87b0672ea751b7f8fabc264dba3e4a45  models/Trinity-Large-Preview-Q4_K_M-00005-of-00007.gguf
   aa22b2e05fb06887078ee92e1239399832f16703c8d4d059a94264c0f020ba97  models/Trinity-Large-Preview-Q4_K_M-00006-of-00007.gguf
   139e9b1c86ed21e570a3ee45244db0b34cc06ca4327aa3740a08b3997ec076d1  models/Trinity-Large-Preview-Q4_K_M-00007-of-00007.gguf
   2f0dbb87667b9c94b3f205ea61fca0752fd02c0eefa20153faea6627aef2c67c  models/tokenizer_config.json
   SHA256
   python3 - <<'PYTEMPLATE'
   import pathlib,json,hashlib
   b=json.loads(pathlib.Path('models/tokenizer_config.json').read_text())['chat_template'].encode('utf-8')
   assert hashlib.sha256(b).hexdigest()=='5aaedc16d1ba0398644bf7caf1226aedebcaf17facf987f7c0336ca24795839d'
   pathlib.Path('models/native-chat-template.jinja').write_bytes(b)
   PYTEMPLATE

   Expected result: All eight files report OK. Complete seven-shard composition required; a first shard alone cannot run this model.

Configure

1. 1. Preserve a private service key

   Create only when absent; an existing empty key fails rather than silently rotating access. This is your local service credential, not a paid provider key.

   CommandCopy command

   set -e
   cd "$HOME/trinity-large-preview-hosting"
   umask 077
   if [ ! -e .api-key ]; then openssl rand -hex 32 > .api-key; fi
   test -s .api-key
   chmod 600 .api-key

   Expected result: Private nonempty key exists without printing it.

2. 2. Write the fixed bounded native diagnostic client

   The client counts the exact same full native-templated request before generation, reserves 512 positions and rejects length truncation, tools or inconsistent usage. It accepts only the fixed original supplied-evidence prompt; factual accuracy still needs human inspection.

   CommandCopy command

   set -e
   cd "$HOME/trinity-large-preview-hosting"
   cat > text-client.py <<'PYCLIENT'
   """Fixed original native text diagnostic, not a public proxy or tool runner."""
   import json,pathlib,urllib.request,urllib.error
   BASE='http://127.0.0.1:8080';CONTEXT=4096;OUTPUT=512
   BODY={'model':'trinity-large-preview','messages':[{'role':'system','content':'Answer only from the supplied passage; do not use tools.'},{'role':'user','content':'Summarize this passage in one sentence without adding facts: The community library opens at 9am on weekdays and closes at 6pm. On Saturday it opens at 10am and closes at 4pm. It is closed on Sunday.'}],'max_tokens':OUTPUT,'temperature':0.8,'top_k':50,'top_p':0.8,'min_p':0,'presence_penalty':0,'frequency_penalty':0,'repeat_penalty':1,'stream':False}
   def positive(v):return type(v) is int and v>0
   def checked(call,body):
    count=call('/v1/chat/completions/input_tokens',body).get('input_tokens')
    if not positive(count) or count+OUTPUT>CONTEXT:raise ValueError('Invalid full input count or context budget')
    result=call('/v1/chat/completions',body);choices=result.get('choices')
    if not isinstance(choices,list) or len(choices)!=1:raise ValueError('One complete response required')
    c=choices[0];message=c.get('message',{});s=message.get('content')
    if c.get('finish_reason')!='stop' or not isinstance(s,str) or not s.strip():raise ValueError('Empty, incomplete or exhausted-cap response')
    if message.get('tool_calls') or '<tool_call>' in s:raise ValueError('No tool result accepted')
    usage=result.get('usage',{});n=usage.get('completion_tokens')
    if usage.get('prompt_tokens')!=count or not positive(n) or n>OUTPUT or count+n>CONTEXT:raise ValueError('Usage mismatch or cap exceeded')
    return {'preflightInputTokens':count,'response':result}
   def http(key):
    def call(route,body=None):
     data=None if body is None else json.dumps(body).encode()
     req=urllib.request.Request(BASE+route,data=data,headers={'Authorization':'Bearer '+key,'Content-Type':'application/json'})
     with urllib.request.urlopen(req,timeout=1800) as r:
      if r.status!=200:raise ValueError('HTTP failure')
      raw=r.read(2000001)
     if len(raw)>2000000:raise ValueError('Oversized response')
     d=json.loads(raw)
     if not isinstance(d,dict):raise ValueError('Object response required')
     return d
    return call
   def main():
    key=pathlib.Path('.api-key').read_text().strip()
    if len(key)!=64 or any(c not in '0123456789abcdef' for c in key):raise ValueError('Restore private service key')
    call=http(key);call('/health')
    req=urllib.request.Request(BASE+'/v1/chat/completions',data=b'{}',headers={'Content-Type':'application/json'})
    try:
     with urllib.request.urlopen(req,timeout=30):raise ValueError('Unauthenticated request accepted')
    except urllib.error.HTTPError as e:
     if e.code!=401:raise
    result=checked(call,BODY);pathlib.Path('response.json').write_text(json.dumps(result,indent=2)+'\n');print(json.dumps(result,ensure_ascii=False))
    print('Manually require the supplied weekday/Saturday hours and Sunday closure; structural success does not prove factual correctness.')
   if __name__=='__main__':main()
   PYCLIENT

   Expected result: Client saved; no inference is performed by writing it.

Start the model

1. 1. Start the full Trinity CPU model

   Run one sequence with 4096 total positions and 16 CPU threads. Exact native BOS/chat boundaries come from the complete native template. Native EOS ends ordinary text; no invented thinking toggle or MTP draft is installed. Context shifting is disabled; the client checks input plus response reserve.

   CommandCopy command

   set -e
   cd "$HOME/trinity-large-preview-hosting"
   export TRINITY_SERVICE_KEY="$(cat .api-key)"
   ./llama.cpp/build/bin/llama-server -m models/Trinity-Large-Preview-Q4_K_M-00001-of-00007.gguf --alias trinity-large-preview --host 127.0.0.1 --port 8080 --api-key "$TRINITY_SERVICE_KEY" --device none -ngl 0 -t 16 -c 4096 -np 1 --no-context-shift --cache-type-k f16 --cache-type-v f16 --jinja --chat-template-file models/native-chat-template.jinja --reasoning-format none --skip-chat-parsing --temp 0.8 --top-k 50 --top-p 0.8 --min-p 0 --presence-penalty 0 --frequency-penalty 0 --repeat-penalty 1

   Expected result: All seven splits load before readiness. Stop on missing tensors, architecture/template, allocation or context errors. A ?B size display is not evidence of successful compatibility; health and real checks remain required.

Verify the setup

1. 1. Check readiness, authentication, budget and supplied evidence

   Use a second terminal inside the host. Read both the native preflight count and saved final response. Require the correct weekday/Saturday hours and Sunday closure from the supplied passage. A stop-finished response or protocol test alone does not prove accuracy.

   CommandCopy command

   set -e
   cd "$HOME/trinity-large-preview-hosting"
   python3 text-client.py

   Expected result: Health ready; unauthenticated 401; positive full input count plus 512 no greater than 4096; nonempty stop-finished text with matching actual usage. Manually reject invented hours or omitted closure.

Performance evidence

No compatible measurement available. Documented support and estimated memory do not establish response speed.

Limits and troubleshooting

• Source-qualified configuration only; no weights/model imports, build, installation, startup, inference, fit, speed or quality was executed.
• All seven publisher Q4_K_M files total 241,632,210,848 bytes. Approximate 13B active computation does not shrink 398B total stored parameters into a 13B memory allowance.
• The native card advertises 512k context, while the pinned config stores 262144 maximum positions; neither is an executed long-context result. Configured sliding window is 4096 with every fourth layer global. This guide allocates 4096 total input/output positions for one sequence, reserving 512 output. Full input/output counts remain unknown until actual reader execution.
• Pinned AFMoE model size labeling lists smaller known sizes and labels 60 layers unknown/?B. Actual tensor creation and graph iteration use loaded dimensions and leading-dense/expert configuration, not that display label. This source argument is not an executed compatibility result; stop on actual loading errors.
• The publisher-owned GGUF explicitly names the native Preview base; its exact quantization input base commit is not established. Native template and GGUF pins are independent, not a weight byte-equivalence claim.
• The native OpenMDW 1.1 grant covers selected artifacts; third-party rights/consents remain reader responsibility. No terms acceptance or legal clearance is performed by the author.
• The OpenRouter free alias is independently documented identity/routing context only, with no assumed endpoint availability. Your owned CPU electricity and cloud rental/storage costs remain separate.
• Preview is a chat-ready lightly post-trained checkpoint. Thinking, tools, other quantizations, native BF16/FP8/GPU, concurrency and long context remain separate open scopes. Required family totals stay null.
• A private loopback service is not a public TLS or multi-user isolation design. Retain logs and stop on unexpected output, decoder/template errors or resource failures.
• The exact selected template is the native tokenizer configuration embedded field. Its separately published standalone Jinja adds generation-mask tags and a trailing newline; these files are not byte-identical. The guide uses the complete embedded native template with a checked derivation, not an edited or sibling template.

Shard/checksum or architecture failure

Stop. Check all seven exact publisher files and the native template hashes and pinned runtime. Do not substitute Mini, Base, Thinking or a partial expert subset.

Out of memory or slow response

Stop the foreground server and inspect actual OS-visible RAM/logs. Capacity beyond serialized bytes is not measured working fit. A changed allocation or context is a different configuration.

Input budget, response cap or usage fails

Do not bypass the native count or silently truncate. Preserve errors; review a changed workload/configuration separately. Wrong factual output fails even when protocol fields pass.

Connection refused or401

Check successful initialization, exact local key/Bearer header and intended loopback/SSH host. Keep the public inference port closed.

Stop the model

1. 1. Stop the foreground server

   Press Ctrl+C in the server terminal, then unset the key in each shell. Verify localhost no longer serves. Stopping the process does not stop EC2 billing.

   CommandCopy command

   unset TRINITY_SERVICE_KEY
   if curl --silent --max-time 2 http://127.0.0.1:8080/health >/dev/null; then echo 'Still serving; stop the foreground process.'; exit 1; fi

   Expected result: No service remains on loopback 8080.

Uninstall

1. 1. Remove the dedicated walkthrough files

   After stopping, back up wanted logs, confirm the dedicated directory and remove only it. Shared apt packages can remain.

   CommandCopy command

   rm -r -- "$HOME/trinity-large-preview-hosting"

   Expected result: Dedicated model, build, responses and key removed; shared packages remain.

Shut down cloud resources

1. 1. End cloud billing resources

   In EC2 us-east-1, stop for later reuse or terminate this exact VM. Review Delete on termination for its root volume. Delete unwanted detached EBS volumes, snapshots and reserved/Elastic/public IP resources. Stopped VM storage and retained resources may still bill. Check Bills/Cost Explorer later for final usage.

   Expected result: Exact instance is stopped/terminated as intended; no unwanted attached/retained billed resources remain. Instance compute stops only after its lifecycle transition, not when llama-server stops.

Related guides

• https://opentools.ai/llms/trinity-large-preview