Run Trinity Mini on a 64 GiB Intel desktop
Serve Arcee AI’s exact 15.94 GB Q4_K_M release of Trinity Mini with pinned llama.cpp and its native reasoning template. Use an owned Intel Core i5-12400 desktop. Allocate 64 GiB RAM, eight threads and 80 GiB disk for one CPU sequence, 8,192 total context tokens and a 4,096-token reasoning-plus-answer cap. Full memory fit and performance remain unknown.
- Hardware
- Intel Core i5-12400 desktop, 8 inference threads, 64 GiB RAM
- Runtime
- llama.cpp v0.7.0 (620d2225cc436ab0b001e3200e590e3b286ef9ab) · CPU
- Model format
- GGUF · GGUF Q4_K_M mixed quantization · Q4_K_M GGUF quantization published by Arcee AI
- Workload
- 8,192 token context · 1 concurrent
Based on source documentation; this setup has not been executed by OpenTools. Memory: unknown. No compatible measurement available.
Read the complete setup guide
Before you start
- Use an Intel Core i5-12400 desktop (6 cores / 12 threads, AVX2) with Ubuntu 24.04 x86_64, 64 GiB installed RAM and 80 GiB free disk. Allocate eight inference threads. Before a RAM upgrade, confirm motherboard, DIMM and OS compatibility; Intel’s 128 GB CPU maximum depends on memory type and does not validate a particular desktop. Build and run natively on this same CPU.
- 64 GiB RAM, 80 GiB free/root disk and eight threads are author allocations for the 15.94 GB full MoE artifact plus runtime and operating-system resources. They are not measured minima.
- Read the exact OpenMDW-1.1 agreement included by Arcee AI in both the native and GGUF repositories. Retain the license and applicable origin notices when distributing Model Materials. This recipe serves private text requests.
- Use the exact Trinity Mini release: AfmoeForCausalLM, 32 layers, hidden size 2,048, 32 attention heads and four KV heads with 128-dimensional heads. The config specifies two leading dense layers, 128 routed experts, eight selected experts per token and one shared expert, with a 2,048-token sliding window in three layers before each full-attention layer.
Exact model and requirements
- Artifact:
- Arcee AI / Trinity Mini Q4_K_M, publisher GGUF · checkpoint arcee-ai/Trinity-Mini-GGUF
- Publisher revision:
- a11737ef4b6f89d958d3f6ab670a9aa26e4b8ad7
- Artifact digest:
- sha256:7c2f0f2991923c21baee495340b79feb2a124b1c93b57dea0652f8aa27ba08e8
- Download size:
- 14.84 GiB
- Memory requirement (unknown):
- Unknown — No full-process memory peak is measured for this exact Trinity Mini Q4_K_M and CPU context configuration.
- System RAM / disk:
- Unknown / Unknown
- Placement:
- single device. One CPU host; no tensor/pipeline sharding. CPU-only execution; --device none and zero GPU layers. No disk or accelerator offload.
- KV cache:
- F16 attention K and V for interleaved sliding-window and full-attention layers; no full cache or process-memory estimate.
- Workload:
- 8,192 token context; Unknown; Unknown; concurrency 1; batch 1; cache unknown; repetitions Unknown.
- Tokenizer:
- Exact Arcee AI GGUF tokenizer with separately pinned native Trinity Mini Jinja template · a11737ef4b6f89d958d3f6ab670a9aa26e4b8ad7; template cdb81e8130815fec299cccc8f8700004b8366e1e
- Sampling / reasoning:
- Publisher recommended temperature 0.15, top_k 50, top_p 0.75 and min_p 0.06. repeat_penalty 1.0 and a combined reasoning-plus-final output cap of 4,096 are author choices; actual output length is unmeasured. Native Trinity Mini template appends <think> to the assistant generation prompt. It strips prior assistant reasoning before replay and supports tool formatting; this single-turn text recipe provides no tools or executor. Raw message.content uses reasoning-format none and skip-chat-parsing. Require a closing </think> followed by a nonempty final answer; no enable_thinking control is documented or passed.
- Runtime prerequisites:
- Ubuntu 24.04 x86_64; See prerequisites above.
Exact native and publisher GGUF repositories carry identical OpenMDW-1.1 license text. It grants permission free of charge to deal in Model Materials, subject to its terms. Distribution must retain the agreement and applicable copyright/origin notices; litigation termination and third-party-rights responsibilities apply. This recipe records open weights, without an OSI certification claim. Read the full pinned agreement before use or redistribution.
- trinity-mini-chat-template.jinja · 0 GiB · revision cdb81e8130815fec299cccc8f8700004b8366e1e
- The full 26B-parameter MoE artifact is 15,935,808,352 bytes. The publisher reports about 3B active parameters per token; active parameter count is not the resident model weight size or total memory requirement.
- Runtime weights, sliding/full-attention cache, routing buffers, OS and build artifacts add requirements. 64 GiB RAM and 80 GiB disk are planning allocations, not measured minimums or fit guarantees.
One-time setup and hardware
- hardware
- Unknown — Existing desktop purchase price and incremental hardware cost depend on the reader; unknown.
- Existing desktop purchase price and incremental hardware cost depend on the reader; unknown. Tax: unknown.
- setup
- Unknown — Initial provisioning/build time and labor cost are unknown.
- Initial provisioning/build time and labor cost are unknown. Tax: unknown.
Recurring costs
- electricity
- Unknown — Actual wall power, electricity rate and powered hours are unknown; no TDP-to-wall-power conversion.
- Actual wall power, electricity rate and powered hours are unknown; no TDP-to-wall-power conversion. Tax: unknown.
- storage
- Unknown — Incremental cost of local disk capacity is unknown. The 80 GiB free-disk walkthrough allocation is an author choice, not a measured requirement.
- Incremental cost of local disk capacity is unknown. The 80 GiB free-disk walkthrough allocation is an author choice, not a measured requirement. Tax: unknown.
- egress
- Unknown — Data transfer price, allowances and traffic volume are unknown.
- Data transfer price, allowances and traffic volume are unknown. Tax: unknown.
- administration
- Unknown — Maintenance labor rate and hours are unknown.
- Maintenance labor rate and hours are unknown. Tax: unknown.
Install the runtime
1. Build the pinned CPU server
Install build dependencies and compile the exact official source revision. These packages are system shared dependencies; no CUDA/GPU driver is required. GGML_NATIVE=ON selects -march=native for this GNU/Linux x86 build, compiling for the host CPU. Build and run on the same machine; no AVX-512 requirement is imposed on the i5-12400.
Commandsudo apt-get update sudo apt-get install -y build-essential cmake git curl ca-certificates openssl mkdir -p "$HOME/trinity-mini-hosting/models" cd "$HOME/trinity-mini-hosting" git clone https://github.com/ggml-org/llama.cpp.git cd llama.cpp git checkout --detach 620d2225cc436ab0b001e3200e590e3b286ef9ab cmake -S . -B build -DCMAKE_BUILD_TYPE=Release -DGGML_METAL=OFF -DGGML_NATIVE=ON cmake --build build --config Release -j 8 ./build/bin/llama-server --versionExpected result: Build succeeds and server version identifies pinned source. Stop if checkout or build fails.
Download the model
1. Download exact publisher GGUF and native Trinity template
Use Arcee AI’s own immutable Q4_K_M GGUF and separately pinned native template. Verify both SHA-256 values. The publisher documents llama.cpp support starting at b7061; this recipe selects the pinned 0.7.0 source with explicit AFMOE support. The exact GGUF has not been loaded here.
Commandcd "$HOME/trinity-mini-hosting" curl --fail --location --output models/Trinity-Mini-Q4_K_M.gguf 'https://huggingface.co/arcee-ai/Trinity-Mini-GGUF/resolve/a11737ef4b6f89d958d3f6ab670a9aa26e4b8ad7/Trinity-Mini-Q4_K_M.gguf' curl --fail --location --output models/trinity-mini-chat-template.jinja 'https://huggingface.co/arcee-ai/Trinity-Mini/raw/cdb81e8130815fec299cccc8f8700004b8366e1e/chat_template.jinja' sha256sum --check <<'SHA256' 7c2f0f2991923c21baee495340b79feb2a124b1c93b57dea0652f8aa27ba08e8 models/Trinity-Mini-Q4_K_M.gguf 97eba9fdb6e0a23122d1aefba507c835a65ad382441fc76c18d2b40981f7f1ae models/trinity-mini-chat-template.jinja SHA256Expected result: Both files report OK; model 15,935,808,352 bytes and template 2,860 bytes. Stop on mismatch.
Configure
1. Create a private service credential
Generate this only on your own host when following the guide. It is a local service password, not an OpenAI/provider credential. Keep it private and never paste it into tickets. Create it only when absent; preserve an existing key so a repeated setup does not silently rotate access. It is not created during this documentation run.
Commandcd "$HOME/trinity-mini-hosting" umask 077 if [ ! -e .api-key ]; then openssl rand -hex 32 > .api-key fi test -s .api-key || { echo 'Existing .api-key is empty; restore it before continuing.'; exit 1; } chmod 600 .api-keyExpected result: A non-empty private .api-key file exists; existing key preserved. No secret is printed.
Start the model
1. Start one CPU reasoning sequence
The pinned converter registers AfmoeForCausalLM as AFMOE. Native runtime recognizes the 32-layer 26B model and handles leading dense layers, routed and shared experts, route normalization/scaling, and interleaved sliding/full attention. The native Jinja parser accepts generation blocks. Use the pinned template verbatim. The 8,192-token total context and 4,096-token combined reasoning/final cap are bounded author choices below the advertised 131,072 maximum positions; adequacy and actual token count remain unmeasured.
Commandcd "$HOME/trinity-mini-hosting" export TRINITY_MINI_SERVICE_KEY="$(cat .api-key)" ./llama.cpp/build/bin/llama-server -m models/Trinity-Mini-Q4_K_M.gguf --alias trinity-mini --host 127.0.0.1 --port 8080 --api-key "$TRINITY_MINI_SERVICE_KEY" --device none -ngl 0 -t 8 -c 8192 -np 1 --cache-type-k f16 --cache-type-v f16 --jinja --chat-template-file models/trinity-mini-chat-template.jinja --reasoning-format none --skip-chat-parsing --temp 0.15 --top-p 0.75 --top-k 50 --min-p 0.06 --repeat-penalty 1.0Expected result: Model initializes and listener starts. Pass readiness and authenticated chat checks before relying on it. Command-line key may be visible to privileged/local users; trusted host required.
Connect to your endpoint
http://127.0.0.1:8080/v1
Same desktop loopback only; no LAN/public listener. Use on a trusted single-user host.
Access: loopback. Reader-generated private service key in Authorization: Bearer header for chat; /health is public on loopback. Run the client on this same desktop. Keep the key private; this is not an OpenAI API key.
Verify the setup
1. Check readiness and authenticated text generation
Use a second terminal on the server. Inspect raw choices[0].message.content for a closing </think>, followed by a nonempty final summary accurately reflecting the supplied passage. The template prefills the opening tag, which may not repeat in returned content. Require finish_reason != length. Missing closure, incomplete final body or added facts fails the check. This is functional verification, not a benchmark.
Commandcd "$HOME/trinity-mini-hosting" export TRINITY_MINI_SERVICE_KEY="$(cat .api-key)" curl --fail http://127.0.0.1:8080/health curl --fail http://127.0.0.1:8080/v1/chat/completions -H "Authorization: Bearer $TRINITY_MINI_SERVICE_KEY" -H 'Content-Type: application/json' -d '{"model":"trinity-mini","messages":[{"role":"user","content":"Summarize this supplied passage in one sentence without adding facts: The community library opens at 9am on weekdays and closes at 6pm. On Saturday it opens at 10am and closes at 4pm. It is closed on Sunday."}],"max_tokens":4096,"temperature":0.15,"top_k":50,"top_p":0.75,"min_p":0.06,"repeat_penalty":1.0}' curl -o /dev/null -s -w '%{http_code}\n' http://127.0.0.1:8080/v1/chat/completions -H 'Content-Type: application/json' -d '{"model":"trinity-mini","messages":[{"role":"user","content":"Hello"}]}'Expected result: Health HTTP 200; authenticated generation contains closed reasoning and an accurate nonempty final summary, finish_reason is not length; unauthenticated chat HTTP 401. Never accept a partial reasoning trace as the final answer.
Performance evidence
No compatible measurement available. Documented support and estimated memory do not establish response speed.
Limits and troubleshooting
- Documented instructions only. No weights, build, inference, peak memory, latency or throughput measurement was executed by OpenTools.
- Trinity Mini is a 26B total / about 3B active MoE according to the publisher. All selected artifact weights must be available; active parameters do not establish model fit or a smaller download.
- Native config specifies 131,072 maximum positions and a 2,048-token sliding window with every fourth layer full attention. This recipe uses only 8,192 combined template, prompt and generated tokens, with a 4,096-token output cap. Neither the full window nor actual prompt/output counts are validated.
- F16 K/V flags specify attention cache precision. Expert-routing buffers and full process memory remain unknown; no generic cache formula or sibling model performance is transferred.
- The publisher documents llama.cpp support from b7061, and the selected newer pinned source explicitly handles AFMOE. Exact GGUF startup compatibility is source-supported but unexecuted.
- The native 2,860-byte Jinja template is used verbatim, including <think> generation prefill and stripping historical assistant reasoning. The pinned runtime parser ignores Transformers generation/endgeneration blocks. No tool executor, thinking-disable switch or structured reasoning parser is configured.
- Temperature 0.15, top_k 50, top_p 0.75 and min_p 0.06 follow the publisher’s model details and generation config. repeat_penalty 1 and the 4,096-token output cap are author choices. The short bounded prompt does not establish adequate reasoning budget for arbitrary tasks.
- 64 GiB RAM, 80 GiB disk and eight threads are resource allocations, not minimum requirements. Original dated cloud compute/storage quotes exclude tax, egress and administration. Local hardware purchase and electricity remain unknown.
- OpenMDW-1.1 is the exact publisher license. Read its grants, notice retention, termination and rights-responsibility terms; this record does not assert OSI approval.
- Loopback and key authentication do not isolate privileged local users. Public or shared operation needs separate TLS/authentication/operations design. OS and build dependencies resolve at installation; retain actual versions and build logs.
- Download or checksum failure
- Verify exact uppercase filename and pinned URL; remove only the incomplete dedicated model file and retry. Do not run a mismatching file.
- Architecture unsupported or build fails
- Confirm exact pinned source commit, Ubuntu compiler/CMake dependencies and GGUF. Do not substitute an older packaged llama.cpp silently.
- Out of memory or context overflow
- Stop server, free/provision RAM or reduce context and restart; shorten input/output within configured total. Allocation is not a measured guarantee; record any changed configuration.
- Connection refused or HTTP 401
- Check foreground server logs, 8080 binding, matching private key and Bearer header. Run the client on this same desktop and keep the loopback listener.
- Template error, incomplete answer or architecture mismatch
- Check exact Trinity Mini GGUF/template hashes and pinned AFMOE runtime. Stop on architecture or Jinja errors. Require </think>, a useful factual final summary and finish_reason other than length. A truncated reasoning trace fails the check. Keep template, input and output within 8,192 tokens. Changes to artifact, context, parser or output cap require a newly validated configuration.
Stop the model
1. Stop the service
Press Ctrl+C in server terminal. Unset key in each shell. Confirm this desktop localhost:8080 no longer serves the model.
Commandunset TRINITY_MINI_SERVICE_KEYExpected result: Server stops and localhost:8080 stops serving.
Uninstall
1. Remove dedicated model, build and credential files
Back up wanted logs/files first. Remove only this walkthrough directory after confirming its path. Shared apt packages can remain; do not remove other workloads.
Commandrm -r -- "$HOME/trinity-mini-hosting"Expected result: Dedicated files and local service credential removed; shared system dependencies remain.
Sources and verification
Sources checked Oct 16, 2026. Verification source
- https://huggingface.co/api/models/arcee-ai/Trinity-Mini?blobs=truepublisher · retrieved Oct 16, 2026
- https://huggingface.co/api/models/arcee-ai/Trinity-Mini-GGUF?blobs=truepublisher · retrieved Oct 16, 2026
- https://huggingface.co/arcee-ai/Trinity-Mini/raw/cdb81e8130815fec299cccc8f8700004b8366e1e/README.mdpublisher · retrieved Oct 16, 2026 · revision cdb81e8130815fec299cccc8f8700004b8366e1e
- https://huggingface.co/arcee-ai/Trinity-Mini/raw/cdb81e8130815fec299cccc8f8700004b8366e1e/LICENSEpublisher · retrieved Oct 16, 2026 · revision cdb81e8130815fec299cccc8f8700004b8366e1e
- https://huggingface.co/arcee-ai/Trinity-Mini/raw/cdb81e8130815fec299cccc8f8700004b8366e1e/config.jsonpublisher · retrieved Oct 16, 2026 · revision cdb81e8130815fec299cccc8f8700004b8366e1e
- https://huggingface.co/arcee-ai/Trinity-Mini/raw/cdb81e8130815fec299cccc8f8700004b8366e1e/chat_template.jinjapublisher · retrieved Oct 16, 2026 · revision cdb81e8130815fec299cccc8f8700004b8366e1e
- https://huggingface.co/arcee-ai/Trinity-Mini/raw/cdb81e8130815fec299cccc8f8700004b8366e1e/generation_config.jsonpublisher · retrieved Oct 16, 2026 · revision cdb81e8130815fec299cccc8f8700004b8366e1e
- https://huggingface.co/arcee-ai/Trinity-Mini-GGUF/raw/a11737ef4b6f89d958d3f6ab670a9aa26e4b8ad7/README.mdpublisher · retrieved Oct 16, 2026 · revision a11737ef4b6f89d958d3f6ab670a9aa26e4b8ad7
- https://huggingface.co/arcee-ai/Trinity-Mini-GGUF/raw/a11737ef4b6f89d958d3f6ab670a9aa26e4b8ad7/LICENSEpublisher · retrieved Oct 16, 2026 · revision a11737ef4b6f89d958d3f6ab670a9aa26e4b8ad7
- https://raw.githubusercontent.com/ggml-org/llama.cpp/620d2225cc436ab0b001e3200e590e3b286ef9ab/conversion/afmoe.pyruntime maintainer · retrieved Oct 16, 2026 · revision 620d2225cc436ab0b001e3200e590e3b286ef9ab
- https://raw.githubusercontent.com/ggml-org/llama.cpp/620d2225cc436ab0b001e3200e590e3b286ef9ab/src/models/afmoe.cppruntime maintainer · retrieved Oct 16, 2026 · revision 620d2225cc436ab0b001e3200e590e3b286ef9ab
- https://raw.githubusercontent.com/ggml-org/llama.cpp/620d2225cc436ab0b001e3200e590e3b286ef9ab/common/jinja/parser.cppruntime maintainer · retrieved Oct 16, 2026 · revision 620d2225cc436ab0b001e3200e590e3b286ef9ab
- https://www.intel.com/content/www/us/en/products/sku/134586/intel-core-i512400-processor-18m-cache-up-to-4-40-ghz/specifications.htmlmanufacturer · retrieved Oct 16, 2026
- https://raw.githubusercontent.com/ggml-org/llama.cpp/620d2225cc436ab0b001e3200e590e3b286ef9ab/docs/build.mdruntime maintainer · retrieved Oct 16, 2026 · revision 620d2225cc436ab0b001e3200e590e3b286ef9ab
- https://raw.githubusercontent.com/ggml-org/llama.cpp/620d2225cc436ab0b001e3200e590e3b286ef9ab/ggml/src/ggml-cpu/CMakeLists.txtruntime maintainer · retrieved Oct 16, 2026 · revision 620d2225cc436ab0b001e3200e590e3b286ef9ab
- https://docs.liquid.ai/lfm/inference/llama-cppruntime maintainer · retrieved Oct 16, 2026
- https://raw.githubusercontent.com/ggml-org/llama.cpp/620d2225cc436ab0b001e3200e590e3b286ef9ab/tools/server/README.mdruntime maintainer · retrieved Oct 16, 2026 · revision 620d2225cc436ab0b001e3200e590e3b286ef9ab
- https://raw.githubusercontent.com/ggml-org/llama.cpp/620d2225cc436ab0b001e3200e590e3b286ef9ab/common/arg.cppruntime maintainer · retrieved Oct 16, 2026 · revision 620d2225cc436ab0b001e3200e590e3b286ef9ab
- https://api.github.com/repos/ggml-org/llama.cpp/releases/latestruntime maintainer · retrieved Oct 16, 2026