Arcee AI logo

Trinity Mini

TrinityCurrent
byArcee AIArcee AI(startup)
Released December 1, 2025
Context131.1K tokens
Price (In / Out)$0.04/M / $0.15/M
CategoryLarge Language Model
Max Output131.1K tokens

About Trinity Mini

Trinity Mini is a large language model from Arcee AI. Supports up to 131,072 token context window. Available from $0.04/M input tokens.

Capabilities

textcode

Input Modalities

text

Output Modalities

text

Technical Details

API Identifier
arcee-ai/trinity-mini
Category
Large Language Model
Context Window
131,072 tokens
Max Output Tokens
131,072 tokens

Tags

budget-friendlylong-context

Run this model yourself

Local and cloud setup for Trinity Mini, with exact configurations, sources and clear evidence limits.

Find a model for your hardware
Local hardwareDocumented setupSources checked Oct 16, 2026

Run Trinity Mini on a 64 GiB Intel desktop

Serve Arcee AI’s exact 15.94 GB Q4_K_M release of Trinity Mini with pinned llama.cpp and its native reasoning template. Use an owned Intel Core i5-12400 desktop. Allocate 64 GiB RAM, eight threads and 80 GiB disk for one CPU sequence, 8,192 total context tokens and a 4,096-token reasoning-plus-answer cap. Full memory fit and performance remain unknown.

Hardware
Intel Core i5-12400 desktop, 8 inference threads, 64 GiB RAM
Runtime
llama.cpp v0.7.0 (620d2225cc436ab0b001e3200e590e3b286ef9ab) · CPU
Model format
GGUF · GGUF Q4_K_M mixed quantization · Q4_K_M GGUF quantization published by Arcee AI
Workload
8,192 token context · 1 concurrent

Based on source documentation; this setup has not been executed by OpenTools. Memory: unknown. No compatible measurement available.

Read the complete setup guide

Before you start

  • Use an Intel Core i5-12400 desktop (6 cores / 12 threads, AVX2) with Ubuntu 24.04 x86_64, 64 GiB installed RAM and 80 GiB free disk. Allocate eight inference threads. Before a RAM upgrade, confirm motherboard, DIMM and OS compatibility; Intel’s 128 GB CPU maximum depends on memory type and does not validate a particular desktop. Build and run natively on this same CPU.
  • 64 GiB RAM, 80 GiB free/root disk and eight threads are author allocations for the 15.94 GB full MoE artifact plus runtime and operating-system resources. They are not measured minima.
  • Read the exact OpenMDW-1.1 agreement included by Arcee AI in both the native and GGUF repositories. Retain the license and applicable origin notices when distributing Model Materials. This recipe serves private text requests.
  • Use the exact Trinity Mini release: AfmoeForCausalLM, 32 layers, hidden size 2,048, 32 attention heads and four KV heads with 128-dimensional heads. The config specifies two leading dense layers, 128 routed experts, eight selected experts per token and one shared expert, with a 2,048-token sliding window in three layers before each full-attention layer.

Exact model and requirements

Artifact:
Arcee AI / Trinity Mini Q4_K_M, publisher GGUF · checkpoint arcee-ai/Trinity-Mini-GGUF
Publisher revision:
a11737ef4b6f89d958d3f6ab670a9aa26e4b8ad7
Artifact digest:
sha256:7c2f0f2991923c21baee495340b79feb2a124b1c93b57dea0652f8aa27ba08e8
Download size:
14.84 GiB
Memory requirement (unknown):
Unknown — No full-process memory peak is measured for this exact Trinity Mini Q4_K_M and CPU context configuration.
System RAM / disk:
Unknown / Unknown
Placement:
single device. One CPU host; no tensor/pipeline sharding. CPU-only execution; --device none and zero GPU layers. No disk or accelerator offload.
KV cache:
F16 attention K and V for interleaved sliding-window and full-attention layers; no full cache or process-memory estimate.
Workload:
8,192 token context; Unknown; Unknown; concurrency 1; batch 1; cache unknown; repetitions Unknown.
Tokenizer:
Exact Arcee AI GGUF tokenizer with separately pinned native Trinity Mini Jinja template · a11737ef4b6f89d958d3f6ab670a9aa26e4b8ad7; template cdb81e8130815fec299cccc8f8700004b8366e1e
Sampling / reasoning:
Publisher recommended temperature 0.15, top_k 50, top_p 0.75 and min_p 0.06. repeat_penalty 1.0 and a combined reasoning-plus-final output cap of 4,096 are author choices; actual output length is unmeasured. Native Trinity Mini template appends <think> to the assistant generation prompt. It strips prior assistant reasoning before replay and supports tool formatting; this single-turn text recipe provides no tools or executor. Raw message.content uses reasoning-format none and skip-chat-parsing. Require a closing </think> followed by a nonempty final answer; no enable_thinking control is documented or passed.
Runtime prerequisites:
Ubuntu 24.04 x86_64; See prerequisites above.

Official artifact · License

Exact native and publisher GGUF repositories carry identical OpenMDW-1.1 license text. It grants permission free of charge to deal in Model Materials, subject to its terms. Distribution must retain the agreement and applicable copyright/origin notices; litigation termination and third-party-rights responsibilities apply. This recipe records open weights, without an OSI certification claim. Read the full pinned agreement before use or redistribution.

  • The full 26B-parameter MoE artifact is 15,935,808,352 bytes. The publisher reports about 3B active parameters per token; active parameter count is not the resident model weight size or total memory requirement.
  • Runtime weights, sliding/full-attention cache, routing buffers, OS and build artifacts add requirements. 64 GiB RAM and 80 GiB disk are planning allocations, not measured minimums or fit guarantees.

One-time setup and hardware

hardware
Unknown — Existing desktop purchase price and incremental hardware cost depend on the reader; unknown.
Existing desktop purchase price and incremental hardware cost depend on the reader; unknown. Tax: unknown.
setup
Unknown — Initial provisioning/build time and labor cost are unknown.
Initial provisioning/build time and labor cost are unknown. Tax: unknown.

Recurring costs

electricity
Unknown — Actual wall power, electricity rate and powered hours are unknown; no TDP-to-wall-power conversion.
Actual wall power, electricity rate and powered hours are unknown; no TDP-to-wall-power conversion. Tax: unknown.
storage
Unknown — Incremental cost of local disk capacity is unknown. The 80 GiB free-disk walkthrough allocation is an author choice, not a measured requirement.
Incremental cost of local disk capacity is unknown. The 80 GiB free-disk walkthrough allocation is an author choice, not a measured requirement. Tax: unknown.
egress
Unknown — Data transfer price, allowances and traffic volume are unknown.
Data transfer price, allowances and traffic volume are unknown. Tax: unknown.
administration
Unknown — Maintenance labor rate and hours are unknown.
Maintenance labor rate and hours are unknown. Tax: unknown.

Install the runtime

  1. 1. Build the pinned CPU server

    Install build dependencies and compile the exact official source revision. These packages are system shared dependencies; no CUDA/GPU driver is required. GGML_NATIVE=ON selects -march=native for this GNU/Linux x86 build, compiling for the host CPU. Build and run on the same machine; no AVX-512 requirement is imposed on the i5-12400.

    Command
    sudo apt-get update
    sudo apt-get install -y build-essential cmake git curl ca-certificates openssl
    mkdir -p "$HOME/trinity-mini-hosting/models"
    cd "$HOME/trinity-mini-hosting"
    git clone https://github.com/ggml-org/llama.cpp.git
    cd llama.cpp
    git checkout --detach 620d2225cc436ab0b001e3200e590e3b286ef9ab
    cmake -S . -B build -DCMAKE_BUILD_TYPE=Release -DGGML_METAL=OFF -DGGML_NATIVE=ON
    cmake --build build --config Release -j 8
    ./build/bin/llama-server --version

    Expected result: Build succeeds and server version identifies pinned source. Stop if checkout or build fails.

Download the model

  1. 1. Download exact publisher GGUF and native Trinity template

    Use Arcee AI’s own immutable Q4_K_M GGUF and separately pinned native template. Verify both SHA-256 values. The publisher documents llama.cpp support starting at b7061; this recipe selects the pinned 0.7.0 source with explicit AFMOE support. The exact GGUF has not been loaded here.

    Command
    cd "$HOME/trinity-mini-hosting"
    curl --fail --location --output models/Trinity-Mini-Q4_K_M.gguf 'https://huggingface.co/arcee-ai/Trinity-Mini-GGUF/resolve/a11737ef4b6f89d958d3f6ab670a9aa26e4b8ad7/Trinity-Mini-Q4_K_M.gguf'
    curl --fail --location --output models/trinity-mini-chat-template.jinja 'https://huggingface.co/arcee-ai/Trinity-Mini/raw/cdb81e8130815fec299cccc8f8700004b8366e1e/chat_template.jinja'
    sha256sum --check <<'SHA256'
    7c2f0f2991923c21baee495340b79feb2a124b1c93b57dea0652f8aa27ba08e8  models/Trinity-Mini-Q4_K_M.gguf
    97eba9fdb6e0a23122d1aefba507c835a65ad382441fc76c18d2b40981f7f1ae  models/trinity-mini-chat-template.jinja
    SHA256

    Expected result: Both files report OK; model 15,935,808,352 bytes and template 2,860 bytes. Stop on mismatch.

Configure

  1. 1. Create a private service credential

    Generate this only on your own host when following the guide. It is a local service password, not an OpenAI/provider credential. Keep it private and never paste it into tickets. Create it only when absent; preserve an existing key so a repeated setup does not silently rotate access. It is not created during this documentation run.

    Command
    cd "$HOME/trinity-mini-hosting"
    umask 077
    if [ ! -e .api-key ]; then
      openssl rand -hex 32 > .api-key
    fi
    test -s .api-key || { echo 'Existing .api-key is empty; restore it before continuing.'; exit 1; }
    chmod 600 .api-key

    Expected result: A non-empty private .api-key file exists; existing key preserved. No secret is printed.

Start the model

  1. 1. Start one CPU reasoning sequence

    The pinned converter registers AfmoeForCausalLM as AFMOE. Native runtime recognizes the 32-layer 26B model and handles leading dense layers, routed and shared experts, route normalization/scaling, and interleaved sliding/full attention. The native Jinja parser accepts generation blocks. Use the pinned template verbatim. The 8,192-token total context and 4,096-token combined reasoning/final cap are bounded author choices below the advertised 131,072 maximum positions; adequacy and actual token count remain unmeasured.

    Command
    cd "$HOME/trinity-mini-hosting"
    export TRINITY_MINI_SERVICE_KEY="$(cat .api-key)"
    ./llama.cpp/build/bin/llama-server -m models/Trinity-Mini-Q4_K_M.gguf --alias trinity-mini --host 127.0.0.1 --port 8080 --api-key "$TRINITY_MINI_SERVICE_KEY" --device none -ngl 0 -t 8 -c 8192 -np 1 --cache-type-k f16 --cache-type-v f16 --jinja --chat-template-file models/trinity-mini-chat-template.jinja --reasoning-format none --skip-chat-parsing --temp 0.15 --top-p 0.75 --top-k 50 --min-p 0.06 --repeat-penalty 1.0

    Expected result: Model initializes and listener starts. Pass readiness and authenticated chat checks before relying on it. Command-line key may be visible to privileged/local users; trusted host required.

Connect to your endpoint

http://127.0.0.1:8080/v1

Same desktop loopback only; no LAN/public listener. Use on a trusted single-user host.

Access: loopback. Reader-generated private service key in Authorization: Bearer header for chat; /health is public on loopback. Run the client on this same desktop. Keep the key private; this is not an OpenAI API key.

Verify the setup

  1. 1. Check readiness and authenticated text generation

    Use a second terminal on the server. Inspect raw choices[0].message.content for a closing </think>, followed by a nonempty final summary accurately reflecting the supplied passage. The template prefills the opening tag, which may not repeat in returned content. Require finish_reason != length. Missing closure, incomplete final body or added facts fails the check. This is functional verification, not a benchmark.

    Command
    cd "$HOME/trinity-mini-hosting"
    export TRINITY_MINI_SERVICE_KEY="$(cat .api-key)"
    curl --fail http://127.0.0.1:8080/health
    curl --fail http://127.0.0.1:8080/v1/chat/completions -H "Authorization: Bearer $TRINITY_MINI_SERVICE_KEY" -H 'Content-Type: application/json' -d '{"model":"trinity-mini","messages":[{"role":"user","content":"Summarize this supplied passage in one sentence without adding facts: The community library opens at 9am on weekdays and closes at 6pm. On Saturday it opens at 10am and closes at 4pm. It is closed on Sunday."}],"max_tokens":4096,"temperature":0.15,"top_k":50,"top_p":0.75,"min_p":0.06,"repeat_penalty":1.0}'
    curl -o /dev/null -s -w '%{http_code}\n' http://127.0.0.1:8080/v1/chat/completions -H 'Content-Type: application/json' -d '{"model":"trinity-mini","messages":[{"role":"user","content":"Hello"}]}'

    Expected result: Health HTTP 200; authenticated generation contains closed reasoning and an accurate nonempty final summary, finish_reason is not length; unauthenticated chat HTTP 401. Never accept a partial reasoning trace as the final answer.

Performance evidence

No compatible measurement available. Documented support and estimated memory do not establish response speed.

Limits and troubleshooting

  • Documented instructions only. No weights, build, inference, peak memory, latency or throughput measurement was executed by OpenTools.
  • Trinity Mini is a 26B total / about 3B active MoE according to the publisher. All selected artifact weights must be available; active parameters do not establish model fit or a smaller download.
  • Native config specifies 131,072 maximum positions and a 2,048-token sliding window with every fourth layer full attention. This recipe uses only 8,192 combined template, prompt and generated tokens, with a 4,096-token output cap. Neither the full window nor actual prompt/output counts are validated.
  • F16 K/V flags specify attention cache precision. Expert-routing buffers and full process memory remain unknown; no generic cache formula or sibling model performance is transferred.
  • The publisher documents llama.cpp support from b7061, and the selected newer pinned source explicitly handles AFMOE. Exact GGUF startup compatibility is source-supported but unexecuted.
  • The native 2,860-byte Jinja template is used verbatim, including <think> generation prefill and stripping historical assistant reasoning. The pinned runtime parser ignores Transformers generation/endgeneration blocks. No tool executor, thinking-disable switch or structured reasoning parser is configured.
  • Temperature 0.15, top_k 50, top_p 0.75 and min_p 0.06 follow the publisher’s model details and generation config. repeat_penalty 1 and the 4,096-token output cap are author choices. The short bounded prompt does not establish adequate reasoning budget for arbitrary tasks.
  • 64 GiB RAM, 80 GiB disk and eight threads are resource allocations, not minimum requirements. Original dated cloud compute/storage quotes exclude tax, egress and administration. Local hardware purchase and electricity remain unknown.
  • OpenMDW-1.1 is the exact publisher license. Read its grants, notice retention, termination and rights-responsibility terms; this record does not assert OSI approval.
  • Loopback and key authentication do not isolate privileged local users. Public or shared operation needs separate TLS/authentication/operations design. OS and build dependencies resolve at installation; retain actual versions and build logs.
Download or checksum failure
Verify exact uppercase filename and pinned URL; remove only the incomplete dedicated model file and retry. Do not run a mismatching file.
Architecture unsupported or build fails
Confirm exact pinned source commit, Ubuntu compiler/CMake dependencies and GGUF. Do not substitute an older packaged llama.cpp silently.
Out of memory or context overflow
Stop server, free/provision RAM or reduce context and restart; shorten input/output within configured total. Allocation is not a measured guarantee; record any changed configuration.
Connection refused or HTTP 401
Check foreground server logs, 8080 binding, matching private key and Bearer header. Run the client on this same desktop and keep the loopback listener.
Template error, incomplete answer or architecture mismatch
Check exact Trinity Mini GGUF/template hashes and pinned AFMOE runtime. Stop on architecture or Jinja errors. Require </think>, a useful factual final summary and finish_reason other than length. A truncated reasoning trace fails the check. Keep template, input and output within 8,192 tokens. Changes to artifact, context, parser or output cap require a newly validated configuration.

Stop the model

  1. 1. Stop the service

    Press Ctrl+C in server terminal. Unset key in each shell. Confirm this desktop localhost:8080 no longer serves the model.

    Command
    unset TRINITY_MINI_SERVICE_KEY

    Expected result: Server stops and localhost:8080 stops serving.

Uninstall

  1. 1. Remove dedicated model, build and credential files

    Back up wanted logs/files first. Remove only this walkthrough directory after confirming its path. Shared apt packages can remain; do not remove other workloads.

    Command
    rm -r -- "$HOME/trinity-mini-hosting"

    Expected result: Dedicated files and local service credential removed; shared system dependencies remain.

Sources and verification

Sources checked Oct 16, 2026. Verification source

Compare this configuration
Cloud serverDocumented setupSources checked Oct 16, 2026

Self-host Trinity Mini on an AWS CPU VM

Serve Arcee AI’s exact 15.94 GB Q4_K_M release of Trinity Mini with pinned llama.cpp and its native reasoning template. Use AWS r7i.2xlarge in us-east-1. Allocate 64 GiB RAM, eight threads and 80 GiB disk for one CPU sequence, 8,192 total context tokens and a 4,096-token reasoning-plus-answer cap. Full memory fit and performance remain unknown.

Hardware
AWS r7i.2xlarge, 64 GiB RAM, Ubuntu x86_64
Runtime
llama.cpp v0.7.0 (620d2225cc436ab0b001e3200e590e3b286ef9ab) · CPU
Model format
GGUF · GGUF Q4_K_M mixed quantization · Q4_K_M GGUF quantization published by Arcee AI
Workload
8,192 token context · 1 concurrent

Based on source documentation; this setup has not been executed by OpenTools. Memory: unknown. No compatible measurement available.

Read the complete setup guide

Before you start

  • Use Ubuntu 24.04 x86_64 on AWS r7i.2xlarge (8 vCPUs, 64 GiB RAM) in us-east-1 with an 80 GiB gp3 root disk. Check current quote, quota and capacity before provisioning; creating resources incurs charges. Use an existing SSH key, allow inbound SSH only from your client IP, and keep port 8080 closed.
  • 64 GiB RAM, 80 GiB free/root disk and eight threads are author allocations for the 15.94 GB full MoE artifact plus runtime and operating-system resources. They are not measured minima.
  • Read the exact OpenMDW-1.1 agreement included by Arcee AI in both the native and GGUF repositories. Retain the license and applicable origin notices when distributing Model Materials. This recipe serves private text requests.
  • Use the exact Trinity Mini release: AfmoeForCausalLM, 32 layers, hidden size 2,048, 32 attention heads and four KV heads with 128-dimensional heads. The config specifies two leading dense layers, 128 routed experts, eight selected experts per token and one shared expert, with a 2,048-token sliding window in three layers before each full-attention layer.

Exact model and requirements

Artifact:
Arcee AI / Trinity Mini Q4_K_M, publisher GGUF · checkpoint arcee-ai/Trinity-Mini-GGUF
Publisher revision:
a11737ef4b6f89d958d3f6ab670a9aa26e4b8ad7
Artifact digest:
sha256:7c2f0f2991923c21baee495340b79feb2a124b1c93b57dea0652f8aa27ba08e8
Download size:
14.84 GiB
Memory requirement (unknown):
Unknown — No full-process memory peak is measured for this exact Trinity Mini Q4_K_M and CPU context configuration.
System RAM / disk:
Unknown / Unknown
Placement:
single device. One CPU host; no tensor/pipeline sharding. CPU-only execution; --device none and zero GPU layers. No disk or accelerator offload.
KV cache:
F16 attention K and V for interleaved sliding-window and full-attention layers; no full cache or process-memory estimate.
Workload:
8,192 token context; Unknown; Unknown; concurrency 1; batch 1; cache unknown; repetitions Unknown.
Tokenizer:
Exact Arcee AI GGUF tokenizer with separately pinned native Trinity Mini Jinja template · a11737ef4b6f89d958d3f6ab670a9aa26e4b8ad7; template cdb81e8130815fec299cccc8f8700004b8366e1e
Sampling / reasoning:
Publisher recommended temperature 0.15, top_k 50, top_p 0.75 and min_p 0.06. repeat_penalty 1.0 and a combined reasoning-plus-final output cap of 4,096 are author choices; actual output length is unmeasured. Native Trinity Mini template appends <think> to the assistant generation prompt. It strips prior assistant reasoning before replay and supports tool formatting; this single-turn text recipe provides no tools or executor. Raw message.content uses reasoning-format none and skip-chat-parsing. Require a closing </think> followed by a nonempty final answer; no enable_thinking control is documented or passed.
Runtime prerequisites:
Ubuntu 24.04 x86_64; See prerequisites above.

Official artifact · License

Exact native and publisher GGUF repositories carry identical OpenMDW-1.1 license text. It grants permission free of charge to deal in Model Materials, subject to its terms. Distribution must retain the agreement and applicable copyright/origin notices; litigation termination and third-party-rights responsibilities apply. This recipe records open weights, without an OSI certification claim. Read the full pinned agreement before use or redistribution.

  • The full 26B-parameter MoE artifact is 15,935,808,352 bytes. The publisher reports about 3B active parameters per token; active parameter count is not the resident model weight size or total memory requirement.
  • Runtime weights, sliding/full-attention cache, routing buffers, OS and build artifacts add requirements. 64 GiB RAM and 80 GiB disk are planning allocations, not measured minimums or fit guarantees.

One-time setup and hardware

hardware
Not applicable — AWS hardware acquisition is included in compute rental; no purchased GPU required. Any separate hardware cost is not applicable.
AWS hardware acquisition is included in compute rental; no purchased GPU required. Any separate hardware cost is not applicable. Tax: unknown.
setup
Unknown — Initial provisioning/build time and labor cost are unknown.
Initial provisioning/build time and labor cost are unknown. Tax: unknown.

Recurring costs

compute
$0.53 / hour
AWS published us-east-1 Linux On-Demand quote USD 0.5292 per billed instance-hour, no paid preinstalled software. Eight billed hours cost USD 4.2336; 24 hours USD 12.7008 compute. Idle running hours bill. No discount, Spot or reservation assumed. Storage, egress, tax and administration additional; capacity unknown. Region: us-east-1. SKU: r7i.2xlarge Linux On-Demand. Tax: excluded. Price dated Oct 15, 2026. Price source
storage
$0.08 / GB month
Captured us-east-1 gp3 quote USD 0.08 per GB-month, with GB defined as 1024^3 bytes. Selected 80 GiB root volume costs USD 6.40 per full month, prorated by provisioned duration, at baseline 3000 IOPS and 125 MB/s. No provisioned extras assumed; retained snapshots/resources separate. Storage persists after stop; tax excluded. Region: us-east-1. SKU: EBS gp3 baseline 3000 IOPS / 125 MB/s. Tax: excluded. Price dated Oct 16, 2026. Price source
egress
Unknown — Data transfer price, allowances and traffic volume are unknown.
Data transfer price, allowances and traffic volume are unknown. Tax: unknown.
administration
Unknown — Maintenance labor rate and hours are unknown.
Maintenance labor rate and hours are unknown. Tax: unknown.

Install the runtime

  1. 1. Build the pinned CPU server

    Install build dependencies and compile the exact official source revision. These packages are system shared dependencies; no CUDA/GPU driver is required. GGML_NATIVE=ON selects -march=native for this GNU/Linux x86 build, compiling for the host CPU. Build and run on the same machine; the binary uses the instruction set exposed by this r7i.2xlarge guest. Do not copy it to a different instance family without rebuilding.

    Command
    sudo apt-get update
    sudo apt-get install -y build-essential cmake git curl ca-certificates openssl
    mkdir -p "$HOME/trinity-mini-hosting/models"
    cd "$HOME/trinity-mini-hosting"
    git clone https://github.com/ggml-org/llama.cpp.git
    cd llama.cpp
    git checkout --detach 620d2225cc436ab0b001e3200e590e3b286ef9ab
    cmake -S . -B build -DCMAKE_BUILD_TYPE=Release -DGGML_METAL=OFF -DGGML_NATIVE=ON
    cmake --build build --config Release -j 8
    ./build/bin/llama-server --version

    Expected result: Build succeeds and server version identifies pinned source. Stop if checkout or build fails.

Download the model

  1. 1. Download exact publisher GGUF and native Trinity template

    Use Arcee AI’s own immutable Q4_K_M GGUF and separately pinned native template. Verify both SHA-256 values. The publisher documents llama.cpp support starting at b7061; this recipe selects the pinned 0.7.0 source with explicit AFMOE support. The exact GGUF has not been loaded here.

    Command
    cd "$HOME/trinity-mini-hosting"
    curl --fail --location --output models/Trinity-Mini-Q4_K_M.gguf 'https://huggingface.co/arcee-ai/Trinity-Mini-GGUF/resolve/a11737ef4b6f89d958d3f6ab670a9aa26e4b8ad7/Trinity-Mini-Q4_K_M.gguf'
    curl --fail --location --output models/trinity-mini-chat-template.jinja 'https://huggingface.co/arcee-ai/Trinity-Mini/raw/cdb81e8130815fec299cccc8f8700004b8366e1e/chat_template.jinja'
    sha256sum --check <<'SHA256'
    7c2f0f2991923c21baee495340b79feb2a124b1c93b57dea0652f8aa27ba08e8  models/Trinity-Mini-Q4_K_M.gguf
    97eba9fdb6e0a23122d1aefba507c835a65ad382441fc76c18d2b40981f7f1ae  models/trinity-mini-chat-template.jinja
    SHA256

    Expected result: Both files report OK; model 15,935,808,352 bytes and template 2,860 bytes. Stop on mismatch.

Configure

  1. 1. Create a private service credential

    Generate this only on your own host when following the guide. It is a local service password, not an OpenAI/provider credential. Keep it private and never paste it into tickets. Create it only when absent; preserve an existing key so a repeated setup does not silently rotate access. It is not created during this documentation run.

    Command
    cd "$HOME/trinity-mini-hosting"
    umask 077
    if [ ! -e .api-key ]; then
      openssl rand -hex 32 > .api-key
    fi
    test -s .api-key || { echo 'Existing .api-key is empty; restore it before continuing.'; exit 1; }
    chmod 600 .api-key

    Expected result: A non-empty private .api-key file exists; existing key preserved. No secret is printed.

Start the model

  1. 1. Start one CPU reasoning sequence

    The pinned converter registers AfmoeForCausalLM as AFMOE. Native runtime recognizes the 32-layer 26B model and handles leading dense layers, routed and shared experts, route normalization/scaling, and interleaved sliding/full attention. The native Jinja parser accepts generation blocks. Use the pinned template verbatim. The 8,192-token total context and 4,096-token combined reasoning/final cap are bounded author choices below the advertised 131,072 maximum positions; adequacy and actual token count remain unmeasured.

    Command
    cd "$HOME/trinity-mini-hosting"
    export TRINITY_MINI_SERVICE_KEY="$(cat .api-key)"
    ./llama.cpp/build/bin/llama-server -m models/Trinity-Mini-Q4_K_M.gguf --alias trinity-mini --host 127.0.0.1 --port 8080 --api-key "$TRINITY_MINI_SERVICE_KEY" --device none -ngl 0 -t 8 -c 8192 -np 1 --cache-type-k f16 --cache-type-v f16 --jinja --chat-template-file models/trinity-mini-chat-template.jinja --reasoning-format none --skip-chat-parsing --temp 0.15 --top-p 0.75 --top-k 50 --min-p 0.06 --repeat-penalty 1.0

    Expected result: Model initializes and listener starts. Pass readiness and authenticated chat checks before relying on it. Command-line key may be visible to privileged/local users; trusted host required.

Connect to your endpoint

http://127.0.0.1:8080/v1

Server loopback only. For remote client run ssh -N -L 8080:127.0.0.1:8080 USER@VM_IP using your existing SSH access, then call localhost:8080 on client with the service key. Keep EC2 8080 closed.

Access: loopback. Reader-generated private service key in Authorization: Bearer header for chat; /health is public on loopback. SSH authenticates the cloud tunnel separately. Keep the key private; this is not an OpenAI API key.

Verify the setup

  1. 1. Check readiness and authenticated text generation

    Use a second terminal on the server. Inspect raw choices[0].message.content for a closing </think>, followed by a nonempty final summary accurately reflecting the supplied passage. The template prefills the opening tag, which may not repeat in returned content. Require finish_reason != length. Missing closure, incomplete final body or added facts fails the check. This is functional verification, not a benchmark.

    Command
    cd "$HOME/trinity-mini-hosting"
    export TRINITY_MINI_SERVICE_KEY="$(cat .api-key)"
    curl --fail http://127.0.0.1:8080/health
    curl --fail http://127.0.0.1:8080/v1/chat/completions -H "Authorization: Bearer $TRINITY_MINI_SERVICE_KEY" -H 'Content-Type: application/json' -d '{"model":"trinity-mini","messages":[{"role":"user","content":"Summarize this supplied passage in one sentence without adding facts: The community library opens at 9am on weekdays and closes at 6pm. On Saturday it opens at 10am and closes at 4pm. It is closed on Sunday."}],"max_tokens":4096,"temperature":0.15,"top_k":50,"top_p":0.75,"min_p":0.06,"repeat_penalty":1.0}'
    curl -o /dev/null -s -w '%{http_code}\n' http://127.0.0.1:8080/v1/chat/completions -H 'Content-Type: application/json' -d '{"model":"trinity-mini","messages":[{"role":"user","content":"Hello"}]}'

    Expected result: Health HTTP 200; authenticated generation contains closed reasoning and an accurate nonempty final summary, finish_reason is not length; unauthenticated chat HTTP 401. Never accept a partial reasoning trace as the final answer.

Performance evidence

No compatible measurement available. Documented support and estimated memory do not establish response speed.

Limits and troubleshooting

  • Documented instructions only. No weights, build, inference, peak memory, latency or throughput measurement was executed by OpenTools.
  • Trinity Mini is a 26B total / about 3B active MoE according to the publisher. All selected artifact weights must be available; active parameters do not establish model fit or a smaller download.
  • Native config specifies 131,072 maximum positions and a 2,048-token sliding window with every fourth layer full attention. This recipe uses only 8,192 combined template, prompt and generated tokens, with a 4,096-token output cap. Neither the full window nor actual prompt/output counts are validated.
  • F16 K/V flags specify attention cache precision. Expert-routing buffers and full process memory remain unknown; no generic cache formula or sibling model performance is transferred.
  • The publisher documents llama.cpp support from b7061, and the selected newer pinned source explicitly handles AFMOE. Exact GGUF startup compatibility is source-supported but unexecuted.
  • The native 2,860-byte Jinja template is used verbatim, including <think> generation prefill and stripping historical assistant reasoning. The pinned runtime parser ignores Transformers generation/endgeneration blocks. No tool executor, thinking-disable switch or structured reasoning parser is configured.
  • Temperature 0.15, top_k 50, top_p 0.75 and min_p 0.06 follow the publisher’s model details and generation config. repeat_penalty 1 and the 4,096-token output cap are author choices. The short bounded prompt does not establish adequate reasoning budget for arbitrary tasks.
  • 64 GiB RAM, 80 GiB disk and eight threads are resource allocations, not minimum requirements. Original dated cloud compute/storage quotes exclude tax, egress and administration. Local hardware purchase and electricity remain unknown.
  • OpenMDW-1.1 is the exact publisher license. Read its grants, notice retention, termination and rights-responsibility terms; this record does not assert OSI approval.
  • Loopback and key authentication do not isolate privileged local users. Public or shared operation needs separate TLS/authentication/operations design. OS and build dependencies resolve at installation; retain actual versions and build logs.
Download or checksum failure
Verify exact uppercase filename and pinned URL; remove only the incomplete dedicated model file and retry. Do not run a mismatching file.
Architecture unsupported or build fails
Confirm exact pinned source commit, Ubuntu compiler/CMake dependencies and GGUF. Do not substitute an older packaged llama.cpp silently.
Out of memory or context overflow
Stop server, free/provision RAM or reduce context and restart; shorten input/output within configured total. Allocation is not a measured guarantee; record any changed configuration.
Connection refused or HTTP 401
Check foreground server logs, 8080 binding, matching private key and Bearer header. Cloud client must have working SSH tunnel to the correct VM; do not expose the port to repair a tunnel.
Template error, incomplete answer or architecture mismatch
Check exact Trinity Mini GGUF/template hashes and pinned AFMOE runtime. Stop on architecture or Jinja errors. Require </think>, a useful factual final summary and finish_reason other than length. A truncated reasoning trace fails the check. Keep template, input and output within 8,192 tokens. Changes to artifact, context, parser or output cap require a newly validated configuration.

Stop the model

  1. 1. Stop the service

    Press Ctrl+C in server terminal. Unset key in each shell. Close the SSH tunnel with Ctrl+C. Stopping the process alone does not stop EC2 billing.

    Command
    unset TRINITY_MINI_SERVICE_KEY

    Expected result: Server stops and localhost:8080 stops serving.

Uninstall

  1. 1. Remove dedicated model, build and credential files

    Back up wanted logs/files first. Remove only this walkthrough directory after confirming its path. Shared apt packages can remain; do not remove other workloads.

    Command
    rm -r -- "$HOME/trinity-mini-hosting"

    Expected result: Dedicated files and local service credential removed; shared system dependencies remain.

Shut down cloud resources

  1. 1. End cloud billing resources

    In EC2 us-east-1, stop for later reuse or terminate this exact VM. Review Delete on termination for its root volume. Delete unwanted detached EBS volumes, snapshots and reserved/Elastic/public IP resources. Stopped VM storage and retained resources may still bill. Check Bills/Cost Explorer later for final usage.

    Expected result: Exact instance is stopped/terminated as intended; no unwanted attached/retained billed resources remain. Instance compute stops only after its lifecycle transition, not when llama-server stops.

Sources and verification

Sources checked Oct 16, 2026. Verification source

Compare this configuration

Pricing

Token pricing for Trinity Mini API usage.

Input Tokens

$0.04/M

per million tokens

Output Tokens

$0.15/M

per million tokens

Competing Models

Same pricing tier — direct alternatives to Trinity Mini

Efficiency

AI Models by Arcee AI

AI models from the same organization.

ModelContext WindowPrice (In / Out per M)
Trinity Large ThinkingCurrent262K$0.22 / $0.85
Trinity Large PreviewCurrent131K-- / --
SpotlightCurrent131K$0.18 / $0.18