Hunyuan A13B Instruct

Exact model and requirements

Artifact:

Tencent / Hunyuan A13B Instruct Q4_K_M publisher GGUF · checkpoint tencent/Hunyuan-A13B-Instruct-GGUF

Publisher revision:

fab91edee31364882a57e601f4b16f12956a2418

Artifact digest:

sha256:7802da9933ad9340064cfeab7edf814b1cdf523d3b23aefaa6ad737d3b27d957

Download size:

45.43 GiB

Memory requirement (unknown):

Unknown — No exact full-process peak for Hunyuan A13B Q4_K_M and CPU workload.

System RAM / disk:

Unknown / Unknown

Placement:

single device. One CPU host; no tensor/pipeline sharding. CPU-only execution; --device none and zero GPU layers. No disk or accelerator offload.

KV cache:

F16 attention K and V for this bounded CPU configuration; exact full cache/process memory unmeasured.

Workload:

4,096 token context; Unknown; Unknown; concurrency 1; batch 1; cache unknown; repetitions Unknown.

Tokenizer:

Exact Tencent GGUF tokenizer with separately pinned native tokenizer_config Jinja · fab91edee31364882a57e601f4b16f12956a2418; native template 290ddb9a56ed23c2c83a1c8081533e58925df952

Sampling / reasoning:

Temperature 0.7, top_k 20, top_p 0.8 and repetition_penalty 1.05 match native generation config. min_p 0.0, 4,096 context and 512 output cap are author walkthrough choices. Publisher documents /no_think prefix for fast mode and /think for slow mode. Every walkthrough user prompt uses /no_think. The pinned native Jinja contains no enable_thinking variable despite README kwargs advice, so this recipe relies on the explicit documented prefix and verifies actual output. Raw content is preserved with reasoning-format none and skip-chat-parsing. No tool executor or separate reasoning parser.

Runtime prerequisites:

Ubuntu 24.04 x86_64; See prerequisites above.

Official artifact · License

Tencent Hunyuan Community License for A13B, June 27, 2025. Territory excludes European Union, United Kingdom and South Korea; use and hosted services must remain within applicable Territory and agreement/AUP conditions. Distribution/services require agreement delivery, modified-file notices and applicable use-restriction notices; non-hosted distributions require the prescribed Notice. The Powered by Tencent Hunyuan marking and experience statement are encouraged, not mandatory. Organizations exceeding 100 million monthly active users on the release-date test need Tencent’s separate express license. Works and outputs must not improve other AI models except Tencent Hunyuan or its derivatives. AUP, termination and compliance terms apply. Read the complete pinned agreement; no Apache, OSI or legal-clearance claim.

• tokenizer_config.json (contains verbatim native chat_template) · 0 GiB · revision 290ddb9a56ed23c2c83a1c8081533e58925df952

• The full 80B-total model artifact is 48,782,257,888 bytes. About 13B active parameters per token does not reduce resident expert weights to 13B.
• All expert weights, attention cache, buffers, OS and build files add memory/storage. 64 GiB RAM and 80 GiB disk are author allocations, not measured minima or fit guarantees.

Install the runtime

1. 1. Build the pinned CPU server

   Install build dependencies and compile the exact official source revision. These packages are system shared dependencies; no CUDA/GPU driver is required. GGML_NATIVE=ON selects -march=native for this GNU/Linux x86 build, compiling for the host CPU. Build and run on the same machine; no AVX-512 requirement is imposed on the i5-12400.

   CommandCopy command

   sudo apt-get update
   sudo apt-get install -y build-essential cmake git curl ca-certificates openssl python3
   mkdir -p "$HOME/hunyuan-a13b-hosting/models"
   cd "$HOME/hunyuan-a13b-hosting"
   git clone https://github.com/ggml-org/llama.cpp.git
   cd llama.cpp
   git checkout --detach 620d2225cc436ab0b001e3200e590e3b286ef9ab
   cmake -S . -B build -DCMAKE_BUILD_TYPE=Release -DGGML_METAL=OFF -DGGML_NATIVE=ON
   cmake --build build --config Release -j 8
   ./build/bin/llama-server --version

   Expected result: Build succeeds and server version identifies pinned source. Stop if checkout or build fails.

Download the model

1. 1. Download exact publisher GGUF and native template configuration

   Verify both inputs, then extract the exact chat_template string without modification. The template extraction uses system Python3 only; no model code runs. Stop on any hash mismatch.

   CommandCopy command

   cd "$HOME/hunyuan-a13b-hosting"
   curl --fail --location --output models/Hunyuan-A13B-Instruct-Q4_K_M.gguf 'https://huggingface.co/tencent/Hunyuan-A13B-Instruct-GGUF/resolve/fab91edee31364882a57e601f4b16f12956a2418/Hunyuan-A13B-Instruct-Q4_K_M.gguf'
   curl --fail --location --output models/tokenizer_config.json 'https://huggingface.co/tencent/Hunyuan-A13B-Instruct/raw/290ddb9a56ed23c2c83a1c8081533e58925df952/tokenizer_config.json'
   sha256sum --check <<'SHA256'
   7802da9933ad9340064cfeab7edf814b1cdf523d3b23aefaa6ad737d3b27d957  models/Hunyuan-A13B-Instruct-Q4_K_M.gguf
   b2e1225428746d48b9fa435a918b7684622bbe0212ce6f822fc6005b157b2bad  models/tokenizer_config.json
   SHA256
   python3 - <<'PYTEMPLATE'
   import json,pathlib
   p=pathlib.Path('models')
   p.joinpath('hunyuan-chat.jinja').write_text(json.loads(p.joinpath('tokenizer_config.json').read_text())['chat_template'],encoding='utf-8')
   PYTEMPLATE
   sha256sum --check <<'SHA256'
   816b4f48ac106aa74c041ae835ececba878e43e50dfa85f1f5426d174bb2275a  models/hunyuan-chat.jinja
   SHA256

   Expected result: Both downloaded inputs and the extracted native Jinja report OK. GGUF is 48,782,257,888 bytes; stop on mismatch.

Configure

1. 1. Create a private service credential

   Generate this only on your own host when following the guide. It is a local service password, not an OpenAI/provider credential. Keep it private and never paste it into tickets. Create it only when absent; preserve an existing key so a repeated setup does not silently rotate access. It is not created during this documentation run.

   CommandCopy command

   cd "$HOME/hunyuan-a13b-hosting"
   umask 077
   if [ ! -e .api-key ]; then
     openssl rand -hex 32 > .api-key
   fi
   test -s .api-key || { echo 'Existing .api-key is empty; restore it before continuing.'; exit 1; }
   chmod 600 .api-key

   Expected result: A non-empty private .api-key file exists; existing key preserved. No secret is printed.

Start the model

1. 1. Start one bounded CPU sequence

   Pinned converter explicitly maps HunYuanMoEV1ForCausalLM to HUNYUAN_MOE and writes routed/shared expert settings and native dynamic-NTK RoPE interpretation; native runtime recognizes the 32-layer A13B variant. This is source compatibility, not an executed model load. Use the exact native Jinja and /no_think on each prompt; the selected 4,096 total context and 512 output cap do not validate larger windows.

   CommandCopy command

   cd "$HOME/hunyuan-a13b-hosting"
   export HUNYUAN_A13B_SERVICE_KEY="$(cat .api-key)"
   ./llama.cpp/build/bin/llama-server -m models/Hunyuan-A13B-Instruct-Q4_K_M.gguf --alias hunyuan-a13b --host 127.0.0.1 --port 8080 --api-key "$HUNYUAN_A13B_SERVICE_KEY" --device none -ngl 0 -t 8 -c 4096 -np 1 --cache-type-k f16 --cache-type-v f16 --jinja --chat-template-file models/hunyuan-chat.jinja --reasoning-format none --skip-chat-parsing --temp 0.7 --top-p 0.8 --top-k 20 --min-p 0.0 --repeat-penalty 1.05

   Expected result: Model initializes and loopback listener starts. Pass readiness and authenticated functional checks. Exact binary fit/startup are unexecuted; local privileged users may inspect command-line key.

Verify the setup

1. 1. Check readiness and authenticated text generation

   Use a second terminal on the server. The user message begins with the publisher’s exact /no_think control. Inspect raw message.content for a nonempty accurate final summary with no substantive thinking trace and finish_reason other than length. Unexpected reasoning, malformed template output, added facts or truncation fails this bounded fast-mode check. No exact wording or speed benefit is guaranteed.

   CommandCopy command

   cd "$HOME/hunyuan-a13b-hosting"
   export HUNYUAN_A13B_SERVICE_KEY="$(cat .api-key)"
   curl --fail http://127.0.0.1:8080/health
   curl --fail http://127.0.0.1:8080/v1/chat/completions -H "Authorization: Bearer $HUNYUAN_A13B_SERVICE_KEY" -H 'Content-Type: application/json' -d '{"model":"hunyuan-a13b","messages":[{"role":"user","content":"/no_think Summarize this supplied passage in one sentence without adding facts: The community library opens at 9am on weekdays and closes at 6pm. On Saturday it opens at 10am and closes at 4pm. It is closed on Sunday."}],"max_tokens":512,"temperature":0.7,"top_k":20,"top_p":0.8,"min_p":0.0,"repeat_penalty":1.05}'
   curl -o /dev/null -s -w '%{http_code}\n' http://127.0.0.1:8080/v1/chat/completions -H 'Content-Type: application/json' -d '{"model":"hunyuan-a13b","messages":[{"role":"user","content":"Hello"}]}'

   Expected result: Health HTTP200; authenticated response has a nonempty accurate completed final summary with no substantive thinking trace and finish_reason not length; missing key returns401. Documented instructions do not claim this check has run.

Performance evidence

No compatible measurement available. Documented support and estimated memory do not establish response speed.

Limits and troubleshooting

• Documented only. No model build, weights, inference, memory peak, throughput, latency or fast-mode success was executed by OpenTools.
• Publisher describes 80B total and 13B active parameters. All routed/shared expert weights remain relevant in this full CPU residency configuration.
• Native config has 32,768 default positions, while publisher documents 262,144 maximum context and converter emits 262,144 metadata under exact dynamic-NTK assumptions. This recipe selects only 4,096 total tokens, one sequence and 512 output tokens; do not infer long-context fit or modify RoPE.
• Native config lists 64 routed experts, eight selected and one shared expert per layer. Native runtime implements shared and routed expert tensors; no dense-only or active-weight-only memory assumption is used.
• The separately pinned tokenizer_config native template is extracted verbatim. It does not reference enable_thinking despite publisher README kwargs advice. Use the official /no_think prompt prefix for this bounded fast-mode task and verify output; /think slow mode requires separate workload budget and review.
• Sampler follows native generation config: temperature 0.7, top_k 20, top_p 0.8, repeat penalty 1.05. min_p 0, context and output caps are explicit walkthrough choices.
• 64 GiB RAM, 80 GiB disk and eight threads are allocations, not measured minima or fit. The 48.78 GB file excludes runtime/cache/OS/build overhead. F16 attention K/V is not a full process memory estimate.
• Exact Tencent custom license and AUP restrict Territory and use. EU, UK and South Korea are excluded; distribution/service notices and special commercial scale terms apply. No broad unrestricted licensing claim.
• Original dated cloud compute/storage excludes tax, egress and administration; local power/hardware costs remain unknown. Loopback/key do not isolate privileged users; public deployment requires separate operations/security design.
• GPU, disk offload, larger context, slow thinking and other supported workflows remain outside this pair’s qualification and are not excluded. Full coverage remains open.

Download or checksum failure

Verify exact uppercase filename and pinned URL; remove only the incomplete dedicated model file and retry. Do not run a mismatching file.

Architecture unsupported or build fails

Confirm exact pinned source commit, Ubuntu compiler/CMake dependencies and GGUF. Do not substitute an older packaged llama.cpp silently.

Out of memory or context overflow

Stop server, free/provision RAM or reduce context and restart; shorten input/output within configured total. Allocation is not a measured guarantee; record any changed configuration.

Connection refused or HTTP 401

Check foreground server logs, 8080 binding, matching private key and Bearer header. Run the client on this same desktop and keep the loopback listener.

Template error, incomplete answer or architecture mismatch

Check exact publisher GGUF/native tokenizer_config hashes and the extracted Jinja. Keep /no_think at the beginning of every user prompt and inspect final output. Do not infer enable_thinking kwargs work in this pinned template. Stop on load/template errors or unexpected reasoning; changing context/mode/parser requires new validation.

Stop the model

1. 1. Stop the service

   Press Ctrl+C in server terminal. Unset key in each shell. Confirm this desktop localhost:8080 no longer serves the model.

   CommandCopy command

   unset HUNYUAN_A13B_SERVICE_KEY

   Expected result: Server stops and localhost:8080 stops serving.

Uninstall

1. 1. Remove dedicated model, build and credential files

   Back up wanted logs/files first. Remove only this walkthrough directory after confirming its path. Shared apt packages can remain; do not remove other workloads.

   CommandCopy command

   rm -r -- "$HOME/hunyuan-a13b-hosting"

   Expected result: Dedicated files and local service credential removed; shared system dependencies remain.

Exact model and requirements

Artifact:

Tencent / Hunyuan A13B Instruct Q4_K_M publisher GGUF · checkpoint tencent/Hunyuan-A13B-Instruct-GGUF

Publisher revision:

fab91edee31364882a57e601f4b16f12956a2418

Artifact digest:

sha256:7802da9933ad9340064cfeab7edf814b1cdf523d3b23aefaa6ad737d3b27d957

Download size:

45.43 GiB

Memory requirement (unknown):

Unknown — No exact full-process peak for Hunyuan A13B Q4_K_M and CPU workload.

System RAM / disk:

Unknown / Unknown

Placement:

single device. One CPU host; no tensor/pipeline sharding. CPU-only execution; --device none and zero GPU layers. No disk or accelerator offload.

KV cache:

F16 attention K and V for this bounded CPU configuration; exact full cache/process memory unmeasured.

Workload:

4,096 token context; Unknown; Unknown; concurrency 1; batch 1; cache unknown; repetitions Unknown.

Tokenizer:

Exact Tencent GGUF tokenizer with separately pinned native tokenizer_config Jinja · fab91edee31364882a57e601f4b16f12956a2418; native template 290ddb9a56ed23c2c83a1c8081533e58925df952

Sampling / reasoning:

Temperature 0.7, top_k 20, top_p 0.8 and repetition_penalty 1.05 match native generation config. min_p 0.0, 4,096 context and 512 output cap are author walkthrough choices. Publisher documents /no_think prefix for fast mode and /think for slow mode. Every walkthrough user prompt uses /no_think. The pinned native Jinja contains no enable_thinking variable despite README kwargs advice, so this recipe relies on the explicit documented prefix and verifies actual output. Raw content is preserved with reasoning-format none and skip-chat-parsing. No tool executor or separate reasoning parser.

Runtime prerequisites:

Ubuntu 24.04 x86_64; See prerequisites above.

Official artifact · License

Tencent Hunyuan Community License for A13B, June 27, 2025. Territory excludes European Union, United Kingdom and South Korea; use and hosted services must remain within applicable Territory and agreement/AUP conditions. Distribution/services require agreement delivery, modified-file notices and applicable use-restriction notices; non-hosted distributions require the prescribed Notice. The Powered by Tencent Hunyuan marking and experience statement are encouraged, not mandatory. Organizations exceeding 100 million monthly active users on the release-date test need Tencent’s separate express license. Works and outputs must not improve other AI models except Tencent Hunyuan or its derivatives. AUP, termination and compliance terms apply. Read the complete pinned agreement; no Apache, OSI or legal-clearance claim.

• tokenizer_config.json (contains verbatim native chat_template) · 0 GiB · revision 290ddb9a56ed23c2c83a1c8081533e58925df952

• The full 80B-total model artifact is 48,782,257,888 bytes. About 13B active parameters per token does not reduce resident expert weights to 13B.
• All expert weights, attention cache, buffers, OS and build files add memory/storage. 64 GiB RAM and 80 GiB disk are author allocations, not measured minima or fit guarantees.

Install the runtime

1. 1. Build the pinned CPU server

   Install build dependencies and compile the exact official source revision. These packages are system shared dependencies; no CUDA/GPU driver is required. GGML_NATIVE=ON selects -march=native for this GNU/Linux x86 build, compiling for the host CPU. Build and run on the same machine; the binary uses the instruction set exposed by this r7i.2xlarge guest. Do not copy it to a different instance family without rebuilding.

   CommandCopy command

   sudo apt-get update
   sudo apt-get install -y build-essential cmake git curl ca-certificates openssl python3
   mkdir -p "$HOME/hunyuan-a13b-hosting/models"
   cd "$HOME/hunyuan-a13b-hosting"
   git clone https://github.com/ggml-org/llama.cpp.git
   cd llama.cpp
   git checkout --detach 620d2225cc436ab0b001e3200e590e3b286ef9ab
   cmake -S . -B build -DCMAKE_BUILD_TYPE=Release -DGGML_METAL=OFF -DGGML_NATIVE=ON
   cmake --build build --config Release -j 8
   ./build/bin/llama-server --version

   Expected result: Build succeeds and server version identifies pinned source. Stop if checkout or build fails.

Download the model

1. 1. Download exact publisher GGUF and native template configuration

   Verify both inputs, then extract the exact chat_template string without modification. The template extraction uses system Python3 only; no model code runs. Stop on any hash mismatch.

   CommandCopy command

   cd "$HOME/hunyuan-a13b-hosting"
   curl --fail --location --output models/Hunyuan-A13B-Instruct-Q4_K_M.gguf 'https://huggingface.co/tencent/Hunyuan-A13B-Instruct-GGUF/resolve/fab91edee31364882a57e601f4b16f12956a2418/Hunyuan-A13B-Instruct-Q4_K_M.gguf'
   curl --fail --location --output models/tokenizer_config.json 'https://huggingface.co/tencent/Hunyuan-A13B-Instruct/raw/290ddb9a56ed23c2c83a1c8081533e58925df952/tokenizer_config.json'
   sha256sum --check <<'SHA256'
   7802da9933ad9340064cfeab7edf814b1cdf523d3b23aefaa6ad737d3b27d957  models/Hunyuan-A13B-Instruct-Q4_K_M.gguf
   b2e1225428746d48b9fa435a918b7684622bbe0212ce6f822fc6005b157b2bad  models/tokenizer_config.json
   SHA256
   python3 - <<'PYTEMPLATE'
   import json,pathlib
   p=pathlib.Path('models')
   p.joinpath('hunyuan-chat.jinja').write_text(json.loads(p.joinpath('tokenizer_config.json').read_text())['chat_template'],encoding='utf-8')
   PYTEMPLATE
   sha256sum --check <<'SHA256'
   816b4f48ac106aa74c041ae835ececba878e43e50dfa85f1f5426d174bb2275a  models/hunyuan-chat.jinja
   SHA256

   Expected result: Both downloaded inputs and the extracted native Jinja report OK. GGUF is 48,782,257,888 bytes; stop on mismatch.

Configure

1. 1. Create a private service credential

   Generate this only on your own host when following the guide. It is a local service password, not an OpenAI/provider credential. Keep it private and never paste it into tickets. Create it only when absent; preserve an existing key so a repeated setup does not silently rotate access. It is not created during this documentation run.

   CommandCopy command

   cd "$HOME/hunyuan-a13b-hosting"
   umask 077
   if [ ! -e .api-key ]; then
     openssl rand -hex 32 > .api-key
   fi
   test -s .api-key || { echo 'Existing .api-key is empty; restore it before continuing.'; exit 1; }
   chmod 600 .api-key

   Expected result: A non-empty private .api-key file exists; existing key preserved. No secret is printed.

Start the model

1. 1. Start one bounded CPU sequence

   Pinned converter explicitly maps HunYuanMoEV1ForCausalLM to HUNYUAN_MOE and writes routed/shared expert settings and native dynamic-NTK RoPE interpretation; native runtime recognizes the 32-layer A13B variant. This is source compatibility, not an executed model load. Use the exact native Jinja and /no_think on each prompt; the selected 4,096 total context and 512 output cap do not validate larger windows.

   CommandCopy command

   cd "$HOME/hunyuan-a13b-hosting"
   export HUNYUAN_A13B_SERVICE_KEY="$(cat .api-key)"
   ./llama.cpp/build/bin/llama-server -m models/Hunyuan-A13B-Instruct-Q4_K_M.gguf --alias hunyuan-a13b --host 127.0.0.1 --port 8080 --api-key "$HUNYUAN_A13B_SERVICE_KEY" --device none -ngl 0 -t 8 -c 4096 -np 1 --cache-type-k f16 --cache-type-v f16 --jinja --chat-template-file models/hunyuan-chat.jinja --reasoning-format none --skip-chat-parsing --temp 0.7 --top-p 0.8 --top-k 20 --min-p 0.0 --repeat-penalty 1.05

   Expected result: Model initializes and loopback listener starts. Pass readiness and authenticated functional checks. Exact binary fit/startup are unexecuted; local privileged users may inspect command-line key.

Verify the setup

1. 1. Check readiness and authenticated text generation

   Use a second terminal on the server. The user message begins with the publisher’s exact /no_think control. Inspect raw message.content for a nonempty accurate final summary with no substantive thinking trace and finish_reason other than length. Unexpected reasoning, malformed template output, added facts or truncation fails this bounded fast-mode check. No exact wording or speed benefit is guaranteed.

   CommandCopy command

   cd "$HOME/hunyuan-a13b-hosting"
   export HUNYUAN_A13B_SERVICE_KEY="$(cat .api-key)"
   curl --fail http://127.0.0.1:8080/health
   curl --fail http://127.0.0.1:8080/v1/chat/completions -H "Authorization: Bearer $HUNYUAN_A13B_SERVICE_KEY" -H 'Content-Type: application/json' -d '{"model":"hunyuan-a13b","messages":[{"role":"user","content":"/no_think Summarize this supplied passage in one sentence without adding facts: The community library opens at 9am on weekdays and closes at 6pm. On Saturday it opens at 10am and closes at 4pm. It is closed on Sunday."}],"max_tokens":512,"temperature":0.7,"top_k":20,"top_p":0.8,"min_p":0.0,"repeat_penalty":1.05}'
   curl -o /dev/null -s -w '%{http_code}\n' http://127.0.0.1:8080/v1/chat/completions -H 'Content-Type: application/json' -d '{"model":"hunyuan-a13b","messages":[{"role":"user","content":"Hello"}]}'

   Expected result: Health HTTP200; authenticated response has a nonempty accurate completed final summary with no substantive thinking trace and finish_reason not length; missing key returns401. Documented instructions do not claim this check has run.

Performance evidence

No compatible measurement available. Documented support and estimated memory do not establish response speed.

Limits and troubleshooting

• Documented only. No model build, weights, inference, memory peak, throughput, latency or fast-mode success was executed by OpenTools.
• Publisher describes 80B total and 13B active parameters. All routed/shared expert weights remain relevant in this full CPU residency configuration.
• Native config has 32,768 default positions, while publisher documents 262,144 maximum context and converter emits 262,144 metadata under exact dynamic-NTK assumptions. This recipe selects only 4,096 total tokens, one sequence and 512 output tokens; do not infer long-context fit or modify RoPE.
• Native config lists 64 routed experts, eight selected and one shared expert per layer. Native runtime implements shared and routed expert tensors; no dense-only or active-weight-only memory assumption is used.
• The separately pinned tokenizer_config native template is extracted verbatim. It does not reference enable_thinking despite publisher README kwargs advice. Use the official /no_think prompt prefix for this bounded fast-mode task and verify output; /think slow mode requires separate workload budget and review.
• Sampler follows native generation config: temperature 0.7, top_k 20, top_p 0.8, repeat penalty 1.05. min_p 0, context and output caps are explicit walkthrough choices.
• 64 GiB RAM, 80 GiB disk and eight threads are allocations, not measured minima or fit. The 48.78 GB file excludes runtime/cache/OS/build overhead. F16 attention K/V is not a full process memory estimate.
• Exact Tencent custom license and AUP restrict Territory and use. EU, UK and South Korea are excluded; distribution/service notices and special commercial scale terms apply. No broad unrestricted licensing claim.
• Original dated cloud compute/storage excludes tax, egress and administration; local power/hardware costs remain unknown. Loopback/key do not isolate privileged users; public deployment requires separate operations/security design.
• GPU, disk offload, larger context, slow thinking and other supported workflows remain outside this pair’s qualification and are not excluded. Full coverage remains open.

Download or checksum failure

Verify exact uppercase filename and pinned URL; remove only the incomplete dedicated model file and retry. Do not run a mismatching file.

Architecture unsupported or build fails

Confirm exact pinned source commit, Ubuntu compiler/CMake dependencies and GGUF. Do not substitute an older packaged llama.cpp silently.

Out of memory or context overflow

Stop server, free/provision RAM or reduce context and restart; shorten input/output within configured total. Allocation is not a measured guarantee; record any changed configuration.

Connection refused or HTTP 401

Check foreground server logs, 8080 binding, matching private key and Bearer header. Cloud client must have working SSH tunnel to the correct VM; do not expose the port to repair a tunnel.

Template error, incomplete answer or architecture mismatch

Check exact publisher GGUF/native tokenizer_config hashes and the extracted Jinja. Keep /no_think at the beginning of every user prompt and inspect final output. Do not infer enable_thinking kwargs work in this pinned template. Stop on load/template errors or unexpected reasoning; changing context/mode/parser requires new validation.

Stop the model

1. 1. Stop the service

   Press Ctrl+C in server terminal. Unset key in each shell. Close the SSH tunnel with Ctrl+C. Stopping the process alone does not stop EC2 billing.

   CommandCopy command

   unset HUNYUAN_A13B_SERVICE_KEY

   Expected result: Server stops and localhost:8080 stops serving.

Uninstall

1. 1. Remove dedicated model, build and credential files

   Back up wanted logs/files first. Remove only this walkthrough directory after confirming its path. Shared apt packages can remain; do not remove other workloads.

   CommandCopy command

   rm -r -- "$HOME/hunyuan-a13b-hosting"

   Expected result: Dedicated files and local service credential removed; shared system dependencies remain.

Shut down cloud resources

1. 1. End cloud billing resources

   In EC2 us-east-1, stop for later reuse or terminate this exact VM. Review Delete on termination for its root volume. Delete unwanted detached EBS volumes, snapshots and reserved/Elastic/public IP resources. Stopped VM storage and retained resources may still bill. Check Bills/Cost Explorer later for final usage.

   Expected result: Exact instance is stopped/terminated as intended; no unwanted attached/retained billed resources remain. Instance compute stops only after its lifecycle transition, not when llama-server stops.