LFM2-24B-A2B

Exact model and requirements

Artifact:

Liquid AI / LFM2-24B-A2B Q4_K_M · checkpoint LiquidAI/LFM2-24B-A2B-GGUF

Publisher revision:

6247d03b87c27c05a258feb5acdfb4d5efbf0bd8

Artifact digest:

sha256:eb4d2d4d4e61b795726c2f526c4434ca6bc725ad7a783691b58681f025cf58f2

Download size:

13.43 GiB

Memory requirement (unknown):

Unknown — No exact full-process peak-memory evidence for this Q4_K_M configuration.

System RAM / disk:

Unknown / Unknown

Placement:

single device. One CPU host; no tensor/pipeline sharding. CPU-only execution; --device none and zero GPU layers. No disk or accelerator offload.

KV cache:

F16 attention K and V; hybrid short-convolution state managed separately by llama.cpp.

Workload:

4,096 token context; Unknown; Unknown; concurrency 1; batch 1; cache unknown; repetitions Unknown.

Tokenizer:

Embedded LFM2-24B-A2B GGUF tokenizer and chat template · 6247d03b87c27c05a258feb5acdfb4d5efbf0bd8

Sampling / reasoning:

temperature 0.1; top_k 50; repeat_penalty 1.05; top_p 1.0; min_p 0.0; validation max_tokens 512. General-purpose instruct model without reasoning traces per publisher. No separate reasoning parser, speculative drafter or tool executor configured.

Runtime prerequisites:

Ubuntu 24.04 x86_64; See prerequisites above.

Official artifact · License

The custom license defines a USD 10 million annual-revenue threshold and conditions commercial rights on not exceeding it; commercial use by entities exceeding it is not licensed under this agreement. Read the full license for exact scope, redistribution and qualified nonprofit terms.

• Full GGUF download is 14415473952 bytes; MoE active parameter count is not weight storage or full working memory.
• CPU system RAM must accommodate model/cache/convolution state/runtime/OS. Publisher generic 32 GB claim is not an exact-host fit measurement.

Install the runtime

1. 1. Build the pinned CPU server

   Install build dependencies and compile the exact official source revision. These packages are system shared dependencies; no CUDA/GPU driver is required. GGML_NATIVE=ON selects -march=native for this GNU/Linux x86 build, compiling for the host CPU. Build and run on the same machine; no AVX-512 requirement is imposed on the i5-12400.

   CommandCopy command

   sudo apt-get update
   sudo apt-get install -y build-essential cmake git curl ca-certificates openssl
   mkdir -p "$HOME/lfm2-24b-hosting/models"
   cd "$HOME/lfm2-24b-hosting"
   git clone https://github.com/ggml-org/llama.cpp.git
   cd llama.cpp
   git checkout --detach 620d2225cc436ab0b001e3200e590e3b286ef9ab
   cmake -S . -B build -DCMAKE_BUILD_TYPE=Release -DGGML_METAL=OFF -DGGML_NATIVE=ON
   cmake --build build --config Release -j 4
   ./build/bin/llama-server --version

   Expected result: Build succeeds and server version identifies pinned source. Stop if checkout or build fails.

Download the model

1. 1. Download and verify the exact GGUF

   Download the complete case-sensitive Q4_K_M GGUF at the pinned publisher commit. Metadata reports public ungated access. All routed expert weights belong to this model artifact; active parameter count is not download/storage size.

   CommandCopy command

   cd "$HOME/lfm2-24b-hosting"
   curl --fail --location --output models/LFM2-24B-A2B-Q4_K_M.gguf 'https://huggingface.co/LiquidAI/LFM2-24B-A2B-GGUF/resolve/6247d03b87c27c05a258feb5acdfb4d5efbf0bd8/LFM2-24B-A2B-Q4_K_M.gguf'
   printf '%s  %s\n' 'eb4d2d4d4e61b795726c2f526c4434ca6bc725ad7a783691b58681f025cf58f2' 'models/LFM2-24B-A2B-Q4_K_M.gguf' | sha256sum --check

   Expected result: LFM2-24B-A2B-Q4_K_M.gguf reports OK; publisher metadata size 14,415,473,952 bytes. Stop on mismatch.

Configure

1. 1. Create a private service credential

   Generate this only on your own host when following the guide. It is a local service password, not an OpenAI/provider credential. Keep it private and never paste it into tickets. Create it only when absent; preserve an existing key so a repeated setup does not silently rotate access. It is not created during this documentation run.

   CommandCopy command

   cd "$HOME/lfm2-24b-hosting"
   umask 077
   if [ ! -e .api-key ]; then
     openssl rand -hex 32 > .api-key
   fi
   test -s .api-key || { echo 'Existing .api-key is empty; restore it before continuing.'; exit 1; }
   chmod 600 .api-key

   Expected result: A non-empty private .api-key file exists; existing key preserved. No secret is printed.

Start the model

1. 1. Start one CPU-only sequence

   Run the exact GGUF using the pinned CPU build. Its native lfm2moe runtime implementation recognizes the 40-layer 24B_A2B variant and expert routing. This is documented architecture support, not proof of this host’s measured fit. Use embedded template, F16 attention cache, CPU-only execution, one slot and 4096 total prompt plus output tokens. Wait for readiness; leave room for 512 output tokens. No thinking parser or tool executor is configured.

   CommandCopy command

   cd "$HOME/lfm2-24b-hosting"
   export LFM2_24B_SERVICE_KEY="$(cat .api-key)"
   ./llama.cpp/build/bin/llama-server -m models/LFM2-24B-A2B-Q4_K_M.gguf --alias lfm2-24b --host 127.0.0.1 --port 8080 --api-key "$LFM2_24B_SERVICE_KEY" --device none -ngl 0 -t 4 -c 4096 -np 1 --cache-type-k f16 --cache-type-v f16 --temp 0.1 --top-k 50 --top-p 1.0 --min-p 0.0 --repeat-penalty 1.05

   Expected result: Model initializes and listener starts. Pass health and authenticated chat checks before relying on the service. Command-line credentials may be visible to privileged/local users; trusted host required.

Verify the setup

1. 1. Check readiness and authenticated text generation

   Run checks from a second terminal on the server. Health is public on loopback. Check the authenticated chat’s non-empty final content against the supplied passage and ensure finish_reason is not length; no exact wording is guaranteed. This is a short functional summarization check, not a quality benchmark.

   CommandCopy command

   cd "$HOME/lfm2-24b-hosting"
   export LFM2_24B_SERVICE_KEY="$(cat .api-key)"
   curl --fail http://127.0.0.1:8080/health
   curl --fail http://127.0.0.1:8080/v1/chat/completions -H "Authorization: Bearer $LFM2_24B_SERVICE_KEY" -H 'Content-Type: application/json' -d '{"model":"lfm2-24b","messages":[{"role":"user","content":"Summarize this supplied passage in one sentence without adding facts: The community library opens at 9am on weekdays and closes at 6pm. On Saturday it opens at 10am and closes at 4pm. It is closed on Sunday."}],"max_tokens":512,"temperature":0.1,"top_k":50,"top_p":1.0,"min_p":0.0,"repeat_penalty":1.05}'
   curl -o /dev/null -s -w '%{http_code}\n' http://127.0.0.1:8080/v1/chat/completions -H 'Content-Type: application/json' -d '{"model":"lfm2-24b","messages":[{"role":"user","content":"Hello"}]}'

   Expected result: Health HTTP 200 when ready; authorized response contains non-empty choices[0].message.content accurately summarizing the supplied opening hours, with finish_reason not length; unauthorized chat returns HTTP 401. Inspect facts yourself; generated text can be wrong.

Performance evidence

No compatible measurement available. Documented support and estimated memory do not establish response speed.

Limits and troubleshooting

• No speed numbers are imported from the old LFM2 desktop table, phone Q4_0, NPU runs or speculative decoding. No OpenTools execution occurred.
• Operating system packages and build tool dependencies resolve when installed. Save tool versions/build logs with any actual execution. Local hardware is Intel Core i5-12400 with four inference threads allocated; no exact-hardware benchmark is claimed.
• The custom license defines a USD 10 million annual-revenue threshold and conditions commercial rights on not exceeding it; commercial use by entities exceeding it is not licensed under this agreement. Read the full license for exact scope, redistribution and qualified nonprofit terms.
• Loopback and a service key do not isolate the service from privileged local users. A shared/public service needs separately designed TLS, authentication and operations.
• The full GGUF is 14,415,473,952 bytes. Full process memory also includes attention cache, convolution state, buffers and OS; exact working-memory requirement and fit remain unknown. The 24B total weights are not reduced to 2.3B because only a subset is active per token.
• The model card’s detailed table gives 24B total parameters and 2.3B active per token, while its introduction rounds active count to 2B. Config declares 64 experts, four selected per token, 40 layers (30 convolution / 10 attention). Do not transfer dense sibling cache estimates or speed.
• The card states 32768 context while pinned config max_position_embeddings is 128000. This source discrepancy is unresolved. The recipe uses only 4096 combined prompt and output tokens and does not endorse either larger limit.
• This is an instruct model without reasoning traces. Publisher recommends summarization, local RAG, customer support and agentic uses, and does not recommend coding. This recipe verifies text summarization only; it does not configure native tool parsing, retrieval, constrained JSON or autonomous actions.
• The publisher’s consumer 32 GB statement and AMD/H100 benchmarks do not establish fit or speed on this exact Intel/AWS allocation. No OpenTools execution, peak-memory, throughput or latency measurement occurred.

Download or checksum failure

Verify exact uppercase filename and pinned URL; remove only the incomplete dedicated model file and retry. Do not run a mismatching file.

Architecture unsupported or build fails

Confirm exact pinned source commit, Ubuntu compiler/CMake dependencies and GGUF. Do not substitute an older packaged llama.cpp silently.

Out of memory or context overflow

Stop server, free/provision RAM or reduce context and restart; shorten input/output within configured total. Allocation is not a measured guarantee; record any changed configuration.

Connection refused or HTTP 401

Check foreground server logs, 8080 binding, matching private key and Bearer header. Run the client on this same desktop and keep the loopback listener.

MoE loader fails or summarization is wrong/truncated

Confirm full file SHA-256 and pinned llama.cpp checkout with lfm2moe 40-layer support. Do not replace with dense sibling files. Keep source passage/prompt plus output within 4096 tokens; inspect finish_reason and reduce input if needed. Validate facts and task suitability; this setup does not guarantee accuracy.

Stop the model

1. 1. Stop the service

   Press Ctrl+C in server terminal. Unset key in each shell. Confirm this desktop localhost:8080 no longer serves the model.

   CommandCopy command

   unset LFM2_24B_SERVICE_KEY

   Expected result: Server stops and localhost:8080 stops serving.

Uninstall

1. 1. Remove dedicated model, build and credential files

   Back up wanted logs/files first. Remove only this walkthrough directory after confirming its path. Shared apt packages can remain; do not remove other workloads.

   CommandCopy command

   rm -r -- "$HOME/lfm2-24b-hosting"

   Expected result: Dedicated files and local service credential removed; shared system dependencies remain.

Related guides

• https://opentools.ai/llms/lfm2-24b-a2b

Exact model and requirements

Artifact:

Liquid AI / LFM2-24B-A2B Q4_K_M · checkpoint LiquidAI/LFM2-24B-A2B-GGUF

Publisher revision:

6247d03b87c27c05a258feb5acdfb4d5efbf0bd8

Artifact digest:

sha256:eb4d2d4d4e61b795726c2f526c4434ca6bc725ad7a783691b58681f025cf58f2

Download size:

13.43 GiB

Memory requirement (unknown):

Unknown — No exact full-process peak-memory evidence for this Q4_K_M configuration.

System RAM / disk:

Unknown / Unknown

Placement:

single device. One CPU host; no tensor/pipeline sharding. CPU-only execution; --device none and zero GPU layers. No disk or accelerator offload.

KV cache:

F16 attention K and V; hybrid short-convolution state managed separately by llama.cpp.

Workload:

4,096 token context; Unknown; Unknown; concurrency 1; batch 1; cache unknown; repetitions Unknown.

Tokenizer:

Embedded LFM2-24B-A2B GGUF tokenizer and chat template · 6247d03b87c27c05a258feb5acdfb4d5efbf0bd8

Sampling / reasoning:

temperature 0.1; top_k 50; repeat_penalty 1.05; top_p 1.0; min_p 0.0; validation max_tokens 512. General-purpose instruct model without reasoning traces per publisher. No separate reasoning parser, speculative drafter or tool executor configured.

Runtime prerequisites:

Ubuntu 24.04 x86_64; See prerequisites above.

Official artifact · License

The custom license defines a USD 10 million annual-revenue threshold and conditions commercial rights on not exceeding it; commercial use by entities exceeding it is not licensed under this agreement. Read the full license for exact scope, redistribution and qualified nonprofit terms.

• Full GGUF download is 14415473952 bytes; MoE active parameter count is not weight storage or full working memory.
• CPU system RAM must accommodate model/cache/convolution state/runtime/OS. Publisher generic 32 GB claim is not an exact-host fit measurement.

Install the runtime

1. 1. Build the pinned CPU server

   Install build dependencies and compile the exact official source revision. These packages are system shared dependencies; no CUDA/GPU driver is required. GGML_NATIVE=ON selects -march=native for this GNU/Linux x86 build, compiling for the host CPU. Build and run on the same machine; the binary uses the instruction set exposed by this r7i.xlarge guest. Do not copy it to a different instance family without rebuilding.

   CommandCopy command

   sudo apt-get update
   sudo apt-get install -y build-essential cmake git curl ca-certificates openssl
   mkdir -p "$HOME/lfm2-24b-hosting/models"
   cd "$HOME/lfm2-24b-hosting"
   git clone https://github.com/ggml-org/llama.cpp.git
   cd llama.cpp
   git checkout --detach 620d2225cc436ab0b001e3200e590e3b286ef9ab
   cmake -S . -B build -DCMAKE_BUILD_TYPE=Release -DGGML_METAL=OFF -DGGML_NATIVE=ON
   cmake --build build --config Release -j 4
   ./build/bin/llama-server --version

   Expected result: Build succeeds and server version identifies pinned source. Stop if checkout or build fails.

Download the model

1. 1. Download and verify the exact GGUF

   Download the complete case-sensitive Q4_K_M GGUF at the pinned publisher commit. Metadata reports public ungated access. All routed expert weights belong to this model artifact; active parameter count is not download/storage size.

   CommandCopy command

   cd "$HOME/lfm2-24b-hosting"
   curl --fail --location --output models/LFM2-24B-A2B-Q4_K_M.gguf 'https://huggingface.co/LiquidAI/LFM2-24B-A2B-GGUF/resolve/6247d03b87c27c05a258feb5acdfb4d5efbf0bd8/LFM2-24B-A2B-Q4_K_M.gguf'
   printf '%s  %s\n' 'eb4d2d4d4e61b795726c2f526c4434ca6bc725ad7a783691b58681f025cf58f2' 'models/LFM2-24B-A2B-Q4_K_M.gguf' | sha256sum --check

   Expected result: LFM2-24B-A2B-Q4_K_M.gguf reports OK; publisher metadata size 14,415,473,952 bytes. Stop on mismatch.

Configure

1. 1. Create a private service credential

   Generate this only on your own host when following the guide. It is a local service password, not an OpenAI/provider credential. Keep it private and never paste it into tickets. Create it only when absent; preserve an existing key so a repeated setup does not silently rotate access. It is not created during this documentation run.

   CommandCopy command

   cd "$HOME/lfm2-24b-hosting"
   umask 077
   if [ ! -e .api-key ]; then
     openssl rand -hex 32 > .api-key
   fi
   test -s .api-key || { echo 'Existing .api-key is empty; restore it before continuing.'; exit 1; }
   chmod 600 .api-key

   Expected result: A non-empty private .api-key file exists; existing key preserved. No secret is printed.

Start the model

1. 1. Start one CPU-only sequence

   Run the exact GGUF using the pinned CPU build. Its native lfm2moe runtime implementation recognizes the 40-layer 24B_A2B variant and expert routing. This is documented architecture support, not proof of this host’s measured fit. Use embedded template, F16 attention cache, CPU-only execution, one slot and 4096 total prompt plus output tokens. Wait for readiness; leave room for 512 output tokens. No thinking parser or tool executor is configured.

   CommandCopy command

   cd "$HOME/lfm2-24b-hosting"
   export LFM2_24B_SERVICE_KEY="$(cat .api-key)"
   ./llama.cpp/build/bin/llama-server -m models/LFM2-24B-A2B-Q4_K_M.gguf --alias lfm2-24b --host 127.0.0.1 --port 8080 --api-key "$LFM2_24B_SERVICE_KEY" --device none -ngl 0 -t 4 -c 4096 -np 1 --cache-type-k f16 --cache-type-v f16 --temp 0.1 --top-k 50 --top-p 1.0 --min-p 0.0 --repeat-penalty 1.05

   Expected result: Model initializes and listener starts. Pass health and authenticated chat checks before relying on the service. Command-line credentials may be visible to privileged/local users; trusted host required.

Verify the setup

1. 1. Check readiness and authenticated text generation

   Run checks from a second terminal on the server. Health is public on loopback. Check the authenticated chat’s non-empty final content against the supplied passage and ensure finish_reason is not length; no exact wording is guaranteed. This is a short functional summarization check, not a quality benchmark.

   CommandCopy command

   cd "$HOME/lfm2-24b-hosting"
   export LFM2_24B_SERVICE_KEY="$(cat .api-key)"
   curl --fail http://127.0.0.1:8080/health
   curl --fail http://127.0.0.1:8080/v1/chat/completions -H "Authorization: Bearer $LFM2_24B_SERVICE_KEY" -H 'Content-Type: application/json' -d '{"model":"lfm2-24b","messages":[{"role":"user","content":"Summarize this supplied passage in one sentence without adding facts: The community library opens at 9am on weekdays and closes at 6pm. On Saturday it opens at 10am and closes at 4pm. It is closed on Sunday."}],"max_tokens":512,"temperature":0.1,"top_k":50,"top_p":1.0,"min_p":0.0,"repeat_penalty":1.05}'
   curl -o /dev/null -s -w '%{http_code}\n' http://127.0.0.1:8080/v1/chat/completions -H 'Content-Type: application/json' -d '{"model":"lfm2-24b","messages":[{"role":"user","content":"Hello"}]}'

   Expected result: Health HTTP 200 when ready; authorized response contains non-empty choices[0].message.content accurately summarizing the supplied opening hours, with finish_reason not length; unauthorized chat returns HTTP 401. Inspect facts yourself; generated text can be wrong.

Performance evidence

No compatible measurement available. Documented support and estimated memory do not establish response speed.

Limits and troubleshooting

• No speed numbers are imported from the old LFM2 desktop table, phone Q4_0, NPU runs or speculative decoding. No OpenTools execution occurred.
• Operating system packages and build tool dependencies resolve when installed. Save tool versions/build logs with any actual execution. Cloud hardware is AWS r7i.xlarge with four vCPUs and 32 GiB host RAM; four inference threads are allocated. No exact-instance benchmark is claimed.
• The custom license defines a USD 10 million annual-revenue threshold and conditions commercial rights on not exceeding it; commercial use by entities exceeding it is not licensed under this agreement. Read the full license for exact scope, redistribution and qualified nonprofit terms.
• Loopback and a service key do not isolate the service from privileged local users. A shared/public service needs separately designed TLS, authentication and operations.
• The full GGUF is 14,415,473,952 bytes. Full process memory also includes attention cache, convolution state, buffers and OS; exact working-memory requirement and fit remain unknown. The 24B total weights are not reduced to 2.3B because only a subset is active per token.
• The model card’s detailed table gives 24B total parameters and 2.3B active per token, while its introduction rounds active count to 2B. Config declares 64 experts, four selected per token, 40 layers (30 convolution / 10 attention). Do not transfer dense sibling cache estimates or speed.
• The card states 32768 context while pinned config max_position_embeddings is 128000. This source discrepancy is unresolved. The recipe uses only 4096 combined prompt and output tokens and does not endorse either larger limit.
• This is an instruct model without reasoning traces. Publisher recommends summarization, local RAG, customer support and agentic uses, and does not recommend coding. This recipe verifies text summarization only; it does not configure native tool parsing, retrieval, constrained JSON or autonomous actions.
• The publisher’s consumer 32 GB statement and AMD/H100 benchmarks do not establish fit or speed on this exact Intel/AWS allocation. No OpenTools execution, peak-memory, throughput or latency measurement occurred.

Download or checksum failure

Verify exact uppercase filename and pinned URL; remove only the incomplete dedicated model file and retry. Do not run a mismatching file.

Architecture unsupported or build fails

Confirm exact pinned source commit, Ubuntu compiler/CMake dependencies and GGUF. Do not substitute an older packaged llama.cpp silently.

Out of memory or context overflow

Stop server, free/provision RAM or reduce context and restart; shorten input/output within configured total. Allocation is not a measured guarantee; record any changed configuration.

Connection refused or HTTP 401

Check foreground server logs, 8080 binding, matching private key and Bearer header. Cloud client must have working SSH tunnel to the correct VM; do not expose the port to repair a tunnel.

MoE loader fails or summarization is wrong/truncated

Confirm full file SHA-256 and pinned llama.cpp checkout with lfm2moe 40-layer support. Do not replace with dense sibling files. Keep source passage/prompt plus output within 4096 tokens; inspect finish_reason and reduce input if needed. Validate facts and task suitability; this setup does not guarantee accuracy.

Stop the model

1. 1. Stop the service

   Press Ctrl+C in server terminal. Unset key in each shell. Close the SSH tunnel with Ctrl+C. Stopping the process alone does not stop EC2 billing.

   CommandCopy command

   unset LFM2_24B_SERVICE_KEY

   Expected result: Server stops and localhost:8080 stops serving.

Uninstall

1. 1. Remove dedicated model, build and credential files

   Back up wanted logs/files first. Remove only this walkthrough directory after confirming its path. Shared apt packages can remain; do not remove other workloads.

   CommandCopy command

   rm -r -- "$HOME/lfm2-24b-hosting"

   Expected result: Dedicated files and local service credential removed; shared system dependencies remain.

Shut down cloud resources

1. 1. End cloud billing resources

   In EC2 us-east-1, stop for later reuse or terminate this exact VM. Review Delete on termination for its root volume. Delete unwanted detached EBS volumes, snapshots and reserved/Elastic/public IP resources. Stopped VM storage and retained resources may still bill. Check Bills/Cost Explorer later for final usage.

   Expected result: Exact instance is stopped/terminated as intended; no unwanted attached/retained billed resources remain. Instance compute stops only after its lifecycle transition, not when llama-server stops.

Related guides

• https://opentools.ai/llms/lfm2-24b-a2b