LFM2.5-1.2B-Instruct

Exact model and requirements

Artifact:

Liquid AI / LFM2.5-1.2B-Instruct Q4_K_M · checkpoint LiquidAI/LFM2.5-1.2B-Instruct-GGUF

Publisher revision:

8ed288026e23958ad9dfa92d53ed773a8eee7125

Artifact digest:

sha256:b1b3de114215d9507409a662a501a631095a479a419584e8a2ded6304b19b4f5

Download size:

0.68 GiB

Memory requirement (unknown):

Unknown — No exact full-process peak memory evidence for this configuration.

System RAM / disk:

Unknown / Unknown

Placement:

single device. One CPU host; no tensor/pipeline sharding. CPU-only execution; --device none and zero GPU layers. No disk or accelerator offload.

KV cache:

F16 K and V; hybrid convolution state managed separately by llama.cpp

Workload:

4,096 token context; Unknown; Unknown; concurrency 1; batch 1; cache unknown; repetitions Unknown.

Tokenizer:

Embedded LFM2.5 Instruct GGUF tokenizer and chat template · 8ed288026e23958ad9dfa92d53ed773a8eee7125

Sampling / reasoning:

temperature 0.1; top_k 50; repeat_penalty 1.05; top_p 1.0 and min_p 0.0; max output 128 in validation request. Instruct text chat; no separate reasoning parser, draft model or tool executor.

Runtime prerequisites:

Ubuntu 24.04 x86_64; See prerequisites above.

Official artifact · License

The custom license defines a USD 10 million annual-revenue threshold and conditions commercial rights on not exceeding it; commercial use by entities exceeding it is not licensed under this agreement. Read the full license for exact scope, redistribution and qualified nonprofit terms.

• CPU weights, attention cache, convolution state, buffers and OS share system RAM. File size alone cannot prove compatibility.
• Use the stated allocation as a starting resource choice, validate locally, and record actual startup and process peak before making fit claims.

Install the runtime

1. 1. Build the pinned CPU server

   Install build dependencies and compile the exact official source revision. These packages are system shared dependencies; no CUDA/GPU driver is required. GGML_NATIVE=ON selects -march=native for this GNU/Linux x86 build, compiling for the host CPU. Build and run on the same machine; no AVX-512 requirement is imposed on the i5-12400.

   CommandCopy command

   sudo apt-get update
   sudo apt-get install -y build-essential cmake git curl ca-certificates openssl
   mkdir -p "$HOME/lfm25-hosting/models"
   cd "$HOME/lfm25-hosting"
   git clone https://github.com/ggml-org/llama.cpp.git
   cd llama.cpp
   git checkout --detach 620d2225cc436ab0b001e3200e590e3b286ef9ab
   cmake -S . -B build -DCMAKE_BUILD_TYPE=Release -DGGML_METAL=OFF -DGGML_NATIVE=ON
   cmake --build build --config Release -j 4
   ./build/bin/llama-server --version

   Expected result: Build succeeds and server version identifies pinned source. Stop if checkout or build fails.

Download the model

1. 1. Download and verify the exact GGUF

   Use publisher GGUF commit and case-sensitive filename. This post-training Q4_K_M artifact differs from QAD Q4_0. The repository is public and ungated as checked; no provider API subscription is required.

   CommandCopy command

   cd "$HOME/lfm25-hosting"
   curl --fail --location --output models/LFM2.5-1.2B-Instruct-Q4_K_M.gguf 'https://huggingface.co/LiquidAI/LFM2.5-1.2B-Instruct-GGUF/resolve/8ed288026e23958ad9dfa92d53ed773a8eee7125/LFM2.5-1.2B-Instruct-Q4_K_M.gguf'
   printf '%s  %s\n' 'b1b3de114215d9507409a662a501a631095a479a419584e8a2ded6304b19b4f5' 'models/LFM2.5-1.2B-Instruct-Q4_K_M.gguf' | sha256sum --check

   Expected result: Downloaded file reports OK against publisher LFS SHA-256; 730,895,168 bytes. Stop on mismatch.

Configure

1. 1. Create a private service credential

   Generate this only on your own host when following the guide. It is a local service password, not an OpenAI/provider credential. Keep it private and never paste it into tickets. Create it only when absent; preserve an existing key so a repeated setup does not silently rotate access. It is not created during this documentation run.

   CommandCopy command

   cd "$HOME/lfm25-hosting"
   umask 077
   if [ ! -e .api-key ]; then
     openssl rand -hex 32 > .api-key
   fi
   test -s .api-key || { echo 'Existing .api-key is empty; restore it before continuing.'; exit 1; }
   chmod 600 .api-key

   Expected result: A non-empty private .api-key file exists; existing key preserved. No secret is printed.

Start the model

1. 1. Start one CPU-only sequence

   Run in a foreground terminal. Bind loopback, disable accelerator execution, use the embedded GGUF chat template, 4096 total context, F16 K/V cache and one slot. Disable extra top-p/min-p filtering to retain the stated sampling choice. Input token count varies; reserve 128 tokens for the validation output.

   CommandCopy command

   cd "$HOME/lfm25-hosting"
   export LFM25_SERVICE_KEY="$(cat .api-key)"
   ./llama.cpp/build/bin/llama-server -m models/LFM2.5-1.2B-Instruct-Q4_K_M.gguf --alias lfm25-instruct --host 127.0.0.1 --port 8080 --api-key "$LFM25_SERVICE_KEY" --device none -ngl 0 -t 4 -c 4096 -np 1 --cache-type-k f16 --cache-type-v f16 --temp 0.1 --top-k 50 --top-p 1.0 --min-p 0.0 --repeat-penalty 1.05

   Expected result: Model initializes and listener starts. Pass health and authenticated chat checks before relying on the service. Command-line credentials may be visible to privileged/local users; trusted host required.

Verify the setup

1. 1. Check readiness and authenticated text generation

   In another terminal on the server, load the private key without printing it. Health is intentionally public in llama-server; it does not prove authenticated chat works. A non-empty completion demonstrates this request only, not a benchmark or guaranteed correct answer.

   CommandCopy command

   cd "$HOME/lfm25-hosting"
   export LFM25_SERVICE_KEY="$(cat .api-key)"
   curl --fail http://127.0.0.1:8080/health
   curl --fail http://127.0.0.1:8080/v1/chat/completions -H "Authorization: Bearer $LFM25_SERVICE_KEY" -H 'Content-Type: application/json' -d '{"model":"lfm25-instruct","messages":[{"role":"user","content":"Return a JSON object containing the city and country from: Toronto is in Canada."}],"max_tokens":128,"temperature":0.1,"top_k":50,"top_p":1.0,"min_p":0.0,"repeat_penalty":1.05}'
   curl -o /dev/null -s -w '%{http_code}\n' http://127.0.0.1:8080/v1/chat/completions -H 'Content-Type: application/json' -d '{"model":"lfm25-instruct","messages":[{"role":"user","content":"Hello"}]}'

   Expected result: Health returns HTTP 200 when ready; authorized request returns non-empty choices[0].message.content; unauthenticated chat returns HTTP 401. Inspect generated JSON/facts yourself.

Performance evidence

No compatible measurement available. Documented support and estimated memory do not establish response speed.

Limits and troubleshooting

• Documented support only. Exact process peak RAM, full fit, latency and throughput are unknown; the 730,895,168-byte weight file is not total working memory.
• The publisher advertises 32768 context; this recipe configures 4096 total prompt plus output and one active slot. Larger context/concurrency or another quantization is a different configuration.
• Text-only Instruct model; do not substitute Thinking, Base, QAD or multimodal variants. Publisher recommends extraction/RAG/agentic uses and cautions against programming and knowledge-intensive tasks. Tool execution/parsing is not configured here.
• No speed numbers are imported from the old LFM2 desktop table, phone Q4_0, NPU runs or speculative decoding. No OpenTools execution occurred.
• Operating system packages and build tool dependencies resolve when installed. Save tool versions/build logs with any actual execution. Local hardware is Intel Core i5-12400 with four inference threads allocated; no exact-hardware benchmark is claimed.
• The custom license defines a USD 10 million annual-revenue threshold and conditions commercial rights on not exceeding it; commercial use by entities exceeding it is not licensed under this agreement. Read the full license for exact scope, redistribution and qualified nonprofit terms.
• Loopback and a service key do not isolate the service from privileged local users. A shared/public service needs separately designed TLS, authentication and operations.

Download or checksum failure

Verify exact uppercase filename and pinned URL; remove only the incomplete dedicated model file and retry. Do not run a mismatching file.

Architecture unsupported or build fails

Confirm exact pinned source commit, Ubuntu compiler/CMake dependencies and GGUF. Do not substitute an older packaged llama.cpp silently.

Out of memory or context overflow

Stop server, free/provision RAM or reduce context and restart; shorten input/output within configured total. Allocation is not a measured guarantee; record any changed configuration.

Connection refused or HTTP 401

Check foreground server logs, 8080 binding, matching private key and Bearer header. Run the client on this same desktop and keep the loopback listener.

Poor extraction or invalid JSON

Verify Instruct file, sampling and embedded template; simplify input and validate output in the application. The small model can be factually wrong; JSON is not constrained in this request.

Stop the model

1. 1. Stop the service

   Press Ctrl+C in server terminal. Unset key in each shell. Confirm this desktop localhost:8080 no longer serves the model.

   CommandCopy command

   unset LFM25_SERVICE_KEY

   Expected result: Server stops and localhost:8080 stops serving.

Uninstall

1. 1. Remove dedicated model, build and credential files

   Back up wanted logs/files first. Remove only this walkthrough directory after confirming its path. Shared apt packages can remain; do not remove other workloads.

   CommandCopy command

   rm -r -- "$HOME/lfm25-hosting"

   Expected result: Dedicated files and local service credential removed; shared system dependencies remain.

Related guides

• https://opentools.ai/llms/lfm25-12b-instruct

Exact model and requirements

Artifact:

Liquid AI / LFM2.5-1.2B-Instruct Q4_K_M · checkpoint LiquidAI/LFM2.5-1.2B-Instruct-GGUF

Publisher revision:

8ed288026e23958ad9dfa92d53ed773a8eee7125

Artifact digest:

sha256:b1b3de114215d9507409a662a501a631095a479a419584e8a2ded6304b19b4f5

Download size:

0.68 GiB

Memory requirement (unknown):

Unknown — No exact full-process peak memory evidence for this configuration.

System RAM / disk:

Unknown / Unknown

Placement:

single device. One CPU host; no tensor/pipeline sharding. CPU-only execution; --device none and zero GPU layers. No disk or accelerator offload.

KV cache:

F16 K and V; hybrid convolution state managed separately by llama.cpp

Workload:

4,096 token context; Unknown; Unknown; concurrency 1; batch 1; cache unknown; repetitions Unknown.

Tokenizer:

Embedded LFM2.5 Instruct GGUF tokenizer and chat template · 8ed288026e23958ad9dfa92d53ed773a8eee7125

Sampling / reasoning:

temperature 0.1; top_k 50; repeat_penalty 1.05; top_p 1.0 and min_p 0.0; max output 128 in validation request. Instruct text chat; no separate reasoning parser, draft model or tool executor.

Runtime prerequisites:

Ubuntu 24.04 x86_64; See prerequisites above.

Official artifact · License

The custom license defines a USD 10 million annual-revenue threshold and conditions commercial rights on not exceeding it; commercial use by entities exceeding it is not licensed under this agreement. Read the full license for exact scope, redistribution and qualified nonprofit terms.

• CPU weights, attention cache, convolution state, buffers and OS share system RAM. File size alone cannot prove compatibility.
• Use the stated allocation as a starting resource choice, validate locally, and record actual startup and process peak before making fit claims.

Install the runtime

1. 1. Build the pinned CPU server

   Install build dependencies and compile the exact official source revision. These packages are system shared dependencies; no CUDA/GPU driver is required. GGML_NATIVE=ON selects -march=native for this GNU/Linux x86 build, compiling for the host CPU. Build and run on the same machine; the binary uses the instruction set exposed by this c7i.xlarge guest. Do not copy it to a different instance family without rebuilding.

   CommandCopy command

   sudo apt-get update
   sudo apt-get install -y build-essential cmake git curl ca-certificates openssl
   mkdir -p "$HOME/lfm25-hosting/models"
   cd "$HOME/lfm25-hosting"
   git clone https://github.com/ggml-org/llama.cpp.git
   cd llama.cpp
   git checkout --detach 620d2225cc436ab0b001e3200e590e3b286ef9ab
   cmake -S . -B build -DCMAKE_BUILD_TYPE=Release -DGGML_METAL=OFF -DGGML_NATIVE=ON
   cmake --build build --config Release -j 4
   ./build/bin/llama-server --version

   Expected result: Build succeeds and server version identifies pinned source. Stop if checkout or build fails.

Download the model

1. 1. Download and verify the exact GGUF

   Use publisher GGUF commit and case-sensitive filename. This post-training Q4_K_M artifact differs from QAD Q4_0. The repository is public and ungated as checked; no provider API subscription is required.

   CommandCopy command

   cd "$HOME/lfm25-hosting"
   curl --fail --location --output models/LFM2.5-1.2B-Instruct-Q4_K_M.gguf 'https://huggingface.co/LiquidAI/LFM2.5-1.2B-Instruct-GGUF/resolve/8ed288026e23958ad9dfa92d53ed773a8eee7125/LFM2.5-1.2B-Instruct-Q4_K_M.gguf'
   printf '%s  %s\n' 'b1b3de114215d9507409a662a501a631095a479a419584e8a2ded6304b19b4f5' 'models/LFM2.5-1.2B-Instruct-Q4_K_M.gguf' | sha256sum --check

   Expected result: Downloaded file reports OK against publisher LFS SHA-256; 730,895,168 bytes. Stop on mismatch.

Configure

1. 1. Create a private service credential

   Generate this only on your own host when following the guide. It is a local service password, not an OpenAI/provider credential. Keep it private and never paste it into tickets. Create it only when absent; preserve an existing key so a repeated setup does not silently rotate access. It is not created during this documentation run.

   CommandCopy command

   cd "$HOME/lfm25-hosting"
   umask 077
   if [ ! -e .api-key ]; then
     openssl rand -hex 32 > .api-key
   fi
   test -s .api-key || { echo 'Existing .api-key is empty; restore it before continuing.'; exit 1; }
   chmod 600 .api-key

   Expected result: A non-empty private .api-key file exists; existing key preserved. No secret is printed.

Start the model

1. 1. Start one CPU-only sequence

   Run in a foreground terminal. Bind loopback, disable accelerator execution, use the embedded GGUF chat template, 4096 total context, F16 K/V cache and one slot. Disable extra top-p/min-p filtering to retain the stated sampling choice. Input token count varies; reserve 128 tokens for the validation output.

   CommandCopy command

   cd "$HOME/lfm25-hosting"
   export LFM25_SERVICE_KEY="$(cat .api-key)"
   ./llama.cpp/build/bin/llama-server -m models/LFM2.5-1.2B-Instruct-Q4_K_M.gguf --alias lfm25-instruct --host 127.0.0.1 --port 8080 --api-key "$LFM25_SERVICE_KEY" --device none -ngl 0 -t 4 -c 4096 -np 1 --cache-type-k f16 --cache-type-v f16 --temp 0.1 --top-k 50 --top-p 1.0 --min-p 0.0 --repeat-penalty 1.05

   Expected result: Model initializes and listener starts. Pass health and authenticated chat checks before relying on the service. Command-line credentials may be visible to privileged/local users; trusted host required.

Verify the setup

1. 1. Check readiness and authenticated text generation

   In another terminal on the server, load the private key without printing it. Health is intentionally public in llama-server; it does not prove authenticated chat works. A non-empty completion demonstrates this request only, not a benchmark or guaranteed correct answer.

   CommandCopy command

   cd "$HOME/lfm25-hosting"
   export LFM25_SERVICE_KEY="$(cat .api-key)"
   curl --fail http://127.0.0.1:8080/health
   curl --fail http://127.0.0.1:8080/v1/chat/completions -H "Authorization: Bearer $LFM25_SERVICE_KEY" -H 'Content-Type: application/json' -d '{"model":"lfm25-instruct","messages":[{"role":"user","content":"Return a JSON object containing the city and country from: Toronto is in Canada."}],"max_tokens":128,"temperature":0.1,"top_k":50,"top_p":1.0,"min_p":0.0,"repeat_penalty":1.05}'
   curl -o /dev/null -s -w '%{http_code}\n' http://127.0.0.1:8080/v1/chat/completions -H 'Content-Type: application/json' -d '{"model":"lfm25-instruct","messages":[{"role":"user","content":"Hello"}]}'

   Expected result: Health returns HTTP 200 when ready; authorized request returns non-empty choices[0].message.content; unauthenticated chat returns HTTP 401. Inspect generated JSON/facts yourself.

Performance evidence

No compatible measurement available. Documented support and estimated memory do not establish response speed.

Limits and troubleshooting

• Documented support only. Exact process peak RAM, full fit, latency and throughput are unknown; the 730,895,168-byte weight file is not total working memory.
• The publisher advertises 32768 context; this recipe configures 4096 total prompt plus output and one active slot. Larger context/concurrency or another quantization is a different configuration.
• Text-only Instruct model; do not substitute Thinking, Base, QAD or multimodal variants. Publisher recommends extraction/RAG/agentic uses and cautions against programming and knowledge-intensive tasks. Tool execution/parsing is not configured here.
• No speed numbers are imported from the old LFM2 desktop table, phone Q4_0, NPU runs or speculative decoding. No OpenTools execution occurred.
• Operating system packages and build tool dependencies resolve when installed. Save tool versions/build logs with any actual execution. Cloud hardware is AWS c7i.xlarge with four vCPUs and 8 GiB host RAM; four inference threads are allocated. No exact-instance benchmark is claimed.
• The custom license defines a USD 10 million annual-revenue threshold and conditions commercial rights on not exceeding it; commercial use by entities exceeding it is not licensed under this agreement. Read the full license for exact scope, redistribution and qualified nonprofit terms.
• Loopback and a service key do not isolate the service from privileged local users. A shared/public service needs separately designed TLS, authentication and operations.

Download or checksum failure

Verify exact uppercase filename and pinned URL; remove only the incomplete dedicated model file and retry. Do not run a mismatching file.

Architecture unsupported or build fails

Confirm exact pinned source commit, Ubuntu compiler/CMake dependencies and GGUF. Do not substitute an older packaged llama.cpp silently.

Out of memory or context overflow

Stop server, free/provision RAM or reduce context and restart; shorten input/output within configured total. Allocation is not a measured guarantee; record any changed configuration.

Connection refused or HTTP 401

Check foreground server logs, 8080 binding, matching private key and Bearer header. Cloud client must have working SSH tunnel to the correct VM; do not expose the port to repair a tunnel.

Poor extraction or invalid JSON

Verify Instruct file, sampling and embedded template; simplify input and validate output in the application. The small model can be factually wrong; JSON is not constrained in this request.

Stop the model

1. 1. Stop the service

   Press Ctrl+C in server terminal. Unset key in each shell. Close the SSH tunnel with Ctrl+C. Stopping the process alone does not stop EC2 billing.

   CommandCopy command

   unset LFM25_SERVICE_KEY

   Expected result: Server stops and localhost:8080 stops serving.

Uninstall

1. 1. Remove dedicated model, build and credential files

   Back up wanted logs/files first. Remove only this walkthrough directory after confirming its path. Shared apt packages can remain; do not remove other workloads.

   CommandCopy command

   rm -r -- "$HOME/lfm25-hosting"

   Expected result: Dedicated files and local service credential removed; shared system dependencies remain.

Shut down cloud resources

1. 1. End cloud billing resources

   In EC2 us-east-1, stop for later reuse or terminate this exact VM. Review Delete on termination for its root volume. Delete unwanted detached EBS volumes, snapshots and reserved/Elastic/public IP resources. Stopped VM storage and retained resources may still bill. Check Bills/Cost Explorer later for final usage.

   Expected result: Exact instance is stopped/terminated as intended; no unwanted attached/retained billed resources remain. Instance compute stops only after its lifecycle transition, not when llama-server stops.

Related guides

• https://opentools.ai/llms/lfm25-12b-instruct