Needle

Exact model and requirements

Artifact:

Cactus Compute / Original Needle nominal 26M full native checkpoint · checkpoint Cactus-Compute/needle

Publisher revision:

ad6574affab1f8f7561c8a8bafc8ff050eb14b06

Artifact digest:

sha256:40a32e91d1d4197bf15ba559b74f6727c342dc8746918742fc7d8e2c1f18df40

Download size:

0.05 GiB

Memory requirement (unknown):

Unknown — Native JAX BF16 process, JIT, full decoder recomputation, loading buffers and OS peak are unmeasured.

System RAM / disk:

Unknown / Unknown

Placement:

single device. One full original Needle model and one prediction at a time on CPU. Full original encoder, decoder, embedding and all native parameters use one CPU host. No GPU, disk offload or model subset.

KV cache:

No persistent autoregressive KV cache; native BF16 attention and full decoder-buffer recomputation

Workload:

1,024 token context; Unknown; Unknown; concurrency 1; batch 1; cache cold; repetitions Unknown.

Tokenizer:

Original native 8192 SentencePiece BPE and explicit tools separator; no chat template · Original model pin; SentencePiece 0.2.0

Sampling / reasoning:

Exact native greedy argmax and native partial tool-name/key constraints with the original constraint-state update order. The reader rejects fallback warnings and missing/off-trie constraints before a fallback token is chosen; this stronger boundary is not a native full-JSON guarantee. Raw logits must be finite before constraint masking. Single-shot original tool prediction, not conversational chat or tool execution. contextTokens 1024 is the encoder allocation only. The separate decoder allocation is 512 positions including initial EOS and at most 511 decisions; require actual terminal EOS and strict inert tool JSON. These source defaults are not native context ceilings; catalog context/output maxima remain unknown. Native architecture, all model parameters and tokenizer are preserved.

Runtime prerequisites:

Ubuntu 24.04 x86_64; See prerequisites above.

Official artifact · License

Exact original model card declares MIT; historical publisher source includes full MIT grant, Cactus Compute copyright, notice condition and warranty disclaimer. Preserve both card and license/notices and identify reader-authored modifications. The standalone historical model LICENSE path returned 404; this does not negate the explicit card grant. Current Needle3 Apache terms are not substituted. No endorsement, action authorization, task suitability or legal clearance is claimed.

• native/needle/__init__.py · 0 GiB · revision b4a4d5bcd3c24d79a42a82357932b256b6786bc2
• native/needle/model/__init__.py · 0 GiB · revision b4a4d5bcd3c24d79a42a82357932b256b6786bc2
• native/needle/dataset/__init__.py · 0 GiB · revision b4a4d5bcd3c24d79a42a82357932b256b6786bc2
• native/needle/model/architecture.py · 0 GiB · revision b4a4d5bcd3c24d79a42a82357932b256b6786bc2
• native/needle/model/run.py · 0 GiB · revision b4a4d5bcd3c24d79a42a82357932b256b6786bc2
• native/needle/model/quantize.py · 0 GiB · revision b4a4d5bcd3c24d79a42a82357932b256b6786bc2
• native/needle/model/export.py · 0 GiB · revision b4a4d5bcd3c24d79a42a82357932b256b6786bc2
• native/needle/model/constrained.py · 0 GiB · revision b4a4d5bcd3c24d79a42a82357932b256b6786bc2
• native/needle/dataset/dataset.py · 0 GiB · revision b4a4d5bcd3c24d79a42a82357932b256b6786bc2
• native/needle/dataset/tokenizer.py · 0 GiB · revision b4a4d5bcd3c24d79a42a82357932b256b6786bc2
• native/LICENSE · 0 GiB · revision b4a4d5bcd3c24d79a42a82357932b256b6786bc2
• needle.model · 0 GiB · revision ad6574affab1f8f7561c8a8bafc8ff050eb14b06
• model-card.md · 0 GiB · revision ad6574affab1f8f7561c8a8bafc8ff050eb14b06

• The native pickle is 52,633,098 serialized bytes, not a process-memory measurement. Actual embedded parameter count/dtypes are verified by the reader; the 26M label is publisher nominal wording.
• 32 GiB RAM and 40 GiB disk are planning allocations. Four logical CPUs are selected by affinity; JAX internal thread count, actual memory fit, startup/JIT time and speed remain unknown.

Install the runtime

1. 1. Prepare the dedicated CPU workspace

   Use Ubuntu 24.04 native Python 3.12. Install ordinary packages only; the original general setup script is not run and no W&B/Gemini credentials are needed for this inference scope.

   CommandCopy command

   set -eu
   sudo apt-get update
   sudo apt-get install -y python3.12 python3.12-venv ca-certificates openssl curl util-linux
   mkdir -p "$HOME/needle26m-prediction"
   cd "$HOME/needle26m-prediction"
   umask 077
   python3.12 -m venv .venv
   .venv/bin/python -c 'import sys; assert sys.version_info[:2] == (3,12)'

   Expected result: Dedicated private workspace and Python 3.12 environment created.

2. 2. Install the source-qualified native CPU dependencies

   The exact CPU JAXlib wheel has a published release-metadata digest. Pin compatible JAX/Flax and dependency branches. Other transitive packages are resolved by pip, checked and recorded; this is not a full transitive hash lock or executed compatibility proof. Do not install CUDA/TPU extras or run native model code in this step.

   CommandCopy command

   set -eu
   cd "$HOME/needle26m-prediction"
   .venv/bin/python -m pip install 'https://files.pythonhosted.org/packages/ba/e8/71c2555431edb5dd115cf86a7b599aa7e1be26728d89ae59aa11251d299c/jaxlib-0.6.2-cp312-cp312-manylinux2014_x86_64.whl#sha256=f1dd09b481a93c1d4c750013f467f74194493ba7bd29fcd4d1cec16e3a214f65' jax==0.6.2 flax==0.10.6 optax==0.2.5 orbax-checkpoint==0.11.13 chex==0.1.89 numpy==2.2.6 scipy==1.15.3 ml-dtypes==0.5.1 datasets==3.6.0 sentencepiece==0.2.0 huggingface-hub==0.36.0
   .venv/bin/python -m pip check
   .venv/bin/python -m pip freeze > reader-environment.txt

   Expected result: Pip checks pass and actual package versions are recorded; no native model was imported by the author.

Download the model

1. 1. Write the complete native inference asset manifest

   Every imported original source file and the complete native checkpoint/tokenizer have exact bytes and digests. The reader tree retains the full original model and inference import closure; training and UI are not invoked. Fixed publisher model-card and MIT notice files are retained.

   CommandCopy command

   set -eu
   cd "$HOME/needle26m-prediction"
   cat > assets.json <<'ASSETS'
   [{"path":"native/needle/__init__.py","bytes":534,"sha256":"dd01263c09a894d937da008ec52d8180014b17598c8ae30ea7e8403aca9b8182","url":"https://raw.githubusercontent.com/cactus-compute/needle/b4a4d5bcd3c24d79a42a82357932b256b6786bc2/needle/__init__.py"},{"path":"native/needle/model/__init__.py","bytes":349,"sha256":"eb4c5e695a862475570bc0506b65f4296d80cec3d8a8e0af5404e4cd39efb352","url":"https://raw.githubusercontent.com/cactus-compute/needle/b4a4d5bcd3c24d79a42a82357932b256b6786bc2/needle/model/__init__.py"},{"path":"native/needle/dataset/__init__.py","bytes":86,"sha256":"c343eac9fcdf8da4d5878fac8b5ff8976c0259a136e53632ef7a764682fcea92","url":"https://raw.githubusercontent.com/cactus-compute/needle/b4a4d5bcd3c24d79a42a82357932b256b6786bc2/needle/dataset/__init__.py"},{"path":"native/needle/model/architecture.py","bytes":20317,"sha256":"f65f22d9e4db1f7ff51480d955160a1acb92ff5b671f6fdd9d79996ff6f7341a","url":"https://raw.githubusercontent.com/cactus-compute/needle/b4a4d5bcd3c24d79a42a82357932b256b6786bc2/needle/model/architecture.py"},{"path":"native/needle/model/run.py","bytes":12716,"sha256":"bb6a748502d0b00c3e1e2b8ef78440d86e6a6975780eb924ad8e3180de6d3402","url":"https://raw.githubusercontent.com/cactus-compute/needle/b4a4d5bcd3c24d79a42a82357932b256b6786bc2/needle/model/run.py"},{"path":"native/needle/model/quantize.py","bytes":2283,"sha256":"0d048a36a29d253fc9fb41facac9e33a55884773c9c5b48e33fdb9d0e460a710","url":"https://raw.githubusercontent.com/cactus-compute/needle/b4a4d5bcd3c24d79a42a82357932b256b6786bc2/needle/model/quantize.py"},{"path":"native/needle/model/export.py","bytes":5348,"sha256":"e9783cbbdebf3f80342fb64c5303409e18b992b2da43e9a518bcfc10c057910d","url":"https://raw.githubusercontent.com/cactus-compute/needle/b4a4d5bcd3c24d79a42a82357932b256b6786bc2/needle/model/export.py"},{"path":"native/needle/model/constrained.py","bytes":13859,"sha256":"bbee312f5758d1d12543d43ae5b0725063971a92a45018512d166c716a7bbb20","url":"https://raw.githubusercontent.com/cactus-compute/needle/b4a4d5bcd3c24d79a42a82357932b256b6786bc2/needle/model/constrained.py"},{"path":"native/needle/dataset/dataset.py","bytes":37086,"sha256":"fac18c264a2a1cec62e62b47d9cbdc3a83e7ef8383fa7e01fc01d1a840e9c381","url":"https://raw.githubusercontent.com/cactus-compute/needle/b4a4d5bcd3c24d79a42a82357932b256b6786bc2/needle/dataset/dataset.py"},{"path":"native/needle/dataset/tokenizer.py","bytes":5241,"sha256":"fc621e6f78f1b4e35aa7788146696846c01e4e46eabc987fc3cac35fb2437b33","url":"https://raw.githubusercontent.com/cactus-compute/needle/b4a4d5bcd3c24d79a42a82357932b256b6786bc2/needle/dataset/tokenizer.py"},{"path":"native/LICENSE","bytes":1071,"sha256":"40150e30169beeb87130c9f66ad56302432788d931eec1e4ce80875d838df855","url":"https://raw.githubusercontent.com/cactus-compute/needle/b4a4d5bcd3c24d79a42a82357932b256b6786bc2/LICENSE"},{"path":"needle.pkl","bytes":52633098,"sha256":"40a32e91d1d4197bf15ba559b74f6727c342dc8746918742fc7d8e2c1f18df40","url":"https://huggingface.co/Cactus-Compute/needle/resolve/ad6574affab1f8f7561c8a8bafc8ff050eb14b06/needle.pkl"},{"path":"needle.model","bytes":124960,"sha256":"0823f5b9133c68a8140addc5d7a425fa9119c4c8cb4a550363b4bffa4ba1c8c7","url":"https://huggingface.co/Cactus-Compute/needle/resolve/ad6574affab1f8f7561c8a8bafc8ff050eb14b06/tokenizer/needle.model"},{"path":"model-card.md","bytes":5681,"sha256":"8141d596a2e4a40e06fdedc68a87adaadd0231e5e728f5bd87ed622477dc7e9d","url":"https://huggingface.co/Cactus-Compute/needle/resolve/ad6574affab1f8f7561c8a8bafc8ff050eb14b06/README.md"}]
   ASSETS

   Expected result: Exact source/checkpoint/tokenizer/license manifest written.

2. 2. Download and stream-verify the original assets

   Use a fresh dedicated workspace. Downloads are exclusive and fail fast on access, byte-size or hash mismatch. All pickle bytes must pass before any deserialization or native import. Do not use an alternate artifact or continue with a partial download.

   CommandCopy command

   set -eu
   cd "$HOME/needle26m-prediction"
   .venv/bin/python - <<'PYDOWNLOAD'
   import json,pathlib,urllib.request,hashlib
   for x in json.load(open('assets.json')):
    p=pathlib.Path(x['path']);p.parent.mkdir(parents=True,exist_ok=True)
    h=hashlib.sha256();n=0
    with urllib.request.urlopen(x['url'],timeout=120) as r,p.open('xb') as f:
     while b:=r.read(1024*1024):
      n+=len(b)
      if n>x['bytes']:raise ValueError('Asset exceeds pinned size')
      h.update(b);f.write(b)
    if n!=x['bytes'] or h.hexdigest()!=x['sha256']:raise ValueError('Pinned asset mismatch: '+x['path'])
    print('Verified',x['path'])
   PYDOWNLOAD

   Expected result: Every native source, model, tokenizer and license asset reports verified. The author fetched metadata/source text only, not these model/tokenizer bodies.

Configure

1. 1. Create a private key

   Keep this key local to the reader. Never print it or put it in a URL. Both health and prediction requests require it.

   CommandCopy command

   set -eu
   cd "$HOME/needle26m-prediction"
   umask 077
   .venv/bin/python - <<'PYKEY'
   import pathlib,secrets
   with pathlib.Path('.api-key').open('x') as f:f.write(secrets.token_hex(32)+'\n')
   PYKEY

   Expected result: Private key created without printing it.

2. 2. Write strict framing, schema and native-unit guards

   Reader-authored source preserves native model decisions and records actual EOS/counts. It rejects input truncation, nonfinite raw logits, exhausted decoder buffers, constraint fallbacks and invalid fixed-tool output. It contains no tool executor. Review the complete source and trusted-pickle boundary before starting.

   CommandCopy command

   cd "$HOME/needle26m-prediction"
   cat > reader-prediction-server.py <<'PYSERVER'
   """Reader-authored protocol adapter for original Needle; no native imports on import."""
   import hashlib,hmac,json,logging,os,pathlib,re,socket
   from collections.abc import Mapping
   from http.server import BaseHTTPRequestHandler,HTTPServer
   ENCODER=1024
   DECODER=512
   PORT=8999
   ASSET_MANIFEST_SHA256='3617a8a288d561f6795a409d8f60d149a6dd594f470e171ab03d7c755e3ffe0a'
   TOOLS=[{'name':'set_timer','parameters':{'minutes':{'type':'integer'}}},{'name':'set_light','parameters':{'room':{'type':'string'},'enabled':{'type':'boolean'}}}]
   TOOLS_JSON=json.dumps(TOOLS,separators=(',',':'))
   
   def sha256(path):
    h=hashlib.sha256()
    with open(path,'rb') as f:
     for b in iter(lambda:f.read(1024*1024),b''):h.update(b)
    return h.hexdigest()
   
   def verify_assets(root,entries):
    if not isinstance(entries,list) or hashlib.sha256(json.dumps(entries,sort_keys=True,separators=(',',':'),allow_nan=False).encode()).hexdigest()!=ASSET_MANIFEST_SHA256:raise ValueError('Complete pinned manifest identity required')
    by_path={e['path']:e for e in entries}
    for path,size,digest in [('needle.pkl',52633098,'40a32e91d1d4197bf15ba559b74f6727c342dc8746918742fc7d8e2c1f18df40'),('needle.model',124960,'0823f5b9133c68a8140addc5d7a425fa9119c4c8cb4a550363b4bffa4ba1c8c7')]:
     if path not in by_path or by_path[path]['bytes']!=size or by_path[path]['sha256']!=digest:raise ValueError('Required original checkpoint/tokenizer identity missing')
    for x in entries:
     p=root/x['path']
     if not p.is_file() or (x['bytes'] is not None and p.stat().st_size!=x['bytes']) or sha256(p)!=x['sha256']:raise ValueError('Pinned asset mismatch: '+x['path'])
   
   def unique(pairs):
    d={}
    for k,v in pairs:
     if k in d:raise ValueError('Duplicate JSON key')
     d[k]=v
    return d
   
   def strict_json(text):
    return json.loads(text,object_pairs_hook=unique,parse_constant=lambda v:(_ for _ in ()).throw(ValueError('Nonfinite JSON')))
   
   def parse_request(body):
    d=strict_json(body.decode('utf-8'))
    if not isinstance(d,dict) or set(d)!={'query'} or not isinstance(d['query'],str) or not d['query'].strip() or len(d['query'].encode())>4096:raise ValueError('Only a bounded query is accepted')
    if '<' in d['query'] or '>' in d['query']:raise ValueError('Native token controls are reserved')
    return d['query']
   
   def framing(headers):
    xs=headers.get_all('Content-Length',[])
    if len(xs)!=1 or not re.fullmatch('[0-9]+',xs[0]) or headers.get('Transfer-Encoding') is not None:raise ValueError('Strict content length required')
    n=int(xs[0])
    if not 1<=n<=8192:raise ValueError('Body bound exceeded')
    return n
   
   def check_config(c):
    expected={'vocab_size':8192,'d_model':512,'num_heads':8,'num_kv_heads':4,'num_encoder_layers':12,'num_decoder_layers':8,'no_feedforward':True,'dtype':'bfloat16','pad_token_id':0}
    if not isinstance(c,dict) or any(c.get(k)!=v for k,v in expected.items()):raise ValueError('Original checkpoint architecture mismatch')
   
   def flattened(tree,prefix=()):
    if isinstance(tree,Mapping):
     out={}
     for k,v in tree.items():
      if not isinstance(k,str):raise ValueError('Parameter key must be a string')
      out.update(flattened(v,prefix+(k,)))
     return out
    return {prefix:tree}
   
   def check_shapes(actual,expected):
    a=flattened(actual);e=flattened(expected)
    if not a or set(a)!=set(e) or any(tuple(a[k].shape)!=tuple(e[k].shape) for k in a):raise ValueError('Missing, unexpected or mismatched native parameter')
    return sum(int(v.size) for v in a.values())
   
   def untruncated_input(query_ids,tool_ids,native_ids):
    full=query_ids+[5]+tool_ids
    if not query_ids or len(query_ids)>ENCODER-2 or len(full)>ENCODER or full!=native_ids:raise ValueError('Encoder budget exceeded or native truncation detected')
    return full
   
   def validate_calls(text):
    x=strict_json(text)
    if not isinstance(x,list) or len(x)>4:raise ValueError('Expected at most four predicted calls')
    for call in x:
     if not isinstance(call,dict) or set(call)!={'name','arguments'} or not isinstance(call['arguments'],dict):raise ValueError('Malformed predicted call')
     args=call['arguments']
     if call['name']=='set_timer':
      if set(args)!={'minutes'} or type(args['minutes']) is not int or not 1<=args['minutes']<=120:raise ValueError('Invalid timer prediction')
     elif call['name']=='set_light':
      if set(args)!={'room','enabled'} or args['room'] not in ('kitchen','bedroom','study') or type(args['enabled']) is not bool:raise ValueError('Invalid light prediction')
     else:raise ValueError('Unknown predicted tool')
    return x
   
   def terminal_decisions(ids,eos=1):
    if not 1<=len(ids)<=DECODER-1 or ids[-1]!=eos or eos in ids[:-1]:raise ValueError('No native terminal EOS within decoder allocation')
    return ids[:-1]
   
   class FailConstraintFallback(logging.Handler):
    def emit(self,record):raise ValueError('Native constraint fallback rejected before choosing a token')
   
   PYSERVER

   Expected result: Reader source written; no model loaded until the next step.

3. 3. Write the checked original loader and native decoder adapter

   Reader-authored source preserves native model decisions and records actual EOS/counts. It rejects input truncation, nonfinite raw logits, exhausted decoder buffers, constraint fallbacks and invalid fixed-tool output. It contains no tool executor. Review the complete source and trusted-pickle boundary before starting.

   CommandCopy command

   cd "$HOME/needle26m-prediction"
   cat >> reader-prediction-server.py <<'PYSERVER'
   class Engine:
    def __init__(self,root):
     verify_assets(root,json.loads((root/'assets.json').read_text()))
     os.environ.update(JAX_PLATFORMS='cpu',HF_HUB_OFFLINE='1',HF_DATASETS_OFFLINE='1',HF_HUB_DISABLE_TELEMETRY='1')
     import sys
     sys.path.insert(0,str(root/'native'))
     import jax,jax.numpy as jnp,numpy as np,pickle
     from needle.model.architecture import SimpleAttentionNetwork,TransformerConfig,make_padding_mask
     from needle.dataset.tokenizer import NeedleTokenizer
     from needle.model.run import _get_decode_fn,_build_encoder_input,normalize_tools
     from needle.model.constrained import build_constrained_decoder
     if any(d.platform!='cpu' for d in jax.devices()):raise ValueError('Non-CPU backend selected')
     # The exact publisher pickle is trusted only after full size/SHA checks above.
     with (root/'needle.pkl').open('rb') as f:data=pickle.load(f)
     if not isinstance(data,dict) or set(data)!={'params','config'}:raise ValueError('Unknown native checkpoint envelope')
     check_config(data['config'])
     valid=set(TransformerConfig.__dataclass_fields__)
     if set(data['config'])-valid:raise ValueError('Unknown embedded configuration field')
     config=TransformerConfig(**data['config']);model=SimpleAttentionNetwork(config)
     dummy=jnp.array([[2]],dtype=jnp.int32);target=jnp.array([[1]],dtype=jnp.int32)
     abstract=jax.eval_shape(lambda:model.init(jax.random.PRNGKey(0),dummy,target,method=model.init_all))['params']
     count=check_shapes(data['params'],abstract)
     for v in flattened(data['params']).values():
      if not isinstance(v,np.ndarray) or (v.dtype.kind!='f' and str(v.dtype)!='bfloat16') or not np.isfinite(v).all():raise ValueError('Invalid native floating parameter')
     # Exact native load_checkpoint conversion: full parameter tree to bfloat16.
     params=jax.tree.map(lambda x:jnp.array(x,dtype=jnp.bfloat16),data['params'])
     tokenizer=NeedleTokenizer(str(root/'needle.model'))
     expected=['<pad>','</s>','<s>','<unk>','<tool_call>','<tools>']
     if tokenizer.vocab_size!=8192 or [tokenizer.sp.IdToPiece(i) for i in range(6)]!=expected:raise ValueError('Original SentencePiece controls mismatch')
     normalized,names=normalize_tools(TOOLS_JSON)
     if normalized!=TOOLS_JSON or any(k!=v for k,v in names.items()):raise ValueError('Fixed tool normalization changed')
     self.jax=jax;self.jnp=jnp;self.np=np;self.model=model;self.params=params;self.tokenizer=tokenizer
     self.padding=make_padding_mask;self.build_input=_build_encoder_input;self.decoder_factory=build_constrained_decoder;self.decode=_get_decode_fn(model,DECODER);self.parameter_elements=count
     self.constraint_logger=logging.getLogger('needle.model.constrained');self.constraint_logger.setLevel(logging.WARNING);self.constraint_logger.addHandler(FailConstraintFallback())
    def predict(self,query):
     t=self.tokenizer;jnp=self.jnp;np=self.np
     ids=untruncated_input(t.encode(query),t.encode(TOOLS_JSON),self.build_input(t,query,TOOLS_JSON,ENCODER))
     enc=jnp.array([ids],dtype=jnp.int32)
     encoder_out,enc_mask=self.model.apply({'params':self.params},enc,src_mask=self.padding(enc,0),method='encode')
     buffer=jnp.full((1,DECODER),0,dtype=jnp.int32).at[0,0].set(1)
     decoder=self.decoder_factory([TOOLS_JSON],t)
     logits=self.decode(self.params,buffer,encoder_out,enc_mask)
     decisions=[]
     for i in range(DECODER-1):
      next_logits=logits[0,i]
      if not np.isfinite(np.array(next_logits)).all():raise ValueError('Nonfinite raw native logits')
      if decoder.is_active(0):
       # Native fallback on missing trie is silent; reject it explicitly as well.
       machine=decoder.machines[0];tc=decoder.tool_constraints[0]
       trie=tc.name_trie if machine.state.name=='IN_NAME' else tc.get_param_trie(machine.current_function)
       if trie is None or trie.get_node(machine.constrained_buf) is None:raise ValueError('Missing native constraint trie')
       selected=decoder.constrain_logits(np.array(next_logits),0)
       if not np.isfinite(selected).any() or np.isnan(selected).any():raise ValueError('Invalid constrained logits')
       token=int(np.argmax(selected))
      else:
       if not np.isfinite(np.array(next_logits)).all():raise ValueError('Invalid native logits')
       token=int(jnp.argmax(next_logits))
      decoder.update(0,token);decisions.append(token)
      if token==1:break
      buffer=buffer.at[0,i+1].set(token)
      logits=self.decode(self.params,buffer,encoder_out,enc_mask)
     generated=terminal_decisions(decisions)
     if not generated or generated[0]!=4:raise ValueError('Missing native tool-call marker')
     text=t.decode(generated)
     if not text.startswith('<tool_call>'):raise ValueError('Native marker decode mismatch')
     calls=validate_calls(text[len('<tool_call>'):])
     return {'predictions':calls,'encoder_tokens':len(ids),'decoder_positions_including_initial_eos':len(decisions)+1,'generated_decisions_including_terminal_eos':len(decisions),'complete_by_native_eos':True,'tool_execution_performed':False,'semantic_quality_measured':False,'observed_parameter_elements':self.parameter_elements}
   
   PYSERVER

   Expected result: Reader source written; no model loaded until the next step.

4. 4. Write the serial authenticated HTTP boundary

   Reader-authored source preserves native model decisions and records actual EOS/counts. It rejects input truncation, nonfinite raw logits, exhausted decoder buffers, constraint fallbacks and invalid fixed-tool output. It contains no tool executor. Review the complete source and trusted-pickle boundary before starting.

   CommandCopy command

   cd "$HOME/needle26m-prediction"
   cat >> reader-prediction-server.py <<'PYSERVER'
   def serve(engine,key):
    class Handler(BaseHTTPRequestHandler):
     protocol_version='HTTP/1.0'
     def setup(self):super().setup();self.connection.settimeout(30)
     def log_message(self,*a):pass
     def reply(self,status,value):
      b=json.dumps(value,allow_nan=False).encode()
      if len(b)>65536:status=500;b=b'{"error":"Response bound exceeded"}'
      self.send_response(status);self.send_header('Content-Type','application/json');self.send_header('Content-Length',str(len(b)));self.end_headers();self.wfile.write(b)
     def authorized(self):
      xs=self.headers.get_all('Authorization',[])
      return len(xs)==1 and xs[0].isascii() and hmac.compare_digest(xs[0],'Bearer '+key)
     def do_GET(self):
      if not self.authorized():return self.reply(401,{'error':'Unauthorized'})
      if self.path!='/health':return self.reply(404,{'error':'Not found'})
      self.reply(200,{'ready':True,'encoder_positions':ENCODER,'decoder_positions':DECODER,'max_decoder_decisions':DECODER-1})
     def do_POST(self):
      if not self.authorized():return self.reply(401,{'error':'Unauthorized'})
      if self.path!='/v1/predict':return self.reply(404,{'error':'Not found'})
      try:
       if self.headers.get('Content-Type')!='application/json':raise ValueError('JSON required')
       n=framing(self.headers);body=self.rfile.read(n)
       if len(body)!=n:raise ValueError('Incomplete body')
       query=parse_request(body)
      except (ValueError,UnicodeError,socket.timeout):return self.reply(400,{'error':'Invalid bounded request'})
      try:result=engine.predict(query)
      except Exception:return self.reply(422,{'error':'Native prediction rejected; inspect local failure and do not relax safeguards'})
      self.reply(200,result)
    server=HTTPServer(('127.0.0.1',PORT),Handler)
    try:server.serve_forever()
    finally:server.server_close()
   
   if __name__=='__main__':
    root=pathlib.Path(__file__).resolve().parent;key=(root/'.api-key').read_text().strip()
    if not re.fullmatch('[a-f0-9]{64}',key):raise ValueError('Private key required')
    serve(Engine(root),key)
   PYSERVER

   Expected result: Reader source written; no model loaded until the next step.

Start the model

1. 1. Start the checked native CPU process

   Keep the foreground terminal open. Startup verifies all assets before trusted native imports/pickle loading, validates embedded config and all parameter keys/shapes against native abstract initialization, then tokenizer controls and CPU devices. Four allowed logical CPUs are an affinity bound, not an internal thread-count setting. Native JIT occurs on actual prediction; readiness is not a successful inference or fit proof.

   CommandCopy command

   set -eu
   cd "$HOME/needle26m-prediction"
   CPU_SET=$(.venv/bin/python -c 'import os; c=sorted(os.sched_getaffinity(0)); assert len(c)>=4; print(",".join(map(str,c[:4])))')
   taskset --cpu-list "$CPU_SET" .venv/bin/python reader-prediction-server.py

   Expected result: The process binds loopback only after all startup checks. Actual first-call compilation, process fit and speed must be observed by the reader.

Verify the setup

1. 1. Check native EOS, three inert predictions and authentication

   Use a second terminal on this same host; the virtual-environment interpreter is explicit. Test timer, light and an uncovered joke query. Record whether actual predicted routing/arguments/abstention match the request; do not force an expected semantic answer. A client timeout does not cancel JAX computation.

   CommandCopy command

   set -eu
   cd "$HOME/needle26m-prediction"
   NEEDLE_KEY=$(cat .api-key)
   curl -fsS --max-time 30 -H "Authorization: Bearer $NEEDLE_KEY" http://127.0.0.1:8999/health -o health.json
   .venv/bin/python - <<'PYFIXTURES'
   import json,pathlib
   queries=['Set a timer for five minutes.','Turn off the light in the study.','Tell me a joke.']
   for i,q in enumerate(queries):pathlib.Path('query-'+str(i)+'.json').write_text(json.dumps({'query':q}))
   PYFIXTURES
   for i in 0 1 2; do
    curl -fsS --max-time 1800 -H "Authorization: Bearer $NEEDLE_KEY" -H 'Content-Type: application/json' --data-binary "@query-$i.json" http://127.0.0.1:8999/v1/predict -o "answer-$i.json"
   done
   .venv/bin/python - <<'PYCHECK'
   import json
   h=json.load(open('health.json'));assert h=={'ready':True,'encoder_positions':1024,'decoder_positions':512,'max_decoder_decisions':511}
   for i in range(3):
    r=json.load(open('answer-'+str(i)+'.json'))
    assert r['complete_by_native_eos'] is True and r['tool_execution_performed'] is False and r['semantic_quality_measured'] is False
    assert 1<=r['encoder_tokens']<=1024 and 2<=r['generated_decisions_including_terminal_eos']<=511
    assert r['decoder_positions_including_initial_eos']==r['generated_decisions_including_terminal_eos']+1<=512
    assert type(r['observed_parameter_elements']) is int and r['observed_parameter_elements']>0
    assert isinstance(r['predictions'],list)
    print(i,r)
   print('Inspect actual routing, argument values and no-call behavior. Valid JSON/EOS is not semantic correctness; record any errors. No tool was executed.')
   PYCHECK
   code=$(curl -sS --max-time 30 -o unauthorized.json -w '%{http_code}' http://127.0.0.1:8999/health)
   test "$code" = 401
   unset NEEDLE_KEY

   Expected result: Three actual responses pass native EOS, separate-unit and JSON protocol checks; unauthorized health returns 401. Semantic errors and actual parameter counts are recorded by the reader, not invented here.

Performance evidence

No compatible measurement available. Documented support and estimated memory do not establish response speed.

Limits and troubleshooting

• Exact original publisher Needle nominal 26M native pickle and historical code only. Current Needle2/3, newer HF-format/quantized artifacts, device/GPU, training and embedding workflows remain open; no broad family completion or API equivalence is claimed.
• Context 1,024 denotes the encoder allocation, not a shared chat window or a native ceiling. The decoder separately has 512 positions including initial EOS and at most 511 decisions including terminal EOS. Catalog/native maxima and actual input/output counts remain unknown until execution.
• Original native construction silently slices input; this boundary enforces full query/tools IDs, query at most 1,022, total encoder at most 1,024 and exact native-ID equality. Native partial constraints do not guarantee JSON or quality; the stronger reader boundary rejects their fallback paths before token choice.
• The exact trusted publisher pickle can execute code while deserializing. Reader verifies its full bytes first, then native architecture, full key/shape tree, floating parameters and tokenizer. No agent model/tokenizer binary download, native import, compilation, inference or actual parameter count was performed.
• JAX CPU recomputes the full decoder buffer and uses native BF16 operands. Four logical CPUs are selected by affinity; internal thread count, fit, first-call compilation time and speed are unknown. Source-compatible package pins are not an executed install or full transitive hash lock.
• The serial service has 30-second socket reads and bounded client requests, but no hard inference deadline or cancellation. Client timeout does not stop JAX compute. No public TLS, multitenant resilience, inference SLA, action authorization or semantic quality is claimed.

Checksum, native checkpoint or tokenizer mismatch

Stop and retain the actual failure. Restore the exact original checkpoint, source and tokenizer. Do not ignore missing/extra parameter keys, change the architecture, load random weights or use a newer Needle variant.

Encoder truncation or decoder exhaustion

Reject the request. Keep the full query and fixed tools intact; do not silently trim either. Native terminal EOS is required, not merely a parsed partial JSON string.

Constraint fallback, invalid JSON or unsupported tool prediction

The reader boundary fails closed before a fallback decision or accepts only validated inert predictions. Record the actual failure; never execute model output or weaken validation to force a successful result.

Slow JIT, out of memory or dependency failure

Stop and record the actual CPU, environment and error. Planning capacities and file sizes do not prove runtime fit. Preserve the source-qualified pins; any changed stack requires requalification.

401 or connection refused

Check the private local key and foreground readiness, then loopback or the private SSH boundary. Keep public service ports closed and never print the key.

Stop the model

1. 1. Stop the native prediction process

   Press Ctrl+C in the foreground terminal, unset the key in each client and confirm the loopback endpoint stops. The local JAX model process must exit.

   CommandCopy command

   unset NEEDLE_KEY

   Expected result: The private Python/JAX process exits and the endpoint is unavailable.

Uninstall

1. 1. Remove the dedicated workspace

   Back up wanted private observations and license records. Verify the exact directory before deleting it. Shared Ubuntu packages remain installed.

   CommandCopy command

   rm -r -- "$HOME/needle26m-prediction"

   Expected result: Dedicated native source/model/tokenizer, virtual environment, reader wrapper, results and private key removed.

Exact model and requirements

Artifact:

Cactus Compute / Original Needle nominal 26M full native checkpoint · checkpoint Cactus-Compute/needle

Publisher revision:

ad6574affab1f8f7561c8a8bafc8ff050eb14b06

Artifact digest:

sha256:40a32e91d1d4197bf15ba559b74f6727c342dc8746918742fc7d8e2c1f18df40

Download size:

0.05 GiB

Memory requirement (unknown):

Unknown — Native JAX BF16 process, JIT, full decoder recomputation, loading buffers and OS peak are unmeasured.

System RAM / disk:

Unknown / Unknown

Placement:

single device. One full original Needle model and one prediction at a time on CPU. Full original encoder, decoder, embedding and all native parameters use one CPU host. No GPU, disk offload or model subset.

KV cache:

No persistent autoregressive KV cache; native BF16 attention and full decoder-buffer recomputation

Workload:

1,024 token context; Unknown; Unknown; concurrency 1; batch 1; cache cold; repetitions Unknown.

Tokenizer:

Original native 8192 SentencePiece BPE and explicit tools separator; no chat template · Original model pin; SentencePiece 0.2.0

Sampling / reasoning:

Exact native greedy argmax and native partial tool-name/key constraints with the original constraint-state update order. The reader rejects fallback warnings and missing/off-trie constraints before a fallback token is chosen; this stronger boundary is not a native full-JSON guarantee. Raw logits must be finite before constraint masking. Single-shot original tool prediction, not conversational chat or tool execution. contextTokens 1024 is the encoder allocation only. The separate decoder allocation is 512 positions including initial EOS and at most 511 decisions; require actual terminal EOS and strict inert tool JSON. These source defaults are not native context ceilings; catalog context/output maxima remain unknown. Native architecture, all model parameters and tokenizer are preserved.

Runtime prerequisites:

Ubuntu 24.04 x86_64; See prerequisites above.

Official artifact · License

Exact original model card declares MIT; historical publisher source includes full MIT grant, Cactus Compute copyright, notice condition and warranty disclaimer. Preserve both card and license/notices and identify reader-authored modifications. The standalone historical model LICENSE path returned 404; this does not negate the explicit card grant. Current Needle3 Apache terms are not substituted. No endorsement, action authorization, task suitability or legal clearance is claimed.

• native/needle/__init__.py · 0 GiB · revision b4a4d5bcd3c24d79a42a82357932b256b6786bc2
• native/needle/model/__init__.py · 0 GiB · revision b4a4d5bcd3c24d79a42a82357932b256b6786bc2
• native/needle/dataset/__init__.py · 0 GiB · revision b4a4d5bcd3c24d79a42a82357932b256b6786bc2
• native/needle/model/architecture.py · 0 GiB · revision b4a4d5bcd3c24d79a42a82357932b256b6786bc2
• native/needle/model/run.py · 0 GiB · revision b4a4d5bcd3c24d79a42a82357932b256b6786bc2
• native/needle/model/quantize.py · 0 GiB · revision b4a4d5bcd3c24d79a42a82357932b256b6786bc2
• native/needle/model/export.py · 0 GiB · revision b4a4d5bcd3c24d79a42a82357932b256b6786bc2
• native/needle/model/constrained.py · 0 GiB · revision b4a4d5bcd3c24d79a42a82357932b256b6786bc2
• native/needle/dataset/dataset.py · 0 GiB · revision b4a4d5bcd3c24d79a42a82357932b256b6786bc2
• native/needle/dataset/tokenizer.py · 0 GiB · revision b4a4d5bcd3c24d79a42a82357932b256b6786bc2
• native/LICENSE · 0 GiB · revision b4a4d5bcd3c24d79a42a82357932b256b6786bc2
• needle.model · 0 GiB · revision ad6574affab1f8f7561c8a8bafc8ff050eb14b06
• model-card.md · 0 GiB · revision ad6574affab1f8f7561c8a8bafc8ff050eb14b06

• The native pickle is 52,633,098 serialized bytes, not a process-memory measurement. Actual embedded parameter count/dtypes are verified by the reader; the 26M label is publisher nominal wording.
• 32 GiB RAM and 40 GiB disk are planning allocations. Four logical CPUs are selected by affinity; JAX internal thread count, actual memory fit, startup/JIT time and speed remain unknown.

Install the runtime

1. 1. Prepare the dedicated CPU workspace

   Use Ubuntu 24.04 native Python 3.12. Install ordinary packages only; the original general setup script is not run and no W&B/Gemini credentials are needed for this inference scope.

   CommandCopy command

   set -eu
   sudo apt-get update
   sudo apt-get install -y python3.12 python3.12-venv ca-certificates openssl curl util-linux
   mkdir -p "$HOME/needle26m-prediction"
   cd "$HOME/needle26m-prediction"
   umask 077
   python3.12 -m venv .venv
   .venv/bin/python -c 'import sys; assert sys.version_info[:2] == (3,12)'

   Expected result: Dedicated private workspace and Python 3.12 environment created.

2. 2. Install the source-qualified native CPU dependencies

   The exact CPU JAXlib wheel has a published release-metadata digest. Pin compatible JAX/Flax and dependency branches. Other transitive packages are resolved by pip, checked and recorded; this is not a full transitive hash lock or executed compatibility proof. Do not install CUDA/TPU extras or run native model code in this step.

   CommandCopy command

   set -eu
   cd "$HOME/needle26m-prediction"
   .venv/bin/python -m pip install 'https://files.pythonhosted.org/packages/ba/e8/71c2555431edb5dd115cf86a7b599aa7e1be26728d89ae59aa11251d299c/jaxlib-0.6.2-cp312-cp312-manylinux2014_x86_64.whl#sha256=f1dd09b481a93c1d4c750013f467f74194493ba7bd29fcd4d1cec16e3a214f65' jax==0.6.2 flax==0.10.6 optax==0.2.5 orbax-checkpoint==0.11.13 chex==0.1.89 numpy==2.2.6 scipy==1.15.3 ml-dtypes==0.5.1 datasets==3.6.0 sentencepiece==0.2.0 huggingface-hub==0.36.0
   .venv/bin/python -m pip check
   .venv/bin/python -m pip freeze > reader-environment.txt

   Expected result: Pip checks pass and actual package versions are recorded; no native model was imported by the author.

Download the model

1. 1. Write the complete native inference asset manifest

   Every imported original source file and the complete native checkpoint/tokenizer have exact bytes and digests. The reader tree retains the full original model and inference import closure; training and UI are not invoked. Fixed publisher model-card and MIT notice files are retained.

   CommandCopy command

   set -eu
   cd "$HOME/needle26m-prediction"
   cat > assets.json <<'ASSETS'
   [{"path":"native/needle/__init__.py","bytes":534,"sha256":"dd01263c09a894d937da008ec52d8180014b17598c8ae30ea7e8403aca9b8182","url":"https://raw.githubusercontent.com/cactus-compute/needle/b4a4d5bcd3c24d79a42a82357932b256b6786bc2/needle/__init__.py"},{"path":"native/needle/model/__init__.py","bytes":349,"sha256":"eb4c5e695a862475570bc0506b65f4296d80cec3d8a8e0af5404e4cd39efb352","url":"https://raw.githubusercontent.com/cactus-compute/needle/b4a4d5bcd3c24d79a42a82357932b256b6786bc2/needle/model/__init__.py"},{"path":"native/needle/dataset/__init__.py","bytes":86,"sha256":"c343eac9fcdf8da4d5878fac8b5ff8976c0259a136e53632ef7a764682fcea92","url":"https://raw.githubusercontent.com/cactus-compute/needle/b4a4d5bcd3c24d79a42a82357932b256b6786bc2/needle/dataset/__init__.py"},{"path":"native/needle/model/architecture.py","bytes":20317,"sha256":"f65f22d9e4db1f7ff51480d955160a1acb92ff5b671f6fdd9d79996ff6f7341a","url":"https://raw.githubusercontent.com/cactus-compute/needle/b4a4d5bcd3c24d79a42a82357932b256b6786bc2/needle/model/architecture.py"},{"path":"native/needle/model/run.py","bytes":12716,"sha256":"bb6a748502d0b00c3e1e2b8ef78440d86e6a6975780eb924ad8e3180de6d3402","url":"https://raw.githubusercontent.com/cactus-compute/needle/b4a4d5bcd3c24d79a42a82357932b256b6786bc2/needle/model/run.py"},{"path":"native/needle/model/quantize.py","bytes":2283,"sha256":"0d048a36a29d253fc9fb41facac9e33a55884773c9c5b48e33fdb9d0e460a710","url":"https://raw.githubusercontent.com/cactus-compute/needle/b4a4d5bcd3c24d79a42a82357932b256b6786bc2/needle/model/quantize.py"},{"path":"native/needle/model/export.py","bytes":5348,"sha256":"e9783cbbdebf3f80342fb64c5303409e18b992b2da43e9a518bcfc10c057910d","url":"https://raw.githubusercontent.com/cactus-compute/needle/b4a4d5bcd3c24d79a42a82357932b256b6786bc2/needle/model/export.py"},{"path":"native/needle/model/constrained.py","bytes":13859,"sha256":"bbee312f5758d1d12543d43ae5b0725063971a92a45018512d166c716a7bbb20","url":"https://raw.githubusercontent.com/cactus-compute/needle/b4a4d5bcd3c24d79a42a82357932b256b6786bc2/needle/model/constrained.py"},{"path":"native/needle/dataset/dataset.py","bytes":37086,"sha256":"fac18c264a2a1cec62e62b47d9cbdc3a83e7ef8383fa7e01fc01d1a840e9c381","url":"https://raw.githubusercontent.com/cactus-compute/needle/b4a4d5bcd3c24d79a42a82357932b256b6786bc2/needle/dataset/dataset.py"},{"path":"native/needle/dataset/tokenizer.py","bytes":5241,"sha256":"fc621e6f78f1b4e35aa7788146696846c01e4e46eabc987fc3cac35fb2437b33","url":"https://raw.githubusercontent.com/cactus-compute/needle/b4a4d5bcd3c24d79a42a82357932b256b6786bc2/needle/dataset/tokenizer.py"},{"path":"native/LICENSE","bytes":1071,"sha256":"40150e30169beeb87130c9f66ad56302432788d931eec1e4ce80875d838df855","url":"https://raw.githubusercontent.com/cactus-compute/needle/b4a4d5bcd3c24d79a42a82357932b256b6786bc2/LICENSE"},{"path":"needle.pkl","bytes":52633098,"sha256":"40a32e91d1d4197bf15ba559b74f6727c342dc8746918742fc7d8e2c1f18df40","url":"https://huggingface.co/Cactus-Compute/needle/resolve/ad6574affab1f8f7561c8a8bafc8ff050eb14b06/needle.pkl"},{"path":"needle.model","bytes":124960,"sha256":"0823f5b9133c68a8140addc5d7a425fa9119c4c8cb4a550363b4bffa4ba1c8c7","url":"https://huggingface.co/Cactus-Compute/needle/resolve/ad6574affab1f8f7561c8a8bafc8ff050eb14b06/tokenizer/needle.model"},{"path":"model-card.md","bytes":5681,"sha256":"8141d596a2e4a40e06fdedc68a87adaadd0231e5e728f5bd87ed622477dc7e9d","url":"https://huggingface.co/Cactus-Compute/needle/resolve/ad6574affab1f8f7561c8a8bafc8ff050eb14b06/README.md"}]
   ASSETS

   Expected result: Exact source/checkpoint/tokenizer/license manifest written.

2. 2. Download and stream-verify the original assets

   Use a fresh dedicated workspace. Downloads are exclusive and fail fast on access, byte-size or hash mismatch. All pickle bytes must pass before any deserialization or native import. Do not use an alternate artifact or continue with a partial download.

   CommandCopy command

   set -eu
   cd "$HOME/needle26m-prediction"
   .venv/bin/python - <<'PYDOWNLOAD'
   import json,pathlib,urllib.request,hashlib
   for x in json.load(open('assets.json')):
    p=pathlib.Path(x['path']);p.parent.mkdir(parents=True,exist_ok=True)
    h=hashlib.sha256();n=0
    with urllib.request.urlopen(x['url'],timeout=120) as r,p.open('xb') as f:
     while b:=r.read(1024*1024):
      n+=len(b)
      if n>x['bytes']:raise ValueError('Asset exceeds pinned size')
      h.update(b);f.write(b)
    if n!=x['bytes'] or h.hexdigest()!=x['sha256']:raise ValueError('Pinned asset mismatch: '+x['path'])
    print('Verified',x['path'])
   PYDOWNLOAD

   Expected result: Every native source, model, tokenizer and license asset reports verified. The author fetched metadata/source text only, not these model/tokenizer bodies.

Configure

1. 1. Create a private key

   Keep this key local to the reader. Never print it or put it in a URL. Both health and prediction requests require it.

   CommandCopy command

   set -eu
   cd "$HOME/needle26m-prediction"
   umask 077
   .venv/bin/python - <<'PYKEY'
   import pathlib,secrets
   with pathlib.Path('.api-key').open('x') as f:f.write(secrets.token_hex(32)+'\n')
   PYKEY

   Expected result: Private key created without printing it.

2. 2. Write strict framing, schema and native-unit guards

   Reader-authored source preserves native model decisions and records actual EOS/counts. It rejects input truncation, nonfinite raw logits, exhausted decoder buffers, constraint fallbacks and invalid fixed-tool output. It contains no tool executor. Review the complete source and trusted-pickle boundary before starting.

   CommandCopy command

   cd "$HOME/needle26m-prediction"
   cat > reader-prediction-server.py <<'PYSERVER'
   """Reader-authored protocol adapter for original Needle; no native imports on import."""
   import hashlib,hmac,json,logging,os,pathlib,re,socket
   from collections.abc import Mapping
   from http.server import BaseHTTPRequestHandler,HTTPServer
   ENCODER=1024
   DECODER=512
   PORT=8999
   ASSET_MANIFEST_SHA256='3617a8a288d561f6795a409d8f60d149a6dd594f470e171ab03d7c755e3ffe0a'
   TOOLS=[{'name':'set_timer','parameters':{'minutes':{'type':'integer'}}},{'name':'set_light','parameters':{'room':{'type':'string'},'enabled':{'type':'boolean'}}}]
   TOOLS_JSON=json.dumps(TOOLS,separators=(',',':'))
   
   def sha256(path):
    h=hashlib.sha256()
    with open(path,'rb') as f:
     for b in iter(lambda:f.read(1024*1024),b''):h.update(b)
    return h.hexdigest()
   
   def verify_assets(root,entries):
    if not isinstance(entries,list) or hashlib.sha256(json.dumps(entries,sort_keys=True,separators=(',',':'),allow_nan=False).encode()).hexdigest()!=ASSET_MANIFEST_SHA256:raise ValueError('Complete pinned manifest identity required')
    by_path={e['path']:e for e in entries}
    for path,size,digest in [('needle.pkl',52633098,'40a32e91d1d4197bf15ba559b74f6727c342dc8746918742fc7d8e2c1f18df40'),('needle.model',124960,'0823f5b9133c68a8140addc5d7a425fa9119c4c8cb4a550363b4bffa4ba1c8c7')]:
     if path not in by_path or by_path[path]['bytes']!=size or by_path[path]['sha256']!=digest:raise ValueError('Required original checkpoint/tokenizer identity missing')
    for x in entries:
     p=root/x['path']
     if not p.is_file() or (x['bytes'] is not None and p.stat().st_size!=x['bytes']) or sha256(p)!=x['sha256']:raise ValueError('Pinned asset mismatch: '+x['path'])
   
   def unique(pairs):
    d={}
    for k,v in pairs:
     if k in d:raise ValueError('Duplicate JSON key')
     d[k]=v
    return d
   
   def strict_json(text):
    return json.loads(text,object_pairs_hook=unique,parse_constant=lambda v:(_ for _ in ()).throw(ValueError('Nonfinite JSON')))
   
   def parse_request(body):
    d=strict_json(body.decode('utf-8'))
    if not isinstance(d,dict) or set(d)!={'query'} or not isinstance(d['query'],str) or not d['query'].strip() or len(d['query'].encode())>4096:raise ValueError('Only a bounded query is accepted')
    if '<' in d['query'] or '>' in d['query']:raise ValueError('Native token controls are reserved')
    return d['query']
   
   def framing(headers):
    xs=headers.get_all('Content-Length',[])
    if len(xs)!=1 or not re.fullmatch('[0-9]+',xs[0]) or headers.get('Transfer-Encoding') is not None:raise ValueError('Strict content length required')
    n=int(xs[0])
    if not 1<=n<=8192:raise ValueError('Body bound exceeded')
    return n
   
   def check_config(c):
    expected={'vocab_size':8192,'d_model':512,'num_heads':8,'num_kv_heads':4,'num_encoder_layers':12,'num_decoder_layers':8,'no_feedforward':True,'dtype':'bfloat16','pad_token_id':0}
    if not isinstance(c,dict) or any(c.get(k)!=v for k,v in expected.items()):raise ValueError('Original checkpoint architecture mismatch')
   
   def flattened(tree,prefix=()):
    if isinstance(tree,Mapping):
     out={}
     for k,v in tree.items():
      if not isinstance(k,str):raise ValueError('Parameter key must be a string')
      out.update(flattened(v,prefix+(k,)))
     return out
    return {prefix:tree}
   
   def check_shapes(actual,expected):
    a=flattened(actual);e=flattened(expected)
    if not a or set(a)!=set(e) or any(tuple(a[k].shape)!=tuple(e[k].shape) for k in a):raise ValueError('Missing, unexpected or mismatched native parameter')
    return sum(int(v.size) for v in a.values())
   
   def untruncated_input(query_ids,tool_ids,native_ids):
    full=query_ids+[5]+tool_ids
    if not query_ids or len(query_ids)>ENCODER-2 or len(full)>ENCODER or full!=native_ids:raise ValueError('Encoder budget exceeded or native truncation detected')
    return full
   
   def validate_calls(text):
    x=strict_json(text)
    if not isinstance(x,list) or len(x)>4:raise ValueError('Expected at most four predicted calls')
    for call in x:
     if not isinstance(call,dict) or set(call)!={'name','arguments'} or not isinstance(call['arguments'],dict):raise ValueError('Malformed predicted call')
     args=call['arguments']
     if call['name']=='set_timer':
      if set(args)!={'minutes'} or type(args['minutes']) is not int or not 1<=args['minutes']<=120:raise ValueError('Invalid timer prediction')
     elif call['name']=='set_light':
      if set(args)!={'room','enabled'} or args['room'] not in ('kitchen','bedroom','study') or type(args['enabled']) is not bool:raise ValueError('Invalid light prediction')
     else:raise ValueError('Unknown predicted tool')
    return x
   
   def terminal_decisions(ids,eos=1):
    if not 1<=len(ids)<=DECODER-1 or ids[-1]!=eos or eos in ids[:-1]:raise ValueError('No native terminal EOS within decoder allocation')
    return ids[:-1]
   
   class FailConstraintFallback(logging.Handler):
    def emit(self,record):raise ValueError('Native constraint fallback rejected before choosing a token')
   
   PYSERVER

   Expected result: Reader source written; no model loaded until the next step.

3. 3. Write the checked original loader and native decoder adapter

   Reader-authored source preserves native model decisions and records actual EOS/counts. It rejects input truncation, nonfinite raw logits, exhausted decoder buffers, constraint fallbacks and invalid fixed-tool output. It contains no tool executor. Review the complete source and trusted-pickle boundary before starting.

   CommandCopy command

   cd "$HOME/needle26m-prediction"
   cat >> reader-prediction-server.py <<'PYSERVER'
   class Engine:
    def __init__(self,root):
     verify_assets(root,json.loads((root/'assets.json').read_text()))
     os.environ.update(JAX_PLATFORMS='cpu',HF_HUB_OFFLINE='1',HF_DATASETS_OFFLINE='1',HF_HUB_DISABLE_TELEMETRY='1')
     import sys
     sys.path.insert(0,str(root/'native'))
     import jax,jax.numpy as jnp,numpy as np,pickle
     from needle.model.architecture import SimpleAttentionNetwork,TransformerConfig,make_padding_mask
     from needle.dataset.tokenizer import NeedleTokenizer
     from needle.model.run import _get_decode_fn,_build_encoder_input,normalize_tools
     from needle.model.constrained import build_constrained_decoder
     if any(d.platform!='cpu' for d in jax.devices()):raise ValueError('Non-CPU backend selected')
     # The exact publisher pickle is trusted only after full size/SHA checks above.
     with (root/'needle.pkl').open('rb') as f:data=pickle.load(f)
     if not isinstance(data,dict) or set(data)!={'params','config'}:raise ValueError('Unknown native checkpoint envelope')
     check_config(data['config'])
     valid=set(TransformerConfig.__dataclass_fields__)
     if set(data['config'])-valid:raise ValueError('Unknown embedded configuration field')
     config=TransformerConfig(**data['config']);model=SimpleAttentionNetwork(config)
     dummy=jnp.array([[2]],dtype=jnp.int32);target=jnp.array([[1]],dtype=jnp.int32)
     abstract=jax.eval_shape(lambda:model.init(jax.random.PRNGKey(0),dummy,target,method=model.init_all))['params']
     count=check_shapes(data['params'],abstract)
     for v in flattened(data['params']).values():
      if not isinstance(v,np.ndarray) or (v.dtype.kind!='f' and str(v.dtype)!='bfloat16') or not np.isfinite(v).all():raise ValueError('Invalid native floating parameter')
     # Exact native load_checkpoint conversion: full parameter tree to bfloat16.
     params=jax.tree.map(lambda x:jnp.array(x,dtype=jnp.bfloat16),data['params'])
     tokenizer=NeedleTokenizer(str(root/'needle.model'))
     expected=['<pad>','</s>','<s>','<unk>','<tool_call>','<tools>']
     if tokenizer.vocab_size!=8192 or [tokenizer.sp.IdToPiece(i) for i in range(6)]!=expected:raise ValueError('Original SentencePiece controls mismatch')
     normalized,names=normalize_tools(TOOLS_JSON)
     if normalized!=TOOLS_JSON or any(k!=v for k,v in names.items()):raise ValueError('Fixed tool normalization changed')
     self.jax=jax;self.jnp=jnp;self.np=np;self.model=model;self.params=params;self.tokenizer=tokenizer
     self.padding=make_padding_mask;self.build_input=_build_encoder_input;self.decoder_factory=build_constrained_decoder;self.decode=_get_decode_fn(model,DECODER);self.parameter_elements=count
     self.constraint_logger=logging.getLogger('needle.model.constrained');self.constraint_logger.setLevel(logging.WARNING);self.constraint_logger.addHandler(FailConstraintFallback())
    def predict(self,query):
     t=self.tokenizer;jnp=self.jnp;np=self.np
     ids=untruncated_input(t.encode(query),t.encode(TOOLS_JSON),self.build_input(t,query,TOOLS_JSON,ENCODER))
     enc=jnp.array([ids],dtype=jnp.int32)
     encoder_out,enc_mask=self.model.apply({'params':self.params},enc,src_mask=self.padding(enc,0),method='encode')
     buffer=jnp.full((1,DECODER),0,dtype=jnp.int32).at[0,0].set(1)
     decoder=self.decoder_factory([TOOLS_JSON],t)
     logits=self.decode(self.params,buffer,encoder_out,enc_mask)
     decisions=[]
     for i in range(DECODER-1):
      next_logits=logits[0,i]
      if not np.isfinite(np.array(next_logits)).all():raise ValueError('Nonfinite raw native logits')
      if decoder.is_active(0):
       # Native fallback on missing trie is silent; reject it explicitly as well.
       machine=decoder.machines[0];tc=decoder.tool_constraints[0]
       trie=tc.name_trie if machine.state.name=='IN_NAME' else tc.get_param_trie(machine.current_function)
       if trie is None or trie.get_node(machine.constrained_buf) is None:raise ValueError('Missing native constraint trie')
       selected=decoder.constrain_logits(np.array(next_logits),0)
       if not np.isfinite(selected).any() or np.isnan(selected).any():raise ValueError('Invalid constrained logits')
       token=int(np.argmax(selected))
      else:
       if not np.isfinite(np.array(next_logits)).all():raise ValueError('Invalid native logits')
       token=int(jnp.argmax(next_logits))
      decoder.update(0,token);decisions.append(token)
      if token==1:break
      buffer=buffer.at[0,i+1].set(token)
      logits=self.decode(self.params,buffer,encoder_out,enc_mask)
     generated=terminal_decisions(decisions)
     if not generated or generated[0]!=4:raise ValueError('Missing native tool-call marker')
     text=t.decode(generated)
     if not text.startswith('<tool_call>'):raise ValueError('Native marker decode mismatch')
     calls=validate_calls(text[len('<tool_call>'):])
     return {'predictions':calls,'encoder_tokens':len(ids),'decoder_positions_including_initial_eos':len(decisions)+1,'generated_decisions_including_terminal_eos':len(decisions),'complete_by_native_eos':True,'tool_execution_performed':False,'semantic_quality_measured':False,'observed_parameter_elements':self.parameter_elements}
   
   PYSERVER

   Expected result: Reader source written; no model loaded until the next step.

4. 4. Write the serial authenticated HTTP boundary

   Reader-authored source preserves native model decisions and records actual EOS/counts. It rejects input truncation, nonfinite raw logits, exhausted decoder buffers, constraint fallbacks and invalid fixed-tool output. It contains no tool executor. Review the complete source and trusted-pickle boundary before starting.

   CommandCopy command

   cd "$HOME/needle26m-prediction"
   cat >> reader-prediction-server.py <<'PYSERVER'
   def serve(engine,key):
    class Handler(BaseHTTPRequestHandler):
     protocol_version='HTTP/1.0'
     def setup(self):super().setup();self.connection.settimeout(30)
     def log_message(self,*a):pass
     def reply(self,status,value):
      b=json.dumps(value,allow_nan=False).encode()
      if len(b)>65536:status=500;b=b'{"error":"Response bound exceeded"}'
      self.send_response(status);self.send_header('Content-Type','application/json');self.send_header('Content-Length',str(len(b)));self.end_headers();self.wfile.write(b)
     def authorized(self):
      xs=self.headers.get_all('Authorization',[])
      return len(xs)==1 and xs[0].isascii() and hmac.compare_digest(xs[0],'Bearer '+key)
     def do_GET(self):
      if not self.authorized():return self.reply(401,{'error':'Unauthorized'})
      if self.path!='/health':return self.reply(404,{'error':'Not found'})
      self.reply(200,{'ready':True,'encoder_positions':ENCODER,'decoder_positions':DECODER,'max_decoder_decisions':DECODER-1})
     def do_POST(self):
      if not self.authorized():return self.reply(401,{'error':'Unauthorized'})
      if self.path!='/v1/predict':return self.reply(404,{'error':'Not found'})
      try:
       if self.headers.get('Content-Type')!='application/json':raise ValueError('JSON required')
       n=framing(self.headers);body=self.rfile.read(n)
       if len(body)!=n:raise ValueError('Incomplete body')
       query=parse_request(body)
      except (ValueError,UnicodeError,socket.timeout):return self.reply(400,{'error':'Invalid bounded request'})
      try:result=engine.predict(query)
      except Exception:return self.reply(422,{'error':'Native prediction rejected; inspect local failure and do not relax safeguards'})
      self.reply(200,result)
    server=HTTPServer(('127.0.0.1',PORT),Handler)
    try:server.serve_forever()
    finally:server.server_close()
   
   if __name__=='__main__':
    root=pathlib.Path(__file__).resolve().parent;key=(root/'.api-key').read_text().strip()
    if not re.fullmatch('[a-f0-9]{64}',key):raise ValueError('Private key required')
    serve(Engine(root),key)
   PYSERVER

   Expected result: Reader source written; no model loaded until the next step.

Start the model

1. 1. Start the checked native CPU process

   Keep the foreground terminal open. Startup verifies all assets before trusted native imports/pickle loading, validates embedded config and all parameter keys/shapes against native abstract initialization, then tokenizer controls and CPU devices. Four allowed logical CPUs are an affinity bound, not an internal thread-count setting. Native JIT occurs on actual prediction; readiness is not a successful inference or fit proof.

   CommandCopy command

   set -eu
   cd "$HOME/needle26m-prediction"
   CPU_SET=$(.venv/bin/python -c 'import os; c=sorted(os.sched_getaffinity(0)); assert len(c)>=4; print(",".join(map(str,c[:4])))')
   taskset --cpu-list "$CPU_SET" .venv/bin/python reader-prediction-server.py

   Expected result: The process binds loopback only after all startup checks. Actual first-call compilation, process fit and speed must be observed by the reader.

Verify the setup

1. 1. Check native EOS, three inert predictions and authentication

   Use a second terminal on this same host; the virtual-environment interpreter is explicit. Test timer, light and an uncovered joke query. Record whether actual predicted routing/arguments/abstention match the request; do not force an expected semantic answer. A client timeout does not cancel JAX computation.

   CommandCopy command

   set -eu
   cd "$HOME/needle26m-prediction"
   NEEDLE_KEY=$(cat .api-key)
   curl -fsS --max-time 30 -H "Authorization: Bearer $NEEDLE_KEY" http://127.0.0.1:8999/health -o health.json
   .venv/bin/python - <<'PYFIXTURES'
   import json,pathlib
   queries=['Set a timer for five minutes.','Turn off the light in the study.','Tell me a joke.']
   for i,q in enumerate(queries):pathlib.Path('query-'+str(i)+'.json').write_text(json.dumps({'query':q}))
   PYFIXTURES
   for i in 0 1 2; do
    curl -fsS --max-time 1800 -H "Authorization: Bearer $NEEDLE_KEY" -H 'Content-Type: application/json' --data-binary "@query-$i.json" http://127.0.0.1:8999/v1/predict -o "answer-$i.json"
   done
   .venv/bin/python - <<'PYCHECK'
   import json
   h=json.load(open('health.json'));assert h=={'ready':True,'encoder_positions':1024,'decoder_positions':512,'max_decoder_decisions':511}
   for i in range(3):
    r=json.load(open('answer-'+str(i)+'.json'))
    assert r['complete_by_native_eos'] is True and r['tool_execution_performed'] is False and r['semantic_quality_measured'] is False
    assert 1<=r['encoder_tokens']<=1024 and 2<=r['generated_decisions_including_terminal_eos']<=511
    assert r['decoder_positions_including_initial_eos']==r['generated_decisions_including_terminal_eos']+1<=512
    assert type(r['observed_parameter_elements']) is int and r['observed_parameter_elements']>0
    assert isinstance(r['predictions'],list)
    print(i,r)
   print('Inspect actual routing, argument values and no-call behavior. Valid JSON/EOS is not semantic correctness; record any errors. No tool was executed.')
   PYCHECK
   code=$(curl -sS --max-time 30 -o unauthorized.json -w '%{http_code}' http://127.0.0.1:8999/health)
   test "$code" = 401
   unset NEEDLE_KEY

   Expected result: Three actual responses pass native EOS, separate-unit and JSON protocol checks; unauthorized health returns 401. Semantic errors and actual parameter counts are recorded by the reader, not invented here.

Performance evidence

No compatible measurement available. Documented support and estimated memory do not establish response speed.

Limits and troubleshooting

• Exact original publisher Needle nominal 26M native pickle and historical code only. Current Needle2/3, newer HF-format/quantized artifacts, device/GPU, training and embedding workflows remain open; no broad family completion or API equivalence is claimed.
• Context 1,024 denotes the encoder allocation, not a shared chat window or a native ceiling. The decoder separately has 512 positions including initial EOS and at most 511 decisions including terminal EOS. Catalog/native maxima and actual input/output counts remain unknown until execution.
• Original native construction silently slices input; this boundary enforces full query/tools IDs, query at most 1,022, total encoder at most 1,024 and exact native-ID equality. Native partial constraints do not guarantee JSON or quality; the stronger reader boundary rejects their fallback paths before token choice.
• The exact trusted publisher pickle can execute code while deserializing. Reader verifies its full bytes first, then native architecture, full key/shape tree, floating parameters and tokenizer. No agent model/tokenizer binary download, native import, compilation, inference or actual parameter count was performed.
• JAX CPU recomputes the full decoder buffer and uses native BF16 operands. Four logical CPUs are selected by affinity; internal thread count, fit, first-call compilation time and speed are unknown. Source-compatible package pins are not an executed install or full transitive hash lock.
• The serial service has 30-second socket reads and bounded client requests, but no hard inference deadline or cancellation. Client timeout does not stop JAX compute. No public TLS, multitenant resilience, inference SLA, action authorization or semantic quality is claimed.

Checksum, native checkpoint or tokenizer mismatch

Stop and retain the actual failure. Restore the exact original checkpoint, source and tokenizer. Do not ignore missing/extra parameter keys, change the architecture, load random weights or use a newer Needle variant.

Encoder truncation or decoder exhaustion

Reject the request. Keep the full query and fixed tools intact; do not silently trim either. Native terminal EOS is required, not merely a parsed partial JSON string.

Constraint fallback, invalid JSON or unsupported tool prediction

The reader boundary fails closed before a fallback decision or accepts only validated inert predictions. Record the actual failure; never execute model output or weaken validation to force a successful result.

Slow JIT, out of memory or dependency failure

Stop and record the actual CPU, environment and error. Planning capacities and file sizes do not prove runtime fit. Preserve the source-qualified pins; any changed stack requires requalification.

401 or connection refused

Check the private local key and foreground readiness, then loopback or the private SSH boundary. Keep public service ports closed and never print the key.

Stop the model

1. 1. Stop the native prediction process

   Press Ctrl+C in the foreground terminal, unset the key in each client and confirm the loopback endpoint stops. Close any SSH tunnel; stopping Python alone does not end AWS billing.

   CommandCopy command

   unset NEEDLE_KEY

   Expected result: The private Python/JAX process exits and the endpoint is unavailable.

Uninstall

1. 1. Remove the dedicated workspace

   Back up wanted private observations and license records. Verify the exact directory before deleting it. Shared Ubuntu packages remain installed.

   CommandCopy command

   rm -r -- "$HOME/needle26m-prediction"

   Expected result: Dedicated native source/model/tokenizer, virtual environment, reader wrapper, results and private key removed.

Shut down cloud resources

1. 1. End cloud billing resources

   In EC2 us-east-1, stop for later reuse or terminate this exact VM. Review Delete on termination for its root volume. Delete unwanted detached EBS volumes, snapshots and reserved/Elastic/public IP resources. Stopped VM storage and retained resources may still bill. Check Bills/Cost Explorer later for final usage.

   Expected result: Exact instance is stopped/terminated as intended; no unwanted attached/retained billed resources remain. Instance compute stops only after its lifecycle transition, not when the Python service stops.