Meta logo

SAM 2

Segment Anything Modelv2.1Current
byMetaMeta(technology company)
Released July 29, 2024
Context--
CategoryImage Model

About SAM 2

SAM 2, or Segment Anything Model 2, is Meta FAIR’s promptable visual segmentation model for images and videos. The official repository provides inference code, SAM 2.1 checkpoints, example notebooks, training resources, and benchmark tables for SA-V, MOSE, and LVOS video object segmentation tasks.

Capabilities

visionimage segmentationvideo segmentationpromptable segmentationopen weights

Input Modalities

imagevideovisual prompts

Output Modalities

segmentation masksmasklets

Technical Details

API Identifier
facebook/sam2-hiera-large
Category
Image Model

Tags

sam-2segment-anythingimage-segmentationvideo-segmentationmetafairopen-weightscomputer-vision

Run this model yourself

Local and cloud setup for SAM 2, with exact configurations, sources and clear evidence limits.

Find a model for your hardware
Local hardwareDocumented setupSources checked Oct 16, 2026

Segment a fixed image with original SAM 2 on a research CPU desktop

Use the exact facebook/sam2-hiera-large original July 2024 checkpoint and native image predictor in CPU float32. An original RGB geometry fixture and one positive point produce a binary mask and estimated IoU through a private educational endpoint. Native input resize is 1024 × 1024; its configured coarse visual embedding has 4,096 positions, separate from higher-resolution features and prompt/decoder tokens. The catalog version says 2.1, but this guide explicitly uses its exact original-SAM2 API artifact. Fit, speed and mask quality are unmeasured.

Hardware
Intel Core i5-12400 desktop, 4 inference threads, 32 GiB RAM
Runtime
Native SAM2 original image predictor CPU private research harness SAM2 source 2b90b9f5ceec907a1c18123530e92e794ad901a4; torch2.6.0+cpu; torchvision0.21.0+cpu; Python3.12; numpy2.2.3; Pillow11.1.0 · CPU
Model format
PyTorch checkpoint · Native CPU float32 eager inference; no autocast, quantization or compile optimization · Unquantized native checkpoint; reader CPU float32
Workload
4,096 token context · 1 concurrent

Based on source documentation; this setup has not been executed by OpenTools. Memory: unknown. No compatible measurement available.

Read the complete setup guide

Before you start

  • Use an owned Intel Core i5-12400 desktop with Ubuntu 24.04 x86_64, 32 GiB RAM, 40 GiB free disk and a four-thread allocation. These are planning choices, not measured minima.
  • The frozen catalog says version 2.1, but its exact API facebook/sam2-hiera-large maps to the original July 29, 2024 checkpoint. This recipe deliberately uses that exact API artifact and native original config; it does not silently upgrade to SAM2.1 or rewrite catalog metadata.
  • Review Apache2.0 obligations for the selected model/runtime. Use only the original geometric fixture generated by this harness; no third-party images, uploads, external URLs, arbitrary file inputs or model choices are accepted.
  • This is a source-documented private educational image segmentation experiment, not measured quality or a production service. Python 3.12 and the source-supported CPU package versions below have not been installed/model-tested by the author. Save dependency resolution and stop on pip check, state-dictionary or shape errors.

Exact model and requirements

Artifact:
Meta / facebookresearch / SAM2 original Hiera Large; exact frozen API artifact, not SAM2.1 · checkpoint facebook/sam2-hiera-large
Publisher revision:
e6a8e8809b8f1bfa2238b6d080f3d05cc76bd251
Artifact digest:
sha256:7442e4e9b732a508f80e141e7c2913437a3610ee0c77381a66658c3a445df87b
Download size:
0.84 GiB
Memory requirement (unknown):
Unknown — No measured full-process peak for original Hiera Large CPU float32,1024image activations, prompt decoder and wrapper.
System RAM / disk:
Unknown / Unknown
Placement:
single device. One CPU host; no tensor or pipeline sharding. Whole native model, image and prompt handling on CPU; no GPU/disk offload.
KV cache:
Image segmentation does not use autoregressive text K/V caching. Image predictor feature state is reset after each request; video memory is outside this recipe.
Workload:
4,096 token context; 4,096 input tokens; Not applicable; concurrency 1; batch 1; cache cold; repetitions Unknown.
Tokenizer:
Native 1024 × 1024 RGB image embedding, stride16 coarse grid64 × 64; no text tokenizer · 2b90b9f5ceec907a1c18123530e92e794ad901a4
Sampling / reasoning:
No text or audio generation sampling. Fixed original224 × 224 RGB geometry plus positive XY point [64,112], label1. Native ToTensor, square1024resize and ImageNet normalization. multimask_output=False selects one candidate; no hole/sprinkle filling, dynamic stability fallback or video propagation. 4,096 is the configured single-image coarse embedding position count only. Native high-resolution256 × 256 and128 × 128 feature maps, sparse spatial prompts and learned decoder tokens are separate; this is not a global/native maximum or text allowance. Point-label embedding types do not cap the number of possible points. This recipe fixes exactly one point and one image; masks do not supply semantic classes.
Runtime prerequisites:
Ubuntu 24.04 x86_64; See prerequisites above.

Official artifact · License

Actual upstream Apache License 2.0 text is archived, and upstream README expressly includes SAM2 model checkpoints. Retain license and relevant copyright/NOTICE material; observe redistribution/change-notice and patent conditions. No trademark endorsement or legal clearance is granted by this guide. Optional connected-component code has its own archived license; this CPU recipe skips that extension. Demo fonts and media are not included or presumed Apache-licensed. The original fixture is author-created geometry.

  • 897,952,466 bytes is official LFS metadata for this exact native checkpoint, not independently downloaded weight-body proof or full runtime RAM.
  • 32 GiB RAM,40 GiB disk and four threads are planning allocations, not measured minima or fit.1024image features, model buffers, native dependencies and OS add overhead.

One-time setup and hardware

hardware
Unknown — Existing desktop purchase price and incremental hardware cost depend on the reader; unknown.
Existing desktop purchase price and incremental hardware cost depend on the reader; unknown. Tax: unknown.
setup
Unknown — Initial provisioning/build time and labor cost are unknown.
Initial provisioning/build time and labor cost are unknown. Tax: unknown.

Recurring costs

electricity
Unknown — Actual wall power, electricity rate and powered hours are unknown; no TDP-to-wall-power conversion.
Actual wall power, electricity rate and powered hours are unknown; no TDP-to-wall-power conversion. Tax: unknown.
storage
Unknown — Incremental cost of local disk capacity is unknown. The 40 GiB free-disk walkthrough allocation is an author choice, not a measured requirement.
Incremental cost of local disk capacity is unknown. The 40 GiB free-disk walkthrough allocation is an author choice, not a measured requirement. Tax: unknown.
egress
Unknown — Data transfer price, allowances and traffic volume are unknown.
Data transfer price, allowances and traffic volume are unknown. Tax: unknown.
administration
Unknown — Maintenance labor rate and hours are unknown.
Maintenance labor rate and hours are unknown. Tax: unknown.

Install the runtime

  1. 1. Create the native CPU environment

    Use exact official CPU wheels and a pinned native checkout. Explicitly skip the optional CUDA extension. Pin direct dependencies, save transitive/system resolution and stop on pip check failure; this is documented support rather than an executed environment.

    Command
    sudo apt-get update
    sudo apt-get install -y python3.12 python3.12-venv git curl ca-certificates openssl
    mkdir -p "$HOME/sam2-image/models"
    cd "$HOME/sam2-image"
    python3.12 -m venv .venv
    .venv/bin/python -m pip install 'https://download.pytorch.org/whl/cpu/torch-2.6.0%2Bcpu-cp312-cp312-linux_x86_64.whl#sha256=59e78aa0c690f70734e42670036d6b541930b8eabbaa18d94e090abf14cc4d91' 'https://download-r2.pytorch.org/whl/cpu/torchvision-0.21.0%2Bcpu-cp312-cp312-linux_x86_64.whl#sha256=d6874431e678ba107b60a83f255c33f3755f06bad587b1b919aa514ec325dcd8'
    .venv/bin/python -m pip install 'numpy==2.2.3' 'Pillow==11.1.0' 'hydra-core==1.3.2' 'omegaconf==2.3.0' 'antlr4-python3-runtime==4.9.3' 'iopath==0.1.10' 'tqdm==4.67.1'
    git clone https://github.com/facebookresearch/sam2.git native-sam2
    git -C native-sam2 checkout --detach 2b90b9f5ceec907a1c18123530e92e794ad901a4
    SAM2_BUILD_CUDA=0 .venv/bin/python -m pip install -e ./native-sam2
    .venv/bin/python -m pip check
    .venv/bin/python -m pip freeze > resolved-environment.txt

    Expected result: Pinned checkout and hashed CPU wheels resolve; extension build is disabled, pip check passes and versions are saved. No CUDA driver/compiler or GPU runtime is required by this chosen path.

Download the model

  1. 1. Verify the exact original checkpoint and config

    Reader download only: the author fetched metadata/text, not weights. Use exact HF revision for native .pt and exact config from pinned runtime. HF safetensors and HF-native YAML are different artifacts/config revisions and are not substituted.

    Command
    cd "$HOME/sam2-image"
    curl --fail --location --output models/sam2_hiera_large.pt 'https://huggingface.co/facebook/sam2-hiera-large/resolve/e6a8e8809b8f1bfa2238b6d080f3d05cc76bd251/sam2_hiera_large.pt'
    sha256sum --check <<'SHA256'
    7442e4e9b732a508f80e141e7c2913437a3610ee0c77381a66658c3a445df87b  models/sam2_hiera_large.pt
    0ee9f2037fd98489212bb0b78a56f33ac82eb177cf5e52c88be3714cb84eea26  native-sam2/sam2/configs/sam2/sam2_hiera_l.yaml
    c71d239df91726fc519c6eb72d318ec65820627232b2f796219e87dcf35d0ab4  native-sam2/LICENSE
    SHA256

    Expected result: All three exact hashes report OK; stop on mismatch. Local native loader uses weights_only=True and checks state-dictionary keys. No native convenience from_pretrained call floats the model revision.

Configure

  1. 1. Create a stable private service key

    Generate only when absent; preserve existing access. Do not print or share this local service key.

    Command
    cd "$HOME/sam2-image"
    umask 077
    if [ ! -e .api-key ]; then openssl rand -hex 32 > .api-key; fi
    chmod 600 .api-key
    test -s .api-key || { echo "Restore empty service key before continuing"; exit 1; }

    Expected result: Private existing key is preserved; server requires 64 lowercase hexadecimal characters.

  2. 2. Create the reader-authored point-prompt harness

    This custom serial HTTPServer binds loopback and accepts only a fixed fixture ID. It generates original 224 RGB geometry and uses a positive point inside its red rectangle. Native preprocessing resizes to 1024; optional CUDA-dependent hole/sprinkle postprocessing and dynamic multimask stability fallback are explicitly disabled. This is not publisher deployment code.

    Command
    cd "$HOME/sam2-image"
    cat > sam2_image.py <<'PYSERVER'
    """Reader-authored educational fixed-input SAM2 original-large harness; not publisher server."""
    import hashlib
    import hmac
    import json
    import math
    import pathlib
    from http.server import BaseHTTPRequestHandler, HTTPServer
    
    ROOT = pathlib.Path(__file__).resolve().parent
    PORT = 8993
    MODEL_SHA = '7442e4e9b732a508f80e141e7c2913437a3610ee0c77381a66658c3a445df87b'
    
    
    def validate_mask(result):
        mask, score, low_shape = result
        if type(score) not in (int, float) or not math.isfinite(score):
            raise ValueError('invalid_score')
        if low_shape != [1, 256, 256] or len(mask) != 224:
            raise ValueError('invalid_shape')
        flat = []
        for row in mask:
            if len(row) != 224 or any(type(v) is not bool for v in row):
                raise ValueError('invalid_mask')
            flat.extend(row)
        runs = []
        i = 0
        while i < len(flat):
            if not flat[i]:
                i += 1
                continue
            start = i
            while i < len(flat) and flat[i]:
                i += 1
            runs.append([start, i-start])
        return {'fixture': 'author-geometry-v1', 'image_shape': [224, 224, 3],
                'transformed_shape': [1, 3, 1024, 1024],
                'coarse_embedding_shape': [1, 256, 64, 64],
                'configured_coarse_vision_positions': 4096,
                'point_xy': [64, 112], 'point_label': 1,
                'mask_shape': [1, 224, 224], 'mask_true_runs': runs,
                'mask_area_pixels': sum(flat), 'estimated_iou': score,
                'low_resolution_logits_shape': low_shape,
                'generated_text_tokens': None, 'semantic_classification': False}
    
    
    def make_encoder():
        # Actual model imports occur only in the reader-run path, not fake boundary tests.
        import numpy as np
        import torch
        from PIL import Image, ImageDraw
        from sam2.build_sam import build_sam2
        from sam2.sam2_image_predictor import SAM2ImagePredictor
        torch.set_num_threads(4)
        artifact = ROOT / 'models/sam2_hiera_large.pt'
        if hashlib.sha256(artifact.read_bytes()).hexdigest() != MODEL_SHA:
            raise RuntimeError('checkpoint_hash_mismatch')
        model = build_sam2('configs/sam2/sam2_hiera_l.yaml', str(artifact),
                           device='cpu', apply_postprocessing=False)
        model = model.float().eval()
        predictor = SAM2ImagePredictor(model, max_hole_area=0, max_sprinkle_area=0)
        # Original author geometry; no third-party input or mutable remote URL.
        image = Image.new('RGB', (224, 224), (240, 240, 240))
        draw = ImageDraw.Draw(image)
        draw.rectangle((24, 24, 104, 200), fill=(220, 40, 40))
        draw.ellipse((120, 64, 208, 152), fill=(30, 90, 220))
        image.save(ROOT / 'fixture.png')
    
        def encode():
            # Set fresh predictor state for each fixed-image request; no video state shared.
            try:
                with torch.inference_mode():
                    predictor.set_image(image)
                    if list(predictor.get_image_embedding().shape) != [1, 256, 64, 64]:
                        raise RuntimeError('coarse_embedding_shape')
                    masks, scores, low = predictor.predict(
                        point_coords=np.array([[64, 112]], dtype=np.float32),
                        point_labels=np.array([1], dtype=np.int32),
                        multimask_output=False, return_logits=False)
                    if list(masks.shape) != [1, 224, 224] or list(scores.shape) != [1]:
                        raise RuntimeError('output_shape')
                    if list(low.shape) != [1, 256, 256] or not np.isfinite(low).all():
                        raise RuntimeError('low_logit_shape_or_finiteness')
                    return masks[0].astype(bool).tolist(), float(scores[0]), list(low.shape)
            finally:
                predictor.reset_predictor()
        return encode
    
    
    def create_server(encoder, key, port=PORT):
        if not isinstance(key, str) or len(key) != 64 or any(c not in '0123456789abcdef' for c in key):
            raise ValueError('key_format')
    
        class Handler(BaseHTTPRequestHandler):
            def setup(self):
                super().setup()
                self.connection.settimeout(5)
    
            def log_message(self, *_):
                pass  # Do not log Bearer keys or request bodies.
    
            def reply(self, status, obj):
                data = json.dumps(obj, allow_nan=False, separators=(',', ':')).encode()
                self.send_response(status)
                self.send_header('Content-Type', 'application/json')
                self.send_header('Content-Length', str(len(data)))
                self.send_header('Connection', 'close')
                self.end_headers()
                self.wfile.write(data)
                self.close_connection = True
    
            def authorized(self):
                values = self.headers.get_all('Authorization', [])
                return len(values) == 1 and hmac.compare_digest(values[0].encode(), ('Bearer ' + key).encode())
    
            def do_GET(self):
                if not self.authorized():
                    return self.reply(401, {'error': 'unauthorized'})
                if self.path != '/health':
                    return self.reply(404, {'error': 'not_found'})
                return self.reply(200, {'ready': True, 'configured_coarse_vision_positions': 4096,
                                        'generated_text_tokens': None})
    
            def do_POST(self):
                if not self.authorized():
                    return self.reply(401, {'error': 'unauthorized'})
                if self.path != '/v1/point-segment':
                    return self.reply(404, {'error': 'not_found'})
                if self.headers.get('Transfer-Encoding') is not None:
                    return self.reply(400, {'error': 'transfer_encoding_unsupported'})
                lengths = self.headers.get_all('Content-Length', [])
                if len(lengths) != 1:
                    return self.reply(411, {'error': 'content_length_required'})
                raw = lengths[0]
                if not raw or len(raw) > 4 or any(c not in '0123456789' for c in raw):
                    return self.reply(413, {'error': 'invalid_or_excessive_length'})
                n = int(raw)
                if n < 1 or n > 256:
                    return self.reply(413, {'error': 'body_limit'})
                if self.headers.get_content_type() != 'application/json':
                    return self.reply(415, {'error': 'json_required'})
                try:
                    body = self.rfile.read(n)
                except (TimeoutError, OSError):
                    return self.reply(408, {'error': 'body_timeout'})
                if len(body) != n:
                    return self.reply(400, {'error': 'incomplete_body'})
                try:
                    def unique(pairs):
                        obj = {}
                        for k, v in pairs:
                            if k in obj:
                                raise ValueError('duplicate_key')
                            obj[k] = v
                        return obj
                    request = json.loads(body.decode('utf-8'), object_pairs_hook=unique,
                                         parse_constant=lambda _: (_ for _ in ()).throw(ValueError('constant')))
                except (ValueError, UnicodeError, RecursionError):
                    return self.reply(400, {'error': 'invalid_json'})
                if type(request) is not dict or request != {'fixture': 'author-geometry-v1'}:
                    return self.reply(400, {'error': 'fixed_fixture_required'})
                try:
                    result = validate_mask(encoder())
                except Exception:
                    return self.reply(500, {'error': 'predictor_or_mask_failure'})
                return self.reply(200, result)
    
        return HTTPServer(('127.0.0.1', port), Handler)
    
    
    if __name__ == '__main__':
        secret = (ROOT / '.api-key').read_text().strip()
        encoder = make_encoder()
        with create_server(encoder, secret) as server:
            server.serve_forever()
    PYSERVER
    .venv/bin/python -m py_compile sam2_image.py
    sha256sum --check <<'SHA256'
    3c1898a944b0ba5de896cba840cd04c9ba1575b408a97054653b6e82f166782f  sam2_image.py
    SHA256

    Expected result: Wrapper syntax and exact bytes verify without loading model weights.

Start the model

  1. 1. Start the fixed point segmentation experiment

    Run from the dedicated workspace, outside the source checkout. Native eager CPU float32 is explicit; no CUDA autocast or optimized compiled path. Predictor state resets after each request, with no shared video/session state.

    Command
    cd "$HOME/sam2-image"
    .venv/bin/python sam2_image.py

    Expected result: After a source-supported but unexecuted load, the server binds 127.0.0.1:8993. Stop on loading errors; no model load or successful mask is claimed by this guide.

Connect to your endpoint

http://127.0.0.1:8993/v1/point-segment

Same trusted desktop loopback only.

Access: loopback. Private Bearer key required for health and segmentation. Send a JSON object with fixture set to author-geometry-v1. Response contains binary-mask foreground runs, shapes, fixed point coordinates and estimated IoU. No chat schema, arbitrary uploads, URLs or file paths are accepted.

Verify the setup

  1. 1. Validate mask shape and inspect spatial quality

    From a second terminal on the server, request the exact fixed fixture. Inspect fixture.png, then reconstruct mask.png from row-major foreground runs. Check finite estimated IoU and exact shapes; compare the mask overlay with the clicked red rectangle yourself. One positive point does not guarantee its entire boundary or any exact area/score. Missing authorization must return 401. Do not turn this smoke check into a quality benchmark.

    Command
    cd "$HOME/sam2-image"
    SAM_KEY="$(cat .api-key)"
    curl --fail http://127.0.0.1:8993/health -H "Authorization: Bearer $SAM_KEY"
    curl --fail http://127.0.0.1:8993/v1/point-segment -H "Authorization: Bearer $SAM_KEY" -H 'Content-Type: application/json' -d '{"fixture":"author-geometry-v1"}' -o mask.json
    curl -s -o /dev/null -w '%{http_code}\n' http://127.0.0.1:8993/health
    unset SAM_KEY
    .venv/bin/python - <<'PYVERIFY'
    import json,math
    from PIL import Image
    d=json.load(open('mask.json'))
    assert d['configured_coarse_vision_positions']==4096 and d['generated_text_tokens'] is None
    assert d['mask_shape']==[1,224,224] and d['low_resolution_logits_shape']==[1,256,256]
    assert math.isfinite(d['estimated_iou'])
    pixels=[0]*(224*224);previous=0
    for start,length in d['mask_true_runs']:
     assert type(start) is int and type(length) is int and start>=previous and length>0 and start+length<=len(pixels)
     pixels[start:start+length]=[255]*length;previous=start+length
    assert sum(x>0 for x in pixels)==d['mask_area_pixels']
    mask=Image.new('L',(224,224));mask.putdata(pixels);mask.save('mask.png')
    image=Image.open('fixture.png').convert('RGB');tint=Image.new('RGB',image.size,(0,255,0))
    image.paste(Image.blend(image,tint,0.4),(0,0),mask);image.save('overlay.png')
    print('Inspect overlay.png and clicked point [64,112]; no exact mask quality promised.')
    PYVERIFY

    Expected result: Authenticated health and fixed segmentation return documented JSON; missing key 401. Mask shapes/RLE/finite checks pass after actual reader execution. Inspect overlay and record empty/incorrect masks as failures of the selected task, not fabricated successful segmentation.

Performance evidence

No compatible measurement available. Documented support and estimated memory do not establish response speed.

Limits and troubleshooting

  • The exact frozen API identifies original July 2024 SAM2 Hiera Large while the catalog version field says 2.1. This guide exposes the discrepancy and uses original checkpoint/config; no 2.1 substitution, catalog rewrite or family-complete claim.
  • 4,096 is the configured coarse 64 × 64 visual embedding position count at 1024 RGB input. Higher-resolution 256×256/128× 128 features, sparse point embeddings and learned decoder tokens are distinct; no text limit, whole internal sequence sum or global maximum is asserted.
  • Exactly one original 224 RGBfixture and one positive point are used. Mask and estimated IoU are not semantic class labels, calibrated quality or a guaranteed full-object segmentation. Manual spatial inspection remains essential.
  • Optional CUDA connected-component hole/sprinkle filling is omitted with SAM2_BUILD_CUDA=0 and predictor areas 0. apply_postprocessing=False also omits dynamic stability fallback. Do not claim identical masks to default GPU/postprocessed configurations.
  • Exact native checkpoint LFS SHA/size and Apache 2 license statements are source metadata/text evidence; no weights were downloaded or model executed. The separately published safetensors and HF-native YAML are not silently treated as identical.
  • Native runtime config RoPE initial grid 64 differs from HF-native YAML 32. Runtime recomputes frequencies for actual query shape; this source behavior is not measured numerical equivalence. The recipe pins native runtime config explicitly.
  • The original checkpoint is 224.4M parameters per publisher table; A100 benchmark rows are not CPU speed or memory evidence. 32 GiB RAM, 40 GiB disk / four threads are chosen allocations; fit, latency and runtime memory remain unknown.
  • This reader-authored serial HTTPServer has fixed input, private loopback/auth, bounded request body and socket timeout. It has no inference deadline, multiuser resilience or public-service suitability guarantee.
  • No autoregressive text cache, tokenizer, chat/reasoning template or generation sampler applies. Output tokens are N/A. Native image features reset after each request; video memory/automatic mask generation and other checkpoints remain separate useful open scopes.
  • Package versions are source-qualified but not installed/tested with the model. Transitive/system resolutions are recorded by pip freeze. Fake-mask transport tests prove request/output boundary handling only, not segmentation quality or runtime compatibility.
Checkpoint, config or state-dictionary mismatch
Stop and restore exact original checkpoint/native runtime config and source pins. Do not substitute SAM2.1 or HF safetensors, relax key checks or follow an unpinned convenience download.
CUDA extension warning or missing postprocessing
This CPU path intentionally omits hole/sprinkle cleanup and dynamic fallback. Confirm SAM2_BUILD_CUDA=0, CPU device and zero predictor cleanup areas. Do not install a GPU compiler or claim full GPU behavior from this guide.
401 or rejected body
Use matching private 64-character lowercase hexadecimal key and exact fixed fixture JSON. Arbitrary image uploads, point changes, paths and URLs are not accepted by this bounded experiment.
Empty, inaccurate or slow mask
Inspect original fixture, point [64,112], native resize and overlay. Record actual mask defects or runtime errors. No exact area, IoU, latency or CPU fit is promised; do not call a finite but wrong mask a validated segmentation.

Stop the model

  1. 1. Stop the research endpoint

    Press Ctrl+C in its foreground terminal and unset client keys. Confirm local loopback port 8993 no longer serves.

    Command
    unset SAM_KEY

    Expected result: Native process and endpoint stop; no active session state remains.

Uninstall

  1. 1. Remove dedicated experiment files

    Preserve desired results and licensing records first; confirm the dedicated directory before deletion. Shared Ubuntu packages remain.

    Command
    rm -r -- "$HOME/sam2-image"

    Expected result: Dedicated checkpoint, native checkout, virtualenv, generated fixture/mask/overlay, wrapper and private key removed.

Sources and verification

Sources checked Oct 16, 2026. Verification source

Compare this configuration
Cloud serverDocumented setupSources checked Oct 16, 2026

Segment a fixed image with original SAM 2 on a private CPU research VM

Use the exact facebook/sam2-hiera-large original July 2024 checkpoint and native image predictor in CPU float32. An original RGB geometry fixture and one positive point produce a binary mask and estimated IoU through a private educational endpoint. Native input resize is 1024 × 1024; its configured coarse visual embedding has 4,096 positions, separate from higher-resolution features and prompt/decoder tokens. The catalog version says 2.1, but this guide explicitly uses its exact original-SAM2 API artifact. Fit, speed and mask quality are unmeasured.

Hardware
AWS r7i.xlarge, 32 GiB RAM, Ubuntu x86_64
Runtime
Native SAM2 original image predictor CPU private research harness SAM2 source 2b90b9f5ceec907a1c18123530e92e794ad901a4; torch2.6.0+cpu; torchvision0.21.0+cpu; Python3.12; numpy2.2.3; Pillow11.1.0 · CPU
Model format
PyTorch checkpoint · Native CPU float32 eager inference; no autocast, quantization or compile optimization · Unquantized native checkpoint; reader CPU float32
Workload
4,096 token context · 1 concurrent

Based on source documentation; this setup has not been executed by OpenTools. Memory: unknown. No compatible measurement available.

Read the complete setup guide

Before you start

  • Use an owned AWS r7i.xlarge Ubuntu 24.04 x86_64 VM in us-east-1 with 32 GiB RAM, four vCPUs and a chosen 40 GiB gp3 volume. Existing SSH access must be limited to your client; keep service port 8993 closed. Provisioning and quoted/other applicable billing are reader actions.
  • The frozen catalog says version 2.1, but its exact API facebook/sam2-hiera-large maps to the original July 29, 2024 checkpoint. This recipe deliberately uses that exact API artifact and native original config; it does not silently upgrade to SAM2.1 or rewrite catalog metadata.
  • Review Apache2.0 obligations for the selected model/runtime. Use only the original geometric fixture generated by this harness; no third-party images, uploads, external URLs, arbitrary file inputs or model choices are accepted.
  • This is a source-documented private educational image segmentation experiment, not measured quality or a production service. Python 3.12 and the source-supported CPU package versions below have not been installed/model-tested by the author. Save dependency resolution and stop on pip check, state-dictionary or shape errors.

Exact model and requirements

Artifact:
Meta / facebookresearch / SAM2 original Hiera Large; exact frozen API artifact, not SAM2.1 · checkpoint facebook/sam2-hiera-large
Publisher revision:
e6a8e8809b8f1bfa2238b6d080f3d05cc76bd251
Artifact digest:
sha256:7442e4e9b732a508f80e141e7c2913437a3610ee0c77381a66658c3a445df87b
Download size:
0.84 GiB
Memory requirement (unknown):
Unknown — No measured full-process peak for original Hiera Large CPU float32,1024image activations, prompt decoder and wrapper.
System RAM / disk:
Unknown / Unknown
Placement:
single device. One CPU host; no tensor or pipeline sharding. Whole native model, image and prompt handling on CPU; no GPU/disk offload.
KV cache:
Image segmentation does not use autoregressive text K/V caching. Image predictor feature state is reset after each request; video memory is outside this recipe.
Workload:
4,096 token context; 4,096 input tokens; Not applicable; concurrency 1; batch 1; cache cold; repetitions Unknown.
Tokenizer:
Native 1024 × 1024 RGB image embedding, stride16 coarse grid64 × 64; no text tokenizer · 2b90b9f5ceec907a1c18123530e92e794ad901a4
Sampling / reasoning:
No text or audio generation sampling. Fixed original224 × 224 RGB geometry plus positive XY point [64,112], label1. Native ToTensor, square1024resize and ImageNet normalization. multimask_output=False selects one candidate; no hole/sprinkle filling, dynamic stability fallback or video propagation. 4,096 is the configured single-image coarse embedding position count only. Native high-resolution256 × 256 and128 × 128 feature maps, sparse spatial prompts and learned decoder tokens are separate; this is not a global/native maximum or text allowance. Point-label embedding types do not cap the number of possible points. This recipe fixes exactly one point and one image; masks do not supply semantic classes.
Runtime prerequisites:
Ubuntu 24.04 x86_64; See prerequisites above.

Official artifact · License

Actual upstream Apache License 2.0 text is archived, and upstream README expressly includes SAM2 model checkpoints. Retain license and relevant copyright/NOTICE material; observe redistribution/change-notice and patent conditions. No trademark endorsement or legal clearance is granted by this guide. Optional connected-component code has its own archived license; this CPU recipe skips that extension. Demo fonts and media are not included or presumed Apache-licensed. The original fixture is author-created geometry.

  • 897,952,466 bytes is official LFS metadata for this exact native checkpoint, not independently downloaded weight-body proof or full runtime RAM.
  • 32 GiB RAM,40 GiB disk and four threads are planning allocations, not measured minima or fit.1024image features, model buffers, native dependencies and OS add overhead.

One-time setup and hardware

hardware
Not applicable — AWS hardware acquisition is included in compute rental; no purchased GPU required. Any separate hardware cost is not applicable.
AWS hardware acquisition is included in compute rental; no purchased GPU required. Any separate hardware cost is not applicable. Tax: unknown.
setup
Unknown — Initial provisioning/build time and labor cost are unknown.
Initial provisioning/build time and labor cost are unknown. Tax: unknown.

Recurring costs

compute
$0.26 / hour
AWS public us-east-1 Linux On-Demand rate USD 0.2646 per billed instance-hour, no paid preinstalled software. Eight hours USD 2.1168; 24 hours USD 6.3504 compute. Idle running hours bill. Storage, egress, tax and administration separate; no Spot/reservation/discount assumed. Capacity/quota unverified. Region: us-east-1. SKU: r7i.xlarge Linux On-Demand. Tax: excluded. Price dated Oct 15, 2026. Price source
storage
$0.08 / GB month
US East (N. Virginia) gp3 USD 0.08 per GB-month; AWS defines GB as 1024^3 bytes. Selected 40 GiB root volume is USD 3.20 per full month, prorated by provisioned duration. Baseline 3000 IOPS / 125 MB/s; no extras assumed. Retained snapshots/resources separate; storage persists after instance stop. Tax excluded. Region: us-east-1. SKU: EBS gp3 baseline 3000 IOPS / 125 MB/s. Tax: excluded. Price dated Oct 16, 2026. Price source
egress
Unknown — Data transfer price, allowances and traffic volume are unknown.
Data transfer price, allowances and traffic volume are unknown. Tax: unknown.
administration
Unknown — Maintenance labor rate and hours are unknown.
Maintenance labor rate and hours are unknown. Tax: unknown.

Install the runtime

  1. 1. Create the native CPU environment

    Use exact official CPU wheels and a pinned native checkout. Explicitly skip the optional CUDA extension. Pin direct dependencies, save transitive/system resolution and stop on pip check failure; this is documented support rather than an executed environment.

    Command
    sudo apt-get update
    sudo apt-get install -y python3.12 python3.12-venv git curl ca-certificates openssl
    mkdir -p "$HOME/sam2-image/models"
    cd "$HOME/sam2-image"
    python3.12 -m venv .venv
    .venv/bin/python -m pip install 'https://download.pytorch.org/whl/cpu/torch-2.6.0%2Bcpu-cp312-cp312-linux_x86_64.whl#sha256=59e78aa0c690f70734e42670036d6b541930b8eabbaa18d94e090abf14cc4d91' 'https://download-r2.pytorch.org/whl/cpu/torchvision-0.21.0%2Bcpu-cp312-cp312-linux_x86_64.whl#sha256=d6874431e678ba107b60a83f255c33f3755f06bad587b1b919aa514ec325dcd8'
    .venv/bin/python -m pip install 'numpy==2.2.3' 'Pillow==11.1.0' 'hydra-core==1.3.2' 'omegaconf==2.3.0' 'antlr4-python3-runtime==4.9.3' 'iopath==0.1.10' 'tqdm==4.67.1'
    git clone https://github.com/facebookresearch/sam2.git native-sam2
    git -C native-sam2 checkout --detach 2b90b9f5ceec907a1c18123530e92e794ad901a4
    SAM2_BUILD_CUDA=0 .venv/bin/python -m pip install -e ./native-sam2
    .venv/bin/python -m pip check
    .venv/bin/python -m pip freeze > resolved-environment.txt

    Expected result: Pinned checkout and hashed CPU wheels resolve; extension build is disabled, pip check passes and versions are saved. No CUDA driver/compiler or GPU runtime is required by this chosen path.

Download the model

  1. 1. Verify the exact original checkpoint and config

    Reader download only: the author fetched metadata/text, not weights. Use exact HF revision for native .pt and exact config from pinned runtime. HF safetensors and HF-native YAML are different artifacts/config revisions and are not substituted.

    Command
    cd "$HOME/sam2-image"
    curl --fail --location --output models/sam2_hiera_large.pt 'https://huggingface.co/facebook/sam2-hiera-large/resolve/e6a8e8809b8f1bfa2238b6d080f3d05cc76bd251/sam2_hiera_large.pt'
    sha256sum --check <<'SHA256'
    7442e4e9b732a508f80e141e7c2913437a3610ee0c77381a66658c3a445df87b  models/sam2_hiera_large.pt
    0ee9f2037fd98489212bb0b78a56f33ac82eb177cf5e52c88be3714cb84eea26  native-sam2/sam2/configs/sam2/sam2_hiera_l.yaml
    c71d239df91726fc519c6eb72d318ec65820627232b2f796219e87dcf35d0ab4  native-sam2/LICENSE
    SHA256

    Expected result: All three exact hashes report OK; stop on mismatch. Local native loader uses weights_only=True and checks state-dictionary keys. No native convenience from_pretrained call floats the model revision.

Configure

  1. 1. Create a stable private service key

    Generate only when absent; preserve existing access. Do not print or share this local service key.

    Command
    cd "$HOME/sam2-image"
    umask 077
    if [ ! -e .api-key ]; then openssl rand -hex 32 > .api-key; fi
    chmod 600 .api-key
    test -s .api-key || { echo "Restore empty service key before continuing"; exit 1; }

    Expected result: Private existing key is preserved; server requires 64 lowercase hexadecimal characters.

  2. 2. Create the reader-authored point-prompt harness

    This custom serial HTTPServer binds loopback and accepts only a fixed fixture ID. It generates original 224 RGB geometry and uses a positive point inside its red rectangle. Native preprocessing resizes to 1024; optional CUDA-dependent hole/sprinkle postprocessing and dynamic multimask stability fallback are explicitly disabled. This is not publisher deployment code.

    Command
    cd "$HOME/sam2-image"
    cat > sam2_image.py <<'PYSERVER'
    """Reader-authored educational fixed-input SAM2 original-large harness; not publisher server."""
    import hashlib
    import hmac
    import json
    import math
    import pathlib
    from http.server import BaseHTTPRequestHandler, HTTPServer
    
    ROOT = pathlib.Path(__file__).resolve().parent
    PORT = 8993
    MODEL_SHA = '7442e4e9b732a508f80e141e7c2913437a3610ee0c77381a66658c3a445df87b'
    
    
    def validate_mask(result):
        mask, score, low_shape = result
        if type(score) not in (int, float) or not math.isfinite(score):
            raise ValueError('invalid_score')
        if low_shape != [1, 256, 256] or len(mask) != 224:
            raise ValueError('invalid_shape')
        flat = []
        for row in mask:
            if len(row) != 224 or any(type(v) is not bool for v in row):
                raise ValueError('invalid_mask')
            flat.extend(row)
        runs = []
        i = 0
        while i < len(flat):
            if not flat[i]:
                i += 1
                continue
            start = i
            while i < len(flat) and flat[i]:
                i += 1
            runs.append([start, i-start])
        return {'fixture': 'author-geometry-v1', 'image_shape': [224, 224, 3],
                'transformed_shape': [1, 3, 1024, 1024],
                'coarse_embedding_shape': [1, 256, 64, 64],
                'configured_coarse_vision_positions': 4096,
                'point_xy': [64, 112], 'point_label': 1,
                'mask_shape': [1, 224, 224], 'mask_true_runs': runs,
                'mask_area_pixels': sum(flat), 'estimated_iou': score,
                'low_resolution_logits_shape': low_shape,
                'generated_text_tokens': None, 'semantic_classification': False}
    
    
    def make_encoder():
        # Actual model imports occur only in the reader-run path, not fake boundary tests.
        import numpy as np
        import torch
        from PIL import Image, ImageDraw
        from sam2.build_sam import build_sam2
        from sam2.sam2_image_predictor import SAM2ImagePredictor
        torch.set_num_threads(4)
        artifact = ROOT / 'models/sam2_hiera_large.pt'
        if hashlib.sha256(artifact.read_bytes()).hexdigest() != MODEL_SHA:
            raise RuntimeError('checkpoint_hash_mismatch')
        model = build_sam2('configs/sam2/sam2_hiera_l.yaml', str(artifact),
                           device='cpu', apply_postprocessing=False)
        model = model.float().eval()
        predictor = SAM2ImagePredictor(model, max_hole_area=0, max_sprinkle_area=0)
        # Original author geometry; no third-party input or mutable remote URL.
        image = Image.new('RGB', (224, 224), (240, 240, 240))
        draw = ImageDraw.Draw(image)
        draw.rectangle((24, 24, 104, 200), fill=(220, 40, 40))
        draw.ellipse((120, 64, 208, 152), fill=(30, 90, 220))
        image.save(ROOT / 'fixture.png')
    
        def encode():
            # Set fresh predictor state for each fixed-image request; no video state shared.
            try:
                with torch.inference_mode():
                    predictor.set_image(image)
                    if list(predictor.get_image_embedding().shape) != [1, 256, 64, 64]:
                        raise RuntimeError('coarse_embedding_shape')
                    masks, scores, low = predictor.predict(
                        point_coords=np.array([[64, 112]], dtype=np.float32),
                        point_labels=np.array([1], dtype=np.int32),
                        multimask_output=False, return_logits=False)
                    if list(masks.shape) != [1, 224, 224] or list(scores.shape) != [1]:
                        raise RuntimeError('output_shape')
                    if list(low.shape) != [1, 256, 256] or not np.isfinite(low).all():
                        raise RuntimeError('low_logit_shape_or_finiteness')
                    return masks[0].astype(bool).tolist(), float(scores[0]), list(low.shape)
            finally:
                predictor.reset_predictor()
        return encode
    
    
    def create_server(encoder, key, port=PORT):
        if not isinstance(key, str) or len(key) != 64 or any(c not in '0123456789abcdef' for c in key):
            raise ValueError('key_format')
    
        class Handler(BaseHTTPRequestHandler):
            def setup(self):
                super().setup()
                self.connection.settimeout(5)
    
            def log_message(self, *_):
                pass  # Do not log Bearer keys or request bodies.
    
            def reply(self, status, obj):
                data = json.dumps(obj, allow_nan=False, separators=(',', ':')).encode()
                self.send_response(status)
                self.send_header('Content-Type', 'application/json')
                self.send_header('Content-Length', str(len(data)))
                self.send_header('Connection', 'close')
                self.end_headers()
                self.wfile.write(data)
                self.close_connection = True
    
            def authorized(self):
                values = self.headers.get_all('Authorization', [])
                return len(values) == 1 and hmac.compare_digest(values[0].encode(), ('Bearer ' + key).encode())
    
            def do_GET(self):
                if not self.authorized():
                    return self.reply(401, {'error': 'unauthorized'})
                if self.path != '/health':
                    return self.reply(404, {'error': 'not_found'})
                return self.reply(200, {'ready': True, 'configured_coarse_vision_positions': 4096,
                                        'generated_text_tokens': None})
    
            def do_POST(self):
                if not self.authorized():
                    return self.reply(401, {'error': 'unauthorized'})
                if self.path != '/v1/point-segment':
                    return self.reply(404, {'error': 'not_found'})
                if self.headers.get('Transfer-Encoding') is not None:
                    return self.reply(400, {'error': 'transfer_encoding_unsupported'})
                lengths = self.headers.get_all('Content-Length', [])
                if len(lengths) != 1:
                    return self.reply(411, {'error': 'content_length_required'})
                raw = lengths[0]
                if not raw or len(raw) > 4 or any(c not in '0123456789' for c in raw):
                    return self.reply(413, {'error': 'invalid_or_excessive_length'})
                n = int(raw)
                if n < 1 or n > 256:
                    return self.reply(413, {'error': 'body_limit'})
                if self.headers.get_content_type() != 'application/json':
                    return self.reply(415, {'error': 'json_required'})
                try:
                    body = self.rfile.read(n)
                except (TimeoutError, OSError):
                    return self.reply(408, {'error': 'body_timeout'})
                if len(body) != n:
                    return self.reply(400, {'error': 'incomplete_body'})
                try:
                    def unique(pairs):
                        obj = {}
                        for k, v in pairs:
                            if k in obj:
                                raise ValueError('duplicate_key')
                            obj[k] = v
                        return obj
                    request = json.loads(body.decode('utf-8'), object_pairs_hook=unique,
                                         parse_constant=lambda _: (_ for _ in ()).throw(ValueError('constant')))
                except (ValueError, UnicodeError, RecursionError):
                    return self.reply(400, {'error': 'invalid_json'})
                if type(request) is not dict or request != {'fixture': 'author-geometry-v1'}:
                    return self.reply(400, {'error': 'fixed_fixture_required'})
                try:
                    result = validate_mask(encoder())
                except Exception:
                    return self.reply(500, {'error': 'predictor_or_mask_failure'})
                return self.reply(200, result)
    
        return HTTPServer(('127.0.0.1', port), Handler)
    
    
    if __name__ == '__main__':
        secret = (ROOT / '.api-key').read_text().strip()
        encoder = make_encoder()
        with create_server(encoder, secret) as server:
            server.serve_forever()
    PYSERVER
    .venv/bin/python -m py_compile sam2_image.py
    sha256sum --check <<'SHA256'
    3c1898a944b0ba5de896cba840cd04c9ba1575b408a97054653b6e82f166782f  sam2_image.py
    SHA256

    Expected result: Wrapper syntax and exact bytes verify without loading model weights.

Start the model

  1. 1. Start the fixed point segmentation experiment

    Run from the dedicated workspace, outside the source checkout. Native eager CPU float32 is explicit; no CUDA autocast or optimized compiled path. Predictor state resets after each request, with no shared video/session state.

    Command
    cd "$HOME/sam2-image"
    .venv/bin/python sam2_image.py

    Expected result: After a source-supported but unexecuted load, the server binds 127.0.0.1:8993. Stop on loading errors; no model load or successful mask is claimed by this guide.

Connect to your endpoint

http://127.0.0.1:8993/v1/point-segment

VM loopback only. Optional client tunnel with existing SSH access: ssh -N -L 8993:127.0.0.1:8993 USER@VM_IP. Keep EC2 port8993 closed; shown checks run on server.

Access: loopback. Private Bearer key required for health and segmentation. Send a JSON object with fixture set to author-geometry-v1. Response contains binary-mask foreground runs, shapes, fixed point coordinates and estimated IoU. No chat schema, arbitrary uploads, URLs or file paths are accepted.

Verify the setup

  1. 1. Validate mask shape and inspect spatial quality

    From a second terminal on the server, request the exact fixed fixture. Inspect fixture.png, then reconstruct mask.png from row-major foreground runs. Check finite estimated IoU and exact shapes; compare the mask overlay with the clicked red rectangle yourself. One positive point does not guarantee its entire boundary or any exact area/score. Missing authorization must return 401. Do not turn this smoke check into a quality benchmark.

    Command
    cd "$HOME/sam2-image"
    SAM_KEY="$(cat .api-key)"
    curl --fail http://127.0.0.1:8993/health -H "Authorization: Bearer $SAM_KEY"
    curl --fail http://127.0.0.1:8993/v1/point-segment -H "Authorization: Bearer $SAM_KEY" -H 'Content-Type: application/json' -d '{"fixture":"author-geometry-v1"}' -o mask.json
    curl -s -o /dev/null -w '%{http_code}\n' http://127.0.0.1:8993/health
    unset SAM_KEY
    .venv/bin/python - <<'PYVERIFY'
    import json,math
    from PIL import Image
    d=json.load(open('mask.json'))
    assert d['configured_coarse_vision_positions']==4096 and d['generated_text_tokens'] is None
    assert d['mask_shape']==[1,224,224] and d['low_resolution_logits_shape']==[1,256,256]
    assert math.isfinite(d['estimated_iou'])
    pixels=[0]*(224*224);previous=0
    for start,length in d['mask_true_runs']:
     assert type(start) is int and type(length) is int and start>=previous and length>0 and start+length<=len(pixels)
     pixels[start:start+length]=[255]*length;previous=start+length
    assert sum(x>0 for x in pixels)==d['mask_area_pixels']
    mask=Image.new('L',(224,224));mask.putdata(pixels);mask.save('mask.png')
    image=Image.open('fixture.png').convert('RGB');tint=Image.new('RGB',image.size,(0,255,0))
    image.paste(Image.blend(image,tint,0.4),(0,0),mask);image.save('overlay.png')
    print('Inspect overlay.png and clicked point [64,112]; no exact mask quality promised.')
    PYVERIFY

    Expected result: Authenticated health and fixed segmentation return documented JSON; missing key 401. Mask shapes/RLE/finite checks pass after actual reader execution. Inspect overlay and record empty/incorrect masks as failures of the selected task, not fabricated successful segmentation.

Performance evidence

No compatible measurement available. Documented support and estimated memory do not establish response speed.

Limits and troubleshooting

  • The exact frozen API identifies original July 2024 SAM2 Hiera Large while the catalog version field says 2.1. This guide exposes the discrepancy and uses original checkpoint/config; no 2.1 substitution, catalog rewrite or family-complete claim.
  • 4,096 is the configured coarse 64 × 64 visual embedding position count at 1024 RGB input. Higher-resolution 256×256/128× 128 features, sparse point embeddings and learned decoder tokens are distinct; no text limit, whole internal sequence sum or global maximum is asserted.
  • Exactly one original 224 RGBfixture and one positive point are used. Mask and estimated IoU are not semantic class labels, calibrated quality or a guaranteed full-object segmentation. Manual spatial inspection remains essential.
  • Optional CUDA connected-component hole/sprinkle filling is omitted with SAM2_BUILD_CUDA=0 and predictor areas 0. apply_postprocessing=False also omits dynamic stability fallback. Do not claim identical masks to default GPU/postprocessed configurations.
  • Exact native checkpoint LFS SHA/size and Apache 2 license statements are source metadata/text evidence; no weights were downloaded or model executed. The separately published safetensors and HF-native YAML are not silently treated as identical.
  • Native runtime config RoPE initial grid 64 differs from HF-native YAML 32. Runtime recomputes frequencies for actual query shape; this source behavior is not measured numerical equivalence. The recipe pins native runtime config explicitly.
  • The original checkpoint is 224.4M parameters per publisher table; A100 benchmark rows are not CPU speed or memory evidence. 32 GiB RAM, 40 GiB disk / four threads are chosen allocations; fit, latency and runtime memory remain unknown.
  • This reader-authored serial HTTPServer has fixed input, private loopback/auth, bounded request body and socket timeout. It has no inference deadline, multiuser resilience or public-service suitability guarantee.
  • No autoregressive text cache, tokenizer, chat/reasoning template or generation sampler applies. Output tokens are N/A. Native image features reset after each request; video memory/automatic mask generation and other checkpoints remain separate useful open scopes.
  • Package versions are source-qualified but not installed/tested with the model. Transitive/system resolutions are recorded by pip freeze. Fake-mask transport tests prove request/output boundary handling only, not segmentation quality or runtime compatibility.
Checkpoint, config or state-dictionary mismatch
Stop and restore exact original checkpoint/native runtime config and source pins. Do not substitute SAM2.1 or HF safetensors, relax key checks or follow an unpinned convenience download.
CUDA extension warning or missing postprocessing
This CPU path intentionally omits hole/sprinkle cleanup and dynamic fallback. Confirm SAM2_BUILD_CUDA=0, CPU device and zero predictor cleanup areas. Do not install a GPU compiler or claim full GPU behavior from this guide.
401 or rejected body
Use matching private 64-character lowercase hexadecimal key and exact fixed fixture JSON. Arbitrary image uploads, point changes, paths and URLs are not accepted by this bounded experiment.
Empty, inaccurate or slow mask
Inspect original fixture, point [64,112], native resize and overlay. Record actual mask defects or runtime errors. No exact area, IoU, latency or CPU fit is promised; do not call a finite but wrong mask a validated segmentation.

Stop the model

  1. 1. Stop the research endpoint

    Press Ctrl+C in its foreground terminal and unset client keys. Close optional SSH tunnel. Stopping Python does not stop VM billing.

    Command
    unset SAM_KEY

    Expected result: Native process and endpoint stop; no active session state remains.

Uninstall

  1. 1. Remove dedicated experiment files

    Preserve desired results and licensing records first; confirm the dedicated directory before deletion. Shared Ubuntu packages remain.

    Command
    rm -r -- "$HOME/sam2-image"

    Expected result: Dedicated checkpoint, native checkout, virtualenv, generated fixture/mask/overlay, wrapper and private key removed.

Shut down cloud resources

  1. 1. End cloud billing resources

    In EC2 us-east-1, stop for later reuse or terminate this exact VM. Review Delete on termination for its root volume. Delete unwanted detached EBS volumes, snapshots and reserved/Elastic/public IP resources. Stopped VM storage and retained resources may still bill. Check Bills/Cost Explorer later for final usage.

    Expected result: Exact instance is stopped/terminated as intended; no unwanted attached/retained billed resources remain. Instance compute stops only after its lifecycle transition, not when the Python service stops.

Sources and verification

Sources checked Oct 16, 2026. Verification source

Compare this configuration
Local hardwareDocumented setupSources checked Oct 17, 2026

Generate automatic masks with original SAM 2 on a private research CPU desktop

Use the exact original facebook/sam2-hiera-large checkpoint and native automatic mask generator on one author-created RGB fixture. A 4 × 4 point grid, batches of four points and three candidates per point produce at most 48 candidates before native IoU and stability filtering and box NMS. Complete masks use column-major uncompressed RLE. This automatic proposal workflow supplies no semantic classes or guaranteed object discovery. The catalog version 2.1 discrepancy stays explicit; CPU fit, speed and quality are unmeasured.

Hardware
Intel Core i5-12400 desktop, 4 inference threads, 32 GiB RAM
Runtime
Native original SAM2 CPU automatic mask generator with private research harness SAM2 source 2b90b9f5ceec907a1c18123530e92e794ad901a4; torch2.6.0+cpu; torchvision0.21.0+cpu; Python3.12; numpy2.2.3; Pillow11.1.0 · CPU
Model format
PyTorch checkpoint · Native CPU float32 eager inference; no autocast, quantization or compile optimization · Unquantized native checkpoint; reader CPU float32
Workload
4,096 token context · 1 concurrent

Based on source documentation; this setup has not been executed by OpenTools. Memory: unknown. No compatible measurement available.

Read the complete setup guide

Before you start

  • Use an owned Intel Core i5-12400 desktop with Ubuntu 24.04 x86_64, 32 GiB RAM, 40 GiB free disk and a four-thread allocation. These are planning choices, not measured minima.
  • The frozen catalog says version 2.1, but its exact API facebook/sam2-hiera-large maps to the original July 29, 2024 checkpoint. This recipe deliberately uses that exact API artifact and native original config; it does not silently upgrade to SAM2.1 or rewrite catalog metadata.
  • Review the exact upstream Apache 2.0 conditions and retain relevant license and notice material. This bounded experiment uses only original author-created geometry, with no third-party media, external URL, arbitrary image path, upload or user-selected model. Automatic proposals carry no semantic labels.
  • This is a source-documented CPU float32 eager experiment; no model execution or quality benchmark has occurred. Use Python 3.12 and the exact direct CPU packages below, save transitive and system resolution, and stop on pip check, checkpoint, NMS or output errors. A foreground process deadline of 900 seconds bounds the demonstration lifetime. A client timeout of 120 seconds may expire earlier while the serial computation is still running.

Exact model and requirements

Artifact:
Meta / facebookresearch / SAM2 original Hiera Large; exact frozen API artifact, not SAM2.1 · checkpoint facebook/sam2-hiera-large
Publisher revision:
e6a8e8809b8f1bfa2238b6d080f3d05cc76bd251
Artifact digest:
sha256:7442e4e9b732a508f80e141e7c2913437a3610ee0c77381a66658c3a445df87b
Download size:
0.84 GiB
Memory requirement (unknown):
Unknown — No measured full-process peak for original Hiera Large CPU float32,1024image activations, prompt decoder and wrapper. Automatic-mask prompt batches/intermediate logits and output add unmeasured overhead; prior prompted-image behavior does not establish fit.
System RAM / disk:
Unknown / Unknown
Placement:
single device. One CPU host; no tensor or pipeline sharding. Whole native model, image and prompt handling on CPU; no GPU/disk offload.
KV cache:
No autoregressive text K/V cache. Native image features are reset after generation/failure. The automatic point-grid decoder batches and intermediate masks are separate from image embedding positions.
Workload:
4,096 token context; 4,096 input tokens; Not applicable; concurrency 1; batch 1; cache cold; repetitions Unknown.
Tokenizer:
Native 1024 × 1024 RGB image embedding, stride16 coarse grid64 × 64; no text tokenizer · 2b90b9f5ceec907a1c18123530e92e794ad901a4
Sampling / reasoning:
One fixed original 224 × 224 RGB geometry image, resized natively to 1024 × 1024. A normalized 4 × 4 grid supplies 16 positive point prompts in four batches of four. multimask_output=True produces at most three candidates per point, or 48 before filtering. crop_n_layers=0 selects one whole-image crop. pred_iou_thresh=0.8 uses strict >; stability_score_thresh=0.95 uses >=; stability offset=1, mask threshold=0 and box NMS threshold=0.7. No M2M refinement, region cleanup or COCO encoder is selected. 4,096 is the configured count of coarse visual positions per whole-image crop. Separate 256 × 256 and 128 × 128 high-resolution features, sparse point embeddings and decoder tokens are outside that count. Workload batchSize=1 means one image per request; internal points_per_batch=4 is a separate decoder batch. The point grid and candidate counts are chosen finite bounds, not native maxima or a text-token allowance.
Runtime prerequisites:
Ubuntu 24.04 x86_64; See prerequisites above.

Official artifact · License

Actual upstream Apache License 2.0 text is archived, and upstream README expressly includes SAM2 model checkpoints. Retain license and relevant copyright/NOTICE material; observe redistribution/change-notice and patent conditions. No trademark endorsement or legal clearance is granted by this guide. Optional connected-component code has its own archived license; this CPU recipe skips that extension. Demo fonts and media are not included or presumed Apache-licensed. The original fixture is author-created geometry.

  • 897,952,466 bytes is official LFS metadata for this exact native checkpoint, not independently downloaded weight-body proof or full runtime RAM.
  • 32 GiB RAM,40 GiB disk and four threads are planning allocations, not measured minima or fit.1024image features, model buffers, native dependencies and OS add overhead.

One-time setup and hardware

hardware
Unknown — Existing desktop purchase price and incremental hardware cost depend on the reader; unknown.
Existing desktop purchase price and incremental hardware cost depend on the reader; unknown. Tax: unknown.
setup
Unknown — Initial provisioning/build time and labor cost are unknown.
Initial provisioning/build time and labor cost are unknown. Tax: unknown.

Recurring costs

electricity
Unknown — Actual wall power, electricity rate and powered hours are unknown; no TDP-to-wall-power conversion.
Actual wall power, electricity rate and powered hours are unknown; no TDP-to-wall-power conversion. Tax: unknown.
storage
Unknown — Incremental cost of local disk capacity is unknown. The 40 GiB free-disk walkthrough allocation is an author choice, not a measured requirement.
Incremental cost of local disk capacity is unknown. The 40 GiB free-disk walkthrough allocation is an author choice, not a measured requirement. Tax: unknown.
egress
Unknown — Data transfer price, allowances and traffic volume are unknown.
Data transfer price, allowances and traffic volume are unknown. Tax: unknown.
administration
Unknown — Maintenance labor rate and hours are unknown.
Maintenance labor rate and hours are unknown. Tax: unknown.

Install the runtime

  1. 1. Create the native CPU environment

    Use exact official CPU wheels and a pinned native checkout. Explicitly skip the optional CUDA extension. Pin direct dependencies, save transitive/system resolution and stop on pip check failure; this is documented support rather than an executed environment.

    Command
    sudo apt-get update
    sudo apt-get install -y python3.12 python3.12-venv git curl ca-certificates openssl
    mkdir -p "$HOME/sam2-automatic/models"
    cd "$HOME/sam2-automatic"
    python3.12 -m venv .venv
    .venv/bin/python -m pip install 'https://download.pytorch.org/whl/cpu/torch-2.6.0%2Bcpu-cp312-cp312-linux_x86_64.whl#sha256=59e78aa0c690f70734e42670036d6b541930b8eabbaa18d94e090abf14cc4d91' 'https://download-r2.pytorch.org/whl/cpu/torchvision-0.21.0%2Bcpu-cp312-cp312-linux_x86_64.whl#sha256=d6874431e678ba107b60a83f255c33f3755f06bad587b1b919aa514ec325dcd8'
    .venv/bin/python -m pip install 'numpy==2.2.3' 'Pillow==11.1.0' 'hydra-core==1.3.2' 'omegaconf==2.3.0' 'antlr4-python3-runtime==4.9.3' 'iopath==0.1.10' 'tqdm==4.67.1'
    git clone https://github.com/facebookresearch/sam2.git native-sam2
    git -C native-sam2 checkout --detach 2b90b9f5ceec907a1c18123530e92e794ad901a4
    SAM2_BUILD_CUDA=0 .venv/bin/python -m pip install -e ./native-sam2
    .venv/bin/python -m pip check
    .venv/bin/python -m pip freeze > resolved-environment.txt

    Expected result: Pinned checkout and hashed CPU wheels resolve; extension build is disabled, pip check passes and versions are saved. No CUDA driver/compiler or GPU runtime is required by this chosen path.

Download the model

  1. 1. Verify the exact original checkpoint and config

    Reader download only: the author fetched metadata/text, not weights. Use exact HF revision for native .pt and exact config from pinned runtime. HF safetensors and HF-native YAML are different artifacts/config revisions and are not substituted.

    Command
    cd "$HOME/sam2-automatic"
    curl --fail --location --output models/sam2_hiera_large.pt 'https://huggingface.co/facebook/sam2-hiera-large/resolve/e6a8e8809b8f1bfa2238b6d080f3d05cc76bd251/sam2_hiera_large.pt'
    sha256sum --check <<'SHA256'
    7442e4e9b732a508f80e141e7c2913437a3610ee0c77381a66658c3a445df87b  models/sam2_hiera_large.pt
    0ee9f2037fd98489212bb0b78a56f33ac82eb177cf5e52c88be3714cb84eea26  native-sam2/sam2/configs/sam2/sam2_hiera_l.yaml
    c71d239df91726fc519c6eb72d318ec65820627232b2f796219e87dcf35d0ab4  native-sam2/LICENSE
    SHA256

    Expected result: All three exact hashes report OK; stop on mismatch. Local native loader uses weights_only=True and checks state-dictionary keys. No native convenience from_pretrained call floats the model revision.

Configure

  1. 1. Create a stable private service key

    Generate only when absent; preserve existing access. Do not print or share this local service key.

    Command
    cd "$HOME/sam2-automatic"
    umask 077
    if [ ! -e .api-key ]; then openssl rand -hex 32 > .api-key; fi
    chmod 600 .api-key
    test -s .api-key || { echo "Restore empty service key before continuing"; exit 1; }

    Expected result: Private existing key is preserved; server requires 64 lowercase hexadecimal characters.

  2. 2. Create the bounded native automatic-mask harness

    The complete reader-authored wrapper is provided in two consecutive steps. It preserves native automatic mask generation with 16 grid points, batches of four points, one whole-image crop, native candidate filtering and NMS, and column-major RLE. It validates output bounds and resets predictor state in finally. Requests cannot change the image, grid or model. Syntax and hash verification do not load a model.

    Command
    cd "$HOME/sam2-automatic"
    cat > sam2_automatic.py <<'PYSERVER'
    """Reader-authored fixed automatic-mask experiment. Native model path is reader-run only."""
    import hashlib
    import hmac
    import json
    import math
    import pathlib
    from http.server import BaseHTTPRequestHandler, HTTPServer
    
    ROOT = pathlib.Path(__file__).resolve().parent
    PORT = 8996
    MODEL_SHA = '7442e4e9b732a508f80e141e7c2913437a3610ee0c77381a66658c3a445df87b'
    
    
    def validate_mask(records):
        if type(records) is not list or len(records) > 48:
            raise ValueError('record_count')
        checked=[]
        for r in records:
            if type(r) is not dict or set(r) != {'segmentation','area','bbox','predicted_iou','point_coords','stability_score','crop_box'}:
                raise ValueError('record_fields')
            seg=r['segmentation']
            if type(seg) is not dict or set(seg) != {'size','counts'} or seg['size'] != [224,224]:
                raise ValueError('rle_shape')
            counts=seg['counts']
            if type(counts) is not list or not 1 <= len(counts) <= 50177 or any(type(v) is not int or v < 0 or v > 50176 for v in counts) or sum(counts) != 50176:
                raise ValueError('rle_counts')
            area=sum(counts[1::2])
            if type(r['area']) is not int or r['area'] != area or area==0:
                raise ValueError('mask_area')
            for k in ['predicted_iou','stability_score']:
                if type(r[k]) not in (float,int) or not math.isfinite(r[k]):
                    raise ValueError('nonfinite_score')
            for k in ['bbox','crop_box']:
                v=r[k]
                if type(v) is not list or len(v)!=4 or any(type(x) not in (float,int) or not math.isfinite(x) or x<0 or x>224 for x in v) or v[0]+v[2]>224 or v[1]+v[3]>224:
                    raise ValueError('xywh_box')
            if r['crop_box'] != [0,0,224,224]:
                raise ValueError('unexpected_crop')
            pts=r['point_coords']
            if type(pts) is not list or len(pts)!=1 or type(pts[0]) is not list or len(pts[0])!=2 or any(type(v) not in (float,int) or not math.isfinite(v) or v not in (28.0,84.0,140.0,196.0) for v in pts[0]):
                raise ValueError('grid_point')
            checked.append(r)
        return {'fixture':'author-geometry-v1','image_shape':[224,224,3],
                'configured_coarse_vision_positions_per_crop':4096,
                'grid_points':16,'points_per_batch':4,'crop_count':1,
                'maximum_candidate_masks_before_filtering':48,
                'mask_count':len(checked),'records':checked,
                'rle_order':'column-major alternating background/foreground starting with background',
                'generated_text_tokens':None,'semantic_classification':False}
    
    
    def make_encoder():
        import numpy as np
        import torch
        from PIL import Image, ImageDraw
        from sam2.build_sam import build_sam2
        from sam2.automatic_mask_generator import SAM2AutomaticMaskGenerator
        torch.set_num_threads(4)
        artifact=ROOT/'models/sam2_hiera_large.pt'
        if hashlib.sha256(artifact.read_bytes()).hexdigest()!=MODEL_SHA:
            raise RuntimeError('checkpoint_hash_mismatch')
        model=build_sam2('configs/sam2/sam2_hiera_l.yaml',str(artifact),device='cpu',apply_postprocessing=False).float().eval()
        generator=SAM2AutomaticMaskGenerator(model,points_per_side=4,points_per_batch=4,
            pred_iou_thresh=0.8,stability_score_thresh=0.95,stability_score_offset=1.0,
            mask_threshold=0.0,box_nms_thresh=0.7,crop_n_layers=0,crop_nms_thresh=0.7,
            crop_overlap_ratio=512/1500,crop_n_points_downscale_factor=1,
            min_mask_region_area=0,output_mode='uncompressed_rle',use_m2m=False,
            multimask_output=True)
        image=Image.new('RGB',(224,224),(240,240,240));draw=ImageDraw.Draw(image)
        draw.rectangle((24,24,104,200),fill=(220,40,40));draw.ellipse((120,64,208,152),fill=(30,90,220))
        image.save(ROOT/'fixture.png')
        def encode():
            try:
                with torch.inference_mode():
                    records=generator.generate(np.array(image,dtype=np.uint8))
                    return records
            finally:
                generator.predictor.reset_predictor()
        return encode
    
    
    def create_server(encoder, key, port=PORT):
        if not isinstance(key, str) or len(key) != 64 or any(c not in '0123456789abcdef' for c in key):
            raise ValueError('key_format')
    
    PYSERVER

    Expected result: Exact wrapper bytes and Python syntax verify; no model load is claimed.

  3. 3. Complete and verify the automatic-mask harness

    Append the second half exactly, then syntax-check and verify the full-file hash before starting.

    Command
    cd "$HOME/sam2-automatic"
    cat >> sam2_automatic.py <<'PYSERVER'
        class Handler(BaseHTTPRequestHandler):
            def setup(self):
                super().setup()
                self.connection.settimeout(5)
    
            def log_message(self, *_):
                pass  # Do not log Bearer keys or request bodies.
    
            def reply(self, status, obj):
                data = json.dumps(obj, allow_nan=False, separators=(',', ':')).encode()
                if len(data)>16*1024*1024:
                    status=500
                    data=b'{"error":"output_bound"}'
                self.send_response(status)
                self.send_header('Content-Type', 'application/json')
                self.send_header('Content-Length', str(len(data)))
                self.send_header('Connection', 'close')
                self.end_headers()
                self.wfile.write(data)
                self.close_connection = True
    
            def authorized(self):
                values = self.headers.get_all('Authorization', [])
                return len(values) == 1 and hmac.compare_digest(values[0].encode(), ('Bearer ' + key).encode())
    
            def do_GET(self):
                if not self.authorized():
                    return self.reply(401, {'error': 'unauthorized'})
                if self.path != '/health':
                    return self.reply(404, {'error': 'not_found'})
                return self.reply(200, {'ready': True, 'configured_coarse_vision_positions_per_crop': 4096,
                                        'generated_text_tokens': None})
    
            def do_POST(self):
                if not self.authorized():
                    return self.reply(401, {'error': 'unauthorized'})
                if self.path != '/v1/automatic-masks':
                    return self.reply(404, {'error': 'not_found'})
                if self.headers.get('Transfer-Encoding') is not None:
                    return self.reply(400, {'error': 'transfer_encoding_unsupported'})
                lengths = self.headers.get_all('Content-Length', [])
                if len(lengths) != 1:
                    return self.reply(411, {'error': 'content_length_required'})
                raw = lengths[0]
                if not raw or len(raw) > 4 or any(c not in '0123456789' for c in raw):
                    return self.reply(413, {'error': 'invalid_or_excessive_length'})
                n = int(raw)
                if n < 1 or n > 256:
                    return self.reply(413, {'error': 'body_limit'})
                if self.headers.get_content_type() != 'application/json':
                    return self.reply(415, {'error': 'json_required'})
                try:
                    body = self.rfile.read(n)
                except (TimeoutError, OSError):
                    return self.reply(408, {'error': 'body_timeout'})
                if len(body) != n:
                    return self.reply(400, {'error': 'incomplete_body'})
                try:
                    def unique(pairs):
                        obj = {}
                        for k, v in pairs:
                            if k in obj:
                                raise ValueError('duplicate_key')
                            obj[k] = v
                        return obj
                    request = json.loads(body.decode('utf-8'), object_pairs_hook=unique,
                                         parse_constant=lambda _: (_ for _ in ()).throw(ValueError('constant')))
                except (ValueError, UnicodeError, RecursionError):
                    return self.reply(400, {'error': 'invalid_json'})
                if type(request) is not dict or request != {'fixture': 'author-geometry-v1'}:
                    return self.reply(400, {'error': 'fixed_fixture_required'})
                try:
                    result = validate_mask(encoder())
                except Exception:
                    return self.reply(500, {'error': 'predictor_or_mask_failure'})
                return self.reply(200, result)
    
        return HTTPServer(('127.0.0.1', port), Handler)
    
    
    if __name__ == '__main__':
        secret = (ROOT / '.api-key').read_text().strip()
        encoder = make_encoder()
        with create_server(encoder, secret) as server:
            server.serve_forever()
    PYSERVER
    .venv/bin/python -m py_compile sam2_automatic.py
    sha256sum --check <<'SHA256'
    273092b59c119a77e8f29e69d650cb19664cf6d8b6083da8038c31914c14e788  sam2_automatic.py
    SHA256

    Expected result: Full wrapper hash matches; no native model operations have run.

Start the model

  1. 1. Start automatic-mask generation privately

    Run in a foreground terminal. timeout bounds the total service lifetime to 900 seconds, sends TERM and then KILL after five more seconds. This is an experiment deadline, not measured inference latency. Reaching it is a failed run. The guide author has not executed CUDA, autocast or the native model.

    Command
    cd "$HOME/sam2-automatic"
    timeout --signal=TERM --kill-after=5s 900s .venv/bin/python sam2_automatic.py

    Expected result: After the reader actually loads the model, 127.0.0.1:8996 serves the fixed-input workflow. Stop on errors; empty or incorrect masks do not establish successful segmentation.

Connect to your endpoint

http://127.0.0.1:8996/v1/automatic-masks

Same trusted desktop loopback only.

Access: loopback. A private 64-character lowercase hexadecimal Bearer key is required for health and automatic masks. Only JSON with fixture=author-geometry-v1 is accepted. Uploads, paths, URLs and point, grid or model overrides are rejected. The serial HTTPServer has a 256-byte body bound, a five-second socket read timeout, a 16 MiB response cap and at most 48 masks. A foreground deadline ends the process after 900 seconds. Keep port 8996 closed publicly; use only an existing SSH tunnel if needed.

Verify the setup

  1. 1. Decode every automatic proposal and inspect spatial quality

    Run on the server from a second terminal. Health without a key must return 401. Decode each mask in native column-major order and inspect its separate colored overlay and seed point against the red rectangle and blue ellipse. Masks may overlap, miss shapes or cover background. Predicted IoU and stability are model and filter scores, not measured correctness. Zero masks is an honest transport result but leaves task quality unverified. A client timeout does not cancel native work; stop the foreground process if it expires.

    Command
    cd "$HOME/sam2-automatic"
    SAM_KEY="$(cat .api-key)"
    curl --fail --max-time 5 http://127.0.0.1:8996/health -H "Authorization: Bearer $SAM_KEY"
    curl --fail --max-time 120 http://127.0.0.1:8996/v1/automatic-masks -H "Authorization: Bearer $SAM_KEY" -H 'Content-Type: application/json' -d '{"fixture":"author-geometry-v1"}' -o masks.json
    curl -s --max-time 5 -o /dev/null -w '%{http_code}\n' http://127.0.0.1:8996/health
    unset SAM_KEY
    .venv/bin/python - <<'PYVERIFY'
    import json,math
    from pathlib import Path
    from PIL import Image,ImageDraw
    raw=Path('masks.json').read_bytes();assert len(raw)<=16*1024*1024
    d=json.loads(raw)
    assert d['configured_coarse_vision_positions_per_crop']==4096 and d['generated_text_tokens'] is None
    assert d['grid_points']==16 and d['crop_count']==1 and d['maximum_candidate_masks_before_filtering']==48
    assert d['mask_count']==len(d['records']) and 0<=d['mask_count']<=48
    original=Image.open('fixture.png').convert('RGB');combined=original.copy()
    for n,r in enumerate(d['records']):
     counts=r['segmentation']['counts'];assert r['segmentation']['size']==[224,224]
     assert 1<=len(counts)<=50177 and all(type(x) is int and 0<=x<=50176 for x in counts) and sum(counts)==50176
     assert r['area']==sum(counts[1::2])>0
     assert all(math.isfinite(r[x]) for x in ['predicted_iou','stability_score'])
     # Native uncompressed RLE is column-major, starting with background.
     mask=Image.new('L',(224,224));offset=0
     for run,count in enumerate(counts):
      if run%2:
       for index in range(offset,offset+count):mask.putpixel((index//224,index%224),255)
      offset+=count
     mask.save(f'mask-{n:02d}.png')
     color=((73*n+40)%256,(127*n+80)%256,(191*n+120)%256)
     tint=Image.new('RGB',original.size,color);overlay=original.copy();overlay.paste(Image.blend(original,tint,0.45),(0,0),mask)
     draw=ImageDraw.Draw(overlay);x,y=r['point_coords'][0];draw.ellipse((x-3,y-3,x+3,y+3),fill='yellow')
     overlay.save(f'overlay-{n:02d}.png');combined.paste(Image.blend(combined,tint,0.35),(0,0),mask)
    combined.save('overlay-all.png')
    print('Inspect every overlay and seed point against fixture.png; duplicates, missed shapes and empty output remain possible.')
    if not d['records']:raise SystemExit('No masks survived: task quality not established. Do not lower thresholds silently.')
    PYVERIFY

    Expected result: Only after real reader execution: finite bounded records pass RLE/shape/area checks and overlays are produced; manual spatial inspection determines usefulness. No exact mask count, quality, latency or fit is promised.

Performance evidence

No compatible measurement available. Documented support and estimated memory do not establish response speed.

Limits and troubleshooting

  • The exact frozen API identifies original July 2024 SAM2 Hiera Large while the catalog version field says 2.1. This guide exposes the discrepancy and uses original checkpoint/config; no 2.1 substitution, catalog rewrite or family-complete claim.
  • 4,096 is the configured count of coarse visual positions per whole-image crop. Separate 256 × 256 and 128 × 128 high-resolution features, sparse point embeddings and decoder tokens are outside that count. Workload batchSize=1 means one image per request; internal points_per_batch=4 is a separate decoder batch. The point grid and candidate counts are chosen finite bounds, not native maxima or a text-token allowance.
  • The native automatic generator applies predicted IoU > 0.8, stability >= 0.95 and within-crop box NMS at 0.7 to at most 48 candidates. These filters can return zero masks. No minimum coverage, exactly two masks, semantic identity or calibrated score is guaranteed.
  • The native default grid of 32 × 32 points is reduced to 4 × 4 for this bounded experiment; quality equivalence is unproven. The workflow uses one whole-image crop, no M2M refinement, min_mask_region_area=0 and no CUDA hole or sprinkle cleanup. COCO, pycocotools and OpenCV paths are not selected.
  • Complete uncompressed RLE uses column-major order, alternating background and foreground runs from background. XYWH boxes follow native conventions and are separate from inclusive pixel area. A dimensionally valid output can still be wrong; inspect every overlay.
  • Exact native checkpoint LFS SHA/size and Apache 2 license statements are source metadata/text evidence; no weights were downloaded or model executed. The separately published safetensors and HF-native YAML are not silently treated as identical.
  • Native runtime config RoPE initial grid 64 differs from HF-native YAML 32. Runtime recomputes frequencies for actual query shape; this source behavior is not measured numerical equivalence. The recipe pins native runtime config explicitly.
  • 32 GiB RAM, 40 GiB free disk and four threads are chosen allocations, not measured minima or proven fit. Automatic decoder batches, intermediate masks and output consume additional unmeasured memory. All model weights remain resident on CPU.
  • The reader-authored serial endpoint accepts only the fixed input and up to 256 bytes of JSON. It uses a five-second socket read timeout and 16 MiB output cap. The 900-second process deadline bounds the demonstration lifetime, with no multiuser resilience or latency guarantee. A client timeout of 120 seconds does not cancel server computation.
  • No autoregressive tokenizer, chat or reasoning template, text sampler or text cache applies. Output text tokens are not applicable. The coarse 4,096-position count excludes high-resolution features, prompt and decoder tokens and intermediates. Prompted image segmentation, video propagation and final family coverage remain distinct.
  • Direct packages are source-qualified but have not been installed or executed with the model. Fake transport and output tests prove wrapper boundaries only. Native torchvision CPU NMS, SAM2 compatibility, numerical quality and peak RAM remain unmeasured.
Empty, missed, duplicate, background or slow proposals
Inspect fixture.png and every overlay. Preserve the documented thresholds and grid for this configuration and record actual task failures. A larger grid, more crops or different filters define another workload requiring resource review. Do not infer correctness from scores.
NMS, checkpoint or state-dictionary failure
Stop and preserve the exact original SAM2 checkpoint, runtime and matching CPU torch and torchvision wheels. Do not switch silently to SAM2.1, safetensors, another backend or a workflow without NMS, or relax load checks.
401, 400, 413, 415 or 500 responses
Use the exact private key and fixed-fixture JSON. Correct content type and body framing. A 500 response indicates native or output failure. Uploads, URLs and grid or model overrides are rejected.
Client or foreground process deadline expires
A client timeout may leave serial native computation running. Stop the foreground process with Ctrl+C, confirm port 8996 is closed, and retain logs without keys. The foreground deadline terminates the process; it does not prove a completed run or measured performance.

Stop the model

  1. 1. Stop the research endpoint

    Press Ctrl+C in its foreground terminal and unset client keys. Confirm local loopback port 8996 no longer serves.

    Command
    unset SAM_KEY

    Expected result: Native process and endpoint stop; no active session state remains.

Uninstall

  1. 1. Remove dedicated experiment files

    Preserve desired results and licensing records first; confirm the dedicated directory before deletion. Shared Ubuntu packages remain.

    Command
    rm -r -- "$HOME/sam2-automatic"

    Expected result: Dedicated checkpoint, native checkout, virtualenv, generated fixture/mask/overlay, wrapper and private key removed.

Sources and verification

Sources checked Oct 17, 2026. Verification source

Compare this configuration
Cloud serverDocumented setupSources checked Oct 17, 2026

Generate automatic masks with original SAM 2 on a private AWS CPU instance

Use the exact original facebook/sam2-hiera-large checkpoint and native automatic mask generator on one author-created RGB fixture. A 4 × 4 point grid, batches of four points and three candidates per point produce at most 48 candidates before native IoU and stability filtering and box NMS. Complete masks use column-major uncompressed RLE. This automatic proposal workflow supplies no semantic classes or guaranteed object discovery. The catalog version 2.1 discrepancy stays explicit; CPU fit, speed and quality are unmeasured.

Hardware
AWS r7i.xlarge, 32 GiB RAM, Ubuntu x86_64
Runtime
Native original SAM2 CPU automatic mask generator with private research harness SAM2 source 2b90b9f5ceec907a1c18123530e92e794ad901a4; torch2.6.0+cpu; torchvision0.21.0+cpu; Python3.12; numpy2.2.3; Pillow11.1.0 · CPU
Model format
PyTorch checkpoint · Native CPU float32 eager inference; no autocast, quantization or compile optimization · Unquantized native checkpoint; reader CPU float32
Workload
4,096 token context · 1 concurrent

Based on source documentation; this setup has not been executed by OpenTools. Memory: unknown. No compatible measurement available.

Read the complete setup guide

Before you start

  • Use an owned AWS r7i.xlarge Ubuntu 24.04 x86_64 VM in us-east-1 with 32 GiB RAM, four vCPUs and a chosen 40 GiB gp3 volume. Existing SSH access must be limited to your client; keep service port 8996 closed. Provisioning and quoted/other applicable billing are reader actions.
  • The frozen catalog says version 2.1, but its exact API facebook/sam2-hiera-large maps to the original July 29, 2024 checkpoint. This recipe deliberately uses that exact API artifact and native original config; it does not silently upgrade to SAM2.1 or rewrite catalog metadata.
  • Review the exact upstream Apache 2.0 conditions and retain relevant license and notice material. This bounded experiment uses only original author-created geometry, with no third-party media, external URL, arbitrary image path, upload or user-selected model. Automatic proposals carry no semantic labels.
  • This is a source-documented CPU float32 eager experiment; no model execution or quality benchmark has occurred. Use Python 3.12 and the exact direct CPU packages below, save transitive and system resolution, and stop on pip check, checkpoint, NMS or output errors. A foreground process deadline of 900 seconds bounds the demonstration lifetime. A client timeout of 120 seconds may expire earlier while the serial computation is still running.

Exact model and requirements

Artifact:
Meta / facebookresearch / SAM2 original Hiera Large; exact frozen API artifact, not SAM2.1 · checkpoint facebook/sam2-hiera-large
Publisher revision:
e6a8e8809b8f1bfa2238b6d080f3d05cc76bd251
Artifact digest:
sha256:7442e4e9b732a508f80e141e7c2913437a3610ee0c77381a66658c3a445df87b
Download size:
0.84 GiB
Memory requirement (unknown):
Unknown — No measured full-process peak for original Hiera Large CPU float32,1024image activations, prompt decoder and wrapper. Automatic-mask prompt batches/intermediate logits and output add unmeasured overhead; prior prompted-image behavior does not establish fit.
System RAM / disk:
Unknown / Unknown
Placement:
single device. One CPU host; no tensor or pipeline sharding. Whole native model, image and prompt handling on CPU; no GPU/disk offload.
KV cache:
No autoregressive text K/V cache. Native image features are reset after generation/failure. The automatic point-grid decoder batches and intermediate masks are separate from image embedding positions.
Workload:
4,096 token context; 4,096 input tokens; Not applicable; concurrency 1; batch 1; cache cold; repetitions Unknown.
Tokenizer:
Native 1024 × 1024 RGB image embedding, stride16 coarse grid64 × 64; no text tokenizer · 2b90b9f5ceec907a1c18123530e92e794ad901a4
Sampling / reasoning:
One fixed original 224 × 224 RGB geometry image, resized natively to 1024 × 1024. A normalized 4 × 4 grid supplies 16 positive point prompts in four batches of four. multimask_output=True produces at most three candidates per point, or 48 before filtering. crop_n_layers=0 selects one whole-image crop. pred_iou_thresh=0.8 uses strict >; stability_score_thresh=0.95 uses >=; stability offset=1, mask threshold=0 and box NMS threshold=0.7. No M2M refinement, region cleanup or COCO encoder is selected. 4,096 is the configured count of coarse visual positions per whole-image crop. Separate 256 × 256 and 128 × 128 high-resolution features, sparse point embeddings and decoder tokens are outside that count. Workload batchSize=1 means one image per request; internal points_per_batch=4 is a separate decoder batch. The point grid and candidate counts are chosen finite bounds, not native maxima or a text-token allowance.
Runtime prerequisites:
Ubuntu 24.04 x86_64; See prerequisites above.

Official artifact · License

Actual upstream Apache License 2.0 text is archived, and upstream README expressly includes SAM2 model checkpoints. Retain license and relevant copyright/NOTICE material; observe redistribution/change-notice and patent conditions. No trademark endorsement or legal clearance is granted by this guide. Optional connected-component code has its own archived license; this CPU recipe skips that extension. Demo fonts and media are not included or presumed Apache-licensed. The original fixture is author-created geometry.

  • 897,952,466 bytes is official LFS metadata for this exact native checkpoint, not independently downloaded weight-body proof or full runtime RAM.
  • 32 GiB RAM,40 GiB disk and four threads are planning allocations, not measured minima or fit.1024image features, model buffers, native dependencies and OS add overhead.

One-time setup and hardware

hardware
Not applicable — AWS hardware acquisition is included in compute rental; no purchased GPU required. Any separate hardware cost is not applicable.
AWS hardware acquisition is included in compute rental; no purchased GPU required. Any separate hardware cost is not applicable. Tax: unknown.
setup
Unknown — Initial provisioning/build time and labor cost are unknown.
Initial provisioning/build time and labor cost are unknown. Tax: unknown.

Recurring costs

compute
$0.26 / hour
AWS public us-east-1 Linux On-Demand rate USD 0.2646 per billed instance-hour, no paid preinstalled software. Eight hours USD 2.1168; 24 hours USD 6.3504 compute. Idle running hours bill. Storage, egress, tax and administration separate; no Spot/reservation/discount assumed. Capacity/quota unverified. Region: us-east-1. SKU: r7i.xlarge Linux On-Demand. Tax: excluded. Price dated Oct 15, 2026. Price source
storage
$0.08 / GB month
US East (N. Virginia) gp3 USD 0.08 per GB-month; AWS defines GB as 1024^3 bytes. Selected 40 GiB root volume is USD 3.20 per full month, prorated by provisioned duration. Baseline 3000 IOPS / 125 MB/s; no extras assumed. Retained snapshots/resources separate; storage persists after instance stop. Tax excluded. Region: us-east-1. SKU: EBS gp3 baseline 3000 IOPS / 125 MB/s. Tax: excluded. Price dated Oct 16, 2026. Price source
egress
Unknown — Data transfer price, allowances and traffic volume are unknown.
Data transfer price, allowances and traffic volume are unknown. Tax: unknown.
administration
Unknown — Maintenance labor rate and hours are unknown.
Maintenance labor rate and hours are unknown. Tax: unknown.

Install the runtime

  1. 1. Create the native CPU environment

    Use exact official CPU wheels and a pinned native checkout. Explicitly skip the optional CUDA extension. Pin direct dependencies, save transitive/system resolution and stop on pip check failure; this is documented support rather than an executed environment.

    Command
    sudo apt-get update
    sudo apt-get install -y python3.12 python3.12-venv git curl ca-certificates openssl
    mkdir -p "$HOME/sam2-automatic/models"
    cd "$HOME/sam2-automatic"
    python3.12 -m venv .venv
    .venv/bin/python -m pip install 'https://download.pytorch.org/whl/cpu/torch-2.6.0%2Bcpu-cp312-cp312-linux_x86_64.whl#sha256=59e78aa0c690f70734e42670036d6b541930b8eabbaa18d94e090abf14cc4d91' 'https://download-r2.pytorch.org/whl/cpu/torchvision-0.21.0%2Bcpu-cp312-cp312-linux_x86_64.whl#sha256=d6874431e678ba107b60a83f255c33f3755f06bad587b1b919aa514ec325dcd8'
    .venv/bin/python -m pip install 'numpy==2.2.3' 'Pillow==11.1.0' 'hydra-core==1.3.2' 'omegaconf==2.3.0' 'antlr4-python3-runtime==4.9.3' 'iopath==0.1.10' 'tqdm==4.67.1'
    git clone https://github.com/facebookresearch/sam2.git native-sam2
    git -C native-sam2 checkout --detach 2b90b9f5ceec907a1c18123530e92e794ad901a4
    SAM2_BUILD_CUDA=0 .venv/bin/python -m pip install -e ./native-sam2
    .venv/bin/python -m pip check
    .venv/bin/python -m pip freeze > resolved-environment.txt

    Expected result: Pinned checkout and hashed CPU wheels resolve; extension build is disabled, pip check passes and versions are saved. No CUDA driver/compiler or GPU runtime is required by this chosen path.

Download the model

  1. 1. Verify the exact original checkpoint and config

    Reader download only: the author fetched metadata/text, not weights. Use exact HF revision for native .pt and exact config from pinned runtime. HF safetensors and HF-native YAML are different artifacts/config revisions and are not substituted.

    Command
    cd "$HOME/sam2-automatic"
    curl --fail --location --output models/sam2_hiera_large.pt 'https://huggingface.co/facebook/sam2-hiera-large/resolve/e6a8e8809b8f1bfa2238b6d080f3d05cc76bd251/sam2_hiera_large.pt'
    sha256sum --check <<'SHA256'
    7442e4e9b732a508f80e141e7c2913437a3610ee0c77381a66658c3a445df87b  models/sam2_hiera_large.pt
    0ee9f2037fd98489212bb0b78a56f33ac82eb177cf5e52c88be3714cb84eea26  native-sam2/sam2/configs/sam2/sam2_hiera_l.yaml
    c71d239df91726fc519c6eb72d318ec65820627232b2f796219e87dcf35d0ab4  native-sam2/LICENSE
    SHA256

    Expected result: All three exact hashes report OK; stop on mismatch. Local native loader uses weights_only=True and checks state-dictionary keys. No native convenience from_pretrained call floats the model revision.

Configure

  1. 1. Create a stable private service key

    Generate only when absent; preserve existing access. Do not print or share this local service key.

    Command
    cd "$HOME/sam2-automatic"
    umask 077
    if [ ! -e .api-key ]; then openssl rand -hex 32 > .api-key; fi
    chmod 600 .api-key
    test -s .api-key || { echo "Restore empty service key before continuing"; exit 1; }

    Expected result: Private existing key is preserved; server requires 64 lowercase hexadecimal characters.

  2. 2. Create the bounded native automatic-mask harness

    The complete reader-authored wrapper is provided in two consecutive steps. It preserves native automatic mask generation with 16 grid points, batches of four points, one whole-image crop, native candidate filtering and NMS, and column-major RLE. It validates output bounds and resets predictor state in finally. Requests cannot change the image, grid or model. Syntax and hash verification do not load a model.

    Command
    cd "$HOME/sam2-automatic"
    cat > sam2_automatic.py <<'PYSERVER'
    """Reader-authored fixed automatic-mask experiment. Native model path is reader-run only."""
    import hashlib
    import hmac
    import json
    import math
    import pathlib
    from http.server import BaseHTTPRequestHandler, HTTPServer
    
    ROOT = pathlib.Path(__file__).resolve().parent
    PORT = 8996
    MODEL_SHA = '7442e4e9b732a508f80e141e7c2913437a3610ee0c77381a66658c3a445df87b'
    
    
    def validate_mask(records):
        if type(records) is not list or len(records) > 48:
            raise ValueError('record_count')
        checked=[]
        for r in records:
            if type(r) is not dict or set(r) != {'segmentation','area','bbox','predicted_iou','point_coords','stability_score','crop_box'}:
                raise ValueError('record_fields')
            seg=r['segmentation']
            if type(seg) is not dict or set(seg) != {'size','counts'} or seg['size'] != [224,224]:
                raise ValueError('rle_shape')
            counts=seg['counts']
            if type(counts) is not list or not 1 <= len(counts) <= 50177 or any(type(v) is not int or v < 0 or v > 50176 for v in counts) or sum(counts) != 50176:
                raise ValueError('rle_counts')
            area=sum(counts[1::2])
            if type(r['area']) is not int or r['area'] != area or area==0:
                raise ValueError('mask_area')
            for k in ['predicted_iou','stability_score']:
                if type(r[k]) not in (float,int) or not math.isfinite(r[k]):
                    raise ValueError('nonfinite_score')
            for k in ['bbox','crop_box']:
                v=r[k]
                if type(v) is not list or len(v)!=4 or any(type(x) not in (float,int) or not math.isfinite(x) or x<0 or x>224 for x in v) or v[0]+v[2]>224 or v[1]+v[3]>224:
                    raise ValueError('xywh_box')
            if r['crop_box'] != [0,0,224,224]:
                raise ValueError('unexpected_crop')
            pts=r['point_coords']
            if type(pts) is not list or len(pts)!=1 or type(pts[0]) is not list or len(pts[0])!=2 or any(type(v) not in (float,int) or not math.isfinite(v) or v not in (28.0,84.0,140.0,196.0) for v in pts[0]):
                raise ValueError('grid_point')
            checked.append(r)
        return {'fixture':'author-geometry-v1','image_shape':[224,224,3],
                'configured_coarse_vision_positions_per_crop':4096,
                'grid_points':16,'points_per_batch':4,'crop_count':1,
                'maximum_candidate_masks_before_filtering':48,
                'mask_count':len(checked),'records':checked,
                'rle_order':'column-major alternating background/foreground starting with background',
                'generated_text_tokens':None,'semantic_classification':False}
    
    
    def make_encoder():
        import numpy as np
        import torch
        from PIL import Image, ImageDraw
        from sam2.build_sam import build_sam2
        from sam2.automatic_mask_generator import SAM2AutomaticMaskGenerator
        torch.set_num_threads(4)
        artifact=ROOT/'models/sam2_hiera_large.pt'
        if hashlib.sha256(artifact.read_bytes()).hexdigest()!=MODEL_SHA:
            raise RuntimeError('checkpoint_hash_mismatch')
        model=build_sam2('configs/sam2/sam2_hiera_l.yaml',str(artifact),device='cpu',apply_postprocessing=False).float().eval()
        generator=SAM2AutomaticMaskGenerator(model,points_per_side=4,points_per_batch=4,
            pred_iou_thresh=0.8,stability_score_thresh=0.95,stability_score_offset=1.0,
            mask_threshold=0.0,box_nms_thresh=0.7,crop_n_layers=0,crop_nms_thresh=0.7,
            crop_overlap_ratio=512/1500,crop_n_points_downscale_factor=1,
            min_mask_region_area=0,output_mode='uncompressed_rle',use_m2m=False,
            multimask_output=True)
        image=Image.new('RGB',(224,224),(240,240,240));draw=ImageDraw.Draw(image)
        draw.rectangle((24,24,104,200),fill=(220,40,40));draw.ellipse((120,64,208,152),fill=(30,90,220))
        image.save(ROOT/'fixture.png')
        def encode():
            try:
                with torch.inference_mode():
                    records=generator.generate(np.array(image,dtype=np.uint8))
                    return records
            finally:
                generator.predictor.reset_predictor()
        return encode
    
    
    def create_server(encoder, key, port=PORT):
        if not isinstance(key, str) or len(key) != 64 or any(c not in '0123456789abcdef' for c in key):
            raise ValueError('key_format')
    
    PYSERVER

    Expected result: Exact wrapper bytes and Python syntax verify; no model load is claimed.

  3. 3. Complete and verify the automatic-mask harness

    Append the second half exactly, then syntax-check and verify the full-file hash before starting.

    Command
    cd "$HOME/sam2-automatic"
    cat >> sam2_automatic.py <<'PYSERVER'
        class Handler(BaseHTTPRequestHandler):
            def setup(self):
                super().setup()
                self.connection.settimeout(5)
    
            def log_message(self, *_):
                pass  # Do not log Bearer keys or request bodies.
    
            def reply(self, status, obj):
                data = json.dumps(obj, allow_nan=False, separators=(',', ':')).encode()
                if len(data)>16*1024*1024:
                    status=500
                    data=b'{"error":"output_bound"}'
                self.send_response(status)
                self.send_header('Content-Type', 'application/json')
                self.send_header('Content-Length', str(len(data)))
                self.send_header('Connection', 'close')
                self.end_headers()
                self.wfile.write(data)
                self.close_connection = True
    
            def authorized(self):
                values = self.headers.get_all('Authorization', [])
                return len(values) == 1 and hmac.compare_digest(values[0].encode(), ('Bearer ' + key).encode())
    
            def do_GET(self):
                if not self.authorized():
                    return self.reply(401, {'error': 'unauthorized'})
                if self.path != '/health':
                    return self.reply(404, {'error': 'not_found'})
                return self.reply(200, {'ready': True, 'configured_coarse_vision_positions_per_crop': 4096,
                                        'generated_text_tokens': None})
    
            def do_POST(self):
                if not self.authorized():
                    return self.reply(401, {'error': 'unauthorized'})
                if self.path != '/v1/automatic-masks':
                    return self.reply(404, {'error': 'not_found'})
                if self.headers.get('Transfer-Encoding') is not None:
                    return self.reply(400, {'error': 'transfer_encoding_unsupported'})
                lengths = self.headers.get_all('Content-Length', [])
                if len(lengths) != 1:
                    return self.reply(411, {'error': 'content_length_required'})
                raw = lengths[0]
                if not raw or len(raw) > 4 or any(c not in '0123456789' for c in raw):
                    return self.reply(413, {'error': 'invalid_or_excessive_length'})
                n = int(raw)
                if n < 1 or n > 256:
                    return self.reply(413, {'error': 'body_limit'})
                if self.headers.get_content_type() != 'application/json':
                    return self.reply(415, {'error': 'json_required'})
                try:
                    body = self.rfile.read(n)
                except (TimeoutError, OSError):
                    return self.reply(408, {'error': 'body_timeout'})
                if len(body) != n:
                    return self.reply(400, {'error': 'incomplete_body'})
                try:
                    def unique(pairs):
                        obj = {}
                        for k, v in pairs:
                            if k in obj:
                                raise ValueError('duplicate_key')
                            obj[k] = v
                        return obj
                    request = json.loads(body.decode('utf-8'), object_pairs_hook=unique,
                                         parse_constant=lambda _: (_ for _ in ()).throw(ValueError('constant')))
                except (ValueError, UnicodeError, RecursionError):
                    return self.reply(400, {'error': 'invalid_json'})
                if type(request) is not dict or request != {'fixture': 'author-geometry-v1'}:
                    return self.reply(400, {'error': 'fixed_fixture_required'})
                try:
                    result = validate_mask(encoder())
                except Exception:
                    return self.reply(500, {'error': 'predictor_or_mask_failure'})
                return self.reply(200, result)
    
        return HTTPServer(('127.0.0.1', port), Handler)
    
    
    if __name__ == '__main__':
        secret = (ROOT / '.api-key').read_text().strip()
        encoder = make_encoder()
        with create_server(encoder, secret) as server:
            server.serve_forever()
    PYSERVER
    .venv/bin/python -m py_compile sam2_automatic.py
    sha256sum --check <<'SHA256'
    273092b59c119a77e8f29e69d650cb19664cf6d8b6083da8038c31914c14e788  sam2_automatic.py
    SHA256

    Expected result: Full wrapper hash matches; no native model operations have run.

Start the model

  1. 1. Start automatic-mask generation privately

    Run in a foreground terminal. timeout bounds the total service lifetime to 900 seconds, sends TERM and then KILL after five more seconds. This is an experiment deadline, not measured inference latency. Reaching it is a failed run. The guide author has not executed CUDA, autocast or the native model.

    Command
    cd "$HOME/sam2-automatic"
    timeout --signal=TERM --kill-after=5s 900s .venv/bin/python sam2_automatic.py

    Expected result: After the reader actually loads the model, 127.0.0.1:8996 serves the fixed-input workflow. Stop on errors; empty or incorrect masks do not establish successful segmentation.

Connect to your endpoint

http://127.0.0.1:8996/v1/automatic-masks

VM loopback only. Optional client tunnel with existing SSH access: ssh -N -L 8996:127.0.0.1:8996 USER@VM_IP. Keep EC2 port8996 closed; shown checks run on server.

Access: loopback. A private 64-character lowercase hexadecimal Bearer key is required for health and automatic masks. Only JSON with fixture=author-geometry-v1 is accepted. Uploads, paths, URLs and point, grid or model overrides are rejected. The serial HTTPServer has a 256-byte body bound, a five-second socket read timeout, a 16 MiB response cap and at most 48 masks. A foreground deadline ends the process after 900 seconds. Keep port 8996 closed publicly; use only an existing SSH tunnel if needed.

Verify the setup

  1. 1. Decode every automatic proposal and inspect spatial quality

    Run on the server from a second terminal. Health without a key must return 401. Decode each mask in native column-major order and inspect its separate colored overlay and seed point against the red rectangle and blue ellipse. Masks may overlap, miss shapes or cover background. Predicted IoU and stability are model and filter scores, not measured correctness. Zero masks is an honest transport result but leaves task quality unverified. A client timeout does not cancel native work; stop the foreground process if it expires.

    Command
    cd "$HOME/sam2-automatic"
    SAM_KEY="$(cat .api-key)"
    curl --fail --max-time 5 http://127.0.0.1:8996/health -H "Authorization: Bearer $SAM_KEY"
    curl --fail --max-time 120 http://127.0.0.1:8996/v1/automatic-masks -H "Authorization: Bearer $SAM_KEY" -H 'Content-Type: application/json' -d '{"fixture":"author-geometry-v1"}' -o masks.json
    curl -s --max-time 5 -o /dev/null -w '%{http_code}\n' http://127.0.0.1:8996/health
    unset SAM_KEY
    .venv/bin/python - <<'PYVERIFY'
    import json,math
    from pathlib import Path
    from PIL import Image,ImageDraw
    raw=Path('masks.json').read_bytes();assert len(raw)<=16*1024*1024
    d=json.loads(raw)
    assert d['configured_coarse_vision_positions_per_crop']==4096 and d['generated_text_tokens'] is None
    assert d['grid_points']==16 and d['crop_count']==1 and d['maximum_candidate_masks_before_filtering']==48
    assert d['mask_count']==len(d['records']) and 0<=d['mask_count']<=48
    original=Image.open('fixture.png').convert('RGB');combined=original.copy()
    for n,r in enumerate(d['records']):
     counts=r['segmentation']['counts'];assert r['segmentation']['size']==[224,224]
     assert 1<=len(counts)<=50177 and all(type(x) is int and 0<=x<=50176 for x in counts) and sum(counts)==50176
     assert r['area']==sum(counts[1::2])>0
     assert all(math.isfinite(r[x]) for x in ['predicted_iou','stability_score'])
     # Native uncompressed RLE is column-major, starting with background.
     mask=Image.new('L',(224,224));offset=0
     for run,count in enumerate(counts):
      if run%2:
       for index in range(offset,offset+count):mask.putpixel((index//224,index%224),255)
      offset+=count
     mask.save(f'mask-{n:02d}.png')
     color=((73*n+40)%256,(127*n+80)%256,(191*n+120)%256)
     tint=Image.new('RGB',original.size,color);overlay=original.copy();overlay.paste(Image.blend(original,tint,0.45),(0,0),mask)
     draw=ImageDraw.Draw(overlay);x,y=r['point_coords'][0];draw.ellipse((x-3,y-3,x+3,y+3),fill='yellow')
     overlay.save(f'overlay-{n:02d}.png');combined.paste(Image.blend(combined,tint,0.35),(0,0),mask)
    combined.save('overlay-all.png')
    print('Inspect every overlay and seed point against fixture.png; duplicates, missed shapes and empty output remain possible.')
    if not d['records']:raise SystemExit('No masks survived: task quality not established. Do not lower thresholds silently.')
    PYVERIFY

    Expected result: Only after real reader execution: finite bounded records pass RLE/shape/area checks and overlays are produced; manual spatial inspection determines usefulness. No exact mask count, quality, latency or fit is promised.

Performance evidence

No compatible measurement available. Documented support and estimated memory do not establish response speed.

Limits and troubleshooting

  • The exact frozen API identifies original July 2024 SAM2 Hiera Large while the catalog version field says 2.1. This guide exposes the discrepancy and uses original checkpoint/config; no 2.1 substitution, catalog rewrite or family-complete claim.
  • 4,096 is the configured count of coarse visual positions per whole-image crop. Separate 256 × 256 and 128 × 128 high-resolution features, sparse point embeddings and decoder tokens are outside that count. Workload batchSize=1 means one image per request; internal points_per_batch=4 is a separate decoder batch. The point grid and candidate counts are chosen finite bounds, not native maxima or a text-token allowance.
  • The native automatic generator applies predicted IoU > 0.8, stability >= 0.95 and within-crop box NMS at 0.7 to at most 48 candidates. These filters can return zero masks. No minimum coverage, exactly two masks, semantic identity or calibrated score is guaranteed.
  • The native default grid of 32 × 32 points is reduced to 4 × 4 for this bounded experiment; quality equivalence is unproven. The workflow uses one whole-image crop, no M2M refinement, min_mask_region_area=0 and no CUDA hole or sprinkle cleanup. COCO, pycocotools and OpenCV paths are not selected.
  • Complete uncompressed RLE uses column-major order, alternating background and foreground runs from background. XYWH boxes follow native conventions and are separate from inclusive pixel area. A dimensionally valid output can still be wrong; inspect every overlay.
  • Exact native checkpoint LFS SHA/size and Apache 2 license statements are source metadata/text evidence; no weights were downloaded or model executed. The separately published safetensors and HF-native YAML are not silently treated as identical.
  • Native runtime config RoPE initial grid 64 differs from HF-native YAML 32. Runtime recomputes frequencies for actual query shape; this source behavior is not measured numerical equivalence. The recipe pins native runtime config explicitly.
  • 32 GiB RAM, 40 GiB free disk and four threads are chosen allocations, not measured minima or proven fit. Automatic decoder batches, intermediate masks and output consume additional unmeasured memory. All model weights remain resident on CPU.
  • The reader-authored serial endpoint accepts only the fixed input and up to 256 bytes of JSON. It uses a five-second socket read timeout and 16 MiB output cap. The 900-second process deadline bounds the demonstration lifetime, with no multiuser resilience or latency guarantee. A client timeout of 120 seconds does not cancel server computation.
  • No autoregressive tokenizer, chat or reasoning template, text sampler or text cache applies. Output text tokens are not applicable. The coarse 4,096-position count excludes high-resolution features, prompt and decoder tokens and intermediates. Prompted image segmentation, video propagation and final family coverage remain distinct.
  • Direct packages are source-qualified but have not been installed or executed with the model. Fake transport and output tests prove wrapper boundaries only. Native torchvision CPU NMS, SAM2 compatibility, numerical quality and peak RAM remain unmeasured.
Empty, missed, duplicate, background or slow proposals
Inspect fixture.png and every overlay. Preserve the documented thresholds and grid for this configuration and record actual task failures. A larger grid, more crops or different filters define another workload requiring resource review. Do not infer correctness from scores.
NMS, checkpoint or state-dictionary failure
Stop and preserve the exact original SAM2 checkpoint, runtime and matching CPU torch and torchvision wheels. Do not switch silently to SAM2.1, safetensors, another backend or a workflow without NMS, or relax load checks.
401, 400, 413, 415 or 500 responses
Use the exact private key and fixed-fixture JSON. Correct content type and body framing. A 500 response indicates native or output failure. Uploads, URLs and grid or model overrides are rejected.
Client or foreground process deadline expires
A client timeout may leave serial native computation running. Stop the foreground process with Ctrl+C, confirm port 8996 is closed, and retain logs without keys. The foreground deadline terminates the process; it does not prove a completed run or measured performance.

Stop the model

  1. 1. Stop the research endpoint

    Press Ctrl+C in its foreground terminal and unset client keys. Close optional SSH tunnel. Stopping Python does not stop VM billing.

    Command
    unset SAM_KEY

    Expected result: Native process and endpoint stop; no active session state remains.

Uninstall

  1. 1. Remove dedicated experiment files

    Preserve desired results and licensing records first; confirm the dedicated directory before deletion. Shared Ubuntu packages remain.

    Command
    rm -r -- "$HOME/sam2-automatic"

    Expected result: Dedicated checkpoint, native checkout, virtualenv, generated fixture/mask/overlay, wrapper and private key removed.

Shut down cloud resources

  1. 1. End cloud billing resources

    In EC2 us-east-1, stop for later reuse or terminate this exact VM. Review Delete on termination for its root volume. Delete unwanted detached EBS volumes, snapshots and reserved/Elastic/public IP resources. Stopped VM storage and retained resources may still bill. Check Bills/Cost Explorer later for final usage.

    Expected result: Exact instance is stopped/terminated as intended; no unwanted attached/retained billed resources remain. Instance compute stops only after its lifecycle transition, not when the Python service stops.

Sources and verification

Sources checked Oct 17, 2026. Verification source

Compare this configuration
Local hardwareDocumented setupSources checked Oct 17, 2026

Propagate an original SAM 2 point prompt through four frames on a research CPU desktop

Use the complete original facebook/sam2-hiera-large checkpoint and native temporal video predictor on four ordered author-created JPEG frames. A positive point seeds object 1 in frame zero; native memory then propagates a mask as a red rectangle moves right. This is a bounded synthetic spatial tracking experiment, not text generation, semantic recognition or measured tracking quality. The frozen version says 2.1 but the selected exact API artifact is original SAM 2. Fit, speed and temporal mask accuracy remain unmeasured.

Hardware
Intel Core i5-12400 desktop, 4 inference threads, 32 GiB RAM
Runtime
Native original SAM 2 finite JPEG temporal video predictor CPU SAM 2 source 2b90b9f; Torch 2.6.0 CPU; torchvision 0.21.0 CPU; Python 3.12 · CPU
Model format
PyTorch checkpoint · CPU model FP32 eager; native stored mask-memory features BF16; no autocast or compile. · Unquantized native checkpoint; reader CPU float32
Workload
4,096 token context · 1 concurrent

Based on source documentation; this setup has not been executed by OpenTools. Memory: unknown. No compatible measurement available.

Read the complete setup guide

Before you start

  • Use an owned Intel Core i5-12400 desktop with Ubuntu 24.04 x86_64, 32 GiB RAM, 40 GiB free disk and a four-thread allocation. These are planning choices, not measured minima.
  • The frozen catalog says version 2.1, but its exact API facebook/sam2-hiera-large maps to the original July 29, 2024 checkpoint. This recipe deliberately uses that exact API artifact and native original config; it does not silently upgrade to SAM 2.1 or rewrite catalog metadata.
  • Review Apache 2.0 obligations for the selected model/runtime. Use only the four original moving-geometry frames generated by this harness; no third-party media, uploads, external URLs, file paths or model choices are accepted.
  • This is a source-documented private educational temporal segmentation experiment, not measured quality or a production service. Python 3.12 and the source-supported CPU package versions below have not been installed/model-tested by the author. Save dependency resolution and stop on pip check, state-dictionary or shape errors.
  • The fixed fixture is original author-created geometry generated locally by the reader using Pillow. No publisher demo-media license is inferred, and no media is downloaded. JPEG encoding bytes are reader-generated and not claimed to have publisher hashes; four filenames, dimensions and deterministic drawing instructions are explicit.

Exact model and requirements

Artifact:
Meta / facebookresearch / SAM 2 original Hiera Large; exact frozen API artifact, not SAM 2.1 · checkpoint facebook/sam2-hiera-large
Publisher revision:
e6a8e8809b8f1bfa2238b6d080f3d05cc76bd251
Artifact digest:
sha256:7442e4e9b732a508f80e141e7c2913437a3610ee0c77381a66658c3a445df87b
Download size:
0.84 GiB
Memory requirement (unknown):
Unknown — No measured whole-process peak with four normalized 1024 frames, model, high-resolution features, memory encoder/attention and output masks.
System RAM / disk:
Unknown / Unknown
Placement:
single device. One CPU host; no tensor or pipeline sharding. Whole predictor executes on CPU. Native offload_video_to_cpu=True and offload_state_to_cpu=True keep frame/state storage on that same CPU; no GPU/disk offload.
KV cache:
Not autoregressive text K/V caching. Native temporal mask-memory features are BF16; positional encodings, object pointers, cached image features and frame/state tensors remain native managed state.
Workload:
4,096 token context; Not applicable; Not applicable; concurrency 1; batch 1; cache cold; repetitions Unknown.
Tokenizer:
Native original SAM 2 per-frame image grid; JPEG folder numeric order and spatial prompts, no text tokenizer · 2b90b9f5ceec907a1c18123530e92e794ad901a4
Sampling / reasoning:
Exactly four original 224 × 224 JPEG frames numbered 00000.jpg through 00003.jpg, generated by the reader from code. The rectangle left edge is 24, 48, 72, 96 pixels; fixed positive XY point [56,112] on frame 0, object ID 1. Native forward propagation start_frame_idx = 0 and max_frame_num_to_track = 3 is inclusive and yields indices 0 through 3. No recorded FPS/duration, frame interpolation, external file/URL input, reverse or correction prompts. The configured context value of 4,096 is the configured per-frame coarse 64 × 64 visual grid at native 1024 × 1024 resize. Four frames do not turn this into a 4096-position whole-video capacity: high-resolution features, prompt/decoder tokens, temporal mask memory and object-pointer sequences are distinct. Native num_maskmem = 7 is a memory-selection configuration, not a global video-frame maximum. This recipe bounds actual input at four frames and one object.
Runtime prerequisites:
Ubuntu 24.04 x86_64; See prerequisites above.

Official artifact · License

Actual upstream Apache License 2.0 text is archived, and upstream README expressly includes SAM 2 model checkpoints. Retain license and relevant copyright/NOTICE material; observe redistribution/change-notice and patent conditions. No trademark endorsement or legal clearance is granted by this guide. Optional connected-component code has its own archived license; this CPU recipe skips that extension. Demo fonts and media are not included or presumed Apache-licensed. The original fixture is author-created geometry.

  • 32 GiB RAM / 40 GiB disk / four threads are existing author planning allocations, not measured minima or fit.
  • The exact checkpoint is 897,952,466 serialized bytes; native video state and FP32 model execution add unmeasured overhead. All four frames are loaded eagerly; no long-video or streaming memory bound is claimed.

One-time setup and hardware

hardware
Unknown — Existing desktop purchase price and incremental hardware cost depend on the reader; unknown.
Existing desktop purchase price and incremental hardware cost depend on the reader; unknown. Tax: unknown.
setup
Unknown — Initial provisioning/build time and labor cost are unknown.
Initial provisioning/build time and labor cost are unknown. Tax: unknown.

Recurring costs

electricity
Unknown — Actual wall power, electricity rate and powered hours are unknown; no TDP-to-wall-power conversion.
Actual wall power, electricity rate and powered hours are unknown; no TDP-to-wall-power conversion. Tax: unknown.
storage
Unknown — Incremental cost of local disk capacity is unknown. The 40 GiB free-disk walkthrough allocation is an author choice, not a measured requirement.
Incremental cost of local disk capacity is unknown. The 40 GiB free-disk walkthrough allocation is an author choice, not a measured requirement. Tax: unknown.
egress
Unknown — Data transfer price, allowances and traffic volume are unknown.
Data transfer price, allowances and traffic volume are unknown. Tax: unknown.
administration
Unknown — Maintenance labor rate and hours are unknown.
Maintenance labor rate and hours are unknown. Tax: unknown.

Install the runtime

  1. 1. Create the native CPU environment

    Use exact official CPU wheels and a pinned native checkout. Explicitly skip the optional CUDA extension. Pin direct dependencies, save transitive/system resolution and stop on pip check failure; this is documented support rather than an executed environment.

    Command
    set -e
    sudo apt-get update
    sudo apt-get install -y python3.12 python3.12-venv git curl ca-certificates openssl
    mkdir -p "$HOME/sam2-video/models"
    cd "$HOME/sam2-video"
    python3.12 -m venv .venv
    .venv/bin/python -m pip install 'https://download.pytorch.org/whl/cpu/torch-2.6.0%2Bcpu-cp312-cp312-linux_x86_64.whl#sha256=59e78aa0c690f70734e42670036d6b541930b8eabbaa18d94e090abf14cc4d91' 'https://download-r2.pytorch.org/whl/cpu/torchvision-0.21.0%2Bcpu-cp312-cp312-linux_x86_64.whl#sha256=d6874431e678ba107b60a83f255c33f3755f06bad587b1b919aa514ec325dcd8'
    .venv/bin/python -m pip install 'numpy==2.2.3' 'Pillow==11.1.0' 'hydra-core==1.3.2' 'omegaconf==2.3.0' 'antlr4-python3-runtime==4.9.3' 'iopath==0.1.10' 'tqdm==4.67.1'
    git clone https://github.com/facebookresearch/sam2.git native-sam2
    git -C native-sam2 checkout --detach 2b90b9f5ceec907a1c18123530e92e794ad901a4
    SAM2_BUILD_CUDA=0 .venv/bin/python -m pip install -e ./native-sam2
    .venv/bin/python -m pip check
    .venv/bin/python -m pip freeze > resolved-environment.txt

    Expected result: Pinned checkout and hashed CPU wheels resolve; extension build is disabled, pip check passes and versions are saved. No CUDA driver/compiler or GPU runtime is required by this chosen path.

Download the model

  1. 1. Verify the exact original checkpoint and config

    Reader download only: the author fetched metadata/text, not weights. Use exact HF revision for native .pt and exact config from pinned runtime. HF safetensors and HF-native YAML are different artifacts/config revisions and are not substituted.

    Command
    set -e
    cd "$HOME/sam2-video"
    curl --fail --location --output models/sam2_hiera_large.pt 'https://huggingface.co/facebook/sam2-hiera-large/resolve/e6a8e8809b8f1bfa2238b6d080f3d05cc76bd251/sam2_hiera_large.pt'
    sha256sum --check <<'SHA256'
    7442e4e9b732a508f80e141e7c2913437a3610ee0c77381a66658c3a445df87b  models/sam2_hiera_large.pt
    0ee9f2037fd98489212bb0b78a56f33ac82eb177cf5e52c88be3714cb84eea26  native-sam2/sam2/configs/sam2/sam2_hiera_l.yaml
    c71d239df91726fc519c6eb72d318ec65820627232b2f796219e87dcf35d0ab4  native-sam2/LICENSE
    SHA256

    Expected result: All three exact hashes report OK; stop on mismatch. Local native loader uses weights_only=True and checks state-dictionary keys. No native convenience from_pretrained call floats the model revision.

Configure

  1. 1. Create a stable private service key

    Generate only when absent; preserve existing access. Do not print or share this local service key.

    Command
    set -e
    cd "$HOME/sam2-video"
    umask 077
    if [ ! -e .api-key ]; then openssl rand -hex 32 > .api-key; fi
    chmod 600 .api-key
    test -s .api-key || { echo "Restore empty service key before continuing"; exit 1; }

    Expected result: Private existing key is preserved; server requires 64 lowercase hexadecimal characters.

  2. 2. Create the reader-authored temporal harness

    The custom serial server accepts only a fixed fixture identifier. It creates four original JPEG frames, uses the complete native video predictor and one positive point, and resets state after every request. Native CPU device, model and source are unmodified; optional CUDA hole filling and dynamic multimask fallback stay disabled. No MP4/decord/FFmpeg codec route is used. This is authored deployment prescription, not publisher server code.

    Command
    set -e
    cd "$HOME/sam2-video"
    cat > sam2_video.py <<'PYCORE'
    """Reader-authored fixed four-JPEG temporal SAM2 research harness; not publisher server."""
    import hashlib,hmac,json,math,pathlib
    from http.server import BaseHTTPRequestHandler,HTTPServer
    ROOT=pathlib.Path(__file__).resolve().parent
    PORT=8994
    MODEL_SHA='7442e4e9b732a508f80e141e7c2913437a3610ee0c77381a66658c3a445df87b'
    
    def validate_video(results):
        if type(results) is not list or len(results)!=4: raise ValueError('frame_count')
        frames=[]
        for expected,result in enumerate(results):
            if type(result) is not dict or result.get('frame_index')!=expected or type(result.get('frame_index'))is not int or (type(result.get('object_ids'))is not list or len(result['object_ids'])!=1 or type(result['object_ids'][0])is not int or result['object_ids'][0]!=1): raise ValueError('order_or_identity')
            mask=result.get('mask')
            if type(mask)is not list or len(mask)!=224:raise ValueError('mask_shape')
            flat=[]
            for row in mask:
                if type(row)is not list or len(row)!=224 or any(type(v)is not bool for v in row):raise ValueError('mask_values')
                flat.extend(row)
            runs=[];i=0
            while i<len(flat):
                if not flat[i]:i+=1;continue
                start=i
                while i<len(flat)and flat[i]:i+=1
                runs.append([start,i-start])
            frames.append({'frame_index':expected,'object_ids':[1],'mask_shape':[1,1,224,224],'mask_true_runs':runs,'mask_area_pixels':sum(flat)})
        return {'fixture':'author-moving-geometry-v1','frame_count':4,'frame_order':[0,1,2,3],'point_xy':[56,112],'point_frame_index':0,'point_label':1,'frames':frames,'configured_coarse_vision_positions_per_frame':4096,'generated_text_tokens':None,'real_time_fps':None,'semantic_classification':False}
    
    def make_encoder():
        # Native imports only in the reader-run path. Tests inject a fake callable.
        import numpy as np
        import torch
        from PIL import Image,ImageDraw
        from sam2.build_sam import build_sam2_video_predictor
        torch.set_num_threads(4)
        artifact=ROOT/'models/sam2_hiera_large.pt'
        if hashlib.sha256(artifact.read_bytes()).hexdigest()!=MODEL_SHA:raise RuntimeError('checkpoint_hash_mismatch')
        predictor=build_sam2_video_predictor('configs/sam2/sam2_hiera_l.yaml',str(artifact),device='cpu',apply_postprocessing=False,vos_optimized=False).float().eval()
        if predictor.fill_hole_area!=0 or predictor.image_size!=1024:raise RuntimeError('native_settings')
        folder=ROOT/'frames';folder.mkdir(exist_ok=True)
        expected_names={f'{i:05d}.jpg'for i in range(4)}
        if any(x.name not in expected_names for x in folder.iterdir()):raise RuntimeError('unexpected_fixture_file')
        # Author-created geometry, four original ordered frames; no external media rights.
        for i in range(4):
            image=Image.new('RGB',(224,224),(240,240,240));draw=ImageDraw.Draw(image)
            x=24+24*i
            draw.rectangle((x,56,x+64,168),fill=(220,40,40))
            draw.ellipse((156,12,208,44),fill=(30,90,220))
            image.save(folder/f'{i:05d}.jpg',format='JPEG',quality=95,subsampling=0)
        if {x.name for x in folder.iterdir()}!=expected_names:raise RuntimeError('fixture_manifest')
        def encode():
            state=None
            try:
                with torch.inference_mode():
                    state=predictor.init_state(str(folder),offload_video_to_cpu=True,offload_state_to_cpu=True,async_loading_frames=False)
                    if state['num_frames']!=4 or state['video_height']!=224 or state['video_width']!=224:raise RuntimeError('native_frame_shape')
                    if list(state['images'].shape)!=[4,3,1024,1024]or not torch.isfinite(state['images']).all():raise RuntimeError('input_shape_or_finite')
                    predictor.add_new_points_or_box(state,frame_idx=0,obj_id=1,points=np.array([[56,112]],dtype=np.float32),labels=np.array([1],dtype=np.int32))
                    results=[]
                    # Native limit is inclusive; four input frames + max3 yields indices0..3.
                    for frame_idx,obj_ids,logits in predictor.propagate_in_video(state,start_frame_idx=0,max_frame_num_to_track=3,reverse=False):
                        if list(logits.shape)!=[1,1,224,224]or not torch.isfinite(logits).all():raise RuntimeError('native_output_shape_or_finite')
                        if len(results)>=4:raise RuntimeError('excess_frames')
                        results.append({'frame_index':frame_idx,'object_ids':list(obj_ids),'mask':(logits[0,0]>0).cpu().tolist()})
                    return results
            finally:
                if state is not None:predictor.reset_state(state)
        return encode
    
    PYCORE

    Expected result: Exact wrapper bytes and Python syntax verify without loading native model code or weights.

  3. 3. Complete the private transport and verify wrapper

    Append the bounded authenticated transport to the preceding native predictor adapter. Verify the complete exact wrapper before starting.

    Command
    set -e
    cd "$HOME/sam2-video"
    cat >> sam2_video.py <<'PYHTTP'
    def create_server(encoder, key, port=PORT):
        if not isinstance(key, str) or len(key) != 64 or any(c not in '0123456789abcdef' for c in key):
            raise ValueError('key_format')
    
        class Handler(BaseHTTPRequestHandler):
            def setup(self):
                super().setup()
                self.connection.settimeout(5)
    
            def log_message(self, *_):
                pass  # Do not log Bearer keys or request bodies.
    
            def reply(self, status, obj):
                data = json.dumps(obj, allow_nan=False, separators=(',', ':')).encode()
                self.send_response(status)
                self.send_header('Content-Type', 'application/json')
                self.send_header('Content-Length', str(len(data)))
                self.send_header('Connection', 'close')
                self.end_headers()
                self.wfile.write(data)
                self.close_connection = True
    
            def authorized(self):
                values = self.headers.get_all('Authorization', [])
                return len(values) == 1 and hmac.compare_digest(values[0].encode(), ('Bearer ' + key).encode())
    
            def do_GET(self):
                if not self.authorized():
                    return self.reply(401, {'error': 'unauthorized'})
                if self.path != '/health':
                    return self.reply(404, {'error': 'not_found'})
                return self.reply(200, {'ready': True, 'configured_coarse_vision_positions_per_frame': 4096,
                                        'generated_text_tokens': None})
    
            def do_POST(self):
                if not self.authorized():
                    return self.reply(401, {'error': 'unauthorized'})
                if self.path != '/v1/video-segment':
                    return self.reply(404, {'error': 'not_found'})
                if self.headers.get('Transfer-Encoding') is not None:
                    return self.reply(400, {'error': 'transfer_encoding_unsupported'})
                lengths = self.headers.get_all('Content-Length', [])
                if len(lengths) != 1:
                    return self.reply(411, {'error': 'content_length_required'})
                raw = lengths[0]
                if not raw or len(raw) > 4 or any(c not in '0123456789' for c in raw):
                    return self.reply(413, {'error': 'invalid_or_excessive_length'})
                n = int(raw)
                if n < 1 or n > 256:
                    return self.reply(413, {'error': 'body_limit'})
                if self.headers.get_content_type() != 'application/json':
                    return self.reply(415, {'error': 'json_required'})
                try:
                    body = self.rfile.read(n)
                except (TimeoutError, OSError):
                    return self.reply(408, {'error': 'body_timeout'})
                if len(body) != n:
                    return self.reply(400, {'error': 'incomplete_body'})
                try:
                    def unique(pairs):
                        obj = {}
                        for k, v in pairs:
                            if k in obj:
                                raise ValueError('duplicate_key')
                            obj[k] = v
                        return obj
                    request = json.loads(body.decode('utf-8'), object_pairs_hook=unique,
                                         parse_constant=lambda _: (_ for _ in ()).throw(ValueError('constant')))
                except (ValueError, UnicodeError, RecursionError):
                    return self.reply(400, {'error': 'invalid_json'})
                if type(request) is not dict or request != {'fixture': 'author-moving-geometry-v1'}:
                    return self.reply(400, {'error': 'fixed_fixture_required'})
                try:
                    result = validate_video(encoder())
                except Exception:
                    return self.reply(500, {'error': 'predictor_or_mask_failure'})
                return self.reply(200, result)
    
        return HTTPServer(('127.0.0.1', port), Handler)
    
    
    if __name__ == '__main__':
        secret = (ROOT / '.api-key').read_text().strip()
        encoder = make_encoder()
        with create_server(encoder, secret) as server:
            server.serve_forever()
    PYHTTP
    .venv/bin/python -m py_compile sam2_video.py
    sha256sum --check <<'SHA256'
    20043053f7e4347547d889bf95e950d41c04988150b0714633dfab918ded44f5  sam2_video.py
    SHA256

    Expected result: Complete wrapper syntax and exact hash verify; no model load occurs during compilation.

Start the model

  1. 1. Start the bounded temporal experiment

    Run in the dedicated workspace. Startup verifies the exact native checkpoint hash and uses build_sam2_video_predictor with the pinned original config, device CPU, apply_postprocessing = False and vos_optimized = False. Model FP32 execution and evaluation mode is explicit; optional CUDA extension and compiled VOS path are absent.

    Command
    set -e
    cd "$HOME/sam2-video"
    .venv/bin/python sam2_video.py

    Expected result: After actual reader load, loopback 127.0.0.1:8994 serves the fixed-input endpoint. No successful native load or tracking run was executed by the guide author.

Connect to your endpoint

http://127.0.0.1:8994/v1/video-segment

Same trusted desktop loopback only.

Access: loopback. Private Bearer key required for health and propagation. Send a JSON object with fixture set to author-moving-geometry-v1. Response contains four ordered binary-mask runs, shapes, frame indices, object IDs and point metadata. No uploads, arbitrary paths, URLs or chat schema.

Verify the setup

  1. 1. Check every ordered mask and inspect temporal overlays

    Run from a second server terminal. Validate exactly four ordered masks, object ID 1, finite native logits before thresholding, bounded row-major foreground runs and dimensions. Reconstruct overlays against each original frame. Visually inspect whether masks follow the red rectangle from left to right while the blue ellipse stays fixed. Empty masks, background capture, jumps or identity loss are task failures to record; no exact shape/area, overlap score or temporal quality is promised. The four samples are an authored ordered sequence, not a recorded real-time clip.

    Command
    set -e
    cd "$HOME/sam2-video"
    SAM_KEY="$(cat .api-key)"
    curl --fail http://127.0.0.1:8994/health -H "Authorization: Bearer $SAM_KEY"
    curl --fail http://127.0.0.1:8994/v1/video-segment -H "Authorization: Bearer $SAM_KEY" -H 'Content-Type: application/json' -d '{"fixture":"author-moving-geometry-v1"}' -o masks.json
    curl -s -o /dev/null -w '%{http_code}\n' http://127.0.0.1:8994/health
    unset SAM_KEY
    .venv/bin/python - <<'PYVERIFY'
    import json
    from PIL import Image
    d=json.load(open('masks.json'))
    assert d['frame_count']==4 and d['frame_order']==[0,1,2,3] and d['generated_text_tokens'] is None
    assert d['configured_coarse_vision_positions_per_frame']==4096 and len(d['frames'])==4
    for i,f in enumerate(d['frames']):
     assert f['frame_index']==i and f['object_ids']==[1] and f['mask_shape']==[1,1,224,224]
     pixels=[0]*(224*224);previous=0
     for start,length in f['mask_true_runs']:
      assert type(start)is int and type(length)is int and start>=previous and length>0 and start+length<=len(pixels)
      pixels[start:start+length]=[255]*length;previous=start+length
     assert sum(x>0 for x in pixels)==f['mask_area_pixels']
     mask=Image.new('L',(224,224));mask.putdata(pixels);mask.save(f'mask-{i}.png')
     image=Image.open(f'frames/{i:05d}.jpg').convert('RGB');tint=Image.new('RGB',image.size,(0,255,0))
     image.paste(Image.blend(image,tint,0.4),(0,0),mask);image.save(f'overlay-{i}.png')
    print('Inspect overlay-0.png through overlay-3.png in order; the original rectangle moves right. Record actual tracking defects.')
    PYVERIFY

    Expected result: Authenticated requests return the documented four-frame JSON; missing authorization returns 401. Reader shape/RLE checks and visual tracking inspection determine the actual result; finite masks alone do not establish correct tracking.

Performance evidence

No compatible measurement available. Documented support and estimated memory do not establish response speed.

Limits and troubleshooting

  • The exact frozen API identifies original July 2024 SAM 2 Hiera Large while the catalog version field says 2.1. This guide exposes the discrepancy and uses original checkpoint and configuration; no 2.1 substitution, catalog rewrite or family-complete claim.
  • 4,096 is the configured coarse 64 × 64 visual embedding position count at 1024 RGB input. Higher-resolution 256×256/128× 128 features, sparse point embeddings and learned decoder tokens are distinct; no text limit, whole internal sequence sum or global maximum is asserted.
  • Exact native checkpoint LFS SHA/size and Apache 2 license statements are source metadata/text evidence; no weights were downloaded or model executed. The separately published safetensors and HF-native YAML are not silently treated as identical.
  • Native runtime config RoPE initial grid 64 differs from HF-native YAML 32. Runtime recomputes frequencies for actual query shape; this source behavior is not measured numerical equivalence. The recipe pins native runtime config explicitly.
  • The original checkpoint is 224.4 million parameters per publisher table; A100 benchmark rows are not CPU speed or memory evidence. 32 GiB RAM, 40 GiB disk and four threads are chosen allocations; fit, latency and runtime memory remain unknown.
  • Package versions are source-qualified but not installed/tested with the model. Transitive/system resolutions are recorded by pip freeze. Fake-mask transport tests prove request/output boundary handling only, not segmentation quality or runtime compatibility.
  • Exactly four original generated JPEG frames and one initial positive point and one object are supported. No additional frames, arbitrary file or media URL, MP4 decoding, correction prompts, reverse tracking or multi-object interface is supplied.
  • Native temporal state differs from image predictor state: full normalized frames, cached features, conditioning/non-conditioning masks, positional encodings and object pointers are retained within one request. Native stored mask-memory features are BF16 even with FP32 model weights; reset_state runs finally and state is discarded between requests.
  • The complete native video path is used. apply_postprocessing = False omits dynamic stability fallback, binarized clicked-mask memory override and hole filling; no identical-to-default-postprocessed-GPU mask claim.
  • Native output logits must be finite at shape [1,1,224,224] before threshold greater than zero. There is no native per-frame estimated IoU in this public propagation result; no score is invented. Masks do not encode semantic class labels.
  • Four original ordered geometry frames supply a reproducible temporal input without third-party media rights or codec dependencies. They are not a natural-video tracking benchmark; no exact segmentation or generalization quality is promised.
  • This serial private educational HTTPServer bounds input body and socket reads and output frames. It has no hard native inference deadline or multiuser resilience, and a failed or slow model call can block the server until stopped.
  • 4096 is per-frame coarse embedding configuration only, not global video-context maximum or total internal sequence. High-resolution, spatial prompt, decoder, temporal-memory and object-pointer terms must remain distinct.
  • Video, model, dependency and memory behavior are source-qualified, not executed. Stdlib fake protocol tests cannot establish actual CPU dtype compatibility, numerical behavior, fit, speed or tracking quality. Other checkpoints, larger/natural videos, corrections, automatic masks and platforms remain open scope.
Checkpoint, native config or state loading fails
Stop and restore the exact original checkpoint hash, source and configuration. Do not switch to SAM 2.1, relax missing/unexpected key checks or use floating from_pretrained downloads.
CPU dtype, shape or propagation error
Stop; retain the native error and pinned environment report. Confirm CPU FP32 model, native BF16 stored memory, four 224 × 224 JPEG frames and source defaults. Do not patch or cast native internals to hide an unqualified incompatibility; this guide has not executed native tracking.
Missing, reordered or incorrect masks
Require indices 0 through 3, exactly object ID 1 and four finite masks. Inspect original moving rectangle and overlays in order. Empty masks and failed tracking are recorded failures; no exact area/IoU or automatic quality approval is supplied.
401, framing or fixed fixture rejection
Use the stable private hexadecimal key and exact fixture JSON. Unsupported URLs, paths, uploads, extra fields and alternate points are rejected.

Stop the model

  1. 1. Stop the research endpoint

    Press Ctrl+C in its foreground terminal and unset client keys. Confirm local loopback port 8994 no longer serves.

    Command
    set -e
    unset SAM_KEY

    Expected result: Foreground process and loopback endpoint stop; per-request native temporal state is no longer held.

Uninstall

  1. 1. Remove dedicated experiment files

    Preserve desired results and licensing records first; confirm the dedicated directory before deletion. Shared Ubuntu packages remain.

    Command
    set -e
    rm -r -- "$HOME/sam2-video"

    Expected result: Dedicated checkpoint, checkout, environment, generated frames, masks, overlays and key removed. Shared system packages remain.

Sources and verification

Sources checked Oct 17, 2026. Verification source

Compare this configuration
Cloud serverDocumented setupSources checked Oct 17, 2026

Propagate an original SAM 2 point prompt through four frames on a private CPU VM

Use the complete original facebook/sam2-hiera-large checkpoint and native temporal video predictor on four ordered author-created JPEG frames. A positive point seeds object 1 in frame zero; native memory then propagates a mask as a red rectangle moves right. This is a bounded synthetic spatial tracking experiment, not text generation, semantic recognition or measured tracking quality. The frozen version says 2.1 but the selected exact API artifact is original SAM 2. Fit, speed and temporal mask accuracy remain unmeasured.

Hardware
AWS r7i.xlarge, 32 GiB RAM, Ubuntu x86_64
Runtime
Native original SAM 2 finite JPEG temporal video predictor CPU SAM 2 source 2b90b9f; Torch 2.6.0 CPU; torchvision 0.21.0 CPU; Python 3.12 · CPU
Model format
PyTorch checkpoint · CPU model FP32 eager; native stored mask-memory features BF16; no autocast or compile. · Unquantized native checkpoint; reader CPU float32
Workload
4,096 token context · 1 concurrent

Based on source documentation; this setup has not been executed by OpenTools. Memory: unknown. No compatible measurement available.

Read the complete setup guide

Before you start

  • Use an owned AWS r7i.xlarge Ubuntu 24.04 x86_64 VM in us-east-1 with 32 GiB RAM, four vCPUs and a chosen 40 GiB gp3 volume. Existing SSH access must be limited to your client; keep service port 8994 closed. Provisioning and quoted/other applicable billing are reader actions.
  • The frozen catalog says version 2.1, but its exact API facebook/sam2-hiera-large maps to the original July 29, 2024 checkpoint. This recipe deliberately uses that exact API artifact and native original config; it does not silently upgrade to SAM 2.1 or rewrite catalog metadata.
  • Review Apache 2.0 obligations for the selected model/runtime. Use only the four original moving-geometry frames generated by this harness; no third-party media, uploads, external URLs, file paths or model choices are accepted.
  • This is a source-documented private educational temporal segmentation experiment, not measured quality or a production service. Python 3.12 and the source-supported CPU package versions below have not been installed/model-tested by the author. Save dependency resolution and stop on pip check, state-dictionary or shape errors.
  • The fixed fixture is original author-created geometry generated locally by the reader using Pillow. No publisher demo-media license is inferred, and no media is downloaded. JPEG encoding bytes are reader-generated and not claimed to have publisher hashes; four filenames, dimensions and deterministic drawing instructions are explicit.

Exact model and requirements

Artifact:
Meta / facebookresearch / SAM 2 original Hiera Large; exact frozen API artifact, not SAM 2.1 · checkpoint facebook/sam2-hiera-large
Publisher revision:
e6a8e8809b8f1bfa2238b6d080f3d05cc76bd251
Artifact digest:
sha256:7442e4e9b732a508f80e141e7c2913437a3610ee0c77381a66658c3a445df87b
Download size:
0.84 GiB
Memory requirement (unknown):
Unknown — No measured whole-process peak with four normalized 1024 frames, model, high-resolution features, memory encoder/attention and output masks.
System RAM / disk:
Unknown / Unknown
Placement:
single device. One CPU host; no tensor or pipeline sharding. Whole predictor executes on CPU. Native offload_video_to_cpu=True and offload_state_to_cpu=True keep frame/state storage on that same CPU; no GPU/disk offload.
KV cache:
Not autoregressive text K/V caching. Native temporal mask-memory features are BF16; positional encodings, object pointers, cached image features and frame/state tensors remain native managed state.
Workload:
4,096 token context; Not applicable; Not applicable; concurrency 1; batch 1; cache cold; repetitions Unknown.
Tokenizer:
Native original SAM 2 per-frame image grid; JPEG folder numeric order and spatial prompts, no text tokenizer · 2b90b9f5ceec907a1c18123530e92e794ad901a4
Sampling / reasoning:
Exactly four original 224 × 224 JPEG frames numbered 00000.jpg through 00003.jpg, generated by the reader from code. The rectangle left edge is 24, 48, 72, 96 pixels; fixed positive XY point [56,112] on frame 0, object ID 1. Native forward propagation start_frame_idx = 0 and max_frame_num_to_track = 3 is inclusive and yields indices 0 through 3. No recorded FPS/duration, frame interpolation, external file/URL input, reverse or correction prompts. The configured context value of 4,096 is the configured per-frame coarse 64 × 64 visual grid at native 1024 × 1024 resize. Four frames do not turn this into a 4096-position whole-video capacity: high-resolution features, prompt/decoder tokens, temporal mask memory and object-pointer sequences are distinct. Native num_maskmem = 7 is a memory-selection configuration, not a global video-frame maximum. This recipe bounds actual input at four frames and one object.
Runtime prerequisites:
Ubuntu 24.04 x86_64; See prerequisites above.

Official artifact · License

Actual upstream Apache License 2.0 text is archived, and upstream README expressly includes SAM 2 model checkpoints. Retain license and relevant copyright/NOTICE material; observe redistribution/change-notice and patent conditions. No trademark endorsement or legal clearance is granted by this guide. Optional connected-component code has its own archived license; this CPU recipe skips that extension. Demo fonts and media are not included or presumed Apache-licensed. The original fixture is author-created geometry.

  • 32 GiB RAM / 40 GiB disk / four threads are existing author planning allocations, not measured minima or fit.
  • The exact checkpoint is 897,952,466 serialized bytes; native video state and FP32 model execution add unmeasured overhead. All four frames are loaded eagerly; no long-video or streaming memory bound is claimed.

One-time setup and hardware

hardware
Not applicable — AWS hardware acquisition is included in compute rental; no purchased GPU required. Any separate hardware cost is not applicable.
AWS hardware acquisition is included in compute rental; no purchased GPU required. Any separate hardware cost is not applicable. Tax: unknown.
setup
Unknown — Initial provisioning/build time and labor cost are unknown.
Initial provisioning/build time and labor cost are unknown. Tax: unknown.

Recurring costs

compute
$0.26 / hour
AWS public us-east-1 Linux On-Demand rate USD 0.2646 per billed instance-hour, no paid preinstalled software. Eight hours USD 2.1168; 24 hours USD 6.3504 compute. Idle running hours bill. Storage, egress, tax and administration separate; no Spot/reservation/discount assumed. Capacity/quota unverified. Region: us-east-1. SKU: r7i.xlarge Linux On-Demand. Tax: excluded. Price dated Oct 15, 2026. Price source
storage
$0.08 / GB month
US East (N. Virginia) gp3 USD 0.08 per GB-month; AWS defines GB as 1024^3 bytes. Selected 40 GiB root volume is USD 3.20 per full month, prorated by provisioned duration. Baseline 3000 IOPS / 125 MB/s; no extras assumed. Retained snapshots/resources separate; storage persists after instance stop. Tax excluded. Region: us-east-1. SKU: EBS gp3 baseline 3000 IOPS / 125 MB/s. Tax: excluded. Price dated Oct 16, 2026. Price source
egress
Unknown — Data transfer price, allowances and traffic volume are unknown.
Data transfer price, allowances and traffic volume are unknown. Tax: unknown.
administration
Unknown — Maintenance labor rate and hours are unknown.
Maintenance labor rate and hours are unknown. Tax: unknown.

Install the runtime

  1. 1. Create the native CPU environment

    Use exact official CPU wheels and a pinned native checkout. Explicitly skip the optional CUDA extension. Pin direct dependencies, save transitive/system resolution and stop on pip check failure; this is documented support rather than an executed environment.

    Command
    set -e
    sudo apt-get update
    sudo apt-get install -y python3.12 python3.12-venv git curl ca-certificates openssl
    mkdir -p "$HOME/sam2-video/models"
    cd "$HOME/sam2-video"
    python3.12 -m venv .venv
    .venv/bin/python -m pip install 'https://download.pytorch.org/whl/cpu/torch-2.6.0%2Bcpu-cp312-cp312-linux_x86_64.whl#sha256=59e78aa0c690f70734e42670036d6b541930b8eabbaa18d94e090abf14cc4d91' 'https://download-r2.pytorch.org/whl/cpu/torchvision-0.21.0%2Bcpu-cp312-cp312-linux_x86_64.whl#sha256=d6874431e678ba107b60a83f255c33f3755f06bad587b1b919aa514ec325dcd8'
    .venv/bin/python -m pip install 'numpy==2.2.3' 'Pillow==11.1.0' 'hydra-core==1.3.2' 'omegaconf==2.3.0' 'antlr4-python3-runtime==4.9.3' 'iopath==0.1.10' 'tqdm==4.67.1'
    git clone https://github.com/facebookresearch/sam2.git native-sam2
    git -C native-sam2 checkout --detach 2b90b9f5ceec907a1c18123530e92e794ad901a4
    SAM2_BUILD_CUDA=0 .venv/bin/python -m pip install -e ./native-sam2
    .venv/bin/python -m pip check
    .venv/bin/python -m pip freeze > resolved-environment.txt

    Expected result: Pinned checkout and hashed CPU wheels resolve; extension build is disabled, pip check passes and versions are saved. No CUDA driver/compiler or GPU runtime is required by this chosen path.

Download the model

  1. 1. Verify the exact original checkpoint and config

    Reader download only: the author fetched metadata/text, not weights. Use exact HF revision for native .pt and exact config from pinned runtime. HF safetensors and HF-native YAML are different artifacts/config revisions and are not substituted.

    Command
    set -e
    cd "$HOME/sam2-video"
    curl --fail --location --output models/sam2_hiera_large.pt 'https://huggingface.co/facebook/sam2-hiera-large/resolve/e6a8e8809b8f1bfa2238b6d080f3d05cc76bd251/sam2_hiera_large.pt'
    sha256sum --check <<'SHA256'
    7442e4e9b732a508f80e141e7c2913437a3610ee0c77381a66658c3a445df87b  models/sam2_hiera_large.pt
    0ee9f2037fd98489212bb0b78a56f33ac82eb177cf5e52c88be3714cb84eea26  native-sam2/sam2/configs/sam2/sam2_hiera_l.yaml
    c71d239df91726fc519c6eb72d318ec65820627232b2f796219e87dcf35d0ab4  native-sam2/LICENSE
    SHA256

    Expected result: All three exact hashes report OK; stop on mismatch. Local native loader uses weights_only=True and checks state-dictionary keys. No native convenience from_pretrained call floats the model revision.

Configure

  1. 1. Create a stable private service key

    Generate only when absent; preserve existing access. Do not print or share this local service key.

    Command
    set -e
    cd "$HOME/sam2-video"
    umask 077
    if [ ! -e .api-key ]; then openssl rand -hex 32 > .api-key; fi
    chmod 600 .api-key
    test -s .api-key || { echo "Restore empty service key before continuing"; exit 1; }

    Expected result: Private existing key is preserved; server requires 64 lowercase hexadecimal characters.

  2. 2. Create the reader-authored temporal harness

    The custom serial server accepts only a fixed fixture identifier. It creates four original JPEG frames, uses the complete native video predictor and one positive point, and resets state after every request. Native CPU device, model and source are unmodified; optional CUDA hole filling and dynamic multimask fallback stay disabled. No MP4/decord/FFmpeg codec route is used. This is authored deployment prescription, not publisher server code.

    Command
    set -e
    cd "$HOME/sam2-video"
    cat > sam2_video.py <<'PYCORE'
    """Reader-authored fixed four-JPEG temporal SAM2 research harness; not publisher server."""
    import hashlib,hmac,json,math,pathlib
    from http.server import BaseHTTPRequestHandler,HTTPServer
    ROOT=pathlib.Path(__file__).resolve().parent
    PORT=8994
    MODEL_SHA='7442e4e9b732a508f80e141e7c2913437a3610ee0c77381a66658c3a445df87b'
    
    def validate_video(results):
        if type(results) is not list or len(results)!=4: raise ValueError('frame_count')
        frames=[]
        for expected,result in enumerate(results):
            if type(result) is not dict or result.get('frame_index')!=expected or type(result.get('frame_index'))is not int or (type(result.get('object_ids'))is not list or len(result['object_ids'])!=1 or type(result['object_ids'][0])is not int or result['object_ids'][0]!=1): raise ValueError('order_or_identity')
            mask=result.get('mask')
            if type(mask)is not list or len(mask)!=224:raise ValueError('mask_shape')
            flat=[]
            for row in mask:
                if type(row)is not list or len(row)!=224 or any(type(v)is not bool for v in row):raise ValueError('mask_values')
                flat.extend(row)
            runs=[];i=0
            while i<len(flat):
                if not flat[i]:i+=1;continue
                start=i
                while i<len(flat)and flat[i]:i+=1
                runs.append([start,i-start])
            frames.append({'frame_index':expected,'object_ids':[1],'mask_shape':[1,1,224,224],'mask_true_runs':runs,'mask_area_pixels':sum(flat)})
        return {'fixture':'author-moving-geometry-v1','frame_count':4,'frame_order':[0,1,2,3],'point_xy':[56,112],'point_frame_index':0,'point_label':1,'frames':frames,'configured_coarse_vision_positions_per_frame':4096,'generated_text_tokens':None,'real_time_fps':None,'semantic_classification':False}
    
    def make_encoder():
        # Native imports only in the reader-run path. Tests inject a fake callable.
        import numpy as np
        import torch
        from PIL import Image,ImageDraw
        from sam2.build_sam import build_sam2_video_predictor
        torch.set_num_threads(4)
        artifact=ROOT/'models/sam2_hiera_large.pt'
        if hashlib.sha256(artifact.read_bytes()).hexdigest()!=MODEL_SHA:raise RuntimeError('checkpoint_hash_mismatch')
        predictor=build_sam2_video_predictor('configs/sam2/sam2_hiera_l.yaml',str(artifact),device='cpu',apply_postprocessing=False,vos_optimized=False).float().eval()
        if predictor.fill_hole_area!=0 or predictor.image_size!=1024:raise RuntimeError('native_settings')
        folder=ROOT/'frames';folder.mkdir(exist_ok=True)
        expected_names={f'{i:05d}.jpg'for i in range(4)}
        if any(x.name not in expected_names for x in folder.iterdir()):raise RuntimeError('unexpected_fixture_file')
        # Author-created geometry, four original ordered frames; no external media rights.
        for i in range(4):
            image=Image.new('RGB',(224,224),(240,240,240));draw=ImageDraw.Draw(image)
            x=24+24*i
            draw.rectangle((x,56,x+64,168),fill=(220,40,40))
            draw.ellipse((156,12,208,44),fill=(30,90,220))
            image.save(folder/f'{i:05d}.jpg',format='JPEG',quality=95,subsampling=0)
        if {x.name for x in folder.iterdir()}!=expected_names:raise RuntimeError('fixture_manifest')
        def encode():
            state=None
            try:
                with torch.inference_mode():
                    state=predictor.init_state(str(folder),offload_video_to_cpu=True,offload_state_to_cpu=True,async_loading_frames=False)
                    if state['num_frames']!=4 or state['video_height']!=224 or state['video_width']!=224:raise RuntimeError('native_frame_shape')
                    if list(state['images'].shape)!=[4,3,1024,1024]or not torch.isfinite(state['images']).all():raise RuntimeError('input_shape_or_finite')
                    predictor.add_new_points_or_box(state,frame_idx=0,obj_id=1,points=np.array([[56,112]],dtype=np.float32),labels=np.array([1],dtype=np.int32))
                    results=[]
                    # Native limit is inclusive; four input frames + max3 yields indices0..3.
                    for frame_idx,obj_ids,logits in predictor.propagate_in_video(state,start_frame_idx=0,max_frame_num_to_track=3,reverse=False):
                        if list(logits.shape)!=[1,1,224,224]or not torch.isfinite(logits).all():raise RuntimeError('native_output_shape_or_finite')
                        if len(results)>=4:raise RuntimeError('excess_frames')
                        results.append({'frame_index':frame_idx,'object_ids':list(obj_ids),'mask':(logits[0,0]>0).cpu().tolist()})
                    return results
            finally:
                if state is not None:predictor.reset_state(state)
        return encode
    
    PYCORE

    Expected result: Exact wrapper bytes and Python syntax verify without loading native model code or weights.

  3. 3. Complete the private transport and verify wrapper

    Append the bounded authenticated transport to the preceding native predictor adapter. Verify the complete exact wrapper before starting.

    Command
    set -e
    cd "$HOME/sam2-video"
    cat >> sam2_video.py <<'PYHTTP'
    def create_server(encoder, key, port=PORT):
        if not isinstance(key, str) or len(key) != 64 or any(c not in '0123456789abcdef' for c in key):
            raise ValueError('key_format')
    
        class Handler(BaseHTTPRequestHandler):
            def setup(self):
                super().setup()
                self.connection.settimeout(5)
    
            def log_message(self, *_):
                pass  # Do not log Bearer keys or request bodies.
    
            def reply(self, status, obj):
                data = json.dumps(obj, allow_nan=False, separators=(',', ':')).encode()
                self.send_response(status)
                self.send_header('Content-Type', 'application/json')
                self.send_header('Content-Length', str(len(data)))
                self.send_header('Connection', 'close')
                self.end_headers()
                self.wfile.write(data)
                self.close_connection = True
    
            def authorized(self):
                values = self.headers.get_all('Authorization', [])
                return len(values) == 1 and hmac.compare_digest(values[0].encode(), ('Bearer ' + key).encode())
    
            def do_GET(self):
                if not self.authorized():
                    return self.reply(401, {'error': 'unauthorized'})
                if self.path != '/health':
                    return self.reply(404, {'error': 'not_found'})
                return self.reply(200, {'ready': True, 'configured_coarse_vision_positions_per_frame': 4096,
                                        'generated_text_tokens': None})
    
            def do_POST(self):
                if not self.authorized():
                    return self.reply(401, {'error': 'unauthorized'})
                if self.path != '/v1/video-segment':
                    return self.reply(404, {'error': 'not_found'})
                if self.headers.get('Transfer-Encoding') is not None:
                    return self.reply(400, {'error': 'transfer_encoding_unsupported'})
                lengths = self.headers.get_all('Content-Length', [])
                if len(lengths) != 1:
                    return self.reply(411, {'error': 'content_length_required'})
                raw = lengths[0]
                if not raw or len(raw) > 4 or any(c not in '0123456789' for c in raw):
                    return self.reply(413, {'error': 'invalid_or_excessive_length'})
                n = int(raw)
                if n < 1 or n > 256:
                    return self.reply(413, {'error': 'body_limit'})
                if self.headers.get_content_type() != 'application/json':
                    return self.reply(415, {'error': 'json_required'})
                try:
                    body = self.rfile.read(n)
                except (TimeoutError, OSError):
                    return self.reply(408, {'error': 'body_timeout'})
                if len(body) != n:
                    return self.reply(400, {'error': 'incomplete_body'})
                try:
                    def unique(pairs):
                        obj = {}
                        for k, v in pairs:
                            if k in obj:
                                raise ValueError('duplicate_key')
                            obj[k] = v
                        return obj
                    request = json.loads(body.decode('utf-8'), object_pairs_hook=unique,
                                         parse_constant=lambda _: (_ for _ in ()).throw(ValueError('constant')))
                except (ValueError, UnicodeError, RecursionError):
                    return self.reply(400, {'error': 'invalid_json'})
                if type(request) is not dict or request != {'fixture': 'author-moving-geometry-v1'}:
                    return self.reply(400, {'error': 'fixed_fixture_required'})
                try:
                    result = validate_video(encoder())
                except Exception:
                    return self.reply(500, {'error': 'predictor_or_mask_failure'})
                return self.reply(200, result)
    
        return HTTPServer(('127.0.0.1', port), Handler)
    
    
    if __name__ == '__main__':
        secret = (ROOT / '.api-key').read_text().strip()
        encoder = make_encoder()
        with create_server(encoder, secret) as server:
            server.serve_forever()
    PYHTTP
    .venv/bin/python -m py_compile sam2_video.py
    sha256sum --check <<'SHA256'
    20043053f7e4347547d889bf95e950d41c04988150b0714633dfab918ded44f5  sam2_video.py
    SHA256

    Expected result: Complete wrapper syntax and exact hash verify; no model load occurs during compilation.

Start the model

  1. 1. Start the bounded temporal experiment

    Run in the dedicated workspace. Startup verifies the exact native checkpoint hash and uses build_sam2_video_predictor with the pinned original config, device CPU, apply_postprocessing = False and vos_optimized = False. Model FP32 execution and evaluation mode is explicit; optional CUDA extension and compiled VOS path are absent.

    Command
    set -e
    cd "$HOME/sam2-video"
    .venv/bin/python sam2_video.py

    Expected result: After actual reader load, loopback 127.0.0.1:8994 serves the fixed-input endpoint. No successful native load or tracking run was executed by the guide author.

Connect to your endpoint

http://127.0.0.1:8994/v1/video-segment

VM loopback only. Optional existing SSH tunnel: ssh -N -L 8994:127.0.0.1:8994 USER@VM_IP. Keep EC2 service port 8994 closed; shown checks run on server.

Access: loopback. Private Bearer key required for health and propagation. Send a JSON object with fixture set to author-moving-geometry-v1. Response contains four ordered binary-mask runs, shapes, frame indices, object IDs and point metadata. No uploads, arbitrary paths, URLs or chat schema.

Verify the setup

  1. 1. Check every ordered mask and inspect temporal overlays

    Run from a second server terminal. Validate exactly four ordered masks, object ID 1, finite native logits before thresholding, bounded row-major foreground runs and dimensions. Reconstruct overlays against each original frame. Visually inspect whether masks follow the red rectangle from left to right while the blue ellipse stays fixed. Empty masks, background capture, jumps or identity loss are task failures to record; no exact shape/area, overlap score or temporal quality is promised. The four samples are an authored ordered sequence, not a recorded real-time clip.

    Command
    set -e
    cd "$HOME/sam2-video"
    SAM_KEY="$(cat .api-key)"
    curl --fail http://127.0.0.1:8994/health -H "Authorization: Bearer $SAM_KEY"
    curl --fail http://127.0.0.1:8994/v1/video-segment -H "Authorization: Bearer $SAM_KEY" -H 'Content-Type: application/json' -d '{"fixture":"author-moving-geometry-v1"}' -o masks.json
    curl -s -o /dev/null -w '%{http_code}\n' http://127.0.0.1:8994/health
    unset SAM_KEY
    .venv/bin/python - <<'PYVERIFY'
    import json
    from PIL import Image
    d=json.load(open('masks.json'))
    assert d['frame_count']==4 and d['frame_order']==[0,1,2,3] and d['generated_text_tokens'] is None
    assert d['configured_coarse_vision_positions_per_frame']==4096 and len(d['frames'])==4
    for i,f in enumerate(d['frames']):
     assert f['frame_index']==i and f['object_ids']==[1] and f['mask_shape']==[1,1,224,224]
     pixels=[0]*(224*224);previous=0
     for start,length in f['mask_true_runs']:
      assert type(start)is int and type(length)is int and start>=previous and length>0 and start+length<=len(pixels)
      pixels[start:start+length]=[255]*length;previous=start+length
     assert sum(x>0 for x in pixels)==f['mask_area_pixels']
     mask=Image.new('L',(224,224));mask.putdata(pixels);mask.save(f'mask-{i}.png')
     image=Image.open(f'frames/{i:05d}.jpg').convert('RGB');tint=Image.new('RGB',image.size,(0,255,0))
     image.paste(Image.blend(image,tint,0.4),(0,0),mask);image.save(f'overlay-{i}.png')
    print('Inspect overlay-0.png through overlay-3.png in order; the original rectangle moves right. Record actual tracking defects.')
    PYVERIFY

    Expected result: Authenticated requests return the documented four-frame JSON; missing authorization returns 401. Reader shape/RLE checks and visual tracking inspection determine the actual result; finite masks alone do not establish correct tracking.

Performance evidence

No compatible measurement available. Documented support and estimated memory do not establish response speed.

Limits and troubleshooting

  • The exact frozen API identifies original July 2024 SAM 2 Hiera Large while the catalog version field says 2.1. This guide exposes the discrepancy and uses original checkpoint and configuration; no 2.1 substitution, catalog rewrite or family-complete claim.
  • 4,096 is the configured coarse 64 × 64 visual embedding position count at 1024 RGB input. Higher-resolution 256×256/128× 128 features, sparse point embeddings and learned decoder tokens are distinct; no text limit, whole internal sequence sum or global maximum is asserted.
  • Exact native checkpoint LFS SHA/size and Apache 2 license statements are source metadata/text evidence; no weights were downloaded or model executed. The separately published safetensors and HF-native YAML are not silently treated as identical.
  • Native runtime config RoPE initial grid 64 differs from HF-native YAML 32. Runtime recomputes frequencies for actual query shape; this source behavior is not measured numerical equivalence. The recipe pins native runtime config explicitly.
  • The original checkpoint is 224.4 million parameters per publisher table; A100 benchmark rows are not CPU speed or memory evidence. 32 GiB RAM, 40 GiB disk and four threads are chosen allocations; fit, latency and runtime memory remain unknown.
  • Package versions are source-qualified but not installed/tested with the model. Transitive/system resolutions are recorded by pip freeze. Fake-mask transport tests prove request/output boundary handling only, not segmentation quality or runtime compatibility.
  • Exactly four original generated JPEG frames and one initial positive point and one object are supported. No additional frames, arbitrary file or media URL, MP4 decoding, correction prompts, reverse tracking or multi-object interface is supplied.
  • Native temporal state differs from image predictor state: full normalized frames, cached features, conditioning/non-conditioning masks, positional encodings and object pointers are retained within one request. Native stored mask-memory features are BF16 even with FP32 model weights; reset_state runs finally and state is discarded between requests.
  • The complete native video path is used. apply_postprocessing = False omits dynamic stability fallback, binarized clicked-mask memory override and hole filling; no identical-to-default-postprocessed-GPU mask claim.
  • Native output logits must be finite at shape [1,1,224,224] before threshold greater than zero. There is no native per-frame estimated IoU in this public propagation result; no score is invented. Masks do not encode semantic class labels.
  • Four original ordered geometry frames supply a reproducible temporal input without third-party media rights or codec dependencies. They are not a natural-video tracking benchmark; no exact segmentation or generalization quality is promised.
  • This serial private educational HTTPServer bounds input body and socket reads and output frames. It has no hard native inference deadline or multiuser resilience, and a failed or slow model call can block the server until stopped.
  • 4096 is per-frame coarse embedding configuration only, not global video-context maximum or total internal sequence. High-resolution, spatial prompt, decoder, temporal-memory and object-pointer terms must remain distinct.
  • Video, model, dependency and memory behavior are source-qualified, not executed. Stdlib fake protocol tests cannot establish actual CPU dtype compatibility, numerical behavior, fit, speed or tracking quality. Other checkpoints, larger/natural videos, corrections, automatic masks and platforms remain open scope.
Checkpoint, native config or state loading fails
Stop and restore the exact original checkpoint hash, source and configuration. Do not switch to SAM 2.1, relax missing/unexpected key checks or use floating from_pretrained downloads.
CPU dtype, shape or propagation error
Stop; retain the native error and pinned environment report. Confirm CPU FP32 model, native BF16 stored memory, four 224 × 224 JPEG frames and source defaults. Do not patch or cast native internals to hide an unqualified incompatibility; this guide has not executed native tracking.
Missing, reordered or incorrect masks
Require indices 0 through 3, exactly object ID 1 and four finite masks. Inspect original moving rectangle and overlays in order. Empty masks and failed tracking are recorded failures; no exact area/IoU or automatic quality approval is supplied.
401, framing or fixed fixture rejection
Use the stable private hexadecimal key and exact fixture JSON. Unsupported URLs, paths, uploads, extra fields and alternate points are rejected.

Stop the model

  1. 1. Stop the research endpoint

    Press Ctrl+C in its foreground terminal and unset client keys. Close optional SSH tunnel. Stopping Python does not stop VM billing.

    Command
    set -e
    unset SAM_KEY

    Expected result: Foreground process and loopback endpoint stop; per-request native temporal state is no longer held.

Uninstall

  1. 1. Remove dedicated experiment files

    Preserve desired results and licensing records first; confirm the dedicated directory before deletion. Shared Ubuntu packages remain.

    Command
    set -e
    rm -r -- "$HOME/sam2-video"

    Expected result: Dedicated checkpoint, checkout, environment, generated frames, masks, overlays and key removed. Shared system packages remain.

Shut down cloud resources

  1. 1. End cloud billing resources

    In EC2 us-east-1, stop for later reuse or terminate this exact VM. Review Delete on termination for its root volume. Delete unwanted detached EBS volumes, snapshots and reserved/Elastic/public IP resources. Stopped VM storage and retained resources may still bill. Check Bills/Cost Explorer later for final usage.

    Expected result: Exact instance is stopped/terminated as intended; no unwanted attached/retained billed resources remain. Instance compute stops only after its lifecycle transition, not when the Python service stops.

Sources and verification

Sources checked Oct 17, 2026. Verification source

Compare this configuration

Benchmarks

Performance scores for SAM 2 across standard benchmarks.

SA-V test J&Fofficial SAM 2 README, sam2.1_hiera_large · Sep 2024
79.5%
MOSE val J&Fofficial SAM 2 README, sam2.1_hiera_large · Sep 2024
74.6%
LVOS v2 J&Fofficial SAM 2 README, sam2.1_hiera_large · Sep 2024
80.6%
Speed FPS A100official SAM 2 README, sam2.1_hiera_large · Sep 2024
39.5 frames/s

Pricing and access

Open checkpoints and code are provided under Apache 2.0 in the official repository; compute costs depend on the user’s hardware or cloud environment.

Meta logo
Metatechnology company

Build the future of human connection and the technology that makes it possible

5 Tools8 ModelsFounded 2004Menlo Park, California, United States
View full profile

Tools by Meta

Other AI tools from the same organization.

AI Models by Meta

AI models from the same organization.

ModelContext WindowPrice (In / Out per M)
Muse Spark 1.1Current1.0M$1.25 / $4.25
DINOv3Current---- / --
Llama Guard 4 12BCurrent164K$0.18 / $0.18
Llama 4 MaverickCurrent1.0M$0.15 / $0.60
Llama 4 ScoutCurrent328K$0.08 / $0.30
Llama Guard 3 8BCurrent131K$0.48 / $0.03

Related News

Latest coverage and updates.