{"id":"local-model-mac","schemaVersion":1,"revision":"2026-09-15.1","status":"researching","title":"Which local model runtime fits a 32GB or 64GB Mac?","area":"AI infrastructure · model runtimes","summary":"Compare a pinned open model across local runtimes, then measure what changes with four simultaneous requests. The procedure is untested; no speed, memory-fit or quality winner has been established.","constraints":["Use separate 32GB and 64GB Apple Silicon machines; record chip, GPU cores, memory bandwidth, macOS and power mode. RAM capacity alone cannot explain cross-machine differences.","Use the same source-model revision, tokenizer, chat template and synthetic prompts. Hash every downloaded artifact.","Compare the identical GGUF artifact across llama.cpp, Ollama and LM Studio; evaluate MLX conversions separately. Matching bit counts do not make quantizations equivalent.","Keep cloud inference and external tools disabled; preload permitted model files before the offline measurement phase."],"collection":{"agent":"Codex","model":"GPT-6","resolvedModelRevision":null,"method":"official-documentation-review","checkedAt":"2026-09-15","review":"unreviewed"},"subjects":[{"id":"llama-cpp","name":"llama.cpp","role":"Inference library","productId":"llama-cpp"},{"id":"mlx-lm","name":"MLX-LM","role":"Inference library","productId":"mlx-lm"},{"id":"ollama-local","name":"Ollama (local runtime)","role":"Packaged runtime","productId":"ollama-local"},{"id":"lm-studio-local","name":"LM Studio (local runtime)","role":"Packaged runtime","productId":"lm-studio-local"}],"sources":[{"id":"llama-cpp-readme","title":"llama.cpp overview","url":"https://github.com/ggml-org/llama.cpp/blob/38a5b42d9a3e82e0a586bcd1caed121f36c87a73/README.md","locator":"Description; supported backends; quick start","checkedAt":"2026-09-15","version":"38a5b42d9a3e82e0a586bcd1caed121f36c87a73"},{"id":"mlx-lm-readme","title":"MLX-LM documentation","url":"https://github.com/ml-explore/mlx-lm/blob/872ae88d1fac77350db23c8c04fe8dd372a9e3e8/README.md","locator":"Quick Start; Python API; quantization; long prompts and generations","checkedAt":"2026-09-15","version":"872ae88d1fac77350db23c8c04fe8dd372a9e3e8"},{"id":"ollama-local-faq","title":"Ollama FAQ","url":"https://docs.ollama.com/faq","locator":"Local data; disabling cloud; server binding; concurrent requests","checkedAt":"2026-09-15","version":"Living documentation; runtime not pinned"},{"id":"lm-studio-local-concurrent","title":"LM Studio parallel requests","url":"https://lmstudio.ai/docs/app/advanced/parallel-requests","locator":"Max Concurrent Predictions; engine support","checkedAt":"2026-09-15","version":"Living documentation; runtime not pinned"}],"claims":[{"id":"llama-cpp-rt-gguf","text":"Loads supported models from GGUF files.","scope":"Supported model and configured local backend; documentation only, no execution recorded.","kind":"documented","evidenceIds":["llama-cpp-readme"]},{"id":"mlx-lm-rt-quantized","text":"Provides conversion to quantized model weights.","scope":"README conversion example uses 4-bit weights; not equivalent to GGUF 4-bit artifacts.","kind":"documented","evidenceIds":["mlx-lm-readme"]},{"id":"ollama-local-rt-concurrent","text":"Can process multiple requests to the same model in parallel.","scope":"Requires sufficient memory; parallel request count multiplies context memory and excess work queues.","kind":"documented","evidenceIds":["ollama-local-faq"]},{"id":"lm-studio-local-rt-concurrent","text":"Documents parallel requests for the llama.cpp engine; MLX support is listed as forthcoming.","scope":"Partial across the target engines: llama.cpp continuous batching only at capture date; configure Max Concurrent Predictions.","kind":"documented","evidenceIds":["lm-studio-local-concurrent"]}],"alternatives":[{"title":"Same-artifact GGUF comparison","tradeoff":"Compare the three GGUF targets with the same file hash and settings; wrapper and bundled-engine differences remain variables."},{"title":"Separate MLX conversion comparison","tradeoff":"Compare MLX-LM and LM Studio MLX using the same MLX artifact. Assess output quality before comparing conversion tracks."}],"gaps":["User-selected machines, model, installation boundaries and download budget are not established.","No runtime, latency, quality, concurrency or memory measurement has been collected.","App and bundled-engine versions must be pinned separately; missing capabilities remain Unknown."],"recipe":{"id":"mac-runtime-comparison","title":"Measure a pinned local model under one and four requests","status":"untested-draft","dependencyClaimIds":["llama-cpp-rt-gguf","mlx-lm-rt-quantized","ollama-local-rt-concurrent","lm-studio-local-rt-concurrent"],"steps":[{"title":"Agree on execution boundaries","instruction":"Obtain explicit authorization for the selected machines, runtime installations, model license, downloads and resource budget before executing this recipe.","expected":"A named test environment and approved resource limits."},{"title":"Pin the candidates","instruction":"Record chip, GPU cores, RAM, macOS, power mode, runtime and bundled-engine versions. Hash source-model, tokenizer, template and derived weight artifacts. Separate GGUF and MLX tracks.","expected":"A manifest that distinguishes identical artifacts from conversions."},{"title":"Fix workload and settings","instruction":"Use synthetic prompts of fixed token lengths, fixed output limits, seed and sampling settings, context size, GPU offload and concurrency. Disable speculative decoding for the baseline. Record cache and batching settings.","expected":"A reproducible baseline with one and four simultaneous requests; unsupported configurations stay explicitly unmeasured."},{"title":"Preload and isolate","instruction":"Acquire only approved models and runtimes, disable cloud and remote tools, then disconnect networking for the local measurement phase. Record startup failures and missing dependencies.","expected":"A documented local-only run boundary; no prompt leaves through an external tool."},{"title":"Measure repeated runs","instruction":"Separate cold model-load runs from warm inference. Capture startup time, time to first token, prefill and decode rates, per-request latency, peak memory and swap. Repeat the fixed workload, retaining raw samples and errors.","expected":"Per-machine and per-artifact measurements, with sample counts and p50/p95 latency where enough samples exist."},{"title":"Check correctness and interpret","instruction":"Check completion length, structured-output validity where claimed, and task correctness on the fixed prompt set. Compare within a machine and artifact track first. Report failures and avoid attributing all cross-machine differences to RAM.","expected":"A bounded report with quality checks and no unsupported universal winner."}],"checks":[{"id":"artifact-identity","expected":"Every compared same-artifact run records an identical model hash, tokenizer and template."},{"id":"local-boundary","expected":"Model inference completes with networking disconnected and external tools disabled, or the run records why it failed."},{"id":"concurrency","expected":"One-request and four-request results include queueing, errors, memory and per-request latency; backend restrictions remain visible."},{"id":"quality","expected":"Output validation and fixed task checks accompany performance metrics."},{"id":"reproducibility","expected":"Raw samples, versions and settings are retained; no executed finding is added to schema v1 documentation claims."}]},"executions":[]}