Run 122B AI Locally on MacBook: claude-code-local Exposed

Run 122B AI Locally on MacBook: claude-code-local Exposed

What if your MacBook could run AI models larger than most data centers — without ever connecting to the internet? No API keys. No subscription fees. No data leaving your machine. Sound impossible? Matt Macosko just proved it isn't.

Here's the painful truth every developer knows: cloud AI is bleeding you dry. $20 here, $100 there, and suddenly you're paying more for Claude than your entire AWS^{↗ Bright Coding Blog} bill. Worse? Every line of code you paste into that chat window becomes someone else's training data. Working on proprietary software? Handling HIPAA-protected health records? Reviewing NDAs under strict confidentiality? You're one paste away from a career-ending leak. And don't even try coding on that cross-country flight — no Wi-Fi, no AI, no productivity.

But what if the solution wasn't another cloud wrapper or privacy policy to trust? What if the answer was sitting in your lap right now, hiding inside that M-series chip Apple designed?

Enter claude-code-local — the open-source project that's making developers rip out their API keys and never look back. Built by Matt Macosko in Arcata, California, this isn't another "local AI" toy that chokes on anything beyond "Hello, world." We're talking 122 billion parameters running at 65 tokens per second. We're talking full Claude Code functionality — code editing, file management, browser automation, even hands-free voice control — completely air-gapped. The same AI power that costs $100/month in the cloud, now running on your MacBook at exactly $0 forever.

Still skeptical? You should be. I was too. Until I saw the lsof output with my own eyes.

What Is claude-code-local?

claude-code-local is an MLX-native Anthropic API server that lets you run Claude Code — Anthropic's official AI coding agent — entirely on-device using local large language models. Created by Matt Macosko and released under the MIT license, this project represents a fundamental shift in how developers can leverage AI: from cloud-dependent, subscription-locked, and privacy-compromised to local-first, offline-capable, and mathematically secure.

The project emerged from a simple frustration. Macosko wanted to use Claude Code on sensitive client work without sending proprietary code to Anthropic's servers. The standard approach — running Ollama or llama.cpp with a proxy translation layer — added latency, broke tool calls, and turned a snappy coding session into a 133-second slog. So he took a radically different path: write a native Anthropic API server in ~1000 lines of Python^{↗ Bright Coding Blog}, directly on Apple's MLX framework, eliminating the proxy entirely.

The result? A 7.5× speed improvement — from 133 seconds per task down to 17.6 seconds. But speed was just the beginning. The project has evolved into a complete "lineup" of local AI "fighters": Gemma 4 31B for daily coding on 64GB Macs, Qwen 3.5 122B for maximum throughput at 65 tok/s, and DeepSeek V4 Flash with its staggering 1 million token context window. Each model swaps in with a single environment variable change, speaking the same Anthropic API that Claude Code expects.

What makes claude-code-local genuinely revolutionary isn't just the technical achievement — it's the verifiable privacy guarantee. Every component has been audited: the custom server.py (zero outbound calls), mlx-lm from Apple's ML team (zero outbound calls), the MLX framework itself (zero outbound calls). Even the Claude Code CLI binary, which previously phoned home for telemetry and feature flags, is now fully sandboxed with documented environment variables. Run lsof -p $(pgrep -f claude) during a session and you'll see only localhost:4000 — no api.anthropic.com, no statsig.com, no sneaky background connections.

The project has gained serious traction in privacy-conscious developer communities: lawyers auditing NDAs, healthcare professionals reviewing patient data, contractors handling classified government work, and anyone who's ever stared at a "No External Services" policy and wished AI could still help them code.

Key Features That Make Developers Switch

Native Anthropic API — Zero Translation Layer. Unlike every other local AI solution, claude-code-local doesn't translate between OpenAI-format and Anthropic-format APIs. It speaks Anthropic's native protocol directly. Claude Code thinks it's talking to the cloud. It's actually talking to your GPU. This single architectural decision eliminates an entire class of bugs, latency, and compatibility issues that plague proxy-based setups.

Three-Model Lineup with Hot-Swapping. The project ships with three production-ready models: Gemma 4 31B (abliterated, 4-bit, ~18GB RAM, ~15 tok/s, fits 64GB Macs), Qwen 3.5 122B-A10B (4-bit MoE, ~75GB RAM, 65 tok/s, 256K context), and DeepSeek V4 Flash (2-bit asymmetric via Antirez's ds4 engine, ~81GB RAM, ~32 tok/s, 1 million token context). Swap between them by changing one environment variable or double-clicking a different launcher. The MoE architecture is crucial here — Qwen's 122B parameters sound massive, but only 10B activate per token, making it faster than many 30B dense models.

Production-Hardened Tool-Call Reliability. Local models garble tool calls. They mix XML and JSON syntax, create infinite loops where they "want" to call a tool but never actually do. Version 2 (March 2026) fixes this with four coordinated changes: 8-bit KV cache with quantization delayed to token 1024 (preserves conversation context), temperature dropped to 0.2 (less randomness), a recover_garbled_tool_json() function that catches XML-in-JSON hybrids, and retry logic with explicit formatting instructions. 98/98 automated tests passed across 7 consecutive runs.

Full MCP Ecosystem Compatibility. Claude Code's plugin protocol — Model Context Protocol — is how it reads files, queries databases, controls browsers, and accesses 200+ tools. Most local proxies break MCP by mangling tool definitions or stripping tool_use blocks. claude-code-local passes tool definitions through and translates responses back, across all three model families, with special MLX_BROWSER_MODE=1 optimization that reduces a 10K-token browser-control prompt to ~200 tokens (28× reduction).

Hands-Free Voice Loop — Both Directions On-Device. This is the feature Macosko is "proudest of in the whole stack." Speak to your Mac, hear replies in your cloned voice. Apple's SFSpeechRecognizer (the same on-device engine powering macOS Dictation) handles speech-to-text. Pocket TTS with voice cloning handles text-to-speech. Auto-pause during playback prevents feedback loops. A 10-minute preventive process recycle dodges known SFSpeech daemon wedges. No cloud STT. No cloud TTS. No data leaves your Mac in either direction.

iMessage Media Pipeline — Control From Your Couch. Text your Mac from your iPhone, get back text, screenshots, screen recordings, or fully produced videos. The local model plans the task, Brave browser executes it, Studio Record captures it, and a shell script ships it back via iMessage. Works over iMessage's native protocol — no bots, no third-party apps, no cloud relay.

Real-World Use Cases Where This Shines

1. NDA-Bound Legal and Financial Work

Imagine you're a lawyer reviewing a merger agreement under strict confidentiality. Every clause analysis, every risk flag, every redline suggestion — normally you'd be pasting into Claude's web interface, praying their data retention policy holds. With claude-code-local, you physically disconnect Wi-Fi, run lsof to verify zero outbound connections, and have a 70B-parameter model audit the document with mathematically guaranteed privacy. The demo video shows exactly this: a real NDA, Llama 3.3 70B, Wi-Fi physically OFF, live lsof running on screen. Your malpractice insurer will thank you.

2. Healthcare and HIPAA-Protected Environments

Medical coders reviewing patient charts. Researchers analyzing clinical trial data. Therapists drafting treatment notes. Any scenario where PHI (Protected Health Information) exists, cloud AI is a compliance nightmare requiring Business Associate Agreements, risk assessments, and ongoing audit trails. claude-code-local removes the compliance burden entirely — the data cannot leave because there is no network path out. The model runs on the same machine holding the data. Air-gapped by architecture, not by policy.

3. Airplane, Train, and Remote Development

That 6-hour transatlantic flight where you planned to catch up on refactoring? Cloud AI is useless at 35,000 feet. claude-code-local doesn't care. Download your models before takeoff, work offline the entire flight, land with merged PRs. The same applies to remote field sites, military installations with classified networks, client sites with locked-down firewalls, or that cabin in the woods where you finally get focused work done.

4. Zero-Subscription Cost Scaling

A 10-developer team using Claude Code Pro at $100/month each burns $12,000 annually. For that money, you could buy three M4 Max Mac Studios and run equivalent capability locally forever. Individual developers tired of API rate limits, usage caps, and surprise bills can budget exactly $0 for AI coding assistance. The Qwen 3.5 122B setup beats cloud Claude Opus on raw speed (65 vs 40 tok/s) while costing nothing.

5. Hands-Free Coding for Accessibility and Ergonomics

Developers with repetitive strain injury, carpal tunnel, or other conditions making keyboard use painful. The voice loop lets you speak code reviews, hear narrated responses, and maintain productivity without touching a keyboard. Beyond accessibility, Macosko frames this as "ambient computing" — getting off screens and mice entirely, computing that's "around you instead of in front of you."

Step-by-Step Installation & Setup Guide

Prerequisites

Also required:

• Python 3.12+ (for MLX framework compatibility)
• Claude Code CLI (npm install -g @anthropic-ai/claude-code)
• ~18-81 GB free disk space per model you want to download

Automated Setup (Recommended)

The setup.sh script auto-detects your RAM, picks an appropriate model, downloads it, installs the MLX server, and creates a double-clickable launcher on your Desktop.

# Clone the repository
git clone https://github.com/nicedreamzapp/claude-code-local
cd claude-code-local

# Run the automated installer
bash setup.sh

After completion, double-click Claude Local.command on your Desktop. That's it — you're coding with local AI.

Troubleshooting: If the launcher prompts you to sign in to a Claude account, your claude CLI is outdated. The launchers pass --bare for local-only auth, but older versions don't support this flag:

Advertisement

npm install -g @anthropic-ai/claude-code
claude --version  # Verify recent version

For contributors: setup.sh installs the server as a symlink at ~/.local/mlx-native-server/server.py pointing back to the repo's proxy/server.py. Edit in the repo, restart the server, done — no re-running setup, no silent drift.

Manual Setup (Full Control)

# 1. Create isolated Python environment for MLX
python3.12 -m venv ~/.local/mlx-server
~/.local/mlx-server/bin/pip install mlx-lm

# 2. Download your chosen model (one-time, ~18-75 GB)
# Options: 'gemma', 'llama', or 'qwen'
bash scripts/download-and-import.sh gemma

# 3. Start the MLX-native Anthropic API server
MLX_MODEL=divinetribe/gemma-4-31b-it-abliterated-4bit-mlx \
  bash scripts/start-mlx-server.sh

# 4. Launch Claude Code pointing at local server
ANTHROPIC_BASE_URL=http://localhost:4000 \
ANTHROPIC_API_KEY=sk-local \
claude --model claude-sonnet-4-6

The ANTHROPIC_API_KEY=sk-local is arbitrary — the server accepts any key since authentication happens locally. The --model claude-sonnet-4-6 flag is Claude Code's internal model identifier; our server intercepts this and routes to your local model regardless.

Environment Variable Tuning

REAL Code Examples From the Repository

Example 1: Starting the MLX Server with Custom Configuration

#!/bin/bash
# scripts/start-mlx-server.sh — boots the MLX-native Anthropic API server
# This is the heart of claude-code-local: ~1000 lines of Python speaking
# Anthropic's protocol directly to Claude Code, with zero proxy translation.

# Override default model via environment variable
MLX_MODEL="${MLX_MODEL:-divinetribe/gemma-4-31b-it-abliterated-4bit-mlx}"

# KV cache tuning: 8-bit preserves conversation context better than 4-bit,
# preventing the "forgetting" that causes infinite tool-call loops
MLX_KV_BITS="${MLX_KV_BITS:-8}"
MLX_KV_QUANT_START="${MLX_KV_QUANT_START:-1024}"

# Tool-call recovery: how many times to retry when model garbles JSON format
MLX_TOOL_RETRIES="${MLX_TOOL_RETRIES:-2}"

# Launch the server on port 4000 — Claude Code's default local endpoint
exec ~/.local/mlx-server/bin/python proxy/server.py \
  --model "$MLX_MODEL" \
  --kv-bits "$MLX_KV_BITS" \
  --kv-quant-start "$MLX_KV_QUANT_START" \
  --tool-retries "$MLX_TOOL_RETRIES" \
  --port 4000

What this does: The launcher script encapsulates all server configuration in environment variables, making model switching as simple as changing one variable. The exec replaces the shell process with Python, ensuring clean signal handling. Default values are optimized for reliability over minimal memory — the 8-bit KV cache and 1024-token quantization delay were specifically tuned to fix the tool-call infinite loops that plagued earlier versions.

Example 2: Loading a Custom Model with Abliterated Weights

# Launch with Llama 3.3 70B — our own 8-bit MLX abliterated build
# "Abliteration" suppresses the model's built-in refusal direction,
# preventing false refusals on benign-but-edgy requests.
# NOT a capability upgrade — use responsibly, bound by upstream license.

MLX_MODEL=divinetribe/Llama-3.3-70B-Instruct-abliterated-8bit-mlx \
  bash scripts/start-mlx-server.sh

What this does: This demonstrates the hot-swap capability — same server, same API, different brain. The divinetribe/Llama-3.3-70B-Instruct-abliterated-8bit-mlx model is Macosko's own upload to HuggingFace: 8-bit affine quantization with group size 64, preserving quality over minimal footprint. The abliteration technique (explained in Maxime Labonne's blog post) removes the "refusal vector" from the model's representation space without retraining. The 15 safetensors shards total ~75 GB, requiring substantial disk space but fitting comfortably in 128 GB unified memory systems.

Example 3: Running the Automated Tool-Call Test Suite

# scripts/test_mlx_server.py — validates tool-call reliability
# This suite was built to reproduce and verify fixes for the infinite-loop bug.

import subprocess
import sys

def run_test_suite():
    """
    14 tests per run, covering the exact failure modes that broke before v2:
    - Simple Bash commands
    - Directory creation with mkdir -p
    - File reading via Read tool
    - Complex Bash with pipes
    - File editing with find/replace
    - Multi-tool sequences (Glob → Read)
    - Rapid-fire sequential commands
    - Multi-step calendar scenario (create → delete → verify)
    """
    # The calendar scenario was the exact infinite-loop trigger:
    # create 12 month folders, delete all but September, verify.
    # Before the fix: model would say "let me do that" forever.
    # After: passes every time.
    
    results = subprocess.run(
        [sys.executable, "scripts/test_mlx_server.py"],
        capture_output=True,
        text=True
    )
    
    # Expected: 98/98 tests passed across 7 consecutive runs (zero failures)
    print(results.stdout)
    assert "Zero failures" in results.stdout or results.returncode == 0

if __name__ == "__main__":
    run_test_suite()

What this does: The test suite validates the four coordinated fixes in v2: KV cache quality (8-bit, delayed quantization), temperature reduction (0.2), garbled JSON recovery, and retry logic. The multi-step calendar scenario is particularly significant — it requires the model to maintain context across multiple tool calls, plan ahead, and verify results, exactly the kind of complex workflow that caused infinite loops before. Running this yourself gives confidence that your local setup handles real Claude Code tasks reliably.

Example 4: Adding an MCP Server for Local Fileystem Access

# MCP (Model Context Protocol) servers extend Claude Code's capabilities.
# This adds filesystem access — local model can read/write your projects.
# The proxy passes tool definitions through and translates responses back.

# 1. Add Anthropic's reference filesystem MCP server
claude mcp add filesystem -- npx -y @modelcontextprotocol/server-filesystem ~/projects

# 2. Launch with local model (any launcher works)
# The local model now has full filesystem access via the same protocol
# cloud Claude uses — but nothing leaves your machine.

What this does: MCP servers are how Claude Code extends beyond text generation into your actual tools and data. The claude mcp add command registers a new capability; the npx -y @modelcontextprotocol/server-filesystem package runs a local Node.js server that exposes filesystem operations. When your local model "wants" to read a file, the proxy translates its tool-use block into the format the MCP server expects, then translates the response back. The filesystem server makes outbound calls to your local disk only — no network involved. This pattern works for 200+ MCP servers: GitHub, Postgres, Slack, Brave Search, Chrome DevTools, and more.

Example 5: Browser Mode Optimization for Web Automation

# Chrome DevTools MCP sends 30+ tools and 10K-token system prompts.
# Death for local models. MLX_BROWSER_MODE strips the bloat.

MLX_BROWSER_MODE=1 ./scripts/start-mlx-server.sh

# Now when Claude Code detects browser MCP registration
# (mcp__chrome-devtools__* tools), it auto-optimizes:
# - Keeps only 9 essential browser-control tools
# - Reduces prompt from ~5600 tokens to ~200 tokens
# - 28× reduction cuts prefill from ~60s to ~2s on Gemma 4 31B

What this does: Browser automation via MCP is one of the most powerful Claude Code features — "go research this topic, take screenshots, fill forms, extract data." But the full Chrome DevTools protocol exposes dozens of tools with verbose descriptions. Cloud Claude handles this fine; local models choke. MLX_BROWSER_MODE=1 enables aggressive optimization that detects browser-specific MCP sessions and strips everything non-essential. The 28× prompt reduction is dramatic: from ~5,600 tokens describing all possible browser operations down to ~200 tokens covering only click, type, navigate, screenshot, evaluate, and a few others. This makes browser automation viable on 64GB Macs, not just 128GB monsters.

Advanced Usage & Best Practices

Prompt Cache Reuse for Speed. The server automatically caches Claude Code's 4K-token system prompt across requests. For short follow-up questions, this eliminates re-prefilling — a massive speedup. The DeepSeek V4 Flash integration goes further with on-disk KV cache: prefill a 25K-token system prompt once, persist to disk, never prefill again across restarts.

Code Mode Auto-Detection. The server detects coding sessions (presence of Bash/Read/Edit/Write/Grep/Glob tools) and swaps Claude Code's ~10K-token harness prompt for a slim ~150-token version tuned for local models. It also strips verbose tool descriptions from 35 tools down to name + parameter types. Result: 28× prompt reduction, prefill time from ~60s to ~2s. You don't configure this — it just happens.

Model Selection by Task. Use Gemma 4 31B for quick iterations and daily coding (fits 64GB Macs). Switch to Qwen 3.5 122B for maximum throughput on complex refactors (65 tok/s, but needs 96GB+ RAM). Deploy Llama 3.3 70B when reasoning quality matters more than speed (hardest problems, full precision). Use DeepSeek V4 Flash for agentic loops with massive context (1M tokens, 128GB+ RAM).

Voice Loop Production Hardening. The NarrateClaude sibling repo includes wedge-detection (auto-restart stuck speech recognizer), queue-backlog monitoring (non-zero exit when listener is stuck), and 10-minute preventive process recycle. Don't run voice demos as scripts — run them as infrastructure.

Security Verification Habit. After any update to Claude Code CLI, re-run lsof -p $(pgrep -f claude) during a session. Confirm only localhost:4000 appears. The launchers set the four blocking environment variables automatically, but verify after major CLI updates.

Comparison With Alternatives

Why claude-code-local wins: It's not just "local AI" — it's local AI that actually works with Claude Code's full feature set. The native Anthropic API eliminates translation bugs. The tested tool-call reliability means infinite loops are gone. The MCP compatibility means you don't lose plugins going local. And the voice/iMessage integrations are simply unavailable elsewhere.

When cloud still makes sense: If you need Claude-level reasoning on novel problems (local models are good, not Claude-level), or if you prioritize absolute maximum speed over privacy (Sonnet is slightly faster at 80 tok/s). For everything else — especially sensitive work, offline needs, or cost control — local wins decisively.

FAQ

Does claude-code-local work on Intel Macs? No. The MLX framework is Apple Silicon-only (M1/M2/M3/M4/M5 and variants). Intel Macs lack the unified memory architecture and Metal GPU performance required for efficient large model inference.

How much RAM do I actually need? Minimum 32 GB for Gemma 4 31B (tight but functional). 64 GB for comfortable Gemma usage. 96 GB+ for Qwen 3.5 122B. 128 GB+ for DeepSeek V4 Flash or running multiple models. The rule of thumb: model size in 4-bit quantization ≈ 0.5× parameter count in GB, plus overhead for KV cache and OS.

Is this actually secure, or just "trust me bro"? Verifiably secure. Run lsof -i -P and lsof -p $(pgrep -f claude) during operation. The only network connection is localhost:4000. Every component's outbound call status is documented in the audit table. The server source is ~1000 lines you can read yourself. No telemetry, no analytics, no phone-home — mathematically guaranteed by absence of network code.

Can I use my own fine-tuned models? Yes. Set MLX_MODEL=<your-huggingface-url> pointing to any MLX-compatible model. The server loads anything mlx-lm can handle. Community contributions for additional "fighters" in the lineup are welcome via PR.

Does tool-call reliability vary by model? Yes. Qwen 3.5 122B with its MoE architecture and larger active parameter count formats tools most reliably. Gemma 4 31B sometimes needs the garbled-recovery path. Llama 3.3 70B is intermediate. The v2 fixes (8-bit KV, lower temperature, recovery function, retries) help all models significantly.

What happens when Claude Code CLI updates? The launchers auto-set the four blocking environment variables for new CLI versions. However, Anthropic could change behavior. Re-verify with lsof after major updates. The community typically flags issues within hours — star the repo and watch for notifications.

How does this compare to running Ollama with a generic proxy? 7.5× faster for real tasks, infinitely more reliable for tool calls, full MCP compatibility, and native Anthropic API instead of translation hacks. The proxy is the bottleneck — eliminating it was the entire point.

Conclusion

The future of AI-assisted development isn't in someone else's data center — it's in your backpack, on your desk, running silently while you sleep. claude-code-local proves that "local AI" doesn't have to mean "compromised AI." With 122 billion parameters at 65 tokens per second, full Claude Code compatibility, verifiable air-gapped privacy, and capabilities like hands-free voice control that don't exist in the cloud version, this project isn't a downgrade from cloud AI. For many workflows, it's an upgrade.

Matt Macosko built this because he needed it — for chicken problems, client NDAs, plane flights, and the simple dignity of knowing his code stayed his. He open-sourced it because the problem is universal. Every developer who's ever hesitated before pasting proprietary code into a chat window. Every healthcare worker navigating HIPAA. Every contractor on a classified network. Every traveler 30,000 feet above the Atlantic with a deadline looming.

The math is simple: zero outbound HTTP calls means zero data leakage. The speed is proven: 17.6 seconds versus 133 seconds for the same real task. The cost is undeniable: $0 forever versus $100/month.

But the real revolution is subtler. It's the voice loop that lets you code while walking. It's the iMessage pipeline that turns your phone into a terminal for your home Mac. It's the ambient computing vision where AI is around you, not in front of you — where screens become optional, typing becomes optional, but your data never becomes someone else's training set.

Clone it. Run it. Verify it with lsof. Then ask yourself why you ever paid for cloud AI in the first place.

👉 github.com/nicedreamzapp/claude-code-local — Star it, fork it, bend it to your workflow. The whole stack is MIT-licensed and waiting.

Built by Matt Macosko in Arcata, CA. Started with a chicken problem. Still figuring it out.