Stop Rewriting Code! Fine-Tune LLMs on Mac with mlx-tune

Stop Rewriting Code! Fine-Tune LLMs on Mac with mlx-tune

What if I told you that your biggest productivity killer isn't slow training—it's context switching?

Picture this: You're deep in flow, iterating on a fine-tuning pipeline for your latest LLM project. Your MacBook hums along, but you hit a wall. The model you need to prototype won't fit in unified memory, or worse—you're forced to rewrite your entire training script because your local setup uses Apple's MLX while your production cluster demands CUDA-optimized Unsloth code. Hours evaporate. Bugs creep in. Your creative momentum shatters against the rocks of platform incompatibility.

Sound familiar? For Apple Silicon developers, this has been the brutal reality. You either eat cloud GPU costs for breakfast just to test ideas, or you maintain two parallel codebases like some kind of digital masochist. Neither option scales. Neither option sparks joy.

Enter mlx-tune—the unofficial bridge that Apple Silicon developers have been screaming for. Born from genuine developer pain, this open-source project wraps Apple's native MLX framework in a 100% Unsloth-compatible API. Write once on your MacBook. Test locally with unified memory advantages up to 512GB. Then push that exact same script to your NVIDIA cluster without touching a single import statement. No rewrites. No surprises. Just pure, portable fine-tuning power.

This isn't vaporware promising to replace Unsloth. It's something smarter: a context-switching antidote that respects your existing workflow while unlocking local prototyping superpowers you didn't know your Mac had.

What is mlx-tune?

mlx-tune (formerly unsloth-mlx) is an open-source fine-tuning framework built specifically for Apple Silicon Macs, created by developer ARahim3 after hitting a personal friction point. The creator explains it best: "I rely on Unsloth for my daily fine-tuning on cloud GPUs—it's the gold standard for me. But recently, I started working on a MacBook M4 and hit a friction point: I wanted to prototype locally on my Mac, then scale up to the cloud without rewriting my entire training script."

The technical constraint? Unsloth depends on Triton kernels, NVIDIA's GPU programming framework that simply doesn't exist on macOS. Rather than accept defeat or maintain divergent codebases, ARahim3 built a compatibility layer that maps Unsloth's beloved FastLanguageModel API directly onto Apple's MLX framework—the machine learning engine designed specifically for Apple Silicon's unified memory architecture.

The project has since exploded in scope far beyond its original LLM focus. What started as a simple API bridge now supports SFT, DPO, GRPO, ORPO, KTO, SimPO training methods across LLMs, vision-language models, text-to-speech, speech-to-text, embedding models, and OCR systems. We're talking about fine-tuning everything from Whisper transcription to Gemma 4's audio tower to DeepSeek-OCR document understanding—all natively on your laptop.

Why it's trending now: With Apple's M4 chips delivering serious ML performance and unified memory configurations reaching 128GB+ on MacBook Pros (and 512GB on Mac Studio), the hardware has finally caught up to ambition. Meanwhile, cloud GPU costs remain punishing for iterative experimentation. mlx-tune arrives at this inflection point offering zero-friction local prototyping with an escape hatch to production-scale training.

The project's philosophy deserves emphasis: "The goal isn't to replace Unsloth or claim superior performance. The goal is code portability." This honesty builds trust. It's a fan-built tool for fans, solving a workflow problem that official channels haven't addressed.

Key Features That Make Developers Switch

Let's dissect what makes mlx-tune genuinely transformative for Apple Silicon workflows:

🚀 Universal Model Support: Fine-tune LLMs, VLMs (Vision-Language Models), TTS (Text-to-Speech), STT (Speech-to-Text), embedding models, and OCR systems through a single, coherent API. This isn't a narrow tool—it's a complete fine-tuning workstation.

💾 Unified Memory Exploitation: Apple Silicon's shared memory pool between CPU and GPU means no painful data transfers, no VRAM ceiling anxiety. Load 70B parameter models on a 128GB MacBook Pro. Train with batch sizes that would choke discrete GPUs. The architecture fundamentally changes what's possible on "laptop" hardware.

🔄 True API Parity: The compatibility claim isn't marketing fluff. Check this actual comparison from the documentation:

# Unsloth (CUDA)                        # MLX-Tune (Apple Silicon)
from unsloth import FastLanguageModel   from mlx_tune import FastLanguageModel
from trl import SFTTrainer              from mlx_tune import SFTTrainer

# Rest of your code stays exactly the same!

Same classes. Same methods. Same training scripts. The import line changes; nothing else does.

📦 Flexible Export: Save adapters, merge to full models, export to HuggingFace format, or generate GGUF for Ollama/llama.cpp deployment. Your trained models aren't trapped in Apple's ecosystem.

🎙️ Audio Frontier: Five TTS architectures (Orpheus, OuteTTS, Spark, Sesame/CSM, Qwen3-TTS) and seven STT variants including streaming real-time transcription. The audio support alone covers use cases most frameworks ignore entirely.

🔬 Research-Grade Methods: From standard SFT to cutting-edge GRPO (Group Relative Policy Optimization for reasoning models, DeepSeek R1-style) to SimPO's reference-free preference optimization. You're not sacrificing methodological sophistication for convenience.

🧠 MoE & Exotic Architectures: 39+ Mixture of Experts architectures auto-detected with per-expert LoRA via LoRASwitchLinear. Arcee Trinity-Nano's 128 experts? Handled. DeepSeek's sparse activations? Supported. Even Liquid AI's hybrid convolutional-GQA LFM2 models work natively.

Real-World Use Cases Where mlx-tune Dominates

1. The Frugal Prototyper

You're a solo developer or small team with ideas but limited cloud budget. Before committing $500+ to A100 hours, you need to validate: Does this dataset work? Is my prompt engineering sound? Will LoRA rank 8 suffice? mlx-tune lets you iterate through twenty experiments on your Mac for the cost of electricity, then promote only the most promising configuration to cloud training.

2. The Privacy-Conscious Enterprise

Healthcare, finance, legal—industries where data can't leave premises. mlx-tune enables full fine-tuning pipelines on air-gapped Apple Silicon servers. Train medical LLMs on patient records, fine-tune legal document OCR, adapt embedding models to internal knowledge bases—all without a network connection to external compute.

3. The Multi-Modal Product Builder

You're building an app that needs vision understanding, voice interaction, and text generation. Traditionally, you'd cobble together three separate training pipelines. With mlx-tune, you fine-tune Gemma 4 for visual question answering, Orpheus for voice personality, and Whisper for command recognition through identical API patterns. Your team learns one abstraction; your product gains multiple intelligences.

4. The Academic Researcher

Reproducibility and rapid iteration matter more than raw throughput. mlx-tune's local execution means no queueing for cluster time, no environment debugging across different CUDA versions. Test a hypothesis, see results in minutes, adjust. The cognitive flow stays intact. Plus, with support for continual pretraining and domain adaptation, you can push model capabilities in genuinely novel directions.

5. The On-Device AI Strategist

Apple's Neural Engine and MLX optimizations mean models trained with mlx-tune can deploy directly to iOS and macOS apps via Core ML conversion. You're not just training—you're building a pipeline from experimentation to App Store submission.

Step-by-Step Installation & Setup Guide

Getting started with mlx-tune is deliberately frictionless. The creator optimized for modern Python^{↗ Bright Coding Blog} tooling.

Prerequisites

• Hardware: Apple Silicon Mac (M1/M2/M3/M4/M5—yes, even M1 works)
• OS: macOS 13.0 or newer
• Memory: 8GB unified RAM minimum; 16GB+ strongly recommended for serious work
• Python: 3.9 or newer

Installation Commands

The project recommends uv for dependency management—it's faster and more reliable than pip:

# Core installation (recommended)
uv pip install mlx-tune

# With audio support for TTS/STT fine-tuning
uv pip install 'mlx-tune[audio]'
brew install ffmpeg  # Required system dependency for audio codecs

# Traditional pip alternative
pip install mlx-tune

# Development installation from source
git clone https://github.com/ARahim3/unsloth-mlx.git
cd mlx-tune  # Note: repository renamed, folder may vary
uv pip install -e .

Verification

Confirm successful installation:

python -c "from mlx_tune import FastLanguageModel; print('mlx-tune ready!')"

Environment Optimization

For maximum training throughput:

1. Close memory-hungry applications—unified memory means Safari tabs compete with your model
2. Use Activity Monitor to verify MLX is utilizing GPU, not falling back to CPU
3. Consider sudo nvram boot-args="vm_compressor=2" on macOS Ventura+ for aggressive memory compression (test stability first)

REAL Code Examples from the Repository

Let's examine actual working code from mlx-tune's documentation, with detailed explanations of what's happening under the hood.

Example 1: Basic LLM Fine-Tuning with SFT

This is the bread-and-butter workflow that converts Unsloth users:

from mlx_tune import FastLanguageModel, SFTTrainer, SFTConfig
from datasets import load_dataset

# Load any HuggingFace model with automatic quantization detection
# The 4-bit MLX-community model loads directly without separate conversion
model, tokenizer = FastLanguageModel.from_pretrained(
    model_name="mlx-community/Llama-3.2-1B-Instruct-4bit",
    max_seq_length=2048,      # Truncation/padding boundary
    load_in_4bit=True,        # QLoRA: 4-bit base with 16-bit adapters
)

# Inject trainable LoRA adapters into attention layers
# r=16 gives 16-dimensional low-rank decomposition—good balance for small models
model = FastLanguageModel.get_peft_model(
    model,
    r=16,
    target_modules=["q_proj", "k_proj", "v_proj", "o_proj"],  # Standard attention targets
    lora_alpha=16,            # Scaling factor: alpha/r = 1.0 (conservative)
)

# Quick dataset slice for validation—swap to full dataset for real training
dataset = load_dataset("yahma/alpaca-cleaned", split="train[:100]")

# SFTTrainer mirrors HuggingFace TRL exactly—zero learning curve
trainer = SFTTrainer(
    model=model,
    train_dataset=dataset,
    tokenizer=tokenizer,
    args=SFTConfig(
        output_dir="outputs",
        per_device_train_batch_size=2,  # Unified memory allows larger batches than you'd expect
        learning_rate=2e-4,             # Standard LoRA LR—won't destabilize 4-bit base
        max_steps=50,                   # Quick demo; production: use epochs or higher steps
    ),
)
trainer.train()

# Three save modes cover all deployment scenarios
model.save_pretrained("lora_model")           # Adapters only (small, shareable)
model.save_pretrained_merged("merged", tokenizer)  # Full fp16 model (inference-ready)
model.save_pretrained_gguf("model", tokenizer)     # GGUF for Ollama (see limitations below)

What's remarkable here: The SFTConfig and SFTTrainer classes are drop-in replacements. If you've used TRL (Transformer Reinforcement Learning), this is muscle memory. The MLX-specific optimizations—memory mapping, unified buffer management, Metal kernel dispatch—happen transparently.

Example 2: Vision-Language Model Fine-Tuning

Fine-tuning multimodal models has traditionally required arcane data collation. mlx-tune abstracts this away:

from mlx_tune import FastVisionModel, UnslothVisionDataCollator, VLMSFTTrainer
from mlx_tune.vlm import VLMSFTConfig

# Load vision model—auto-detects image processor architecture
model, processor = FastVisionModel.from_pretrained(
    "mlx-community/Qwen3.5-0.8B-bf16",  # Vision-language checkpoint
)

# Fine-tune both modalities or lock vision encoder for efficiency
model = FastVisionModel.get_peft_model(
    model,
    finetune_vision_layers=True,    # Update image understanding (costly but flexible)
    finetune_language_layers=True,  # Update text generation (usually necessary)
    r=16, lora_alpha=16,
)

# Required: flag model for training mode (sets dropout, gradient checkpointing)
FastVisionModel.for_training(model)

# UnslothVisionDataCollator handles image tensor batching automatically
# No manual padding, resizing, or attention mask construction
trainer = VLMSFTTrainer(
    model=model,
    tokenizer=processor,  # Vision processors double as tokenizers
    data_collator=UnslothVisionDataCollator(model, processor),
    train_dataset=dataset,  # Format: {"images": [...], "messages": [...]}
    args=VLMSFTConfig(max_steps=30, learning_rate=2e-4),
)
trainer.train()

Critical insight: The UnslothVisionDataCollator is doing heavy lifting invisible to the user—handling variable image sizes, creating pixel attention masks, and ensuring vision tokens align with language positions. On CUDA, this often requires custom DataLoader subclasses. Here, it's one argument.

Example 3: Speech-to-Text with Advanced Loss Functions

For developers building voice interfaces, mlx-tune's STT support includes cutting-edge architectures:

from mlx_tune import FastSTTModel, STTSFTTrainer, STTSFTConfig, STTDataCollator

# Auto-detects architecture: Whisper encoder-decoder vs. CTC vs. RNN-T
model, processor = FastSTTModel.from_pretrained(
    "mlx-community/whisper-tiny-asr-fp16"
)

# Fine-tune both encoder (acoustic modeling) and decoder (language modeling)
# Or freeze encoder for transfer learning to new languages
model = FastSTTModel.get_peft_model(
    model, r=8,
    finetune_encoder=True,   # Adapt to new accents, noise conditions
    finetune_decoder=True,   # Adapt to domain vocabulary
)

# STTDataCollator handles: mel-spectrogram computation, padding, token masking
# language="en" and task="transcribe" set Whisper's special tokens
trainer = STTSFTTrainer(
    model=model, processor=processor,
    data_collator=STTDataCollator(
        model, processor,
        language="en",
        task="transcribe",    # vs. "translate" for cross-lingual
    ),
    train_dataset=dataset,    # Audio file paths + transcripts
    args=STTSFTConfig(output_dir="./stt_output", max_steps=60),
)
trainer.train()

Pro tip: The finetune_encoder=True flag is powerful but memory-intensive. For quick domain adaptation (medical terminology, brand names), finetune_decoder=False with higher LoRA rank often suffices.

Example 4: Embedding Models with Contrastive Learning

Semantic search and retrieval-augmented generation demand quality embeddings. mlx-tune supports modern architectures:

from mlx_tune import FastEmbeddingModel, EmbeddingSFTTrainer, EmbeddingSFTConfig, EmbeddingDataCollator

# Pooling strategy critical for downstream task: "mean" for similarity, "cls" for classification
model, tokenizer = FastEmbeddingModel.from_pretrained(
    "mlx-community/all-MiniLM-L6-v2-bf16",
    pooling_strategy="mean",  # Average all token embeddings
)
model = FastEmbeddingModel.get_peft_model(model, r=16, lora_alpha=16)

# InfoNCE loss with in-batch negatives: efficient, no hard negative mining needed
trainer = EmbeddingSFTTrainer(
    model=model, tokenizer=tokenizer,
    data_collator=EmbeddingDataCollator(model, tokenizer),
    train_dataset=[
        {"anchor": "query about neural networks", "positive": "relevant paper on backprop"},
        # ... more (anchor, positive) pairs
    ],
    args=EmbeddingSFTConfig(
        loss_type="infonce",
        temperature=0.05,      # Lower = sharper distinction, harder training
        per_device_train_batch_size=32,  # Embeddings allow large batches
        max_steps=50,
    ),
)
trainer.train()

# Inference: encode and compute cosine similarity via dot product
embeddings = model.encode(["Hello world", "Hi there"])
similarity = (embeddings[0] * embeddings[1]).sum().item()  # MLX native tensor op

The temperature parameter: At 0.05, this is aggressive contrastive learning. For noisy real-world data, consider 0.1-0.2 to prevent collapse. The in-batch negatives mean every other positive in your batch becomes a negative for each anchor—massively efficient.

Advanced Usage & Best Practices

Memory-First Training Strategy: Apple Silicon's unified memory is generous but not infinite. Use load_in_4bit=True for base models, keep max_seq_length tight to your actual data distribution, and monitor with mlx.core.metal.get_active_memory(). For 70B models on 128GB systems, gradient checkpointing (gradient_checkpointing=True in configs) trades ~30% speed for 50% memory reduction.

LoRA Target Selection: Default attention targets (q_proj, k_proj, v_proj, o_proj) work universally. For domain adaptation with plentiful data, add MLP layers (gate_proj, up_proj, down_proj). For MoE models, mlx-tune auto-routes to expert-specific layers—don't manually specify SwitchLinear paths.

Response-Only Training Efficiency: Use train_on_responses_only() to mask loss computation on instruction tokens. This can reduce effective sequence length by 60%+ for chat datasets, directly translating to faster training and lower memory.

Continual Pretraining Discipline: When using CPTTrainer, always use base (not instruction-tuned) models. The decoupled learning rates—embedding_learning_rate 10x smaller than learning_rate—prevent catastrophic forgetting of token representations while adapting to new domains.

Export Strategy: For GGUF deployment, plan ahead. If you need Ollama compatibility, start with non-quantized base models or accept the dequantize-and-re-quantize workflow. For pure Python/MLX serving, save_pretrained_merged() is simpler and equally performant.

Comparison with Alternatives

When to choose native mlx-lm: You're doing pure research, need maximum flexibility, and don't mind writing custom training loops.

When to choose PyTorch MPS: You have existing PyTorch code that can't be refactored, and you accept 2-3x memory overhead.

When to choose mlx-tune: You value developer velocity, maintain both Mac and Linux workflows, or need multi-modal capabilities without framework juggling.

FAQ: Developer Concerns Addressed

Q: Is mlx-tune officially affiliated with Unsloth? A: No—it's a community project by a developer who uses Unsloth daily. The API compatibility is a deliberate design choice, not an official partnership. The project was renamed from unsloth-mlx to mlx-tune precisely to avoid confusion.

Q: Can I really run the same script on CUDA and Apple Silicon? A: Yes, with one import change. from mlx_tune import FastLanguageModel becomes from unsloth import FastLanguageModel. All method signatures, config parameters, and save formats are mirrored.

Q: What about performance compared to Unsloth on A100s? A: Apple Silicon won't beat NVIDIA at raw throughput for equivalent cost. The value proposition is workflow efficiency: prototype 10x more iterations locally, then promote the best to cloud. For many model sizes, local training is surprisingly competitive.

Q: Which Mac should I use? A: M2 Pro/Max or newer with 32GB+ unified memory for serious work. M1 works for 7B models. M4 Max with 128GB handles 70B parameter models in 4-bit. Mac Studio with 512GB is essentially a research workstation.

Q: Does mlx-tune support distributed training? A: Not currently—single-device only. For multi-GPU scaling, export your validated script to Unsloth on CUDA. This aligns with the project's "prototype local, scale cloud" philosophy.

Q: What's the catch with GGUF export? A: A known limitation in upstream mlx-lm prevents direct GGUF export from 4-bit quantized base models. Workarounds: use non-quantized bases, dequantize during export, or skip GGUF for MLX-native deployment.

Q: Can I fine-tune on proprietary data without internet? A: Absolutely. All training happens locally. You only need internet for initial model download from HuggingFace, which can be pre-cached.

Conclusion: The Context-Switching Killer Has Arrived

The silent productivity destroyer in modern ML engineering isn't slow code or buggy frameworks—it's the cognitive and technical overhead of maintaining parallel workflows across platforms. mlx-tune strikes at this root cause with surgical precision, delivering genuine Unsloth API compatibility on Apple's increasingly formidable Silicon.

What started as one developer's personal friction point has evolved into a remarkably comprehensive fine-tuning ecosystem. From whisper-tiny STT adaptation to 128-expert MoE tuning to vision-language reasoning with GRPO, the breadth exceeds what many commercial platforms offer. The unified memory architecture isn't just a spec sheet bullet point here—it's actively exploited to enable model sizes and batch configurations impossible on discrete GPU laptops.

My assessment: If you develop on Apple Silicon and ever touch fine-tuning, mlx-tune belongs in your toolkit. Not as a Unsloth replacement, but as its natural complement—the local yin to cloud CUDA's yang. The code portability promise is real, validated, and transformative for iterative development workflows.

Ready to stop rewriting and start training? Head to github.com/ARahim3/unsloth-mlx (now mlx-tune on PyPI), star the repository to support open-source development, and run your first SFTTrainer in the next ten minutes. Your future self—prototyping freely on a MacBook, deploying confidently to A100s—will thank you.

Community project. Not affiliated with Unsloth AI or Apple. Built by a fan, for fans who happen to use Macs.