Ornith MLX Guide: Running Ornith 1.0 on Apple Silicon Macs

July 15, 2026 · 7 min read

Running Ornith MLX locally on a MacBook or Mac Studio is entirely possible — and for many developers it is the most practical path to a private, zero-cost Ornith 1.0 deployment. This guide covers the two main routes (mlx-lm and LM Studio), how to match model size to your Mac’s unified memory, and what performance to realistically expect in 2026.

If you have not yet read the Ornith 1.0 model overview, start there to understand how the 9B, 35B, and 31B variants differ before deciding which one to download.

Why Apple Silicon Is a Good Fit for Ornith

Apple Silicon Macs use unified memory — a single pool shared by the CPU, GPU, and Neural Engine. Unlike a discrete GPU with a fixed VRAM ceiling, a Mac with 32 GB of RAM can address all 32 GB for model weights. That means a quantized model that would exceed a typical consumer GPU’s VRAM can run locally on a Mac that costs far less than a workstation with an NVIDIA A100.

The trade-off is throughput. Apple Silicon is notably slower than a modern discrete GPU for dense transformer inference. For personal coding tasks, document summarisation, or exploratory use, the speed is workable. For production pipelines that need server-grade throughput, a cloud GPU is the right tool. But for a fully private, always-available local model, the Mac path is hard to beat.

The Ornith 1.0 family’s 262K context window is supported on all variants — though on memory-constrained Macs you will want to cap context to 32K or 64K to reduce RAM pressure.

Choosing the Right Ornith Model for Your Mac

Not every Ornith model fits every Mac. Here is the decision matrix based on unified memory size:

8 GB Macs are not recommended for Ornith. At Q2 or Q3 quantization the model fits, but quality degrades significantly and generation speed is poor. If you are on an 8 GB machine, consider using the hosted try page instead.

A 16 GB Mac — the M2 Pro MacBook Pro baseline — is the sweet spot for the 9B. The Q4_K_M GGUF weighs roughly 6 GB, leaving ample headroom for macOS and other applications. See the VRAM requirements post for a full breakdown across all quantization levels.

The 32 GB Mac is where things get interesting. Ornith-1.0-35B is a Mixture-of-Experts model (~3B active parameters per token), which means it does far less computation per token than a dense 35B would. The Q5_K_M GGUF sits at ~25 GB — comfortable on a 32 GB machine. On Apple Silicon, MoE models tend to be relatively efficient because the per-token compute cost is low even if total parameter count is high.

The 397B model requires around 200 GB at FP8 across eight 80 GB GPUs. It is not runnable on any Mac.

Path 1: Ornith MLX via mlx-lm

MLX is Apple’s open-source machine learning framework designed specifically for Apple Silicon. The mlx-lm package provides a command-line interface that loads GGUF models directly into unified memory with Metal acceleration.

Installation

pip install mlx-lm

Python 3.10+ is required. A virtual environment is recommended:

python -m venv ornith-env
source ornith-env/bin/activate
pip install mlx-lm

Running Ornith-1.0-9B with mlx-lm

python -m mlx_lm.generate \
  --model deepreinforce-ai/Ornith-1.0-9B-GGUF \
  --prompt "Write a Python function that reads a CSV file and returns a list of dicts"

mlx-lm will download the GGUF weights from Hugging Face on first run and cache them locally. Subsequent runs load from cache.

Running Ornith-1.0-35B with mlx-lm

python -m mlx_lm.generate \
  --model deepreinforce-ai/Ornith-1.0-35B-GGUF \
  --prompt "Explain the MoE architecture and why it is efficient for inference"

On a 32 GB Mac you can also pass --max-tokens 2048 and --context-length 32768 to cap memory usage if the OS is competing for RAM.

Capping Context to Save Memory

The full 262K context window is memory-hungry. For typical coding tasks, 32K is more than sufficient:

python -m mlx_lm.generate \
  --model deepreinforce-ai/Ornith-1.0-9B-GGUF \
  --context-length 32768 \
  --prompt "Your prompt here"

For interactive chat with mlx-lm, replace generate with chat:

python -m mlx_lm.chat \
  --model deepreinforce-ai/Ornith-1.0-9B-GGUF \
  --context-length 32768

Path 2: Ornith 1.0 via LM Studio

LM Studio is a GUI application for running local models. It has native Apple Silicon support, handles quantization selection automatically, and requires no terminal work.

# 1. Open LM Studio
# 2. Search "Ornith-1.0" in the model browser
# 3. Download Q4_K_M (for 9B) or Q5_K_M (for 35B)
# 4. Load and start chatting

LM Studio handles the Metal backend and unified memory allocation transparently. For users who prefer not to work in the terminal, it is the fastest path to a running Ornith MLX session. The model browser pulls directly from Hugging Face, so the same GGUF files used by mlx-lm are available here.

One limitation: LM Studio’s API server uses OpenAI-compatible endpoints, which makes it easy to point coding tools like Cursor or VS Code extensions at your local Ornith instance.

Path 3: Ollama on Mac

Ollama is a third option for Mac users who want a lightweight background service. The command is straightforward:

ollama run hf.co/deepreinforce-ai/Ornith-1.0-9B-GGUF

Ollama will pull the GGUF, create a local model entry, and drop you into an interactive session. It also exposes an OpenAI-compatible HTTP server at localhost:11434. See the full Ollama walkthrough for more detail on configuration options.

Performance Expectations on Apple Silicon

Do not expect cloud-GPU speed. Apple Silicon inference is meaningfully slower than a discrete GPU for dense transformer models like the 9B. On a 16 GB M2 Pro MacBook Pro, Ornith-9B Q4_K_M produces tokens at a rate that is workable for interactive use — you will not be staring at a blank screen — but a batch coding task that takes seconds on a cloud A100 may take minutes locally.

The 35B MoE is a different story. Because only ~3B parameters are active per token, the per-token compute is much lower than a dense 35B would require. On a 32 GB M3 Max, the 35B MoE can be surprisingly responsive for its apparent size. It is the model we recommend for 32 GB+ Macs. Compare the two models head-to-head at 9B vs 35B.

What the Mac path does offer: complete privacy. No request leaves your machine. No API keys. No rate limits. No costs beyond electricity. For a developer working with proprietary codebases, that is a meaningful advantage over cloud inference.

Check the benchmarks page for task-level comparisons if you want to calibrate expectations before committing to a download.

Choosing Between mlx-lm and LM Studio

Both use the same underlying GGUF files and Metal backend. The performance difference between them is negligible; choose based on your workflow. If you plan to call Ornith programmatically from a script, mlx-lm is cleaner. If you want to explore the model’s capabilities quickly, LM Studio gets you there faster.

FAQ

Can I run Ornith MLX on an M1 MacBook Air with 8 GB?

Technically yes, at Q2 or Q3 quantization. In practice the output quality at those quantization levels is noticeably degraded, and generation speed on an 8 GB M1 Air is slow enough to be frustrating. The try page gives you cloud-hosted Ornith without the hardware constraint.

Does Ornith 1.0 support the full 262K context on Mac?

The model supports 262K context, but loading a full-length context on a memory-constrained Mac will exhaust RAM quickly. For most coding and document tasks, 32K–64K is sufficient and much more memory-efficient. Pass --context-length 32768 with mlx-lm to cap it.

Which Ornith variant should I run on a 32 GB Mac?

Ornith-1.0-35B Q5_K_M (~25 GB) is the recommended choice. It fits comfortably with room for the OS, and the MoE architecture makes it relatively efficient on Apple Silicon. The 9B Q4 also fits and is faster; if you need quick iterations rather than maximum quality, the 9B is a reasonable alternative. See the model alternatives page for more guidance.

Is mlx-lm faster than Ollama on Apple Silicon?

Both use Metal acceleration under the hood. Reported differences in practice are small and depend on the specific model, quantization, and Mac chip. For the Ornith GGUF models, either tool will give you comparable throughput. The choice comes down to whether you want a Python library (mlx-lm) or a background service with an HTTP API (Ollama).

Whether you choose the mlx-lm terminal path, LM Studio’s GUI, or Ollama’s background service, running Ornith MLX locally on Apple Silicon puts a capable self-scaffolding coding model entirely under your control. The 16 GB / 9B and 32 GB / 35B pairings are the two setups most worth targeting — everything else is either too constrained or more hardware than most developers own.