Run Ornith with Ollama: Complete 2026 Guide

July 14, 2026 · 7 min read

The easiest way to get Ornith running locally is with Ollama. In five minutes you can have Ornith 1.0 responding to prompts on your own hardware — no Python environment, no CUDA drivers to wrangle, no API key. This guide covers everything: install, pulling models, writing a Modelfile for the full ornith ollama experience, context-length settings, troubleshooting, and when to reach for vLLM instead.

Install Ollama

Download Ollama from ollama.com for macOS, Linux, or Windows. On macOS it ships as a menu-bar app; on Linux the one-liner installer also starts the background daemon:

curl -fsSL https://ollama.com/install.sh | sh

Verify the daemon is running:

ollama --version
ollama list   # empty on first install

Ollama stores models in ~/.ollama/models (Linux/macOS) or %USERPROFILE%\.ollama\models (Windows). Make sure the target drive has enough space before pulling — see the VRAM and disk requirements in the VRAM requirements guide.

Pull and Run Ornith 1.0 Models

Ornith 1.0 GGUF weights are hosted on Hugging Face and can be pulled directly through Ollama using the hf.co/ prefix. No manual download required.

Ornith-1.0-9B (recommended for most users)

ollama run hf.co/deepreinforce-ai/Ornith-1.0-9B-GGUF

This pulls the default quantization (Q4_K_M, ~6 GB) and opens an interactive chat session immediately after download. The 9B model fits in a single consumer GPU with 8 GB VRAM.

Ornith-1.0-35B (MoE — faster than it looks)

ollama run hf.co/deepreinforce-ai/Ornith-1.0-35B-GGUF

The 35B model is a Mixture-of-Experts architecture — only ~3B parameters activate per token, so generation speed is comparable to or faster than the 9B Dense model on the same hardware. The default quantization is Q5_K_M (~25 GB). You need roughly 28–30 GB of VRAM or a unified-memory Mac with 32 GB+ RAM.

For a detailed comparison between the two sizes see Ornith 9B vs 35B.

Write a Modelfile for Full 262K Context

The single most important Ollama tuning step for Ornith is setting num_ctx explicitly. Ollama’s built-in default context window is often 2048 or 4096 tokens, which is far below Ornith 1.0’s 262K-token capability. Without overriding it you silently truncate long documents and agent conversations.

Create a file named Modelfile-ornith-9b (name is arbitrary):

FROM hf.co/deepreinforce-ai/Ornith-1.0-9B-GGUF
PARAMETER num_ctx 262144
PARAMETER num_predict -1
PARAMETER temperature 1.0
SYSTEM "You are a helpful agentic coding assistant."

Build and run the custom model:

ollama create ornith-9b -f Modelfile-ornith-9b
ollama run ornith-9b

For the 35B model the Modelfile is identical except for the FROM line:

FROM hf.co/deepreinforce-ai/Ornith-1.0-35B-GGUF
PARAMETER num_ctx 262144
PARAMETER num_predict -1
PARAMETER temperature 1.0
SYSTEM "You are a helpful agentic coding assistant."

ollama create ornith-35b -f Modelfile-ornith-35b
ollama run ornith-35b

Modelfile Parameter Reference

Use Ornith via the Ollama API

Once a model is running, Ollama exposes an OpenAI-compatible endpoint on http://localhost:11434. You can query it with curl:

curl http://localhost:11434/v1/chat/completions \
  -H "Content-Type: application/json" \
  -d '{
    "model": "ornith-9b",
    "messages": [{"role": "user", "content": "Explain prefix caching in three sentences."}]
  }'

Or use the native Ollama endpoint:

ollama chat ornith-9b --message "Write a Python function that reverses a linked list."

Point any OpenAI SDK at base_url="http://localhost:11434/v1" and model="ornith-9b" to integrate with your own apps. See the how-to-run guide for full API examples.

Troubleshooting Common Ornith Ollama Issues

Out of VRAM / model won’t load

Ollama will error with not enough memory if the model doesn’t fit. Options, in order of preference:

1. Lower quantization — the Q4_K_M quant for 9B is ~6 GB, well within an 8 GB GPU. If you accidentally pulled a higher quant, specify the tag explicitly (e.g., append :Q4_K_M when Ollama supports per-tag selection, or pre-download via huggingface-cli and point FROM at the local path).
2. Reduce num_ctx — full 262K context reserves significant KV-cache VRAM. Setting num_ctx 65536 cuts KV-cache to roughly a quarter.
3. CPU offload — add PARAMETER num_gpu 0 to your Modelfile to run entirely on CPU + RAM (slow but functional on machines with 32 GB+ system RAM).

Slow generation on 35B

If Ornith-35B feels slow, confirm that GPU layers are actually being used. Run ollama ps while the model is loaded — the PROCESSOR column should show 100% GPU. If it shows mixed CPU/GPU, you need more VRAM or should lower num_ctx. Because 35B is MoE, it should outpace 9B Dense in tokens-per-second on the same hardware once fully GPU-resident.

Context silently truncated

If responses seem to ignore content from earlier in long conversations, you likely haven’t set num_ctx in your Modelfile. Ollama does not automatically inherit the model’s maximum context length from its metadata. Always set PARAMETER num_ctx 262144 explicitly as shown above.

Model outputs malformed JSON or tool calls

Ollama does not support the qwen3_xml tool-call parser that Ornith 1.0 requires for structured tool use. For agent workflows that depend on reliable JSON tool calls, use vLLM instead (see next section). Plain chat and code generation work fine in Ollama.

When to Use vLLM Instead of Ornith Ollama

Ollama is ideal for local experimentation, single-user workloads, and quick prototyping. For production or agent-heavy use, vLLM gives you capabilities Ollama can’t match:

The vLLM serve command for Ornith 1.0 9B:

vllm serve deepreinforce-ai/Ornith-1.0-9B \
  --served-model-name Ornith-1.0-9B \
  --host 0.0.0.0 \
  --port 8000 \
  --max-model-len 262144 \
  --gpu-memory-utilization 0.90 \
  --enable-prefix-caching \
  --enable-auto-tool-choice \
  --tool-call-parser qwen3_xml \
  --reasoning-parser qwen3 \
  --trust-remote-code

The --tool-call-parser qwen3_xml flag is critical for correct function-calling output. Without it, tool-call JSON is unreliable. Use vLLM any time your application depends on structured outputs, runs multiple concurrent users, or needs the prefix-caching speedup for repetitive agent loops.

For Mac users without an NVIDIA GPU, the MLX backend is a third option — see Running Ornith on Mac with MLX.

You can also compare all deployment options side-by-side on the alternatives page.

FAQ

What is the fastest way to start chatting with Ornith using Ollama?

Run ollama run hf.co/deepreinforce-ai/Ornith-1.0-9B-GGUF in a terminal. Ollama will pull the Q4_K_M quant (~6 GB) and open an interactive session. No Modelfile or configuration needed for a first test — just add a Modelfile to unlock the full 262K context afterward.

How do I set the full 262K context window in Ollama?

Create a Modelfile with PARAMETER num_ctx 262144, build it with ollama create <name> -f <Modelfile>, and run that named model. Ollama does not automatically use the model’s maximum context; the explicit override is required.

Can I use Ornith tool calling with Ollama?

Basic prompt-based tool calling works, but Ornith 1.0’s qwen3_xml parser — which governs reliable structured JSON output for function calls — is not supported by Ollama. For production agent workflows that depend on tool calls, switch to vLLM with --tool-call-parser qwen3_xml. See the how-to-run page for the full vLLM command.

Which Ornith model should I run locally with Ollama?

Start with Ornith-1.0-9B if you have 8–12 GB VRAM. Move to Ornith-1.0-35B if you have 28+ GB VRAM (or a 64 GB unified-memory Mac) — the MoE architecture makes it faster than 9B Dense once it’s fully GPU-resident, and it scores higher on most benchmarks. Check the benchmarks page and the full review to see where each model shines.

Whether you’re spinning up a quick prototype or setting up a permanent local assistant, the ornith ollama combination covers the vast majority of use cases with minimal setup. Pull the GGUF, set num_ctx 262144 in your Modelfile, and you have the full 262K-context Ornith experience running entirely on your own hardware. When you hit the ceiling — parallel requests, tool-call reliability, or the 397B scale — vLLM is the natural next step.