Ornith 1.0 VRAM Requirements: Complete Quantization Guide

July 13, 2026 · 7 min read

If you want to run Ornith locally, the first question is always the same: does it fit in my GPU? This guide covers every Ornith 1.0 VRAM scenario — from the 9B on a gaming GPU to the 397B MoE across a multi-GPU server rig — with concrete quantization choices for each tier.

Internal pages that complement this guide: model overview, live benchmarks, and step-by-step how-to-run instructions.

Why Quantization Changes Everything

A model’s raw parameter count does not determine whether it fits in your GPU — its precision and quantization level do. Ornith 1.0 ships in bf16 (full precision), FP8 (for the 397B), and GGUF quantizations ranging from Q4 to Q8. Each step down in precision cuts memory footprint by roughly half, with a corresponding but often small quality trade-off.

For most local deployments the sweet spot is Q4_K_M or Q5_K_M: you keep >95% of the original model quality while fitting comfortably in consumer hardware. GGUF quantizations for all Ornith 1.0 models are published by DeepReinforce on HuggingFace.

Ornith 1.0 Model Sizes at a Glance

The family currently has four public checkpoints, spanning dense and MoE architectures:

The MoE models (35B and 397B) activate only a small fraction of parameters per token, which is why the 35B slots into ~25 GB despite its headline parameter count. See the Ornith 1.0-35B MoE deep dive for more on that architecture.

VRAM Tier Breakdown — What Fits Where

6 GB VRAM (Entry-Level Consumer GPU)

The only Ornith 1.0 VRAM option here is Ornith-1.0-9B at Q4_K_M (~6 GB). You are at the edge of the envelope — keep context short (under ~16K tokens) to avoid overflow, and use llama.cpp with its layer-offloading flag (-ngl) to push as many layers to the GPU as possible while letting the remainder run on CPU RAM.

# llama.cpp with full GPU offload attempt (adjust -ngl down if OOM)
llama-server -hf deepreinforce-ai/Ornith-1.0-9B-GGUF --port 8000 -c 16384

12 GB VRAM (Mid-Range Consumer GPU)

At 12 GB you have comfortable headroom for Ornith-1.0-9B at Q8 (~10 GB), which is noticeably sharper than Q4 on code and long-context reasoning. You can also run the full 262K context window with vLLM if you accept some CPU spillover for the KV cache.

# Ollama one-liner — pulls the GGUF automatically
ollama run hf.co/deepreinforce-ai/Ornith-1.0-9B-GGUF

24 GB VRAM (Prosumer / A10)

This is where Ornith gets genuinely powerful. The Ornith-1.0-35B MoE at Q5_K_M (~25 GB) sits just above 24 GB, so you technically need 24 GB+ (an RTX 3090/4090 or A10G). The Q5_K_M quant is the recommended quality-to-size ratio for the 35B according to the DeepReinforce release notes.

vllm serve deepreinforce-ai/Ornith-1.0-35B --max-model-len 262144 \
  --gpu-memory-utilization 0.90 --enable-prefix-caching \
  --enable-auto-tool-choice --tool-call-parser qwen3_xml \
  --reasoning-parser qwen3 --trust-remote-code

The Ornith-1.0-31B Q4 (~20 GB) is also a strong option at this tier, leaving ~4 GB headroom for the KV cache.

48 GB VRAM (A40 / 2× 24 GB NVLink)

Ornith-1.0-35B Q5_K_M fits easily with full context. The 31B bf16 (~62 GB) does not fit in a single 48 GB card — you would need multi-GPU or accept the Q4 quant instead. This tier is ideal for agentic workloads where you need the reasoning depth of the 35B with fast token throughput. Compare the two in the 9B vs 35B comparison post.

80 GB VRAM (A100 / H100)

At 80 GB the Ornith-1.0-31B bf16 (~62 GB) runs natively with ~18 GB of headroom for KV cache. This is the production-grade inference tier for the dense 31B.

vllm serve deepreinforce-ai/Ornith-1.0-9B --max-model-len 262144 \
  --gpu-memory-utilization 0.90 --enable-prefix-caching \
  --enable-auto-tool-choice --tool-call-parser qwen3_xml \
  --reasoning-parser qwen3 --trust-remote-code

(Swap the model name for 31B or 35B as needed — the flags are identical.)

200 GB+ (Multi-GPU Server)

The Ornith-1.0-397B FP8 (~200 GB) requires at least 8× 80 GB GPUs with NVLink. The FP8 quantization is a structured hardware-native format supported by H100 SXM and H200 — not a GGUF quant. For API access without owning the hardware, see the Ornith 1.0-397B API guide.

vllm serve deepreinforce-ai/Ornith-1.0-397B \
  --quantization fp8 --tensor-parallel-size 8 \
  --max-model-len 262144 --gpu-memory-utilization 0.90 \
  --enable-prefix-caching --enable-auto-tool-choice \
  --tool-call-parser qwen3_xml --reasoning-parser qwen3 \
  --trust-remote-code

Recommended Quantization Per Model

CPU Offloading: When Your GPU Is Too Small

If you have a compatible GPU but it falls short of the required VRAM, llama.cpp’s -ngl (number of GPU layers) flag lets you split the model between GPU VRAM and system RAM. GPU layers run fast; CPU layers are slower but still functional for light workloads.

# Example: run 9B with partial GPU offload on a 4 GB GPU
llama-server -hf deepreinforce-ai/Ornith-1.0-9B-GGUF \
  --port 8000 -c 4096 --n-gpu-layers 20

Reduce --n-gpu-layers until the model loads without OOM. A higher value means more layers on GPU (faster), lower means more on CPU (slower but possible). LM Studio provides the same slider in a GUI if you prefer not to use the command line.

Keep system RAM in mind: the layers not on the GPU still need to fit in RAM, so for the 9B Q4 (~6 GB total) you need at least ~6 GB of combined GPU + CPU RAM available.

Context Length and KV Cache Overhead

All Ornith 1.0 models support a 262K context window, but the KV cache required to hold that context adds VRAM on top of the model weights. At full 262K context with the 9B model, the KV cache alone can exceed several gigabytes depending on batch size and sequence length.

Practical advice: set --max-model-len to match your actual use case. For chat with < 32K turns, --max-model-len 32768 reduces overhead substantially. Only expand toward 262K if your workload genuinely needs long documents.

FAQ

Can Ornith-1.0-9B run on a laptop GPU?

Yes — if the laptop has at least 6 GB of dedicated VRAM (e.g., RTX 4060 Mobile), the Q4_K_M quant fits. Lower the context window to 8K–16K to leave room for the KV cache. Ollama handles this automatically on supported hardware.

Does the 35B MoE use more VRAM than a 9B dense model at runtime?

For the weights themselves, yes — Q5_K_M weighs ~25 GB versus ~6 GB for the 9B Q4. But because the 35B MoE only activates ~3B parameters per token, its compute cost per token is closer to a 3B model. You pay the memory cost upfront, and get strong reasoning without proportional slowdown.

What is the minimum GPU for Ornith-1.0-31B?

The Q4 quant (~20 GB) requires a GPU with at least 20 GB VRAM, such as an RTX 3090 (24 GB) or A10G (24 GB). The bf16 (~62 GB) needs an 80 GB card or multi-GPU setup. There is no published Q8 for the 31B in the current release. See the full 1.0 review for more details on the 31B performance profile.

Can I run Ornith-1.0-397B without 8× H100s?

Not at full precision. The FP8 checkpoint alone is ~200 GB. You could theoretically offload heavily to CPU with llama.cpp, but throughput would be impractical. For most users, the API endpoint is the realistic path to 397B inference.

Choosing the right Ornith 1.0 VRAM configuration comes down to three variables: which model size you need, what quantization trade-off you accept, and whether you can use multiple GPUs. For most local use cases, the 9B Q4_K_M on 6 GB and the 35B Q5_K_M on 24 GB+ cover the full range from personal experiments to production agentic workloads. Check the alternatives page if Ornith does not fit your hardware and you need a comparable option.