Ternary-Bonsai-2 packs a 27B model into 2-bit form

The team at prism-ml has published Ternary-Bonsai-2-27B, a 27-billion-parameter language model distributed in GGUF format and quantized down to roughly two bits per weight. The release, hosted on Hugging Face, pairs aggressive ternary quantization with a hybrid attention design and is aimed squarely at running large models on everyday hardware.

The pitch here is memory. A 27B model at full or even 8-bit precision is a heavy lift for a single consumer GPU or a laptop, but ternary weights slash the footprint dramatically. That opens the door to fitting a mid-to-large model on machines that would otherwise be limited to something in the 7B–13B range.

Why it matters

Quantization is where much of the practical work in open models now happens, and the GGUF ecosystem has made local deployment routine. Ternary approaches push that trend further by trading some numerical precision for a much smaller model that can live in constrained memory.

• Parameters: roughly 27B, in the 13B–34B class
• Precision: ternary, about 2 bits per weight
• Architecture: dense model with hybrid attention
• Targets: CUDA and Metal for GPU-accelerated local inference

The cross-platform focus on both CUDA and Metal signals an intent to reach NVIDIA GPUs and Apple silicon alike, the two most common setups for hobbyists and developers running models at home. As always with heavily quantized releases, the open question is how much quality survives the compression — something worth testing against your own workloads before relying on it in production.