bonsai 2 27b
An open 27B-class ternary GGUF reasoning model for local llama.cpp inference, with compact 5.95–7.21 GB packs and CUDA, Metal, or CPU support.
Screenshots
Overview
Product overview
Bonsai 2 27B — GGUF is Prism ML's open ternary transformer model for local reasoning and text generation. The model card describes a 27B-class Qwen3.8-27B hybrid-attention backbone for llama.cpp on CUDA, Metal, and CPU, under Apache-2.0.
Core features
- Ternary g128 weights with an effective 1.72 bits per weight.
- PTQ1_0 and PQ2_0 GGUF packings at about 5.95 GB and 7.21 GB.
- 262K-token context and a Qwen3.8-27B hybrid-attention architecture.
- The published 14-benchmark thinking-mode average retains 98.2% of the FP16 reference.
- Optional Q8_0 vision-tower packs for image input and an MLX companion for Apple Silicon.
- Local llama.cpp backends for CUDA, Metal, and CPU, with serving and CLI examples.
How to use
Download a pack from Hugging Face and use the Prism ML llama.cpp fork or Bonsai demo setup. Load the model, configure GPU offload, context, and sampling parameters, then run a local CLI or OpenAI-compatible server. The model card warns that stock llama.cpp cannot run these ternary files; the required low-bit kernels and Hadamard activation runtime come from the Prism ML fork.
Audience and availability
Bonsai 2 27B is for developers and researchers exploring local reasoning, coding, agentic tool calling, long-context generation, and compact on-device inference. The model files are openly available under Apache-2.0. There is no hosted Inference Provider deployment, so compatible local hardware and runtime setup are required.
