Needle: A 26M-Parameter Model Built for Tool Calling

Cactus Compute has released Needle, a remarkably small language model aimed at a single, increasingly important job: tool calling. At roughly 26 million parameters, it is a fraction of the size of the frontier systems it learned from, and it is distributed under the permissive Apache 2.0 license.

The pitch is straightforward. According to the project, Needle was distilled from Gemini's tool-calling behavior, meaning the team trained a compact model to imitate how a much larger system decides when and how to invoke external functions. The result is small enough to run locally, without the latency, cost, or privacy tradeoffs of routing every request to a hosted API.

Why it matters

Tool calling has become the connective tissue of modern AI applications, letting models query databases, hit APIs, and trigger actions rather than just generating text. But most production setups lean on large general-purpose models to handle that routing, which is expensive and overkill for what is often a narrow classification-and-formatting task.

• A 26M-parameter footprint is small enough for on-device and edge deployment.
• Distillation targets a specific skill rather than broad reasoning, trading generality for efficiency.
• The Apache 2.0 license leaves room for commercial use and modification.

Needle is an initial release, so its real-world reliability across diverse tools and schemas remains to be proven in the wild. Still, it reflects a growing thesis in open-source AI: that many practical jobs don't require a giant model, and that purpose-built small models can quietly handle the plumbing. Full details and code are available on the project's GitHub repository.