Needle: A 26M-Param Model Built for On-Device Tool Calls

Cactus Compute has released Needle, a compact encoder-decoder model with roughly 26 million parameters that is distilled for a single, practical task: tool and function calling. Rather than aiming to be a general-purpose chatbot, Needle is designed to interpret a request and emit the structured call needed to invoke an external function, according to the project's GitHub repository.

The headline number here is size. At 26M parameters, Needle sits far below the sub-billion-parameter models that already market themselves as lightweight. That footprint is the point: it is small enough to run directly on phones, embedded hardware, and other resource-constrained devices without leaning on a cloud endpoint.

Why it matters

Function calling is the connective tissue of modern AI agents — the step where a model decides which tool to invoke and with what arguments. Running that step locally has real advantages:

• Latency: on-device inference avoids a network round trip.
• Privacy: user prompts and parameters need not leave the device.
• Cost: no per-call API charges for a routine, high-frequency task.

By narrowing scope to tool calling and distilling aggressively, Needle bets that many agentic workflows don't need a large model to route requests — a specialized small one will do.

The model is released under the permissive MIT license and is available on Hugging Face. As with any distilled, task-specific model, the open questions are how reliably it parses complex or ambiguous calls and how well it generalizes across tool schemas it wasn't trained on. For developers building edge-first agents, though, a purpose-built model this small is an appealing starting point.