STORY · MODELLER_
Needle: 26 million parameters sufficient for function calling on mobile devices
The Cactus team has open-sourced Needle, a distilled AI model with just 26M parameters that handles function calling (tool use) on ordinary consumer devices. The model runs at 6000 tokens per second during prefill and 1200 during decode, and is built with a novel architecture (Simple Attention Networks) without FFN layers—because tool calling is retrieval-and-assembly, not complex reasoning.
WHY IT MATTERS
This demonstrates that agentic AI doesn't require massive parameter counts: with the right architecture, small models can handle tools on smartphones and wearables. The finding that FFN layers are wasteful when structured knowledge comes from input (RAG, tool use) could reshape the efficiency of entire agentic systems.
SOURCES
MACHINE-GENERATED SUMMARY This summary is written by machine from the sources below. We sort and explain — but we are a way into the field, not the final word. Check the source when something matters to you.