Better orchestratrion leads to cost efficient AI Agents
Published in Artificial Intelligence.
🚨 Breaking: First Andrew Ng, now NVIDIA’s own researchers are backing the same two conclusions:
– Small Language models are better than LLMs for Agentic AI
– Better orchestratrion leads to cost efficient AI Agents
Here are two papers explaining this 👇
📌 Paper 1: “SLM is the Future of Agentic AI”
– NVIDIA argues that agents don’t need general conversational genius 100% of the time.
– Actual agentic workflows are mostly narrow tasks :
• “Format this JSON”
• “Call the weather API”
• “Check if this date is valid”
The SLM Advantage
– For these tasks, Small Language Models (SLMs <10B params) are not just "good enough", they are better
• Latency: They react instantly (critical for multi-step agents)
• Cost: 10-30x cheaper per token
• Control: Easier to fine-tune for strict protocols than a stubborn giant model
But wait...
"If we use small models, won't the agent get dumber?"
This is where the second paper changes the game
"Intelligence isn't just raw parameter count; it's about organization"
🔗 https://lnkd.in/dajzm3TV
📌 Paper 2: "ToolOrchestra"
NVIDIA introduced Orchestrator-8B
It’s not a genius at everything, but it’s a genius at management.
It acts as a "Router"
It analyzes a request and decides: "Do I need the expensive reasoning model for this? Or can I use a cheap calculator tool?"
- The Results are Wild
They tested this on the "Humanity's Last Exam" (HLE) benchmark.
• GPT-5 (Monolithic): 35.1% accuracy
• Orchestrator-8B (Router): 37.1% accuracy
The kicker?
The Orchestrator system was 2.5x more efficient and cost ~30% of the monolithic baseline
TLDR:
Building "Agentic Systems" by just wrapping a prompt around a massive API is a financial dead end
As usage scales, your margins vanish
Orchestrated approach keeps intelligence high but slashes the compute bill by ~70%
• Monolithic Agents: Expensive, slow, wasteful
• The NVIDIA Way: Orchestrate specialized SLMs.
🔗 https://lnkd.in/dJc8-Res
📌 Key Insight: Intelligence is about routing to the right tool, not being the tool