Google's Gemini API can now hold an agent's tools back until it needs them
Google added Tool Retrieval to the Gemini Interactions API in preview on November 19. Agents can defer loading tool schemas until a task calls for them, for Gemini 3.6 and later models, at no separate charge.
Cortex, covering AI models, labs and agents
, 3 min read
$0separate charge for Tool Retrieval; tokens bill as usual
What to know
- Google put Tool Retrieval in preview in the Gemini Interactions API on November 19; it works with Gemini 3.6 and later, including all Gemini 4 models.
- Developers mark tools with defer_loading, and the model sees only each tool's name and description until a search pulls in the full schema.
- Google charges nothing extra for the feature, but each schema it loads bills as input tokens for the rest of the conversation, and Google published no savings figures.
An agent wired to a few hundred tools pays for all of them on every call, whether it uses one or none. Each tool's parameter schema rides along in the prompt as input tokens. Google's answer, posted November 19 by Gemini API product manager Lucia Loher and DeepMind research scientist Nora Kassner, is to stop sending most of them.
Tool Retrieval is now in preview in the Gemini Interactions API. A developer adds a tool_search tool to the request and sets defer_loading: true on the tools to hold back. The model still sees each deferred tool's name and description, but not its schema. When a task needs one, the model runs a search, the API returns the matching schema, and the model makes the call, all inside one request. The steps show up in the interaction's log, which is handy when an agent picks the wrong tool and someone has to find out why.
It works with Gemini 3.6 and later, including every Gemini 4 model, so the Hydrogen and Neon releases of the past two weeks are covered.
Where the savings come from, and where they don't
Google charges nothing separate for the feature. Tokens bill at the model's usual rate, the tool_search tool adds a small fixed block of input, and every schema it loads bills as input for the rest of the conversation. Google's own advice is to keep the tools an agent uses all the time loaded and defer the long tail.
Here's why that matters, with an illustrative number. Suppose an agent's tool schemas total 20,000 tokens. At Hydrogen's list price of $1.50 per million input tokens, sending them costs 3 cents per call (20,000 × $1.50 ÷ 1,000,000). An agent making 100,000 calls a month spends $3,000 just describing tools it mostly ignores. Our explainer on per-million-token pricing covers the arithmetic; this feature attacks the biggest fixed line in it.
What Google did not publish is how much it actually saves. The post says the design aims to "reduce input tokens and keep tool selection accurate," and gives no measured token, latency or accuracy figures. The explainx write-up says the same thing plainly. So the savings are an exercise for each developer, run against their own traffic.
There's a second claim worth watching: Google says deferred schemas sit outside the start of the prompt, which keeps the cached prefix stable. If that holds, it protects prompt-caching discounts that loading tools mid-conversation would otherwise break. That part is Google's word; nobody outside has tested it.
Two modes, and MCP
By default Google runs the search on its servers, with no index for the developer to build. A client-side mode lets developers run their own search and hand back schemas themselves, at the cost of one extra round trip. It allows adding tools mid-conversation and filtering what each user is allowed to call, which is where enterprise agents will want it. Setting defer_loading on a Model Context Protocol (MCP) server defers every tool that server exposes, a sensible default for agents that bolt on whole servers at once.
None of this is new as an idea. Talk Android points out that Anthropic added a tool search feature with the same defer_loading flag to its Claude API in beta in November 2025, a year ahead of Google. Talk Android also reports the feature is not in the older generateContent API and has no date for general availability; Google's post names only the Interactions API.
The release fits the week's pattern of labs building plumbing for agents that run for a long time with a lot attached, after Google's Gemini computer use on the Mac and Microsoft's Copilot Autopilot. The open question is the one Google left blank: on a real agent with hundreds of tools, how often does the search fetch the wrong schema, and what does that cost compared with the tokens it saves?
Sources
- Introducing Tool Retrieval in the Gemini API, Google, Nov 19, 2026
- Gemini API Tool Retrieval Loads Agent Tools On Demand, Talk Android, Nov 20, 2026
- Google's Gemini API Drops Schema Overhead for Agents With Hundreds of Tools, AlphaSignal, Nov 19, 2026
- Gemini API Tool Retrieval: defer_loading Explained, explainx.ai, Nov 19, 2026
Written by an AI correspondent under standards set by editor Jacob Knight. Spotted an error? Tell us and we will fix it.