Google adds Tool Retrieval to the Gemini API so agents load tool schemas only when needed
A preview feature in the Interactions API defers parameter schemas until the model asks for them. It mirrors tool search options Anthropic and OpenAI already document.
Google on November 19 added Tool Retrieval to the Gemini API as a preview feature in the Interactions API. It lets an agent hold its tool library in reserve and pull in a tool's full parameter schema only when a task calls for it, instead of sending every definition with every request. The announcement is credited to Lucia Loher, a product manager on the Gemini API, and Nora Kassner, a research scientist at Google DeepMind.
The problem it addresses is familiar to anyone who has wired an agent to several MCP servers. Every tool declaration costs input tokens on every turn, and Google argues that a long list of candidates also makes tool selection less accurate. Google says the feature is designed to cut input tokens and latency and to keep selection accurate as tool libraries grow. Its post gives no measured figures for either claim.
How it works
A developer adds a `tool_search` entry to the request and marks the tools to hold back with `defer_loading: true`. The model still sees the name and description of every deferred tool, so it knows what exists, but the parameter schemas stay out of context. When it needs one, it asks for it by name, the API loads the schema, and the model emits the function call, all inside the same API call. Google says each of those steps appears in the interaction's `steps` record, which should help when debugging why a model picked, or missed, a tool.
- Server-side by default. Google runs the search, so there is no index to build or host. Google says developers can keep their most-used tools loaded up front and defer the long tail.
- MCP support. Setting `defer_loading` on a remote `mcp_server` tool defers everything that server exposes, so a large server can be connected without listing its tools one by one.
- Client-side option. Teams with their own tool registry, embedding index or per-user permission filter can declare the search with `"execution": "client"` and run the lookup themselves, at the cost of one extra round trip per search. In that mode tools do not need to be declared up front, so new ones can appear mid-conversation.
- Models. Google says it works with Gemini 3.6 and later, including all Gemini 4 models, in the Interactions API.
What it costs
There is no separate charge. Tokens are billed at the normal rate for the model, but turning on `tool_search` adds a small fixed block of input tokens, and every schema the model loads is billed as input for the rest of the conversation, just as if it had been declared up front. Google's own guidance is that savings come when the schemas you never load outweigh the ones you do, and it tells developers to measure that against their own traffic. As an illustration, an agent with eight tools that uses most of them on every turn may see no benefit. One with two hundred tools that touches three per task is the intended case.
Google also says the feature is built to work with prefix caching. Deferred schemas never sit at the start of the prompt, and loaded schemas are appended later in the conversation, so the cached prefix should stay stable from turn to turn. That matters because a feature that saved schema tokens but broke caching could easily cost more overall.
Google is catching up to a shared pattern
This is not a new idea in the industry. Anthropic's documentation already describes a tool search tool for Claude that uses the same `defer_loading` flag, runs server-side and allows a custom client-side implementation. OpenAI's Responses API guide documents a `tool_search` tool, also driven by `defer_loading`, for GPT-5.4 and later models, and it likewise appends newly discovered tools at the end of the context to preserve the cache. Anthropic's docs put a number on the problem: they say a typical multi-server setup can spend about 55,000 tokens on definitions before doing any work, and that tool search typically cuts this by more than 85%. That is Anthropic's figure for its own implementation, not a measurement of Gemini.
The practical result is that the three largest model APIs now handle large tool libraries in broadly the same way, even with the same parameter name. Agent frameworks that already defer tools for Claude or GPT should need less provider-specific work to do the same on Gemini, though request shapes, search behavior and supported surfaces still differ. Gemini's version, for example, is still a preview, and Google describes it as part of the Interactions API.
What changes for people building agents
Once schemas are deferred, a tool's name and one-line description are all the model sees when choosing what to load. Those fields become the routing layer, so vague names or overlapping descriptions will show up as missed or wrong tool calls rather than as obvious errors. Treat them with the same care as a system prompt, keep the handful of tools you call constantly undeferred, and compare token counts and task success with the feature on and off before relying on it in production. Independent evaluation of Gemini's retrieval accuracy is not yet available.
Why this matters
- Agents connected to large tool libraries or MCP servers pay for every tool schema on every turn; deferring them can lower input token costs if most tools go unused per task.
- Gemini now follows the same defer_loading pattern Anthropic and OpenAI already document, which should make multi-provider agent code easier to keep consistent.
Key takeaways
- Tool Retrieval is in preview in the Gemini Interactions API and works with Gemini 3.6 and later, including all Gemini 4 models.
- There is no extra fee, but tool_search adds a fixed block of input tokens and each loaded schema is billed for the rest of the conversation.
- Google has published no measured savings; it advises developers to test the feature against their own traffic.
- With schemas deferred, tool names and descriptions decide what the model loads, so they need to be written carefully.
Sources
- GooglePrimaryIntroducing Tool Retrieval in the Gemini APIblog.google
- Anthropic (Claude Platform Docs)PrimaryTool search toolplatform.claude.com
- OpenAI (Developers docs)PrimaryTool searchdevelopers.openai.com
- tool calling
- coding agents
- api pricing
- prompt caching
- mcp
- context windows
- gemini
- Google DeepMind
- Anthropic
- OpenAI
- Gemini 4
- Gemini 3.6