Skip to content
Threadbare
Tools··11 min read

Natural Language Tool Discovery for Agents

Let your agents discover tools without paying the enumeration tax on every run.

Remember the whole CLI vs tools debate? Late in 2025 and early in 2026 we saw a lot of chatter online to the effect of:

“CLIs are more optimal. Tools cost tokens.”

Sometimes. Not always. CLIs cost tokens too. A better way to handle a large number of tools is letting the agent search for what it needs instead of shoving n tools into context. This post isn’t a PSA for some FOSS project I made. It’s about leveling up how your agents discover tools. Let’s dive in.

Token cost: measured, not assumed

The “CLIs save tokens” argument usually assumes the agent already knows the exact commands and flags. In practice, agents pay an orientation tax:

  • running --help
  • reading schemas / docs
  • guessing parameters and retrying
  • writing small scripts to transform output

MCP tooling can have the opposite failure mode: agents often need the full tool list in-context to decide what to call, so you pay an enumeration tax up front. It’s not small. Measured MCP tool schemas run roughly 280-470 tokens each, so 50 tools is 10-20K tokens before the agent does anything. It also hurts accuracy: Anthropic’s docs say tool selection degrades past 30-50 tools, and their own testing saw MCP eval accuracy jump from 49% to 74% on Opus 4 once tools were discovered on demand instead of loaded up front.

So rather than argue by vibes, I ran a small set of experiments.

Method

  • Runs: one representative session per row in the results table (not averaged)
  • Agent runners: OpenCode for gws CLI runs; GPT-6.1 Sol (gpt-6.1-sol) for MCP runs (Arcade.dev + a semantic search layer I built on top)
  • Token measurement: total session tokens per run, including each runner’s system prompt
  • Prompt: identical across runs (unless stated)
  • Freedom: the agent could use help commands, inspect schemas, and write scripts as needed
  • Caveat (the gap): at ~280-470 tokens per tool, Arcade’s 33 Gmail tool definitions are roughly 9-15K tokens. That covers most, but not all, of the 18K gap between the two MCP runs. The rest is agent behavior, runner overhead, and not-quite-identical setups (33 tools loaded vs a 117-tool index), so don’t read the whole gap as “enumeration tax”
  • Caveat (caching): tool definitions are cacheable on both Anthropic and OpenAI, so repeat sessions pay less than a cold run. These runs didn’t measure that
  • Caveat: these are representative runs, not a peer-reviewed benchmark — the shape of the results matters more than the exact numbers

Prompt used for all tests:

Find all JavaScript Weekly emails in my inbox from the
last year. I want to see how the ecosystem has changed
and where you think it might be going.

Results

ApproachTokensNotes
gws CLI24,418Agent had to discover flags + write a script (with retries)
gws CLI + gws-gmail-read skill25,099Still required a lot of discovery to find correct params
gws CLI + gws-gmail skill21,845Best CLI run: good upfront context reduced exploration
Arcade.dev MCP (33 Gmail tools loaded)32,538Expected overhead: tools list added to context
Arcade.dev • tool discovery (117 tools semantic search index)14,339Winner: discovery + execute beats enumeration

What’s happening (the scaling argument)

  • CLIs can be token-efficient when the agent already “knows the CLI.” But that knowledge usually has to come from somewhere (help output, skills, docs, examples), and you often pay for it repeatedly.
  • MCP tool lists create a fixed overhead: if the agent needs the full list to decide, your token spend grows with tool surface area.
  • Discovery changes the curve: the agent asks for capabilities, retrieves only relevant tools, and then executes.

In other words: the question isn’t “CLI vs tools.” The question is enumeration vs discovery.

Enter: natural language indexes

Tool discovery for MCP is literally one tool:

{
"name": "tools.search",
"description": "Look up tools that might satisfy the user's request",
"inputSchema": {
"type": "object",
"properties": {
"query": {
"type": "string",
"description": "A natural language search request or relevant keywords"
},
"k": {
"type": "integer",
"minimum": 1,
"maximum": 50,
"default": 10,
"description": "The number of results to return (default: 10)"
}
},
"required": ["query"]
}
}

and the secret sauce is the next three steps:

  1. The agent uses relevant keywords or natural language to describe what it wants to find
  2. The proxy converts the query into an embedding vector and performs a cosine similarity search against the tool index, ordered by similarity
  3. Tools are returned to the agent and the agent chooses one to execute via tools.execute
flowchart LR
  A["Agent"] --> W[Proxy]
  W -->|tools.search| V["Tool index<br>(vector DB)"]
  V --> W
  W -->|tools.execute| M["MCP server(s)<br>Gmail, Linear, Stripe, etc."]
  M --> W
  W --> A

The two “meta tools”

You only need to expose two tools to the agent:

  • tools.search - Find tools that might help with this request.
  • tools.execute - Run one of the tools you just found.

That’s the whole trick: the agent doesn’t need the entire universe of tools in-context; it needs a way to find them and a way to execute them. That’s literally it. The big labs landed on the same idea: Anthropic shipped a tool search tool in November 2025, and OpenAI has a tool_search of its own in the Responses API. I built my own semantic search layer well before either of them showed up, so yes, I’m biased. It’s not free, though. Discovery adds a round-trip, and if the right tool isn’t in the shortlist the agent never sees it. Anthropic’s own guidance is that it pays off once you’re past roughly 10 tools. Anyway, back to the experiments.

The CLI experiments

Below are the key details for each run, minus the terminal transcript.

Project setup

Terminal window
mkdir -p ~/token_experiment/{gws,arcade,arcade_proxy}

GWS CLI (no skill)

The initial prompt didn’t work unless I appended: use the gws cli to complete this task.

After analysis + some trial-and-error scripting 😅, the session came in at 24,418 tokens.

Example command sequence (abridged):

Terminal window
gws gmail users messages list --params \
'{"userId":"me","q":"subject:\"JavaScript Weekly\" newer_than:1y"}' \
--page-all --page-limit 30

GWS CLI + gws-gmail-read

GWS CLI has skills, so I tried a narrower “read” skill:

Terminal window
bunx skills@latest add googleworkspace/cli --skill gws-gmail-read gws-shared

This run landed at 25,099 tokens — still a lot of discovery. In hindsight, that’s on me: gws-gmail-read is the +read helper, which reads a single message by ID. It can’t list or search, so for a “find all the emails” task the agent had to pull help output anyway. Wrong skill, not a verdict on CLI skills.

GWS CLI + gws-gmail

Adding the broader Gmail skill:

Terminal window
bunx skills@latest add googleworkspace/cli --skill gws-gmail

This was the best CLI result: 21,845 tokens.

The difference wasn’t that CLIs magically saved tokens — it was that good, immediate context reduced exploration. That’s the real lesson: orientation is the cost center.

Arcade.dev MCP (33 tools)

With a typical MCP pattern, you load the tool list so the agent can decide what to call. That run clocked in at 32,538 tokens.

If you’re unfamiliar with how most MCP clients work today, the shape often looks like:

const { tools } = await mcpClient.listTools();
const result = await anthropic.beta.messages.toolRunner({
model: "claude-sonnet-5-5",
max_tokens: 1024,
messages: [{ role: "user", content: "Use the available tools to help me." }],
tools: mcpTools(tools, mcpClient),
});

This is totally reasonable when you have a small set of tools. It gets painful when you have dozens… or hundreds.

Tool discovery: why it wins

The discovery approach flips the interaction:

  • Don’t enumerate tools.
  • Give the agent a search capability.
  • Return only a shortlist.
  • Execute only what’s chosen.

In the tool discovery run (117 tools indexed), the same task landed at 14,339 tokens.

That’s the punchline: discovery scales.


Technical deep dive

Starting tool discovery from scratch can be daunting, but it decomposes cleanly:

  1. An MCP client to fetch tools from a server (e.g., Arcade.dev) via StreamableHTTPClientTransport
  2. A database with vector support (e.g., pgvector or SQLite + sqlite-vec)
  3. A way to generate embeddings (e.g., OpenAI text-embedding-3-small or an open model like qwen3-embedding)
  4. An MCP server that exposes meta tools and proxies calls:
    • tools.search
    • tools.execute

At a high level:

search → embed → vector query → shortlist → execute

Meta tool: tools.search

server.registerTool(
"tools.search",
{
description: "Look up tools that might satisfy the user's request",
annotations: { readOnlyHint: true, idempotentHint: true },
inputSchema: {
query: z.string().describe("A natural language search request"),
k: z.number().int().positive().max(50).optional().default(10),
},
},
async ({ query, k }) => {
const matches = await searchTools(query, k);
return { content: [{ type: "text", text: JSON.stringify(matches, null, 2) }] };
},
);

Meta tool: tools.execute

server.registerTool(
"tools.execute",
{
description: "Execute a tool that was found in the search results.",
annotations: { openWorldHint: true },
inputSchema: {
toolName: z.string().describe("The tool name to execute"),
input: z.record(z.string(), z.unknown()).describe("Tool arguments matching inputSchema"),
},
},
async ({ toolName, input }) => {
return await callRemoteTool(toolName, input);
},
);

Storage: tools table + vector index

Example schema (illustrative):

CREATE TABLE tools (
id INTEGER PRIMARY KEY,
name TEXT NOT NULL,
description TEXT,
input_schema TEXT NOT NULL,
content_hash TEXT NOT NULL,
embedding VECTOR(1536),
created_at TEXT DEFAULT CURRENT_TIMESTAMP NOT NULL,
updated_at TEXT DEFAULT CURRENT_TIMESTAMP NOT NULL
);
CREATE UNIQUE INDEX tools_name_idx ON tools(name);
CREATE INDEX tools_content_hash_idx ON tools(content_hash);
-- Example for pgvector:
-- CREATE INDEX tools_embedding_cosine_idx ON tools USING hnsw (embedding vector_cosine_ops);

Embeddings: one paragraph version

Embeddings translate text into vectors such that “similar meaning” tends to be “close together.” Tool discovery works because we can embed both:

  • tool descriptions + schemas
  • the agent’s search query

…and then do a nearest-neighbor search.

Searching + executing (the loop)

The agent asks for a tool by intent:

{ "query": "search for emails", "k": 5 }

The proxy returns candidates; the agent picks one and executes it through the proxy.

The key point: the agent never needed the full tool universe in-context.

When CLIs are actually the right answer

The “CLI vs tools” debate gets dumb when it turns into absolutism. There are plenty of cases where CLIs are great:

  • You have a small, stable surface area of commands.
  • You want strong sandboxing and simple auditing.
  • You want a hard boundary between the agent and the environment.
  • Your team already has strong CLI patterns and docs.

And there are plenty of cases where CLIs lose their edge:

  • You have many integrations or fast-changing toolsets.
  • You’re dealing with a long tail of capabilities (rare-but-important actions).
  • You don’t want to keep paying the orientation tax (help output, parameter discovery) across runs.

Even current arguments like, “CLIs are ideal for data-heavy flows” are breaking down with sub-agents automatically teeing large datasets to files.

Summary

What I took from these experiments:

  • CLIs aren’t inherently “token optimal.” They can be great — but the agent often pays an orientation cost to use them.
  • MCP tooling isn’t inherently “token wasteful.” The waste comes from enumerating a large tool surface area in-context.
  • Tool discovery is the scalable move: search → shortlist → execute.

The question was never CLI vs tools. It’s whether your agent has to read the whole menu before it orders.