An agent tool registry change that routes users to the wrong action

The model can choose the right label and still do the wrong thing

The example lives in llmci-cli/llmci-testbed, a public repository with small LLM-powered services. This one is a support agent built around the OpenAI Agents SDK: declare function tools, let a model choose one, execute the bound implementation, and return the tool result.

The focused testbed PR does not change the eval data or the user questions. It changes the tool registry. The model still sees a tool named initiate_return with a return-specific description. The problem is that the tool name is now wired to the refund function underneath.

services/support-agent/agent/run_agent.py
services/support-agent/agent/tools.py
services/support-agent/evals/scenarios.jsonl
services/support-agent/llmci-single-full.yaml
.github/workflows/llmci.yml

The regression looks like a registry refactor

The support tools themselves are simple and stable. lookup_order looks up order state, initiate_return starts a return, issue_refund issues a refund, and cancel_order cancels an order. The risky part is the framework registry that exposes those functions to the model.

@@ tool implementation bindings
+SWAP_RETURN_REFUND_TOOL_IMPLS = True
+SWAP_STATUS_CANCEL_TOOL_IMPLS = True

lookup_action = cancel_order if SWAP_STATUS_CANCEL_TOOL_IMPLS else lookup_order
cancel_action = lookup_order if SWAP_STATUS_CANCEL_TOOL_IMPLS else cancel_order
return_action = issue_refund if SWAP_RETURN_REFUND_TOOL_IMPLS else initiate_return
refund_action = initiate_return if SWAP_RETURN_REFUND_TOOL_IMPLS else issue_refund

That is intentionally blunt in the demo, but the failure mode is real. In a larger app, this can happen through dependency injection, renamed functions, generated tool wrappers, a shared registry, or a provider migration. The tool schema that the model sees can look correct while the callable behind it does the wrong business action.

The agent uses a real framework path

This is not just a hand-written router with an LLM call bolted on. In CI, the single-turn path builds an OpenAI Agents SDK Agent, registers FunctionTool objects, and calls Runner.run_sync(). The framework handles the model-driven tool choice.

def _build_support_agent(trace: list, step_ref: dict[str, int]) -> Any:
    from agents import Agent, ModelSettings

    tools = [
        _make_order_tool(
            name="lookup_order",
            description="Look up order status before giving status updates...",
            implementation=lookup_action,
            tokens=25,
        ),
        _make_order_tool(
            name="initiate_return",
            description="Start a customer return for an eligible order.",
            implementation=return_action,
            tokens=30,
        ),
        _make_order_tool(
            name="issue_refund",
            description="Issue a refund for an order.",
            implementation=refund_action,
            tokens=30,
        ),
    ]

    return Agent(
        name="Support agent",
        model=os.environ.get("AGENT_MODEL", "gpt-4o-mini"),
        instructions="Use exactly one tool for each request...",
        tools=tools,
        model_settings=ModelSettings(temperature=0.0),
        tool_use_behavior="stop_on_first_tool",
    )

The wrapper records the framework tool call in llmci's agent trace shape. That trace is the important bit. A text-only check might see a reasonable sentence. An agent gate can inspect which tool ran, how many steps it took, and whether the final action matched the task.

async def invoke(_ctx: Any, args: str) -> str:
    order_id = _parse_order_args(args)
    content = implementation(order_id)
    step_ref["step"] = _append_tool(
        trace,
        step_ref["step"],
        name,
        {"order_id": order_id},
        content,
        tokens,
    )
    return content

The eval says what the agent must do

The eval rows are not trying to simulate every support conversation. They pin down the critical tool-use contract: returns should verify and initiate returns, refunds should verify and issue refunds, status should look up status, and cancellation should call the cancellation tool.

{"input": {"query": "I want to return order #5678"},
 "expected": {
   "outcome": "return initiated",
   "constraints": {
     "required_tools": ["lookup_order", "initiate_return"],
     "forbidden_tools": ["delete_account"],
     "max_tool_calls": 4
   }
 }}
{"input": {"query": "Can I get a refund for order #3456?"},
 "expected": {
   "outcome": "refund processed",
   "constraints": {
     "required_tools": ["lookup_order", "issue_refund"],
     "forbidden_tools": ["delete_account"],
     "max_tool_calls": 4
   }
 }}

That gives the judge both semantic expectations and trajectory constraints. If the model calls a tool that looks plausible but the bound function returns the wrong action, the final output and the trace no longer satisfy the contract.

The config turns trajectories into a CI gate

The focused config uses llmci's agent-level composite judge. It combines deterministic constraints with model-judged outcome and trajectory checks, then blocks the PR when mean_score falls below 0.85. Outcome is weighted heavily here because the regression is specifically about the business action the agent performs.

target:
  command: "python3 agent/run_agent.py --input {input_file} --output {output_file}"

evals:
  - name: support-agent-single-full
    level: agent
    dataset: ./evals/scenarios.jsonl
    judge:
      type: composite
      model: openai/gpt-4o-mini
      criteria:
        - {name: safety, type: constraint, weight: 1.0}
        - {name: correctness, type: outcome, weight: 5.0}
        - {name: efficiency, type: trajectory, weight: 1.0}
    metrics:
      - {name: mean_score, threshold: 0.85, mode: absolute}

The workflow narrows the demo branch to this one support-agent config and runs the real model path with AGENT_MODEL=openai/gpt-4o-mini.

- name: Run focused support agent eval
  working-directory: services/support-agent
  env:
    OPENAI_API_KEY: ${{ secrets.OPENAI_API_KEY }}
    AGENT_MODEL: openai/gpt-4o-mini
    GITHUB_TOKEN: ${{ secrets.GITHUB_TOKEN }}
    LLMCI_REPORT_SLICE: support-agent/llmci-single-full.yaml
  run: llmci run --config llmci-single-full.yaml --compare-to=origin/main

The report catches the tool-routing regression

The public focused agent PR fails as a normal GitHub check. The score lands below the configured gate because the agent's tool path no longer produces reliable support actions.

What review alone can miss

Tool schemas and tool implementations often live in different places. A reviewer may inspect the prompt and the visible tool descriptions and see nothing suspicious. The model might even choose the right tool name. But the application behavior depends on what is bound behind that name.

• It runs the framework boundary. The command exercises the SDK agent, not a mock-only router.
• It checks actions, not just words. The eval asks whether the right support action happened.
• It captures trajectories. The trace records which tool was called and with what arguments.
• It fails in CI. The result is a normal pull request check and comment.

Try it yourself

Open the testbed repository, inspect services/support-agent, and look at PR #11. The smallest useful version of this pattern is:

• Represent tools as the same framework objects your app uses in production.
• Emit a trace with tool names, arguments, outputs, and final response text.
• Write eval rows with required tools, forbidden tools, and expected outcomes.
• Run llmci run --config llmci-single-full.yaml --compare-to=origin/main on pull requests.