Skip to content

ToolRAG Agent

ToolRAG Agents operate by first doing RAG on the set of available tools based on the query and only show a subset of the toolset to the model to select from for a given query. This pre-filtering approach reduces the likelihood of the model choosing the wrong tools as it is already being shown a smaller set of relevant tools to pick from.

Pros:

  • Reduced likelihood of wrong tool selection.

  • Reduce number of input tokens used for model invocation.

  • Easy way to extend with new tools without incurring concerns of context windows or agent accuracy regression.

Cons:

  • Tool pre-filtering may not raise up the relevant tool for model to select from.

When to use:

This approach is effective when you have a large set of tools and not all are relevant for any single query.

This dynamic filtering is different to the Route and Solve Agents which is a static grouping defined by the agent dev at build time. This approach instead dynamically filters at runtime based on the incoming query.

Prerequisites

This lab is a Jupyter notebook. Please follow the instructions in pre-work to run the lab.

Lab

# ToolRAG Agent # ToolRAG Agent

To run the notebook from your command line in Jupyter using the active virtual environment from the pre-work, run:

jupyter notebook notebooks/ToolRAG_Agent.ipynb

The path of the notebook file above is relative to the granite-agent-workshop folder from the git clone in the pre-work.