A concept not unique to RAG, query rewriting is the act of adjusting the user’s input to enhance results. In a RAG application, this can mean providing additional context beyond the query itself.

A common example:

User: How many zip codes are there in Texas?

AI: There are approximately 1,900–1,935 zip codes.

User: And how about Colorado?

Sending the last message from the user raw would result in no context for what was being asked.

For conversations with few turns and short messages, it may be enough to send the message history with context instructing the model to answer the last user message based on conversational context.

To narrow the focus, though, it would be beneficial to add a step where another model call rephrases the query.

AI Refined query: How many zip codes are in Colorado?

Moving query rewriting into a separate process allows for greater complexity. Say that this is an AI application integration within a specialized business. For a more specific question, such as "How many zip codes are in our third market?", the query rewriting process may include a step that involves retrieving the markets served and identifying which one we haven’t already addressed. This could be surfaced through a suite of read-only tools that connect to your data.

Another dimension of this includes multi-query messages:

const initialQuery = "Which state has the most zip codes? Which state has the fewest? Which state has the zip code with the highest numerical value?"

// invocation to break up sub-queries...

// resulting query
const query = [
  "Which state has the most zip codes?",
  "Which state has the fewest?",
  "Which state has the zip code with the highest numerical value?"
]

These can then be handled individually, later to be merged into a cohesive response by a process that composes the final message.