The Retrieval Augmented Generation (RAG) feature of LLM systems allows businesses to utilise their own data for generating responses.

This technique enables in-context learning without costly fine-tuning, making the use of LLMs more cost-efficient.

By leveraging RAG, companies can use the same model to process and generate responses based on new data, while being able to customise their solution and maintain data relevance.

On the contrary, without RAG, models may return incorrect knowledge as they are trained on a broader range of data, and more intensive training resources are required for fine-tuning.

Thus, RAG allows organisations to optimise the integration of LLMs while gaining various benefits such as fact-checking components, up to date data and business-specific data. Hence circumnavigating the problem of a LLM model being trained and frozen in time.

The image below shows a straight forward query being posed to gpt-4–0613:with the question For which club does Leonel Messi Play?

A caveat is included by the model with a dated answer.

Considering the image below, the RAG approach is taken, where, shown in red, a contextual reference is included in the prompt, and the model responds with the correct answer in this instance.

Below is the full code to run the example, first without the RAG approach:

‍

And the answer received is:

As of my latest update in 2021, Lionel Messi plays for Paris Saint-Germain Football Club.

‍

And then supplying the contextual reference in the prompt:

With the correct contextual answer: Leonel Messi plays for Inter Miami CF.

The challenge of course is including the correct amount of context, at the right time, at scale.

A Machine Learning pipeline approach needs to be taken when compiling the prompt in real time. While being able to measure and enhance the RAG workflow and testing elements like:

• Data Generation
• Automatic prompt creation
• Observe, inspect and optimise prompt evaluation metrics, etc.

‍