Retrieval-Augmented Generation (RAG)

Without retrieval, a model fills gaps with plausible guesses, which is what a hallucination is. With it, the answer is grounded in a real document that can be named and checked. A support assistant can quote the current returns policy. A sales tool can quote the actual price list instead of an outdated number remembered by a rep. The same idea explains how AI search engines choose what to cite: they retrieve pages, then summarise them, so a page that cannot be found or quoted cannot be cited.

• Keep the source documents current. The answer is only as reliable as what is retrieved.
• Write clear, self-contained statements that a chunk can carry on its own, with the heading and the facts together.
• Check which sources get cited. A wrong answer is usually fixed in the document, not by arguing with the model.

A language model answers from what it learned in training, which may be out of date and knows nothing about your private files. RAG fixes that by adding a lookup step. When a question arrives, the system searches a body of documents, picks the passages most likely to help, pastes them into the prompt and asks the model to answer using only that material. The documents can be your handbook, a price list, a support archive or the open web.

• Assuming RAG removes hallucination. It reduces it, but poor retrieval or contradictory documents still produce wrong answers.
• Feeding in everything, which crowds the context window and hides the useful passage.
• Forgetting access rules, so the assistant quotes a document the asker should not see.

• The documents are split into short chunks and stored in a searchable index, usually by meaning as well as by keyword.
• A question is matched against the index, and the best few chunks are returned.
• Those chunks go into the model's context window with the question.
• The model writes an answer from them, and the system can show which chunks were used.