Retrieval-Augmented Generation (RAG): Making AI Answer from Your Own Documents
RAG combines a language model with document retrieval so AI answers are grounded in relevant, up-to-date sources. Learn the architecture and best practices.

Large language models store knowledge in their parameters, but that knowledge is limited to the training data, can become outdated, and does not include an organisation's internal documents. Retrieval-Augmented Generation (RAG) addresses this by giving the model access to an external knowledge source when answering.
The term RAG was introduced in a paper by Lewis et al. (2020), which combined parametric memory (a generative model) with non-parametric memory in the form of a vector index of documents. As a result, answers become more factual and their sources can be traced.
How RAG works
- Indexing — documents are split into chunks, converted into embedding vectors, and stored in a vector database or search engine.
- Retrieval — when a user asks a question, the question is also embedded to find the most relevant document chunks.
- Augmentation — the selected chunks are inserted into the prompt as context.
- Generation — the model composes an answer based on that context, ideally with references to the source documents.
Advantages
- Knowledge is easy to update without retraining the model — just update the document index.
- Fewer hallucinations, because answers are grounded in relevant text.
- Transparency — users can check the cited sources.
- Access control — search results can be filtered by the user's permissions.
Best practices
- Chunking strategy: chunks that are too small lose context; chunks that are too large reduce precision.
- Hybrid search: combine semantic (vector) search with keyword search (for example BM25) for technical terms, names and numbers.
- Rerank search results before sending them to the model.
- Continuous evaluation: measure retrieval relevance and answer accuracy with a set of test questions.
- Security: beware of prompt injection hidden inside documents, and never include confidential data that the user is not allowed to see.
RAG has become the standard pattern for internal knowledge assistants, customer service, regulatory search and many other document-based AI applications.
Sources & references
This article is a summary and interpretation by the Oemah Solution team of the sources below. Refer to the original sources for full details and context.


