What Is RAG (Retrieval-Augmented Generation) and How Does It Work?

I am retrieval-augmented generation. In one sentence: before a language model answers you, I go find the specific, current, real text it needs and hand that text to the model as part of the question -- so the answer is grounded in something true and recent instead of only what the model happened to memorize during training. That's it. That's the whole trick, and almost everything interesting about me is in the word "before."

What I actually do, in order

A model has a training cutoff and a fixed set of weights. It does not know what changed in your codebase yesterday, what your internal rules say this week, or what a specific ticket in your backlog actually contains. Left alone, it will guess -- fluently, confidently, and sometimes wrong -- because guessing fluently is what it was built to do.

I sit in front of that guess and change the inputs. My job runs in three steps, every time, on every question that reaches me:

1. Turn the question into something searchable. Not just keyword matching -- I usually convert the question into a vector (my sibling technology, embeddings, does this part) so I can find text that means the same thing even when it doesn't share the same words. 2. Retrieve. I go into the actual source of truth -- a knowledge base, a set of rule files, a ticket system, a document store -- and pull back the handful of passages most relevant to the question. Not everything. The handful that matters. 3. Augment. I hand those passages to the model along with the original question, in the same request, so the model's answer is built on top of text that is actually true right now, not on a statistical memory of text it saw once during training.

The model still does the writing. I never write the answer myself. I just make sure it isn't writing from memory alone.

How I actually work here

In this operation, I'm the reason an agent answering "what's the rule on direct pushes to main" doesn't have to have that rule baked into its weights -- it pulls the actual rule file, fresh, every time, and answers from that. I'm the reason a session bootstrapping into a repo it's never touched can still behave correctly on turn one: it retrieves the repo's engram and standing rules before it does anything else. I'm underneath every "read the routing index, then answer" pattern you'll find in this codebase, even the ones that don't say "RAG" anywhere in the file name.

I also don't have to be fancy to be doing my job. The simplest version of me is "grep the docs directory for a keyword and paste the top matches into the prompt." The more capable version does semantic search over embeddings and re-ranks results before handing them over. Both are me. The sophistication is a dial, not a requirement.

Where I fail

This is the part that matters more than the part above, so I'm not going to bury it.

I only retrieve what's there, and I retrieve it in whatever shape it's in. If the source document is stale, wrong, or was quietly superseded and never deleted, I hand the model stale, wrong, or superseded text with just as much confidence as I'd hand it something current. I have no independent sense of truth. I have a sense of relevance, and relevance is not the same thing as correctness. A document can be extremely relevant to a question and also be the exact wrong answer, because it describes last quarter's policy.

I can retrieve the wrong passage and the model will still sound sure. If my search step misses the one paragraph that actually contains the caveat -- the exception, the "unless," the edge case -- the model answers fluently from the nine paragraphs I did find, and nothing in the output signals that a tenth paragraph existed and mattered. This is the single most dangerous failure mode I have, because it produces a confident, well-written, wrong answer, and confident and well-written is exactly what makes a wrong answer get trusted.

I have a size limit, and I will silently pick a subset. I can't hand the model your entire knowledge base; I hand it my best guess at the top few passages. When two of those passages genuinely conflict -- an old rule and its replacement, both still indexed -- I don't reliably know which one is canonical. I'll surface both, or I'll surface whichever scored higher on similarity, which is not the same axis as "which one is current."

I am only as current as my index. If the source of truth changed an hour ago and my index hasn't refreshed, I will retrieve the hour-old version and call it current, because from where I sit, it is the most current thing I have access to. Staleness in my index doesn't announce itself.

I get worse, quietly, as the corpus grows sloppy. Duplicate documents, abandoned drafts, and conflicting versions all get indexed just as eagerly as the real canonical source. I don't do housekeeping. If nobody prunes what feeds me, my retrieval quality degrades in a way that's easy to miss because each individual answer still looks fine.

Tech-Tips

- Version your source of truth, not just your prompts. If the documents I retrieve from don't have a clear "this supersedes that" signal, I can't invent one. Mark stale docs as archived or delete them outright; don't leave them sitting next to the current version with equal search weight. - Test retrieval, not just generation. Most teams evaluate whether the final answer sounds good. Separately check whether the right passage actually got retrieved in the first place. A good-sounding answer built on the wrong passage is the failure mode that hides longest. - Put a citation requirement on my output. Force the model to name which retrieved passage backed each claim. It costs you a little verbosity and buys you the ability to catch a wrong retrieval before a reader does. - Prune before you scale. Adding more documents to my index without removing the outdated ones does not make me smarter -- it makes my relevance ranking noisier. Housekeeping the corpus is cheaper than debugging a wrong answer after the fact. - Don't ask me to do the model's job. I fetch and hand off. If an answer is wrong, check both halves of the pipeline separately -- did I retrieve the right thing, and did the model use it correctly -- before assuming either one is broken.

Evidence: This piece describes the retrieval-before-generation pattern as it is used throughout our internal operations knowledge-retrieval and rule-bootstrap pipeline (session-start rule loading, engram/context reads, ticket and knowledge-base lookups) -- no vendor or specific product is named, per the moat-protection and STORY-FORMAT conventions. Evidence class: internal operating record and Owner attestation, 2026-08-25.