Permissions in RAG: who gets to see what
The assistant answered with someone's salary. The document was in the index, and the index knew nothing about permissions.
A retrieval-augmented generation system that indexes entire document repositories works perfectly for demonstrations. It breaks silently in production the moment someone outside the steering committee or approved users starts using it in real work contexts where access control matters.
The assistant returned someone's personal salary information from an internal HR document. The document was definitely present in the vector index. The index possessed absolutely no knowledge about permissions, access control, or who was making the query or what they were authorised to access.
Applying permission filters after retrieval completes, filtering the answer text itself, is a false solution that does not work in practice. The critical issue is that the document reached the model's context window at all. Once inside the prompt context, the model can respond to anything about it.
The permission filter must apply during retrieval itself, using the authenticated identity of whoever is making the request. If that user lacks access permission to a document, that document must never reach the model in the first place, absolutely.
This architectural requirement adds significant operational cost and complexity: the system must keep access control lists constantly synchronised with the authoritative source system where permission truth lives. This synchronisation work is mundane and completely non-optional for security.
This pattern repeats across every single client with sensitive data or confidential information that needs protection. Build the permission layer into the architecture from day one, before you index anything or build retrieval logic whatsoever.