Most retrieval-augmented systems I get called into do access control in the same place: after the search.
Embed the question. Pull the top ten most similar chunks. Loop over them, drop the ones this user isn't cleared for, send what's left to the model. It reads as obviously correct — the user never sees a restricted document, so the permission boundary holds.
It doesn't hold. It leaks in three ways, and only one of them is obvious.
The obvious leak: metadata
Filtering after retrieval means the restricted chunks were retrieved. They existed, briefly, inside your request handler — and they tend to leave fingerprints.
The citation count that says "found 8 relevant passages" before you drop six. The trace span with document IDs in it. The token count you log for billing. The debug endpoint someone added during the pilot and nobody removed. Each of those is a surface where the filtered items still show up, and each one tells a user something about documents they aren't cleared to know exist.
Most teams find and plug these individually. That's the problem — it's a whack-a-mole where every new observability feature reopens the hole.
The subtle leak: top-K starvation
This is the one that gets missed, and it's worse.
Suppose you retrieve the top ten chunks by similarity and then filter. A user asks a question whose best answers are all in restricted documents. The ten nearest chunks come back, all ten get filtered out, and the user receives "I don't have information about that."
Now the same question from a user who is cleared returns a confident, well-cited answer.
The difference between those two responses is itself information. Run a handful of questions and the shape of what the corpus contains becomes inferable — not the contents, but the existence and approximate subject matter of material you're not cleared for. In a company where restricted documents are things like acquisition-target-diligence.md or performance-review-process-changes.md, the existence signal is often the sensitive part.
There's a quieter version too. Say six of the ten are restricted. The cleared user gets an answer grounded in the six best passages; the uncleared user gets one grounded in whatever ranked seventh through tenth. Both get an answer. One of them is quietly worse, and neither user can tell. That's a correctness bug wearing a security costume.
The fix is a one-line change of order
Permission shouldn't be a filter applied to results. It should be a precondition of candidacy — the restricted chunks are never in the pool that gets scored.
Here's the actual implementation from the demo, which is public:
// src/Api/Features/Rag/InMemoryVectorStore.cs
public IReadOnlyList<RetrievedChunk> Search(
float[] queryEmbedding,
IEnumerable<string> userRoles,
int topK,
float minScore)
{
var roles = userRoles.ToArray();
List<PolicyChunk> snapshot;
lock (_gate)
{
snapshot = _chunks.Where(c => c.IsVisibleTo(roles)).ToList();
}
return snapshot
.Select(c => new
{
Chunk = c,
Score = VectorMath.CosineSimilarity(queryEmbedding, c.Embedding)
})
.Where(x => x.Score >= minScore)
.OrderByDescending(x => x.Score)
.Take(topK)
// ...
}The whole argument is the position of one Where. IsVisibleTo runs while building snapshot. CosineSimilarity never sees a chunk the caller isn't cleared for. topK is applied to an already-authorised set, so starvation degrades into "fewer results" rather than "different results" — and there is no filtered-out set to leak, because it was never assembled.
The visibility rule itself is deliberately boring:
// src/Api/Features/Rag/PolicyChunk.cs
public bool IsVisibleTo(IEnumerable<string> userRoles) =>
AllowedRoles.Any(ar =>
userRoles.Any(ur => string.Equals(ar, ur, StringComparison.OrdinalIgnoreCase)));Boring is the point. This is the function that decides whether a permission boundary holds, so it should be short enough to read in one breath and obvious enough to be wrong loudly rather than quietly.
The caller passes roles into the search rather than around it:
// src/Api/Features/Ask/AskQuestionHandler.cs
var queryVec = await embeddings.EmbedAsync(question, ct);
var hits = store.Search(queryVec, roleList, opts.TopK, opts.MinScore);There is no overload of Search that omits roles. You cannot call this API and forget to pass permissions, because the compiler won't let you. That's worth more than a code review comment.
Prove it with a test, not a paragraph
Security properties that live only in prose die during a refactor. This one has a test:
// tests/Contoso.PolicyAssistant.Api.Tests/RagPipelineTests.cs
[Fact]
public void Employee_retrieval_excludes_supervisor_chunk()
{
// ... store contains a leave policy (Employee, Supervisor)
// and a safety escalation doc (Supervisor, Admin)
var q = LexicalEmbeddingClient.Embed("How many leave days do employees get?");
var hits = store.Search(q, ["Employee"], topK: 4, minScore: 0.05f);
Assert.DoesNotContain(hits, h => h.Chunk.FileName == "safety-escalate.md");
Assert.Contains(hits, h => h.Chunk.FileName == "leave-policy.md");
}Note it asserts both directions. The negative assertion catches a leak. The positive one catches the equally bad failure where somebody "fixes" security by returning nothing useful to anyone.
What changes at scale
The demo holds chunks in a List<PolicyChunk> and scores them in-process. That is fine into the tens of thousands of chunks and dishonest to pretend beyond it.
At real corpus sizes the same principle applies with the enforcement point moved into the store. With pgvector, you push the predicate into the query rather than filtering the result set:
SELECT id, text, embedding <=> $1 AS distance
FROM chunks
WHERE allowed_roles && $2 -- array overlap, evaluated first
ORDER BY embedding <=> $1
LIMIT $3;The WHERE narrows the candidate set before the ANN index ranks it. With a partial index per role group, or row-level security, the database enforces it rather than your application layer remembering to.
The trap at scale is that approximate nearest-neighbour indexes and filters interact badly: some HNSW implementations apply filters after traversing the graph, which reintroduces exactly the starvation problem you were trying to remove — now with worse recall and no obvious symptom. If you're using pgvector, understand whether your query plan is doing a filtered index scan or a post-filter, because the two look identical from the outside and only one of them is correct.
Multi-tenant systems often skip all of this by giving each tenant a separate collection or index. That's the strongest boundary available and it's usually the right call when tenants are organisations. It doesn't help with role-based access within a tenant, which is where most of this difficulty actually lives.
On the demo's retrieval provider
The public instance at policy.compcodesolutions.com runs a lexical embedding provider — hashed bag-of-words, computed locally — rather than a hosted embedding model. That's deliberate: it costs nothing to leave running, and it keeps the repository free of anything that needs a key to work.
The access-control path is identical either way. Search doesn't know or care where the vector came from; swapping the provider changes retrieval quality and nothing about who can see what. That separation is worth designing for on purpose — if your permission logic is entangled with your embedding provider, you have two problems.
Sign in as Alice, Bob and Ada, ask all three the same question, and compare what comes back. The full source, tests and architecture decisions are public.
I build AI systems that reach production — retrieval over document estates that are actually messy, permissions that match a real org chart, guardrails on anything that writes, and evaluation so answers can be measured rather than assumed. If you're stuck between a working demo and a security review, that's the conversation I have most often.