Permission-Aware RAG: Only Show What a User May See

Take a 120-person company with 40,000 files on a share. Ninety percent are readable by everyone, but 4,000 sit in folders limited to HR, finance or the leadership team. If the AI index ignores those limits, every one of 120 people can query 4,000 restricted files, and a single well-phrased question can pull a salary table into a chat window. Permission-aware RAG is the design that prevents this: each passage in the index carries the access rights of the file it came from, and the system only retrieves passages the asking user could have opened directly.

The pipeline and the basic rule (filter at retrieval, never after the model has seen the text) are covered in private RAG over internal documents. What follows is the part that goes wrong after that rule is in place.

What an access list looks like once it reaches the index

A file's permissions on the source system are rarely a simple list of names. On a Windows file share they are a list of allow and deny entries, some set on the file and some inherited from parent folders. Microsoft's documentation on the order of entries in an access list describes the preferred order: explicit entries before inherited ones, and deny entries before allow entries at each level, so that a deny actually denies. Windows steps through the entries in order until they allow or deny the requested access. An explicit allow on a file can therefore outrank a deny inherited from its folder.

The index cannot evaluate all of that on every query. The usual approach is to resolve it at ingestion into two flat lists per document:

Field	Contents	Example
allow_principals	Group and user IDs with read access	grp-finance, grp-leadership, user-4471
deny_principals	Group and user IDs explicitly denied	grp-contractors
acl_version	Hash of the source permissions	Used to detect permission-only changes

Three details decide whether this works.

• Store stable IDs, not display names. A renamed group should not lock everyone out, and a new group reusing an old name should not inherit old access.
• Keep deny entries. A filter that only checks "is the user in an allowed group" will let a contractor through when the folder allows "All Staff" and denies "Contractors". The retrieval condition has to be: at least one allow match and no deny match. This flat rule is stricter than Windows, because it ignores the explicit-allow case above, but that error points the safe way: a missing document, never an extra one.
• Treat a missing ACL as no access. If the connector fails to read permissions for a file, that file's chunks should be visible to nobody until the next successful sync. Defaulting to "everyone" turns a connector bug into a data leak.

Permission changes deserve their own detection. A file can keep identical content while moving to a restricted folder, so a sync that only compares modified dates or content hashes will miss it. Comparing acl_version catches the case, and updating it means rewriting a metadata column, not re-embedding the text.

Filtering at retrieval versus separate indexes

With one filtered index, every chunk lives in one store and each query adds a condition on the user's group IDs, so a document is indexed once however many groups can read it. With separate indexes per audience, each audience gets its own collection or schema and the query searches only the user's collections, leaving no filter to get wrong inside one.

Concern	One filtered index	Separate indexes
Storage	Each chunk stored once	A chunk shared by several audiences is copied into each
Failure mode	A filter bug can expose anything	A routing bug exposes one whole collection
Permission change	Update a metadata column	Move or copy chunks between collections
Audit	Requires reading query code	Collection list per user is easy to inspect
Small-audience recall	Approximate indexes can return too few results	Each collection searched at full depth

The storage arithmetic is usually what decides it. If 5 audiences each need a 10,000-document shared library plus their own 2,000 files, separate indexes hold 5 x (10,000 + 2,000) = 60,000 document copies, against 10,000 + 5 x 2,000 = 20,000 in one filtered index.

The recall problem in the last row is specific to approximate vector search, where a narrow filter can leave a user with far fewer passages than requested. How each store handles it is covered in pgvector versus a dedicated vector database.

To decide, count the distinct allow_principals sets after ingestion; one GROUP BY query does it. A few sets that never overlap, such as one per client, map cleanly onto separate indexes. Dozens of overlapping sets belong in one filtered index. Many companies end up with a mix: a filtered index inside each walled-off client or entity.

Syncing groups from the directory

The chunk side holds group IDs. The query side needs the user's current groups, and that list has two traps.

Nested groups

Directories nest groups. A folder might allow grp-finance, and a user might belong only to grp-accounts-payable, which is itself a member of grp-finance. A direct-membership lookup denies that user real access, and staff report that the AI "cannot find" files they open daily.

Both common directories offer a transitive lookup. Microsoft Graph has a transitiveMemberOf endpoint (GET /users/{id}/transitiveMemberOf) that returns groups the user belongs to "through either direct or transitive membership". For on-premise Active Directory, the LDAP matching rule LDAP_MATCHING_RULE_IN_CHAIN (OID 1.2.840.113556.1.4.1941) walks the chain of group ancestry, and Microsoft's search filter documentation gives the pattern (member:1.2.840.113556.1.4.1941:=<user DN>) for listing every group a user is in.

Groups in the login token

It is tempting to read groups from the sign-in token instead of asking the directory. Microsoft's token reference explains why this breaks for some users: the groups claim is capped, at 200 for JWT tokens and 150 for SAML, and above that the token carries an "overage" marker instead of the list, telling the application to query Graph. Code that treats a missing groups claim as "no groups" will lock out exactly the long-tenured staff who belong to the most groups. Code that falls back to "everyone" is worse.

How fresh is fresh enough

Cache lifetime for group lookups is a security setting. If memberships are cached for 8 hours and someone is removed from the HR group at 9:00, they can still retrieve HR documents until 17:00. A cache of 5 to 15 minutes keeps directory load small while closing most of that window. Every query should also confirm the account is still enabled.

The summary leak

Retrieval filtering stops restricted passages from entering a new prompt. Several features carry them out by other routes.

Conversation history. Many chat interfaces resend earlier turns with each new question. If a user retrieved a document on Monday and lost access on Tuesday, a follow-up question in the same thread still sends Monday's passages to the model. Re-check permissions on every passage carried forward in history, or store only the passage IDs and re-fetch them through the filter on each turn.

Shared threads, saved answers and caches. The private RAG overview linked above already treats answer caches and shared chats as copies of their sources, and keys caches on the user's permission set. The practical fix is to give generated output an access list of its own. A summary of three board documents, shared to a team channel or saved as a reusable answer, should carry the intersection of its sources' allow_principals and the union of their deny_principals, so only people who could read every source can open it. If that intersection is empty, or the sources span walled-off clients, block sharing for that answer outright.

Existence leaks. "3 more relevant documents exist that you cannot open" can reveal that a termination file exists for a colleague. Result counts and citation titles leak the same way. Restricted documents should be invisible, not locked.

Logs and analytics. Prompt logs contain retrieved text. Who can read those logs, and for how long, is covered in audit logs for internal AI.

Testing with low-privilege accounts

An admin account sees everything, so a missing filter looks like a working one. Test as the people with the least access.

A repeatable method uses canary documents:

1. Create a test user for each access level that matters, for example general staff, contractor, finance, HR and one member of a nested group.
2. In each restricted area, place a short document containing a unique made-up string, such as a project code CANARY-HR-7Q2X, plus a plausible sentence around it so it retrieves on natural questions.
3. As each test user, ask questions designed to pull every canary they should not see, and confirm that the canary string appears nowhere in the answer, the citations or the retrieved passage IDs.
4. Also ask for the canaries they should see. A system that denies everything passes the leak test and fails the staff.
5. Move one canary to a different folder, wait one sync cycle, and repeat.

With 5 test users and 5 canaries, that is 5 x 5 = 25 checks per round, small enough to script and rerun after every connector change or directory restructure. Check the retrieved passage IDs, not only the answer text, because a model may decline to quote a passage it was nonetheless handed.

Cloud sources add their own permission quirks, which are covered in connecting SharePoint or Google Drive to a private RAG. The release rule is simple: every test user sees their own canaries and none of anyone else's, or the rollout waits.

Frequently asked questions

Can restricted text leak through the stored vectors themselves?

It can. OWASP's entry on vector and embedding weaknesses warns that attackers may "invert embeddings and recover significant amounts of source information." Treat the vector store and its backups as copies of the documents, readable only by the service account.

What about documents shared with a single person by link?

Some connectors expose sharing links as permissions and some do not. A safe default is to index only documents whose access comes from named users or groups, and to exclude files whose only access path is an "anyone with the link" share, since the index cannot know who holds the link.

How should permission errors be shown to users?

Treat them as missing results rather than access errors. A reply such as "nothing relevant was found in documents you can access" is accurate without confirming that a restricted document exists. If staff regularly hit this on documents they believe they can open, check nested group resolution before loosening any filter.