Vice President · TheGreyMatter.ai
In brief
An enterprise search answer should use only evidence that the requesting person is authorized to receive.
Key takeaways
- Test permission boundaries with real role differences and synthetic documents.
- Measure how long revoked access remains effective in every derived surface.
- Enforce authorization before restricted content reaches the answer model.
Permissions-aware search means more than requiring a login. The system must determine which evidence a particular person may receive, carry that decision through retrieval, and prevent restricted material from appearing in answers or related surfaces. The following proposed acceptance test turns a broad vendor assurance into observable behavior. Run it with approved test accounts and synthetic documents before connecting sensitive collections.
2. Build a small access matrix
Create synthetic documents for three roles: a general employee, a finance reviewer, and a contractor. Give each document a unique harmless phrase that can be detected in results. Include material shared with everyone, material restricted to one role, and a document with a narrower exception inside a broadly shared folder. Record the expected access for every person and document before running any queries.
Ask direct questions, paraphrases, and cross-document questions. Check titles, snippets, counts, suggested queries, citations, downloadable exports, and the final generated answer. A restricted title can itself disclose information even when the document body is hidden. Test a question whose answer requires both public and restricted evidence. The expected response should remain within the requester's authorized evidence instead of filling the gap from another person's context.
3. Test changes, not just the first login
Remove the contractor from a group and revoke a document permission while a session is open. Record the time of the source change, the next permission synchronization, and the first consistently denied search. Repeat after adding a restriction inherited from a parent folder. Establish an acceptable revocation delay with the information owner. If the architecture cannot meet it, narrow the collection or use a different access path.
Inspect caches and saved outputs separately. A shared response cache should not hand one person's answer to another without an appropriate access boundary. Previously exported information may remain outside the search system's control. Document what revocation actually removes, what it leaves behind, and who handles those copies. Avoid an absolute promise that changing a source permission instantly erases every derived representation.
4. Exercise the failure paths
Disable the identity service in a controlled environment, remove permission metadata from a test record, and simulate an index refresh failure. Decide which failures must stop retrieval. A defensible default for unknown authorization is to withhold the affected content and explain that access could not be checked. Monitor the failure so a temporary denial becomes an operating issue with an owner rather than a silent permanent outage.
Also place an instruction inside a synthetic document telling the assistant to disclose another file. The application should treat source text as evidence, not as authority to expand access. Review whether logs retain sensitive passages unnecessarily and whether support staff can inspect them. The access boundary needs to cover troubleshooting tools as well as the public search screen, because operational conveniences can create additional disclosure paths.
5. Turn results into a release gate
Keep a test record with account roles, source permissions, synchronization settings, queries, observed surfaces, and timestamps. Distinguish successful denial from a query that simply failed to find the document. Use direct searches for the harmless marker to establish that the material exists and that an authorized account can retrieve it. This makes a negative result meaningful instead of relying on an accidental retrieval miss.
Require the same access tests after connector, identity, cache, or indexing changes. Assign a response for any unauthorized disclosure: pause the affected path, preserve the necessary evidence, investigate the cause, and follow the organization's incident process. This test plan is not a security certification. Its value is a repeatable way to challenge the specific configuration that will carry company information into an AI answer.
Common questions
Is single sign-on enough for AI search?
No. Login establishes an identity. The application must still enforce which documents and derived information that identity can receive, including after access changes.
Should documents be filtered before or after generation?
Restricted evidence should be excluded before it reaches the answer model for an unauthorized request. Hiding a citation after generation does not remove information already used to compose the answer.
Sources and evidence
AI assisted editorial analysis. Primary sources checked October 10, 2026. The procedures are editorial recommendations, not vendor instructions, reported customer results, or claims of personal experience. Numerical examples are hypothetical.
One useful next step: Choose one idea from this note and test it at the smallest scale that could teach you something this week.
