how does enterprise RAG leak sensitive data
Where your sensitive data can leak when AI reads your documents - and how to close both gaps
Your embeddings aren't anonymous.
They reverse to names. Govern the data, not just the output.
In short
A data leak in an AI document system is the unintended release of sensitive information from your files. The real question is not access control, but whether the data is still recoverable once turned into vectors. Embeddings are not anonymization - they reverse to names and records. The leak opens in two places: indexing and query time.
On this page
Where can sensitive information escape when AI reads your documents?
Most tools that put AI on your files work the same way. The system reads your documents and stores them in a searchable form. When someone asks a question, it pulls the right pieces into the model. The technical name for this is retrieval-augmented generation, or RAG.
Two things can go wrong, and they happen at two different moments. The first is at ingestion, when the system loads files, breaks them up, and indexes them. The second is at inference, when a query pulls data back and the model answers. These are separate failures. They need separate fixes. Guarding only one is the common mistake.
Most advice frames this as a security problem to solve at the output, and output scanners do have a role. The leak itself, though, forms earlier, in how you governed the information before anything was indexed. Roughly 80 to 90% of enterprise data is unstructured [5], and that is exactly where names, account numbers, and contract terms live.
What is an ingestion-time leak?
An ingestion-time leak happens while the system prepares your files. It parses each document. It splits the text into chunks, which are small passages. It turns every chunk into an embedding, the numeric form it searches on. Three things go wrong here.
Turning text into numbers does not hide it
To search your files, the system turns them into vectors - long strings of numbers. It is tempting to assume that scrambles the data. It does not. Researchers have shown the numbers can be turned back into the original words: the Vec2Text attack rebuilt 92% of short passages verbatim, names and medical details included [6], and a 2026 follow-up does it with no training data at all [7]. Europe's data regulator puts it plainly - information counts as anonymous only if pulling it back out is highly unlikely [1], and most systems do not clear that bar.
Copies multiply
One file becomes many. It turns into extracted text, chunks, embeddings, an index, summaries, and logs. Delete the original PDF, and the data can still surface in five other places. Each copy carries the same duty as the source.
Permission gets stripped
Access rights rarely travel with the data. If nobody writes them into each chunk, the index forgets who could see what. OWASP now tracks this as its own risk category, LLM08, and calls for permission-aware vector databases. [8] So govern and permission the information before you index it, not after.
What is an inference-time leak?
An inference-time leak happens at query time. A question comes in. The system retrieves data. The model answers. A log records the exchange. Each step can expose something it should not.
Retrieval-layer oversharing
This is the most common version. The system checks access at the application layer, not at the retrieval layer where it actually selects files. So it runs exactly as built and still surfaces records the asker was never cleared to see. The fix is one line: access control has to extend to what the AI can reason over, not just what a human user can view [5].
Three more ways out
Three more paths open the same vector.
Prompt injection. A document pulled into the answer can carry hidden instructions - text the model reads as a command rather than as data - and may follow them.
Multi-tenant bleed. When customers share one system, a single query can reach across the boundary and pull back another customer's files.
Log leaks. Prompt and response logs quietly keep the question, the retrieved passages, and the identifiers - a store of personal records nobody approved.
The two leak vectors at a glance
| The two leak vectors | When it happens | What can escape | How you close it |
|---|---|---|---|
| Ingestion-time | The system loads, chunks, embeds, and indexes files | Reconstructable embeddings, multiplied copies, permission stripped from chunks | Govern and permission the data before indexing; write access rights into every chunk; keep lineage across copies |
| Inference-time | A query retrieves, the model answers, a log records it | Over-broad retrieval, injected instructions, cross-tenant bleed, identifiers in logs | Enforce access at the retrieval layer; isolate customers; trace and control what is logged |
The split is really a continuum. Storage and deletion sit in the middle. But two controls make it closable. Fix what goes in. Fix what comes out.
Which regulations put this on your desk?
Most write-ups stop at the mechanisms, but the rules are the next layer, and a compliance officer owns them. Four of them point the same way: every derived copy carries the duty of the source.
Europe: EDPB Opinion 28/2024
A model trained on personal data is not automatically anonymous. You have to show that extraction is unlikely [1]. Since embeddings can be reversed, the embedding and the index carry the original information - so they count as derived personal data, with the same duty attached.
United States: NIST AI 600-1
NIST released the Generative AI Profile in 2024. It names data privacy and information security among 12 risk areas [2]. It ties AI governance to controls you already run.
India: the DPDP Rules
India notified its Digital Personal Data Protection Rules in November 2025 [3]. Core duties phase in through 2027, so firms handling Indian personal data are already in scope.
The EU AI Act
The EU AI Act took force in 2024 [4]. Its duties follow the system and your role. Running something "internally" does not exempt it. To a compliance lead, the embedding, the index, and the log are one regulated record in different forms.
How do you close both gaps?
Closing both gaps takes three controls. One is an access discipline you enforce. The other two SageX gives you - and the strongest is architectural: it removes the exposure instead of policing it.
Lock the retrieval layer
Start where the common leak is. Permission has to be checked at the retrieval layer - before any data reaches the model - not only at the application layer, where most systems stop. Filter what a query can pull back; don't retrieve everything and then hide part of the answer [5]. This is an access discipline you hold any system to, whoever builds it.
Keep the data in your cloud
The scariest ingestion-time leak - embeddings reversing into names - only matters if the vectors can be reached. Keep them inside your own perimeter and that path closes by construction. SageX runs inside your own cloud; your data never leaves your walls [5]. You don't police this leak check-by-check; you remove it.
Trace every answer
Every answer should trace to its source - which page, which section, which word - and the system should say it has no authorized source rather than guess. SageX does this at every level: click an answer and the source snaps to the exact line. Traceability is what turns "trust the AI" into "check the AI."
Containment and traceability are what SageX gives you. The access-layer controls are the discipline you hold the system to. A system you can audit is worth more than a promise that nothing ever leaks.
Do you have to build these controls yourself?
You can. It takes months of governance, permissioning, and lineage work on top of the AI. Teams underestimate it. The model demo works long before the governance does. The other path is simpler. Govern the data layer once, and reuse it everywhere. SageX runs that layer inside your own cloud, with five live, revenue-generating deployments behind it. Containment and traceability come built in; the access controls are yours to set on top.
Encrypting the vector database does not stop the leak.
Encryption guards storage. Governance closes the leak.
References (8 sources)
[1] EDPB, "Opinion 28/2024 on certain data protection aspects related to the processing of personal data in the context of AI models," 2024. https://www.edpb.europa.eu/system/files/2024-12/edpb_opinion_202428_ai-models_en.pdf
[2] NIST, "AI Risk Management Framework: Generative AI Profile (NIST AI 600-1)," 2024. https://nvlpubs.nist.gov/nistpubs/ai/NIST.AI.600-1.pdf
[3] Government of India, "Digital Personal Data Protection Rules, 2025," 2025.
[4] European Union, "The Artificial Intelligence Act," 2024.
[5] SageX, "The hidden cost of production AI," 2026.
[6] Morris et al., "Text Embeddings Reveal (Almost) As Much As Text," EMNLP 2023, arXiv:2310.06816.
[7] Kim et al., "Zero2Text: Zero-Training Cross-Domain Inversion Attacks on Textual Embeddings," arXiv:2602.01757, 2026.
[8] OWASP, "Top 10 for LLM Applications 2025 - LLM08: Vector and Embedding Weaknesses," 2024. https://genai.owasp.org/llmrisk/llm082025-vector-and-embedding-weaknesses/