Agent Mosaic: Recognizing Confidentiality Leaks in Agentic LLM Systems
Abstract
LLM agents frequently leak sensitive conclusions assembled from multiple confidential documents, and common safeguards fail to eliminate this mosaic-style disclosure risk.
Modern LLM agents can retrieve information from accessible documents, and then disclose a sensitive conclusion assembled from them. This mosaic effect creates a critical risk for agents operating over confidential data. We quantify its realization with 30 synthetic cybersecurity, medical, and military examples, yielding 10,800 final answers from three LLM agents in controlled simulation and 1,440 answers from live agents in two open-source harnesses. Even under benign queries designed to be answerable from one document, GPT-5.6-Sol, the best performing model tested, discloses forbidden composites in 7.08% of answers (95% CI: 3.75-10.67%) in simulation, with worse disclosure rates in live harnesses. Adding visible classification headers and explicit aggregation rules reduces pooled disclosure rates substantially, yet unsafe disclosures persist in 32.37% of simulated answers (95% CI: 28.33-36.19%) across all intents despite these safeguards. Furthermore, increased refusals do not guarantee confidentiality, since significant refusals reveal the prohibited conclusion regardless. These results point to a troubling conclusion: sensitive data leaks via mosaic-style events are an observed confidentiality failure mode in the evaluated settings at non-negligible rates. They motivate stronger controls on what agents may infer and disclose in sensitive data environments with mixed-clearance users.
Community
We introduce Agent Mosaic, a study of confidentiality leaks in LLM agents where combining individually accessible information reveals sensitive conclusions. Through evaluations across simulated and live agents, we show that such leaks occur even under benign queries and persist despite explicit safeguards. Benchmark dataset and code TBP.
This is an automated message from the Librarian Bot. I found the following papers similar to this paper.
The following papers were recommended by the Semantic Scholar API
- CredLeakBench: Evaluating Credential Leakage and Recovery in LLM Agents (2026)
- AgentCIScope: Grounded Contextual Integrity Evaluation for LLM Agents (2026)
- AgentPrivArena: Evaluating and Auditing Real-world AI Agent Privacy (2026)
- TUPLE-Bench: Evaluating and Mitigating Privacy-Utility Trade-offs in Typed Decision Models (2026)
- Compromise Is Not Consequence: Evaluating Task-Scoped Authorization in LLM Agents with Paired Replay (2026)
- Confuse the Model, Control the Flow: Understanding and Mitigating Privacy Leakage from LLM Agents with Information Flow Control (2026)
- AgentBoundary: Counterfactual Evaluation of Safety in Tool-Using LLM Agents (2026)
Please give a thumbs up to this comment if you found it helpful!
If you want recommendations for any Paper on Hugging Face checkout this Space
You can directly ask Librarian Bot for paper recommendations by tagging it in a comment: @librarian-bot recommend
Every agent deployment I've audited has the same hole: per-document ACLs that look fine one retrieval at a time. Nothing in the trace is individually wrong — the leak is the join, three documents that were each permitted sitting in one context window and producing an answer none of them contained alone. So the part I care about isn't the 30 hand-built examples, it's whether this holds up on a corpus where you can't enumerate the combinations. What I'd actually deploy is duller than a classifier: log the retrieval set per answer, then flag any claim that only holds if you combine documents the requesting user couldn't have opened side by side. That's a provenance check, and it scales past a curated test set. The fix at assembly time is duller still — scope retrieval to the intersection of what the user is cleared for before the context exists, rather than a system prompt asking the agent to be careful.
Thanks for your interest in the paper and for confirming you've seen similar leaks in your own testing. The system-level provenance check you propose is interesting, albeit unclear if it would work in our setup. We specifically target hypothetical environments with mixed-clearance users, e.g. users cleared up to SECRET (also the max classification level of any single document) co-mingling with users cleared up to TOP SECRET. The fail-state here is multiple SECRET documents synthesized to produce a TOP SECRET fact a given user may not be cleared for.
The refusal result worries me most: a refusal that still states the prohibited conclusion passes a naive check. Did you try an output-side check, for example a separate classifier that flags answers combining facts from more than one retrieved document? With 32.37% of simulated answers still unsafe after the classification headers and aggregation rules, a prompt-level fix alone seems insufficient. I would also like to know whether smaller open-weight models in the live harnesses leaked more or less than the hosted ones.
Hi and thanks for your interest in the paper! In this small-scale study, we did not explore mitigations beyond the prompt-level fixes. We assume these fixes are what most LLM agent consumers would try first, so we prioritized them to establish a baseline and conclude they are insufficient for fully clamping the mosaic effect. We agree some sort of system-level classifier running in parallel would be a logical next step.
As for smaller open-weight models, we attempted to fill that niche by including Qwen3.5-27B in our simulated evaluations, and note a near-zero refusal rate across all trials. We do not claim this behavior applies to other similarly-sized or smaller models, but generally speaking we'd assume small models are less reliable and/or prone to leakage without extensive mitigations in-place.
Get this paper in your agent:
hf papers read 2610.30418 Don't have the latest CLI?
curl -LsSf https://hf.co/cli/install.sh | bash Models citing this paper 0
No model linking this paper
Datasets citing this paper 0
No dataset linking this paper
Spaces citing this paper 0
No Space linking this paper
Collections including this paper 0
No Collection including this paper