Preprint · 2025 · Open access
The Sum Leaks More Than Its Parts: Compositional Privacy Risks and Mitigations in Multi-Agent Collaboration
arXiv (Cornell University) · Published 16 September 2025 · DOI 10.48550/arxiv.2509.14284
In brief
“The Sum Leaks More Than Its Parts: Compositional Privacy Risks and Mitigations in Multi-Agent Collaboration” is a 2025 preprint by Vaidehi Patil and 2 co-authors, published in arXiv (Cornell University). OpenAlex counts 33 citations as of 19 November 2026. It is open access: the full text is free to read at arXiv (Cornell University).
- Citations
- 33
- Most cited in
- 2026 (31)
- References
- 37
- Authors
- 3
Abstract
As large language models (LLMs) become integral to multi-agent systems, new privacy risks emerge that extend beyond memorization, direct inference, or single-turn evaluations. In particular, seemingly innocuous responses, when composed across interactions, can cumulatively enable adversaries to recover sensitive information, a phenomenon we term compositional privacy leakage. We present the first systematic study of such compositional privacy leaks and possible mitigation methods in multi-agent LLM systems. First, we develop a framework that models how auxiliary knowledge and agent interactions jointly amplify privacy risks, even when each response is benign in isolation. Next, to mitigate this, we propose and evaluate two defense strategies: (1) Theory-of-Mind defense (ToM), where defender agents infer a questioner's intent by anticipating how their outputs may be exploited by adversaries, and (2) Collaborative Consensus Defense (CoDef), where responder agents collaborate with peers who vote based on a shared aggregated state to restrict sensitive information spread. Crucially, we balance our evaluation across compositions that expose sensitive information and compositions that yield benign inferences. Our experiments quantify how these defense strategies differ in balancing the privacy-utility trade-off. We find that while chain-of-thought alone offers limited protection to leakage (~39% sensitive blocking rate), our ToM defense substantially improves sensitive query blocking (up to 97%) but can reduce benign task success. CoDef achieves the best balance, yielding the highest Balanced Outcome (79.8%), highlighting the benefit of combining explicit reasoning with defender collaboration. Together, our results expose a new class of risks in collaborative LLM deployments and provide actionable insights for designing safeguards against compositional, context-driven privacy leakage.
Cite this paper
Formatted from the catalogue record. Check names, capitals and page numbers against the paper before you submit.
APA 7
Patil, V., Stengel-Eskin, E., & Mohit, B. (2025). The Sum Leaks More Than Its Parts: Compositional Privacy Risks and Mitigations in Multi-Agent Collaboration. arXiv (Cornell University). https://doi.org/10.48550/arxiv.2509.14284
MLA 9
Patil, Vaidehi, et al. “The Sum Leaks More Than Its Parts: Compositional Privacy Risks and Mitigations in Multi-Agent Collaboration.” arXiv (Cornell University), 2025. https://doi.org/10.48550/arxiv.2509.14284.
Chicago
Patil, Vaidehi, Elias Stengel-Eskin, and Bansal, Mohit. 2025. “The Sum Leaks More Than Its Parts: Compositional Privacy Risks and Mitigations in Multi-Agent Collaboration.” arXiv (Cornell University). https://doi.org/10.48550/arxiv.2509.14284.
Harvard
Patil, V., Stengel-Eskin, E. and Mohit, B. (2025) ‘The Sum Leaks More Than Its Parts: Compositional Privacy Risks and Mitigations in Multi-Agent Collaboration’, arXiv (Cornell University). doi:10.48550/arxiv.2509.14284.
IEEE
V. Patil, E. Stengel-Eskin, and B. Mohit, “The Sum Leaks More Than Its Parts: Compositional Privacy Risks and Mitigations in Multi-Agent Collaboration,” arXiv (Cornell University), 2025, doi: 10.48550/arxiv.2509.14284.
BibTeX
@misc{patil2025leaks,
title = {{The Sum Leaks More Than Its Parts: Compositional Privacy Risks and Mitigations in Multi-Agent Collaboration}},
author = {Patil, Vaidehi and Stengel-Eskin, Elias and Mohit, Bansal,},
howpublished = {arXiv (Cornell University)},
year = {2025},
doi = {10.48550/arxiv.2509.14284},
url = {https://doi.org/10.48550/arxiv.2509.14284}
}
Download for a reference manager: BibTeX (.bib) · RIS for Zotero, Mendeley and EndNote (.ris)
Topic and keywords
Physical Sciences › Computer Science › Artificial Intelligence › Privacy-Preserving Technologies in Data
- multi-agent systems
- large language models
- privacy-utility trade-off
- chain-of-thought
- privacy risk mitigation
Work with this paper
Most-cited papers in Computer Science
- ImageNet classification with deep convolutional neural networksAlex Krizhevsky, Ilya Sutskever, Geoffrey E. Hinton · Communications of the ACM · 2017 · 114,726 citations
- Long Short-Term MemorySepp Hochreiter, Jürgen Schmidhuber · Neural Computation · 1997 · 109,097 citations
- A Mathematical Theory of CommunicationClaude E. Shannon · Bell System Technical Journal · 1948 · 91,736 citations
- Deep learningYann LeCun, Yoshua Bengio, Geoffrey E. Hinton · Nature · 2015 · 90,226 citations
- Faster R-CNN: Towards Real-Time Object Detection with Region Proposal NetworksShaoqing Ren, Kaiming He, Ross Girshick and 1 more · IEEE Transactions on Pattern Analysis and Machine Intelligence · 2016 · 74,143 citations
Questions about this paper
How do I cite this paper in APA style?
Patil, V., Stengel-Eskin, E., & Mohit, B. (2025). The Sum Leaks More Than Its Parts: Compositional Privacy Risks and Mitigations in Multi-Agent Collaboration. arXiv (Cornell University). https://doi.org/10.48550/arxiv.2509.14284
How many times has this paper been cited?
33 times, according to OpenAlex on 19 November 2026. Google Scholar and Scopus count different sets of sources, so their numbers differ.
Is this paper free to read?
Yes. It is open access at arXiv (Cornell University): sign in and open the PDF from this page.
Who wrote this paper?
Vaidehi Patil, Elias Stengel-Eskin, Bansal, Mohit.
What is the DOI of this paper?
10.48550/arxiv.2509.14284, permanently at https://doi.org/10.48550/arxiv.2509.14284. Sign in and open it from this page.
Bibliographic data from OpenAlex (CC0), last checked 19 November 2026. ResearchGuru does not host this paper; once you are signed in, the links open it at the publisher or the repository that does.