Machine learning models are susceptible to a variety of attacks that compromise the privacy of underlying data via various aspects of the training process. One such inference risk is distribution inference, where the adversary seeks to infer properties of the underlying training distribution. The goal of this dissertation is to make advances in distribution inference that bring it to a level of understanding comparable to that we have for dataset notions of privacy like membership inference. In our preliminary work, we formalized distribution inference as a cryptographic game and developed metrics for measuring leakage across different kinds of properties. We further improve the state-of-the-art on distribution inference attacks in our follow-up work and show that this risk leads to non-trivial leakage even in non-contrived threat scenarios, like hard-label API access and model architecture mismatch. Going forward, we plan to quantify inference risk under various training configurations and use insights from this work to augment the development of more effective and efficient black-box attacks. Then, we plan to study the granularity and variety of properties that can be inferred successfully, exploring links with causal learning. Connections and observations from these studies will be used to develop theoretical connections with other useful notions of machine learning, ultimately leading to the development of principled defenses.
- Tianhao Wang, Commitee Chair, (CS/SEAS/UVA)
- David Evans, Advisor, (CS/SEAS/UVA)
- Sheng Li, (School of Data Science/UVA)
- Mohammad Mahmoody, (CS/SEAS/UVA)
- Giuseppe Ateniese, (CS/CSYE, George Mason University)