Event Actions
Defending Generative Models through Privacy Auditing and Content Detection
Abstract:
Generative models for images, video, and text have achieved unprecedented capabilities, driven not only by advances in model architectures but also by training on massive, high-quality datasets. While such data and the resulting generative power underpin impressive performance, they also introduce significant privacy and security risks. This dissertation focuses on these risks from both the training data and content-generation perspectives and develops auditing and defense mechanisms across modalities.
For training data privacy, we design membership inference attacks (MIAs) tailored to diffusion models. Our white-box, gradient-based MIAs serve as auditing tools that allow data owners to verify whether their content was used without consent. We further extend auditing to a practical black-box setting through score-based MIAs that rely only on model outputs.
We then examine risks arising from powerful generative capabilities. For video generation models, we develop VGMShield, a framework for detecting AI-generated videos, attributing them to source models, and preventing realistic forgeries. We additionally conduct a systematic study of unsafe video generation, build the first dataset of unsafe VGM outputs, and propose a latent-variable defense that blocks harmful content efficiently. Finally, we introduce Paladin, a trigger–tag paradigm that embeds robust markers into AI-generated phishing emails, enabling scalable detection without affecting normal model utility.
Committee:
- Wajih Ul Hassan, Committee Chair (CS/SEAS, SDS/UVA)
- Tianhao Wang, Advisor (CS/SEAS/UVA)
- Jundong Li (ECE, CS/SEAS/UVA)
- Miaomiao Zhang (ECE,CS/SEAS/UVA)
- Xiaojing Liao (University of Illinois Urbana-Champaign)