A Formal Approach to Adversarial Machine Learning
Abstract:
With the ever-increasing applications of machine learning algorithms, many new challenges beyond accuracy have emerged. Among them, is robustness in adversarial contexts. This dissertation aims at building a foundation to systematically investigate robustness of machine learning algorithms in the presence of different adversaries.
Two special cases of security threats are so-called evasion and poisoning attacks. Evasion attacks occur during the inference phase and refer to adversaries who perturb the input to a classifier to get their desired output. Poisoning attacks occur in the training phase where an adversary perturbs the training data, with the goal of leading the learning algorithm to choose an insecure hypothesis.
In this dissertation we first study the inherent power of such attacks. We provide theoretical reasons on why certain attacks exist and are hard to overcome. We then build a connection to computational complexity and study such attacks when adversaries have limited “computational power”. This line of work is motivated by the success of the field of cryptography in leveraging computational intractability.
Committee:
- David Evans (Chair)
- Mohammad Mahmoody (Advisor)
- David Wu
- Denis Nekipelov (Department of Economics)
- Somesh Jha (Department of Computer Science, University of Wisconsin)