Learning by Exploration with Information Advantage
Abstract:
Learning is a predominant theme for any intelligent system, humans, or machines. Moving beyond the classical paradigm of learning from past experiences, e.g., offline supervised learning from given labels, an intelligent learner needs to actively, or even proactively, collect human feedback to learn from the unknowns, i.e., learning through exploration. The growing needs of interactive intelligent systems in practice, such as recommender systems, smart homes, conversational systems and self-driving cars, urge the research in the learning by exploration paradigm. My dissertation focuses on this key ingredient in interactive online learning problems, with the goal of designing algorithms that efficiently interact with and learn from human feedback in real-world environments. There are several challenges in realizing this goal including huge exploration space, missing information and privacy and security concerns. The key insight to overcome the challenges is that the information advantage, i.e., leveraging additional information regarding the structure of the problem such as social connectivity and context structure, offers a unique opportunity to develop advanced intelligent systems. Based on this insight, I developed advanced interactive online learning systems from three perspectives: 1) sample efficient online learning with explicit structural information; 2) efficient exploration in implicitly structured environments; and 3) privacy and security in online learning. By harnessing the power of information in exploration, the research has been applied to high-impact real-world problems such as interactive recommendation, search result ranking, and social influence maximization.
Committee:
- David Evans, Committee Chair (CS/SEAS/UVA)
- Hongning Wang, Advisor (CS/SEAS/UVA)
- Haifeng Xu (CS/SEAS/UVA)
- Cong Shen (ECE/SEAS/UVA)
- Mengdi Wang (EE/Princeton University)