Toward Reliable Decision-making in Information Systems
Abstract:
Nowadays information systems have been increasingly used in assisting our everyday decision-makings, from recommending movies to watch, products to purchase, to even providing treatment to patients. Such systems enable us to have every kind of data we could possibly want at our fingertips. However, as such systems become more involved in our everyday lives in almost all aspects, it also brings up serious concerns on whether the systems could provide reliable service and gain users' trust. Arguably, users' confidence and trust in a system will rapidly deteriorate if the system cannot provide expected service. For example, a couple of inferior recommendations or unfair ranking results will push users away. This calls for a more reliable decision-making in information systems.
To improve the reliability of the decision-making in information systems, we focus on three specific aspects: accuracy, explainability, and fairness. First, an accurate prediction is the foundation of a reliable information system. Users' trust on the system will be hurt if they are presented with instances that are subsequently found to be inferior. Second, only returning the most relevance results to users is insufficient to help users to perceive their value. To gain users' trust, the system also needs to make it more explicit why the users should pay attention to those returned results. Third, served as an intermedia between information provider and consumer, modern information systems need to be trustful not only for the consumer, but also for the information provider. For the candidate instances, a reliable decision-making should not create discriminatory or unjust impacts when comparing across different demographics.
Based on this insight, we develop effective, transparent, and fair models for reliable information systems in this thesis: 1) efficient and effective online learning to rank with pairwise exploration; 2) a multi-task learning model for explainable recommendation systems; 3) online ranking policy with fair exposure across information providers. Our study provides a deep and thorough understanding of the importance of reliability in information systems and enhances the reliable service of the systems in three specific perspectives.
Rigorous theoretical analysis and extensive empirical evaluation validated the approaches' applicability in various contexts and applications.
Committee:
- David Evans, Committee Chair, (CS/SEAS/UVA)
- Hongning Wang, Advisor, (CS/SEAS/UVA)
- Haifeng Xu (CS/SEAS/UVA)
- Cong Shen (ECE/SEAS/UVA)
- Liangjie Hong (AI at LinkedIn Corporation)