ABSTRACT

Decision making tasks like contextual bandit and reinforcement learning often need to be conducted under data distribution shifts. For example, we may need to utilize off-policy data to evaluate a target policy and/or learn an optimal policy utilizing logged data. We may also need to deal with sim2real problem when there is a dynamics shift between training and testing environments. In this talk, I am going to introduce two threads of my work in the domain of robust decision making under distribution shifts. First, I will introduce distributionally robust off-policy evaluation and learning techniques that feature a more conservative reward estimation component. This pessimistic reward estimation will benefit both off-policy evaluation and learning under various distribution shifts. These are based on publications in AISTAT2023 and TMLR2024. Second, I will introduce off-dynamics reinforcement learning via domain adaptation and reward augmented imitation. We recognize the previous methods in off-dynamics reinforcement learning can suffer performance degradation and propose an imitation from observation approach to mitigate it. This is based on a work accepted in NeurIPS 2024. Finally, I will cover future work and new developments in uncertainty-aware approaches to safe decision making problems.

BIOGRAPHY

Anqi (Angie) Liu is an assistant professor in the Department of Computer Science at the Whiting School of Engineering, Johns Hopkins University. She is broadly interested in developing principled machine learning algorithms for building more reliable, trustworthy, and human-compatible AI systems in the real world. Her research focuses on enabling the machine learning algorithms to be robust to the changing data and environments, to provide accurate and honest uncertainty estimates, and to consider human preferences and values in AI interactions. She obtained her PhD in computer science from the University of Illinois Chicago. Prior to joining Johns Hopkins, she completed her postdoctoral research in the Department of Computing + Mathematical Sciences at the California Institute of Technology. She is a recipient of the Amazon Research Award and her research has been supported by a JHU discovery award, NIH, Moore Foundation, and ONR.