CFDP 2155

Efficient Counterfactual Learning from Bandit Feedback


Publication Date: December 2018

Pages: 15


What is the most statistically efficient way to do off-policy optimization with batch data from bandit feedback? For log data generated by contextual bandit algorithms, we consider offline estimators for the expected reward from a counterfactual policy. Our estimators are shown to have lowest variance in a wide class of estimators, achieving variance reduction relative to standard estimators. We then apply our estimators to improve advertisement design by a major advertisement company. Consistent with the theoretical result, our estimators allow us to improve on the existing bandit algorithm with more statistical confidence compared to a state-of-theart benchmark.

Keywords: Machine Learning, Artificial Intelligence, Bandit Algorithm, Counterfactual Prediction, Propensity Score, Semiparametric Efficiency Bound, Advertisement Design

JEL Classification Codes: C1, C5, C9