Publication Date: December 2018
What is the most statistically eﬀicient way to do oﬀ-policy optimization with batch data from bandit feedback? For log data generated by contextual bandit algorithms, we consider oﬀline estimators for the expected reward from a counterfactual policy. Our estimators are shown to have lowest variance in a wide class of estimators, achieving variance reduction relative to standard estimators. We then apply our estimators to improve advertisement design by a major advertisement company. Consistent with the theoretical result, our estimators allow us to improve on the existing bandit algorithm with more statistical conﬁdence compared to a state-of-theart benchmark.
Keywords: Machine Learning, Artificial Intelligence, Bandit Algorithm, Counterfactual Prediction, Propensity Score, Semiparametric Efficiency Bound, Advertisement Design
JEL Classification Codes: C1, C5, C9