Probability Tools, Tricks, and Miracles

Originally the author envisioned this book as an exposition of some asymptotic methods used for developing statistical and econometric theory. However, over the years the focus shifted towards the inequalities and approximations at the core of those methods. In part, the book evolved into an attempt to explain how tricks invented to solve specific problems in one area can turn into general tools applicable in many areas; it attempts to shed light on the mystery of how anyone could come up with such clever ideas. As the book tries to explain, the story often starts from a small insight that slowly gets transformed into an imposing theory where motivating ideas lie hidden behind clever definitions. (Sometimes, plain old Calculus plus a smidgen of convexity magic are at work behind the scenes.) The main topics are: exponential inequalities for both sums of independent random variables and martingales; path methods for gaussian processes; maximal inequalities, with their extension via chaining arguments to uniform bounds for large (or infinite) index sets; symmetrization and the combinatorial methods initiated by Vapnik and Chervonenkis; and concentration inequalities. The final chapters also describe some ways to handle dependence.