Comparison of distributions; Density approach; Stein's method; Statistics and Probability
Abstract :
[en] We propose a new general version of Stein's method for univariate distributions. In particular we propose a canonical definition of the Stein operator of a probability distribution which is based on a linear difference or differential-type operator. The resulting Stein identity highlights the unifying theme behind the literature on Stein's method (both for continuous and discrete distributions). Viewing the Stein operator as an operator acting on pairs of functions, we provide an extensive toolkit for distributional comparisons. Several abstract approximation theorems are provided. Our approach is illustrated for comparison of several pairs of distributions: normal vs normal, sums of independent Rademacher vs normal, normal vs Student, and maximum of random variables vs exponential, Fŕechet and Gumbel.
Disciplines :
Mathematics
Author, co-author :
LEY, Christophe ; University of Luxembourg > Faculty of Science, Technology and Medicine (FSTM) > Department of Mathematics (DMATH) ; Ghent University Department of Applied Mathematics, Computer Science and Statistics, Gent, Belgium
Reinert, Gesine; University of Oxford, Department of Statistics, Oxford, United Kingdom
Swan, Yvik; Universite de Líege, Departement de Mathematique, Líege, Belgium
External co-authors :
yes
Language :
English
Title :
Stein's method for comparison of univariate distributions
Publication date :
2017
Journal title :
Probability Surveys
ISSN :
1549-5787
Publisher :
Institute of Mathematical Statistics
Volume :
14
Issue :
2017
Pages :
1 - 52
Peer reviewed :
Peer reviewed
Funding text :
This work has been initiated when Christophe Ley and Yvik Swan were visiting Keble College, Oxford. Substantial progress was also made during a stay at the CIRM in Luminy. Christophe Ley thanks the Fonds National de la Recherche Scientifique, Communaute fraņcaise de Belgique, for support via a Mandat de Charǵe de Recherche FNRS. Gesine Reinert was supported in part by EPSRC grant EP/K032402/1. Yvik Swan gratefully acknowledges support from the IAP Research Network P7/06 of the Belgian State (Belgian Science Policy). We thank Carine Bartholḿe for discussions which led to the application given in Section 2.6. The authors would further like to thank Oliver Johnson, Larry Goldstein, Giovanni Peccati and Christian Dobler for the many discussions about Stein's method which have helped shape part of this work. In particular, we thank Larry for his input on Section 3.6, Christian for the idea behind Section 4.6 and Oliver for the impetus behind the computations shown in Section 6.1. Finally, we thank an anonymous referee for their suggestions.
Afendras, G., Balakrishnan, N. and Papadatos, N. (2014). Orthogonal polynomials in the cumulative Ord family and its application to variance bounds, preprint arXiv:1408.1849. MR3160581
Afendras, G., Papadatos, N. and Papathanasiou, V. (2011). An extended Stein-type covariance identity for the Pearson family with applications to lower variance bounds, Bernoulli 17, 507-529. MR2787602
Arras, B., Azmoodeh, E., Poly, G. and Swan, Y. (2016). Stein's method on the second Wiener chaos: 2-Wasserstein distance, preprint arXiv:1601.03301.
Baldi, P., Rinott, Y. and Stein, C. (1989). A normal approximation for the number of local maxima of a random function on a graph, Probability, Statistics, and Mathematics, Academic Press, Boston, MA, 59-81. MR1031278
Barbour, A. D. (1990). Stein's method for diffusion approximations, Probability Theory and Related Fields 84, 297-322. MR1035659
Barbour, A. D. and Chen L. H. Y. (2005). An introduction to Stein's method, Lecture Notes Series Inst. Math. Sci. Natl. Univ. Singap. 4, Singapore University Press, Singapore. MR2205339
Barbour, A. D., Gan, H. L. and Xia, A. (2015). Stein factors for negative binomial approximation in Wasserstein distance, Bernoulli 21, 1002-1007. MR3338654
Barbour, A. D., Holst, L. and Janson, S. (1992). Poisson approximation, Oxford Studies in Probability, 2, The Clarendon Press Oxford University Press, New York, Oxford Science Publications. MR1163825
Barbour, A. D. andEagleson, G. K. (1985). Multiple comparisons and sums of dissociated random variables, Advances in Applied Probability 17, 147-162. MR0778597
Barbour, A. D., Luczak M. J. and Xia A. (2015). Multivariate approximation in total variation, preprint arXiv:1512.07400.
Bonis T. (2015). Stable measures and Stein's method: rates in the Central Limit Theorem and diffusion approximation, preprint arXiv:1506.06966.
Brown, T. C. and Xia, A. (1995). On Stein-Chen factors for Poisson approximation, Statistics & Probability Letters 23, 327-332. MR1342742
Cacoullos, T., Papadatos, N. and Papathanasiou, V. (2001). An application of a density transform and the local limit theorem, Teor. Veroyatnost. i Primenen. 46, 803-810. MR1971837
Cacoullos, T. and Papathanasiou, V. (1989). Characterizations of distributions by variance bounds, Statistics & Probability Letters 7, 351-356. MR1001133
Cacoullos, T., Papadatos, N. and Papathanasiou, V. (1998). Variance inequalities for covariance kernels and applications to central limit theorems, Theory of Probability & Its Applications 42, 149-155. MR1453340
Cacoullos, T. and Papathanasiou, V. (1995). A generalization of covariance identity and related characterizations, Mathematical Methods of Statistics 4, 106-113. MR1324694
Cacoullos, T., Papathanasiou, V. and Utev, S. A. (1994). Variational inequalities with examples and an application to the central limit theorem, Annals of Probability 22, 1607-1618. MR1303658
Chatterjee, S. (2014). A short survey of Stein's method, Proceedings of ICM 2014, to appear.
Chatterjee, S., Fulman, J. and Rollin, A. (2011). Exponential approximation by exchangeable pairs and spectral graph theory, ALEA Latin American Journal of Probability and Mathematical Statistics 8, 1-27. MR2802856
Chatterjee, S. and Meckes, E. (2008). Multivariate normal approximation using exchangeable pairs, ALEA Latin American Journal of Probability and Mathematical Statistics 4, 257-283. MR2453473
Chatterjee, S. and Shao, Q.-M. (2011). Nonnormal approximation by Stein's method of exchangeable pairs with application to the Curie-Weiss model, Annals of Applied Probability 21, 464-483. MR2807964
Chen, L. H. Y. (1975). Poisson approximation for dependent trials, Annals of Probability 3, 534-545. MR0428387
Chen, L. H. Y. (1980). An inequality for multivariate normal distribution, Technical report, MIT.
Chen, L. H. Y.,Goldstein, L. and Shao, Q.-M. (2011). Normal approximation by Stein's method, Probability and its Applications (New York), Springer, Heidelberg. MR2732624
Daly, F. (2008). Upper bounds for Stein-type operators, Electronic Journal of Probability 13, 566-587. MR2399291
Diaconis, P. and Zabell, S. (1991). Closed form summation for classical distributions: variations on a theme of de Moivre, Statistical Science 6, 284-302. MR1144242
Dobler, C. (2012). On rates of convergence and Berry-Esseen bounds for random sums of centered random variables with finite third moments, preprint arXiv:1212.5401.
Dobler, C. (2015). Stein's method of exchangeable pairs for the beta distribution and generalizations, Electronic Journal of Probability 20, 1-28. MR3418541
Dobler, C., Gaunt, R. E. and Vollmer, S. J. (2015). An iterative technique for bounding derivatives of solutions of Stein equations, preprint arXiv:1510.02623.
Eden, R. and Viquez, J. (2015). Nourdin-Peccati analysis on Wiener and Wiener-Poisson space for general distributions, Stochastic Processes and their Applications 125, 182-216. MR3274696
Ehm, W. (1991). Binomial approximation to the Poisson binomial distribution, Statistics & Probability Letters 11, 7-16. MR1093412
Eichelsbacher, P. and Lowe, M. (2010). Stein's method for dependent random variables occurring in statistical mechanics, Electronic Journal of Probability 15, 962-988. MR2659754
Eichelsbacher, P. and Martschink, B. (2014). Rates of convergence in the Blume-Emery-Griffiths model, Journal of Statistical Physics 154, 1483-1507. MR3176416
Eichelsbacher, P. and Reinert, G. (2008). Stein's method for discrete Gibbs measures, Annals of Applied Probability 18, 1588-1618. MR2434182
Fulman, J. and Goldstein, L. (2015). Stein's method and the rank distribution of random matrices over finite fields, Annals of Probability 43, 1274-1314. MR3342663
Fulman, J. and Goldstein, L. (2014). Stein's method, semicircle distribution, and reduced decompositions of the longest element in the symmetric group, preprint arXiv:1405.1088.
Gaunt, R. E. (2016). On Stein's method for products of normal random variables and zero bias couplings, Bernoulli, to appear.
Gaunt, R. E. (2014). Variance-Gamma approximation via Stein's method, Electronic Journal of Probability 19, 1-33. MR3194737
Gaunt, R. E., Mijoule, G. and Swan, Y. (2016). Stein operators for product distributions, with applications. preprint arXiv:1604.06819.
Gibbs, A. L. and Su, F. E. (2002). On choosing and bounding probability metrics, International Statistical Review / Revue Internationale de Statistique 70, 419-435.
Goldstein, L. and Reinert, G. (1997). Stein's method and the zero bias transformation with application to simple random sampling, Annals of Applied Probability 7, 935-952. MR1484792
Goldstein, L. and Reinert, G. (2005). Distributional transformations, orthogonal polynomials, and Stein characterizations, Journal of Theoretical Probability 18, 237-260. MR2132278
Goldstein, L. and Reinert, G. (2013). Stein's method for the Beta distribution and the Ṕolya-Eggenberger urn, Journal of Applied Probability 50, 1187-1205. MR3161381
Goldstein, L. and Rinott, Y. (1996). Multivariate normal approximations by Stein's method and size bias couplings, Journal of Applied Probability 33, 1-17. MR1371949
Gorham, J. and Mackey, L. (2015). Measuring sample quality with Stein's method. In: Advances in Neural Information Processing Systems (NIPS), 226-234.
Gotze, F. and Tikhomirov, A. N. (2003). Rate of convergence to the semi-circular law, Probability Theory and Related Fields 127, 228-276. MR2013983
Gotze, F. and Tikhomirov, A. N. (2006). Limit theorems for spectra of random matrices with martingale structure, Teor. Veroyatnost. i Primenen. 51, 171-192. MR2324173
Gotze, F. (1991). On the rate of convergence in the multivariate CLT, Annals of Probability 19, 724-739. MR1106283
Haagerup, U. and Thorbjørnsen, S. (2012). Asymptotic expansions for the Gaussian unitary ensemble, Infinite Dimensional Analysis, Quantum Probability and Related Topics 15, no. 01. MR2922846
Hall, W. J. and Wellner, J. A. (1979). The rate of convergence in law of the maximum of an exponential sample, Statistica Neerlandica 33, 151-154. MR0552259
Hillion, E., Johnson, O. and Yu, Y. (2014). A natural derivative on [0, n] and a binomial Poincaŕe inequality. ESAIM: Probability and Statistics 18, 703-712. MR3334010
Holmes, S. (2004). Stein's method for birth and death chains, Stein's Method: Expository Lectures and Applications, IMS Lecture Notes Monogr. Ser., 46, 45-67. MR2118602
Johnson, O. (2004). Information Theory and the Central Limit Theorem, Imperial College Press, London. MR2109042
Johnson, O. and Barron, A. (2004). Fisher information inequalities and the central limit theorem, Probability Theory and Related Fields 129, 391-404. MR2128239
Johnson, R. W. (1993). A note on variance bounds for a function of a Pearson variate, Statistics & Risk Modeling 11, 273-278. MR1257861
Klaassen, C. A. J. (1985). On an inequality of Chernoff, Annals of Probability 13, 966-974. MR0799431
Korwar, R. M. (1991). On characterizations of distributions by mean absolute deviation and variance bounds, Annals of the Institute of Statistical Mathematics 43, 287-295. MR1128869
Kusuoka, S. and Tudor, C. A. (2012). Stein's method for invariant measures of diffusions via Malliavin calculus, Stochastic Processes and Their Applications 122, 1627-1651. MR2914766
Kusuoka, S. and Tudor, C. A. (2013). Extension of the fourth moment theorem to invariant measures of diffusions, preprint arXiv:1310.3785.
Ledoux, M., Nourdin, I. and Peccati, G. (2015). Stein's method, logarithmic Sobolev and transport inequalities, Geometric and Functional Analysis 25, 256-306. MR3320893
Lefevre, C., Papathanasiou, V. and Utev, S. (2002). Generalized Pearson distributions and related characterization problems, Annals of the Institute of Statistical Mathematics 54, 731-742. MR1954042
Ley, C., Reinert, G. and Swan, Y. (2016). Distances between nested densities and a measure of the impact of the prior in Bayesian statistics, Annals of Applied Probability, to appear.
Ley, C. and Swan, Y. (2016). A general parametric Stein characterization, Statistics & Probability Letters 111, 67-71. MR3474784
Ley, C. and Swan, Y. (2013). Local Pinsker inequalities via Stein's discrete density approach, IEEE Transactions on Information Theory 59, 5584-4491. MR3096942
Ley, C. and Swan, Y. (2013). Stein's density approach and information inequalities, Electronic Communications in Probability 18, 1-14. MR3019670
Ley, C. and Swan, Y. (2016). Parametric Stein operators and variance bounds, Brazilian Journal of Probability and Statistics 30, 171-195. MR3481100
Loh, W.-L. (2004). On the characteristic function of Pearson type IV distributions, A Festschrift for Herman Rubin, Institute of Mathematical Statistics, 171-179. MR2126896
Luk, H. M. (1994). Stein's method for the Gamma distribution and related statistical applications, Ph.D. thesis, University of Southern California. MR2693204
Mackey, L. and Gorham, J. (2016). Multivariate Stein factors for a class of strongly log-concave distributions. Electronic Communications in Probability 21 paper no. 56, 14 pp.
Nourdin, I. and Peccati, G. (2009). Stein's method on Wiener chaos, Probability Theory and Related Fields 145, 75-118. MR2520122
Nourdin, I. and Peccati, G. (2012). Normal approximations with Malliavin calculus: from Stein's method to universality, Cambridge Tracts in Mathematics, Cambridge University Press. MR2962301
Nourdin, I., Peccati, G. and Reinert, G. (2009). Second order Poincaŕe inequalities and CLTs on Wiener space, Journal of Functional Analysis 257, 593-609. MR2527030
Nourdin, I., Peccati, G. and Swan, Y. (2014). Entropy and the fourth moment phenomenon, Journal of Functional Analysis 266, 3170-3207. MR3158721
Nourdin, I., Peccati, G. and Swan, Y. (2014). Integration by parts and representation of information functionals, IEEE International Symposium on Information Theory (ISIT), 2217-2221.
Novak, S. Y. (2011). Extreme Value Methods with Applications to Finance, Chapman & Hall/CRC Press, Boca Raton. MR2933280
Ord, J. K. (1967). On a system of discrete distributions, Biometrika 54, 649-656. MR0224192
Papadatos, N. and Papathanasiou, V. (1995). Distance in variation between two arbitrary distributions via the associated w-functions, Theory of Probability & Its Applications 40, 567-575. MR1402000
Papathanasiou, V. (1995). A characterization of the Pearson system of distributions and the associated orthogonal polynomials, Annals of the Institute of Statistical Mathematics 47, 171-176. MR1341214
Pekoz, E. and Rollin, A. (2011). New rates for exponential approximation and the theorems of Ŕenyi and Yaglom, Annals of Probability 39, 587-608. MR2789507
Pekoz, E. Rollin, A. and Ross, N. (2013). Degree asymptotics with rates for preferential attachment random graphs, Annals of Applied Probability 23, 1188-1218. MR3076682
Pickett, A. (2004). Rates of convergence of ?2 approximations via Stein's method, Ph.D. thesis, Lincoln College, University of Oxford.
Pike, J. and Ren, H. (2014). Stein's method and the Laplace distribution, ALEA Latin American Journal of Probability and Mathematical Statistics 11, 571-587. MR3283586
Rachev, S. T. (1991). Probability Metrics and the Stability of Stochastic Models, vol. 334, Wiley New York. MR1105086
Reinert, G. (1998). Couplings for normal approximations with Stein's method, DIMACS Ser. Discrete Math. Theoret. Comput. Sci., 41, 193-200. MR1630415
Reinert, G. and Rollin, A. (2009). Multivariate normal approximation with Stein's method of exchangeable pairs under a general linearity condition, Annals of Probability 37, 2150-2173. MR2573554
Rollin, A. (2012). On magic factors and the construction of examples with sharp rates in Stein's method. Probability Approximations and Beyond, Lecture Notes in Statistics 205, Springer. MR3289376
Ross, N. (2011). Fundamentals of Stein's method, Probability Surveys 8, 210-293. MR2861132
Schoutens, W. (2001). Orthogonal polynomials in Stein's method, Journal of Mathematical Analysis and Applications 253, 515-531. MR1808151
Shimizu, R. (1975). On Fisher's amount of information for location family, A Modern Course on Statistical Distributions in Scientific Work, Springer, 305-312.
Stein, C. (1972). A bound for the error in the normal approximation to the distribution of a sum of dependent random variables, Proceedings of the Sixth Berkeley Symposium on Mathematical Statistics and Probability (Univ. California, Berkeley, Calif., 1970/1971), Vol. II: Probability theory (Berkeley, Calif.), Univ. California Press, 583-602. MR0402873
Stein, C. (1986). Approximate computation of expectations, Institute of Mathematical Statistics Lecture Notes-Monograph Series, 7, Institute of Mathematical Statistics, Hayward, CA. MR0882007
Stein, C., Diaconis, P., Holmes, S. and Reinert, G. (2004). Use of exchangeable pairs in the analysis of simulations, Stein's method: expository lectures and applications (Persi Diaconis and Susan Holmes, eds.), IMS Lecture Notes Monogr. Ser, vol. 46, Beachwood, Ohio, USA: Institute of Mathematical Statistics, 1-26. MR2118600
Upadhye, N. S., Cekanavicius, V. and Vellaisamy, P. (2016). On Stein operators for discrete approximations, Bernoulli, to appear.
