Finite mixture model; robust estimation; supremum of an empirical process
Abstract :
[en] We observe a n-sample, the distribution of which is assumed to belong, or at least to be close enough, to a given mixture model. We propose an estimator of this distribution that belongs to our model and possesses some robustness properties with respect to a possible misspecification of it. We establish a non-asymptotic deviation bound for the Hellinger distance between the target distribution and its estimator when the model consists of a mixture of densities that belong to VC-subgraph classes. Under suitable assumptions and when the mixture model is well-specified, we derive risk bounds for the parameters of the mixture. Finally, we design a statistical procedure that allows us to select from the data the number of components as well as suitable models for each of the densities that are involved in the mixture. These models are chosen among a collection of candidate ones and we show that our selection rule combined with our estimation strategy result in an estimator which satisfies an oracle-type inequality.
Disciplines :
Mathematics
Author, co-author :
LECESTRE, Alexandre ; University of Luxembourg > Faculty of Science, Technology and Medicine (FSTM) > Department of Mathematics (DMATH)
scite shows how a scientific paper has been cited by providing the context of the citation, a classification describing whether it supports, mentions, or contrasts the cited claim, and a label indicating in which section the citation was made.
Bibliography
E. S. Allman, C. Matias and J. A. Rhodes, Identifiability of parameters in latent structure models with many observed variables. Ann. Stat. 37 (2009) 3099-3132.
A. Azzalini and A. Capitanio, The Skew-Normal and Related Families. Institute of Mathematical Statistics Monographs, Cambridge University Press (2013).
Y. Baraud and L. Birge, Rho-estimators revisited: General theory and applications. Ann. Stat. 46 (2018) 3767-3804.
Y. Baraud, L. Birge and M. Sart, A new method for estimation and model selection: rho-estimation. Invent. Math. 207 (2017) 425-517.
Y. Baraud and J. Chen, Robust estimation of a regression function in exponential families (2020).
L. Birge, Approximation dans les espaces metriques et theorie de l'estimation. Zeitsch. Wahrscheinlichkeitstheorie Verwand. Gebiete 65 (1983).
L. Birge, On estimating a density using Hellinger distance and some other strange facts. Prob. Theory Related Fields 71 (1986).
I. Diakonikolas, D. M. Kane and A. Stewart, List-decodable robust mean estimation and learning mixtures of spherical Gaussians, in Proceedings of the 50th Annual ACM SIGACT Symposium on Theory of Computing, STOC 2018. Association for Computing Machinery, New York, NY, USA (2018) pp. 1047-1060.
C. R. Doss and J. A. Wellner, Global rates of convergence of the MLEs of log-concave and s-concave densities. Ann. Stat. 44 (2016) 954-981.
N. Doss, Y. Wu, P. Yang and H. H. Zhou, Optimal estimation of high-dimensional location Gaussian mixtures (2020).
B. Everitt and D. J. Hand, Finite mixture distributions. Chapman and Hall London; New York (1981).
S. Fruhwirth-Schnatter, Finite Mixture and Markov Switching Models. Springer Series in Statistics, Springer New York (2006).
S. Gadat, C. Marteau and C. Maugis-Rabusseau, Parameter recovery in two-component contamination mixtures: The L2 strategy. Ann. l'Institut Henri Poincare, Prob. Stat. 56 (2020) 1391-1418.
C. Genovese and L. Wasserman, Convergence rates for the Gaussian mixture sieve, Ann. Stat. 28 (2000) 10. 1214/aos/1015956709.
S. Ghosal and A. W. van der Vaart, Entropies and rates of convergence for maximum likelihood and Bayes estimation for mixtures of normal densities, Ann. Statist. 29 (2001) 1233-1263.
P. Heinrich and J. Kahn, Strong identifiability and optimal minimax rates for finite mixture estimation. Ann. Stat. 46 (2018) 2844-2870.
I. A. Ibragimov and H. R. Z., Statistical Estimation. Springer, New York (1981).
W. Kruijer, J. Rousseau and A. van der Vaart, Adaptive Bayesian density estimation with location-scale mixtures. Electr. J. Stat. 4 (2010) 1225-1257.
P. Massart, Concentration Inequalities and Model Selection. Vol. 1896 of Lect. Notes Math. Springer, Berlin, Heidelberg (2007).
C. Maugis and B. Michel, A non asymptotic penalized criterion for Gaussian mixture model selection. ESAIM: PS 15 (2011) 41-68.
C. Maugis-Rabusseau and B. Michel, Adaptive density estimation for clustering with Gaussian mixtures. ESAIM: PS 17 (2013) 698-724.
G. McLachlan and D. Peel, Finite mixture models. Vol. 44 of Wiley Series in Probability and Statistics. Wiley (2000).
E. Meijer and J. Y. Ypma, A simple identification proof for a mixture of two univariate normal distributions. J. Classif. 25 (2008) 113-123.
R. T. Rockafellar, Convex Analysis. Princeton University Press (2015).
T. Sapatinas, Identifiability of mixtures of power-series distributions and related characterizations. Ann. Inst. Stat. Math. 47 (1995) 447-459.
H. Teicher, Identifiability of mixtures. Ann. Math. Stat. 32 (1961) 244-248.
D. Titterington, A. Smith and U. Makov, Statistical Analysis of Finite Mixture Distributions, Applied section. Wiley (1985).
A. W. van der Vaart and J. A. Wellner, Weak Convergence and Empirical Processes. Springer, New York (1996).
Y. Wu and P. Yang, Optimal estimation of Gaussian mixtures via denoised method of moments. Ann. Stat. 48 (2020) 1981-2007.
