From robust tests to Bayes-like posterior distributions

E-print/Working paper (2021)

In the Bayes paradigm and for a given loss function, we propose the construction of a new type of posterior distributions for estimating the law of an n-sample. The loss functions we have in mind are ... [more ▼]

In the Bayes paradigm and for a given loss function, we propose the construction of a new type of posterior distributions for estimating the law of an n-sample. The loss functions we have in mind are based on the total variation distance, the Hellinger distance as well as some 𝕃j-distances. We prove that, with a probability close to one, this new posterior distribution concentrates its mass in a neighbourhood of the law of the data, for the chosen loss function, provided that this law belongs to the support of the prior or, at least, lies close enough to it. We therefore establish that the new posterior distribution enjoys some robustness properties with respect to a possible misspecification of the prior, or more precisely, its support. For the total variation and squared Hellinger losses, we also show that the posterior distribution keeps its concentration properties when the data are only independent, hence not necessarily i.i.d., provided that most of their marginals are close enough to some probability distribution around which the prior puts enough mass. The posterior distribution is therefore also stable with respect to the equidistribution assumption. We illustrate these results by several applications. We consider the problems of estimating a location parameter or both the location and the scale of a density in a nonparametric framework. Finally, we also tackle the problem of estimating a density, with the squared Hellinger loss, in a high-dimensional parametric model under some sparcity conditions. The results established in this paper are non-asymptotic and provide, as much as possible, explicit constants. [less ▲]

Robust Estimation of a Regression Function in Exponential Families

E-print/Working paper (2020)

ROBUST BAYES-LIKE ESTIMATION: RHO-BAYES ESTIMATION

in Annals of Statistics (2020)

We observe n independent random variables with joint distribution P and pretend that they are i.i.d. with some common density s (with respect to a known measure μ) that we wish to estimate. We consider a ... [more ▼]

We observe n independent random variables with joint distribution P and pretend that they are i.i.d. with some common density s (with respect to a known measure μ) that we wish to estimate. We consider a density model S for s that we endow with a prior distribution π (with support in S) and build a robust alternative to the classical Bayes posterior distribution which possesses similar concentration properties around s whenever the data are truly i.i.d. and their density s belongs to the model S. Furthermore, in this case, the Hellinger distance between the classical and the robust posterior distributions tends to 0, as the number of observations tends to infinity, under suitable assumptions on the model and the prior. However, unlike what happens with the classical Bayes posterior distribution, we show that the concentration properties of this new posterior distribution are still preserved when the model is misspecified or when the data are not i.i.d. but the marginal densities of their joint distribution are close enough in Hellinger distance to the model S. [less ▲]

Rho-estimators revisited: general theory and applications

in Annals of Statistics (2018), 46(6B), 3767--3804

A new method for estimation and model selection: $\rho$-estimation

in Inventiones Mathematicae (2017), 207(2), 425--517

Bounding the expectation of the supremum of an empirical process over a (weak) VC-major class

in Electronic Journal of Statistics (2016), 10(2), 1709--1728

Estimating composite functions by model selection

in Ann. Inst. Henri Poincaré Probab. Stat. (2014), 50(1), 285--314

Estimator selection with respect to Hellinger-type risks

in Probab. Theory Related Fields (2011), 151(1-2), 353--401

Gaussian model selection with an unknown variance

in Annals of Statistics (2009), 37(2), 630--672

Testing convex hypotheses on the mean of a Gaussian vector. Application to testing qualitative hypotheses on a regression function

in Annals of Statistics (2005), 33(1), 214--257