Aggregated hold-out for sparse linear regression with a robust loss function

in Electronic Journal of Statistics (2022), 16(1), 935-997

Sparse linear regression methods generally have a free hyperparameter which controls the amount of sparsity, and is subject to a bias-variance tradeoff. This article considers the use of Aggregated hold ... [more ▼]

Sparse linear regression methods generally have a free hyperparameter which controls the amount of sparsity, and is subject to a bias-variance tradeoff. This article considers the use of Aggregated hold-out to aggregate over values of this hyperparameter, in the context of linear regression with the Huber loss function. Aggregated hold-out (Agghoo) is a procedure which averages estimators selected by hold-out (cross-validation with a single split). In the theoretical part of the article, it is proved that Agghoo satisfies a non-asymptotic oracle inequality when it is applied to sparse estimators which are parametrized by their zero-norm. In particular, this includes a variant of the Lasso introduced by Zou, Hastié and Tibshirani \cite{Zou_Has_Tib:2007}. Simulations are used to compare Agghoo with cross-validation. They show that Agghoo performs better than CV when the intrinsic dimension is high and when there are confounders correlated with the predictive covariates. [less ▲]

Bounding the expectation of the supremum of an empirical process over a (weak) VC-major class

in Electronic Journal of Statistics (2016), 10(2), 1709--1728

Exact confidence intervals for the Hurst parameter of a fractional Brownian motion

in Electronic Journal of Statistics (2009), 3