Reference : Aggregated hold-out for sparse linear regression with a robust loss function
E-prints/Working papers : Already available on another site
Physical, chemical, mathematical & earth Sciences : Mathematics
Aggregated hold-out for sparse linear regression with a robust loss function
Maillard, Guillaume mailto [University of Luxembourg > Faculty of Science, Technology and Medicine (FSTM) > Department of Mathematics (DMATH) >]
[en] Hyperparameter selection ; Sparse regression ; Cross-validation ; Robust regression ; Lasso ; Aggregation ; Model selection
[en] Sparse linear regression methods generally have a free hyperparameter which controls the amount of sparsity, and is subject to a bias-variance tradeoff. This article considers the use of Aggregated hold-out to aggregate over values of this hyperparameter, in the context of linear regression with the Huber loss function. Aggregated hold-out (Agghoo) is a procedure which averages estimators selected by hold-out (cross-validation with a single split). In the theoretical part of the article, it is proved that Agghoo satisfies a non-asymptotic oracle inequality when it is applied to sparse estimators which are parametrized by their zero-norm. In particular, this includes a variant of the Lasso introduced by Zou, HastiƩ and Tibshirani \cite{Zou_Has_Tib:2007}. Simulations are used to compare Agghoo with cross-validation. They show that Agghoo performs better than CV when the intrinsic dimension is high and when there are confounders correlated with the predictive covariates.
European Union Horizon 2020

File(s) associated to this reference

Fulltext file(s):

Open access
sparse_huber_hal.pdfAuthor preprint1.02 MBView/Open

Bookmark and Share SFX Query

All documents in ORBilu are protected by a user license.