Reference : Aggregated hold-out for sparse linear regression with a robust loss function
E-prints/Working papers : Already available on another site
Physical, chemical, mathematical & earth Sciences : Mathematics
http://hdl.handle.net/10993/47481
Aggregated hold-out for sparse linear regression with a robust loss function
English
Maillard, Guillaume mailto [University of Luxembourg > Faculty of Science, Technology and Medicine (FSTM) > Department of Mathematics (DMATH) >]
7-May-2021
2
64
No
[en] Hyperparameter selection ; Sparse regression ; Cross-validation ; Robust regression ; Lasso ; Aggregation ; Model selection
[en] Sparse linear regression methods generally have a free hyperparameter which controls the amount of sparsity, and is subject to a bias-variance tradeoff. This article considers the use of Aggregated hold-out to aggregate over values of this hyperparameter, in the context of linear regression with the Huber loss function. Aggregated hold-out (Agghoo) is a procedure which averages estimators selected by hold-out (cross-validation with a single split). In the theoretical part of the article, it is proved that Agghoo satisfies a non-asymptotic oracle inequality when it is applied to sparse estimators which are parametrized by their zero-norm. In particular, this includes a variant of the Lasso introduced by Zou, HastiƩ and Tibshirani \cite{Zou_Has_Tib:2007}. Simulations are used to compare Agghoo with cross-validation. They show that Agghoo performs better than CV when the intrinsic dimension is high and when there are confounders correlated with the predictive covariates.
European Union Horizon 2020
Professionals
http://hdl.handle.net/10993/47481
https://hal.archives-ouvertes.fr/hal-02485694

File(s) associated to this reference

Fulltext file(s):

FileCommentaryVersionSizeAccess
Open access
sparse_huber_hal.pdfAuthor preprint1.02 MBView/Open

Bookmark and Share SFX Query

All documents in ORBilu are protected by a user license.