DISCUSSION: A TALE OF THREE COUSINS: LASSO, L2BOOSTING AND DANTZIG
Contributor(s)The Pennsylvania State University CiteSeerX Archives
Full recordShow full item record
AbstractWe would like to congratulate the authors for their thought-provoking and interesting paper. The Dantzig paper is on the timely topic of highdimensional data modeling that has been the center of much research lately and where many exciting results have been obtained. It also falls in the very hot area at the interface of statistics and optimization: ℓ1-constrained minimization in linear models for computationally efficient model selection, or sparse model estimation (Chen, Donoho and Saunders  and Tibshirani ). The sparsity consideration indicates a trend in high-dimensional data modeling advancing from prediction, the hallmark of machine learning, to sparsity—a proxy for interpretability. This trend has been greatly fueled by the participation of statisticians in machine learning research. In particular, Lasso (Tibshirani ) is the focus of many sparsity studies in terms of both theoretical analysis (Knight and Fu , Greenshtein and Ritov , van de Geer , Bunea, Tsybakov and Wegkamp , Meinshausen and Bühlmann , Zhao and Yu  and Wainwright ) and fast algorithm development (Osborne, Presnell and Turlach  and Efron et al. ). Given n units of data Zi = (Xi,Yi) with Yi ∈ R and X T i ∈ Rp for i = 1,...,n, let Y = (Y1,...,Yn) T ∈ R n be the continuous vector response variable and X = (X1,...,Xn) T the n × p design matrix and let the columns of X be normalized to have ℓ2-norm 1. It is often useful to assume a linear regression model, (1) Y = Xβ + ε, where ε is an i.i.d. N(0,σ 2) vector of size n.