Expand description
§Lasso
Linear regression is the standard algorithm for predicting a quantitative response \(y\) on the basis of a linear combination of explanatory variables \(X\) that assumes that there is approximately a linear relationship between \(X\) and \(y\). Lasso is an extension to linear regression that adds L1 regularization term to the loss function during training.
Similar to ridge regression, the lasso shrinks the coefficient estimates towards zero when. However, in the case of the lasso, the l1 penalty has the effect of forcing some of the coefficient estimates to be exactly equal to zero when the tuning parameter \(\alpha\) is sufficiently large.
Lasso coefficient estimates solve the problem:
\[\underset{\beta}{minimize} \space \space \frac{1}{n} \sum_{i=1}^n \left( y_i - \beta_0 - \sum_{j=1}^p \beta_jx_{ij} \right)^2 + \alpha \sum_{j=1}^p \lVert \beta_j \rVert_1\]
This problem is solved with an interior-point method that is comparable to coordinate descent in solving large problems with modest accuracy, but is able to solve them with high accuracy with relatively small additional computational cost.
§References:
Structs§
- Lasso
- Lasso regressor
- Lasso
Parameters - Lasso regression parameters
- Lasso
Search Parameters - Lasso grid search parameters
- Lasso
Search Parameters Iterator - Lasso grid search iterator