elastic net

How Does Elastic Net Work?

Elastic Net introduces two hyperparameters:

1. Alpha (α): This parameter controls the balance between Lasso and Ridge penalties. When α=1, Elastic Net becomes Lasso; when α=0, it becomes Ridge.
2. Lambda (λ): This parameter controls the overall strength of the penalty. Higher λ values lead to more regularization.

The Elastic Net objective function is:

\[ \min_{\beta} \left( \frac{1}{2N} \sum_{i=1}^{N} (y_i - \beta_0 - \sum_{j=1}^{p} x_{ij}\beta_j)^2 + \lambda \left( \alpha \sum_{j=1}^{p} |\beta_j| + \frac{1-\alpha}{2} \sum_{j=1}^{p} \beta_j^2 \right) \right) \]

Here, \( \beta \) represents the coefficients, \( N \) is the number of observations, \( y_i \) are the observed outcomes, and \( x_{ij} \) are the predictor values.