glmnet in R

Load glmnet to make the functionalities available.

library(glmnetUtils)
library(tidyverse)

Ressources on glmnet

• The package vignette vignette("glmnet_beta") is highly recommendable.
• Many questions have already been asked and answered at https://stackoverflow.com/questions/tagged/glmnet.

Example

crime <- read_csv("day06_crime.csv", col_types = cols())

Note: Due to different scales of the variables, the estimated parameters may be different in size simply due to different units. Hence, The glmnet function automatically standardises both the response (for family = "gaussian") y and the covariates x.

crime_lasso <- glmnet(`crime rate` ~ ., alpha = 1, data = crime) ## alpha = 1: LASSO
plot(crime_lasso)

library(plotmo) # for plot_glmnet # install.packages("plotmo")
par(mfrow = c(2,2))
plot_glmnet(crime_lasso, xvar = "norm")
plot_glmnet(crime_lasso, xvar = "rlambda")
plot_glmnet(crime_lasso, xvar = "lambda")
plot_glmnet(crime_lasso, xvar = "dev")

crime_ridge <- glmnet(`crime rate` ~ ., alpha = 0, data = crime) ## alpha = 0: Ridge Regression
plot(crime_ridge)

plot_glmnet(crime_ridge, xvar = "norm")

What should \(\lambda\) be?

LASSO

cv_glmnet_lasso <- cv.glmnet(`crime rate` ~ ., alpha = 1, data = crime)
plot(cv_glmnet_lasso)

Ridge

cv_glmnet_ridge <- cv.glmnet(`crime rate` ~ ., alpha = 0, data = crime) ## alpha = 0: Ridge
plot(cv_glmnet_ridge)

Elastic net approach: What should \(\alpha\) be?

cv_glmnet_enet <- cva.glmnet(`crime rate` ~ ., data = crime)

plot(cv_glmnet_enet)
alpha_cols <- cv_glmnet_enet$alpha %>% set_names(topo.colors(length(.)), nm = .)
legend("topleft", bty = "n", title = expression(alpha),
       legend = names(alpha_cols), lty = 1, col = alpha_cols)

minlossplot(cv_glmnet_enet)

Other types of regression

There are many different types of regressions one would be interested in:

• Logistic regression (binary outcomes): family = "binomial"
• Count regression (Poisson distribution): family = "poisson"
• Multiclass (multinomial data): family = "multinomial"
• Survival analysis (Cox proportional hazard model): family = "cox"
• Multivariate normal (multivariate Gaussian): family = "mgaussian"