You are building a linear model with over 100 input features, all with values between -1 and 1. You suspect that many features are non-informative. You want to remove the non-informative features from your model while keeping the informative ones in their original form. Which technique should you use?

L1 regularization, also known as Lasso (Least Absolute Shrinkage and Selection Operator), is a technique used in model training that adds a penalty term to the loss function proportional to the absolute value of the coefficient magnitudes. A key property of this penalty is its ability to shrink the coefficients of non-informative features to exactly zero. This process effectively performs automatic feature selection by removing irrelevant features from the model. The remaining features with non-zero coefficients are retained in their original, interpretable form, directly addressing the requirements of the problem.

A. Principal Component Analysis (PCA) is a dimensionality reduction technique, not feature selection. It creates new, uncorrelated features (principal components) from linear combinations of the original ones, failing to keep informative features in their original form.
C. Shapley values are a post-hoc explainability method used to understand a trained model's predictions. While they can identify important features, this is an indirect, multi-step process (train, analyze, remove, retrain) rather than an integrated feature selection technique.
D. This describes a heuristic approach similar to recursive feature elimination. While viable, L1 regularization is a more standard, computationally efficient, and integrated method for inducing sparsity and performing feature selection directly during the training of a linear model.

1. Google Cloud Documentation: In BigQuery ML, the L1REG option in the CREATE MODEL statement for linear models is used to apply L1 regularization. The documentation notes its use for feature selection: "L1 regularization... can be useful if the goal is to select only a subset of the most important features."

Source: Google Cloud, BigQuery ML Documentation, CREATE MODEL statement for linear regression models, L1REG option description.

2. Academic Textbook: Hastie, T., Tibshirani, R., & Friedman, J. (2009). The Elements of Statistical Learning: Data Mining, Inference, and Prediction. Springer.

In Section 3.4.2, "Lasso," it is stated: "The lasso shrinkage causes some of the coefficients to be exactly zero... Thus the lasso does a kind of continuous subset selection." This highlights its dual role in regularization and feature selection.

In the module on "Regularization," the course explains that L1 regularization leads to a sparse model where many feature weights become zero. This sparsity is explicitly mentioned as a mechanism for performing feature selection, especially when dealing with a large number of features.

4. Academic Textbook (Contrasting PCA): James, G., Witten, D., Hastie, T., & Tibshirani, R. (2013). An Introduction to Statistical Learning: with Applications in R. Springer.

In Chapter 6, Section 6.2.2 "The Lasso," the text states, "...the lasso performs variable selection." In contrast, Chapter 10, Section 10.2 "Principal Components Analysis," describes PCA as a tool to "summarize a set of correlated variables with a smaller number of representative variables," confirming it transforms rather than selects original features.