For a machine learning progress, how should you split data for training and evaluation?

The standard practice in machine learning is to partition the dataset into two or more subsets. The largest subset, the training set, is used to train the model by allowing it to learn the relationship between features and labels. A separate, unseen subset, the evaluation or test set, is used to assess the model's performance on new data. This is accomplished by randomly splitting the rows (which represent individual data records or observations), ensuring that both the training and evaluation sets have the same features and are statistically representative of the original dataset. This process helps to produce an unbiased evaluation of the model's ability to generalize.

A. Use features for training and labels for evaluation.

Incorrect. Supervised model training requires both features (inputs) and corresponding labels (known outcomes) to learn from.

C. Use labels for training and features for evaluation.

Incorrect. A model cannot be trained on labels alone; it needs the features to identify patterns that predict the labels.

D. Randomly split the data into columns for training and columns for evaluation.

Incorrect. The set of features (columns) must be consistent between the training and evaluation datasets. The model is trained to expect a specific set of input features.

---

1. Microsoft Learn

AI-900 Documentation: In the "Train and evaluate a simple model" module

the process is described: "To train the model

we'll split the dataset into two subsets: a training set (which we'll use to train the model) and a validation or test set (which we'll use to evaluate the trained model)." This split is performed on the data records (rows).

Source: Microsoft

"AI-900: Describe features of machine learning on Azure

" Module: "Train and evaluate a simple model

" Unit: "Training and evaluation."

2. Microsoft Azure Machine Learning Documentation: The documentation for the "Split Data" component in Azure Machine Learning designer explicitly states its function: "This article describes the Split Data component in Azure Machine Learning designer. Use the Split Data component to divide a dataset into two distinct sets... The component works by randomly dividing the rows of data to create the two datasets."

Source: Microsoft Docs

"Split Data component

" Azure Machine Learning documentation

"Component details" section.

3. Stanford University Courseware (CS229): In the course notes on model selection and evaluation

the standard procedure is outlined: "The standard way to do this is to split S into a training set... and a test set... We then train our algorithm on the training set

and evaluate its performance on the test set." Here

S represents the entire dataset of examples (rows).

Source: Ng

A. (2023). "CS229: Machine Learning Course Notes

" Stanford University

Section: "Model Selection and Cross Validation

" Subsection: "Hold-out cross validation and K-fold cross validation."