During the process of Machine Learning, when should you review evaluation metrics?

The standard machine learning workflow involves training a model on a known dataset and then evaluating its performance on data it has not seen before. This evaluation step is performed by using the trained model to make predictions on a validation or test dataset. The predictions are then compared to the actual known values (ground truth) in that dataset to calculate performance metrics (e.g., accuracy, precision, RMSE). Therefore, reviewing evaluation metrics is a critical activity that occurs after a model has been tested on validation data to assess its effectiveness and generalization capabilities.

A. After you clean the data.

Data cleaning is a preprocessing step. At this stage, a model has not yet been trained, so there are no performance metrics to review.

B. Before you train a model.

Before training, a model does not exist in a trained state. Evaluation metrics measure a trained model's performance, so they cannot be reviewed yet.

C. Before you choose the type of model.

Model selection is based on the nature of the problem (e.g., classification, regression). Evaluation metrics are generated by a trained model, not before one is chosen.

---

1. Microsoft Learn

AI-900: Explore the machine learning process. This module outlines the typical workflow for a machine learning project. The "Evaluate the model" step is explicitly placed after the "Train the model" step. The documentation states

"After you've trained a model

you need to evaluate it to determine how well it predicts... To evaluate a model

you use data that was held back from the training set... and compare the predictions it makes with the known actual values."

Source: Microsoft Learn

"Describe the machine learning process

" under the section "Evaluate the model."

2. Microsoft Learn

Evaluate model performance in Azure Machine Learning. This document details the evaluation phase. It describes the process of using a holdout dataset (test data) to score a trained model and then reviewing the resulting metrics to understand its performance.

Source: Microsoft Learn

"Introduction to model evaluation

" under the section "Evaluate a model."

3. Microsoft Learn

What is automated machine learning (AutoML)? The description of the AutoML process reinforces this sequence. It explains that the service iterates through algorithms

producing a model with a training score in each iteration. It explicitly states

"You can review the metrics for each model to determine its performance

" which happens after each model is trained and tested.

Source: Microsoft Learn

"What is automated machine learning (AutoML)?

" under the section "When to use AutoML: Classify

regression

& forecast."