PROFESSIONAL-MACHINE-LEARNING-ENGINEER practice question 15 of 35

You are an ML engineer at a travel company. You have been researching customers’ travel behavior for many years, and you have deployed models that…

Full set: 283 questions
You are an ML engineer at a travel company. You have been researching customers’ travel behavior for many years, and you have deployed models that predict customers’ vacation patterns. You have observed that customers’ vacation destinations vary based on seasonality and holidays; however, these seasonal variations are similar across years. You want to quickly and easily store and compare the model versions and performance statistics across years. What should you do?
Answer options
A: Store the performance statistics in Cloud SQL. Query that database to compare the performance statistics across the model versions.
B: Create versions of your models for each season per year in Vertex AI. Compare the performance statistics across the models in the Evaluate tab of the Vertex AI UI.
C: Store the performance statistics of each pipeline run in Kubeflow under an experiment for each season per year. Compare the results across the experiments in the Kubeflow UI.
D: Store the performance statistics of each version of your models using seasons and years as events in Vertex ML Metadata. Compare the results across the slices. Correct answer
Correct Answer:
D. Store the performance statistics of each version of your models using seasons and years as events in Vertex ML Metadata. Compare the results across the slices.
Explanation

Vertex ML Metadata is a managed service specifically designed to track and analyze the artifacts, executions, and metadata of ML workflows. By storing performance statistics as metadata and tagging them with custom properties like 'season' and 'year', you can create a queryable history of your model's performance. This allows you to easily "slice" the data (e.g., compare all "summer" models across different years) to analyze the impact of seasonality, which directly addresses the core requirement of the question. This approach provides a robust and scalable solution for long-term model performance analysis.

Why Incorrect

A: Cloud SQL is a generic database. It would require significant custom development to build the necessary schemas, ingestion pipelines, and querying interfaces, making it not a "quick and easy" solution compared to a purpose-built ML tool. B: The "Evaluate" tab in the Vertex AI Model Registry is primarily for comparing model versions against a specific evaluation dataset, not for complex, multi-dimensional performance analysis across different time periods like seasons and years. C: While Kubeflow Experiments can group pipeline runs, comparing performance across different experiments (e.g., one for each season/year) is less straightforward than using a dedicated metadata querying service designed for this type of analysis.

References

1. Google Cloud Documentation, "Introduction to Vertex ML Metadata": "Vertex ML Metadata lets you track the artifacts and metadata produced by your ML systems. You can query the metadata to understand your ML workflow and make informed decisions... For example, you can query for all the models trained with a certain dataset, or the experiments from a specific day." This supports the concept of querying metadata based on custom properties like time or season.

2. Google Cloud Documentation, "Track and analyze ML metadata and artifacts": "By storing information about the executions, artifacts, and contexts of your ML workflow in Vertex ML Metadata, you can answer questions about your ML workflow to help you... compare the performance of pipeline runs." This directly aligns with the solution of using ML Metadata to store and compare performance statistics.

3. Google Cloud Documentation, "Query your ML metadata": This page provides examples of filtering artifacts and executions using the API. A user could, for instance, "List all artifact resources where the displayname contains my-artifact and the createtime is later than T". This demonstrates the powerful filtering capability that can be used to slice data by properties like season and year.

Scroll to Top