PROFESSIONAL-MACHINE-LEARNING-ENGINEER practice question 28 of 35

You need to execute a batch prediction on 100 million records in a BigQuery table with a custom TensorFlow DNN regressor model, and then store the predicted results in a BigQuery table. You want to minimize the effort required to build this inference pipeline. What should you do?

Full set: 283 questions
Answer options
A: Import the TensorFlow model with BigQuery ML, and run the ml.predict function. Correct answer
B: Use the TensorFlow BigQuery reader to load the data, and use the BigQuery API to write the results to BigQuery.
C: Create a Dataflow pipeline to convert the data in BigQuery to TFRecords. Run a batch inference on Vertex AI Prediction, and write the results to BigQuery.
D: Load the TensorFlow SavedModel in a Dataflow pipeline. Use the BigQuery I/O connector with a custom function to perform the inference within the pipeline, and write the results to BigQuery.
Correct Answer:
A. Import the TensorFlow model with BigQuery ML, and run the ml.predict function.
Explanation

BigQuery ML is designed to bring machine learning to the data. It supports importing pre-trained TensorFlow models directly into the BigQuery environment. Once the model is imported and registered using a CREATE MODEL statement, you can perform batch predictions on petabyte-scale data using a single, simple SQL query with the ML.PREDICT function. This serverless approach abstracts away all infrastructure management and data movement pipelines, requiring significantly less development and operational effort compared to building custom pipelines with Dataflow or managing separate prediction jobs. This makes it the ideal choice for minimizing effort.

Why Incorrect

B. This requires writing a custom application with TensorFlow and BigQuery client libraries, managing the execution environment, and handling data I/O manually, which is more complex than a single SQL query.
C. This is a multi-stage process involving a Dataflow job for data conversion, a Vertex AI Batch Prediction job, and managing data flow between services, representing a high-effort solution.
D. Building a custom Dataflow pipeline to read data, load a model, perform inference, and write results requires significant Apache Beam programming effort, which is not minimal.

References

1. Google Cloud Documentation, "Create a model in BigQuery ML using TensorFlow": This document explicitly describes the process of importing a TensorFlow SavedModel into BigQuery ML. The syntax CREATE MODEL mydataset.importedtfmodel OPTIONS(modeltype='TENSORFLOW', modelpath='gs://yourbucket/yourmodel/') demonstrates the straightforward, SQL-based method for making the model available within BigQuery.

2. Google Cloud Documentation, "The ML.PREDICT function": This page details the usage of ML.PREDICT for inference. It states, "You can use the ML.PREDICT function to predict outcomes using your model... For imported models, the output of the ML.PREDICT function is the output of the model's serving signature." This confirms that a simple SQL function call is all that is needed after the model is imported.

3. Google Cloud Documentation, "Introduction to BigQuery ML": This overview highlights the core value proposition relevant to the question. In the "BigQuery ML capabilities" section, it lists the ability to "Import previously trained TensorFlow... models for evaluation or prediction," reinforcing that this is a primary, built-in feature designed for ease of use.

Scroll to Top