ASSOCIATE-DATA-PRACTITIONER practice question 20 of 25

You have a Dataflow pipeline that processes website traffic logs stored in Cloud Storage and writes the processed data to BigQuery. You noticed that the pipeline is failing intermittently. You need to troubleshoot the issue. What should you do?

Full set: 106 questions
Answer options
A: Use Cloud Logging to identify error groups in the pipeline's logs. Use Cloud Monitoring to create a dashboard that tracks the number of errors in each group.
B: Use Cloud Logging to create a chart displaying the pipeline’s error logs. Use Metrics Explorer to validate the findings from the chart.
C: Use Cloud Logging to view error messages in the pipeline's logs. Use Cloud Monitoring to analyze the pipeline's metrics, such as CPU utilization and memory usage. Correct answer
D: Use the Dataflow job monitoring interface to check the pipeline's status every hour. Use Cloud Profiler to analyze the pipeline’s metrics, such as CPU utilization and memory usage.
Correct Answer:
C. Use Cloud Logging to view error messages in the pipeline's logs. Use Cloud Monitoring to analyze the pipeline's metrics, such as CPU utilization and memory usage.
Explanation

The most effective and standard troubleshooting process for an intermittent Dataflow pipeline failure involves a two-step approach. First, use Cloud Logging to inspect the job logs for specific error messages, which reveals what is failing. Second, use Cloud Monitoring to analyze performance metrics (e.g., CPU utilization, memory, system lag) around the time of the failures. This helps correlate the errors with potential resource constraints or bottlenecks, revealing why the failure is occurring.

Why Incorrect

A. While error grouping is useful, viewing specific error messages first is more direct. A dashboard is for ongoing monitoring, not initial troubleshooting of an unknown intermittent issue.
B. Metrics Explorer is for analyzing metrics, not for validating findings from log charts. The primary step should be to read the error messages directly.
D. Manually checking the UI is inefficient for intermittent issues. Cloud Profiler is for deep code-level performance optimization, not the first tool for general job failures.

References

1. Google Cloud Documentation, "Monitor a pipeline," Dataflow. "You can use Cloud Monitoring to view metrics about your Dataflow jobs... You can also use Cloud Logging to view log messages from your Dataflow workers."

2. Google Cloud Documentation, "Troubleshoot Dataflow errors," Dataflow. "To troubleshoot a failed Dataflow job, first identify the cause of the error by looking at the job logs... After you identify the error, you might need to debug your pipeline code."

3. Google Cloud Documentation, "Build observable pipelines," Dataflow. "Cloud Logging provides logs from your Dataflow pipeline... Cloud Monitoring provides visibility into the performance, uptime, and overall health of your Dataflow jobs."

Scroll to Top