PROFESSIONAL-CLOUD-DEVOPS-ENGINEER practice question 3 of 25

You are responding to a high-priority incident where a critical, user-facing payment service is experiencing a 50% error rate. The cause is a…

Full set: 205 questions
You are responding to a high-priority incident where a critical, user-facing payment service is experiencing a 50% error rate. The cause is a non-critical, batch analytics Dataflow pipeline flooding a shared Memorystore for Redis instance with writes, which has spiked read latency for the payment service. A full rollback of the Dataflow pipeline's deployment will take 15 minutes to complete through your CI/CD process. You need to restore the payment service as quickly as possible. What should you do?
Answer options
A: Use Cloud Profiler to inspect the Dataflow pipeline's execution graph to pinpoint the source of the excessive writes.
B: In the Google Cloud console, edit the Memorystore for Redis instance and increase its capacity tier.
C: Initiate an automated rollback of the Dataflow pipeline's deployment to revert to the last stable version.
D: Cancel the active Dataflow job. Correct answer
Correct Answer:
D. Cancel the active Dataflow job.
Explanation

The immediate priority is to mitigate the impact on the critical payment service. The problem is caused by a non-critical Dataflow pipeline flooding the shared Memorystore instance. Canceling the Dataflow job is the fastest and most direct action to stop the excessive writes. This will immediately relieve the load on Memorystore, allowing the payment service's read latency to return to normal and restore service. Other options are either too slow or do not address the immediate cause of the incident.

Why Incorrect

A. Profiling is for analysis, not immediate remediation. It takes time to gather data and would not stop the ongoing impact on the payment service.
B. Increasing the Memorystore capacity might eventually help, but scaling operations are not instantaneous and don't stop the source of the abusive traffic.
C. The question explicitly states that a full rollback through the CI/CD pipeline will take 15 minutes, which is too slow for a critical incident.

References

1. Google Cloud Documentation, Dataflow, "Update a pipeline". States that canceling a job is an immediate action to stop it. "To stop your running job, you can cancel it."

2. Google, "Site Reliability Engineering: How Google Runs Production Systems," Chapter 12, "Effective Troubleshooting". Emphasizes the principle of stopping the cause of the problem as quickly as possible during an incident. "The first step of any incident response is to triage the issue... The fastest way to restore service is often to roll back the change that introduced the problem." In this case, canceling the job is faster than a full rollback.

Scroll to Top