ASSOCIATE-DATA-PRACTITIONER practice question 11 of 25

Your organization’s ecommerce website collects user activity logs using a Pub/Sub topic. Your organization’s leadership team wants a dashboard that contains aggregated user engagement metrics. You need to create a solution that transforms the user activity logs into aggregated metrics, while ensuring that the raw data can be easily queried. What should you do?

Full set: 106 questions
Answer options
A: Create a Dataflow subscription to the Pub/Sub topic, and transform the activity logs. Load the transformed data into a BigQuery table for reporting.
B: Create an event-driven Cloud Run function to trigger a data transformation pipeline to run. Load the transformed activity logs into a BigQuery table for reporting.
C: Create a Cloud Storage subscription to the Pub/Sub topic. Load the activity logs into a bucket using the Avro file format. Use Dataflow to transform the data, and load it into a BigQuery table for reporting.
D: Create a BigQuery subscription to the Pub/Sub topic, and load the activity logs into the table. Create a materialized view in BigQuery using SQL to transform the data for reporting Correct answer
Correct Answer:
D. Create a BigQuery subscription to the Pub/Sub topic, and load the activity logs into the table. Create a materialized view in BigQuery using SQL to transform the data for reporting
Explanation

This solution is the most efficient and modern approach. A BigQuery subscription directly streams raw Pub/Sub messages into a BigQuery table, perfectly satisfying the requirement for easily queryable raw data. A materialized view can then be created on top of this raw data table. It automatically and incrementally pre-computes the aggregated user engagement metrics. This pre-computation significantly speeds up dashboard queries, providing a low-latency experience for the leadership team. This architecture is simpler and more integrated than a multi-service Dataflow pipeline for this specific use case.

Why Incorrect

A. This is a valid but less efficient pattern. It requires managing a separate Dataflow job for transformation, adding operational overhead compared to a serverless BigQuery-native solution.
B. Using Cloud Run is overly complex for this task. Dataflow or BigQuery's native capabilities are better suited for continuous data transformation pipelines than a generic container service.
C. This adds an unnecessary intermediate step. Storing data in Cloud Storage first before processing introduces latency and complexity compared to streaming directly into BigQuery.

References

1. Google Cloud Documentation, "Stream data from Pub/Sub to BigQuery". This page describes the BigQuery subscription type, which writes messages directly to an existing BigQuery table.

2. Google Cloud Documentation, "Introduction to materialized views". This document explains how materialized views precompute results, improving dashboarding performance by querying pre-aggregated data.

3. Google Cloud Architecture Center, "Serverless data analysis reference architecture". This guide shows a pattern where Pub/Sub streams directly to BigQuery for real-time analytics, which is then used by BI tools.

Scroll to Top