ASSOCIATE-DATA-PRACTITIONER practice question 22 of 25
Your company is building a near real-time streaming pipeline to process JSON telemetry data from small appliances. You need to process messages arriving at a Pub/Sub topic, capitalize letters in the serial number field, and write results to BigQuery. You want to use a managed service and write a minimal amount of code for underlying transformations. What should you do?
Full set: 106 questionsThe "Pub/Sub to BigQuery" Dataflow template is a managed, serverless solution designed for streaming data from Pub/Sub to BigQuery. This approach minimizes operational overhead. The template supports User-Defined Functions (UDFs) written in JavaScript, which allows for lightweight, inline transformations like capitalizing a field. This combination perfectly meets the requirements of using a managed service to process a stream in near real-time while writing a minimal amount of custom code for the transformation logic.
A. Using a scheduled query introduces latency (up to five minutes) and is not a near real-time streaming solution.
B. This architecture (Pub/Sub -> GCS -> Cloud Run) is overly complex, adds unnecessary latency, and requires more code than a Dataflow template.
D. While Cloud Run can work, it requires writing more boilerplate code for message handling and BigQuery API interaction compared to using a pre-built Dataflow template with a UDF.
1. Google Cloud. (2024). Pub/Sub to BigQuery template. Dataflow documentation. "The Pub/Sub to BigQuery template is a streaming pipeline that reads JSON-formatted messages from a Pub/Sub topic and writes them to a BigQuery table." The page also details the javascriptTextTransformGcsPath and javascriptTextTransformFunctionName parameters for applying a UDF. Retrieved from https://cloud.google.com/dataflow/docs/guides/templates/provided/pubsub-to-bigquery
2. Google Cloud. (2024). Create user-defined functions for Dataflow templates. Dataflow documentation. This guide provides examples and instructions on how to write and apply a JavaScript UDF to a Dataflow template for custom transformations. Retrieved from https://cloud.google.com/dataflow/docs/guides/templates/create-udf
Keep practicing
Take the free ASSOCIATE-DATA-PRACTITIONER practice test
Ten exam-style questions with answers and explanations, plus the exam facts and study guides.
More questions
Other ASSOCIATE-DATA-PRACTITIONER practice questions
- Question 1You are designing a pipeline to process data files that arrive in Cloud Storage by 3:00 a…
- Question 2Your organization plans to move their on-premises environment to Google Cloud. Your organ…
- Question 3You are responsible for managing Cloud Storage buckets for a research company. Your compa…
- Question 4You are predicting customer churn for a subscription-based service. You have a 50 PB hist…
- Question 5Your retail company collects customer data from various sources:Online transactions: Stor…
- Question 6Your company has developed a website that allows users to upload and share video files. T…
