PROFESSIONAL-DATA-ENGINEER practice question 10 of 10
You are using Cloud Bigtable to persist and serve stock market data for each of the major indices. To serve the trading application, you need to access only the most recent stock prices that are streaming in How should you design your row key and tables to ensure that you can access the data with the most simple query?
Full set: 401 questionsThe optimal design for this time-series use case is to use a single table with a composite row key that facilitates the primary query: retrieving the most recent entry. The row key [index]#[reversetimestamp] achieves this. By prefixing with the stock index, all data for a given index is grouped together. Using a reverse timestamp (e.g., Long.MAXVALUE - timestamp) ensures that the most recent data (with the highest original timestamp) has the lowest reverse timestamp value. This places the newest entries at the very beginning of each index's row range, allowing the most recent price to be retrieved with a simple and highly efficient prefix scan limited to one row.
A. Using a standard timestamp ([index]#[timestamp]) sorts data chronologically, placing the most recent entries at the end of the row range. Retrieving the latest entry would require a less efficient reverse scan.
C. Creating a separate table for each index is a Bigtable anti-pattern that increases management overhead. Additionally, using a standard timestamp presents the same query inefficiency as option A.
D. While using a reverse timestamp is correct for the query pattern, creating a separate table for each index is a suboptimal design practice for storing related data in Cloud Bigtable.
1. Google Cloud Documentation, "Schema design for time series data": This document explicitly recommends using a reverse timestamp for this exact use case. It states, "To make it easy to read the latest values first, use a reversed timestamp in your row key. A reversed timestamp is a timestamp that is subtracted from the maximum possible value for a timestamp...". It also provides an example row key [metrictype]#[reversedtimestamp], which directly maps to the structure in the correct answer.
Source: Google Cloud Documentation, Cloud Bigtable > Documentation > Guides > Schema design for time series data.
2. Google Cloud Documentation, "Cloud Bigtable schema design": This guide outlines general best practices, including the preference for a single table over multiple small tables for related data. It notes, "In general, a schema with one table is the most efficient." This supports the "one unique table" approach in options A and B and refutes the multi-table approach in C and D.
Source: Google Cloud Documentation, Cloud Bigtable > Documentation > Guides > Schema design.
3. Designing Data-Intensive Applications by Martin Kleppmann (ISBN-13: 978-1449373320): While not a Google-specific source, this is a foundational academic text on data systems. Chapter 3 discusses SSTables and LSM-Trees, the underlying technology for Bigtable. It explains how data is stored sorted by key, which is the fundamental principle making a reverse-timestamp key efficient for "latest-first" queries, as it turns the query into a simple scan from the beginning of a key range.
Source: Chapter 3, "Storage and Retrieval", Section on "SSTables and LSM-Trees".
Keep practicing
Take the free PROFESSIONAL-DATA-ENGINEER practice test
Ten exam-style questions with answers and explanations, plus the exam facts and study guides.
More questions
Other PROFESSIONAL-DATA-ENGINEER practice questions
- Question 1You orchestrate ETL pipelines by using Cloud Composer One of the tasks in the Apache Airf…
- Question 2You want to rebuild your batch pipeline for structured data on Google Cloud You are using…
- Question 3You are designing an Apache Beam pipeline to enrich data from Cloud Pub/Sub with static r…
- Question 4You need to deploy additional dependencies to all of a Cloud Dataproc cluster at startup …
- Question 5A shipping company has live package-tracking data that is sent to an Apache Kafka stream …
- Question 6You use a dataset in BigQuery for analysis. You want to provide third-party companies wit…
