QSDA2024 practice question 9 of 10

A data architect needs to acquire social media data for the past 10 years. The data architect needs to track all changes made to the source data, include all relevant fields, and reload the application four times a day. What information does the data architect need?

Full set: 50 questions
Answer options
A: A field with social media source, a set of key fields to sort out updated records, configure reload task to load four times a day
B: A field with record creation time, a secondary key field to remove deleted records, configure reload task to load four times a day
C: A field with ModificationTime, a primary key field to sort out updated records, insert and append records, update records
D: A field with ModificationTime, a primary key field to sort out updated records, insert and update records, remove records Correct answer
Correct Answer:
D. A field with ModificationTime, a primary key field to sort out updated records, insert and update records, remove records
Explanation

To implement a robust incremental load strategy that tracks all changes, a data architect requires specific information and capabilities. A ModificationTime field is essential to efficiently identify which records have been created or updated since the last data load. A primary key field is non-negotiable, as it provides a unique identifier for each record, enabling the script to correctly apply updates to existing records and identify which records to remove. Finally, the overall process must account for all three types of data modifications: inserting new records, updating changed records, and removing records that have been deleted from the source system to ensure the Qlik application remains a true reflection of the source.

Why Incorrect

A. This option lacks a modification timestamp, which is the most efficient way to identify changed records. Relying only on key fields to "sort out" updates is insufficient and inefficient.
B. Using only a "record creation time" fails to capture any subsequent updates to a record. The method for handling deletes ("secondary key field") is also non-standard and incorrect.
C. This option is close but incomplete. While it correctly identifies the need for a modification time and primary key, it omits the crucial step of removing records deleted from the source.

References

1. Qlik Sense on Windows Help, "Loading new and updated records with incremental load": This official documentation outlines the complete strategy. It explicitly states the need for a field indicating when data was last modified (e.g., ModificationTime). It also details the process for handling inserts, updates (by using a primary key to find and replace records), and deletes (by loading all source keys and removing records from the existing data file that no longer exist in the source).

Section: "Deleting records that have been removed from the source database"

Section: "Example - incremental load"

2. Qlik Sense on Windows Help, "Strategies for managing changing data": This document discusses various data management techniques, highlighting incremental loading as a key strategy. It emphasizes using a date field to select new records, which aligns with the need for a ModificationTime.

Section: "Incremental load"

Scroll to Top