Datadog-APM-and-Distributed-Tracing-Fundamentals Practice Questions

Trace metrics (e.g., request.count, error.count, latency) are generated by the Agent from every span before any sampling or indexing decisions and are stored as standard custom metrics with Datadog’s default metric-retention schedule (high-resolution 30 days, down-sampled 15 months). Thus, none of the given statements accurately describe trace metrics.

A. They are based on all ingested spans, not only indexed traces.

B. Metric retention far exceeds 30 days; down-sampled data is kept 15 months.

C. Because metrics are produced before sampling, ingestion controls do not change them.

• Datadog Docs – Trace Metrics → “Computed by the Agent before sampling… metrics reflect 100 % of traffic.” (docs.datadoghq.com/tracing/tracemetrics/)

• Datadog Docs – Metrics Retention → “Metrics retained for 15 months.” (docs.datadoghq.com/metrics/retention/)

When you click on a span in the Trace Explorer, it opens the detailed Trace View panel. This view contains a flame graph of the entire trace and several tabs for deeper analysis. The "Errors" tab is specifically designed to aggregate all errors that occurred within that trace, displaying the error type, message, and a full stack trace. This is the most direct and comprehensive way to investigate the root cause of a specific error.

C. The span summary (or Metadata tab) shows tags and metadata, which might include an error flag, but the full message and stack trace are in the Errors tab.

D. The service map provides a high-level view of dependencies and health metrics, not the specific error message for an individual trace.

E. The service page aggregates data for the entire service. While it has an error section, it does not drill down to the specific error instance you selected.

1. Datadog Documentation, "Explore Traces". This page describes the user interface for trace investigation, showing that clicking a span opens the Trace View. Screenshots and descriptions highlight the "Errors" tab as the location for error messages and stack traces. (See section: "Trace View").

The firstPartyHosts property is used during the initialization of the RUM React Native SDK to identify which network requests are made to your own backend services. For any request matching a host in this list, the SDK automatically adds Datadog tracing headers. This allows the backend APM agent to link the server-side trace with the originating frontend user action, creating a complete distributed trace from the client to the backend.

This question was presented in a fill-in-the-blank format; therefore, no incorrect options were available for evaluation.

1. Datadog Documentation, "React Native Monitoring," Connect RUM and Traces. (https://docs.datadoghq.com/realusermonitoring/reactnative/datacollected/connectrumandtraces/). This page explicitly states: "To link RUM data with traces, you need to specify the hosts of your first party requests in the firstPartyHosts array during the SDK initialization."

2. Datadog Github Repository, dd-sdk-react-native, "Advanced Usage" documentation. The README.md file in the repository details the firstPartyHosts configuration option for enabling distributed tracing.

To identify the 100 restaurants with the most errors, you must first filter the traces to the relevant context: the specific service, the ordering resource, and only traces marked as errors. Then, to count errors per restaurant, you group the results by the restaurant name tag. Finally, visualizing this data as a "Top List" and limiting it to 100 entries directly answers the request by showing the restaurants with the highest error counts in descending order.

C. Grouping by HTTP Status Code would show the most frequent error codes, not the restaurants that experienced them most often.

D. Grouping by service and resource name would aggregate errors by code function, not by the affected restaurant entity.

1. Datadog Documentation, "Trace Explorer". In the "Analytics" section, it describes the process of creating queries, grouping by facets (tags), and using visualizations. The "Top List" visualization is explicitly for ranking items by a measure. (See section: "Visualize trace analytics").

2. Datadog Documentation, "Search and Analyze". This page details how to build a query by filtering on services and resources, then using the "Group by" field with a tag like restaurant name to aggregate results.

Creating a facet is not universally required for the actions listed. You can perform searching, generate metrics, and configure retention filters using default attributes like service, env, operationname, or error without creating any new facets. Facets are only required when you want to use a custom tag for these operations. Since the question asks what action requires creating facets, and all these actions can be performed in some capacity without creating new facets, none of the options are universally correct. Therefore, "None of the choices" is the most accurate answer based on the absolute language of the question.

A. You can search spans using default attributes like service or env which are indexed by default and do not require you to create a facet.

B. You can generate metrics from spans filtered or grouped by default attributes like service or resourcename without needing to create a new facet.

C. Retention filters can be configured based on default attributes such as service or operationname, which does not necessitate creating a facet first.

1. Datadog Documentation, "Search and Manage Spans". Section: Search Syntax. This section lists default attributes like service, env, and operationname that are searchable without user-created facets.

2. Datadog Documentation, "Generate Metrics from Spans". This guide shows that metrics can be created by filtering on default attributes like service, which do not require facet creation.

3. Datadog Documentation, "Retention Filters". The documentation shows that filter queries can be based on default attributes, such as service:my-backend, without creating a new facet.

Datadog APM retains all ingested spans for 15 days by default. For traces to be searchable and explorable in the Trace Explorer, their spans must be indexed by a retention filter. These indexed spans are also retained for 15 days. While extended retention is available as a paid add-on, the standard, default retention period is a fixed duration. Therefore, the most accurate and fundamental answer is 15 days, which is the baseline for all plans before any add-ons.

A. 7 days is incorrect. The standard retention period for ingested and indexed spans in Datadog APM is 15 days.

C. 1 month (or 30 days) is available only as a paid add-on for extended retention, it is not the standard duration.

D. While extended retention can be part of a contract, the fundamental retention period is a standard 15 days, making this answer less precise.

1. Datadog Documentation, "Trace Retention". In the section "Ingested Spans", it states: "Datadog APM retains all of your ingested spans for 15 days." In the section "Indexed Spans", it states: "Spans that are indexed by retention filters are retained for 15 days."

2. Datadog Documentation, "APM Pricing". The pricing page for both Pro and Enterprise plans explicitly lists "15-day trace & profile retention" as a standard feature.

When a trace with an error is selected from the Live Search view, a side panel opens. This panel contains an "Errors" tab. Clicking this tab displays the specific error message, type, and a full stack trace. This provides the most direct and detailed information about the root cause of the error, which is essential for debugging 500 errors. The span summary and service map provide contextual information but not the specific error details.

B. The span summary provides metadata like duration, host, and tags, but it does not contain the detailed error message or stack trace needed for debugging.

C. The service map shows dependencies and error propagation, which is useful for high-level analysis but does not reveal the specific reason for a single error instance.

1. Datadog Documentation, "Explore Traces". In the "Trace View" section, the documentation describes the side panel that appears when a trace is selected. It explicitly mentions the "Errors" tab for viewing "error messages, types, and stack traces". (Datadog APM > Traces > Explore Traces)

2. Datadog Documentation, "APM Terminology". Defines a span and its associated tags, which can include error.msg, error.stack, and error.type. The "Errors" tab in the UI is the dedicated view for these tags. (Datadog APM > Getting Started > APM Terminology)

The primary purpose of enabling tracer debug mode is for troubleshooting and diagnostics. When enabled, the tracer outputs verbose internal logs about its operations, such as its configuration, sampling decisions, and communication with the Datadog Agent. This detailed information is invaluable for diagnosing instrumentation problems, connectivity issues, or unexpected behavior. It can also be used to audit the trace data being generated by the application before it is sent, helping to verify custom instrumentation.

B. While enabling debug mode does increase the number of log events, this is the mechanism, not the purpose. The purpose is diagnostics.

C. Debug mode increases logging verbosity; it does not reduce the number of events.

D. Using debug mode in production is strongly discouraged due to the potential for significant performance overhead and excessive log volume. It is intended for temporary use during troubleshooting.

1. Datadog Documentation, "Python APM and Distributed Tracing," Advanced Configuration section. The documentation for the DDTRACEDEBUG environment variable states its purpose is to "Enable debug logging from the tracer." This logging is used for troubleshooting.

2. Datadog Documentation, "Java APM and Distributed Tracing," Troubleshooting section. This section frequently advises enabling debug logging (-Ddd.trace.debug=true) as a first step to "gather more information about what the tracer is doing" when diagnosing issues.

Datadog's platform is designed for unified observability, seamlessly correlating distributed traces with other key telemetry data. Traces can be linked to corresponding logs via injected trace IDs, to infrastructure metrics from the host or container, to code-level performance data via profiles, and to network communication data. This allows users to pivot between different data types from a single trace view, providing a comprehensive context for troubleshooting and performance analysis. This interconnectedness is a core value proposition of the Datadog APM.

A. This is incomplete. While traces correlate with profiles, they also correlate with logs, metrics, and network data, making "All choices" a better answer.

B. This is incomplete. While traces correlate with logs, they also correlate with profiles, metrics, and network data, making "All choices" a better answer.

C. This is incomplete. While traces correlate with network data, they also correlate with logs, metrics, and profiles, making "All choices" a better answer.

E. This is incomplete. While traces correlate with infrastructure metrics, they also correlate with logs, profiles, and network data, making "All choices" a better answer.

1. Datadog Documentation, "APM & Distributed Tracing," Section: "Full-stack visibility". States that Datadog APM provides "seamless navigation to correlated logs, runtime metrics, and more."

2. Datadog Documentation, "Connect Logs and Traces," Section: "Trace and Log Correlation". Explains how trace and span IDs are automatically added to logs for correlation.

3. Datadog Documentation, "Continuous Profiler," Section: "How Continuous Profiler Works with APM Tracing". Describes how profiles are linked to traces to identify code-level bottlenecks.

4. Datadog Documentation, "Network Performance Monitoring," Section: "Getting Started". Shows how network data is correlated with services and traces.

The Datadog Java tracer provides multiple ways to create new spans for custom instrumentation. Developers can use the @Trace annotation directly in their code to mark specific methods for tracing. Alternatively, they can configure the tracer to trace specific methods without code changes by using the dd.trace.methods system property or the DDTRACEMETHODS environment variable. Since both methods are valid and supported features, the correct choice encompasses both.

A. This is only partially correct. While tracing methods via configuration is a feature, it omits the equally valid annotation-based approach.

B. This is also only partially correct. Using trace annotations is a key feature, but it ignores the method of instrumenting via configuration.

D. This is incorrect because the Datadog Java tracer explicitly supports both trace annotations and method tracing through configuration for custom instrumentation.

1. Datadog Documentation, "Java Custom Instrumentation". Section: "Trace a method using annotations". This section details the use of the @Trace annotation.

2. Datadog Documentation, "Java Custom Instrumentation". Section: "Trace a method using configuration". This section explains how to use the dd.trace.methods property to instrument methods without code changes.

3. Datadog Documentation, "Configuration". Section: "Java". This page lists DDTRACEMETHODS as an environment variable for "A list of methods that should be traced."