TM12 Practice Questions

Demotivating factors for testers often stem from repetitive, unchallenging work and unfair performance metrics. Manually executing the same regression tests repeatedly (C) is tedious, lacks intellectual stimulation, and is inefficient, which can significantly lower morale. Using the number of customer-reported problems as a primary measure of test quality (E) is also highly demotivating. This metric unfairly places blame on the test team for defects that escape to production, ignoring factors like project constraints, inherent code quality, and the impossibility of exhaustive testing. It fosters a culture of blame rather than collaborative quality improvement.

A. Management staying informed about testing is a positive sign that the work is considered important and visible, which is typically motivating, not demotivating.

B. When a tester's recommendations are adopted, it demonstrates that their expertise is valued and their contributions have a tangible impact, which is a strong motivating factor.

D. Assessing testers on finding important failures is a core part of their role. While it can be implemented poorly, the principle itself recognizes their primary value contribution.

1. ISTQB® Advanced Level Syllabus, Test Manager (Version 2012), Section 5.2.3, "Individual Motivation." This syllabus, foundational for BCS certifications, explicitly lists "Repetitive work (e.g., running manual regression tests)" as a common demotivator for testers. This directly supports option C. The section also emphasizes the need for fair evaluation, contrasting with the blame-oriented metric described in option E.

2. Kaner, C., Bach, J., & Pettichord, B. (2002). Lessons Learned in Software Testing: A Context-Driven Approach. Wiley. In Chapter 2, "The Role of the Tester," the authors discuss various testing anti-patterns. Lesson 29, "Don't use bug counts as a measure of tester performance," explains how such metrics can be misleading and demotivating, creating perverse incentives. This reasoning directly supports why option E is a demotivating factor.

3. Kasurinen, J. (2017). Software Test Management. Springer International Publishing. In Chapter 4, "Test Team and People Management," Section 4.3, "Motivation," the author discusses that "monotonous and repetitive work" is a key reason for low motivation in testing teams. This provides academic support for option C being a significant demotivating factor.

Specifying test conditions at a detailed level creates a strong dependency on the test basis (e.g., requirements specifications). In an environment where this basis is continuously changing, detailed test conditions become a significant maintenance burden. Each change to the test basis would require a corresponding, and often complex, update to the detailed test conditions. Therefore, in such volatile environments, it is more practical to specify test conditions at a higher, more abstract level to improve maintainability. High-level conditions are less sensitive to minor changes in the underlying documentation.

B. When formal requirements are unavailable, the process of creating detailed test conditions (e.g., from stakeholder interviews or analysis of a legacy system) helps to formalize and document the expected behavior, providing a clear and explicit basis for testing.

C. To ensure consistency, clarity, and reusability of detailed test conditions across a team, a certain level of formality (e.g., using standardized templates, terminology, and review processes) is necessary.

D. Deriving detailed test conditions from the test basis early in the lifecycle is a form of static analysis. This process often reveals ambiguities, omissions, and inconsistencies in the requirements, allowing these defects to be fixed before they are implemented in code, thereby preventing defects.

---

1. ISTQB® Certified Tester Advanced Level Syllabus, Test Manager (Version 2012):

Section 4.2.3, Identifying Test Conditions: The syllabus discusses that the level of detail for test conditions depends on various factors. It implies that in unstable projects, high detail leads to high maintenance. It states, "The level of detail to which the test conditions should be specified depends on the test level, the level of detail of the test basis, the complexity of the system, the project and product risks, the software development lifecycle, the available tools, the skills and knowledge of the test team, and the level of maturity of the test and development processes." An unstable test basis (continuously changing) would necessitate a lower level of detail to manage maintenance effort.

2. Black, R. (2014). Advanced Software Testing - Vol. 2, 2nd Edition: Guide to the ISTQB Advanced Certification as an Advanced Test Manager. Rocky Nook.

Chapter 3, Test Analysis and Design, Page 96: The text explains the trade-offs in the level of detail for test documentation. It explicitly states, "If the requirements are unstable, the cost of maintenance for detailed test cases can be very high." This directly supports the reasoning that statement A is false, as detailed specification is not recommended for better maintainability in such a scenario.

3. Van Veenendaal, E. (Ed.). (2012). Improving the Test Process: Implementing Improvement and Change - A Study Guide for the ISTQB Expert Level Module. Rocky Nook.

Chapter 4, Assessing the Test Process: This book discusses process maturity and formality. It supports option C by explaining that consistent and repeatable processes, which are necessary for managing detailed test artifacts, require an appropriate level of formality and definition within the team. It also supports option D by linking early test analysis (a key part of defect prevention) to mature test processes.

A full risk-based testing strategy prioritizes test activities according to the identified level of product risk. The highest-risk items are tested first and most rigorously. The provided chart shows that 97% of risk items are covered by passed tests. Following the risk-based approach, these would be the highest and medium-risk items. The remaining 3% (2% covered by failed tests and 1% not yet covered) would logically correspond to the lowest-risk items. Therefore, a release can be considered, subject to a business decision on managing the known, low-risk failures and accepting the minimal risk of the untested items.

A. This is factually incorrect. The chart clearly shows 2% of items have failed tests and 1% are not yet covered by tests.

B. This contradicts the fundamental principle of risk-based testing, where high-risk items are prioritized and tested first, not last.

D. This is factually incorrect. The chart explicitly shows 2% of risk items are covered by failed tests, indicating open bugs or failures exist.

---

1. ISTQB Advanced Level Test Manager Syllabus (Version 2012). Section 3.2.2, "Using Risk Analysis to Determine Test Priority," states: "The risk rating is used to order the execution of the tests... Higher-risk test cases are tested first." This principle directly supports the conclusion that the remaining untested or failed items (the red and blue areas) are the lowest-risk ones.

2. Black, R. (2009). Pragmatic Software Testing: Becoming an Effective and Efficient Test Professional. In Chapter 10, "Risk-Based Testing," the author explains that the process involves identifying risks and then using that information to guide the testing effort. This guidance inherently means prioritizing testing of high-risk features, leaving lower-risk items for later in the cycle.

3. Felderer, M., & Ramler, R. (2016). A multiple case study on risk-based testing in industry. International Journal on Software Tools for Technology Transfer, 18(5), 505-523. Section 2.1, "Risk-Based Testing," describes the process: "The identified risks are used to select, prioritize, and/or design test cases." This academic source confirms that prioritization is a key output of risk analysis in testing, validating that the last items to be addressed are the lowest priority (i.e., lowest risk). DOI: https://doi.org/10.1007/s10009-015-0399-z

4. University of Minnesota, Course CSci 5801: Software Engineering I, Lecture Notes on Software Testing. Course materials typically explain that risk-based testing allocates the greatest test effort to high-risk components. This implies that as testing progresses and coverage increases, the remaining uncovered or failing items are those that were initially assessed as having the lowest risk.

The exit criterion EX1 states, "100% of tests of priority 1 have been executed and passed." By analyzing the provided table, we can identify a total of seven test cases with Priority 1: TC-001, TC-002, TC-003, TC-004, TC-005, TC-006, and TC-007.

The scenario specifies that three of these Priority 1 test cases (TC-001, TC-002, and TC-007) have already been executed and passed. To satisfy the 100% completion requirement of EX1, the remaining Priority 1 test cases must also be successfully executed. The remaining test cases are TC-003, TC-004, TC-005, and TC-006, which amounts to a minimum of 4 test cases.

B. 5: This is an incorrect calculation of the remaining Priority 1 test cases required to meet the exit criterion.

C. 6: This is an incorrect calculation of the remaining Priority 1 test cases required to meet the exit criterion.

D. 7: This represents the total number of Priority 1 test cases, not the number remaining to be executed to meet the criterion.

1. ISTQB® Foundation Level Syllabus (Version 2018 V3.1.1): Section 5.2.4, "Test Plan," defines Exit Criteria as "The set of generic and specific conditions... for permitting a process to be officially completed." The question requires applying this principle by evaluating progress against a specific exit criterion (EX1). The calculation involves determining the remaining work to satisfy the condition of "100% of tests of priority 1 have been executed and passed."

2. Kaner, C., Falk, J., & Nguyen, H. Q. (1999). Testing Computer Software. John Wiley & Sons. In Chapter 8, "Test Planning," the concept of "Release Criteria" or "Exit Criteria" is discussed. The text emphasizes that these criteria are specific, measurable conditions that must be met, such as "all high-priority tests must pass." The problem presented is a direct application of this test management principle.

3. MIT OpenCourseWare, 6.170 Software Studio, Spring 2013: Lecture 10, "Testing," discusses the components of a test plan, including test completion criteria. These criteria define when testing is considered finished, for example, when a certain percentage of test cases have passed. The question assesses the ability to calculate progress towards such a pre-defined completion criterion. (Available at: https://ocw.mit.edu/courses/6-170-software-studio-spring-2013/)

To determine the most risky subsystem, the total risk score for each must be calculated by summing the individual risk scores from the three categories provided: 'Failed Tests', 'Not Run Tests', and 'Open Bugs'. This aggregate score represents the overall quality risk for each component.

The calculations are as follows:

SOA: 1.2 + 0.8 + 1.5 = 3.5

CRM: 2.5 + 1.2 + 2.1 = 5.8

BRM: 1.8 + 0.5 + 1.1 = 3.4

SMS: 0.9 + 0.2 + 0.7 = 1.8

Based on these totals, the CRM subsystem has the highest aggregate risk score of 5.8, making it the most risky subsystem.

A. SOA: The total risk score for SOA is 3.5, which is lower than the CRM's score of 5.8.

C. BRM: The total risk score for BRM is 3.4, which is lower than the scores for both SOA and CRM.

D. SMS: The total risk score for SMS is 1.8, the lowest of all subsystems, indicating it is the least risky.

1. Black, R. (2014). Advanced Software Testing - Vol. 2: Guide to the ISTQB Advanced Certification as an Advanced Test Manager (2nd ed.). Rocky Nook. In Chapter 3, "Risk-Based Testing," the methodology of identifying, analyzing, and quantifying risks is detailed. The process involves assigning a level of risk (e.g., a risk score or priority number) to test items, which can then be aggregated to assess the overall risk of a component or system, aligning with the summation method used here.

2. Software Engineering Institute, Carnegie Mellon University. (1996). Continuous Risk Management Guidebook. (CMU/SEI-96-HB-001). Chapter 4, "Analyze," discusses the process of converting risk information into a form that allows for prioritization. This involves evaluating risks in terms of their probability and impact, often resulting in a quantitative score that can be summed or aggregated to compare different areas of a project, as demonstrated in the question's table.

This option directly addresses all of the senior management's objectives. Planning for "formal exit-phase reviews" satisfies the need for a well-documented, formal bug-removal activity with defined roles. The process of analyzing bugs to "determine the phase in which they have been introduced" is the exact mechanism required to measure and subsequently improve "phase containment effectiveness." By identifying where defects originate, the team can enhance the review and testing processes in those specific early phases. This shift-left approach of finding and fixing defects earlier is a proven method for reducing the overall costs associated with later-stage dynamic testing.

A. "Lightweight" reviews contradict the specific objective for a "formal process with well-defined roles." Lightweight reviews lack the rigor needed for the specified bug-removal and documentation goals.

B. While proposing formal reviews is correct, analyzing defects only to identify the "larger cluster of defects" (defect clustering) does not directly address the objective of measuring phase containment effectiveness.

C. This option is fundamentally flawed because it promotes a blame culture by focusing on "the people responsible for those bugs." Effective process improvement focuses on identifying and fixing flaws in the process, not on blaming individuals.

1. Pressman, R. S., & Maxim, B. R. (2020). Software Engineering: A Practitioner's Approach (9th ed.). McGraw-Hill.

Chapter 17, Section 17.5, "The Defect Amplification Model": This section explains how defects found later in the process are more costly to correct. It states, "A key goal of software quality assurance is to find and correct defects as early as possible... The model can be used to illustrate the cost-effectiveness of an early SQA activity like formal technical reviews." This supports the objective of reducing dynamic testing costs by using effective reviews.

Chapter 20, Section 20.3, "Formal Technical Reviews": This section details the structure of formal reviews, including defined roles, a specific agenda, and follow-up verification, which aligns with the management objective for a "formal process with well-defined roles."

2. Kan, S. H. (2003). Metrics and Models in Software Quality Engineering (2nd ed.). Addison-Wesley.

Chapter 4, "In-Process Metrics for Software Testing," Section on "Defect Removal Effectiveness": This chapter defines Defect Removal Effectiveness (DRE), a key metric for phase containment. It is calculated as DRE = (Defects removed during a phase) / (Defects latent in the product at the beginning of the phase + Defects injected during the phase). To calculate this, one must track the phase in which defects are introduced and the phase in which they are found, directly supporting the analysis described in option D.

3. MIT OpenCourseWare. (2005). 6.170 Laboratory in Software Engineering, Fall 2005. Massachusetts Institute of Technology.

Lecture 15 Notes, "Testing II": The lecture notes discuss the economics of testing, emphasizing that the cost to fix a bug rises dramatically the later it is found. The notes state, "Find bugs early... Reviews, inspections, and walkthroughs can find bugs before code is ever run." This reinforces the principle that using formal reviews to improve phase containment (finding bugs in the phase they are introduced) is the most effective way to reduce overall project costs.

The scenario describes activities related to several phases of the tool's lifecycle. The selection process covers Acquisition. The creation of backup/restore procedures and standards for managing assets covers Support and Maintenance. Analyzing data export formats for future migration explicitly addresses the Retirement phase.

However, the 'Evolution' phase is not adequately considered. This phase involves the ongoing adaptation and growth of the tool's usage, such as rolling it out to other projects beyond the pilot, integrating it with other tools in the software development lifecycle (e.g., CI/CD pipelines), and planning for version upgrades. The description focuses on the initial setup and eventual replacement, omitting the strategy for its growth and adaptation over time.

A. Acquisition: This phase is explicitly mentioned in the first sentence: "After a selection process you have selected a test management tool..."

B. Support and maintenance: This is covered by the statement about devising "standard ways of managing, storing and maintaining the tool and its assets including backup/restore procedures."

D. Retirement: This phase is considered by analyzing "standard formats... to export, import and archive all the information... to minimize the impacts of migrating this information to a new tool."

---

1. ISTQB Advanced Level Syllabus, Test Manager (Version 2012), Section 6.2, "Introducing a Tool into an Organization." This section outlines the lifecycle of tool implementation, including the pilot project (6.2.2), deployment (6.2.3), and defining usage guidelines (6.2.4). The scenario covers these initial steps but lacks consideration for the ongoing evolution, such as adapting the tool for new projects or integrating it further into the organization's processes, which is a critical part of its long-term lifecycle.

2. Black, R. (2014). Advanced Software Testing - Vol. 2: Guide to the ISTQB Advanced Certification as an Advanced Test Manager (2nd ed.). Rocky Nook. In Chapter 4, "Test Tools and Automation," the tool lifecycle is discussed. The text differentiates between initial deployment and ongoing support, which includes "upgrading the tool and test assets as new versions are released" and adapting its use. The scenario's focus on static maintenance (backups) and retirement planning neglects this dynamic 'Evolution' aspect.

The process described is a textbook application of Failure Mode and Effect Analysis (FMEA). This technique systematically identifies potential "failure modes" (ways a system can fail), analyzes their "effects," and determines their root "causes." A key component of FMEA is the calculation of a Risk Priority Number (RPN) to prioritize risks. The question explicitly states this calculation as the product of three factors: severity, occurrence (likelihood), and detection. This structured, bottom-up approach is characteristic of FMEA and is widely used for risk analysis in safety-critical systems, as described in the scenario.

A. The described method is a formal and structured technique (FMEA), not a generic or informal "lightweight" approach.

C. Wide Band Delphi is an expert-based estimation technique for forecasting, not a method for analyzing system failure modes.

D. Cost of Exposure calculates risk as the product of failure probability and cost, not the three-factor (S x O x D) model described.

1. Teoh, P. C., & Case, K. (2004). Failure modes and effects analysis through knowledge modelling. Journal of Materials Processing Technology, 153-154, 253-260. (See Section 2, "Failure Modes and Effects Analysis," which describes the RPN calculation as Severity × Occurrence × Detection). https://doi.org/10.1016/j.jmatprotec.2004.04.258

2. University of Washington, Department of Mechanical Engineering. (n.d.). ME 395: Introduction to Mechanical Design - Failure Modes and Effects Analysis (FMEA). Courseware. (See slides 5-10, which detail the FMEA process and define the Risk Priority Number (RPN) as the product of Severity (S), Occurrence (O), and Detection (D)).

3. Stamatis, D. H. (2003). Failure Mode and Effect Analysis: FMEA from Theory to Execution (2nd ed.). ASQ Quality Press. (See Chapter 4, "The FMEA Process," which details the calculation of the RPN from Severity, Occurrence, and Detection ratings).

The scenario explicitly confirms the presence of management support, participant involvement, and training. A business case was examined, and the pilot was effective. However, a critical success factor for the long-term institutionalization of a new process is the ongoing measurement of its value. The scenario does not mention the definition and systematic use of metrics to calculate and demonstrate the Return on Investment (ROI). While the pilot was successful, sustaining management support and optimizing the process requires quantitative proof of its benefits, which is achieved through metrics.

A. Management support is explicitly stated in the scenario ("the management finally decides to adopt formal reviews").

B. Participant support is clearly described as being present ("positive involvement from all the participants").

D. Training is explicitly mentioned as having been provided ("spent to provide training in the review process").

---

1. ISTQB Advanced Level Syllabus, Test Manager (Version 2012): Section 3.4, "Introducing Reviews," lists critical success factors for implementing a review process. It states, "The following are critical success factors for a review program... Management support... A pilot project... Training for participants... The definition and use of metrics for the review process to measure the return on investment (ROI)." The scenario covers all listed factors except the ongoing definition and use of metrics to measure ROI.

2. Black, R. (2014). Advanced Software Testing - Vol. 2: Guide to the ISTQB Advanced Certification as an Advanced Test Manager (2nd ed.). Rocky Nook. In Chapter 2, "Reviews," the section on "Implementing Reviews" emphasizes creating a business case. It details the importance of gathering metrics to calculate ROI, stating, "To make the business case for reviews, you should gather metrics on the costs and benefits of reviews... These metrics can be used to calculate the return on investment (ROI) for reviews." This highlights that measuring ROI is a distinct and crucial step beyond simply getting initial approval.

3. van Veenendaal, E. (Ed.). (2012). The Little Black Book on Test Process Improvement. TMap. Chapter 4, "Implementing Improvements," discusses the importance of measurement to anchor changes within an organization. It explains that after a pilot, "measurement is essential to show the results and benefits of the improvement." This supports the idea that without formal metrics, the success of the pilot cannot be translated into a sustained, organization-wide practice.