CT-UT Practice Questions

A well-defined usability task should be realistic, goal-oriented, and allow for the observation of a user's natural interaction with the system. Option B provides a clear scenario with context (broken phone) and constraints (budget, feature), which represents a typical user's goal. This task structure enables the examiner to assess the entire process from searching and filtering to completing the order, directly addressing the test's main objective of evaluating the ease of placing an order. It avoids prescribing specific steps, which is crucial for identifying potential usability issues.

A: This task is too simplistic and not representative of a real user's goal-oriented shopping behavior.

C: This is a prescriptive script, not a usability task. It tells the user how to act, preventing genuine usability evaluation.

D: This is a background or preference question, not an interactive task that tests the system's functionality or flow.

1. Dumas, J. S., & Redish, J. C. (1999). A practical guide to usability testing. Intellect books. In Chapter 6, "Developing the Test Plan," the authors state that tasks should be "framed as work goals, not as instructions on how to use the interface" and must be "representative of the work users do" (pp. 116-119). This supports option B's scenario-based approach and refutes option C's instructional nature.

2. Rubin, J., & Chisnell, D. (2008). Handbook of Usability Testing: How to Plan, Design, and Conduct Effective Tests (2nd ed.). Wiley. Chapter 6, "Developing the Test Plan," in the section "Creating the Test Scenarios," explains that scenarios should describe a user's motivation and goal, not the interface steps (pp. 128-131). This aligns directly with the structure of option B.

3. ISO 9241-210:2019, Ergonomics of human-system interaction — Part 210: Human-centred design for interactive systems. Section 5.2, "Understand and specify the context of use," highlights the importance of defining tasks based on user goals within a realistic context, which is the core principle demonstrated in option B.

This option provides the most accurate and complete summary of the standard process for conducting a user survey in usability testing. The process begins with planning and stakeholder interviews to define goals. It is followed by selecting or designing the questionnaire, deploying it to users, analyzing the collected data, and finally, communicating the findings. This sequence represents a logical, end-to-end workflow, capturing all critical phases from inception to reporting.

A. This option omits the crucial step of deploying the questionnaire. It incorrectly jumps from recruiting users directly to analyzing results without mentioning data collection.

C. This option is fundamentally flawed as it omits the "analyze" step. Communicating results is impossible without first analyzing the data collected from the survey.

D. Similar to option C, this option also omits the essential "analyze" step, making the summarized process incomplete and illogical.

1. ISTQB® Certified Tester Usability Testing Syllabus (Version 1.0, 2018): Section 4.3.3, "Surveys," outlines the process which includes: 1. Plan the survey, 2. Select or create the questionnaire, 3. Recruit participants, 4. Deploy the questionnaire, 5. Analyze the results, and 6. Communicate the findings. Option B is the closest representation of this official process among the choices, as it correctly includes the critical sequence of deployment, analysis, and communication, which other options fail to do.

2. Lazar, J., Feng, J. H., & Hochheiser, H. (2017). Research Methods in Human-Computer Interaction (2nd ed.). Morgan Kaufmann. Chapter 5, "Surveys," details the process which involves identifying goals and the user population (planning/interviews), developing questions (selecting a questionnaire), distributing the survey (deployment), and analyzing the results. This academic text validates the core sequence presented in option B. (DOI not applicable for the entire book, but it is a standard university textbook).

3. Sauro, J., & Lewis, J. R. (2016). Quantifying the User Experience: Practical Statistics for User Research (2nd ed.). Morgan Kaufmann. Chapter 3, "Standardized Usability Questionnaires," discusses the process of selecting, administering (deploying), and analyzing data from surveys like the SUS, which aligns with the steps in option B. The chapter emphasizes the sequence of selection, deployment, and analysis as fundamental.

A usability requirement is a non-functional requirement that specifies the extent to which a product can be used by specified users to achieve specified goals with effectiveness, efficiency, and satisfaction in a specified context of use. It defines measurable criteria for the ease of use and user-friendliness of a system or component. These requirements are essential for guiding design and for establishing the basis against which the final product's usability will be formally tested and validated. They are distinct from functional requirements, which define what the system does.

A. This describes a detail of a test environment or context of use, not the definition of a usability requirement for the system itself.

B. This refers to the qualifications or prerequisites for the person conducting the test, not a quality attribute of the system under test.

C. This describes a requirement for the test process or methodology, not a requirement for the product's usability.

1. ISTQB® Certified Tester Foundation Level - Usability Testing Syllabus (Version 2018). Section 1.2, "What is Usability?", defines usability based on the ISO 9241-11 standard, stating it is the "extent to which a system, product or service can be used by specified users to achieve specified goals with effectiveness, efficiency and satisfaction in a specified context of use." A usability requirement is a formal statement of these desired characteristics.

2. ISO/IEC/IEEE 29148:2018, "Systems and software engineering — Life cycle processes — Requirements engineering." Section 6.3.3 discusses quality requirements, including usability. It specifies that such requirements should define the performance of the system from the user's perspective, directly supporting that a usability requirement is placed on the system.

3. Sommerville, I. (2011). Software Engineering (9th ed.). Pearson. In Chapter 4, "Requirements Engineering," Section 4.2.2, "Non-functional requirements," usability requirements are classified as product requirements that specify how well the system should perform regarding its ease of use for operators.

The System Usability Scale (SUS) is a standardized, post-test questionnaire designed to measure the subjective perception of usability. Developed by John Brooke in 1986, it consists of a 10-item questionnaire with five response options for respondents, based on a Likert scale. It is technology-agnostic and provides a single, global score representing a composite measure of perceived usability. Its simplicity, reliability, and validity have made it a widely adopted industry standard for collecting user feedback on a system's ease of use.

A. This is the formal definition of usability testing itself, as outlined in the ISO 9241-11 standard, not the SUS tool.

B. This describes a usability test task or scenario, which is an activity a participant performs, not a measurement instrument.

D. This defines a usability requirement, which is a criterion or constraint specified for a system, not a method for evaluation.

1. Brooke, J. (1996). SUS: A "quick and dirty" usability scale. In P. W. Jordan, B. Thomas, B. A. Weerdmeester, & A. L. McClelland (Eds.), Usability Evaluation in Industry (pp. 189-194). Taylor & Francis.

Reference Point: In the abstract and introduction (p. 189), Brooke describes SUS as a "ten-item scale giving a global view of subjective assessments of usability."

2. Bangor, A., Kortum, P. T., & Miller, J. T. (2008). An empirical evaluation of the System Usability Scale. International Journal of Human-Computer Interaction, 24(6), 574-594. https://doi.org/10.1080/10447310802205776

Reference Point: The abstract (p. 574) states, "The System Usability Scale (SUS) is a simple, ten-item attitude Likert scale giving a global view of subjective usability."

3. Sauro, J., & Lewis, J. R. (2012). Quantifying the User Experience: Practical Statistics for User Research. Morgan Kaufmann.

Reference Point: Chapter 8, "Standardized Usability Questionnaires," describes the SUS as a "ten-item instrument for the subjective assessment of usability" (p. 186).

Given the project is in an early stage with a "rudimentary prototype," a usability review is the most appropriate and efficient approach. This method, also known as an expert review or heuristic evaluation, allows the usability expert to leverage their knowledge to inspect the prototype against established usability principles. It is a cost-effective method that provides rapid, actionable feedback to the development team. This is ideal for early-stage artifacts where formal user-based testing would be premature and less efficient. The established positive relationship with the development team further enhances the effectiveness of this collaborative, expert-driven feedback loop.

B. Unmoderated usability test: This method requires a more stable and functional prototype to allow users to complete tasks independently without real-time guidance.

C. Usability maturity assessment: This evaluates an organization's overall usability processes and capabilities, not the usability of a specific product prototype.

D. Usability test in the lab: This is a formal, resource-intensive method better suited for more mature, interactive prototypes or finished products, not a rudimentary one.

---

1. ISTQB® Certified Tester Foundation Level Usability Testing Syllabus (Version 2018).

Section 2.2.1, Usability Review: "Usability reviews can be applied at any stage of the development lifecycle, from early concepts and prototypes to the finished product... Usability reviews are relatively inexpensive to perform compared to usability tests." This supports using reviews on early prototypes.

Section 2.2.2, Usability Testing: This section describes methods involving end-users (like lab tests), which generally require more mature artifacts and are more resource-intensive than expert reviews.

2. Nielsen, J. (1994). Heuristic evaluation. In J. Nielsen & R. L. Mack (Eds.), Usability inspection methods (pp. 25–62). John Wiley & Sons.

Chapter 2, Heuristic Evaluation: This foundational text establishes heuristic evaluation (a type of usability review) as a "discount usability engineering method." It emphasizes that it is fast, inexpensive, and effective for finding usability problems early in the design process, even with low-fidelity prototypes.

3. Lazar, J., Feng, J. H., & Hochheiser, H. (2017). Research Methods in Human-Computer Interaction (2nd ed.). Morgan Kaufmann.

Chapter 8, Usability Testing: This chapter contrasts expert reviews with user-based testing, noting that "Expert reviews can be useful at any stage of the development process, but they are especially useful at the early stages of design, when it may be too early to get meaningful feedback from users." (p. 201).

The user's suggestion for a new feature, like a quick search box, is not a direct problem with the existing interface but an idea for enhancement. In the context of usability testing, such findings are classified as "good ideas" or "user suggestions." They are valuable input for future development but are distinct from usability problems, which are issues that hinder users from completing tasks with the current design. This classification helps teams prioritize fixes for existing problems separately from considering new features.

A. Positive finding: This would be a comment on something the user liked or found easy to use in the current system, not a suggestion for something new.

B. Usability problem: This is an obstacle that makes the system difficult to use. The suggestion is a potential solution, not the problem itself.

D. Functional problem: This refers to a feature that is broken or does not work as specified, which is not the case here.

1. ISTQB® Certified Tester Foundation Level Usability Testing Syllabus (Version 2018):

Section 4.3.1, The Usability Test Report, page 21: "The report should distinguish between findings that are usability problems and other findings such as positive findings and user suggestions (sometimes called 'good ideas')." This directly supports classifying the user's suggestion as a "good idea."

2. Dumas, J. S., & Loring, B. A. (2008). Moderating Usability Tests: Principles and Practices for Interacting. Morgan Kaufmann.

Chapter 10, Analyzing and Reporting Data, Section "Types of Findings", pp. 258-259: The authors differentiate between "Problems" (issues that impede task completion) and "Suggestions for Improvement." The user's idea for a search box falls squarely into the latter category, which is synonymous with a "good idea."

3. Rubin, J., & Chisnell, D. (2008). Handbook of Usability Testing: How to Plan, Design, and Conduct Effective Tests (2nd ed.). Wiley Publishing, Inc.

Chapter 12, Compiling and Analyzing Data, p. 299: The text advises categorizing observations and user comments, distinguishing between "usability findings (problems)" and "user suggestions or requests for new features." This aligns with the principle of separating identified problems from enhancement ideas.

The Equality Act 2010 is a key piece of UK legislation that consolidates and replaces previous anti-discrimination laws. Its primary purpose is to legally protect people from discrimination in the workplace and in wider society. It covers "protected characteristics," including disability. For digital services, this means organizations have a legal duty to make "reasonable adjustments" to ensure their websites and apps are accessible to people with disabilities, as failure to do so can be considered a form of discrimination.

A. The Act requires "reasonable adjustments," which considers the cost and practicality of changes, not accessibility "at all costs."

C. The Act does not explicitly name or mandate adherence to any specific technical standard, including WCAG conformity level AA.

D. The Act does not explicitly name or mandate adherence to any specific technical standard, including WCAG conformity level A.

1. UK Government. (2010). Equality Act 2010. legislation.gov.uk. Part 2, Chapter 2, Section 20 outlines the "Duty to make adjustments," which is the core of the accessibility requirement without prescribing specific technical standards. Part 3, Section 29 covers the provision of services.

2. International Software Testing Qualifications Board (ISTQB®). (2018). Certified Tester Usability Testing Syllabus. Version 2018. Section 3.3, "Accessibility Standards and Guidelines," p. 21. The syllabus notes, "In some countries, accessibility is a legal requirement (e.g., the Americans with Disabilities Act (ADA) in the USA, or the Equality Act in the UK)." This frames the Act as a legal requirement against discrimination, not a technical standard.

3. University of Edinburgh. (n.d.). UK law and digital accessibility. Website and Communications. Retrieved from https://www.ed.ac.uk/information-services/websites/accessibility/uk-law. The guidance states, "The Equality Act 2010 says that you have to make ‘reasonable adjustments’ for disabled people... It does not lay out specific technical requirements for websites to follow."

The Web Content Accessibility Guidelines (WCAG) are built on four principles: Perceivable, Operable, Understandable, and Robust. Within these principles, WCAG specifies success criteria for various elements. It requires resizable text (SC 1.4.4), text alternatives for non-text content (SC 1.1.1), and full keyboard accessibility (Guideline 2.1). It also addresses pointer inputs (like a mouse) under Guideline 2.5, covering aspects like pointer gestures and target size. WCAG does not, however, mandate subjective aesthetic qualities like a "visually appealing design"; its focus is on functional accessibility, not on visual appeal.

A. This option incorrectly states that keyboard accessibility (iv) and mouse input (v) are false; both are explicitly addressed in WCAG.

B. This option incorrectly includes "visually appealing design" (ii) as a WCAG element and excludes keyboard and mouse input (iv, v).

C. This option incorrectly excludes resizability of text (i) and text alternatives (iii), which are fundamental WCAG requirements.

1. W3C, Web Content Accessibility Guidelines (WCAG) 2.1, "Guideline 1.1 Text Alternatives". This guideline's first success criterion, SC 1.1.1, requires text alternatives for any non-text content. (Covers element iii).

2. W3C, Web Content Accessibility Guidelines (WCAG) 2.1, "Success Criterion 1.4.4 Resize text". This criterion mandates that text can be resized up to 200 percent without loss of content or functionality. (Covers element i).

3. W3C, Web Content Accessibility Guidelines (WCAG) 2.1, "Guideline 2.1 Keyboard Accessible". This guideline states, "Make all functionality available from a keyboard." (Covers element iv).

4. W3C, Web Content Accessibility Guidelines (WCAG) 2.1, "Guideline 2.5 Input Modalities". This guideline makes it easier for users to operate functionality through various inputs beyond a keyboard, including pointer inputs (mouse, touch). For example, SC 2.5.1 addresses "Pointer Gestures". (Covers element v).

5. W3C, Web Content Accessibility Guidelines (WCAG) 2.1, "Introduction". The introduction and principles focus on functional access for people with disabilities, not on subjective aesthetic design. The term "visually appealing" is not a testable criterion within the guidelines. (Covers element ii).

The fundamental difference between an informal usability review and an expert usability review is the required qualification of the reviewers. An expert review, such as a heuristic evaluation, mandates that the reviewers are specialists in usability, human-computer interaction (HCI), and user experience. These experts use established principles (heuristics) to systematically identify potential usability issues. In contrast, an informal review does not require participants to possess formal usability qualifications. It can be conducted by project team members, stakeholders, or peers to gather quick feedback, making it a less rigorous but often faster evaluation method.

A. An expert review is typically more structured, but the defining characteristic is the reviewer's expertise, not the formality of the process itself.

C. Both review types involve professional judgment, but expert reviews are based on established usability principles (heuristics), not just arbitrary opinion.

D. The number of reviewers is not a distinguishing factor; both review types can be performed by one or more individuals, though multiple are recommended.

---

1. ISTQB® Certified Tester Usability Testing Syllabus (Version 1.0, 2018):

Section 3.2.2.3, Expert review: "An expert review (also known as a heuristic evaluation) is a more formal method of review where a number of usability experts review a user interface to check for problems against a list of recognized usability principles (the “heuristics”)."

Section 3.2.2.2, Informal review: "Reviewers may be colleagues from the same team, from other teams, or anyone from the organization." This section does not specify any required qualifications, contrasting with the "usability experts" required for an expert review.

2. Nielsen, J. (1994). Heuristic evaluation. In J. Nielsen & R. L. Mack (Eds.), Usability Inspection Methods (pp. 25-62). John Wiley & Sons.

Chapter 2, Page 29: Nielsen specifies that for a heuristic evaluation (a type of expert review), "the evaluators should be usability specialists." This foundational text emphasizes that the expertise of the evaluator is a critical component of the method, distinguishing it from reviews conducted by non-specialists.

3. Hartson, H. R., & Pyla, P. S. (2012). The UX Book: Process and Guidelines for Ensuring a Quality User Experience. Morgan Kaufmann.

Chapter 19, Section 19.2.1, "Expert Reviews": The text states, "Expert reviews, also called usability inspections, are a collection of evaluation methods in which an expert in usability and user experience (UX) evaluates a user interface against a set of guidelines or principles." This directly supports that the reviewer's qualification is the key differentiator.

The primary distinction between formative and summative usability evaluation lies in their goals and timing within the development lifecycle. Formative evaluation is conducted during the design and development process. Its main purpose is to identify usability issues and provide feedback to "form" or improve the product. In contrast, summative evaluation is typically performed at the end of a development cycle on a finished or near-finished product. Its goal is to "sum up" or assess the overall usability, often by measuring it against predefined benchmarks or competitors, thereby judging the final outcome.

A. This option incorrectly reverses the definitions. Formative evaluations focus on improvement, while summative evaluations assess the final outcome.

B. This is an incorrect generalization. Both evaluation types can use either experts or users; the choice of participants is method-dependent, not type-dependent.

D. This is also an incorrect generalization. For instance, formative evaluations frequently use informal user tests, while summative evaluations can employ expert-based methods.

---

1. ISTQB® Certified Tester Usability Testing Syllabus (Version 1.0, 2018).

Section 3.2.1, Formative Evaluation: "The main goal of a formative evaluation is to find usability problems in a product and to get recommendations for the improvement of the product." (p. 20)

Section 3.2.2, Summative Evaluation: "The main goal of a summative evaluation is to assess the level of usability of a product. A summative evaluation is often used to determine if a product meets its usability goals..." (p. 21)

These sections directly support that formative evaluation is for improvement and summative evaluation is for assessment of the outcome.

2. Preece, J., Rogers, Y., & Sharp, H. (2019). Interaction Design: Beyond Human-Computer Interaction (5th ed.). Wiley.

Chapter 13, Evaluation Studies: From Controlled to Natural Settings: The text distinguishes between the two by stating, "Formative evaluations are done to help designers improve a design... Summative evaluations are done to assess the quality of a finished product." (p. 418). This aligns perfectly with the correct answer.

3. Sauro, J., & Lewis, J. R. (2016). Quantifying the User Experience: Practical Statistics for User Research (2nd ed.). Morgan Kaufmann.

Chapter 1, Introduction: The authors define the terms: "Formative evaluations are done as part of an iterative design process... The goal of a formative evaluation is to find and fix problems. In contrast, the goal of a summative evaluation is to describe the user experience of a finished product..." (p. 6). This confirms that formative focuses on improvement ("fix problems") and summative on assessment ("describe the user experience of a finished product"). DOI: https://doi.org/10.1016/C2014-0-03229-9