Pass the ISTQB AI Testing CT-AI Questions and answers with ExamsMirror

Section4.2 – Test Coverage Criteria for AI Modelsof the ISTQB CT-AI syllabus describes neural network–specific coverage methods. Among the techniques,threshold coverageis explicitly noted asadaptable, meaning testers may choose different thresholds to determine whether neuron activation is considered “covered.” This flexibility makes threshold coverage adjustable to the model architecture, problem domain, and required test thoroughness.

Options A and B (Sign–Sign and Sign–Change coverage) are more rigid structural criteria and are not described as adaptable within the syllabus. They focus on sign patterns of neuron activations and do not allow altering thresholds. Option D, neuron coverage, measures the proportion of neurons activated at least once. Although simple, it is not defined as an adaptable criterion. Its limitations are documented: it provides shallow insight and too easily achieves high coverage.

Onlythreshold coverageallows testers to adjust activation thresholds for more refined coverage measurement, makingOption Cthe correct choice.

The ISTQB CT-AI syllabus (Section5.3 – AI Support for Defect Analysis) explains that AI can categorize defect reports using natural language processing or classification models. Categorization helps route defects efficiently and determine which areas of the system are affected. Thus, OptionCis correct: AI canidentify defect categories, supporting assignment and triage.

Option A is incorrect because ML cannot infer severity or consequence without domain-specific risk modeling. Option B inaccurately ties duplicate detection to criticality; the syllabus separatesduplicate detectionfromseverity analysis. Option D is incorrect because AI may assist in routing defects using categories, but the syllabus does not claim AI identifies specific developers.

Thus,Option Cis the syllabus-accurate statement.

A. Identifying suitable tests by looking at the complexity of the test cases.

While complexity analysis can help in selecting important test cases, it does not directly address the issue of optimizing the entire regression suite effectively.

B. Using a random subset of tests.

Randomly selecting test cases may miss critical tests and does not ensure an optimized regression suite. This approach lacks a systematic method for ensuring comprehensive coverage.

C. Automating test scripts using AI-based test automation tools.

Automation helps in running tests efficiently but does not inherently optimize the selection of tests to prevent the suite from growing too large.

D. Using an AI-based tool to optimize the regression test suite by analyzing past test results.

This is the most effective approach as AI-based tools can analyze historical test data, identify patterns, and prioritize tests that are more likely to catch defects based on past results. This method ensures an optimized and manageable regression test suite by focusing on the most impactful test cases.

Therefore, the correct answer isDbecause using an AI-based tool to analyze past test results is the best option to optimize regression test selection and manage the size of the regression suite effectively​​.

A. Feature selection

Feature selection is the process of selecting the most relevant features from the data. While important, it is not directly about handling excess data.

B. Data sampling

Data sampling involves selecting a representative subset of the data for training. When there is more data than needed, sampling can be used to create a manageable dataset that maintains the statistical properties of the full dataset.

C. Data labeling

Data labeling involves annotating data for supervised learning. It is necessary for training models but does not address the issue of having excess data.

D. Data augmentation

Data augmentation is used to increase the size of the training dataset by creating modified versions of existing data. It is useful when there is insufficient data, not when there is excess data.

Therefore, the correct answer isBbecause data sampling is the most relevant activity when dealing with an excess amount of data for training​​.

Section1.6.3 – Reinforcement Learningof the ISTQB CT-AI syllabus states that reinforcement learning (RL) is based on anagent interacting with an environment, performing actions, and receivingrewards or penalties. The core concept is thereward function, which guides the agent’s learning process. The syllabus emphasizes that training in RL isdriven by rewards, and the agent aims to maximize cumulative reward over time. Therefore, OptionCdirectly reflects the correct description: the agent learns by being rewarded for successful actions .

Option A is incorrect because RL doesnotuse labeled data; that applies to supervised learning. Option B contradicts the syllabus definition: RL fundamentallyrequiresinteraction with the environment. Option D is incorrect because the reward function isdefined by humans, not learned by the agent; the agent learns apolicy, not the reward function itself.

Thus, OptionCis the only statement consistent with RL as defined in the syllabus.

The syllabus mentions that AI can help optimize regression test suites:

"An AI-based tool can perform optimization of the regression test suite by analyzing… the information from previous test results, associated defects, and the latest changes that have been made, such as features which are broken more frequently and which tests exercise code impacted by recent changes."

(Reference: ISTQB CT-AI Syllabus v1.0, Section 11.4, page 79 of 99)

The syllabus states that non-determinism is one of the key challenges for ensuring safety in AI-based systems:

"The characteristics of AI-based systems that make it more difficult to ensure they are safe... include: complexity, non-determinism, probabilistic nature, self-learning, lack of transparency, interpretability and explainability, and lack of robustness."

(Reference: ISTQB CT-AI Syllabus v1.0, Section 2.8, page 25 of 99)

In the syllabus, theevolutioncharacteristic for AI-based systems means the ability of the system to evolve and adapt its behavior in response to changes in the environment or in its own performance:

"Evolution is the system’s ability to change itself to adapt to new situations, different hardware, or a changing operational environment."

(Reference: ISTQB CT-AI Syllabus v1.0, Section 2.3)

The syllabus states that while pairwise testing is effective at finding defects by reducing the number of test cases needed, the resulting test suite can still be extensive and require automation:

"Even the use of pairwise testing can result in extensive test suites… automation and virtual test environments often become necessary to allow the required tests to be run."

(Reference: ISTQB CT-AI Syllabus v1.0, Section 9.2, Page 67 of 99)

The syllabus explains:

"Where the internal structure of an AI-based system is too complex for humans to understand, the system can only be tested as a black box. Even when the internal structure is visible, this provides no additional useful information to help with testing."

This confirms that black-box testing is chosen because the tester does not need to understand the system’s internal structure.

(Reference: ISTQB CT-AI Syllabus v1.0, Section 8.5, page 61 of 99)