Free Practice Questions for ISTQB CT-AI Exam

The ISTQB CT-AI syllabus lists several characteristics that make it difficult to ensure safety in AI-based systems. Section2.8 – Safety and AIexplicitly names the characteristics that complicate safety assurance:complexity,non-determinism,probabilistic behavior,self-learning,lack of transparency, andlack of robustness. Among these,complexityis a core challenge because modern AI systems—particularly those using deep learning—have highly non-linear behavior, large numbers of parameters, and intricate interactions that are hard to predict.

Option B (Complexity) directly aligns with the syllabus and is therefore correct.

Option A (Determinism) is the opposite of AI behavior; AI is oftennon-deterministic, and determinism doesnotmake systems unsafe. Option C (Interpretability) does impact trust and explainability, but the syllabus positions it as a transparency challenge, not the primary difficulty in ensuring safety. Option D (Robustness) is a desired quality, not a reason safety is hard; alackof robustness would be a challenge, not robustness itself.

Thus,complexitybest reflects the syllabus’ explicit safety-related difficulty.

The syllabus explains that supervised learning requires correctly labeled data so the algorithm can learn the relationship between input features and output labels:

"In supervised learning, the algorithm creates the ML model from labeled data during the training phase. The labeled data is used to infer the relationship between the input data and output labels."

(Reference: ISTQB CT-AI Syllabus v1.0, Section 3.1.1)

The syllabus explains that ML models can be used to analyze reported defects and suggest which developers are best suited to fix them based on historical data about defect assignment and resolution speed:

"Assignment: ML models can suggest which developers are best suited to fix particular defects, based on the defect content and previous developer assignments."

(Reference: ISTQB CT-AI Syllabus v1.0, Section 11.2, page 78 of 99)

The ISTQB CT-AI syllabus describes severaldata quality aspectsthat affect ML performance. In Section2.2 – Data Preparation, it explains that datasets may suffer from issues such asincomplete data,irrelevant data,incorrect data,unbalanced data, or data lacking preprocessing. “Incomplete data” means thatportions of the required data are missing, often because some time periods, records, or sources were not captured. This aligns exactly with Option A, which correctly identifies missing intervals as incomplete data.

Option B is incorrect: “data not preprocessed” refers to data that has not undergone normalization, cleaning, or transformation—not data recorded incorrectly. Option C is wrong because irrelevant datadoesnegatively affect ML models by introducing noise and unnecessary features. The syllabus explicitly states that including irrelevant features can degrade model learning. Option D is incorrect: “unbalanced data” relates todisproportionate class distribution, not recency or freshness of data.

Thus, OptionAis the only statement that correctly matches the syllabus definition of this data quality aspect.

A/B testing, also known as split testing, is a method used to compare two versions of a product or system to determine which one performs better. It is widely used in web development, marketing, and machine learning to optimize user experiences and model performance. Here’s why option C is the least descriptive of an A/B testing scenario:

Understanding A/B Testing:

In A/B testing, two versions (A and B) of a system or feature are tested against each other. The objective is to measure which version performs better based on predefined metrics such as user engagement, conversion rates, or other performance indicators.

Application in Machine Learning:

In ML systems, A/B testing might involve comparing two different models, algorithms, or system configurations on the same set of data to observe which yields better results.

Why Option C is the Least Descriptive:

Option C describes comparing the performance of an ML system on two different input datasets. This scenario focuses on the input data variation rather than the comparison of system versions or features, which is the essence of A/B testing. A/B testing typically involves a controlled experiment with two versions being tested under the same conditions, not different datasets.

Clarifying the Other Options:

A. A comparison of two different websites for the same company to observe from a user acceptance perspective: This is a classic example of A/B testing where two versions of a website are compared.

B. A comparison of two different offers in a recommendation system to decide on the more effective offer for the same users: This is another example of A/B testing in a recommendation system.

D. A comparison of the performance of two different ML implementations on the same input data: This fits the A/B testing model where two implementations are compared under the same conditions.

Section2.10 – Explainability and Transparencyof the ISTQB CT-AI syllabus describes explainable AI as the ability of a system to provide human-understandable insight into its decisions. The syllabus referencesThe Royal Society’s reportas a foundational source explaining why explainability is important. Among the stated motivations is the need toincrease user trust and confidencein AI systems by making their decisions understandable and justifiable. Therefore, OptionCdirectly reflects the syllabus content .

Option A is incorrect because interpretability doesnotrefer to determining correctness of outputs; rather, it refers to understandinghowthe model arrives at outputs. Option B incorrectly frames explainability as the ability to investigate algorithms or training data; explainability is aboutunderstanding the model’s decision-making, not reverse engineering its components. Option D is incorrect because explainability doesnoteliminate the need for risk and vulnerability assessments; the syllabus clearly emphasizes that testing, risk assessment, and robustness checks remain critical even when a model is explainable.

Thus, the only statement consistent with the syllabus isOption C.

AI test environments have several unique characteristics that differentiate them from traditional test environments. Let’s evaluate each option:

A. Challenges resulting from low accuracy of the models.

Low accuracy is a common challenge in AI systems, especially during initial development and training phases. Ensuring the model performs accurately in varied and unpredictable scenarios is a critical aspect of AI testing.

B. The challenge of mimicking undefined scenarios generated due to self-learning.

AI systems, particularly those that involve machine learning, can generate undefined or unexpected scenarios due to their self-learning capabilities. Mimicking and testing these scenarios is a unique challenge in AI environments.

C. The challenge of providing explainability to the decisions made by the system.

Explainability, or the ability to understand and articulate how an AI system arrives at its decisions, is a significant and unique challenge in AI testing. This is crucial for trust and transparency in AI systems.

D. Challenges in the creation of scenarios of human handover for autonomous systems.

While important, the creation of scenarios for human handover in autonomous systems is not a characteristic unique to AI test environments. It is more related to the operational and deployment challenges of autonomous systems rather than the intrinsic technology-related characteristics of AI.

Given the above points, optionDis the correct answer because it describes a challenge related to operational deployment rather than a technology-related characteristic unique to AI test environments.

The ISTQB CT-AI syllabus explains in Section1.6 – Machine Learning Approachesthatsupervised learningis appropriate when labeled data exists and the goal is to predict an output based on known historical examples. Predicting a customer’spurchase probabilityis aclassificationtask when the output corresponds to discrete categories such as“likely to purchase”vs.“not likely to purchase.”The syllabus gives similar examples in describing classification as the process of assigning instances to predefined classes based on learned patterns in labeled data. Because the retail company wants to determine whether a customer will make a purchase based on marketing actions, classification is the most appropriate choice .

Option A (regression) predicts continuous numeric values and is less suitable because the task centers oncategorical likelihood, not estimating exact monetary values. Option C (unsupervised learning) is used when labels are not available—here, the company has labeled purchase histories. Option D (reinforcement learning) requires an interactive environment with reward-driven behavior, which is not applicable to this scenario.

Thus,supervised learning (classification)is the most suitable approach according to the syllabus.

TheISTQB Certified Tester AI Testing Syllabus v1.0explicitly categorizes different AI implementation technologies in Section1.4 – AI Technologies. Within this section, AI methods are grouped into categories, one of which is“Reasoning techniques.”These reasoning techniques includerule engines, deductive classifiers, case-based reasoning, and procedural reasoning. Because deductive classifiers are directly listed under this set of reasoning approaches, they are recognized as a reasoning-based AI technology.

Reasoning techniques differ from machine learning approaches because they rely onstructured, predefined rules or logicto reach conclusions. Deductive classifiers use logical inference and symbolic reasoning to classify inputs by applying encoded knowledge. This makes them fundamentally different from statistical or data-driven ML algorithms.

The other options—Linear regression,Random Forest, andGenetic algorithms—are listed by the syllabus asmachine learning techniques, not reasoning methods. Linear regression performs numerical prediction, Random Forest is an ensemble decision-tree ML model, and genetic algorithms are optimization-based ML approaches inspired by evolutionary processes. None of these involve symbolic logical deduction.

Thus, based on the authoritative definitions in the syllabus,Deductive classifiers (Option A)is the only technology classified as a reasoning technique.