Learn

What Is Computerized Adaptive Testing (CAT)?

Published October 31, 2026

A computerized adaptive test (CAT) chooses each question based on how the test taker answered the ones before it. After every answer it re-estimates ability, picks the question that tells it the most at that estimate, and stops once the estimate is precise enough.

The model behind it

Most adaptive tests rest on item response theory (IRT). Each test taker has an ability, written θ (theta), on a scale where 0 is average and most people fall between −3 and +3. Each question has a difficulty b on the same scale and a discrimination a, which says how sharply the question separates people just below its difficulty from people just above it. The two-parameter logistic (2PL) model gives the chance of a correct answer:

P(correct | θ) = 1 / (1 + e^(−a(θ − b)))

When θ equals b, the chance is exactly 50%. Easier questions push it up, harder ones push it down, and a larger a makes the change steeper.

The loop

1. Pick the most informative question. A question's Fisher information at ability θ is

   I(θ) = a² · P(θ) · (1 − P(θ))

   It is highest when the chance of a correct answer is near 50%, that is, when the question's difficulty is close to the current estimate, and it grows with discrimination. The test picks the unused question with the most information at the current estimate.
2. Re-estimate ability. After each answer, the test finds the θ that makes the whole pattern of right and wrong answers most likely (maximum likelihood, with a weak pull toward 0 that keeps the estimate stable). When every answer so far is right, or every answer is wrong, there is no finite best θ, so the estimate is set one point above the hardest (or below the easiest) question seen, capped at ±3.
3. Decide whether to stop. The standard error of the estimate is

   SE = 1 / √(sum of I(θ) over the questions answered)

   Every answered question adds information, so the standard error shrinks. The test stops when it falls below a target, such as 0.3, or when it reaches a maximum number of questions.

One test, step by step

This is one simulated test taker with a true ability of 1.2, drawn from a pool of 200 questions. The answers are simulated from the 2PL model itself. The table is the output of the code further down.

The first two right answers send the estimate up to 1.98. The test then serves harder questions, gets three wrong answers in a row and pulls the estimate back down. From there it keeps serving questions near the estimate, and the standard error falls with each one. (Question 20 shows 0.300 because of rounding; the unrounded value was just above the target.) The test stops after 21 questions with θ = 1.45 and SE = 0.294. The 95% interval, 1.45 ± 1.96 × 0.294, runs from 0.88 to 2.03 and contains the true 1.2.

How many questions does it save?

It depends on who is taking the test. We compared the adaptive test with a well-built fixed form: the 60 questions in the same pool that are most informative for an average test taker, given in that order. Each row is 300 simulated test takers.

Error is the distance between the estimate and the true θ. All three columns count only test takers with at least one right and one wrong answer, because the engine cannot give a real standard error for an all-right or all-wrong pattern; SE and error after 20 are means over those test takers, and questions to SE 0.3 is the median. At question 20 that is everyone on the adaptive test, and 175 of 300 low-band and 185 of 300 high-band test takers on the fixed form. *Only 170 of 300 low-band and 45 of 300 high-band test takers reached SE 0.3 within the fixed form's 60 questions; the median is over those who did.

For test takers near the middle, a fixed form aimed at the middle does as well as the adaptive test. The adaptive test pulls ahead at the ends of the scale. There, a fixed form spends most of its questions far from the test taker's level, and it either needs about 50 questions or never gets there. The adaptive test reaches the same precision in 17 to 19 questions across all three bands. How many questions a test needs overall depends on the precision target and the question pool; see how many questions an adaptive test needs.

These numbers come from a tidy simulation: question parameters are known exactly, and the answers follow the model. Real tests calibrate questions from response data, which adds error. They also add rules for content balance and for how often each question is shown, and those rules cost some efficiency. Treat the table as the shape of the difference, not a guarantee for any particular test.

Run it yourself

The engine is open source: adaptive-question-selector on GitHub. Clone it, install its requirements, save this as a file in the repository root and run it with Python 3.12. It prints the table above.

import random

from src.irt import AbilityEstimate, IRTEngine, Question

rng = random.Random(42)
engine = IRTEngine()

# A 200-question pool: difficulty b from -3 to +3, discrimination a from 0.5 to 2.0
pool = [
    Question(id=f"q{i}", difficulty=rng.uniform(-3, 3), discrimination=rng.uniform(0.5, 2.0))
    for i in range(200)
]

true_theta = 1.2  # the simulated test taker's real ability
ability = AbilityEstimate()
available = list(pool)

while ability.standard_error > 0.3 and len(ability.responses) < 60:
    question = engine.select_next_question(ability, available)
    available.remove(question)
    correct = rng.random() < engine.probability_correct(true_theta, question)
    ability = engine.update_ability(ability, question, correct)
    print(
        f"{len(ability.responses):2d}  b={question.difficulty:+.2f}  "
        f"{'right' if correct else 'wrong'}  theta={ability.theta:+.2f}  SE={ability.standard_error:.3f}"
    )

Hosted version

The AdaptiveTest API runs adaptive sessions over HTTP. You create a test, start a session and submit answers, and each response returns the next question and the current ability estimate. See the developer page for the endpoints and how to get access.