Level 60: The Verification Lab

Open level →

Task

Goal: design the verification scenarios — you write them now. Two cores — the golden one and the subject with ONE planted defect; the symptom is in the subject card bug report. A scenario is a program + initial state + expectation table (cycle/field/value); meet four gates: at least three scenarios in the suite and at least one expectation on an intermediate cycle; the whole suite passes on the golden core; at least one scenario catches the subject (passes on golden — fails on it); after the "fix", the regression is green on the entire suite again. This is test design, not a Verilog testbench: pin the initial state yourself, check intermediate states along the way.

A program can produce the right answer on one example while containing a bug that appears on another data set. That is why engineers not only run ready-made tests but also design the verification themselves: choose stimuli, state expectations, catch known defects. This level is about designing tests, not building circuits.

All RISC Lab levels check you with a scenario — a sequence of steps with stimuli and expectations. Now the role changes: you write the scenarios. A scenario can:

  • set inputs — for example, register values and an opcode;
  • advance a cycle — execute one simulation step;
  • check state — not only outputs but also intermediate values of registers and memory;
  • compare with the expectation — a mismatch is recorded as a failure.

Level tasks:

  1. State the expected behavior. For a short instruction sequence, write down what PC, registers and memory should be after every cycle.
  2. Check edge cases. Zero, negative numbers, overflow, equal operands, a write to x0, an unaligned address.
  3. Write a test that must fail. The platform plants a known bug in the processor — modeled on level 43 with a crossed wire. Your scenario must detect it: a test that passes on both the correct and the broken circuit checks nothing.
  4. Build a regression. Save your scenarios and run the whole set after the "fix" — all must pass.

Outcome of the level: you learn to design verification scenarios for digital blocks — choose stimuli, state expectations, find defects and run a regression. This is not writing a Verilog testbench: the directives initial, $display and delays are deliberately forbidden in the course dialect — scenario checking is done with the platform's tools.

Solution for self-check

First write your own scenarios. The solution below is a verifier's checklist.

What you should get: all four acceptance gates are met: at least three scenarios with one expectation on an intermediate cycle; the whole suite is green on the golden core; at least one scenario catches the planted defect; after the fix the regression is green again on the entire suite.

Step 1 — fix the initial state. A good scenario leaves no "empty" registers: define which values are in registers and memory before the run. Otherwise the test checks something random.

Step 2 — check along the way, not only at the end. The final value can be correct while the path is wrong. Add an intermediate-state check: for example, after the first cycle of the addition x5 should already be 42 — long before halt.

Step 3 — edge cases as a list. A minimal set for the blocks you built: 0 and −1 in arithmetic; 0x80000000 for slt/sltu; writing 42 to x0 (after the cycle x0 = 0); we=0 (the register value has not changed); an offset with a one in bit 11 for the S-format.

Step 4 — the trap test. A planted bug manifests on a specific vector. Before writing the scenario, answer: on which input must the correct and the broken circuits diverge? Then write a scenario for exactly that input and make sure: it passes on the reference and fails on the mutant.

Step 5 — regression. Return the circuit to the correct state and run all your scenarios in sequence. This is the regression suite: after every future processor change it runs again — you will use your own test collection more than once in levels 58–66.

Check yourself

Why does a test that passes on both the golden and the mutant check nothing?

It does not tell correct behaviour from broken behaviour: the value of a scenario is its ability to fail on a known defect. That is why the suite runs on the golden (green), on the subject (catches it), and the fix is closed by a regression over the whole suite.