Lab: cocotb on a counter
In this lab you run every cocotb example from the guide pages on an 8-bit counter: your first test, the clock-edge race, the triggers, the 1.x forms that warn and the ones that fail, a two-sided verdict, and the counter’s waveforms in Surfer and GTKWave. Nothing here is graded; the lab is practice with the tools before the processor arrives in Lab 02. Twice along the way you’ll break something on purpose and check that the right tests notice.
This lab takes about 75 minutes.
Package: labs/cocotb-intro/
Before you start
Finish Lab 01.
Read Your first cocotb test, Clocks, resets and triggers, cocotb 2.x and older tutorials, Driving and checking a design and Waveforms.
Work in a terminal on the noVNC desktop (Part 5 opens graphical viewers), in the package directory:
mod dd104 # in a new terminal, load .designinit first (Lab 01, Part 1) cd labs/cocotb-intro
The design is rtl/counter.sv: an 8-bit counter with an enable, a parallel load that takes priority over the enable, and an active-low synchronous reset. Its tc output is high on the cycle before the count wraps. rtl/counter_broken.sv is the same file with en & deleted from the expression for tc.
Part 1: Your first test
What does the smallest cocotb test print, and what goes wrong if it reads q at the clock edge?
Run the minimal test:
make helloYou should see
Running on Icarus Verilog version 14.0 (devel), thenq = 1 after one enabled clock. That is the whole test.at21.00ns, andTESTS=1 PASS=1 FAIL=0 SKIP=0with21.00ns of simulated time.Try the two commands that look like they should run the test:
python3 test_01_hello.py; echo "exit=$?" python3 -m pytest test_01_hello.py; echo "exit=$?"The first prints nothing and
exit=0, and the second printscollected 0 itemsandno tests ran, withexit=5. Neither of them started a simulator, so neither ran a test; in this package, onlymakestarts one.Next, make the clock-edge race happen by copying the test and deleting its
Timerline. Start with the copy:cp test_01_hello.py test_norace.pyOpen
test_norace.pyand delete the lineawait Timer(1, unit="ns"), the one between the lastawait RisingEdge(dut.clk)and the finalassert. Before you run it, write down what you expect theassertto read fromq. Then run it throughsim.mkdirectly, with its own build directory:make -f sim.mk sim TB=test_norace SIM_BUILD=build/noraceYou should see the test fail at
20.00nswithafter one enabled clock q is 00000000, expected 1, thenTESTS=1 PASS=0 FAIL=1. Themakeerror lines after the table are cocotb’s makefile reporting the failed test.Read the failure. Which lines are your assertion message, and which are pytest’s assertion rewriting (
assert ... + where ...)? Why does the message print00000000and not0?TipAnswerYour message is the text after
AssertionError:, and it also appears on the first line. Theassert ... + where ...lines are pytest taking yourassertapart and showing what each piece evaluated to. The message prints00000000becausedut.q.valueis aLogicArray, which can holdXandZ, so printing it gives the bit pattern.int(dut.q.value)would give0. Your first cocotb test has the full explanation.
Check: make hello passes, and test_norace fails with q is 00000000 one clock early.
Part 2: Triggers
How long does each trigger really wait, and is the counter’s reset synchronous?
Run the trigger tests:
make triggersYou should see
TESTS=6 PASS=6 FAIL=0 SKIP=0and1832.51ns of simulated time.Find these lines in the log and match each to its section of Clocks, resets and triggers:
10 edges counted from t=0 took 90 ns (nine periods -- the edge at t=0 was free) q read AT the rising edge : 00000000 q read 1 ns after the edge: 00000001 ClockCycles(clk, 5) from 1 ns past an edge: 49 ns rst_n low, no clock edge yet: q = 0x5A one rising edge later : q = 0x00 First(RisingEdge(dut.tc), Timer(200, unit='ns')) -> the Timer wonWhy did ten edges counted from t = 0 take 90 ns and not 100?
TipAnswerClock.start()drives the clock high at t = 0. That first0→1transition is a rising edge and takes no time, so ten edges from t = 0 span nine periods.The two reset lines also tell you something about the design: whether the counter’s reset is synchronous or asynchronous. Write down which it is, and which line tells you.
TipAnswerSynchronous.
rst_n low, no clock edge yet: q = 0x5Ashowsqkeeping its value with reset asserted, andone rising edge latershows it cleared on the first edge. If the first line had printed0x00, the reset would be asynchronous.
Check: six tests pass in 1832.51 ns, and you can say why ten edges from t = 0 took 90 ns.
Part 3: The 1.x traps
Which cocotb 1.x forms still run on cocotb 2.0.1, and which fail?
Before you run them, write down which of the four files in
oldapi/you expect to pass. The file names say which 1.x form each one uses. Then run them:make oldapiYou should see four results in order:
File Result oldapi/test_1x_units.pyDeprecationWarning: The 'units' argument has been renamed to 'unit'., thenTESTS=1 PASS=1oldapi/test_1x_fork.pyAttributeError: module 'cocotb' has no attribute 'fork',TESTS=1 PASS=0 FAIL=1oldapi/test_1x_assign.pyTypeError: '<=' not supported between instances of 'LogicObject' and 'int',FAIL=1oldapi/test_1x_imports.pyModuleNotFoundError: No module named 'cocotb.binary', and no results table, because it failed while cocotb was importing the fileWhich of the four is the most dangerous to find in old code?
TipAnswerunits=, the one that passed. Old code that still runs gives you no sign that it will break in a later release, and the only hint is a one-line deprecation warning that scrolls past.Run the suite with a mistyped test filter:
make nothingYou should see
No tests left after filtering with: chekcing$, no results table, andexit status 0, no FAIL anywhere, and zero tests ran.Look at what cocotb recorded:
grep -c "<skipped" build/nothing.xmlYou should see
6: six tests, all skipped.Ask the installed cocotb which keywords
@cocotb.test()takes:python3 -c "import cocotb, inspect; print(inspect.signature(cocotb.test))"You should see one long line starting
(obj: Union[Callable[..., Coroutine[...and listingtimeout_time,timeout_unit,expect_fail,expect_error,skip,stageandname.
Check: you can say which of the four 1.x forms passed, and why make nothing exits 0.
Part 4: A two-sided verdict
Does the checking testbench pass the good counter, fail the broken one, and notice a counter you break yourself?
Run everything:
makeThis runs
hello,triggers,simandbroken, which comes to four simulator runs and nineteen tests, with 0.3 s of simulation. The whole command takes 8–14 s of wall-clock time, depending on how busy the machine is. You should see:------------------------------------------------------------------------ VERDICT ------------------------------------------------------------------------ test_03_checking.py PASSED on rtl/counter.sv test_03_checking.py FAILED on rtl/counter_broken.svScroll up to the
make brokensection, where one test fails and five pass. Find the failing test’s message, which startsq is 0xFF and en is 0, so the count is NOT about to wrap, but tc is 1.Which line of RTL does it send you to?TipAnswerThe
tcassignment. The message goes on to say thattcshould been & (q == 8'hFF)and that atcof 1 here means theen &is missing, which is exactly the difference betweenrtl/counter.svandrtl/counter_broken.sv.Find the two log lines from
against_a_reference_model. How many of its 600 random clocks hadq = 0xFFwithenlow?TipAnswerNone:
of those 600 clocks, 0 had q = 0xFF with en low. The counter only reaches0xFFwhile it is running, and while it is running the brokentcand the correct one agree, so the most thorough test in the file never reaches the one situation that exposes the bug.Now break the good counter yourself. Open
rtl/counter.svand changeelse if (en) q <= q + 8'd1;to
q <= q + 8'd2. Before you run, write down which of the six tests intest_03_checking.pyyou expect to fail. Then run:make simYou should see three of the six tests fail:
load_then_count,the_wrap_is_where_the_bugs_liveandagainst_a_reference_model, withTESTS=6 PASS=3 FAIL=3. The first failure is on the first comparison:31.00ns WARNING .. load_then_count.load_then_count counting from 0x30: expected 0x31, got 0x32 -- a difference of +1The
assert 50 == 49line under it is pytest’s assertion rewriting showing the same numbers in decimal.the_wrap_is_where_the_bugs_liveshows the whole trajectory,[(255, 1), (1, 0), (3, 0), (5, 0), (7, 0)], which is the counter stepping by two.What do
a difference of +1and theq=0x32 ... at t=31 nsin the full message tell you?TipAnswerThe
+1tells you what went wrong: one counted edge movedqone step further than it should have. Thet=31 nstells you when, which was on the first counted edge, so the error was there from the start rather than drifting in later.Undo the edit and run
make simagain. You should seeTESTS=6 PASS=6 FAIL=0 SKIP=0.
Check: make prints the verdict above, and rtl/counter.sv is back to q + 8'd1.
Part 5: Waveforms
The broken counter prints the same log as the good one, so what does its bug look like in a waveform?
Make a waveform:
make wavesYou should see the log lines from
reset released at t = 11 nstofinished at t = 211 ns with q = 0x05, includingloaded 0xFF at t = 121 ns <-- zoom here, thenFST info: dumpfile ... opened for output.after the results table (which is normal), and twowrotelines forcounter.fstandcounter.vcd.Open it in Surfer, from this terminal:
surfer counter.fstClick
counterin the Scopes pane, then addclk,rst_n,en,load,qandtcfrom the Variables pane. Then zoom to 121 ns; Surfer’s ruler is in picoseconds, so look near121000 ps.You should see
qreachffat 121 ns, hold for three clocks withenlow, andtcgo high for exactly one clock beforeqwraps to00.A command file saves you all that clicking next time:
printf 'module_add counter\n' > view.sf surfer -c view.sf counter.fstYou should see every signal in
counteradded at once.Open the same file in GTKWave, with the shipped save file:
gtkwave counter.fst counter.gtkwIf the wave pane looks empty, that’s because it is showing only the first few picoseconds. Use
View → Zoom → Zoom Best Fit(CtrlAltF), and you should then see the same 211 ns run.Make a waveform of the broken counter. Before you open it, sketch what you expect
tcto do from 121 ns, whileqsits at0xFFwithenlow.make -f sim.mk sim TB=test_04_waves WAVES=1 RTL=rtl/counter_broken.sv \ SIM_BUILD=build/bw COCOTB_RESULTS_FILE=build/bw.xml surfer build/bw/counter.fstThe test passes, and its log is identical line for line to the good run’s. Add the same six signals.
You should see
tcgo high at 121 ns, whenqbecomes0xFF, and stay high for four clocks, including the three whereenis low and the counter is not moving. On the good counter it was one clock wide.
Check: you have seen tc one clock wide on the good counter and four clocks wide on the broken one.
Expected results
| Check | Expected | |
|---|---|---|
make hello |
TESTS=1 PASS=1, 21.00 ns |
exact |
make triggers |
TESTS=6 PASS=6, 1832.51 ns |
exact |
make nothing |
no results table, exit 0, six <skipped in build/nothing.xml |
exact |
make, verdict |
PASSED on rtl/counter.sv, FAILED on rtl/counter_broken.sv |
exact |
make wall clock |
8–14 s | ballpark |
make sim with q + 8'd2 |
TESTS=6 PASS=3 FAIL=3 |
exact |
make waves, VCD size |
1098 bytes |
exact |
Finishing
You’re done when your test_norace fails one clock early, your q + 8'd2 counter fails 3 of 6 tests, and make on the restored rtl/counter.sv prints both halves of the verdict.
Submit: nothing for this lab. make clean removes build/, the waveform files and the caches; it does not remove test_norace.py, view.sf or a test_mine.py from Going further, so delete those yourself if you want a clean package.
Going further (optional)
To write your own variant of the checking tests, copy the file and run it through sim.mk, giving each run its own build directory:
cp test_03_checking.py test_mine.py
make -f sim.mk sim TB=test_mine SIM_BUILD=build/mine
make -f sim.mk sim TB=test_mine RTL=rtl/counter_broken.sv SIM_BUILD=build/mine-brokenYou should see TESTS=6 PASS=6 on the first and TESTS=6 PASS=5 FAIL=1 on the second. The failure on the second run is the half of the two-sided verdict that shows your testbench can see the bug. (make sim TB=... won’t work for this, because the package’s sim target always runs test_03_checking.)
Next on the Course path: read Two simulators and SimProc in one page, then do Lab 02: The only test this chip has.