Lab 05: Find the commit that broke it
You’ll take over the history of a small Tiny Tapeout project, an LED dimmer that three teammates wrote over five weeks. Its test passed at the commit they tagged v1.0, and it fails now, 27 commits later. You’ll write a script that tells git bisect run correctly whether each commit is good, bad or can’t be tested, use it to find the commit that broke the test, revert that commit on a branch, and get the revert through every workflow in a pull request on a repository of your own.
Package: labs/lab-05-find-the-commit-that-broke-it/
Before you start
Read History as evidence.
Finish Lab 03. This lab uses the test job’s tools you installed in Lab 01, Part 1, your access token from Lab 01, Part 2, and the way you turned on Pages in Lab 01, Part 5.
Every command runs in a terminal on the workbench desktop, where
git bisect runcan call the samemake testas Lab 03. In each new terminal, run these first; the steps use$Lfor this lab’s package and$TTfor the test job’s tools:. /foss/designs/common/.designinit L=/foss/designs/modules/ic102-d/labs/lab-05-find-the-commit-that-broke-it TT=/foss/designs/modules/ic102-d/labs/common/tt-tools
The teammates are made up, and so is their project, which the course builds from Tiny Tapeout’s template with a script. Every commit is an ordinary one, though: features, tests, documentation, two half-finished edits, a new file left out of the test’s source list, and one tidy-up that went wrong.
This lab takes about an hour, including the 5 to 10 minutes each push takes on GitHub’s runners.
Part 1: Clone the history
What does the test say now, and what did it say at v1.0?
Clone the history. The course ships it as a git bundle, a single file that holds every commit and tag, and
git clonereads it like a repository:mkdir -p /foss/designs/projects && cd /foss/designs/projects git clone $L/data/history.bundle ttsky-ic102-d-bisect cd ttsky-ic102-d-bisect git config user.name "Your Name" git config user.email "ID+USERNAME@users.noreply.github.com" git config credential.helper 'cache --timeout=28800'Use the same no-reply address as in Lab 01. You should see
Cloning into 'ttsky-ic102-d-bisect'.... The folder is named after the repository you’ll push it to in Part 4.Look at the history:
git log --oneline | wc -l git log --oneline v1.0..main | wc -l git log --oneline -3You should see
36, then27, then:00c3d87 test: name the LED's pin in failure messages 35f4b2e info.yaml: a longer description 14d59b6 test: run fade mode for 100 stepsThe hashes in your clone are the same as the ones on this page, because the course built each commit with a fixed author, date and message.
Run the test on
main, as the test job would:make -C "$TT" test PROJECT="$PWD"You should see three of seven tests fail:
AssertionError: with duty 0 the LED (uo_out[7]) was on for 8 of 2048 cycles, expected 0 AssertionError: with duty 64 the LED (uo_out[7]) was on for 260 of 1024 cycles, expected 256 AssertionError: with duty 255 the LED (uo_out[7]) was on for 1024 of 1024 cycles, expected 1020and
** TESTS=7 PASS=4 FAIL=3 SKIP=0in the summary table, followed bymake: *** [Makefile:42: test] Error 2.Before you run it, write down whether the test passes at
v1.0. Then check:git switch --detach v1.0 make -C "$TT" test PROJECT="$PWD" git switch mainYou should see
** TESTS=3 PASS=3 FAIL=0 SKIP=0. The test had only three cases then, all of them about the PWM, and all three pass.Somewhere in the 27 commits between
v1.0andmain, the PWM broke. How many tests willgit bisectneed to find the commit?TipAnswerAbout five. Each test halves the number of suspects, and \(2^5 = 32 \ge 27\). One commit that can’t be tested may add a test or two, as you’ll see in Part 2.
You should see: three PWM failures on main, and a pass at v1.0.
Part 2: The bisect script
What should the script say about a commit whose design doesn’t compile?
Copy the starting script to the top of your clone, and read it:
cp $L/bisect-template.sh bisect.shIt runs the test job’s steps with
run-test.sh, the script behind Lab 03’smake test, and then always exits 0. The comment at the top lists the three answersgit bisect rununderstands: 0 for good, 1 to 127 except 125 for bad, and 125 for “can’t test this one”.bisect.shisn’t tracked by git, so checking out old commits leaves it alone.Why can’t the script simply exit with
run-test.sh’s status? Try it onmain, where a test fails, and oncbba6a3, a commit from the middle of the history:for c in main cbba6a3; do git checkout -q $c; rm -f test/results.xml "$TT/run-test.sh" . > /dev/null 2>&1 echo "$c: exit $?; $(ls test/results.xml 2>&1)" done git checkout -q mainYou should see:
main: exit 2; test/results.xml cbba6a3: exit 2; ls: cannot access 'test/results.xml': No such file or directoryBoth exit with 2. With cocotb 2.x,
makeruns cocotb’s own results check at the end of the simulation, so it stops withError 2when a test fails, and it stops with the same status when the design doesn’t compile. The difference shows intest/results.xml: cocotb writes it whenever the simulation runs, and never when the compile fails. (This is also why the script deletesresults.xmlbefore each run. An old one, left from the previous commit, would make a commit that never ran look as if it had.)Replace the last line of
bisect.sh(exit 0, under the TODO) with the rule the History as evidence script uses forprotocol-emulator, where a missingresults.xmlcounts as bad:if [ ! -f test/results.xml ]; then exit 1 fi if grep -q "<failure" test/results.xml; then exit 1 fi exit 0Run the bisect between the last known good commit and
main:git bisect start main v1.0 git bisect run ./bisect.shIt takes about ten seconds. You should see it test five commits and end with:
85133d69064177106e14d05cf0e7b97a7add19f5 is the first bad commit commit 85133d69064177106e14d05cf0e7b97a7add19f5 Author: Sam Rivera <sam.rivera@example.com> Date: Thu Oct 15 20:25:00 2026 -0400 Move the seven-segment table into seg7_table.v info.yaml | 1 + src/seg7.v | 24 +++--------------------- src/seg7_table.v | 35 +++++++++++++++++++++++++++++++++++ 3 files changed, 39 insertions(+), 21 deletions(-) create mode 100644 src/seg7_table.v bisect found first bad commitIs
85133d6the commit that broke the PWM? The script kept a log of each commit it tested; read this one, and the next commit’s message:grep error /tmp/bisect-85133d6.log git log --oneline -1 ae2c625TipAnswerNo. The log has a line ending in
src/seg7.v:15: error: Unknown module type: seg7_table: the commit added a new file toinfo.yamlbut not toPROJECT_SOURCESintest/Makefile, so the design didn’t compile and the test never ran. The very next commit,ae2c625 test/Makefile: add seg7_table.v to PROJECT_SOURCES, fixed that, and nothing about the PWM changed. The script reported a commit that couldn’t be tested at all as bad, andgit bisectgoes by whatever the script reports.The same rule was right for
protocol-emulator, because there the failure being hunted was a compile error (a module the testbench could no longer find). Here the failure being hunted is a wrong value from the PWM, and a commit that doesn’t compile tells you nothing about it.End this bisect, change the first
exit 1inbisect.sh(the one underif [ ! -f test/results.xml ]) toexit 125, and run it again:git bisect reset git bisect start main v1.0 git bisect run ./bisect.sh git bisect logYou should see the first commit it checks out,
cbba6a3 WIP: preset brightness levels on uio_in[2:1], reported as skipped in the log, five commits tested in all, and a different first bad commit. Write down its hash and message before you open the answer.TipAnswer: the bisect, from git bisect log# bad: [00c3d87894d0c8bce7a127e029cd9f6cbbb36da2] test: name the LED's pin in failure messages # good: [4e04d775e153a5c2526f9687f9ef83a185f62b7d] docs: say how the dimmer works git bisect start 'main' 'v1.0' # skip: [cbba6a3831a677c6552cf449fd3ae56728bf2565] WIP: preset brightness levels on uio_in[2:1] git bisect skip cbba6a3831a677c6552cf449fd3ae56728bf2565 # good: [921b40562b8fbf021b317bc475c61ee5f525c1b7] presets: finish the case statement git bisect good 921b40562b8fbf021b317bc475c61ee5f525c1b7 # bad: [d18e3169a759beec08f58e0f6327310941a607de] pwm: tidy up the comparator and its comments git bisect bad d18e3169a759beec08f58e0f6327310941a607de # good: [3ad66712b887c83d196dd5e4038357ec66c468b8] test: the three presets git bisect good 3ad66712b887c83d196dd5e4038357ec66c468b8 # good: [60651505fa672660194fbb3cb412ff63d39e56f9] test: run the testbench clock at 10 MHz, to match clock_hz git bisect good 60651505fa672660194fbb3cb412ff63d39e56f9 # first bad commit: [d18e3169a759beec08f58e0f6327310941a607de] pwm: tidy up the comparator and its commentsRead the commit, then end the bisect:
git show <the first bad commit> git bisect resetIts message says it tidied up the comparator and its comments. Which line changed what the circuit does, and why does that turn the LED on for exactly 8 of 2048 cycles at duty 0?
TipAnswerassign out = (phase < duty);becameassign out = (phase <= duty);. The phase counts 0 to 255, once per PWM period, so with duty 0 the new comparison is true for one step in every period, when the phase is 0. A step lasts four clock cycles (the prescaler divides by 4), and the test watches two periods: \(2 \times 4 = 8\) cycles. The same extra step gives 260 instead of 256 cycles at duty 64, and turns duty 255 into always on. The four changed comment lines around it make the diff look like a comment edit, which is how it got past its author.Check your script against three commits whose answers are known:
python3 $L/check.py --offlineYou should see, after a few seconds (5 s in our run):
Your bisect.sh, run at three commits 4e04d77 v1.0, test passes exit 0 ok cbba6a3 preset WIP, doesn't compile exit 125 ok 00c3d87 last commit, test fails exit 1 okfollowed by
no commit made by git revert on this branch yet. You’ll make that commit in Part 3.
You should see: git bisect name a commit whose diff changes the PWM, and three ok lines from the check.
Part 3: The revert
Can you undo that one commit without undoing the seven commits that came after it?
Before you run anything, look at what the seven later commits touched (
git log --stat --oneline <the first bad commit>..main). Write down whether you expectgit revertto apply cleanly.TipAnswerIt should. None of the later commits touch
src/pwm.v, so the lines the revert puts back haven’t moved, and git can apply the reverse of the old diff directly.Make a branch for the fix and revert the commit on it:
git switch -c revert-pwm-compare main git revert --no-edit <the first bad commit> git log -1 --format=%BYou should see
1 file changed, 4 insertions(+), 4 deletions(-)and a message that ends withThis reverts commit <its full hash>.The new commit’s own hash will differ from anyone else’s, because it carries your name and the time.Run the test again:
make -C "$TT" test PROJECT="$PWD"You should see all seven pass, among them
test.test_duty_0_is_off PASS 205250.00and** TESTS=7 PASS=7 FAIL=0 SKIP=0. The fade, preset and display tests added after the bad commit still pass, because the revert undid only the comparator.Commit your script on the branch, so that the pull request carries the evidence:
git add bisect.sh git commit -m "Add the bisect script that found the PWM regression"
You should see: seven passing tests on your branch, with the later features still in place.
Part 4: The pull request
Does GitHub’s runner agree with your workbench?
On GitHub, create a new public repository called
ttsky-ic102-d-bisectin your own account. Leave it empty, as in Lab 04, Part 1: no README, no.gitignoreand no licence, because the history you push already has its own first commit.Point your clone at it and push
mainand the tag. The push sends files under.github/workflows/, which GitHub accepts only from a token with the Workflows permission you gave yours in Lab 01, Part 2:git remote set-url origin https://github.com/YOUR-USERNAME/ttsky-ic102-d-bisect.git git push -u origin main --tagsBefore you look at the Actions tab, predict the result of each job on
main:docs,test,gds,precheck,gl_testandviewer.TipAnswerdocs,gdsandprecheckshould pass, because the files they check are fine.testfails on the three PWM cases, and so doesgl_test, which runs the same test on the gate-level netlist.viewerfails with the 404 from Lab 01, because a new repository has Pages off.That’s what the course’s practice repository got.
testandgl_testboth stopped atAssertionError: with duty 0 the LED (uo_out[7]) was on for 8 of 2048 cycles, expected 0, the first of the three failures you saw in Part 1, andvieweratError: Failed to create deployment (status: 404).You’ll also see the three workflows run a second time, on
v1.0. Tiny Tapeout’s workflows run on every push, and pushing a tag counts as one. Onv1.0onlyviewerfails, with the sameTurn Pages on for
ttsky-ic102-d-bisectthe way you did in Lab 01, Part 5. You don’t need Run workflow this time, because the push in the next step starts a new run.Push your branch and open a pull request from
revert-pwm-compareintomain:git push -u origin revert-pwm-compareGitHub prints a link for creating the pull request. In the description, name the commit you reverted and why it was wrong, say how you found it, and paste the output of
git bisect logfrom Part 2. If an AI tool helped you write the script or the description, say so and say what you checked yourself, as Pull requests and review asks.Wait for the runs on your branch to finish (about 5 to 10 minutes), then run the full check:
python3 $L/check.pyIt repeats Part 2’s three runs and then reads your branch’s latest commit from GitHub. In the course’s practice repository it printed:
Checking /foss/designs/projects/ttsky-ic102-d-bisect Your bisect.sh, run at three commits 4e04d77 v1.0, test passes exit 0 ok cbba6a3 preset WIP, doesn't compile exit 125 ok 00c3d87 last commit, test fails exit 1 ok Your revert reverts d18e316, the commit that broke the test GitHub: vproHacks/ic102-test-d at 6b2ae2a docs success test success gds success precheck success gl_test success viewer success VERDICT: PASSviewerpasses on a branch too: it publishes the branch’s layout to your project’s page.Merge the pull request. The repository is yours and has no rules, so you can merge it yourself. The merge is a push to
main, somain’s workflows run again, now with the revert in them and Pages on. In the course’s practice repository all six were green 5 min 45 s after the merge,viewerincluded.
You should see: all six jobs green on the pull request, VERDICT: PASS, and all six green on main after the merge.
Finishing
You’re done when check.py, run on your revert-pwm-compare branch, prints VERDICT: PASS (your script gives the right answer at all three commits, your branch reverts the commit that broke the test, and all six jobs passed on the branch’s latest commit), and every job is green on main after you’ve merged the pull request.
Questions:
Your script and the
protocol-emulatorscript in History as evidence give opposite answers for a commit that doesn’t compile. Can both be right?TipAnswerYes, because they hunt different failures. In
protocol-emulatorthe test broke because the testbench stopped compiling, so a commit that doesn’t compile shows the bug. In the dimmer the bug is a wrong value from a design that compiles, so a commit that doesn’t compile can’t show it either way, and 125 tells git to look at a neighbour instead. So decide what you’re hunting before you decide what the script says about a commit that doesn’t compile.The
v1.0tag saved you a step. What would you have had to do without it?TipAnswerFind a good commit yourself before
git bisect startcould begin, by checking out older commits and running the test until one passed. A tag on a commit that everyone saw green is a known-good starting point for anyone who comes later, which is why History as evidence suggests marking one.The bad commit was made on Oct 25, and seven more commits followed before anyone noticed. In the team’s real repositories the test workflow runs on every push. What does that suggest the made-up team was doing wrong?
TipAnswerThey weren’t looking at their runs. On GitHub, the
testworkflow would have gone red on the push that includedd18e316, and on every push after it. A red cross only stays red for a week when nobody is reading it, and that’s howhart-1’sdocsandgdsstayed red for four weeks.
Submit: the link to your pull request.
Going further (optional)
When you already know which line is wrong, git can find the commit that touched it without running anything. git log -G lists the commits whose diffs add or remove a line matching a pattern:
git log --oneline -G "phase <" -- src/pwm.vOn the revert-pwm-compare branch of the course’s practice repository it printed:
840c251 Revert "pwm: tidy up the comparator and its comments"
d18e316 pwm: tidy up the comparator and its comments
c6218cc Add an 8-bit PWM
From the bottom up, those are the commit that wrote the comparison, the commit that changed it, and the revert that changed it back, which has a hash of its own in your clone. git bisect is for the more common case, where you know only that the test fails.
This is the last row of the Course path.