Problem and objective
A regression needs to do more than start several simulator commands in succession. It must use the correct sources, name tests unambiguously, propagate failures reliably and distinguish each run’s results from old files. This note builds a small executable workflow with VUnit. The same four-bit counter is checked in two tests. An additional Python wrapper accepts success only when the newly generated report contains tests that actually ran and passed.
Prerequisites and tested versions
The original example was executed with Python 3.12.14, VUnit 4.7.1 and GHDL 6.0.0 mcode on Windows x64. The Python version recorded in the test report is authoritative and is read from the actual interpreter. VUnit 4.7.1 is deliberately pinned; no later prerelease was used for this test. A local package directory was prepared for the trial. VUnit 4.7.1 package release.
For transfer to Aldec, the official VUnit installation documentation lists Riviera-PRO and Active-HDL in the VHDL context. Your particular simulator version and licence must match separately. A successful GHDL run demonstrates neither the Aldec installation nor arbitrary SystemVerilog features. Explicitly record the combination of language, test framework and simulator. VUnit installation and supported simulators.
Architecture and directory layout
counter/
rtl/counter.vhd
tb/tb_counter_vunit.vhd
run.py # VUnit-Projekt und Testauswahl
ci.py # Zeitgrenze und Berichtprüfung
requirements.txt # festgelegte Python-Pakete
build/<lauf-id>/ # neuer Ordner je RegressionThe files are supplied as original examples in the project. The DUT is fully explained in the testbench note. Each wrapper run creates a new directory containing a timestamp and a random suffix. An old successful report therefore cannot hide a new failed run. Directories are not deleted automatically; retention and controlled cleanup are separate operational decisions.
Step 1: Prepare the Python environment
python -m venv .venv
# Windows PowerShell:
.\.venv\Scripts\python.exe -m pip install -r requirements.txt
# Linux entsprechend:
# .venv/bin/python -m pip install -r requirements.txtThe requirements.txt file contains vunit_hdl==4.7.1 and colorama==0.4.6. Use the same virtual-environment interpreter for every subsequent command. Also retain its version output and the simulator version actually used. For long-term reproducibility, an internal package archive with hash verification can be added; a version pin alone does not guarantee that every external package remains available indefinitely.
Step 2: Define the VUnit project
"""VUnit 4.7.1, eigenes Beispiel; Simulator ueber VUNIT_SIMULATOR waehlen."""
from pathlib import Path
import os
from vunit import VUnit
root = Path(__file__).resolve().parent
vu = VUnit.from_argv(compile_builtins=False)
vu.add_vhdl_builtins()
lib = vu.add_library("counter_lib")
lib.add_source_file(root / "rtl" / "counter.vhd")
lib.add_source_file(root / "tb" / "tb_counter_vunit.vhd")
lib.test_bench("tb_counter_vunit").set_generic(
"inject_failure", os.environ.get("EDA_INJECT_FAILURE") == "1"
)
vu.main()The runner derives the project path from its own file, so it does not depend on whichever directory someone happens to launch it from. Only the explicitly listed source files are added to counter_lib. This prevents an experimental extra testbench from entering the regression unnoticed through an overly broad filename pattern. Larger projects can later generate the source list from a reviewed project description.
The EDA_INJECT_FAILURE environment variable exists solely for the negative check. It sets a generic that deliberately fails a check. Leave it unset in normal operation. A parent system should record such test parameters in its run configuration without publishing arbitrary other environment variables or secrets. The checking behaviour must remain reproducible from the documented inputs.
Step 3: Implement two named tests
library ieee;
use ieee.std_logic_1164.all;
use ieee.numeric_std.all;
library vunit_lib;
context vunit_lib.vunit_context;
entity tb_counter_vunit is
generic (
runner_cfg : string;
inject_failure : boolean := false
);
end entity;
architecture test of tb_counter_vunit is
signal clk : std_logic := '0';
signal rst : std_logic := '1';
signal enable : std_logic := '0';
signal q : unsigned(3 downto 0);
begin
clk <= not clk after 5 ns;
dut : entity work.counter(rtl) port map (clk, rst, enable, q);
test_runner_watchdog(runner, 2 us);
main : process
procedure step(r, e : std_logic; expected : natural) is
begin
wait until falling_edge(clk);
rst <= r;
enable <= e;
wait until rising_edge(clk);
wait for 1 ns;
check_equal(q, to_unsigned(expected, q'length), "Zaehlerwert");
end procedure;
begin
test_runner_setup(runner, runner_cfg);
while test_suite loop
if run("reset_and_priority") then
step('1', '0', 0);
step('0', '1', 1);
step('1', '1', 0);
elsif run("count_wrap_and_hold") then
step('1', '0', 0);
for n in 1 to 20 loop
step('0', '1', n mod 16);
end loop;
step('0', '0', 4);
step('0', '0', 4);
end if;
check(not inject_failure, "Absichtlich ausgeloester Negativtest");
end loop;
test_runner_cleanup(runner);
wait;
end process;
end architecture;The reset_and_priority test examines reset and its priority over enable. count_wrap_and_hold examines counting, overflow and holding. Each test establishes its initial state through reset. Test order can therefore change without one test silently depending on the previous test’s final state. In larger environments, reference models, queues and protocol states also need this deliberate initialisation.
VUnit provides test control and comparison functions. The original test content remains the selection of situations and expected values. The official guide explains how the Python project connects to the VHDL runner; this note applies that mechanism to a small original design. The watchdog limits simulation time. An additional outer timeout is still needed if a tool hangs or the simulation makes no time progress. VUnit user guide.
Step 4: Select the simulator and list the tests
# Windows PowerShell, wenn GHDL installiert und im PATH ist:
$env:VUNIT_SIMULATOR = 'ghdl'
.\.venv\Scripts\python.exe run.py --list
.\.venv\Scripts\python.exe run.py 'counter_lib.tb_counter_vunit.reset_and_priority'
# Für einen separat eingerichteten Aldec-Arbeitsplatz:
# $env:VUNIT_SIMULATOR = 'rivierapro'
# oder: $env:VUNIT_SIMULATOR = 'activehdl'
# Erst nach Prüfung von Installation, Lizenz und VUnit-Kompatibilität verwenden.Listing the tests is a quick defence against accidentally empty regressions. Check that both expected names appear. A single test can then be started for diagnosis. Selection syntax and xUnit export are documented in the official command-line reference. Listing is not, however, evidence of simulation: only an actual run exercises the compiler, elaboration and functionality. VUnit command-line interface.
Step 5: Use a strict overall run
"""Strikter Runner: kein Erfolg ohne ausgefuehrte, erfolgreiche Testfaelle."""
from pathlib import Path
from datetime import datetime, timezone
import os
import signal
import subprocess
import sys
import uuid
import xml.etree.ElementTree as ET
ROOT = Path(__file__).resolve().parent
def report_status(path):
try:
tree = ET.parse(path)
except (OSError, ET.ParseError):
return 3 # Bericht fehlt oder ist ungueltig.
cases = list(tree.iter("testcase"))
if not cases:
return 3
if any(c.find(tag) is not None for c in cases for tag in ("failure", "error", "skipped")):
return 1
return 0
def run():
run_id = datetime.now(timezone.utc).strftime("%Y%m%dT%H%M%SZ") + "-" + uuid.uuid4().hex[:8]
out = ROOT / "build" / run_id
out.mkdir(parents=True)
report = out / "results.xml"
command = [sys.executable, str(ROOT / "run.py"),
"--output-path", str(out / "vunit"), "--xunit-xml", str(report)]
options = {"creationflags": subprocess.CREATE_NEW_PROCESS_GROUP} if os.name == "nt" else {"start_new_session": True}
print("Ergebnisse:", out, flush=True)
with (out / "runner.log").open("w", encoding="utf-8") as log:
try:
process = subprocess.Popen(command, cwd=ROOT, stdout=log, stderr=subprocess.STDOUT, **options)
except OSError:
return 2
try:
returncode = process.wait(timeout=120)
except subprocess.TimeoutExpired:
# Nur den von diesem Lauf gestarteten Prozessbaum beenden.
if os.name == "nt":
subprocess.run(["taskkill", "/PID", str(process.pid), "/T", "/F"], check=False,
stdout=log, stderr=subprocess.STDOUT)
else:
os.killpg(process.pid, signal.SIGKILL)
process.wait()
return 124
if returncode != 0:
return 1
return report_status(report)
if __name__ == "__main__":
raise SystemExit(run())Start the wrapper with the virtual environment’s Python interpreter and ci.py. It writes the complete runner log into the new output location. Process status is checked first. The XML report is then read: a missing, invalid or empty report is an infrastructure problem. Failed and errored tests—and skipped tests under this example’s strict policy—produce a negative overall result.
The return values have deliberately simple meanings: 0 for executed tests that passed, 1 for a negative test or runner outcome, 2 for a process that could not be started, 3 for a missing or unusable report and 124 for the outer timeout. This classification is an original project convention. It does not claim that all participating simulators use identical error codes. Their original diagnostics remain in the log.
The example’s time limit is 120 seconds. On timeout, only the process tree or process group started by the wrapper is terminated. This logic was implemented for Windows; a Linux branch is present but was not exercised in this assignment. A production CI service also needs its own job isolation and time limits. The small limit suits the demonstration design and must be adjusted with justification for real projects.
Step 6: Exercise the negative path
# Windows PowerShell, innerhalb der vorbereiteten Umgebung:
$env:EDA_INJECT_FAILURE = '1'
.\.venv\Scripts\python.exe ci.py
$edaResult = $LASTEXITCODE
Remove-Item Env:EDA_INJECT_FAILURE
if ($edaResult -eq 0) { throw 'Negativtest wurde unerwartet als Erfolg gewertet' }In the actual trial, the normal overall run completed with two passing tests and wrapper exit code 0. The injected error produced failed tests and wrapper exit code 1. Report evaluation was also checked with missing, empty, invalid, successful, failed and skipped-test reports. These negative checks are particularly valuable because an incorrect success signal would undermine every subsequent quality decision.
Diagnosis, limitations and transfer
For a compiler error, first open runner.log and find the earliest relevant diagnostic. For an empty report, inspect test selection and whether the runner stopped before simulation began. A licence problem on a commercial workstation is a failure, not a skipped successful regression. Concurrent operation requires separate output directories for each run and a process count compatible with the available licences.
The timeout with an actually hanging simulator, the Linux process-group branch and the Aldec adapters were not tested. No runtime comparisons or coverage measurements were performed either. The small design is not a comprehensive acceptance test for a large FPGA project. It provides a tested starting point that can be extended with real sources, libraries and requirements.
As a next step, transfer an existing module test and run the normal case, an intentional failure and a fresh build. The project structure fundamentals help when adding more libraries. An automation consultation should then consider the actual simulator installation, licence situation and expected test volumes together.
Sources and date
Research checked on 19 October 2026. VUnit 4.7.1 package release, VUnit installation and supported simulators, VUnit user guide, VUnit command-line interface, GHDL simulation runtime options.