diff --git a/doc/03_reference/images/tb2.svg b/doc/03_reference/images/tb2.svg new file mode 100644 index 00000000..1145b1c1 --- /dev/null +++ b/doc/03_reference/images/tb2.svg @@ -0,0 +1,994 @@ + + + + + + + + + + image/svg+xml + + + + + + + + Legend: + + + + + + ... + + SV classes (UVM; dynamically created) + + + + + + + ... + + SV modules / interfaces (static)Dotted boxes are bound-in interfaces. + + + + Code block (a UVM phase) + + + + + + + + + + + + + core_ibex_base_test + + + + run_phase: + In the UVM run phase, the core_ibex_base_testclass loads the binary specified by the +bin= plusarg and calls send_stimulus. Other testsderive from core_ibex_base_test and overridesend_stimulus to provide the stimulusappropriate to the test + + + + + + + core_ibex_env + + + + ibex_mem_intf_agent (dside) + + + + + ibex_cosim_agent + + + ibex_cosim_scoreboard + + + + + ibex_mem_intf_agent (iside) + + + + ibex_irq_agent + + + + core_ibex_env_cfg + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + core_ibex_ifetch_if ifetch_vif + IF -> ID/EXInterface + + + core_ibex_fcov_if + + + + + imem_model + + + diff --git a/doc/03_reference/index.rst b/doc/03_reference/index.rst index 3cb9655f..aa36abfb 100644 --- a/doc/03_reference/index.rst +++ b/doc/03_reference/index.rst @@ -23,6 +23,7 @@ It describes the design in detail, discusses the verification approach and the r tracer verification cosim + testplan coverage_plan rvfi history diff --git a/doc/03_reference/testplan.rst b/doc/03_reference/testplan.rst new file mode 100644 index 00000000..88927610 --- /dev/null +++ b/doc/03_reference/testplan.rst @@ -0,0 +1,109 @@ +.. _testplan: + +.. todo:: + + Add detail about security hardening verification. + +.. note:: + + This testplan is a work in progress still being implemented so this document may not match the implemented verification in the repository. + +Test Plan +========= + +Goals +----- + +* Verify compliance with all the RISC-V specifications Ibex supports. +* Verify Ibex's security hardening features. +* Ensure correct functionality is maintained across all possible behaviours of external interfaces (interrupts, memory responses, debug requests etc). +* Hit all functional coverage points, described in :ref:`coverage-plan`. + +Testbench Architecture +---------------------- + +.. figure:: images/tb2.svg + :alt: Testbench Architecture + + Architecture of the UVM testbench for Ibex core + +Ibex utilises a co-simulation checking approach described in detail in :ref:`cosim`. +With the co-simulation system all instructions Ibex executes and all external events such as an interrupts or memory errors are fed to a golden model. +The results of every instruction execution and every memory access are crossed checked against the golden model with any mismatches resulting in a test failure. +The aim is to check all possible externally observable behaviours of ``ibex_top`` against the golden model. +The golden model used is the `Spike RISC-V ISS `_. + +The testbench uses UVM. +It consists of 3 agents: + +Co-simulation Agent: + This has multiple monitors. + One monitors the RVFI interface which provides details of retired instructions. + The other monitors relate to fetched instructions and instruction memory errors; more details are provided in :ref:`coverage-plan`. + Additionally it connects to the monitor of the Memory Interface Agent for the instruction and data side via analysis ports. + The monitored transactions are used by a scoreboard to provide information to the co-simulation system allowing it to step the golden model and check its execution and memory activity against Ibex's behaviour. + +Memory Interface Agent + This provides a driver and a monitor for the :ref:`Ibex Memory Interface Protocol`. + The driver provides fully randomised and configurable timings for responses and randomisation of error responses. + Two agents are instantiated; one for the data memory interface the other for the instruction memory interface. + Read data for memory responses is provided from a backing memory; write requests update the contents of the backing memory. + This is separate from the memory used by the golden model in the co-simulation agent. + The contents of these two memories will be identical unless there is a mismatch resulting in a failure. + The backing memory is held in a memory model as a separate UVM component. + The two agents use the same backing memory so they have a coherent view of memory. + +IRQ Agent + This provides a driver and a monitor for the IRQ interface. + It provides randomised interrupt stimulus to Ibex when a test requests it. + Constraints can be used to control types of interrupts generated (e.g. NMI or not) and whether multiple interrupts should be raised together. + +Debug and reset signals are a single wire each so do not have a dedicated agent. +Instead any sequence that wishes to use them will directly manipulate them via a virtual interface + +The testbench instantiates the agents described above along with the memory model used by both the data and instruction side memory agents. +A test consists of executing a pre-built binary (which is loaded into the memory model at the start of the test via backdoor accesses) along with configuring agents to provide appropriate stimulus for the test. +Some tests may use the agents to generate stimulus at particular times (e.g. interrupts). +A test may perform additional checking on top of the co-simulation golden model comparison where appropriate (e.g. ensuring a raised interrupt has caused an exception). + +Stimulus Strategy +----------------- + +Stimulus falls into two categories: + +* Instructions to execute: These are generated by the `RISC-V DV random instruction `_ generator and provided to the testbench via a raw binary file. +* Activity on external interfaces. + +Instructions are generated ahead of time so the test has no control over them at run time. +All external interfaces have their stimulus generated at run time so can be controlled by the test. +It is the responsibility of the regression run environment to ensure generated instructions are matched with appropriate tests (e.g. ensuring an exception handler is present where interrupts are expected). + +Stimulus generation will use a coverage based approach. +Stimulus is developed based upon the :ref:`coverage-plan`. +Where coverage is not being hit stimulus will be added to hit it. + +Tests +----- + +As with stimulus, test sequence development uses a coverage based approach. +Tests will be added such that all coverage in the :ref:`coverage-plan` can be hit. +Not all the details of specific tests will be documented here. +The test list (`dv/uvm/core_ibex/riscv_dv_extension/testlist.yaml `_), provides an exhaustive list of all tests along with a brief description of what the test does. + +A test will execute a binary whilst running zero or more sequences that provide stimulus to external interfaces of ``ibex_top``. +As the memory interfaces are all driven by Ibex, with any testbench generated activity in response to a request from Ibex, they do not require explicit sequences run by the test. +Instead the test can configure the randomisation of memory delays as it wishes. +Memory errors can be configured to always occur in statically defined areas of the memory map or a sequence can be used to inject them via the memory interface agent. + +The following sequences are available for tests to use. +Each sequence is derived from a base sequence which provides controls to repeat the sequence at fixed or random internals, forever or after a random number of repeats. +Full details can be found in `dv/uvm/core_ibex/tests/core_ibex_seq_lib.sv `_. + +* ``irq_raise_seq`` - Raises one or more interrupts. + The testbench binary can write to a special memory location to acknowledge the interrupt and cause it to drop. + Alternatively the testbench can drop it after a given amount of time. +* ``debug_seq`` - Raises the external debug request. + The testbench binary can write to a special memory location to acknowledge the request and cause it to drop. + Alternatively the testbench can drop it after a given amount of time. +* ``mem_error_seq`` - Injects a memory error in either the instruction side or data side, so the next access results in an error response. +* ``reset_seq`` - Resets the core.