mirror of
https://github.com/lowRISC/ibex.git
synced 2025-04-24 22:07:43 -04:00
109 lines
6.8 KiB
ReStructuredText
109 lines
6.8 KiB
ReStructuredText
.. _testplan:
|
|
|
|
.. todo::
|
|
|
|
Add detail about security hardening verification.
|
|
|
|
.. note::
|
|
|
|
This testplan is a work in progress still being implemented so this document may not match the implemented verification in the repository.
|
|
|
|
Test Plan
|
|
=========
|
|
|
|
Goals
|
|
-----
|
|
|
|
* Verify compliance with all the RISC-V specifications Ibex supports.
|
|
* Verify Ibex's security hardening features.
|
|
* Ensure correct functionality is maintained across all possible behaviours of external interfaces (interrupts, memory responses, debug requests etc).
|
|
* Hit all functional coverage points, described in :ref:`coverage-plan`.
|
|
|
|
Testbench Architecture
|
|
----------------------
|
|
|
|
.. figure:: images/tb2.svg
|
|
:alt: Testbench Architecture
|
|
|
|
Architecture of the UVM testbench for Ibex core
|
|
|
|
Ibex utilises a co-simulation checking approach described in detail in :ref:`cosim`.
|
|
With the co-simulation system all instructions Ibex executes and all external events such as an interrupts or memory errors are fed to a golden model.
|
|
The results of every instruction execution and every memory access are crossed checked against the golden model with any mismatches resulting in a test failure.
|
|
The aim is to check all possible externally observable behaviours of ``ibex_top`` against the golden model.
|
|
The golden model used is the `Spike RISC-V ISS <https://github.com/riscv-software-src/riscv-isa-sim>`_.
|
|
|
|
The testbench uses UVM.
|
|
It consists of 3 agents:
|
|
|
|
Co-simulation Agent:
|
|
This has multiple monitors.
|
|
One monitors the RVFI interface which provides details of retired instructions.
|
|
The other monitors relate to fetched instructions and instruction memory errors; more details are provided in :ref:`coverage-plan`.
|
|
Additionally it connects to the monitor of the Memory Interface Agent for the instruction and data side via analysis ports.
|
|
The monitored transactions are used by a scoreboard to provide information to the co-simulation system allowing it to step the golden model and check its execution and memory activity against Ibex's behaviour.
|
|
|
|
Memory Interface Agent
|
|
This provides a driver and a monitor for the :ref:`Ibex Memory Interface Protocol<lsu-protocol>`.
|
|
The driver provides fully randomised and configurable timings for responses and randomisation of error responses.
|
|
Two agents are instantiated; one for the data memory interface the other for the instruction memory interface.
|
|
Read data for memory responses is provided from a backing memory; write requests update the contents of the backing memory.
|
|
This is separate from the memory used by the golden model in the co-simulation agent.
|
|
The contents of these two memories will be identical unless there is a mismatch resulting in a failure.
|
|
The backing memory is held in a memory model as a separate UVM component.
|
|
The two agents use the same backing memory so they have a coherent view of memory.
|
|
|
|
IRQ Agent
|
|
This provides a driver and a monitor for the IRQ interface.
|
|
It provides randomised interrupt stimulus to Ibex when a test requests it.
|
|
Constraints can be used to control types of interrupts generated (e.g. NMI or not) and whether multiple interrupts should be raised together.
|
|
|
|
Debug and reset signals are a single wire each so do not have a dedicated agent.
|
|
Instead any sequence that wishes to use them will directly manipulate them via a virtual interface
|
|
|
|
The testbench instantiates the agents described above along with the memory model used by both the data and instruction side memory agents.
|
|
A test consists of executing a pre-built binary (which is loaded into the memory model at the start of the test via backdoor accesses) along with configuring agents to provide appropriate stimulus for the test.
|
|
Some tests may use the agents to generate stimulus at particular times (e.g. interrupts).
|
|
A test may perform additional checking on top of the co-simulation golden model comparison where appropriate (e.g. ensuring a raised interrupt has caused an exception).
|
|
|
|
Stimulus Strategy
|
|
-----------------
|
|
|
|
Stimulus falls into two categories:
|
|
|
|
* Instructions to execute: These are generated by the `RISC-V DV random instruction <https://github.com/google/riscv-dv>`_ generator and provided to the testbench via a raw binary file.
|
|
* Activity on external interfaces.
|
|
|
|
Instructions are generated ahead of time so the test has no control over them at run time.
|
|
All external interfaces have their stimulus generated at run time so can be controlled by the test.
|
|
It is the responsibility of the regression run environment to ensure generated instructions are matched with appropriate tests (e.g. ensuring an exception handler is present where interrupts are expected).
|
|
|
|
Stimulus generation will use a coverage based approach.
|
|
Stimulus is developed based upon the :ref:`coverage-plan`.
|
|
Where coverage is not being hit stimulus will be added to hit it.
|
|
|
|
Tests
|
|
-----
|
|
|
|
As with stimulus, test sequence development uses a coverage based approach.
|
|
Tests will be added such that all coverage in the :ref:`coverage-plan` can be hit.
|
|
Not all the details of specific tests will be documented here.
|
|
The test list (`dv/uvm/core_ibex/riscv_dv_extension/testlist.yaml <https://github.com/lowRISC/ibex/blob/master/dv/uvm/core_ibex/riscv_dv_extension/testlist.yaml>`_), provides an exhaustive list of all tests along with a brief description of what the test does.
|
|
|
|
A test will execute a binary whilst running zero or more sequences that provide stimulus to external interfaces of ``ibex_top``.
|
|
As the memory interfaces are all driven by Ibex, with any testbench generated activity in response to a request from Ibex, they do not require explicit sequences run by the test.
|
|
Instead the test can configure the randomisation of memory delays as it wishes.
|
|
Memory errors can be configured to always occur in statically defined areas of the memory map or a sequence can be used to inject them via the memory interface agent.
|
|
|
|
The following sequences are available for tests to use.
|
|
Each sequence is derived from a base sequence which provides controls to repeat the sequence at fixed or random internals, forever or after a random number of repeats.
|
|
Full details can be found in `dv/uvm/core_ibex/tests/core_ibex_seq_lib.sv <https://github.com/lowRISC/ibex/blob/master/dv/uvm/core_ibex/tests/core_ibex_seq_lib.sv>`_.
|
|
|
|
* ``irq_raise_seq`` - Raises one or more interrupts.
|
|
The testbench binary can write to a special memory location to acknowledge the interrupt and cause it to drop.
|
|
Alternatively the testbench can drop it after a given amount of time.
|
|
* ``debug_seq`` - Raises the external debug request.
|
|
The testbench binary can write to a special memory location to acknowledge the request and cause it to drop.
|
|
Alternatively the testbench can drop it after a given amount of time.
|
|
* ``mem_error_seq`` - Injects a memory error in either the instruction side or data side, so the next access results in an error response.
|
|
* ``reset_seq`` - Resets the core.
|