documentation updates

README.md

@@ -1,6 +1,6 @@
 # Vortex GPGPU
 
-Vortex is a full-stack open-source RISC-V GPGPU. Vortex supports multiple *backend drivers*, including our C++ simulator (simx), an RTL simulator, and physical Xilinx and Altera FPGAs-- all controlled by a single driver script. The chosen driver determines the corresponding code invoked to run Vortex. Generally, developers will prototype their intended design in simx, before completing going forward with an RTL implementation. Alternatively, you can get up and running by selecting a driver of your choice and running a demo program.
+Vortex is a full-stack open-source RISC-V GPGPU. Vortex supports multiple **backend drivers**, including our C++ simulator (simx), an RTL simulator, and physical Xilinx and Altera FPGAs-- all controlled by a single driver script. The chosen driver determines the corresponding code invoked to run Vortex. Generally, developers will prototype their intended design in simx, before completing going forward with an RTL implementation. Alternatively, you can get up and running by selecting a driver of your choice and running a demo program.
 
 ## Website
 Vortex news can be found on its [website](https://vortex.cc.gatech.edu/)

docs/fpga_setup.md

@@ -52,9 +52,9 @@ To request 16 cores and 64GB of RAM for 6 hours on flubber9, a fpga dev node:
 ```bash
 salloc -p rg-fpga --nodes=1 --ntasks-per-node=16 --mem=64G --nodelist flubber9 --time=06:00:00
 ```
-
-## Environment Setup
-Once you are logged in, you will need to complete some first time configurations.
+Synthesis for Xilinx Boards
+----------------------
+Once you are logged in, you will need to complete some first time configurations. If you are interested in the Intel (Altera) synthesis steps, scroll down below.
 
 ### Source Configuration Scripts
 ```
@@ -89,7 +89,7 @@ The build is complete when the bitstream file `vortex_afu.xclbin` exists in `
 
 ### Running a Program on Xilinx FPGA
 
-The blackbox.sh script in `ci` can be used to run a test with Vortex’s xrt driver using the following command:
+The [blackbox.sh](./simulation.md) script within the build directory can be used to run a test with Vortex’s xrt driver using the following command:
 
 `FPGA_BIN_DIR=<path to bitstream directory> TARGET=hw|hw_emu PLATFORM=<platform baseName> ./ci/blackbox.sh --driver=xrt --app=<test name>`
 
@@ -97,20 +97,12 @@ For example:
 
 ```FPGA_BIN_DIR=<realpath> hw/syn/xilinx/xrt/build_4c_u280_xilinx_u280_gen3x16_xdma_1_202211_1_hw/bin TARGET=hw PLATFORM=xilinx_u280_gen3x16_xdma_1_202211_1 ./ci/blackbox.sh --driver=xrt --app=demo```
 
-### Synthesis for Intel (Altera) Boards
-
-To set up the environment, source the XRT setup.sh and other Xilinx scripts. For example:
-
-```
-source /opt/xilinx/xrt/setup.sh
-source /tools/reconfig/xilinx/Vivado/2022.1/settings64.sh
-source /tools/reconfig/xilinx/Vitis/2022.1/settings64.sh
-
-```
-
-OPAE Environment Setup
+Synthesis for Intel (Altera) Boards
 ----------------------
 
+### OPAE Environment Setup
+
+
     $ source /opt/inteldevstack/init_env_user.sh
     $ export OPAE_HOME=/opt/opae/1.1.2
     $ export PATH=$OPAE_HOME/bin:$PATH
@@ -118,8 +110,7 @@ OPAE Environment Setup
     $ export LIBRARY_PATH=$OPAE_HOME/lib:$LIBRARY_PATH
     $ export LD_LIBRARY_PATH=$OPAE_HOME/lib:$LD_LIBRARY_PATH
 
-OPAE Build
-------------------
+### OPAE Build
 
 The FPGA has to following configuration options:
 - DEVICE_FAMILY=arria10 | stratix10
@@ -134,8 +125,7 @@ A new folder (ex: `test1_xxx_4c`) will be created and the build will start and t
 Setting TARGET=ase will build the project for simulation using Intel ASE.
 
 
-OPAE Build Configuration
-------------------------
+### OPAE Build Configuration
 
 The hardware configuration file `/hw/rtl/VX_config.vh` defines all the hardware parameters that can be modified when build the processor.For example, have the following parameters that can be configured:
 - `NUM_WARPS`:   Number of warps per cores
@@ -146,8 +136,7 @@ You configure the syntesis build from the command line:
 
     $ CONFIGS="-DPERF_ENABLE -DNUM_THREADS=8" make
 
-OPAE Build Progress
--------------------
+### OPAE Build Progress
 
 You could check the last 10 lines in the build log for possible errors until build completion.
 
@@ -166,17 +155,40 @@ The file `vortex_afu.gbs` should exist when the build is done:
     $ ls -lsa <build_dir>/synth/vortex_afu.gbs
 
 
-Signing the bitstream and Programming the FPGA
-----------------------------------------------
+### Signing the bitstream and Programming the FPGA
 
     $ cd <build_dir>
     $ PACSign PR -t UPDATE -H openssl_manager -i vortex_afu.gbs -o vortex_afu_unsigned_ssl.gbs
     $ fpgasupdate vortex_afu_unsigned_ssl.gbs
 
-FPGA sample test running OpenCL sgemm kernel
---------------------------------------------
+### Sample FPGA Run Test
+Ensure you have the correct opae runtime for the FPGA target
 
-Run the following from the Vortex root directory
+```
+$ TARGET=FPGA make -C runtime/opae
+```
+
+Run the [blackbox.sh](./simulation.md) from your Vortex build directory
+
+```
+$ TARGET=fpga ./ci/blackbox.sh --driver=opae --app=sgemm --args="-n128"
+```
+
+### FPGA sample test running OpenCL sgemm kernel
+
+You can use the `blackbox.sh` script to run the following from your Vortex build directory
 
     $ TARGET=fpga ./ci/blackbox.sh --driver=opae --app=sgemm --args="-n128"
 
+### Testing Vortex using OPAE with Intel ASE Simulation
+Building ASE synthesis
+
+```$ TARGET=asesim make -C runtime/opae```
+
+Building ASE runtime
+
+```$ TARGET=asesim make -C runtime/opae```
+
+Running ASE simulation
+
+```$ ASE_LOG=0 ASE_WORKDIR=<build_dir>/synth/work TARGET=asesim ./ci/blackbox.sh --driver=opae --app=sgemm --args="-n16"```

docs/index.md

@@ -2,9 +2,8 @@
 
 ## Table of Contents
 
-- [Codebase Layout](codebase.md)
-- [Microarchitecture](microarchitecture.md)
-- [Cache Subsystem](cache_subsystem.md)
-- [Simulation](simulation.md)
-- [Contributing](contributing.md)
-- [Debugging](debugging.md)
+- [Codebase Layout](codebase.md): Summary of repo file tree
+- [Microarchitecture](microarchitecture.md): Vortex Pipeline and cache microarchitectural details and reconfigurability
+- [Simulation](simulation.md): Details for building and running each simulation driver
+- [Contributing](contributing.md): Process for contributing your own features including repo semantics and testing
+- [Debugging](debugging.md): Debugging configurations for each Vortex driver

docs/microarchitecture.md

@@ -77,4 +77,7 @@ Vortex has a 6-stage pipeline:
 - Sockets
   - Grouping multiple cores sharing L1 cache
 - Clusters
-  - Grouping of sockets sharing L2 cache
+  - Grouping of sockets sharing L2 cache
+
+### Vortex Cache Subsystem
+More details about the cache subsystem are provided [here](./cache_subsystem.md).

docs/simulation.md

@@ -15,7 +15,7 @@ SimX is a C++ cycle-level in-house simulator developed for Vortex. The relevant
 
 The guide to build the fpga with specific configurations is located [here.](fpga_setup.md) You can find instructions for both Xilinx and Altera based FPGAs.
 
-### How to Test
+### How to Test (using `blackbox.sh`)
 
 Running tests under specific drivers (rtlsim,simx,fpga) is done using the script named `blackbox.sh` located in the `ci` folder. Running command `./ci/blackbox.sh --help` from the Vortex root directory will display the following command line arguments for `blackbox.sh`:
 
@@ -54,7 +54,8 @@ PERF: instrs=363180, cycles=53108, IPC=6.838518
 
 ## Additional Quick Start Scenarios
 
-Running Vortex simulators with different configurations:
+Running Vortex simulators with different configurations and drivers is supported. For example:
+
 - Run basic driver test with rtlsim driver and Vortex config of 2 clusters, 2 cores, 2 warps, 4 threads
 
     $ ./ci/blackbox.sh --driver=rtlsim --clusters=2 --cores=2 --warps=2 --threads=4  --app=basic

docs/testing.md

@@ -2,7 +2,7 @@
 
 ## Running a Vortex application
 
-The framework provides a utility script: blackbox.sh under the /ci/ folder for executing applications in the tests tree.
+The framework provides a utility script: blackbox.sh under the /ci/ folder for executing applications in the tests tree. It gets copied into the `build` directory with all the environment variables resolved, so you should run it from the `build` directory as follows:
 You can query the commandline options of the tool using:
 
     $ ./ci/blackbox.sh --help
@@ -49,4 +49,4 @@ Compile your test: `$ make -C tests/regression/<test-name>`
 Run your test: `$ ./ci/blackbox.sh --driver=simx --app=<test-name> --debug`
 
 ## Adding Your Tests to the CI Pipeline
-If you are a contributor, then you will need to add tests that integrate into the continuous integration pipeline. Remember, Pull Requests cannot be merged unless new code has tests and existing tests do not regress. See more at [contributing.md](contributing.md) and [continuous_integration.md](continuous_integration.md).
+If you are a contributor, then you will need to add tests that integrate into the continuous integration pipeline. Remember, Pull Requests cannot be merged unless new code has tests and existing tests do not regress. Furthermore, if you are contributing a new feature, it is recommended that you add the ability to enable / disable the new feature that you are adding. See more at [contributing.md](contributing.md) and [continuous_integration.md](continuous_integration.md).