Direct notes from talks on uBoot, UEFI, and new board spinup on uboot --- # UEFI in U-boot efi_loader: UEFI interface translation layer for uboot How do we call it though? `bootefi` requires a kernel and devicetree loaded into memory. call `bootefi $kernel_addr_r $fdt_addr_r` This runs an EFI stub, and boots as normal. You also need a `setenv bootargs console=tty earlycon` for boot debugging so you can watch it boot properly. `fdt chosen` updates current devicetree Seems like it might not require an initramfs? Enables bidi interface for UEFI, which is increasingly more important. # distroboot `boot_targets=mmc0 usb0 pxe dhcp` defines which targets to look for. Search for `extlinux` file, boot script, or an `EFI` executable. `EFI` assumes first partition is EFI System Partition uboot knows your wanted devicetree, so it searches under: - `/` - `/dtb/` - `/dtb/current/` - `/boot/` - `/boot/dtb/` - `/boot/dtb/current/` ##SD Also looks for `efi/boot/boot(arch).efi` "Is that signed?" - we'll come back to it. Doable, not implemented at 2017 Loads grub, plays like normal. Grub can load a kernel, an initrd, an initramfs, whatever grub normally does. ## ISO We have SD cards implemented, which is all we need, right? Well, not really. We need ISO and Network support too. Some PowerPC forks hacked in a solution. There's a module for uboot that exposes an ISO as a partition table. Then we can just play like normal. Stable code since 2016ish. ## networkboot 2 kinds, PXE boot and DHCP boot. PXE is extlinux specific. DHCP is what you would call PXE. Sends a DHCP request, with a hint token saying "I'm an EFI system on this architecture" in the "Vendor [something] specific interface". REturns a filename property in its ACK, which is then downloaded from a TFTP server. Works the same as tianocore or AMI. # EFI Tables Pointers to binary blobs with a UUID prepended. DeviceTree is in there, ACPI is in there, etc. U-Boot doesn't implement ACPI. "If anybody wants to implement that in a uboot environment, be my guest! But I don't know why you would want to do that" This is where DeviceTree is stored and used after initial boot! And since you can only have one at a time, this makes DeviceTree compatible. # What doesn't work? - NVRAM runtime services No generic RTC interface, no configurable boot order (which.... we need that???) - Editable object bucket Loading stuff into Uboot gets thrown out in the next use; can't boot from BTRFS - Libraries in object bucket ... which means you don't have an EFI shell either. But we're in embedded land, so its not that useful anyways. # Why? Complete separation between hardware and software Removes generic implementation value add from firmware, takes it back to software Current situation is tweaking uboot until it doesn't behave normally, or it becomes a maintenance mess EFI is a generic value add, and it means you can switch to a different architecture "One day when I get around to it I'll write a mainframe port" removes dependency on Linux as well uboot as OS "Don't reinvent the wheel" "How do we get from a firmware that goes away to a firmware that stays resident?" Specially marked points in functions so they don't get overwritten Linux can also move those runtimes around, so we need to know about that Optional implementation, but no real overhead PSCI vs EFI PSCI lives in EO3, EO2 is hypervisor, EO1 is system mode, EO0 is userspace. PSCI and EFI live in EO3, so it can't get messed with by the kernel ------- New board running uboot: 1. RAM 2. Serial There's an internal printlog for the kernel and uboot; you can listen to this by JTAG 3. Networking Netboot means you're no longer the problem Ethernet driver and PHY are needed here. If you're lucky, you have a driver from a similar SOC 1. Or just get PCI working, then use an existing PCI-driven network driver. This isn't necessarily easier though But I don't have network, or no TFTP allowed, or ethernet is not working in A0 Option 1: SPI programmable via JTAG Option 2: Removable SPI chip (very very tedious) Uboot needs drivers for SPI or NAND to boot 4. Other peripherals 5. Diagnostics (uboot has a POST option)o Make sure your uboot partition is generous, and/or check for that size beforehand Upstreaming? 1. Mailing list Push early, push often. Each patch needs to be a singular logical change Follow the model inhouse so you can just push direct from internal Run checkpatch! 2. Upstreaming is good for your project, try to push for it early. You might need to upstream until after release. 1. Start with a rebase to latest version. This might just magically work! ....... or maybe not. In which case start from scratch. 2. Squash No one cares externally 3. Carve into submittable chunks 1. Basic enablement 2. DeviceTree 3. Individual driver in a separate patch 4. diags and anything else (might need company permissions) If you've released this thing, it's GPLv2, it's no longer proprietary. --- Intro to U-Boot; behan webster FSBL lives in the BootROM of the chip. Initialises HW, and loads U-Boot first U-Boot SPL (minimal, for HW init and loading U-Boot/linux direct) then U-Boot proper (shell, boot monitor, debug tool, etc) U-Boot SPL printout is unique from U-Boot proper boot log OneNAND(?) TPL is what we're developing (EDK2) ## Uboot shell 2 shells; original (no name) and HUSH Similar to bourne, persistent env support scripting support `echo` does not interpret control sequences (except `\\c` to suppress newline `bdinfo` prints a lot of the envvars in a pretty way More docs at https://git.denx.de/?p=u-boot.git;a=tree;f=doc Also available on irc under #u-boot, and a mailing list `mw` (Memory Write) `md` (Memory Display) for memory access commands `mm` (memory modify) `nm` nm is single entry `cp` copy memory `cmp` compare memory `bootz` for Image or zImage `booti` for a standard Image `bootm` for uImage or fitImage fdt can be manipulated once loaded https://beagleboard.org/pocket https://beagleboard.org/techlab Use QEMU for testing configs --- # U-Boot (and EDK2) porting Step 1: Get RAM PHYs setup (not necessary if we're loading from Uboot-SPL) Step 2: get serial working (Probably not necessary?) Step 3: networking (N/A) Step 4: Diag (use u-boot if possible?) --- ## U-boot early stages: Platform-specific reset vector code `crt0.S`: C runtime setup first C code run is `common/board_f.c` (u-boot running from FLASH); first item is to load the devicetree Next is `common/board_r.c` execute u-boot running from RAM; autoboot prompt shows up here `lib/initcall.c` is a nice debug aid. replace debug statements with printf, and it'll print everything DT access: - `fdt_*()` functions in `include/fdt_support.h`; rudamentary - `fdtdec_*()` functions in `include/fdtdec.h`; convenience wrappers around the above - `dev_read_*()` DriverModel-specific DT access functions (`include/dm/read.h`) DriverModel: a way to transition from `#ifdef` into something reasonable; consists of - classes: groups of devices with the the same behaviour (uclass) - drivers: code talks to device and presents standard high-level interface for class; "specific register poking" - devices: each device with a fitting driver gets an instance DriverModel core: - handles device life-cycle - inherently lazy for low boot time - creates the root driver; everything else under that. Scans stuff, but doesn't initialise anything `dm tree` lists all drivers found and initialised (these are mutually exclusive) Driver lifecycle: - defined by `U_BOOT_DRIVER` macro - pre-allocate private data if necessary - .ofdata_to_platdata converts data to generic platform-level data - .bind binds the driver to DM, but does not activate it - .probe activates the driver - .remove deactivates - .unbind unbinds Porting to a new board 101: start small: Clock, pinmux, and serial drivers are all you need for initial boot and serial console Debugging is easier with smaller patches start with hardcoding in DT Clock and pinctrl, design serial. Transition clock away from DT Populate `board/mymfg/myboard` - `configs/myboard_defconfig` - `include/configs/myboard.h` - `board/mymfg/myboard/Kconfig` - `board/mymfg/myboard/myboard.c` (ideally empty/placeholder file) - `arch/foo/mach-bar/Kconfig` entry - check `common/board_f.c` and `common/board_r.c` for hooks - most code should live in drivers `board/mymfg/myboard/myboard.c` ```c //Include statements DECLARE_GLOBAL_DATA_PTR; int dram_init(void) { //GD is global data pointer //Check include/asm-generic/global_data.h for more info gd->ram_size = imx_ddr_size(); return 0; } int board_init(void) { return 0; } ``` `board/mymfg/myboard/Kconfig` ``` if TARGET_MY_BOARD # Define where to find the make file config SYS_BOARD default "my_board" # Define where to find the make file config SYS_VENDOR default "my_vendor" # Identifies board header file # include/configs/SYS_CONFIG_NAME.h config SYS_CONFIG_NAME default "my_board" endif ``` `Makefile` ```make obj-y := my_board.o ``` `my_board_defconfig` ``` CONFIG_RISCV=y CONFIG_ARCH_RISCV64=y CONFIG_TARGET_MY_BOARD=y CONFIG_UART=y ``` `include/configs/my_board.h` should have an `ifndev` for your board header, along with include statements and defines Add your board to `arch/xxx/Kconfig` or `arch/xxx/mach-yyy/zzz/Kconfig` Add the target to the arch Kconfig U-Boot serial driver example: ```c U_BOOT_DRIVER(serial_sh) = { .name = "serial_sh", //Can be different name than structure .id = UCLASS_SERIAL, //What driver is this? .ops = &sh_serial_ops, //match these to uclass .of_match = of_match_ptr(sh_serial_id), //support devicetree data .probe = sh_serial_probe, //Function pointer .priv_auto_alloc_size = sizeof(struct uart_port), //private info is allocated at instantiation; no malloc/calloc .ofdata_to_platdata = of_match_ptr(sh_serial_ofdata_to_platdata), //convert devicetree data into platform data .platdata_auto_alloc_size = sizeof(struct sh_serial_platdata), .flags = DM_FLAG_PRE_RELOC, //Make available early on }; ``` Special-purpose code allowing for very early prints `printch`, `printascii`, `printhex2`; most don't support this Clock framework: - uses `UCLASS_CLK` - Implements enable, disable, get and set - kinda boring Pinctrl framework: - `UCLASS_PINCTRL` - implements pinctrl_ops to configure pins; operates per-pin, per-group, and per-function - PINMUX for multiplexing - PINCONF for pin properties - `pinctrl_select_state` does the IO stuff for you MTD needs DM conversion in 2018; DM can trigger size limitations U-Boot SPL board-specific; DM and DT support is optional. `CONFIG_SPL_*` Kconfig options controls what goes in SPL versus U-Boot proper `CONFIG_TPL_*` also exists for tertiary loading --- [link to suggested documentation to read](https://stackoverflow.com/questions/69807637/is-there-a-guide-on-porting-edk2-to-a-new-arm64-platform) > After 1 month of trial and error, today I managed to bring my ARM64 platform into a UEFI Shell environment. I treat it as my 1st milestone on the EDK2 journey. Below I will try to summarize the steps I took so far, as a tentative answer to my question above. Guidance/corrections/comments are welcomed. > > 0. Get familiar with UEFI/PI spec and EDK2 implementation by reading books/specs/articles. Well, UEFI/PI specs are thousands of pages long...how to start? My main reading list is: > "Beyond Bios--Developing with the Unified Extensible Firmware Interface", 3rd ed, by Vincent Zimmer, et al. As the authors explained, the book is a kind of high level summary of the thousands-paged specs. And I find that the book is well organized for a new comer to get familiar with various UEFI related concepts. The main purpose of the 1st read (before playing with edk2 code base) is to get familiar with concepts and architectural ideas, not the details yet. Related sections need to be consulted later when reading EDK2 implementations. > EDK2 specs, including: > - EDKII User Manual > - EDKII Build Specification > - EDKII DSC/FDF/DEC/INF File Specification > Various articles on the web... > > 1. Get a reference platform which can correctly boot a FD image built from latest EDK2 source, and play with the boot manager and Shell environment a bit. In my case, I chose RPi4B. For me, this is very important, as the reference platform serves as a handrail during the whole process, that whenever I encounter bugs or have doubts, I check the source/log of the reference platform. This solves most of the problems I encountered. Btw, always generating "build log" and "build report" for both reference platform and the target platform, as the two files contains very detailed information for comparison and check. Consult the EDK2 build spec on how to generate these two files during build. > > I use the following script to build for RPi4B platform: > ``` > #!/bin/bash > > # https://github.com/tianocore/edk2-platforms#how-to-build-linux-environment > > export WORKSPACE=/home/bruin/work/tianocore > export PACKAGES_PATH=$WORKSPACE/edk2:$WORKSPACE/edk2-platforms:$WORKSPACE/edk2-non-osi > > pushd $WORKSPACE > > rm -rf ./Build/RPi4 > > source edk2/edksetup.sh > > echo "Building BaseTools..." > make -C edk2/BaseTools all > > #sudo apt install acpica-tools # iasl > # pip install antlr4-python3-runtime # -Y EXECUTION_ORDER > > echo "Building firmware for Pi4B..." > GCC5_AARCH64_PREFIX=aarch64-none-linux-gnu- build \ > -n 4 \ > -a AARCH64 \ > -p Platform/RaspberryPi/RPi4/RPi4.dsc \ > -t GCC5 \ > -b NOOPT \ > -v -d 9 -j RPi4-build.log \ > -y RPi4-build-report.txt \ > -Y PCD \ > -Y LIBRARY \ > -Y DEPEX \ > -Y HASH \ > -Y BUILD_FLAGS \ > -Y FLASH \ > -Y FIXED_ADDRESS \ > -Y EXECUTION_ORDER \ > all > ``` > How to use the build result RPI_EFI.fd on RPi4B, consult the following: > > edk2-platforms/Platform/RaspberryPi/RPi4/Readme.md > > readme.md inside https://github.com/pftf/RPi4/releases/download/v1.17/RPi4_UEFI_Firmware_v1.32.zip. btw, I need to replace the original start4.elf and fixup4.dat with the ones in the zip file, otherwise, the boot of RPi4 will fail, complaining something like below: > > RpiFirmwareGetClockRate: Get Clock Rate return: ClockRate=0 ClockId=C > ASSERT [ArasanMMCHost] /home/bruin/work/tianocore/edk2-platforms/Platform/RaspberryPi/ > Drivers/ArasanMmcHostDxe/ArasanMmcHostDxe.c(263): BaseFrequency != 0 > > It's worth to analysis the RPI_EFI.fd content to some extend, by using some UEFI utilities. I mainly use the GUI version UEFITool of sudo apt install uefitool uefitool-cli. Other tools are also available. The anotomy of RPI_EFI.fd is of help when reading EDK2 build specs for checking understanding of the concepts. > One special aspect of RPI_EFI.fd is that the 1st 128K is bl31.bin binary from ATF. I guess this is due to the special booting connfiguration methods for RPi. For my platform, I don't need such kind of packaging, I only need to build the UEFI image MY.fd, which is treated as BL33 image and packaged into fip.bin togehter with BL2 and BL31 images by ATF build script. > Another aspect to notice is the "reset vector" in the begining of the .fd file. This related to the entry point of UEFI image (and entry point of each EDK2 modules), as well as interpreting the BL instruction for AArch64. Basically, it can be summarized as below: > > The first [Components] in RPI_EFI.fd is ArmPlatformPkg/PrePi/PeiUniCore.inf, which is of MODULE_TYPE = SEC. > > What's this component: this is the first (and only) SEC (Security) module in RPi4. What the name PrePi and Pei implies? > > ... the PI spec is not tied to edk2 PEIMs, and I don't see where EDKII PEI modules are currently the only "acknowledged" silicon init environment. The edk2 tree itself seems to contain platforms that don't use the edk2 PEI module set at all, but (IIRC) jump from SEC to DXE. I believe "ArmPlatformPkg/PrePi" and "ArmVirtPkg/PrePi" are related to this. > > --- https://listman.redhat.com/archives/edk2-devel-archive/2020-November/msg00021.html > > Its entry point: all UEFI components have the same entry point (_ModuleEntryPoint). > By "component", it means either a UEFI driver and UEFI app, both are PE32 executables, usually with suffix .efi. > The .efis are converted from ELF executables (.dll) by GenFw tool: modifying the file headers. > To verify that "all components' entry point is _ModuleEntryPoint": > Check the .dll generating command line in build report (build -y ), we have two flags "aarch64-none-linux-gnu-gcc" -o xxx.dll -u _ModuleEntryPoint -Wl,-e,_ModuleEntryPoint ...: > -u: gcc --help -v|grep "undefined SYMBOL" gives -u SYMBOL --undefined SYMBOL: star with undefined reference to SYMBOL. > Wl,-e: ld --help|grep "entry" gives -e ADDRESS, --entry ADDRESS Set start address. > Check all .dll files that Entry point address == _ModuleEntryPoint: find . -type f -name "*.dll" -exec sh -c "readelf -a {} |grep -E 'Entry point address|_ModuleEntryPoint'" \; > > Its entry point is the entry point of whole UEFI FD image (i.e., from bl33_base_addr jump to this _ModuleEntryPoint): > > Topology of the UEFI Firmware File > > A UEFI Firmware File (actually a UEFI Firmware Device - FD file) is a collection of UEFI binaries encapsulated into a single image. The format of this image is defined by the Platform Initialization Specification Volume 3. A Vector Table is located at the base of this file. A 'BL' branch instruction at the base of the firwmare (location of the Reset Entry into the Vector Table) will jump to the first 'SEC' module of the UEFI Firmware Image. > > --- https://github.com/lzeng14/tianocore/wiki/ArmPkg-Debugging > > To verify the statements above: > > Disassember the reset vector (i.e., the 1st word) of generated .FD (we got offset=0x360): > > $ xxd -l 4 -e TEST.fd <== dump 4 bytes in little endian > 00000000: 140000d8 <== BL {PC}+(0xd8<<2); offset=0x360 > > Check the Entry point in .dll (we got offset=0x240): > > $ aarch64-none-elf-objdump -t ArmPlatformPrePiUniCore.dll|grep _ModuleEntryPoint > 0000000000000240 g F .text 0000000000000000 _ModuleEntryPoint > $ readelf -h ArmPlatformPrePiUniCore.dll|grep Entry > Entry point address: 0x240 > > Compare contents of two files at different offset (we got identicial content): > > $ xxd -s 0x360 -l 64 TEST.fd <== skip 0x360 bytes, dump 64 bytes > 00000360: 901e 0094 050a 0094 ea03 00aa a1cd 0a58 ...............X > 00000370: 0200 e0d2 2200 c0f2 0240 a0f2 0200 80f2 ...."....@...... > 00000380: c303 a0d2 e3ff 9ff2 6304 00d1 6300 028b ........c...c... > 00000390: 0400 a1d2 0400 80f2 2000 03eb 8400 0054 ........ ......T > $ xxd -s 0x240 -l 64 ArmPlatformPrePiUniCore.dll <== skip 0x240 bytes > 00000240: 901e 0094 050a 0094 ea03 00aa a1cd 0a58 ...............X > 00000250: 0200 e0d2 2200 c0f2 0240 a0f2 0200 80f2 ...."....@...... > 00000260: c303 a0d2 e3ff 9ff2 6304 00d1 6300 028b ........c...c... > 00000270: 0400 a1d2 0400 80f2 2000 03eb 8400 0054 ........ ......T > > Prepare an empty pkg, and make it build ok. The main purpuse is to do some exercise with EDK2 build system, and use the empty pkg as the start point for the new platform. > > Make a copy of RaspberryPi.dec, change all gRaspberry to gMyPlatform. > > Make a copy of RPi4.dsc and RPi4.fdf, and comment out all stuff in DSC and FDF file. > > Replace all GUIDs in DSC/FDF/DEC files, generating new ones using online guid generator. > > Note that PCD are declared in DEC files, and DEC files are refered by modules (INF files). As the empty package contains no module, no PCD definition will be available in FDF. So for a success build of the empty package, we need to comment out all PCD reference in FDF. > > The NOOPT build command for MyPlatform is as below: > > #!/bin/bash > > export WORKSPACE=/home/bruin/work/tianocore > export PACKAGES_PATH=$WORKSPACE/edk2:$WORKSPACE/edk2-platforms:$WORKSPACE/edk2-non-osi > pushd $WORKSPACE > source edk2/edksetup.sh > > echo "Building BaseTools..." > make -C edk2/BaseTools all > > echo "Building UEFI firmware for MyPlatform..." > GCC5_AARCH64_PREFIX=aarch64-none-linux-gnu- build \ > -n 4 \ > -a AARCH64 \ > -p Platform/MyCorp/MyPlatform/MyPlatform.dsc \ > -t GCC5 \ > -b NOOPT \ > -v -d 9 -j MyPlatform-build.log \ > -y MyPlatform-build-report.txt \ > -Y EXECUTION_ORDER \ > -Y PCD \ > -Y LIBRARY \ > -Y DEPEX \ > -Y HASH \ > -Y BUILD_FLAGS \ > -Y FLASH \ > -Y FIXED_ADDRESS \ > all > popd > > Add the 1st component ArmPlatformPrePiUniCore. This component is to prepare the HOBs for DXE phae. The main purpose is to get serial port working and memory config correct. Another purpose of this step is to familiar with steps for adding a component/module/lib. Below is a brief summary of the steps: > Uncomment the module's INF into both DSC ([Components] section), and FDF ([FV.FVMAIN_COMPACT]). > Rebuild the pkg, and resolve all Instance of library class [xxxLib] is not found errors reported, by updating [LibraryClasses] sections of DSC. > This step is a repeating process for dozens of times. > Some lib-class has multiple lib-instances, making sure choose the appropriate lib-instance (ref the build-report of RPi4). > if encounter ModuleEntryPoint.iiii:31: Error: immediate out of range: enable gArmTokenSpaceGuid.PcdFdBaseAddress and gArmTokenSpaceGuid.PcdFdSize in FDF. > if encounter undefined reference to _gPcd_BinaryPatch_PcdSerialClockRate: set PcdSerialClockRate in [PcdsPatchableInModule] section in DSC. FIXME: why? ref. > Check the PCDs listed in build log: inspect any abnormal PCD values, and supply correct values. > Customize platform-specific drivers or libraries. > SerialPortLib: locate the lib-class header file (MdePkg/Include/Library/SerialPortLib.h) by find edk2 -type f -name "*.dec" -exec grep -Hn SerialPortLib. The following functions are required: > SerialPortInitialize() > SerialPortWrite() > SerialPortRead() > SerialPortPoll() > SerialPortSetControl(): RETURN_UNSUPPORTED > SerialPortGetControl(): RETURN_UNSUPPORTED > SerialPortSetAttributes(): RETURN_UNSUPPORTED > ArmPlatformLib: interface header at Include/Library/ArmPlatformLib.h. The following functions are required: > ArmPlatformGetCorePosition(): return cpu idx in the cluster given the MPIDR value. this function is used in _ModuleEntryPoint for setting stack for secondary cores. Assuming one cluster for now. > ArmPlatformIsPrimaryCore() > ArmPlatformGetPrimaryCoreMpId() > ArmPlatformGetBootMode() > ArmPlatformPeiBootAction() > ArmPlatformInitialize() > ArmPlatformGetVirtualMemoryMap() > ArmPlatformGetPlatformPpiList() > etc... > > Uncomment more modules in DSC/FDF, module by module...For driver/libs which are RPi platform specific, we can: > either search the edk2/edk2-platform for similiar driver or lib instances, or > copy the RPi4 implementation and comment out most of the content, make the pkg build success first, and then bug fixing. > > Debugging: my current main debugging method is through adding "printf()", i.e., the edk2 macro DEBUG((DEBUG_INFO,)). One needs to set gEfiMdePkgTokenSpaceGuid.PcdDebugPrintErrorLevel to an appropriate value to see more debug info. >