Support editing Linux image and toolchain configuration

[paulmnt]

Moving discussion from #61

[paulmnt]

I wonder when we build code for FPGAs like VCU128, which of the following variants will be used?

esp/third-party/ariane/
esp/third-party/ariane/fpga/
esp/third-party/ariane/openpiton/
esp/soft/ariane/

ESP includes the entire Ariane's repo, including its submodules, but it only compiles RTL source files. The list of source files compiled is here.

As you can see, we use ariane/src and some files from ariane/fpga. The ariane/openpiton source files are only needed if you build the OpenPiton system with Ariane, but not for ESP.

Note that when compiling Ariane's source files, we set the flag to use the write-through cache subsystem and we keep the original AXI interface from Ariane, not the L1.5 adapter.

The esp/soft/ariane folder contains just software, including the boot loader, Linux and the ESP device drivers. These are built by the make linux and make soft targets.

[Kendidi]

Thank you @paulmnt !

If I want to customize an ESP build with Ethernet and SPI disabled, and with Bootrom code modified, what's the best way to approach? I tried "make grlib-xconfig" but I wasn't able to find SPI options there.

This is for an experiment with OpenOCD and GDB. Thanks.

[paulmnt]

SPI is not enabled nor actually supported at the moment. We don't have the SPI controller instance in any of the ESP tiles.

Ethernet, on the other hand, is required, because we use it as a debug interface. Without the Ethernet block, you would not be able to load programs in memory and reset the processor to start executing. You can disable Ethernet for Linux by removing it from the device tree ariane.dts, or by deleting the S40 and S50 init scripts from sysroot/etc/init.d, but you must keep the module in the FPGA bitstream.

OpenOCD and GDB cannot be used to load programs and debug the processor core at this time. If you wish for ESP to support that, feel free to open a new issue. This implies adding the RISC-V debug unit to the design and, while we can't guarantee an exact timeline, we will certainly add it to the list of features to implement.

[Kendidi]

@paulmnt

I see. Thank you very much for the info.!

[Kendidi]

Not sure if this it the right pace for this question. If not, please let me know.

After Linux has booted up, I issued command "lsmod" to see what modules are loaded and the following were listed.

vitbfly2 2843 0 - Live 0xffffffd004196000 (O)                                                                                                                                        
visionchip 3073 0 - Live 0xffffffd00418f000 (O)                                                                                                                                      
synth 3084 0 - Live 0xffffffd004188000 (O)                                                                                                                                           
spmv 3109 0 - Live 0xffffffd004181000 (O)                                                                                                                                            
sort 3072 0 - Live 0xffffffd004078000 (O)                                                                                                                                            
fft 2946 0 - Live 0xffffffd004071000 (O)                                                                                                                                             
dummy 2892 0 - Live 0xffffffd00406a000 (O)                                                                                                                                           
adder 2892 0 - Live 0xffffffd004063000 (O)                                                                                                                                           
FFTAccelerator 3001 0 - Live 0xffffffd00405c000 (O)                                                                                                                                  
CounterAccelerator 2957 0 - Live 0xffffffd004055000 (O)                                                                                                                              
AdderAccelerator 3011 0 - Live 0xffffffd00404e000 (O)                                                                                                                                
esp 9520 11 vitbfly2,visionchip,synth,spmv,sort,fft,dummy,adder,FFTAccelerator,CounterAccelerator,AdderAccelerator, Live 0xffffffd004043000 (O)                                      
esp_private_cache 4308 0 - Live 0xffffffd004037000 (O)                                                                                                                               
esp_cache 3901 0 - Live 0xffffffd00400d000 (O)                                                                                                                                       
contig_alloc 10954 1 esp, Live 0xffffffd004009000 (O)  

I was expecting to find other drivers like networking or serial communications, etc. Are they not loaded or are built-in instead?

And if I want to add functions like "modprobe" or "modinfo", what will be the right approach? Thanks a lot in advance!

[paulmnt]

The modules you see with lsmod are the ones that we load after the kernel has booted using insmod. There is a script in sysroot/etc/init.d that does so. Those are ESP-specific and include the drivers of the example accelerators that we provide within the ESP repository.

The other drivers that you mentioned are built-in, as you guessed.

To enable modprobe, you should change the busybox configuration script esp/utils/scripts/riscv_busybox.config

#                                                                                                                                                                                                                                             
# Linux Module Utilities                                                                                                                                                                                                                      
#                                                                                                                                                                                                                                             
CONFIG_MODPROBE_SMALL=y
# CONFIG_DEPMOD is not set                                                                                                                                                                                                                    
CONFIG_INSMOD=y
CONFIG_LSMOD=y
# CONFIG_FEATURE_LSMOD_PRETTY_2_6_OUTPUT is not set                                                                                                                                                                                           
CONFIG_MODINFO=y                                                                                                                                                                                                          
CONFIG_MODPROBE=y                                                                                                                                                                                                               
# CONFIG_FEATURE_MODPROBE_BLACKLIST is not set                                                                                                                                                                                                
CONFIG_RMMOD=y

Then rebuild sysroot with esp/utils/scripts/build_riscv_toolchain.sh. You can skip all steps related to the toolchain, but you must run the last one: *** Populating root file system w/ buildroot ... ***

[Kendidi]

@paulmnt

Cool! Thank you!!

How about those applications (e.g. FIO) that are not in busybox configuration files?

[Kendidi]

By the way, I just made some adjustment in esp\soft\ariane\linux\arch\riscv\configs\ariane_defconfig.
When I run make linux, it started asking me a lot of config questions. I think I may have broke something. Is there a way to recover that?

[paulmnt]

How about those applications (e.g. FIO) that are not in busybox configuration files?

We are booting just the kernel with a minimal disk image, so there is no packet manager (e.g.apt). You need to get the source files and cross compile the application for RISC-V with the tool chain you built for ESP. Once you have the binary, you can just copy it into sysroot and rebuild Linux with ‘make Linux’.

In general, Some applications might be available through buildroot, but I’m not sure about fio.

[paulmnt]

By the way, I just made some adjustment in esp\soft\ariane\linux\arch\riscv\configs\ariane_defconfig.
When I run make linux, it started asking me a lot of config questions. I think I may have broke something. Is there a way to recover that?

To recover the original config file, just go to the Linux folder in esp/soft/ariane/linux, then use

git checkout .

Don’t forget the period at the end.

If Linux was making interactive configuration questions, you probably changed something in the configuration manually that did not account for dependencies.
If you need to change Linux configuration, do it with menuconfig: first compile the default, then enter the linux-build folder and type

ARCH=riscv make menuconfig

[Kendidi]

Update: Config issue with make linux resolved. Thanks!

[Kendidi]

You need to get the source files and cross compile the application for RISC-V with the tool chain you built for ESP.
Once you have the binary, you can just copy it into sysroot and rebuild Linux with ‘make Linux’.

Thanks a lot @paulmnt !

[Kendidi]

1. When I have hardware design change made in top.bit (e.g. memory size changed), which ariane.dts file should be modified?
2. And what command should be run after ariane.dts has been modified so that the changes will be reflected in next FPGA run?

Thanks a lot in advance.

[Kendidi]

It appears socs//socgen/esp/riscv.dts is the file to edit nowadays.
And running make linux afterwards would do. Thanks.

[davide-giri]

Yes, that's the new location of riscv.dts. We restructured the repository ahead of release 2021.1.0, so the position of some files has changed.

[Kendidi]

I see. Thank you @davide-giri !!

[Kendidi]

I realize script, build_riscv_toolchain.sh by default only builds tools for riscv64. How can it install riscv32 instead, or as well? Thanks.

[paulmnt]

There is another script for the riscv32imc toolchain.

You need to run the script on a clean shell, where the RISCV variable is not defined, because the script is going to define it based on the target folder to install the toolchain (e.g. /opt/riscv32imc.

The 64 and 32 bits toolchains can coexist and ESP will use the appropriate compiler based on the target processor core.

Note that the 32-bit script is configured to compile the toolchain for the Ibex core. If you need your gcc to support different ISA extensions, you may need to edit some of the flags passed to configure.

[Kendidi]

I see. Thank you @paulmnt for the info.!!