Xilinx Ultrascale Vitis on Rocky 9

Xilinx Ultrascale Vitis on Rocky 9

I’ve been using Xilinx chip and toolchains since the late 1980s, they have moved on a lot but still are tied together with tcl underneath and push the boundaries of what your PC can handle… the 2023.01 default “Vitis” download for Linux is nearly 100GB plus other needed pieces.

They’ve moved on from Java to Electron, both make sense at the time for cross- platform solutions that feel native, and added this “Vitis” app on top in an attempt to bring the whole flow management GUI thing to cross development as well.

They officially support specific distros, RHEL type and Ubuntu, but I was always able to coax it to run OK on Fedora, and now my machines have switched to Rocky 9, I tried installing on that.

Vitis vs Rhel 9

Difficulties with Vitis cross-distro packaging

Although the download recommended earlier RHEL, it mostly worked out of the box, the first sign of madness was Vitis complaining that it couldn’t start “gitlens”. Git was installed fine, but poking around with strace, Vitis has taken the approach that everything it runs should inherit a special LD_LIBRARY_PATH environment that allows it to override some key libs, namely OpenSSL 1.0.2k. This EOL’d in 2019 and has no security updates since then, it’s not a great way around the multi-distro packaging problem in 2023.

Putting the security failure on one side, the bigger problem is that the install chose to use the override path as <install base dir>/Vitis/2023.2/tps/lnx64/cmake-3.24.2/libs/Default which does not exist, so anything wanting openssl was confused to see the current OpenSSL 3 pieces from Centos instead of the trick ones the Vitis pieces were linked against, and fails.

The solution for these problems was to create a symlink of the Default name it already chose, to the Rhel/9 one that contains the insecure tls libs it was built to use.

$ cd ~/Xilinx/Vitis/2023.2/tps/lnx64/cmake-3.24.2/libs
$ ln -sf Rhel/9 Default

Restarting vitis that leaves two much smaller problems

• if you open a shell console in Vitis, anything in your .bashrc that wants to bind to openssl will inherit the Vitis LD_LIBRARY_PATH and see the 1.0.2k era libraries and die after being unable to bind its imports that exist on the distro openssl versio (like flatpak on my box) but it’s basically cosmetic

flatpak: symbol lookup error: /lib64/libldap.so.2: undefined symbol: EVP_md2, version OPENSSL_3.0.0

• the links in the Vitis “Welcome” page to the internet don’t do anything probably again because of it spawning something that was actually built for OpenSSL 3 getting surprised it’s being handed 1.0.2k api. They’re just a convenience and not neccessary.

The business end of Vivado (which is still its own standalone thing although Vitis is intimately aware of its output products) and Vitis seem to work fine so far.

Serial terminal arrangements on ZCU-104

ZCU-104 has a micro-B USB connector which comes out into four ttyUSB. These unfortunately move around between suspends, but there’s a workaround, which is to go by /dev/serial/by-id/usb-Xilinx_JTAG+3serial_81472-if01-port0.

The Vitis docs claim you have to use root in linux to access the serial ports, but typically you’ll only need to one time add your user to dialout group with

$ sudo usermod -a -G dialout myuser

… and then reboot.

The documentation shows an integrated serial terminal in Vitis, but I couldn’t find it in 2023.2 release, I build gtkterm from source since it’s not in Rocky.

When you get to download things over JTAG, the synchronization of connectivity between the host and the dev board is a bit shaky, as described by this complaint four years ago

https://support.xilinx.com/s/question/0D52E00006hprI5SAI/error-while-launching-program-invalid-target?language=en_US

For me initially it was really shaky, because powertop was enabling suspending my hosts’s USB controllers causing them to reset randomly, I disabled the powertop service and it just became what is apparently “normally shaky”.

Running Vitis etc over a network

I recently upgraded my network here to 2.5Gbps, although the license is node- locked (Mac address locked), it’s perfectly feasible to run Vitis and Vivado over ssh -Y at 4K, at least locally. So you can use a noisy desktop with lots of RAM + cores to run it from somewhere else away from you, and control it perfectly well from a laptop.

There are other problems specific to Rocky 9’s Gnome 40, like there is no UI for creating a desktop session headless, in Gnome 44 it’s supposedly possible to use RDP instead of VNC. This would be desirable because the state of what you’re working on with the remote box will survive suspending either side then, with ssh -Y the app instances will die and need saving / restoring.

Vivado pieces

I followed the flow here

https://xilinx.github.io/Embedded-Design-Tutorials/docs/2023.1/build/html/docs/Introduction/ZynqMPSoC-EDT/3-system-configuration.html

to do the Vivado work to create a default .xsa hardware description (you’ll need it later) and bitstream. This was relatively straightforward and no quirks for Rocky.

Petalinux

More downloads are needed to work with Linux, Xilinx provide their own Yocto distro needed in two parts, a commandline “installer” and a “BSP” for your specific dev board.

The download page says:

Supported OS:

    Completely removed RHEL and CENTOS to align with upstream Yocto. 

https://docs.xilinx.com/r/en-US/ug1144-petalinux-tools-reference-guide/Installation-Requirements

But I have used Yocto on Rocky for years, whatever it’s trying to say there I doubt it can’t work. After “install” as we’ll see there is RHEL9 support in there just not linked up.

Petalinux / Yocto is also a discontiguity for cross-platform Vitis… that supports Windows via electron, but Yocto doesn’t, so they just announce that you “need a Linux box to use Petalinux”.

The “.bsp” file for your platform is actually just a .tar.gz without the file suffix, but don’t unpack it - the petalinux tools consume these “.bsp” files as compressed files with the suffix .bsp directly.

1) First create a directory for petalinux to live in at ~/Xilinx/petalinux.

2) Move the .bsp file(s) you downloaded into there.

3) Run the “petalinux installer” packed shell script you downloaded so it installs into ~/Xilinx/petalinux, and since it also does not really understand Rocky is Rhel you must also create the neccessary link similar to Vitis:

$ cd ~/Xilinx/petalinux/tools/xsct/lib/lnx64.o
$ mv Default old-Default
$ ln -sf Rhel/9 Default

4) It also needs xterm installed on the host to proceed.

5) After that you simply source a toplevel script to customize your shell, Yocto style:

$ cd ~/Xilinx/petalinux
$ . settings.sh
PetaLinux environment set to '/home/agreen/Xilinx/petalinux'
WARNING: This is not a supported OS
INFO: Checking free disk space
INFO: Checking installed tools
INFO: Checking installed development libraries
INFO: Checking network and other services
WARNING: No tftp server found - please refer to "UG1144 2023.2 PetaLinux Tools Documentation Reference Guide" for its impact and solution
$ 

I looked at UG1144 but didn’t see anything useful for this, I guess it’s only needed for network boot, which isn’t today’s problem.

Practical meaning of integration for Vitis

The scope of the tools needed for development on ultrascale is pretty scary, not only Linux, yocto packages, a-t-f, userland libs, but hardware definition, toolchains, debugger, JTAG and the whole HDL -> bitstream flow in Vivado. Knowing and having experience in each of these separately (in particular having the VHDL + Vivado flow knowledge) initially doesn’t help much dealing with how Vitis takes the rest away from you and provides access to them under its own rules.

Vitis “integration” doesn’t always mean that it brings them inside Vitis UI or idioms in the same way the source debugger is subsumed into Vitis, Vivado remains a separate app with its own UI and “integrated” mainly in the sense a “hardware description” .xsa file produced from Vivado-land is the source of truth about the platform in the rest of Vitis.

The jury’s out whether this is overall better than just providing idiomatic customization for each tool - eg, I’m very familiar with Yocto, the old way was to just provide the necessary Yocto config for the dev boards and let me get on with it.

Petalinux is again its own wrapped, unintegrated external stack and again requires its own glue flow outside vitis.

Creating a linux “bsp” project and adapting it to the hardware .xsa

First source the yocto/petalinux environment in a terminal

$ cd ~/Xilinx/petalinux
$ . settings.sh
...
$

Then create a “petalinux project” from the BSP

$ cd ~/xilinx-workspace
$ petalinux-create -t project -n plx-zcu104-1 -s ~/Xilinx/petalinux/xilinx-zcu104-v2023.2-10140544.bsp
INFO: create project plx-zcu104-1
INFO: new project successfully created in /home/agreen/
$

For the next step to customize the “petalinux project”, we need to “be inside the petalinux project” directory, so enter the project we just created as the cwd and then “configure” the petalinux project we created from the bsp reference just now

$ cd ~/xilinx-workspace/plx-zcu104-1
$ petalinux-config --get-hw-description ~/xilinx-tutorials/test1/design_2_wrapper.xsa

This now does a kernel menuconfig / buildroot style menu so we can see what’s going to be included in the build

… on exiting the menu, having made changes or not, it will spend a few seconds modifying the petalinux project to follow the .xsa we just gave it, and any config overrides from the menuconfig.

Finally you can start the yocto build

$ petalinux-build