In the previous two posts, we talked about the basics of Trusted Execution Environment (TEE), various terms that are used in this context and what they mean. Also, we talked more about the OP-TEE software offering, its various components and finally talked a little about GlobalPlatform and why it is relevant in the context of TEEs.
In case you missed these posts, you can find them here.
In this post, we will talk about setting up OP-TEE and running a working TEE on QEMU and Raspberry Pi 3 platforms. In both of these cases, the TEE will be talking to a rich world based on a linux kernel where the Buildroot build system is used to generate the entire platform image.
You can click on any of the below items to navigate this post.
OP-TEE on QEMU
According to the official documentation, QEMU is a generic and open-source machine emulator and virtualizer. QEMU is very popular with the embedded linux community as it allows generation and testing of linux images for various architectures without having a hardware platform ready or even if you have it, you don’t need to do the tedious steps necessary to generate the SD card compatible image with the right filesystem and so on. In fact, QEMU tends to be a de-facto supported platform for most releases of popular build systems like Yocto, Buildroot, etc.
Software Pre-requisites
Although any linux distribution can be used to do OP-TEE development, the maintainers of OP-TEE use Ubuntu for their development and release work. In that light, it is highly recommended to use Ubuntu as the host for your development.
Most of the pre-requisites for the OP-TEE builds can be obtained from the apt package manager. The package requirements specific to a particular Ubuntu release are documented on the official OP-TEE pre-requisites page.
Another key pre-requisite is the repo package. Repo is a tool built on top of git. Repo helps manage many git repositories, does the uploads to revision control systems, and automates parts of the development workflow. Please note that repo is not meant to replace git, only to make it easier to work with git.
To install repo, you can simply use apt as below.
$ sudo apt update
$ sudo apt install repoProcedure to build and run OP-TEE for QEMU
Once the software pre-requisites have been met, the following steps need to be performed to build and run an OP-TEE image for the QEMU platform. Do ensure that you have a fast internet connection!
Create a working directory
It is ideal to create a new empty directory and navigate into it so that all the work gets done within a known, desired location.
$ mkdir optee_qemu
$ cd optee_qemuRepo init
In this step, we point the repo tool to the URL where to find the manifest files that contains the relevant information about the packages needed, the version to be used and also the name of the manifest file to be used i.e. specify the platform to be used. In the OP-TEE software, the default platform is qemu (v7) – the manifest file is named default.xml.
Execute the below command.
$ repo init -u https://github.com/OP-TEE/manifest.git -m default.xml -b 3.19.0Repo sync
The repo sync step downloads the necessary git repos and checks out the versions as specified in the manifest file. Note the usage of the –no-clone-bundle option – this disables any attempt to use $URL/clone.bundle to bootstrap a new git repository from a resumeable bundle file on a content delivery network.
Execute the below command.
$ repo sync --no-clone-bundleGetting the toolchains
Once the sync is done, we will navigate into the build directory and get the toolchains that we will need to build the bootloaders, kernel, OP-TEE and finally the filesystem that we will be using.
$ cd build
$ make toolchainsMake and run the image
We are now ready to run possibly the longest running step of the entire process! In this single step, we will make and run the QEMU image. This may run for anywhere between few tens of minutes to a few hours depending on your internet connection and the host PC CPU specifications.
$ make runStart the QEMU Image
After the previous step is done successfully, the cursor is at a qemu prompt. Also, two new shell windows have opened – one each for the normal world (linux – rich world) and the secure world (OP-TEE – trusted world).
To start the qemu image, just type the character c or the word cont at the qemu prompt and observe the logs in the newly opened shell windows. Don’t forget to press the ENTER key!
(qemu) c
OR
(qemu) contTest OP-TEE using xtest
Once the normal and the secure worlds have booted up, log in at the buildroot login prompt using the user name as root. There is no password associated with this user in the image. Don’t forget to press the ENTER key!
Welcome to Buildroot, type root or test to login
buildroot login: rootYou can confirm that a TEE device was loaded correctly in the rich world by inspecting the boot log using dmesg. You can filter the output using the tee string using grep.
# dmesg | grep tee
optee: probing for conduit method
optee: revision 3.19 (afacf356)
...
...
optee: initialized driverIf you see output like the above, then we are ready to run the OP-TEE test suite bundled inside the application called xtest. Since the image is generated such that the tee-supplicant is already started, we don’t need to restart it. To know what the tee-supplicant is, you can check out this info from the last post.
Run the xtest from the command line. Don’t forget to press the ENTER key!
# xtestThe test suite will rattle down each and every test and subtest built into it. If everything goes well, you should see that all tests and subtests passed.
OP-TEE on Raspberry Pi 3
The Raspberry Pi or simply the RPi is one of the most popular single-board computers available today. They have revolutionised computing for the masses and made open-source embedded computing very accessible to a large section of the world owing to their affordability and the support ecosystem. Check out this video about the history of the RPi and this playlist for playing around with various peripherals on the RPi boards.
The good folks at Linaro working on the OP-TEE have an official support for the Raspberry Pi 3 SBC. Let us now take a look at how one can build an SD card image for the RPi 3 that allows us to execute the test suite as well as build trusted applications for the SBC.
Software Pre-requisites
Although any linux distribution can be used to do OP-TEE development, the maintainers of OP-TEE use Ubuntu for their development and release work. In that light, it is highly recommended to use Ubuntu as the host for your development.
Most of the pre-requisites for the OP-TEE builds can be obtained from the apt package manager. The package requirements specific to a particular Ubuntu release are documented on the official OP-TEE pre-requisites page.
Another key pre-requisite is the repo package. Repo is a tool built on top of git. Repo helps manage many git repositories, does the uploads to revision control systems, and automates parts of the development workflow. Please note that repo is not meant to replace git, only to make it easier to work with git.
To install repo, you can simply use apt as below.
$ sudo apt update
$ sudo apt install repoIn order to log into the RPi 3 after it has booted and to run the test suite, etc., we need to be able to connect to it using the serial console. You need a host software like minicom or picocom (your choice!) so that you can access the serial port using a USB to serial converter dongle. To install these tools, simply execute the below.
$ sudo apt update
$ sudo apt install minicom picocomThe default settings for the RPi 3 serial console is 115200-8-N-1.
Procedure to build and run OP-TEE for RPi 3
The procedure to build a working image for the RPi 3 platform is pretty similar to that for QEMU except for a few minor changes here and there. Do ensure that you have a fast internet connection!
Create a working directory
It is ideal to create a new empty directory and navigate into it so that all the work gets done within a known, desired location.
$ mkdir optee_rpi3
$ cd optee_rpi3Repo init
In this step, we point the repo tool to the URL where to find the manifest files that contains the relevant information about the packages needed, the version to be used and also the name of the manifest file to be used i.e. specify the platform to be used. In the OP-TEE software, the manifest file for the RPi 3 is named rpi3.xml.
Execute the below command.
$ repo init -u https://github.com/OP-TEE/manifest.git -m rpi3.xml -b 3.19.0Repo sync
The repo sync step downloads the necessary git repos and checks out the versions as specified in the manifest file. Note the usage of the –no-clone-bundle option – this disables any attempt to use $URL/clone.bundle to bootstrap a new git repository from a resumeable bundle file on a content delivery network.
Execute the below command.
$ repo sync --no-clone-bundleGetting the toolchains
Once the sync is done, we will navigate into the build directory and get the toolchains that we will need to build the bootloaders, kernel, OP-TEE and finally the filesystem that we will be using.
$ cd build
$ make toolchainsMake the image
We are now ready to run possibly the longest running step of the entire process! In this single step, we will prepare the image that we will use for the RPi 3. This may run for anywhere between few tens of minutes to a few hours depending on your internet connection and the host PC CPU specifications.
$ make -j `nproc`The nproc helps to most efficiently use your host CPU specifications by allocating the right number of threads for doing this build.
Prepare the SD card for RPi 3
In order to make the process of building the SD card image hassle-free as well as safe, the OP-TEE maintainers have decided to make the process manual instead of automated. In addition, they have provided clear instructions to do the SD card operations using the fdisk utility – which when followed exactly, create a usable SD card image for you.
To see these instructions, execute the below after the build is done successfully.
$ make img-helpYou should see the instructions as below.
$ fdisk /dev/sdx # where sdx is the name of your sd-card
> p # prints partition table
> d # repeat until all partitions are deleted
> n # create a new partition
> p # create primary
> 1 # make it the first partition
> <enter> # use the default sector
> +64M # create a boot partition with 64MB of space
> n # create rootfs partition
> p
> 2
> <enter>
> <enter> # fill the remaining disk, adjust size to fit your needs
> t # change partition type
> 1 # select first partition
> e # use type 'e' (FAT16)
> a # make partition bootable
> 1 # select first partition
> p # double check everything looks right
> w # write partition table to disk.
run the following as root
$ mkfs.vfat -F16 -n BOOT /dev/sdx1
$ mkdir -p /media/boot
$ mount /dev/sdx1 /media/boot
$ cd /media
$ gunzip -cd /location/of/optee_rpi3/build/../out-br/images/rootfs.cpio.gz | sudo cpio -idmv "boot/*"
$ umount boot
run the following as root
$ mkfs.ext4 -L rootfs /dev/sdx2
$ mkdir -p /media/rootfs
$ mount /dev/sdx2 /media/rootfs
$ cd rootfs
$ gunzip -cd /location/of/optee_rpi3/build/../out-br/images/rootfs.cpio.gz | sudo cpio -idmv
$ rm -rf /media/rootfs/boot/*
$ cd .. && umount rootfsJust follow the instructions diligently and you should have yourself an SD card – ready for use!
NOTE: Choose the correct /dev/sdx device! If you don’t do so, you risk wiping out your PC disk itself!
Boot up the RPi 3 and log into the serial console
Now, plug in the SD card into the RPi 3. Connect the GND-TXD-RXD pins of the RPi 3 to the GND-RXD-TXD pins of your USB to serial converter dongle. For your reference, below is the pinout of the RPi GPIO connector with the serial pins inside the green box.
Power on the RPi 3 and you should see the boot-up logs inside your favourite serial console. Like the QEMU case above, we log in as root as below. Don’t forget to press ENTER key!
Welcome to Buildroot, type root or test to login
buildroot login: rootYou can confirm that a TEE device was loaded correctly in the rich world by inspecting the boot log using dmesg. You can filter the output using the tee string using grep.
# dmesg | grep tee
optee: probing for conduit method
optee: revision 3.19 (afacf356)
optee: initialized driverTest OP-TEE using xtest
If you see output like the above, then we are ready to run the OP-TEE test suite bundled inside the application called xtest. Since the image is generated such that the tee-supplicant is already started, we don’t need to restart it. To know what the tee-supplicant is, you can check out this info from the last post.
Run the xtest from the command line. Don’t forget to press the ENTER key!
# xtestThe test suite will rattle down each and every test and subtest built into it. If everything goes well, you should see that all tests and subtests passed successfully.
Conclusion
In this post, we saw how to set up OP-TEE on two very popular platforms used for OP-TEE development – QEMU and Raspberry Pi 3. We did not only set it up but also executed the OP-TEE test suite bundled within the xtest application which is a part of the images we built.
In the next post, we will study the anatomy of a typical trusted application executing within OP-TEE and which talks to the rich world using the tee-supplicant which itself is executing within the rich world.
See you in the next post!