super_build_and_cmake

Super build and CMake, what and why?

Introduction

First of all we are going to present the operations needed to transform the code into binaries. This will lead us to explain why we use tools like CMake, catkin (CMake module) and ament (CMake module) and what they do. We then present the challenges of building a code base (several package together). This will lead to the introduction of the super build.

Compilation chain for 1 package:

First of all let's talk about 1 package. and the list of operations to get the binaries out of the source code.

What do we need?

We need:

• the list of dependencies that will be use by this package (e.g. Eigen, boost, ...).
• the list of files currently present in the package that needs compiling.
• a way for others to use our code (install, export).
• the arguments to give to the compiler (gcc, clang, VisualStudio, ...)

And this in a cross-platform way if possible. The tool that helps us performing and ordering all this task is CMake.

The compilation steps:

With the following steps we build a package:

cd package_name # change directory to the package repository
mkdir build # create a temporary folder
cd build # change directory to the folder
# calling cmake. This steps create configuration files that will
# - defines the dependencies location and how to use it
# - look for all files to be compiled and setup the compiler for it to his job later on
cmake ..
cmake --build # calls the compiler and fill in the build folder with the newly created binaries
cmake --install . --prefix "/my/arbitary/path/" # install in the directory '/my/arbitary/path/'

A CMake module (catkin, ament, mpi_cmake_modules)

CMake has a cubbersome programming language and not super easy to apprehend. Hence people have been writing CMake code (macros and functions) to ease their usage of CMake. The robotics community (ROS) decided to create a CMake module to ease the creation of dependencies between packages. The two version of these module are called catkin (ROS1) and ament (ROS2).

note : the name catkin is also used for a super build, the difference will be explained later on. Idem for ament.

These tools make things simpler but then we require to heavily depend on them for our build system. In our lab we decided to create our own CMake module called mpi_cmake_modules. This package allows us to create a very thin layer on top of CMake (which becomes easier and easier every year), in the hope that one day we would only use native CMake macros to build our code base.

How to?

If you are wondering how to use CMake and the mpi_cmake_modules please look at this tutorial.

Compiling a lot of different packages.

We have seen earlier the requirement for compiling a package alone. Now the question is how do I do if I have interdependent packages to build?

What do we need?

We need:

• a system to know where are all the packages to be built,
• a system that will detect the build order depending on the dependency tree,
• a system that will be able to actually build all package (using CMake for example)
• a system that is user friendly enough to gain time,
• a system that will handle the temporary generated files (created by CMake and the compilers)
• cross platform is even better.

The tool that respect these constraint is called a Super Build. It is usually a collection of python scripts that perform the above list.

List of super-builders.

Here a quick list of super-build that we tend to use in the robotics community (due to ROS).

catkin_make

catkin_make is the super build developed by the ROS community in parallel of their CMake module catkin. The name is clear and unfortunate as it creates a bit of confusion on what is what? You can of course use catkin_make without using the CMake module caktin and back.

catkin-tools

catkin-tools is the next version of catkin_make this time it's not part of the catkin ROS repository. It is therefore independent of it. I simply does the same things as caktin_make in a more fancy and useful way. Right now the naming convention should already spin your head... This is normal, take the time to re-read this.

A tutorial on how to use it is here

ament-tools

This time ament-tools is part of the ament ROS eco system. It's purpose was to create a yet another version of catkin_make. The success is yet to be seen as another tool is now present: colcon

colcon

Colcon is the new catkin-tools, it interface with CMake, catkin_make, ament-tools, python-setuptools, and potentially more. This is the new super build design to be interfaced universally among all compiler and code structures. The community is going for this tool (at least ROS community does). This is the tool we use in our lab.

A tutorial on how to use it is here