feat(examples): Add Caddy example

README.maintainers.md

@@ -0,0 +1,119 @@
+# Catalog for Maintainers
+
+These are common instructions for applications in the extended maintainers catalog.
+Each application gets their own directory, used for documenting, setting up, configuring, building, running, testing and debugging the application.
+Common information, configuration scripts are collected together in other directories such as `utils/`.
+
+## Instructions for Bincompat Apps
+
+The following are common information for examples of applications running in binary compatibility (i.e. **bincompat**) mode.
+Instructions assume you are running commands in the application example subdirectory.
+
+Applications running in binary compatibility mode use [`app-elfloader`](https://github.com/unikraft/app-elfloader) images to load and run native Linux ELFs (*Executable and Linking Format* files).
+The images are generated using `library/base/`.
+We call them `base` images or `base` kernels.
+
+The images are also stored in the Unikraft registry and can be pulled from there.
+
+### Directory Contents
+
+A typical directory for a bincompat app contains:
+
+* `Kraftfile`: build / run rules, including pulling the `base` image
+* `Dockerfile`: filesystem, including binary and libraries
+* `Makefile`: used to generate the root filesystem from the `Dockerfile` rules
+* `README.md`: specific application instructions, such as starting and testing an application
+* `config.yaml`: configuration file to generate script files to run the application
+* specific application files, such as configuration files and source code files
+
+### Common Setup
+
+The following are required before building and running a bincompat app.
+In case you have already done them for other bincompat runs, you can skip them:
+
+1. [Install Unikraft's companion command-line toolchain `kraft`](https://unikraft.org/docs/cli).
+
+1. Configure `kraft` to be able to pull contents from GitHub.
+   For this, first [generate a GitHub token](https://github.com/settings/tokens/new) that has all the `repo` list items checked.
+   Configure `kraft` to use the token:
+
+   ```console
+   kraft login -u <username> -t <token> github.com
+   ```
+
+   In the command above, replace `<username>` with your GitHub username and `<token>` with the generated token.
+
+1. Install Docker following the [official instructions](https://docs.docker.com/engine/install/).
+   This can be either [Docker Engine](https://docs.docker.com/engine/) or [Docker Desktop](https://docs.docker.com/desktop/).
+
+1. Start a [BuildKit](https://docs.docker.com/build/buildkit/) container to be used by KraftKit to build the filesystem from the Dockerfile.
+   Source the corresponding script from the `utils/` directory.
+   This can be one of the following commands, depending on the location of your application directory:
+
+   ```console
+   source ../../utils/start-buildkit.sh
+   source ../../../utils/start-buildkit.sh
+   ```
+
+1. Running the application requires kernel images (called `base` kernel images).
+   If not already generated, generate the kernels in the `kernels/` directory, while inside the `library/base/` directory:
+   This can be one of the following commands, depending on the location of your application directory:
+
+   ```console
+   ../../utils/bincompat/base-build-all.sh
+   ../../../utils/bincompat/base-build-all.sh
+   ```
+
+## Specific Setup
+
+`config.yaml` is used to generate the scripts and configuration files required to build and run the application.
+To generate these files, run one of the following commands, depending on the location of your application directory:
+
+```console
+../../utils/bincompat/generate.py
+```
+
+It generates the `run*` scripts and configuration files used to run the application in bincompat mode using different tools / VMMs.
+
+### Build, Run and Use
+
+You build and run the application using [KraftKit](https://github.com/unikraft/kraftkit).
+Follow the specific instructions in the `README.md` file in the application directory for specifics on building and running with KraftKit and on using the application.
+
+### Scripted Run
+
+To make it easier to run application in bincompat mode, use the generated scripts.
+Follow the steps:
+
+1. Build the application root filesystem as an initial ramdisk from the `Dockerfile` using:
+
+   ```console
+   make initrd
+   ```
+
+   This creates the `rootfs/` directory and the `rootfs.cpio` file with the root filesystem:
+
+   ```console
+   tree rootfs/
+   ```
+
+1. Run the scripts:
+
+   ```console
+   ./run-qemu-x86_64.sh
+   ./run-fc-x86_64.sh
+   ./kraft-run-qemu.sh
+   ```
+
+   All scripts start a local instance of the `base` image with the application running in bincompat mode.
+   The application filesystem (binary executable, libraries, configuration files) are packed inside the `rootfs.cpio` initrd file.
+   There is no Kraft script to run Firecracker, as Kraft doesn't currently support running images with networking support using Firecracker.
+
+1. You can run the `strace` or `debug` kernels with the scripts:
+
+   ```console
+   ./run-qemu-x86_64.sh ../../kernels/base_qemu-x86_64-strace
+   ./run-qemu-x86_64.sh ../../kernels/base_qemu-x86_64-debug
+   ```
+
+1. Build, run and use the application as detailed in the application `README.md`.

examples/caddy-2.7/.gitignore

@@ -0,0 +1,6 @@
+/rootfs/
+/rootfs.cpio
+/run-qemu*
+/run-fc*
+/kraft-run-*
+/fc*.json

examples/caddy-2.7/Caddyfile

@@ -0,0 +1,3 @@
+:2015
+
+respond "Hello, World!"

examples/caddy-2.7/Dockerfile

@@ -0,0 +1,34 @@
+FROM --platform=linux/x86_64 golang:1.21.4-bookworm AS build
+
+# Inspired from: https://github.com/caddyserver/caddy-docker/blob/master/2.7/builder/Dockerfile
+
+RUN set -xe; apt-get update ; \
+    apt-get install -y --no-install-recommends \
+    ca-certificates \
+    git \
+    libcap2-bin \
+    ;
+
+ENV XCADDY_VERSION v0.3.5
+# Configures xcaddy to build with this version of Caddy
+ENV CADDY_VERSION v2.7.5
+# Configures xcaddy to not clean up post-build (unnecessary in a container)
+ENV XCADDY_SKIP_CLEANUP 1
+# Sets capabilities for output caddy binary to be able to bind to privileged ports
+ENV XCADDY_SETCAP 1
+
+RUN set -xe; \
+    wget -O /tmp/xcaddy.tar.gz "https://github.com/caddyserver/xcaddy/releases/download/v0.3.5/xcaddy_0.3.5_linux_amd64.tar.gz"; \
+    tar x -z -f /tmp/xcaddy.tar.gz -C /usr/bin xcaddy; \
+    rm -f /tmp/xcaddy.tar.gz; \
+    chmod +x /usr/bin/xcaddy;
+
+ENV CGO_ENABLED=1
+ENV XCADDY_GO_BUILD_FLAGS "-buildmode=pie -ldflags '-linkmode external -extldflags -static-pie'"
+
+RUN /usr/bin/xcaddy build
+
+FROM scratch
+
+COPY --from=build /go/caddy /usr/local/bin/caddy
+COPY Caddyfile /etc/caddy/Caddyfile

examples/caddy-2.7/Kraftfile

@@ -0,0 +1,9 @@
+spec: v0.6
+
+name: caddy
+
+runtime: index.unikraft.io/unikraft.org/base:latest
+
+rootfs: ./rootfs
+
+cmd: ["/usr/local/bin/caddy", "run", "--config", "/etc/caddy/Caddyfile"]

examples/caddy-2.7/Makefile

@@ -0,0 +1,8 @@
+IMAGE_NAME = unikraft-caddy
+
+include ../../utils/bincompat/docker.Makefile
+
+export: clean create cleanfs
+	mkdir rootfs
+	docker export $(CONTAINER_NAME) | tar -x -C rootfs
+	cp hosts rootfs/etc/hosts

examples/caddy-2.7/README.md

@@ -0,0 +1,34 @@
+# Caddy 2.7
+
+This directory contains the definition to run [Caddy](https://github.com/caddyserver/caddy) on Unikraft in binary compatibility mode.
+
+Follow the instructions in the common `../README.md` file to set up and configure the application.
+
+## Quick Run
+
+Use `kraft` to run the image:
+
+```console
+kraft run -M 256M -p 2015:2015
+```
+
+Once executed, it will open port `2015` and wait for connections, and can be queried.
+
+Query the service using:
+
+```console
+curl localhost:2015
+```
+
+This doesn't work, however, probably due to some Caddy specifics.
+
+## Scripted Run
+
+Use the scripted runs, detailed in the common `../README.md`.
+Once you run the the scripts, query the service:
+
+```console
+curl 172.44.0.2:2015
+```
+
+This will show a `Hello, World!` message, provided by the Caddy, based on the configuration on `Caddyfile`.

examples/caddy-2.7/config.yaml

@@ -0,0 +1,3 @@
+networking: True
+accel: True
+memory: 512

examples/caddy-2.7/hosts

@@ -0,0 +1,2 @@
+127.0.0.1	localhost
+::1     ip6-localhost ip6-loopback

library/base/Kraftfile

@@ -2,26 +2,34 @@ spec: v0.6
 
 template:
   source: https://github.com/unikraft/app-elfloader.git
-  version: stable
+  version: staging
 
 unikraft:
   source: https://github.com/unikraft/unikraft.git
   version: staging
   kconfig:
+    # Configurations options for app-elfloader
+    # (they can't be part of the template atm)
     CONFIG_APPELFLOADER_ARCH_PRCTL: 'y'
     CONFIG_APPELFLOADER_BRK: 'y'
     CONFIG_APPELFLOADER_CUSTOMAPPNAME: 'y'
     CONFIG_APPELFLOADER_STACK_NBPAGES: 128
     CONFIG_APPELFLOADER_VFSEXEC_EXECBIT: 'n'
     CONFIG_APPELFLOADER_VFSEXEC: 'y'
+    CONFIG_APPELFLOADER_HFS: 'y'
+    CONFIG_APPELFLOADER_HFS_ETCRESOLVCONF: 'y'
+    CONFIG_APPELFLOADER_HFS_ETCHOSTS: 'y'
+    CONFIG_APPELFLOADER_HFS_ETCHOSTNAME: 'y'
+    CONFIG_APPELFLOADER_HFS_REPLACEEXIST: 'y'
+    # Unikraft options
     CONFIG_HAVE_PAGING_DIRECTMAP: 'y'
     CONFIG_HAVE_PAGING: 'y'
+    CONFIG_I8042: 'y'
     CONFIG_LIBDEVFS_AUTOMOUNT: 'y'
     CONFIG_LIBDEVFS_DEV_NULL: 'y'
     CONFIG_LIBDEVFS_DEV_STDOUT: 'y'
     CONFIG_LIBDEVFS_DEV_ZERO: 'y'
     CONFIG_LIBDEVFS: 'y'
-    CONFIG_LIBISRLIB: 'y'
     CONFIG_LIBPOSIX_ENVIRON_ENVP0: "PATH=/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin"
     CONFIG_LIBPOSIX_ENVIRON_ENVP1: "LD_LIBRARY_PATH=/usr/local/lib:/usr/lib:/lib"
     CONFIG_LIBPOSIX_ENVIRON_ENVP2: "HOME=/"
@@ -55,17 +63,29 @@ unikraft:
     CONFIG_LIBUKBLKDEV: 'y'
     CONFIG_LIBUKBOOT_BANNER_MINIMAL: 'y'
     CONFIG_LIBUKBOOT_HEAP_BASE: '0x400000000'
+    CONFIG_LIBUKBOOT_MAINTHREAD: 'y'
+    CONFIG_LIBUKBOOT_SHUTDOWNREQ_HANDLER: 'y'
     CONFIG_LIBUKCPIO: 'y'
+    CONFIG_LIBUKDEBUG_CRASH_SCREEN: 'y'
+    CONFIG_LIBUKDEBUG_ENABLE_ASSERT: 'y'
+    CONFIG_LIBUKDEBUG_PRINT_SRCNAME: 'n'
+    CONFIG_LIBUKDEBUG_PRINT_TIME: 'y'
+    CONFIG_LIBUKDEBUG_PRINTK_ERR: 'y'
+    CONFIG_LIBUKDEBUG_PRINTK: 'y'
+    CONFIG_LIBUKDEBUG: 'y'
     CONFIG_LIBUKFALLOC: 'y'
     CONFIG_LIBUKMPI: 'n'
     CONFIG_LIBUKSIGNAL: 'y'
+    CONFIG_LIBUKSWRAND_DEVFS: 'y'
+    CONFIG_LIBUKSWRAND: 'y'
     CONFIG_LIBUKVMEM_DEFAULT_BASE: '0x0000001000000000'
     CONFIG_LIBUKVMEM_DEMAND_PAGE_IN_SIZE: 12
     CONFIG_LIBUKVMEM_PAGEFAULT_HANDLER_PRIO: 4
     CONFIG_LIBUKVMEM: 'y'
     CONFIG_LIBVFSCORE_AUTOMOUNT_ROOTFS: 'n'
     CONFIG_LIBVFSCORE_FSTAB: 'y'
     CONFIG_LIBVFSCORE_NONLARGEFILE: 'y'
+    CONFIG_LIBVFSCORE: 'y'
     CONFIG_OPTIMIZE_DEADELIM: 'y'
     CONFIG_OPTIMIZE_LTO: 'y'
     CONFIG_PAGING: 'y'
@@ -74,19 +94,37 @@ unikraft:
     CONFIG_UKPLAT_MEMREGION_MAX_COUNT: 64
 
     # Debug options
-    CONFIG_KVM_DEBUG_VGA_CONSOLE: 'n'
-    CONFIG_KVM_KERNEL_VGA_CONSOLE: 'n'
-    CONFIG_LIBUKDEBUG_CRASH_SCREEN: 'n'
-    CONFIG_LIBUKDEBUG_ENABLE_ASSERT: 'n'
-    CONFIG_LIBUKDEBUG_PRINT_TIME: 'n'
-
-    # CONFIG_LIBSYSCALL_SHIM_STRACE: 'y'
     # CONFIG_LIBUKDEBUG_PRINTD: 'y'
     # CONFIG_LIBUKDEBUG_PRINTK_INFO: 'y'
+    # CONFIG_LIBSYSCALL_SHIM_STRACE: 'y'
+    # CONFIG_LIBSYSCALL_SHIM_DEBUG: 'y'
 
 libraries:
-  lwip: staging
-  libelf: stable
+  lwip:
+    source: https://github.com/unikraft/lib-lwip.git
+    version: staging
+    kconfig:
+      CONFIG_LWIP_LOOPIF: 'y'
+      CONFIG_LWIP_UKNETDEV: 'y'
+      CONFIG_LWIP_LOOPBACK: 'y'
+      CONFIG_LWIP_TCP: 'y'
+      CONFIG_LWIP_UDP: 'y'
+      CONFIG_LWIP_RAW: 'y'
+      CONFIG_LWIP_WND_SCALE: 'y'
+      CONFIG_LWIP_TCP_KEEPALIVE: 'y'
+      CONFIG_LWIP_THREADS: 'y'
+      CONFIG_LWIP_HEAP: 'y'
+      CONFIG_LWIP_SOCKET: 'y'
+      CONFIG_LWIP_AUTOIFACE: 'y'
+      CONFIG_LWIP_IPV4: 'y'
+      CONFIG_LWIP_DHCP: 'n'
+      CONFIG_LWIP_DNS: 'n'
+      CONFIG_LWIP_NUM_TCPCON: 64
+      CONFIG_LWIP_NUM_TCPLISTENERS: 64
+      CONFIG_LWIP_ICMP: 'y'
+  libelf:
+    source: https://github.com/unikraft/lib-libelf.git
+    version: staging
 
 targets:
 - fc/x86_64

utils/bincompat/base-build-all.sh

@@ -0,0 +1,47 @@
+#!/bin/sh
+
+# SPDX-License-Identifier: BSD-3-Clause
+# Copyright (c) 2023, Unikraft GmbH and The Unikraft Authors.
+#
+# Generate strace and debug Kraftfiles and build all kernel images
+# for app compat:
+#   - qemu-x86_64
+#   - fc-x86_64
+#   - qemu-x86_64-strace
+#   - fc-x86_64-strace
+#   - qemu-x86_64-debug
+#   - fc-x86_64-debug
+#
+# This script is to be run from the `library/base/` directory.
+# Kernel image files are generated in the `$KERNELDIR` directory.
+
+# Generate Kraftfile for strace-enabled build.
+tac Kraftfile | sed '/^\([ \t]\+\)\(CONFIG_.*\)$/ {s/^\([ \t]\+\)\(.*\)$/\1CONFIG_LIBSYSCALL_SHIM_STRACE: '\'y\''\n\1\2/; :loop; n; b loop}' | tac > Kraftfile.strace
+
+# Generate Kraftfile for all debug-enabled build.
+tac Kraftfile | sed '/^\([ \t]\+\)\(CONFIG_.*\)$/ {s/^\([ \t]\+\)\(.*\)$/\1CONFIG_LIBSYSCALL_SHIM_STRACE: '\'y\''\n\1CONFIG_LIBSYSCALL_SHIM_DEBUG: '\'y\''\n\1CONFIG_LIBUKDEBUG_PRINTK_INFO: '\'y\''\n\1CONFIG_LIBUKDEBUG_PRINTD: '\'y\''\n\1\2/; :loop; n; b loop}' | tac > Kraftfile.debug
+
+KERNELDIR="../../kernels"
+
+test -d "$KERNELDIR" || mkdir "$KERNELDIR"
+
+rm -f .config*
+rm -fr .unikraft
+
+kraft build --no-cache --plat qemu --arch x86_64
+cp .unikraft/build/base_qemu-x86_64 "$KERNELDIR"/base_qemu-x86_64
+
+kraft build --no-cache --plat fc --arch x86_64
+cp .unikraft/build/base_fc-x86_64 "$KERNELDIR"/base_fc-x86_64
+
+kraft build -K Kraftfile.strace --no-cache --plat qemu --arch x86_64
+cp .unikraft/build/base_qemu-x86_64 "$KERNELDIR"/base_qemu-x86_64-strace
+
+kraft build -K Kraftfile.strace --no-cache --plat fc --arch x86_64
+cp .unikraft/build/base_fc-x86_64 "$KERNELDIR"/base_fc-x86_64-strace
+
+kraft build -K Kraftfile.debug --no-cache --plat qemu --arch x86_64
+cp .unikraft/build/base_qemu-x86_64 "$KERNELDIR"/base_qemu-x86_64-debug
+
+kraft build -K Kraftfile.debug --no-cache --plat fc --arch x86_64
+cp .unikraft/build/base_fc-x86_64 "$KERNELDIR"/base_fc-x86_64-debug