Skip to content

Commit

Permalink
Delete libraries and user-side programs (#172)
Browse files Browse the repository at this point in the history
  • Loading branch information
vlvkobal authored Jul 27, 2020
1 parent 6a598c6 commit bfd03ff
Show file tree
Hide file tree
Showing 165 changed files with 78 additions and 55,887 deletions.
1 change: 0 additions & 1 deletion .dockerfiles/build.sh
Original file line number Diff line number Diff line change
Expand Up @@ -5,7 +5,6 @@ set -e
build() {
echo "[XXX]: Building against Kernel ${KERNEL_VERSION} for libc ${_LIBC} ..."
(
cd user || exit 1
if [ "${DEBUG:-0}" -eq 1 ]; then
echo "[XXX]: Building with debug symbols ..."
make CFLAGS='-fno-stack-protector -I /usr/src/linux/usr/include' EXTRA_CFLAGS='-g -fno-stack-protector'
Expand Down
9 changes: 0 additions & 9 deletions .gitignore
Original file line number Diff line number Diff line change
Expand Up @@ -5,14 +5,5 @@
*.tar.*
*.tar

*.so*
*.ll
*.o*

user/process_monitor
user/process_latency
user/process_test

libbpf/src/build/
libbpf/src/root/
library/libbpf.a
38 changes: 38 additions & 0 deletions Makefile
Original file line number Diff line number Diff line change
@@ -0,0 +1,38 @@
CC=gcc

KERNEL_DIR = kernel/
KERNEL_PROGRAM = $(KERNEL_DIR)process_kern.o

KERNEL_VERSION="$(shell if [ -f /usr/src/linux/include/config/kernel.release ]; then cat /usr/src/linux/include/config/kernel.release; else cat /proc/sys/kernel/osrelease; fi)"
FIRST_KERNEL_VERSION=$(shell sh tools/complement.sh "$(KERNEL_VERSION)")

NETDATA_KERNEL_VERSION=$(shell echo $(KERNEL_VERSION) | tr -s "." "_")

VER_MAJOR=$(shell echo $(KERNEL_VERSION) | cut -d. -f1)
VER_MINOR=$(shell echo $(KERNEL_VERSION) | cut -d. -f2)
VER_PATCH=$(shell echo $(KERNEL_VERSION) | cut -d. -f3)

_LIBC ?= glibc

EXTRA_CFLAGS += -fno-stack-protector

all: $(KERNEL_PROGRAM)
cp $(KERNEL_DIR)rprocess_kern.o rnetdata_ebpf_process.$(VER_MAJOR).$(VER_MINOR).$(VER_PATCH).o
cp $(KERNEL_DIR)pprocess_kern.o pnetdata_ebpf_process.$(VER_MAJOR).$(VER_MINOR).$(VER_PATCH).o
cp $(KERNEL_DIR)rnetwork_viewer_kern.o rnetdata_ebpf_socket.$(VER_MAJOR).$(VER_MINOR).$(VER_PATCH).o
cp $(KERNEL_DIR)pnetwork_viewer_kern.o pnetdata_ebpf_socket.$(VER_MAJOR).$(VER_MINOR).$(VER_PATCH).o
if [ -f pnetdata_ebpf_process.$(VER_MAJOR).$(VER_MINOR).$(VER_PATCH).o ]; then tar -cf artifacts/netdata_ebpf-$(FIRST_KERNEL_VERSION)_$(NETDATA_KERNEL_VERSION)-$(_LIBC).tar [pr]netdata_ebpf_*.$(VER_MAJOR).$(VER_MINOR).$(VER_PATCH).o; else echo "ERROR: Cannot find BPF programs"; exit 1; fi
if [ "$${DEBUG:-0}" -eq 1 ]; then tar -uvf artifacts/netdata_ebpf-$(FIRST_KERNEL_VERSION)_$(NETDATA_KERNEL_VERSION)-$(_LIBC).tar tools/check-kernel-config.sh; fi
xz artifacts/netdata_ebpf-$(FIRST_KERNEL_VERSION)_$(NETDATA_KERNEL_VERSION)-$(_LIBC).tar
( cd artifacts; sha256sum netdata_ebpf-$(FIRST_KERNEL_VERSION)_$(NETDATA_KERNEL_VERSION)-$(_LIBC).tar.xz > netdata_ebpf-$(FIRST_KERNEL_VERSION)_$(NETDATA_KERNEL_VERSION)-$(_LIBC).tar.xz.sha256sum )

$(KERNEL_PROGRAM):
cd $(KERNEL_DIR) && $(MAKE) all;

clean:
if [ -f pnetdata_ebpf_process.$(VER_MAJOR).$(VER_MINOR).$(VER_PATCH).o ] ; then rm *.o; fi
cd $(KERNEL_DIR) && $(MAKE) clean;
rm artifacts/*

install:
cp *netdata_ebpf_process.$(VER_MAJOR).$(VER_MINOR).$(VER_PATCH).o /usr/libexec/netdata/plugins.d/
59 changes: 18 additions & 41 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -9,22 +9,17 @@ Linux Kernel eBPF Collectors

The respository has the following directory structure:

- `artifacts`: Directory that will have the eBPF programs and shared libraries when the compilation process ends.
- `artifacts`: Directory that will have the eBPF programs when the compilation process ends.
- `includes`: Common headers
- `kernel`: The eBPF programs source code
- `lib`: The libelf static used to create the shared library.
- `library`: Codes from Linux kernel-source changed to create the shared library.
- `libbpf_0_0_1`: An inexistent version of the libbpf library, we changed it to support old Linux kernels.
- `libbpf`: The latest libbpf version from kernel.
- `tools`: scripts used to verify system status before to install or test any eBPF code.
- `user`: Software to tests the eBPF program
- `tools`: scripts used to verify system status before installing eBPF code.

## Necessary packages

To compile the shared libraries and the eBPF programs, it will be necessary to have the following packages:
To compile the eBPF programs, it will be necessary to have the following packages:

- Libelf headers
- llvm/clang , because GCC cannot compile eBPF codes.
- llvm/clang, because GCC prior to 10.0 cannot compile eBPF code.
- Kernel headers

The last group of files can be extracted direct from kernel source doing the following steps:
Expand All @@ -37,7 +32,7 @@ make prepare
make headers_install
```

Case you are using the kernel `5.4` or newer, it is necessary to comment the following line inside the file
In case you are using the kernel `5.4` or newer, it is necessary to comment the following line inside the file
`generated/autoconf.h`:

```
Expand All @@ -46,13 +41,13 @@ Case you are using the kernel `5.4` or newer, it is necessary to comment the fol

## Necessary changes

Before to compile this repository, it is necessary to change the Makefiles according your environment. The original
Before compilation of this repository, it is necessary to change the Makefiles according your environment. The original
files were adjusted to compile on Slackware Linux Current.


### `kernel/Makefile`

Inside this file probably it will be neecssary to change the following variable:
Inside this file probably it will be necessary to change the following variable:

- `KERNELSOURCE`: Where is your kernel-source? This variable was set initially to work on Slackware, Fedora and Ubuntu
- `KERNELBUILD`: Directory where the headers are expected to be stored.
Expand Down Expand Up @@ -112,46 +107,28 @@ This sets `EXTRA_CFLAGS=-g` up before building.

## Compilation (manually)

After to do the necessaries changes inside the file `kernel/Makefille`, to compile the libraries
and the eBPF programs, you only need to do the following steps:
After the necessary changes have been done inside the `kernel/Makefille` file, you need to run the following
command to compile the eBPF programs:

```bash
# cd user
# make
```

When the compilation finishes, you will have inside `artificats` directory a file with the following
When the compilation finishes, you will have a file inside `artifacts` directory with the following
content:

```
usr/lib64/libbpf_kernel.so
libnetdata_ebpf.so
dnetdata_ebpf_process.o
pnetdata_ebpf_process.o
rnetdata_ebpf_process.o
pnetdata_ebpf_process.<kernel version>.o
pnetdata_ebpf_socket.<kernel version>.o
rnetdata_ebpf_process.<kernel version>.o
rnetdata_ebpf_socket.<kernel version>.o
```

We can group these files as:
`p*.o`: eBPF programs used with entry mode, this is the default mode.
`r*.o`: eBPF programs used with return mode.

- `libbpf_kernel.so`: This is the libbpf shared library that must be moved to a directory listed inside
`/etc/ld.so.conf`, when you move it for one of the directory, it will be necessary to create a symbolic link,
for example, let us assume that the distribution uses `/usr/lib64/` to store shared libraries, the following
commands are necessaries:

```bash
cp usr/lib64/libbpf_kernel.so /usr/lib64/
ln -s /usr/lib64/libbpf_kernel.so /usr/lib64/libbpf_kernel.so.0
```

- `Collector files`: the collector works with all files created during the compilation, but the next 4 files
need to be copied to `/usr/libexec/netdata/plugins.d` for the collector to have condition to access them:
- `libnetdata_ebpf.so`: Shared library used to load the eBPF programs.
- `dnetdata_ebpf_process.o`: eBPF program used with developer mode.
- `pnetdata_ebpf_process.o`: eBPF program used with entry mode, this is the default mode.
- `rnetdata_ebpf_process.o`: eBPF program used with return mode.


After this you can start the new collector `ebpf_program.plugin`.
These files have to be copied to your plugins directory, which is usually at `/usr/libexec/netdata/plugins.d/`,
for the collector to be able to access them. After this you can start the new collector `ebpf_program.plugin`.

## Releasing

Expand Down
17 changes: 8 additions & 9 deletions docs/Compilation-as-normal-user.md
Original file line number Diff line number Diff line change
@@ -1,25 +1,24 @@
# Compilation as normal user

The `kernel-collector` repository can be compiled as normal user, but before to do this it is necessary
some steps as `root`.
The `kernel-collector` repository can be compiled as normal user, but before doing this it is necessary
to do some steps as `root`.

## Steps as root

Before to compile this repository, it is necessary to install the nessary packages available on your
Linux distribution. The complete list of packages can be found inside the Docker files on this repository.
Before compiling this repository, you have to install necessary packages available on your
Linux distribution. The complete list of packages can be found inside the Docker files in this repository.

Case you are trying to compile on a kernel newer than `5.0`, it will be necessary to disable the
`assembly inline`, this can be done editing the file `/usr/src/linux/include/generated/autoconf.h`
In case you are trying to compile on a kernel newer than `5.0`, it will be necessary to disable the
`assembly inline`, this can be done by editing the file `/usr/src/linux/include/generated/autoconf.h`
and commenting the line `//#define CONFIG_CC_HAS_ASM_INLINE 1`.

Finally case your distribution does not create the symbolic link `/usr/src/linux` to your kernel source,
Finally, if your distribution does not create the symbolic link `/usr/src/linux` to your kernel source,
you will need to do it manually.

## Step as normal user

After to do the necessary installation, you only need to run the next commands to compile the repository:
After the necessary installation has been prepared, you need to run the next command to compile the repository:

```bash
$ cd user/
$ make
```
2 changes: 1 addition & 1 deletion docs/README.md
Original file line number Diff line number Diff line change
@@ -1,3 +1,3 @@
# Docs

The following directory contains general documentation related to this repository.
This directory contains general documentation related to this repository.
67 changes: 0 additions & 67 deletions docs/Update-libbpf.md

This file was deleted.

3 changes: 1 addition & 2 deletions includes/README.md
Original file line number Diff line number Diff line change
@@ -1,4 +1,3 @@
# Include files

This directory has only one file that Netdata developed, the file `api.h`, all theother files were
copied from kernel source repository.
Copied from the kernel source repository.
22 changes: 0 additions & 22 deletions includes/api.h

This file was deleted.

58 changes: 0 additions & 58 deletions includes/bpf_endian.h

This file was deleted.

Loading

0 comments on commit bfd03ff

Please sign in to comment.