Vector Operator

Description

The operator deploys and configures a vector agent daemonset on every node to collect container and application logs from the node file system.

Connect us in Telegram - https://t.me/+Y0PzGa1d5DFiYThi

Features

• Building vector config from namespaced custom resources (kind: VectorPipeline)
• Configuration validation
• Full support of vector config options
• Namespace isolation
• Vector config optimization
• Vector aggregator support

RoadMap

Documentation

• Quick start doc
• Design doc
• Specification doc
• Secure credentials doc
• Collect logs from file doc
• Collect journald services logs doc

Configuration Examples

Configuration for CR Vector:

apiVersion: observability.kaasops.io/v1alpha1
kind: Vector
metadata:
  name: vector-sample
  namespace: vector
spec:
  agent:
    service: true
    image: "timberio/vector:0.24.0-distroless-libc"

Configuration for CR VectorPipeline:

apiVersion: observability.kaasops.io/v1alpha1
kind: VectorPipeline
metadata:
  name: vectorpipeline-sample
spec:
  sources:
    source1:
      type: "kubernetes_logs"
      extra_label_selector: "app!=testdeployment"
    source2:
      type: "kubernetes_logs"
      extra_label_selector: "app!=testdeployment1"
  transforms:
    remap:
      type: "remap"
      inputs:
        - source1
      source: |
        .@timestamp = del(.timestamp)

        .testField = "testValuevalue"
    filter:
      type: "filter"
      inputs:
        - source2
      condition:
        type: "vrl"
        source: ".status != 200"
  sinks:
    test222:
      type: "console"
      encoding:
        codec: "json"
      inputs:
        - filter
        - remap

Contributing

How it works

This project aims to follow the Kubernetes Operator pattern

It uses Controllers which provides a reconcile function responsible for synchronizing resources untile the desired state is reached on the cluster

Test It Out

1. Install the CRDs into the cluster:

make install

2. Run your controller (this will run in the foreground, so switch to a new terminal if you want to leave it running):

make run

NOTE: You can also run this in one step by running: make install run

Modifying the API definitions

If you are editing the API definitions, generate the manifests such as CRs or CRDs using:

make manifests

NOTE: Run make --help for more information on all potential make targets

More information can be found via the Kubebuilder Documentation