Consider using Kubernetes API Aggregation

[mrueg]

What would you like to be added?

Use API Aggregation to allow Kubernetes to access Netbox data directly without replicating and storing the data in etcd.

An API Extension Service could register the CRDs using the APIService.apiregistration.k8s/v1 and serve as a translation layer between netbox client calls and the CR it represents.

See also:
https://kubernetes.io/blog/2024/11/21/dynamic-kubernetes-api-server-for-cozystack/

Why is this needed?

A Netbox instance can store a lot of data, and as far as I can see this operator replicates data from Netbox and stores it in ETCD via the Kubernetes API. This causes additional load on etcd and creates the risk of being out of sync in etcd or causing issues with the Cluster if a lot of Custom Resources get created or change at the same time.

[jstudler]

Hey @mrueg

Thanks for opening this issue. I can see that there might be use cases where the kube-api/etcd limits can affect scalability of NetBox Operator. It didn't come up as a requirement until now but I can certainly see it as a feature as long as it's not breaking for users that just want to leverage the default kube-api/CRD backend.

Note that NetBox Operator does not replicate all data that is stored in NetBox, it only replicates the relevant data it needs to e.g. do IPAM.

Another note on scalability: Since NetBox Operator uses NetBox as a source of truth it is possible to use NetBox Operator on more than one Kubernetes Cluster with the same NetBox backend. Each Cluster can have a different set of resources without the risk of e.g. IP overlap as NetBox still acts as IPAM source of truth. In case you consider this, be aware that there is currently a limitation with the distributed locking using leaslocker which only locks on cluster scope and doesn't support cross cluster locking of NetBox Resources for NetBox Operator.