Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Unable to run the targetAllocator in namespace mode #3086

Closed
alita1991 opened this issue Jul 1, 2024 · 4 comments · Fixed by #3763 · May be fixed by #3573
Closed

Unable to run the targetAllocator in namespace mode #3086

alita1991 opened this issue Jul 1, 2024 · 4 comments · Fixed by #3763 · May be fixed by #3573
Assignees
Labels
area:rbac Issues relating to RBAC area:target-allocator Issues for target-allocator enhancement New feature or request good first issue Good for newcomers

Comments

@alita1991
Copy link

alita1991 commented Jul 1, 2024

Component(s)

target allocator

Describe the issue you're reporting

Hi,

I'm trying to run the targetAllocator in namespace mode, but I encountered the following errors:

{"level":"error","ts":"2024-07-01T09:59:11Z","logger":"setup.prometheus-cr-watcher","msg":"Failed to create namespace informer in promOperator CRD watcher","error":"missing list/watch permissions on the 'namespaces' resource: missing \"list\" permission on resource \"namespaces\" (group: \"\") for all namespaces: missing \"watch\" permission on resource \"namespaces\" (group: \"\") for all namespaces","stacktrace":"[github.com/open-telemetry/opentelemetry-operator/cmd/otel-allocator/watcher.NewPrometheusCRWatcher\n\t/home/runner/work/opentelemetry-operator/opentelemetry-operator/cmd/otel-allocator/watcher/promOperator.go:99\nmain.main\n\t/home/runner/work/opentelemetry-operator/opentelemetry-operator/cmd/otel-allocator/main.go:119\nruntime.main\n\t/opt/hostedtoolcache/go/1.22.4/x64/src/runtime/proc.go:271](http://github.com/open-telemetry/opentelemetry-operator/cmd/otel-allocator/watcher.NewPrometheusCRWatcher/n/t/home/runner/work/opentelemetry-operator/opentelemetry-operator/cmd/otel-allocator/watcher/promOperator.go:99/nmain.main/n/t/home/runner/work/opentelemetry-operator/opentelemetry-operator/cmd/otel-allocator/main.go:119/nruntime.main/n/t/opt/hostedtoolcache/go/1.22.4/x64/src/runtime/proc.go:271)"}
{"level":"error","ts":"2024-07-01T09:59:14Z","msg":"pkg/mod/[k8s.io/[email protected]/tools/cache/reflector.go:232](http://k8s.io/[email protected]/tools/cache/reflector.go:232): Failed to watch *v1.PodMonitor: failed to list *v1.PodMonitor: [podmonitors.monitoring.coreos.com](http://podmonitors.monitoring.coreos.com/) is forbidden: User \"system:serviceaccount:argocd-openshift:observability-cr-argocd-openshift-sa\" cannot list resource \"podmonitors\" in API group \"[monitoring.coreos.com](http://monitoring.coreos.com/)\" at the cluster scope","stacktrace":"[k8s.io/client-go/tools/cache.DefaultWatchErrorHandler\n\t/home/runner/go/pkg/mod/k8s.io/[email protected]/tools/cache/reflector.go:150\nk8s.io/client-go/tools/cache.(*Reflector).Run.func1\n\t/home/runner/go/pkg/mod/k8s.io/[email protected]/tools/cache/reflector.go:299\nk8s.io/apimachinery/pkg/util/wait.BackoffUntil.func1\n\t/home/runner/go/pkg/mod/k8s.io/[email protected]/pkg/util/wait/backoff.go:226\nk8s.io/apimachinery/pkg/util/wait.BackoffUntil\n\t/home/runner/go/pkg/mod/k8s.io/[email protected]/pkg/util/wait/backoff.go:227\nk8s.io/client-go/tools/cache.(*Reflector).Run\n\t/home/runner/go/pkg/mod/k8s.io/[email protected]/tools/cache/reflector.go:297\nk8s.io/client-go/tools/cache.(*controller).Run.(*Group).StartWithChannel.func2\n\t/home/runner/go/pkg/mod/k8s.io/[email protected]/pkg/util/wait/wait.go:55\nk8s.io/apimachinery/pkg/util/wait.(*Group).Start.func1\n\t/home/runner/go/pkg/mod/k8s.io/[email protected]/pkg/util/wait/wait.go:72](http://k8s.io/client-go/tools/cache.DefaultWatchErrorHandler/n/t/home/runner/go/pkg/mod/k8s.io/[email protected]/tools/cache/reflector.go:150/nk8s.io/client-go/tools/cache.(*Reflector).Run.func1/n/t/home/runner/go/pkg/mod/k8s.io/[email protected]/tools/cache/reflector.go:299/nk8s.io/apimachinery/pkg/util/wait.BackoffUntil.func1/n/t/home/runner/go/pkg/mod/k8s.io/[email protected]/pkg/util/wait/backoff.go:226/nk8s.io/apimachinery/pkg/util/wait.BackoffUntil/n/t/home/runner/go/pkg/mod/k8s.io/[email protected]/pkg/util/wait/backoff.go:227/nk8s.io/client-go/tools/cache.(*Reflector).Run/n/t/home/runner/go/pkg/mod/k8s.io/[email protected]/tools/cache/reflector.go:297/nk8s.io/client-go/tools/cache.(*controller).Run.(*Group).StartWithChannel.func2/n/t/home/runner/go/pkg/mod/k8s.io/[email protected]/pkg/util/wait/wait.go:55/nk8s.io/apimachinery/pkg/util/wait.(*Group).Start.func1/n/t/home/runner/go/pkg/mod/k8s.io/[email protected]/pkg/util/wait/wait.go:72)"}
{"level":"error","ts":"2024-07-01T12:10:48Z","msg":"pkg/mod/[k8s.io/[email protected]/tools/cache/reflector.go:232](http://k8s.io/[email protected]/tools/cache/reflector.go:232): Failed to watch *v1.ServiceMonitor: failed to list *v1.ServiceMonitor: [servicemonitors.monitoring.coreos.com](http://servicemonitors.monitoring.coreos.com/) is forbidden: User \"system:serviceaccount:argocd-openshift:observability-cr-argocd-openshift-sa\" cannot list resource \"servicemonitors\" in API group \"[monitoring.coreos.com](http://monitoring.coreos.com/)\" at the cluster scope","stacktrace":"[k8s.io/client-go/tools/cache.DefaultWatchErrorHandler\n\t/home/runner/go/pkg/mod/k8s.io/[email protected]/tools/cache/reflector.go:150\nk8s.io/client-go/tools/cache.(*Reflector).Run.func1\n\t/home/runner/go/pkg/mod/k8s.io/[email protected]/tools/cache/reflector.go:299\nk8s.io/apimachinery/pkg/util/wait.BackoffUntil.func1\n\t/home/runner/go/pkg/mod/k8s.io/[email protected]/pkg/util/wait/backoff.go:226\nk8s.io/apimachinery/pkg/util/wait.BackoffUntil\n\t/home/runner/go/pkg/mod/k8s.io/[email protected]/pkg/util/wait/backoff.go:227\nk8s.io/client-go/tools/cache.(*Reflector).Run\n\t/home/runner/go/pkg/mod/k8s.io/[email protected]/tools/cache/reflector.go:297\nk8s.io/client-go/tools/cache.(*controller).Run.(*Group).StartWithChannel.func2\n\t/home/runner/go/pkg/mod/k8s.io/[email protected]/pkg/util/wait/wait.go:55\nk8s.io/apimachinery/pkg/util/wait.(*Group).Start.func1\n\t/home/runner/go/pkg/mod/k8s.io/[email protected]/pkg/util/wait/wait.go:72](http://k8s.io/client-go/tools/cache.DefaultWatchErrorHandler/n/t/home/runner/go/pkg/mod/k8s.io/[email protected]/tools/cache/reflector.go:150/nk8s.io/client-go/tools/cache.(*Reflector).Run.func1/n/t/home/runner/go/pkg/mod/k8s.io/[email protected]/tools/cache/reflector.go:299/nk8s.io/apimachinery/pkg/util/wait.BackoffUntil.func1/n/t/home/runner/go/pkg/mod/k8s.io/[email protected]/pkg/util/wait/backoff.go:226/nk8s.io/apimachinery/pkg/util/wait.BackoffUntil/n/t/home/runner/go/pkg/mod/k8s.io/[email protected]/pkg/util/wait/backoff.go:227/nk8s.io/client-go/tools/cache.(*Reflector).Run/n/t/home/runner/go/pkg/mod/k8s.io/[email protected]/tools/cache/reflector.go:297/nk8s.io/client-go/tools/cache.(*controller).Run.(*Group).StartWithChannel.func2/n/t/home/runner/go/pkg/mod/k8s.io/[email protected]/pkg/util/wait/wait.go:55/nk8s.io/apimachinery/pkg/util/wait.(*Group).Start.func1/n/t/home/runner/go/pkg/mod/k8s.io/[email protected]/pkg/util/wait/wait.go:72)"}

Config

  targetAllocator:
    allocationStrategy: consistent-hashing
    enabled: true
    filterStrategy: relabel-config
    observability:
      metrics: {}
    podSecurityContext:
      fsGroup: 1000700000
      seccompProfile:
        type: RuntimeDefault
    prometheusCR:
      enabled: true
      podMonitorSelector: {}
      scrapeInterval: 30s
      serviceMonitorSelector: {}

The collector is configured to run with a serviceAccount bound to a Role, limiting access to a namespace only.

@jaronoff97
Copy link
Contributor

the target allocator already has a servicemonitor and podmonitor namespace selectro, we just need to expose it in the operator here.

@RichardChukwu
Copy link

Hi @jaronoff97 I'd love to know if this is still being worked on?

@jaronoff97
Copy link
Contributor

@RichardChukwu I don't believe so... cc @alita1991 are you still interested in working on this?

@dexter0195
Copy link
Contributor

I would love to contribute on this 😄 I submitted a PR here, let me know what do you think #3573

@swiatekm swiatekm assigned dexter0195 and unassigned alita1991 Jan 16, 2025
CharlieTLe added a commit to CharlieTLe/opentelemetry-operator that referenced this issue Mar 3, 2025
It's currently not possible to deploy the TA without cluster-wide
permisisons.

This change introduces a new env variable to the TA, WATCH_NAMESPACE,
which allows for specifying which namespaces to watch. This approach is
similar to how the opentelemetry-operator can be scoped to watch a
single namespace.

This does mean that cluster-wide resource like node metrics (cAdvisor)
are no longer accessible, but this is acceptable since we only want the
TA to know about targets that exist a specific namespaces.

Fixes: open-telemetry#3086

Signed-off-by: Charlie Le <[email protected]>
CharlieTLe added a commit to CharlieTLe/opentelemetry-operator that referenced this issue Mar 3, 2025
It's currently not possible to deploy the TA without cluster-wide
permisisons.

This change introduces a new env variable to the TA, WATCH_NAMESPACE,
which allows for specifying which namespaces to watch. This approach is
similar to how the opentelemetry-operator can be scoped to watch a
single namespace.

This does mean that cluster-wide resource like node metrics (cAdvisor)
are no longer accessible, but this is acceptable since we only want the
TA to know about targets that exist a specific namespaces.

Fixes: open-telemetry#3086

Signed-off-by: Charlie Le <[email protected]>
CharlieTLe added a commit to CharlieTLe/opentelemetry-operator that referenced this issue Mar 15, 2025
It's currently not possible to deploy the TA without cluster-wide
permisisons.

This change introduces a new env variable to the TA, WATCH_NAMESPACE,
which allows for specifying which namespaces to watch. This approach is
similar to how the opentelemetry-operator can be scoped to watch a
single namespace.

This does mean that cluster-wide resource like node metrics (cAdvisor)
are no longer accessible, but this is acceptable since we only want the
TA to know about targets that exist a specific namespaces.

Fixes: open-telemetry#3086

Signed-off-by: Charlie Le <[email protected]>
CharlieTLe added a commit to CharlieTLe/opentelemetry-operator that referenced this issue Mar 15, 2025
When the target allocator is configured to watch Prometheus custom
resources in the cluster to discover targets, it is currently hard-coded
to require a ClusterRole with a policy rule of listing namespaces. This
prevents usage in environments where configuring ClusterRoles is not
permitted i.e. in namespace-as-a-service setups where only Roles can be
created.

This change introduces two fields in the prometheusCR specification to
allow configuring the namespaces that can be interacted with by the
target allocator.

- allowNamespaces is a comma-separated list of namespaces for the target
  allocator to watch. If set to an empty string, it will list all
  list all namespaces in the cluster. This is mutually exclusive with
  denyNamespaces. Defaults to an empty string.
- denyNamespaces is a comma-separated list of namespaces for the target
  allocator to not watch. If set to an empty string, it will not exclude
  any namespaces. This is mutually exclusive with allowNamespaces.
  Defaults to an empty string.

Fixes: open-telemetry#3086

Signed-off-by: Charlie Le <[email protected]>
CharlieTLe added a commit to CharlieTLe/opentelemetry-operator that referenced this issue Mar 15, 2025
When the target allocator is configured to watch Prometheus custom
resources in the cluster to discover targets, it is currently hard-coded
to require a ClusterRole with a policy rule of listing namespaces. This
prevents usage in environments where configuring ClusterRoles is not
permitted i.e. in namespace-as-a-service setups where only Roles can be
created.

This change introduces two fields in the prometheusCR specification to
allow configuring the namespaces that can be interacted with by the
target allocator.

- allowNamespaces is a comma-separated list of namespaces for the target
  allocator to watch. If set to an empty string, it will list all
  list all namespaces in the cluster. This is mutually exclusive with
  denyNamespaces. Defaults to an empty string.
- denyNamespaces is a comma-separated list of namespaces for the target
  allocator to not watch. If set to an empty string, it will not exclude
  any namespaces. This is mutually exclusive with allowNamespaces.
  Defaults to an empty string.

Fixes: open-telemetry#3086

Signed-off-by: Charlie Le <[email protected]>
CharlieTLe added a commit to CharlieTLe/opentelemetry-operator that referenced this issue Mar 16, 2025
When the target allocator is configured to watch Prometheus custom
resources in the cluster to discover targets, it is currently hard-coded
to require a ClusterRole with a policy rule of listing namespaces. This
prevents usage in environments where configuring ClusterRoles is not
permitted i.e. in namespace-as-a-service setups where only Roles can be
created.

This change introduces two fields in the prometheusCR specification to
allow configuring the namespaces that can be interacted with by the
target allocator.

- allowNamespaces is a comma-separated list of namespaces for the target
  allocator to watch. If set to an empty string, it will list all
  list all namespaces in the cluster. This is mutually exclusive with
  denyNamespaces. Defaults to an empty string.
- denyNamespaces is a comma-separated list of namespaces for the target
  allocator to not watch. If set to an empty string, it will not exclude
  any namespaces. This is mutually exclusive with allowNamespaces.
  Defaults to an empty string.

A new end to end test was created,
e2e-targetallocator/targetallocator-namespace, to test setting
allowNamespaces and denyNamespaces.

The first part of the test sets up a a collector and TA in a namespace suffixed with
'-top-secret'. The TA in this namespace is configured with the
allowNamespaces to by this '-top-secret' namespace. Then a check is
performed to make sure that it can discover the service monitor that is
created by the otel-operator. This setup also doesn't make use of any
ClusterRole as it only creates a Role to perform its task of discovering
targets in its own namespace.

The second part of the test sets up another collector and TA are created
in a separate namespace without the '-top-secret' namespace with a
ClusterRole. The denyNamespaces is set to the namespace suffixed with
the '-top-secret' namespace, and later we check that we can't discover
the ServiceMonitor that is in the 'top-secret' namespace. We also check
that it can still discover the ServiceMonitor that is in its own
namespace.

Documentation was added to the otel-allocator README on how to set
allowNamespaces to configure the scope that the TA is allowed to
interact with.

Fixes: open-telemetry#3086

Signed-off-by: Charlie Le <[email protected]>
CharlieTLe added a commit to CharlieTLe/opentelemetry-operator that referenced this issue Mar 17, 2025
When the target allocator is configured to watch Prometheus custom
resources in the cluster to discover targets, it is currently hard-coded
to require a ClusterRole with a policy rule of listing namespaces. This
prevents usage in environments where configuring ClusterRoles is not
permitted i.e. in namespace-as-a-service setups where only Roles can be
created.

This change introduces two fields in the prometheusCR specification to
allow configuring the namespaces that can be interacted with by the
target allocator.

- allowNamespaces is a list of namespaces for the target allocator to
  watch. If set to an empty list, it will list all list all namespaces
  in the cluster. This is mutually exclusive with denyNamespaces.
  Defaults to an empty list.
- denyNamespaces is a list of namespaces for the target allocator to not
  watch. If set to an empty list, it will not exclude any namespaces.
  This is mutually exclusive with allowNamespaces.  Defaults to an empty
  list.

A new end to end test was created,
e2e-targetallocator/targetallocator-namespace, to test setting
allowNamespaces and denyNamespaces.

The first part of the test sets up a a collector and TA in a namespace suffixed with
'-top-secret'. The TA in this namespace is configured with the
allowNamespaces to by this '-top-secret' namespace. Then a check is
performed to make sure that it can discover the service monitor that is
created by the otel-operator. This setup also doesn't make use of any
ClusterRole as it only creates a Role to perform its task of discovering
targets in its own namespace.

The second part of the test sets up another collector and TA are created
in a separate namespace without the '-top-secret' namespace with a
ClusterRole. The denyNamespaces is set to the namespace suffixed with
the '-top-secret' namespace, and later we check that we can't discover
the ServiceMonitor that is in the 'top-secret' namespace. We also check
that it can still discover the ServiceMonitor that is in its own
namespace.

Documentation was added to the otel-allocator README on how to set
allowNamespaces to configure the scope that the TA is allowed to
interact with.

Fixes: open-telemetry#3086

Signed-off-by: Charlie Le <[email protected]>
CharlieTLe added a commit to CharlieTLe/opentelemetry-operator that referenced this issue Mar 18, 2025
When the target allocator is configured to watch Prometheus custom
resources in the cluster to discover targets, it is currently hard-coded
to require a ClusterRole with a policy rule of listing namespaces. This
prevents usage in environments where configuring ClusterRoles is not
permitted i.e. in namespace-as-a-service setups where only Roles can be
created.

This change introduces two fields in the prometheusCR specification to
allow configuring the namespaces that can be interacted with by the
target allocator.

- allowNamespaces is a list of namespaces for the target allocator to
  watch. If set to an empty list, it will list all list all namespaces
  in the cluster. This is mutually exclusive with denyNamespaces.
  Defaults to an empty list.
- denyNamespaces is a list of namespaces for the target allocator to not
  watch. If set to an empty list, it will not exclude any namespaces.
  This is mutually exclusive with allowNamespaces.  Defaults to an empty
  list.

A new end to end test was created,
e2e-targetallocator/targetallocator-namespace, to test setting
allowNamespaces and denyNamespaces.

The first part of the test sets up a a collector and TA in a namespace suffixed with
'-top-secret'. The TA in this namespace is configured with the
allowNamespaces to by this '-top-secret' namespace. Then a check is
performed to make sure that it can discover the service monitor that is
created by the otel-operator. This setup also doesn't make use of any
ClusterRole as it only creates a Role to perform its task of discovering
targets in its own namespace.

The second part of the test sets up another collector and TA are created
in a separate namespace without the '-top-secret' namespace with a
ClusterRole. The denyNamespaces is set to the namespace suffixed with
the '-top-secret' namespace, and later we check that we can't discover
the ServiceMonitor that is in the 'top-secret' namespace. We also check
that it can still discover the ServiceMonitor that is in its own
namespace.

Documentation was added to the otel-allocator README on how to set
allowNamespaces to configure the scope that the TA is allowed to
interact with.

Fixes: open-telemetry#3086

Signed-off-by: Charlie Le <[email protected]>
CharlieTLe added a commit to CharlieTLe/opentelemetry-operator that referenced this issue Mar 18, 2025
When the target allocator is configured to watch Prometheus custom
resources in the cluster to discover targets, it is currently hard-coded
to require a ClusterRole with a policy rule of listing namespaces. This
prevents usage in environments where configuring ClusterRoles is not
permitted i.e. in namespace-as-a-service setups where only Roles can be
created.

This change introduces two fields in the prometheusCR specification to
allow configuring the namespaces that can be interacted with by the
target allocator.

- allowNamespaces is a list of namespaces for the target allocator to
  watch. If set to an empty list, it will list all list all namespaces
  in the cluster. This is mutually exclusive with denyNamespaces.
  Defaults to an empty list.
- denyNamespaces is a list of namespaces for the target allocator to not
  watch. If set to an empty list, it will not exclude any namespaces.
  This is mutually exclusive with allowNamespaces.  Defaults to an empty
  list.

A new end to end test was created,
e2e-targetallocator/targetallocator-namespace, to test setting
allowNamespaces and denyNamespaces.

The first part of the test sets up a a collector and TA in a namespace suffixed with
'-top-secret'. The TA in this namespace is configured with the
allowNamespaces to by this '-top-secret' namespace. Then a check is
performed to make sure that it can discover the service monitor that is
created by the otel-operator. This setup also doesn't make use of any
ClusterRole as it only creates a Role to perform its task of discovering
targets in its own namespace.

The second part of the test sets up another collector and TA are created
in a separate namespace without the '-top-secret' namespace with a
ClusterRole. The denyNamespaces is set to the namespace suffixed with
the '-top-secret' namespace, and later we check that we can't discover
the ServiceMonitor that is in the 'top-secret' namespace. We also check
that it can still discover the ServiceMonitor that is in its own
namespace.

Documentation was added to the otel-allocator README on how to set
allowNamespaces to configure the scope that the TA is allowed to
interact with.

Fixes: open-telemetry#3086

Signed-off-by: Charlie Le <[email protected]>
CharlieTLe added a commit to CharlieTLe/opentelemetry-operator that referenced this issue Mar 18, 2025
When the target allocator is configured to watch Prometheus custom
resources in the cluster to discover targets, it is currently hard-coded
to require a ClusterRole with a policy rule of listing namespaces. This
prevents usage in environments where configuring ClusterRoles is not
permitted i.e. in namespace-as-a-service setups where only Roles can be
created.

This change introduces two fields in the prometheusCR specification to
allow configuring the namespaces that can be interacted with by the
target allocator.

- allowNamespaces is a list of namespaces for the target allocator to
  watch. If set to an empty list, it will list all list all namespaces
  in the cluster. This is mutually exclusive with denyNamespaces.
  Defaults to an empty list.
- denyNamespaces is a list of namespaces for the target allocator to not
  watch. If set to an empty list, it will not exclude any namespaces.
  This is mutually exclusive with allowNamespaces.  Defaults to an empty
  list.

A new end to end test was created,
e2e-targetallocator/targetallocator-namespace, to test setting
allowNamespaces and denyNamespaces.

The first part of the test sets up a a collector and TA in a namespace suffixed with
'-top-secret'. The TA in this namespace is configured with the
allowNamespaces to by this '-top-secret' namespace. Then a check is
performed to make sure that it can discover the service monitor that is
created by the otel-operator. This setup also doesn't make use of any
ClusterRole as it only creates a Role to perform its task of discovering
targets in its own namespace.

The second part of the test sets up another collector and TA are created
in a separate namespace without the '-top-secret' namespace with a
ClusterRole. The denyNamespaces is set to the namespace suffixed with
the '-top-secret' namespace, and later we check that we can't discover
the ServiceMonitor that is in the 'top-secret' namespace. We also check
that it can still discover the ServiceMonitor that is in its own
namespace.

Documentation was added to the otel-allocator README on how to set
allowNamespaces to configure the scope that the TA is allowed to
interact with.

Fixes: open-telemetry#3086

Signed-off-by: Charlie Le <[email protected]>
CharlieTLe added a commit to CharlieTLe/opentelemetry-operator that referenced this issue Mar 18, 2025
When the target allocator is configured to watch Prometheus custom
resources in the cluster to discover targets, it is currently hard-coded
to require a ClusterRole with a policy rule of listing namespaces. This
prevents usage in environments where configuring ClusterRoles is not
permitted i.e. in namespace-as-a-service setups where only Roles can be
created.

This change introduces two fields in the prometheusCR specification to
allow configuring the namespaces that can be interacted with by the
target allocator.

- allowNamespaces is a list of namespaces for the target allocator to
  watch. If set to an empty list, it will list all list all namespaces
  in the cluster. This is mutually exclusive with denyNamespaces.
  Defaults to an empty list.
- denyNamespaces is a list of namespaces for the target allocator to not
  watch. If set to an empty list, it will not exclude any namespaces.
  This is mutually exclusive with allowNamespaces.  Defaults to an empty
  list.

A new end to end test was created,
e2e-targetallocator/targetallocator-namespace, to test setting
allowNamespaces and denyNamespaces.

The first part of the test sets up a a collector and TA in a namespace suffixed with
'-top-secret'. The TA in this namespace is configured with the
allowNamespaces to by this '-top-secret' namespace. Then a check is
performed to make sure that it can discover the service monitor that is
created by the otel-operator. This setup also doesn't make use of any
ClusterRole as it only creates a Role to perform its task of discovering
targets in its own namespace.

The second part of the test sets up another collector and TA are created
in a separate namespace without the '-top-secret' namespace with a
ClusterRole. The denyNamespaces is set to the namespace suffixed with
the '-top-secret' namespace, and later we check that we can't discover
the ServiceMonitor that is in the 'top-secret' namespace. We also check
that it can still discover the ServiceMonitor that is in its own
namespace.

Documentation was added to the otel-allocator README on how to set
allowNamespaces to configure the scope that the TA is allowed to
interact with.

Fixes: open-telemetry#3086

Signed-off-by: Charlie Le <[email protected]>
CharlieTLe added a commit to CharlieTLe/opentelemetry-operator that referenced this issue Mar 18, 2025
When the target allocator is configured to watch Prometheus custom
resources in the cluster to discover targets, it is currently hard-coded
to require a ClusterRole with a policy rule of listing namespaces. This
prevents usage in environments where configuring ClusterRoles is not
permitted i.e. in namespace-as-a-service setups where only Roles can be
created.

This change introduces two fields in the prometheusCR specification to
allow configuring the namespaces that can be interacted with by the
target allocator.

- allowNamespaces is a list of namespaces for the target allocator to
  watch. If set to an empty list, it will list all namespaces in the
  cluster. This is mutually exclusive with denyNamespaces.  Defaults to
  an empty list.
- denyNamespaces is a list of namespaces for the target allocator to not
  watch. If set to an empty list, it will not exclude any namespaces.
  This is mutually exclusive with allowNamespaces.  Defaults to an empty
  list.

Validation is performed at the admission webhook and TA startup to
validate that both fields are not set at the same time.

A new end to end test was created,
e2e-targetallocator/targetallocator-namespace, to test setting
allowNamespaces and denyNamespaces.

The first part of the test sets up a a collector and TA in a namespace suffixed with
'-top-secret'. The TA in this namespace is configured with the
allowNamespaces to the '-top-secret' namespace. Then a check is
performed to make sure that it can discover the service monitor that is
created by the otel-operator. This setup also doesn't make use of any
ClusterRole as it only creates a Role to perform its task of discovering
targets in its own namespace.

The second part of the test sets up another collector and TA in a
separate namespace without the '-top-secret' namespace with a
ClusterRole. The denyNamespaces is set to the namespace suffixed with
the '-top-secret' namespace, and later we check that we can't discover
the ServiceMonitor that is in the 'top-secret' namespace. We also check
that it can still discover the ServiceMonitor that is in its own
namespace.

Documentation was added to the otel-allocator README on how to set
allowNamespaces to configure the scope that the TA is allowed to
interact with.

Fixes: open-telemetry#3086

Signed-off-by: Charlie Le <[email protected]>
CharlieTLe added a commit to CharlieTLe/opentelemetry-operator that referenced this issue Mar 18, 2025
When the target allocator is configured to watch Prometheus custom
resources in the cluster to discover targets, it is currently hard-coded
to require a ClusterRole with a policy rule of listing namespaces. This
prevents usage in environments where configuring ClusterRoles is not
permitted i.e. in namespace-as-a-service setups where only Roles can be
created.

This change introduces two fields in the prometheusCR specification to
allow configuring the namespaces that can be interacted with by the
target allocator.

- allowNamespaces is a list of namespaces for the target allocator to
  watch. If set to an empty list, it will list all namespaces in the
  cluster. This is mutually exclusive with denyNamespaces.  Defaults to
  an empty list.
- denyNamespaces is a list of namespaces for the target allocator to not
  watch. If set to an empty list, it will not exclude any namespaces.
  This is mutually exclusive with allowNamespaces.  Defaults to an empty
  list.

Validation is performed at the admission webhook and TA startup to
validate that both fields are not set at the same time.

A new end to end test was created,
e2e-targetallocator/targetallocator-namespace, to test setting
allowNamespaces and denyNamespaces.

The first part of the test sets up a a collector and TA in a namespace suffixed with
'-top-secret'. The TA in this namespace is configured with the
allowNamespaces to the '-top-secret' namespace. Then a check is
performed to make sure that it can discover the service monitor that is
created by the otel-operator. This setup also doesn't make use of any
ClusterRole as it only creates a Role to perform its task of discovering
targets in its own namespace.

The second part of the test sets up another collector and TA in a
separate namespace without the '-top-secret' namespace with a
ClusterRole. The denyNamespaces is set to the namespace suffixed with
the '-top-secret' namespace, and later we check that we can't discover
the ServiceMonitor that is in the 'top-secret' namespace. We also check
that it can still discover the ServiceMonitor that is in its own
namespace.

Documentation was added to the otel-allocator README on how to set
allowNamespaces to configure the scope that the TA is allowed to
interact with.

Fixes: open-telemetry#3086

Signed-off-by: Charlie Le <[email protected]>
swiatekm pushed a commit that referenced this issue Mar 18, 2025
When the target allocator is configured to watch Prometheus custom
resources in the cluster to discover targets, it is currently hard-coded
to require a ClusterRole with a policy rule of listing namespaces. This
prevents usage in environments where configuring ClusterRoles is not
permitted i.e. in namespace-as-a-service setups where only Roles can be
created.

This change introduces two fields in the prometheusCR specification to
allow configuring the namespaces that can be interacted with by the
target allocator.

- allowNamespaces is a list of namespaces for the target allocator to
  watch. If set to an empty list, it will list all namespaces in the
  cluster. This is mutually exclusive with denyNamespaces.  Defaults to
  an empty list.
- denyNamespaces is a list of namespaces for the target allocator to not
  watch. If set to an empty list, it will not exclude any namespaces.
  This is mutually exclusive with allowNamespaces.  Defaults to an empty
  list.

Validation is performed at the admission webhook and TA startup to
validate that both fields are not set at the same time.

A new end to end test was created,
e2e-targetallocator/targetallocator-namespace, to test setting
allowNamespaces and denyNamespaces.

The first part of the test sets up a a collector and TA in a namespace suffixed with
'-top-secret'. The TA in this namespace is configured with the
allowNamespaces to the '-top-secret' namespace. Then a check is
performed to make sure that it can discover the service monitor that is
created by the otel-operator. This setup also doesn't make use of any
ClusterRole as it only creates a Role to perform its task of discovering
targets in its own namespace.

The second part of the test sets up another collector and TA in a
separate namespace without the '-top-secret' namespace with a
ClusterRole. The denyNamespaces is set to the namespace suffixed with
the '-top-secret' namespace, and later we check that we can't discover
the ServiceMonitor that is in the 'top-secret' namespace. We also check
that it can still discover the ServiceMonitor that is in its own
namespace.

Documentation was added to the otel-allocator README on how to set
allowNamespaces to configure the scope that the TA is allowed to
interact with.

Fixes: #3086

Signed-off-by: Charlie Le <[email protected]>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
area:rbac Issues relating to RBAC area:target-allocator Issues for target-allocator enhancement New feature or request good first issue Good for newcomers
Projects
None yet
4 participants