Forward metrics to another GCP Project #2309
Description
Describe the bug
Similarly to #885 we want to forward to a specific GCP project. In my case I'm hosting the GKE cluster in project A and using a pod service account with workload identity towards a GCP service account in Project B.
It seems like it's not respecting the GOOGLE_CLOUD_PROJECT for the java apps.
Logs
prePopulateVerifiedDescriptors | Failed to pre populate verified descriptors for PROJECT-A
com.google.api.gax.rpc.PermissionDeniedException: io.grpc.StatusRuntimeException: PERMISSION_DENIED: Permission monitoring.metricDescriptors.create denied (or the resource may not exist).
The environment variable GOOGLE_CLOUD_PROJECT is set to Project B
The SA in project B have the necessary IAM permission