feat(profilecli): add kube-proxy command for unified microservice access

[simonswine]

This adds a reverse proxy command that unifies all Pyroscope microservices in Kubernetes into a single endpoint.

The command:

• Auto-discovers services using label selectors
• Provides path-based routing to correct services
• Uses kubectl proxy on Unix socket for secure K8s API access
• Listens on configurable TCP address (default 127.0.0.1:4242)
• Filters to user-facing services only (distributor, query-frontend,
  tenant-settings, ad-hoc-profiles)

cc @jake-kramer we discussed the best way for debugging a problem, not too sure that is. But it works for me. In theory we could also inject a tenant id using a flag.

[jake-kramer]

Thanks for putting this together! To get it to work with drilldown locally, I needed the following changes:

diff --git a/cmd/profilecli/kube_proxy.go b/cmd/profilecli/kube_proxy.go
index e5696af8f..1037a8c80 100644
--- a/cmd/profilecli/kube_proxy.go
+++ b/cmd/profilecli/kube_proxy.go
@@ -381,13 +381,24 @@ func (m *reverseProxyManager) proxyToService(w http.ResponseWriter, r *http.Requ
 		return
 	}
 
-	// Copy headers
+	// Copy headers, but skip Authorization to avoid interfering with kubectl's auth
 	for key, values := range r.Header {
+		// Skip Authorization header - kubectl proxy uses kubeconfig auth, not incoming auth
+		if key == "Authorization" {
+			level.Debug(logger).Log("msg", "skipping Authorization header from incoming request")
+			continue
+		}
 		for _, value := range values {
 			proxyReq.Header.Add(key, value)
 		}
 	}
 
+	// Inject tenant ID if configured
+	if m.params.TenantID != "" {
+		proxyReq.Header.Set("X-Scope-OrgID", m.params.TenantID)
+		level.Debug(logger).Log("msg", "injected tenant ID", "tenant_id", m.params.TenantID)
+	}
+
 	// Execute request via Unix socket
 	client := &http.Client{
 		Transport: &http.Transport{

I was able to successfully run against a local drilldown with:

profilecli admin kube-proxy -v 
--url=<cluster url>
--tenant-id=<tenant id>
-n <namespace>
-c <cluster>
--listen-addr="127.0.0.1:4040"
-l "app.kubernetes.io/name=fire" # for dev

where drilldown Pyroscope datasource is set to :4040

[jake-kramer]

Is there any straightforward way of programmatically generating this list?