Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Update prometheus-istio-integration.md #102

Open
wants to merge 1 commit into
base: main
Choose a base branch
from

Conversation

pratiksharma-dev
Copy link

Istio addon has a custom namespace for Istio, so this should be aks-istio-system instead of istio-system

Istio addon has a custom namespace for Istio, so this should be aks-istio-system instead of istio-system
Copy link
Contributor

@pratiksharma-dev : Thanks for your contribution! The author(s) have been notified to review your proposed change.

Copy link
Contributor

Learn Build status updates of commit 9f9e8fb:

✅ Validation status: passed

File Status Preview URL Details
articles/azure-monitor/containers/prometheus-istio-integration.md ✅Succeeded

For more details, please refer to the build report.

For any questions, please:

@pratiksharma-dev
Copy link
Author

pratiksharma-dev commented Mar 2, 2025

This configuration is not working, not able to get istio metrics by enabling annotation based scraping for aks-istio-system and default namespace:
image
Able to make it work by configuring a scraping job:

scrape_configs:
  - job_name: 'kubernetespods-scrape'

    kubernetes_sd_configs:
    - role: pod

    relabel_configs:
    # Scrape only pods with the annotation: prometheus.io/scrape = true
    - source_labels: [__meta_kubernetes_pod_annotation_prometheus_io_scrape]
      action: keep
      regex: true

    # If prometheus.io/path is specified, scrape this path instead of /metrics
    - source_labels: [__meta_kubernetes_pod_annotation_prometheus_io_path]
      action: replace
      target_label: __metrics_path__
      regex: (.+)

    # If prometheus.io/port is specified, scrape this port instead of the default
    - source_labels: [__address__, __meta_kubernetes_pod_annotation_prometheus_io_port]
      action: replace
      regex: ([^:]+)(?::\d+)?;(\d+)
      replacement: $1:$2
      target_label: __address__

    # If prometheus.io/scheme is specified, scrape with this scheme instead of http
    - source_labels: [__meta_kubernetes_pod_annotation_prometheus_io_scheme]
      action: replace
      regex: (http|https)
      target_label: __scheme__

    # Include the pod namespace as a label for each metric
    - source_labels: [__meta_kubernetes_namespace]
      action: replace
      target_label: kubernetes_namespace

    # Include the pod name as a label for each metric
    - source_labels: [__meta_kubernetes_pod_name]
      action: replace
      target_label: kubernetes_pod_name

    # [Optional] Include all pod labels as labels for each metric
    - action: labelmap
      regex: __meta_kubernetes_pod_label_(.+)

This configmap is applied to ama-metrics-prometheus-config as it has definition for scrape jobs

kubectl create configmap ama-metrics-prometheus-config --from-file=prometheus-config -n kube-system

The benefit here is that scraping will work even when mTLS is enforced by istio

image

I can help with editing of this article, feel free to reach out to me.

@Court72
Copy link
Contributor

Court72 commented Mar 3, 2025

@bwren

Can you review the proposed changes?

Important: When the changes are ready for publication, adding a #sign-off comment is the best way to signal that the PR is ready for the review team to merge.

#label:"aq-pr-triaged"
@MicrosoftDocs/public-repo-pr-review-team

@prmerger-automator prmerger-automator bot added the aq-pr-triaged C+L Pull Request Review Team label label Mar 3, 2025
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants