Skip to content

feat: add alertmanager-0.28.1 #1617

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Open
wants to merge 2 commits into
base: stackhpc/2024.1
Choose a base branch
from
Open
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
15 changes: 13 additions & 2 deletions doc/source/configuration/monitoring.rst
Original file line number Diff line number Diff line change
Expand Up @@ -81,8 +81,8 @@ on the overcloud hosts:
SMART reporting should now be enabled along with a Prometheus alert for
unhealthy disks and a Grafana dashboard called ``Hardware Overview``.

Alertmanager and Slack
======================
Alertmanager, Slack and Microsoft Teams
=======================================

StackHPC Kayobe configuration comes bundled with an array of alerts but does not
enable any receivers for notifications by default. Various receivers can be
Expand Down Expand Up @@ -119,6 +119,17 @@ available `here <https://awesome-prometheus-alerts.grep.to/>`__. They simply
need to be added to one of the ``*.rules`` files in the prometheus configuration
directory.

If however you are using Microsoft Teams instead of Slack, you can use Prometheus
Alertmanager's built-in support for the new message format based on Power Automate flows.
You will need an incoming webhook URL for your Teams channel.
This can be done by following `these instructions <https://support.microsoft.com/en-gb/office/create-incoming-webhooks-with-workflows-for-microsoft-teams-8ae491c7-0394-4861-ba59-055e33f75498>`__.
To set up a receiver, create a ``prometheus-alertmanager.yml`` file under
``etc/kayobe/kolla/config/prometheus/``.
An example config is stored in this directory known as ``prometheus-alertmanager.msteamvs2.yml.example``.
The example configuration uses two Slack channels.
One channel receives all alerts while the other only receives alerts tagged as critical.
Feel free to modify the example configuration to suit your needs.

Ceph Monitoring
===============

Expand Down
3 changes: 3 additions & 0 deletions etc/kayobe/kolla-image-tags.yml
Original file line number Diff line number Diff line change
Expand Up @@ -49,6 +49,9 @@ kolla_image_tags:
ovn:
rocky-9: 2024.1-rocky-9-20250219T113722
ubuntu-jammy: 2024.1-ubuntu-jammy-20250219T113722
prometheus_alertmanager:
rocky-9: 2024.1-rocky-9-20250422T103147
ubuntu-jammy: 2024.1-ubuntu-jammy-20250422T103147
skyline_apiserver:
rocky-9: 2024.1-rocky-9-20250408T133253
ubuntu-jammy: 2024.1-ubuntu-jammy-20250408T133253
Expand Down
Original file line number Diff line number Diff line change
@@ -0,0 +1,29 @@
---
global:
resolve_timeout: 5m
smtp_require_tls: true

route:
receiver: 'msteamsv2-notifications'
group_by: [alertname]
group_wait: 30s
group_interval: 5m
repeat_interval: 4h

routes:
- matchers:
- severity=~"critical|alert"
receiver: 'msteamvs2-critical-notifications'

receivers:
- name: 'msteamsv2-notifications'
msteamsv2_configs:
- webhook_url: '{{ secrets_msteams_notification_channel_url }}'
send_resolved: true
- name: 'msteamsv2-critical-notifications'
msteamsv2_configs:
- webhook_url: '{{ secrets_msteams_notification_critical_channel_url }}'
send_resolved: true

templates:
- '/etc/prometheus/*.tmpl'
5 changes: 5 additions & 0 deletions releasenotes/notes/bump-alertmanager-958f90fa2bc9b562.yaml
Original file line number Diff line number Diff line change
@@ -0,0 +1,5 @@
---
features:
- |
Prometheus Alertmanager has been updated to ``0.28.1``. This release
includes support for Microsoft Teams notifications.
Loading