Skip to content

Commit 5269652

Browse files
mstyushinmstyushin
committed
Merge branch 'dev'
2 parents d29da5d + 1319701 commit 5269652

File tree

12 files changed

+123
-37
lines changed

12 files changed

+123
-37
lines changed

Diff for: ‎.github/workflows/mamonsu-tests-dev.yml

+1-1
Original file line numberDiff line numberDiff line change
@@ -86,7 +86,7 @@ jobs:
8686
echo "zabbix_address=$(hostname -I | awk '{print $1}')" >> $GITHUB_OUTPUT
8787
id: zabbix_address
8888
- name: Edit Zabbix address in agent.conf
89-
run: sed -i "s/\(address *= *\).*/\1 ${{ steps.zabbix_address.outputs.zabbix_address }}/" ${{ env.MAMONSU_PATH }}/github-actions-tests/sources/agent_3.5.10.conf
89+
run: sed -i "s/\(address *= *\).*/\1 ${{ steps.zabbix_address.outputs.zabbix_address }}/" ${{ env.MAMONSU_PATH }}/github-actions-tests/sources/agent_3.5.11.conf
9090

9191
- name: Copy test scripts to container
9292
run: docker exec $( echo "${{ matrix.docker_os }}" | sed 's/://' | sed 's/\.//' ) mkdir -p -m 755 /mamonsu/

Diff for: ‎.github/workflows/mamonsu-tests-master.yml

+1-1
Original file line numberDiff line numberDiff line change
@@ -91,7 +91,7 @@ jobs:
9191
echo "zabbix_address=$(hostname -I | awk '{print $1}')" >> $GITHUB_OUTPUT
9292
id: zabbix_address
9393
- name: Edit Zabbix address in agent.conf
94-
run: sed -i "s/\(address *= *\).*/\1 ${{ steps.zabbix_address.outputs.zabbix_address }}/" ${{ env.MAMONSU_PATH }}/github-actions-tests/sources/agent_3.5.10.conf
94+
run: sed -i "s/\(address *= *\).*/\1 ${{ steps.zabbix_address.outputs.zabbix_address }}/" ${{ env.MAMONSU_PATH }}/github-actions-tests/sources/agent_3.5.11.conf
9595

9696
- name: Copy test scripts to container
9797
run: docker exec $( echo "${{ matrix.docker_os }}" | sed 's/://' | sed 's/\.//' ) mkdir -p -m 755 /mamonsu/

Diff for: ‎github-actions-tests/mamonsu_build.sh

+2-2
Original file line numberDiff line numberDiff line change
@@ -41,7 +41,7 @@ if [ "${OS%:*}" = "centos" ]; then
4141
python3 setup.py build && python3 setup.py install
4242
make rpm
4343
sudo rpm -i ./mamonsu*.rpm
44-
cat /mamonsu/github-actions-tests/sources/agent_3.5.10.conf > /etc/mamonsu/agent.conf
44+
cat /mamonsu/github-actions-tests/sources/agent_3.5.11.conf > /etc/mamonsu/agent.conf
4545
# ensuring mamonsu can actually start
4646
sudo su -s /bin/bash -c "mamonsu bootstrap -x --user postgres -d mamonsu_test_db" mamonsu
4747
/etc/init.d/mamonsu restart
@@ -65,7 +65,7 @@ elif [ "${OS%:*}" = "ubuntu" ]; then
6565
python3 setup.py build && python3 setup.py install
6666
make deb
6767
sudo dpkg -i ./mamonsu*.deb
68-
cat /mamonsu/github-actions-tests/sources/agent_3.5.10.conf > /etc/mamonsu/agent.conf
68+
cat /mamonsu/github-actions-tests/sources/agent_3.5.11.conf > /etc/mamonsu/agent.conf
6969
# ensuring mamonsu can actually start
7070
sudo su -s /bin/bash -c "mamonsu bootstrap -x --user postgres -d mamonsu_test_db" mamonsu
7171
service mamonsu restart

Diff for: ‎github-actions-tests/sources/agent_3.5.10.conf renamed to ‎github-actions-tests/sources/agent_3.5.11.conf

File renamed without changes.

Diff for: ‎mamonsu/__init__.py

+1-1
Original file line numberDiff line numberDiff line change
@@ -1,7 +1,7 @@
11
__author__ = 'Dmitry Vasilyev'
22
__author_email__ = '[email protected]'
33
__description__ = 'Monitoring agent for PostgreSQL'
4-
__version__ = '3.5.10'
4+
__version__ = '3.5.11'
55
__licence__ = 'BSD'
66

77
__url__ = 'https://github.com/postgrespro/mamonsu'

Diff for: ‎mamonsu/plugins/pgsql/driver/pool.py

+22
Original file line numberDiff line numberDiff line change
@@ -110,6 +110,7 @@ def __init__(self, params=None):
110110
"bootstrap": {"storage": {}, "counter": 0, "cache": 10, "version": False},
111111
"recovery": {"storage": {}, "counter": 0, "cache": 10},
112112
"extension_schema": {"pg_buffercache": {}, "pg_stat_statements": {}, "pg_wait_sampling": {}, "pgpro_stats": {}},
113+
"extension_versions" : {},
113114
"pgpro": {"storage": {}},
114115
"pgproee": {"storage": {}}
115116
}
@@ -135,6 +136,19 @@ def server_version(self, db=None):
135136
result.decode("ascii"))
136137
return self._cache["server_version"]["storage"][db]
137138

139+
def extension_version(self, extension, db=None):
140+
db = self._normalize_db(db)
141+
if extension in self._cache["extension_versions"] and db in self._cache["extension_versions"][extension][db]:
142+
return self._cache["extension_versions"][extension][db]
143+
144+
version_string = self.query("select extversion from pg_catalog.pg_extension where lower(extname) = lower('{0}');".format(extension), db)[0][0]
145+
result = bytes(
146+
version_string.split(" ")[0], "utf-8")
147+
self._cache["extension_versions"][extension] = {}
148+
self._cache["extension_versions"][extension][db] = "{0}".format(
149+
result.decode("ascii"))
150+
return self._cache["extension_versions"][extension][db]
151+
138152
def server_version_greater(self, version, db=None):
139153
db = self._normalize_db(db)
140154
return packaging.version.parse(self.server_version(db)) >= packaging.version.parse(version)
@@ -229,6 +243,14 @@ def is_pgpro_ee(self, db=None):
229243
self._cache["pgproee"][db] = False
230244
return self._cache["pgproee"][db]
231245

246+
def extension_version_greater(self, extension, version, db=None):
247+
db = self._normalize_db(db)
248+
return packaging.version.parse(self.extension_version(extension, db)) >= packaging.version.parse(version)
249+
250+
def extension_version_less(self, extension, version, db=None):
251+
db = self._normalize_db(db)
252+
return packaging.version.parse(self.extension_version(extension, db)) <= packaging.version.parse(version)
253+
232254
def extension_installed(self, ext, db=None):
233255
db = self._normalize_db(db)
234256
result = self.query("""

Diff for: ‎mamonsu/plugins/pgsql/statements.py

+60-10
Original file line numberDiff line numberDiff line change
@@ -27,7 +27,7 @@ class Statements(Plugin):
2727

2828
query_info = """
2929
SELECT {metrics}
30-
FROM {extension_schema}.pg_stat_statements_info;
30+
FROM {extension_schema}.{info_view_name};
3131
"""
3232
key = "pgsql."
3333
# zbx_key, sql, desc, unit, delta, (Graph, color, side)
@@ -88,6 +88,32 @@ class Statements(Plugin):
8888
("PostgreSQL Statements Info: Last Statistics Reset Time", "9C8A4E", 0))
8989
]
9090

91+
Items_pgpro_stats_1_8 = [
92+
("stat[read_bytes]",
93+
"(sum(shared_blks_read+local_blks_read+temp_blks_read)*8*1024)::bigint",
94+
"Read bytes/s", Plugin.UNITS.bytes_per_second, Plugin.DELTA.speed_per_second,
95+
("PostgreSQL Statements: Bytes", "87C2B9", 0)),
96+
("stat[write_bytes]",
97+
"(sum(shared_blks_written+local_blks_written+temp_blks_written)*8*1024)::bigint",
98+
"Write bytes/s", Plugin.UNITS.bytes_per_second, Plugin.DELTA.speed_per_second,
99+
("PostgreSQL Statements: Bytes", "793F5D", 0)),
100+
("stat[dirty_bytes]",
101+
"(sum(shared_blks_dirtied+local_blks_dirtied)*8*1024)::bigint",
102+
"Dirty bytes/s", Plugin.UNITS.bytes_per_second, Plugin.DELTA.speed_per_second,
103+
("PostgreSQL Statements: Bytes", "9C8A4E", 0)),
104+
("stat[read_time]",
105+
"(sum(shared_blk_read_time+local_blk_read_time+temp_blk_read_time)/float4(100))::bigint",
106+
"Read IO Time", Plugin.UNITS.s, Plugin.DELTA.speed_per_second,
107+
("PostgreSQL Statements: Spent Time", "87C2B9", 0)),
108+
("stat[write_time]",
109+
"(sum(shared_blk_write_time+local_blk_write_time+temp_blk_write_time)/float4(100))::bigint",
110+
"Write IO Time", Plugin.UNITS.s, Plugin.DELTA.speed_per_second,
111+
("PostgreSQL Statements: Spent Time", "793F5D", 0)),
112+
["stat[other_time]",
113+
"(sum(total_exec_time+total_plan_time-shared_blk_read_time-local_blk_read_time-temp_blk_read_time-shared_blk_write_time-local_blk_write_time-temp_blk_write_time)/float4(100))::bigint",
114+
"Other (mostly CPU) Time", Plugin.UNITS.s, Plugin.DELTA.speed_per_second,
115+
("PostgreSQL Statements: Spent Time", "9C8A4E", 0)]]
116+
91117
all_graphs = [
92118
("PostgreSQL Statements: Bytes", None),
93119
("PostgreSQL Statements: Spent Time", 1),
@@ -115,21 +141,45 @@ def run(self, zbx):
115141

116142
# TODO: add 13 and 14 items when pgpro_stats added new WAL metrics
117143
all_items = self.Items.copy()
118-
if Pooler.server_version_greater("14"):
144+
145+
if Pooler.extension_installed("pgpro_stats") and Pooler.extension_version_greater("pgpro_stats", "1.8"):
146+
info_view = 'pg_stat_statements_info'
147+
if self.extension == "pgpro_stats":
148+
info_view = 'pgpro_stats_info'
149+
150+
info_items = self.Items_pg_14
151+
info_params = [x[1] for x in info_items]
152+
info_result = Pooler.query(
153+
self.query_info.format(metrics=(", ".join(info_params)), extension_schema=extension_schema, info_view_name=info_view))
154+
for key, value in enumerate(info_result[0]):
155+
zbx_key, value = "pgsql.{0}".format(
156+
info_items[key][0]), int(value)
157+
zbx.send(zbx_key, value, info_items[key][4])
158+
159+
all_items = self.Items_pgpro_stats_1_8.copy()
160+
all_items += self.Items_pg_13
161+
162+
elif Pooler.server_version_greater("14"):
119163
self.Items[5][1] = self.Items[5][1].format("total_exec_time+total_plan_time")
120164
all_items += self.Items_pg_13
165+
info_view = 'pgpro_stats_info'
121166
if self.extension == "pg_stat_statements":
122-
info_items = self.Items_pg_14
123-
info_params = [x[1] for x in info_items]
124-
info_result = Pooler.query(
125-
self.query_info.format(metrics=(", ".join(info_params)), extension_schema=extension_schema))
126-
for key, value in enumerate(info_result[0]):
127-
zbx_key, value = "pgsql.{0}".format(
128-
info_items[key][0]), int(value)
129-
zbx.send(zbx_key, value, info_items[key][4])
167+
info_view = 'pg_stat_statements_info'
168+
info_items = self.Items_pg_14
169+
info_params = [x[1] for x in info_items]
170+
info_result = Pooler.query(
171+
self.query_info.format(metrics=(", ".join(info_params)),
172+
extension_schema=extension_schema,
173+
info_view_name=info_view))
174+
for key, value in enumerate(info_result[0]):
175+
zbx_key, value = "pgsql.{0}".format(
176+
info_items[key][0]), int(value)
177+
zbx.send(zbx_key, value, info_items[key][4])
178+
130179
elif Pooler.server_version_greater("13"):
131180
self.Items[5][1] = self.Items[5][1].format("total_exec_time+total_plan_time")
132181
all_items += self.Items_pg_13
182+
133183
else:
134184
self.Items[5][1] = self.Items[5][1].format("total_time")
135185
columns = [x[1] for x in all_items]

Diff for: ‎mamonsu/tools/bootstrap/sql.py

+19-19
Original file line numberDiff line numberDiff line change
@@ -7,7 +7,7 @@
77
$do$
88
BEGIN
99
IF NOT EXISTS (
10-
SELECT FROM pg_catalog.pg_roles
10+
SELECT FROM pg_catalog.pg_roles
1111
WHERE rolname = '{0}') THEN
1212
CREATE ROLE {0} LOGIN PASSWORD '{0}';
1313
IF EXISTS (
@@ -166,15 +166,15 @@
166166
DROP FUNCTION IF EXISTS mamonsu.get_oldest_transaction();
167167
CREATE or REPLACE FUNCTION mamonsu.get_oldest_transaction()
168168
RETURNS DOUBLE PRECISION AS $$
169-
SELECT
170-
CASE WHEN extract(epoch from max(now() - xact_start)) IS NOT null
169+
SELECT
170+
CASE WHEN extract(epoch from max(now() - xact_start)) IS NOT null
171171
AND extract(epoch from max(now() - xact_start))>0
172-
THEN extract(epoch from max(now() - xact_start))
173-
ELSE 0
174-
END
175-
FROM pg_catalog.pg_stat_activity
176-
WHERE
177-
pid NOT IN(select pid from pg_stat_replication) AND
172+
THEN extract(epoch from max(now() - xact_start))
173+
ELSE 0
174+
END
175+
FROM pg_catalog.pg_stat_activity
176+
WHERE
177+
pid NOT IN(select pid from pg_stat_replication) AND
178178
pid <> pg_backend_pid()
179179
$$ LANGUAGE SQL SECURITY DEFINER;
180180
@@ -225,15 +225,15 @@
225225
CREATE OR REPLACE FUNCTION mamonsu.prepared_transaction()
226226
RETURNS TABLE(count_prepared BIGINT, oldest_prepared BIGINT) AS $$
227227
SELECT COUNT(*) AS count_prepared,
228-
coalesce (ROUND(MAX(EXTRACT (EPOCH FROM (now() - prepared)))),0)::bigint AS oldest_prepared
228+
coalesce (ROUND(MAX(EXTRACT (EPOCH FROM (now() - prepared)))),0)::bigint AS oldest_prepared
229229
FROM pg_catalog.pg_prepared_xacts$$ LANGUAGE SQL SECURITY DEFINER;
230230
231231
DROP FUNCTION IF EXISTS mamonsu.count_{3}_lag_lsn();
232232
CREATE OR REPLACE FUNCTION mamonsu.count_{3}_lag_lsn()
233-
RETURNS TABLE(application_name TEXT, {8} total_lag INTEGER) AS $$
233+
RETURNS TABLE(application_name TEXT, {8} total_lag BIGINT) AS $$
234234
SELECT application_name,
235-
{6}
236-
coalesce((pg_{7}_diff(pg_current_{7}(), replay_{9}))::int, 0) AS total_lag
235+
{6}
236+
coalesce((pg_{7}_diff(pg_current_{7}(), replay_{9}))::bigint, 0) AS total_lag
237237
FROM pg_stat_replication
238238
$$ LANGUAGE SQL SECURITY DEFINER;
239239
"""
@@ -287,7 +287,7 @@
287287
FROM pg_extension e
288288
JOIN pg_namespace n
289289
ON e.extnamespace = n.oid
290-
WHERE e.extname = 'pgpro_stats';
290+
WHERE e.extname = 'pgpro_stats';
291291
EXECUTE 'DROP FUNCTION IF EXISTS mamonsu.wait_sampling_all_locks();
292292
CREATE OR REPLACE FUNCTION mamonsu.wait_sampling_all_locks()
293293
RETURNS TABLE(lock_type text, count bigint) AS $$
@@ -298,7 +298,7 @@
298298
FROM (SELECT key, value AS locktuple
299299
FROM jsonb_each((SELECT wait_stats
300300
FROM ' || extension_schema || '.pgpro_stats_totals()
301-
WHERE object_type = ''cluster''))) setoflocks,
301+
WHERE object_type = ''cluster''))) setoflocks,
302302
jsonb_each(setoflocks.locktuple) AS json_data)
303303
SELECT
304304
CASE
@@ -327,7 +327,7 @@
327327
FROM (SELECT key, value AS locktuple
328328
FROM jsonb_each((SELECT wait_stats
329329
FROM ' || extension_schema || '.pgpro_stats_totals()
330-
WHERE object_type = ''cluster''))) setoflocks,
330+
WHERE object_type = ''cluster''))) setoflocks,
331331
jsonb_each(setoflocks.locktuple) AS json_data)
332332
SELECT
333333
lock_type,
@@ -347,7 +347,7 @@
347347
FROM (SELECT key, value AS locktuple
348348
FROM jsonb_each((SELECT wait_stats
349349
FROM ' || extension_schema || '.pgpro_stats_totals()
350-
WHERE object_type = ''cluster''))) setoflocks,
350+
WHERE object_type = ''cluster''))) setoflocks,
351351
jsonb_each(setoflocks.locktuple) AS json_data
352352
WHERE setoflocks.key IN (''Lock'', ''LWLock'', ''LWLockTranche'', ''LWLockNamed''))
353353
SELECT
@@ -415,13 +415,13 @@
415415
FROM pg_extension e
416416
JOIN pg_namespace n
417417
ON e.extnamespace = n.oid
418-
WHERE e.extname = 'pgpro_stats';
418+
WHERE e.extname = 'pgpro_stats';
419419
EXECUTE 'DROP FUNCTION IF EXISTS mamonsu.statements_pro();
420420
CREATE OR REPLACE FUNCTION mamonsu.statements_pro()
421421
RETURNS TABLE({columns}) AS $$
422422
SELECT {metrics}
423423
FROM ' || extension_schema || '.pgpro_stats_totals()
424-
WHERE object_type = ''cluster'';
424+
WHERE object_type = ''cluster'';
425425
$$ LANGUAGE SQL SECURITY DEFINER;';
426426
ELSE
427427
EXIT functions_creation;

Diff for: ‎mamonsu/tools/bootstrap/start.py

+7-1
Original file line numberDiff line numberDiff line change
@@ -237,7 +237,13 @@ def run_deploy():
237237
if Pooler.is_pgpro() or Pooler.is_pgpro_ee():
238238
bootstrap_extension_queries = fill_query_params(CreateWaitSamplingFunctionsSQL)
239239
Pooler.query(bootstrap_extension_queries)
240-
if Pooler.server_version_greater("12"):
240+
if Pooler.extension_installed("pgpro_stats") and Pooler.extension_version_greater("pgpro_stats", "1.8"):
241+
statements_items = [x[1] for x in Statements.Items_pgpro_stats_1_8] + [x[1] for x in Statements.Items_pg_13]
242+
statements_columns = [x[0][x[0].find("[")+1:x[0].find("]")] for x in Statements.Items_pgpro_stats_1_8] + [x[0][x[0].find("[")+1:x[0].find("]")] for x in Statements.Items_pg_13]
243+
bootstrap_extension_queries = CreateStatementsFunctionsSQL.format(
244+
columns=" bigint, ".join(statements_columns) + " bigint", metrics=(", ".join(statements_items)))
245+
Pooler.query(bootstrap_extension_queries)
246+
elif Pooler.server_version_greater("12"):
241247
statements_items = [x[1] for x in Statements.Items] + ([x[1] for x in Statements.Items_pg_13] if Pooler.server_version_greater("13") else [])
242248
statements_items[5] = statements_items[5].format("total_exec_time+total_plan_time")
243249
statements_columns = [x[0][x[0].find("[")+1:x[0].find("]")] for x in Statements.Items] + ([x[0][x[0].find("[")+1:x[0].find("]")] for x in Statements.Items_pg_13] if Pooler.server_version_greater("13") else [])

Diff for: ‎packaging/debian/changelog

+4
Original file line numberDiff line numberDiff line change
@@ -1,3 +1,7 @@
1+
mamonsu (3.5.11-1) stable; urgency=low
2+
* Updated statements plugin: added support for pgpro_stats 1.8;
3+
* Fixed types for count_wal_lag_lsn() function (int to bigint);
4+
15
mamonsu (3.5.10-1) stable; urgency=low
26
* Updated checkpoint plugin: added support for new view pg_stat_checkpointer;
37
* Updated bgwriter plugin: consider updated view pg_stat_bgwriter in postgres 17;

Diff for: ‎packaging/rpm/SPECS/mamonsu.spec

+5-1
Original file line numberDiff line numberDiff line change
@@ -1,5 +1,5 @@
11
Name: mamonsu
2-
Version: 3.5.10
2+
Version: 3.5.11
33
Release: 1%{?dist}
44
Summary: Monitoring agent for PostgreSQL
55
Group: Applications/Internet
@@ -73,6 +73,10 @@ chown -R mamonsu:mamonsu /var/log/mamonsu
7373
chown -R mamonsu:mamonsu /etc/mamonsu
7474

7575
%changelog
76+
* Wed Jan 15 2025 Maxim Styushin <[email protected]> - 3.5.11-1
77+
- Updated statements plugin: added support for pgpro_stats 1.8;
78+
- Fixed types for count_wal_lag_lsn() function (int to bigint);
79+
7680
* Sat Dec 14 2024 Maxim Styushin <[email protected]> - 3.5.10-1
7781
- Updated checkpoint plugin: added support for new view pg_stat_checkpointer;
7882
- Updated bgwriter plugin: consider updated view pg_stat_bgwriter in postgres 17;

Diff for: ‎packaging/win/mamonsu.def.nsh

+1-1
Original file line numberDiff line numberDiff line change
@@ -1,5 +1,5 @@
11
!define NAME Mamonsu
2-
!define VERSION 3.5.10
2+
!define VERSION 3.5.11
33
!define MAMONSU_REG_PATH "Software\PostgresPro\Mamonsu"
44
!define MAMONSU_REG_UNINSTALLER_PATH "Software\Microsoft\Windows\CurrentVersion\Uninstall"
55
!define EDB_REG "SOFTWARE\Postgresql"

0 commit comments

Comments
 (0)