Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Using show all slaves status when using MariaDB to be consistent with MySQL #602

Merged
merged 17 commits into from
Jan 19, 2024

Conversation

wfelipew
Copy link
Contributor

@wfelipew wfelipew commented Jan 4, 2024

SUMMARY

Using show all slaves status when using MariaDB to be consistent with MySQL behaviour

According to MySQL documentation if we run SHOW SLAVE STATUS (without a specific replication channel) it should return all replication channels.
But In MariaDB, if we want to get all replication channels we need to use the SHOW ALL SLAVES STATUS otherwise it will only return the default channel.

Fixes #603

ISSUE TYPE
  • Bugfix Pull Request
COMPONENT NAME

mysql_info

ADDITIONAL INFORMATION
Before the change:
$ ansible-playbook -i inventories/ --check --diff  -l mariadb.example.com playbooks/mariadb.yml
PLAY [Install/Configure MariaDB] ****************************************************************************************************************************************

TASK [Gathering Facts] **************************************************************************************************************************************************
Thursday 04 January 2024  19:01:25 +0000 (0:00:00.035)       0:00:00.079 ****** 
ok: [mariadb.example.com]

TASK [example.mariadb : Populate facts] *************************************************************************************************************************************
Thursday 04 January 2024  19:01:44 +0000 (0:00:19.369)       0:00:19.448 ****** 
included: /home/example/Documents/example-ansible_base/roles/example.mariadb/tasks/facts.yml for mariadb.example.com

TASK [example.mariadb : Get mysql_info] *************************************************************************************************************************************
Thursday 04 January 2024  19:01:44 +0000 (0:00:00.023)       0:00:19.471 ****** 
ok: [mariadb.example.com]

TASK [example.mariadb : debug] **********************************************************************************************************************************************
Thursday 04 January 2024  19:02:04 +0000 (0:00:20.055)       0:00:39.526 ****** 
ok: [mariadb.example.com] => {
    "msg": {}
}

After the change:
$ ansible-playbook -i inventories/ --check --diff  -l mariadb.example.com playbooks/mariadb.yml
PLAY [Install/Configure MariaDB] ****************************************************************************************************************************************

TASK [Gathering Facts] **************************************************************************************************************************************************
Thursday 04 January 2024  19:03:03 +0000 (0:00:00.035)       0:00:00.079 ****** 
ok: [mariadb.example.com]

TASK [example.mariadb : Populate facts] *************************************************************************************************************************************
Thursday 04 January 2024  19:03:23 +0000 (0:00:19.892)       0:00:19.971 ****** 
included: /home/example/Documents/example-ansible_base/roles/example.mariadb/tasks/facts.yml for mariadb.example.com

TASK [example.mariadb : Get mysql_info] *************************************************************************************************************************************
Thursday 04 January 2024  19:03:23 +0000 (0:00:00.022)       0:00:19.993 ****** 
ok: [mariadb.example.com]

TASK [example.mariadb : debug] **********************************************************************************************************************************************
Thursday 04 January 2024  19:03:42 +0000 (0:00:19.362)       0:00:39.355 ****** 
ok: [mariadb.example.com] => {
    "msg": {
        "replication.example.com": {
            "3306": {
                "replicate_db_001": {
                    "Connect_Retry": 60,
                    "Connection_name": "example1",
                    "Exec_Master_Log_Pos": 170041708,
                    "Executed_log_entries": 25973857,
                    "Gtid_IO_Pos": "0-8723-1512347238,366-8723-1534994784,466-233-464430931",
                    "Gtid_Slave_Pos": "0-22190-1512603882,366-8723-1536396267,466-233-464430931",
                    "Last_Errno": 0,
                    "Last_Error": "",
                    "Last_IO_Errno": 0,
                    "Last_IO_Error": "",
                    "Last_SQL_Errno": 0,
                    "Last_SQL_Error": "",
                    "Master_Log_File": "log-bin.006904",
                    "Master_SSL_Allowed": "No",
                    "Master_SSL_CA_File": "",
                    "Master_SSL_CA_Path": "",
                    "Master_SSL_Cert": "",
                    "Master_SSL_Cipher": "",
                    "Master_SSL_Crl": "",
                    "Master_SSL_Crlpath": "",
                    "Master_SSL_Key": "",
                    "Master_SSL_Verify_Server_Cert": "No",
                    "Master_Server_Id": 233,
                    "Max_relay_log_size": 1073741824,
                    "Parallel_Mode": "optimistic",
                    "Read_Master_Log_Pos": 170041708,
                    "Relay_Log_File": "relay-bin-example1.000028",
                    "Relay_Log_Pos": 101696731,
                    "Relay_Log_Space": 101697091,
                    "Relay_Master_Log_File": "log-bin.006904",
                    "Replicate_Do_DB": "",
                    "Replicate_Do_Domain_Ids": "",
                    "Replicate_Do_Table": "",
                    "Replicate_Ignore_DB": "mysql,performance_schema,information_schema",
                    "Replicate_Ignore_Domain_Ids": "",
                    "Replicate_Ignore_Server_Ids": "",
                    "Replicate_Ignore_Table": "",
                    "Replicate_Wild_Do_Table": "",
                    "Replicate_Wild_Ignore_Table": "mysql.%,performance_schema.%,information_schema.%",
                    "Retried_transactions": 0,
                    "SQL_Delay": 0,
                    "SQL_Remaining_Delay": null,
                    "Seconds_Behind_Master": 0,
                    "Skip_Counter": 0,
                    "Slave_DDL_Groups": 0,
                    "Slave_IO_Running": "Yes",
                    "Slave_IO_State": "Waiting for master to send event",
                    "Slave_Non_Transactional_Groups": 0,
                    "Slave_SQL_Running": "Yes",
                    "Slave_SQL_Running_State": "Slave has read all relay log; waiting for more updates",
                    "Slave_SQL_State": "Slave has read all relay log; waiting for more updates",
                    "Slave_Transactional_Groups": 1125363,
                    "Slave_heartbeat_period": 30,
                    "Slave_received_heartbeats": 0,
                    "Until_Condition": "None",
                    "Until_Log_File": "",
                    "Until_Log_Pos": 0,
                    "Using_Gtid": "Slave_Pos"
                }
            }
        },
        "master2.example.com": {
            "3306": {
                "replicate_db_001": {
                    "Connect_Retry": 60,
                    "Connection_name": "example2",
                    "Exec_Master_Log_Pos": 298086447,
                    "Executed_log_entries": 9608214,
                    "Gtid_IO_Pos": "0-8723-1512347238,366-8723-1536396267,466-233-463305575",
                    "Gtid_Slave_Pos": "0-22190-1512603882,366-8723-1536396267,466-233-464430931",
                    "Last_Errno": 0,
                    "Last_Error": "",
                    "Last_IO_Errno": 0,
                    "Last_IO_Error": "",
                    "Last_SQL_Errno": 0,
                    "Last_SQL_Error": "",
                    "Master_Log_File": "binlog.000784",
                    "Master_SSL_Allowed": "No",
                    "Master_SSL_CA_File": "",
                    "Master_SSL_CA_Path": "",
                    "Master_SSL_Cert": "",
                    "Master_SSL_Cipher": "",
                    "Master_SSL_Crl": "",
                    "Master_SSL_Crlpath": "",
                    "Master_SSL_Key": "",
                    "Master_SSL_Verify_Server_Cert": "No",
                    "Master_Server_Id": 273,
                    "Max_relay_log_size": 1073741824,
                    "Parallel_Mode": "optimistic",
                    "Read_Master_Log_Pos": 298086447,
                    "Relay_Log_File": "relay-bin-example2.000012",
                    "Relay_Log_Pos": 177215035,
                    "Relay_Log_Space": 177215386,
                    "Relay_Master_Log_File": "binlog.000784",
                    "Replicate_Do_DB": "",
                    "Replicate_Do_Domain_Ids": "",
                    "Replicate_Do_Table": "",
                    "Replicate_Ignore_DB": "mysql,performance_schema,information_schema",
                    "Replicate_Ignore_Domain_Ids": "",
                    "Replicate_Ignore_Server_Ids": "",
                    "Replicate_Ignore_Table": "",
                    "Replicate_Wild_Do_Table": "",
                    "Replicate_Wild_Ignore_Table": "mysql.%,performance_schema.%,information_schema.%",
                    "Retried_transactions": 0,
                    "SQL_Delay": 0,
                    "SQL_Remaining_Delay": null,
                    "Seconds_Behind_Master": 0,
                    "Skip_Counter": 0,
                    "Slave_DDL_Groups": 0,
                    "Slave_IO_Running": "Yes",
                    "Slave_IO_State": "Waiting for master to send event",
                    "Slave_Non_Transactional_Groups": 0,
                    "Slave_SQL_Running": "Yes",
                    "Slave_SQL_Running_State": "Slave has read all relay log; waiting for more updates",
                    "Slave_SQL_State": "Slave has read all relay log; waiting for more updates",
                    "Slave_Transactional_Groups": 1401474,
                    "Slave_heartbeat_period": 30,
                    "Slave_received_heartbeats": 0,
                    "Until_Condition": "None",
                    "Until_Log_File": "",
                    "Until_Log_Pos": 0,
                    "Using_Gtid": "Slave_Pos"
                }
            }
        }
    }
}

Copy link

codecov bot commented Jan 4, 2024

Codecov Report

All modified and coverable lines are covered by tests ✅

Comparison is base (81ab18d) 76.66% compared to head (6ac515b) 74.86%.

Additional details and impacted files
@@            Coverage Diff             @@
##             main     #602      +/-   ##
==========================================
- Coverage   76.66%   74.86%   -1.81%     
==========================================
  Files          28       18      -10     
  Lines        2447     2327     -120     
  Branches      603      585      -18     
==========================================
- Hits         1876     1742     -134     
- Misses        388      405      +17     
+ Partials      183      180       -3     
Flag Coverage Δ
integration 74.17% <81.81%> (+<0.01%) ⬆️
sanity ?
units ?

Flags with carried forward coverage won't be shown. Click here to find out more.

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

Copy link
Collaborator

@Andersson007 Andersson007 left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@wfelipew thanks for the issue and the PR!

@wfelipew @laurent-indermuehle i see you folks are still discussing some implementation things. I'll be happy with everything you'll agree on as I'm not a user.

This is a kind reminder that we need to add a changelog fragment here. @wfelipew could you please add it?

Thanks

@wfelipew
Copy link
Contributor Author

Hey @Andersson007 just added the changelog fragment, could you take a look?

Copy link
Collaborator

@laurent-indermuehle laurent-indermuehle left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks for your contribution!
Finally someone added a way to tell the module if we are running on MySQL and MariaDB. Thank you.
Thought I feel like we could generalize this by moving that logic in module_utils.

EDIT : typo

Comment on lines 392 to 400
def is_mariadb(self):
if self.server_implementation == "mariadb":
return True
else:
return False

def is_mysql(self):
if self.server_implementation == "mysql":
return True
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This would be more at home in module_utils/mysql.py

And it's crazy to think we have a variable db_engine in the integrations tests but we don't have an equivalent in the modules. If we implement this in Python, we could then use it in the integrations tests.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Just moved it to the module_utils/mysql.py

Comment on lines 758 to 763
cursor.execute("SELECT VERSION()")
if 'mariadb' in cursor.fetchone()["VERSION()"].lower():
server_implementation = "mariadb"
else:
server_implementation = "mysql"

Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This could use get_server_version() from module_utils/mysql.py.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Just moved it to the module_utils/mysql.py and using the get_server_version() instead of running the SELECT VERSION()

@laurent-indermuehle
Copy link
Collaborator

@Andersson007 It's a bit late to fire up my brain, but don't you think the module_utils @wfelipew is implementing to differentiate the behavior between MySQL and MariaDb deserve it's own PR?

Also, rather than is_mysql() and is_mariadb() it may be more extensible to set a variable db_engine and test that instead? If someday we add, let's say, 'mariadb-galera', we won't have to add another method is_mariadb_galera.

@Andersson007
Copy link
Collaborator

@Andersson007 It's a bit late to fire up my brain, but don't you think the module_utils @wfelipew is implementing to differentiate the behavior between MySQL and MariaDb deserve it's own PR?

Also, rather than is_mysql() and is_mariadb() it may be more extensible to set a variable db_engine and test that instead? If someday we add, let's say, 'mariadb-galera', we won't have to add another method is_mariadb_galera.

@laurent-indermuehle SGTM, how about implementing the refactoring in a separate PR?

Copy link
Collaborator

@laurent-indermuehle laurent-indermuehle left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I'm sorry to add one more review, but I really don't like those is_mariadb and is_mysql. I think get_server_implementation is a single point were we have to add logic for future implementations and is already easy enough to use.

Comment on lines 215 to 228


def is_mariadb(implementation):
if implementation == "mariadb":
return True
else:
return False


def is_mysql(implementation):
if implementation == "mysql":
return True
else:
return False
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Suggested change
def is_mariadb(implementation):
if implementation == "mariadb":
return True
else:
return False
def is_mysql(implementation):
if implementation == "mysql":
return True
else:
return False

@@ -292,6 +293,8 @@
mysql_driver_fail_msg,
get_connector_name,
get_connector_version,
get_server_implementation,
is_mariadb,
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Suggested change
is_mariadb,

@@ -497,7 +501,10 @@ def __get_master_status(self):

def __get_slave_status(self):
"""Get slave status if the instance is a slave."""
res = self.__exec_sql('SHOW SLAVE STATUS')
if is_mariadb(self.server_implementation):
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Suggested change
if is_mariadb(self.server_implementation):
if self.server_implementation == "mariadb":

__metaclass__ = type

import pytest

from ansible_collections.community.mysql.plugins.module_utils.mysql import get_server_version
from ansible_collections.community.mysql.plugins.module_utils.mysql import get_server_version, get_server_implementation, is_mariadb, is_mysql
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Suggested change
from ansible_collections.community.mysql.plugins.module_utils.mysql import get_server_version, get_server_implementation, is_mariadb, is_mysql
from ansible_collections.community.mysql.plugins.module_utils.mysql import get_server_version, get_server_implementation

Comment on lines 45 to 59

def test_is_mysql():
"""
Test that server is_mysql return expect results
"""
assert is_mysql("mysql") is True
assert is_mysql("mariadb") is False


def test_is_mariadb():
"""
Test that server is_mariadb return expect results
"""
assert is_mariadb("mariadb") is True
assert is_mariadb("mysql") is False
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Suggested change
def test_is_mysql():
"""
Test that server is_mysql return expect results
"""
assert is_mysql("mysql") is True
assert is_mysql("mariadb") is False
def test_is_mariadb():
"""
Test that server is_mariadb return expect results
"""
assert is_mariadb("mariadb") is True
assert is_mariadb("mysql") is False

@wfelipew
Copy link
Contributor Author

@laurent-indermuehle, just removed the is_mariadb and is_mysql. Looking better now?

@Andersson007
Copy link
Collaborator

As @laurent-indermuehle is busy now, I'm merging the PR.

@wfelipew thanks for the contribution! Any other from you will be much appreciated!
@laurent-indermuehle thanks for reviewing! Please create refactoring issues we discussed once you're back

@wfelipew if you're not already there, we have a Matrix channel and a forum group. All details are in the communication guide, fyi and welcome to join

@Andersson007 Andersson007 merged commit 852c19a into ansible-collections:main Jan 19, 2024
40 of 41 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

mysql_info returns an empty list for slave_status when using MariaDB with multiple replication channels
3 participants