Skip to content

WSREP: server: node3 unallowed state transition: disconnecting -> synced #251

Description

@ayurchen

To avoid concurrent calls to provider().disconnect() wsrep-lib switches to disconnecting state before the call:

int wsrep::server_state::disconnect()
{
    {
        wsrep::unique_lock<wsrep::mutex> lock(mutex_);
        // In case of failure situations which are caused by provider
        // being shut down some failing operation may also try to shut
        // down the replication. Check the state here and
        // return success if the provider disconnect is already in progress
        // or has completed.
        if (state(lock) == s_disconnecting || state(lock) == s_disconnected)
        {
            return 0;
        }
        state(lock, s_disconnecting);
        interrupt_state_waiters(lock);
    }
    return provider().disconnect();
}

However this creates a race where wsrep-lib is already disconnecting while provider is still connected and is delivering events including state changing events:

2025-04-07 10:42:44 0 [Note] WSREP: Shutdown replication
2025-04-07 10:42:44 0 [Note] WSREP: Disconnect provider
2025-04-07 10:42:44 0 [Note] WSREP: server node3 state change: joined -> disconnecting
2025-04-07 10:42:44 0 [Note] WSREP: Server status change joined -> disconnecting
2025-04-07 10:42:44 0 [Note] WSREP: Member 1.0 (node3) synced with group.
2025-04-07 10:42:44 0 [Note] WSREP: Setting wsrep_ready to 0
2025-04-07 10:42:44 0 [Note] WSREP: wsrep_notify_status server not yet ready : wsrep_ready=0 status 8
2025-04-07 10:42:44 0 [Note] WSREP: Processing event queue:... 100.0% (1/1 events) complete.
2025-04-07 10:42:44 0 [Note] WSREP: Shifting JOINED -> SYNCED (TO: 1761)
2025-04-07 10:42:44 0 [Note] WSREP: Closing send monitor...
2025-04-07 10:42:44 0 [Note] WSREP: Closed send monitor.
2025-04-07 10:42:44 0 [Note] WSREP: gcomm: terminating thread
2025-04-07 10:42:44 0 [Note] WSREP: gcomm: joining thread
2025-04-07 10:42:44 0 [Note] WSREP: gcomm: closing backend
2025-04-07 10:42:44 1 [Note] WSREP: Server node3 synced with group
2025-04-07 10:42:44 1 [Warning] WSREP: server: node3 unallowed state transition: disconnecting -> synced

Such events should be simply ignored in disconnecting state.

Moved here from codership/galera-bugs#1063

Activity

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

Labels

No labels
No labels

Type

No type

Projects

No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions