NATS supercluster server/cluster discovery from client

[NevinDry]

Hi,

I’ve been experimenting with NATS superclusters and I’m a bit confused about the client discovery behavior.

When a client connects to a seed node in one cluster and asks for the available server connections, it only receives the nodes from that same cluster. It doesn’t get the nodes from the other/all clusters in the supercluster.

Is this the expected behavior? My assumption was that when you deploy a new cluster into the supercluster, it would connect to the others through gateways, and its seed/node connection URLs would become discoverable to clients that already know about other clusters.

Right now, if I don’t explicitly provide the seed URL of the new cluster to the client, it never learns about that cluster’s nodes.

I might be missing some configuration?

I’m using the Rust client to connect to NATS. The client fetches available node connection URLs from server_info.connect_urls. This works fine within a single cluster, but when I add a new cluster to my supercluster, the server_info response never includes the connection URLs from that new cluster.

Right now, the options I see are:

• Hot-reload the seed server list and manually add the new cluster’s seed URL.
• Query the gatewayz endpoint to discover gateways and connection URLs.

However, I was expecting that once a new cluster is joined into the supercluster, clients connected to existing clusters would be able to discover the new cluster automatically through the normal server_info mechanism.

Is this by design? Or is there a recommended way for clients to learn about new clusters dynamically without having to explicitly add their seeds?

Has anyone run into something similar?

Thanks!

[derekcollison]

Yes auto-discovery, and all INFO async updates are cluster scoped.

There are ways to allow failover to other clusters but that is a bit clunky atm. We are looking at ways to formalize a better DR (with switchback) story here.

[NevinDry]

Wow wasn't expecting an answer that fast !
Thank you so much for the clarification.
We will see how we can handle that for our usecase.

[derekcollison]

The way you could do it today would be to use two DNS names, one for the primary cluster and one for the DR one, and tell the client not to shuffle the entries when it resolves the A records. This would force the client to go through all the servers in the primary and iff they are not available fall over to the backup cluster servers.

This would not do proper "switch back" - but again we have some thoughts here on how to pull this off /cc @ripienaar