Connect to all nodes of a RedisCluster at startup #3197
-
|
Hello community, I am wondering if it is possible to open the connection to all nodes within a redis cluster on startup. Right now, we are calling Is it possible to have this method create connections to all nodes (essentially to warm them up)? Or is there another way to warm up connections to all nodes in the cluster? |
Beta Was this translation helpful? Give feedback.
Replies: 2 comments 6 replies
-
|
A somewhat similar question was asked in #1111 where warming the connections up could help reduce the problem they are having with manual flushing. By design the driver does not pre-warm cluster connections:
You could iterate over all connections of the cluster, as suggested by @mp911de in #1111 (comment) and - for instance - send a Have in mind - though - that if there is a topology change and a new node is added the connection to it would not be pre-warmed. In this case you might want to listen for ClusterTopologyChangedEvent too. Ultimately we might consider adding such a built-in functionality for edge cases like the one in #1111 but for now it is not planned. Does that help? |
Beta Was this translation helpful? Give feedback.
-
|
Hey @tishun another follow up question on this. We are also using mutual TLS for these connections. Would that affect anything? We deployed a change to send a PING to all master (upstream) nodes in the cluster before serving traffic but we are still seeing high latency on the first request. |
Beta Was this translation helpful? Give feedback.
-
|
I'm having the line of thought about a flag in I am not sure how to think about this in production use yet it might be a valid approach to simplify initialization so that not all clients wanting to have initialized connections must write the the same code. Clearly, there could be variants (pre-init all masters, all replicas, or just pre-init without wait) and I'm also not sure about complexity vs. benefits. |
Beta Was this translation helpful? Give feedback.
-
|
@mp911de thanks for the response. I think such an option would be beneficial. Particularly because it's still not clear to me exactly how to warm up the right connections. I have the following code that is not yet working: public void warmUp() {
this.lettuceClusterConnection.getPartitions().stream()
.filter(node -> node.getRole().isUpstream())
.forEach(node -> {
StatefulRedisConnection<byte[], byte[]> conn = this.lettuceClusterConnection.getConnection(node.getNodeId());
conn.sync().ping();
});
}This does open some connection, but then when I later issue a Some Logs during the warm-up PING, which runs first: Some Logs during the GET command, which runs right after the warmup: Notice how both logs create new connections to I'm hoping someone can show me how to properly warm up the connections that will be used as the |
Beta Was this translation helpful? Give feedback.
-
|
I think the issue is related to this: vs In public CompletableFuture<StatefulRedisConnection<K, V>> getConnectionAsync(ConnectionIntent connectionIntent, String nodeId) {
if (this.debugEnabled) {
logger.debug("getConnection(" + connectionIntent + ", " + nodeId + ")");
}
return this.getConnectionAsync(new ClusterNodeConnectionFactory.ConnectionKey(connectionIntent, nodeId)).toCompletableFuture();
}I can warm up both I did try code to warmup both |
Beta Was this translation helpful? Give feedback.
-
|
Aha ok, sorry for the influx of messages but I think using is what I needed to do. When I warm up these connections for both |
Beta Was this translation helpful? Give feedback.
-
|
Going to write a succinct answer here for others who have the same question about warming up all Redis Cluster connections. As @tishun mentioned in the other answer,
In order to successfully do this, try the following code RedisClusterClient lettuceClient = RedisClusterClient.create(resources, redisURI);
StatefulRedisClusterConnection<byte[], byte[]> lettuceClusterConnection = lettuceClusterClient.connect(ByteArrayCodec.INSTANCE);
// Warm up connections to upstream (primary) nodes
lettuceClusterConnection.getPartitions().stream()
.filter(node -> node.getRole().isUpstream())
.forEach(node -> {
String hostname = node.getUri().getHost();
int port = node.getUri().getPort();
// Each node has a separate connection for read and write operations, so we need to warm up both here.
// Additionally, using the connections returned by getConnection(String nodeId, ConnectionIntent) is NOT the right connection
// to warm up as that connection is not used by redis operations, so warming up that connection does nothing.
// Instead, we must use the connections returned by getConnection(String host, int port, ConnectionIntent)
StatefulRedisConnection<byte[], byte[]> readConn = this.lettuceClusterConnection.getConnection(hostname, port, ConnectionIntent.READ);
StatefulRedisConnection<byte[], byte[]> writeConn = this.lettuceClusterConnection.getConnection(hostname, port, ConnectionIntent.WRITE);
readConn.sync().ping();
writeConn.sync().ping();
}); |
Beta Was this translation helpful? Give feedback.
Going to write a succinct answer here for others who have the same question about warming up all Redis Cluster connections.
As @tishun mentioned in the other answer,
In order to successfully do this, try the following code