kad: Implement client-server mode for Kademlia

[lexnv]

This PR implements the client-server mode of operation for Kademlia.

Full nodes and substrate-nodes utilize the server mode by default:

• nodes respond to queries
• They are added to the routing table in automatic mode

Light clients or DHT crawlers are encouraged to use the client mode:

• To preserve resources, they are not responding to queries
• Client nodes are not added to the routing table, therefore not polluting the DHT (or even worse, intertwining multiple DHTs)

To prepare for supporting the dynamic addition and removal of multiple DHTs at runtime, this PR introduces dynamic protocol configuration for the identify protocol.
By default, Kademlia implementations running in client mode are not advertised via the identify protocol.

cc @dmitry-markin @skunert

[skunert]

In general looking good. From looking into this I was wondering about the impact on the routing table. So basically after this change the routing tables will be filles more conservatively which peers which have answered to some of our queries. Peers we discovered on the way but did not query will not be added, even though they might be in server mode (we just don't know). It sounds good to me, I assume we expect no impact? Or even positive impact, because these peers have already answered us.

[gab8i]

Nice! So few lines of code for quite a feature!

[dmitry-markin]

The logic seems good with some minor nits. I am going to analyze some general client-server operation consequences before the final approval, but should be good to go.

[dmitry-markin]

We should also make it possible to switch from client to server mode at runtime — e.g., when we discover a reachable external address. Can be a follow-up to make the PR smaller.

[lexnv]

Have left this as a follow-up work 🙏

[dmitry-markin]

We need to make sure this won't cause the nodes without client-server understanding to busy-loop on trying to open a substream. Reading and silently discarding all the input might be better if it is the case.

[lexnv]

Have added on the register_protocol:

• InboundProtocol::Accept - negotiates inbound substreams over multistream-select + advertise over identify
• InboundProtocol::Deny - rejects inbound substreams over multistream-select + doesnt advertise over identify

This way a client mode Kademlia won't accept any substreams at all coming from server mode Kademlia (ie maybe an old litep2p version).

Litep2p Side

Even we matching different versions: old litep2p instance (server mode without proven peer response) <-> litep2p client mode:

• KadMode::client dials and requests a query from old-litep2p
• old-litep2p doens't add the client mode instance to the routing table unless it is provided as a response peer to a FIND_NODE query
  

  litep2p/src/protocol/libp2p/kademlia/mod.rs

  Lines 482 to 483 in 6e21525

  	// update routing table and inform user about the update
  	self.update_routing_table(&peers).await;

Substrate Side

Substrate does call into the routing table to add a peer as response to the identify protocol:

https://github.com/paritytech/polkadot-sdk/blob/dda8013292008c5a5d105622cd423d47df342bfe/substrate/client/network/src/litep2p/mod.rs#L1180-L1182

This is then gated by the disjoint list of kad protocols:

https://github.com/paritytech/polkadot-sdk/blob/1407d3feba9d3fc95e20823cd0b74acbe1ca24ee/substrate/client/network/src/litep2p/discovery.rs#L371

I. Case old-litep2p + new litep2p client on the same /0x1/kad protocol:

• old-litep2p has /0x1/kad advertised
• new client mode doesnt advertise /0x1/kad (client mode deliberately omits this)

II. Case old-litep2p + multi-genesis (speculative messaging)

• old-litep2p has /0x1/kad advertised (foreign DHT)
• new client mode for /0x1/kad and normal server mode for its own genesis /0x2/kad => the advertised list over identify is /0x2/kad which doesnt match /0x1/kad from old-litep2p

We should be safe here as well