ggml : replace reallocation to reuse vector

[lexasub]

Good afternoon! I've found a way to optimize tensor transmission a bit further.
The improvement is approximately -10% in terms of time and -20% according to the profiler. The changes are still in draft form, as I'm unsure how much to allocate initially for the vector.
Perhaps we could extract the "maximum tensor size" from somewhere if it's not too large, or set a constant value—just not as a hardcoded magic number like I used for now (I picked one arbitrarily). I believe the changes are clear to everyone: there won't be any reallocation, whereas previously we were simply calling resize on an empty vector in the recv_msg function.
Also, i go doing experiments with batching sending))
@rgerganov This change is very minor, sorry for bothering you over such a small thing. On one hand, if I create a draft MR, it might go unnoticed. On the other hand, if I open it for merging, someone might accidentally merge my arbitrary constant.
Alse i see warn msg in rpc-server "check_node_graph_compatibility_and_refresh_copy_ops: disabling CUDA graphs due to batch size > 1 [ffn_inp-0] [5120 2 1 1]" what is it?

[ggerganov]

I think the initial reserve value is not very important to be the most optimal. The vector size would settle after a few calls anyway.

[rgerganov]

The problem with this change is that it will make rpc-server consume a lot of RAM because the capacity of set_tensor_vec will be equal to the size of the largest tensor.

I think the proper way for speeding up the model transfer over the network is using hashes and local copies as I have explained here