at_c: discussion on connection receive buffers

[JeremyTubongbanua]

Arch discussion happened on July 18

The Problem At Hand

There are 2 issues with our CRUD operations. I will take atclient_get_publickey as an example.

Problem 1

This is the function signature:

 int atclient_get_publickey(atclient *atclient, atclient_atkey *atkey, char *value, const size_t valuesize, size_t *valuelen, bool bypasscache)

We realized that there is a potential for wasted memory space by the caller. If the value was "Hi" (3 bytes) and the caller allocated 4096 bytes, that would be 4093 bytes unused.

Solution to Problem 1

Change function signature to this:

int atclient_get_publickey(atclient *atclient, atclient_atkey *atkey, char **value);

Usage would be like this:

char *value = NULL;
atclient_get_publickey(&atclient, &atkey, &value);
free(value);

value would be allocated to the brim of whatever value it would hold. For example, if it held the value "Hi", then 3 bytes would be allocated.

*value is allocated by the function and is allocated just enough to hold the word "Hi\0" so that no memory is wasted. The caller would be responsible for free-ing the *value and this would be mentioned in the function documentation.

Problem 2

Inside of atclient_get_publickey, we use the valuesize as the buffer size for the network receive buffer. But this is poor implementation. For example, if we were expecting a value of let's say "Hi" for example, this is 3 bytes, so I would allocate a buffer of 3 bytes. While 3 bytes is sufficient to hold the value "Hi", it is not a sufficient buffer size to hold the resposne from the atServer when in the function is in the process of obtaining this value. The function would send a command to the atServer like plookup:all:message@bob and then receive data:{"value": "Hi", "metadata": {...}}. This network buffer has to be bigger than 3 in order to receive that long message that we are getting back from the atServer.

Solution to Problem 2

Option A: Use realloc

Use a fixed buffer size (like 1024) and reuse that same buffer to continuously read from that atServer.

We would also have to reduce the coupling of atclient_connection_send and re-write two entirely new functions: atclient_connection_write and atclient_connection_read

Option B: Have the caller specify network receive buffer size

We could have the caller of the function speciy a network buffer size.

For example,

atclient_get_publickey(&atclient, &atkey, &value, 8192);

The 8192 bytes specified would be a sufficient size to receive and hold the response from the atServer which would be sufficient to hold a string similar to: data:{"value": "Hi", "metadata": {...}}

The caveat to this is the caller shouldn't have to care about what size the buffer should be when the connection receives a response back from the atServer

[JeremyTubongbanua]

#337 implemented atclient_connection_read/write