`UTF8JsonGenerator.writeBinary()` should allocate buffer based on supplied length

[kilink]

Currently UTF8JsonGenerator#writeBinary does not pass the size hint to allocBase64Buffer:

jackson-core/src/main/java/tools/jackson/core/json/UTF8JsonGenerator.java

Line 944 in 4ba686e

byte[] encodingBuffer = _ioContext.allocBase64Buffer();

Any reason it couldn't? I am writing binary from a protobuf InputStream, and if the byte[] array passed to its read method is smaller than the proto's serialized size, it triggers serialization to an in memory byte array first before filling the passed in buffer. I'm trying to think of a reason why we can't use the supplied size to allocate the encode buffer? It seems the default is 2000 bytes which is pretty small.

[cowtowncoder]

There's limited benefit from creating huge encoding buffers in general case, since buffers are recycled (so memory retention is relevant too wrt ThreadLocal-based recycling).

But it would perhaps make sense to use some min/max boundaries for buffer size -- like, go up to 64k or so; never smaller than 2k.

[kilink]

Yeah I was thinking that too, 2KB seems very small, but maybe capping it makes sense. In our case, we deal with lots of large binary blobs, and it's only a single thread doing the serialization, a larger buffer (of even 4MB) would help us avoid serializing the proto object to a throwaway byte array first, but I get that it's maybe not the general use case. I'd still be in favor of at least using the size hint to size it up to some limit.

One thing to note, since it's an InputStream we're reading from, internally methods like readNBytes or transferTo use a buffer size of 16KB; just another data point arguing for at least increasing the buffer size.

[cowtowncoder]

I'd be open to PR that passes size hint, capped at say 64kb, as a compromise -- code would base this on dataLength if > 0.
Increasing default from 2kb to... I guess 16kB? Sounds reasonable too.

Not sure if this should go just in 3.x or 2.x and 3.x -- seems simple enough to go in 2.x too, so one PR would be fine (I can roll forward from 2.x to 3.x).

[cowtowncoder]

Note: will update default size only to 4000 since the biggest impact is wrt supplied length; but also since change affects reader buffers too.

[cowtowncoder]

Merged in 2.x for 2.23.0; 3.2 for 3.2.1.