(2.12.1+) Atomic batch: R1 async flush

[MauriceVanVeen]

At the time of writing using batching with a replicated stream has increased performance over using async publish. However, when using a R1 stream, batching has worse performance than using async publish.

This PR fixes part of that, by having all streams (also R1) use async flush, but ensure we flush manually when required. Allowing batching to not flush for each individual processJetStreamMsg call, and just flush after the last message. This is safe, because we don't clean up the batch until after we've flushed.

Some local benchmarking has shown ~25% performance increase when using batching for R1, but it's still ~20% slower than using async publish. This is because async publish with R1 can simply immediately persist the data, whereas batching needs to persist and flush this into a staging batch-stream, and only persist into the stream itself after commit. All while holding the stream lock, so ingest of new batches is also halted during commit.

When also swapping the file-based stream for storing the R1 batch prior to commit with an in-memory stream (that replicated streams use to stage batches), the performance increases by ~60%. But, that's totally unsafe and doesn't survive hard kills, so that only illustrates staging the batch on disk prior to commit is what's the primary bottleneck.

Signed-off-by: Maurice van Veen [email protected]

[MauriceVanVeen]

Think we should land this after 2.12.0 ships. Couple other fixes in the pipeline that are more important to be included than this one.