wip: design doc

Signed-off-by: Matt Peterson <[email protected]>

Author: mattp-swirldslabs

server/docs/design/bidi-producer-consumer-streaming.md

@@ -0,0 +1,95 @@
+# Bi-directional Producer/Consumer Streaming with gRPC
+
+## Purpose
+The `hedera-block-node` relies on the Helidon 4.x.x server implementations of 
+HTTP/2 and gRPC services to ingest block data from Consensus Nodes and to stream
+block data to downstream consumers.  It does this by defining bidirectional 
+streaming services based on protobuf definitions.
+
+Helidon provides well-defined APIs and extension points to implement business
+logic for these services. The main entry point for custom logic is an implementation
+of `GrpcService`.
+
+---
+A bidirectional consumer service receiving BlockItems from an upstream Consensus Node must fulfill the client-side
+contract:
+
+```java
+BidiStreamingMethod<Stream<T>, StreamObserver<T>> clientBidiStreamingMethod;
+```
+* At runtime, Helidon provides the latest inbound BlockItem to the client-side implementation as well as
+   an object to send a response back to the producer by calling the `onNext()` method.
+
+---
+
+A bidirectional producer service handling downstream BlockItems to consumers must 
+fulfill the server-side contract:
+
+```java
+BidiStreamingMethod<Stream<Response>, StreamObserver<BlockItem>> serverBidiStreamingMethod;
+```
+* At runtime, Helidon provides the latest inbound response from the downstream consumer to the server-side
+   implementation by calling the `onNext()` method.
+
+
+## Entities
+
+**ProducerBlockStreamObserver** - A custom implementation of StreamObserver called by Helidon which is responsible for:
+1) Receiving the latest BlockItem from the producer (e.g. Consensus Node).
+2) Returning a response to the producer.
+
+**ConsumerBlockStreamObserver** - A custom implementation of StreamObserver called by Helidon which is responsible for:
+1) Receiving the latest response from the downstream consumer.
+2) Sending the latest BlockItem to the downstream consumer.
+
+## Approaches:
+
+---
+
+### Approach 1: Directly passing BlockItems from the producer bidirectional service to N bidirectional consumer services
+
+Directly passing BlockItems from the producer bidirectional service to N bidirectional consumer services has the 
+following drawbacks:
+
+1) Each producer must iterate over the list of consumers to pass the BlockItem to each consumer before saving the 
+BlockItem to disk and issuing a response to the producer.  Linear scaling of consumers will aggregate latency resulting
+in the last consumer in the list being delayed by the sum of the latencies of all consumers before it.
+2) Dynamically subscribing/unsubscribing consumers while deterministically broadcasting BlockItems to each consumer in
+the correct order complicates and slows down the process.  It requires thread-safe data structures and synchronization
+on all reads and writes to ensure new/removed subscribers do not disrupt the iteration order of the consumers.
+
+### Approach 2: Shared data structure between producer and consumer services.  Consumers busy-wait for new BlockItems.
+
+Alternatively, if producers store BlockItems in a shared data structure before immediately returning a response to the
+producer, the BlockItem is then immediately available for all consumers to read asynchronously.  Consumers can repeatedly
+poll the shared data structure for new BlockItems.  This approach has the following drawbacks:
+
+1) Busy-waiting consumers will consume more CPU resources polling the shared data structure for new BlockItems.
+2) It is difficult to anticipate an optimal polling interval for consumers as the number of consumers scales up or down.
+3) While prototyping this approach, it appeared that using a busy-wait on a consumer hijacked the thread from responding
+to the responses of the downstream consumer.
+
+### Approach 3: Shared data structure between producer and consumer services.  Consumer responses drive BlockItems sent to the consumers.
+
+With this approach, producers will also store BlockItems in a shared data structure before immediately returning a 
+response to the producer.  However, rather than using a busy-wait to poll for new BlockItems, consumers will be triggered
+to return all the newest BlockItems from the shared data structure when receiving a response from the downstream consumer.
+
+This approach has the following advantages:
+1) It will not consume CPU resources polling.
+2) It will not hijack the thread from responding to the downstream consumer.  Rather, it uses the interaction with the 
+consumer to trigger sending the newest BlockItems downstream.
+3) The shared data structure will need to be concurrent but, after the initial write operation, all subsequent reads
+should not require synchronization.
+4) The shared data structure should decouple the producer from the consumers.  The producer(s) and consumers should
+operate independently of each other and not accrue the same latency issues as Approach 1.
+
+Possible drawbacks:
+1) With this approach, BlockItems sent to the consumer are driven by the downstream consumer's responses.  When Helidon
+invokes `onNext()` with the consumer response, can send the all the latest BlockItems (it will keep track of the last BlockItem sent
+as well as the last BlockItem received from the producer).
+
+## Goals
+
+## Design
+