Skip to content

Commit 8c5794e

Browse files
fix: changed double-spaces to single-spaces after a .
Signed-off-by: Matt Peterson <[email protected]>
1 parent d3c5487 commit 8c5794e

File tree

1 file changed

+39
-39
lines changed

1 file changed

+39
-39
lines changed

server/docs/design/bidi-producer-consumers-streaming.md

Lines changed: 39 additions & 39 deletions
Original file line numberDiff line numberDiff line change
@@ -4,9 +4,9 @@
44

55
A primary use case of the `hedera-block-node` is to stream live BlockItems (see Terms section) from a producer
66
(e.g. Consensus Node) to N consumers (e.g. Mirror Node) with the lowest possible latency while correctly preserving the
7-
order of the BlockItems. This document outlines several possible strategies to implement this use case and the design
7+
order of the BlockItems. This document outlines several possible strategies to implement this use case and the design
88
of the recommended approach. All strategies rely on the Helidon 4.x.x server implementations of HTTP/2 and gRPC
9-
services to ingest BlockItem data from a producer and then to stream the same BlockItems to downstream consumers. It
9+
services to ingest BlockItem data from a producer and then to stream the same BlockItems to downstream consumers. It
1010
does this by defining bidirectional gRPC streaming services based on protobuf definitions.
1111

1212
Helidon provides well-defined APIs and extension points to implement business logic for these services. The main entry
@@ -27,18 +27,18 @@ point for custom logic is an implementation of `GrpcService`.
2727
### Terms
2828

2929
**BlockItem** - The BlockItem is the primary data structure passed between the producer, the `hedera-block-node`
30-
and consumers. A defined sequence of BlockItems represent a Block when stored on the `hedera-block-node`.
30+
and consumers. A defined sequence of BlockItems represent a Block when stored on the `hedera-block-node`.
3131

3232
**Bidirectional Streaming** - Bidirectional streaming is an [HTTP/2 feature](https://datatracker.ietf.org/doc/html/rfc9113#name-streams-and-multiplexing) allowing both a client and a server to emit
33-
a continuous stream of frames without waiting for responses. In this way, gRPC services can be used to efficiently
33+
a continuous stream of frames without waiting for responses. In this way, gRPC services can be used to efficiently
3434
transmit a continuous flow of BlockItem messages while the HTTP/2 connection is open.
3535

3636
**Producer StreamObserver** - The Producer StreamObserver is a custom implementation of the [gRPC StreamObserver
37-
interface](https://github.com/grpc/grpc-java/blob/0ff3f8e4ac4c265e91b4a0379a32cf25a0a2b2f7/stub/src/main/java/io/grpc/stub/StreamObserver.java#L45) used by Helidon. It is initialized by the BlockItemStreamService (see Entities section). Helidon invokes the Producer
37+
interface](https://github.com/grpc/grpc-java/blob/0ff3f8e4ac4c265e91b4a0379a32cf25a0a2b2f7/stub/src/main/java/io/grpc/stub/StreamObserver.java#L45) used by Helidon. It is initialized by the BlockItemStreamService (see Entities section). Helidon invokes the Producer
3838
StreamObserver at runtime when the producer sends a new BlockItem to the `StreamSink` gRPC service.
3939

4040
**Consumer StreamObserver** - The Consumer StreamObserver is a custom implementation of the [gRPC StreamObserver
41-
interface](https://github.com/grpc/grpc-java/blob/0ff3f8e4ac4c265e91b4a0379a32cf25a0a2b2f7/stub/src/main/java/io/grpc/stub/StreamObserver.java#L45) used by Helidon. It is initialized by the BlockItemStreamService (see Entities section). Helidon invokes the Consumer
41+
interface](https://github.com/grpc/grpc-java/blob/0ff3f8e4ac4c265e91b4a0379a32cf25a0a2b2f7/stub/src/main/java/io/grpc/stub/StreamObserver.java#L45) used by Helidon. It is initialized by the BlockItemStreamService (see Entities section). Helidon invokes the Consumer
4242
StreamObserver at runtime when the downstream consumer of the `StreamSource` gRPC service sends HTTP/2 responses to
4343
sent BlockItems.
4444

@@ -61,45 +61,45 @@ The Block Node gRPC Streaming Services API is now aligned with the names and sim
6161
## Approaches:
6262

6363
All the following approaches require integrating with Helidon 4.x.x gRPC services to implement the bidirectional
64-
streaming API methods defined above. The following objects are used in all approaches:
64+
streaming API methods defined above. The following objects are used in all approaches:
6565

6666
`BlockItemStreamService` is a custom implementation of the Helidon gRPC `GrpcService`. It is responsible for binding
6767
the Helidon routing mechanism to the gRPC streaming methods called by producers and consumers.
6868

6969
`ProducerBlockItemObserver` is a custom implementation of the Helidon gRPC `StreamObserver` interface.
7070
`BlockItemStreamService` instantiates a new `ProducerBlockItemObserver` instance when the `StreamSink` gRPC method is
71-
called by a producer. Thereafter, Helidon invokes `ProducerBlockItemObserver` methods to receive the latest BlockItem
71+
called by a producer. Thereafter, Helidon invokes `ProducerBlockItemObserver` methods to receive the latest BlockItem
7272
from the producer and return BlockItemResponses via a bidirectional stream.
7373

7474
`ConsumerBlockItemObserver` is also a custom implementation of the Helidon gRPC `StreamObserver` interface.
7575
`BlockItemStreamService` instantiates a new `ConsumerBlockItemObserver` instance when the `StreamSource` gRPC method
76-
is called by each consumer. The `ConsumerBlockItemObserver` wraps an instance of `StreamObserver` provided by Helidon
77-
when the connection is established. The `ConsumerBlockItemObserver` uses the `StreamObserver` to send the latest
78-
BlockItem to the downstream consumer. Helidon invokes `ConsumerBlockItemObserver` methods to deliver BlockItemResponses
76+
is called by each consumer. The `ConsumerBlockItemObserver` wraps an instance of `StreamObserver` provided by Helidon
77+
when the connection is established. The `ConsumerBlockItemObserver` uses the `StreamObserver` to send the latest
78+
BlockItem to the downstream consumer. Helidon invokes `ConsumerBlockItemObserver` methods to deliver BlockItemResponses
7979
from the consumer in receipt of BlockItems.
8080

8181

8282
### Approach 1: Directly passing BlockItems from `ProducerBlockItemObserver` to N `ConsumerBlockItemObserver`s.
8383

8484
Directly passing BlockItems from the `ProducerBlockItemObserver` to N `ConsumerBlockItemObserver`s without storing
8585
BlockItems in an intermediate data structure. This approach was the basis for one of the first implementations of gRPC
86-
Live Streaming (see [BlockNode Issue 21](https://github.com/hashgraph/hedera-block-node/issues/21)). Unfortunately, this approach has the following problems:
86+
Live Streaming (see [BlockNode Issue 21](https://github.com/hashgraph/hedera-block-node/issues/21)). Unfortunately, this approach has the following problems:
8787

8888
Drawbacks:
8989
1) Each `ProducerBlockItemObserver` must iterate over the list of subscribed consumers to pass the BlockItem to each
9090
`ConsumerBlockItemObserver` before saving the BlockItem to disk and issuing a BlockItemResponse back to the producer.
9191
The linear scaling of consumers will aggregate latency resulting in the last consumer in the list to be penalized
9292
with the sum of the latencies of all consumers before it.
9393
2) Dynamically subscribing/unsubscribing `ConsumerBlockItemObserver`s while deterministically broadcasting BlockItems
94-
to each consumer in the correct order complicates and slows down the process. It requires thread-safe data
94+
to each consumer in the correct order complicates and slows down the process. It requires thread-safe data
9595
structures and synchronization on all reads and writes to ensure new/removed subscribers do not disrupt the
9696
iteration order of the `ConsumerBlockItemObserver`s.
9797

98-
### Approach 2: Use a shared data structure between `ProducerBlockItemObserver` and `ConsumerBlockItemObserver`s. Consumers busy-wait for new BlockItems.
98+
### Approach 2: Use a shared data structure between `ProducerBlockItemObserver` and `ConsumerBlockItemObserver`s. Consumers busy-wait for new BlockItems.
9999

100100
Alternatively, if `ProducerBlockItemObserver`s store BlockItems in a shared data structure before immediately returning
101101
a response to the producer, the BlockItem is then immediately available for all `ConsumerBlockItemObserver`s to read
102-
asynchronously. Consumers can repeatedly poll the shared data structure for new BlockItems. This approach has the
102+
asynchronously. Consumers can repeatedly poll the shared data structure for new BlockItems. This approach has the
103103
following consequences:
104104

105105
Advantages:
@@ -113,22 +113,22 @@ Drawbacks:
113113
up or down.
114114
3) While prototyping this approach, it appeared that `ConsumerBlockItemObserver`s using a busy-wait to watch for new
115115
BlockItems impaired the ability of the Helidon Virtual Thread instance to process the inbound responses from the
116-
downstream consumer in a timely way. The aggressive behavior of the busy-wait could complicate future use cases
116+
downstream consumer in a timely way. The aggressive behavior of the busy-wait could complicate future use cases
117117
requiring downstream consumer response processing.
118118

119119

120-
### Approach 3: Use a shared data structure between `ProducerBlockItemObserver` and `ConsumerBlockItemObserver`s. Use downstream consumer BlockItemResponses to drive the process of sending new BlockItems.
120+
### Approach 3: Use a shared data structure between `ProducerBlockItemObserver` and `ConsumerBlockItemObserver`s. Use downstream consumer BlockItemResponses to drive the process of sending new BlockItems.
121121

122122
With this approach, the `ProducerBlockItemObserver` will store BlockItems in a shared data structure before immediately
123-
returning a BlockItemResponse to the producer. However, rather than using a busy-wait to poll for new BlockItems,
123+
returning a BlockItemResponse to the producer. However, rather than using a busy-wait to poll for new BlockItems,
124124
`ConsumerBlockItemObserver`s will send new BlockItems only upon receipt of BlockItemResponses from previously sent
125-
BlockItems. When Helidon invokes `onNext()` with a BlockItemResponse, the `ConsumerBlockItemObserver` (using an
125+
BlockItems. When Helidon invokes `onNext()` with a BlockItemResponse, the `ConsumerBlockItemObserver` (using an
126126
internal counter) will calculate and send all newest BlockItems available from the shared data structure to the
127-
downstream consumer. In this way, the downstream consumer responses will drive the process of sending new BlockItems.
127+
downstream consumer. In this way, the downstream consumer responses will drive the process of sending new BlockItems.
128128

129129
Advantages:
130130
1) It will not consume CPU resources polling.
131-
2) It will not hijack the thread from responding to the downstream consumer. Rather, it uses the interaction with the
131+
2) It will not hijack the thread from responding to the downstream consumer. Rather, it uses the interaction with the
132132
consumer to trigger sending the newest BlockItems downstream.
133133
3) The shared data structure will need to be concurrent but, after the initial write operation, all subsequent reads
134134
should not require synchronization.
@@ -141,10 +141,10 @@ Drawbacks:
141141
BlockItemResponses. Given, the latency of a network request/response round-trip, this approach will likely be far
142142
too slow to be considered effective even when sending a batch of all the latest BlockItems.
143143

144-
### Approach 4: Shared data structure between producer and consumer services. Leveraging the LMAX Disruptor library to manage inter-process pub/sub message-passing between producer and consumers via RingBuffer.
144+
### Approach 4: Shared data structure between producer and consumer services. Leveraging the LMAX Disruptor library to manage inter-process pub/sub message-passing between producer and consumers via RingBuffer.
145145

146146
The LMAX Disruptor library is a high-performance inter-process pub/sub message passing library that could be used to
147-
efficiently pass BlockItems between a `ProducerBlockItemObserver` and `ConsumerBlockItemObserver`s. The Disruptor
147+
efficiently pass BlockItems between a `ProducerBlockItemObserver` and `ConsumerBlockItemObserver`s. The Disruptor
148148
library is designed to minimize latency as well as CPU cycles to by not blocking while maintaining concurrency
149149
guarantees.
150150

@@ -161,64 +161,64 @@ Drawbacks:
161161
effectively.
162162
2) Leveraging the Disruptor library requires the communication between the `ProducerBlockItemObserver` and
163163
`ConsumerBlockItemObserver`s to be affiliated by subscribing/unsubscribing the downstream consumers to receive the
164-
latest BlockItems from the producer via the Disruptor RingBuffer. The process of managing these subscriptions to
164+
latest BlockItems from the producer via the Disruptor RingBuffer. The process of managing these subscriptions to
165165
the RingBuffer can be complex.
166166

167167
---
168168

169169
## Design
170170

171171
Given the goals and the proposed approaches, Approach #4 has significant advantages and fewer significant drawbacks.
172-
Using the LMAX Disruptor offers low latency and CPU consumption via a well-maintained and tested API. The RingBuffer
172+
Using the LMAX Disruptor offers low latency and CPU consumption via a well-maintained and tested API. The RingBuffer
173173
intermediate data structure should serve to decouple the producer bidirectional stream from the consumer bidirectional
174-
streams. Please see the following Entities section and Diagrams for a visual representation of the design.
174+
streams. Please see the following Entities section and Diagrams for a visual representation of the design.
175175

176176
### Producer Registration Flow
177177

178178
At boot time, the `BlockItemStreamService` will initialize the `StreamMediator` with the LMAX Disruptor RingBuffer.
179179

180180
When a producer calls the `StreamSink` gRPC method, the `BlockItemStreamService` will create a new
181181
`ProducerBlockItemObserver` instance for Helidon to invoke during the lifecycle of the bidirectional connection to the
182-
upstream producer. The `ProducerBlockItemObserver` is constructed with a reference to the `StreamMediator` and to
182+
upstream producer. The `ProducerBlockItemObserver` is constructed with a reference to the `StreamMediator` and to
183183
the `ResponseStreamObserver` managed by Helidon for transmitting BlockItemResponses to the producer.
184184
See the Producer Registration Flow diagram for more details.
185185

186186
### Consumer Registration Flow
187187

188188
When a consumer calls the `StreamSource` gRPC method, the `BlockItemStreamService` will create a new
189189
`ConsumerBlockItemObserver` instance for Helidon to invoke during the lifecycle of the bidirectional connection to the
190-
downstream consumer. The `ConsumerBlockItemObserver` is constructed with a reference to the `StreamMediator` and to
191-
the `ResponseStreamObserver` managed by Helidon for transmitting BlockItemResponses to the downstream consumer. The
190+
downstream consumer. The `ConsumerBlockItemObserver` is constructed with a reference to the `StreamMediator` and to
191+
the `ResponseStreamObserver` managed by Helidon for transmitting BlockItemResponses to the downstream consumer. The
192192
`BlockItemStreamService` will also subscribe the `ConsumerBlockItemObserver` to the `StreamMediator` to receive the
193193
streaming BlockItems from the producer.
194194

195195

196196
### Runtime Streaming
197197

198198
At runtime, the `ProducerBlockItemObserver` will receive the latest BlockItem from the producer via Helidon and will
199-
invoke publishEvent(BlockItem) on the `StreamMediator` to write the BlockItem to the RingBuffer. The
199+
invoke publishEvent(BlockItem) on the `StreamMediator` to write the BlockItem to the RingBuffer. The
200200
`ProducerBlockItemObserver` will then persist the BlockItem and return a BlockItemResponse to the producer via
201201
its reference to `ResponseStreamObserver`.
202202

203203
Asynchronously, the RingBuffer will invoke the onEvent(BlockItem) method of all the subscribed
204-
`ConsumerBlockItemObserver`s passing them the latest BlockItem. The `ConsumerBlockItemObserver` will then transmit
205-
the BlockItem downstream to the consumer via its reference to the `ResponseStreamObserver`. Downstream consumers will
206-
respond with a BlockItemResponse. Helidon will call the onNext() method of the `ConsumerBlockItemObserver` with the
204+
`ConsumerBlockItemObserver`s passing them the latest BlockItem. The `ConsumerBlockItemObserver` will then transmit
205+
the BlockItem downstream to the consumer via its reference to the `ResponseStreamObserver`. Downstream consumers will
206+
respond with a BlockItemResponse. Helidon will call the onNext() method of the `ConsumerBlockItemObserver` with the
207207
BlockItemResponse.
208208

209209
BlockItems sent to the `ConsumerBlockItemObserver` via the RingBuffer and BlockItemResponses passed by Helidon from
210-
the downstream consumer are used to refresh internal timeouts maintained by the `ConsumerBlockItemObserver`. If a
210+
the downstream consumer are used to refresh internal timeouts maintained by the `ConsumerBlockItemObserver`. If a
211211
configurable timeout threshold is exceeded, the `ConsumerBlockItemObserver` will unsubscribe itself from the
212-
`StreamMediator`. This mechanism is necessary because producers and consumers may not send HTTP/2 `End Stream` DATA
213-
frames to terminate their bidirectional connection. Moreover, Helidon does not throw an exception back up to
214-
`ConsumerBlockItemObserver` when the downstream consumer disconnects. Internal timeouts ensure objects are not
212+
`StreamMediator`. This mechanism is necessary because producers and consumers may not send HTTP/2 `End Stream` DATA
213+
frames to terminate their bidirectional connection. Moreover, Helidon does not throw an exception back up to
214+
`ConsumerBlockItemObserver` when the downstream consumer disconnects. Internal timeouts ensure objects are not
215215
permanently subscribed to the `StreamMediator`.
216216

217217
### Entities
218218

219219
**BlockItemStreamService** - The BlockItemStreamService is a custom implementation of the Helidon gRPC GrpcService.
220220
It is responsible for initializing the StreamMediator and instantiating ProducerBlockItemObserver and
221-
ConsumerBlockItemObserver instances on-demand when the gRPC API is called by producers and consumers. It is
221+
ConsumerBlockItemObserver instances on-demand when the gRPC API is called by producers and consumers. It is
222222
the primary binding between the Helidon routing mechanisms and the `hedera-block-node` custom business logic.
223223

224224
**StreamObserver** - StreamObserver is the main interface through which Helidon 4.x.x invokes custom business logic
@@ -238,7 +238,7 @@ the producer and consumers.
238238
The RingBuffer is a fixed-sized array of ConsumerBlockItemObservers that is managed by the Disruptor library.
239239

240240
**EventHandler** - The EventHandler is an integration interface provided by the Disruptor library as a mechanism to
241-
invoke callback logic when a new BlockItem is written to the RingBuffer. The EventHandler is responsible for passing
241+
invoke callback logic when a new BlockItem is written to the RingBuffer. The EventHandler is responsible for passing
242242
the latest BlockItem to the ConsumerBlockItemObserver when it is available in the RingBuffer.
243243

244244
**ConsumerBlockItemObserver** - A custom implementation of StreamObserver called by Helidon which is responsible for:

0 commit comments

Comments
 (0)