input.subscription
Required, e.g. projects/myproject/subscriptions/snowplow-enriched. Name of the Pub/Sub subscription with the enriched events
input.parallelPullFactor
Optional. Default value 0.5. parallelPullFactor * cpu count will determine the number of threads used internally by the Pub/Sub client library for fetching events
input.durationPerAckExtension
Optional. Default value 60 seconds. Pubsub ack deadlines are extended for this duration when needed.
input.minRemainingAckDeadline
Optional. Default value 0.1.
Controls when ack deadlines are re-extended, for a message that is close to exceeding its ack deadline.
For example, if durationPerAckExtension is 60 seconds and minRemainingAckDeadline is 0.1 then the loader
will wait until there is 6 seconds left of the remining deadline, before re-extending the message deadline.
input.maxMessagesPerPull
Optional. Default value 1000. How many Pub/Sub messages to pull from the server in a single request.
input.debounceRequests
Optional. Default value 100 millis.
Adds an artifical delay between consecutive requests to Pub/Sub for more messages.
Under some circumstances, this was found to slightly alleviate a problem in which Pub/Sub might re-deliver the same messages multiple times.
output.bad.topic
Required, e.g. projects/myproject/topics/snowplow-bad. Name of the Pub/Sub topic that will receive failed events.
output.bad.batchSize
Optional. Default value 1000. Bad events are sent to Pub/Sub in batches not exceeding this count.
output.bad.requestByteThreshold
Optional. Default value 1000000. Bad events are sent to Pub/Sub in batches with a total size not exceeding this byte threshold
output.bad.maxRecordSize
Optional. Default value 10000000. Any single failed event sent to Pub/Sub should not exceed this size in bytes