Implement event batching in the miniframework - #364
Open
asubah wants to merge 3 commits into
Open
Conversation
Add an AsyncState to communicate between acquire() and produce()
Add a template type parameter to the edm::EDProducerExternalWork class.
The acquire() method is now const, and it communicates its asynchronous
state to the the produce() method via an object of this type, rather
than using data members.
Implementations that do not need to pass any state between acquire() and
produce() can use "void" or leave the template parameter empty.
Implement simple support for event batching
Add EDBatchingProducer and EDBatchingProducerExternalWork base classes
Implement tests for all kind of EDProducer
Reorganise the existing EDProducer tests, and add tests for
- EDBatchingProducer
- EDBatchingProducerExternalWork
Wrap ownership of and access to event batches
Introduce three new edm classes:
- edm::EventBatch
implements the ownership of a batch of events, implemented as an
std::vector<edm::Event>; used by the Source to provide events to the
StreamSchedule.
- edm::EventRange
implements non-owning, non-const access to a batch of events;
wraps a pair of Event* to the begin and end of an event batch;
passed (by value) to doProduce() and produce().
- edm::ConstEventRange
implements non-owning const access to a batch of events;
wraps a pair of Event* to the begin and end of an event batch;
passed (by value) to doAcquire() and acquire().
Avoid potential overrun while reading multiple events per batch with multiple threads
Adapt the EDProducerExternalWork modules to the new interface
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
This PR picks up from where @fwyzard left in #143.
It contains 3 main commits:
Add a doneWaiting overload without the exception argument:
This commit is not related to the implementation of event batching. It just allows us to call
doneWaiting()without passing an exception pointer instead of passing an empty pointer.Add event batching support for fwtest:
The event batching feature is enabled by passing the
--batchEvents <batch-size>option.From original PR
Note:
The commit deletes the files
src/fwtest/plugin-Test2/TestProducer2.ccandsrc/fwtest/plugin-Test2/TestProducer3.ccand replaces them with batching test.Copy event batching support from fwtest to cuda:
This commit copies the same event baching implementation to cuda.
The cuda modules are not changed at the moment, they still process each event individually even if batching is used. The modules will be updated in the future to take advantage of batching.