Allow additional transaction items

[manosbatsis]

Does it make sense to modify the SDK flows in such a way as to allow additional transaction items? For example, consider custom contracts/states used on the road to issuance in some cordapp; wouldn't it be useful to have those updated within e.g. the IssueToken.Initiator's transaction by injecting the corresponding, custom input/output/command TX items?

[RyanGled]

Keen for this.
Unless I'm misunderstanding, this should allow for transaction atomicity of two separate, but accompanying CorDapps when using the Token SDK as a payment layer.

[manosbatsis]

@RyanGled exactly. Perhaps an approach would be refactoring in a way to provide abstract/base flow implementations that can be extended for this by cordapps, without necessarily making it possible with the ready-to-use flows.

[RyanGled]

That sounds like a great idea.
Enabling the effortless interoperability of Tokens/Payment layers with existing CorDapps should encourage SDK use considerably...

[roger-that-dev]

Hey guys (@manosbatsis @RyanGled),

Sorry for the slow reply here. I need to pay more attention to the github issues - I'll remind myself to check it every day :)

If we can come up with a neat way to build these types of flows from simple abstractions then I'm all ears but I don't think we should extend the default issue/move/redeem flows to allow provision of other state types. The business logic will quickly become very complex and I doubt we'll be able to cater to all use-cases. So, unless we can come up with a nice way to extent the existing flows, I suspect we'll add DvP as separate flows. See TwoPartyTradeFlow in the main Corda repository as an example.

What I would like to do is apply the unix philosophy to the tokens SDK. The idea would be to have many simple flows with common and well defined interfaces such that the flows can be easily joined together with minimal fuss. For issuing, an example would be:

1. A flow that creates a transaction with an arbitrary number of output states and commands.
2. From that you can get a flow that issues one single output state.
3. From that you can create a token issue flow.
4. You can then wrap the token issue flow with an additional flow to handle confidential identities.
5. If additional business logic is needed, e.g. for dealing with issuing depository receipts then that can be added as a wrapper flow.
6. Etc.

A very cool way to go about this would be to create flows which pass around, and mutate, transaction builders instead of always returning signed transaction this way you could do the following to get a nice DvP flow:

1. Define a flow which takes a tx builder and adds an arbitrary number of inputs and outputs to it, of some type.
2. Define a move flow which wraps the above but specifically adds a move command and some other bits.
3. Define a flow which calls the above flow twice for two different state types, this is the DvP bit!
4. Wrap the above flow with some additional checks then finalise and commit the transaction.

The key is mutating the builder and passing it around the flows. i'll experiment this week to see what I can come up with. I think this might provide what you need.

On another note... We might look at doing notary batching in the future where multiple transactions can be submitted to a notary in one go such that all transactions are committed if and only if all transactions are valid and all inputs are unique. This will allow you to do DvP without mixing states in flows. The other advantage is that you get a major privacy boost as there's now less need to co-mingle states. Clearly this approach cannot be used all the time as sometimes you just need different state types in the same transaction.

[RyanGled]

@roger3cev The notary batching feature sounds like exactly what we need, it would allow effortless interop between any number of CorDapps - huge.

Would be worth collating a list of people interested, perhaps we can set up some sort of working group to help get this off the ground.

[mikehearn]

Ryan asked me to comment on the app interop and notary batching aspects.

1 - notary batching; to be clear, what Roger is talking about here is a privacy optimisation, it doesn't change what combinations of apps you can do or anything else about the data model. A single Corda transaction can have many different contributions from many different apps all at once, and they all apply atomically. Also, you can "tear off" parts of tx if you want to partially reveal them, which is conceptually very similar. Let's shelve discussion of new privacy mechanisms as we don't really have any plans to do that right now (or at least let's take it to corda-dev)

2 - how best to structure apps for interop.

This is the more interesting topic. I think Roger is on the right lines. The finance app gave a rough first draft: the cash app has an API that lets you say "add inputs+outputs to this TX builder, that makes it pay X tokens to recipient Y".

https://github.com/corda/corda/blob/c39c61ecab7a02e499c9b69e0403260aebd67a3a/finance/workflows/src/main/kotlin/net/corda/finance/flows/TwoPartyTradeFlow.kt#L227

This API isn't a flow. Generally, an API should only be modelled as a subflow if it actually needs to communicate with peers. Otherwise it's just unnecessarily complicated and restricting the situations where it can be called, e.g. it'll be harder to unit test.

But it might be that "add a payment to this tx builder" should be a flow when confidential-identities is considered. There may also be complex business logic involved in making a payment, for instance, doing an on-demand FX trade? So probably it shouldn't be a simple method call like in CashUtils, but a flow that adds stuff to a tx builder. This suggests an 'inner' and 'outer' scheme, where there'd be two flows:

• MakePaymentFlow: outer, designed to be invoked from the shell, RPC at the top level. makes a payment and is done.
• AddPaymentFlow: inner subflow, designed to take a TransactionBuilder in the c'tor (i.e. can't be invoked from the shell, meaningless to start from RPC), doesn't finalize any transaction.

If this naming convention takes off, then any app can be used both for doing the action 'direct' and also composing it with other apps.

Regardless of how it's modelled, the basic idea for app interop is that each app exposes APIs that let them contribute to a TX that's in the middle of being built. Then at the end you finalise it, and the FinalityFlow takes care of distributing the finished transaction to all the participants that were added by the sub-apps. In this way you can do something like the TwoPartyTradeFlow (which potentially can be improved and moved into the tokens SDK?).

[IgorWolkov]

@manosbatsis I made an attempt to solve partially the problem of additional transaction items.
#47

[manosbatsis]

I think that with the new structure (RPC flows, inline flows, util methods) there should be enough legroom and expect to verify early next week

[manosbatsis]

Great work, thanks everyone.