Skip to content

Commit b9cf921

Browse files
szynwelskiszynwelski
committed
Documentation of a decentralized sequencer
1 parent 671a928 commit b9cf921

19 files changed

+937
-12
lines changed

‎warp-academy-docs/docs/sdk/advanced/bundled-interaction.md

Lines changed: 0 additions & 5 deletions
Original file line numberDiff line numberDiff line change
@@ -200,8 +200,3 @@ query {
200200
## Interaction retrieval via Warp gateway
201201

202202
The Warp `/gateway/interactions/:id` endpoint allows to retrieve the interaction info based on its original id.
203-
204-
## Further development
205-
206-
1. Blind/trustless sequencing - https://github.com/warp-contracts/gateway/issues/48
207-
2. Sequencer decentralization - https://github.com/warp-contracts/gateway/issues/93

‎warp-academy-docs/docs/sdk/advanced/vrf.md

Lines changed: 6 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -1,5 +1,11 @@
11
# Verifiable randomness
22

3+
:::caution
4+
Random values will be handled in a new way once the decentralized sequencer is launched.
5+
More details can be found [here](/docs/sequencer/lifecycle/ordering#random-values).
6+
For backward compatibility, the function `SmartWeave.vrf.randomInt(maxValue)` will still work, but it will use the new function `SmartWeave.randomNumber(maxValue)` under the hood.
7+
:::
8+
39
You have an option to use random values in contracts via Verifiable Random Functions.
410
This may be a very useful feature in gaming, dynamic NFT generation, etc.
511

‎warp-academy-docs/docs/sdk/legacy/bundled-interaction-legacy.md

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -1,7 +1,7 @@
11
# Warp Sequencer - legacy
22

33
:::caution
4-
Please note, that you are viewing old Warp Sequencer docs. Since `[email protected]` nested bundles have been introduced instead of the format described in this file. In order to read about current interaction format, please head to [Warp Sequencer](../advanced/bundled-interaction.md).
4+
Please note, that you are viewing old Warp Sequencer docs. Since `[email protected]` nested bundles have been introduced instead of the format described in this file. In order to read about current interaction format, please head to [Warp Sequencer](../../sequencer/introduction.md).
55
:::
66

77
Warp Sequencer is a module that is responsible for submitting users' transactions into Arweave (via [Bundlr Network](https://bundlr.network/)).

‎warp-academy-docs/docs/sequencer/centralized.md

Lines changed: 158 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,158 @@
1+
# Warp Centralized Sequencer
2+
3+
:::caution
4+
This section describes the operation of the current centralized sequencer, which will soon be replaced by the new [decentralized solution](/docs/sequencer/decentralized).
5+
:::
6+
7+
## How it works
8+
9+
Instead of posting the interaction transactions directly to Arweave mainnet, Warp SDK creates a data item with interaction input and tags which is then signed with user's wallet. Data item is then sent to Warp Sequencer (`/gateway/v2/sequencer/register` endpoint) (this is the default behaviour of Warp's SDK `contract.writeInteraction`
10+
function, when `forMainnet` instance is being used).
11+
12+
The Warp Sequencer then:
13+
14+
#### 1. Generates a sort key
15+
16+
A sort key is generated from:
17+
18+
1. current mainnet network height
19+
2. current sequence value
20+
3. original transaction id
21+
4. current mainnet block hash
22+
23+
In the original SmartWeave protocol specification, a _sort key_ is defined
24+
as a `[ block_height,sha256(transactionId + blockHash) ]`, where:
25+
26+
1. `block_height` - current network height, l-padded with `0` to 12 chars, e.g. for block height `1015556`, the
27+
result is `000001015556`
28+
2. `sha256(transactionId + blockHash)` - a `sha256` hash of the concatenated buffers of the transaction id and block
29+
hash,
30+
e.g. for txId `UR_35HORbjjZ_NnUqinkZuWkcNB1-gBST3Rezt5JrDs` and block
31+
hash `ixWCxRN36DjVUxQRa68xIeoZLfvLDTtX78e0ae8RAAJjOPpDBuVKVaEKYOpq7bLS`,
32+
the result is `44edd70f2018924f22a878a558a8f2d5cae8bc1f718d567df43bf52b6384d260`.
33+
34+
The complete _sort key_ for the above values would
35+
be: `000001015556,44edd70f2018924f22a878a558a8f2d5cae8bc1f718d567df43bf52b6384d260`.
36+
37+
The generated sort keys are then used by the SmartWeave protocol to lexicographically sort the transactions.
38+
39+
The Warp Sequencer extends this default mechanism by the current sequence value.
40+
The formula for the _sort key_ is extended to:
41+
`[ block_height,sequence_value,sha256(transactionId + blockHash) ]`
42+
43+
This sequence value can be obtained from the Sequencer's node timestamp, database or other sources.
44+
In its current implementation - a Sequencer node timestamp value is being used.
45+
This in effect gives a fair transactions ordering - the transactions will have the sequence assigned in order in which
46+
they are processed by the Sequencer.
47+
48+
Assuming transaction id `La_NpAFAWxGj-VIiLfg7NbBfox0RZ8uuEJSOOZykd48`, block
49+
hash `-o88tFYsMG9RXSGcNXX5sVDuSV5uHy7zuFRj6vYo91e3mXpmng6qw322Ip0-EguA`,
50+
block height `1015560` and current Sequencer value `1663069424541`, the generated _sort key_ would
51+
be `000001015560,1663069424541,a21ac8a60326ba8c2bb8caa05cff3334a22e9960ef55de0b5392caa30b484d0a`
52+
53+
**NOTE** All the transactions sent to Arweave directly, have the sequence value assigned to `0000000000000000`.
54+
This effectively means that if transactions to a given contract are sent both directly to Arweave mainnet and Warp
55+
Sequencer -
56+
if two transactions happen to be at the same block height, the "direct" Arweave transactions take precedence.
57+
This also means that the sequencing algorithm is fully backwards compatible with the original SmartWeave protocol.
58+
59+
#### 2. Generates tags for the Bundlr transaction
60+
61+
| Tag Name | Tag Value |
62+
| ------------------------------------------- | -------------------------------------------------------------------- |
63+
| `Sequencer` | `Warp` |
64+
| `Sequencer-Owner` | The original owner/signar of the contract transaction |
65+
| `Sequencer-Mills` | The sequence value used by the Sequencer for this transaction |
66+
| `Sequencer-Sort-Key` | The generated sort key for this transaction |
67+
| `Sequencer-Prev-Sort-Key` | The sort key of the previous transaction |
68+
| `Sequencer-Tx-Id` | The original transaction id |
69+
| `Sequencer-Block-Height` | The block height used for generating the sort key |
70+
| `Sequencer-Block-Id` | The block hash used for generating the sort key |
71+
| `Sequencer-Block-Timestamp` | The timestamp of the block that was used for generating the sort key |
72+
| ...all the tags of the original transaction | |
73+
74+
:::tip
75+
The `Sequencer-Prev-Sort-Key` tells what is the sort key of the 'previous' transaction in the sequencer and
76+
can be used to verify whether all transactions have been properly loaded (i.e. if one
77+
decides to load them directly from L1 nodes) and none is missing.
78+
:::
79+
80+
Additional set of tags are added in case user requests generating a random value using VRF (Verifiable Random Function):
81+
82+
| Tag Name | Tag Value |
83+
| ------------ | ----------------------------------------------------------------------- |
84+
| `vrf-index` | The original hash generated by the VRF (using `sort_key` as input data) |
85+
| `vrf-proof` | The original proof generated by the VRF |
86+
| `vrf-bigint` | A BigInt value evaluated from the hash generated by the VRF |
87+
| `vrf-pubkey` | The public key used by the VRF |
88+
89+
Verifiable randomness can be used by contracts that require using random values - e.g. gaming contracts, nft/loot
90+
generating contracts, etc.
91+
Using the `sort_key`, `vrf-proof` and `vrf-pubkey`, the client can always verify the generated random value.
92+
93+
#### 3. Wrap original data item in a bundle and uploads the bundle to Bundlr
94+
95+
Apart from all the tags from point 2, some additional tags are added to the wrapping bundle:
96+
97+
| Tag Name | Tag Value |
98+
| ---------------- | ----------------- |
99+
| `Bundle-Format` | `binary` |
100+
| `Bundle-Version` | `2.0.0` |
101+
| `App-Name` | `Warp` |
102+
| `Action` | `WarpInteraction` |
103+
104+
In order to send original data item to Bundlr, we use the concept of nested bundles and set an interaction data item inside a bundle. If you're not faimiliar with this concept, here is a quick summary:
105+
106+
:::info
107+
Bundling allows to write multiple data items into one top level transaction. A data item differs from a regular transaction by not allowing to transfer AR tokens and passing reward but it has most of the transaction's properties - such as owner, data, tags, owner and id.
108+
109+
In a nutshell, the nested bundles concept means that a data item of a bundle can also be a bundle containg other data items. According to ANS-104 standard it can lead to theoretically unbounded levels of nesting.
110+
111+
You can read the specification for nested bundles standard in [ANS-104](https://github.com/ArweaveTeam/arweave-standards/blob/master/ans/ANS-104.md#31-nested-bundle).
112+
:::
113+
114+
:::info
115+
Obviously the data-item is signed by the Sequencer's wallet - and as such can be easily verified.
116+
The Sequencer wallet address is `jnioZFibZSCcV8o-HkBXYPYEYNib4tqfexP0kCBXX_M`.
117+
:::
118+
119+
**NOTE** The original data item is not modified in any way - this is to preserve the original
120+
signature!
121+
122+
After receiving proper response and receipt from Bundlr, the Warp gateway indexes the contract interaction
123+
internally - to make it instantly available.
124+
125+
#### 4. Finally, the Warp gateway returns the response from the Bundlr to the client.
126+
127+
## Interaction data item retrieval (generated by the Warp Sequencer) via Arweave gateway
128+
129+
Use the GQL endpoint, with the original data item id passed in the `Contract` tag. Note that all the
130+
interactions will be part of a bundle (i.e. will have the `edges.node.bundledIn.id` value set).
131+
132+
```qql
133+
query {
134+
transactions(
135+
ids: ["1UIhK4vL5lc2X4aMsJFmMpJqfdgrjznVzi2F17yLBlc"]
136+
) {
137+
edges {
138+
node {
139+
id
140+
tags {
141+
name
142+
value
143+
}
144+
block {
145+
height
146+
}
147+
bundledIn {
148+
id
149+
}
150+
}
151+
}
152+
}
153+
}
154+
```
155+
156+
## Interaction retrieval via Warp gateway
157+
158+
The Warp `/gateway/interactions/:id` endpoint allows to retrieve the interaction info based on its original id.

‎warp-academy-docs/docs/sequencer/data-and-api.md

Lines changed: 99 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,99 @@
1+
# Data and API Specification
2+
3+
This chapter specifies the format of data stored by the sequencer network and the API provided by each sequencer node.
4+
5+
## Genesis state
6+
7+
The genesis state of the Warp Sequencer is the initial data that is used to start the blockchain. This data includes:
8+
- Information about the Arweave block from which the sequencer will start.
9+
- Data required to calculate the prev sort key for interactions handled by the sequencer.
10+
11+
### Arweave Block Information
12+
13+
The first type of data needed for the sequencer to start is information about the Arweave block that was last processed by the previous centralized sequencer,
14+
as well as data about the Arweave block that will be added first to the sequencer blockchain, along with the interactions it contains.
15+
These details are extracted from the `arweave_block.json` file, and a sample representation is provided below:
16+
17+
```json
18+
{"lastArweaveBlock":
19+
{"height":1318268,
20+
"timestamp":1702040807,
21+
"hash":"VwqjV_2VMlh5DzLHUbk0n3EN_a5aRbe-H2xIFACZ9AmZrujSwwmEdxt4B_M0jYxl"},
22+
"nextArweaveBlock":
23+
{"blockInfo":
24+
{"height":1318269,
25+
"timestamp":1702040913,
26+
"hash":"CQCojSIk5irYvdnsvWKm1d62Oh5z9G3X7qAbNjcu2Yh-KYzgz1IYT9GIpaYIs6cJ"},
27+
"transactions":
28+
[{"id":"UAie4Tyqlz5eCTQK_aqoRzWznZ0UCdckTm6E4pjA_HE",
29+
"contract":"KTzTXT_ANmF84fWEKHzWURD1LWd9QaFR9yfYUwH2Lxw",
30+
"sort_key":"000001318269,0000000000000,56802e09c3d3310140df996fa70be6eccc8c28e715c601e39d820a6be816f54c"},
31+
{"id":"6t-OqbyGeQ-yigDbwRcChtjwpB8w5uswg_yvd9LHE5A",
32+
"contract":"KTzTXT_ANmF84fWEKHzWURD1LWd9QaFR9yfYUwH2Lxw",
33+
"sort_key":"000001318269,0000000000000,6849c6ab2791631ed9645dfb9a7eea6868bcb5f6aa1fea19700bc48a4f9345b9"},
34+
{"id":"1iJSdydrcR2YqEsZ8DVrKIrIIdv-MG7WvRL6cCe76Is",
35+
"contract":"KTzTXT_ANmF84fWEKHzWURD1LWd9QaFR9yfYUwH2Lxw",
36+
"sort_key":"000001318269,0000000000000,e6472aac3115163f3a14de480744be6dc6fc29f860397e50662ea18d7a9bff2f"}]}}
37+
```
38+
Where:
39+
- `lastArweaveBlock` is information about the block that was last processed by the previous sequencer.
40+
- `nextArweaveBlock` is the Arweave block that will be added first to the sequencer.
41+
42+
### Previous Sort Keys for Contracts
43+
44+
Information about the sort key of the last interaction for each contract is crucial for assigning prev sort keys to interactions handled by the sequencer.
45+
This data is sourced from the `prev_sort_keys.json` file, structured as follows:
46+
```json
47+
[{"contract":"007g_77MJ1eJaeRvDYwj4Po-6_2_G-5DGyN7eX3AL4o",
48+
"sortKey":"000001025889,1664397554325,6dca2bac0ee65c6f3a3b0f3e5fe5d162c130bc5c39ea203524b5b799d1ad7f04"},
49+
{"contract":"008p0JdJA0XY79o139Us9OgXbcLmxiKubZ_7v31HZOI",
50+
"sortKey":"000001119402,1676452495692,13468b18dd98a9c0fe9cb3e6f5b482ad14da1b744dea660a373fadb0ff0360d9"},
51+
{"contract":"0094QB2iMzuy7h-W5_577li7kat2jb3EXYmDxRYm2yM",
52+
"sortKey":"000001237223,1691593745386,b4c26ef252025f97b6684fb88e5a2c66ec4f8b1b9d5899e1c05859550aa3265b"},
53+
{"contract":"00Bh4GLtT1kY4sAu3QbsXH3ZSuXa2iBaVXO64LJovp8",
54+
"sortKey":"000000801138,0000000000000,1ed5d58b01f4deda06dcd61bd4e1d8494e82142df0d8333758d8a6d0648eb78f"},
55+
...
56+
]
57+
```
58+
59+
## Messages format
60+
61+
Sequencer blocks encapsulate [transactions](https://docs.cosmos.network/main/learn/advanced/transactions), which in turn contain [messages](https://docs.cosmos.network/main/build/building-modules/messages-and-queries).
62+
Within the realm of Warp Sequencer, two fundamental types of messages play a pivotal role: those tailored for incoming interactions and those designed for Arweave blocks retrieved by the sequencer.
63+
Messages are defined and serialized in the [Protocol Buffers](https://protobuf.dev/) format.
64+
65+
### Interaction Message
66+
67+
The `MsgDataItem` message is used to store an interaction sent to the sequencer. The message has the following structure:
68+
```protobuf
69+
message MsgDataItem {
70+
bytes data_item = 1;
71+
string sort_key = 2;
72+
string prev_sort_key = 3;
73+
bytes random = 4;
74+
}
75+
```
76+
77+
### Arweave Block Message
78+
79+
The `MsgArweaveBlock` message is used to store an Arweave block that has been downloaded by the sequencer. The message has the following fields:
80+
81+
```protobuf
82+
message MsgArweaveBlock {
83+
ArweaveBlockInfo block_info = 1;
84+
repeated ArweaveTransactionWithInfo transactions = 2;
85+
}
86+
```
87+
88+
TODO
89+
90+
## Data stored in sequencer nodes
91+
92+
- last_arweave_block
93+
- prev_sort_key
94+
- next_arweave_block (?)
95+
96+
## API methods
97+
- Open API link
98+
- list of the most important API methods
99+
- instructions on how to connect to notifications (?)

0 commit comments

Comments
 (0)