Skip to content

Edit chapter 14 #37

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
merged 3 commits into from
Jun 10, 2024
Merged
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
62 changes: 38 additions & 24 deletions 14_json_serialization.md
Original file line number Diff line number Diff line change
@@ -1,6 +1,7 @@
# JSON Serialization

If you have set up Bitcoin Core and run a `bitcoin-cli` command, you will notice that the output is returned in a certain format. For example:
If you have set up Bitcoin Core and run a `bitcoin-cli` command, you will notice that the output is returned in a certain format.
For example:
```shell
$ bitcoin-cli decoderawtransaction 010000000242d5c1d6f7308bbe95c0f6e1301dd73a8da77d2155b0773bc297ac47f9cd7380010000006a4730440220771361aae55e84496b9e7b06e0a53dd122a1425f85840af7a52b20fa329816070220221dd92132e82ef9c133cb1a106b64893892a11acf2cfa1adb7698dcdc02f01b0121030077be25dc482e7f4abad60115416881fe4ef98af33c924cd8b20ca4e57e8bd5feffffff75c87cc5f3150eefc1c04c0246e7e0b370e64b17d6226c44b333a6f4ca14b49c000000006b483045022100e0d85fece671d367c8d442a96230954cdda4b9cf95e9edc763616d05d93e944302202330d520408d909575c5f6976cc405b3042673b601f4f2140b2e4d447e671c47012103c43afccd37aae7107f5a43f5b7b223d034e7583b77c8cd1084d86895a7341abffeffffff02ebb10f00000000001976a9144ef88a0b04e3ad6d1888da4be260d6735e0d308488ac508c1e000000000017a91476c0c8f2fc403c5edaea365f6a284317b9cdf7258700000000
```
Expand Down Expand Up @@ -64,9 +65,12 @@ This will print the following:

This is the [JSON](https://www.json.org/json-en.html) format, a human-readable data-interchange format that transmits data objects consisting of key-value pairs and arrays.

In Rust, there is a popular crate available for taking some data and converting it into or "**serializing**" it in the JSON format. We can use the [`serde_json`](https://docs.rs/serde_json/latest/serde_json/) crate for this.
In Rust, there is a popular crate available for taking some data and converting it (or "**serializing**" it) into the JSON format.
We can use the [`serde_json`](https://docs.rs/serde_json/latest/serde_json/) crate for this.

Let's add this to our Cargo.toml file. We'll need to bring in the `derive` feature from `serde` so that we can derive the `Serialize` implementation for our structs. We'll also need to bring `serde_json` so that we can convert our struct to a JSON formatted `String`.
Let's add this to our Cargo.toml file.
We'll need to bring in the `derive` feature from `serde` so that we can derive the `Serialize` implementation for our structs.
We'll also need to bring `serde_json` so that we can convert our struct to a JSON formatted `String`.

```toml
[package]
Expand All @@ -84,9 +88,13 @@ serde_json = "1.0.115"

We can follow the example from the documentation here: https://docs.rs/serde_json/latest/serde_json/index.html#creating-json-by-serializing-data-structures.

First, we'll have to import the `Serialize` feature with the `use serde::Serialize;` statement. Then, we can add the `#[derive(Serialize)]` attribute to our `Input` struct. This will work as long as all of the primitive types it contains can be serialized. Finally, we'll call `to_string()` method on `serde_json` to convert inputs into a JSON formatted string. Let's try that and see what happens.
First, we'll have to import the `Serialize` feature with the `use serde::Serialize;` statement.
Then, we can add the `#[derive(Serialize)]` attribute to our `Input` struct.
This will work as long as all of the primitive types it contains can be serialized.
Finally, we'll call `to_string()` method on `serde_json` to convert inputs into a JSON formatted string.
Let's try that and see what happens.

*Reminder: We don't need to import top-level modules such as `serde_json` as those will already be included by default in our `main.rs` file.*
*Note: We don't need to import top-level modules such as `serde_json` as those will already be included by default in our `main.rs` file.*

```rust
use std::io::Read;
Expand All @@ -100,7 +108,6 @@ struct Input {
sequence: u32,
}
...
...
fn main() {
let transaction_hex = "010000000242d5c1d6f7308bbe95c0f6e1301dd73a8da77d2155b0773bc297ac47f9cd7380010000006a4730440220771361aae55e84496b9e7b06e0a53dd122a1425f85840af7a52b20fa329816070220221dd92132e82ef9c133cb1a106b64893892a11acf2cfa1adb7698dcdc02f01b0121030077be25dc482e7f4abad60115416881fe4ef98af33c924cd8b20ca4e57e8bd5feffffff75c87cc5f3150eefc1c04c0246e7e0b370e64b17d6226c44b333a6f4ca14b49c000000006b483045022100e0d85fece671d367c8d442a96230954cdda4b9cf95e9edc763616d05d93e944302202330d520408d909575c5f6976cc405b3042673b601f4f2140b2e4d447e671c47012103c43afccd37aae7107f5a43f5b7b223d034e7583b77c8cd1084d86895a7341abffeffffff02ebb10f00000000001976a9144ef88a0b04e3ad6d1888da4be260d6735e0d308488ac508c1e000000000017a91476c0c8f2fc403c5edaea365f6a284317b9cdf7258700000000";
let transaction_bytes = hex::decode(transaction_hex).unwrap();
Expand Down Expand Up @@ -135,12 +142,15 @@ Ok if we print this out now, we'll see that it's now JSON formatted, but it's st

```console
Version: 1
Inputs: [{"txid":[248,198,147,119,27,41,146,161,27,83,192,69,54,154,179,26,146,13,225,217,33,255,60,20,138,157,4,135,200,249,11,175],"output_index":16,"script":[72,48,69,2,33,0,144,74,46,14,143,89,127,193,204,39,27,98,148,176,151,166,237,201,82,227,12,69,62,53,48,249,36,146,116,151,105,168,2,32,24,70,76,34,91,3,194,135,145,175,6,188,127,237,18,157,202,174,255,158,200,19,90,218,31,177,23,98,206,8,30,169,1,65,4,218,40,145,146,176,132,93,91,137,206,130,102,93,136,172,137,215,87,207,197,253,153,123,29,232,174,71,247,120,12,230,163,34,7,88,59,116,88,209,210,243,253,107,58,59,132,42,234,158,183,137,226,190,165,123,3,212,14,104,77,142,30,5,105],"sequence":4294967295},{"txid":[229,29,33,119,51,43,175,249,207,187,192,132,39,207,13,133,210,138,253,200,20,17,205,187,132,244,12,149,133,139,8,13],"output_index":1,"script":[72,48,69,2,32,54,157,247,212,39,149,35,158,171,249,212,26,238,117,227,255,32,82,23,84,82,43,208,103,137,15,142,237,246,4,76,109,2,33,0,154,207,189,136,213,29,132,45,184,122,185,144,164,139,237,18,177,248,22,233,85,2,208,25,142,208,128,222,69,106,152,141,1,65,4,224,236,152,138,103,153,54,206,168,10,136,230,6,61,98,220,133,24,46,84,138,83,95,174,205,110,86,159,181,101,99,61,229,180,232,61,90,17,251,173,139,1,144,140,231,30,3,116,176,6,216,70,148,176,111,16,189,193,83,202,88,165,63,135],"sequence":4294967295}]
Inputs: [{"txid":[128,115,205,249,71,172,151,194,59,119,176,85,33,125,167,141,58,215,29,48,225,246,192,149,190,139,48,247,214,193,213,66],"output_index":1,"script":[71,48,68,2,32,119,19,97,170,229,94,132,73,107,158,123,6,224,165,61,209,34,161,66,95,133,132,10,247,165,43,32,250,50,152,22,7,2,32,34,29,217,33,50,232,46,249,193,51,203,26,16,107,100,137,56,146,161,26,207,44,250,26,219,118,152,220,220,2,240,27,1,33,3,0,119,190,37,220,72,46,127,74,186,214,1,21,65,104,129,254,78,249,138,243,60,146,76,216,178,12,164,229,126,139,213],"sequence":4294967294},{"txid":[156,180,20,202,244,166,51,179,68,108,34,214,23,75,230,112,179,224,231,70,2,76,192,193,239,14,21,243,197,124,200,117],"output_index":0,"script":[72,48,69,2,33,0,224,216,95,236,230,113,211,103,200,212,66,169,98,48,149,76,221,164,185,207,149,233,237,199,99,97,109,5,217,62,148,67,2,32,35,48,213,32,64,141,144,149,117,197,246,151,108,196,5,179,4,38,115,182,1,244,242,20,11,46,77,68,126,103,28,71,1,33,3,196,58,252,205,55,170,231,16,127,90,67,245,183,178,35,208,52,231,88,59,119,200,205,16,132,216,104,149,167,52,26,191],"sequence":4294967294}]
```

Our `txid` and `script` fields need to be converted to hexadecimal format so that they appear more human readable. For now, we can store them as `String` types instead. This way we can leverage the `hex` library we've already included, and call `hex::encode` to encode both of these types as hex strings.
Our `txid` and `script` fields need to be converted to hexadecimal format so that they appear more human readable.
For now, we can store them as `String` types instead.
This way we can leverage the `hex` library we've already included, and call `hex::encode` to encode both of these types as hex strings.

*Note: It's probably a better idea to store these as their original types for internal purposes and calculations and separate the logic for how we display or serialize data from how it is stored. We'll revisit this separation of concerns in lesson 16 and talk about how to do that.*
*Note: It's probably a better idea to store these as their original types for internal purposes and calculations and separate the logic for how we display or serialize data from how it is stored.
We'll revisit this separation of concerns in lesson 16 and talk about how to do that.*

First, we'll update our `Input` string to change both of these fields to `String` types:
```rust
Expand Down Expand Up @@ -175,34 +185,36 @@ Let's run this now and see what happens.

```console
Version: 1
Inputs: [{"txid":"f8c693771b2992a11b53c045369ab31a920de1d921ff3c148a9d0487c8f90baf","output_index":16,"script":"483045022100904a2e0e8f597fc1cc271b6294b097a6edc952e30c453e3530f92492749769a8022018464c225b03c28791af06bc7fed129dcaaeff9ec8135ada1fb11762ce081ea9014104da289192b0845d5b89ce82665d88ac89d757cfc5fd997b1de8ae47f7780ce6a32207583b7458d1d2f3fd6b3a3b842aea9eb789e2bea57b03d40e684d8e1e0569","sequence":4294967295},{"txid":"e51d2177332baff9cfbbc08427cf0d85d28afdc81411cdbb84f40c95858b080d","output_index":1,"script":"4830450220369df7d42795239eabf9d41aee75e3ff20521754522bd067890f8eedf6044c6d0221009acfbd88d51d842db87ab990a48bed12b1f816e95502d0198ed080de456a988d014104e0ec988a679936cea80a88e6063d62dc85182e548a535faecd6e569fb565633de5b4e83d5a11fbad8b01908ce71e0374b006d84694b06f10bdc153ca58a53f87","sequence":4294967295}]
Inputs: [{"txid":"8073cdf947ac97c23b77b055217da78d3ad71d30e1f6c095be8b30f7d6c1d542","output_index":1,"script":"4730440220771361aae55e84496b9e7b06e0a53dd122a1425f85840af7a52b20fa329816070220221dd92132e82ef9c133cb1a106b64893892a11acf2cfa1adb7698dcdc02f01b0121030077be25dc482e7f4abad60115416881fe4ef98af33c924cd8b20ca4e57e8bd5","sequence":4294967294},{"txid":"9cb414caf4a633b3446c22d6174be670b3e0e746024cc0c1ef0e15f3c57cc875","output_index":0,"script":"483045022100e0d85fece671d367c8d442a96230954cdda4b9cf95e9edc763616d05d93e944302202330d520408d909575c5f6976cc405b3042673b601f4f2140b2e4d447e671c47012103c43afccd37aae7107f5a43f5b7b223d034e7583b77c8cd1084d86895a7341abf","sequence":4294967294}]
```

Ok, a little better but it's still hard to read. Let's look at the documentation and see if there's a better method we can use: https://docs.rs/serde_json/latest/serde_json/#functions. Looks like there's a method called `to_string_pretty`. Let's try that instead of `to_string` and see how that looks.
Ok, a little better but it's still hard to read.
Let's look at the documentation and see if there's a better method we can use: https://docs.rs/serde_json/latest/serde_json/#functions.
Looks like there's a method called `to_string_pretty`.
Let's try that instead of `to_string` and see how that looks.

```console
Version: 1
Inputs: [
{
"txid": "f8c693771b2992a11b53c045369ab31a920de1d921ff3c148a9d0487c8f90baf",
"output_index": 16,
"script": "483045022100904a2e0e8f597fc1cc271b6294b097a6edc952e30c453e3530f92492749769a8022018464c225b03c28791af06bc7fed129dcaaeff9ec8135ada1fb11762ce081ea9014104da289192b0845d5b89ce82665d88ac89d757cfc5fd997b1de8ae47f7780ce6a32207583b7458d1d2f3fd6b3a3b842aea9eb789e2bea57b03d40e684d8e1e0569",
"sequence": 4294967295
"txid": "8073cdf947ac97c23b77b055217da78d3ad71d30e1f6c095be8b30f7d6c1d542",
"output_index": 1,
"script": "4730440220771361aae55e84496b9e7b06e0a53dd122a1425f85840af7a52b20fa329816070220221dd92132e82ef9c133cb1a106b64893892a11acf2cfa1adb7698dcdc02f01b0121030077be25dc482e7f4abad60115416881fe4ef98af33c924cd8b20ca4e57e8bd5",
"sequence": 4294967294
},
{
"txid": "e51d2177332baff9cfbbc08427cf0d85d28afdc81411cdbb84f40c95858b080d",
"output_index": 1,
"script": "4830450220369df7d42795239eabf9d41aee75e3ff20521754522bd067890f8eedf6044c6d0221009acfbd88d51d842db87ab990a48bed12b1f816e95502d0198ed080de456a988d014104e0ec988a679936cea80a88e6063d62dc85182e548a535faecd6e569fb565633de5b4e83d5a11fbad8b01908ce71e0374b006d84694b06f10bdc153ca58a53f87",
"sequence": 4294967295
"txid": "9cb414caf4a633b3446c22d6174be670b3e0e746024cc0c1ef0e15f3c57cc875",
"output_index": 0,
"script": "483045022100e0d85fece671d367c8d442a96230954cdda4b9cf95e9edc763616d05d93e944302202330d520408d909575c5f6976cc405b3042673b601f4f2140b2e4d447e671c47012103c43afccd37aae7107f5a43f5b7b223d034e7583b77c8cd1084d86895a7341abf",
"sequence": 4294967294
}
]
```

Ok, that's way better! Starting to look much more similar to the output from `bitcoin-cli`.

Let's make one more modification to place all the different pieces of a transaction, such as the version, inputs and outputs all into one `Transaction` struct.

We'll add the `Transaction` struct:
Let's make one more modification to place all the different pieces of a transaction, such as the version, inputs and outputs all into a `Transaction` struct.
First, declare the `Transaction` struct:
```rust
#[derive(Debug, Serialize)]
struct Transaction {
Expand Down Expand Up @@ -244,7 +256,7 @@ fn main() {
}
```

And now we should see an output with everything neatly printed out under one `Transaction` JSON object!
Now we should see an output with everything neatly printed out under one `Transaction` JSON object!

```console
Transaction: {
Expand All @@ -266,7 +278,9 @@ Transaction: {
}
```

Pretty neat! Let's keep moving and finish decoding the transaction in the next lesson. After that, we'll set up our program to handle user inputs and deal with handling errors. Onwards!
Pretty neat! Let's keep moving and finish decoding the transaction in the next lesson.
After that, we'll set up our program to handle user inputs and deal with handling errors.
Onwards!

<hr/>

Expand Down