Skip to content

Commit cc092de

Browse files
pinheadmzpinheadmz
authored
Merge pull request #316 from pinheadmz/docs-march2024
Update docs for march 2024
2 parents 125c410 + 6aa76f7 commit cc092de

24 files changed

+319
-2302
lines changed

‎README.md

+5-6
Original file line numberDiff line numberDiff line change
@@ -5,14 +5,13 @@ Monitor and analyze the emergent behaviors of Bitcoin networks.
55

66
## Major Features
77

8-
* Launch a bitcoin network with a specified number of nodes connected to each other according to a network topology from a specification file.
9-
* Nodes are assigned random "reachable" IP addresses in the 100.0.0.0/8 subnet which ensures randomized `addrman` bucketing [by Bitcoin Core.](https://github.com/bitcoin/bitcoin/blob/8372ab0ea3c88308aabef476e3b9d023ba3fd4b7/src/addrman.h#L66) (docker only)
8+
* Launch a bitcoin network with a specified number of nodes connected to each other according to a network topology from a graphml file.
109
* Scenarios can be run across the network which can be programmed using the Bitcoin Core functional [test_framework language](https://github.com/bitcoin/bitcoin/tree/master/test/functional).
1110
* Nodes can have traffic shaping parameters assigned to them via the graph using [tc-netem](https://manpages.ubuntu.com/manpages/trusty/man8/tc-netem.8.html) tool.
12-
* Log files from nodes can be accessed directly
13-
* A unified log file can be grepped using regex
14-
* Container resource metrics are reported via a Graphana dashboard.
15-
* P2P messages between any two nodes can be retrieved in chronological order.
11+
* Data from nodes can be collected and searched including log files and p2p messages.
12+
* Performance data from containers can be monitored and visualized.
13+
* Lightning Network nodes can be deployed and operated.
14+
* Small networks can be deployed locally with Docker Compose, larger networks can be deployed remotely with Kubernetes
1615

1716
## Documentation
1817

‎docs/data.md

-2
Original file line numberDiff line numberDiff line change
@@ -40,5 +40,3 @@ warnet_test_uhynisdj_tank_000002: 2023-10-11T17:44:52.325691Z [mempool] AcceptTo
4040
warnet_test_uhynisdj_tank_000002: 2023-10-11T17:44:52.325838Z [validation] TransactionAddedToMempool: txid=94cacabc09b024b56dcbed9ccad15c90340c596e883159bcb5f1d2152997322d wtxid=0cc875e73bb0bd8f892b70b8d1e5154aab64daace8d571efac94c62b8c1da3cf
4141
... (etc)
4242
```
43-
44-
# Next: [Monitoring](monitoring.md)

‎docs/graph.md

+47-98
Original file line numberDiff line numberDiff line change
@@ -5,8 +5,7 @@ Warnet creates a Bitcoin network using a network topology from a [graphml](https
55
Before any scenarios or RPC commands can be executed, a Warnet network must be started from a graph.
66
See [warcli.md](warcli.md) for more details on these commands.
77

8-
To start a network called `"warnet"` from the [default graph file](../src/graphs/default.graphml)
9-
which consists of 12 Bitcoin Core v26.0 nodes connected in a ring:
8+
To start a network called `"warnet"` from the [default graph file](../src/graphs/default.graphml):
109
```
1110
warcli network start
1211
```
@@ -16,113 +15,63 @@ To start a network with custom configurations:
1615
warcli network start <path/to/file.graphml> --network="network_name"
1716
```
1817

19-
## GraphML file specification
20-
21-
```graphml
22-
<?xml version="1.0" encoding="UTF-8"?><graphml xmlns="http://graphml.graphdrawing.org/xmlns">
23-
<key attr.name="label" attr.type="string" for="node" id="label"/>
24-
<key attr.name="x" attr.type="float" for="node" id="x"/>
25-
<key attr.name="y" attr.type="float" for="node" id="y"/>
26-
<key attr.name="version" attr.type="string" for="node" id="version"/>
27-
<key attr.name="bitcoin_config" attr.type="string" for="node" id="bitcoin_config"/>
28-
<key attr.name="tc_netem" attr.type="string" for="node" id="tc_netem"/>
29-
<graph edgedefault="directed">
30-
31-
<!-- <nodes> -->
32-
<!-- <edges> -->
33-
34-
</graph>
35-
</graphml>
36-
37-
```
38-
### Node attributes
39-
40-
* `id` should be a unique integer identifier
41-
* `label` [optional] specifies the node's label
42-
* `x` specifies the node's x position when rendered in a GUI
43-
* `y` specifies the node's y position when rendered in a GUI
44-
* `version` specifies the node's Bitcoin Core release version, or GitHub branch
45-
* `bitcoin_config` is a comma-separated list of values the node should apply to it's bitcoin.conf, using bitcoin.conf syntax
46-
* `tc_netem` is a `tc-netem` command as a string beginning with "tc qdisc add dev eth0 root netem"
47-
48-
`version` should be either a version number from the pre-compiled binary list on https://bitcoincore.org/bin/ **or** a branch to be compiled from GitHub using `<user>/<repo>#<branch>` syntax.
49-
50-
Nodes can be added to the graph as follows:
51-
52-
53-
#### Release binaries
54-
55-
```graphml
56-
<node id="0">
57-
<data key="x">5.5</data>
58-
<data key="y">2.5</data>
59-
<data key="version">26.0</data>
60-
<data key="bitcoin_config">uacomment=warnet0_v24,debugexclude=libevent</data>
61-
<data key="tc_netem"></data>
62-
</node>
63-
```
64-
#### Pre-built custom image
65-
66-
For a pre-built custom image, remove the `version` tag, and add the image repository to an `image` tag.
67-
These should be built using the `warcli image build` command to ensure they have the needed patches.
68-
69-
```graphml
70-
<node id="0">
71-
<data key="x">5.5</data>
72-
<data key="y">2.5</data>
73-
<data key="image">bitcoindevproject/bitcoin:26.0</data>
74-
<data key="bitcoin_config">uacomment=warnet0_v24,debugexclude=libevent</data>
75-
<data key="tc_netem"></data>
76-
</node>
77-
```
78-
79-
#### On-demand built branch
80-
81-
For an on-demand built branch with traffic shaping rules applied using `tc_netem`:
82-
83-
```graphml
84-
<node id="0">
85-
<data key="x">2.5</data>
86-
<data key="y">5.0</data>
87-
<data key="version">vasild/bitcoin#relay_tx_to_priv_nets</data>
88-
<data key="bitcoin_config">uacomment=warnet1_custom,debug=1</data>
89-
<data key="tc_netem">tc qdisc add dev eth0 root netem delay 100ms</data>
90-
</node>
91-
```
92-
93-
`x`, `y`, `version`, `bitcoin_config` and `tc_netem` data fields are optional for all nodes.
94-
95-
### Edges
96-
97-
Edges can be added between the nodes as follows:
98-
99-
```graphml
100-
<edge id="0" source="0" target="1">
101-
</edge>
102-
```
103-
104-
## Creating graphs
18+
## Creating graphs automatically
10519

10620
Graphs can be created via the graph menu:
10721

10822
```bash
10923
# show graph commands
11024
warcli graph --help
11125

112-
# Create an erdos renyi graph of 12 nodes using edge connection probability of 0.3 and default bitcoin version (v26.0)
113-
warcli graph create n=12 p=0.3 --outfile=erdos-renyi_n12_v26.0.graphml
26+
# Create a cycle graph of 12 nodes using default Bitcoin Core version (v26.0)
27+
warcli graph create 12 --outfile=./12_x_v26.0.graphml
11428

115-
# Create an erdos renyi graph of 15 nodes using default edge connection probability of p=0.2 and using random bitcoin versions
116-
warcli graph create n=15 --outfile=erdos-renyi_n15_random.graphml --random
29+
# Start network with default name "warnet"
30+
warcli network start ./12_x_v26.0.graphml
11731
```
11832

119-
## Examples
33+
## Warnet graph nodes and edges
34+
35+
Nodes in a Warnet graph MUST have either a `"version"` key or an `"image"` key.
36+
These dictate what version of Bitcoin Core to deploy in a fiven tank.
12037

121-
1. [example.graphml](../src/graphs/default.graphml) -- This is the default graph.
122-
2. [random_internet_as_graph_n100_pos.graphml](../src/graphs/random_internet_as_graph_n100_pos.graphml)
38+
Edges without additional properties are interpreted as Bitcoin p2p connections.
39+
If an edge has additional key-value properties, it will be interpreted as a
40+
lightning network channel (see [lightning.md](lightning.md)).
12341

124-
![random_internet_as_graph_n100](../docs/random_internet_as_graph_n100.png)
125-
*random_internet_as_graph_n100*
42+
## GraphML file specification
12643

44+
### GraphML file format and headers
45+
```xml
46+
<?xml version="1.0" encoding="UTF-8"?><graphml xmlns="http://graphml.graphdrawing.org/xmlns">
47+
<key id="version" attr.name="version" attr.type="string" for="node" />
48+
<key id="image" attr.name="image" attr.type="string" for="node" />
49+
<key id="bitcoin_config" attr.name="bitcoin_config" attr.type="string" for="node" />
50+
<key id="tc_netem" attr.name="tc_netem" attr.type="string" for="node" />
51+
<key id="exporter" attr.name="exporter" attr.type="boolean" for="node" />
52+
<key id="collect_logs" attr.name="collect_logs" attr.type="boolean" for="node" />
53+
<key id="build_args" attr.name="build_args" attr.type="string" for="node" />
54+
<key id="ln" attr.name="ln" attr.type="string" for="node" />
55+
<key id="ln-image" attr.name="ln-image" attr.type="string" for="node" />
56+
<key id="ln-cb-image" attr.name="ln-cb-image" attr.type="string" for="node" />
57+
<key id="channel" attr.name="channel" attr.type="number" for="edge" />
58+
<graph edgedefault="directed">
59+
<!-- <nodes> -->
60+
<!-- <edges> -->
61+
</graph>
62+
</graphml>
63+
```
12764

128-
# Next: [`warcli` commands](warcli.md)
65+
| key | for | type | default | explanation |
66+
|----------------|-------|---------|-----------|---------------------------------------------------------------------------------------------------------------------------------------------------------------------|
67+
| version | node | string | | Bitcoin Core version with an available Warnet tank image on Dockerhub. May also be a GitHub repository with format user/repository:branch to build from source code |
68+
| image | node | string | | Bitcoin Core Warnet tank image on Dockerhub with the format repository/image:tag |
69+
| bitcoin_config | node | string | | A string of Bitcoin Core options in command-line format, e.g. '-debug=net -blocksonly' |
70+
| tc_netem | node | string | | A tc-netem command as a string beginning with 'tc qdisc add dev eth0 root netem' |
71+
| exporter | node | boolean | False | Whether to attach a Prometheus data exporter to the tank |
72+
| collect_logs | node | boolean | False | Whether to collect Bitcoin Core debug logs with Promtail |
73+
| build_args | node | string | | A string of configure options used when building Bitcoin Core from source code, e.g. '--without-gui --disable-tests' |
74+
| ln | node | string | | Attach a lightning network node of this implementation (currently only supports 'lnd') |
75+
| ln-image | node | string | | Specify a lightning network node image from Dockerhub with the format repository/image:tag |
76+
| ln-cb-image | node | string | | Specify a lnd Circuit Breaker image from Dockerhub with the format repository/image:tag |
77+
| channel | edge | number | | Indicate that this edge is a lightning channel with this specified capacity |

‎docs/install.md

-2
Original file line numberDiff line numberDiff line change
@@ -81,5 +81,3 @@ source .venv/bin/activate
8181
pip install --upgrade pip
8282
pip install -e .
8383
```
84-
85-
# Next: [Running Warnet](running.md)

‎docs/lightning.md

-4
Original file line numberDiff line numberDiff line change
@@ -89,7 +89,3 @@ $ sim-cli --data-dir /Users/bitcoin-dev-project/.warnet/warnet/warnet/simln
8989
2023-11-18T16:58:28.826Z INFO [sim_lib] Started random activity producer for warnet_ln_000001(02318b...c7bf05): activity generator for capacity: 100000000 with multiplier 2: 52.63157894736842 payments per month (0.07309941520467836 per hour).
9090
2023-11-18T16:58:28.826Z INFO [sim_lib] Started random activity producer for warnet_ln_000002(0393aa...9804d5): activity generator for capacity: 50000000 with multiplier 2: 26.31578947368421 payments per month (0.03654970760233918 per hour).
9191
```
92-
93-
94-
95-

‎docs/monitoring.md

-2
Original file line numberDiff line numberDiff line change
@@ -30,5 +30,3 @@ The default dashboard is called "Docker Container & Host Metrics" and can be
3030
accessed via the "dashboards" tab, or from the bottom right of the home screen.
3131

3232
Additional dashboards and datasources may be added in the future.
33-
34-
# Next: [Lightning Network](lightning.md)

‎docs/running.md

-2
Original file line numberDiff line numberDiff line change
@@ -115,5 +115,3 @@ sudo systemctl restart docker
115115
```
116116

117117
On Ubuntu this file is located at `/lib/systemd/system/docker.service` but you can find it using `sudo systemctl status docker`.
118-
119-
# Next: [Network Topology](graph.md)

‎docs/scenarios.md

-2
Original file line numberDiff line numberDiff line change
@@ -45,5 +45,3 @@ Stopped scenario with PID 14683.
4545

4646
To add your own scenario make a copy of one of the existing python tests in src/scenarios/ and write the desired scenario.
4747
Save this file back into the same src/scenarios/ directory and it will be listed and available for running using the aforementioned commands.
48-
49-
# Next: [Collecting Data](data.md)

0 commit comments

Comments
 (0)