Skip to content

Commit 45f128b

Browse files
adamjonasadamjonas
committed
update running docs, added scaling, and rm services
1 parent 6d348e7 commit 45f128b

File tree

9 files changed

+252
-215
lines changed

9 files changed

+252
-215
lines changed

‎README.md

+10-10
Original file line numberDiff line numberDiff line change
@@ -15,15 +15,15 @@ Monitor and analyze the emergent behaviors of Bitcoin networks.
1515

1616
## Documentation
1717

18-
- [Installation](https://github.com/bitcoin-dev-project/warnet/blob/main/docs/install.md)
19-
- [Quick Run](https://github.com/bitcoin-dev-project/warnet/blob/main/docs/quickrun.md)
20-
- [Running Warnet](https://github.com/bitcoin-dev-project/warnet/blob/main/docs/running.md)
21-
- [Network Topology](https://github.com/bitcoin-dev-project/warnet/blob/main/docs/graph.md)
22-
- [CLI Commands](https://github.com/bitcoin-dev-project/warnet/blob/main/docs/warcli.md)
23-
- [Scenarios](https://github.com/bitcoin-dev-project/warnet/blob/main/docs/scenarios.md)
24-
- [Data Collection](https://github.com/bitcoin-dev-project/warnet/blob/main/docs/data.md)
25-
- [Monitoring](https://github.com/bitcoin-dev-project/warnet/blob/main/docs/monitoring.md)
26-
- [Lightning Network](https://github.com/bitcoin-dev-project/warnet/blob/main/docs/lightning.md)
27-
- [Connecting to local nodes](https://github.com/bitcoin-dev-project/warnet/blob/main/docs/connecting-local-nodes.md)
18+
- [Installation](/docs/install.md)
19+
- [Quick Run](/docs/quickrun.md)
20+
- [Running Warnet](/docs/running.md)
21+
- [Network Topology](/docs/graph.md)
22+
- [CLI Commands](/docs/warcli.md)
23+
- [Scenarios](/docs/scenarios.md)
24+
- [Monitoring](/docs/logging_monitoring.md)
25+
- [Lightning Network](/docs/lightning.md)
26+
- [Scaling](/docs/scaling.md)
27+
- [Connecting to local nodes](https://github.com/bitcoin-dev-project/warnet/blob/main/docs/
2828

2929
![warnet-art](https://raw.githubusercontent.com/bitcoin-dev-project/warnet/main/docs/machines.webp)

‎docs/data.md

-42
This file was deleted.

‎docs/logging_monitoring.md

+159
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,159 @@
1+
# Logging and Monitoring
2+
3+
Warnet allows different granularity of logging.
4+
5+
## Logging
6+
7+
### Warnet network level logging
8+
9+
For information like:
10+
11+
- how many tanks are running
12+
- what scenarios are running
13+
- RPC requests
14+
15+
To run these `warcli network logs` or `warcli network logs --follow`.
16+
17+
These logs are fetched from the warnet RPC server `rpc-0`, which is in charge of orchestrating the network.
18+
19+
### Bitcoin Core logs
20+
21+
These are tank level or pod level log output from a Bitcoin Core node, useful for things like net logging and transaction propagation, retrieved by RPC `debug-log` using its network name and graph node index.
22+
23+
Example:
24+
25+
```sh
26+
$ warcli bitcoin debug-log 0
27+
28+
29+
2023-10-11T17:54:39.616974Z Bitcoin Core version v25.0.0 (release build)
30+
2023-10-11T17:54:39.617209Z Using the 'arm_shani(1way,2way)' SHA256 implementation
31+
2023-10-11T17:54:39.628852Z Default data directory /home/bitcoin/.bitcoin
32+
... (etc)
33+
```
34+
35+
For logs of lightning nodes, kubectl is required.
36+
37+
### Aggregated logs from all nodes
38+
39+
Aggregated logs can be searched using RPC `grep-logs` with regex patterns.
40+
41+
Example:
42+
43+
```sh
44+
$ warcli bitcoin grep-logs 94cacabc09b024b56dcbed9ccad15c90340c596e883159bcb5f1d2152997322d
45+
46+
warnet_test_uhynisdj_tank_000001: 2023-10-11T17:44:48.716582Z [miner] AddToWallet 94cacabc09b024b56dcbed9ccad15c90340c596e883159bcb5f1d2152997322d newupdate
47+
warnet_test_uhynisdj_tank_000001: 2023-10-11T17:44:48.717787Z [miner] Submitting wtx 94cacabc09b024b56dcbed9ccad15c90340c596e883159bcb5f1d2152997322d to mempool for relay
48+
warnet_test_uhynisdj_tank_000001: 2023-10-11T17:44:48.717929Z [validation] Enqueuing TransactionAddedToMempool: txid=94cacabc09b024b56dcbed9ccad15c90340c596e883159bcb5f1d2152997322d wtxid=0cc875e73bb0bd8f892b70b8d1e5154aab64daace8d571efac94c62b8c1da3cf
49+
warnet_test_uhynisdj_tank_000001: 2023-10-11T17:44:48.718040Z [validation] TransactionAddedToMempool: txid=94cacabc09b024b56dcbed9ccad15c90340c596e883159bcb5f1d2152997322d wtxid=0cc875e73bb0bd8f892b70b8d1e5154aab64daace8d571efac94c62b8c1da3cf
50+
warnet_test_uhynisdj_tank_000001: 2023-10-11T17:44:48.723017Z [miner] AddToWallet 94cacabc09b024b56dcbed9ccad15c90340c596e883159bcb5f1d2152997322d
51+
warnet_test_uhynisdj_tank_000007: 2023-10-11T17:44:52.173199Z [validation] Enqueuing TransactionAddedToMempool: txid=94cacabc09b024b56dcbed9ccad15c90340c596e883159bcb5f1d2152997322d wtxid=0cc875e73bb0bd8f892b70b8d1e5154aab64daace8d571efac94c62b8c1da3cf
52+
... (etc)
53+
```
54+
55+
## Monitoring and Metrics
56+
57+
## Install logging infrastructure
58+
59+
Ensure that [`helm`](https://helm.sh/docs/intro/install/) is installed, then simply run the following scripts:
60+
61+
```bash
62+
./resources/scripts/install_logging.sh
63+
```
64+
65+
To forward port `3000` and view the [Grafana](#grafana) dashboard run the `connect_logging` script:
66+
67+
```bash
68+
./resources/scripts/connect_logging.sh
69+
```
70+
71+
It might take a couple minutes to get the pod running. If you see `error: unable to forward port because pod is not running. Current status=Pending`, hang tight.
72+
73+
The Grafana dashboard (and API) will be accessible without requiring authentication
74+
at `http://localhost:3000`.
75+
76+
The below logging scripts need to be installed before starting the network in order to collect the information for monitoring and metrics. `warcli network down && warcli network up` should do the trick.
77+
78+
### Prometheus
79+
80+
To monitor RPC return values over time, a Prometheus data exporter can be connected
81+
to any Bitcoin Tank and configured to scrape any available RPC results.
82+
83+
The `bitcoin-exporter` image is defined in `resources/images/exporter` and
84+
maintained in the BitcoinDevProject dockerhub organization. To add the exporter
85+
in the Tank pod with Bitcoin Core add the `"exporter"` key to the node in the graphml file:
86+
87+
```xml
88+
<node id="0">
89+
<data key="version">27.0</data>
90+
<data key="exporter">true</data>
91+
</node>
92+
```
93+
94+
The default metrics are defined in the `bitcoin-exporter` image:
95+
- Block count
96+
- Number of inbound peers
97+
- Number of outbound peers
98+
- Mempool size (# of TXs)
99+
100+
Metrics can be configured by setting a `"metrics"` key to the node in the graphml file.
101+
The metrics value is a space-separated list of labels, RPC commands with arguments, and
102+
JSON keys to resolve the desired data:
103+
104+
```
105+
label=method(arguments)[JSON result key][...]
106+
```
107+
108+
For example, the default metrics listed above are defined as:
109+
110+
```xml
111+
<node id="0">
112+
<data key="version">27.0</data>
113+
<data key="exporter">true</data>
114+
<data key="metrics">blocks=getblockcount() inbounds=getnetworkinfo()["connections_in"] outbounds=getnetworkinfo()["connections_in"] mempool_size=getmempoolinfo()["size"]</data>
115+
</node>
116+
```
117+
118+
The data can be retrieved from the Prometheus exporter on port `9332`, example:
119+
120+
```
121+
# HELP blocks getblockcount()
122+
# TYPE blocks gauge
123+
blocks 704.0
124+
# HELP inbounds getnetworkinfo()["connections_in"]
125+
# TYPE inbounds gauge
126+
inbounds 0.0
127+
# HELP outbounds getnetworkinfo()["connections_in"]
128+
# TYPE outbounds gauge
129+
outbounds 0.0
130+
# HELP mempool_size getmempoolinfo()["size"]
131+
# TYPE mempool_size gauge
132+
mempool_size 0.0
133+
```
134+
135+
### Grafana
136+
137+
Data from Prometheus exporters can be collected and fed into Grafana for a
138+
web-based interface.
139+
140+
#### Dashboards
141+
142+
To view the default metrics in the included default dashboard, upload the dashboard
143+
JSON file to the Grafana server:
144+
145+
```sh
146+
curl localhost:3000/api/dashboards/db \
147+
-H "Content-Type: application/json" \
148+
--data "{\"dashboard\": $(cat resources/configs/grafana/default_dashboard.json)}"
149+
```
150+
151+
Note the URL in the reply from the server (example):
152+
153+
```sh
154+
{"folderUid":"","id":2,"slug":"default-warnet-dashboard","status":"success","uid":"fdu0pda1z6a68b","url":"/d/fdu0pda1z6a68b/default-warnet-dashboard","version":1}(
155+
```
156+
157+
Open the dashboard in your browser (example):
158+
159+
`http://localhost:3000/d/fdu0pda1z6a68b/default-warnet-dashboard`

‎docs/monitoring.md

-102
This file was deleted.

0 commit comments

Comments
 (0)