docs: more cleanup, details, and remove compose

Author: pinheadmz

README.md

@@ -24,6 +24,6 @@ Monitor and analyze the emergent behaviors of Bitcoin networks.
 - [Monitoring](/docs/logging_monitoring.md)
 - [Lightning Network](/docs/lightning.md)
 - [Scaling](/docs/scaling.md)
-- [Connecting to local nodes](https://github.com/bitcoin-dev-project/warnet/blob/main/docs/
+- [Connecting to local nodes](https://github.com/bitcoin-dev-project/warnet/blob/main/docs/)
 
 ![warnet-art](https://raw.githubusercontent.com/bitcoin-dev-project/warnet/main/docs/machines.webp)

docs/install.md

@@ -1,26 +1,27 @@
 # Install Warnet
 
-Warnet requires _either_ Kubernetes or Docker to be installed in order to run the network.
-Instructions for both can be found below.
+Warnet requires Kubernetes in order to run the network. Kubernetes can be run
+remotely or locally (with minikube or Docker Desktop). `kubectl` must be run
+locally to administer the network.
 
 ## Dependencies
 
 ### Kubernetes
 
 Install [`kubectl`](https://kubernetes.io/docs/setup/) (or equivalent) and
-configure your cluster. This can be done locally with `minikube` or using a
-managed cluster.
+configure your cluster. This can be done locally with `minikube` (or Docker Desktop)
+or using a managed cluster.
 
 #### Docker engine with minikube
 
 If using Minikube to run a smaller-sized local cluster, you will require docker engine.
-To install docker engine and its `compose` plugin for your system, see: https://docs.docker.com/engine/install/
+To install docker engine, see: https://docs.docker.com/engine/install/
 
 e.g. For Ubuntu:
 
 ```bash
 # First uninstall any old versions
-for pkg in docker.io docker-doc docker-compose podman-docker containerd runc; do sudo apt-get remove $pkg; done
+for pkg in docker.io docker-doc podman-docker containerd runc; do sudo apt-get remove $pkg; done
 
 # Add Docker's official GPG key:
 sudo apt-get update
@@ -36,7 +37,7 @@ echo \
   sudo tee /etc/apt/sources.list.d/docker.list > /dev/null
 sudo apt-get update
 
-# Install the docker packages and docker compose plugin
+# Install the docker packages
 sudo apt-get install docker-ce docker-ce-cli containerd.io docker-buildx-plugin
 ```
 
@@ -50,28 +51,30 @@ The Docker daemon MUST be running before stating Warnet.
 - [Check Docker user/group permissions](https://stackoverflow.com/a/48957722/1653320)
 - or [`chmod` the Docker UNIX socket](https://stackoverflow.com/a/51362528/1653320)
 
-##### macOS
+## Install Warnet
 
-On macOS, a bridge to the docker subnet is required, such as https://github.com/chipmk/docker-mac-net-connect
+### Recommended: use a virtual Python environment such as `venv`
 
 ```bash
-# Install via Homebrew
-brew install chipmk/tap/docker-mac-net-connect
+python3 -m venv .venv # Use alternative venv manager if desired
+source .venv/bin/activate
+```
 
-# Run the service and register it to launch at boot
-sudo brew services start chipmk/tap/docker-mac-net-connect
+```bash
+pip install --upgrade pip
+pip install warnet
 ```
 
-## Download Warnet
+## Contributing / Local Warnet Development
+
+### Download the code repository
 
 ```bash
 git clone https://github.com/bitcoin-dev-project/warnet
 cd warnet
 ```
 
-## Install Warnet
-
-### Optional: use a virtual Python environment such as `venv`
+### Recommended: use a virtual Python environment such as `venv`
 
 ```bash
 python3 -m venv .venv # Use alternative venv manager if desired
@@ -82,3 +85,4 @@ source .venv/bin/activate
 pip install --upgrade pip
 pip install -e .
 ```
+

docs/logging_monitoring.md

@@ -6,15 +6,17 @@ Warnet allows different granularity of logging.
 
 ### Warnet network level logging
 
-For information like:
+Fetch logs from the warnet RPC server `rpc-0`, which is in charge of orchestrating the network.
+
+Examples of information provided:
 
 - how many tanks are running
 - what scenarios are running
-- RPC requests
+- warnet RPC requests
 
-To run these `warcli network logs` or `warcli network logs --follow`.
+Commands: `warcli network logs` or `warcli network logs --follow`.
 
-These logs are fetched from the warnet RPC server `rpc-0`, which is in charge of orchestrating the network.
+See more details in [warcli](/docs/warcli.md#warcli-network-logs)
 
 ### Bitcoin Core logs
 
@@ -36,7 +38,7 @@ For logs of lightning nodes, kubectl is required.
 
 ### Aggregated logs from all nodes
 
-Aggregated logs can be searched using RPC `grep-logs` with regex patterns.
+Aggregated logs can be searched using `warcli bitcoin grep-logs` with regex patterns.
 
 Example:
 
@@ -52,6 +54,8 @@ warnet_test_uhynisdj_tank_000007: 2023-10-11T17:44:52.173199Z [validation] Enque
 ... (etc)
 ```
 
+See more details in [warcli](/docs/warcli.md#warcli-bitcoin-grep-logs)
+
 ## Monitoring and Metrics
 
 ## Install logging infrastructure
@@ -73,7 +77,7 @@ It might take a couple minutes to get the pod running. If you see `error: unable
 The Grafana dashboard (and API) will be accessible without requiring authentication
 at `http://localhost:3000`.
 
-The below logging scripts need to be installed before starting the network in order to collect the information for monitoring and metrics. `warcli network down && warcli network up` should do the trick.
+The `install_logging` script will need to be installed before starting the network in order to collect the information for monitoring and metrics. Restart the network with `warcli network down && warcli network up` if necessary.
 
 ### Prometheus
 

docs/running.md

@@ -1,48 +1,32 @@
 # Running Warnet
 
-Warnet runs a server which can be used to manage multiple networks. On docker
-this runs locally, but on Kubernetes this runs as a `statefulSet` in the
-cluster.
+Warnet runs a server which can be used to manage multiple networks. On Kubernetes
+this runs as a `statefulSet` in the cluster.
 
-If the `$XDG_STATE_HOME` environment variable is set, the server will log to
-a file `$XDG_STATE_HOME/warnet/warnet.log`, otherwise it will use `$HOME/.warnet/warnet.log`.
+See more details in [warcli](/docs/warcli.md), examples:
 
-## Kubernetes
-
-// TODO
-
-### Install logging infrastructure
-
-First make sure you have `helm` installed, then simply run the following script:
-
-```bash
-./scripts/install_logging.sh
-```
-
-To forward port to view Grafana dashboard:
+To start the server run:
 
 ```bash
-./scripts/connect_logging.sh
+warcli cluster deploy
 ```
 
-## Kubernetes (e.g. minikube)
-
-To start the server run:
+Start a network from a graph file:
 
 ```bash
-warcli network start
+warcli network start resources/graphs/default.graphml
 ```
 
-Make sure the tanks are running with:
+Make sure all tanks are running with:
 
 ```bash
- warcli network status
+warcli network status
 ```
 
-Check if the edges of the nodes are connected with:
+Check if the edges of the graph (bitcoin p2p connections) are complete:
 
 ```bash
- warcli network connected
+warcli network connected
 ```
 
 _Optional_ Check out the logs with:

docs/scaling.md

@@ -1,6 +1,6 @@
 # Running large networks
 
-When running a large number of containers on a single host machine (i.e. with the Docker interface), the system may run out of various resources.
+When running a large number of containers on a single host machine, the system may run out of various resources.
 We recommend setting the following values in /etc/sysctl.conf:
 
 ```sh

docs/scenarios.md

@@ -2,8 +2,8 @@
 
 Scenarios are written using the Bitcoin Core test framework for functional testing,
 with some modifications: most notably that `self.nodes[]` represents an array of
-dockerized `bitcoind` nodes. Scenario files can be run with a python interpreter
-or from `warcli` commands and used to control many nodes in the network simultaneously.
+containerized `bitcoind` nodes ("tanks"). Scenario files are run with a python interpreter
+inside the server and can control many nodes in the network simultaneously.
 
 See [`src/warnet/scenarios`](../src/warnet/scenarios) for examples of how these can be written.
 
@@ -13,7 +13,7 @@ To see available scenarios (loaded from the default directory):
 warcli scenarios available
 ```
 
-Once a scenarios is selected it can be run with `warcli scenarios run [--network=warnet] <scenario_name> [scenario_params]`.
+Once a scenario is selected it can be run with `warcli scenarios run [--network=warnet] <scenario_name> [scenario_params]`.
 
 The [`miner_std`](../src/warnet/scenarios/miner_std.py) scenario is a good one to start with as it automates block generation:
 
@@ -22,7 +22,7 @@ The [`miner_std`](../src/warnet/scenarios/miner_std.py) scenario is a good one t
 warcli scenarios run miner_std --allnodes --interval=5
 ```
 
-This will run the run the scenario in the background until it exits or is stopped by the user.
+This will run the scenario in a background thread on the server until it exits or is stopped by the user.
 
 Active scenarios can be listed and terminated by PID:
 
@@ -44,7 +44,11 @@ $ warcli scenarios stop 14683
 Stopped scenario with PID 14683.
 ```
 
-## Add scenarios
+## Running a custom scenario
 
-To add your own scenario make a copy of one of the existing python tests in src/scenarios/ and write the desired scenario.
-Save this file back into the same `src/warnet/scenarios/` directory and it will be listed and available for running using the aforementioned commands.
+You can write your own scenario file locally and upload it to the server with
+the [run-file](/docs/warcli.md#warcli-scenarios-run-file) command (example):
+
+```bash
+warcli scenarios run-file /home/me/bitcoin_attack.py
+```

resources/scripts/connect_logging.sh

@@ -8,7 +8,7 @@ echo "Grafana pod name: ${POD_NAME}"
 
 while true; do
   echo "Attempting to start Grafana port forwarding"
-  echo "(Hang tight... it might take a couple of minutes to get the pod running)"
+  echo "Please wait... it might take a few minutes for all logging pods to start"
   kubectl --namespace warnet-logging port-forward "${POD_NAME}" 3000 2>&1
   echo "Grafana port forwarding exited with status: $?"
   sleep 5