From 48bef38f83be76bdbd37e1cb4317622c8a5a014a Mon Sep 17 00:00:00 2001 From: johnnymatthews <9611008+johnnymatthews@users.noreply.github.com> Date: Mon, 25 Nov 2024 22:52:42 -0400 Subject: [PATCH 1/4] Adds missing dependencies for local devnet install. --- content/guides/spin-up-a-devnet.md | 81 ++++++++++++++++++++++++------ 1 file changed, 67 insertions(+), 14 deletions(-) diff --git a/content/guides/spin-up-a-devnet.md b/content/guides/spin-up-a-devnet.md index 5818549c..9a28cabd 100644 --- a/content/guides/spin-up-a-devnet.md +++ b/content/guides/spin-up-a-devnet.md @@ -11,28 +11,49 @@ Spinning up a devnet using the Docker images supplied in the Entropy Core repo i ### Prerequisites -You need to have [Docker](https://docs.docker.com/engine/install/) and [Docker Compose](https://docs.docker.com/compose/install/) installed. Verify you have them both installed by running: +You need to have the following software installed: -```shell -docker version && docker compose version -``` +- Docker and Docker Compose +- Rust +- A common C Compiler +- OpenSSL +- PKG-config -```output -Client: - Cloud integration: v1.0.35+desktop.13 - Version: 26.1.1 +{{< tabs items="Debian/Ubuntu, MacOS, Arch" >}} -... + {{< tab >}} + ```shell + curl -fsSL https://get.docker.com -o install-docker.sh + sudo sh install-docker.sh + curl --proto '=https' --tlsv1.2 -sSf https://sh.rustup.rs | sh + sudo apt update && sudo apt install build-essential clang libssl-dev pkg-config -y + ``` + {{< /tab >}} -Docker Compose version v2.27.0-desktop.2 -``` + {{< tab >}} + ```shell + brew install --cask docker + curl --proto '=https' --tlsv1.2 -sSf https://sh.rustup.rs | sh + brew install openssl pkg-config + ``` + {{< /tab >}} + + {{< tab >}} + ```shell + sudo pacman -S docker docker-compose + curl --proto '=https' --tlsv1.2 -sSf https://sh.rustup.rs | sh + sudo pacman -S clang libssl-dev pkg-config + ``` + {{< /tab >}} + +{{< /tabs >}} ### Steps 1. Clone the Entropy Core repository and move into the new `entropy-core` directory: ```shell - git clone https://github.com/entropyxyz/entropy-core.git + git clone https://github.com/entropyxyz/entropy-core cd entropy-core ``` @@ -42,13 +63,31 @@ Docker Compose version v2.27.0-desktop.2 echo "127.0.0.1 alice-tss-server bob-tss-server" | sudo tee -a /etc/hosts ``` - You may need to enter your computer's password when prompted. + Enter an admin password if prompted. + +1. Start the Docker daemon, if it isn't running already: + + {{< tabs items="Linux, MacOS" >}} + + {{< tab >}} + ```shell + sudo systemctl start docker + ``` + {{< /tab >}} + + {{< tab >}} + ```shell + dockerd + ``` + {{< /tab >}} + + {{< /tabs >}} 1. Start the Docker containers: ```shell - docker compose up --detach # Detaching is optional. + docker compose up --detach ``` ```output @@ -212,3 +251,17 @@ Developers should strive to mirror the mainnet environment as closely as possibl **Cannot connect to the Docker daemon**: If you see the error message `Cannot connect to the Docker daemon at unix:///Users/johnny/.docker/run/docker.sock. Is the docker daemon running?` it's likely because your Docker daemon isn't running. Double-check that you've opened the Docker application. **I can't build from source**: there are quite a few dependencies for building Substrate-based nodes. Run through the [official Substrate documentation](https://docs.substrate.io/install/) and make sure you have everything installed. + +**Permission denied while trying to connect to the Docker daemon socket**: you likely don't have the correct permissions and user-groups set. Verify that the Docker socket file /var/run/docker.sock has the correct permissions. It should be owned by the `root` user and have appropriate permissions for the `docker` group: + +```shell +sudo chown root:docker /var/run/docker.sock +sudo chmod 0660 /var/run/docker.sock +``` + +Also, make sure that your current user is in the `docker` group: + +```shell +sudo su +usermod -aG docker your_username +``` From 34f8c313d273e9617ed799ff19b8498ee16bb4a5 Mon Sep 17 00:00:00 2001 From: johnnymatthews <9611008+johnnymatthews@users.noreply.github.com> Date: Fri, 3 Jan 2025 11:30:07 -0400 Subject: [PATCH 2/4] Improves devnet structure. Updates build-from-source steps. --- content/guides/spin-up-a-devnet.md | 251 +++++++++-------------------- 1 file changed, 78 insertions(+), 173 deletions(-) diff --git a/content/guides/spin-up-a-devnet.md b/content/guides/spin-up-a-devnet.md index 9a28cabd..658b490d 100644 --- a/content/guides/spin-up-a-devnet.md +++ b/content/guides/spin-up-a-devnet.md @@ -1,74 +1,42 @@ --- -title: "Spin up a devnet" -lead: "A developer network (devnet) is a private, isolated blockchain network that developers use to test and experiment with features and programs without affecting other Entropy networks or risking real-world assets. This guide will walk you through creating a local devnet on your machine." +title: "Spin up a devnet for Entropy" +lead: "A developer network (devnet) is a private blockchain network that mimics the mainnet but is isolated for testing and development purposes. This allows developers to make mistakes and iterate quickly without impacting real users or risking real-world assets. This guide will walk you through setting up a local devnet for the Entropy." --- -Developers should use a devnet when testing new features, experimenting with network parameters, or during initial development stages. However, developers should avoid using it for final production deployments, security audits requiring mainnet conditions, or when real-world economic incentives need to be tested. +A devnet is an essential tool for devs working with Entropy. It provides a safe and controlled environment to: -## Docker image +- Test new features and functionalities. +- Experiment with network parameters. +- Debug and troubleshoot issues. +- Develop and test Entropy Programs without impacting mainnet. -Spinning up a devnet using the Docker images supplied in the Entropy Core repo is the easiest way to get up and running. The requirements are fairly minimal, and everything should work straight out of the box. +This guide will cover two primary methods for setting up a local Entropy devnet: -### Prerequisites - -You need to have the following software installed: - -- Docker and Docker Compose -- Rust -- A common C Compiler -- OpenSSL -- PKG-config - -{{< tabs items="Debian/Ubuntu, MacOS, Arch" >}} +- [Using Docker containers]({{< relref "#docker-containers" >}}): The recommended method for most users due to its ease of use and simplicity. +- [Building from source]({{< relref "building-from-source" >}}): For developers who require more control or are unable to use Docker. - {{< tab >}} - ```shell - curl -fsSL https://get.docker.com -o install-docker.sh - sudo sh install-docker.sh - curl --proto '=https' --tlsv1.2 -sSf https://sh.rustup.rs | sh - sudo apt update && sudo apt install build-essential clang libssl-dev pkg-config -y - ``` - {{< /tab >}} +## Docker containers - {{< tab >}} - ```shell - brew install --cask docker - curl --proto '=https' --tlsv1.2 -sSf https://sh.rustup.rs | sh - brew install openssl pkg-config - ``` - {{< /tab >}} +This method leverages pre-built Docker images to quickly and easily spin up a local devnet. - {{< tab >}} - ```shell - sudo pacman -S docker docker-compose - curl --proto '=https' --tlsv1.2 -sSf https://sh.rustup.rs | sh - sudo pacman -S clang libssl-dev pkg-config - ``` - {{< /tab >}} +### Prerequisites -{{< /tabs >}} +- [Docker](https://docs.docker.com/engine/install/). +- [Docker Compose](https://docs.docker.com/compose/install/). +- Basic understanding of Docker commands. ### Steps -1. Clone the Entropy Core repository and move into the new `entropy-core` directory: +1. Clone the Entropy Core repo: - ```shell - git clone https://github.com/entropyxyz/entropy-core - cd entropy-core - ``` - -1. Add the Alice and Bob threshold-signing services (TSS) to your local `hosts` file: - - ```shell - echo "127.0.0.1 alice-tss-server bob-tss-server" | sudo tee -a /etc/hosts - ``` + ```bash + git clone https://github.com/entropyxyz/entropy-core.git + cd entropy-core + ``` - Enter an admin password if prompted. - -1. Start the Docker daemon, if it isn't running already: - - {{< tabs items="Linux, MacOS" >}} +1. Start the Docker daemon: + {{< tabs items="MacOS, Linux" >}} {{< tab >}} ```shell sudo systemctl start docker @@ -77,174 +45,110 @@ You need to have the following software installed: {{< tab >}} ```shell - dockerd + dockerd ``` {{< /tab >}} - {{< /tabs >}} 1. Start the Docker containers: + ```bash + docker compose up --detach + ``` - ```shell - docker compose up --detach - ``` - - ```output - [+] Running 0/17 - ⠸ bob-tss-server [⠀] Pulling - ⠏ b3d3cc4a5268 Waiting - ⠏ dec0c2d4580b Waiting - - ... - - ✔ Container entropy-devnet-local-bob-chain-node-1 Started - ✔ Container entropy-devnet-local-alice-tss-server-1 Started - ✔ Container entropy-devnet-local-bob-tss-server-1 Started - ``` - -1. Confirm that the containers are up and running: - - ```shell - docker ps - ``` - - ```output - CONTAINER ID IMAGE COMMAND CREATED STATUS PORTS NAMES - 23116711e503 entropyxyz/entropy-tss "/usr/local/bin/entr…" 1 minutes ago Up 4 seconds 9615/tcp, 9944/tcp, 127.0.0.1:3001->3001/tcp, 30333/tcp entropy-devnet-local-alice-tss-server-1 - c83c2ae9da20 entropyxyz/entropy "/usr/local/bin/entr…" 1 minutes ago Up 4 seconds 3001/tcp, 9615/tcp, 30333/tcp, 127.0.0.1:9944->9944/tcp entropy-devnet-local-alice-chain-node-1 - 5088bb75951c entropyxyz/entropy-tss "/usr/local/bin/entr…" 1 minutes ago Up 4 seconds 3001/tcp, 9615/tcp, 9944/tcp, 30333/tcp, 127.0.0.1:3002->3002/tcp entropy-devnet-local-bob-tss-server-1 - 3b0048bcaa00 entropyxyz/entropy "/usr/local/bin/entr…" 1 minutes ago Up 4 seconds 3001/tcp, 9615/tcp, 30333/tcp, 127.0.0.1:9945->9944/tcp entropy-devnet-local-bob-chain-node-1 - ``` - -1. Confirm that the local devnet is functioning by using the Rust test interface within the Entropy Core repo: - - ```shell - cargo run -p entropy-test-cli -- --chain-endpoint="ws://127.0.0.1:9944" status - ``` - - ```output - Finished `dev` profile [unoptimized + debuginfo] target(s) in 0.60s - Running `target/debug/entropy-test-cli '--chain-endpoint=ws://127.0.0.1:9944' status` - - ... - - Hash Stored by: Times used: Size in bytes: Configurable? Has auxiliary? - 0x0000…0000 5GrwvaEF5zXb26Fz9rcQpDWS57CtERHpNehXCPcNoHGKutQY 10 300498 true true - Success: Got status - That took 224.958542ms - ``` - - If this is the first time you are running the Rust testing interface, the `cargo` command above will take a few minutes to complete. +1. Verify container status: -1. You can also verify that things are working as expected by checking the server logs: + ```bash + docker ps + ``` - ```shell - docker compose logs - ``` + This command lists all running Docker containers. Look for containers like `entropy-devnet-local-alice-chain-node-1`. - ```output - alice-chain-node-1 | 2024-06-24 19:41:06 Unexpected status code: 204 - alice-chain-node-1 | 2024-06-24 19:41:06 💤 Idle (1 peers), best: #116 (0xd68c…bfed), finalized #113 (0x06df…be36), ⬇ 0.6kiB/s ⬆ 0.6kiB/s - alice-chain-node-1 | 2024-06-24 19:41:11 💤 Idle (1 peers), best: #116 (0xd68c…bfed), finalized #114 (0xb994…0299), ⬇ 0.6kiB/s ⬆ 0.5kiB/s - ``` +1. (Optional) Check server logs: -1. To stop the network, simply use the `docker stop` command followed by the ID of each Docker container: + ```bash + docker compose logs + ``` - ```shell - docker stop $(docker ps -a -q) - ``` + While optional, this command shows logs from running containers which can be helpful for troubleshooting. - ```output - 23116711e503 - c83c2ae9da20 - 5088bb75951c - 3b0048bcaa00 - ``` +1. Stop all running containers: - Alternatively, you can stop each container individually. + ```bash + docker stop $(docker ps -a -q) + ``` - ```shell - docker stop 23116711 - docker stop c83c2... - - ... - ``` - -1. That's it! - -## Build from source +## Building from Source It is possible to build the chain node and threshold-signature scheme server binaries. However, the process for spinning up a devnet with this method is slightly more involved than the Docker method outlined above. We recommend that you only follow this method if you have a specific reason to _not_ run Docker. ### Prerequisites -You must have the latest LTS version of [Rust](https://www.rust-lang.org/tools/install) installed, along with all the [Substrate dependencies](https://docs.substrate.io/install/) for your operating system. +- [Latest LTS version of Rust](https://www.rust-lang.org/) +- [Substrate dependencies](https://docs.substrate.io/install/) ### Steps -1. Clone the Entropy Core repository and move into the new `entropy-core` directory: +1. Clone the Entropy Core repository: - ```shell - git clone https://github.com/entropyxyz/entropy-core.git - cd entropy-core - ``` + ```bash + git clone https://github.com/entropyxyz/entropy-core.git + cd entropy-core + ``` -1. Build the chain node and threshold signature scheme server binaries: +1. Compile the source into an executable binary: - ```shell - cargo build --release - ``` + ```bash + cargo build --release + ``` - ```output - Downloaded asn1-rs-derive v0.4.0 - Downloaded byte-tools v0.3.1 - Downloaded const-random-macro v0.1.16 + ```output + Downloaded asn1-rs-derive v0.4.0 + Downloaded byte-tools v0.3.1 + Downloaded const-random-macro v0.1.16 - ... - ``` + ... + ``` - Cargo is downloading and compiling a lot of tooling for the binaries. This process may take upwards of 10 minutes, depending on your system. + Cargo is downloading and compiling a lot of tooling for the binaries. This process may take upwards of 10 minutes, depending on your system. 1. Run the node binary: - ```shell - ./target/release/entropy --dev --rpc-external - ``` + ```bash + ./target/release/entropy --dev --rpc-external + ``` - ```output - 2024-06-24 18:36:10 💤 Idle (0 peers), best: #4 (0xe3da…d11b), finalized #0 (0xe938…3b8f), ⬇ 0 ⬆ 0 - 2024-06-24 18:36:12 🙌 Starting consensus session on top of parent 0xe3da43079cb427b60ca77cee0fe206b933ec9df57ece549ad46a5681ea95d11b - 2024-06-24 18:36:12 🎁 Prepared block for proposing at 5 (2 ms) [hash: 0x636c606f7d66d8c25bc64956c14b1a9c209d035279ff4f7dccd629c346d81047; parent_hash: 0xe3da…d11b; extrinsics (1): [0x7f45…6999 - ``` + ```output + 2024-06-24 18:36:10 💤 Idle (0 peers), best: #4 (0xe3da…d11b), finalized #0 (0xe938…3b8f), ⬇ 0 ⬆ 0 + 2024-06-24 18:36:12 🙌 Starting consensus session on top of parent 0xe3da43079cb427b60ca77cee0fe206b933ec9df57ece549ad46a5681ea95d11b + 2024-06-24 18:36:12 🎁 Prepared block for proposing at 5 (2 ms) [hash: 0x636c606f7d66d8c25bc64956c14b1a9c209d035279ff4f7dccd629c346d81047; parent_hash: 0xe3da…d11b; extrinsics (1): [0x7f45…6999 + ``` -1. Confirm that the local devnet is functioning by using the Rust test interface within the Entropy Core repo: +4. (Optional) Test with the Rust test interface: - ```shell - cargo run -p entropy-test-cli -- --chain-endpoint="ws://127.0.0.1:9944" status + ```bash + cargo run -p entropy-test-cli -- --chain-endpoint="ws://127.0.0.1:9944" status ``` ```output Finished `dev` profile [unoptimized + debuginfo] target(s) in 0.83s - Running `target/debug/entropy-test-cli '--chain-endpoint=ws://127.0.0.1:9944' status` + Running `target/debug/entropy-test-cli '--chain-endpoint=ws://127.0.0.1:9944' status` ... - Hash Stored by: Times used: Size in bytes: Configurable? Has auxiliary? - 0x0000…0000 5GrwvaEF5zXb26Fz9rcQpDWS57CtERHpNehXCPcNoHGKutQY 10 300498 true true + Hash Stored by: Times used: Size in bytes: Configurable? Has auxiliary? + 0x0000…0000 5GrwvaEF5zXb26Fz9rcQpDWS57CtERHpNehXCPcNoHGKutQY 10 300498 true true Success: Got status That took 182.155ms ``` - If this is the first time you are running the Rust testing interface, the `cargo` command above will take a few minutes to complete. - -1. That's it! +If this is the first time you are running the Rust testing interface, the `cargo` command above will take a few minutes to complete. ## Best Practices -It's important to regularly reset the network to maintain a clean testing environment, thoroughly document all configuration settings for reproducibility, and simulate various network conditions to ensure robustness. +It's important to regularly reset the network to maintain a clean testing environment, thoroughly document all configuration settings for reproducibility, and simulate various network conditions to ensure robustness. -Developers should strive to mirror the mainnet environment as closely as possible while still maintaining flexibility for rapid iteration. If you plan to share access to the devnet, it's essential to establish a clear protocol for managing and distributing test tokens, implement monitoring and logging systems to track network behaviour, and regularly update the devnet software to match planned mainnet upgrades. +Developers should strive to mirror the mainnet environment as closely as possible while still maintaining flexibility for rapid iteration. If you plan to share access to the devnet, it's essential to establish a clear protocol for managing and distributing test tokens, implement monitoring and logging systems to track network behaviour, and regularly update the devnet software to match planned mainnet upgrades. ## Troubleshooting @@ -254,6 +158,7 @@ Developers should strive to mirror the mainnet environment as closely as possibl **Permission denied while trying to connect to the Docker daemon socket**: you likely don't have the correct permissions and user-groups set. Verify that the Docker socket file /var/run/docker.sock has the correct permissions. It should be owned by the `root` user and have appropriate permissions for the `docker` group: + ```shell sudo chown root:docker /var/run/docker.sock sudo chmod 0660 /var/run/docker.sock From d44972bc706a703e709b86fc53c87e25eb230475 Mon Sep 17 00:00:00 2001 From: johnnymatthews <9611008+johnnymatthews@users.noreply.github.com> Date: Mon, 6 Jan 2025 12:39:39 -0400 Subject: [PATCH 3/4] Adds summary intro to Programs. --- content/concepts/programs.md | 35 ++++++++++++++++++++++------------- 1 file changed, 22 insertions(+), 13 deletions(-) diff --git a/content/concepts/programs.md b/content/concepts/programs.md index 0f169859..c72bd309 100644 --- a/content/concepts/programs.md +++ b/content/concepts/programs.md @@ -3,22 +3,31 @@ title: "Programs" lead: "The purpose of an Entropy program is to determine whether a group of nodes should generate a signature or not. Developers can create and deploy programs, but validator nodes are the only agents that will directly interact with the programs once deployed. Programs do not return any data other than a _success_ or _failed_ response." --- +## Quick summary + +1. **What are Entropy Programs**: WebAssembly (WASM) components used by signing nodes to determine whether they should generate a signature and how to generate that signature. +1. **Who uses them**: validator nodes are the only agents directly interacting with deployed programs. However, Entropy Program developers will create, test, and deploy them. End-users do not directly interact with Entropy programs. +1. **What can they do**: define which accounts can generate specific signatures and the process by which those nodes generate the signatures. Programs can contain custom hashing functions to create arbitrary signatures. +1. **What they can't do**: return any data other than success/failure responses, call external chains, access external data, or access any non-deterministic data. + +## Simple example + As a simple example, a program could be designed to check the length of a message. If the message is more than 10 characters, then the program returns `OK`, and the signing nodes create and return a valid signature to the account that submits the message. If the message is more than 10 characters, then the program fails, and no signature is created. ```mermaid flowchart LR - A[Entropy account] - B{Length > 10} - C[Signing nodes] - D[Fail] - E[Success] - - A --> | send message | B - B -- true --> E - E --> | generate signature | C - C --> | valid signature | A - - B -- false --> D + A[Entropy account] + B{Length > 10} + C[Signing nodes] + D[Fail] + E[Success] + + A --> | send message | B + B -- true --> E + E --> | generate signature | C + C --> | valid signature | A + + B -- false --> D ``` {{< callout "info" >}} @@ -78,7 +87,7 @@ The workflow is as follows: - The signing key signs the transaction and becomes the deployer key - A reference counter gets set to 0 when uploading and is used to track how many users are using a program 2. A program then gets stored in the Programs storage slot with the key being `H(bytecode + configuration_interface)`. The hash is used by a user to point to the programs they want applied to their key. Every time a program is referenced, the reference counter increments -3. Since the key is a hash, there is no editing programs (since that would change the hash) +3. Since the key is a hash, it is not possible to edit or modify programs (since that would change the hash). 4. Programs can be removed if the ref count is zero by the deploy key ## Device-proxy From e179cac2a05bb9645705ef54f1bafaf33c9e1d9e Mon Sep 17 00:00:00 2001 From: johnnymatthews <9611008+johnnymatthews@users.noreply.github.com> Date: Mon, 6 Jan 2025 13:01:55 -0400 Subject: [PATCH 4/4] Fixes mermaid diagram. --- content/concepts/programs.md | 24 ++++++++++++------------ 1 file changed, 12 insertions(+), 12 deletions(-) diff --git a/content/concepts/programs.md b/content/concepts/programs.md index c72bd309..6cb1c843 100644 --- a/content/concepts/programs.md +++ b/content/concepts/programs.md @@ -16,18 +16,18 @@ As a simple example, a program could be designed to check the length of a messag ```mermaid flowchart LR - A[Entropy account] - B{Length > 10} - C[Signing nodes] - D[Fail] - E[Success] - - A --> | send message | B - B -- true --> E - E --> | generate signature | C - C --> | valid signature | A - - B -- false --> D + A[Entropy account] + B{Length > 10} + C[Signing nodes] + D[Fail] + E[Success] + + A --> | send message | B + B -- true --> E + E --> | generate signature | C + C --> | valid signature | A + + B -- false --> D ``` {{< callout "info" >}}