Skip to content

Latest commit

 

History

History
280 lines (210 loc) · 12.5 KB

fullnode.md

File metadata and controls

280 lines (210 loc) · 12.5 KB
title
Run a Sui Full Node with Indexer

Note: These instructions are for advanced users. If you just need a local development environment, you should instead follow the instructions in Create a Local Sui Network to create a local Full node, validators, and faucet.

Sui Full nodes validate blockchain activities, including transactions, checkpoints, and epoch changes. Each Full node stores and services the queries for the blockchain state and history.

This role enables validators to focus on servicing and processing transactions. When a validator commits a new set of transactions (or a block of transactions), the validator pushes that block to all connected Full nodes that then service the queries from clients.

The Sui Full node indexer offloads functions from the Sui Full node, processing most API endpoints to lessen the computational burden of the Full node. Without the indexer running, you might not have all available RPC functions available to optimize transaction processing on the network.

Features

Sui Full nodes with indexer:

  • Track and verify the state of the blockchain, independently and locally.
  • Serve read requests from clients.
  • Maximize Sui network throughput.
  • Offer scalable data retention and processing capabilities.

State synchronization

Sui Full nodes sync with validators to receive new transactions on the network.

A transaction requires a few round trips to 2f+1 validators to form a transaction certificate (TxCert).

This synchronization process includes:

  1. Following 2f+1 validators and listening for newly committed transactions.
  2. Making sure that 2f+1 validators recognize the transaction and that it reaches finality.
  3. Executing the transaction locally and updating the local DB.

This synchronization process requires listening to at a minimum 2f+1 validators to ensure that a Full node has properly processed all new transactions. Sui will improve the synchronization process with the introduction of checkpoints and the ability to synchronize with other Full nodes.

Architecture

A Sui Full node is essentially a read-only view of the network state. Unlike validator nodes, Full nodes cannot sign transactions, although they can validate the integrity of the chain by re-executing transactions that a quorum of validators previously committed.

Complete instances of a Sui Full node include an indexer with Postgres database integration that adds support for all JSON-RPC endpoints. The indexer processes transactions that don't require immediate validation, allowing the Full node to more effectively interact with the network.

Today, a Sui Full node maintains the full history of the chain.

Validator nodes store only the latest transactions on the frontier of the object graph (for example, transactions with >0 unspent output objects).

Full node setup

Follow the instructions here to run your own Sui Full node with indexer.

Hardware requirements

Suggested minimum hardware to run a Sui Full node:

  • CPUs: 10 core
  • RAM: 32 GB
  • Storage (SSD): 2 TB

Software requirements

Sui recommends running Sui Full nodes on Linux. Sui supports the Ubuntu and Debian distributions. You can also run a Sui Full node on macOS.

Make sure to update Rust.

Use the following command to install additional Linux dependencies.

sudo apt-get update \
&& sudo apt-get install -y --no-install-recommends \
tzdata \
libprotobuf-dev \
ca-certificates \
build-essential \
libssl-dev \
libclang-dev \
pkg-config \
openssl \
protobuf-compiler \
git \
clang \
cmake

Configure a Full node

You can configure a Sui Full node either using Docker or by building from source.

Using Docker Compose

Follow the instructions in the Full node Docker Readme to run a Sui Full node using Docker, including resetting the environment.

To get a Full node with indexer running in Docker, follow the instructions in the Docker Readme for fullnode-x.

Setting up a local Sui repository

You must get the latest source files from the Sui GitHub repository.

  1. Set up your fork of the Sui repository:
    1. Go to the Sui repository on GitHub and click the Fork button in the top right-hand corner of the screen.
    2. Clone your personal fork of the Sui repository to your local machine (ensure that you insert your GitHub username into the URL): 
      git clone https://github.com/<YOUR-GITHUB-USERNAME>/sui.git
  2. cd into your sui repository: 
    cd sui
  3. Set up the Sui repository as a git remote: 
    git remote add upstream https://github.com/MystenLabs/sui
  4. Sync your fork: 
    git fetch upstream
  5. Check out the branch associated with the network version you want to run (for example, devnet to run a Devnet Full node): 
    git checkout --track upstream/<BRANCH-NAME>

Setting up a Full node from source

Open a Terminal or Console to the sui directory you downloaded in the previous steps to complete the following:

  1. Install the required Prerequisites.
  2. Make a copy of the Full node YAML template: 
    cp crates/sui-config/data/fullnode-template.yaml fullnode.yaml
  3. Download the genesis blob for the network to use:
    • Devnet genesis blob: 
      curl -fLJO https://github.com/MystenLabs/sui-genesis/raw/main/devnet/genesis.blob
    • Testnet genesis blob - Supported only when there is an active public Testnet network. 
      curl -fLJO https://github.com/MystenLabs/sui-genesis/raw/main/testnet/genesis.blob
  4. Optional: Skip this step to accept the default paths to resources. Edit the fullnode.yaml file to use custom paths.
  • Update the db-path field with the path to the Full node database. 
    db-path: "/db-files/sui-fullnode"
  • Update the genesis-file-location with the path to genesis.blob. 
    genesis:
   genesis-file-location: "/sui-fullnode/genesis.blob"

After completing these steps, continue with setting up your indexer before starting the Full node.

Setting up a Sui indexer from source

Before setting up your Sui indexer, make sure you have the following prerequisites on your system:

With Postgres and Diesel CLI installed, follow these steps to set up the indexer:

  1. Create a Postgres database to use for the indexer.
  2. Open a Terminal or Console to the sui/crates/sui-indexer directory.
  3. Pass your database connection URL to the diesel setup command. The connection URL is in the form of postgres://<POSTGRES-USER>/<POSTGRES-USER-PASSWORD>@localhost/<DATABASE-NAME>. 
    diesel setup --database-url="<DATABASE-URL>"
  4. Run the Diesel migration with the following command: 
    diesel migration run --database-url="<DATABASE-URL>"

Refer to the sui-indexer module Readme file for additional information and troubleshooting.

After completing these steps, you're ready to start your Sui Full node and indexer.

Starting services

At this point, your Sui Full node and indexer are ready to connect to the Sui network.

  1. Open a Terminal or Console to the sui directory.
  2. Start the Sui Full node: 
    cargo run --release --bin sui-node -- --config-path fullnode.yaml
  3. Optional: Publish/subscribe to notifications using JSON-RPC via websocket.
  4. To start the Sui indexer binary, open a new Terminal or Console to the sui/crates/sui-indexer directory.
  5. Using the database connection URL you used when setting up the indexer, run the following command: 
    cargo run --bin sui-indexer -- --db-url "<DATABASE-URL>" --rpc-client-url "http://0.0.0.0:9000"

If your setup is successful, your Sui Full node is now connected to the appropriate network and your indexer is processing transactions.

Your Full node serves the read endpoints of the Sui JSON-RPC API at: http://127.0.0.1:9000.

Troubleshooting

If you receive a cannot find -lpq error, you are missing the libpq library. Use sudo apt-get install libpq-dev to install on Linux, or brew install libpq on MacOS. After you install on MacOS, create a Homebrew link using brew link --force libpq. For further context, reference the issue on Stack Overflow.

If you receive the following error:

panicked at 'error binding to 0.0.0.0:9184: error creating server listener: Address already in use (os error 98)

Then update the metrics address in your fullnode.yaml file to use port 9180.

metrics-address: "0.0.0.0:9180"

Sui Explorer with your Full node

Sui Explorer supports connections to custom RPC URLS and local networks. You can point the Explorer to your local Full node and see the transactions it syncs from the network. To make this change:

  1. Open a browser and go to: https://explorer.sui.io/
  2. Click the Devnet button in the top right-hand corner of Sui Explorer (or menu icon on smaller screens) and select Local or Testnet from the drop-down menu.
  3. Close the Choose a Network menu to see the latest transactions. If you chose the Local network, Sui Explorer now uses your local Full node to explore the state of the chain.

Monitoring

Monitor your Full node using the instructions at Logging, Tracing, Metrics, and Observability.

Note the default metrics port is 9184. To change the port, edit your fullnode.yaml file.

Update your Full node and indexer

Whenever Sui releases a new version, the network resets and restarts as a new network with no data. You must update your Full node and indexer with each Sui release to ensure compatibility with the network.

Update with Docker Compose

Follow the instructions to reset the environment, namely by running the command:

docker-compose down --volumes

Update from source

If you followed the instructions for Building from Source, update your Full node as follows:

  1. Shut down your running Full node and indexer.
  2. cd into your local Sui repository: 
    cd sui
  3. Remove the old on-disk database and 'genesis.blob' file: 
    rm -r suidb genesis.blob
  4. Fetch the source from the latest release: 
    git fetch upstream
  5. Reset your branch: 
    git checkout -B <BRANCH-NAME> --track upstream/<BRANCH-NAME>
  6. Download the latest genesis blob:
    • Devnet genesis blob: 
      curl -fLJO https://github.com/MystenLabs/sui-genesis/raw/main/devnet/genesis.blob
    • Testnet genesis blob - supported only when there is an active public Testnet network 
      curl -fLJO https://github.com/MystenLabs/sui-genesis/raw/main/testnet/genesis.blob
  7. Update your fullnode.yaml configuration file if needed.
  8. Restart your Sui Full node: 
    cargo run --release --bin sui-node -- --config-path fullnode.yaml
  9. Restart your indexer: 
    cargo run --bin sui-indexer -- --db-url "<DATABASE-URL>" --rpc-client-url "http://0.0.0.0:9000"

Your Full node starts on: http://127.0.0.1:9000.