Split out the Snapshots page as its own nav entry

- refresh the latest config schema from restate
- add Minio and S3 local dev examples

docs/operate/data-backup.mdx

@@ -1,93 +1,14 @@
 ---
-sidebar_position: 12
+sidebar_position: 13
 description: "Backing up and restoring the Restate data store on single nodes"
 ---
 
 import Admonition from '@theme/Admonition';
 
-# Snapshots & Backups
+# Data Backups
 
-## Snapshotting
-
-Restate workers can be configured to periodically publish snapshots of their partition state to a shared destination.
-Snapshots act as a form of backup and allow nodes that had not previously served a partition to bootstrap a copy of its state.
-Without snapshots, rebuilding a partition processor would require the full replay of the partition's log. Replaying the log might take a long time or even be impossible if the log was trimmed.
-
-<Admonition type="note" title="Architectural overview">
-    To understand the terminology used on this page, it might be helpful to read through the [architecture reference](/references/architecture).
-</Admonition>
-
-### Configuring Snapshots
-Restate clusters should always be configured with a snapshot repository to allow nodes to efficiently share partition state.
-Restate currently supports using Amazon S3 (or another API-compatible object store) as a shared snapshot repository.
-To set up a snapshot destination, update your [server configuration](/operate/configuration/server) as follows:
-
-```toml
-[worker.snapshots]
-destination = "s3://snapshots-bucket/cluster-prefix"
-snapshot-interval-num-records = 10000
-```
-
-This enables automated periodic snapshots to be written to the specified bucket.
-You can also trigger snapshot creation manually using the [`restatectl`](/deploy/server/cluster/deployment#controlling-clusters):
-```shell
-restatectl snapshots create-snapshot --partition-id <PARTITION_ID>
-```
-
-We recommend testing the snapshot configuration by requesting a snapshot and examining the contents of the bucket.
-You should see a new prefix with each partition's id, and a `latest.json` file pointing to the most recent snapshot.
-
-No additional configuration is required to enable restoring snapshots.
-When partition processors first start up, and no local partition state is found, the processor will attempt to restore the latest snapshot from the repository.
-This allows for efficient bootstrapping of additional partition workers.
-
-<Admonition type="tip" title="Experimenting with snapshots without an object store">
-    For testing purposes, you can also use the `file://` protocol to publish snapshots to a local directory. This is mostly useful when experimenting with multi-node configurations on a single machine. The `file` provider does not support conditional updates, which makes it unsuitable for potentially contended operation.
-</Admonition>
-
-For S3 bucket destinations, Restate will use the AWS credentials available from the environment, or the configuration profile specified by `AWS_PROFILE` environment variable, falling back to the default AWS profile.
-
-### Log trimming and Snapshots
-
-In a distributed environment, the shared log is the mechanism for replicating partition state among nodes.
-Therefore it is critical to that all cluster members can get all the relevant events recorded in the log, even newly built nodes that will join the cluster in the future.
-This requirement is at odds with an immutable log growing unboundedly. Snapshots enable log trimming - the proces of removing older segments of the log.
-
-When partition processors successfully publish a snapshot, they update their "archived" log sequence number (LSN).
-This reflects the position in the log at which the snapshot was taken and allows the cluster to safely trim its logs.
-
-By default, Restate will attempt to trim logs once an hour which you can override or disable in the server configuration:
-
-```toml
-[admin]
-log-trim-check-interval = "1h"
-```
-
-This interval determines only how frequently the check is performed, and not a guarantee that logs will be trimmed. Restate will automatically determine the appropriate safe trim point for each partition's log.
-
-If replicated logs are in use in a clustered environment, the log safe trim point will be determined based on the archived LSN.
-If a snapshot repository is not configured, then archived LSNs are not reported.
-Instead, the safe trim point will be determined by the smallest reported persisted LSN across all known processors for the given partition.
-Single-node local-only logs are also trimmed based on the partitions' persisted LSNs.
-
-The presence of any dead nodes in a cluster will cause trimming to be suspended for all partitions, unless a snapshot repository is configured.
-This is because we can not know what partitions may reside on the unreachable nodes, which will become stuck when the node comes back.
-
-When a node starts up with pre-existing partition state and finds that the partition's log has been trimmed to a point beyond the most recent locally-applied LSN, the node will attempt to download the latest snapshot from the configured repository. If a suitable snapshot is available, the processor will re-bootstrap its local state and resume applying the log.
-
-<Admonition type="note" title="Handling log trim gap errors">
-    If you observe repeated `Shutting partition processor down because it encountered a trim gap in the log.` errors in the Restate server log, it is an indication that a processor is failing to start up due to missing log records. To recover, you must ensure that a snapshot repository is correctly configured and accessible from the node reporting errors. You can still recover even if no snapshots were taken previously as long as there is at least one healthy node with a copy of the partition data. In that case, you must first configure the existing node(s) to publish snapshots for the affected partition(s) to a shared destination.
-</Admonition>
-
-### Pruning the snapshot repository
-
-<Admonition type="warning" title="Pruning">
-    Restate does not currently support pruning older snapshots from the snapshot repository. We recommend implementing an object lifecycle policy directly in the object store to manage retention.
-</Admonition>
-
-## Data Backups
 <Admonition type="note">
-    This page covers backing up individual Restate server instances. For sharing snapshots in a Restate cluster environment, see [Logs and Snapshots](/operate/data-backup#snapshotting).
+    This page covers backing up individual Restate server instances. For sharing snapshots in a Restate cluster environment, see [Snapshots](/operate/snapshots).
 </Admonition>
 
 The Restate server persists both metadata (such as the details of deployed services, in-flight invocations) and data (e.g., virtual object and workflow state keys) in its data store, which is located in its base directory (by default, the `restate-data` path relative to the startup working directory). Restate is configured to perform write-ahead logging with fsync enabled to ensure that effects are fully persisted before being acknowledged to participating services.

docs/operate/snapshots.mdx

@@ -0,0 +1,126 @@
+---
+sidebar_position: 12
+description: "Backing up and restoring the Restate data store on single nodes"
+---
+
+import Admonition from '@theme/Admonition';
+
+# Snapshots
+
+<Admonition type="note">
+    This page covers configuring a Restate cluster to share partition snapshots for fast fail-over and bootstrapping new nodes. For backup of Restate nodes, see [Data Backup](/operate/data-backup).
+</Admonition>
+
+Restate workers can be configured to periodically publish snapshots of their partition state to a shared destination. Snapshots are not necessarily backups. Rather, snapshots allow nodes that had not previously served a partition to bootstrap a copy of its state. Without snapshots, placing a partition processor on a node that wasn't previously a follower would require the full replay of that partition's log. Replaying the log might take a long time - and is impossible if the log gets trimmed.
+
+<Admonition type="note" title="Architectural overview">
+    To understand the terminology used on this page, it might be helpful to read through the [architecture reference](/references/architecture).
+</Admonition>
+
+## Configuring Snapshots
+Restate clusters should always be configured with a snapshot repository to allow nodes to efficiently share partition state, and for new nodes to be added to the cluster in the future.
+Restate currently supports using Amazon S3 (or an API-compatible object store) as a shared snapshot repository.
+To set up a snapshot destination, update your [server configuration](/operate/configuration/server) as follows:
+
+```toml
+[worker.snapshots]
+destination = "s3://snapshots-bucket/cluster-prefix"
+snapshot-interval-num-records = 10000
+```
+
+This enables automated periodic snapshots to be written to the specified bucket.
+You can also trigger snapshot creation manually using the [`restatectl`](/deploy/server/cluster/deployment#controlling-clusters):
+```shell
+restatectl snapshots create-snapshot --partition-id <PARTITION_ID>
+```
+
+We recommend testing the snapshot configuration by requesting a snapshot and examining the contents of the bucket.
+You should see a new prefix with each partition's id, and a `latest.json` file pointing to the most recent snapshot.
+
+No additional configuration is required to enable restoring snapshots.
+When partition processors first start up, and no local partition state is found, the processor will attempt to restore the latest snapshot from the repository.
+This allows for efficient bootstrapping of additional partition workers.
+
+<Admonition type="tip" title="Experimenting with snapshots without an object store">
+    For testing purposes, you can also use the `file://` protocol to publish snapshots to a local directory. This is mostly useful when experimenting with multi-node configurations on a single machine. The `file` provider does not support conditional updates, which makes it unsuitable for potentially contended operation.
+</Admonition>
+
+## Object Store endpoint and access credentials
+
+Restate supports Amazon S3 and S3-compatible object stores. In typical server deployments to AWS, the configuration will be automatically inferred. Object store locations are specified in the form of a URL where the scheme is `s3://` and the authority is the name of the _bucket_. Optionally, you may supply an additonal path within the bucket, which will be used as a common prefix for all operations. If you need to specify a custom endpoint for S3-compatible stores, you can override the API endpoint using the `aws-endpoint-url` config key.
+
+For typical server deployments in AWS, you might not need to set region or credentials at all when using Amazon S3 beyond setting the path. Restate's object store support uses the conventional [AWS SDKs and Tools](https://docs.aws.amazon.com/sdkref/latest/guide/file-format.html) credentials discovery. We strongly recommend against using long-lived credentials in configuration. For development, you can use short-term credentials provided by a profile.
+
+### Local development with Minio
+
+Minio is a common target while developing locally. You can configure it as follows:
+
+```toml
+[worker.snapshots]
+destination = "s3://bucket/cluster-name"
+snapshot-interval-num-records = 1000
+
+aws-region = "local"
+aws-access-key-id = "minioadmin"
+aws-secret-access-key = "minioadmin"
+aws-endpoint-url = "http://localhost:9000"
+aws-allow-http = true
+```
+
+### Local development with S3
+
+Assuming you have a profile set up to assume a specific role granted access to your bucket, you can work with S3 directly using a configuration like:
+
+```toml
+[worker.snapshots]
+destination = "s3://bucket/cluster-name"
+snapshot-interval-num-records = 1000
+aws-profile = "restate-dev"
+```
+
+This assumes that in your `~/.aws/config` you have a profile similar to:
+
+```
+[profile restate-dev]
+source_profile = ...
+region = us-east-1
+role_arn = arn:aws:iam::123456789012:role/restate-local-dev-role
+```
+
+## Log trimming and Snapshots
+
+In a distributed environment, the shared log is the mechanism for replicating partition state among nodes.
+Therefore it is critical to that all cluster members can get all the relevant events recorded in the log, even newly built nodes that will join the cluster in the future.
+This requirement is at odds with an immutable log growing unboundedly. Snapshots enable log trimming - the proces of removing older segments of the log.
+
+When partition processors successfully publish a snapshot, they update their "archived" log sequence number (LSN).
+This reflects the position in the log at which the snapshot was taken and allows the cluster to safely trim its logs.
+
+By default, Restate will attempt to trim logs once an hour which you can override or disable in the server configuration:
+
+```toml
+[admin]
+log-trim-check-interval = "1h"
+```
+
+This interval determines only how frequently the check is performed, and not a guarantee that logs will be trimmed. Restate will automatically determine the appropriate safe trim point for each partition's log.
+
+If replicated logs are in use in a clustered environment, the log safe trim point will be determined based on the archived LSN.
+If a snapshot repository is not configured, then archived LSNs are not reported.
+Instead, the safe trim point will be determined by the smallest reported persisted LSN across all known processors for the given partition.
+Single-node local-only logs are also trimmed based on the partitions' persisted LSNs.
+
+The presence of any dead nodes in a cluster will cause trimming to be suspended for all partitions, unless a snapshot repository is configured.
+This is because we can not know what partitions may reside on the unreachable nodes, which will become stuck when the node comes back.
+
+When a node starts up with pre-existing partition state and finds that the partition's log has been trimmed to a point beyond the most recent locally-applied LSN, the node will attempt to download the latest snapshot from the configured repository. If a suitable snapshot is available, the processor will re-bootstrap its local state and resume applying the log.
+
+<Admonition type="note" title="Handling log trim gap errors">
+    If you observe repeated `Shutting partition processor down because it encountered a trim gap in the log.` errors in the Restate server log, it is an indication that a processor is failing to start up due to missing log records. To recover, you must ensure that a snapshot repository is correctly configured and accessible from the node reporting errors. You can still recover even if no snapshots were taken previously as long as there is at least one healthy node with a copy of the partition data. In that case, you must first configure the existing node(s) to publish snapshots for the affected partition(s) to a shared destination.
+</Admonition>
+
+## Pruning the snapshot repository
+
+<Admonition type="warning" title="Pruning">
+    Restate does not currently support pruning older snapshots from the snapshot repository. We recommend implementing an object lifecycle policy directly in the object store to manage retention.
+</Admonition>

docs/operate/troubleshooting.mdx

@@ -1,5 +1,5 @@
 ---
-sidebar_position: 13
+sidebar_position: 14
 description: "Resolving common problems with Restate deployments"
 ---
 

static/schemas/restate-server-configuration-schema.json

@@ -1089,12 +1089,11 @@
     },
     "SnapshotsOptions": {
       "title": "Snapshot options.",
-      "description": "Partition store snapshotting settings.",
+      "description": "Partition store snapshotting settings. At a minimum, set `destination` and `snapshot-interval-num-records` to enable snapshotting. For a complete example, see [Snapshots](https://docs.restate.dev/operate/snapshots).",
       "type": "object",
       "properties": {
         "destination": {
-          "title": "Destination",
-          "description": "Newly produced snapshots are published to this location. Supports `s3://` and `file://` protocol scheme.\n\nFor S3 bucket destinations, Restate will use the AWS credentials available from the environment, or the configuration profile specified by `AWS_PROFILE` environment variable, falling back to the default AWS profile.\n\nExample: `s3://bucket/prefix` will put snapshots in the specified bucket and (optional) prefix.\n\nDefault: `None`",
+          "description": "Base URL for cluster snapshots. Supports `s3://` and `file://` protocol scheme.\n\nDefault: `None`",
           "default": null,
           "type": [
             "string",
@@ -1113,56 +1112,49 @@
           "minimum": 1.0
         },
         "aws-profile": {
-          "description": "The AWS configuration profile to use for S3 object store destinations.",
-          "default": null,
+          "description": "The AWS configuration profile to use for S3 object store destinations. If you use named profiles in your AWS configuration, you can replace all the other settings with a single profile reference. See the [AWS documentation on profiles] (https://docs.aws.amazon.com/sdkref/latest/guide/file-format.html) for more.",
           "type": [
             "string",
             "null"
           ]
         },
         "aws-region": {
-          "description": "AWS region to use with S3 object store destinations.",
-          "default": null,
+          "description": "AWS region to use with S3 object store destinations. This may be inferred from the environment, for example the current region when running in EC2. Because of the request signing algorightm this must have a value. For Minio, you can generally set this to any string, such as `us-east-1`.",
           "type": [
             "string",
             "null"
           ]
         },
         "aws-access-key-id": {
-          "description": "AWS access key. We strongly recommend against using long-lived credentials; set up a configuration profile with a role-based session credentials provider instead.",
-          "default": null,
+          "description": "AWS access key (or username in Minio / S3-compatible stores).",
           "type": [
             "string",
             "null"
           ]
         },
         "aws-secret-access-key": {
-          "description": "AWS secret key. We strongly recommend against using long-lived credentials; set up a configuration profile with a role-based session credentials provider instead.",
-          "default": null,
+          "description": "AWS secret key (or password in Minio / S3-compatible stores).",
           "type": [
             "string",
             "null"
           ]
         },
         "aws-session-token": {
-          "description": "AWS session token. We strongly recommend against using long-lived credentials; set up a configuration profile with a role-based session credentials provider instead.",
-          "default": null,
+          "description": "AWS session token, needed with short-term STS session credentials.",
           "type": [
             "string",
             "null"
           ]
         },
         "aws-endpoint-url": {
-          "description": "AWS endpoint URL for S3 object store destinations. Some S3-compatible stores may require you to use this.",
-          "default": null,
+          "description": "Endpoint URL override. When you use Amazon S3, this is typically inferred from the region and there is no need to set it. With other object stores, you will have to provide an appropriate HTTP(S) endpoint. If *not* using HTTPS, also set `aws-allow-http` to `true`.",
           "type": [
             "string",
             "null"
           ]
         },
         "aws-allow-http": {
-          "description": "Allow plain HTTP to be used with the object store endpoint.",
-          "default": null,
+          "description": "Allow plain HTTP to be used with the object store endpoint. Required when the endpoint URL that isn't using HTTPS.",
           "type": [
             "boolean",
             "null"