Skip to content

Commit 51264c3

Browse files
nipunn1313Convex, Inc
nipunn1313
authored and
Convex, Inc
committed
Update / make terse the self-hosted docs (#34058)
Move some of the disclaimers to appropriate sub-docs GitOrigin-RevId: 33066bbf31fe8b4188c8593fda47a5b223f99dba
1 parent 25d433c commit 51264c3

File tree

5 files changed

+154
-126
lines changed

5 files changed

+154
-126
lines changed

‎BUILD.md

+94
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,94 @@
1+
# Building from source
2+
3+
## Installing dependencies
4+
5+
You will need to first install the following dependencies if you don't already
6+
have them on your machine:
7+
8+
To use the scripts set up in this repo:
9+
10+
- [`Just`](https://github.com/casey/just)
11+
- Just is used to execute scripts set up in the `Justfile`.
12+
- To install it see
13+
[Packages](https://github.com/casey/just?tab=readme-ov-file#packages), for
14+
example `cargo install just` or `brew install just`
15+
16+
To run the Convex CLI:
17+
18+
- [Node.js](https://nodejs.org/en)
19+
- Make sure you have the version specified in `.nvmrc`
20+
- We recommend installing Node.js via
21+
[nvm](https://github.com/nvm-sh/nvm#installing-and-updating).
22+
- Run `nvm use` from the root of the repo.
23+
24+
To [build the backend from source](#building-from-source):
25+
26+
- Cargo
27+
- The convex local backend is written in Rust. Cargo is the build system.
28+
- We recommend installing Cargo via [rustup](https://rustup.rs/).
29+
- The Rust nightly version specified in `rust-toolchain`
30+
- Assuming you installed Rust/Cargo with `rustup`, this will install
31+
automatically.
32+
- Rush
33+
- `npm clean-install --prefix scripts`
34+
- We manage the packages in a monorepo using [Rush](https://rushjs.io/).
35+
- Convex JavaScript dependencies
36+
- `just rush install`
37+
38+
### Building from source
39+
40+
Build and run the local backend from the source in this repo:
41+
42+
```sh
43+
just run-local-backend
44+
```
45+
46+
## Provisioning a demo app locally
47+
48+
This example will go through running the backend with the included demo project.
49+
50+
**1. Start the backend**
51+
52+
[Run the backend](#running-the-convex-backend)
53+
54+
If this fails with an error "persisted db metadata ..." you might need to erase
55+
the local database, in root directory run `rm convex_local_backend.sqlite3`.
56+
57+
**2. Develop against the backend**
58+
59+
The Convex CLI watches for changes in the application source code and pushes the
60+
code to backend.
61+
62+
To make the local backend run the included demo project, do:
63+
64+
```bash
65+
cd npm-packages/demos/tutorial
66+
npm i
67+
just convex dev
68+
```
69+
70+
The `convex` script in `Justfile` automatically adds appropriate `--url` and
71+
`--admin-key` flags to point the CLI to the local backend.
72+
73+
To run the client web application you can run the demo Vite server via:
74+
75+
```bash
76+
npm run dev:frontend
77+
```
78+
79+
Note that unlike the hosted Convex workflow, we don't want to run the
80+
`dev:backend` command since `convex dev` is already running.
81+
82+
_The following CLI commands may be useful when interacting with your backend:_
83+
84+
- `just convex data` - Lists tables in your Convex deployment
85+
- `just convex env` - Allows you to list/set/update/delete environment variables
86+
- `just convex logs` - Streams out log lines to the terminal (it includes all
87+
successful executions if `--success` is passed in)
88+
- `just convex import` - Allows you to import tables
89+
- `just convex export` - Allows you to export tables
90+
91+
If you're using both the local backend and the hosted cloud platform, make sure
92+
to run `npx convex dev` or `just convex dev` before you start testing your
93+
client. The `dev` command will take care of updating your `.env.local` file with
94+
the correct `CONVEX_URL`.

‎README.md

+2-114
Original file line numberDiff line numberDiff line change
@@ -54,125 +54,15 @@ To see how to contribute, visit [Contributing.md](./CONTRIBUTING.md).
5454
- GitHub Issues. Best for: surfacing bugs and errors you encounter while
5555
building and using the open source Convex backend
5656

57-
## Running on your own machine
58-
59-
Instead of self-hosting via docker, you can run the Convex backend locally on
60-
your own machine.
61-
6257
## Building from source
6358

64-
To get started, clone this repo:
65-
66-
```sh
67-
git clone https://github.com/get-convex/convex-backend.git
68-
cd convex-backend
69-
```
70-
71-
## Dependencies
72-
73-
You will need to first install the following dependencies if you don't already
74-
have them on your machine:
75-
76-
To use the scripts set up in this repo:
77-
78-
- [`Just`](https://github.com/casey/just)
79-
- Just is used to execute scripts set up in the `Justfile`.
80-
- To install it see
81-
[Packages](https://github.com/casey/just?tab=readme-ov-file#packages), for
82-
example `cargo install just` or `brew install just`
83-
84-
To run the Convex CLI:
85-
86-
- [Node.js](https://nodejs.org/en)
87-
- Make sure you have the version specified in `.nvmrc`
88-
- We recommend installing Node.js via
89-
[nvm](https://github.com/nvm-sh/nvm#installing-and-updating).
90-
- Run `nvm use` from the root of the repo.
91-
92-
To [build the backend from source](#building-from-source):
93-
94-
- Cargo
95-
- The convex local backend is written in Rust. Cargo is the build system.
96-
- We recommend installing Cargo via [rustup](https://rustup.rs/).
97-
- The Rust nightly version specified in `rust-toolchain`
98-
- Assuming you installed Rust/Cargo with `rustup`, this will install
99-
automatically.
100-
- Rush
101-
- `npm clean-install --prefix scripts`
102-
- We manage the packages in a monorepo using [Rush](https://rushjs.io/).
103-
- Convex JavaScript dependencies
104-
- `just rush install`
105-
106-
### Building from source
107-
108-
Build and run the local backend from the source in this repo:
109-
110-
```sh
111-
just run-local-backend
112-
```
113-
114-
Under the hood, this builds with Cargo:
115-
116-
```sh
117-
cargo run -p local_backend --bin convex-local-backend
118-
```
119-
120-
## Provisioning a demo app locally
121-
122-
This example will go through running the backend with the included demo project.
123-
124-
**1. Start the backend**
125-
126-
[Run the backend](#running-the-convex-backend)
127-
128-
If this fails with an error "persisted db metadata ..." you might need to erase
129-
the local database, in root directory run `rm convex_local_backend.sqlite3`.
130-
131-
**2. Develop against the backend**
132-
133-
The Convex CLI watches for changes in the application source code and pushes the
134-
code to backend.
135-
136-
To make the local backend run the included demo project, do:
137-
138-
```bash
139-
cd demo
140-
npm i
141-
just convex dev
142-
```
143-
144-
The `convex` script in `Justfile` automatically adds appropriate `--url` and
145-
`--admin-key` flags to point the CLI to the local backend.
146-
147-
To run the client web application you can run the demo Vite server via:
148-
149-
```bash
150-
npm run dev:frontend
151-
```
152-
153-
Note that unlike the hosted Convex workflow, we don't want to run the
154-
`dev:backend` command since `convex dev` is already running.
155-
156-
_The following CLI commands may be useful when interacting with your backend:_
157-
158-
- `just convex data` - Lists tables in your Convex deployment
159-
- `just convex env` - Allows you to list/set/update/delete environment variables
160-
- `just convex logs` - Streams out log lines to the terminal (it includes all
161-
successful executions if `--success` is passed in)
162-
- `just convex import` - Allows you to import tables
163-
- `just convex export` - Allows you to export tables
59+
See [BUILD.md](./BUILD.md).
16460

16561
## Disclaimers
16662

16763
- If you choose to self-host, we recommend following the self-hosting guide. If
168-
you are going off the beaten path, make sure to change your instance secret
64+
you are instead building from source, make sure to change your instance secret
16965
and admin key from the defaults in the repo.
170-
- Migrating to a new version of self-hosted requires carefully following the
171-
migration guide in our release notes.
172-
- If you're using both the local backend and the hosted cloud platform, make
173-
sure to run `npx convex dev` or `just convex dev` before you start testing
174-
your client. The `dev` command will take care of updating your `.env.local`
175-
file with the correct `CONVEX_URL`.
17666
- Convex is battle tested most thoroughly on Linux and Mac. On Windows, it has
17767
less experience. If you run into issues, please message us on
17868
[Discord](https://convex.dev/community) in the #open-source channel.
@@ -201,5 +91,3 @@ _The following CLI commands may be useful when interacting with your backend:_
20191
- `udf-tests/` is a collection of functions used in testing the isolate
20292
layer
20393
- `system-udfs/` contains functions used by the Convex system e.g. the CLI
204-
- `demo/` contains a demo project that showcases the basic functionality of
205-
Convex using React

‎demo/README.md

+4-6
Original file line numberDiff line numberDiff line change
@@ -1,8 +1,6 @@
1-
# Convex local backend demo
1+
# Convex tutorial
22

3-
You're just a few clicks away from having a webapp powered by Convex.
3+
You're just a few minutes away from having a chat app powered by Convex.
44

5-
Get started by running `npm install` inside of this directory. Then, run
6-
`npx convex dev --url [local url] --admin-key [key]` to get the backend running
7-
locally. Finally, you can use `npm run dev:frontend` to view the frontend. We
8-
hope you enjoy using Convex!
5+
Follow the tutorial at
6+
[docs.convex.dev/tutorial](https://docs.convex.dev/tutorial) for instructions.

‎npm-packages/demos/tutorial/README.md

+4-6
Original file line numberDiff line numberDiff line change
@@ -1,8 +1,6 @@
1-
# Convex local backend demo
1+
# Convex tutorial
22

3-
You're just a few clicks away from having a webapp powered by Convex.
3+
You're just a few minutes away from having a chat app powered by Convex.
44

5-
Get started by running `npm install` inside of this directory. Then, run
6-
`npx convex dev --url [local url] --admin-key [key]` to get the backend running
7-
locally. Finally, you can use `npm run dev:frontend` to view the frontend. We
8-
hope you enjoy using Convex!
5+
Follow the tutorial at
6+
[docs.convex.dev/tutorial](https://docs.convex.dev/tutorial) for instructions.

‎self-hosted/SELFHOSTING.md

+50
Original file line numberDiff line numberDiff line change
@@ -126,3 +126,53 @@ cd your_project
126126
npm install
127127
npx convex dev --admin-key 'flying-fox-123|01c046ab1512d9306a6abda3eedec5dfe862f1fe0f66a5aee774fb9ae3fda87706facaf682b9d4f9209a05e038cbd6e9b8' --url "http://127.0.0.1:3210"
128128
```
129+
130+
# Upgrading your self-hosted backend on a production instance.
131+
132+
In order to safely migrate to a new version of self-hosted, there are two
133+
options.
134+
135+
## Option 1: Export/Import your database (higher downtime + easy, recommended)
136+
137+
1. Take down external traffic to your backend
138+
2. Export your database with `npx convex export`
139+
3. Save your environment variables with `npx convex env list` (or via
140+
dashboard).
141+
4. Upgrade the backend docker image (or binary)
142+
5. Import from your backup with `npx convex import --replace-all`
143+
6. Bring back your environment variables with `npx convex env set` (or via
144+
dashboard)
145+
7. Bring back external traffic to your backend
146+
147+
Given that exports/imports can be expensive if you have a lot of data, this can
148+
incur downtime. You can get a sense of how much downtime safely, by running an
149+
export while your self-hosted instance is up. For smaller instances, this may be
150+
quick and easy.
151+
152+
However to safely avoid losing data, it's important that the final export is
153+
done after load is stopped from your instance, since exports are taken at a
154+
snapshot in time.
155+
156+
## Option 2: Upgrade in-place (lower downtime)
157+
158+
This is a more manual, more fiddly process, but it incurs less downtime. If you
159+
choose to go this route, please be careful, and feel free to reach out for
160+
guidance.
161+
162+
You will need to upgrade through each intermediate binary revision specified via
163+
`git log crates/model/src/migrations.rs`.
164+
165+
Each upgrade will incur a small amount of downtime, but the underlying database
166+
will be upgraded in-place while your app still functions. You need to allow the
167+
backend to run at each intermediate revision until it is ready.
168+
169+
Look for loglines like this - and follow those instructions to complete the
170+
in-place upgrade. Each migration will let you know which logline to wait for to
171+
determine that the in-place upgrade is complete.
172+
173+
```
174+
Executing Migration 114/115. MigrationComplete(115)
175+
```
176+
177+
Please feel free to reach out to us on [Discord](https://convex.dev/community)
178+
if you have any questions.

0 commit comments

Comments
 (0)