From 326b0e504903c7daadbf0ce0812e23e3d9eca123 Mon Sep 17 00:00:00 2001 From: Bastian Hofmann Date: Fri, 1 Dec 2023 16:15:10 +0100 Subject: [PATCH 01/29] Improve installation docs * Add a section about requirements for Qdrant * Separate installation instructions between recommended installations for prod and dev * Add cloud option * Add enterprise operator option * Add details about configuring the database with docker --- .../documentation/guides/configuration.md | 17 +- .../documentation/guides/installation.md | 146 ++++++++++++++---- .../content/documentation/interfaces.md | 2 +- 3 files changed, 119 insertions(+), 46 deletions(-) diff --git a/qdrant-landing/content/documentation/guides/configuration.md b/qdrant-landing/content/documentation/guides/configuration.md index fe1050913..f482c8bbc 100644 --- a/qdrant-landing/content/documentation/guides/configuration.md +++ b/qdrant-landing/content/documentation/guides/configuration.md @@ -17,22 +17,7 @@ production mode, you could also choose to overwrite `config/production.yaml`. See [ordering](#order-and-priority) for details on how configurations are loaded. -To use Qdrant in Docker and overwrite the production configuration use: - -```bash -docker run -p 6333:6333 \ - -v $(pwd)/path/to/custom_config.yaml:/qdrant/config/production.yaml \ - qdrant/qdrant -``` - -Or use your own configuration file and specify it: - -```bash -docker run -p 6333:6333 \ - -v $(pwd)/path/to/custom_config.yaml:/qdrant/config/custom_config.yaml \ - qdrant/qdrant \ - ./qdrant --config-path config/custom_config.yaml -``` +The [Installation](../installation) guide contains examples of how to set up Qdrant with a custom configuration for the different deployment methods. ## Order and priority diff --git a/qdrant-landing/content/documentation/guides/installation.md b/qdrant-landing/content/documentation/guides/installation.md index 92118fa24..f0cf3279b 100644 --- a/qdrant-landing/content/documentation/guides/installation.md +++ b/qdrant-landing/content/documentation/guides/installation.md @@ -6,21 +6,74 @@ aliases: - ../installation --- -# Installation options +## Installation requirements -## Docker +### CPU and memory -The easiest way to start using Qdrant is to run it from a ready-made Docker image. -The latest versions are always available on [DockerHub](https://hub.docker.com/r/qdrant/qdrant/tags?page=1&ordering=last_updated). +The amount of CPU and memory that Qdrant needs depends on the amount of vectors, their dimensions, the additional payload data that you store, and their indexes, as well as storage, replication and quantization configuration. + +Our [Cloud Pricing Calculator](https://cloud.qdrant.io/calculator) can give you a very rough approximation of the necessary resources to store the vectors without any payload or index data. + +### Storage + +For persistent storage, Qdrant requires a block storage device with a proper file system such es ext4. This is especially important, if you mount an external volume inside the Qdrant container or if you use a Kubernetes Persistent Volume. +Qdrant won't work on a file storage device, such as NFS, or an object storage such as S3. + +The amount of storage depends on the size of your data. + +If you offload vectors to disk, you also have to ensure that your disk storage is performant. You should at least use SSDs or NVMEs. HDDs are not recommended. + +### Networking + +Each Qdrant instance opens 3 ports: + +* `6333` - For the HTTP API, including the health and metric endpoints +* `6334` - For the gRPC API +* `6335` - For internal replication communication + +All Qdrant instances in a cluster must be able to communicate with each other over these ports. + +Each Qdrant instance must also allow incoming connections to ports `6333` and `6334` from the clients that will be using Qdrant. + +## Installation options + +Qdrant can be installed in different ways depending on your needs: + +For testing or development setups, you can run the Qdrant container or as a binary executable. -Make sure that Docker daemon is installed and running: +For production, you can use our Qdrant Cloud to run Qdrant either fully managed in our infrastructure or with Hybrid SaaS in yours. + +If you want to run Qdrant in your own infrastructure, without any cloud connection, we recommend to install Qdrant in a Kubernetes cluster with our Helm chart, or to use our Qdrant Enterprise Operator + +## Production + +### Qdrant Cloud + +The easiest way to run Qdrant for a production setup is to use the [Qdrant Cloud](https://qdrant.to/cloud), which provides fully managed Qdrant databases. Out-of-the-box you will get easy horizontal and vertical scaling, one click installation and upgrades, monitoring, logging and backup and disaster recovery. For more information, see the [Qdrant Cloud documentention](../../cloud). + +### Kubernetes + +You can use a ready-made [Helm Chart](https://helm.sh/docs/) to run Qdrant in your Kubeternetes cluster. ```bash -sudo docker info +helm repo add qdrant https://qdrant.to/helm +helm install qdrant qdrant/qdrant ``` -* If you do not see the server listed, start the Docker daemon. -* On Linux, Docker needs `sudo` privileges. To run Docker commands without `sudo` privileges, create a docker group and add your users (see [Post-installation Steps for Linux](https://docs.docker.com/engine/install/linux-postinstall/) for details). +Read further instructions at [qdrant-helm](https://github.com/qdrant/qdrant-helm/tree/main/charts/qdrant). + +### Qdrant Operator + +We provide a Qdrant Enterprise Operator for Kubernetes installations. If you are interested in using it, please [contact us](https://qdrant.to/contact-us). + +## Development + +### Docker + +The easiest way to start using Qdrant for testing or development is to run the Qdrant container image. +The latest versions are always available on [DockerHub](https://hub.docker.com/r/qdrant/qdrant/tags?page=1&ordering=last_updated). + +Make sure that [Docker](https://docs.docker.com/engine/install/), [Podman](https://podman.io/docs/installation) or the container runtime of your choice is installed and running. The following instructions use Docker. Pull the image: @@ -37,11 +90,63 @@ docker run -p 6333:6333 \ ``` With this command, you will start a Qdrant instance with the default configuration. -It will store all data in `./path/to/data` directory. +It will store all data in the `./path/to/data` directory. By default, Qdrant uses port 6333, so at [localhost:6333](http://localhost:6333) you should see the welcome message. -## From source +To change the Qdrant configuration, you can overwrite the production configuration: + +```bash +docker run -p 6333:6333 \ + -v $(pwd)/path/to/data:/qdrant/storage \ + -v $(pwd)/path/to/custom_config.yaml:/qdrant/config/production.yaml \ + qdrant/qdrant +``` + +Or use your own configuration file and specify it: + +```bash +docker run -p 6333:6333 \ + -v $(pwd)/path/to/data:/qdrant/storage \ + -v $(pwd)/path/to/custom_config.yaml:/qdrant/config/custom_config.yaml \ + qdrant/qdrant \ + ./qdrant --config-path config/custom_config.yaml +``` + +For more details see the [Configuration](../configuration) section. + +### Docker Compose + +You can also use [Docker Compose](https://docs.docker.com/compose/) to run Qdrant. + +Here is an example compose file for a single node Qdrant cluster with custom configuration: + +```yaml +services: + qdrant: + image: qdrant/qdrant:v1.6.1 + restart: always + container_name: qdrant + ports: + - 6333:6333 + - 6334:6334 + expose: + - 6333 + - 6334 + - 6335 + configs: + - source: qdrant_config + target: /qdrant/config/production.yaml + volumes: + - ./qdrant_data:/qdrant_data + +configs: + qdrant_config: + content: | + log_level: INFO +``` + +### From source Qdrant is written in Rust and can be compiled into a binary executable. This installation method can be helpful if you want to compile Qdrant for a specific processor architecture or if you do not want to use Docker for some reason. @@ -57,23 +162,6 @@ cargo build --release --bin qdrant After a successful build, the binary is available at `./target/release/qdrant`. -## Python client - -In addition to the service itself, Qdrant has a distinct python client, which has some additional features compared to [clients](https://qdrant.tech/documentation/quick_start/#clients) generated from OpenAPI directly. - -To install this client, just run the following command: - -```bash -pip install qdrant-client -``` - -## Kubernetes - -You can use a ready-made [Helm Chart](https://helm.sh/docs/) to run Qdrant in your Kubeternetes cluster. - -```bash -helm repo add qdrant https://qdrant.to/helm -helm install qdrant-release qdrant/qdrant -``` +## Client libraries -Read further instructions in [qdrant-helm](https://github.com/qdrant/qdrant-helm) repository. \ No newline at end of file +In addition to the service itself, Qdrant provides client libraries for different programming languages. For a full list and documentation, see the [Client libraries](../../interfaces/#client-libraries) section. diff --git a/qdrant-landing/content/documentation/interfaces.md b/qdrant-landing/content/documentation/interfaces.md index 83b95874c..858a56d46 100644 --- a/qdrant-landing/content/documentation/interfaces.md +++ b/qdrant-landing/content/documentation/interfaces.md @@ -13,7 +13,7 @@ or [protobuf](https://github.com/qdrant/qdrant/tree/master/lib/api/src/grpc/prot ||Client Repository|Installation|Version| |-|-|-|-| |[![python](/docs/misc/python.webp)](https://python-client.qdrant.tech/)|**[Python](https://github.com/qdrant/qdrant-client)**|`pip install qdrant-client[fastembed]`|**Latest Release**, [API Docs](https://python-client.qdrant.tech/)| -|![typescript](/docs/misc/ts.webp)|**[Typescript](https://github.com/qdrant/qdrant-js)**|`npm install @qdrant/js-client-rest`|**Latest Release**| +|![typescript](/docs/misc/ts.webp)|**[JavaScript / Typescript](https://github.com/qdrant/qdrant-js)**|`npm install @qdrant/js-client-rest`|**Latest Release**| |![rust](/docs/misc/rust.webp)|**[Rust](https://github.com/qdrant/rust-client)**|`cargo add qdrant-client`|**Latest Release**| |![golang](/docs/misc/go.webp)|**[Go](https://github.com/qdrant/go-client)**|`go get github.com/qdrant/go-client`|**Latest Release**| |![.net](/docs/misc/dotnet.webp)|**[.NET](https://github.com/qdrant/qdrant-dotnet)**|`dotnet add package Qdrant.Client`|**Latest Release**| From 96d16dd27d65a697c19f973c730bfabbfe8d8202 Mon Sep 17 00:00:00 2001 From: Bastian Hofmann Date: Fri, 5 Jan 2024 16:48:38 +0100 Subject: [PATCH 02/29] Update qdrant-landing/content/documentation/guides/installation.md Co-authored-by: Mike Jang --- qdrant-landing/content/documentation/guides/installation.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/qdrant-landing/content/documentation/guides/installation.md b/qdrant-landing/content/documentation/guides/installation.md index f0cf3279b..89a9d6baa 100644 --- a/qdrant-landing/content/documentation/guides/installation.md +++ b/qdrant-landing/content/documentation/guides/installation.md @@ -149,7 +149,7 @@ configs: ### From source Qdrant is written in Rust and can be compiled into a binary executable. -This installation method can be helpful if you want to compile Qdrant for a specific processor architecture or if you do not want to use Docker for some reason. +This installation method can be helpful if you want to compile Qdrant for a specific processor architecture or if you do not want to use Docker. Before compiling, make sure that the necessary libraries and the [rust toolchain](https://www.rust-lang.org/tools/install) are installed. The current list of required libraries can be found in the [Dockerfile](https://github.com/qdrant/qdrant/blob/master/Dockerfile). From 700c79881abd979b3fcb41589c4ac67f6caa7a90 Mon Sep 17 00:00:00 2001 From: Bastian Hofmann Date: Fri, 5 Jan 2024 16:49:10 +0100 Subject: [PATCH 03/29] Update qdrant-landing/content/documentation/guides/installation.md Co-authored-by: Mike Jang --- qdrant-landing/content/documentation/guides/installation.md | 2 ++ 1 file changed, 2 insertions(+) diff --git a/qdrant-landing/content/documentation/guides/installation.md b/qdrant-landing/content/documentation/guides/installation.md index 89a9d6baa..4f87b1e6e 100644 --- a/qdrant-landing/content/documentation/guides/installation.md +++ b/qdrant-landing/content/documentation/guides/installation.md @@ -8,6 +8,8 @@ aliases: ## Installation requirements +The following sections describe the requirements for deploying Qdrant. + ### CPU and memory The amount of CPU and memory that Qdrant needs depends on the amount of vectors, their dimensions, the additional payload data that you store, and their indexes, as well as storage, replication and quantization configuration. From 2fa55e6ffb2246dc6a494038c8721113c7d48465 Mon Sep 17 00:00:00 2001 From: Bastian Hofmann Date: Fri, 5 Jan 2024 16:49:36 +0100 Subject: [PATCH 04/29] Update qdrant-landing/content/documentation/guides/installation.md Co-authored-by: Mike Jang --- .../content/documentation/guides/installation.md | 9 ++++++++- 1 file changed, 8 insertions(+), 1 deletion(-) diff --git a/qdrant-landing/content/documentation/guides/installation.md b/qdrant-landing/content/documentation/guides/installation.md index 4f87b1e6e..4a5d28b7d 100644 --- a/qdrant-landing/content/documentation/guides/installation.md +++ b/qdrant-landing/content/documentation/guides/installation.md @@ -12,7 +12,14 @@ The following sections describe the requirements for deploying Qdrant. ### CPU and memory -The amount of CPU and memory that Qdrant needs depends on the amount of vectors, their dimensions, the additional payload data that you store, and their indexes, as well as storage, replication and quantization configuration. +The CPU and RAM that you need depends on: + +- Number of vectors +- Vector dimensions +- [Payloads](/documentation/concepts/payload/) and their indexes +- Storage +- Replication +- How you configure quantization Our [Cloud Pricing Calculator](https://cloud.qdrant.io/calculator) can give you a very rough approximation of the necessary resources to store the vectors without any payload or index data. From 607c7312eea59082176f7ba63969e887ccd798b8 Mon Sep 17 00:00:00 2001 From: Bastian Hofmann Date: Fri, 5 Jan 2024 16:49:56 +0100 Subject: [PATCH 05/29] Update qdrant-landing/content/documentation/guides/installation.md Co-authored-by: Mike Jang --- qdrant-landing/content/documentation/guides/installation.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/qdrant-landing/content/documentation/guides/installation.md b/qdrant-landing/content/documentation/guides/installation.md index 4a5d28b7d..68d740423 100644 --- a/qdrant-landing/content/documentation/guides/installation.md +++ b/qdrant-landing/content/documentation/guides/installation.md @@ -21,7 +21,7 @@ The CPU and RAM that you need depends on: - Replication - How you configure quantization -Our [Cloud Pricing Calculator](https://cloud.qdrant.io/calculator) can give you a very rough approximation of the necessary resources to store the vectors without any payload or index data. +Our [Cloud Pricing Calculator](https://cloud.qdrant.io/calculator) can help you estimate required resources without payload or index data. ### Storage From d26b61d49e24db9393b639ab65d0045784fbdb4a Mon Sep 17 00:00:00 2001 From: Bastian Hofmann Date: Fri, 5 Jan 2024 16:51:36 +0100 Subject: [PATCH 06/29] Update qdrant-landing/content/documentation/guides/installation.md Co-authored-by: Mike Jang --- qdrant-landing/content/documentation/guides/installation.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/qdrant-landing/content/documentation/guides/installation.md b/qdrant-landing/content/documentation/guides/installation.md index 68d740423..19ef1bba4 100644 --- a/qdrant-landing/content/documentation/guides/installation.md +++ b/qdrant-landing/content/documentation/guides/installation.md @@ -112,7 +112,7 @@ docker run -p 6333:6333 \ qdrant/qdrant ``` -Or use your own configuration file and specify it: +Alternatively, you can use your own `custom_config.yaml` configuration file: ```bash docker run -p 6333:6333 \ From 49ed7e0d08ff093a64456fb8e417883e132185bd Mon Sep 17 00:00:00 2001 From: Bastian Hofmann Date: Fri, 5 Jan 2024 16:51:47 +0100 Subject: [PATCH 07/29] Update qdrant-landing/content/documentation/guides/installation.md Co-authored-by: Mike Jang --- qdrant-landing/content/documentation/guides/installation.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/qdrant-landing/content/documentation/guides/installation.md b/qdrant-landing/content/documentation/guides/installation.md index 19ef1bba4..dfa3537ea 100644 --- a/qdrant-landing/content/documentation/guides/installation.md +++ b/qdrant-landing/content/documentation/guides/installation.md @@ -122,7 +122,7 @@ docker run -p 6333:6333 \ ./qdrant --config-path config/custom_config.yaml ``` -For more details see the [Configuration](../configuration) section. +For more information, see the [Configuration](/documentation/guides/configuration/) documentation. ### Docker Compose From 041e34553074b69843b2cfaa5fbd5c5a2abc3e15 Mon Sep 17 00:00:00 2001 From: Bastian Hofmann Date: Fri, 5 Jan 2024 16:52:04 +0100 Subject: [PATCH 08/29] Update qdrant-landing/content/documentation/guides/installation.md Co-authored-by: Mike Jang --- qdrant-landing/content/documentation/guides/installation.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/qdrant-landing/content/documentation/guides/installation.md b/qdrant-landing/content/documentation/guides/installation.md index dfa3537ea..9954a54b4 100644 --- a/qdrant-landing/content/documentation/guides/installation.md +++ b/qdrant-landing/content/documentation/guides/installation.md @@ -128,7 +128,7 @@ For more information, see the [Configuration](/documentation/guides/configuratio You can also use [Docker Compose](https://docs.docker.com/compose/) to run Qdrant. -Here is an example compose file for a single node Qdrant cluster with custom configuration: +Here is an example customized compose file for a single node Qdrant cluster: ```yaml services: From 6bdeb8bf20e4d84064bab1685facabbc3f61ec45 Mon Sep 17 00:00:00 2001 From: Bastian Hofmann Date: Fri, 5 Jan 2024 16:58:57 +0100 Subject: [PATCH 09/29] Update qdrant-landing/content/documentation/guides/installation.md Co-authored-by: Mike Jang --- qdrant-landing/content/documentation/guides/installation.md | 2 -- 1 file changed, 2 deletions(-) diff --git a/qdrant-landing/content/documentation/guides/installation.md b/qdrant-landing/content/documentation/guides/installation.md index 9954a54b4..4cd50cd72 100644 --- a/qdrant-landing/content/documentation/guides/installation.md +++ b/qdrant-landing/content/documentation/guides/installation.md @@ -28,8 +28,6 @@ Our [Cloud Pricing Calculator](https://cloud.qdrant.io/calculator) can help you For persistent storage, Qdrant requires a block storage device with a proper file system such es ext4. This is especially important, if you mount an external volume inside the Qdrant container or if you use a Kubernetes Persistent Volume. Qdrant won't work on a file storage device, such as NFS, or an object storage such as S3. -The amount of storage depends on the size of your data. - If you offload vectors to disk, you also have to ensure that your disk storage is performant. You should at least use SSDs or NVMEs. HDDs are not recommended. ### Networking From 89a79fafc9cad39f44547200df45ba82fe579d73 Mon Sep 17 00:00:00 2001 From: Bastian Hofmann Date: Fri, 5 Jan 2024 16:59:21 +0100 Subject: [PATCH 10/29] Update qdrant-landing/content/documentation/guides/installation.md Co-authored-by: Mike Jang --- qdrant-landing/content/documentation/guides/installation.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/qdrant-landing/content/documentation/guides/installation.md b/qdrant-landing/content/documentation/guides/installation.md index 4cd50cd72..0dbae7b48 100644 --- a/qdrant-landing/content/documentation/guides/installation.md +++ b/qdrant-landing/content/documentation/guides/installation.md @@ -28,7 +28,7 @@ Our [Cloud Pricing Calculator](https://cloud.qdrant.io/calculator) can help you For persistent storage, Qdrant requires a block storage device with a proper file system such es ext4. This is especially important, if you mount an external volume inside the Qdrant container or if you use a Kubernetes Persistent Volume. Qdrant won't work on a file storage device, such as NFS, or an object storage such as S3. -If you offload vectors to disk, you also have to ensure that your disk storage is performant. You should at least use SSDs or NVMEs. HDDs are not recommended. +If you offload vectors to a local disk, we recommend you use a solid-state (SSD or NVMe) drive. ### Networking From ba82f3965a06fcbbfb645c1b15f8bb311c9569c3 Mon Sep 17 00:00:00 2001 From: Bastian Hofmann Date: Fri, 5 Jan 2024 17:00:05 +0100 Subject: [PATCH 11/29] Update qdrant-landing/content/documentation/guides/installation.md Co-authored-by: Mike Jang --- qdrant-landing/content/documentation/guides/installation.md | 5 +++-- 1 file changed, 3 insertions(+), 2 deletions(-) diff --git a/qdrant-landing/content/documentation/guides/installation.md b/qdrant-landing/content/documentation/guides/installation.md index 0dbae7b48..af5620afb 100644 --- a/qdrant-landing/content/documentation/guides/installation.md +++ b/qdrant-landing/content/documentation/guides/installation.md @@ -38,9 +38,10 @@ Each Qdrant instance opens 3 ports: * `6334` - For the gRPC API * `6335` - For internal replication communication -All Qdrant instances in a cluster must be able to communicate with each other over these ports. +All Qdrant instances in a cluster must be able to: -Each Qdrant instance must also allow incoming connections to ports `6333` and `6334` from the clients that will be using Qdrant. +- Communicate with each other over these ports +- Allow incoming connections to ports `6333` and `6334` from clients that use Qdrant. ## Installation options From 28929226de0b9fa1b2215b5001b5192747ee7d23 Mon Sep 17 00:00:00 2001 From: Bastian Hofmann Date: Fri, 5 Jan 2024 17:00:19 +0100 Subject: [PATCH 12/29] Update qdrant-landing/content/documentation/guides/installation.md Co-authored-by: Mike Jang --- qdrant-landing/content/documentation/guides/installation.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/qdrant-landing/content/documentation/guides/installation.md b/qdrant-landing/content/documentation/guides/installation.md index af5620afb..f49b87901 100644 --- a/qdrant-landing/content/documentation/guides/installation.md +++ b/qdrant-landing/content/documentation/guides/installation.md @@ -98,7 +98,7 @@ docker run -p 6333:6333 \ ``` With this command, you will start a Qdrant instance with the default configuration. -It will store all data in the `./path/to/data` directory. +It stores all data in the `./path/to/data` directory. By default, Qdrant uses port 6333, so at [localhost:6333](http://localhost:6333) you should see the welcome message. From 93998fc902c13700061c2692a53f3ab1aabd0310 Mon Sep 17 00:00:00 2001 From: Bastian Hofmann Date: Fri, 5 Jan 2024 17:00:33 +0100 Subject: [PATCH 13/29] Update qdrant-landing/content/documentation/guides/installation.md Co-authored-by: Mike Jang --- qdrant-landing/content/documentation/guides/installation.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/qdrant-landing/content/documentation/guides/installation.md b/qdrant-landing/content/documentation/guides/installation.md index f49b87901..bffcfff2f 100644 --- a/qdrant-landing/content/documentation/guides/installation.md +++ b/qdrant-landing/content/documentation/guides/installation.md @@ -72,7 +72,7 @@ Read further instructions at [qdrant-helm](https://github.com/qdrant/qdrant-helm ### Qdrant Operator -We provide a Qdrant Enterprise Operator for Kubernetes installations. If you are interested in using it, please [contact us](https://qdrant.to/contact-us). +We provide a Qdrant Enterprise Operator for Kubernetes installations. For more information, [use this form](https://qdrant.to/contact-us) to contact us. ## Development From 0c6964eefc84424d87b322ad9df00391953260e6 Mon Sep 17 00:00:00 2001 From: Bastian Hofmann Date: Fri, 5 Jan 2024 17:00:50 +0100 Subject: [PATCH 14/29] Update qdrant-landing/content/documentation/guides/installation.md Co-authored-by: Mike Jang --- qdrant-landing/content/documentation/guides/installation.md | 2 ++ 1 file changed, 2 insertions(+) diff --git a/qdrant-landing/content/documentation/guides/installation.md b/qdrant-landing/content/documentation/guides/installation.md index bffcfff2f..9d0931ef8 100644 --- a/qdrant-landing/content/documentation/guides/installation.md +++ b/qdrant-landing/content/documentation/guides/installation.md @@ -10,6 +10,8 @@ aliases: The following sections describe the requirements for deploying Qdrant. +The following sections describe the requirements for deploying Qdrant. + ### CPU and memory The CPU and RAM that you need depends on: From 26d6b35296e7a5e25d6b8bd9c8b516835628dce0 Mon Sep 17 00:00:00 2001 From: Bastian Hofmann Date: Fri, 5 Jan 2024 17:01:58 +0100 Subject: [PATCH 15/29] Update qdrant-landing/content/documentation/guides/installation.md Co-authored-by: Mike Jang --- qdrant-landing/content/documentation/guides/installation.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/qdrant-landing/content/documentation/guides/installation.md b/qdrant-landing/content/documentation/guides/installation.md index 9d0931ef8..6a4e76c9a 100644 --- a/qdrant-landing/content/documentation/guides/installation.md +++ b/qdrant-landing/content/documentation/guides/installation.md @@ -34,7 +34,7 @@ If you offload vectors to a local disk, we recommend you use a solid-state (SSD ### Networking -Each Qdrant instance opens 3 ports: +Each Qdrant instance requires three open ports: * `6333` - For the HTTP API, including the health and metric endpoints * `6334` - For the gRPC API From c90dd427fde2f1198f85be045789a9991d96e170 Mon Sep 17 00:00:00 2001 From: Bastian Hofmann Date: Fri, 5 Jan 2024 17:02:27 +0100 Subject: [PATCH 16/29] Update qdrant-landing/content/documentation/guides/installation.md Co-authored-by: Mike Jang --- qdrant-landing/content/documentation/guides/installation.md | 6 +++--- 1 file changed, 3 insertions(+), 3 deletions(-) diff --git a/qdrant-landing/content/documentation/guides/installation.md b/qdrant-landing/content/documentation/guides/installation.md index 6a4e76c9a..50c17dbd5 100644 --- a/qdrant-landing/content/documentation/guides/installation.md +++ b/qdrant-landing/content/documentation/guides/installation.md @@ -36,9 +36,9 @@ If you offload vectors to a local disk, we recommend you use a solid-state (SSD Each Qdrant instance requires three open ports: -* `6333` - For the HTTP API, including the health and metric endpoints -* `6334` - For the gRPC API -* `6335` - For internal replication communication +* `6333` - For the HTTP API, for the [Monitoring](/documentation/guides/monitoring/) health and metrics endpoints +* `6334` - For the [gRPC](documentation/interfaces/#grpc-interface) API +* `6335` - For [Distributed deployment](/documentation/guides/distributed_deployment/) All Qdrant instances in a cluster must be able to: From 28c444014f13d178c91a75a314b947b350c6beaa Mon Sep 17 00:00:00 2001 From: Bastian Hofmann Date: Fri, 5 Jan 2024 17:02:55 +0100 Subject: [PATCH 17/29] Update qdrant-landing/content/documentation/guides/installation.md Co-authored-by: Mike Jang --- qdrant-landing/content/documentation/guides/installation.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/qdrant-landing/content/documentation/guides/installation.md b/qdrant-landing/content/documentation/guides/installation.md index 50c17dbd5..aaf0ba53e 100644 --- a/qdrant-landing/content/documentation/guides/installation.md +++ b/qdrant-landing/content/documentation/guides/installation.md @@ -70,7 +70,7 @@ helm repo add qdrant https://qdrant.to/helm helm install qdrant qdrant/qdrant ``` -Read further instructions at [qdrant-helm](https://github.com/qdrant/qdrant-helm/tree/main/charts/qdrant). +For more information, see the [qdrant-helm](https://github.com/qdrant/qdrant-helm/tree/main/charts/qdrant) README. ### Qdrant Operator From 6cac7529f3bea5f26b2771a288b9daa109693354 Mon Sep 17 00:00:00 2001 From: Bastian Hofmann Date: Fri, 5 Jan 2024 17:03:29 +0100 Subject: [PATCH 18/29] Update qdrant-landing/content/documentation/guides/installation.md Co-authored-by: Mike Jang --- qdrant-landing/content/documentation/guides/installation.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/qdrant-landing/content/documentation/guides/installation.md b/qdrant-landing/content/documentation/guides/installation.md index aaf0ba53e..1ae2133b9 100644 --- a/qdrant-landing/content/documentation/guides/installation.md +++ b/qdrant-landing/content/documentation/guides/installation.md @@ -99,7 +99,7 @@ docker run -p 6333:6333 \ qdrant/qdrant ``` -With this command, you will start a Qdrant instance with the default configuration. +With this command, you start a Qdrant instance with the default configuration. It stores all data in the `./path/to/data` directory. By default, Qdrant uses port 6333, so at [localhost:6333](http://localhost:6333) you should see the welcome message. From 906a62f594e6975e772ac0d9a4de5feee26c199b Mon Sep 17 00:00:00 2001 From: Bastian Hofmann Date: Fri, 5 Jan 2024 17:04:09 +0100 Subject: [PATCH 19/29] Update qdrant-landing/content/documentation/guides/installation.md Co-authored-by: Mike Jang --- qdrant-landing/content/documentation/guides/installation.md | 3 ++- 1 file changed, 2 insertions(+), 1 deletion(-) diff --git a/qdrant-landing/content/documentation/guides/installation.md b/qdrant-landing/content/documentation/guides/installation.md index 1ae2133b9..f04777cf6 100644 --- a/qdrant-landing/content/documentation/guides/installation.md +++ b/qdrant-landing/content/documentation/guides/installation.md @@ -59,7 +59,8 @@ If you want to run Qdrant in your own infrastructure, without any cloud connecti ### Qdrant Cloud -The easiest way to run Qdrant for a production setup is to use the [Qdrant Cloud](https://qdrant.to/cloud), which provides fully managed Qdrant databases. Out-of-the-box you will get easy horizontal and vertical scaling, one click installation and upgrades, monitoring, logging and backup and disaster recovery. For more information, see the [Qdrant Cloud documentention](../../cloud). +You can set up production with the [Qdrant Cloud](https://qdrant.to/cloud), which provides fully managed Qdrant databases. +It provides horizontal and vertical scaling, one click installation and upgrades, monitoring, logging, as well as backup and disaster recovery. For more information, see the [Qdrant Cloud documentation](/documentation/cloud). ### Kubernetes From f3b02950f17b6255bc43011cef16093a3725dbaf Mon Sep 17 00:00:00 2001 From: Bastian Hofmann Date: Fri, 5 Jan 2024 17:06:10 +0100 Subject: [PATCH 20/29] Update qdrant-landing/content/documentation/guides/installation.md Co-authored-by: Mike Jang --- qdrant-landing/content/documentation/guides/installation.md | 2 ++ 1 file changed, 2 insertions(+) diff --git a/qdrant-landing/content/documentation/guides/installation.md b/qdrant-landing/content/documentation/guides/installation.md index f04777cf6..b3c2844a6 100644 --- a/qdrant-landing/content/documentation/guides/installation.md +++ b/qdrant-landing/content/documentation/guides/installation.md @@ -79,6 +79,8 @@ We provide a Qdrant Enterprise Operator for Kubernetes installations. For more i ## Development +For development and testing, we recommend that you set up Qdrant in Docker. We also have different client libraries. + ### Docker The easiest way to start using Qdrant for testing or development is to run the Qdrant container image. From 195db33bfd11a869fd472f403e9600fe9eee269f Mon Sep 17 00:00:00 2001 From: Bastian Hofmann Date: Fri, 5 Jan 2024 17:06:25 +0100 Subject: [PATCH 21/29] Update qdrant-landing/content/documentation/guides/installation.md Co-authored-by: Mike Jang --- qdrant-landing/content/documentation/guides/installation.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/qdrant-landing/content/documentation/guides/installation.md b/qdrant-landing/content/documentation/guides/installation.md index b3c2844a6..548901bb0 100644 --- a/qdrant-landing/content/documentation/guides/installation.md +++ b/qdrant-landing/content/documentation/guides/installation.md @@ -64,7 +64,7 @@ It provides horizontal and vertical scaling, one click installation and upgrades ### Kubernetes -You can use a ready-made [Helm Chart](https://helm.sh/docs/) to run Qdrant in your Kubeternetes cluster. +You can use a ready-made [Helm Chart](https://helm.sh/docs/) to run Qdrant in your Kubernetes cluster: ```bash helm repo add qdrant https://qdrant.to/helm From b72f88bef94b305408e2f997b55e742c05db9801 Mon Sep 17 00:00:00 2001 From: Bastian Hofmann Date: Fri, 5 Jan 2024 17:06:42 +0100 Subject: [PATCH 22/29] Update qdrant-landing/content/documentation/guides/installation.md Co-authored-by: Mike Jang --- qdrant-landing/content/documentation/guides/installation.md | 2 ++ 1 file changed, 2 insertions(+) diff --git a/qdrant-landing/content/documentation/guides/installation.md b/qdrant-landing/content/documentation/guides/installation.md index 548901bb0..110b7942d 100644 --- a/qdrant-landing/content/documentation/guides/installation.md +++ b/qdrant-landing/content/documentation/guides/installation.md @@ -57,6 +57,8 @@ If you want to run Qdrant in your own infrastructure, without any cloud connecti ## Production +For production, we recommend that you configure Qdrant in the cloud, with Kubernetes, or with a Qdrant Enterprise Operator. + ### Qdrant Cloud You can set up production with the [Qdrant Cloud](https://qdrant.to/cloud), which provides fully managed Qdrant databases. From 27d18b246edc0164fcb7fdf9d9161af4e5bf7191 Mon Sep 17 00:00:00 2001 From: Bastian Hofmann Date: Mon, 8 Jan 2024 09:52:19 +0100 Subject: [PATCH 23/29] Update qdrant-landing/content/documentation/guides/installation.md Co-authored-by: Mike Jang --- qdrant-landing/content/documentation/guides/installation.md | 2 -- 1 file changed, 2 deletions(-) diff --git a/qdrant-landing/content/documentation/guides/installation.md b/qdrant-landing/content/documentation/guides/installation.md index 110b7942d..1ab2c602a 100644 --- a/qdrant-landing/content/documentation/guides/installation.md +++ b/qdrant-landing/content/documentation/guides/installation.md @@ -10,8 +10,6 @@ aliases: The following sections describe the requirements for deploying Qdrant. -The following sections describe the requirements for deploying Qdrant. - ### CPU and memory The CPU and RAM that you need depends on: From 0667558629ffffa9b945174bc231858104871d8b Mon Sep 17 00:00:00 2001 From: Bastian Hofmann Date: Mon, 8 Jan 2024 09:52:44 +0100 Subject: [PATCH 24/29] Update qdrant-landing/content/documentation/guides/installation.md Co-authored-by: Mike Jang --- qdrant-landing/content/documentation/guides/installation.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/qdrant-landing/content/documentation/guides/installation.md b/qdrant-landing/content/documentation/guides/installation.md index 1ab2c602a..468c656e6 100644 --- a/qdrant-landing/content/documentation/guides/installation.md +++ b/qdrant-landing/content/documentation/guides/installation.md @@ -177,4 +177,4 @@ After a successful build, the binary is available at `./target/release/qdrant`. ## Client libraries -In addition to the service itself, Qdrant provides client libraries for different programming languages. For a full list and documentation, see the [Client libraries](../../interfaces/#client-libraries) section. +In addition to the service, Qdrant provides a variety of client libraries for different programming languages. For a full list, see our [Client libraries](../../interfaces/#client-libraries) documentation. From ba4c7114bd3125f5d6d3bc543fdce50c9bcdf2a4 Mon Sep 17 00:00:00 2001 From: Bastian Hofmann Date: Mon, 8 Jan 2024 09:53:31 +0100 Subject: [PATCH 25/29] Update qdrant-landing/content/documentation/guides/installation.md Co-authored-by: Mike Jang --- qdrant-landing/content/documentation/guides/installation.md | 4 ++-- 1 file changed, 2 insertions(+), 2 deletions(-) diff --git a/qdrant-landing/content/documentation/guides/installation.md b/qdrant-landing/content/documentation/guides/installation.md index 468c656e6..86e422640 100644 --- a/qdrant-landing/content/documentation/guides/installation.md +++ b/qdrant-landing/content/documentation/guides/installation.md @@ -25,8 +25,8 @@ Our [Cloud Pricing Calculator](https://cloud.qdrant.io/calculator) can help you ### Storage -For persistent storage, Qdrant requires a block storage device with a proper file system such es ext4. This is especially important, if you mount an external volume inside the Qdrant container or if you use a Kubernetes Persistent Volume. -Qdrant won't work on a file storage device, such as NFS, or an object storage such as S3. +For persistent storage, Qdrant requires block-level access to storage devices with a [POSIX-compatible file system](https://www.quobyte.com/storage-explained/posix-filesystem/). Network systems such as [iSCSI](https://en.wikipedia.org/wiki/ISCSI) that provide block-level access are also acceptable. +Qdrant won't work with [Network file systems](https://en.wikipedia.org/wiki/File_system#Network_file_systems) such such as NFS, or [Object storage](https://en.wikipedia.org/wiki/Object_storage) systems such as S3. If you offload vectors to a local disk, we recommend you use a solid-state (SSD or NVMe) drive. From 82a117e68aecf97d24d65e3d6774bfcb184ce266 Mon Sep 17 00:00:00 2001 From: Bastian Hofmann Date: Mon, 8 Jan 2024 09:59:19 +0100 Subject: [PATCH 26/29] Update qdrant-landing/content/documentation/guides/installation.md Co-authored-by: Mike Jang --- qdrant-landing/content/documentation/guides/installation.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/qdrant-landing/content/documentation/guides/installation.md b/qdrant-landing/content/documentation/guides/installation.md index 86e422640..a497e8dbd 100644 --- a/qdrant-landing/content/documentation/guides/installation.md +++ b/qdrant-landing/content/documentation/guides/installation.md @@ -173,7 +173,7 @@ Build Qdrant with Cargo: cargo build --release --bin qdrant ``` -After a successful build, the binary is available at `./target/release/qdrant`. +After a successful build, you can find the binary in the following subdirectory `./target/release/qdrant`. ## Client libraries From 3dacdb248fe7a09eac61a3317f98c4a4a52fee87 Mon Sep 17 00:00:00 2001 From: Bastian Hofmann Date: Mon, 8 Jan 2024 10:39:57 +0100 Subject: [PATCH 27/29] Switch installation options to match document structure. --- qdrant-landing/content/documentation/guides/installation.md | 4 ++-- 1 file changed, 2 insertions(+), 2 deletions(-) diff --git a/qdrant-landing/content/documentation/guides/installation.md b/qdrant-landing/content/documentation/guides/installation.md index a497e8dbd..49375bef2 100644 --- a/qdrant-landing/content/documentation/guides/installation.md +++ b/qdrant-landing/content/documentation/guides/installation.md @@ -47,10 +47,10 @@ All Qdrant instances in a cluster must be able to: Qdrant can be installed in different ways depending on your needs: -For testing or development setups, you can run the Qdrant container or as a binary executable. - For production, you can use our Qdrant Cloud to run Qdrant either fully managed in our infrastructure or with Hybrid SaaS in yours. +For testing or development setups, you can run the Qdrant container or as a binary executable. + If you want to run Qdrant in your own infrastructure, without any cloud connection, we recommend to install Qdrant in a Kubernetes cluster with our Helm chart, or to use our Qdrant Enterprise Operator ## Production From a3a2725cccdc2072b10362dac28c0d55ba16d078 Mon Sep 17 00:00:00 2001 From: Bastian Hofmann Date: Mon, 8 Jan 2024 10:52:27 +0100 Subject: [PATCH 28/29] Add notes on using Docker in production --- .../documentation/guides/installation.md | 18 ++++++++++++++++-- 1 file changed, 16 insertions(+), 2 deletions(-) diff --git a/qdrant-landing/content/documentation/guides/installation.md b/qdrant-landing/content/documentation/guides/installation.md index 49375bef2..d18f118f5 100644 --- a/qdrant-landing/content/documentation/guides/installation.md +++ b/qdrant-landing/content/documentation/guides/installation.md @@ -26,7 +26,7 @@ Our [Cloud Pricing Calculator](https://cloud.qdrant.io/calculator) can help you ### Storage For persistent storage, Qdrant requires block-level access to storage devices with a [POSIX-compatible file system](https://www.quobyte.com/storage-explained/posix-filesystem/). Network systems such as [iSCSI](https://en.wikipedia.org/wiki/ISCSI) that provide block-level access are also acceptable. -Qdrant won't work with [Network file systems](https://en.wikipedia.org/wiki/File_system#Network_file_systems) such such as NFS, or [Object storage](https://en.wikipedia.org/wiki/Object_storage) systems such as S3. +Qdrant won't work with [Network file systems](https://en.wikipedia.org/wiki/File_system#Network_file_systems) such as NFS, or [Object storage](https://en.wikipedia.org/wiki/Object_storage) systems such as S3. If you offload vectors to a local disk, we recommend you use a solid-state (SSD or NVMe) drive. @@ -73,10 +73,24 @@ helm install qdrant qdrant/qdrant For more information, see the [qdrant-helm](https://github.com/qdrant/qdrant-helm/tree/main/charts/qdrant) README. -### Qdrant Operator +### Qdrant Kubernetes Operator We provide a Qdrant Enterprise Operator for Kubernetes installations. For more information, [use this form](https://qdrant.to/contact-us) to contact us. +### Docker and Docker Compose + +Usually, we recommend to run Qdrant in Kubernetes, or use the Qdrant Cloud for production setups. This makes setting up highly available and scalable Qdrant clusters with backups and disaster recovery a lot easier. + +However, you can also use Docker and Docker Compose to run Qdrant in production, by following the setup instructions in the [Docker](#docker) and [Docker Compose](#docker-compose) Development sections. +In addition, you have to make sure: + +* To use a performant [persistent storage](#storage) for your data +* To configure the [security settings](/documentation/guides/security/) for your deployment +* To set up and configure Qdrant on multiple nodes for a highly available [distributed deployment](/documentation/guides/distributed_deployment/) +* To set up a load balancer for your Qdrant cluster +* To create a [backup and disaster recovery strategy](/documentation/concepts/snapshots/) for your data +* To integrate Qdrant with your [monitoring](/documentation/guides/monitoring/) and logging solutions + ## Development For development and testing, we recommend that you set up Qdrant in Docker. We also have different client libraries. From 751b3332d01a533b479d4df8ef13f6866e8b30c5 Mon Sep 17 00:00:00 2001 From: mjang Date: Wed, 10 Jan 2024 10:01:02 -0800 Subject: [PATCH 29/29] Make sure users modify the code sample as needed --- qdrant-landing/content/documentation/guides/installation.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/qdrant-landing/content/documentation/guides/installation.md b/qdrant-landing/content/documentation/guides/installation.md index d18f118f5..8338f221c 100644 --- a/qdrant-landing/content/documentation/guides/installation.md +++ b/qdrant-landing/content/documentation/guides/installation.md @@ -108,7 +108,7 @@ Pull the image: docker pull qdrant/qdrant ``` -Run the container: +In the following command, revise `$(pwd)/path/to/data` for your Docker configuration. Then use the updated command to run the container: ```bash docker run -p 6333:6333 \