Skip to content

Commit 183c79d

Browse files
soyeric128soyeric128
authored
docs: iceberg catalog updates (#1688)
1 parent 6302247 commit 183c79d

File tree

5 files changed

+23
-70
lines changed

5 files changed

+23
-70
lines changed

‎docs/en/guides/51-access-data-lake/02-iceberg/iceberg-catalog.md renamed to ‎docs/en/guides/51-access-data-lake/02-iceberg.md

Lines changed: 21 additions & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -1,5 +1,5 @@
11
---
2-
title: Iceberg Catalog
2+
title: Apache Iceberg
33
---
44
import FunctionDescription from '@site/src/components/FunctionDescription';
55

@@ -63,7 +63,26 @@ CONNECTION=(
6363
| `TYPE` (inside `CONNECTION`) | Yes | The connection type. For Iceberg, it is typically set to `rest` for REST-based connection. |
6464
| `ADDRESS` | Yes | The address or URL of the Iceberg service (e.g., `http://127.0.0.1:8181`). |
6565
| `WAREHOUSE` | Yes | The location of the Iceberg warehouse, usually an S3 bucket or compatible object storage system. |
66-
| `<connection_parameter>` | Yes | Connection parameters to establish connections with external storage. The required parameters vary based on the specific storage service and authentication methods. Refer to [Connection Parameters](/sql/sql-reference/connect-parameters) for detailed information. If you're using Amazon S3 or S3-compatible storage systems, make sure to prefix the parameters with `s3.` (e.g., `s3.region`, `s3.endpoint`). |
66+
| `<connection_parameter>` | Yes | Connection parameters to establish connections with external storage. The required parameters vary based on the specific storage service and authentication methods. See the table below for a full list of the available parameters. |
67+
68+
| Connection Parameter | Description |
69+
|-----------------------------------|----------------------------------------------------------------------------------------------------------------------------------------|
70+
| `s3.endpoint` | S3 endpoint. |
71+
| `s3.access-key-id` | S3 access key ID. |
72+
| `s3.secret-access-key` | S3 secret access key. |
73+
| `s3.session-token` | S3 session token, required when using temporary credentials. |
74+
| `s3.region` | S3 region. |
75+
| `client.region` | Region to use for the S3 client, takes precedence over `s3.region`. |
76+
| `s3.path-style-access` | S3 Path Style Access. |
77+
| `s3.sse.type` | S3 Server-Side Encryption (SSE) type. |
78+
| `s3.sse.key` | S3 SSE key. If encryption type is `kms`, this is a KMS Key ID. If encryption type is `custom`, this is a base-64 AES256 symmetric key. |
79+
| `s3.sse.md5` | S3 SSE MD5 checksum. |
80+
| `client.assume-role.arn` | ARN of the IAM role to assume instead of using the default credential chain. |
81+
| `client.assume-role.external-id` | Optional external ID used to assume an IAM role. |
82+
| `client.assume-role.session-name` | Optional session name used to assume an IAM role. |
83+
| `s3.allow-anonymous` | Option to allow anonymous access (e.g., for public buckets/folders). |
84+
| `s3.disable-ec2-metadata` | Option to disable loading credentials from EC2 metadata (typically used with `s3.allow-anonymous`). |
85+
| `s3.disable-config-load` | Option to disable loading configuration from config files and environment variables. |
6786

6887
:::note
6988
To read data from HDFS, you need to set the following environment variables before starting Databend. These environment variables ensure that Databend can access the necessary Java and Hadoop dependencies to interact with HDFS effectively. Make sure to replace "/path/to/java" and "/path/to/hadoop" with the actual paths to your Java and Hadoop installations, and adjust the CLASSPATH to include all the required Hadoop JAR files.

‎docs/en/guides/51-access-data-lake/02-iceberg/_category_.json

Lines changed: 0 additions & 3 deletions
This file was deleted.

‎docs/en/guides/51-access-data-lake/02-iceberg/iceberg-engine.md

Lines changed: 0 additions & 57 deletions
This file was deleted.

‎docs/en/guides/51-access-data-lake/index.md

Lines changed: 2 additions & 3 deletions
Original file line numberDiff line numberDiff line change
@@ -4,9 +4,8 @@ title: Accessing Data Lake
44

55
Databend presents a seamless integration with three robust Data Lake technologies—[Apache Hive](https://hive.apache.org/), [Apache Iceberg](https://iceberg.apache.org/), and [Delta Lake](https://delta.io/). This integration brings a distinct advantage by supporting multiple facets of Data Lake functionality. Databend offers a versatile and comprehensive platform, empowering users with increased flexibility and efficiency in handling diverse datasets within the Data Lake environment.
66

7-
Furthermore, the integration of these three technologies within Databend is characterized by varying approaches. While some, like Apache Hive, integrate at the catalog level, others, such as Delta Lake, operate at the table engine level. Apache Iceberg supports integration at both levels. The catalog-based integration establishes a centralized connection to the Data Lake, streamlining access and management across multiple tables. On the other hand, table engine-level integration provides a more granular control, allowing for tailored optimization and fine-tuning at the individual table level.
7+
Furthermore, the integration of these three technologies within Databend is characterized by varying approaches. While some, like Apache Hive, integrate at the catalog level, others, such as Delta Lake, operate at the table engine level. The catalog-based integration establishes a centralized connection to the Data Lake, streamlining access and management across multiple tables. On the other hand, table engine-level integration provides a more granular control, allowing for tailored optimization and fine-tuning at the individual table level.
88

99
- [Apache Hive Catalog](01-hive.md)
10-
- [Apache Iceberg Catalog](02-iceberg/iceberg-catalog.md)
11-
- [Apache Iceberg Table Engine](02-iceberg/iceberg-engine.md)
10+
- [Apache Iceberg Catalog](02-iceberg.md)
1211
- [Delta Lake Table Engine](03-delta.md)

‎docs/en/sql-reference/00-sql-reference/30-table-engines/03-iceberg.md

Lines changed: 0 additions & 5 deletions
This file was deleted.

0 commit comments

Comments
 (0)