Skip to content

Commit 14be227

Browse files
CopilotBohuTANG
andcommitted
Add USE_LOGIC_TYPE format option for PARQUET and AVRO files
Co-authored-by: BohuTANG <[email protected]>
1 parent b649308 commit 14be227

File tree

3 files changed

+44
-2
lines changed

3 files changed

+44
-2
lines changed

docs/cn/sql-reference/00-sql-reference/50-file-format-options.md

Lines changed: 20 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -36,6 +36,7 @@ formatTypeOptions ::=
3636
ESCAPE = '<character>'
3737
NAN_DISPLAY = '<string>'
3838
ROW_TAG = '<string>'
39+
USE_LOGIC_TYPE = TRUE | FALSE
3940
COMPRESSION = AUTO | GZIP | BZ2 | BROTLI | ZSTD | DEFLATE | RAW_DEFLATE | XZ | NONE
4041
```
4142

@@ -229,6 +230,15 @@ Databend 在处理 TSV 文件时受以下条件约束:
229230
| `ERROR` (默认) | 如果遇到缺失字段,则生成错误。 |
230231
| `FIELD_DEFAULT` | 对缺失字段使用字段的默认值。 |
231232

233+
### USE_LOGIC_TYPE (仅加载)
234+
235+
控制加载期间如何解释时间数据类型(日期和时间戳)。
236+
237+
| 可用值 | 描述 |
238+
|-----------------|---------------------------------------------------------------------------------------------------------------------|
239+
| `TRUE` (默认) | 日期和时间戳值作为其逻辑数据类型(DATE 和 TIMESTAMP)加载。 |
240+
| `FALSE` | 日期和时间戳值作为原始整数值加载(日期为 INT32,时间戳为 INT64)。 |
241+
232242
### COMPRESSION (仅卸载)
233243

234244
指定压缩算法,该算法用于压缩文件的内部块,而不是整个文件,因此输出仍为 Parquet 格式。
@@ -258,4 +268,13 @@ Databend 在处理 TSV 文件时受以下条件约束:
258268
| 可选值 | 描述 |
259269
|------------------|----------------------------------------------------------------------------------------------------|
260270
| `ERROR` (默认) | 如果遇到缺失字段,则会生成错误。 |
261-
| `FIELD_DEFAULT` | 对于缺失的字段,使用该字段的默认值。 |
271+
| `FIELD_DEFAULT` | 对于缺失的字段,使用该字段的默认值。 |
272+
273+
### USE_LOGIC_TYPE (仅加载)
274+
275+
控制加载期间如何解释时间数据类型(日期和时间戳)。
276+
277+
| 可用值 | 描述 |
278+
|-----------------|---------------------------------------------------------------------------------------------------------------------|
279+
| `TRUE` (默认) | 日期和时间戳值作为其逻辑数据类型(DATE 和 TIMESTAMP)加载。 |
280+
| `FALSE` | 日期和时间戳值作为原始整数值加载(日期为 INT32,时间戳为 INT64)。 |

docs/en/sql-reference/00-sql-reference/50-file-format-options.md

Lines changed: 20 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -36,6 +36,7 @@ formatTypeOptions ::=
3636
ESCAPE = '<character>'
3737
NAN_DISPLAY = '<string>'
3838
ROW_TAG = '<string>'
39+
USE_LOGIC_TYPE = TRUE | FALSE
3940
COMPRESSION = AUTO | GZIP | BZ2 | BROTLI | ZSTD | DEFLATE | RAW_DEFLATE | XZ | NONE
4041
```
4142

@@ -231,6 +232,15 @@ Determines the behavior when encountering missing fields during data loading. Re
231232
| `ERROR` (Default)| Generates an error if a missing field is encountered. |
232233
| `FIELD_DEFAULT` | Uses the default value of the field for missing fields. |
233234

235+
### USE_LOGIC_TYPE (Load Only)
236+
237+
Controls how temporal data types (date and timestamp) are interpreted during loading.
238+
239+
| Available Values | Description |
240+
|------------------|----------------------------------------------------------------------------------------------------------------------------|
241+
| `TRUE` (Default) | Date and timestamp values are loaded as their logical data types (DATE and TIMESTAMP). |
242+
| `FALSE` | Date and timestamp values are loaded as raw integer values (INT32 for dates, INT64 for timestamps). |
243+
234244
### COMPRESSION (Unload Only)
235245

236246
Specifies the compression algorithm, which is used for compressing internal blocks of the file rather than the entire file, so the output remains in Parquet format.
@@ -262,4 +272,13 @@ Determines the behavior when encountering missing fields during data loading. Re
262272
| Available Values | Description |
263273
|------------------|-----------------------------------------------------------------------------------------------|
264274
| `ERROR` (Default)| Generates an error if a missing field is encountered. |
265-
| `FIELD_DEFAULT` | Uses the default value of the field for missing fields. |
275+
| `FIELD_DEFAULT` | Uses the default value of the field for missing fields. |
276+
277+
### USE_LOGIC_TYPE (Load Only)
278+
279+
Controls how temporal data types (date and timestamp) are interpreted during loading.
280+
281+
| Available Values | Description |
282+
|------------------|----------------------------------------------------------------------------------------------------------------------------|
283+
| `TRUE` (Default) | Date and timestamp values are loaded as their logical data types (DATE and TIMESTAMP). |
284+
| `FALSE` | Date and timestamp values are loaded as raw integer values (INT32 for dates, INT64 for timestamps). |

static/robots.txt

Lines changed: 4 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,4 @@
1+
User-agent: *
2+
Disallow: /404
3+
Disallow: /tags/
4+
Sitemap: https://www.databend.cn/sitemap.xml

0 commit comments

Comments
 (0)