You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
Copy file name to clipboardExpand all lines: website/docs/docs/build/snapshots.md
+61-64Lines changed: 61 additions & 64 deletions
Original file line number
Diff line number
Diff line change
@@ -437,103 +437,100 @@ Snapshot <Term id="table">tables</Term> will be created as a clone of your sourc
437
437
438
438
In dbt Core v1.9+ (or available sooner in [the "Latest" release track in dbt Cloud](/docs/dbt-versions/cloud-release-tracks)):
439
439
- These column names can be customized to your team or organizational conventions using the [`snapshot_meta_column_names`](/reference/resource-configs/snapshot_meta_column_names) config.
440
-
ess)
441
440
- Use the [`dbt_valid_to_current` config](/reference/resource-configs/dbt_valid_to_current) to set a custom indicator for the value of `dbt_valid_to` in current snapshot records (like a future date such as `9999-12-31`). By default, this value is `NULL`. When set, dbt will use this specified value instead of `NULL` for `dbt_valid_to` for current records in the snapshot table.
442
441
- Use the [`hard_deletes`](/reference/resource-configs/hard-deletes) config to track deleted records as new rows with the `dbt_is_deleted` meta field when using the `hard_deletes='new_record'` field.
443
442
444
443
445
-
| Field | Meaning | Usage |
446
-
| -------------- | ------- | ----- |
447
-
| dbt_valid_from | The timestamp when this snapshot row was first inserted | This column can be used to order the different "versions" of a record. |
448
-
| dbt_valid_to | The timestamp when this row became invalidated. <br /> For current records, this is `NULL` by default <VersionBlock firstVersion="1.9"> or the value specified in `dbt_valid_to_current`.</VersionBlock> | The most recent snapshot record will have `dbt_valid_to` set to `NULL` <VersionBlock firstVersion="1.9"> or the specified value. </VersionBlock> |
449
-
| dbt_scd_id | A unique key generated for each snapshotted record. | This is used internally by dbt |
450
-
| dbt_updated_at | The updated_at timestamp of the source record when this snapshot row was inserted. | This is used internally by dbt |
451
-
| dbt_is_deleted | A boolean value indicating if the record has been deleted. `True` if deleted, `False` otherwise. |Added when `hard_deletes='new_record'` is configured. This is used internally by dbt |
444
+
| Field | <div style={{width:'250px'}}>Meaning</div> | Notes | Example|
445
+
| -------------- | ------- | ----- | ------- |
446
+
| `dbt_valid_from` | The timestamp when this snapshot row was first inserted and became valid. | This column can be used to order the different "versions" of a record. | `snapshot_meta_column_names: {dbt_valid_from: start_date}` |
447
+
| `dbt_valid_to` | The timestamp when this row became invalidated. For current records, this is `NULL` by default or the value specified in `dbt_valid_to_current`. | The most recent snapshot record will have `dbt_valid_to` set to `NULL` or the specified value. | `snapshot_meta_column_names: {dbt_valid_to: end_date}` |
448
+
| `dbt_scd_id` | A unique key generated for each snapshot row. | This is used internally by dbt. | `snapshot_meta_column_names: {dbt_scd_id: scd_id}` |
449
+
| `dbt_updated_at` | The `updated_at` timestamp of the source record when this snapshot row was inserted. | This is used internally by dbt. | `snapshot_meta_column_names: {dbt_updated_at: modified_date}` |
450
+
| `dbt_is_deleted` | A string value indicating if the record has been deleted. (`True` if deleted, `False` if not deleted). |Added when `hard_deletes='new_record'` is configured. | `snapshot_meta_column_names: {dbt_is_deleted: is_deleted}` |
452
451
453
-
*The timestamps used for each column are subtly different depending on the strategy you use:
452
+
All of these column names can be customized using the `snapshot_meta_column_names` config. Refer to this [example](/reference/resource-configs/snapshot_meta_column_names#example) for more details.
454
453
455
-
For the `timestamp` strategy, the configured `updated_at` column is used to populate the `dbt_valid_from`, `dbt_valid_to` and `dbt_updated_at` columns.
454
+
*The timestamps used for each column are subtly different depending on the strategy you use:
456
455
457
-
<details>
458
-
<summary> Details for the timestamp strategy </summary>
456
+
- For the `timestamp` strategy, the configured `updated_at` column is used to populate the `dbt_valid_from`, `dbt_valid_to` and `dbt_updated_at` columns.
459
457
460
-
Snapshot query results at `2024-01-01 11:00`
458
+
<Expandable alt_header="Sample results for the timestamp strategy">
461
459
462
-
| id | status | updated_at |
463
-
| -- | ------- | ---------------- |
464
-
| 1 | pending | 2024-01-01 10:47 |
460
+
Snapshot query results at `2024-01-01 11:00`
465
461
466
-
Snapshot results (note that `11:00` is not used anywhere):
462
+
| id | status | updated_at |
463
+
| -- | ------- | ---------------- |
464
+
| 1 | pending | 2024-01-01 10:47 |
467
465
468
-
| id | status | updated_at | dbt_valid_from | dbt_valid_to | dbt_updated_at |
For the `check` strategy, the current timestamp is used to populate each column. If configured, the `check` strategy uses the `updated_at` column instead, as with the timestamp strategy.
496
+
- For the `check` strategy, the current timestamp is used to populate each column. If configured, the `check` strategy uses the `updated_at` column instead, as with the timestamp strategy.
499
497
500
-
<details>
501
-
<summary> Details for the check strategy </summary>
498
+
<Expandable alt_header="Sample results for the check strategy">
502
499
503
-
Snapshot query results at `2024-01-01 11:00`
500
+
Snapshot query results at `2024-01-01 11:00`
504
501
505
-
| id | status |
506
-
| -- | ------- |
507
-
| 1 | pending |
502
+
| id | status |
503
+
| -- | ------- |
504
+
| 1 | pending |
508
505
509
-
Snapshot results:
506
+
Snapshot results:
510
507
511
-
| id | status | dbt_valid_from | dbt_valid_to | dbt_updated_at |
Copy file name to clipboardExpand all lines: website/docs/reference/resource-configs/snapshot_meta_column_names.md
+12-12Lines changed: 12 additions & 12 deletions
Original file line number
Diff line number
Diff line change
@@ -19,7 +19,7 @@ snapshots:
19
19
dbt_valid_to: <string>
20
20
dbt_scd_id: <string>
21
21
dbt_updated_at: <string>
22
-
dbt_is_deleted: <boolean>
22
+
dbt_is_deleted: <string>
23
23
24
24
```
25
25
@@ -35,7 +35,7 @@ snapshots:
35
35
"dbt_valid_to": "<string>",
36
36
"dbt_scd_id": "<string>",
37
37
"dbt_updated_at": "<string>",
38
-
"dbt_is_deleted": "<boolean>",
38
+
"dbt_is_deleted": "<string>",
39
39
}
40
40
)
41
41
}}
@@ -54,7 +54,7 @@ snapshots:
54
54
dbt_valid_to: <string>
55
55
dbt_scd_id: <string>
56
56
dbt_updated_at: <string>
57
-
dbt_is_deleted: <boolean>
57
+
dbt_is_deleted: <string>
58
58
```
59
59
60
60
</File>
@@ -67,17 +67,17 @@ In order to align with an organization's naming conventions, the `snapshot_meta_
67
67
68
68
By default, dbt snapshots use the following column names to track change history using [Type 2 slowly changing dimension](https://en.wikipedia.org/wiki/Slowly_changing_dimension#Type_2:_add_new_row) records:
69
69
70
-
| Field | Meaning | Notes |
71
-
| -------------- | ------- | ----- |
72
-
| `dbt_valid_from` | The timestamp when this snapshot row was first inserted and became valid. | The value is affected by the [`strategy`](/reference/resource-configs/strategy). |
73
-
| `dbt_valid_to` | The timestamp when this row is no longer valid. | |
74
-
| `dbt_scd_id` | A unique key generated for each snapshot row. | This is used internally by dbt. |
75
-
| `dbt_updated_at` | The `updated_at` timestamp of the source record when this snapshot row was inserted. | This is used internally by dbt. |
76
-
| `dbt_is_deleted` | A boolean value indicating if the record has been deleted. `True` if deleted, `False` otherwise. |Added when `hard_deletes='new_record'` is configured. |
70
+
| Field | <div style={{width:'250px'}}>Meaning</div> | Notes | Example |
71
+
| -------------- | ------- | ----- | ------- |
72
+
| `dbt_valid_from` | The timestamp when this snapshot row was first inserted and became valid. | The value is affected by the [`strategy`](/reference/resource-configs/strategy). | `snapshot_meta_column_names: {dbt_valid_from: start_date}` |
73
+
| `dbt_valid_to` | The timestamp when this row is no longer valid. | | `snapshot_meta_column_names: {dbt_valid_to: end_date}` |
74
+
| `dbt_scd_id` | A unique key generated for each snapshot row. | This is used internally by dbt. | `snapshot_meta_column_names: {dbt_scd_id: scd_id}` |
75
+
| `dbt_updated_at` | The `updated_at` timestamp of the source record when this snapshot row was inserted. | This is used internally by dbt. | `snapshot_meta_column_names: {dbt_updated_at: modified_date}` |
76
+
| `dbt_is_deleted` | A string value indicating if the record has been deleted. (`True` if deleted, `False` if not deleted). |Added when `hard_deletes='new_record'` is configured. | `snapshot_meta_column_names: {dbt_is_deleted: is_deleted}` |
77
77
78
-
However, these column names can be customized using the `snapshot_meta_column_names` config.
78
+
All of these column names can be customized using the `snapshot_meta_column_names` config. Refer to the [Example](#example) for more details.
79
79
80
-
:::warning
80
+
:::warning
81
81
82
82
To avoid any unintentional data modification, dbt will **not** automatically apply any column renames. So if a user applies `snapshot_meta_column_names` config for a snapshot without updating the pre-existing table, they will get an error. We recommend either only using these settings for net-new snapshots, or arranging an update of pre-existing tables prior to committing a column name change.
0 commit comments