Skip to content

Commit e84b8f3

Browse files
authored
Merge branch 'current' into er/compatible-2025-01
2 parents ab9181c + f43ba0e commit e84b8f3

File tree

8 files changed

+118
-27
lines changed

8 files changed

+118
-27
lines changed

website/docs/docs/build/custom-aliases.md

Lines changed: 5 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -157,3 +157,8 @@ If these models should indeed have the same database identifier, you can work ar
157157

158158
By default, dbt will create versioned models with the alias `<model_name>_v<v>`, where `<v>` is that version's unique identifier. You can customize this behavior just like for non-versioned models by configuring a custom `alias` or re-implementing the `generate_alias_name` macro.
159159

160+
## Related docs
161+
162+
- [Customize dbt models database, schema, and alias](/guides/customize-schema-alias?step=1) to learn how to customize dbt models database, schema, and alias
163+
- [Custom schema](/docs/build/custom-schemas) to learn how to customize dbt schema
164+
- [Custom database](/docs/build/custom-databases) to learn how to customize dbt database

website/docs/docs/build/custom-databases.md

Lines changed: 6 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -98,3 +98,9 @@ See docs on macro `dispatch`: ["Managing different global overrides across packa
9898
### BigQuery
9999

100100
When dbt opens a BigQuery connection, it will do so using the `project_id` defined in your active `profiles.yml` target. This `project_id` will be billed for the queries that are executed in the dbt run, even if some models are configured to be built in other projects.
101+
102+
## Related docs
103+
104+
- [Customize dbt models database, schema, and alias](/guides/customize-schema-alias?step=1) to learn how to customize dbt models database, schema, and alias
105+
- [Custom schema](/docs/build/custom-schemas) to learn how to customize dbt model schema
106+
- [Custom aliases](/docs/build/custom-aliases) to learn how to customize dbt model alias name

website/docs/docs/build/custom-schemas.md

Lines changed: 6 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -207,3 +207,9 @@ In the `generate_schema_name` macro examples shown in the [built-in alternative
207207
If your schema names are being generated incorrectly, double-check your target name in the relevant environment.
208208

209209
For more information, consult the [managing environments in dbt Core](/docs/core/dbt-core-environments) guide.
210+
211+
## Related docs
212+
213+
- [Customize dbt models database, schema, and alias](/guides/customize-schema-alias?step=1) to learn how to customize dbt models database, schema, and alias
214+
- [Custom database](/docs/build/custom-databases) to learn how to customize dbt model database
215+
- [Custom aliases](/docs/build/custom-aliases) to learn how to customize dbt model alias name

website/docs/docs/build/unit-tests.md

Lines changed: 11 additions & 5 deletions
Original file line numberDiff line numberDiff line change
@@ -10,9 +10,6 @@ keywords:
1010

1111
<VersionCallout version="1.8" />
1212

13-
14-
15-
1613
Historically, dbt's test coverage was confined to [“data” tests](/docs/build/data-tests), assessing the quality of input data or resulting datasets' structure. However, these tests could only be executed _after_ building a model.
1714

1815
Starting in dbt Core v1.8, we have introduced an additional type of test to dbt - unit tests. In software programming, unit tests validate small portions of your functional code, and they work much the same way here. Unit tests allow you to validate your SQL modeling logic on a small set of static inputs _before_ you materialize your full model in production. Unit tests enable test-driven development, benefiting developer efficiency and code reliability.
@@ -219,10 +216,19 @@ dbt test --select test_is_valid_email_address
219216

220217
Your model is now ready for production! Adding this unit test helped catch an issue with the SQL logic _before_ you materialized `dim_customers` in your warehouse and will better ensure the reliability of this model in the future.
221218

222-
223219
## Unit testing incremental models
224220

225-
When configuring your unit test, you can override the output of macros, vars, or environment variables. This enables you to unit test your incremental models in "full refresh" and "incremental" modes.
221+
When configuring your unit test, you can override the output of macros, vars, or environment variables. This enables you to unit test your incremental models in "full refresh" and "incremental" modes.
222+
223+
:::note
224+
Incremental models need to exist in the database first before running unit tests or doing a `dbt build`. Use the [`--empty` flag](/reference/commands/build#the---empty-flag) to build an empty version of the models to save warehouse spend. You can also optionally select only your incremental models using the [`--select` flag](/reference/node-selection/syntax#shorthand).
225+
226+
```shell
227+
dbt run --select "config.materialized:incremental" --empty
228+
```
229+
230+
After running the command, you can then perform a regular `dbt build` for that model and then run your unit test.
231+
:::
226232

227233
When testing an incremental model, the expected output is the __result of the materialization__ (what will be merged/inserted), not the resulting model itself (what the final table will look like after the merge/insert).
228234

website/docs/docs/deploy/deploy-jobs.md

Lines changed: 3 additions & 3 deletions
Original file line numberDiff line numberDiff line change
@@ -32,9 +32,9 @@ You can create a deploy job and configure it to run on [scheduled days and times
3232
- (Optional) **Description** &mdash; Provide a description of what the job does (for example, what the job consumes and what the job produces).
3333
- **Environment** &mdash; By default, it’s set to the deployment environment you created the deploy job from.
3434
3. Options in the **Execution settings** section:
35-
- **Commands** &mdash; By default, it includes the `dbt build` command. Click **Add command** to add more [commands](/docs/deploy/job-commands) that you want to be invoked when the job runs.
36-
- **Generate docs on run** &mdash; Enable this option if you want to [generate project docs](/docs/collaborate/build-and-view-your-docs) when this deploy job runs.
37-
- **Run source freshness** &mdash; Enable this option to invoke the `dbt source freshness` command before running the deploy job. Refer to [Source freshness](/docs/deploy/source-freshness) for more details.
35+
- [**Commands**](/docs/deploy/job-commands#built-in-commands) &mdash; By default, it includes the `dbt build` command. Click **Add command** to add more [commands](/docs/deploy/job-commands) that you want to be invoked when the job runs. During a job run, [built-in commands](/docs/deploy/job-commands#built-in-commands) are "chained" together and if one run step fails, the entire job fails with an "Error" status.
36+
- [**Generate docs on run**](/docs/deploy/job-commands#checkbox-commands) &mdash; Enable this option if you want to [generate project docs](/docs/collaborate/build-and-view-your-docs) when this deploy job runs. If the step fails, the job can succeed if subsequent steps pass.
37+
- [**Run source freshness**](/docs/deploy/job-commands#checkbox-commands) &mdash; Enable this option to invoke the `dbt source freshness` command before running the deploy job. If the step fails, the job can succeed if subsequent steps pass. Refer to [Source freshness](/docs/deploy/source-freshness) for more details.
3838
4. Options in the **Triggers** section:
3939
- **Run on schedule** &mdash; Run the deploy job on a set schedule.
4040
- **Timing** &mdash; Specify whether to [schedule](#schedule-days) the deploy job using **Intervals** that run the job every specified number of hours, **Specific hours** that run the job at specific times of day, or **Cron schedule** that run the job specified using [cron syntax](#cron-schedule).

website/docs/guides/customize-schema-alias.md

Lines changed: 1 addition & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -9,6 +9,7 @@ icon: 'guides'
99
hide_table_of_contents: true
1010
level: 'Advanced'
1111
recently_updated: true
12+
keywords: ["generate", "schema name", "guide", "dbt", "schema customization", "custom schema"]
1213
---
1314

1415
<div style={{maxWidth: '900px'}}>

website/docs/reference/resource-configs/event-time.md

Lines changed: 17 additions & 10 deletions
Original file line numberDiff line numberDiff line change
@@ -21,7 +21,6 @@ models:
2121
```
2222
</File>
2323
24-
2524
<File name='models/properties.yml'>
2625
2726
```yml
@@ -139,20 +138,28 @@ sources:
139138
140139
## Definition
141140
142-
Set the `event_time` to the name of the field that represents the timestamp of the event -- "at what time did the row occur" -- as opposed to an event ingestion date. You can configure `event_time` for a [model](/docs/build/models), [seed](/docs/build/seeds), or [source](/docs/build/sources) in your `dbt_project.yml` file, property YAML file, or config block.
141+
You can configure `event_time` for a [model](/docs/build/models), [seed](/docs/build/seeds), or [source](/docs/build/sources) in your `dbt_project.yml` file, property YAML file, or config block.
142+
143+
`event_time` is required for the [incremental microbatch](/docs/build/incremental-microbatch) strategy and highly recommended for [Advanced CI's compare changes](/docs/deploy/advanced-ci#optimizing-comparisons) in CI/CD workflows, where it ensures the same time-slice of data is correctly compared between your CI and production environments.
144+
145+
### Best practices
146+
147+
Set the `event_time` to the name of the field that represents the actual timestamp of the event (like `account_created_at`). The timestamp of the event should represent "at what time did the row occur" rather than an event ingestion date. Marking a column as the `event_time` when it isn't, diverges from the semantic meaning of the column which may result in user confusion when other tools make use of the metadata.
143148

144-
Here are some examples of good and bad `event_time` columns:
149+
However, if an ingestion date (like `loaded_at`, `ingested_at`, or `last_updated_at`) are the only timestamps you use, you can set `event_time` to these fields. Here are some considerations to keep in mind if you do this:
145150

146-
- ✅ Good:
147-
- `account_created_at` &mdash; This represents the specific time when an account was created, making it a fixed event in time.
148-
- `session_began_at` &mdash; This captures the exact timestamp when a user session started, which won’t change and directly ties to the event.
151+
- Using `last_updated_at` or `loaded_at` &mdash; May result in duplicate entries in the resulting table in the data warehouse over multiple runs. Setting an appropriate [lookback](/reference/resource-configs/lookback) value can reduce duplicates but it can't fully eliminate them since some updates outside the lookback window won't be processed.
152+
- Using `ingested_at` &mdash; Since this column is created by your ingestion/EL tool instead of coming from the original source, it will change if/when you need to resync your connector for some reason. This means that data will be reprocessed and loaded into your warehouse for a second time against a second date. As long as this never happens (or you run a full refresh when it does), microbatches will be processed correctly when using `ingested_at`.
149153

150-
- ❌ Bad:
154+
Here are some examples of recommended and not recommended `event_time` columns:
151155

152-
- `_fivetran_synced` &mdash; This isn't the time that the event happened, it's the time that the event was ingested.
153-
- `last_updated_at` &mdash; This isn't a good use case as this will keep changing over time.
154156

155-
`event_time` is required for [Incremental microbatch](/docs/build/incremental-microbatch) and highly recommended for [Advanced CI's compare changes](/docs/deploy/advanced-ci#optimizing-comparisons) in CI/CD workflows, where it ensures the same time-slice of data is correctly compared between your CI and production environments.
157+
| <div style={{width:'200px'}}>Status</div> | Column name | Description |
158+
|--------------------|---------------------|----------------------|
159+
| ✅ Recommended | `account_created_at` | Represents the specific time when an account was created, making it a fixed event in time. |
160+
| ✅ Recommended | `session_began_at` | Captures the exact timestamp when a user session started, which won’t change and directly ties to the event. |
161+
| ❌ Not recommended | `_fivetran_synced` | This represents the time the event was ingested, not when it happened. |
162+
| ❌ Not recommended | `last_updated_at` | Changes over time and isn't tied to the event itself. If used, note the considerations mentioned earlier in [best practices](#best-practices). |
156163

157164
## Examples
158165

website/docs/reference/resource-configs/tags.md

Lines changed: 69 additions & 9 deletions
Original file line numberDiff line numberDiff line change
@@ -93,6 +93,21 @@ resource_type:
9393
```
9494
9595
</File>
96+
97+
To apply tags to a model in your `models/` directory, add the `config` property similar to the following example:
98+
99+
<File name='models/model.yml'>
100+
101+
```yaml
102+
models:
103+
- name: my_model
104+
description: A model description
105+
config:
106+
tags: ['example_tag']
107+
```
108+
109+
</File>
110+
96111
</TabItem>
97112

98113
<TabItem value="config">
@@ -126,10 +141,24 @@ You can use the [`+` operator](/reference/node-selection/graph-operators#the-plu
126141
- `dbt run --select +model_name+` &mdash; Run a model, its upstream dependencies, and its downstream dependencies.
127142
- `dbt run --select tag:my_tag+ --exclude tag:exclude_tag` &mdash; Run model tagged with `my_tag` and their downstream dependencies, and exclude models tagged with `exclude_tag`, regardless of their dependencies.
128143

144+
145+
:::tip Usage notes about tags
146+
147+
When using tags, consider the following:
148+
149+
- Tags are additive across project hierarchy.
150+
- Some resource types (like sources, exposures) require tags at the top level.
151+
152+
Refer to [usage notes](#usage-notes) for more information.
153+
:::
154+
129155
## Examples
156+
157+
The following examples show how to apply tags to resources in your project. You can configure tags in the `dbt_project.yml`, `schema.yml`, or SQL files.
158+
130159
### Use tags to run parts of your project
131160

132-
Apply tags in your `dbt_project.yml` as a single value or a string:
161+
Apply tags in your `dbt_project.yml` as a single value or a string. In the following example, one of the models, the `jaffle_shop` model, is tagged with `contains_pii`.
133162

134163
<File name='dbt_project.yml'>
135164

@@ -153,16 +182,52 @@ models:
153182
- "published"
154183
155184
```
185+
</File>
186+
187+
188+
### Apply tags to models
189+
190+
This section demonstrates applying tags to models in the `dbt_project.yml`, `schema.yml`, and SQL files.
191+
192+
To apply tags to a model in your `dbt_project.yml` file, you would add the following:
193+
194+
<File name='dbt_project.yml'>
195+
196+
```yaml
197+
models:
198+
jaffle_shop:
199+
+tags: finance # jaffle_shop model is tagged with 'finance'.
200+
```
201+
202+
</File>
203+
204+
To apply tags to a model in your `models/` directory YAML file, you would add the following using the `config` property:
205+
206+
<File name='models/stg_customers.yml'>
207+
208+
```yaml
209+
models:
210+
- name: stg_customers
211+
description: Customer data with basic cleaning and transformation applied, one row per customer.
212+
config:
213+
tags: ['santi'] # stg_customers.yml model is tagged with 'santi'.
214+
columns:
215+
- name: customer_id
216+
description: The unique key for each customer.
217+
data_tests:
218+
- not_null
219+
- unique
220+
```
156221

157222
</File>
158223

159-
You can also apply tags to individual resources using a config block:
224+
To apply tags to a model in your SQL file, you would add the following:
160225

161226
<File name='models/staging/stg_payments.sql'>
162227

163228
```sql
164229
{{ config(
165-
tags=["finance"]
230+
tags=["finance"] # stg_payments.sql model is tagged with 'finance'.
166231
) }}
167232
168233
select ...
@@ -211,14 +276,10 @@ seeds:
211276

212277
<VersionBlock lastVersion="1.8">
213278

214-
:::tip Upgrade to dbt Core 1.9
215-
216-
Applying tags to saved queries is only available in dbt Core versions 1.9 and later.
217-
:::
279+
<VersionCallout version="1.9" />
218280

219281
</VersionBlock>
220282

221-
<VersionBlock firstVersion="1.9">
222283

223284
This following example shows how to apply a tag to a saved query in the `dbt_project.yml` file. The saved query is then tagged with `order_metrics`.
224285

@@ -263,7 +324,6 @@ Run resources with multiple tags using the following commands:
263324
# Run all resources tagged "order_metrics" and "hourly"
264325
dbt build --select tag:order_metrics tag:hourly
265326
```
266-
</VersionBlock>
267327

268328
## Usage notes
269329

0 commit comments

Comments
 (0)