From 35a260a5de3810e56c2783ecd941b09bfc67be73 Mon Sep 17 00:00:00 2001 From: Matt Shaver <60105315+matthewshaver@users.noreply.github.com> Date: Tue, 26 Nov 2024 16:13:01 -0500 Subject: [PATCH 1/7] Cumulative metrics behavior flag --- .../core-upgrade/06-upgrading-to-v1.9.md | 1 + .../global-configs/behavior-changes.md | 50 +++++++++++++++++++ 2 files changed, 51 insertions(+) diff --git a/website/docs/docs/dbt-versions/core-upgrade/06-upgrading-to-v1.9.md b/website/docs/docs/dbt-versions/core-upgrade/06-upgrading-to-v1.9.md index 7ac5a743995..c293c1a8cc6 100644 --- a/website/docs/docs/dbt-versions/core-upgrade/06-upgrading-to-v1.9.md +++ b/website/docs/docs/dbt-versions/core-upgrade/06-upgrading-to-v1.9.md @@ -83,6 +83,7 @@ You can read more about each of these behavior changes in the following links: - (Introduced, disabled by default) [`skip_nodes_if_on_run_start_fails` project config flag](/reference/global-configs/behavior-changes#behavior-change-flags). If the flag is set and **any** `on-run-start` hook fails, mark all selected nodes as skipped. - `on-run-start/end` hooks are **always** run, regardless of whether they passed or failed last time. - (Introduced, disabled by default) [[Redshift] `restrict_direct_pg_catalog_access`](/reference/global-configs/behavior-changes#redshift-restrict_direct_pg_catalog_access). If the flag is set the adapter will use the Redshift API (through the Python client) if available, or query Redshift's `information_schema` tables instead of using `pg_` tables. +- (Introduced, disabled by default) [`require_nested_cumulative_type_params`](/reference/global-configs/behavior-changes#cumulative-metrics). If the flag is set to `True`, users will receive an error instead of a warning if they're not proprly formatting cumulative metrics using the new [`cumulative_type_params`](/docs/build/cumulative#parameters) nesting. ## Adapter specific features and functionalities diff --git a/website/docs/reference/global-configs/behavior-changes.md b/website/docs/reference/global-configs/behavior-changes.md index 7b69ac44084..2a332a10413 100644 --- a/website/docs/reference/global-configs/behavior-changes.md +++ b/website/docs/reference/global-configs/behavior-changes.md @@ -76,6 +76,7 @@ When we use dbt Cloud in the following table, we're referring to accounts that h | [state_modified_compare_more_unrendered_values](#source-definitions-for-state) | 2024.10 | TBD* | 1.9.0 | TBD* | | [require_yaml_configuration_for_mf_time_spines](#metricflow-time-spine-yaml) | 2024.10 | TBD* | 1.9.0 | TBD* | | [require_batched_execution_for_custom_microbatch_strategy](#custom-microbatch-strategy) | 2024.11 | TBD* | 1.9.0 | TBD* | +| [cumulative_type_params](#cumulative-metrics-parameter) | 2024.11 | TBD* | 1.9.0 | TBD* | When the dbt Cloud Maturity is "TBD," it means we have not yet determined the exact date when these flags' default values will change. Affected users will see deprecation warnings in the meantime, and they will receive emails providing advance warning ahead of the maturity date. In the meantime, if you are seeing a deprecation warning, you can either: - Migrate your project to support the new behavior, and then set the flag to `True` to stop seeing the warnings. @@ -175,3 +176,52 @@ Set the flag is set to `True` if you have a custom microbatch macro set up in yo If you have a custom microbatch macro and the flag is left as `False`, dbt will issue a deprecation warning. Previously, users needed to set the `DBT_EXPERIMENTAL_MICROBATCH` environment variable to `True` to prevent unintended interactions with existing custom incremental strategies. But this is no longer necessary, as setting `DBT_EXPERMINENTAL_MICROBATCH` will no longer have an effect on runtime functionality. + +### Cumulative metrics + +[Cumulative-type metrics](/docs/build/cumulative#parameters) are nested under the `cumulative_type_params` field in Versionless dbt Cloud and Core v1.9 and newer. Currently, users will be warned if they have cumulative metrics imroperly nested. To enforce the new format (resulting in an error instead of a warning), set the `require_nested_cumulative_type_params` to `True`. + +Use the following metric as an example: + +```yaml + + type: cumulative + type_params: + measure: order_count + window: 7 days + +``` + +You will see the following warning on v1.9 or Versionless dbt Cloud if you run `dbt parse`: + +```bash + +15:36:22 [WARNING]: Cumulative fields `type_params.window` and +`type_params.grain_to_date` has been moved and will soon be deprecated. Please +nest those values under `type_params.cumulative_type_params.window` and +`type_params.cumulative_type_params.grain_to_date`. See documentation on +behavior changes: +https://docs.getdbt.com/reference/global-configs/behavior-changes. + +``` + +If you set `require_nested_cumulative_type_params` to `True` and run `dbt parse`, you will now receive an error: + +```bash + +21:39:18 Cumulative fields `type_params.window` and `type_params.grain_to_date` should be nested under `type_params.cumulative_type_params.window` and `type_params.cumulative_type_params.grain_to_date`. Invalid metrics: orders_last_7_days. See documentation on behavior changes: https://docs.getdbt.com/reference/global-configs/behavior-changes. + +``` + +Once the metric is updated with the proper, updated syntax, it will work as expected: + +```yaml + + type: cumulative + type_params: + measure: + name: order_count + cumulative_type_params: + window: 7 days + +``` \ No newline at end of file From 372859cdb20e7386ab1ff82a6e4bfd6e719db641 Mon Sep 17 00:00:00 2001 From: Matt Shaver <60105315+matthewshaver@users.noreply.github.com> Date: Tue, 26 Nov 2024 22:38:07 -0500 Subject: [PATCH 2/7] Update website/docs/reference/global-configs/behavior-changes.md --- website/docs/reference/global-configs/behavior-changes.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/website/docs/reference/global-configs/behavior-changes.md b/website/docs/reference/global-configs/behavior-changes.md index 2a332a10413..e1053b66a2a 100644 --- a/website/docs/reference/global-configs/behavior-changes.md +++ b/website/docs/reference/global-configs/behavior-changes.md @@ -179,7 +179,7 @@ Previously, users needed to set the `DBT_EXPERIMENTAL_MICROBATCH` environment va ### Cumulative metrics -[Cumulative-type metrics](/docs/build/cumulative#parameters) are nested under the `cumulative_type_params` field in Versionless dbt Cloud and Core v1.9 and newer. Currently, users will be warned if they have cumulative metrics imroperly nested. To enforce the new format (resulting in an error instead of a warning), set the `require_nested_cumulative_type_params` to `True`. +[Cumulative-type metrics](/docs/build/cumulative#parameters) are nested under the `cumulative_type_params` field in Versionless dbt Cloud and Core v1.9 and newer. Currently, dbt will warn users if they have cumulative metrics improperly nested. To enforce the new format (resulting in an error instead of a warning), set the `require_nested_cumulative_type_params` to `True`. Use the following metric as an example: From 1c651b43de564204eae4498610d48748125c4dc3 Mon Sep 17 00:00:00 2001 From: Matt Shaver <60105315+matthewshaver@users.noreply.github.com> Date: Tue, 26 Nov 2024 22:39:18 -0500 Subject: [PATCH 3/7] Update website/docs/reference/global-configs/behavior-changes.md --- website/docs/reference/global-configs/behavior-changes.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/website/docs/reference/global-configs/behavior-changes.md b/website/docs/reference/global-configs/behavior-changes.md index e1053b66a2a..919251b4673 100644 --- a/website/docs/reference/global-configs/behavior-changes.md +++ b/website/docs/reference/global-configs/behavior-changes.md @@ -181,7 +181,7 @@ Previously, users needed to set the `DBT_EXPERIMENTAL_MICROBATCH` environment va [Cumulative-type metrics](/docs/build/cumulative#parameters) are nested under the `cumulative_type_params` field in Versionless dbt Cloud and Core v1.9 and newer. Currently, dbt will warn users if they have cumulative metrics improperly nested. To enforce the new format (resulting in an error instead of a warning), set the `require_nested_cumulative_type_params` to `True`. -Use the following metric as an example: +Use the following metric configured with the syntax before v1.9 as an example: ```yaml From 8410d83a541a01accd67ce028f3582b205a97c83 Mon Sep 17 00:00:00 2001 From: Matt Shaver <60105315+matthewshaver@users.noreply.github.com> Date: Tue, 26 Nov 2024 22:40:23 -0500 Subject: [PATCH 4/7] Update website/docs/reference/global-configs/behavior-changes.md --- website/docs/reference/global-configs/behavior-changes.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/website/docs/reference/global-configs/behavior-changes.md b/website/docs/reference/global-configs/behavior-changes.md index 919251b4673..e475373390f 100644 --- a/website/docs/reference/global-configs/behavior-changes.md +++ b/website/docs/reference/global-configs/behavior-changes.md @@ -192,7 +192,7 @@ Use the following metric configured with the syntax before v1.9 as an example: ``` -You will see the following warning on v1.9 or Versionless dbt Cloud if you run `dbt parse`: +If you run `dbt parse` with that syntax on Core v1.9 or Versionless dbt Cloud, you will receive a warning like: ```bash From 6bada5d507105e042f6dd1802fe68428d68f1332 Mon Sep 17 00:00:00 2001 From: Matt Shaver <60105315+matthewshaver@users.noreply.github.com> Date: Tue, 26 Nov 2024 22:41:07 -0500 Subject: [PATCH 5/7] Update website/docs/reference/global-configs/behavior-changes.md --- website/docs/reference/global-configs/behavior-changes.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/website/docs/reference/global-configs/behavior-changes.md b/website/docs/reference/global-configs/behavior-changes.md index e475373390f..72490c69d40 100644 --- a/website/docs/reference/global-configs/behavior-changes.md +++ b/website/docs/reference/global-configs/behavior-changes.md @@ -205,7 +205,7 @@ https://docs.getdbt.com/reference/global-configs/behavior-changes. ``` -If you set `require_nested_cumulative_type_params` to `True` and run `dbt parse`, you will now receive an error: +If you set `require_nested_cumulative_type_params` to `True` and re-run `dbt parse` you will now receive an error like: ```bash From eb4b806e41cb844295d32fcd0d5af3178337c9ff Mon Sep 17 00:00:00 2001 From: Matt Shaver <60105315+matthewshaver@users.noreply.github.com> Date: Tue, 26 Nov 2024 22:41:54 -0500 Subject: [PATCH 6/7] Update website/docs/reference/global-configs/behavior-changes.md --- website/docs/reference/global-configs/behavior-changes.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/website/docs/reference/global-configs/behavior-changes.md b/website/docs/reference/global-configs/behavior-changes.md index 72490c69d40..b16e6b3169e 100644 --- a/website/docs/reference/global-configs/behavior-changes.md +++ b/website/docs/reference/global-configs/behavior-changes.md @@ -213,7 +213,7 @@ If you set `require_nested_cumulative_type_params` to `True` and re-run `dbt par ``` -Once the metric is updated with the proper, updated syntax, it will work as expected: +Once the metric is updated, it will work as expected: ```yaml From 76b1091a3af67ab42165d8493d2aee5b42b68ccb Mon Sep 17 00:00:00 2001 From: nataliefiann <120089939+nataliefiann@users.noreply.github.com> Date: Wed, 27 Nov 2024 10:15:10 +0000 Subject: [PATCH 7/7] Update website/docs/reference/global-configs/behavior-changes.md --- website/docs/reference/global-configs/behavior-changes.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/website/docs/reference/global-configs/behavior-changes.md b/website/docs/reference/global-configs/behavior-changes.md index b16e6b3169e..bccf96eb728 100644 --- a/website/docs/reference/global-configs/behavior-changes.md +++ b/website/docs/reference/global-configs/behavior-changes.md @@ -179,7 +179,7 @@ Previously, users needed to set the `DBT_EXPERIMENTAL_MICROBATCH` environment va ### Cumulative metrics -[Cumulative-type metrics](/docs/build/cumulative#parameters) are nested under the `cumulative_type_params` field in Versionless dbt Cloud and Core v1.9 and newer. Currently, dbt will warn users if they have cumulative metrics improperly nested. To enforce the new format (resulting in an error instead of a warning), set the `require_nested_cumulative_type_params` to `True`. +[Cumulative-type metrics](/docs/build/cumulative#parameters) are nested under the `cumulative_type_params` field in Versionless dbt Cloud, dbt Core v1.9 and newer. Currently, dbt will warn users if they have cumulative metrics improperly nested. To enforce the new format (resulting in an error instead of a warning), set the `require_nested_cumulative_type_params` to `True`. Use the following metric configured with the syntax before v1.9 as an example: