Skip to content

Commit ed0b407

Browse files
authored
dbt-adapter: normalize cluster name generation in deploy operation (#31086)
<!-- Describe the contents of the PR briefly but completely. If you write detailed commit messages, it is acceptable to copy/paste them here, or write "see commit messages for details." If there is only one commit in the PR, GitHub will have already added its commit message above. --> ### Motivation Updating the dbt blue-green deployment actions so that the `adapter.generate_final_cluster_name` is used consistently when checking source clusters to respect customer-defined cluster naming logic using `generate_cluster_name` in their projects. This allows us to use a correct cluster validation when users override the `generate_cluster_name` macro. <!-- Which of the following best describes the motivation behind this PR? * This PR fixes a recognized bug. [Ensure issue is linked somewhere.] * This PR adds a known-desirable feature. [Ensure issue is linked somewhere.] * This PR fixes a previously unreported bug. [Describe the bug in detail, as if you were filing a bug report.] * This PR adds a feature that has not yet been specified. [Write a brief specification for the feature, including justification for its inclusion in Materialize, as if you were writing the original feature specification.] * This PR refactors existing code. [Describe what was wrong with the existing code, if it is not obvious.] --> ### Tips for reviewer <!-- Leave some tips for your reviewer, like: * The diff is much smaller if viewed with whitespace hidden. * [Some function/module/file] deserves extra attention. * [Some function/module/file] is pure code movement and only needs a skim. Delete this section if no tips. --> ### Checklist - [ ] This PR has adequate test coverage / QA involvement has been duly considered. ([trigger-ci for additional test/nightly runs](https://trigger-ci.dev.materialize.com/)) - [ ] This PR has an associated up-to-date [design doc](https://github.com/MaterializeInc/materialize/blob/main/doc/developer/design/README.md), is a design doc ([template](https://github.com/MaterializeInc/materialize/blob/main/doc/developer/design/00000000_template.md)), or is sufficiently small to not require a design. <!-- Reference the design in the description. --> - [ ] If this PR evolves [an existing `$T ⇔ Proto$T` mapping](https://github.com/MaterializeInc/materialize/blob/main/doc/developer/command-and-response-binary-encoding.md) (possibly in a backwards-incompatible way), then it is tagged with a `T-proto` label. - [ ] If this PR will require changes to cloud orchestration or tests, there is a companion cloud PR to account for those changes that is tagged with the release-blocker label ([example](MaterializeInc/cloud#5021)). <!-- Ask in #team-cloud on Slack if you need help preparing the cloud PR. --> - [ ] If this PR includes major [user-facing behavior changes](https://github.com/MaterializeInc/materialize/blob/main/doc/developer/guide-changes.md#what-changes-require-a-release-note), I have pinged the relevant PM to schedule a changelog post.
1 parent 138e4bf commit ed0b407

File tree

3 files changed

+18
-12
lines changed

3 files changed

+18
-12
lines changed

misc/dbt-materialize/dbt/include/materialize/macros/deploy/deploy_init.sql

Lines changed: 8 additions & 6 deletions
Original file line numberDiff line numberDiff line change
@@ -44,12 +44,13 @@
4444
{% endfor %}
4545

4646
{% for cluster in clusters %}
47-
{% if not cluster_exists(cluster) %}
48-
{{ exceptions.raise_compiler_error("Production cluster " ~ cluster ~ " does not exist") }}
47+
{% set origin_cluster = adapter.generate_final_cluster_name(cluster, force_deploy_suffix=False) %}
48+
{% if not cluster_exists(origin_cluster) %}
49+
{{ exceptions.raise_compiler_error("Production cluster " ~ origin_cluster ~ " does not exist") }}
4950
{% endif %}
50-
{% if cluster_contains_sinks(cluster) %}
51+
{% if cluster_contains_sinks(origin_cluster) %}
5152
{{ exceptions.raise_compiler_error("""
52-
Production cluster " ~ cluster ~ " contains sinks.
53+
Production cluster " ~ origin_cluster ~ " contains sinks.
5354
Blue/green deployments require sinks to be in a dedicated cluster.
5455
""") }}
5556
{% endif %}
@@ -107,6 +108,7 @@
107108
{% endfor %}
108109

109110
{% for cluster in clusters %}
111+
{% set origin_cluster = adapter.generate_final_cluster_name(cluster, force_deploy_suffix=False) %}
110112
{% set cluster_configuration %}
111113
SELECT
112114
c.managed,
@@ -118,7 +120,7 @@
118120
cs.refresh_hydration_time_estimate
119121
FROM mz_clusters c
120122
LEFT JOIN mz_internal.mz_cluster_schedules cs ON cs.cluster_id = c.id
121-
WHERE c.name = {{ dbt.string_literal(cluster) }}
123+
WHERE c.name = {{ dbt.string_literal(origin_cluster) }}
122124
{% endset %}
123125

124126
{% set cluster_config_results = run_query(cluster_configuration) %}
@@ -133,7 +135,7 @@
133135
{% set refresh_hydration_time_estimate = results[6] %}
134136

135137
{% if not managed %}
136-
{{ exceptions.raise_compiler_error("Production cluster " ~ cluster ~ " is not managed") }}
138+
{{ exceptions.raise_compiler_error("Production cluster " ~ origin_cluster ~ " is not managed") }}
137139
{% endif %}
138140

139141
{% set deploy_cluster = create_cluster(

misc/dbt-materialize/dbt/include/materialize/macros/deploy/deploy_permission_validation.sql

Lines changed: 2 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -89,7 +89,8 @@ WITH clusters_under_deployment AS (
8989
WHERE name IN (
9090
{% if clusters|length > 0 %}
9191
{% for cluster in clusters %}
92-
{{ dbt.string_literal(cluster) }}{% if not loop.last %},{% endif %}
92+
{% set origin_cluster = adapter.generate_final_cluster_name(cluster, force_deploy_suffix=False) %}
93+
{{ dbt.string_literal(origin_cluster) }}{% if not loop.last %},{% endif %}
9394
{% endfor %}
9495
{% else %}
9596
NULL

misc/dbt-materialize/dbt/include/materialize/macros/deploy/deploy_promote.sql

Lines changed: 8 additions & 5 deletions
Original file line numberDiff line numberDiff line change
@@ -67,8 +67,9 @@
6767

6868
{% for cluster in clusters %}
6969
{% set deploy_cluster = adapter.generate_final_cluster_name(cluster, force_deploy_suffix=True) %}
70-
{% if not cluster_exists(cluster) %}
71-
{{ exceptions.raise_compiler_error("Production cluster " ~ cluster ~ " does not exist") }}
70+
{% set origin_cluster = adapter.generate_final_cluster_name(cluster, force_deploy_suffix=False) %}
71+
{% if not cluster_exists(origin_cluster) %}
72+
{{ exceptions.raise_compiler_error("Production cluster " ~ origin_cluster ~ " does not exist") }}
7273
{% endif %}
7374
{% if not cluster_exists(deploy_cluster) %}
7475
{{ exceptions.raise_compiler_error("Deployment cluster " ~ deploy_cluster ~ " does not exist") }}
@@ -91,8 +92,9 @@
9192

9293
{% for cluster in clusters %}
9394
{% set deploy_cluster = adapter.generate_final_cluster_name(cluster, force_deploy_suffix=True) %}
94-
{{ log("Swapping clusters " ~ adapter.generate_final_cluster_name(cluster) ~ " and " ~ deploy_cluster, info=True) }}
95-
ALTER CLUSTER {{ adapter.quote(cluster) }} SWAP WITH {{ adapter.quote(deploy_cluster) }};
95+
{% set origin_cluster = adapter.generate_final_cluster_name(cluster, force_deploy_suffix=False) %}
96+
{{ log("Swapping clusters " ~ origin_cluster ~ " and " ~ deploy_cluster, info=True) }}
97+
ALTER CLUSTER {{ adapter.quote(origin_cluster) }} SWAP WITH {{ adapter.quote(deploy_cluster) }};
9698
{% endfor %}
9799

98100
COMMIT;
@@ -108,8 +110,9 @@
108110

109111
{% for cluster in clusters %}
110112
{% set deploy_cluster = adapter.generate_final_cluster_name(cluster, force_deploy_suffix=True) %}
113+
{% set origin_cluster = adapter.generate_final_cluster_name(cluster, force_deploy_suffix=False) %}
111114
{{ log("DRY RUN: Swapping clusters " ~ adapter.generate_final_cluster_name(cluster) ~ " and " ~ deploy_cluster, info=True) }}
112-
{{ log("DRY RUN: ALTER CLUSTER " ~ adapter.quote(cluster) ~ " SWAP WITH " ~ adapter.quote(deploy_cluster), info=True) }}
115+
{{ log("DRY RUN: ALTER CLUSTER " ~ adapter.quote(origin_cluster) ~ " SWAP WITH " ~ adapter.quote(deploy_cluster), info=True) }}
113116
{% endfor %}
114117
{{ log("Dry run completed. The statements above were **not** executed against Materialize.", info=True) }}
115118
{% endif %}

0 commit comments

Comments
 (0)