[Feature] Allow for Setting of IAM Tags

[anaghshineh]

Is this your first time submitting a feature request?

• I have read the expectations for open source contributors
• I have searched the existing issues, and I could not find an existing issue for this feature
• I am requesting a straightforward extension of existing dbt-bigquery functionality, rather than a Big Idea better suited to a discussion

Describe the feature

I'd love to be able to set IAM tags on generated datasets and tables via dbt YAML! Similar to how we can specify labels and policy tags. This would let us manage access to generated assets conditionally. In my opinion, this provides a more flexible and scalable approach to access management over how grants work.

Describe alternatives you've considered

No response

Who will this benefit?

This will benefit organizations who leverage or want to leverage tags for conditional access to resources in their cloud environments. This is a common approach for access management. It allows for more streamlined, consistent access patterns across different projects and environments.

Are you interested in contributing this feature?

Perhaps! I'd be interested to hear from you all re: what you think the difficulty level of this is.

Anything else?

No response

[dataders]

@anaghshineh! 👋🏻 !

I'll say right away that I've never before encountered IAM Tags on BigQuery, so please enlighten me if I'm off as I try to figure it out and how you'd like to see it applied to dbt. I hope I'm providing helpful context

To me there are two questions to be answered:

How does one go about setting IAM tags programmatically in BigQuery (without dbt)

1. Is it available possible via REST API?
2. Is the REST API method surfaced within the python-bigquery client library?

After poking around the docs and got lost quickly. Help!

• BQ Python Client's Dataset Class docs
• BQ REST API reference for tables.setIamPolicy

Can IAM tags be applied to individual models, or only to a Dataset or Project?

If there's a python-bigquery way to do it, there's precedent for modifying/adding a new method to the BigQueryAdapter class. In virtually all cases it happens via a call to client.update_table or client.update_dataset (for example how PolicyTags are currently applied)

What should dbt UX look like?

1. Should they work like grants?

   models:
     - name: specific_model
       config:
         grants:
           select: ['SOME_IAM_TAG']
   

2. Should they work like tags?

   {{
     config(
       materialized = "table",
       labels = {'SOME_IAM_TAG': ''}
     )
   }}

[dataders]

also @anaghshineh I'd appreciate your insight on dbt-labs/dbt-external-tables#263 if you have a chance.

[Junobijlard]

Hey,

I was also looking for this feature and stumbled upon this issue.
You can use these BigQuery tags to set conditional access (although it's still in preview I see).

To chip in on your question:

• Tags can technically be applied to both datasets and tables
• Programmatically, you can call tables.update and include tags in resource_tags source
• For datasets, there is no support for the REST API (but i can ask my Google contacts if needed)

Regarding the dbt UX:

• I like the label approach better -- with our org, we would like to have the flexibility to use our own tags / naming conventions (managed by terraform) -- for me it's not obvious how that would work with the grants approach;

[akerone]

Hi everyone!

I just wanted to +1 this feature and second everything @Junobijlard said in his comment, adding that it seems that today it is already GA and that it seems they have added the ability to add resource tags to datasets. Both great news!

If it helps the case for prioritizing this enhancement, Resource Tags are the only way to apply conditions for "Deny Policies". In my case, I need to apply Deny Policies to protect sensitive data from organization-level grants which I don't have control over.

Being able to attach Resource Tags to tables or datasets from dbt (i.e.: dbt_project.yml) would greatly simplify the whole process.

Thank you!

[oscar-macho-ingka]

I'm eagerly looking for this feature so +1 too!

[keunsoopark]

At the end of 2024, there is REST API for adding resource tags (referred as IAM tags in this issue): https://cloud.google.com/bigquery/docs/tags#tag_datasets

I could not find resource_tags field in Dataset class in python-bigquery, and I think resource_tags is different with Etag according to bq dataset API docs.

So my question is how can we implement this feature as dbt-UX style with rest api? Have not done this through rest api, so any idea/help would be appreciated!

[akerone]

Hi @keunsoopark

I've actually tried to dig deep on the Resource Manager API for this, and can't figure out how to specify the location for the CreateTagBindingRequest. This is needed because datasets aren't "global" resources. So far, I think Terraform are the only ones who have been able to pull it off. link to the relevant resource here. I haven't had the time to try an reverse-engineer their provider code.

I've looked both into the Python Client and REST API with no luck.

It seems to me that this method that @Junobijlard mentioned looks like the best approach at this time.

[keunsoopark]

Started fixing this first for dataset. Let's see how fast PR process is there :)
googleapis/python-bigquery#2090

[geoHeil]

Great to see googleapis/python-bigquery#2090 merged. What else needs to be done here to move this MR forward?