Databricks on GCP data exfiltration protection workspace deployment #172
Conversation
The commit contains the implementation for the workspace resource group. However this change requires to no more use the local.rg_location variable, since the value is known after the apply, and rhis force the replacement of all of the resources
Most of README files were already defined. TFDocs updated in each of them
|
@bhavink - wdyt? |
|
@alexott I do not think on GCP we need traditional hub/spoke based arch. Shared vpc based deployment is a common and popular arch where one could use vpc f/w rules along with vpc sc to prevent data exfiltration. TF support for CMv1 will be available by early March 2025 so may I suggest that we wait for it to be released and then update the GCP specific module? |
|
I agree about waiting for CMv1 migration |
There was a problem hiding this comment.
Pull Request Overview
This PR adds documentation to support the deployment of Databricks on GCP with data exfiltration protection using a Hub & Spoke network architecture while still using the CMv2 architecture.
- Added an example README for provisioning the workspace in the examples directory.
- Introduced a module README that details resource outcomes and the network architecture for the deployment.
Reviewed Changes
Copilot reviewed 21 out of 21 changed files in this pull request and generated 2 comments.
| File | Description |
|---|---|
| examples/gcp-with-psc-exfiltration-protection/README.md | New documentation for workspace provisioning using hub & spoke architecture |
| modules/gcp-with-psc-exfiltration-protection/README.md | Detailed module documentation including architecture and resource listings |
|
@micheledaddetta-databricks can you update the code to use provider >= 1.71 - it includes changes for CMv1 support |
|
@alexott I'll update it during next week |
Starting from provider version 1.71 CMv1 is supported for Databricks on GCP
|
@alexott here you can find updated code |
This is an initial implementation. I will enhance it in the future commits in order to include metastore admin assignment, workspace-metastore binding, catalog owner, catalog-workspace binding. In case the module can be built in order to be cloud agnostic.
alexott
left a comment
alexott
left a comment
There was a problem hiding this comment.
Minor changes required, like, update image
3 participants
The module still uses the CMv2 architecture. When the CMv1 architecture will be released and supported by Terraform provider, the implementation will be reviewed