Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

NTD: enrich mart tables with caltrans_district, handle ntd_id type #3608

Draft
wants to merge 1 commit into
base: main
Choose a base branch
from

Conversation

charlie-costanzo
Copy link
Member

Description

The PR enriches the newly ingested NTD data tables with caltrans_district to more easily enable analysis by caltrans district in dashboards and elsewhere.

Type of change

  • New feature

How has this been tested?

Screenshot 2024-12-19 at 4 35 58 PM

Post-merge follow-ups

  • Actions required (specified below)
    This enrichment should be further inspected to make sure it's best portraying california agency data

@charlie-costanzo charlie-costanzo added the data-pipeline-ingestion-and-modeling Ingesting, parsing and modeling data. Evan Siroky is product owner. label Dec 19, 2024
@charlie-costanzo charlie-costanzo self-assigned this Dec 19, 2024
Copy link

Warehouse report 📦

DAG

Legend (in order of precedence)

Resource type Indicator Resolution
Large table-materialized model Orange Make the model incremental
Large model without partitioning or clustering Orange Add partitioning and/or clustering
View with more than one child Yellow Materialize as a table or incremental
Incremental Light green
Table Green
View White

@charlie-costanzo charlie-costanzo force-pushed the ntd-enrich-marts-with-caltrans-district branch from b96b8c0 to 81afacb Compare January 14, 2025 15:15
dim_2022_agency_information AS (
SELECT
staging_agency_information.*,
dim_organizations.caltrans_district
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I don't think this is appropriate since we usually have used the bridge_organizations_x_headquarters_county_geography and dim_county_geography to join to the appropriate Caltrans District. The Caltrans District column within Airtable's Organizations table has been deprecated in favor of County Geography which contains Caltrans District.

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I don't think this is appropriate since we usually have used the bridge_organizations_x_headquarters_county_geography and dim_county_geography to join to the appropriate Caltrans District. The Caltrans District column within Airtable's Organizations table has been deprecated in favor of County Geography which contains Caltrans District.

Hey @evansiroky! Thank you so much for this feedback, this is exactly what I was hoping to validate next when I put up this draft PR.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
data-pipeline-ingestion-and-modeling Ingesting, parsing and modeling data. Evan Siroky is product owner.
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants