Version 0.9.0
[v0.9.0] - 2022-09-13
New Platform Support
Databricks
✨ Staging
✨ Hubs
✨ Links
✨ Satellites
Postgres**
✨ Staging
✨ Hubs
✨ Links
✨ Satellites
**There are currently limitations on Postgres. Click here for more details
New Features
Introducing:
🌟 Payload column exclusion: Satellite's payload can now be configured to select all columns, except a user-defined list using an exclude configuration (#128)
🌟 Null Business Key Handling: Now users can provide a configuration in their staging tables to handle NULL keys elegantly, according to business needs (#133)
🌟 Extra Columns Parameter: All table macros now provide a src_extra_columns parameter which allows users to add extra columns outside of the standard template for business needs.
🌟 More logging: Hubs and Links now provide additional logging about the number of sources they are loading from. Minor but helpful! This is our first step towards giving our users more information.
Fixes
✅ Fixed an edge case for vault_insert_by_period when the staging table and the target table were in different databases (#121)
✅ Removed the uppercase conversion in the staging macro (#122, #134)
✅ Fixed an issue where duplicate records (same hashdiff) would sometimes be loaded into a Satellite (#126)
✅ Disabled automatic column name escaping in derived columns when using the stage() macro. Escaping can now be configured on a case-by-case basis for each column to escape when they are reserved words etc. (#114, #141)
Breaking changes
Behind the scenes
- Re-factor PIT and Bridge macros to ensure better maintainability and readability
Docs
- Moved stage configuration details to a new page
- Updated packages behind the scenes for security and bug fixes
- Created landing pages for sections, which should make navigation easier, e.g. getting started is now the home page when clicking 'Tutorials' in the menu, instead of having to click twice.
With thanks to:
Thank you to the following people/teams for making this release possible and working closely with the dbtvault team over the last few months to give the community an initial set of macros for new platform support! Fantastic work. We are proud to have such a great community behind us.
@kk-ais - Thank you to KK and the AIS team for the initial Databricks support.
@johnoscott - Thank you for the hard work on initial Postgres support.
@eamontaaffe - Migrate flags.FULL_REFRESH to should_full_refresh() macro #154
Honourable Mentions
Thank you to @NikkaZ and their team for their eagerness for Databricks support and their support, advice and patience anticipating its release.
