Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

ADAP-1129: Migrate dbt-snowflake into dbt-adapters #681

Merged
merged 831 commits into from
Jan 24, 2025

Conversation

mikealfare
Copy link
Contributor

Migrate dbt-snowflake into this repo using the following shell script and call:

repo=$1
source_branch=$2
target_branch=$3

# create a remote for the legacy adapter repo and fetch the latest commits
git remote remove old || true
git remote add old https://github.com/dbt-labs/$repo.git
git fetch old

# merge the updated branch from the legacy repo into the dbt-adapters repo
git checkout $target_branch
git merge old/$source_branch --allow-unrelated-histories

# remove the remote that was created by this process
git remote remove old || true
source ./scripts/migrate-adapter.sh dbt-snowflake monorepo/stage monorepo/migrate-dbt-snowflake

Update workflows to include dbt-snowflake.

Make updates related with code quality checks due to slightly different config.

Merge the last commit from dbt-snowflake/main into this branch and move the new files into the package subdirectory. Altered files merged as expected.

Merge this PR as a regular merge instead of a squash merge in order to retain git blame history.

emmyoop and others added 30 commits May 15, 2023 09:28
* more github deprecations fixes

* swap out abandonded action
* Add fix and simple test case.

* Add changelog and cleanup test

---------

Co-authored-by: Mila Page <[email protected]>
* Bump mypy from 1.2.0 to 1.3.0

Bumps [mypy](https://github.com/python/mypy) from 1.2.0 to 1.3.0.
- [Commits](python/mypy@v1.2.0...v1.3.0)

---
updated-dependencies:
- dependency-name: mypy
  dependency-type: direct:development
  update-type: version-update:semver-minor
...

Signed-off-by: dependabot[bot] <[email protected]>

* Add automated changelog yaml from template for bot PR

* fixed file format on dependencies type

---------

Signed-off-by: dependabot[bot] <[email protected]>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
Co-authored-by: Github Build Bot <[email protected]>
Co-authored-by: Mike Alfare <[email protected]>
Co-authored-by: Mike Alfare <[email protected]>
* add missing space

* remove newline at end

---------

Co-authored-by: Mike Alfare <[email protected]>
* swap out for reusable release branch testing

* cleanup workflow

* fix triage label workflow

* point to main
* bumping .latest branch variable in update_dependencies.sh to 1.5.latest

* updating env variable to 1.5.latest in nightly-release.yml

* created 1.5.0rc1 changelog (#566)

* updated changelog (#569)

* Bumping version to 1.5.0 and generate changelog

* Fix Issue URLs in 1.5.0 Changelog (#582)

* Fix 1.5.0 changelog links

* Patch changie for Spark->Snowflake

* fix regular expression for redaction of row values

redaction of row values did not work if value contained '\n' characters, eg in JSON, format because the regular expression would fail to detect such values and thus, the data would not get redacted. I added the newline character to the regular expression to fix this.

* finish rebase

* added changelog

---------

Co-authored-by: Github Build Bot <[email protected]>
Co-authored-by: Mike Alfare <[email protected]>
Co-authored-by: Kevin Wang <[email protected]>
Co-authored-by: Matthew McKnight <[email protected]>
* update RELEASE_BRANCH env

* drop support for python 3.7

* add changelog entry
* update RELEASE_BRANCH env

* adding link to 1.5 release notes

* run changie merge
* create feature branch, point to core feature branch, add changie

* ADAP-541: Stub out dynamic table (#616)

* stub jinja templates with a traditional table as proxy
* add dynamic_table as a relation type
* setup testing framework for dynamic tables

* updated dbt-snowflake to reflect changes in dbt-core

* ADAP-542: Add configuration options for dynamic tables (#636)

* add new config enums

* added refresh strategy query and relation method to determine differences

* added dynamic table ddl

* tests mostly pass, fail due to dynamic table being unavailable

* updated with materialized views pushed to main, added retry to get_row_count for dynamic table initialization

* lag > target_lag; remove refresh statement that wasn't doing anything

* fixed some typos, readability updates on tests
* update RELEASE_BRANCH env

* add clean_up methods to drop alt schemas made by tests not caught by defualt teardown method
* test constraints on model with sql header

* revert dev-requirements.txt changes

---------

Co-authored-by: Mike Alfare <[email protected]>
* Bump mypy from 1.3.0 to 1.4.0

Bumps [mypy](https://github.com/python/mypy) from 1.3.0 to 1.4.0.
- [Commits](python/mypy@v1.3.0...v1.4.0)

---
updated-dependencies:
- dependency-name: mypy
  dependency-type: direct:development
  update-type: version-update:semver-minor
...

Signed-off-by: dependabot[bot] <[email protected]>

* Add automated changelog yaml from template for bot PR

---------

Signed-off-by: dependabot[bot] <[email protected]>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
Co-authored-by: Github Build Bot <[email protected]>
Co-authored-by: Mike Alfare <[email protected]>
* Update tox requirement from ~=4.4 to ~=4.6

Updates the requirements on [tox](https://github.com/tox-dev/tox) to permit the latest version.
- [Release notes](https://github.com/tox-dev/tox/releases)
- [Changelog](https://github.com/tox-dev/tox/blob/main/docs/changelog.rst)
- [Commits](tox-dev/tox@4.4.0...4.6.0)

---
updated-dependencies:
- dependency-name: tox
  dependency-type: direct:development
...

Signed-off-by: dependabot[bot] <[email protected]>

* Add automated changelog yaml from template for bot PR

* remove pyversion restrictions post py37 retirement

---------

Signed-off-by: dependabot[bot] <[email protected]>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
Co-authored-by: Github Build Bot <[email protected]>
Co-authored-by: Mike Alfare <[email protected]>
Co-authored-by: Mike Alfare <[email protected]>
* Bump mypy from 1.4.0 to 1.4.1

Bumps [mypy](https://github.com/python/mypy) from 1.4.0 to 1.4.1.
- [Commits](python/mypy@v1.4.0...v1.4.1)

---
updated-dependencies:
- dependency-name: mypy
  dependency-type: direct:development
  update-type: version-update:semver-patch
...

Signed-off-by: dependabot[bot] <[email protected]>

* Add automated changelog yaml from template for bot PR

---------

Signed-off-by: dependabot[bot] <[email protected]>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
Co-authored-by: Github Build Bot <[email protected]>
* Update pytest requirement from ~=7.3 to ~=7.4

Updates the requirements on [pytest](https://github.com/pytest-dev/pytest) to permit the latest version.
- [Release notes](https://github.com/pytest-dev/pytest/releases)
- [Changelog](https://github.com/pytest-dev/pytest/blob/main/CHANGELOG.rst)
- [Commits](pytest-dev/pytest@7.3.0...7.4.0)

---
updated-dependencies:
- dependency-name: pytest
  dependency-type: direct:development
...

Signed-off-by: dependabot[bot] <[email protected]>

* Add automated changelog yaml from template for bot PR

---------

Signed-off-by: dependabot[bot] <[email protected]>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
Co-authored-by: Github Build Bot <[email protected]>
Co-authored-by: Mike Alfare <[email protected]>
* Update types-requests requirement from ~=2.28 to ~=2.31

Updates the requirements on [types-requests](https://github.com/python/typeshed) to permit the latest version.
- [Commits](https://github.com/python/typeshed/commits)

---
updated-dependencies:
- dependency-name: types-requests
  dependency-type: direct:development
...

Signed-off-by: dependabot[bot] <[email protected]>

* Add automated changelog yaml from template for bot PR

* added missing space

---------

Signed-off-by: dependabot[bot] <[email protected]>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
Co-authored-by: Github Build Bot <[email protected]>
Co-authored-by: Mike Alfare <[email protected]>
Co-authored-by: Mike Alfare <[email protected]>
* Update pytest-xdist requirement from ~=3.2 to ~=3.3

Updates the requirements on [pytest-xdist](https://github.com/pytest-dev/pytest-xdist) to permit the latest version.
- [Changelog](https://github.com/pytest-dev/pytest-xdist/blob/master/CHANGELOG.rst)
- [Commits](pytest-dev/pytest-xdist@v3.2.0...v3.3.0)

---
updated-dependencies:
- dependency-name: pytest-xdist
  dependency-type: direct:development
...

Signed-off-by: dependabot[bot] <[email protected]>

* Add automated changelog yaml from template for bot PR

---------

Signed-off-by: dependabot[bot] <[email protected]>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
Co-authored-by: Mike Alfare <[email protected]>
Co-authored-by: Github Build Bot <[email protected]>
* Update pre-commit requirement from ~=2.21 to ~=3.3

Updates the requirements on [pre-commit](https://github.com/pre-commit/pre-commit) to permit the latest version.
- [Release notes](https://github.com/pre-commit/pre-commit/releases)
- [Changelog](https://github.com/pre-commit/pre-commit/blob/main/CHANGELOG.md)
- [Commits](https://github.com/pre-commit/pre-commit/commits/v3.3.0)

---
updated-dependencies:
- dependency-name: pre-commit
  dependency-type: direct:development
...

Signed-off-by: dependabot[bot] <[email protected]>

* Add automated changelog yaml from template for bot PR

---------

Signed-off-by: dependabot[bot] <[email protected]>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
Co-authored-by: Mike Alfare <[email protected]>
Co-authored-by: Github Build Bot <[email protected]>
…ests (#681)

* update RELEASE_BRANCH env

* remove dependent_projects_dict from utils.py
* change on catalog.sql in case big information_schema

* adding log change

---------

Co-authored-by: Florian Eiden <[email protected]>
Co-authored-by: Mike Alfare <[email protected]>
* Update pip-tools requirement from ~=6.13 to ~=6.14

Updates the requirements on [pip-tools](https://github.com/jazzband/pip-tools) to permit the latest version.
- [Release notes](https://github.com/jazzband/pip-tools/releases)
- [Changelog](https://github.com/jazzband/pip-tools/blob/main/CHANGELOG.md)
- [Commits](jazzband/pip-tools@6.13.0...6.14.0)

---
updated-dependencies:
- dependency-name: pip-tools
  dependency-type: direct:development
...

Signed-off-by: dependabot[bot] <[email protected]>

* Add automated changelog yaml from template for bot PR

---------

Signed-off-by: dependabot[bot] <[email protected]>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
Co-authored-by: Github Build Bot <[email protected]>
Co-authored-by: Mike Alfare <[email protected]>
@mikealfare
Copy link
Contributor Author

Integration tests run against this branch: https://github.com/dbt-labs/dbt-adapters/actions/runs/12956948840/job/36144339933

@mikealfare mikealfare marked this pull request as ready for review January 24, 2025 20:52
@mikealfare mikealfare requested a review from a team as a code owner January 24, 2025 20:52
@mikealfare mikealfare merged commit aa29250 into main Jan 24, 2025
284 of 298 checks passed
@mikealfare mikealfare deleted the monorepo/migrate-dbt-snowflake branch January 24, 2025 23:45
colin-rogers-dbt pushed a commit that referenced this pull request Feb 3, 2025
Signed-off-by: dependabot[bot] <[email protected]>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
cla:yes The PR author has signed the CLA
Projects
None yet
Development

Successfully merging this pull request may close these issues.