-
There are two main ways to develop with dbt: using the web-based IDE in dbt Cloud or using the command-line interface (CLI) in dbt Core:
-
—
-
dbt Cloud IDE dbt Cloud is a web-based application that allows you to develop dbt projects with the IDE, includes a purpose-built scheduler, and provides an easier way to share your dbt documentation with your team. The IDE is a faster and more reliable way to deploy your dbt models and provides a real-time editing and execution environment for your dbt project.
-
—
-
dbt Core CLI The command line interface (CLI) uses
dbt Core, an
open-source software that’s freely available. You can build your dbt project in a code editor, like Jetbrains or VSCode, and run dbt commands from the command line.
+
You can develop dbt using the web-based IDE in dbt Cloud or on the command line interface using the dbt Cloud CLI or open-source dbt Core, all of which enable you to execute dbt commands. The key distinction between the dbt Cloud CLI and dbt Core is the dbt Cloud CLI is tailored for dbt Cloud's infrastructure and integrates with all its features.
+
—
+
dbt Cloud IDE: dbt Cloud is a web-based application that allows you to develop dbt projects with the IDE, includes a purpose-built scheduler, and provides an easier way to share your dbt documentation with your team. The IDE is a faster and more reliable way to deploy your dbt models and provides a real-time editing and execution environment for your dbt project.
+
—
+
dbt Cloud CLI: The dbt Cloud CLI allows you to run dbt commands against your dbt Cloud development environment from your local command line or code editor. It supports cross-project ref, speedier, lower-cost builds, automatic deferral of build artifacts, and more.
+
—
+
dbt Core: dbt Core is an
open-sourced software that’s freely available. You can build your dbt project in a code editor, and run dbt commands from the command line.
diff --git a/website/docs/docs/cloud/dbt-cloud-ide/ide-user-interface.md b/website/docs/docs/cloud/dbt-cloud-ide/ide-user-interface.md
index de643413a8a..05910b23e7f 100644
--- a/website/docs/docs/cloud/dbt-cloud-ide/ide-user-interface.md
+++ b/website/docs/docs/cloud/dbt-cloud-ide/ide-user-interface.md
@@ -36,11 +36,13 @@ The IDE streamlines your workflow, and features a popular user interface layout
* Added (A) — The IDE detects added files
* Deleted (D) — The IDE detects deleted files.
-
+
5. **Command bar —** The Command bar, located in the lower left of the IDE, is used to invoke [dbt commands](/reference/dbt-commands). When a command is invoked, the associated logs are shown in the Invocation History Drawer.
-6. **IDE Status button —** The IDE Status button, located on the lower right of the IDE, displays the current IDE status. If there is an error in the status or in the dbt code that stops the project from parsing, the button will turn red and display "Error". If there aren't any errors, the button will display a green "Ready" status. To access the [IDE Status modal](#modals-and-menus), simply click on this button.
+6. **Defer to production —** The **Defer to production** toggle allows developers to only build and run and test models they've edited without having to first run and build all the models that come before them (upstream parents). Refer to [Using defer in dbt Cloud](/docs/cloud/about-cloud-develop-defer#defer-in-the-dbt-cloud-ide) for more info.
+
+7. **Status button —** The IDE Status button, located on the lower right of the IDE, displays the current IDE status. If there is an error in the status or in the dbt code that stops the project from parsing, the button will turn red and display "Error". If there aren't any errors, the button will display a green "Ready" status. To access the [IDE Status modal](#modals-and-menus), simply click on this button.
## Editing features
diff --git a/website/docs/docs/cloud/dbt-cloud-ide/lint-format.md b/website/docs/docs/cloud/dbt-cloud-ide/lint-format.md
index 8ffd83ef00e..6a86f1aa14b 100644
--- a/website/docs/docs/cloud/dbt-cloud-ide/lint-format.md
+++ b/website/docs/docs/cloud/dbt-cloud-ide/lint-format.md
@@ -45,7 +45,11 @@ With the dbt Cloud IDE, you can seamlessly use [SQLFluff](https://sqlfluff.com/)
- Works with Jinja and SQL,
- Comes with built-in [linting rules](https://docs.sqlfluff.com/en/stable/rules.html). You can also [customize](#customize-linting) your own linting rules.
- Empowers you to [enable linting](#enable-linting) with options like **Lint** (displays linting errors and recommends actions) or **Fix** (auto-fixes errors in the IDE).
-- Displays a **Code Quality** tab to view code errors, and provides code quality visibility and management.
+- Displays a **Code Quality** tab to view code errors, and provides code quality visibility and management.
+
+:::info Ephemeral models not supported
+Linting doesn't support ephemeral models in dbt v1.5 and lower. Refer to the [FAQs](#faqs) for more info.
+:::
### Enable linting
@@ -223,6 +227,12 @@ Currently, running SQLFluff commands from the terminal isn't supported.
Make sure you're on a development branch. Formatting or Linting isn't available on "main" or "read-only" branches.
+
+Why is there inconsistent SQLFluff behavior when running outside the dbt Cloud IDE (such as a GitHub Action)?
+— Double-check your SQLFluff version matches the one in dbt Cloud IDE (found in the Code Quality tab after a lint operation).
+— If your lint operation passes despite clear rule violations, confirm you're not linting models with ephemeral models. Linting doesn't support ephemeral models in dbt v1.5 and lower.
+
+
## Related docs
- [User interface](/docs/cloud/dbt-cloud-ide/ide-user-interface)
diff --git a/website/docs/docs/cloud/git/authenticate-azure.md b/website/docs/docs/cloud/git/authenticate-azure.md
index 03020ccca73..42028bf993b 100644
--- a/website/docs/docs/cloud/git/authenticate-azure.md
+++ b/website/docs/docs/cloud/git/authenticate-azure.md
@@ -3,10 +3,11 @@ title: "Authenticate with Azure DevOps"
id: "authenticate-azure"
description: "dbt Cloud developers need to authenticate with Azure DevOps."
sidebar_label: "Authenticate with Azure DevOps"
+pagination_next: null
---
-If you use the dbt Cloud IDE to collaborate on your team's Azure DevOps dbt repo, you need to [link your dbt Cloud profile to Azure DevOps](#link-your-dbt-cloud-profile-to-azure-devops), which provides an extra layer of authentication.
+If you use the dbt Cloud IDE or dbt Cloud CLI to collaborate on your team's Azure DevOps dbt repo, you need to [link your dbt Cloud profile to Azure DevOps](#link-your-dbt-cloud-profile-to-azure-devops), which provides an extra layer of authentication.
## Link your dbt Cloud profile to Azure DevOps
diff --git a/website/docs/docs/cloud/git/connect-azure-devops.md b/website/docs/docs/cloud/git/connect-azure-devops.md
index bc5bb81dd24..c138e042abc 100644
--- a/website/docs/docs/cloud/git/connect-azure-devops.md
+++ b/website/docs/docs/cloud/git/connect-azure-devops.md
@@ -1,6 +1,7 @@
---
title: "Connect to Azure DevOps"
id: "connect-azure-devops"
+pagination_next: "docs/cloud/git/setup-azure"
---
@@ -13,7 +14,7 @@ Connect your Azure DevOps cloud account in dbt Cloud to unlock new product exper
- Import new Azure DevOps repos with a couple clicks during dbt Cloud project setup.
- Clone repos using HTTPS rather than SSH
- Enforce user authorization with OAuth 2.0.
-- Carry Azure DevOps user repository permissions (read / write access) through to dbt Cloud IDE's git actions.
+- Carry Azure DevOps user repository permissions (read / write access) through to dbt Cloud IDE or dbt Cloud CLI's git actions.
- Trigger Continuous integration (CI) builds when pull requests are opened in Azure DevOps.
diff --git a/website/docs/docs/cloud/git/connect-github.md b/website/docs/docs/cloud/git/connect-github.md
index 771e4286ef6..ff0f2fff18f 100644
--- a/website/docs/docs/cloud/git/connect-github.md
+++ b/website/docs/docs/cloud/git/connect-github.md
@@ -74,7 +74,7 @@ To connect a personal GitHub account:
4. Once you approve authorization, you will be redirected to dbt Cloud, and you should now see your connected account.
-The next time you log into dbt Cloud, you will be able to do so via OAuth through GitHub, and if you're on the Enterprise plan, you're ready to use the dbt Cloud IDE.
+The next time you log into dbt Cloud, you will be able to do so via OAuth through GitHub, and if you're on the Enterprise plan, you're ready to use the dbt Cloud IDE or dbt Cloud CLI.
## FAQs
diff --git a/website/docs/docs/cloud/git/connect-gitlab.md b/website/docs/docs/cloud/git/connect-gitlab.md
index 53fde5f4878..e55552e2d86 100644
--- a/website/docs/docs/cloud/git/connect-gitlab.md
+++ b/website/docs/docs/cloud/git/connect-gitlab.md
@@ -8,7 +8,7 @@ id: "connect-gitlab"
Connecting your GitLab account to dbt Cloud provides convenience and another layer of security to dbt Cloud:
- Import new GitLab repos with a couple clicks during dbt Cloud project setup.
- Clone repos using HTTPS rather than SSH.
-- Carry GitLab user permissions through to dbt Cloud IDE's git actions.
+- Carry GitLab user permissions through to dbt Cloud or dbt Cloud CLI's git actions.
- Trigger [Continuous integration](/docs/deploy/continuous-integration) builds when merge requests are opened in GitLab.
The steps to integrate GitLab in dbt Cloud depend on your plan. If you are on:
@@ -35,7 +35,7 @@ Once you've accepted, you should be redirected back to dbt Cloud, and you'll see
dbt Cloud enterprise customers have the added benefit of bringing their own GitLab OAuth application to dbt Cloud. This tier benefits from extra security, as dbt Cloud will:
- Enforce user authorization with OAuth.
-- Carry GitLab's user repository permissions (read / write access) through to dbt Cloud IDE's git actions.
+- Carry GitLab's user repository permissions (read / write access) through to dbt Cloud or dbt Cloud CLI's git actions.
In order to connect GitLab in dbt Cloud, a GitLab account admin must:
1. [Set up a GitLab OAuth application](#setting-up-a-gitlab-oauth-application).
@@ -97,7 +97,7 @@ You will then be redirected to GitLab and prompted to sign into your account. Gi
Once you've accepted, you should be redirected back to dbt Cloud, and your integration is ready for developers on your team to [personally authenticate with](#personally-authenticating-with-gitlab).
### Personally authenticating with GitLab
-dbt Cloud developers on the Enterprise plan must each connect their GitLab profiles to dbt Cloud, as every developer's read / write access for the dbt repo is checked in the dbt Cloud IDE.
+dbt Cloud developers on the Enterprise plan must each connect their GitLab profiles to dbt Cloud, as every developer's read / write access for the dbt repo is checked in the dbt Cloud IDE or dbt Cloud CLI.
To connect a personal GitLab account, dbt Cloud developers should navigate to Your Profile settings by clicking the gear icon in the top right, then select **Linked Accounts** in the left menu.
@@ -105,7 +105,7 @@ If your GitLab account is not connected, you’ll see "No connected account". Se
-Once you approve authorization, you will be redirected to dbt Cloud, and you should see your connected account. You're now ready to start developing in the dbt Cloud IDE.
+Once you approve authorization, you will be redirected to dbt Cloud, and you should see your connected account. You're now ready to start developing in the dbt Cloud IDE or dbt Cloud CLI.
## Troubleshooting
diff --git a/website/docs/docs/cloud/git/git-configuration-in-dbt-cloud.md b/website/docs/docs/cloud/git/git-configuration-in-dbt-cloud.md
new file mode 100644
index 00000000000..fb8c0186236
--- /dev/null
+++ b/website/docs/docs/cloud/git/git-configuration-in-dbt-cloud.md
@@ -0,0 +1,37 @@
+---
+title: "Git configuration in dbt Cloud"
+description: "Learn about the Git providers supported in dbt Cloud"
+pagination_next: "docs/cloud/git/import-a-project-by-git-url"
+pagination_prev: null
+---
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
\ No newline at end of file
diff --git a/website/docs/docs/cloud/git/import-a-project-by-git-url.md b/website/docs/docs/cloud/git/import-a-project-by-git-url.md
index ba53baa33ea..83846bb1f0b 100644
--- a/website/docs/docs/cloud/git/import-a-project-by-git-url.md
+++ b/website/docs/docs/cloud/git/import-a-project-by-git-url.md
@@ -1,6 +1,8 @@
---
title: "Import a project by git URL"
id: "import-a-project-by-git-url"
+pagination_next: "docs/cloud/git/connect-github"
+pagination_prev: null
---
In dbt Cloud, you can import a git repository from any valid git URL that points to a dbt project. There are some important considerations to keep in mind when doing this.
diff --git a/website/docs/docs/cloud/git/setup-azure.md b/website/docs/docs/cloud/git/setup-azure.md
index 9eca77d7014..843371be6ea 100644
--- a/website/docs/docs/cloud/git/setup-azure.md
+++ b/website/docs/docs/cloud/git/setup-azure.md
@@ -93,7 +93,7 @@ Once you connect your Azure AD app and Azure DevOps, you need to provide dbt Clo
- **Directory(tenant) ID:** Found in the Azure AD App.
-Your Azure AD app should now be added to your dbt Cloud Account. People on your team who want to develop in dbt Cloud's IDE can now personally [authorize Azure DevOps from their profiles](/docs/cloud/git/authenticate-azure).
+Your Azure AD app should now be added to your dbt Cloud Account. People on your team who want to develop in the dbt Cloud IDE or dbt Cloud CLI can now personally [authorize Azure DevOps from their profiles](/docs/cloud/git/authenticate-azure).
## Connect a service user
diff --git a/website/docs/docs/cloud/manage-access/about-access.md b/website/docs/docs/cloud/manage-access/about-access.md
index f9f97bc555d..d394c79baa3 100644
--- a/website/docs/docs/cloud/manage-access/about-access.md
+++ b/website/docs/docs/cloud/manage-access/about-access.md
@@ -2,6 +2,8 @@
title: "About user access in dbt Cloud"
description: "Learn how dbt Cloud administrators can use dbt Cloud's permissioning model to control user-level access in a dbt Cloud account."
id: "about-user-access"
+pagination_next: "docs/cloud/manage-access/seats-and-users"
+pagination_prev: null
---
:::info "User access" is not "Model access"
diff --git a/website/docs/docs/cloud/manage-access/audit-log.md b/website/docs/docs/cloud/manage-access/audit-log.md
index 98bf660b259..b90bceef570 100644
--- a/website/docs/docs/cloud/manage-access/audit-log.md
+++ b/website/docs/docs/cloud/manage-access/audit-log.md
@@ -3,6 +3,8 @@ title: "The audit log for dbt Cloud Enterprise"
id: audit-log
description: "You can troubleshoot possible issues and provide security audits by reviewing event activity in your organization."
sidebar_label: "Audit log"
+pagination_next: null
+pagination_prev: "docs/cloud/manage-access/about-user-access"
---
To review actions performed by people in your organization, dbt provides logs of audited user and system events in real time. The audit log appears as events happen and includes details such as who performed the action, what the action was, and when it was performed. You can use these details to troubleshoot access issues, perform security audits, or analyze specific events.
diff --git a/website/docs/docs/cloud/manage-access/cloud-seats-and-users.md b/website/docs/docs/cloud/manage-access/cloud-seats-and-users.md
index 04dfbe093c3..24c64a5abed 100644
--- a/website/docs/docs/cloud/manage-access/cloud-seats-and-users.md
+++ b/website/docs/docs/cloud/manage-access/cloud-seats-and-users.md
@@ -3,6 +3,8 @@ title: "Users and licenses"
description: "Learn how dbt Cloud administrators can use licenses and seats to control access in a dbt Cloud account."
id: "seats-and-users"
sidebar: "Users and licenses"
+pagination_next: "docs/cloud/manage-access/self-service-permissions"
+pagination_prev: null
---
In dbt Cloud, _licenses_ are used to allocate users to your account. There are three different types of licenses in dbt Cloud:
@@ -16,6 +18,7 @@ The user's assigned license determines the specific capabilities they can access
| Functionality | Developer User | Read-Only Users | IT Users* |
| ------------- | -------------- | --------------- | -------- |
| Use the dbt Cloud IDE | ✅ | ❌ | ❌ |
+| Use the dbt Cloud CLI | ✅ | ❌ | ❌ |
| Use Jobs | ✅ | ❌ | ❌ |
| Manage Account | ✅ | ❌ | ✅ |
| API Access | ✅ | ❌ | ❌ |
diff --git a/website/docs/docs/cloud/manage-access/enterprise-permissions.md b/website/docs/docs/cloud/manage-access/enterprise-permissions.md
index 5bf3623b105..dcacda20deb 100644
--- a/website/docs/docs/cloud/manage-access/enterprise-permissions.md
+++ b/website/docs/docs/cloud/manage-access/enterprise-permissions.md
@@ -3,6 +3,7 @@ title: "Enterprise permissions"
id: "enterprise-permissions"
description: "Permission sets for Enterprise plans."
hide_table_of_contents: true #For the sake of the tables on this page
+pagination_next: null
---
import Permissions from '/snippets/_enterprise-permissions-table.md';
@@ -21,10 +22,6 @@ The following roles and permission sets are available for assignment in dbt Clou
-## Diagram of the permission sets
-
-
-
## How to set up RBAC Groups in dbt Cloud
Role-Based Access Control (RBAC) is helpful for automatically assigning permissions to dbt admins based on their SSO provider group associations.
diff --git a/website/docs/docs/cloud/manage-access/self-service-permissions.md b/website/docs/docs/cloud/manage-access/self-service-permissions.md
index 21cc765b76d..d3c9cf8f5ea 100644
--- a/website/docs/docs/cloud/manage-access/self-service-permissions.md
+++ b/website/docs/docs/cloud/manage-access/self-service-permissions.md
@@ -12,7 +12,8 @@ The permissions afforded to each role are described below:
| ------ | ------ | ----- |
| View and edit resources | ✅ | ✅ |
| Trigger runs | ✅ | ✅ |
-| Access the IDE | ✅ | ✅ |
+| Access the dbt Cloud IDE | ✅ | ✅ |
+| Access the dbt Cloud CLI | ✅ | ✅ |
| Invite Members to the account | ✅ | ✅ |
| Manage billing | ❌ | ✅ |
| Manage team permissions | ❌ | ✅ |
diff --git a/website/docs/docs/cloud/manage-access/set-up-bigquery-oauth.md b/website/docs/docs/cloud/manage-access/set-up-bigquery-oauth.md
index 516a340c951..87018b14d56 100644
--- a/website/docs/docs/cloud/manage-access/set-up-bigquery-oauth.md
+++ b/website/docs/docs/cloud/manage-access/set-up-bigquery-oauth.md
@@ -1,7 +1,8 @@
---
title: "Set up BigQuery OAuth"
-description: "Learn how dbt Cloud administrators can use licenses and seats to control access in a dbt Cloud account."
+description: "Learn how dbt Cloud administrators can use BigQuery OAuth to control access in a dbt Cloud account"
id: "set-up-bigquery-oauth"
+pagination_next: null
---
:::info Enterprise Feature
@@ -73,3 +74,7 @@ You will then be redirected to BigQuery and asked to approve the drive, cloud pl
Select **Allow**. This redirects you back to dbt Cloud. You should now be an authenticated BigQuery user, ready to use the dbt Cloud IDE.
+
+## FAQs
+
+
diff --git a/website/docs/docs/cloud/manage-access/set-up-databricks-oauth.md b/website/docs/docs/cloud/manage-access/set-up-databricks-oauth.md
new file mode 100644
index 00000000000..679133b7844
--- /dev/null
+++ b/website/docs/docs/cloud/manage-access/set-up-databricks-oauth.md
@@ -0,0 +1,77 @@
+---
+title: "Set up Databricks OAuth"
+description: "Learn how dbt Cloud administrators can use Databricks OAuth to control access in a dbt Cloud account."
+id: "set-up-databricks-oauth"
+---
+
+:::info Enterprise Feature
+
+This guide describes a feature of the dbt Cloud Enterprise plan. If you’re interested in learning more about an Enterprise plan, contact us at sales@getdbt.com.
+
+:::
+
+dbt Cloud supports developer OAuth ([OAuth for partner solutions](https://docs.databricks.com/en/integrations/manage-oauth.html)) with Databricks, providing an additional layer of security for dbt enterprise users. When you enable Databricks OAuth for a dbt Cloud project, all dbt Cloud developers must authenticate with Databricks in order to use the dbt Cloud IDE. The project's deployment environments will still leverage the Databricks authentication method set at the environment level.
+
+:::tip Beta Feature
+
+Databricks OAuth support in dbt Cloud is a [beta feature](/docs/dbt-versions/product-lifecycles#dbt-cloud) and subject to change without notification. More updates to this feature coming soon.
+
+Current limitations:
+- Databrick's OAuth applications are in public preview
+- The current experience requires the IDE to be restarted every hour (access tokens expire after 1 hour - [workaround](https://docs.databricks.com/en/integrations/manage-oauth.html#override-the-default-token-lifetime-policy-for-dbt-core-power-bi-or-tableau-desktop))
+
+:::
+
+### Configure Databricks OAuth (Databricks admin)
+
+To get started, you will need to [add dbt as an OAuth application](https://docs.databricks.com/en/integrations/configure-oauth-dbt.html) with Databricks, in 2 steps:
+
+1. From your terminal, [authenticate to the Databricks Account API](https://docs.databricks.com/en/integrations/configure-oauth-dbt.html#authenticate-to-the-account-api) with the Databricks CLI. You authenticate using:
+ - OAuth for users ([prerequisites](https://docs.databricks.com/en/dev-tools/auth.html#oauth-u2m-auth))
+ - Oauth for service principals ([prerequisites](https://docs.databricks.com/en/dev-tools/auth.html#oauth-m2m-auth))
+ - Username and password (must be account admin)
+2. In the same terminal, **add dbt Cloud as an OAuth application** using `curl` and the [OAuth Custom App Integration API](https://docs.databricks.com/api/account/customappintegration/create)
+
+For the second step, you can use this example `curl` to authenticate with your username and password, replacing values as defined in the following table:
+
+```shell
+curl -u USERNAME:PASSWORD https://accounts.cloud.databricks.com/api/2.0/accounts/ACCOUNT_ID/oauth2/custom-app-integrations -d '{"redirect_urls": ["https://YOUR_ACCESS_URL", "https://YOUR_ACCESS_URL/complete/databricks"], "confidential": true, "name": "NAME", "scopes": ["sql", "offline_access"]}'
+```
+
+These parameters and descriptions will help you authenticate with your username and password:
+
+| Parameter | Description |
+| ------ | ----- |
+| **USERNAME** | Your Databricks username (account admin level) |
+| **PASSWORD** | Your Databricks password (account admin level) |
+| **ACCOUNT_ID** | Your Databricks [account ID](https://docs.databricks.com/en/administration-guide/account-settings/index.html#locate-your-account-id) |
+| **YOUR_ACCESS_URL** | The [appropriate Access URL](/docs/cloud/about-cloud/regions-ip-addresses) for your dbt Cloud account region and plan |
+| **NAME** | The integration name (i.e 'databricks-dbt-cloud')
+
+After running the `curl`, you'll get an API response that includes the `client_id` and `client_secret` required in the following section. At this time, this is the only way to retrieve the secret. If you lose the secret, then the integration needs to be [deleted](https://docs.databricks.com/api/account/customappintegration/delete) and re-created.
+
+
+### Configure the Connection in dbt Cloud (dbt Cloud project admin)
+
+Now that you have an OAuth app set up in Databricks, you'll need to add the client ID and secret to dbt Cloud. To do so:
+ - go to Settings by clicking the gear in the top right.
+ - on the left, select **Projects** under **Account Settings**
+ - choose your project from the list
+ - select **Connection** to edit the connection details
+ - add the `OAuth Client ID` and `OAuth Client Secret` from the Databricks OAuth app under the **Optional Settings** section
+
+
+
+### Authenticating to Databricks (dbt Cloud IDE developer)
+
+Once the Databricks connection via OAuth is set up for a dbt Cloud project, each dbt Cloud user will need to authenticate with Databricks in order to use the IDE. To do so:
+
+- Click the gear icon at the top right and select **Profile settings**.
+- Select **Credentials**.
+- Choose your project from the list
+- Select `OAuth` as the authentication method, and click **Save**
+- Finalize by clicking the **Connect Databricks Account** button
+
+
+
+You will then be redirected to Databricks and asked to approve the connection. This redirects you back to dbt Cloud. You should now be an authenticated Databricks user, ready to use the dbt Cloud IDE.
diff --git a/website/docs/docs/cloud/manage-access/set-up-sso-azure-active-directory.md b/website/docs/docs/cloud/manage-access/set-up-sso-azure-active-directory.md
index 349c3d8ecd7..28d20b526db 100644
--- a/website/docs/docs/cloud/manage-access/set-up-sso-azure-active-directory.md
+++ b/website/docs/docs/cloud/manage-access/set-up-sso-azure-active-directory.md
@@ -144,7 +144,7 @@ To complete setup, follow the steps below in the dbt Cloud application.
| ----- | ----- |
| **Log in with** | Azure AD Single Tenant |
| **Client ID** | Paste the **Application (client) ID** recorded in the steps above |
-| **Client Secret** | Paste the **Client Secret** (remember to use the Secret Value instead of the Secret ID) recorded in the steps above |
+| **Client Secret** | Paste the **Client Secret** (remember to use the Secret Value instead of the Secret ID) recorded in the steps above;
**Note:** When the client secret expires, an Azure AD admin will have to generate a new one to be pasted into dbt Cloud for uninterrupted application access. |
| **Tenant ID** | Paste the **Directory (tenant ID)** recorded in the steps above |
| **Domain** | Enter the domain name for your Azure directory (such as `fishtownanalytics.com`). Only use the primary domain; this won't block access for other domains. |
| **Slug** | Enter your desired login slug. Users will be able to log into dbt Cloud by navigating to `https://YOUR_ACCESS_URL/enterprise-login/LOGIN-SLUG`, replacing `YOUR_ACCESS_URL` with the [appropriate Access URL](/docs/cloud/manage-access/sso-overview#auth0-multi-tenant-uris) for your region and plan. Login slugs must be unique across all dbt Cloud accounts, so pick a slug that uniquely identifies your company. |
diff --git a/website/docs/docs/cloud/manage-access/set-up-sso-okta.md b/website/docs/docs/cloud/manage-access/set-up-sso-okta.md
index 5ec70443d1f..4079cc488c4 100644
--- a/website/docs/docs/cloud/manage-access/set-up-sso-okta.md
+++ b/website/docs/docs/cloud/manage-access/set-up-sso-okta.md
@@ -171,7 +171,7 @@ configured in the steps above.
| **Log in with** | Okta |
| **Identity Provider SSO Url** | Paste the **Identity Provider Single Sign-On URL** shown in the Okta setup instructions |
| **Identity Provider Issuer** | Paste the **Identity Provider Issuer** shown in the Okta setup instructions |
-| **X.509 Certificate** | Paste the **X.509 Certificate** shown in the Okta setup instructions |
+| **X.509 Certificate** | Paste the **X.509 Certificate** shown in the Okta setup instructions;
**Note:** When the certificate expires, an Okta admin will have to generate a new one to be pasted into dbt Cloud for uninterrupted application access. |
| **Slug** | Enter your desired login slug. Users will be able to log into dbt Cloud by navigating to `https://YOUR_ACCESS_URL/enterprise-login/LOGIN-SLUG`, replacing `YOUR_ACCESS_URL` with the [appropriate Access URL](/docs/cloud/about-cloud/regions-ip-addresses) for your region and plan. Login slugs must be unique across all dbt Cloud accounts, so pick a slug that uniquely identifies your company. |
**Note:** When the certificate expires, an Idp admin will have to generate a new one to be pasted into dbt Cloud for uninterrupted application access. |
| Slug | Enter your desired login slug. |
diff --git a/website/docs/docs/cloud/manage-access/sso-overview.md b/website/docs/docs/cloud/manage-access/sso-overview.md
index 7e44859c73a..f613df7907e 100644
--- a/website/docs/docs/cloud/manage-access/sso-overview.md
+++ b/website/docs/docs/cloud/manage-access/sso-overview.md
@@ -1,7 +1,8 @@
---
-title: "SSO Overview"
+title: "Single sign-on (SSO) Overview"
id: "sso-overview"
-
+pagination_next: "docs/cloud/manage-access/set-up-sso-saml-2.0"
+pagination_prev: null
---
This overview explains how users are provisioned in dbt Cloud via Single Sign-On (SSO).
diff --git a/website/docs/docs/cloud/secure/databricks-privatelink.md b/website/docs/docs/cloud/secure/databricks-privatelink.md
index c136cd8a0f9..a2c9e208459 100644
--- a/website/docs/docs/cloud/secure/databricks-privatelink.md
+++ b/website/docs/docs/cloud/secure/databricks-privatelink.md
@@ -3,6 +3,7 @@ title: "Configuring Databricks PrivateLink"
id: databricks-privatelink
description: "Configuring PrivateLink for Databricks"
sidebar_label: "PrivateLink for Databricks"
+pagination_next: null
---
The following steps will walk you through the setup of a Databricks AWS PrivateLink endpoint in the dbt Cloud multi-tenant environment.
diff --git a/website/docs/docs/cloud/secure/ip-restrictions.md b/website/docs/docs/cloud/secure/ip-restrictions.md
index 237de991c02..093d2a1c876 100644
--- a/website/docs/docs/cloud/secure/ip-restrictions.md
+++ b/website/docs/docs/cloud/secure/ip-restrictions.md
@@ -3,6 +3,8 @@ title: "Configuring IP restrictions"
id: ip-restrictions
description: "Configuring IP restrictions to outside traffic from accessing your dbt Cloud environment"
sidebar_label: "IP restrictions"
+pagination_next: "docs/cloud/secure/about-privatelink"
+pagination_prev: null
---
import SetUpPages from '/snippets/_available-tiers-iprestrictions.md';
diff --git a/website/docs/docs/cloud/secure/secure-your-tenant.md b/website/docs/docs/cloud/secure/secure-your-tenant.md
new file mode 100644
index 00000000000..95cb8adffba
--- /dev/null
+++ b/website/docs/docs/cloud/secure/secure-your-tenant.md
@@ -0,0 +1,49 @@
+---
+title: "Secure your tenant"
+description: "Learn how to secure your tenant for dbt Cloud"
+pagination_next: "docs/cloud/secure/ip-restrictions"
+pagination_prev: null
+---
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
\ No newline at end of file
diff --git a/website/docs/docs/collaborate/cloud-build-and-view-your-docs.md b/website/docs/docs/collaborate/cloud-build-and-view-your-docs.md
index 36f4781bfde..b387c64788f 100644
--- a/website/docs/docs/collaborate/cloud-build-and-view-your-docs.md
+++ b/website/docs/docs/collaborate/cloud-build-and-view-your-docs.md
@@ -2,6 +2,7 @@
title: "Build and view your docs with dbt Cloud"
id: "build-and-view-your-docs"
description: "Automatically generate project documentation as you run jobs."
+pagination_next: null
---
dbt enables you to generate documentation for your project and data warehouse, and renders the documentation in a website. For more information, see [Documentation](/docs/collaborate/documentation).
@@ -39,16 +40,17 @@ To create and schedule documentation-only jobs at the end of your production job
You configure project documentation to generate documentation when the job you set up in the previous section runs. In the project settings, specify the job that generates documentation artifacts for that project. Once you configure this setting, subsequent runs of the job will automatically include a step to generate documentation.
1. Click the gear icon in the top right.
-2. Select **Projects** and click the project that needs documentation.
-3. Click **Edit**.
-4. Under "Artifacts," select the job that should generate docs when it runs.
+2. Select **Account Settings**.
+3. Navigate to **Projects** and select the project that needs documentation.
+4. Click **Edit**.
+5. Under **Artifacts**, select the job that should generate docs when it runs.
-5. Click **Save**.
+6. Click **Save**.
## Generating documentation
-To generate documentation in the IDE, run the `dbt docs generate` command in the
-Command Bar in the IDE. This command will generate the Docs for your dbt project as it exists in development in your IDE session.
+To generate documentation in the dbt Cloud IDE, run the `dbt docs generate` command in the
+Command Bar in the dbt Cloud IDE. This command will generate the Docs for your dbt project as it exists in development in your IDE session.
diff --git a/website/docs/docs/collaborate/collaborate-with-others.md b/website/docs/docs/collaborate/collaborate-with-others.md
new file mode 100644
index 00000000000..7875a8044b6
--- /dev/null
+++ b/website/docs/docs/collaborate/collaborate-with-others.md
@@ -0,0 +1,38 @@
+---
+title: "Collaborate with others"
+description: "Learn how dbt Cloud makes it easier to collaborate with others"
+pagination_next: "docs/collaborate/explore-projects"
+pagination_prev: null
+---
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
\ No newline at end of file
diff --git a/website/docs/docs/collaborate/documentation.md b/website/docs/docs/collaborate/documentation.md
index 429b5187152..0fa00c7cca2 100644
--- a/website/docs/docs/collaborate/documentation.md
+++ b/website/docs/docs/collaborate/documentation.md
@@ -2,6 +2,8 @@
title: "About documentation"
description: "Learn how good documentation for your dbt models helps stakeholders discover and understand your datasets."
id: "documentation"
+pagination_next: "docs/collaborate/build-and-view-your-docs"
+pagination_prev: null
---
## Related documentation
diff --git a/website/docs/docs/collaborate/explore-projects.md b/website/docs/docs/collaborate/explore-projects.md
index a4c914259ef..b041cd0c915 100644
--- a/website/docs/docs/collaborate/explore-projects.md
+++ b/website/docs/docs/collaborate/explore-projects.md
@@ -1,25 +1,16 @@
---
-title: "Explore your dbt projects (beta)"
-sidebar_label: "Explore dbt projects (beta)"
+title: "Explore your dbt projects"
+sidebar_label: "Explore dbt projects"
description: "Learn about dbt Explorer and how to interact with it to understand, improve, and leverage your data pipelines."
+pagination_next: null
+pagination_prev: null
---
-With dbt Explorer, you can view your project's [resources](/docs/build/projects) (such as models, tests, and metrics) and their
lineage to gain a better understanding of its latest production state. Navigate and manage your projects within dbt Cloud to help your data consumers discover and leverage your dbt resources.
+With dbt Explorer, you can view your project's [resources](/docs/build/projects) (such as models, tests, and metrics) and their
lineage to gain a better understanding of its latest production state. Navigate and manage your projects within dbt Cloud to help you and other data developers, analysts, and consumers discover and leverage your dbt resources.
-To display the details about your [project state](/docs/dbt-cloud-apis/project-state), dbt Explorer utilizes the metadata provided through the [Discovery API](/docs/dbt-cloud-apis/discovery-api). The metadata that's available on your project depends on the [deployment environment](/docs/deploy/deploy-environments) you've designated as _production_ in your dbt Cloud project. dbt Explorer automatically retrieves the metadata updates after each job run in the production deployment environment so it will always have the latest state on your project. The metadata it displays depends on the [commands executed by the jobs](/docs/deploy/job-commands). For instance:
+:::tip Public preview
-- To update model details or results, you must run `dbt run` or `dbt build` on a given model within a job in the environment.
-- To view catalog statistics and columns, you must run `dbt docs generate` within a job in the environment.
-- To view test results, you must run `dbt test` or `dbt build` within a job in the environment.
-- To view source freshness check results, you must run `dbt source freshness` within a job in the environment.
-
-The need to run these commands will diminish, and richer, more timely metadata will become available as the Discovery API and its underlying platform evolve.
-
-:::tip Join the beta
-
-dbt Explorer is a [beta feature](/docs/dbt-versions/product-lifecycles#dbt-cloud) and subject to change without notification. More updates to this feature coming soon.
-
-If you’re interested in joining the beta, please contact your account team.
+Try dbt Explorer! It's available in [Public Preview](/docs/dbt-versions/product-lifecycles#dbt-cloud) as of October 17, 2023 for dbt Cloud customers. More updates coming soon.
:::
@@ -28,115 +19,218 @@ If you’re interested in joining the beta, please contact your account team.
- You have a [multi-tenant](/docs/cloud/about-cloud/tenancy#multi-tenant) or AWS single-tenant dbt Cloud account on the [Team or Enterprise plan](https://www.getdbt.com/pricing/).
- You have set up a [production deployment environment](/docs/deploy/deploy-environments#set-as-production-environment-beta) for each project you want to explore.
- There has been at least one successful job run in the production deployment environment.
-- You are on the dbt Explorer page. This requires the feature to be enabled for your account.
- - To go to the page, select **Explore (Beta)** from the top navigation bar in dbt Cloud.
+- You are on the dbt Explorer page. To do this, select **Explore** from the top navigation bar in dbt Cloud.
+
+
+## Generate metadata
+
+dbt Explorer uses the metadata provided by the [Discovery API](/docs/dbt-cloud-apis/discovery-api) to display the details about [the state of your project](/docs/dbt-cloud-apis/project-state). The metadata that's available depends on the [deployment environment](/docs/deploy/deploy-environments) you've designated as _production_ in your dbt Cloud project. dbt Explorer automatically retrieves the metadata updates after each job run in the production deployment environment so it always has the latest results for your project.
+
+To view a resource and its metadata, you must define the resource in your project and run a job in the production environment. The resulting metadata depends on the [commands executed by the jobs](/docs/deploy/job-commands).
+
+For a richer experience with dbt Explorer, you must:
+
+- Run [dbt run](/reference/commands/run) or [dbt build](/reference/commands/build) on a given model within a job in the environment to update model details or results.
+- Run [dbt docs generate](/reference/commands/cmd-docs) within a job in the environment to view catalog statistics and columns for models, sources, and snapshots.
+- Run [dbt test](/reference/commands/test) or [dbt build](/reference/commands/build) within a job in the environment to view test results.
+- Run [dbt source freshness](/reference/commands/source#dbt-source-freshness) within a job in the environment to view source freshness data.
+- Run [dbt snapshot](/reference/commands/snapshot) or [dbt build](/reference/commands/build) within a job in the environment to view snapshot details.
+
+Richer and more timely metadata will become available as dbt, the Discovery API, and the underlying dbt Cloud platform evolves.
-## Explore the project’s lineage
+## Explore your project's lineage graph {#project-lineage}
-dbt Explorer provides a visualization of your project’s
DAG that you can interact with. To start, select **Overview** in the left sidebar and click the **Explore Lineage** button on the main (center) section of the page.
+dbt Explorer provides a visualization of your project’s
DAG that you can interact with. To access the project's full lineage graph, select **Overview** in the left sidebar and click the **Explore Lineage** button on the main (center) section of the page.
-If you don't see the lineage graph immediately, click **Render Lineage**. It can take some time for the graph to render depending on the size of your project and your computer’s available memory. The graph of very large projects might not render so, instead, you can select a subset of nodes by using selectors.
+If you don't see the project lineage graph immediately, click **Render Lineage**. It can take some time for the graph to render depending on the size of your project and your computer’s available memory. The graph of very large projects might not render so you can select a subset of nodes by using selectors, instead.
-The nodes in the lineage graph represent the project’s resources and the edges represent the relationships between the nodes. Resources like tests and macros display in the lineage within their [resource details pages](#view-resource-details) but not within the overall project lineage graph. Nodes are color-coded and include iconography according to their resource type.
+The nodes in the lineage graph represent the project’s resources and the edges represent the relationships between the nodes. Nodes are color-coded and include iconography according to their resource type.
-To interact with the lineage graph, you can:
+To explore the lineage graphs of tests and macros, view [their resource details pages](#view-resource-details). By default, dbt Explorer excludes these resources from the full lineage graph unless a search query returns them as results.
+
+To interact with the full lineage graph, you can:
- Hover over any item in the graph to display the resource’s name and type.
- Zoom in and out on the graph by mouse-scrolling.
-- Grab and move the graph.
-- Click on a resource to highlight its relationship with other resources in your project.
-- [Search and select specific resources](#search-resources) or a subset of the DAG using selectors and lineage (for example, `+[YOUR_RESOURCE_NAME]` displays all nodes upstream of a particular resource).
-- [View resource details](#view-resource-details) by selecting a node in the graph (double-clicking).
+- Grab and move the graph and the nodes.
+- Select a resource to highlight its relationship with other resources in your project. A panel opens on the graph’s right-hand side that displays a high-level summary of the resource’s details. The side panel includes a **General** tab for information like description, materialized type, and other details.
+ - Click the Share icon in the side panel to copy the graph’s link to your clipboard.
+ - Click the View Resource icon in the side panel to [view the resource details](#view-resource-details).
+- [Search and select specific resources](#search-resources) or a subset of the DAG using selectors and graph operators. For example:
+ - `+[RESOURCE_NAME]` — Displays all parent nodes of the resource
+ - `resource_type:model [RESOURCE_NAME]` — Displays all models matching the name search
+- [View resource details](#view-resource-details) by selecting a node (double-clicking) in the graph.
+- Click the List view icon in the graph's upper right corner to return to the main **Explore** page.
-
+
## Search for resources {#search-resources}
-With the search bar (on the upper left of the page or in a lineage graph), you can search using keywords or selectors (also known as *selector methods*). The resources that match your search criteria will display as a table in the main section of the page. When you select a resource in the table, its [resource details page](#view-resource-details) will display.
+With the search bar (on the upper left corner of the page or in a lineage graph), you can search with keywords or by using [node selection syntax](/reference/node-selection/syntax). The resources that match your search criteria will display as a lineage graph and a table in the main section of the page.
+
+Select a node (single-click) in the lineage graph to highlight its relationship with your other search results and to display which project contains the resource's definition. When you choose a node (double-click) in the lineage graph or when you select a resource in the table, dbt Explorer displays the [resource's details page](#view-resource-details).
-When using keyword search, dbt Explorer will search through your resources using metadata such as resource type, resource name, column name, source name, tags, schema, database, version, alias/identifier, and package name.
+### Search with keywords
+When searching with keywords, dbt Explorer searches through your resource metadata (such as resource type, resource name, column name, source name, tags, schema, database, version, alias/identifier, and package name) and returns any matches.
-When using selector search, you can utilize the dbt node selection syntax including set and graph operators (like `+`). To learn more about selectors, refer to [Syntax overview](/reference/node-selection/syntax), [Graph operators](/reference/node-selection/graph-operators), and [Set operators](/reference/node-selection/set-operators).
+### Search with selector methods
-Below are the selection methods currently available in dbt Explorer. For more information about each of them, refer to [Methods](/reference/node-selection/methods).
+You can search with [selector methods](/reference/node-selection/methods). Below are the selectors currently available in dbt Explorer:
-- **fqn:** — Find resources by [file or fully qualified name](/reference/node-selection/methods#the-file-or-fqn-method).
-- **source:** — Find resources by a specified [source](/reference/node-selection/methods#the-source-method).
-- **resource_type:** — Find resources by their [type](/reference/node-selection/methods#the-resource_type-method).
-- **package:** — Find resources by the [dbt package](/reference/node-selection/methods#the-package-method) that defines them.
-- **tag:** — Find resources by a specified [tag](/reference/node-selection/methods#the-tag-method).
+- `fqn:` — Find resources by [file or fully qualified name](/reference/node-selection/methods#the-fqn-method). This selector is the search bar's default. If you want to use the default, it's unnecessary to add `fqn:` before the search term.
+- `source:` — Find resources by a specified [source](/reference/node-selection/methods#the-source-method).
+- `resource_type:` — Find resources by their [type](/reference/node-selection/methods#the-resource_type-method).
+- `package:` — Find resources by the [dbt package](/reference/node-selection/methods#the-package-method) that defines them.
+- `tag:` — Find resources by a specified [tag](/reference/node-selection/methods#the-tag-method).
-- **group:** — Find models defined within a specified [group](/reference/node-selection/methods#the-group-method).
-- **access:** — Find models based on their [access](/reference/node-selection/methods#the-access-method) property.
+- `group:` — Find models defined within a specified [group](/reference/node-selection/methods#the-group-method).
+- `access:` — Find models based on their [access](/reference/node-selection/methods#the-access-method) property.
-
+### Search with graph operators
+
+You can use [graph operators](/reference/node-selection/graph-operators) on keywords or selector methods. For example, `+orders` returns all the parents of `orders`.
+
+### Search with set operators
+
+You can use multiple selector methods in your search query with [set operators](/reference/node-selection/set-operators). A space implies a union set operator and a comma for an intersection. For example:
+- `resource_type:metric,tag:nightly` — Returns metrics with the tag `nightly`
+- `+snowplow_sessions +fct_orders` — Returns resources that are parent nodes of either `snowplow_sessions` or `fct_orders`
-## Use the catalog sidebar
+### Search with both keywords and selector methods
-By default, the catalog sidebar lists all your project’s resources. Select any resource type in the list and all those resources in the project will display as a table in the main section of the page. For a description on the different resource types (like models, metrics, and so on), refer to [About dbt projects](https://docs.getdbt.com/docs/build/projects).
+You can use keyword search to highlight results that are filtered by the selector search. For example, if you don't have a resource called `customers`, then `resource_type:metric customers` returns all the metrics in your project and highlights those that are related to the term `customers` in the name, in a column, tagged as customers, and so on.
+
+When searching in this way, the selectors behave as filters that you can use to narrow the search and keywords as a way to find matches within those filtered results.
+
+
+
+## Browse with the sidebar
+
+By default, the catalog sidebar lists all your project’s resources. Select any resource type in the list and all those resources in the project will display as a table in the main section of the page. For a description on the different resource types (like models, metrics, and so on), refer to [About dbt projects](/docs/build/projects).
To browse using a different view, you can choose one of these options from the **View by** dropdown:
- **Resources** (default) — All resources in the project organized by type.
-- **Packages** — All resources in the project organized by the project in which they are defined.
+- **Packages** — All resources in the project organized by the dbt package in which they are defined.
- **File Tree** — All resources in the project organized by the file in which they are defined. This mirrors the file tree in your dbt project repository.
-- **Database** — All resources in the project organized by the database and schema in which they are built. This mirrors your data platform structure.
+- **Database** — All resources in the project organized by the database and schema in which they are built. This mirrors your data platform's structure that represents the [applied state](/docs/dbt-cloud-apis/project-state) of your project.
+
+
-
+## View model versions
+
+If models in the project are versioned, you can see which [version of the model](/docs/collaborate/govern/model-versions) is being applied — `prerelease`, `latest`, and `old` — in the title of the model’s details page and in the model list from the sidebar.
## View resource details {#view-resource-details}
-You can view the definition and latest run results of any resource in your project. To find a resource and view its details, you can interact with the lineage graph, use search, or browse the catalog. The details (metadata) available to you depends on the resource’s type, its definition, and the [commands](/docs/deploy/job-commands) run within jobs in the production environment.
+You can view the definition and latest run results of any resource in your project. To find a resource and view its details, you can interact with the lineage graph, use search, or browse the catalog.
-
+The details (metadata) available to you depends on the resource’s type, its definition, and the [commands](/docs/deploy/job-commands) that run within jobs in the production environment.
+
### Example of model details
An example of the details you might get for a model:
-- **General** — The model’s lineage graph that you can interact with.
-- **Code** — The source code and compiled code for the model.
-- **Columns** — The available columns in the model.
-- **Description** — A [description of the model](/docs/collaborate/documentation#adding-descriptions-to-your-project).
-- **Recent** — Information on the last time the model ran, how long it ran for, whether the run was successful, the job ID, and the run ID.
-- **Tests** — [Tests](/docs/build/tests) for the model.
-- **Details** — Key properties like the model’s relation name (for example, how it’s represented and how you can query it in the data platform: `database.schema.identifier`); model governance attributes like access, group, and if contracted; and more.
-- **Relationships** — The nodes the model **Depends On** and is **Referenced by.**
+- Status bar (below the page title) — Information on the last time the model ran, whether the run was successful, how the data is materialized, number of rows, and the size of the model.
+- **General** tab includes:
+ - **Lineage** graph — The model’s lineage graph that you can interact with. The graph includes one parent node and one child node from the model. Click the Expand icon in the graph's upper right corner to view the model in full lineage graph mode.
+ - **Description** section — A [description of the model](/docs/collaborate/documentation#adding-descriptions-to-your-project).
+ - **Recent** section — Information on the last time the model ran, how long it ran for, whether the run was successful, the job ID, and the run ID.
+ - **Tests** section — [Tests](/docs/build/tests) for the model.
+ - **Details** section — Key properties like the model’s relation name (for example, how it’s represented and how you can query it in the data platform: `database.schema.identifier`); model governance attributes like access, group, and if contracted; and more.
+ - **Relationships** section — The nodes the model **Depends On**, is **Referenced by**, and (if applicable) is **Used by** for projects that have declared the models' project as a dependency.
+- **Code** tab — The source code and compiled code for the model.
+- **Columns** tab — The available columns in the model. This tab also shows tests results (if any) that you can select to view the test's details page. A :white_check_mark: denotes a passing test.
+
### Example of exposure details
An example of the details you might get for an exposure:
-- **Status** — The status on data freshness and data quality.
-- **Lineage** — The exposure’s lineage graph.
-- **Description** — A description of the exposure.
-- **Details** — Details like exposure type, maturity, owner information, and more.
-- **Relationships** — The nodes the exposure **Depends On**.
+- Status bar (below the page title) — Information on the last time the exposure was updated.
+- **General** tab includes:
+ - **Status** section — The status on data freshness and data quality.
+ - **Lineage** graph — The exposure’s lineage graph. Click the Expand icon in the graph's upper right corner to view the exposure in full lineage graph mode.
+ - **Description** section — A description of the exposure.
+ - **Details** section — Details like exposure type, maturity, owner information, and more.
+ - **Relationships** section — The nodes the exposure **Depends On**.
### Example of test details
An example of the details you might get for a test:
-- **General** — The test’s lineage graph that you can interact with.
-- **Code** — The source code and compiled code for the test.
-- **Description** — A description of the test.
-- **Recent** — Information on the last time the test ran, how long it ran for, whether the test passed, the job ID, and the run ID.
-- **Details** — Details like schema, severity, package, and more.
-- **Relationships** — The nodes the test **Depends On**.
+- Status bar (below the page title) — Information on the last time the test ran, whether the test passed, test name, test target, and column name.
+- **General** tab includes:
+ - **Lineage** graph — The test’s lineage graph that you can interact with. The graph includes one parent node and one child node from the test resource. Click the Expand icon in the graph's upper right corner to view the test in full lineage graph mode.
+ - **Description** section — A description of the test.
+ - **Recent** section — Information on the last time the test ran, how long it ran for, whether the test passed, the job ID, and the run ID.
+ - **Details** section — Details like schema, severity, package, and more.
+ - **Relationships** section — The nodes the test **Depends On**.
+- **Code** tab — The source code and compiled code for the test.
+
### Example of source details
An example of the details you might get for each source table within a source collection:
-- **General** — The source’s lineage graph that you can interact with.
-- **Columns** — The available columns in the source.
-- **Description** — A description of the source.
-- **Source freshness** — Information on whether refreshing the data was successful, the last time the source was loaded, the timestamp of when a run generated data, and the run ID.
-- **Details** — Details like database, schema, and more.
-- **Relationships** — A table that lists all the sources used with their freshness status, the timestamp of when freshness was last checked, and the timestamp of when the source was last loaded.
\ No newline at end of file
+- Status bar (below the page title) — Information on the last time the source was updated and the number of tables the source uses.
+- **General** tab includes:
+ - **Lineage** graph — The source’s lineage graph that you can interact with. The graph includes one parent node and one child node from the source. Click the Expand icon in the graph's upper right corner to view the source in full lineage graph mode.
+ - **Description** section — A description of the source.
+ - **Source freshness** section — Information on whether refreshing the data was successful, the last time the source was loaded, the timestamp of when a run generated data, and the run ID.
+ - **Details** section — Details like database, schema, and more.
+ - **Relationships** section — A table that lists all the sources used with their freshness status, the timestamp of when freshness was last checked, and the timestamp of when the source was last loaded.
+- **Columns** tab — The available columns in the source. This tab also shows tests results (if any) that you can select to view the test's details page. A :white_check_mark: denotes a passing test.
+
+## About project-level lineage
+You can also view all the different projects and public models in the account, where the public models are defined, and how they are used to gain a better understanding about your cross-project resources.
+
+When viewing the resource-level lineage graph for a given project that uses cross-project references, you can see cross-project relationships represented in the DAG. The iconography is slightly different depending on whether you're viewing the lineage of an upstream producer project or a downstream consumer project.
+
+When viewing an upstream (parent) project that produces public models that are imported by downstream (child) projects, public models will have a counter icon in their upper right corner that indicates the number of projects that declare the current project as a dependency. Selecting that model reveals the lineage to show the specific projects that are dependent on this model. Projects show up in this counter if they declare the parent project as a dependency in its `dependencies.yml` regardless of whether or not there's a direct `{{ ref() }}` against the public model. Selecting a project node from a public model opens the resource-level lineage graph for that project, which is subject to your permissions.
+
+
+
+When viewing a downstream (child) project that imports and refs public models from upstream (parent) projects, public models will show up in the lineage graph and display an icon on the graph edge that indicates what the relationship is to a model from another project. Hovering over this icon indicates the specific dbt Cloud project that produces that model. Double-clicking on a model from another project opens the resource-level lineage graph of the parent project, which is subject to your permissions.
+
+
+
+
+### Explore the project-level lineage graph
+
+For cross-project collaboration, you can interact with the DAG in all the same ways as described in [Explore your project's lineage](#project-lineage) but you can also interact with it at the project level and view the details.
+
+To get a list view of all the projects, select the account name at the top of the **Explore** page near the navigation bar. This view includes a public model list, project list, and a search bar for project searches. You can also view the project-level lineage graph by clicking the Lineage view icon in the page's upper right corner.
+
+If you have permissions for a project in the account, you can view all public models used across the entire account. However, you can only view full public model details and private models if you have permissions for a project where the models are defined.
+
+From the project-level lineage graph, you can:
+
+- Click the Lineage view icon (in the graph’s upper right corner) to view the cross-project lineage graph.
+- Click the List view icon (in the graph’s upper right corner) to view the project list.
+ - Select a project from the **Projects** tab to switch to that project’s main **Explore** page.
+ - Select a model from the **Public Models** tab to view the [model’s details page](#view-resource-details).
+ - Perform searches on your projects with the search bar.
+- Select a project node in the graph (double-clicking) to switch to that particular project’s lineage graph.
+
+When you select a project node in the graph, a project details panel opens on the graph’s right-hand side where you can:
+
+- View counts of the resources defined in the project.
+- View a list of its public models, if any.
+- View a list of other projects that uses the project, if any.
+- Click **Open Project Lineage** to switch to the project’s lineage graph.
+- Click the Share icon to copy the project panel link to your clipboard so you can share the graph with someone.
+
+
+
+## Related content
+- [Enterprise permissions](/docs/cloud/manage-access/enterprise-permissions)
+- [About model governance](/docs/collaborate/govern/about-model-governance)
+- [What is data mesh?](https://www.getdbt.com/blog/what-is-data-mesh-the-definition-and-importance-of-data-mesh) blog
diff --git a/website/docs/docs/collaborate/git-version-control.md b/website/docs/docs/collaborate/git-version-control.md
index 4444f381bb5..392e2c3baa5 100644
--- a/website/docs/docs/collaborate/git-version-control.md
+++ b/website/docs/docs/collaborate/git-version-control.md
@@ -3,6 +3,8 @@ title: "About git"
id: git-version-control
description: "Git overview"
sidebar_label: "About git"
+pagination_next: "docs/collaborate/git/version-control-basics"
+pagination_prev: null
---
A [version control](https://en.wikipedia.org/wiki/Version_control) system allows you and your teammates to work collaboratively, safely, and simultaneously on a single project. Version control helps you track all the code changes made in your dbt project.
@@ -22,3 +24,4 @@ When you develop in the command line interface (CLI) or Cloud integrated develo
- [Merge conflicts](/docs/collaborate/git/merge-conflicts)
- [Connect to GitHub](/docs/cloud/git/connect-github)
- [Connect to GitLab](/docs/cloud/git/connect-gitlab)
+- [Connect to Azure DevOps](/docs/cloud/git/connect-azure-devops)
diff --git a/website/docs/docs/collaborate/git/merge-conflicts.md b/website/docs/docs/collaborate/git/merge-conflicts.md
index b109cacb511..c3c19b1e2a1 100644
--- a/website/docs/docs/collaborate/git/merge-conflicts.md
+++ b/website/docs/docs/collaborate/git/merge-conflicts.md
@@ -1,6 +1,7 @@
---
title: "Merge conflicts"
id: "merge-conflicts"
+pagination_next: null
---
[Merge conflicts](https://docs.github.com/en/pull-requests/collaborating-with-pull-requests/addressing-merge-conflicts/about-merge-conflicts) in the [dbt Cloud IDE](/docs/cloud/dbt-cloud-ide/develop-in-the-cloud) often occur when multiple users are simultaneously making edits to the same section in the same file. This makes it difficult for Git to decide what changes to incorporate in the final merge.
diff --git a/website/docs/docs/collaborate/govern/about-model-governance.md b/website/docs/docs/collaborate/govern/about-model-governance.md
index efeb2836bc6..bbc430845d2 100644
--- a/website/docs/docs/collaborate/govern/about-model-governance.md
+++ b/website/docs/docs/collaborate/govern/about-model-governance.md
@@ -2,6 +2,8 @@
title: "About model governance"
id: about-model-governance
description: "Information about new features related to model governance"
+pagination_next: "docs/collaborate/govern/model-access"
+pagination_prev: null
---
diff --git a/website/docs/docs/collaborate/govern/model-access.md b/website/docs/docs/collaborate/govern/model-access.md
index 64b70416a2f..765e833ac0c 100644
--- a/website/docs/docs/collaborate/govern/model-access.md
+++ b/website/docs/docs/collaborate/govern/model-access.md
@@ -25,7 +25,7 @@ The two concepts will be closely related, as we develop multi-project collaborat
## Related documentation
* [`groups`](/docs/build/groups)
-* [`access`](/reference/resource-properties/access)
+* [`access`](/reference/resource-configs/access)
## Groups
diff --git a/website/docs/docs/collaborate/govern/project-dependencies.md b/website/docs/docs/collaborate/govern/project-dependencies.md
index 1dbc967e74e..9a1d8b59b68 100644
--- a/website/docs/docs/collaborate/govern/project-dependencies.md
+++ b/website/docs/docs/collaborate/govern/project-dependencies.md
@@ -3,18 +3,17 @@ title: "Project dependencies"
id: project-dependencies
sidebar_label: "Project dependencies"
description: "Reference public models across dbt projects"
+pagination_next: null
---
-:::caution Closed Beta - dbt Cloud Enterprise
-"Project" dependencies and cross-project `ref` are features of dbt Cloud Enterprise, currently in Closed Beta. To access these features while they are in beta, please contact your account team at dbt Labs.
+:::info Available in Public Preview for dbt Cloud Enterprise accounts
-**Prerequisites:** In order to add project dependencies and resolve cross-project `ref`, you must:
-- Have the feature enabled (speak to your account team)
-- Use dbt v1.6 for **both** the upstream ("producer") project and the downstream ("consumer") project.
-- Have a deployment environment in the upstream ("producer") project [that is set to be your production environment](/docs/deploy/deploy-environments#set-as-production-environment-beta)
-- Have a successful run of the upstream ("producer") project
+Project dependencies and cross-project `ref` are features available in [dbt Cloud Enterprise](https://www.getdbt.com/pricing), currently in [Public Preview](/docs/dbt-versions/product-lifecycles#dbt-cloud).
+
+Enterprise users can use these features by designating a [public model](/docs/collaborate/govern/model-access) and adding a [cross-project ref](#how-to-use-ref).
:::
+
For a long time, dbt has supported code reuse and extension by installing other projects as [packages](/docs/build/packages). When you install another project as a package, you are pulling in its full source code, and adding it to your own. This enables you to call macros and run models defined in that other project.
While this is a great way to reuse code, share utility macros, and establish a starting point for common transformations, it's not a great way to enable collaboration across teams and at scale, especially at larger organizations.
@@ -23,6 +22,33 @@ This year, dbt Labs is introducing an expanded notion of `dependencies` across m
- **Packages** — Familiar and pre-existing type of dependency. You take this dependency by installing the package's full source code (like a software library).
- **Projects** — A _new_ way to take a dependency on another project. Using a metadata service that runs behind the scenes, dbt Cloud resolves references on-the-fly to public models defined in other projects. You don't need to parse or run those upstream models yourself. Instead, you treat your dependency on those models as an API that returns a dataset. The maintainer of the public model is responsible for guaranteeing its quality and stability.
+
+Starting in dbt v1.6 or higher, `packages.yml` has been renamed to `dependencies.yml`. However, if you need use Jinja within your packages config, such as an environment variable for your private package, you need to keep using `packages.yml` for your packages for now. Refer to the [FAQs](#faqs) for more info.
+
+## Prerequisites
+
+In order to add project dependencies and resolve cross-project `ref`, you must:
+- Use dbt v1.6 or higher for **both** the upstream ("producer") project and the downstream ("consumer") project.
+- Have a deployment environment in the upstream ("producer") project [that is set to be your production environment](/docs/deploy/deploy-environments#set-as-production-environment-beta)
+- Have a successful run of the upstream ("producer") project
+- Have a multi-tenant or single-tenant [dbt Cloud Enterprise](https://www.getdbt.com/pricing) account (Azure ST is not supported but coming soon)
+
+
## Example
As an example, let's say you work on the Marketing team at the Jaffle Shop. The name of your team's project is `jaffle_marketing`:
@@ -36,7 +62,7 @@ name: jaffle_marketing
As part of your modeling of marketing data, you need to take a dependency on two other projects:
-- `dbt_utils` as a [package](#packages-use-case): An collection of utility macros that you can use while writing the SQL for your own models. This package is, open-source public, and maintained by dbt Labs.
+- `dbt_utils` as a [package](#packages-use-case): A collection of utility macros that you can use while writing the SQL for your own models. This package is, open-source public, and maintained by dbt Labs.
- `jaffle_finance` as a [project use-case](#projects-use-case): Data models about the Jaffle Shop's revenue. This project is private and maintained by your colleagues on the Finance team. You want to select from some of this project's final models, as a starting point for your own work.
@@ -66,7 +92,7 @@ When you're building on top of another team's work, resolving the references in
- You don't need to mirror any conditional configuration of the upstream project such as `vars`, environment variables, or `target.name`. You can reference them directly wherever the Finance team is building their models in production. Even if the Finance team makes changes like renaming the model, changing the name of its schema, or [bumping its version](/docs/collaborate/govern/model-versions), your `ref` would still resolve successfully.
- You eliminate the risk of accidentally building those models with `dbt run` or `dbt build`. While you can select those models, you can't actually build them. This prevents unexpected warehouse costs and permissions issues. This also ensures proper ownership and cost allocation for each team's models.
-### Usage
+### How to use ref
**Writing `ref`:** Models referenced from a `project`-type dependency must use [two-argument `ref`](/reference/dbt-jinja-functions/ref#two-argument-variant), including the project name:
@@ -87,6 +113,8 @@ with monthly_revenue as (
**Cycle detection:** Currently, "project" dependencies can only go in one direction, meaning that the `jaffle_finance` project could not add a new model that depends, in turn, on `jaffle_marketing.roi_by_channel`. dbt will check for cycles across projects and raise errors if any are detected. We are considering support for this pattern in the future, whereby dbt would still check for node-level cycles while allowing cycles at the project level.
+For more guidance on how to use dbt Mesh, refer to the dedicated [dbt Mesh guide](/guides/best-practices/how-we-mesh/mesh-1-intro).
+
### Comparison
If you were to instead install the `jaffle_finance` project as a `package` dependency, you would instead be pulling down its full source code and adding it to your runtime environment. This means:
@@ -99,4 +127,16 @@ There are a few cases where installing another internal project as a package can
- Unified deployments — In a production environment, if the central data platform team of Jaffle Shop wanted to schedule the deployment of models across both `jaffle_finance` and `jaffle_marketing`, they could use dbt's [selection syntax](/reference/node-selection/syntax) to create a new "passthrough" project that installed both projects as packages.
- Coordinated changes — In development, if you wanted to test the effects of a change to a public model in an upstream project (`jaffle_finance.monthly_revenue`) on a downstream model (`jaffle_marketing.roi_by_channel`) _before_ introducing changes to a staging or production environment, you can install the `jaffle_finance` package as a package within `jaffle_marketing`. The installation can point to a specific git branch, however, if you find yourself frequently needing to perform end-to-end testing across both projects, we recommend you re-examine if this represents a stable interface boundary.
-These are the exceptions, rather than the rule. Installing another team's project as a package adds complexity, latency, and risk of unnecessary costs. By defining clear interface boundaries across teams, by serving one team's public models as "APIs" to another, and by enabling practitioners to develop with a more narrowly-defined scope, we can enable more people to contribute, with more confidence, while requiring less context upfront.
+These are the exceptions, rather than the rule. Installing another team's project as a package adds complexity, latency, and risk of unnecessary costs. By defining clear interface boundaries across teams, by serving one team's public models as "APIs" to another, and by enabling practitioners to develop with a more narrowly defined scope, we can enable more people to contribute, with more confidence, while requiring less context upfront.
+
+## FAQs
+
+
+Can I define private packages in the dependencies.yml
file?
+
+If you're using private packages with the [git token method](/docs/build/packages#git-token-method), you must define them in the `packages.yml` file instead of the `dependencies.yml` file. This is because conditional rendering (like Jinja-in-yaml) is not supported.
+
+
+
+## Related docs
+- Refer to the [dbt Mesh](/guides/best-practices/how-we-mesh/mesh-1-intro) guide for more guidance on how to use dbt Mesh.
diff --git a/website/docs/docs/community-adapters.md b/website/docs/docs/community-adapters.md
index 87d1bd4981e..444ea0e04b4 100644
--- a/website/docs/docs/community-adapters.md
+++ b/website/docs/docs/community-adapters.md
@@ -11,10 +11,10 @@ Community adapters are adapter plugins contributed and maintained by members of
| [Clickhouse](/docs/core/connect-data-platform/clickhouse-setup) | [Hive](/docs/core/connect-data-platform/hive-setup) | [Rockset](/docs/core/connect-data-platform/rockset-setup) |
| [IBM DB2](/docs/core/connect-data-platform/ibmdb2-setup) | [Impala](/docs/core/connect-data-platform/impala-setup) | [SingleStore](/docs/core/connect-data-platform/singlestore-setup) |
| [Doris & SelectDB](/docs/core/connect-data-platform/doris-setup) | [Infer](/docs/core/connect-data-platform/infer-setup) | [SQLite](/docs/core/connect-data-platform/sqlite-setup) |
-| [DuckDB](/docs/core/connect-data-platform/duckdb-setup) | [iomete](/docs/core/connect-data-platform/iomete-setup) | [SQL Server & Azure SQL](/docs/core/connect-data-platform/mssql-setup) |
-| [Dremio](/docs/core/connect-data-platform/dremio-setup) | [Layer](/docs/core/connect-data-platform/layer-setup) | [Teradata](/docs/core/connect-data-platform/teradata-setup) |
-| [Exasol Analytics](/docs/core/connect-data-platform/exasol-setup) | [Materialize](/docs/core/connect-data-platform/materialize-setup) | [TiDB](/docs/core/connect-data-platform/tidb-setup) |
-| [Firebolt](/docs/core/connect-data-platform/firebolt-setup) | [MindsDB](/docs/core/connect-data-platform/mindsdb-setup) | [Vertica](/docs/core/connect-data-platform/vertica-setup) |
-| [AWS Glue](/docs/core/connect-data-platform/glue-setup) | [MySQL](/docs/core/connect-data-platform/mysql-setup)| [Upsolver](/docs/core/connect-data-platform/upsolver-setup) |
-| [Databend Cloud](/docs/core/connect-data-platform/databend-setup) | [fal - Python models](/docs/core/connect-data-platform/fal-setup) | |
+| [Starrocks](/docs/core/connect-data-platform/starrocks-setup) | [DuckDB](/docs/core/connect-data-platform/duckdb-setup) | [iomete](/docs/core/connect-data-platform/iomete-setup)
+| [SQL Server & Azure SQL](/docs/core/connect-data-platform/mssql-setup) | [Dremio](/docs/core/connect-data-platform/dremio-setup) | [Layer](/docs/core/connect-data-platform/layer-setup)
+| [Teradata](/docs/core/connect-data-platform/teradata-setup) | [Exasol Analytics](/docs/core/connect-data-platform/exasol-setup) | [Materialize](/docs/core/connect-data-platform/materialize-setup)
+| [TiDB](/docs/core/connect-data-platform/tidb-setup) | [Firebolt](/docs/core/connect-data-platform/firebolt-setup) | [MindsDB](/docs/core/connect-data-platform/mindsdb-setup)
+| [Vertica](/docs/core/connect-data-platform/vertica-setup) | [AWS Glue](/docs/core/connect-data-platform/glue-setup) | [MySQL](/docs/core/connect-data-platform/mysql-setup) |
+| [Upsolver](/docs/core/connect-data-platform/upsolver-setup) | [Databend Cloud](/docs/core/connect-data-platform/databend-setup) | [fal - Python models](/docs/core/connect-data-platform/fal-setup) |
diff --git a/website/docs/docs/connect-adapters.md b/website/docs/docs/connect-adapters.md
index f45da732abb..77ead34e51d 100644
--- a/website/docs/docs/connect-adapters.md
+++ b/website/docs/docs/connect-adapters.md
@@ -11,9 +11,9 @@ This section provides more details on different ways you can connect dbt to an a
Explore the fastest and most reliable way to deploy dbt using dbt Cloud, a hosted architecture that runs dbt Core across your organization. dbt Cloud lets you seamlessly [connect](/docs/cloud/about-cloud-setup) with a variety of [verified](/docs/supported-data-platforms) data platform providers directly in the dbt Cloud UI.
-### Install using the CLI
+### Install with dbt Core
-Install dbt Core, which is an open-source tool, locally using the CLI. dbt communicates with a number of different data platforms by using a dedicated adapter plugin for each. When you install dbt Core, you'll also need to install the specific adapter for your database, [connect to dbt Core](/docs/core/about-core-setup), and set up a `profiles.yml` file.
+Install dbt Core, an open-source tool, locally using the command line. dbt communicates with a number of different data platforms by using a dedicated adapter plugin for each. When you install dbt Core, you'll also need to install the specific adapter for your database, [connect to dbt Core](/docs/core/about-core-setup), and set up a `profiles.yml` file.
With a few exceptions [^1], you can install all [Verified adapters](/docs/supported-data-platforms) from PyPI using `pip install adapter-name`. For example to install Snowflake, use the command `pip install dbt-snowflake`. The installation will include `dbt-core` and any other required dependencies, which may include both other dependencies and even other adapter plugins. Read more about [installing dbt](/docs/core/installation).
diff --git a/website/docs/docs/contribute-core-adapters.md b/website/docs/docs/contribute-core-adapters.md
index 6e66a5d28ff..553361ee1a2 100644
--- a/website/docs/docs/contribute-core-adapters.md
+++ b/website/docs/docs/contribute-core-adapters.md
@@ -1,6 +1,7 @@
---
title: "Contribute to adapters"
id: "contribute-core-adapters"
+pagination_next: null
---
The dbt Community exists to allow analytics practitioners share their knowledge, help others and collectively to drive forward the discipline of analytics engineering. There are opportunities here for everyone to contribute whether you're at the beginning your analytics engineering journey or you are a seasoned data professional.
diff --git a/website/docs/docs/core/about-core-setup.md b/website/docs/docs/core/about-core-setup.md
index 0408e529b2d..a4d5ff09ee3 100644
--- a/website/docs/docs/core/about-core-setup.md
+++ b/website/docs/docs/core/about-core-setup.md
@@ -3,13 +3,15 @@ title: About dbt Core setup
id: about-core-setup
description: "Configuration settings for dbt Core."
sidebar_label: "About dbt Core setup"
+pagination_next: "docs/core/about-dbt-core"
+pagination_prev: null
---
dbt Core is an [open-source](https://github.com/dbt-labs/dbt-core) tool that enables data teams to transform data using analytics engineering best practices. You can install dbt locally in your environment and use dbt Core on the command line. It can communicate with databases through adapters.
This section of our docs will guide you through various settings to get started:
-- [About the CLI](/docs/core/about-the-cli)
+- [About dbt Core](/docs/core/about-dbt-core)
- [Installing dbt](/docs/core/installation)
- [Connecting to a data platform](/docs/core/connect-data-platform/profiles.yml)
- [How to run your dbt projects](/docs/running-a-dbt-project/run-your-dbt-projects)
diff --git a/website/docs/docs/core/about-dbt-core.md b/website/docs/docs/core/about-dbt-core.md
new file mode 100644
index 00000000000..a35d92420f3
--- /dev/null
+++ b/website/docs/docs/core/about-dbt-core.md
@@ -0,0 +1,25 @@
+---
+title: "About dbt Core"
+id: "about-dbt-core"
+sidebar_label: "About dbt Core"
+---
+
+[dbt Core](https://github.com/dbt-labs/dbt-core) is an open sourced project where you can develop from the command line and run your dbt project.
+
+To use dbt Core, your workflow generally looks like:
+
+1. **Build your dbt project in a code editor —** popular choices include VSCode and Atom.
+
+2. **Run your project from the command line —** macOS ships with a default Terminal program, however you can also use iTerm or the command line prompt within a code editor to execute dbt commands.
+
+:::info How we set up our computers for working on dbt projects
+
+We've written a [guide](https://discourse.getdbt.com/t/how-we-set-up-our-computers-for-working-on-dbt-projects/243) for our recommended setup when running dbt projects using dbt Core.
+
+:::
+
+If you're using the command line, we recommend learning some basics of your terminal to help you work more effectively. In particular, it's important to understand `cd`, `ls` and `pwd` to be able to navigate through the directory structure of your computer easily.
+
+You can find more information on installing and setting up the dbt Core [here](/docs/core/installation).
+
+**Note** — dbt supports a dbt Cloud CLI and dbt Core, both command line interface tools that enable you to run dbt commands. The key distinction is the dbt Cloud CLI is tailored for dbt Cloud's infrastructure and integrates with all its [features](/docs/cloud/about-cloud/dbt-cloud-features).
diff --git a/website/docs/docs/core/about-the-cli.md b/website/docs/docs/core/about-the-cli.md
deleted file mode 100644
index d05fb514dfa..00000000000
--- a/website/docs/docs/core/about-the-cli.md
+++ /dev/null
@@ -1,22 +0,0 @@
----
-title: "About the CLI"
-id: "about-the-cli"
-sidebar_label: "About the CLI"
----
-
-dbt ships with a command line interface (CLI) for running your dbt project. This way of running dbt and a dbt project is free and open source.
-
-To use the CLI, your workflow generally looks like:
-1. **Build your dbt project in a code editor —** popular choices include VSCode and Atom.
-
-1. **Run your project from the command line —** macOS ships with a default Terminal program, however you can also use iTerm or the command line prompt within a code editor to execute dbt commands.
-
-:::info How we set up our computers for working on dbt projects
-
-We've written a [guide](https://discourse.getdbt.com/t/how-we-set-up-our-computers-for-working-on-dbt-projects/243) for our recommended setup when running dbt projects using the CLI.
-
-:::
-
-If you're using the CLI, we recommend learning some basics of your terminal to help you work more effectively. In particular, it's important to understand `cd`, `ls` and `pwd` to be able to navigate through the directory structure of your computer easily.
-
-You can find more information on installing and setting up the dbt CLI [here](/dbt-cli/cli-overview).
diff --git a/website/docs/docs/core/connect-data-platform/about-core-connections.md b/website/docs/docs/core/connect-data-platform/about-core-connections.md
index 802e197514c..a85a32cc031 100644
--- a/website/docs/docs/core/connect-data-platform/about-core-connections.md
+++ b/website/docs/docs/core/connect-data-platform/about-core-connections.md
@@ -4,6 +4,8 @@ id: "about-core-connections"
description: "Information about data platform connections in dbt Core"
sidebar_label: "About data platform connections in dbt Core"
hide_table_of_contents: true
+pagination_next: "docs/core/connect-data-platform/profiles.yml"
+pagination_prev: null
---
dbt Core can connect with a variety of data platform providers including:
diff --git a/website/docs/docs/core/connect-data-platform/alloydb-setup.md b/website/docs/docs/core/connect-data-platform/alloydb-setup.md
index c3f3ee9cfca..c01ba06d887 100644
--- a/website/docs/docs/core/connect-data-platform/alloydb-setup.md
+++ b/website/docs/docs/core/connect-data-platform/alloydb-setup.md
@@ -3,7 +3,7 @@ title: "AlloyDB setup"
meta:
maintained_by: Community?
authors: 'dbt-labs'
- github_repo: 'dbt-labs/dbt-postgres'
+ github_repo: 'dbt-labs/dbt-core'
pypi_package: 'dbt-postgres'
min_core_version: 'v1.0.0'
cloud_support: Not Supported
diff --git a/website/docs/docs/core/connect-data-platform/bigquery-setup.md b/website/docs/docs/core/connect-data-platform/bigquery-setup.md
index 7a2a445be3f..4169b782594 100644
--- a/website/docs/docs/core/connect-data-platform/bigquery-setup.md
+++ b/website/docs/docs/core/connect-data-platform/bigquery-setup.md
@@ -74,10 +74,10 @@ my-bigquery-db:
dev:
type: bigquery
method: oauth
- project: [GCP project id]
- dataset: [the name of your dbt dataset] # You can also use "schema" here
- threads: [1 or more]
- [](#optional-configurations):
+ project: GCP_PROJECT_ID
+ dataset: DBT_DATASET_NAME # You can also use "schema" here
+ threads: 4 # Must be a value of 1 or greater
+ [OPTIONAL_CONFIG](#optional-configurations): VALUE
```
@@ -90,14 +90,7 @@ If you do not specify a `project`/`database` and are using the `oauth` method, d
See [docs](https://developers.google.com/identity/protocols/oauth2) on using OAuth 2.0 to access Google APIs.
-
-
-
+#### Refresh token
Using the refresh token and client information, dbt will mint new access tokens as necessary.
@@ -110,21 +103,19 @@ my-bigquery-db:
dev:
type: bigquery
method: oauth-secrets
- project: [GCP project id]
- dataset: [the name of your dbt dataset] # You can also use "schema" here
- threads: [1 or more]
- refresh_token: [token]
- client_id: [client id]
- client_secret: [client secret]
- token_uri: [redirect URI]
- [](#optional-configurations):
+ project: GCP_PROJECT_ID
+ dataset: DBT_DATASET_NAME # You can also use "schema" here
+ threads: 4 # Must be a value of 1 or greater
+ refresh_token: TOKEN
+ client_id: CLIENT_ID
+ client_secret: CLIENT_SECRET
+ token_uri: REDIRECT_URI
+ [OPTIONAL_CONFIG](#optional-configurations): VALUE
```
-
-
-
+#### Temporary token
dbt will use the one-time access token, no questions asked. This approach makes sense if you have an external deployment process that can mint new access tokens and update the profile file accordingly.
@@ -137,18 +128,15 @@ my-bigquery-db:
dev:
type: bigquery
method: oauth-secrets
- project: [GCP project id]
- dataset: [the name of your dbt dataset] # You can also use "schema" here
- threads: [1 or more]
- token: [temporary access token] # refreshed + updated by external process
- [](#optional-configurations):
+ project: GCP_PROJECT_ID
+ dataset: DBT_DATASET_NAME # You can also use "schema" here
+ threads: 4 # Must be a value of 1 or greater
+ token: TEMPORARY_ACCESS_TOKEN # refreshed + updated by external process
+ [OPTIONAL_CONFIG](#optional-configurations): VALUE
```
-
-
-
### Service Account File
@@ -161,11 +149,11 @@ my-bigquery-db:
dev:
type: bigquery
method: service-account
- project: [GCP project id]
- dataset: [the name of your dbt dataset]
- threads: [1 or more]
- keyfile: [/path/to/bigquery/keyfile.json]
- [](#optional-configurations):
+ project: GCP_PROJECT_ID
+ dataset: DBT_DATASET_NAME
+ threads: 4 # Must be a value of 1 or greater
+ keyfile: /PATH/TO/BIGQUERY/keyfile.json
+ [OPTIONAL_CONFIG](#optional-configurations): VALUE
```
@@ -189,10 +177,10 @@ my-bigquery-db:
dev:
type: bigquery
method: service-account-json
- project: [GCP project id]
- dataset: [the name of your dbt dataset]
- threads: [1 or more]
- [](#optional-configurations):
+ project: GCP_PROJECT_ID
+ dataset: DBT_DATASET_NAME
+ threads: 4 # Must be a value of 1 or greater
+ [OPTIONAL_CONFIG](#optional-configurations): VALUE
# These fields come from the service account json keyfile
keyfile_json:
diff --git a/website/docs/docs/core/connect-data-platform/profiles.yml.md b/website/docs/docs/core/connect-data-platform/profiles.yml.md
index 67b0eb15fbe..97254dda1c4 100644
--- a/website/docs/docs/core/connect-data-platform/profiles.yml.md
+++ b/website/docs/docs/core/connect-data-platform/profiles.yml.md
@@ -3,7 +3,7 @@ title: "About profiles.yml"
id: profiles.yml
---
-If you're using dbt from the [command line (CLI)](/docs/core/about-the-cli), you'll need a `profiles.yml` file that contains the connection details for your data platform. When you run dbt from the CLI, it reads your `dbt_project.yml` file to find the `profile` name, and then looks for a profile with the same name in your `profiles.yml` file. This profile contains all the information dbt needs to connect to your data platform.
+If you're using [dbt Core](/docs/core/about-dbt-core), you'll need a `profiles.yml` file that contains the connection details for your data platform. When you run dbt Core from the command line, it reads your `dbt_project.yml` file to find the `profile` name, and then looks for a profile with the same name in your `profiles.yml` file. This profile contains all the information dbt needs to connect to your data platform.
For detailed info, you can refer to the [Connection profiles](/docs/core/connect-data-platform/connection-profiles).
diff --git a/website/docs/docs/core/connect-data-platform/starrocks-setup.md b/website/docs/docs/core/connect-data-platform/starrocks-setup.md
new file mode 100644
index 00000000000..e5c1abac037
--- /dev/null
+++ b/website/docs/docs/core/connect-data-platform/starrocks-setup.md
@@ -0,0 +1,103 @@
+---
+title: "Starrocks setup"
+description: "Read this guide to learn about the Starrocks warehouse setup in dbt."
+id: "starrocks-setup"
+meta:
+ maintained_by: Starrocks
+ authors: Astralidea
+ github_repo: 'StarRocks/starrocks/tree/main/contrib/dbt-connector'
+ pypi_package: 'dbt-starrocks'
+ min_core_version: 'v1.6.2'
+ min_supported_version: 'Starrocks 2.5'
+ cloud_support: Not Supported
+ slack_channel_name: '#db-starrocks'
+ slack_channel_link: 'https://www.getdbt.com/community'
+ platform_name: 'Starrocks'
+ config_page: '/reference/resource-configs/starrocks-configs'
+---
+
+ Overview of {frontMatter.meta.pypi_package}
+
+
+ - Maintained by: {frontMatter.meta.maintained_by}
+ - Authors: {frontMatter.meta.authors}
+ - GitHub repo: {frontMatter.meta.github_repo}
![]({`https://img.shields.io/github/stars/${frontMatter.meta.github_repo}?style=for-the-badge`}/)
+ - PyPI package:
{frontMatter.meta.pypi_package}
![]({`https://badge.fury.io/py/${frontMatter.meta.pypi_package}.svg`}/)
+ - Slack channel: {frontMatter.meta.slack_channel_name}
+ - Supported dbt Core version: {frontMatter.meta.min_core_version} and newer
+ - dbt Cloud support: {frontMatter.meta.cloud_support}
+ - Minimum data platform version: {frontMatter.meta.min_supported_version}
+
+
+
+ Installing {frontMatter.meta.pypi_package}
+
+pip is the easiest way to install the adapter:
+
+pip install {frontMatter.meta.pypi_package}
+
+Installing {frontMatter.meta.pypi_package}
will also install dbt-core
and any other dependencies.
+
+ Configuring {frontMatter.meta.pypi_package}
+
+For {frontMatter.meta.platform_name}-specifc configuration please refer to {frontMatter.meta.platform_name} Configuration
+
+For further info, refer to the GitHub repository: {frontMatter.meta.github_repo}
+
+
+## Authentication Methods
+
+### User / Password Authentication
+
+Starrocks can be configured using basic user/password authentication as shown below.
+
+
+
+```yaml
+my-starrocks-db:
+ target: dev
+ outputs:
+ dev:
+ type: starrocks
+ host: localhost
+ port: 9030
+ schema: analytics
+
+ # User/password auth
+ username: your_starrocks_username
+ password: your_starrocks_password
+```
+
+
+
+#### Description of Profile Fields
+| Option | Description | Required? | Example |
+|----------|--------------------------------------------------------|-----------|--------------------------------|
+| type | The specific adapter to use | Required | `starrocks` |
+| host | The hostname to connect to | Required | `192.168.100.28` |
+| port | The port to use | Required | `9030` |
+| schema | Specify the schema (database) to build models into | Required | `analytics` |
+| username | The username to use to connect to the server | Required | `dbt_admin` |
+| password | The password to use for authenticating to the server | Required | `correct-horse-battery-staple` |
+| version | Let Plugin try to go to a compatible starrocks version | Optional | `3.1.0` |
+
+## Supported features
+
+| Starrocks <= 2.5 | Starrocks 2.5 ~ 3.1 | Starrocks >= 3.1 | Feature |
+|:----------------:|:--------------------:|:-----------------:|:---------------------------------:|
+| ✅ | ✅ | ✅ | Table materialization |
+| ✅ | ✅ | ✅ | View materialization |
+| ❌ | ❌ | ✅ | Materialized View materialization |
+| ❌ | ✅ | ✅ | Incremental materialization |
+| ❌ | ✅ | ✅ | Primary Key Model |
+| ✅ | ✅ | ✅ | Sources |
+| ✅ | ✅ | ✅ | Custom data tests |
+| ✅ | ✅ | ✅ | Docs generate |
+| ❌ | ❌ | ❌ | Kafka |
+
+### Notice
+1. When StarRocks Version < 2.5, `Create table as` can only set engine='OLAP' and table_type='DUPLICATE'
+2. When StarRocks Version >= 2.5, `Create table as` supports table_type='PRIMARY'
+3. When StarRocks Version < 3.1 distributed_by is required
+
+It is recommended to use the latest starrocks version and dbt-starrocks version for the best experience.
\ No newline at end of file
diff --git a/website/docs/docs/core/connect-data-platform/teradata-setup.md b/website/docs/docs/core/connect-data-platform/teradata-setup.md
index 1fe33ff8929..1ba8e506b88 100644
--- a/website/docs/docs/core/connect-data-platform/teradata-setup.md
+++ b/website/docs/docs/core/connect-data-platform/teradata-setup.md
@@ -4,7 +4,7 @@ description: "Read this guide to learn about the Teradata warehouse setup in dbt
id: "teradata-setup"
meta:
maintained_by: Teradata
- authors: Doug Beatty and Adam Tworkiewicz
+ authors: Teradata
github_repo: 'Teradata/dbt-teradata'
pypi_package: 'dbt-teradata'
min_core_version: 'v0.21.0'
@@ -41,6 +41,28 @@ pip is the easiest way to install the adapter:
Installing {frontMatter.meta.pypi_package}
will also install dbt-core
and any other dependencies.
+ Python compatibility
+
+| Plugin version | Python 3.6 | Python 3.7 | Python 3.8 | Python 3.9 | Python 3.10 | Python 3.11 |
+| -------------- | ----------- | ----------- | ----------- | ----------- | ----------- | ------------ |
+| 0.19.0.x | ✅ | ✅ | ✅ | ❌ | ❌ | ❌
+| 0.20.0.x | ✅ | ✅ | ✅ | ✅ | ❌ | ❌
+| 0.21.1.x | ✅ | ✅ | ✅ | ✅ | ❌ | ❌
+| 1.0.0.x | ❌ | ✅ | ✅ | ✅ | ❌ | ❌
+|1.1.x.x | ❌ | ✅ | ✅ | ✅ | ✅ | ❌
+|1.2.x.x | ❌ | ✅ | ✅ | ✅ | ✅ | ❌
+|1.3.x.x | ❌ | ✅ | ✅ | ✅ | ✅ | ❌
+|1.4.x.x | ❌ | ✅ | ✅ | ✅ | ✅ | ✅
+|1.5.x | ❌ | ✅ | ✅ | ✅ | ✅ | ✅
+|1.6.x | ❌ | ❌ | ✅ | ✅ | ✅ | ✅
+
+ dbt dependent packages version compatibility
+
+| dbt-teradata | dbt-core | dbt-teradata-util | dbt-util |
+|--------------|------------|-------------------|----------------|
+| 1.2.x | 1.2.x | 0.1.0 | 0.9.x or below |
+
+
Configuring {frontMatter.meta.pypi_package}
For {frontMatter.meta.platform_name}-specifc configuration please refer to {frontMatter.meta.platform_name} Configuration
@@ -88,11 +110,15 @@ The plugin also supports the following optional connection parameters:
Parameter | Default | Type | Description
----------------------- | ----------- | -------------- | ---
`account` | | string | Specifies the database account. Equivalent to the Teradata JDBC Driver `ACCOUNT` connection parameter.
+`browser` | | string | Specifies the command to open the browser for Browser Authentication, when logmech is BROWSER. Browser Authentication is supported for Windows and macOS. Equivalent to the Teradata JDBC Driver BROWSER connection parameter.
+`browser_tab_timeout` | `"5"` | quoted integer | Specifies the number of seconds to wait before closing the browser tab after Browser Authentication is completed. The default is 5 seconds. The behavior is under the browser's control, and not all browsers support automatic closing of browser tabs.
+`browser_timeout` | `"180"` | quoted integer | Specifies the number of seconds that the driver will wait for Browser Authentication to complete. The default is 180 seconds (3 minutes).
`column_name` | `"false"` | quoted boolean | Controls the behavior of cursor `.description` sequence `name` items. Equivalent to the Teradata JDBC Driver `COLUMN_NAME` connection parameter. False specifies that a cursor `.description` sequence `name` item provides the AS-clause name if available, or the column name if available, or the column title. True specifies that a cursor `.description` sequence `name` item provides the column name if available, but has no effect when StatementInfo parcel support is unavailable.
`connect_failure_ttl` | `"0"` | quoted integer | Specifies the time-to-live in seconds to remember the most recent connection failure for each IP address/port combination. The driver subsequently skips connection attempts to that IP address/port for the duration of the time-to-live. The default value of zero disables this feature. The recommended value is half the database restart time. Equivalent to the Teradata JDBC Driver `CONNECT_FAILURE_TTL` connection parameter.
+`connect_timeout` | `"10000"` | quoted integer | Specifies the timeout in milliseconds for establishing a TCP socket connection. Specify 0 for no timeout. The default is 10 seconds (10000 milliseconds).
`cop` | `"true"` | quoted boolean | Specifies whether COP Discovery is performed. Equivalent to the Teradata JDBC Driver `COP` connection parameter.
`coplast` | `"false"` | quoted boolean | Specifies how COP Discovery determines the last COP hostname. Equivalent to the Teradata JDBC Driver `COPLAST` connection parameter. When `coplast` is `false` or omitted, or COP Discovery is turned off, then no DNS lookup occurs for the coplast hostname. When `coplast` is `true`, and COP Discovery is turned on, then a DNS lookup occurs for a coplast hostname.
-`dbs_port` | `"1025"` | quoted integer | Specifies the database port number. Equivalent to the Teradata JDBC Driver `DBS_PORT` connection parameter.
+`port` | `"1025"` | quoted integer | Specifies the database port number. Equivalent to the Teradata JDBC Driver `DBS_PORT` connection parameter.
`encryptdata` | `"false"` | quoted boolean | Controls encryption of data exchanged between the driver and the database. Equivalent to the Teradata JDBC Driver `ENCRYPTDATA` connection parameter.
`fake_result_sets` | `"false"` | quoted boolean | Controls whether a fake result set containing statement metadata precedes each real result set.
`field_quote` | `"\""` | string | Specifies a single character string used to quote fields in a CSV file.
@@ -102,11 +128,18 @@ Parameter | Default | Type | Description
`lob_support` | `"true"` | quoted boolean | Controls LOB support. Equivalent to the Teradata JDBC Driver `LOB_SUPPORT` connection parameter.
`log` | `"0"` | quoted integer | Controls debug logging. Somewhat equivalent to the Teradata JDBC Driver `LOG` connection parameter. This parameter's behavior is subject to change in the future. This parameter's value is currently defined as an integer in which the 1-bit governs function and method tracing, the 2-bit governs debug logging, the 4-bit governs transmit and receive message hex dumps, and the 8-bit governs timing. Compose the value by adding together 1, 2, 4, and/or 8.
`logdata` | | string | Specifies extra data for the chosen logon authentication method. Equivalent to the Teradata JDBC Driver `LOGDATA` connection parameter.
+`logon_timeout` | `"0"` | quoted integer | Specifies the logon timeout in seconds. Zero means no timeout.
`logmech` | `"TD2"` | string | Specifies the logon authentication method. Equivalent to the Teradata JDBC Driver `LOGMECH` connection parameter. Possible values are `TD2` (the default), `JWT`, `LDAP`, `KRB5` for Kerberos, or `TDNEGO`.
`max_message_body` | `"2097000"` | quoted integer | Specifies the maximum Response Message size in bytes. Equivalent to the Teradata JDBC Driver `MAX_MESSAGE_BODY` connection parameter.
`partition` | `"DBC/SQL"` | string | Specifies the database partition. Equivalent to the Teradata JDBC Driver `PARTITION` connection parameter.
+`request_timeout` | `"0"` | quoted integer | Specifies the timeout for executing each SQL request. Zero means no timeout.
+`retries` | `0` | integer | Allows an adapter to automatically try again when the attempt to open a new connection on the database has a transient, infrequent error. This option can be set using the retries configuration. Default value is 0. The default wait period between connection attempts is one second. retry_timeout (seconds) option allows us to adjust this waiting period.
+`runstartup` | "false" | quoted boolean | Controls whether the user's STARTUP SQL request is executed after logon. For more information, refer to User STARTUP SQL Request. Equivalent to the Teradata JDBC Driver RUNSTARTUP connection parameter. If retries is set to 3, the adapter will try to establish a new connection three times if an error occurs.
+`sessions` | | quoted integer | Specifies the number of data transfer connections for FastLoad or FastExport. The default (recommended) lets the database choose the appropriate number of connections. Equivalent to the Teradata JDBC Driver SESSIONS connection parameter.
`sip_support` | `"true"` | quoted boolean | Controls whether StatementInfo parcel is used. Equivalent to the Teradata JDBC Driver `SIP_SUPPORT` connection parameter.
+`sp_spl` | `"true"` | quoted boolean | Controls whether stored procedure source code is saved in the database when a SQL stored procedure is created. Equivalent to the Teradata JDBC Driver SP_SPL connection parameter.
`sslca` | | string | Specifies the file name of a PEM file that contains Certificate Authority (CA) certificates for use with `sslmode` values `VERIFY-CA` or `VERIFY-FULL`. Equivalent to the Teradata JDBC Driver `SSLCA` connection parameter.
+`sslcrc` | `"ALLOW"` | string | Equivalent to the Teradata JDBC Driver SSLCRC connection parameter. Values are case-insensitive.
• ALLOW provides "soft fail" behavior such that communication failures are ignored during certificate revocation checking.
• REQUIRE mandates that certificate revocation checking must succeed.
`sslcapath` | | string | Specifies a directory of PEM files that contain Certificate Authority (CA) certificates for use with `sslmode` values `VERIFY-CA` or `VERIFY-FULL`. Only files with an extension of `.pem` are used. Other files in the specified directory are not used. Equivalent to the Teradata JDBC Driver `SSLCAPATH` connection parameter.
`sslcipher` | | string | Specifies the TLS cipher for HTTPS/TLS connections. Equivalent to the Teradata JDBC Driver `SSLCIPHER` connection parameter.
`sslmode` | `"PREFER"` | string | Specifies the mode for connections to the database. Equivalent to the Teradata JDBC Driver `SSLMODE` connection parameter.
• `DISABLE` disables HTTPS/TLS connections and uses only non-TLS connections.
• `ALLOW` uses non-TLS connections unless the database requires HTTPS/TLS connections.
• `PREFER` uses HTTPS/TLS connections unless the database does not offer HTTPS/TLS connections.
• `REQUIRE` uses only HTTPS/TLS connections.
• `VERIFY-CA` uses only HTTPS/TLS connections and verifies that the server certificate is valid and trusted.
• `VERIFY-FULL` uses only HTTPS/TLS connections, verifies that the server certificate is valid and trusted, and verifies that the server certificate matches the database hostname.
@@ -124,6 +157,91 @@ For the full description of the connection parameters see https://github.com/Ter
* `ephemeral`
* `incremental`
+#### Incremental Materialization
+The following incremental materialization strategies are supported:
+* `append` (default)
+* `delete+insert`
+* `merge`
+
+To learn more about dbt incremental strategies please check [the dbt incremental strategy documentation](https://docs.getdbt.com/docs/build/incremental-models#about-incremental_strategy).
+
### Commands
All dbt commands are supported.
+
+## Support for model contracts
+Model contracts are not yet supported with dbt-teradata.
+
+## Support for `dbt-utils` package
+`dbt-utils` package is supported through `teradata/teradata_utils` dbt package. The package provides a compatibility layer between `dbt_utils` and `dbt-teradata`. See [teradata_utils](https://hub.getdbt.com/teradata/teradata_utils/latest/) package for install instructions.
+
+### Cross DB macros
+Starting with release 1.3, some macros were migrated from [teradata-dbt-utils](https://github.com/Teradata/dbt-teradata-utils) dbt package to the connector. See the table below for the macros supported from the connector.
+
+For using cross DB macros, teradata-utils as a macro namespace will not be used, as cross DB macros have been migrated from teradata-utils to Dbt-Teradata.
+
+
+#### Compatibility
+
+| Macro Group | Macro Name | Status | Comment |
+|:---------------------:|:-----------------------------:|:---------------------:|:----------------------------------------------------------------------:|
+| Cross-database macros | current_timestamp | :white_check_mark: | custom macro provided |
+| Cross-database macros | dateadd | :white_check_mark: | custom macro provided |
+| Cross-database macros | datediff | :white_check_mark: | custom macro provided, see [compatibility note](#datediff) |
+| Cross-database macros | split_part | :white_check_mark: | custom macro provided |
+| Cross-database macros | date_trunc | :white_check_mark: | custom macro provided |
+| Cross-database macros | hash | :white_check_mark: | custom macro provided, see [compatibility note](#hash) |
+| Cross-database macros | replace | :white_check_mark: | custom macro provided |
+| Cross-database macros | type_string | :white_check_mark: | custom macro provided |
+| Cross-database macros | last_day | :white_check_mark: | no customization needed, see [compatibility note](#last_day) |
+| Cross-database macros | width_bucket | :white_check_mark: | no customization
+
+
+#### examples for cross DB macros
+ ##### replace
+ {{ dbt.replace("string_text_column", "old_chars", "new_chars") }}
+ {{ replace('abcgef', 'g', 'd') }}
+
+ ##### date_trunc
+ {{ dbt.date_trunc("date_part", "date") }}
+ {{ dbt.date_trunc("DD", "'2018-01-05 12:00:00'") }}
+
+ ##### datediff
+ `datediff` macro in teradata supports difference between dates. Differece between timestamps is not supported.
+
+ ##### hash
+
+ `Hash` macro needs an `md5` function implementation. Teradata doesn't support `md5` natively. You need to install a User Defined Function (UDF):
+ 1. Download the md5 UDF implementation from Teradata (registration required): https://downloads.teradata.com/download/extensibility/md5-message-digest-udf.
+ 1. Unzip the package and go to `src` directory.
+ 1. Start up `bteq` and connect to your database.
+ 1. Create database `GLOBAL_FUNCTIONS` that will host the UDF. You can't change the database name as it's hardcoded in the macro:
+ ```sql
+ CREATE DATABASE GLOBAL_FUNCTIONS AS PERMANENT = 60e6, SPOOL = 120e6;
+ ```
+ 1. Create the UDF. Replace `` with your current database user:
+ ```sql
+ GRANT CREATE FUNCTION ON GLOBAL_FUNCTIONS TO ;
+ DATABASE GLOBAL_FUNCTIONS;
+ .run file = hash_md5.btq
+ ```
+ 1. Grant permissions to run the UDF with grant option.
+ ```sql
+ GRANT EXECUTE FUNCTION ON GLOBAL_FUNCTIONS TO PUBLIC WITH GRANT OPTION;
+ ```
+ ##### last_day
+
+ `last_day` in `teradata_utils`, unlike the corresponding macro in `dbt_utils`, doesn't support `quarter` datepart.
+
+## Limitations
+
+### Transaction mode
+Only ANSI transaction mode is supported.
+
+## Credits
+
+The adapter was originally created by [Doug Beatty](https://github.com/dbeatty10). Teradata took over the adapter in January 2022. We are grateful to Doug for founding the project and accelerating the integration of dbt + Teradata.
+
+## License
+
+The adapter is published using Apache-2.0 License. Refer to the [terms and conditions](https://github.com/dbt-labs/dbt-core/blob/main/License.md) to understand items such as creating derivative work and the support model.
diff --git a/website/docs/docs/core/connect-data-platform/trino-setup.md b/website/docs/docs/core/connect-data-platform/trino-setup.md
index 396634dc6e6..39d8ed8ab3f 100644
--- a/website/docs/docs/core/connect-data-platform/trino-setup.md
+++ b/website/docs/docs/core/connect-data-platform/trino-setup.md
@@ -83,7 +83,7 @@ The following profile fields are optional to set up. They let you configure your
| Profile field | Example | Description |
| ----------------------------- | -------------------------------- | ----------------------------------------------------------------------------------------------------------- |
| `threads` | `8` | How many threads dbt should use (default is `1`) |
-| `roles` | `system: analyst` | Catalog roles |
+| `roles` | `system: analyst` | Catalog roles can be set under the optional `roles` parameter using the following format: `catalog: role`. |
| `session_properties` | `query_max_run_time: 4h` | Sets Trino session properties used in the connection. Execute `SHOW SESSION` to see available options |
| `prepared_statements_enabled` | `true` or `false` | Enable usage of Trino prepared statements (used in `dbt seed` commands) (default: `true`) |
| `retries` | `10` | Configure how many times all database operation is retried when connection issues arise (default: `3`) |
diff --git a/website/docs/docs/core/connect-data-platform/upsolver-setup.md b/website/docs/docs/core/connect-data-platform/upsolver-setup.md
index 68cfa3045cd..6b2f410fc07 100644
--- a/website/docs/docs/core/connect-data-platform/upsolver-setup.md
+++ b/website/docs/docs/core/connect-data-platform/upsolver-setup.md
@@ -14,6 +14,7 @@ meta:
slack_channel_link: 'https://join.slack.com/t/upsolvercommunity/shared_invite/zt-1zo1dbyys-hj28WfaZvMh4Z4Id3OkkhA'
platform_name: 'Upsolver'
config_page: '/reference/resource-configs/upsolver-configs'
+pagination_next: null
---
Overview of {frontMatter.meta.pypi_package}
diff --git a/website/docs/docs/core/dbt-core-environments.md b/website/docs/docs/core/dbt-core-environments.md
index 5daf17bddf9..c7f340557fd 100644
--- a/website/docs/docs/core/dbt-core-environments.md
+++ b/website/docs/docs/core/dbt-core-environments.md
@@ -1,6 +1,7 @@
---
title: "dbt Core environments"
id: "dbt-core-environments"
+pagination_next: "docs/running-a-dbt-project/run-your-dbt-projects"
---
dbt makes it easy to maintain separate production and development environments through the use of [targets](/reference/dbt-jinja-functions/target.md) within a [profile](/docs/core/connect-data-platform/profiles.yml). A typical profile, when using dbt locally (for example, running from your command line), will have a target named `dev` and have this set as the default. This means that while making changes, your objects will be built in your _development_ target without affecting production queries made by your end users. Once you are confident in your changes, you can deploy the code to _production_, by running your dbt project with a _prod_ target.
diff --git a/website/docs/docs/core/installation-overview.md b/website/docs/docs/core/installation-overview.md
index f1fdb800fdf..cb1df26b0f8 100644
--- a/website/docs/docs/core/installation-overview.md
+++ b/website/docs/docs/core/installation-overview.md
@@ -2,6 +2,8 @@
title: "About installing dbt"
id: "installation"
description: "You can install dbt Core using a few different tested methods."
+pagination_next: "docs/core/homebrew-install"
+pagination_prev: null
---
You can install dbt Core on the command line by using one of these methods:
@@ -11,9 +13,17 @@ You can install dbt Core on the command line by using one of these methods:
- [Use a Docker image to install dbt](/docs/core/docker-install)
- [Install dbt from source](/docs/core/source-install)
+:::tip Pro tip: Using the --help flag
+
+Most command-line tools, including dbt, have a `--help` flag that you can use to show available commands and arguments. For example, you can use the `--help` flag with dbt in two ways:
+— `dbt --help`: Lists the commands available for dbt
+— `dbt run --help`: Lists the flags available for the `run` command
+
+:::
+
## Upgrading dbt Core
-dbt provides a number of resources for understanding [general best practices](/blog/upgrade-dbt-without-fear) while upgrading your dbt project as well as detailed [migration guides](/guides/migration/versions/upgrading-to-v1.4) highlighting the changes required for each minor and major release, and [core versions](/docs/dbt-versions/core)
+dbt provides a number of resources for understanding [general best practices](/blog/upgrade-dbt-without-fear) while upgrading your dbt project as well as detailed [migration guides](/docs/dbt-versions/core-upgrade/upgrading-to-v1.4) highlighting the changes required for each minor and major release, and [core versions](/docs/dbt-versions/core)
- [Upgrade Homebrew](/docs/core/homebrew-install#upgrading-dbt-and-your-adapter)
- [Upgrade `pip`](/docs/core/pip-install#change-dbt-core-versions)
diff --git a/website/docs/docs/core/pip-install.md b/website/docs/docs/core/pip-install.md
index a35ad5f0d77..44fac00e493 100644
--- a/website/docs/docs/core/pip-install.md
+++ b/website/docs/docs/core/pip-install.md
@@ -5,7 +5,7 @@ description: "You can use pip to install dbt Core and adapter plugins from the c
You need to use `pip` to install dbt Core on Windows or Linux operating systems. You can use `pip` or [Homebrew](/docs/core/homebrew-install) for installing dbt Core on a MacOS.
-You can install dbt Core and plugins using `pip` because they are Python modules distributed on [PyPI](https://pypi.org/project/dbt/).
+You can install dbt Core and plugins using `pip` because they are Python modules distributed on [PyPI](https://pypi.org/project/dbt-core/).
diff --git a/website/docs/docs/core/source-install.md b/website/docs/docs/core/source-install.md
index be9918223fe..42086159c03 100644
--- a/website/docs/docs/core/source-install.md
+++ b/website/docs/docs/core/source-install.md
@@ -1,6 +1,7 @@
---
title: "Install from source"
description: "You can install dbt Core from its GitHub code source."
+pagination_next: null
---
dbt Core and almost all of its adapter plugins are open source software. As such, the codebases are freely available to download and build from source. You might install from source if you want the latest code or want to install dbt from a specific commit. This might be helpful when you are contributing changes, or if you want to debug a past change.
diff --git a/website/docs/docs/dbt-cloud-apis/admin-cloud-api.md b/website/docs/docs/dbt-cloud-apis/admin-cloud-api.md
index 8a5712f40df..168ec0c80f4 100644
--- a/website/docs/docs/dbt-cloud-apis/admin-cloud-api.md
+++ b/website/docs/docs/dbt-cloud-apis/admin-cloud-api.md
@@ -1,6 +1,7 @@
---
title: "dbt Cloud Administrative API"
id: "admin-cloud-api"
+pagination_next: "docs/dbt-cloud-apis/discovery-api"
---
The dbt Cloud Administrative API is enabled by default for [Team and Enterprise plans](https://www.getdbt.com/pricing/). It can be used to:
diff --git a/website/docs/docs/dbt-cloud-apis/apis-overview.md b/website/docs/docs/dbt-cloud-apis/apis-overview.md
index b7d722747d8..eef64992af9 100644
--- a/website/docs/docs/dbt-cloud-apis/apis-overview.md
+++ b/website/docs/docs/dbt-cloud-apis/apis-overview.md
@@ -2,6 +2,8 @@
title: "APIs Overview"
description: "Learn how dbt accounts on the Team and Enterprise plans can query the dbt Cloud APIs."
id: "overview"
+pagination_next: "docs/dbt-cloud-apis/user-tokens"
+pagination_prev: null
---
## Overview
diff --git a/website/docs/docs/dbt-cloud-apis/authentication.md b/website/docs/docs/dbt-cloud-apis/authentication.md
new file mode 100644
index 00000000000..7deadd68f18
--- /dev/null
+++ b/website/docs/docs/dbt-cloud-apis/authentication.md
@@ -0,0 +1,22 @@
+---
+title: "Authentication"
+description: "Learn how to authenticate with user tokens and service account tokens "
+pagination_next: "docs/dbt-cloud-apis/user-tokens"
+pagination_prev: null
+---
+
+
+
+
+
+
+
+
\ No newline at end of file
diff --git a/website/docs/docs/dbt-cloud-apis/discovery-api.md b/website/docs/docs/dbt-cloud-apis/discovery-api.md
index e4441aa55a2..747128cf7bc 100644
--- a/website/docs/docs/dbt-cloud-apis/discovery-api.md
+++ b/website/docs/docs/dbt-cloud-apis/discovery-api.md
@@ -1,5 +1,6 @@
---
title: "About the Discovery API"
+pagination_next: "docs/dbt-cloud-apis/discovery-use-cases-and-examples"
---
Every time dbt Cloud runs a project, it generates and stores information about the project. The metadata includes details about your project’s models, sources, and other nodes along with their execution results. With the dbt Cloud Discovery API, you can query this comprehensive information to gain a better understanding of your DAG and the data it produces.
diff --git a/website/docs/docs/dbt-cloud-apis/discovery-querying.md b/website/docs/docs/dbt-cloud-apis/discovery-querying.md
index ba1365e632b..35c092adb4b 100644
--- a/website/docs/docs/dbt-cloud-apis/discovery-querying.md
+++ b/website/docs/docs/dbt-cloud-apis/discovery-querying.md
@@ -2,6 +2,7 @@
title: "Query the Discovery API"
id: "discovery-querying"
sidebar_label: "Query the Discovery API"
+pagination_next: "docs/dbt-cloud-apis/discovery-schema-environment"
---
The Discovery API supports ad-hoc queries and integrations. If you are new to the API, refer to [About the Discovery API](/docs/dbt-cloud-apis/discovery-api) for an introduction.
diff --git a/website/docs/docs/dbt-cloud-apis/schema-discovery-job.mdx b/website/docs/docs/dbt-cloud-apis/schema-discovery-job.mdx
index bb30786e19d..8b02c5601ad 100644
--- a/website/docs/docs/dbt-cloud-apis/schema-discovery-job.mdx
+++ b/website/docs/docs/dbt-cloud-apis/schema-discovery-job.mdx
@@ -2,6 +2,8 @@
title: "Job object schema"
sidebar_label: "Job"
id: "discovery-schema-job"
+pagination_next: "docs/dbt-cloud-apis/discovery-schema-job-model"
+pagination_prev: null
---
import { QueryArgsTable, SchemaTable } from "./schema";
diff --git a/website/docs/docs/dbt-cloud-apis/sl-api-overview.md b/website/docs/docs/dbt-cloud-apis/sl-api-overview.md
index 42416765904..3ddbf76d152 100644
--- a/website/docs/docs/dbt-cloud-apis/sl-api-overview.md
+++ b/website/docs/docs/dbt-cloud-apis/sl-api-overview.md
@@ -4,6 +4,7 @@ id: sl-api-overview
description: "Integrate and query metrics and dimensions in downstream tools using the Semantic Layer APIs"
tags: [Semantic Layer, API]
hide_table_of_contents: true
+pagination_next: "docs/dbt-cloud-apis/sl-jdbc"
---
@@ -31,10 +32,8 @@ You can use the dbt Semantic Layer for a variety of tools and applications of da
import Features from '/snippets/_sl-plan-info.md'
diff --git a/website/docs/docs/dbt-cloud-apis/sl-jdbc.md b/website/docs/docs/dbt-cloud-apis/sl-jdbc.md
index 02d26229794..e10d057dc75 100644
--- a/website/docs/docs/dbt-cloud-apis/sl-jdbc.md
+++ b/website/docs/docs/dbt-cloud-apis/sl-jdbc.md
@@ -5,7 +5,6 @@ description: "Integrate and use the JDBC API to query your metrics."
tags: [Semantic Layer, API]
---
-
import LegacyInfo from '/snippets/_legacy-sl-callout.md';
@@ -59,11 +58,13 @@ jdbc:arrow-flight-sql://semantic-layer.cloud.getdbt.com:443?&environmentId=20233
## Querying the API for metric metadata
-The Semantic Layer JDBC API has built-in metadata calls which can provide a user with information about their metrics and dimensions. Here are some metadata commands and examples:
+The Semantic Layer JDBC API has built-in metadata calls which can provide a user with information about their metrics and dimensions.
+
+Refer to the following tabs for metadata commands and examples:
-
+
Use this query to fetch all defined metrics in your dbt project:
@@ -74,7 +75,7 @@ select * from {{
```
-
+
Use this query to fetch all dimensions for a metric.
@@ -87,7 +88,7 @@ select * from {{
-
+
Use this query to fetch dimension values for one or multiple metrics and single dimension.
@@ -100,7 +101,7 @@ semantic_layer.dimension_values(metrics=['food_order_amount'], group_by=['custom
-
+
Use this query to fetch queryable granularities for a list of metrics. This API request allows you to only show the time granularities that make sense for the primary time dimension of the metrics (such as `metric_time`), but if you want queryable granularities for other time dimensions, you can use the `dimensions()` call, and find the column queryable_granularities.
@@ -113,6 +114,9 @@ select * from {{
+
+
+
@@ -144,9 +148,10 @@ select NAME, QUERYABLE_GRANULARITIES from {{
-
+
It may be useful in your application to expose the names of the time dimensions that represent `metric_time` or the common thread across all metrics.
+
You can first query the `metrics()` argument to fetch a list of measures, then use the `measures()` call which will return the name(s) of the time dimensions that make up metric time.
```bash
@@ -167,12 +172,13 @@ To query metric values, here are the following parameters that are available:
| `metrics` | The metric name as defined in your dbt metric configuration | `metrics=['revenue']` | Required |
| `group_by` | Dimension names or entities to group by. We require a reference to the entity of the dimension (other than for the primary time dimension), which is pre-appended to the front of the dimension name with a double underscore. | `group_by=['user__country', 'metric_time']` | Optional |
| `grain` | A parameter specific to any time dimension and changes the grain of the data from the default for the metric. | `group_by=[Dimension('metric_time')`
`grain('week\|day\|month\|quarter\|year')]` | Optional |
-| `where` | A where clause that allows you to filter on dimensions and entities using parameters - comes with `TimeDimension`, `Dimension`, and `Entity` objects. Granularity is required with `TimeDimension` | `"{{ where=Dimension('customer__country') }} = 'US')"` | Optional |
+| `where` | A where clause that allows you to filter on dimensions and entities using parameters. This takes a filter list OR string. Inputs come with `Dimension`, and `Entity` objects. Granularity is required if the `Dimension` is a time dimension | `"{{ where=Dimension('customer__country') }} = 'US')"` | Optional |
| `limit` | Limit the data returned | `limit=10` | Optional |
-|`order` | Order the data returned | `order_by=['-order_gross_profit']` (remove `-` for ascending order) | Optional |
+|`order` | Order the data returned by a particular field | `order_by=['order_gross_profit']`, use `-` for descending, or full object notation if the object is operated on: `order_by=[Metric('order_gross_profit').descending(True)`] | Optional |
| `compile` | If true, returns generated SQL for the data platform but does not execute | `compile=True` | Optional |
+
## Note on time dimensions and `metric_time`
You will notice that in the list of dimensions for all metrics, there is a dimension called `metric_time`. `Metric_time` is a reserved keyword for the measure-specific aggregation time dimensions. For any time-series metric, the `metric_time` keyword should always be available for use in queries. This is a common dimension across *all* metrics in a semantic graph.
@@ -246,47 +252,92 @@ select * from {{
Where filters in API allow for a filter list or string. We recommend using the filter list for production applications as this format will realize all benefits from the where possible.
-Where filters have the following components that you can use:
+Where Filters have a few objects that you can use:
-- `Dimension()` - This is used for any categorical or time dimensions. If used for a time dimension, granularity is required - `Dimension('metric_time').grain('week')` or `Dimension('customer__country')`
+- `Dimension()` - Used for any categorical or time dimensions. If used for a time dimension, granularity is required - `Dimension('metric_time').grain('week')` or `Dimension('customer__country')`
-- `TimeDimension()` - This is used for all time dimensions and requires a granularity argument - `TimeDimension('metric_time', 'MONTH)`
+- `Entity()` - Used for entities like primary and foreign keys - `Entity('order_id')`
-- `Entity()` - This is used for entities like primary and foreign keys - `Entity('order_id')`
+Note: If you prefer a more explicit path to create the `where` clause, you can optionally use the `TimeDimension` feature. This helps separate out categorical dimensions from time-related ones. The `TimeDimesion` input takes the time dimension name and also requires granularity, like this: `TimeDimension('metric_time', 'MONTH')`.
-Use the following example to query using a `where` filter with the string format:
+- Use the following example to query using a `where` filter with the string format:
```bash
select * from {{
semantic_layer.query(metrics=['food_order_amount', 'order_gross_profit'],
group_by=[Dimension('metric_time').grain('month'),'customer__customer_type'],
-where="{{ TimeDimension('metric_time', 'MONTH') }} >= '2017-03-09' AND {{ Dimension('customer__customer_type' }} in ('new') AND {{ Entity('order_id') }} = 10")
+where="{{ Dimension('metric_time').grain('month') }} >= '2017-03-09' AND {{ Dimension('customer__customer_type' }} in ('new') AND {{ Entity('order_id') }} = 10")
}}
```
-Use the following example to query using a `where` filter with a filter list format:
+- (Recommended for better performance) Use the following example to query using a `where` filter with a filter list format:
```bash
select * from {{
semantic_layer.query(metrics=['food_order_amount', 'order_gross_profit'],
group_by=[Dimension('metric_time').grain('month'),'customer__customer_type'],
-where=[{{ TimeDimension('metric_time', 'MONTH')}} >= '2017-03-09', {{ Dimension('customer__customer_type' }} in ('new'), {{ Entity('order_id') }} = 10])
+where=["{{ Dimension('metric_time').grain('month') }} >= '2017-03-09'", "{{ Dimension('customer__customer_type' }} in ('new')", "{{ Entity('order_id') }} = 10"]
}}
```
-### Query with a limit and order by
+### Query with a limit
Use the following example to query using a `limit` or `order_by` clauses:
+```bash
+select * from {{
+semantic_layer.query(metrics=['food_order_amount', 'order_gross_profit'],
+ group_by=[Dimension('metric_time')],
+ limit=10)
+ }}
+```
+### Query with Order By Examples
+
+Order By can take a basic string that's a Dimension, Metric, or Entity and this will default to ascending order
+
```bash
select * from {{
semantic_layer.query(metrics=['food_order_amount', 'order_gross_profit'],
group_by=[Dimension('metric_time')],
limit=10,
- order_by=['order_gross_profit'])
+ order_by=['order_gross_profit']
}}
```
+
+For descending order, you can add a `-` sign in front of the object. However, you can only use this short hand notation if you aren't operating on the object or using the full object notation.
+
+```bash
+select * from {{
+semantic_layer.query(metrics=['food_order_amount', 'order_gross_profit'],
+ group_by=[Dimension('metric_time')],
+ limit=10,
+ order_by=[-'order_gross_profit']
+ }}
+```
+If you are ordering by an object that's been operated on (e.g., change granularity), or you are using the full object notation, descending order must look like:
+
+```bash
+select * from {{
+semantic_layer.query(metrics=['food_order_amount', 'order_gross_profit'],
+ group_by=[Dimension('metric_time').grain('week')],
+ limit=10,
+ order_by=[Metric('order_gross_profit').descending(True), Dimension('metric_time').grain('week').descending(True) ]
+ }}
+```
+
+Similarly, this will yield ascending order:
+
+```bash
+select * from {{
+semantic_layer.query(metrics=['food_order_amount', 'order_gross_profit'],
+ group_by=[Dimension('metric_time').grain('week')],
+ limit=10,
+ order_by=[Metric('order_gross_profit'), Dimension('metric_time').grain('week')]
+ }}
+```
+
+
### Query with compile keyword
Use the following example to query using a `compile` keyword:
diff --git a/website/docs/docs/dbt-cloud-apis/sl-manifest.md b/website/docs/docs/dbt-cloud-apis/sl-manifest.md
index 47304accea3..6ecac495869 100644
--- a/website/docs/docs/dbt-cloud-apis/sl-manifest.md
+++ b/website/docs/docs/dbt-cloud-apis/sl-manifest.md
@@ -4,6 +4,7 @@ id: sl-manifest
description: "Learn about the semantic manifest.json file and how you can use artifacts to gain insights about your dbt Semantic Layer."
tags: [Semantic Layer, APIs]
sidebar_label: "Semantic manifest"
+pagination_next: null
---
diff --git a/website/docs/docs/dbt-cloud-apis/user-tokens.md b/website/docs/docs/dbt-cloud-apis/user-tokens.md
index e56d8b2f974..77e536b12a5 100644
--- a/website/docs/docs/dbt-cloud-apis/user-tokens.md
+++ b/website/docs/docs/dbt-cloud-apis/user-tokens.md
@@ -1,6 +1,7 @@
---
title: "User tokens"
id: "user-tokens"
+pagination_next: "docs/dbt-cloud-apis/service-tokens"
---
## User API tokens
@@ -13,7 +14,7 @@ permissions of the user the that they were created for.
You can find your User API token in the Profile page under the `API Access`
label.
-
+
## FAQs
diff --git a/website/docs/docs/dbt-cloud-environments.md b/website/docs/docs/dbt-cloud-environments.md
index f61ec5ef72b..8fa4522d47c 100644
--- a/website/docs/docs/dbt-cloud-environments.md
+++ b/website/docs/docs/dbt-cloud-environments.md
@@ -2,9 +2,10 @@
title: "dbt Cloud environments"
id: "dbt-cloud-environments"
description: "Learn about dbt Cloud's development environment to execute your project in the IDE"
+pagination_next: null
---
-An environment determines how dbt Cloud will execute your project in both the dbt Cloud IDE (for development) and scheduled jobs (for deployment).
+An environment determines how dbt Cloud will execute your project in the [dbt Cloud IDE](/docs/cloud/dbt-cloud-ide/develop-in-the-cloud) or [dbt Cloud CLI](/docs/cloud/cloud-cli-installation) (for development) and scheduled jobs (for deployment).
Critically, in order to execute dbt, environments define three variables:
@@ -34,7 +35,7 @@ To create a new dbt Cloud development environment:
### Set developer credentials
-To use the IDE, each developer will need to set up [personal development credentials](/docs/cloud/dbt-cloud-ide/develop-in-the-cloud#access-the-cloud-ide) to your warehouse connection in their **Profile Settings**. This allows you to set separate target information and maintain individual credentials to connect to your warehouse via the dbt Cloud IDE.
+To use the dbt Cloud IDE or dbt Cloud CLI, each developer will need to set up [personal development credentials](/docs/cloud/dbt-cloud-ide/develop-in-the-cloud#access-the-cloud-ide) to your warehouse connection in their **Profile Settings**. This allows you to set separate target information and maintain individual credentials to connect to your warehouse.
diff --git a/website/docs/docs/dbt-support.md b/website/docs/docs/dbt-support.md
index f63e016b03e..513d5fff588 100644
--- a/website/docs/docs/dbt-support.md
+++ b/website/docs/docs/dbt-support.md
@@ -1,6 +1,8 @@
---
title: "dbt support"
id: "dbt-support"
+pagination_next: null
+pagination_prev: null
---
## dbt Core support
diff --git a/website/docs/docs/dbt-versions/core-upgrade/00-upgrading-to-v1.7.md b/website/docs/docs/dbt-versions/core-upgrade/00-upgrading-to-v1.7.md
new file mode 100644
index 00000000000..9ebd3c64cf3
--- /dev/null
+++ b/website/docs/docs/dbt-versions/core-upgrade/00-upgrading-to-v1.7.md
@@ -0,0 +1,70 @@
+---
+title: "Upgrading to v1.7 (latest)"
+id: upgrading-to-v1.7
+description: New features and changes in dbt Core v1.7
+displayed_sidebar: "docs"
+---
+
+import UpgradeMove from '/snippets/_upgrade-move.md';
+
+
+
+## Resources
+
+- [Changelog](https://github.com/dbt-labs/dbt-core/blob/8aaed0e29f9560bc53d9d3e88325a9597318e375/CHANGELOG.md)
+- [CLI Installation guide](/docs/core/installation)
+- [Cloud upgrade guide](/docs/dbt-versions/upgrade-core-in-cloud)
+- [Release schedule](https://github.com/dbt-labs/dbt-core/issues/8260)
+
+## What to know before upgrading
+
+dbt Labs is committed to providing backward compatibility for all versions 1.x, with the exception of any changes explicitly mentioned below. If you encounter an error upon upgrading, please let us know by [opening an issue](https://github.com/dbt-labs/dbt-core/issues/new).
+
+### Behavior changes
+
+dbt Core v1.7 expands the amount of sources you can configure freshness for. Previously, freshness was limited to sources with a `loaded_at_field`; now, freshness can be generated from warehouse metadata tables when available.
+
+As part of this change, the `loaded_at_field` is no longer required to generate source freshness. If a source has a `freshness:` block, dbt will attempt to calculate freshness for that source:
+- If a `loaded_at_field` is provided, dbt will calculate freshness via a select query (previous behavior).
+- If a `loaded_at_field` is _not_ provided, dbt will calculate freshness via warehouse metadata tables when possible (new behavior).
+
+This is a relatively small behavior change, but worth calling out in case you notice that dbt is calculating freshness for _more_ sources than before. To exclude a source from freshness calculations, you have two options:
+- Don't add a `freshness:` block.
+- Explicitly set `freshness: null`
+
+## New and changed features and functionality
+
+- [`dbt docs generate`](/reference/commands/cmd-docs) now supports `--select` to generate [catalog metadata](/reference/artifacts/catalog-json) for a subset of your project. Currently available for Snowflake and Postgres only, but other adapters are coming soon.
+- [Source freshness](/docs/deploy/source-freshness) can now be generated from warehouse metadata tables, currently Snowflake only, but other adapters that have metadata tables are coming soon.
+
+### MetricFlow enhancements
+
+- Automatically create metrics on measures with [`create_metric: true`](/docs/build/semantic-models).
+- Optional [`label`](/docs/build/semantic-models) in semantic_models, measures, dimensions and entities.
+- New configurations for semantic models - [enable/disable](/reference/resource-configs/enabled), [group](/reference/resource-configs/group), and [meta](/reference/resource-configs/meta).
+- Support `fill_nulls_with` and `join_to_timespine` for metric nodes.
+- `saved_queries` extends governance beyond the semantic objects to their consumption.
+
+### For consumers of dbt artifacts (metadata)
+
+- The [manifest](/reference/artifacts/manifest-json) schema version has been updated to v11.
+- The [run_results](/reference/artifacts/run-results-json) schema version has been updated to v5.
+- There are a few specific changes to the [catalog.json](/reference/artifacts/catalog-json):
+ - Added [node attributes](/reference/artifacts/run-results-json) related to compilation (`compiled`, `compiled_code`, `relation_name`) to the `catalog.json`.
+ - The nodes dictionary in the `catalog.json` can now be "partial" if `dbt docs generate` is run with a selector.
+
+### Model governance
+
+dbt Core v1.5 introduced model governance which we're continuing to refine. v1.7 includes these additional features and functionality:
+
+- **[Breaking change detection](/reference/resource-properties/versions#detecting-breaking-changes) for models with contracts enforced:** When dbt detects a breaking change to a model with an enforced contract during state comparison, it will now raise an error for versioned models and a warning for models that are not versioned.
+- **[Set `access` as a config](/reference/resource-configs/access):** You can now set a model's `access` within config blocks in the model's file or in the `dbt_project.yml` for an entire subfolder at once.
+- **[Type aliasing for model contracts](/reference/resource-configs/contract):** dbt will use each adapter's built-in type aliasing for user-provided data types—meaning you can now write `string` always, and dbt will translate to `text` on Postgres/Redshift. This is "on" by default, but you can opt-out.
+- **[Raise warning for numeric types](/reference/resource-configs/contract):** Because of issues when putting `numeric` in model contracts without considering that default values such as `numeric(38,0)` might round decimals accordingly. dbt will now warn you if it finds a numeric type without specified precision/scale.
+
+### Quick hits
+
+With these quick hits, you can now:
+- Configure a [`delimiter`](/reference/resource-configs/delimiter) for a seed file.
+- Use packages with the same git repo and unique subdirectory.
+- Access the `date_spine` macro directly from dbt-core (moved over from dbt-utils).
diff --git a/website/docs/guides/migration/versions/01-upgrading-to-v1.6.md b/website/docs/docs/dbt-versions/core-upgrade/01-upgrading-to-v1.6.md
similarity index 93%
rename from website/docs/guides/migration/versions/01-upgrading-to-v1.6.md
rename to website/docs/docs/dbt-versions/core-upgrade/01-upgrading-to-v1.6.md
index bdb47bbf2ea..f62b6308ce6 100644
--- a/website/docs/guides/migration/versions/01-upgrading-to-v1.6.md
+++ b/website/docs/docs/dbt-versions/core-upgrade/01-upgrading-to-v1.6.md
@@ -1,8 +1,14 @@
---
-title: "Upgrading to v1.6 (latest)"
+title: "Upgrading to v1.6"
description: New features and changes in dbt Core v1.6
+id: "upgrading-to-v1.6"
+displayed_sidebar: "docs"
---
+import UpgradeMove from '/snippets/_upgrade-move.md';
+
+
+
dbt Core v1.6 has three significant areas of focus:
1. Next milestone of [multi-project deployments](https://github.com/dbt-labs/dbt-core/discussions/6725): improvements to contracts, groups/access, versions; and building blocks for cross-project `ref`
1. Semantic layer re-launch: dbt Core and [MetricFlow](https://docs.getdbt.com/docs/build/about-metricflow) integration
@@ -59,7 +65,7 @@ Supported on:
- [Postgres](/reference/resource-configs/postgres-configs#materialized-view)
- [Redshift](/reference/resource-configs/redshift-configs#materialized-view)
- [Snowflake](/reference/resource-configs/snowflake-configs#dynamic-tables)
-- Databricks (docs forthcoming)
+- [Databricks](/reference/resource-configs/databricks-configs#materialized-views-and-streaming-tables)
Support for BigQuery coming soon.
@@ -90,4 +96,5 @@ More consistency and flexibility around packages. Resources defined in a package
- [`dbt debug --connection`](/reference/commands/debug) to test just the data platform connection specified in a profile
- [`dbt docs generate --empty-catalog`](/reference/commands/cmd-docs) to skip catalog population while generating docs
- [`--defer-state`](/reference/node-selection/defer) enables more-granular control
+- [`dbt ls`](/reference/commands/list) adds the Semantic model selection method to allow for `dbt ls -s "semantic_model:*"` and the ability to execute `dbt ls --resource-type semantic_model`.
diff --git a/website/docs/guides/migration/versions/02-upgrading-to-v1.5.md b/website/docs/docs/dbt-versions/core-upgrade/02-upgrading-to-v1.5.md
similarity index 95%
rename from website/docs/guides/migration/versions/02-upgrading-to-v1.5.md
rename to website/docs/docs/dbt-versions/core-upgrade/02-upgrading-to-v1.5.md
index 0c7fc7ebcad..dded8a690fe 100644
--- a/website/docs/guides/migration/versions/02-upgrading-to-v1.5.md
+++ b/website/docs/docs/dbt-versions/core-upgrade/02-upgrading-to-v1.5.md
@@ -1,8 +1,14 @@
---
title: "Upgrading to v1.5"
description: New features and changes in dbt Core v1.5
+id: "upgrading-to-v1.5"
+displayed_sidebar: "docs"
---
+import UpgradeMove from '/snippets/_upgrade-move.md';
+
+
+
dbt Core v1.5 is a feature release, with two significant additions:
1. [**Model governance**](/docs/collaborate/govern/about-model-governance) — access, contracts, versions — the first phase of [multi-project deployments](https://github.com/dbt-labs/dbt-core/discussions/6725)
2. A Python entry point for [**programmatic invocations**](/reference/programmatic-invocations), at parity with the CLI
@@ -148,4 +154,4 @@ Run `dbt --help` to see new & improved help documentation :)
- The [`version: 2` top-level key](/reference/project-configs/version) is now **optional** in all YAML files. Also, the [`config-version: 2`](/reference/project-configs/config-version) and `version:` top-level keys are now optional in `dbt_project.yml` files.
- [Events and logging](/reference/events-logging): Added `node_relation` (`database`, `schema`, `identifier`) to the `node_info` dictionary, available on node-specific events
- Support setting `--project-dir` via environment variable: [`DBT_PROJECT_DIR`](/reference/dbt_project.yml)
-- More granular [configurations](/reference/global-configs/about-global-configs) for logging (to set log format, log levels, and colorization) and cache population
+- More granular configurations for logging (to set [log format](/reference/global-configs/logs#log-formatting), [log levels](/reference/global-configs/logs#log-level), and [colorization](/reference/global-configs/logs#color)) and [cache population](/reference/global-configs/cache#cache-population)
diff --git a/website/docs/guides/migration/versions/03-upgrading-to-dbt-utils-v1.0.md b/website/docs/docs/dbt-versions/core-upgrade/03-upgrading-to-dbt-utils-v1.0.md
similarity index 99%
rename from website/docs/guides/migration/versions/03-upgrading-to-dbt-utils-v1.0.md
rename to website/docs/docs/dbt-versions/core-upgrade/03-upgrading-to-dbt-utils-v1.0.md
index 72c6fc3c968..a7b302c9a58 100644
--- a/website/docs/guides/migration/versions/03-upgrading-to-dbt-utils-v1.0.md
+++ b/website/docs/docs/dbt-versions/core-upgrade/03-upgrading-to-dbt-utils-v1.0.md
@@ -3,6 +3,10 @@ title: "Upgrading to dbt utils v1.0"
description: New features and breaking changes to consider as you upgrade to dbt utils v1.0.
---
+import UpgradeMove from '/snippets/_upgrade-move.md';
+
+
+
# Upgrading to dbt utils v1.0
For the first time, [dbt utils](https://hub.getdbt.com/dbt-labs/dbt_utils/latest/) is crossing the major version boundary. From [last month’s blog post](https://www.getdbt.com/blog/announcing-dbt-v1.3-and-utils/):
diff --git a/website/docs/guides/migration/versions/04-upgrading-to-v1.4.md b/website/docs/docs/dbt-versions/core-upgrade/04-upgrading-to-v1.4.md
similarity index 97%
rename from website/docs/guides/migration/versions/04-upgrading-to-v1.4.md
rename to website/docs/docs/dbt-versions/core-upgrade/04-upgrading-to-v1.4.md
index 3537eb1677a..6c6d96b2326 100644
--- a/website/docs/guides/migration/versions/04-upgrading-to-v1.4.md
+++ b/website/docs/docs/dbt-versions/core-upgrade/04-upgrading-to-v1.4.md
@@ -1,7 +1,14 @@
---
title: "Upgrading to v1.4"
description: New features and changes in dbt Core v1.4
+id: "upgrading-to-v1.4"
+displayed_sidebar: "docs"
---
+
+import UpgradeMove from '/snippets/_upgrade-move.md';
+
+
+
### Resources
- [Changelog](https://github.com/dbt-labs/dbt-core/blob/1.4.latest/CHANGELOG.md)
diff --git a/website/docs/guides/migration/versions/05-upgrading-to-v1.3.md b/website/docs/docs/dbt-versions/core-upgrade/05-upgrading-to-v1.3.md
similarity index 97%
rename from website/docs/guides/migration/versions/05-upgrading-to-v1.3.md
rename to website/docs/docs/dbt-versions/core-upgrade/05-upgrading-to-v1.3.md
index 5fdf559a267..f66d9bb9706 100644
--- a/website/docs/guides/migration/versions/05-upgrading-to-v1.3.md
+++ b/website/docs/docs/dbt-versions/core-upgrade/05-upgrading-to-v1.3.md
@@ -1,7 +1,14 @@
---
title: "Upgrading to v1.3"
description: New features and changes in dbt Core v1.3
+id: "upgrading-to-v1.3"
+displayed_sidebar: "docs"
---
+
+import UpgradeMove from '/snippets/_upgrade-move.md';
+
+
+
### Resources
- [Changelog](https://github.com/dbt-labs/dbt-core/blob/1.3.latest/CHANGELOG.md)
diff --git a/website/docs/guides/migration/versions/06-upgrading-to-v1.2.md b/website/docs/docs/dbt-versions/core-upgrade/06-upgrading-to-v1.2.md
similarity index 96%
rename from website/docs/guides/migration/versions/06-upgrading-to-v1.2.md
rename to website/docs/docs/dbt-versions/core-upgrade/06-upgrading-to-v1.2.md
index 91ffadf9093..16825ff4e2b 100644
--- a/website/docs/guides/migration/versions/06-upgrading-to-v1.2.md
+++ b/website/docs/docs/dbt-versions/core-upgrade/06-upgrading-to-v1.2.md
@@ -1,7 +1,14 @@
---
title: "Upgrading to v1.2"
description: New features and changes in dbt Core v1.2
+id: "upgrading-to-v1.2"
+displayed_sidebar: "docs"
---
+
+import UpgradeMove from '/snippets/_upgrade-move.md';
+
+
+
### Resources
- [Changelog](https://github.com/dbt-labs/dbt-core/blob/1.2.latest/CHANGELOG.md)
diff --git a/website/docs/guides/migration/versions/07-upgrading-to-v1.1.md b/website/docs/docs/dbt-versions/core-upgrade/07-upgrading-to-v1.1.md
similarity index 97%
rename from website/docs/guides/migration/versions/07-upgrading-to-v1.1.md
rename to website/docs/docs/dbt-versions/core-upgrade/07-upgrading-to-v1.1.md
index 131ecc97657..7819709558e 100644
--- a/website/docs/guides/migration/versions/07-upgrading-to-v1.1.md
+++ b/website/docs/docs/dbt-versions/core-upgrade/07-upgrading-to-v1.1.md
@@ -1,7 +1,14 @@
---
title: "Upgrading to v1.1"
description: New features and changes in dbt Core v1.1
+id: "upgrading-to-v1.1"
+displayed_sidebar: "docs"
---
+
+import UpgradeMove from '/snippets/_upgrade-move.md';
+
+
+
### Resources
- [Changelog](https://github.com/dbt-labs/dbt-core/blob/1.1.latest/CHANGELOG.md)
diff --git a/website/docs/guides/migration/versions/08-upgrading-to-v1.0.md b/website/docs/docs/dbt-versions/core-upgrade/08-upgrading-to-v1.0.md
similarity index 98%
rename from website/docs/guides/migration/versions/08-upgrading-to-v1.0.md
rename to website/docs/docs/dbt-versions/core-upgrade/08-upgrading-to-v1.0.md
index 9fc7991c087..7c67a1849a1 100644
--- a/website/docs/guides/migration/versions/08-upgrading-to-v1.0.md
+++ b/website/docs/docs/dbt-versions/core-upgrade/08-upgrading-to-v1.0.md
@@ -1,7 +1,14 @@
---
title: "Upgrading to v1.0"
description: New features and changes in dbt Core v1.0
+id: "upgrading-to-v1.0"
+displayed_sidebar: "docs"
---
+
+import UpgradeMove from '/snippets/_upgrade-move.md';
+
+
+
### Resources
- [Discourse](https://discourse.getdbt.com/t/3180)
diff --git a/website/docs/guides/migration/versions/09-upgrading-to-v0.21.md b/website/docs/docs/dbt-versions/core-upgrade/09-upgrading-to-v0.21.md
similarity index 97%
rename from website/docs/guides/migration/versions/09-upgrading-to-v0.21.md
rename to website/docs/docs/dbt-versions/core-upgrade/09-upgrading-to-v0.21.md
index e5fbdf3fc7c..d5b429132cd 100644
--- a/website/docs/guides/migration/versions/09-upgrading-to-v0.21.md
+++ b/website/docs/docs/dbt-versions/core-upgrade/09-upgrading-to-v0.21.md
@@ -1,8 +1,15 @@
---
title: "Upgrading to v0.21"
+id: "upgrading-to-v0.21"
+displayed_sidebar: "docs"
---
+import UpgradeMove from '/snippets/_upgrade-move.md';
+
+
+
+
:::caution Unsupported version
dbt Core v0.21 has reached the end of critical support. No new patch versions will be released, and it will stop running in dbt Cloud on June 30, 2022. Read ["About dbt Core versions"](/docs/dbt-versions/core) for more details.
:::
diff --git a/website/docs/guides/migration/versions/10-upgrading-to-v0.20.md b/website/docs/docs/dbt-versions/core-upgrade/10-upgrading-to-v0.20.md
similarity index 96%
rename from website/docs/guides/migration/versions/10-upgrading-to-v0.20.md
rename to website/docs/docs/dbt-versions/core-upgrade/10-upgrading-to-v0.20.md
index 8b33bfa3879..61a7120370a 100644
--- a/website/docs/guides/migration/versions/10-upgrading-to-v0.20.md
+++ b/website/docs/docs/dbt-versions/core-upgrade/10-upgrading-to-v0.20.md
@@ -1,8 +1,13 @@
---
title: "Upgrading to v0.20"
-
+id: "upgrading-to-v0.20"
+displayed_sidebar: "docs"
---
+import UpgradeMove from '/snippets/_upgrade-move.md';
+
+
+
:::caution Unsupported version
dbt Core v0.20 has reached the end of critical support. No new patch versions will be released, and it will stop running in dbt Cloud on June 30, 2022. Read ["About dbt Core versions"](/docs/dbt-versions/core) for more details.
:::
diff --git a/website/docs/guides/migration/versions/11-Older versions/upgrading-to-0-11-0.md b/website/docs/docs/dbt-versions/core-upgrade/11-Older versions/upgrading-to-0-11-0.md
similarity index 95%
rename from website/docs/guides/migration/versions/11-Older versions/upgrading-to-0-11-0.md
rename to website/docs/docs/dbt-versions/core-upgrade/11-Older versions/upgrading-to-0-11-0.md
index e307c46fdf9..e91dde4c923 100644
--- a/website/docs/guides/migration/versions/11-Older versions/upgrading-to-0-11-0.md
+++ b/website/docs/docs/dbt-versions/core-upgrade/11-Older versions/upgrading-to-0-11-0.md
@@ -1,8 +1,13 @@
---
title: "Upgrading to 0.11.0"
id: "upgrading-to-0-11-0"
+displayed_sidebar: "docs"
---
+import UpgradeMove from '/snippets/_upgrade-move.md';
+
+
+
## Schema.yml v2 syntax
dbt v0.11.0 adds an auto-generated docs site to your dbt project. To make effective use of the documentation site, you'll need to use the new "version 2" schema.yml syntax. For a full explanation of the version 2 syntax, check out the [schema.yml Files](/reference/configs-and-properties) section of the documentation.
diff --git a/website/docs/guides/migration/versions/11-Older versions/upgrading-to-0-12-0.md b/website/docs/docs/dbt-versions/core-upgrade/11-Older versions/upgrading-to-0-12-0.md
similarity index 76%
rename from website/docs/guides/migration/versions/11-Older versions/upgrading-to-0-12-0.md
rename to website/docs/docs/dbt-versions/core-upgrade/11-Older versions/upgrading-to-0-12-0.md
index 60900d3c1a4..b3d4e9d9bcb 100644
--- a/website/docs/guides/migration/versions/11-Older versions/upgrading-to-0-12-0.md
+++ b/website/docs/docs/dbt-versions/core-upgrade/11-Older versions/upgrading-to-0-12-0.md
@@ -1,8 +1,13 @@
---
title: "Upgrading to 0.12.0"
id: "upgrading-to-0-12-0"
+displayed_sidebar: "docs"
---
+import UpgradeMove from '/snippets/_upgrade-move.md';
+
+
+
## End of support
Support for the `repositories:` block in `dbt_project.yml` (deprecated in 0.10.0) was removed.
diff --git a/website/docs/guides/migration/versions/11-Older versions/upgrading-to-0-13-0.md b/website/docs/docs/dbt-versions/core-upgrade/11-Older versions/upgrading-to-0-13-0.md
similarity index 94%
rename from website/docs/guides/migration/versions/11-Older versions/upgrading-to-0-13-0.md
rename to website/docs/docs/dbt-versions/core-upgrade/11-Older versions/upgrading-to-0-13-0.md
index 14a70e177e8..bb15d1a73b0 100644
--- a/website/docs/guides/migration/versions/11-Older versions/upgrading-to-0-13-0.md
+++ b/website/docs/docs/dbt-versions/core-upgrade/11-Older versions/upgrading-to-0-13-0.md
@@ -1,8 +1,13 @@
---
title: "Upgrading to 0.13.0"
id: "upgrading-to-0-13-0"
+displayed_sidebar: "docs"
---
+import UpgradeMove from '/snippets/_upgrade-move.md';
+
+
+
## Breaking changes
### on-run-start and on-run-end
diff --git a/website/docs/guides/migration/versions/11-Older versions/upgrading-to-0-14-0.md b/website/docs/docs/dbt-versions/core-upgrade/11-Older versions/upgrading-to-0-14-0.md
similarity index 99%
rename from website/docs/guides/migration/versions/11-Older versions/upgrading-to-0-14-0.md
rename to website/docs/docs/dbt-versions/core-upgrade/11-Older versions/upgrading-to-0-14-0.md
index 3b9c8560230..036a9a2aedf 100644
--- a/website/docs/guides/migration/versions/11-Older versions/upgrading-to-0-14-0.md
+++ b/website/docs/docs/dbt-versions/core-upgrade/11-Older versions/upgrading-to-0-14-0.md
@@ -1,8 +1,13 @@
---
title: "Upgrading to 0.14.0"
id: "upgrading-to-0-14-0"
+displayed_sidebar: "docs"
---
+import UpgradeMove from '/snippets/_upgrade-move.md';
+
+
+
This guide outlines migration instructions for:
1. [Upgrading archives to snapshots](#upgrading-to-snapshot-blocks)
diff --git a/website/docs/guides/migration/versions/11-Older versions/upgrading-to-0-14-1.md b/website/docs/docs/dbt-versions/core-upgrade/11-Older versions/upgrading-to-0-14-1.md
similarity index 98%
rename from website/docs/guides/migration/versions/11-Older versions/upgrading-to-0-14-1.md
rename to website/docs/docs/dbt-versions/core-upgrade/11-Older versions/upgrading-to-0-14-1.md
index a81740d5a68..215385acf0f 100644
--- a/website/docs/guides/migration/versions/11-Older versions/upgrading-to-0-14-1.md
+++ b/website/docs/docs/dbt-versions/core-upgrade/11-Older versions/upgrading-to-0-14-1.md
@@ -1,8 +1,13 @@
---
title: "Upgrading to 0.14.1"
id: "upgrading-to-0-14-1"
+displayed_sidebar: "docs"
---
+import UpgradeMove from '/snippets/_upgrade-move.md';
+
+
+
The dbt v0.14.1 release _does not_ contain any breaking code changes for users upgrading from v0.14.0. If you are upgrading from a version less than 0.14.0, consult the [Upgrading to 0.14.0](upgrading-to-0-14-0) migration guide. The following section contains important information for users of the `check` strategy on Snowflake and BigQuery. Action may be required in your database.
## Changes to the Snapshot "check" algorithm
diff --git a/website/docs/guides/migration/versions/11-Older versions/upgrading-to-0-15-0.md b/website/docs/docs/dbt-versions/core-upgrade/11-Older versions/upgrading-to-0-15-0.md
similarity index 93%
rename from website/docs/guides/migration/versions/11-Older versions/upgrading-to-0-15-0.md
rename to website/docs/docs/dbt-versions/core-upgrade/11-Older versions/upgrading-to-0-15-0.md
index 02ab297c07a..6dd2b6fb9eb 100644
--- a/website/docs/guides/migration/versions/11-Older versions/upgrading-to-0-15-0.md
+++ b/website/docs/docs/dbt-versions/core-upgrade/11-Older versions/upgrading-to-0-15-0.md
@@ -1,10 +1,16 @@
---
title: "Upgrading to 0.15.0"
id: "upgrading-to-0-15-0"
+displayed_sidebar: "docs"
---
+import UpgradeMove from '/snippets/_upgrade-move.md';
+
+
+
The dbt v0.15.0 release contains a handful of breaking code changes for users upgrading from v0.14.0.
+
## Breaking changes
### Stricter YML compilation
diff --git a/website/docs/guides/migration/versions/11-Older versions/upgrading-to-0-16-0.md b/website/docs/docs/dbt-versions/core-upgrade/11-Older versions/upgrading-to-0-16-0.md
similarity index 98%
rename from website/docs/guides/migration/versions/11-Older versions/upgrading-to-0-16-0.md
rename to website/docs/docs/dbt-versions/core-upgrade/11-Older versions/upgrading-to-0-16-0.md
index a34f23c4c89..076e6fc4e88 100644
--- a/website/docs/guides/migration/versions/11-Older versions/upgrading-to-0-16-0.md
+++ b/website/docs/docs/dbt-versions/core-upgrade/11-Older versions/upgrading-to-0-16-0.md
@@ -1,8 +1,13 @@
---
title: "Upgrading to 0.16.0"
id: "upgrading-to-0-16-0"
+displayed_sidebar: "docs"
---
+import UpgradeMove from '/snippets/_upgrade-move.md';
+
+
+
dbt v0.16.0 contains many new features, bug fixes, and improvements. This guide
covers all of the important information to consider when upgrading from an earlier
version of dbt to 0.16.0.
diff --git a/website/docs/guides/migration/versions/11-Older versions/upgrading-to-0-17-0.md b/website/docs/docs/dbt-versions/core-upgrade/11-Older versions/upgrading-to-0-17-0.md
similarity index 98%
rename from website/docs/guides/migration/versions/11-Older versions/upgrading-to-0-17-0.md
rename to website/docs/docs/dbt-versions/core-upgrade/11-Older versions/upgrading-to-0-17-0.md
index 1f891ebc0f4..5b863777df9 100644
--- a/website/docs/guides/migration/versions/11-Older versions/upgrading-to-0-17-0.md
+++ b/website/docs/docs/dbt-versions/core-upgrade/11-Older versions/upgrading-to-0-17-0.md
@@ -1,9 +1,14 @@
---
title: "Upgrading to 0.17.0"
id: "upgrading-to-0-17-0"
+displayed_sidebar: "docs"
---
+import UpgradeMove from '/snippets/_upgrade-move.md';
+
+
+
dbt v0.17.0 makes compilation more consistent, improves performance, and fixes a number of bugs.
## Articles:
diff --git a/website/docs/guides/migration/versions/11-Older versions/upgrading-to-0-18-0.md b/website/docs/docs/dbt-versions/core-upgrade/11-Older versions/upgrading-to-0-18-0.md
similarity index 97%
rename from website/docs/guides/migration/versions/11-Older versions/upgrading-to-0-18-0.md
rename to website/docs/docs/dbt-versions/core-upgrade/11-Older versions/upgrading-to-0-18-0.md
index 8092ad807b8..545bfd41ac6 100644
--- a/website/docs/guides/migration/versions/11-Older versions/upgrading-to-0-18-0.md
+++ b/website/docs/docs/dbt-versions/core-upgrade/11-Older versions/upgrading-to-0-18-0.md
@@ -1,8 +1,13 @@
---
title: "Upgrading to 0.18.0"
+displayed_sidebar: "docs"
---
+import UpgradeMove from '/snippets/_upgrade-move.md';
+
+
+
### Resources
- [Changelog](https://github.com/dbt-labs/dbt-core/blob/dev/marian-anderson/CHANGELOG.md)
diff --git a/website/docs/guides/migration/versions/11-Older versions/upgrading-to-0-19-0.md b/website/docs/docs/dbt-versions/core-upgrade/11-Older versions/upgrading-to-0-19-0.md
similarity index 96%
rename from website/docs/guides/migration/versions/11-Older versions/upgrading-to-0-19-0.md
rename to website/docs/docs/dbt-versions/core-upgrade/11-Older versions/upgrading-to-0-19-0.md
index 0dd428780e0..db825d8af9c 100644
--- a/website/docs/guides/migration/versions/11-Older versions/upgrading-to-0-19-0.md
+++ b/website/docs/docs/dbt-versions/core-upgrade/11-Older versions/upgrading-to-0-19-0.md
@@ -1,8 +1,13 @@
---
title: "Upgrading to 0.19.0"
+displayed_sidebar: "docs"
---
+import UpgradeMove from '/snippets/_upgrade-move.md';
+
+
+
### Resources
- [Discourse](https://discourse.getdbt.com/t/1951)
@@ -23,7 +28,7 @@ See the docs below for more details. We don't expect these to require action in
#### Deprecations
-Removed support for `config-version: 1` of dbt_project.yml, which was deprecated in v0.17.0. Use `config-version: 2` in all projects and installed packages. Otherwise, dbt will raise an error. See docs on [config-version](/reference/project-configs/config-version) and the [v0.17.0 Migration Guide](/guides/migration/versions) for details.
+Removed support for `config-version: 1` of dbt_project.yml, which was deprecated in v0.17.0. Use `config-version: 2` in all projects and installed packages. Otherwise, dbt will raise an error. See docs on [config-version](/reference/project-configs/config-version) and the [v0.17.0 Migration Guide](/docs/dbt-versions/core-upgrade) for details.
### For dbt plugin maintainers
diff --git a/website/docs/docs/dbt-versions/core-versions.md b/website/docs/docs/dbt-versions/core-versions.md
index 2a5ce6daeb7..5e8e437f0b1 100644
--- a/website/docs/docs/dbt-versions/core-versions.md
+++ b/website/docs/docs/dbt-versions/core-versions.md
@@ -2,6 +2,8 @@
title: "About dbt Core versions"
id: "core"
description: "Learn about semantic versioning for dbt Core, and how long those versions are supported."
+pagination_next: "docs/dbt-versions/upgrade-core-in-cloud"
+pagination_prev: null
---
dbt Core releases follow [semantic versioning](https://semver.org/) guidelines. For more on how we use semantic versions, see [How dbt Core uses semantic versioning](#how-dbt-core-uses-semantic-versioning).
diff --git a/website/docs/docs/dbt-versions/experimental-features.md b/website/docs/docs/dbt-versions/experimental-features.md
index 5ed0cf037ca..a621bd4ac44 100644
--- a/website/docs/docs/dbt-versions/experimental-features.md
+++ b/website/docs/docs/dbt-versions/experimental-features.md
@@ -3,6 +3,7 @@ title: "Preview new and experimental features in dbt Cloud"
id: "experimental-features"
sidebar_label: "Preview new dbt Cloud features"
description: "Gain early access to many new dbt Labs experimental features by enabling this in your profile."
+pagination_next: null
---
dbt Labs often tests experimental features before deciding to continue on the [Product lifecycle](https://docs.getdbt.com/docs/dbt-versions/product-lifecycles#dbt-cloud).
diff --git a/website/docs/docs/dbt-versions/release-notes.md b/website/docs/docs/dbt-versions/release-notes.md
index db25af163ae..6f7be90e60d 100644
--- a/website/docs/docs/dbt-versions/release-notes.md
+++ b/website/docs/docs/dbt-versions/release-notes.md
@@ -2,6 +2,8 @@
title: "About dbt Cloud Release Notes"
id: "dbt-cloud-release-notes"
description: "Release notes for dbt Cloud"
+pagination_next: null
+pagination_prev: null
---
dbt provides release notes for dbt Cloud so you can see recent and historical changes. Generally, you'll see release notes for these changes:
diff --git a/website/docs/docs/dbt-versions/release-notes/03-Oct-2023/api-v2v3-limit.md b/website/docs/docs/dbt-versions/release-notes/03-Oct-2023/api-v2v3-limit.md
new file mode 100644
index 00000000000..9768886d5fb
--- /dev/null
+++ b/website/docs/docs/dbt-versions/release-notes/03-Oct-2023/api-v2v3-limit.md
@@ -0,0 +1,15 @@
+---
+title: "API results limited to `100`"
+id: apiv3-limit"
+description: "Oct 2023: In order to enhance the efficiency and stability of our services, we will limit all API results to `100` records. This limit is applicable to multi-tenant instances only."
+sidebar_label: "Update: API results limited to `100`"
+sidebar_position: 04
+tags: [Oct-2023, API]
+---
+
+
+Beginning December 1, 2023, the [Administrative API](/docs/dbt-cloud-apis/admin-cloud-api) v2 and v3 will expect you to limit all "list" or `GET` API methods to 100 results per API request. This limit enhances the efficiency and stability of our services. If you need to handle more than 100 results, then use the `limit` and `offset` query parameters to paginate those results; otherwise, you will receive an error.
+
+This maximum limit applies to [multi-tenant instances](/docs/cloud/about-cloud/regions-ip-addresses) only, and _does not_ apply to single tenant instances.
+
+Refer to the [API v3 Pagination](https://docs.getdbt.com/dbt-cloud/api-v3#/) or [API v2 Pagination](https://docs.getdbt.com/dbt-cloud/api-v2#/) sections for more information on how to paginate your API responses.
diff --git a/website/docs/docs/dbt-versions/release-notes/03-Oct-2023/cloud-cli-pp.md b/website/docs/docs/dbt-versions/release-notes/03-Oct-2023/cloud-cli-pp.md
new file mode 100644
index 00000000000..d96b82636f8
--- /dev/null
+++ b/website/docs/docs/dbt-versions/release-notes/03-Oct-2023/cloud-cli-pp.md
@@ -0,0 +1,31 @@
+---
+title: "New: dbt Cloud CLI in Public Preview"
+description: "October 2023: Learn about the new dbt Cloud CLI development experience, now in public preview,"
+sidebar_position: 04
+sidebar_label: "New: dbt Cloud CLI in Public Preview"
+tags: [Oct-2023, CLI, dbt Cloud]
+date: 2023-10-17
+---
+
+We are excited to announce the dbt Cloud CLI, **unified command line for dbt**, is available in public preview. It’s a local development experience, powered by dbt Cloud. It’s easy to get started: `pip3 install dbt` or `brew install dbt` and you’re ready to go.
+
+We will continue to invest in the dbt Cloud IDE as the easiest and most accessible way to get started using dbt, especially for data analysts who have never developed software using the command line before. We will keep improving the speed, stability, and feature richness of the IDE, as we have been [all year long](https://www.getdbt.com/blog/improvements-to-the-dbt-cloud-ide/).
+
+We also know that many people developing in dbt have a preference for local development, where they can use their favorite terminal, text editor, keybindings, color scheme, and so on. This includes people with data engineering backgrounds, as well as those analytics engineers who started writing code in the dbt Cloud IDE and have expanded their skills.
+
+The new dbt Cloud CLI offers the best of both worlds, including:
+
+- The power of developing against the dbt Cloud platform
+- The flexibility of your own local setup
+
+Run whichever community-developed plugins, pre-commit hooks, or other arbitrary scripts you like.
+
+Some of the unique capabilities of this dbt Cloud CLI include:
+
+- Automatic deferral of build artifacts to your Cloud project's production environment
+- Secure credential storage in the dbt Cloud platform
+- Support for dbt Mesh ([cross-project `ref`](/docs/collaborate/govern/project-dependencies))
+- Development workflow for dbt Semantic Layer
+- Speedier, lower cost builds
+
+Refer to [dbt Cloud CLI](/docs/cloud/cloud-cli-installation) to learn more.
diff --git a/website/docs/docs/dbt-versions/release-notes/03-Oct-2023/custom-branch-fix-rn.md b/website/docs/docs/dbt-versions/release-notes/03-Oct-2023/custom-branch-fix-rn.md
new file mode 100644
index 00000000000..06550b7d863
--- /dev/null
+++ b/website/docs/docs/dbt-versions/release-notes/03-Oct-2023/custom-branch-fix-rn.md
@@ -0,0 +1,14 @@
+---
+title: "Fix: Default behavior for CI job runs without a custom branch"
+description: "October 2023: CI job runs now default to the main branch of the Git repository when a custom branch isn't set"
+sidebar_label: "Fix: Default behavior for CI job runs without a custom branch"
+tags: [Oct-2023, CI]
+date: 2023-10-06
+sidebar_position: 08
+---
+
+If you don't set a [custom branch](/docs/dbt-cloud-environments#custom-branch-behavior) for your dbt Cloud environment, it now defaults to the default branch of your Git repository (for example, `main`). Previously, [CI jobs](/docs/deploy/ci-jobs) would run for pull requests (PRs) that were opened against _any branch_ or updated with new commits if the **Custom Branch** option wasn't set.
+
+## Azure DevOps
+
+Your Git pull requests (PRs) might not trigger against your default branch if you're using Azure DevOps and the default branch isn't `main` or `master`. To resolve this, [set up a custom branch](/faqs/Environments/custom-branch-settings) with the branch you want to target.
diff --git a/website/docs/docs/dbt-versions/release-notes/03-Oct-2023/dbt-deps-auto-install.md b/website/docs/docs/dbt-versions/release-notes/03-Oct-2023/dbt-deps-auto-install.md
new file mode 100644
index 00000000000..80963a9d550
--- /dev/null
+++ b/website/docs/docs/dbt-versions/release-notes/03-Oct-2023/dbt-deps-auto-install.md
@@ -0,0 +1,21 @@
+---
+title: "Enhancement: dbt Cloud auto-installs 'dbt deps' on startup"
+description: "October 2023 :The dbt Cloud IDE and dbt Cloud CLI auto-handles 'dbt deps' on startup; manual run needed for 'packages.yml' changes. Available for multi-tenant users (single-tenant support coming soon) and applies to all dbt versions."
+sidebar_label: "Enhancement: dbt Cloud auto-installs 'dbt deps' on startup"
+tags: [Oct-2023, IDE]
+date: 2023-10-17
+sidebar_position: 06
+---
+
+The dbt Cloud IDE and dbt Cloud CLI now automatically installs `dbt deps` when your environment starts or when necessary. Previously, it would prompt you to run `dbt deps` during initialization.
+
+This improved workflow is available to all multi-tenant dbt Cloud users (Single-tenant support coming next week) and applies to dbt versions.
+
+However, you should still run the `dbt deps` command in these situations:
+
+- When you make changes to the `packages.yml` or `dependencies.yml` file during a session
+- When you update the package version in the `packages.yml` or `dependencies.yml` file.
+- If you edit the `dependencies.yml` file and the number of packages remains the same, run `dbt deps`. (Note that this is a known bug dbt Labs will fix in the future.)
+
+
+
diff --git a/website/docs/docs/dbt-versions/release-notes/03-Oct-2023/explorer-public-preview-rn.md b/website/docs/docs/dbt-versions/release-notes/03-Oct-2023/explorer-public-preview-rn.md
new file mode 100644
index 00000000000..ebf5add8d03
--- /dev/null
+++ b/website/docs/docs/dbt-versions/release-notes/03-Oct-2023/explorer-public-preview-rn.md
@@ -0,0 +1,13 @@
+---
+title: "New: dbt Explorer Public Preview"
+description: "October 2023: dbt Explorer is now available in Public Preview. You can use it to understand, improve, and leverage your dbt projects."
+sidebar_label: "New: dbt Explorer Public Preview"
+tags: [Oct-2023, Explorer]
+date: 2023-10-13
+sidebar_position: 07
+---
+
+On Oct 17, 2023, a Public Preview of dbt Explorer will become available to dbt Cloud customers. With dbt Explorer, you can view your project's resources (such as models, tests, and metrics) and their lineage — including interactive DAGs — to gain a better understanding of its latest production state. Navigate and manage your projects within dbt Cloud to help you and other data developers, analysts, and consumers discover and leverage your dbt resources.
+
+For details, refer to [Explore your dbt projects](/docs/collaborate/explore-projects).
+
diff --git a/website/docs/docs/dbt-versions/release-notes/03-Oct-2023/native-retry-support-rn.md b/website/docs/docs/dbt-versions/release-notes/03-Oct-2023/native-retry-support-rn.md
new file mode 100644
index 00000000000..20e56879940
--- /dev/null
+++ b/website/docs/docs/dbt-versions/release-notes/03-Oct-2023/native-retry-support-rn.md
@@ -0,0 +1,15 @@
+---
+title: "Enhancement: Native support for the dbt retry command"
+description: "October 2023: Rerun errored jobs from start or from the failure point"
+sidebar_label: "Enhancement: Support for dbt retry"
+tags: [Oct-2023, Scheduler]
+date: 2023-10-06
+sidebar_position: 10
+---
+
+Previously in dbt Cloud, you could only rerun an errored job from start but now you can also rerun it from its point of failure.
+
+You can view which job failed to complete successully, which command failed in the run step, and choose how to rerun it. To learn more, refer to [Retry jobs](/docs/deploy/retry-jobs).
+
+
+
diff --git a/website/docs/docs/dbt-versions/release-notes/03-Oct-2023/product-docs-sept-rn.md b/website/docs/docs/dbt-versions/release-notes/03-Oct-2023/product-docs-sept-rn.md
new file mode 100644
index 00000000000..e669b037d17
--- /dev/null
+++ b/website/docs/docs/dbt-versions/release-notes/03-Oct-2023/product-docs-sept-rn.md
@@ -0,0 +1,38 @@
+---
+title: "September 2023 product docs updates"
+id: "product-docs-sept"
+description: "September 2023: The Product docs team merged 107 PRs, made various updates to dbt Cloud and Core, such as GAing continuous integration jobs, Semantic Layer GraphQL API doc, a new community plugin, and more"
+sidebar_label: "Update: Product docs changes"
+tags: [Sept-2023, product-docs]
+date: 2023-10-10
+sidebar_position: 09
+---
+
+Hello from the dbt Docs team: @mirnawong1, @matthewshaver, @nghi-ly, and @runleonarun! First, we’d like to thank the 15 new community contributors to docs.getdbt.com. We merged [107 PRs](https://github.com/dbt-labs/docs.getdbt.com/pulls?q=is%3Apr+merged%3A2023-09-01..2023-09-31) in September.
+
+Here's what's new to [docs.getdbt.com](http://docs.getdbt.com/):
+
+* Migrated docs.getdbt.com from Netlify to Vercel.
+
+## ☁ Cloud projects
+- Continuous integration jobs are now generally available and no longer in beta!
+- Added [Postgres PrivateLink set up page](/docs/cloud/secure/postgres-privatelink)
+- Published beta docs for [dbt Explorer](/docs/collaborate/explore-projects).
+- Added a new Semantic Layer [GraphQL API doc](/docs/dbt-cloud-apis/sl-graphql) and updated the [integration docs](/docs/use-dbt-semantic-layer/avail-sl-integrations) to include Hex. Responded to dbt community feedback and clarified Metricflow use cases for dbt Core and dbt Cloud.
+- Added an [FAQ](/faqs/Git/git-migration) describing how to migrate from one git provider to another in dbt Cloud.
+- Clarified an example and added a [troubleshooting section](/docs/cloud/connect-data-platform/connect-snowflake#troubleshooting) to Snowflake connection docs to address common errors and provide solutions.
+
+
+## 🎯 Core projects
+
+- Deprecated dbt Core v1.0 and v1.1 from the docs.
+- Added configuration instructions for the [AWS Glue](/docs/core/connect-data-platform/glue-setup) community plugin.
+- Revised the dbt Core quickstart, making it easier to follow. Divided this guide into steps that align with the [other guides](/quickstarts/manual-install?step=1).
+
+## New 📚 Guides, ✏️ blog posts, and FAQs
+
+Added a [style guide template](/guides/best-practices/how-we-style/6-how-we-style-conclusion#style-guide-template) that you can copy & paste to make sure you adhere to best practices when styling dbt projects!
+
+## Upcoming changes
+
+Stay tuned for a flurry of releases in October and a filterable guides section that will make guides easier to find!
diff --git a/website/docs/docs/dbt-versions/release-notes/03-Oct-2023/sl-ga.md b/website/docs/docs/dbt-versions/release-notes/03-Oct-2023/sl-ga.md
new file mode 100644
index 00000000000..5e53363f62a
--- /dev/null
+++ b/website/docs/docs/dbt-versions/release-notes/03-Oct-2023/sl-ga.md
@@ -0,0 +1,29 @@
+---
+title: "Update: dbt Cloud Semantic Layer is Generally Available"
+description: "October 2023: dbt Cloud Semantic Layer is Generally Available for all users"
+sidebar_label: "Update: dbt Cloud Semantic Layer is GA"
+sidebar_position: 05
+date: 2023-10-17
+tags: [Oct-2023]
+---
+
+:::important
+If you're using the legacy Semantic Layer, we **highly** recommend you [upgrade your dbt version](/docs/dbt-versions/upgrade-core-in-cloud) to dbt v1.6 or higher and [migrate](/guides/migration/sl-migration) to the latest Semantic Layer.
+:::
+
+dbt Labs is thrilled to announce that the [dbt Semantic Layer](/docs/use-dbt-semantic-layer/dbt-sl) is now generally available. It offers consistent data organization, improved governance, reduced costs, enhanced efficiency, and accessible data for better decision-making and collaboration across organizations.
+
+It aims to bring the best of modeling and semantics to downstream applications by introducing:
+
+- Brand new [integrations](/docs/use-dbt-semantic-layer/avail-sl-integrations) such as Tableau, Google Sheets, Hex, Mode, and Lightdash.
+- New [Semantic Layer APIs](/docs/dbt-cloud-apis/sl-api-overview) using GraphQL and JDBC to query metrics and build integrations.
+- dbt Cloud [multi-tenant regional](/docs/cloud/about-cloud/regions-ip-addresses) support for North America, EMEA, and APAC. Single-tenant support coming soon.
+- Use the APIs to call an export (a way to build tables in your data platform), then access them in your preferred BI tool. Starting from dbt v1.7 or higher, you will be able to schedule exports as part of your dbt job.
+
+
+
+The dbt Semantic Layer is available to [dbt Cloud Team or Enterprise](https://www.getdbt.com/) multi-tenant plans on dbt v1.6 or higher.
+- Team and Enterprise customers can use 1,000 Queried Units per month for no additional cost on a limited trial basis, subject to reasonable use limitations. Refer to [Billing](/docs/cloud/billing#what-counts-as-a-query-unit) for more information.
+- dbt Cloud Developer plans and dbt Core users can define metrics but won't be able to query them with integrated tools.
+
+
diff --git a/website/docs/docs/dbt-versions/release-notes/04-Sept-2023/product-docs-summer-rn.md b/website/docs/docs/dbt-versions/release-notes/04-Sept-2023/product-docs-summer-rn.md
index a647bb5f585..d8148542eef 100644
--- a/website/docs/docs/dbt-versions/release-notes/04-Sept-2023/product-docs-summer-rn.md
+++ b/website/docs/docs/dbt-versions/release-notes/04-Sept-2023/product-docs-summer-rn.md
@@ -35,7 +35,7 @@ You can provide feedback by opening a pull request or issue in [our repo](https:
* Added a section to introduce a new beta feature [**Extended Attributes**](/docs/dbt-cloud-environments#extended-attributes-beta), which allows users to set a flexible `profiles.yml` snippet in their dbt Cloud Environment settings.
## 🎯 Core projects
-* We released [dbt 1.6](/guides/migration/versions/upgrading-to-v1.6)! We added docs for the new commands `dbt retry` and `dbt clone`
+* We released [dbt 1.6](/docs/dbt-versions/core-upgrade/upgrading-to-v1.6)! We added docs for the new commands `dbt retry` and `dbt clone`
## New 📚 Guides, ✏️ blog posts, and FAQs
* Check out how these community members use the dbt community in the [Community spotlight](/community/spotlight).
diff --git a/website/docs/docs/dbt-versions/release-notes/09-April-2023/product-docs.md b/website/docs/docs/dbt-versions/release-notes/09-April-2023/product-docs.md
index d30bcf85b99..d78040ea7e4 100644
--- a/website/docs/docs/dbt-versions/release-notes/09-April-2023/product-docs.md
+++ b/website/docs/docs/dbt-versions/release-notes/09-April-2023/product-docs.md
@@ -26,7 +26,7 @@ Hello from the dbt Docs team: @mirnawong1, @matthewshaver, @nghi-ly, and @runleo
## 🎯 Core projects
- Clearer descriptions in the [Jinja functions page](/reference/dbt-jinja-functions), that improve content for each card.
-- [1.5 Docs](/guides/migration/versions/upgrading-to-v1.5) have been released as an RC!
+- [1.5 Docs](/docs/dbt-versions/core-upgrade/upgrading-to-v1.5) have been released as an RC!
- See the beautiful [work captured in Core v 1.5](https://github.com/dbt-labs/docs.getdbt.com/issues?q=is%3Aissue+label%3A%22dbt-core+v1.5%22+is%3Aclosed).
## New 📚 Guides and ✏️ blog posts
diff --git a/website/docs/docs/dbt-versions/release-notes/10-Mar-2023/1.0-deprecation.md b/website/docs/docs/dbt-versions/release-notes/10-Mar-2023/1.0-deprecation.md
index b11bf702330..6b6f646e40e 100644
--- a/website/docs/docs/dbt-versions/release-notes/10-Mar-2023/1.0-deprecation.md
+++ b/website/docs/docs/dbt-versions/release-notes/10-Mar-2023/1.0-deprecation.md
@@ -17,5 +17,5 @@ Refer to some additional info and resources to help you upgrade your dbt version
- [How to upgrade dbt without fear](https://docs.getdbt.com/blog/upgrade-dbt-without-fear)
- [Upgrade Q&A on breaking changes](/docs/dbt-versions/upgrade-core-in-cloud#upgrading-legacy-versions-under-10)
-- [Version migration guides](/guides/migration/versions)
+- [Version migration guides](/docs/dbt-versions/core-upgrade)
diff --git a/website/docs/docs/dbt-versions/release-notes/35-dbt-cloud-changelog-2019-2020.md b/website/docs/docs/dbt-versions/release-notes/35-dbt-cloud-changelog-2019-2020.md
index b8e15b993de..a6b68cf9d51 100644
--- a/website/docs/docs/dbt-versions/release-notes/35-dbt-cloud-changelog-2019-2020.md
+++ b/website/docs/docs/dbt-versions/release-notes/35-dbt-cloud-changelog-2019-2020.md
@@ -197,7 +197,7 @@ initial support for a GitLab integration and self-service RBAC configuration.
## dbt Cloud v1.1.7 [September 3, 2020]
This release adds a Release Candidate for [dbt
-v0.18.0](/guides/migration/versions) and
+v0.18.0](/docs/dbt-versions/core-upgrade) and
includes bugfixes and improvements to the Cloud IDE
and job scheduler.
diff --git a/website/docs/docs/dbt-versions/upgrade-core-in-cloud.md b/website/docs/docs/dbt-versions/upgrade-core-in-cloud.md
index d143aab5ef1..e46294029ec 100644
--- a/website/docs/docs/dbt-versions/upgrade-core-in-cloud.md
+++ b/website/docs/docs/dbt-versions/upgrade-core-in-cloud.md
@@ -47,7 +47,7 @@ For more on version support and future releases, see [Understanding dbt Core ver
#### Need help upgrading?
-If you want more advice on how to upgrade your dbt projects, check out our [migration guides](/guides/migration/versions/) and our [upgrading Q&A page](/docs/dbt-versions/upgrade-core-in-cloud#upgrading-legacy-versions-under-10).
+If you want more advice on how to upgrade your dbt projects, check out our [migration guides](/docs/dbt-versions/core-upgrade/) and our [upgrading Q&A page](/docs/dbt-versions/upgrade-core-in-cloud#upgrading-legacy-versions-under-10).
## Upgrading legacy versions under 1.0
@@ -96,7 +96,7 @@ clean-targets:
- Do you have custom scripts that parse dbt artifacts?
- (BigQuery only) Do you use dbt's legacy capabilities around ingestion-time-partitioned tables?
-If you believe your project might be affected, read more details in the migration guide [here](/guides/migration/versions/upgrading-to-v1.0).
+If you believe your project might be affected, read more details in the migration guide [here](/docs/dbt-versions/core-upgrade/upgrading-to-v1.0).
@@ -109,7 +109,7 @@ If you believe your project might be affected, read more details in the migratio
- Do you have custom scripts that parse dbt JSON artifacts?
- (Snowflake only) Do you have custom macros or materializations that depend on using transactions, such as statement blocks with `auto_begin=True`?
-If you believe your project might be affected, read more details in the migration guide [here](/guides/migration/versions).
+If you believe your project might be affected, read more details in the migration guide [here](/docs/dbt-versions/core-upgrade).
@@ -123,7 +123,7 @@ If you believe your project might be affected, read more details in the migratio
- Does your project use `adapter.dispatch` or the `spark_utils` package?
- Do you have custom scripts that parse dbt JSON artifacts?
-If you believe your project might be affected, read more details in the migration guide [here](/guides/migration/versions).
+If you believe your project might be affected, read more details in the migration guide [here](/docs/dbt-versions/core-upgrade).
@@ -146,7 +146,7 @@ See **Upgrading to v0.17.latest from v0.16** below for more details.
- Do you have custom scripts that parse dbt JSON artifacts?
- Do you have any custom materializations?
-If you believe your project might be affected, read more details in the migration guide [here](/guides/migration/versions).
+If you believe your project might be affected, read more details in the migration guide [here](/docs/dbt-versions/core-upgrade).
@@ -157,7 +157,7 @@ If you believe your project might be affected, read more details in the migratio
- Do you directly call `adapter_macro`?
-If you believe your project might be affected, read more details in the migration guide [here](/guides/migration/versions).
+If you believe your project might be affected, read more details in the migration guide [here](/docs/dbt-versions/core-upgrade).
@@ -235,7 +235,7 @@ models:
```
-If you believe your project might be affected, read more details in the migration guide [here](/guides/migration/versions).
+If you believe your project might be affected, read more details in the migration guide [here](/docs/dbt-versions/core-upgrade).
@@ -247,7 +247,7 @@ If you believe your project might be affected, read more details in the migratio
- Do you use the custom `generate_schema_name` macro?
- Do you use `partition_by` config for BigQuery models?
-If you believe your project might be affected, read more details in the migration guide [here](/guides/migration/versions).
+If you believe your project might be affected, read more details in the migration guide [here](/docs/dbt-versions/core-upgrade).
@@ -259,7 +259,7 @@ If you believe your project might be affected, read more details in the migratio
- Do you have a custom materialization?
- Do you have a macro that accesses `Relations` directly?
-If you believe your project might be affected, read more details in the migration guide [here](/guides/migration/versions).
+If you believe your project might be affected, read more details in the migration guide [here](/docs/dbt-versions/core-upgrade).
@@ -270,7 +270,7 @@ If you believe your project might be affected, read more details in the migratio
- Do you use the custom `generate_schema_name` macro?
- Do you use the `—non-destructive` flag?
-If you believe your project might be affected, read more details in the migration guide [here](/guides/migration/versions).
+If you believe your project might be affected, read more details in the migration guide [here](/docs/dbt-versions/core-upgrade).
diff --git a/website/docs/docs/deploy/ci-jobs.md b/website/docs/docs/deploy/ci-jobs.md
index fb603e2864e..d10bc780fc2 100644
--- a/website/docs/docs/deploy/ci-jobs.md
+++ b/website/docs/docs/deploy/ci-jobs.md
@@ -27,6 +27,7 @@ To make CI job creation easier, many options on the **CI job** page are set to d
- **Job Name** — Specify the name for this CI job.
- **Environment** — By default, it’s set to the environment you created the CI job from.
- **Triggered by pull requests** — By default, it’s enabled. Every time a developer opens up a pull request or pushes a commit to an existing pull request, this job will get triggered to run.
+ - **Run on Draft Pull Request** — Enable this option if you want to also trigger the job to run every time a developer opens up a draft pull request or pushes a commit to that draft pull request.
3. Options in the **Execution Settings** section:
- **Commands** — By default, it includes the `dbt build --select state:modified+` command. This informs dbt Cloud to build only new or changed models and their downstream dependents. Importantly, state comparison can only happen when there is a deferred environment selected to compare state to. Click **Add command** to add more [commands](/docs/deploy/job-commands) that you want to be invoked when this job runs.
@@ -62,13 +63,13 @@ If you're not using dbt Cloud’s native Git integration with [GitHub](/docs/cl
1. Set up a CI job with the [Create Job](/dbt-cloud/api-v2#/operations/Create%20Job) API endpoint using `"job_type": ci` or from the [dbt Cloud UI](#set-up-ci-jobs).
-1. Call the [Trigger Job Run](/dbt-cloud/api-v2#/operations/Trigger%20Job%20Run) API endpoint to trigger the CI job. Provide the pull request (PR) ID to the payload using one of these fields, even if you're using a different Git provider (like Bitbucket):
+1. Call the [Trigger Job Run](/dbt-cloud/api-v2#/operations/Trigger%20Job%20Run) API endpoint to trigger the CI job. You must include these fields to the payload:
+ - Provide the pull request (PR) ID with one of these fields, even if you're using a different Git provider (like Bitbucket). This can make your code less human-readable but it will _not_ affect dbt functionality.
- - `github_pull_request_id`
- - `gitlab_merge_request_id`
- - `azure_devops_pull_request_id`
-
- This can make your code less human-readable but it will _not_ affect dbt functionality.
+ - `github_pull_request_id`
+ - `gitlab_merge_request_id`
+ - `azure_devops_pull_request_id`
+ - Provide the `git_sha` or `git_branch` to target the correct commit or branch to run the job against.
## Example pull requests
@@ -94,10 +95,18 @@ If you're experiencing any issues, review some of the common questions and answe
Temporary schemas aren't dropping
-
If your temporary schemas aren't dropping after a PR merges or closes, this typically indicates you have overridden the
generate_schema_name
macro and it isn't using
dbt_cloud_pr_
as the prefix.
To resolve this, change your macro so that the temporary PR schema name contains the required prefix. For example:
+
If your temporary schemas aren't dropping after a PR merges or closes, this typically indicates one of these issues:
+
+ - You have overridden the
generate_schema_name
macro and it isn't using dbt_cloud_pr_
as the prefix.
To resolve this, change your macro so that the temporary PR schema name contains the required prefix. For example:
- • ✅ Temporary PR schema name contains the prefix dbt_cloud_pr_
(like dbt_cloud_pr_123_456_marketing
)
- • ❌ Temporary PR schema name doesn't contain the prefix dbt_cloud_pr_
(like marketing
).
+ ✅ Temporary PR schema name contains the prefix dbt_cloud_pr_
(like dbt_cloud_pr_123_456_marketing
).
+ ❌ Temporary PR schema name doesn't contain the prefix dbt_cloud_pr_
(like marketing
).
+
+
+ -
+ A macro is creating a schema but there are no dbt models writing to that schema. dbt Cloud doesn't drop temporary schemas that weren't written to as a result of running a dbt model.
+
+
@@ -153,6 +162,3 @@ If you're experiencing any issues, review some of the common questions and answe
If you're on a Virtual Private dbt Enterprise plan using security features like ingress PrivateLink or IP Allowlisting, registering CI hooks may not be available and can cause the job to fail silently.
-
-
-
diff --git a/website/docs/docs/deploy/continuous-integration.md b/website/docs/docs/deploy/continuous-integration.md
index cc856f97f22..0f87965aada 100644
--- a/website/docs/docs/deploy/continuous-integration.md
+++ b/website/docs/docs/deploy/continuous-integration.md
@@ -16,7 +16,7 @@ Using CI helps:
## How CI works
-When you [set up CI jobs](/docs/deploy/ci-jobs#set-up-ci-jobs), dbt Cloud listens for webhooks from your Git provider indicating that a new PR has been opened or updated with new commits. When dbt Cloud receives one of these webhooks, it enqueues a new run of the CI job. If you want CI checks to run on each new commit, you need to mark your PR as **Ready for review** in your Git provider — draft PRs _don't_ trigger CI jobs.
+When you [set up CI jobs](/docs/deploy/ci-jobs#set-up-ci-jobs), dbt Cloud listens for webhooks from your Git provider indicating that a new PR has been opened or updated with new commits. When dbt Cloud receives one of these webhooks, it enqueues a new run of the CI job.
dbt Cloud builds and tests the models affected by the code change in a temporary schema, unique to the PR. This process ensures that the code builds without error and that it matches the expectations as defined by the project's dbt tests. The unique schema name follows the naming convention `dbt_cloud_pr__` (for example, `dbt_cloud_pr_1862_1704`) and can be found in the run details for the given run, as shown in the following image:
diff --git a/website/docs/docs/deploy/deployment-overview.md b/website/docs/docs/deploy/deployment-overview.md
index 5883ecaa3f1..29934663544 100644
--- a/website/docs/docs/deploy/deployment-overview.md
+++ b/website/docs/docs/deploy/deployment-overview.md
@@ -4,6 +4,8 @@ id: "deployments"
sidebar: "Use dbt Cloud's capabilities to seamlessly run a dbt job in production."
hide_table_of_contents: true
tags: ["scheduler"]
+pagination_next: "docs/deploy/job-scheduler"
+pagination_prev: null
---
Use dbt Cloud's capabilities to seamlessly run a dbt job in production or staging environments. Rather than run dbt commands manually from the command line, you can leverage the [dbt Cloud's in-app scheduling](/docs/deploy/job-scheduler) to automate how and when you execute dbt.
@@ -58,6 +60,12 @@ Learn how to use dbt Cloud's features to help your team ship timely and quality
link="/docs/deploy/run-visibility"
icon="dbt-bit"/>
+
+
+
+## Related content
+- [Retry a failed run for a job](/dbt-cloud/api-v2#/operations/Retry%20a%20failed%20run%20for%20a%20job) API endpoint
+- [Run visibility](/docs/deploy/run-visibility)
+- [Jobs](/docs/deploy/jobs)
+- [Job commands](/docs/deploy/job-commands)
\ No newline at end of file
diff --git a/website/docs/docs/environments-in-dbt.md b/website/docs/docs/environments-in-dbt.md
index 54eaa68f667..70bc096cf4f 100644
--- a/website/docs/docs/environments-in-dbt.md
+++ b/website/docs/docs/environments-in-dbt.md
@@ -2,6 +2,7 @@
title: "About environments"
id: "environments-in-dbt"
hide_table_of_contents: true
+pagination_next: null
---
In software engineering, environments are used to enable engineers to develop and test code without impacting the users of their software. Typically, there are two types of environments in dbt:
@@ -18,7 +19,7 @@ Configure environments to tell dbt Cloud or dbt Core how to build and execute yo
diff --git a/website/docs/docs/introduction.md b/website/docs/docs/introduction.md
index c4cfd6e45ac..0aeef0201cb 100644
--- a/website/docs/docs/introduction.md
+++ b/website/docs/docs/introduction.md
@@ -1,6 +1,8 @@
---
title: "What is dbt?"
id: "introduction"
+pagination_next: null
+pagination_prev: null
---
@@ -28,6 +30,7 @@ Read more about why we want to enable analysts to work more like software engine
You can access dbt using dbt Core or dbt Cloud. dbt Cloud is built around dbt Core, but it also provides:
- Web-based UI so it’s more accessible
+- dbt Cloud-powered command line (CLI) to develop, test, version control dbt projects, and run dbt commands
- Hosted environment so it’s faster to get up and running
- Differentiated features, such as metadata, in-app job scheduler, observability, integrations with other tools, integrated development environment (IDE), and more.
@@ -35,7 +38,8 @@ You can learn about plans and pricing on [www.getdbt.com](https://www.getdbt.com
### dbt Cloud
-dbt Cloud is the fastest and most reliable way to deploy dbt. Develop, test, schedule, and investigate data models all in one web-based UI. Learn more about [dbt Cloud features](/docs/cloud/about-cloud/dbt-cloud-features) and try one of the [dbt Cloud quickstarts](/quickstarts).
+dbt Cloud is the fastest and most reliable way to deploy dbt. Develop, test, schedule, and investigate data models all in one web-based UI. It also natively supports developing using a command line with the [dbt Cloud CLI](/docs/cloud/cloud-cli-installation).
+Learn more about [dbt Cloud features](/docs/cloud/about-cloud/dbt-cloud-features) and try one of the [dbt Cloud quickstarts](/quickstarts).
### dbt Core
diff --git a/website/docs/docs/running-a-dbt-project/run-your-dbt-projects.md b/website/docs/docs/running-a-dbt-project/run-your-dbt-projects.md
index 9bd57e0b280..b3b6ffb3e45 100644
--- a/website/docs/docs/running-a-dbt-project/run-your-dbt-projects.md
+++ b/website/docs/docs/running-a-dbt-project/run-your-dbt-projects.md
@@ -1,14 +1,25 @@
---
title: "Run your dbt projects"
id: "run-your-dbt-projects"
+pagination_prev: null
---
-You can run your dbt projects with [dbt Cloud](/docs/cloud/about-cloud/dbt-cloud-features) and [dbt Core](https://github.com/dbt-labs/dbt-core). dbt Cloud is a hosted application where you can develop directly from a web browser. dbt Core is an open source project where you can develop from the command line.
+You can run your dbt projects with [dbt Cloud](/docs/cloud/about-cloud/dbt-cloud-features) or [dbt Core](https://github.com/dbt-labs/dbt-core):
-Among other features, dbt Cloud provides a development environment to help you build, test, run, and [version control](/docs/collaborate/git-version-control) your project faster. It also includes an easier way to share your [dbt project's documentation](/docs/collaborate/build-and-view-your-docs) with your team. These development tasks are directly built into dbt Cloud for an _integrated development environment_ (IDE). Refer to [Develop in the Cloud](/docs/cloud/dbt-cloud-ide/develop-in-the-cloud) for more details.
+- **dbt Cloud**: A hosted application where you can develop directly from a web browser using the [dbt Cloud IDE](/docs/cloud/dbt-cloud-ide/develop-in-the-cloud). It also natively supports developing using a command line interface, [dbt Cloud CLI](/docs/cloud/cloud-cli-installation). Among other features, dbt Cloud provides:
-With dbt Core, you can run your dbt projects from the command line. The command line interface (CLI) is available from your computer's terminal application such as Terminal and iTerm. When using the command line, you can run commands and do other work from the current working directory on your computer. Before running the dbt project from the command line, make sure you are working in your dbt project directory. Learning terminal commands such as `cd` (change directory), `ls` (list directory contents), and `pwd` (present working directory) can help you navigate the directory structure on your system.
+ - Development environment to help you build, test, run, and [version control](/docs/collaborate/git-version-control) your project faster.
+ - Share your [dbt project's documentation](/docs/collaborate/build-and-view-your-docs) with your team.
+ - Integrates with the dbt Cloud IDE, allowing you to run development tasks and environment in the dbt Cloud UI for a seamless experience.
+ - The dbt Cloud CLI to develop and run dbt commands against your dbt Cloud development environment from your local command line.
+ - For more details, refer to [Develop in the Cloud](/docs/cloud/about-cloud-develop).
-When running your project from dbt Core or dbt Cloud, the commands you commonly use are:
+- **dbt Core**: An open source project where you can develop from the [command line](/docs/core/about-dbt-core).
+
+The dbt Cloud CLI and dbt Core are both command line tools that enable you to run dbt commands. The key distinction is the dbt Cloud CLI is tailored for dbt Cloud's infrastructure and integrates with all its [features](/docs/cloud/about-cloud/dbt-cloud-features).
+
+The command line is available from your computer's terminal application such as Terminal and iTerm. With the command line, you can run commands and do other work from the current working directory on your computer. Before running the dbt project from the command line, make sure you are working in your dbt project directory. Learning terminal commands such as `cd` (change directory), `ls` (list directory contents), and `pwd` (present working directory) can help you navigate the directory structure on your system.
+
+In dbt Cloud or dbt Core, the commands you commonly use are:
- [dbt run](/reference/commands/run) — Runs the models you defined in your project
- [dbt build](/reference/commands/build) — Builds and tests your selected resources such as models, seeds, snapshots, and tests
@@ -20,6 +31,7 @@ For information on all dbt commands and their arguments (flags), see the [dbt co
- [How we set up our computers for working on dbt projects](https://discourse.getdbt.com/t/how-we-set-up-our-computers-for-working-on-dbt-projects/243)
- [Model selection syntax](/reference/node-selection/syntax)
+- [dbt Cloud CLI](/docs/cloud/cloud-cli-installation)
- [Cloud IDE features](/docs/cloud/dbt-cloud-ide/develop-in-the-cloud#ide-features)
- [Does dbt offer extract and load functionality?](/faqs/Project/transformation-tool)
- [Why does dbt compile need a data platform connection](/faqs/Warehouse/db-connection-dbt-compile)
diff --git a/website/docs/docs/running-a-dbt-project/using-threads.md b/website/docs/docs/running-a-dbt-project/using-threads.md
index 519ce8aab81..5eede7abc27 100644
--- a/website/docs/docs/running-a-dbt-project/using-threads.md
+++ b/website/docs/docs/running-a-dbt-project/using-threads.md
@@ -3,7 +3,7 @@ title: "Using threads"
id: "using-threads"
sidebar_label: "Use threads"
description: "Understand what threads mean and how to use them."
-
+pagination_next: null
---
When dbt runs, it creates a directed acyclic graph (DAG) of links between models. The number of threads represents the maximum number of paths through the graph dbt may work on at once – increasing the number of threads can minimize the run time of your project.
@@ -18,7 +18,7 @@ Generally the optimal number of threads depends on your data warehouse and its c
You can use a different number of threads than the value defined in your target by using the `--threads` option when executing a dbt command.
-You will define the number of threads in your `profiles.yml` file (for CLI-users only), dbt Cloud job definition, and dbt Cloud development credentials under your profile.
+You will define the number of threads in your `profiles.yml` file (for dbt Core users only), dbt Cloud job definition, and dbt Cloud development credentials under your profile.
## Related docs
diff --git a/website/docs/docs/supported-data-platforms.md b/website/docs/docs/supported-data-platforms.md
index 8ac782991c8..a8e146f49d0 100644
--- a/website/docs/docs/supported-data-platforms.md
+++ b/website/docs/docs/supported-data-platforms.md
@@ -4,14 +4,20 @@ id: "supported-data-platforms"
sidebar_label: "Supported data platforms"
description: "Connect dbt to any data platform in dbt Cloud or dbt Core, using a dedicated adapter plugin"
hide_table_of_contents: true
+pagination_next: "docs/connect-adapters"
+pagination_prev: null
---
dbt connects to and runs SQL against your database, warehouse, lake, or query engine. These SQL-speaking platforms are collectively referred to as _data platforms_. dbt connects with data platforms by using a dedicated adapter plugin for each. Plugins are built as Python modules that dbt Core discovers if they are installed on your system. Read [What are Adapters](/guides/dbt-ecosystem/adapter-development/1-what-are-adapters) for more info.
-You can [connect](/docs/connect-adapters) to adapters and data platforms either directly in the dbt Cloud user interface (UI) or install them manually using the command line (CLI).
+You can [connect](/docs/connect-adapters) to adapters and data platforms natively in dbt Cloud or install them manually using dbt Core.
You can also further customize how dbt works with your specific data platform via configuration: see [Configuring Postgres](/reference/resource-configs/postgres-configs) for an example.
+import MSCallout from '/snippets/_microsoft-adapters-soon.md';
+
+
+
## Types of Adapters
There are three types of adapters available today:
@@ -36,5 +42,5 @@ import AdaptersTrusted from '/snippets/_adapters-trusted.md';
-
* Install these adapters using the CLI as they're not currently supported in dbt Cloud.
+
* Install these adapters using dbt Core as they're not currently supported in dbt Cloud.
diff --git a/website/docs/docs/trusted-adapters.md b/website/docs/docs/trusted-adapters.md
index e19bb40785f..08191e8ea42 100644
--- a/website/docs/docs/trusted-adapters.md
+++ b/website/docs/docs/trusted-adapters.md
@@ -6,7 +6,7 @@ hide_table_of_contents: true
Trusted adapters are adapters not maintained by dbt Labs, that we feel comfortable recommending to users for use in production.
-Free and open-source tools for the data professional are increasingly abundant. This is by-and-large a *good thing*, however it requires due dilligence that wasn't required in a paid-license, closed-source software world. As a user, there are questions to answer important before taking a dependency on an open-source project. The trusted adapter designation is meant to streamline this process for end users.
+Free and open-source tools for the data professional are increasingly abundant. This is by-and-large a *good thing*, however it requires due diligence that wasn't required in a paid-license, closed-source software world. As a user, there are questions to answer important before taking a dependency on an open-source project. The trusted adapter designation is meant to streamline this process for end users.
Considerations for depending on an open-source project
diff --git a/website/docs/docs/use-dbt-semantic-layer/avail-sl-integrations.md b/website/docs/docs/use-dbt-semantic-layer/avail-sl-integrations.md
index b084dedc305..22178a14b5b 100644
--- a/website/docs/docs/use-dbt-semantic-layer/avail-sl-integrations.md
+++ b/website/docs/docs/use-dbt-semantic-layer/avail-sl-integrations.md
@@ -4,36 +4,35 @@ id: avail-sl-integrations
description: "Discover the diverse range of partners that seamlessly integrate with the powerful dbt Semantic Layer, allowing you to query and unlock valuable insights from your data ecosystem."
tags: [Semantic Layer]
sidebar_label: "Available integrations"
+hide_table_of_contents: true
meta:
api_name: dbt Semantic Layer APIs
---
-import NewSLChanges from '/snippets/_new-sl-changes.md';
-
-
-
There are a number of data applications that seamlessly integrate with the dbt Semantic Layer, powered by MetricFlow, from business intelligence tools to notebooks, spreadsheets, data catalogs, and more. These integrations allow you to query and unlock valuable insights from your data ecosystem.
Use the [dbt Semantic Layer APIs](/docs/dbt-cloud-apis/sl-api-overview) to simplify metric queries, optimize your development workflow, and reduce coding. This approach also ensures data governance and consistency for data consumers.
-
-
-
import AvailIntegrations from '/snippets/_sl-partner-links.md';
### Custom integration
-You can create custom integrations using different languages and tools. We support connecting with JDBC, ADBC, and a GraphQL API. For more info, check out [our examples on GitHub](https://github.com/dbt-labs/example-semantic-layer-clients/).
+- You can create custom integrations using different languages and tools. We support connecting with JDBC, ADBC, and GraphQL APIs. For more info, check out [our examples on GitHub](https://github.com/dbt-labs/example-semantic-layer-clients/).
+- You can also connect to tools that allow you to write SQL. These tools must meet one of the two criteria:
+
+ - Supports a generic JDBC driver option (such as DataGrip) or
+ - Uses Arrow Flight SQL JDBC driver version 12.0.0 or higher.
## Related docs
-- {frontMatter.meta.api_name} to learn how to integrate with JDBC and GraphQL to query your metrics in downstream tools.
-- [dbt Semantic Layer APIs query syntax](/docs/dbt-cloud-apis/sl-jdbc#querying-the-api-for-metric-metadata)
+- {frontMatter.meta.api_name} to learn how to integrate and query your metrics in downstream tools.
+- [dbt Semantic Layer API query syntax](/docs/dbt-cloud-apis/sl-jdbc#querying-the-api-for-metric-metadata)
+- [Hex dbt Semantic Layer cells](https://learn.hex.tech/docs/logic-cell-types/transform-cells/dbt-metrics-cells) to set up SQL cells in Hex.
diff --git a/website/docs/docs/use-dbt-semantic-layer/dbt-sl.md b/website/docs/docs/use-dbt-semantic-layer/dbt-sl.md
index 76753b41ffa..8c78d556a67 100644
--- a/website/docs/docs/use-dbt-semantic-layer/dbt-sl.md
+++ b/website/docs/docs/use-dbt-semantic-layer/dbt-sl.md
@@ -5,13 +5,12 @@ description: "Learn how the dbt Semantic Layer enables data teams to centrally d
sidebar_label: "About the dbt Semantic Layer"
tags: [Semantic Layer]
hide_table_of_contents: true
+pagination_next: "docs/use-dbt-semantic-layer/quickstart-sl"
+pagination_prev: null
---
-import NewSLChanges from '/snippets/_new-sl-changes.md';
-
-
The dbt Semantic Layer, powered by [MetricFlow](/docs/build/about-metricflow), simplifies the process of defining and using critical business metrics, like `revenue` in the modeling layer (your dbt project). By centralizing metric definitions, data teams can ensure consistent self-service access to these metrics in downstream data tools and applications. The dbt Semantic Layer eliminates duplicate coding by allowing data teams to define metrics on top of existing models and automatically handles data joins.
@@ -26,10 +25,8 @@ Refer to the [Why we need a universal semantic layer](https://www.getdbt.com/blo
import Features from '/snippets/_sl-plan-info.md'
@@ -46,12 +43,6 @@ instance="hosted in North America"
link="/docs/use-dbt-semantic-layer/setup-sl"
icon="dbt-bit"/>
-
-
+
+
diff --git a/website/docs/docs/use-dbt-semantic-layer/gsheets.md b/website/docs/docs/use-dbt-semantic-layer/gsheets.md
new file mode 100644
index 00000000000..ee391c91b70
--- /dev/null
+++ b/website/docs/docs/use-dbt-semantic-layer/gsheets.md
@@ -0,0 +1,63 @@
+---
+title: "Google Sheets (beta)"
+description: "Integrate with Google Sheets to query your metrics in a spreadsheet."
+tags: [Semantic Layer]
+sidebar_label: "Google Sheets (beta)"
+---
+
+:::info Beta functionality
+Google Sheets integration with the dbt Semantic Layer is a [beta](/docs/dbt-versions/product-lifecycles#dbt-cloud) feature.
+:::
+
+The dbt Semantic Layer offers a seamless integration with Google Sheets through a custom menu. This add-on allows you to build dbt Semantic Layer queries and return data on your metrics directly within Google Sheet.
+
+## Prerequisites
+
+1. You have a Google account with access to Google Sheets.
+2. You can install Google add-ons.
+3. You have [set up the dbt Semantic Layer](/docs/use-dbt-semantic-layer/setup-sl).
+4. You have a dbt Cloud Environment ID and a [service token](/docs/dbt-cloud-apis/service-tokens) to authenticate with from a dbt Cloud account.
+
+## Installing the add-on
+
+1. Navigate to the [dbt Semantic Layer for Sheets App](https://gsuite.google.com/marketplace/app/foo/392263010968) to install the add-on.
+
+ - You can also find it in Google Sheets by going to [**Extensions -> Add-on -> Get add-ons**](https://support.google.com/docs/answer/2942256?hl=en&co=GENIE.Platform%3DDesktop&oco=0#zippy=%2Cinstall-add-ons%2Cinstall-an-add-on) and searching for it there.
+2. After installing, open the Add-On menu and select the "dbt Semantic Layer for Sheets". This will open a custom menu to the right-hand side of your screen.
+3. Authenticate with your Host, dbt Cloud Environment ID, and Service Token.
+4. Start querying your metrics using the **Query Builder**. For more info on the menu functions, refer to [Custom menu functions](#custom-menu-functions).
+
+When querying your data with Google Sheets:
+
+- It returns the data to the cell you have clicked on.
+- The custom menu operation has a timeout limit of six (6) minutes.
+- If you're using this extension, make sure you're signed into Chrome with the same Google profile you used to set up the Add-On. Log in with one Google profile at a time as using multiple Google profiles at once might cause issues.
+
+
+## Custom menu functions
+
+The custom menu provides the following capabilities:
+
+| Menu items | Description |
+|---------------|-------------------------------------------------------|
+| Metrics | Search and select metrics. |
+| Group By | Search and select dimensions to group by. Dimensions are grouped by the entity of the semantic model they come from. |
+| Granularity | Modify the granularity of the primary time dimension. |
+| Where | Filter your data. This includes categorical and time filters. |
+| Order By | Return your data ordered. |
+| Limit | Set a limit for the rows of your output. |
+
+
+## Filtering data
+
+To use the filter functionality, choose the [dimension](docs/build/dimensions) you want to filter by and select the operation you want to filter on.
+ - For categorical dimensiosn, type in the dimension value you want to filter by (no quotes needed) and press enter.
+ - Continue adding additional filters as needed with AND and OR. If it's a time dimension, choose the operator and select from the calendar.
+
+
+
+**Limited Use Policy Disclosure**
+
+The dbt Semantic Layer for Sheet's use and transfer to any other app of information received from Google APIs will adhere to [Google API Services User Data Policy](https://developers.google.com/terms/api-services-user-data-policy), including the Limited Use requirements.
+
+
diff --git a/website/docs/docs/use-dbt-semantic-layer/quickstart-sl.md b/website/docs/docs/use-dbt-semantic-layer/quickstart-sl.md
index 3bbc11cea3f..cca899d227e 100644
--- a/website/docs/docs/use-dbt-semantic-layer/quickstart-sl.md
+++ b/website/docs/docs/use-dbt-semantic-layer/quickstart-sl.md
@@ -10,17 +10,15 @@ meta:
-import NewSLChanges from '/snippets/_new-sl-changes.md';
-import InstallMetricFlow from '/snippets/_sl-install-metricflow.md';
+
import CreateModel from '/snippets/_sl-create-semanticmodel.md';
import DefineMetrics from '/snippets/_sl-define-metrics.md';
import ConfigMetric from '/snippets/_sl-configure-metricflow.md';
import TestQuery from '/snippets/_sl-test-and-query-metrics.md';
+import ConnectQueryAPI from '/snippets/_sl-connect-and-query-api.md';
+import RunProdJob from '/snippets/_sl-run-prod-job.md';
-
-
-
The dbt Semantic Layer, powered by [MetricFlow](/docs/build/about-metricflow), simplifies defining and using critical business metrics. It centralizes metric definitions, eliminates duplicate coding, and ensures consistent self-service access to metrics in downstream tools.
MetricFlow, a powerful component of the dbt Semantic Layer, simplifies the creation and management of company metrics. It offers flexible abstractions, SQL query generation, and enables fast retrieval of metric datasets from a data platform.
@@ -29,15 +27,15 @@ Use this guide to fully experience the power of the universal dbt Semantic Layer
- [Create a semantic model](#create-a-semantic-model) in dbt Cloud using MetricFlow
- [Define metrics](#define-metrics) in dbt Cloud using MetricFlow
-- [Test and query metrics locally](#test-and-query-metrics) using MetricFlow
+- [Test and query metrics](#test-and-query-metrics) with MetricFlow
- [Run a production job](#run-a-production-job) in dbt Cloud
- [Set up dbt Semantic Layer](#setup) in dbt Cloud
- [Connect and query API](#connect-and-query-api) with dbt Cloud
-
-MetricFlow allows users to define metrics in their dbt project whether in dbt Cloud or in dbt Core. dbt Core users can use the [MetricFlow CLI](/docs/build/metricflow-cli) to define metrics in their local dbt Core project.
+MetricFlow allows you to define metrics in your dbt project and query them whether in dbt Cloud or dbt Core with [MetricFlow commands](/docs/build/metricflow-commands).
However, to experience the power of the universal [dbt Semantic Layer](/docs/use-dbt-semantic-layer/dbt-sl) and query those metrics in downstream tools, you'll need a dbt Cloud [Team or Enterprise](https://www.getdbt.com/pricing/) account.
+
## Prerequisites
import SetUp from '/snippets/_v2-sl-prerequisites.md';
@@ -62,13 +60,8 @@ New to dbt or metrics? Try our [Jaffle shop example project](https://github.com/
## Run a production job
-Once you’ve defined metrics in your dbt project, you can perform a job run in your deployment environment in dbt Cloud to materialize your metrics. The deployment environment is only supported for the dbt Semantic Layer at this moment.
-1. Go to **Deploy** in the navigation header
-2. Select **Jobs** to re-run the job with the most recent code in the deployment environment.
-3. Your metric should appear as a red node in the dbt Cloud IDE and dbt directed acyclic graphs (DAG).
-
-
+
@@ -88,16 +81,7 @@ import SlSetUp from '/snippets/_new-sl-setup.md';
## Connect and query API
-You can query your metrics in a JDBC-enabled tool or use existing first-class integrations with the dbt Semantic Layer.
-
-You must have a dbt Cloud Team or Enterprise [multi-tenant](/docs/cloud/about-cloud/regions-ip-addresses) deployment, hosted in North America (Additional region support coming soon).
-
-- To learn how to use the JDBC or GraphQL API and what tools you can query it with, refer to the {frontMatter.meta.api_name}.
-
- * To authenticate, you need to [generate a service token](/docs/dbt-cloud-apis/service-tokens) with Semantic Layer Only and Metadata Only permissions.
- * Refer to the [SQL query syntax](/docs/dbt-cloud-apis/sl-jdbc#querying-the-api-for-metric-metadata) to query metrics using the APIs.
-
-- To learn more about the sophisticated integrations that connect to the dbt Semantic Layer, refer to [Available integrations](/docs/use-dbt-semantic-layer/avail-sl-integrations) for more info.
+
## FAQs
@@ -124,6 +108,7 @@ The dbt Semantic Layer is proprietary, however, some components of the dbt Seman
- [Build your metrics](/docs/build/build-metrics-intro)
- [Set up dbt Semantic Layer](docs/use-dbt-semantic-layer/setup-dbt-sl)
- [Available integrations](/docs/use-dbt-semantic-layer/avail-sl-integrations)
+- Demo on [how to define and query metrics with MetricFlow](https://www.loom.com/share/60a76f6034b0441788d73638808e92ac?sid=861a94ac-25eb-4fd8-a310-58e159950f5a)
diff --git a/website/docs/docs/use-dbt-semantic-layer/setup-sl.md b/website/docs/docs/use-dbt-semantic-layer/setup-sl.md
index a2395d367e7..4c88ee50b25 100644
--- a/website/docs/docs/use-dbt-semantic-layer/setup-sl.md
+++ b/website/docs/docs/use-dbt-semantic-layer/setup-sl.md
@@ -8,9 +8,6 @@ tags: [Semantic Layer]
-import NewSLChanges from '/snippets/_new-sl-changes.md';
-
-
With the dbt Semantic Layer, you can centrally define business metrics, reduce code duplication and inconsistency, create self-service in downstream tools, and more. Configure the dbt Semantic Layer in dbt Cloud to connect with your integrated partner tool.
diff --git a/website/docs/docs/use-dbt-semantic-layer/sl-architecture.md b/website/docs/docs/use-dbt-semantic-layer/sl-architecture.md
index 89cd9bc6ddc..152821b7e59 100644
--- a/website/docs/docs/use-dbt-semantic-layer/sl-architecture.md
+++ b/website/docs/docs/use-dbt-semantic-layer/sl-architecture.md
@@ -4,12 +4,9 @@ id: sl-architecture
description: "dbt Semantic Layer product architecture and related questions."
sidebar_label: "Architecture"
tags: [Semantic Layer]
+pagination_next: null
---
-import NewSLChanges from '/snippets/_new-sl-changes.md';
-
-
-
@@ -22,12 +19,12 @@ The dbt Semantic Layer allows you to define metrics and use various interfaces t
The dbt Semantic Layer includes the following components:
-| Components | Information | Developer plans | Team plans | Enterprise plans | License |
+| Components | Information | dbt Core users | Developer plans | Team plans | Enterprise plans | License |
| --- | --- | :---: | :---: | :---: | --- |
-| **[MetricFlow](/docs/build/about-metricflow)** | MetricFlow in dbt allows users to centrally define their semantic models and metrics with YAML specifications. | ✅ | ✅ | ✅ | BSL package (code is source available) |
-| **MetricFlow Server**| A proprietary server that takes metric requests and generates optimized SQL for the specific data platform. | ❌ | ✅ | ✅ | Proprietary, Cloud (Team & Enterprise)|
-| **Semantic Layer Gateway** | A service that passes queries to MetricFlow server and executes the SQL generated by MetricFlow against the data platform|
❌| ✅ | ✅ | Proprietary, Cloud (Team & Enterprise) |
-| **Semantic Layer API** | The interfaces that allow users to submit metric queries include the MetricFlow CLI and JDBC API. They also serve as the foundation for building first-class integrations with various tools. | ❌ | ✅ | ✅ | Proprietary, Cloud (Team & Enterprise)|
+| **[MetricFlow](/docs/build/about-metricflow)** | MetricFlow in dbt allows users to centrally define their semantic models and metrics with YAML specifications. | ✅ | ✅ | ✅ | ✅ | BSL package (code is source available) |
+| **MetricFlow Server**| A proprietary server that takes metric requests and generates optimized SQL for the specific data platform. | ❌ | ❌ | ✅ | ✅ | Proprietary, Cloud (Team & Enterprise)|
+| **Semantic Layer Gateway** | A service that passes queries to the MetricFlow server and executes the SQL generated by MetricFlow against the data platform|
❌ | ❌ |✅ | ✅ | Proprietary, Cloud (Team & Enterprise) |
+| **Semantic Layer APIs** | The interfaces allow users to submit metric queries using GraphQL and JDBC APIs. They also serve as the foundation for building first-class integrations with various tools. | ❌ | ❌ | ✅ | ✅ | Proprietary, Cloud (Team & Enterprise)|
## Related questions
diff --git a/website/docs/docs/use-dbt-semantic-layer/tableau.md b/website/docs/docs/use-dbt-semantic-layer/tableau.md
new file mode 100644
index 00000000000..c93643354aa
--- /dev/null
+++ b/website/docs/docs/use-dbt-semantic-layer/tableau.md
@@ -0,0 +1,72 @@
+---
+title: "Tableau (beta)"
+description: "Use Tableau worksheets to query the dbt Semantic Layer and produce dashboards with trusted date."
+tags: [Semantic Layer]
+sidebar_label: "Tableau (beta)"
+---
+
+:::info Beta functionality
+The Tableau integration with the dbt Semantic Layer is a [beta feature](/docs/dbt-versions/product-lifecycles#dbt-cloud).
+:::
+
+
+The Tableau integration allows you to use worksheets to query the Semantic Layer directly and produce your dashboards with trusted data.
+
+This integration provides a live connection to the dbt Semantic Layer through Tableau Desktop.
+
+## Prerequisites
+
+1. You must have [Tableau Desktop](https://www.tableau.com/en-gb/products/desktop) installed with version 2021.1 or greater
+ - Note that Tableau Online does not currently support custom connectors natively.
+2. Log in to Tableau Desktop using either your license or the login details you use for Tableau Server or Tableau Online.
+3. You need your dbt Cloud host, [Environment ID](/docs/use-dbt-semantic-layer/setup-sl#set-up-dbt-semantic-layer) and [service token](/docs/dbt-cloud-apis/service-tokens) to log in. This account should be set up with the dbt Semantic Layer.
+4. You must have a dbt Cloud Team or Enterprise [account](https://www.getdbt.com/pricing) and multi-tenant [deployment](/docs/cloud/about-cloud/regions-ip-addresses). (Single-Tenant coming soon)
+
+
+## Installing
+
+1. Download the GitHub [connector file](https://github.com/dbt-labs/semantic-layer-tableau-connector/releases/download/v1.0.2/dbt_semantic_layer.taco) locally and add it to your default folder:
+ - Windows: `C:\Users\\[Windows User]\Documents\My Tableau Repository\Connectors`
+ - Mac: `/Users/[user]/Documents/My Tableau Repository/Connectors`
+ - Linux: `/opt/tableau/connectors`
+2. Install the [JDBC driver](/docs/dbt-cloud-apis/sl-jdbc) to the folder based on your operating system:
+ - Windows: `C:\Program Files\Tableau\Drivers`
+ - Mac: `~/Library/Tableau/Drivers`
+ - Linux: ` /opt/tableau/tableau_driver/jdbc`
+3. Open Tableau Desktop and find the **dbt Semantic Layer by dbt Labs** connector on the left-hand side.
+4. Connect with your Host, Environment ID, and service token information that's provided to you in your dbt Cloud Semantic Layer configuration.
+
+
+## Using the integration
+
+Once you authenticate, the system will direct you to the data source page with all the metrics and dimensions configured in your Semantic Layer.
+
+- From there, go directly to a worksheet in the bottom left-hand corner.
+- Then, you'll find all the metrics and dimensions that are available to query on the left-hand side of your window.
+
+Visit the [Tableau documentation](https://help.tableau.com/current/pro/desktop/en-us/gettingstarted_overview.htm) to learn more about how to use Tableau worksheets and dashboards.
+
+## Things to note
+
+- All metrics use the "SUM" aggregation type, and this can't be altered. The dbt Semantic Layer controls the aggregation type and it is intentionally fixed. Keep in mind that the underlying aggregation in the dbt Semantic Layer might not be "SUM" (even though "SUM" is Tableau's default).
+- Tableau surfaces all metrics and dimensions from the dbt Semantic Layer on the left-hand side. Note, that not all metrics and dimensions can be combined with one another. You will receive an error message if a particular dimension cannot be sliced with a metric (or vice versa).
+ - To display available metrics and dimensions, dbt Semantic Layer returns metadata for a fake table with the dimensions and metrics as 'columns' on this table. Because of this, you can't actually query this table for previews or extracts.
+ - Since this is treated as a table, dbt Semantic Layer can't dynamically change what is available. This means we display _all_ available metrics and dimensions even if a particular metric and dimension combination isn't available.
+
+- Certain Table calculations like "Totals" and "Percent Of" may not be accurate when using metrics aggregated in a non-additive way (such as count distinct)
+- In any of our Semantic Layer interfaces (not only Tableau), you must include a [time dimension](/docs/build/cumulative#limitations) when working with any cumulative metric that has a time window or granularity.
+
+## Unsupported functionality
+
+The following Tableau features aren't supported at this time, however, the dbt Semantic Layer may support some of this functionality in a future release:
+
+- Updating the data source page
+- Using "Extract" mode to view your data
+- Unioning Tables
+- Writing Custom SQL
+- Table Extensions
+- Cross Database Joins
+- All functions in Analysis --> Create Calculated Field
+- Filtering on a Date Part time dimension for a Cumulative metric type
+- Changing your date dimension to use "Week Number"
+
diff --git a/website/docs/docs/verified-adapters.md b/website/docs/docs/verified-adapters.md
index a2d28a612d6..170bc8f885b 100644
--- a/website/docs/docs/verified-adapters.md
+++ b/website/docs/docs/verified-adapters.md
@@ -13,6 +13,10 @@ The verification process serves as the on-ramp to integration with dbt Cloud. As
To learn more, see [Verifying a new adapter](/guides/dbt-ecosystem/adapter-development/7-verifying-a-new-adapter).
+import MSCallout from '/snippets/_microsoft-adapters-soon.md';
+
+
+
Here are the verified data platforms that connect to dbt and its latest version.
import AdaptersVerified from '/snippets/_adapters-verified.md';
diff --git a/website/docs/faqs/Environments/custom-branch-settings.md b/website/docs/faqs/Environments/custom-branch-settings.md
index 95929d2d393..4bc4b85be02 100644
--- a/website/docs/faqs/Environments/custom-branch-settings.md
+++ b/website/docs/faqs/Environments/custom-branch-settings.md
@@ -1,7 +1,7 @@
---
-title: How do I use the `Custom Branch` settings in a dbt Cloud Environment?
+title: How do I use the 'Custom Branch' settings in a dbt Cloud Environment?
description: "Use custom code from your repository"
-sidebar_label: 'Custom Branch settings'
+sidebar_label: 'Custom branch settings'
id: custom-branch-settings
---
@@ -15,12 +15,21 @@ To specify a custom branch:
## Development
-In a development environment, the default branch (commonly the `main` branch) is a read-only branch found in the IDE's connected repositories, which you can use to create development branches. Identifying a custom branch overrides this default behavior. Instead, your custom branch becomes read-only and can be used to create development branches. You will no longer be able to make commits to the custom branch from within the dbt Cloud IDE.
+In a development environment, the default branch (usually named `main`) is a read-only branch in your connected repositories, which allows you to create new branches for development from it.
-For example, you can use the `develop` branch of a connected repository. Edit an environment, select **Only run on a custom branch** in **General settings** , enter **develop** as the name of your custom branch.
+Specifying a **Custom branch** overrides the default behavior. It makes the custom branch 'read-only' and enables you to create new development branches from it. This also means you can't edit this custom branch directly.
-
+Only one branch can be read-only, which means when you set up a custom branch, your `main` branch (usually read-only) becomes editable. If you want to protect the `main` branch and prevent any commits on it, you need to set up branch protection rules in your git provider settings. This ensures your `main` branch remains secure and no new commits can be made to it.
+
+For example, if you want to use the `develop` branch of a connected repository:
+
+- Go to an environment and select **Settings** to edit it
+- Select **Only run on a custom branch** in **General settings**
+- Enter **develop** as the name of your custom branch
+- Click **Save**
+
+
## Deployment
-When running jobs in a deployment environment, dbt will clone your project from your connected repository before executing your models. By default, dbt uses the default branch of your repository (commonly the `main` branch). To specify a different version of your project for dbt to execute during job runs in a particular environment, you can edit the Custom Branch setting as shown in the previous steps.
\ No newline at end of file
+When running jobs in a deployment environment, dbt will clone your project from your connected repository before executing your models. By default, dbt uses the default branch of your repository (commonly the `main` branch). To specify a different version of your project for dbt to execute during job runs in a particular environment, you can edit the Custom Branch setting as shown in the previous steps.
diff --git a/website/docs/faqs/Models/reference-models-in-another-project.md b/website/docs/faqs/Models/reference-models-in-another-project.md
deleted file mode 100644
index 19f3f52da31..00000000000
--- a/website/docs/faqs/Models/reference-models-in-another-project.md
+++ /dev/null
@@ -1,11 +0,0 @@
----
-title: How can I reference models or macros in another project?
-description: "Use packages to add another project to your dbt project"
-sidebar_label: 'Reference models or macros in another project'
-id: reference-models-in-another-project
-
----
-
-You can use [packages](/docs/build/packages) to add another project to your dbt
-project, including other projects you've created. Check out the [docs](/docs/build/packages)
-for more information!
diff --git a/website/docs/faqs/Project/which-schema.md b/website/docs/faqs/Project/which-schema.md
index f0634ac8c85..2c21cba3c6a 100644
--- a/website/docs/faqs/Project/which-schema.md
+++ b/website/docs/faqs/Project/which-schema.md
@@ -7,7 +7,7 @@ id: which-schema
---
By default, dbt builds models in your target schema. To change your target schema:
* If you're developing in **dbt Cloud**, these are set for each user when you first use a development environment.
-* If you're developing with the **dbt CLI**, this is the `schema:` parameter in your `profiles.yml` file.
+* If you're developing with **dbt Core**, this is the `schema:` parameter in your `profiles.yml` file.
If you wish to split your models across multiple schemas, check out the docs on [using custom schemas](/docs/build/custom-schemas).
diff --git a/website/docs/faqs/Runs/checking-logs.md b/website/docs/faqs/Runs/checking-logs.md
index dbfdb6806a1..ff5e6f5cf04 100644
--- a/website/docs/faqs/Runs/checking-logs.md
+++ b/website/docs/faqs/Runs/checking-logs.md
@@ -10,7 +10,7 @@ To check out the SQL that dbt is running, you can look in:
* dbt Cloud:
* Within the run output, click on a model name, and then select "Details"
-* dbt CLI:
+* dbt Core:
* The `target/compiled/` directory for compiled `select` statements
* The `target/run/` directory for compiled `create` statements
* The `logs/dbt.log` file for verbose logging.
diff --git a/website/docs/faqs/Runs/failed-tests.md b/website/docs/faqs/Runs/failed-tests.md
index bfee565ef61..d19023d035d 100644
--- a/website/docs/faqs/Runs/failed-tests.md
+++ b/website/docs/faqs/Runs/failed-tests.md
@@ -10,7 +10,7 @@ To debug a failing test, find the SQL that dbt ran by:
* dbt Cloud:
* Within the test output, click on the failed test, and then select "Details"
-* dbt CLI:
+* dbt Core:
* Open the file path returned as part of the error message.
* Navigate to the `target/compiled/schema_tests` directory for all compiled test queries
diff --git a/website/docs/faqs/Warehouse/bq-oauth-drive-scope.md b/website/docs/faqs/Warehouse/bq-oauth-drive-scope.md
new file mode 100644
index 00000000000..ae6da82c47a
--- /dev/null
+++ b/website/docs/faqs/Warehouse/bq-oauth-drive-scope.md
@@ -0,0 +1,8 @@
+---
+title: Why does the BigQuery OAuth application require scopes to Google Drive?
+description: "Learn more about Google Drive scopes in the BigQuery OAuth application"
+sidebar_label: "BigQuery OAuth Drive Scopes"
+id: bq-oauth-drive-scope
+---
+
+BigQuery supports external tables over both personal Google Drive files and shared files. For more information, refer to [Create Google Drive external tables](https://cloud.google.com/bigquery/docs/external-data-drive).
diff --git a/website/docs/faqs/Warehouse/database-privileges.md b/website/docs/faqs/Warehouse/database-privileges.md
index 73e0549f130..3761b81fe67 100644
--- a/website/docs/faqs/Warehouse/database-privileges.md
+++ b/website/docs/faqs/Warehouse/database-privileges.md
@@ -12,8 +12,8 @@ schema¹
* read system views to generate documentation (i.e. views in
`information_schema`)
-On Postgres, Redshift, and Snowflake, use a series of `grants` to ensure that
-your user has the correct privileges.
+On Postgres, Redshift, Databricks, and Snowflake, use a series of `grants` to ensure that
+your user has the correct privileges. Check out [example permissions](/reference/database-permissions/about-database-permissions) for these warehouses.
On BigQuery, use the "BigQuery User" role to assign these privileges.
diff --git a/website/docs/guides/advanced/using-jinja.md b/website/docs/guides/advanced/using-jinja.md
index 40cfd2af298..1cbe88dc9ca 100644
--- a/website/docs/guides/advanced/using-jinja.md
+++ b/website/docs/guides/advanced/using-jinja.md
@@ -9,7 +9,7 @@ If you'd like to work through this query, add [this CSV](https://github.com/dbt-
While working through the steps of this model, we recommend that you have your compiled SQL open as well, to check what your Jinja compiles to. To do this:
* **Using dbt Cloud:** Click the compile button to see the compiled SQL in the right hand pane
-* **Using the dbt CLI:** Run `dbt compile` from the command line. Then open the compiled SQL file in the `target/compiled/{project name}/` directory. Use a split screen in your code editor to keep both files open at once.
+* **Using dbt Core:** Run `dbt compile` from the command line. Then open the compiled SQL file in the `target/compiled/{project name}/` directory. Use a split screen in your code editor to keep both files open at once.
## Write the SQL without Jinja
Consider a data model in which an `order` can have many `payments`. Each `payment` may have a `payment_method` of `bank_transfer`, `credit_card` or `gift_card`, and therefore each `order` can have multiple `payment_methods`
diff --git a/website/docs/guides/best-practices/debugging-errors.md b/website/docs/guides/best-practices/debugging-errors.md
index 39670820ddd..fe600ec4f67 100644
--- a/website/docs/guides/best-practices/debugging-errors.md
+++ b/website/docs/guides/best-practices/debugging-errors.md
@@ -17,7 +17,7 @@ Learning how to debug is a skill, and one that will make you great at your role!
- The `target/run` directory contains the SQL dbt executes to build your models.
- The `logs/dbt.log` file contains all the queries that dbt runs, and additional logging. Recent errors will be at the bottom of the file.
- **dbt Cloud users**: Use the above, or the `Details` tab in the command output.
- - **dbt CLI users**: Note that your code editor _may_ be hiding these files from the tree [VSCode help](https://stackoverflow.com/questions/42891463/how-can-i-show-ignored-files-in-visual-studio-code)).
+ - **dbt Core users**: Note that your code editor _may_ be hiding these files from the tree [VSCode help](https://stackoverflow.com/questions/42891463/how-can-i-show-ignored-files-in-visual-studio-code)).
5. If you are really stuck, try [asking for help](/community/resources/getting-help). Before doing so, take the time to write your question well so that others can diagnose the problem quickly.
@@ -184,7 +184,7 @@ hello: world # this is not allowed
## Compilation Errors
-_Note: if you're using the dbt Cloud IDE to work on your dbt project, this error often shows as a red bar in your command prompt as you work on your dbt project. For dbt CLI users, these won't get picked up until you run `dbt run` or `dbt compile`._
+_Note: if you're using the dbt Cloud IDE to work on your dbt project, this error often shows as a red bar in your command prompt as you work on your dbt project. For dbt Core users, these won't get picked up until you run `dbt run` or `dbt compile`._
### Invalid `ref` function
@@ -228,7 +228,7 @@ To fix this:
- Use the error message to find your mistake
To prevent this:
-- _(dbt CLI users only)_ Use snippets to auto-complete pieces of Jinja ([atom-dbt package](https://github.com/dbt-labs/atom-dbt), [vscode-dbt extestion](https://marketplace.visualstudio.com/items?itemName=bastienboutonnet.vscode-dbt))
+- _(dbt Core users only)_ Use snippets to auto-complete pieces of Jinja ([atom-dbt package](https://github.com/dbt-labs/atom-dbt), [vscode-dbt extestion](https://marketplace.visualstudio.com/items?itemName=bastienboutonnet.vscode-dbt))
@@ -280,7 +280,7 @@ To fix this:
- Find the mistake and fix it
To prevent this:
-- (dbt CLI users) Turn on indentation guides in your code editor to help you inspect your files
+- (dbt Core users) Turn on indentation guides in your code editor to help you inspect your files
- Use a YAML validator ([example](http://www.yamllint.com/)) to debug any issues
@@ -341,10 +341,10 @@ Database Error in model customers (models/customers.sql)
90% of the time, there's a mistake in the SQL of your model. To fix this:
1. Open the offending file:
- **dbt Cloud:** Open the model (in this case `models/customers.sql` as per the error message)
- - **dbt CLI:** Open the model as above. Also open the compiled SQL (in this case `target/run/jaffle_shop/models/customers.sql` as per the error message) — it can be useful to show these side-by-side in your code editor.
+ - **dbt Core:** Open the model as above. Also open the compiled SQL (in this case `target/run/jaffle_shop/models/customers.sql` as per the error message) — it can be useful to show these side-by-side in your code editor.
2. Try to re-execute the SQL to isolate the error:
- **dbt Cloud:** Use the `Preview` button from the model file
- - **dbt CLI:** Copy and paste the compiled query into a query runner (e.g. the Snowflake UI, or a desktop app like DataGrip / TablePlus) and execute it
+ - **dbt Core:** Copy and paste the compiled query into a query runner (e.g. the Snowflake UI, or a desktop app like DataGrip / TablePlus) and execute it
3. Fix the mistake.
4. Rerun the failed model.
@@ -356,7 +356,7 @@ In some cases, these errors might occur as a result of queries that dbt runs "be
In these cases, you should check out the logs — this contains _all_ the queries dbt has run.
- **dbt Cloud**: Use the `Details` in the command output to see logs, or check the `logs/dbt.log` file
-- **dbt CLI**: Open the `logs/dbt.log` file.
+- **dbt Core**: Open the `logs/dbt.log` file.
:::tip Isolating errors in the logs
If you're hitting a strange `Database Error`, it can be a good idea to clean out your logs by opening the file, and deleting the contents. Then, re-execute `dbt run` for _just_ the problematic model. The logs will _just_ have the output you're looking for.
@@ -379,6 +379,6 @@ Using the `Preview` button is useful when developing models and you want to visu
We’ve all been there. dbt uses the last-saved version of a file when you execute a command. In most code editors, and in the dbt Cloud IDE, a dot next to a filename indicates that a file has unsaved changes. Make sure you hit `cmd + s` (or equivalent) before running any dbt commands — over time it becomes muscle memory.
### Editing compiled files
-_(More likely for dbt CLI users)_
+_(More likely for dbt Core users)_
If you just opened a SQL file in the `target/` directory to help debug an issue, it's not uncommon to accidentally edit that file! To avoid this, try changing your code editor settings to grey out any files in the `target/` directory — the visual cue will help avoid the issue.
diff --git a/website/docs/guides/best-practices/how-we-build-our-metrics/semantic-layer-2-setup.md b/website/docs/guides/best-practices/how-we-build-our-metrics/semantic-layer-2-setup.md
index 34c0e813725..801227924dd 100644
--- a/website/docs/guides/best-practices/how-we-build-our-metrics/semantic-layer-2-setup.md
+++ b/website/docs/guides/best-practices/how-we-build-our-metrics/semantic-layer-2-setup.md
@@ -13,7 +13,7 @@ git clone git@github.com:dbt-labs/jaffle-sl-template.git
cd path/to/project
```
-Next before we start writing code, we'll need to install the MetricFlow CLI as an extension of a dbt adapter from PyPI. The MetricFlow CLI is compatible with Python versions 3.8 through 3.11.
+Next, before you start writing code, you need to install MetricFlow as an extension of a dbt adapter from PyPI (dbt Core users only). The MetricFlow is compatible with Python versions 3.8 through 3.11.
We'll use pip to install MetricFlow and our dbt adapter:
@@ -33,7 +33,7 @@ Lastly, to get to the pre-Semantic Layer starting state, checkout the `start-her
git checkout start-here
```
-For more information you can [look at the docs](/docs/build/metricflow-cli) or checkout a [Quickstart](https://docs.getdbt.com/quickstarts) to get more familiar with setting up a dbt project.
+For more information, refer to the [MetricFlow commands](/docs/build/metricflow-commands) or a [quickstart](/quickstarts) to get more familiar with setting up a dbt project.
## Basic commands
diff --git a/website/docs/guides/best-practices/how-we-mesh/mesh-1-intro.md b/website/docs/guides/best-practices/how-we-mesh/mesh-1-intro.md
new file mode 100644
index 00000000000..ba1660a8d82
--- /dev/null
+++ b/website/docs/guides/best-practices/how-we-mesh/mesh-1-intro.md
@@ -0,0 +1,39 @@
+---
+title: "Intro to dbt Mesh"
+description: Getting started with dbt Mesh patterns
+hoverSnippet: Learn how to get started with dbt Mesh
+---
+
+## What is dbt Mesh?
+
+Organizations of all sizes rely upon dbt to manage their data transformations, from small startups to large enterprises. At scale, it can be challenging to coordinate all the organizational and technical requirements demanded by your stakeholders within the scope of a single dbt project. To date, there also hasn't been a first-class way to effectively manage the dependencies, governance, and workflows between multiple dbt projects.
+
+Regardless of your organization's size and complexity, dbt should empower data teams to work independently and collaboratively; sharing data, code, and best practices without sacrificing security or autonomy. dbt Mesh provides the tooling for teams to finally achieve this.
+
+dbt Mesh is not a single product: it is a pattern enabled by a convergence of several features in dbt:
+
+- **[Cross-project references](/docs/collaborate/govern/project-dependencies#how-to-use-ref)** - this is the foundational feature that enables the multi-project deployments. `{{ ref() }}`s now work across dbt Cloud projects on Enterprise plans.
+- **[dbt Explorer](/docs/collaborate/explore-projects)** - dbt Cloud's metadata-powered documentation platform, complete with full, cross-project lineage.
+- **Governance** - dbt's new governance features allow you to manage access to your dbt models both within and across projects.
+ - **[Groups](/docs/collaborate/govern/model-access#groups)** - groups allow you to assign models to subsets within a project.
+ - **[Access](/docs/collaborate/govern/model-access#access-modifiers)** - access configs allow you to control who can reference models.
+- **[Model Versions](/docs/collaborate/govern/model-versions)** - when coordinating across projects and teams, we recommend treating your data models as stable APIs. Model versioning is the mechanism to allow graceful adoption and deprecation of models as they evolve.
+- **[Model Contracts](/docs/collaborate/govern/model-contracts)** - data contracts set explicit expectations on the shape of the data to ensure data changes upstream of dbt or within a project's logic don't break downstream consumers' data products.
+
+## Who is dbt Mesh for?
+
+The multi-project architecture helps organizations with mature, complex transformation workflows in dbt increase the flexibility and performance of their dbt projects. If you're already using dbt and your project has started to experience any of the following, you're likely ready to start exploring this paradigm:
+
+- The **number of models** in your project is degrading performance and slowing down development.
+- Teams have developed **separate workflows** and need to decouple development from each other.
+- **Security and governance** requirements are increasing and would benefit from increased isolation.
+
+dbt Cloud is designed to coordinate the features above and simplify the complexity to solve for these problems.
+
+If you're just starting your dbt journey, don't worry about building a multi-project architecture right away. You can _incrementally_ adopt the features in this guide as you scale. The collection of features work effectively as independent tools. Familiarizing yourself with the tooling and features that make up a multi-project architecture, and how they can apply to your organization will help you make better decisions as you grow.
+
+## Learning goals
+
+- Understand the **purpose and tradeoffs** of building a multi-project architecture.
+- Develop an intuition for various **dbt Mesh patterns** and how to design a multi-project architecture for your organization.
+- Establish recommended steps to **incrementally adopt** these patterns in your dbt implementation.
diff --git a/website/docs/guides/best-practices/how-we-mesh/mesh-2-structures.md b/website/docs/guides/best-practices/how-we-mesh/mesh-2-structures.md
new file mode 100644
index 00000000000..937515954af
--- /dev/null
+++ b/website/docs/guides/best-practices/how-we-mesh/mesh-2-structures.md
@@ -0,0 +1,52 @@
+---
+title: Deciding how to structure your dbt Mesh
+description: Getting started with dbt Mesh patterns
+hoverSnippet: Learn how to get started with dbt Mesh
+---
+## Exploring mesh patterns
+
+When adopting a multi-project architecture, where do you draw the lines between projects?
+
+How should you organize data workflows in a world where instead of having a single dbt DAG, you have multiple projects speaking to each other, each comprised of their own DAG?
+
+Adopting the dbt Mesh pattern is not a one-size-fits-all process. In fact, it's the opposite! It's about customizing your project structure to fit _your_ team and _your_ data. Now you can mold your organizational knowledge graph to your organizational people graph, bringing people and data closer together rather than compromising one for the other.
+
+While there is not a single best way to implement this pattern, there are some common decision points that will be helpful for you to consider.
+
+At a high level, you’ll need to decide:
+
+- Where to draw the lines between your dbt Projects -- i.e. how do you determine where to split your DAG and which models go in which project?
+- How to manage your code -- do you want multiple dbt Projects living in the same repository (mono-repo) or do you want to have multiple repos with one repo per project?
+
+## Define your project interfaces by splitting your DAG
+
+The first (and perhaps most difficult!) decision when migrating to a multi-project architecture is deciding where to draw the line in your DAG to define the interfaces between your projects. Let's explore some language for discussing the design of these patterns.
+
+### Vertical splits
+
+Vertical splits separate out layers of transformation in DAG order. Let's look at some examples.
+
+- **Splitting up staging and mart layers** to create a more tightly-controlled, shared set of components that other projects build on but can't edit.
+- **Isolating earlier models for security and governance requirements** to separate out and mask PII data so that downstream consumers can't access it is a common use case for a vertical split.
+- **Protecting complex or expensive data** to isolate large or complex models that are expensive to run so that they are safe from accidental selection, independently deployable, and easier to debug when they have issues.
+
+### Horizontal splits
+
+Horizontal splits separate your DAG based on source or domain. These splits are often based around the shape and size of the data and how it's used. Let's consider some possibilities for horizontal splitting.
+
+- **Team consumption patterns.** For example, splitting out the marketing team's data flow into a separate project.
+- **Data from different sources.** For example, clickstream event data and transactional ecommerce data may need to be modeled independently of each other.
+- **Team workflows.** For example, if two embedded groups operate at different paces, you may want to split the projects up so they can move independently.
+
+### Combining these strategies
+
+- **These are not either/or techniques**. You should consider both types of splits, and combine them in any way that makes sense for your organization.
+- **Pick one type of split and focus on that first**. If you have a hub-and-spoke team topology for example, handle breaking out the central platform project before you split the remainder into domains. Then if you need to break those domains up horizontally you can focus on that after the fact.
+- **DRY applies to underlying data, not just code.** Regardless of your strategy, you should not be sourcing the same rows and columns into multiple nodes. When working within a mesh pattern it becomes increasingly important that we don't duplicate logic or data.
+
+## Determine your git strategy
+
+A multi-project architecture can exist in a single repo (monorepo) or as multiple projects, with each one being in their own repository (multi-repo).
+
+- If you're a **smaller team** looking primarily to speed up and simplify development, a **monorepo** is likely the right choice, but can become unwieldy as the number of projects, models and contributors grow.
+- If you’re a **larger team with multiple groups**, and need to decouple projects for security and enablement of different development styles and rhythms, a **multi-repo setup** is your best bet.
diff --git a/website/docs/guides/best-practices/how-we-mesh/mesh-3-implementation.md b/website/docs/guides/best-practices/how-we-mesh/mesh-3-implementation.md
new file mode 100644
index 00000000000..cfbbc7a1f28
--- /dev/null
+++ b/website/docs/guides/best-practices/how-we-mesh/mesh-3-implementation.md
@@ -0,0 +1,130 @@
+---
+title: "Implementing your mesh plan"
+description: Getting started with dbt Mesh patterns
+hoverSnippet: Learn how to get started with dbt Mesh
+---
+
+As mentioned before, the key decision in migrating to a multi-project architecture is understanding how your project is already being grouped, built, and deployed. We can use this information to inform our decision to split our project apart.
+
+- **Examine your jobs** - which sets of models are most often built together?
+- **Look at your lineage graph** - how are models connected?
+- **Look at your selectors** defined in `selectors.yml` - how do people already define resource groups?
+- **Talk to teams** about what sort of separation naturally exists right now.
+ - Are there various domains people are focused on?
+ - Are there various sizes, shapes, and sources of data that get handled separately (such as click event data)?
+ - Are there people focused on separate levels of transformation, such as landing and staging data or building marts?
+
+## Add groups and access
+
+Once you have a sense of some initial groupings, you can first implement **group and access permissions** within a single project.
+
+- First you can create a [group](/docs/build/groups) to define the owner of a set of models.
+
+```yml
+# in models/__groups.yml
+
+groups:
+ - name: marketing
+ owner:
+ - name: Ben Jaffleck
+ email: ben.jaffleck@jaffleshop.com
+```
+
+- Then, we can add models to that group using the `group:` key in the model's YAML entry.
+
+```yml
+# in models/marketing/__models.yml
+
+models:
+ - name: fct_marketing_model
+ group: marketing
+ - name: stg_marketing_model
+ group: marketing
+```
+
+- Once you've added models to the group, you can **add [access](/docs/collaborate/govern/model-access) settings to the models** based on their connections between groups, *opting for the most private access that will maintain current functionality*. This means that any model that has *only* relationships to other models in the same group should be `private` , and any model that has cross-group relationships, or is a terminal node in the group DAG should be `protected` so that other parts of the DAG can continue to reference it.
+
+```yml
+# in models/marketing/__models.yml
+
+models:
+ - name: fct_marketing_model
+ group: marketing
+ access: protected
+ - name: stg_marketing_model
+ group: marketing
+ access: private
+```
+
+- **Validate these groups by incrementally migrating your jobs** to execute these groups specifically via selection syntax. We would recommend doing this in parallel to your production jobs until you’re sure about them. This will help you feel out if you’ve drawn the lines in the right place.
+- If you find yourself **consistently making changes across multiple groups** when you update logic, that’s a sign that **you may want to rethink your groups**.
+
+## Split your projects
+
+1. **Move your grouped models into a subfolder**. This will include any model in the selected group, it's associated YAML entry, as well as its parent or child resources as appropriate depending on where this group sits in your DAG.
+ 1. Note that just like in your dbt project, circular refereneces are not allowed! Project B cannot have parents and children in Project A, for example.
+2. **Create a new `dbt_project.yml` file** in the subdirectory.
+3. **Copy any macros** used by the resources you moved.
+4. **Create a new `packages.yml` file** in your subdirectory with the packages that are used by the resources you moved.
+5. **Update `{{ ref }}` functions** — For any model that has a cross-project dependency (this may be in the files you moved, or in the files that remain in your project):
+ 1. Update the `{{ ref() }}` function to have two arguments, where the first is the name of the source project and the second is the name of the model: e.g. `{{ ref('jaffle_shop', 'my_upstream_model') }}`
+ 2. Update the upstream, cross-project parents’ `access` configs to `public` , ensuring any project can safely `{{ ref() }}` those models.
+ 3. We *highly* recommend adding a [model contract](/docs/collaborate/govern/model-contracts) to the upstream models to ensure the data shape is consistent and reliable for your downstream consumers.
+6. **Create a `dependencies.yml` file** ([docs](/docs/collaborate/govern/project-dependencies)) for the downstream project, declaring the upstream project as a dependency.
+
+```yml
+
+# in dependencies.yml
+projects:
+ - name: jaffle_shop
+```
+
+### Best practices
+
+- When you’ve **confirmed the right groups**, it's time to split your projects.
+ - **Do *one* group at a time**!
+ - **Do *not* refactor as you migrate**, however tempting that may be. Focus on getting 1-to-1 parity and log any issues you find in doing the migration for later. Once you’ve fully migrated the project then you can start optimizing it for its new life as part of your mesh.
+- Start by splitting your project within the same repository for full git tracking and easy reversion if you need to start from scratch.
+
+
+## Connecting existing projects
+
+Some organizations may already be coordinating across multiple dbt projects. Most often this is via:
+
+1. Installing parent projects as dbt packages
+2. Using `{{ source() }}` functions to read the outputs of a parent project as inputs to a child project.
+
+This has a few drawbacks:
+
+1. If using packages, each project has to include *all* resources from *all* projects in its manifest, slowing down dbt and the development cycle.
+2. If using sources, there are breakages in the lineage, as there's no real connection between the parent and child projects.
+
+The migration steps here are much simpler than splitting up a monolith!
+
+1. If using the `package` method:
+ 1. In the parent project:
+ 1. mark all models being referenced downstream as `public` and add a model contract.
+ 2. In the child project:
+ 1. Remove the package entry from `packages.yml`
+ 2. Add the upstream project to your `dependencies.yml`
+ 3. Update the `{{ ref() }}` functions to models from the upstream project to include the project name argument.
+1. If using `source` method:
+ 1. In the parent project:
+ 1. mark all models being imported downstream as `public` and add a model contract.
+ 2. In the child project:
+ 1. Add the upstream project to your `dependencies.yml`
+ 2. Replace the `{{ source() }}` functions with cross project `{{ ref() }}` functions.
+ 3. Remove the unnecessary `source` definitions.
+
+## Additional Resources
+### Our example projects
+
+We've provided a set of example projects you can use to explore the topics covered here. We've split our [Jaffle Shop](https://github.com/dbt-labs/jaffle-shop) project into 3 separate projects in a multi-repo dbt Mesh. Note that you'll need to leverage dbt Cloud to use multi-project architecture, as cross-project references are powered via dbt Cloud's APIs.
+
+- **[Platform](https://github.com/dbt-labs/jaffle-shop-mesh-platform)** - containing our centralized staging models.
+- **[Marketing](https://github.com/dbt-labs/jaffle-shop-mesh-marketing)** - containing our marketing marts.
+- **[Finance](https://github.com/dbt-labs/jaffle-shop-mesh-finance)** - containing our finance marts.
+
+### dbt-meshify
+
+We recommend using the `dbt-meshify` [command line tool]() to help you do this. This comes with CLI operations to automate most of the above steps.
diff --git a/website/docs/guides/best-practices/materializations/materializations-guide-4-incremental-models.md b/website/docs/guides/best-practices/materializations/materializations-guide-4-incremental-models.md
index 603cbc8cda1..cd4264bafd3 100644
--- a/website/docs/guides/best-practices/materializations/materializations-guide-4-incremental-models.md
+++ b/website/docs/guides/best-practices/materializations/materializations-guide-4-incremental-models.md
@@ -29,7 +29,7 @@ We did our last `dbt build` job on `2022-01-31`, so any new orders since that ru
- 🏔️ build the table from the **beginning of time again — a _table materialization_**
- Simple and solid, if we can afford to do it (in terms of time, compute, and money — which are all directly correlated in a cloud warehouse). It’s the easiest and most accurate option.
- 🤏 find a way to run **just new and updated rows since our previous run — _an_ _incremental materialization_**
- - If we _can’t_ realistically afford to run the whole table — due to complex transformations or big source data, it takes too long — then we want to build incrementally. We want to just transform and add the row with id 567 below, _not_ the previous two with ids 123 and 456 that are already in the table.
+ - If we _can’t_ realistically afford to run the whole table — due to complex transformations or big source data, it takes too long — then we want to build incrementally. We want to just transform and add the row with id 567 below, _not_ the previous two with ids 123 and 234 that are already in the table.
| order_id | order_status | customer_id | order_item_id | ordered_at | updated_at |
| -------- | ------------ | ----------- | ------------- | ---------- | ---------- |
diff --git a/website/docs/guides/best-practices/materializations/materializations-guide-6-examining-builds.md b/website/docs/guides/best-practices/materializations/materializations-guide-6-examining-builds.md
index 07811b42594..909618ef8a5 100644
--- a/website/docs/guides/best-practices/materializations/materializations-guide-6-examining-builds.md
+++ b/website/docs/guides/best-practices/materializations/materializations-guide-6-examining-builds.md
@@ -12,7 +12,7 @@ hoverSnippet: Read this guide to understand how to examine your builds in dbt.
- ⌚ dbt keeps track of how **long each model took to build**, when it started, when it finished, its completion status (error, warn, or success), its materialization type, and _much_ more.
- 🖼️ This information is stored in a couple files which dbt calls **artifacts**.
- 📊 Artifacts contain a ton of information in JSON format, so aren’t easy to read, but **dbt Cloud** packages the most useful bits of information into a tidy **visualization** for you.
-- ☁️ If you’re not using Cloud, we can still use the output of the **dbt CLI to understand our runs**.
+- ☁️ If you’re not using Cloud, we can still use the output of the **dbt Core CLI to understand our runs**.
### Model Timing
@@ -23,9 +23,9 @@ That’s where dbt Cloud’s Model Timing visualization comes in extremely handy
- 🧵 This view lets us see our **mapped out in threads** (up to 64 threads, we’re currently running with 4, so we get 4 tracks) over time. You can think of **each thread as a lane on a highway**.
- ⌛ We can see above that `customer_status_histories` is **taking by far the most time**, so we may want to go ahead and **make that incremental**.
-If you aren’t using dbt Cloud, that’s okay! We don’t get a fancy visualization out of the box, but we can use the output from the dbt CLI to check our model times, and it’s a great opportunity to become familiar with that output.
+If you aren’t using dbt Cloud, that’s okay! We don’t get a fancy visualization out of the box, but we can use the output from the dbt Core CLI to check our model times, and it’s a great opportunity to become familiar with that output.
-### dbt CLI output
+### dbt Core CLI output
If you’ve ever run dbt, whether `build`, `test`, `run` or something else, you’ve seen some output like below. Let’s take a closer look at how to read this.
diff --git a/website/docs/guides/dbt-ecosystem/dbt-python-snowpark/6-foundational-structure.md b/website/docs/guides/dbt-ecosystem/dbt-python-snowpark/6-foundational-structure.md
index e387b208dd1..8a938e10c34 100644
--- a/website/docs/guides/dbt-ecosystem/dbt-python-snowpark/6-foundational-structure.md
+++ b/website/docs/guides/dbt-ecosystem/dbt-python-snowpark/6-foundational-structure.md
@@ -71,7 +71,7 @@ In this step, we’ll need to create a development branch and set up project lev
- `materialized` — Tells dbt how to materialize models when compiling the code before it pushes it down to Snowflake. All models in the `marts` folder will be built as tables.
- `tags` — Applies tags at a directory level to all models. All models in the `aggregates` folder will be tagged as `bi` (abbreviation for business intelligence).
- `docs` — Specifies the `node_color` either by the plain color name or a hex value.
-5. [Materializations](/docs/build/materializations) are strategies for persisting dbt models in a warehouse, with `tables` and `views` being the most commonly utilized types. By default, all dbt models are materialized as views and other materialization types can be configured in the `dbt_project.yml` file or in a model itself. It’s very important to note *Python models can only be materialized as tables or incremental models.* Since all our Python models exist under `marts`, the following portion of our `dbt_project.yml` ensures no errors will occur when we run our Python models. Starting with [dbt version 1.4](/guides/migration/versions/upgrading-to-v1.4#updates-to-python-models), Python files will automatically get materialized as tables even if not explicitly specified.
+5. [Materializations](/docs/build/materializations) are strategies for persisting dbt models in a warehouse, with `tables` and `views` being the most commonly utilized types. By default, all dbt models are materialized as views and other materialization types can be configured in the `dbt_project.yml` file or in a model itself. It’s very important to note *Python models can only be materialized as tables or incremental models.* Since all our Python models exist under `marts`, the following portion of our `dbt_project.yml` ensures no errors will occur when we run our Python models. Starting with [dbt version 1.4](/docs/dbt-versions/core-upgrade/upgrading-to-v1.4#updates-to-python-models), Python files will automatically get materialized as tables even if not explicitly specified.
```yaml
marts:
diff --git a/website/docs/guides/dbt-ecosystem/sl-partner-integration-guide.md b/website/docs/guides/dbt-ecosystem/sl-partner-integration-guide.md
index 68037bfd0cd..936a54465e8 100644
--- a/website/docs/guides/dbt-ecosystem/sl-partner-integration-guide.md
+++ b/website/docs/guides/dbt-ecosystem/sl-partner-integration-guide.md
@@ -4,11 +4,6 @@ id: "sl-partner-integration-guide"
description: Learn about partner integration guidelines, roadmap, and connectivity.
---
-
-import NewChanges from '/snippets/_new-sl-changes.md';
-
-
-
To fit your tool within the world of the Semantic Layer, dbt Labs offers some best practice recommendations for how to expose metrics and allow users to interact with them seamlessly.
:::note
@@ -20,7 +15,7 @@ This is an evolving guide that is meant to provide recommendations based on our
To build a dbt Semantic Layer integration:
-- We offer a [JDBC](/docs/dbt-cloud-apis/sl-jdbc) API (and will soon offer a GraphQL API). Refer to the dedicated [dbt Semantic Layer API](/docs/dbt-cloud-apis/sl-api-overview) for more technical integration details.
+- We offer a [JDBC](/docs/dbt-cloud-apis/sl-jdbc) API and [GraphQL API](/docs/dbt-cloud-apis/sl-graphql). Refer to the dedicated [dbt Semantic Layer API](/docs/dbt-cloud-apis/sl-api-overview) for more technical integration details.
- Familiarize yourself with the [dbt Semantic Layer](/docs/use-dbt-semantic-layer/dbt-sl) and [MetricFlow](/docs/build/about-metricflow)'s key concepts. There are two main objects:
@@ -33,6 +28,14 @@ The dbt Semantic Layer APIs authenticate with `environmentId`, `SERVICE_TOKEN`,
We recommend you provide users with separate input fields with these components for authentication (dbt Cloud will surface these parameters for the user).
+### Exposing metadata to dbt Labs
+
+When building an integration, we recommend you expose certain metadata in the request for analytics purposes. Among other items, it is helpful to have the following:
+
+- Your application's name (such as 'Tableau')
+- The email of the person querying your application
+- The version of dbt they are on.
+
## Best practices on exposing metrics
@@ -85,7 +88,7 @@ Allow users to query either one metric alone without dimensions or multiple metr
- Allow toggling between metrics/dimensions seamlessly.
-- Be clear on exposing what dimensions are queryable with what metrics and hide things that don’t apply, and vice versa.
+- Be clear on exposing what dimensions are queryable with what metrics and hide things that don’t apply. (Our APIs provide calls for you to get relevant dimensions for metrics, and vice versa).
- Only expose time granularities (monthly, daily, yearly) that match the available metrics.
* For example, if a dbt model and its resulting semantic model have a monthly granularity, make sure querying data with a 'daily' granularity isn't available to the user. Our APIs have functionality that will help you surface the correct granularities
@@ -109,6 +112,15 @@ For better analysis, it's best to have the context of the metrics close to where
- Allow for creating other metadata that’s useful for the metric. We can provide some of this information in our configuration (Display name, Default Granularity for View, Default Time range), but there may be other metadata that your tool wants to provide to make the metric richer.
+### Transparency and using compile
+
+For transparency and additional context, we recommend you have an easy way for the user to obtain the SQL that MetricFlow generates. Depending on what API you are using, you can do this by using our `compile` parameter. This is incredibly powerful and emphasizes transparency and openness, particularly for technically inclined users.
+
+
+### Where filters and optimization
+
+In the cases where our APIs support either a string or a filter list for the `where` clause, we always recommend that your application utilizes the filter list in order to gain maximum pushdown benefits. The `where` string may be more intuitive for users writing queries during testing, but it will not have the performance benefits of the filter list in a production environment.
+
## Example stages of an integration
These are recommendations on how to evolve a Semantic Layer integration and not a strict runbook.
@@ -136,14 +148,6 @@ These are recommendations on how to evolve a Semantic Layer integration and not
* Querying dimensions without metrics and other more advanced querying functionality
* Suggest metrics to users based on teams/identity, and so on.
-### A note on transparency and using compile
-
-For transparency and additional context, we recommend you have an easy way for the user to obtain the SQL that MetricFlow generates. Depending on what API you are using, you can do this by using our compile parameter. This is incredibly powerful because we want to be very transparent to the user about what we're doing and do not want to be a black box. This would be mostly beneficial to a technical user.
-
-
-### A note on where filters
-
-In the cases where our APIs support either a string or a filter list for the `where` clause, we always recommend that your application utilizes the filter list in order to gain maximum pushdown benefits. The `where` string may be more intuitive for users writing queries during testing, but it will not have the performance benefits of the filter list in a production environment.
## Related docs
diff --git a/website/docs/guides/migration/sl-migration.md b/website/docs/guides/migration/sl-migration.md
index c9def4537a3..56cd6dc9d80 100644
--- a/website/docs/guides/migration/sl-migration.md
+++ b/website/docs/guides/migration/sl-migration.md
@@ -12,7 +12,7 @@ The legacy Semantic Layer will be deprecated in H2 2023. Additionally, the `dbt_
The metrics specification in dbt Core is changed in v1.6 to support the integration of MetricFlow. It's strongly recommended that you refer to [Build your metrics](/docs/build/build-metrics-intro) and before getting started so you understand the core concepts of the Semantic Layer.
-dbt Labs recommends completing these steps in a local dev environment instead of the IDE:
+dbt Labs recommends completing these steps in a local dev environment (such as the [dbt Cloud CLI](/docs/cloud/cloud-cli-installation)) instead of the dbt Cloud IDE:
1. Create new Semantic Model configs as YAML files in your dbt project.*
1. Upgrade the metrics configs in your project to the new spec.*
@@ -63,11 +63,19 @@ You might need to audit metric values during the migration to ensure that the hi
This step is only relevant to users who want the legacy and new semantic layer to run in parallel for a short time. This will let you recreate content in downstream tools like Hex and Mode with minimal downtime. If you do not need to recreate assets in these tools skip to step 5.
1. Create a new deployment environment in dbt Cloud and set the dbt version to 1.6 or higher.
-2. Choose `Only run on a custom branch` and point to the branch that has the updated metric definition
+
+2. Select **Only run on a custom branch** and point to the branch that has the updated metric definition.
+
3. Set the deployment schema to a temporary migration schema, such as `tmp_sl_migration`. Optional, you can create a new database for the migration.
-4. Create a job to parse your project, such as `dbt parse`, and run it. Make sure this job succeeds, There needs to be a successful job in your environment in order to set up the semantic layer
-5. In Account Settings > Projects > Project details click `Configure the Semantic Layer`. Under **Environment**select the deployment environment you created in the previous step. Save your configuration.
-6. In the Project details page, click `Generate service token` and grant it `Semantic Layer Only` and `Metadata Only` permissions. Save this token securely - you will need it to connect to the semantic layer.
+
+4. Create a job to parse your project, such as `dbt parse`, and run it. Make sure this job succeeds. There needs to be a successful job in your environment in order to set up the semantic layer.
+
+5. Select **Account Settings** -> **Projects** -> **Project details** and choose **Configure the Semantic Layer**.
+
+6. Under **Environment**, select the deployment environment you created in the previous step. Save your configuration.
+
+7. In the **Project details** page, click **Generate service token** and grant it **Semantic Layer Only** and **Metadata Only** permissions. Save this token securely. You will need it to connect to the semantic layer.
+
At this point, both the new semantic layer and the old semantic layer will be running. The new semantic layer will be pointing at your migration branch with the updated metrics definitions.
@@ -106,7 +114,7 @@ To learn more about integrating with Hex, check out their [documentation](https:
If you created a new environment in [Step 3](#step-3-setup-the-semantic-layer-in-a-new-environment):
-3. Update your Environment in Account Settings > Project Details > Edit Semantic Layer Configuration to point to your production environment
+3. Update your Environment in **Account Settings** -> **Project Details** -> **Edit Semantic Layer Configuration** to point to your production environment
4. Delete your migration environment. Be sure to update your connection details in any downstream tools to account for the environment change.
diff --git a/website/docs/guides/migration/tools/migrating-from-spark-to-databricks.md b/website/docs/guides/migration/tools/migrating-from-spark-to-databricks.md
index f5549c58416..cd0577c2d96 100644
--- a/website/docs/guides/migration/tools/migrating-from-spark-to-databricks.md
+++ b/website/docs/guides/migration/tools/migrating-from-spark-to-databricks.md
@@ -35,7 +35,7 @@ In both dbt Core and dbt Cloud, you can migrate your projects to the Databricks-
### Prerequisites
-- Your project must be compatible with dbt 1.0 or greater. Refer to [Upgrading to v1.0](/guides/migration/versions/upgrading-to-v1.0) for details. For the latest version of dbt, refer to [Upgrading to v1.3](/guides/migration/versions/upgrading-to-v1.3).
+- Your project must be compatible with dbt 1.0 or greater. Refer to [Upgrading to v1.0](/docs/dbt-versions/core-upgrade/upgrading-to-v1.0) for details. For the latest version of dbt, refer to [Upgrading to v1.3](/docs/dbt-versions/core-upgrade/upgrading-to-v1.3).
- For dbt Cloud, you need administrative (admin) privileges to migrate dbt projects.
diff --git a/website/docs/guides/migration/versions/00-upgrading-to-v1.7.md b/website/docs/guides/migration/versions/00-upgrading-to-v1.7.md
deleted file mode 100644
index 036c734dfb1..00000000000
--- a/website/docs/guides/migration/versions/00-upgrading-to-v1.7.md
+++ /dev/null
@@ -1,24 +0,0 @@
----
-title: "Upgrading to v1.7 (beta)"
-description: New features and changes in dbt Core v1.7
----
-
-## Resources
-
-- [Changelog](https://github.com/dbt-labs/dbt-core/blob/8aaed0e29f9560bc53d9d3e88325a9597318e375/CHANGELOG.md)
-- [CLI Installation guide](/docs/core/installation)
-- [Cloud upgrade guide](/docs/dbt-versions/upgrade-core-in-cloud)
-- [Release schedule](https://github.com/dbt-labs/dbt-core/issues/7481)
-
-## What to know before upgrading
-
-dbt Labs is committed to providing backward compatibility for all versions 1.x, with the exception of any changes explicitly mentioned below. If you encounter an error upon upgrading, please let us know by [opening an issue](https://github.com/dbt-labs/dbt-core/issues/new).
-
-### Behavior changes
-
-**COMING SOON**
-
-### Quick hits
-
-**COMING SOON**
-
diff --git a/website/docs/guides/orchestration/airflow-and-dbt-cloud/1-airflow-and-dbt-cloud.md b/website/docs/guides/orchestration/airflow-and-dbt-cloud/1-airflow-and-dbt-cloud.md
index d453106eead..d6760771b79 100644
--- a/website/docs/guides/orchestration/airflow-and-dbt-cloud/1-airflow-and-dbt-cloud.md
+++ b/website/docs/guides/orchestration/airflow-and-dbt-cloud/1-airflow-and-dbt-cloud.md
@@ -19,7 +19,7 @@ There are [so many great examples](https://gitlab.com/gitlab-data/analytics/-/bl
### Airflow + dbt Cloud API w/Custom Scripts
-This has served as a bridge until the fabled Astronomer + dbt Labs-built dbt Cloud provider became generally available [here](https://registry.astronomer.io/providers/dbt-cloud?type=Sensors&utm_campaign=Monthly%20Product%20Updates&utm_medium=email&_hsmi=208603877&utm_content=208603877&utm_source=hs_email).
+This has served as a bridge until the fabled Astronomer + dbt Labs-built dbt Cloud provider became generally available [here](https://registry.astronomer.io/providers/dbt%20Cloud/versions/latest).
There are many different permutations of this over time:
diff --git a/website/docs/guides/orchestration/custom-cicd-pipelines/4-dbt-cloud-job-on-pr.md b/website/docs/guides/orchestration/custom-cicd-pipelines/4-dbt-cloud-job-on-pr.md
index b58bab175b3..1a75fdc17ac 100644
--- a/website/docs/guides/orchestration/custom-cicd-pipelines/4-dbt-cloud-job-on-pr.md
+++ b/website/docs/guides/orchestration/custom-cicd-pipelines/4-dbt-cloud-job-on-pr.md
@@ -94,7 +94,7 @@ Add this as a macro to your project. It takes 2 arguments that lets you control
```sql
{#
This macro finds PR schemas older than a set date and drops them
- The maco defaults to 10 days old, but can be configued with the input argument age_in_days
+ The macro defaults to 10 days old, but can be configured with the input argument age_in_days
Sample usage with different date:
dbt run-operation pr_schema_cleanup --args "{'database_to_clean': 'analytics','age_in_days':'15'}"
#}
diff --git a/website/docs/quickstarts/bigquery-qs.md b/website/docs/quickstarts/bigquery-qs.md
index 7f7f9aa7655..546b56c234c 100644
--- a/website/docs/quickstarts/bigquery-qs.md
+++ b/website/docs/quickstarts/bigquery-qs.md
@@ -73,7 +73,6 @@ In order to let dbt connect to your warehouse, you'll need to generate a keyfile
1. Start the [GCP credentials wizard](https://console.cloud.google.com/apis/credentials/wizard). Make sure your new project is selected in the header. If you do not see your account or project, click your profile picture to the right and verify you are using the correct email account. For **Credential Type**:
- From the **Select an API** dropdown, choose **BigQuery API**
- Select **Application data** for the type of data you will be accessing
- - Select **No, I’m not using them** and click **Next**.
- Click **Next** to create a new service account.
2. Create a service account for your new project from the [Service accounts page](https://console.cloud.google.com/projectselector2/iam-admin/serviceaccounts?supportedpurview=project). For more information, refer to [Create a service account](https://developers.google.com/workspace/guides/create-credentials#create_a_service_account) in the Google Cloud docs. As an example for this guide, you can:
- Type `dbt-user` as the **Service account name**
@@ -89,25 +88,22 @@ In order to let dbt connect to your warehouse, you'll need to generate a keyfile
4. Click **Upload a Service Account JSON File** in settings.
5. Select the JSON file you downloaded in [Generate BigQuery credentials](#generate-bigquery-credentials) and dbt Cloud will fill in all the necessary fields.
6. Click **Test Connection**. This verifies that dbt Cloud can access your BigQuery account.
-7. Click **Next** if the test succeeded. If it failed, you might need to go back and regenerate your BigQuery credentials.
+7. Click **Next** if the test succeeds. If it fails, you might need to go back and regenerate your BigQuery credentials.
## Set up a dbt Cloud managed repository
-## Initialize your dbt project and start developing
+## Initialize your dbt project
Now that you have a repository configured, you can initialize your project and start development in dbt Cloud:
1. Click **Start developing in the IDE**. It might take a few minutes for your project to spin up for the first time as it establishes your git connection, clones your repo, and tests the connection to the warehouse.
2. Above the file tree to the left, click **Initialize dbt project**. This builds out your folder structure with example models.
3. Make your initial commit by clicking **Commit and sync**. Use the commit message `initial commit` and click **Commit**. This creates the first commit to your managed repo and allows you to open a branch where you can add new dbt code.
4. You can now directly query data from your warehouse and execute `dbt run`. You can try this out now:
- - Click **+ Create new file**, add this query to the new file, and click **Save as** to save the new file:
- ```sql
- select * from `dbt-tutorial.jaffle_shop.customers`
- ```
- In the command line bar at the bottom, enter `dbt run` and click **Enter**. You should see a `dbt run succeeded` message.
+ - To confirm the success of the above command, navigate to the BigQuery Console and discover the newly created models.
## Build your first model
1. Under **Version Control** on the left, click **Create branch**. You can name it `add-customers-model`. You need to create a new branch since the main branch is set to read-only mode.
@@ -175,7 +171,7 @@ select * from final
6. Enter `dbt run` in the command prompt at the bottom of the screen. You should get a successful run and see the three models.
-Later, you can connect your business intelligence (BI) tools to these views and tables so they only read cleaned up data rather than raw data in your BI tool.
+Later, you can connect your business intelligence (BI) tools to these views and tables so they only read cleaned-up data rather than raw data in your BI tool.
#### FAQs
@@ -283,7 +279,7 @@ Later, you can connect your business intelligence (BI) tools to these views and
4. Execute `dbt run`.
- This time, when you performed a `dbt run`, separate views/tables were created for `stg_customers`, `stg_orders` and `customers`. dbt inferred the order to run these models. Because `customers` depends on `stg_customers` and `stg_orders`, dbt builds `customers` last. You do not need to explicitly define these dependencies.
+ This time, when you performed a `dbt run`, separate views/tables were created for `stg_customers`, `stg_orders`, and `customers`. dbt inferred the order to run these models. Because `customers` depends on `stg_customers` and `stg_orders`, dbt builds `customers` last. You do not need to explicitly define these dependencies.
#### FAQs {#faq-2}
diff --git a/website/docs/quickstarts/manual-install-qs.md b/website/docs/quickstarts/manual-install-qs.md
index 05336178ff6..fc43d38115b 100644
--- a/website/docs/quickstarts/manual-install-qs.md
+++ b/website/docs/quickstarts/manual-install-qs.md
@@ -9,11 +9,11 @@ hide_table_of_contents: true
---
## Introduction
-When you use dbt Core to work with dbt, you will be editing files locally using a code editor, and running projects using the dbt command line interface (dbt CLI). If you'd rather edit files and run projects using the web-based Integrated Development Environment (IDE), you should refer to the [dbt Cloud quickstarts](/quickstarts).
+When you use dbt Core to work with dbt, you will be editing files locally using a code editor, and running projects using a command line interface (CLI). If you'd rather edit files and run projects using the web-based Integrated Development Environment (IDE), you should refer to the [dbt Cloud quickstarts](/quickstarts). You can also develop and run dbt commands using the [dbt Cloud CLI](/docs/cloud/cloud-cli-installation) — a dbt Cloud powered command line.
### Prerequisites
-* To use the dbt CLI, it's important that you know some basics of the Terminal. In particular, you should understand `cd`, `ls` and `pwd` to navigate through the directory structure of your computer easily.
+* To use dbt Core, it's important that you know some basics of the Terminal. In particular, you should understand `cd`, `ls` and `pwd` to navigate through the directory structure of your computer easily.
* Install dbt Core using the [installation instructions](/docs/core/installation) for your operating system.
* Complete [Setting up (in BigQuery)](/quickstarts/bigquery?step=2) and [Loading data (BigQuery)](/quickstarts/bigquery?step=3).
* [Create a GitHub account](https://github.com/join) if you don't already have one.
@@ -103,16 +103,16 @@ When developing locally, dbt connects to your using
jaffle_shop: # this needs to match the profile in your dbt_project.yml file
target: dev
outputs:
- dev:
- type: bigquery
- method: service-account
- keyfile: /Users/BBaggins/.dbt/dbt-tutorial-project-331118.json # replace this with the full path to your keyfile
- project: grand-highway-265418 # Replace this with your project id
- dataset: dbt_bbagins # Replace this with dbt_your_name, e.g. dbt_bilbo
- threads: 1
- timeout_seconds: 300
- location: US
- priority: interactive
+ dev:
+ type: bigquery
+ method: service-account
+ keyfile: /Users/BBaggins/.dbt/dbt-tutorial-project-331118.json # replace this with the full path to your keyfile
+ project: grand-highway-265418 # Replace this with your project id
+ dataset: dbt_bbagins # Replace this with dbt_your_name, e.g. dbt_bilbo
+ threads: 1
+ timeout_seconds: 300
+ location: US
+ priority: interactive
```
@@ -196,7 +196,7 @@ $ git checkout -b add-customers-model
4. From the command line, enter `dbt run`.
-
+
When you return to the BigQuery console, you can `select` from this model.
diff --git a/website/docs/reference/artifacts/manifest-json.md b/website/docs/reference/artifacts/manifest-json.md
index 5e8dcedd2d5..47a9849eda5 100644
--- a/website/docs/reference/artifacts/manifest-json.md
+++ b/website/docs/reference/artifacts/manifest-json.md
@@ -3,15 +3,9 @@ title: "Manifest JSON file"
sidebar_label: "Manifest"
---
-| dbt Core version | Manifest version |
-|------------------|---------------------------------------------------------------|
-| v1.6 | [v10](https://schemas.getdbt.com/dbt/manifest/v10/index.html) |
-| v1.5 | [v9](https://schemas.getdbt.com/dbt/manifest/v9/index.html) |
-| v1.4 | [v8](https://schemas.getdbt.com/dbt/manifest/v8/index.html) |
-| v1.3 | [v7](https://schemas.getdbt.com/dbt/manifest/v7/index.html) |
-| v1.2 | [v6](https://schemas.getdbt.com/dbt/manifest/v6/index.html) |
-| v1.1 | [v5](https://schemas.getdbt.com/dbt/manifest/v5/index.html) |
-| v1.0 | [v4](https://schemas.getdbt.com/dbt/manifest/v4/index.html) |
+import ManifestVersions from '/snippets/_manifest-versions.md';
+
+
**Produced by:** Any command that parses your project. This includes all commands **except** [`deps`](/reference/commands/deps), [`clean`](/reference/commands/clean), [`debug`](/reference/commands/debug), [`init`](/reference/commands/init)
diff --git a/website/docs/reference/artifacts/other-artifacts.md b/website/docs/reference/artifacts/other-artifacts.md
index d776bc8a099..205bdfc1a14 100644
--- a/website/docs/reference/artifacts/other-artifacts.md
+++ b/website/docs/reference/artifacts/other-artifacts.md
@@ -39,4 +39,8 @@ This file is useful for investigating performance issues in dbt Core's graph alg
It is more anonymized and compact than [`manifest.json`](/reference/artifacts/manifest-json) and [`graph.gpickle`](#graph.gpickle).
-It contains only the `name` and `type` of each node along with IDs of its child nodes (`succ`). It includes that information at two separate points in time: immediately after the graph is linked together (`linked`), and after test edges have been added (`with_test_edges`).
+It includes that information at two separate points in time:
+1. `linked` — immediately after the graph is linked together, and
+2. `with_test_edges` — after test edges have been added.
+
+Each of those points in time contains the `name` and `type` of each node and `succ` contains the keys of its child nodes.
diff --git a/website/docs/reference/commands/clone.md b/website/docs/reference/commands/clone.md
index a3c8bb236c7..9852ce84c17 100644
--- a/website/docs/reference/commands/clone.md
+++ b/website/docs/reference/commands/clone.md
@@ -21,7 +21,7 @@ The `clone` command is useful for:
dbt clone --state path/to/artifacts
# clone one_specific_model of my models from specified state to my target schema(s)
-dbt clone --select one_specific_model --state path/to/artifacts
+dbt clone --select "one_specific_model" --state path/to/artifacts
# clone all of my models from specified state to my target schema(s) and recreate all pre-existing relations in the current target
dbt clone --state path/to/artifacts --full-refresh
@@ -37,3 +37,5 @@ Unlike deferral, `dbt clone` requires some compute and creation of additional ob
For example, by creating actual data warehouse objects, `dbt clone` allows you to test out your code changes on downstream dependencies _outside of dbt_ (such as a BI tool).
As another example, you could `clone` your modified incremental models as the first step of your dbt Cloud CI job to prevent costly `full-refresh` builds for warehouses that support zero-copy cloning.
+
+Check out [this Developer blog post](https://docs.getdbt.com/blog/to-defer-or-to-clone) for more details on best practices when to use `dbt clone` vs. deferral.
diff --git a/website/docs/reference/commands/cmd-docs.md b/website/docs/reference/commands/cmd-docs.md
index 754c5e93baf..bc4840464b8 100644
--- a/website/docs/reference/commands/cmd-docs.md
+++ b/website/docs/reference/commands/cmd-docs.md
@@ -19,6 +19,18 @@ The command is responsible for generating your project's documentation website b
dbt docs generate
```
+
+
+Use the `--select` argument to limit the nodes included within `catalog.json`. When this flag is provided, step (3) will be restricted to the selected nodes. All other nodes will be excluded. Step (2) is unaffected.
+
+**Example**:
+```shell
+dbt docs generate --select +orders
+```
+
+
+
+
Use the `--no-compile` argument to skip re-compilation. When this flag is provided, `dbt docs generate` will skip step (2) described above.
**Example**:
diff --git a/website/docs/reference/commands/compile.md b/website/docs/reference/commands/compile.md
index ed403d2af32..cde65b7c6b6 100644
--- a/website/docs/reference/commands/compile.md
+++ b/website/docs/reference/commands/compile.md
@@ -29,7 +29,7 @@ This will log the compiled SQL to the terminal, in addition to writing to the `t
For example:
```bash
-dbt compile --select stg_payments
+dbt compile --select "stg_payments"
dbt compile --inline "select * from {{ ref('raw_orders') }}"
```
@@ -37,7 +37,7 @@ returns the following:
```bash
-dbt compile --select stg_orders
+dbt compile --select "stg_orders"
21:17:09 Running with dbt=1.5.0-b5
21:17:09 Found 5 models, 20 tests, 0 snapshots, 0 analyses, 425 macros, 0 operations, 3 seed files, 0 sources, 0 exposures, 0 metrics, 0 groups
21:17:09
diff --git a/website/docs/reference/commands/deps.md b/website/docs/reference/commands/deps.md
index 4c7a36606e2..f4f8153c115 100644
--- a/website/docs/reference/commands/deps.md
+++ b/website/docs/reference/commands/deps.md
@@ -57,3 +57,31 @@ Installing calogica/dbt_date@0.4.0
Updates available for packages: ['tailsdotcom/dbt_artifacts', 'dbt-labs/snowplow']
Update your versions in packages.yml, then run dbt deps
```
+
+
+
+dbt generates the `package-lock.yml` file in the _project_root_ where `packages.yml` is recorded, which contains all the resolved packages, the first time you run `dbt deps`. Each subsequent run records the packages installed in this file. If the subsequent `dbt deps` runs contain no updated packages in `depenedencies.yml` or `packages.yml`, dbt-core installs from `package-lock.yml`.
+
+When you update the package spec and run `dbt deps` again, the package-lock and package files update accordingly. You can run `dbt deps --lock` to update the `package-lock.yml` with the most recent dependencies from `packages`.
+
+The `--add` flag allows you to add a package to the `packages.yml` with configurable `--version` and `--source` information. The `--dry-run` flag, when set to `False`(default), recompiles the `package-lock.yml` file after a new package is added to the `packages.yml` file. Set the flag to `True` for the changes to not persist.
+
+Examples of the `--add` flag:
+```shell
+# add package from hub (--source arg defaults to "hub")
+dbt deps add --package dbt-labs/dbt_utils --version 1.0.0
+
+# add package from hub with semantic version
+dbt deps add --package dbt-labs/snowplow --version ">=0.7.0,<0.8.0"
+
+# add package from git
+dbt deps add --package https://github.com/fivetran/dbt_amplitude --version v0.3.0 --source git
+
+# add package from local (--version not required for local)
+dbt deps add --package /opt/dbt/redshift --source local
+
+# add package to packages.yml WITHOUT updating package-lock.yml
+dbt deps add --package dbt-labs/dbt_utils --version 1.0.0 --dry-run True
+
+```
+
\ No newline at end of file
diff --git a/website/docs/reference/commands/init.md b/website/docs/reference/commands/init.md
index 873647814ec..ac55717c0ec 100644
--- a/website/docs/reference/commands/init.md
+++ b/website/docs/reference/commands/init.md
@@ -17,10 +17,21 @@ Then, it will:
- Create a new folder with your project name and sample files, enough to get you started with dbt
- Create a connection profile on your local machine. The default location is `~/.dbt/profiles.yml`. Read more in [configuring your profile](/docs/core/connect-data-platform/connection-profiles).
+
+
+When using `dbt init` to initialize your project, include the `--profile` flag to specify an existing `profiles.yml` as the `profile:` key to use instead of creating a new one. For example, `dbt init --profile`.
+
+
+
+If the profile does not exist in `profiles.yml` or the command is run inside an existing project, the command raises an error.
+
+
+
## Existing project
If you've just cloned or downloaded an existing dbt project, `dbt init` can still help you set up your connection profile so that you can start working quickly. It will prompt you for connection information, as above, and add a profile (using the `profile` name from the project) to your local `profiles.yml`, or create the file if it doesn't already exist.
+
## profile_template.yml
`dbt init` knows how to prompt for connection information by looking for a file named `profile_template.yml`. It will look for this file in two places:
diff --git a/website/docs/reference/commands/list.md b/website/docs/reference/commands/list.md
index 6084b3dec70..5caabdc2b2e 100644
--- a/website/docs/reference/commands/list.md
+++ b/website/docs/reference/commands/list.md
@@ -8,9 +8,10 @@ id: "list"
The `dbt ls` command lists resources in your dbt project. It accepts selector arguments that are similar to those provided in [dbt run](/reference/commands/run). `dbt list` is an alias for `dbt ls`. While `dbt ls` will read your [connection profile](/docs/core/connect-data-platform/connection-profiles) to resolve [`target`](/reference/dbt-jinja-functions/target)-specific logic, this command will not connect to your database or run any queries.
### Usage
+
```
dbt ls
- [--resource-type {model,source,seed,snapshot,metric,test,exposure,analysis,default,all}]
+ [--resource-type {model,semantic_model,source,seed,snapshot,metric,test,exposure,analysis,default,all}]
[--select SELECTION_ARG [SELECTION_ARG ...]]
[--models SELECTOR [SELECTOR ...]]
[--exclude SELECTOR [SELECTOR ...]]
@@ -85,7 +86,7 @@ $ dbt ls --select snowplow.* --output json --output-keys "name resource_type des
```
-$ dbt ls --select snowplow.* --output json --output-keys name resource_type description
+$ dbt ls --select snowplow.* --output json --output-keys "name resource_type description"
{"name": "snowplow_events", "description": "This is a pretty cool model", ...}
{"name": "snowplow_page_views", "description": "This model is even cooler", ...}
...
@@ -93,6 +94,16 @@ $ dbt ls --select snowplow.* --output json --output-keys name resource_type desc
+
+
+**Listing Semantic models**
+
+List all resources upstream of your orders semantic model:
+```
+dbt ls -s +semantic_model:orders
+```
+
+
**Listing file paths**
```
diff --git a/website/docs/reference/commands/retry.md b/website/docs/reference/commands/retry.md
index d494a46cf1f..8da5d5a77a6 100644
--- a/website/docs/reference/commands/retry.md
+++ b/website/docs/reference/commands/retry.md
@@ -4,14 +4,6 @@ sidebar_label: "retry"
id: "retry"
---
-:::info Support in dbt Cloud
-
-`dbt retry` is supported in the dbt Cloud IDE.
-
-Native support for restarting scheduled runs from point of failure is currently in development & coming soon.
-
-:::
-
`dbt retry` re-executes the last `dbt` command from the node point of failure. If the previously executed `dbt` command was successful, `retry` will finish as `no operation`.
Retry works with the following commands:
diff --git a/website/docs/reference/commands/seed.md b/website/docs/reference/commands/seed.md
index 8a410706842..d0cd199ea12 100644
--- a/website/docs/reference/commands/seed.md
+++ b/website/docs/reference/commands/seed.md
@@ -12,7 +12,7 @@ The `dbt seed` command will load `csv` files located in the `seed-paths` directo
Specific seeds can be run using the `--select` flag to `dbt seed`. Example:
```
-$ dbt seed --select country_codes
+$ dbt seed --select "country_codes"
Found 2 models, 3 tests, 0 archives, 0 analyses, 53 macros, 0 operations, 2 seed files
14:46:15 | Concurrency: 1 threads (target='dev')
diff --git a/website/docs/reference/commands/show.md b/website/docs/reference/commands/show.md
index 5bdcfacc1e8..a0e5d68c83f 100644
--- a/website/docs/reference/commands/show.md
+++ b/website/docs/reference/commands/show.md
@@ -16,7 +16,7 @@ The results of the preview query are not materialized in the data warehouse, or
Example:
```
-dbt show --select model_name.sql
+dbt show --select "model_name.sql"
```
or
```
@@ -26,7 +26,7 @@ dbt show --inline "select * from {{ ref('model_name') }}"
The following is an example of `dbt show` output for a model named `stg_orders`:
```bash
-dbt show --select stg_orders
+dbt show --select "stg_orders"
21:17:38 Running with dbt=1.5.0-b5
21:17:38 Found 5 models, 20 tests, 0 snapshots, 0 analyses, 425 macros, 0 operations, 3 seed files, 0 sources, 0 exposures, 0 metrics, 0 groups
21:17:38
@@ -46,7 +46,7 @@ dbt show --select stg_orders
For example, if you've just built a model that has a failing test, you can quickly preview the test failures right in the terminal, to find values of `id` that are duplicated:
```bash
-$ dbt build -s my_model_with_duplicates
+$ dbt build -s "my_model_with_duplicates"
13:22:47 Running with dbt=1.5.0
...
13:22:48 Completed with 1 error and 0 warnings:
@@ -58,7 +58,7 @@ $ dbt build -s my_model_with_duplicates
13:22:48
13:22:48 Done. PASS=1 WARN=0 ERROR=1 SKIP=0 TOTAL=2
-$ dbt show -s unique_my_model_with_duplicates_id
+$ dbt show -s "unique_my_model_with_duplicates_id"
13:22:53 Running with dbt=1.5.0
13:22:53 Found 4 models, 2 tests, 0 snapshots, 0 analyses, 309 macros, 0 operations, 0 seed files, 0 sources, 0 exposures, 0 metrics, 0 groups
13:22:53
diff --git a/website/docs/reference/commands/source.md b/website/docs/reference/commands/source.md
index b29bf7dadc6..697ae2b5fcc 100644
--- a/website/docs/reference/commands/source.md
+++ b/website/docs/reference/commands/source.md
@@ -20,10 +20,10 @@ By default, `dbt source freshness` will calculate freshness information for all
```bash
# Snapshot freshness for all Snowplow tables:
-$ dbt source freshness --select source:snowplow
+$ dbt source freshness --select "source:snowplow"
# Snapshot freshness for a particular source table:
-$ dbt source freshness --select source:snowplow.event
+$ dbt source freshness --select "source:snowplow.event"
```
### Configuring source freshness output
diff --git a/website/docs/reference/commands/test.md b/website/docs/reference/commands/test.md
index a1a63729568..c050d82a0ab 100644
--- a/website/docs/reference/commands/test.md
+++ b/website/docs/reference/commands/test.md
@@ -10,22 +10,22 @@ The tests to run can be selected using the `--select` flag discussed [here](/ref
```bash
# run tests for one_specific_model
-dbt test --select one_specific_model
+dbt test --select "one_specific_model"
# run tests for all models in package
-dbt test --select some_package.*
+dbt test --select "some_package.*"
# run only tests defined singularly
-dbt test --select test_type:singular
+dbt test --select "test_type:singular"
# run only tests defined generically
-dbt test --select test_type:generic
+dbt test --select "test_type:generic"
# run singular tests limited to one_specific_model
-dbt test --select one_specific_model,test_type:singular
+dbt test --select "one_specific_model,test_type:singular"
# run generic tests limited to one_specific_model
-dbt test --select one_specific_model,test_type:generic
+dbt test --select "one_specific_model,test_type:generic"
```
For more information on writing tests, see the [Testing Documentation](/docs/build/tests).
diff --git a/website/docs/reference/database-permissions/about-database-permissions.md b/website/docs/reference/database-permissions/about-database-permissions.md
new file mode 100644
index 00000000000..76fff517646
--- /dev/null
+++ b/website/docs/reference/database-permissions/about-database-permissions.md
@@ -0,0 +1,36 @@
+---
+title: "Database permissions"
+id: about-database-permissions
+description: "Database permissions are access rights and privileges granted to users or roles within a database management system."
+sidebar_label: "About database permissions"
+pagination_next: "reference/database-permissions/databricks-permissions"
+pagination_prev: null
+---
+
+Database permissions are access rights and privileges granted to users or roles within a database or data platform. They help you specify what actions users or roles can perform on various database objects, like tables, views, schemas, or even the entire database.
+
+
+### Why are they useful
+
+- Database permissions are essential for security and data access control.
+- They ensure that only authorized users can perform specific actions.
+- They help maintain data integrity, prevent unauthorized changes, and limit exposure to sensitive data.
+- Permissions also support compliance with data privacy regulations and auditing.
+
+### How to use them
+
+- Users and administrators can grant and manage permissions at various levels (such as table, schema, and so on) using SQL statements or through the database system's interface.
+- Assign permissions to individual users or roles (groups of users) based on their responsibilities.
+ - Typical permissions include "SELECT" (read), "INSERT" (add data), "UPDATE" (modify data), "DELETE" (remove data), and administrative rights like "CREATE" and "DROP."
+- Users should be assigned permissions that ensure they have the necessary access to perform their tasks without overextending privileges.
+
+Something to note is that each data platform provider might have different approaches and names for privileges. Refer to their documentation for more details.
+
+### Examples
+
+Refer to the following database permission pages for more info on examples and how to set up database permissions:
+
+- [Databricks](/reference/database-permissions/databricks-permissions)
+- [Postgres](/reference/database-permissions/postgres-permissions)
+- [Redshift](/reference/database-permissions/redshift-permissions)
+- [Snowflake](/reference/database-permissions/snowflake-permissions)
diff --git a/website/docs/reference/database-permissions/databricks-permissions.md b/website/docs/reference/database-permissions/databricks-permissions.md
new file mode 100644
index 00000000000..12e24652ae3
--- /dev/null
+++ b/website/docs/reference/database-permissions/databricks-permissions.md
@@ -0,0 +1,20 @@
+---
+title: "Databricks permissions"
+---
+
+In Databricks, permissions are used to control who can perform certain actions on different database objects. Use SQL statements to manage permissions in a Databricks database.
+
+## Example Databricks permissions
+
+The following example provides you with the SQL statements you can use to manage permissions.
+
+**Note** that you can grant permissions on `securable_objects` to `principals` (This can be user, service principal, or group). For example, `grant privilege_type` on `securable_object` to `principal`.
+
+```
+
+grant all privileges on schema schema_name to principal;
+grant create table on schema schema_name to principal;
+grant create view on schema schema_name to principal;
+```
+
+Check out the [official documentation](https://docs.databricks.com/en/data-governance/unity-catalog/manage-privileges/privileges.html#privilege-types-by-securable-object-in-unity-catalog) for more information.
diff --git a/website/docs/reference/database-permissions/postgres-permissions.md b/website/docs/reference/database-permissions/postgres-permissions.md
new file mode 100644
index 00000000000..da56e9b45f2
--- /dev/null
+++ b/website/docs/reference/database-permissions/postgres-permissions.md
@@ -0,0 +1,25 @@
+---
+title: "Postgres Permissions"
+---
+
+
+In Postgres, permissions are used to control who can perform certain actions on different database objects. Use SQL statements to manage permissions in a Postgres database.
+
+## Example Postgres permissions
+
+The following example provides you with the SQL statements you can use to manage permissions. These examples allow you to run dbt smoothly without encountering permission issues, such as creating schemas, reading existing data, and accessing the information schema.
+
+**Note** that `database_name`, `database.schema_name`, and `user_name` are placeholders and you can replace them as needed for your organization's naming convention.
+
+```
+grant usage on database database_name to user_name;
+grant create schema on database database_name to user_name;
+grant usage on schema database.schema_name to user_name;
+grant create table on schema database.schema_name to user_name;
+grant create view on schema database.schema_name to user_name;
+grant usage on all schemas in database database_name to user_name;
+grant select on all tables in database database_name to user_name;
+grant select on all views in database database_name to user_name;
+```
+
+Check out the [official documentation](https://www.postgresql.org/docs/current/sql-grant.html) for more information.
diff --git a/website/docs/reference/database-permissions/redshift-permissions.md b/website/docs/reference/database-permissions/redshift-permissions.md
new file mode 100644
index 00000000000..5f0949a3528
--- /dev/null
+++ b/website/docs/reference/database-permissions/redshift-permissions.md
@@ -0,0 +1,25 @@
+---
+title: "Redshift permissions"
+---
+
+In Redshift, permissions are used to control who can perform certain actions on different database objects. Use SQL statements to manage permissions in a Redshift database.
+
+## Example Redshift permissions
+
+The following example provides you with the SQL statements you can use to manage permissions.
+
+**Note** that `database_name`, `database.schema_name`, and `user_name` are placeholders and you can replace them as needed for your organization's naming convention.
+
+
+```
+grant usage on database database_name to user_name;
+grant create schema on database database_name to user_name;
+grant usage on schema database.schema_name to user_name;
+grant create table on schema database.schema_name to user_name;
+grant create view on schema database.schema_name to user_name;
+grant usage on all schemas in database database_name to user_name;
+grant select on all tables in database database_name to user_name;
+grant select on all views in database database_name to user_name;
+```
+
+Check out the [official documentation](https://docs.aws.amazon.com/redshift/latest/dg/r_GRANT.html) for more information.
diff --git a/website/docs/reference/database-permissions/snowflake-permissions.md b/website/docs/reference/database-permissions/snowflake-permissions.md
new file mode 100644
index 00000000000..3f474242834
--- /dev/null
+++ b/website/docs/reference/database-permissions/snowflake-permissions.md
@@ -0,0 +1,154 @@
+---
+title: "Snowflake permissions"
+---
+
+In Snowflake, permissions are used to control who can perform certain actions on different database objects. Use SQL statements to manage permissions in a Snowflake database.
+
+## Set up Snowflake account
+
+This section explains how to set up permissions and roles within Snowflake. In Snowflake, you would perform these actions using SQL commands and set up your data warehouse and access control within Snowflake's ecosystem.
+
+1. Set up databases
+```
+use role sysadmin;
+create database raw;
+create database analytics;
+```
+2. Set up warehouses
+```
+create warehouse loading
+ warehouse_size = xsmall
+ auto_suspend = 3600
+ auto_resume = false
+ initially_suspended = true;
+
+create warehouse transforming
+ warehouse_size = xsmall
+ auto_suspend = 60
+ auto_resume = true
+ initially_suspended = true;
+
+create warehouse reporting
+ warehouse_size = xsmall
+ auto_suspend = 60
+ auto_resume = true
+ initially_suspended = true;
+```
+
+3. Set up roles and warehouse permissions
+```
+use role securityadmin;
+
+create role loader;
+grant all on warehouse loading to role loader;
+
+create role transformer;
+grant all on warehouse transforming to role transformer;
+
+create role reporter;
+grant all on warehouse reporting to role reporter;
+```
+
+4. Create users, assigning them to their roles
+
+Every person and application gets a separate user and is assigned to the correct role.
+
+```
+create user stitch_user -- or fivetran_user
+ password = '_generate_this_'
+ default_warehouse = loading
+ default_role = loader;
+
+create user claire -- or amy, jeremy, etc.
+ password = '_generate_this_'
+ default_warehouse = transforming
+ default_role = transformer
+ must_change_password = true;
+
+create user dbt_cloud_user
+ password = '_generate_this_'
+ default_warehouse = transforming
+ default_role = transformer;
+
+create user looker_user -- or mode_user etc.
+ password = '_generate_this_'
+ default_warehouse = reporting
+ default_role = reporter;
+
+-- then grant these roles to each user
+grant role loader to user stitch_user; -- or fivetran_user
+grant role transformer to user dbt_cloud_user;
+grant role transformer to user claire; -- or amy, jeremy
+grant role reporter to user looker_user; -- or mode_user, periscope_user
+```
+
+5. Let loader load data
+Give the role unilateral permission to operate on the raw database
+```
+use role sysadmin;
+grant all on database raw to role loader;
+```
+
+6. Let transformer transform data
+The transformer role needs to be able to read raw data.
+
+If you do this before you have any data loaded, you can run:
+```
+grant usage on database raw to role transformer;
+grant usage on future schemas in database raw to role transformer;
+grant select on future tables in database raw to role transformer;
+grant select on future views in database raw to role transformer;
+```
+If you already have data loaded in the raw database, make sure also you run the following to update the permissions
+```
+grant usage on all schemas in database raw to role transformer;
+grant select on all tables in database raw to role transformer;
+grant select on all views in database raw to role transformer;
+```
+transformer also needs to be able to create in the analytics database:
+```
+grant all on database analytics to role transformer;
+```
+7. Let reporter read the transformed data
+A previous version of this article recommended this be implemented through hooks in dbt, but this way lets you get away with a one-off statement.
+```
+grant usage on database analytics to role reporter;
+grant usage on future schemas in database analytics to role reporter;
+grant select on future tables in database analytics to role reporter;
+grant select on future views in database analytics to role reporter;
+```
+Again, if you already have data in your analytics database, make sure you run:
+```
+grant usage on all schemas in database analytics to role reporter;
+grant select on all tables in database analytics to role transformer;
+grant select on all views in database analytics to role transformer;
+```
+8. Maintain
+When new users are added, make sure you add them to the right role! Everything else should be inherited automatically thanks to those `future` grants.
+
+For more discussion and legacy information, refer to [this Discourse article](https://discourse.getdbt.com/t/setting-up-snowflake-the-exact-grant-statements-we-run/439).
+
+## Example Snowflake permissions
+
+The following example provides you with the SQL statements you can use to manage permissions.
+
+**Note** that `warehouse_name`, `database_name`, and `role_name` are placeholders and you can replace them as needed for your organization's naming convention.
+
+```
+
+grant all on warehouse warehouse_name to role role_name;
+grant usage on database database_name to role role_name;
+grant create schema on database database_name to role role_name;
+grant usage on schema database.an_existing_schema to role role_name;
+grant create table on schema database.an_existing_schema to role role_name;
+grant create view on schema database.an_existing_schema to role role_name;
+grant usage on future schemas in database database_name to role role_name;
+grant monitor on future schemas in database database_name to role role_name;
+grant select on future tables in database database_name to role role_name;
+grant select on future views in database database_name to role role_name;
+grant usage on all schemas in database database_name to role role_name;
+grant monitor on all schemas in database database_name to role role_name;
+grant select on all tables in database database_name to role role_name;
+grant select on all views in database database_name to role role_name;
+```
+
diff --git a/website/docs/reference/dbt-commands.md b/website/docs/reference/dbt-commands.md
index 862829ef809..4bc3ddc24d7 100644
--- a/website/docs/reference/dbt-commands.md
+++ b/website/docs/reference/dbt-commands.md
@@ -2,54 +2,59 @@
title: "dbt Command reference"
---
-dbt is typically run one of two ways:
+You can run dbt using the following tools:
-* In [dbt Cloud](/docs/cloud/dbt-cloud-ide/develop-in-the-cloud)
-* On the [command line interface](/docs/core/about-the-cli) (CLI)
+- In your browser with the [dbt Cloud IDE](/docs/cloud/dbt-cloud-ide/develop-in-the-cloud)
+- On the command line interface using the [dbt Cloud CLI](/docs/cloud/cloud-cli-installation) or open-source [dbt Core](/docs/core/about-dbt-core), both of which enable you to execute dbt commands. The key distinction is the dbt Cloud CLI is tailored for dbt Cloud's infrastructure and integrates with all its [features](/docs/cloud/about-cloud/dbt-cloud-features).
The following sections outline the commands supported by dbt and their relevant flags. For information about selecting models on the command line, consult the docs on [Model selection syntax](/reference/node-selection/syntax).
### Available commands
-
-
-Use the following dbt commands in the [dbt Cloud IDE](/docs/cloud/dbt-cloud-ide/develop-in-the-cloud) or [CLI](/docs/core/about-the-cli). Use the `dbt` prefix. For example, to run the `test` command, type `dbt test`.
-
-| Command | Description | Version |
-| ------- | ----------- | ------- |
-| [build](/reference/commands/build) | Build and test all selected resources (models, seeds, snapshots, tests) | All [supported versions](/docs/dbt-versions/core) |
-| [clean](/reference/commands/clean) | Deletes artifacts present in the dbt project | All [supported versions](/docs/dbt-versions/core) |
-| [clone](/reference/commands/clone) | Clone selected models from the specified state | Requires [dbt v1.6 or higher](/docs/dbt-versions/core) |
-| [compile](/reference/commands/compile) | Compiles (but does not run) the models in a project | All [supported versions](/docs/dbt-versions/core) |
-| [debug](/reference/commands/debug) | Debugs dbt connections and projects | All [supported versions](/docs/dbt-versions/core) |
-| [deps](/reference/commands/deps) | Downloads dependencies for a project | All [supported versions](/docs/dbt-versions/core) |
-| [docs](/reference/commands/cmd-docs) | Generates documentation for a project | All [supported versions](/docs/dbt-versions/core) |
-| [list](/reference/commands/list) | Lists resources defined in a dbt project | All [supported versions](/docs/dbt-versions/core) |
-| [parse](/reference/commands/parse) | Parses a project and writes detailed timing info | All [supported versions](/docs/dbt-versions/core) |
-| [retry](/reference/commands/retry) | Retry the last run `dbt` command from the point of failure | Requires [dbt v1.6 or higher](/docs/dbt-versions/core) |
-| [run](/reference/commands/run) | Runs the models in a project | All [supported versions](/docs/dbt-versions/core) |
-| [run-operation](/reference/commands/run-operation) | Invoke a macro, including running arbitrary maintenance SQL against
the database | All [supported versions](/docs/dbt-versions/core) |
-| [seed](/reference/commands/seed) | Loads CSV files into the database | All [supported versions](/docs/dbt-versions/core) |
-| [show](/reference/commands/show) | Preview table rows post-transformation | All [supported versions](/docs/dbt-versions/core) |
-| [snapshot](/reference/commands/snapshot) | Executes "snapshot" jobs defined in a project | All [supported versions](/docs/dbt-versions/core) |
-| [source](/reference/commands/source) | Provides tools for working with source data (including validating that
sources are "fresh") | All [supported versions](/docs/dbt-versions/core) |
-| [test](/reference/commands/test) | Executes tests defined in a project | All [supported versions](/docs/dbt-versions/core) |
-| [init](/reference/commands/init) | Initializes a new dbt project (CLI only) | All [supported versions](/docs/dbt-versions/core) |
+
+
+All commands in the table are compatible with either the dbt Cloud IDE, dbt Cloud CLI, or dbt Core.
+
+You can run dbt commands in your specific tool by prefixing them with `dbt`. For example, to run the `test` command, type `dbt test`.
+
+| Command | Description | Compatible tools | Version |
+| ------- | ----------- | ---------------- | ------- |
+| [build](/reference/commands/build) | Build and test all selected resources (models, seeds, snapshots, tests) | All | All [supported versions](/docs/dbt-versions/core) |
+| cancel | Cancels the most recent invocation.| dbt Cloud CLI | Requires [dbt v1.6 or higher](/docs/dbt-versions/core) |
+| [clean](/reference/commands/clean) | Deletes artifacts present in the dbt project | All | All [supported versions](/docs/dbt-versions/core) |
+| [clone](/reference/commands/clone) | Clone selected models from the specified state | dbt Cloud CLI
dbt Core | Requires [dbt v1.6 or higher](/docs/dbt-versions/core) |
+| [compile](/reference/commands/compile) | Compiles (but does not run) the models in a project | All | All [supported versions](/docs/dbt-versions/core) |
+| [debug](/reference/commands/debug) | Debugs dbt connections and projects | dbt Core | All [supported versions](/docs/dbt-versions/core) |
+| [deps](/reference/commands/deps) | Downloads dependencies for a project | All | All [supported versions](/docs/dbt-versions/core) |
+| [docs](/reference/commands/cmd-docs) | Generates documentation for a project | All | All [supported versions](/docs/dbt-versions/core) |
+| help | Displays help information for any command | dbt Core
dbt Cloud CLI | All [supported versions](/docs/dbt-versions/core) |
+| [init](/reference/commands/init) | Initializes a new dbt project | dbt Core | All [supported versions](/docs/dbt-versions/core) |
+| [list](/reference/commands/list) | Lists resources defined in a dbt project | All | All [supported versions](/docs/dbt-versions/core) |
+| [parse](/reference/commands/parse) | Parses a project and writes detailed timing info | All | All [supported versions](/docs/dbt-versions/core) |
+| reattach | Reattaches to the most recent invocation to retrieve logs and artifacts. | dbt Cloud CLI | Requires [dbt v1.6 or higher](/docs/dbt-versions/core) |
+| [retry](/reference/commands/retry) | Retry the last run `dbt` command from the point of failure | All | Requires [dbt v1.6 or higher](/docs/dbt-versions/core) |
+| [run](/reference/commands/run) | Runs the models in a project | All | All [supported versions](/docs/dbt-versions/core) |
+| [run-operation](/reference/commands/run-operation) | Invoke a macro, including running arbitrary maintenance SQL against the database | All | All [supported versions](/docs/dbt-versions/core) |
+| [seed](/reference/commands/seed) | Loads CSV files into the database | All | All [supported versions](/docs/dbt-versions/core) |
+| [show](/reference/commands/show) | Preview table rows post-transformation | All | All [supported versions](/docs/dbt-versions/core) |
+| [snapshot](/reference/commands/snapshot) | Executes "snapshot" jobs defined in a project | All | All [supported versions](/docs/dbt-versions/core) |
+| [source](/reference/commands/source) | Provides tools for working with source data (including validating that sources are "fresh") | All | All [supported versions](/docs/dbt-versions/core) |
+| [test](/reference/commands/test) | Executes tests defined in a project | All | All [supported versions](/docs/dbt-versions/core) |
-
+
Select the tabs that are relevant to your development workflow. For example, if you develop in the dbt Cloud IDE, select **dbt Cloud**.
-
+
Use the following dbt commands in the [dbt Cloud IDE](/docs/cloud/dbt-cloud-ide/develop-in-the-cloud) and use the `dbt` prefix. For example, to run the `test` command, type `dbt test`.
- [build](/reference/commands/build): build and test all selected resources (models, seeds, snapshots, tests)
-- [clone](/reference/commands/clone): clone selected nodes from specified state (requires dbt 1.6 or higher)
+- [clone](/reference/commands/clone): clone selected nodes from the specified state (requires dbt 1.6 or higher)
- [compile](/reference/commands/compile): compiles (but does not run) the models in a project
- [deps](/reference/commands/deps): downloads dependencies for a project
- [docs](/reference/commands/cmd-docs) : generates documentation for a project
@@ -64,13 +69,13 @@ Use the following dbt commands in the [dbt Cloud IDE](/docs/cloud/dbt-cloud-ide/
-
+
-Use the following dbt commands in the [CLI](/docs/core/about-the-cli) and use the `dbt` prefix. For example, to run the `test` command, type `dbt test`.
+Use the following dbt commands in [dbt Core](/docs/core/about-dbt-core) and use the `dbt` prefix. For example, to run the `test` command, type `dbt test`.
- [build](/reference/commands/build): build and test all selected resources (models, seeds, snapshots, tests)
- [clean](/reference/commands/clean): deletes artifacts present in the dbt project
-- [clone](/reference/commands/clone): clone selected models from specified state (requires dbt 1.6 or higher)
+- [clone](/reference/commands/clone): clone selected models from the specified state (requires dbt 1.6 or higher)
- [compile](/reference/commands/compile): compiles (but does not run) the models in a project
- [debug](/reference/commands/debug): debugs dbt connections and projects
- [deps](/reference/commands/deps): downloads dependencies for a project
diff --git a/website/docs/reference/dbt-jinja-functions/model.md b/website/docs/reference/dbt-jinja-functions/model.md
index e967debd01f..9ccf0759470 100644
--- a/website/docs/reference/dbt-jinja-functions/model.md
+++ b/website/docs/reference/dbt-jinja-functions/model.md
@@ -52,15 +52,9 @@ To view the structure of `models` and their definitions:
Use the following table to understand how the versioning pattern works and match the Manifest version with the dbt version:
-| dbt version | Manifest version |
-| ----------- | ---------------- |
-| `v1.5` | [Manifest v9](https://schemas.getdbt.com/dbt/manifest/v9/index.html)
-| `v1.4` | [Manifest v8](https://schemas.getdbt.com/dbt/manifest/v8/index.html)
-| `v1.3` | [Manifest v7](https://schemas.getdbt.com/dbt/manifest/v7/index.html)
-| `v1.2` | [Manifest v6](https://schemas.getdbt.com/dbt/manifest/v6/index.html)
-| `v1.1` | [Manifest v5](https://schemas.getdbt.com/dbt/manifest/v5/index.html)
-
+import ManifestVersions from '/snippets/_manifest-versions.md';
+
## Related docs
diff --git a/website/docs/reference/dbt-jinja-functions/target.md b/website/docs/reference/dbt-jinja-functions/target.md
index 7d6627c5a4b..e7d08db592f 100644
--- a/website/docs/reference/dbt-jinja-functions/target.md
+++ b/website/docs/reference/dbt-jinja-functions/target.md
@@ -7,7 +7,7 @@ description: "Contains information about your connection to the warehouse."
`target` contains information about your connection to the warehouse.
-* **dbt CLI:** These values are based on the target defined in your [`profiles.yml` file](/docs/core/connect-data-platform/profiles.yml)
+* **dbt Core:** These values are based on the target defined in your [`profiles.yml` file](/docs/core/connect-data-platform/profiles.yml)
* **dbt Cloud Scheduler:**
* `target.name` is defined per job as described [here](/docs/build/custom-target-names).
* For all other attributes, the values are defined by the deployment connection. To check these values, click **Deploy** from the upper left and select **Environments**. Then, select the relevant deployment environment, and click **Settings**.
diff --git a/website/docs/reference/dbt_project.yml.md b/website/docs/reference/dbt_project.yml.md
index c706b57a73b..9bd85d0d5dd 100644
--- a/website/docs/reference/dbt_project.yml.md
+++ b/website/docs/reference/dbt_project.yml.md
@@ -11,6 +11,8 @@ By default, dbt will look for `dbt_project.yml` in your current working director
By default, dbt will look for `dbt_project.yml` in your current working directory and its parents, but you can set a different directory using the `--project-dir` flag or the `DBT_PROJECT_DIR` environment variable.
+Starting from dbt v1.5 and higher, you can specify your dbt Cloud project ID in the `dbt_project.yml` file using `project-id` under the `dbt-cloud` config. To find your project ID, check your dbt Cloud project URL, such as `https://cloud.getdbt.com/11/projects/123456`, where the project ID is `123456`.
+
The following is a list of all available configurations in the `dbt_project.yml` file.
@@ -19,6 +21,81 @@ The following is a list of all available configurations in the `dbt_project.yml`
dbt uses YAML in a few different places. If you're new to YAML, it would be worth taking the time to learn how arrays, dictionaries and strings are represented.
:::
+
+
+
+
+
+```yml
+[name](/reference/project-configs/name): string
+
+[config-version](/reference/project-configs/config-version): 2
+[version](/reference/project-configs/version): version
+
+[profile](/reference/project-configs/profile): profilename
+
+[model-paths](/reference/project-configs/model-paths): [directorypath]
+[seed-paths](/reference/project-configs/seed-paths): [directorypath]
+[test-paths](/reference/project-configs/test-paths): [directorypath]
+[analysis-paths](/reference/project-configs/analysis-paths): [directorypath]
+[macro-paths](/reference/project-configs/macro-paths): [directorypath]
+[snapshot-paths](/reference/project-configs/snapshot-paths): [directorypath]
+[docs-paths](/reference/project-configs/docs-paths): [directorypath]
+[asset-paths](/reference/project-configs/asset-paths): [directorypath]
+
+[target-path](/reference/project-configs/target-path): directorypath
+[log-path](/reference/project-configs/log-path): directorypath
+[packages-install-path](/reference/project-configs/packages-install-path): directorypath
+
+[clean-targets](/reference/project-configs/clean-targets): [directorypath]
+
+[query-comment](/reference/project-configs/query-comment): string
+
+[require-dbt-version](/reference/project-configs/require-dbt-version): version-range | [version-range]
+
+[dbt-cloud](/docs/cloud/cloud-cli-installation):
+ [project-id](/docs/cloud/configure-cloud-cli#configure-the-dbt-cloud-cli): project_id # Required
+ [defer-env-id](/docs/cloud/about-cloud-develop-defer#defer-in-dbt-cloud-cli): environment_id # Optional
+
+[quoting](/reference/project-configs/quoting):
+ database: true | false
+ schema: true | false
+ identifier: true | false
+
+models:
+ [](/reference/model-configs)
+
+seeds:
+ [](/reference/seed-configs)
+
+snapshots:
+ [](/reference/snapshot-configs)
+
+sources:
+ [](source-configs)
+
+tests:
+ [](/reference/test-configs)
+
+vars:
+ [](/docs/build/project-variables)
+
+[on-run-start](/reference/project-configs/on-run-start-on-run-end): sql-statement | [sql-statement]
+[on-run-end](/reference/project-configs/on-run-start-on-run-end): sql-statement | [sql-statement]
+
+[dispatch](/reference/project-configs/dispatch-config):
+ - macro_namespace: packagename
+ search_order: [packagename]
+
+[restrict-access](/docs/collaborate/govern/model-access): true | false
+
+```
+
+
+
+
+
+
```yml
@@ -79,6 +156,9 @@ vars:
search_order: [packagename]
[restrict-access](/docs/collaborate/govern/model-access): true | false
+
```
+
+
diff --git a/website/docs/reference/global-configs/about-global-configs.md b/website/docs/reference/global-configs/about-global-configs.md
index 42819cdac8f..9d1691812b5 100644
--- a/website/docs/reference/global-configs/about-global-configs.md
+++ b/website/docs/reference/global-configs/about-global-configs.md
@@ -8,4 +8,11 @@ Global configs enable you to fine-tune _how_ dbt runs projects on your machine
Global configs control things like the visual output of logs, the manner in which dbt parses your project, and what to do when dbt finds a version mismatch or a failing model. These configs are "global" because they are available for all dbt commands, and because they can be set for all projects running on the same machine or in the same environment.
-Starting in v1.0, you can set global configs in three places. When all three are set, command line flags take precedence, then environment variables, and last yaml configs (usually `profiles.yml`).
\ No newline at end of file
+### Global config precedence
+
+Starting in v1.0, you can set global configs in three places. dbt will evaluate the configs in the following order:
+1. [user config](https://docs.getdbt.com/reference/global-configs/yaml-configurations)
+1. [environment variable](https://docs.getdbt.com/reference/global-configs/environment-variable-configs)
+1. [CLI flag](https://docs.getdbt.com/reference/global-configs/command-line-flags)
+
+Each config is prioritized over the previous one. For example, if all three are provided, then the CLI flag takes precedence.
diff --git a/website/docs/reference/global-configs/command-line-flags.md b/website/docs/reference/global-configs/command-line-flags.md
index 6496c92da6d..fbe89ce28f1 100644
--- a/website/docs/reference/global-configs/command-line-flags.md
+++ b/website/docs/reference/global-configs/command-line-flags.md
@@ -4,60 +4,95 @@ id: "command-line-flags"
sidebar: "Command line flags"
---
-Command line (CLI) flags immediately follow `dbt` and precede your subcommand. When set, CLI flags override environment variables and profile configs.
+For consistency, command-line interface (CLI) flags should come right after the `dbt` prefix and its subcommands. This includes "global" flags (supported for all commands). When set, CLI flags override environment variables and profile configs.
-Use this non-boolean config structure, replacing `` with the config you are enabling or disabling, `` with the new setting for the config, and `` with the command this config applies to:
+For example, instead of using:
+
+```bash
+dbt --no-populate-cache run
+```
+
+You should use:
+
+```bash
+dbt run --no-populate-cache
+```
+
+Historically, passing flags (such as "global flags") _before_ the subcommand is a legacy functionality that dbt Labs can remove at any time. We do not support using the same flag before and after the subcommand.
+
+## Using boolean and non-boolean flags
+
+You can construct your commands with boolean flags to enable or disable or with non-boolean flags that use specific values, such as strings.
+
+
+
+
+
+Use this non-boolean config structure:
+- Replacing `` with the command this config applies to.
+- `` with the config you are enabling or disabling, and
+- `` with the new setting for the config.
```text
-$ --=
+ --=
```
-Non-boolean config examples:
+### Example
```text
-dbt --printer-width=80 run
-dbt --indirect-selection=eager test
+dbt run --printer-width=80
+dbt test --indirect-selection=eager
```
-To turn on boolean configs, you would use the `--` CLI flag, and a `--no-` CLI flag to turn off boolean configs, replacing `` with the config you are enabling or disabling and `` with the command this config applies to.
+
+
+
+
+To enable or disable boolean configs:
+- Use `` this config applies to.
+- Followed by `--` to turn it on, or `--no-` to turn it off.
+- Replace `` with the config you are enabling or disabling
-Boolean config structure:
```text
-dbt --
-dbt --no-
+dbt --
+dbt --no-
```
-Boolean config example:
+### Example
```text
-dbt --version-check run
-dbt --no-version-check run
+dbt run --version-check
+dbt run --no-version-check
```
-
\ No newline at end of file
+
+
+
+
+
diff --git a/website/docs/reference/model-properties.md b/website/docs/reference/model-properties.md
index 730432c88af..63adc1f0d63 100644
--- a/website/docs/reference/model-properties.md
+++ b/website/docs/reference/model-properties.md
@@ -18,7 +18,7 @@ models:
show: true | false
[latest_version](/reference/resource-properties/latest_version):
[deprecation_date](/reference/resource-properties/deprecation_date):
- [access](/reference/resource-properties/access): private | protected | public
+ [access](/reference/resource-configs/access): private | protected | public
[config](/reference/resource-properties/config):
[](/reference/model-configs):
[constraints](/reference/resource-properties/constraints):
@@ -46,7 +46,7 @@ models:
[description](/reference/resource-properties/description):
[docs](/reference/resource-configs/docs):
show: true | false
- [access](/reference/resource-properties/access): private | protected | public
+ [access](/reference/resource-configs/access): private | protected | public
[constraints](/reference/resource-properties/constraints):
-
[config](/reference/resource-properties/config):
diff --git a/website/docs/reference/node-selection/defer.md b/website/docs/reference/node-selection/defer.md
index e13a4f6648a..03c3b2aac12 100644
--- a/website/docs/reference/node-selection/defer.md
+++ b/website/docs/reference/node-selection/defer.md
@@ -17,16 +17,16 @@ It is possible to use separate state for `state:modified` and `--defer`, by pass
### Usage
```shell
-$ dbt run --select [...] --defer --state path/to/artifacts
-$ dbt test --select [...] --defer --state path/to/artifacts
+dbt run --select [...] --defer --state path/to/artifacts
+dbt test --select [...] --defer --state path/to/artifacts
```
```shell
-$ dbt run --models [...] --defer --state path/to/artifacts
-$ dbt test --models [...] --defer --state path/to/artifacts
+dbt run --models [...] --defer --state path/to/artifacts
+dbt test --models [...] --defer --state path/to/artifacts
```
@@ -101,7 +101,7 @@ I want to test my changes. Nothing exists in my development schema, `dev_alice`.
```shell
-$ dbt run --select model_b
+dbt run --select "model_b"
```
@@ -128,7 +128,7 @@ Unless I had previously run `model_a` into this development environment, `dev_al
```shell
-$ dbt run --select model_b --defer --state prod-run-artifacts
+dbt run --select "model_b" --defer --state prod-run-artifacts
```
@@ -186,7 +186,7 @@ models:
```shell
-dbt test --select model_b
+dbt test --select "model_b"
```
@@ -211,7 +211,7 @@ The `relationships` test requires both `model_a` and `model_b`. Because I did no
```shell
-dbt test --select model_b --defer --state prod-run-artifacts
+dbt test --select "model_b" --defer --state prod-run-artifacts
```
diff --git a/website/docs/reference/node-selection/exclude.md b/website/docs/reference/node-selection/exclude.md
index 9ad4bd1cc0e..d2c140d1bb5 100644
--- a/website/docs/reference/node-selection/exclude.md
+++ b/website/docs/reference/node-selection/exclude.md
@@ -7,19 +7,19 @@ sidebar_label: "Exclude"
dbt provides an `--exclude` flag with the same semantics as `--select`. Models specified with the `--exclude` flag will be removed from the set of models selected with `--select`.
```bash
-$ dbt run --select my_package.*+ --exclude my_package.a_big_model+ # select all models in my_package and their children except a_big_model and its children
+dbt run --select "my_package".*+ --exclude "my_package.a_big_model+" # select all models in my_package and their children except a_big_model and its children
```
Exclude a specific resource by its name or lineage:
```bash
# test
-$ dbt test --exclude not_null_orders_order_id # test all models except the not_null_orders_order_id test
-$ dbt test --exclude orders # test all models except tests associated with the orders model
+dbt test --exclude "not_null_orders_order_id" # test all models except the not_null_orders_order_id test
+dbt test --exclude "orders" # test all models except tests associated with the orders model
# seed
-$ dbt seed --exclude account_parent_mappings # load all seeds except account_parent_mappings
+dbt seed --exclude "account_parent_mappings" # load all seeds except account_parent_mappings
# snapshot
-$ dbt snapshot --exclude snap_order_statuses # execute all snapshots except snap_order_statuses
+dbt snapshot --exclude "snap_order_statuses" # execute all snapshots except snap_order_statuses
```
diff --git a/website/docs/reference/node-selection/graph-operators.md b/website/docs/reference/node-selection/graph-operators.md
index 4fdc2f10628..8cba43e1b52 100644
--- a/website/docs/reference/node-selection/graph-operators.md
+++ b/website/docs/reference/node-selection/graph-operators.md
@@ -7,9 +7,9 @@ If placed at the front of the model selector, `+` will select all parents of the
```bash
- $ dbt run --select my_model+ # select my_model and all children
- $ dbt run --select +my_model # select my_model and all parents
- $ dbt run --select +my_model+ # select my_model, and all of its parents and children
+dbt run --select "my_model+" # select my_model and all children
+dbt run --select "+my_model" # select my_model and all parents
+dbt run --select "+my_model+" # select my_model, and all of its parents and children
```
@@ -20,9 +20,9 @@ to step through.
```bash
- $ dbt run --select my_model+1 # select my_model and its first-degree children
- $ dbt run --select 2+my_model # select my_model, its first-degree parents, and its second-degree parents ("grandparents")
- $ dbt run --select 3+my_model+4 # select my_model, its parents up to the 3rd degree, and its children down to the 4th degree
+dbt run --select "my_model+1" # select my_model and its first-degree children
+dbt run --select "2+my_model" # select my_model, its first-degree parents, and its second-degree parents ("grandparents")
+dbt run --select "3+my_model+4" # select my_model, its parents up to the 3rd degree, and its children down to the 4th degree
```
@@ -32,5 +32,5 @@ The `@` operator is similar to `+`, but will also include _the parents of the ch
```bash
-$ dbt run --models @my_model # select my_model, its children, and the parents of its children
+dbt run --models @my_model # select my_model, its children, and the parents of its children
```
diff --git a/website/docs/reference/node-selection/methods.md b/website/docs/reference/node-selection/methods.md
index 2647f3416a3..e29612e3401 100644
--- a/website/docs/reference/node-selection/methods.md
+++ b/website/docs/reference/node-selection/methods.md
@@ -34,8 +34,8 @@ The `tag:` method is used to select models that match a specified [tag](/referen
```bash
- $ dbt run --select tag:nightly # run all models with the `nightly` tag
- ```
+dbt run --select "tag:nightly" # run all models with the `nightly` tag
+```
### The "source" method
@@ -43,22 +43,22 @@ The `source` method is used to select models that select from a specified [sourc
```bash
- $ dbt run --select source:snowplow+ # run all models that select from Snowplow sources
- ```
+dbt run --select "source:snowplow+" # run all models that select from Snowplow sources
+```
### The "resource_type" method
Use the `resource_type` method to select nodes of a particular type (`model`, `test`, `exposure`, and so on). This is similar to the `--resource-type` flag used by the [`dbt ls` command](/reference/commands/list).
```bash
- $ dbt build --select resource_type:exposure # build all resources upstream of exposures
- $ dbt list --select resource_type:test # list all tests in your project
- ```
+dbt build --select "resource_type:exposure" # build all resources upstream of exposures
+dbt list --select "resource_type:test" # list all tests in your project
+```
Note: This method doesn't work for sources, so use the [`--resource-type`](/reference/commands/list) option of the list command instead:
```bash
- $ dbt list --resource-type source
- ```
+dbt list --resource-type source
+```
### The "path" method
The `path` method is used to select models/sources defined at or under a specific path.
@@ -69,12 +69,12 @@ selectors unambiguous.
```bash
# These two selectors are equivalent
- dbt run --select path:models/staging/github
- dbt run --select models/staging/github
+ dbt run --select "path:models/staging/github"
+ dbt run --select "models/staging/github"
# These two selectors are equivalent
- dbt run --select path:models/staging/github/stg_issues.sql
- dbt run --select models/staging/github/stg_issues.sql
+ dbt run --select "path:models/staging/github/stg_issues.sql"
+ dbt run --select "models/staging/github/stg_issues.sql"
```
@@ -85,9 +85,9 @@ The `file` method can be used to select a model by its filename, including the f
```bash
# These are equivalent
-dbt run --select file:some_model.sql
-dbt run --select some_model.sql
-dbt run --select some_model
+dbt run --select "file:some_model.sql"
+dbt run --select "some_model.sql"
+dbt run --select "some_model"
```
@@ -96,10 +96,10 @@ dbt run --select some_model
The `fqn` method is used to select nodes based off their "fully qualified names" (FQN) within the dbt graph. The default output of [`dbt list`](/reference/commands/list) is a listing of FQN.
-```
-dbt run --select fqn:some_model
-dbt run --select fqn:your_project.some_model
-dbt run --select fqn:some_package.some_other_model
+```bash
+dbt run --select "fqn:some_model"
+dbt run --select "fqn:your_project.some_model"
+dbt run --select "fqn:some_package.some_other_model"
```
### The "package" method
@@ -111,10 +111,10 @@ selectors unambiguous.
```bash
# These three selectors are equivalent
- dbt run --select package:snowplow
- dbt run --select snowplow
- dbt run --select snowplow.*
- ```
+ dbt run --select "package:snowplow"
+ dbt run --select "snowplow"
+ dbt run --select "snowplow.*"
+```
### The "config" method
@@ -124,10 +124,10 @@ The `config` method is used to select models that match a specified [node config
```bash
- $ dbt run --select config.materialized:incremental # run all models that are materialized incrementally
- $ dbt run --select config.schema:audit # run all models that are created in the `audit` schema
- $ dbt run --select config.cluster_by:geo_country # run all models clustered by `geo_country`
- ```
+dbt run --select "config.materialized:incremental" # run all models that are materialized incrementally
+dbt run --select "config.schema:audit" # run all models that are created in the `audit` schema
+dbt run --select "config.cluster_by:geo_country" # run all models clustered by `geo_country`
+```
@@ -135,7 +135,8 @@ The `config` method is used to select models that match a specified [node config
While most config values are strings, you can also use the `config` method to match boolean configs, dictionary keys, and values in lists.
For example, given a model with the following configurations:
-```
+
+```bash
{{ config(
materialized = 'incremental',
unique_key = ['column_a', 'column_b'],
@@ -148,10 +149,10 @@ select ...
You can select using any of the following:
```bash
-$ dbt ls -s config.materialized:incremental
-$ dbt ls -s config.unique_key:column_a
-$ dbt ls -s config.grants.select:reporter
-$ dbt ls -s config.transient:true
+dbt ls -s config.materialized:incremental
+dbt ls -s config.unique_key:column_a
+dbt ls -s config.grants.select:reporter
+dbt ls -s config.transient:true
```
@@ -162,10 +163,10 @@ The `test_type` method is used to select tests based on their type, `singular` o
- ```bash
- $ dbt test --select test_type:generic # run all generic tests
- $ dbt test --select test_type:singular # run all singular tests
- ```
+```bash
+dbt test --select "test_type:generic" # run all generic tests
+dbt test --select "test_type:singular" # run all singular tests
+```
### The "test_name" method
@@ -176,10 +177,10 @@ that defines it. For more information about how generic tests are defined, read
```bash
- $ dbt test --select test_name:unique # run all instances of the `unique` test
- $ dbt test --select test_name:equality # run all instances of the `dbt_utils.equality` test
- $ dbt test --select test_name:range_min_max # run all instances of a custom schema test defined in the local project, `range_min_max`
- ```
+dbt test --select "test_name:unique" # run all instances of the `unique` test
+dbt test --select "test_name:equality" # run all instances of the `dbt_utils.equality` test
+dbt test --select "test_name:range_min_max" # run all instances of a custom schema test defined in the local project, `range_min_max`
+```
### The "state" method
@@ -204,9 +205,9 @@ The `state` method is used to select nodes by comparing them against a previous
```bash
- $ dbt test --select state:new # run all tests on new models + and new tests on old models
- $ dbt run --select state:modified # run all models that have been modified
- $ dbt ls --select state:modified # list all modified nodes (not just models)
+dbt test --select "state:new " # run all tests on new models + and new tests on old models
+dbt run --select "state:modified" # run all models that have been modified
+dbt ls --select "state:modified" # list all modified nodes (not just models)
```
@@ -236,18 +237,18 @@ The `exposure` method is used to select parent resources of a specified [exposur
```bash
- $ dbt run --select +exposure:weekly_kpis # run all models that feed into the weekly_kpis exposure
- $ dbt test --select +exposure:* # test all resources upstream of all exposures
- $ dbt ls --select +exposure:* --resource-type source # list all sources upstream of all exposures
- ```
+dbt run --select "+exposure:weekly_kpis" # run all models that feed into the weekly_kpis exposure
+dbt test --select "+exposure:*" # test all resources upstream of all exposures
+dbt ls --select "+exposure:*" --resource-type source # list all sources upstream of all exposures
+```
### The "metric" method
The `metric` method is used to select parent resources of a specified [metric](/docs/build/metrics). Use in conjunction with the `+` operator.
```bash
-$ dbt build --select +metric:weekly_active_users # build all resources upstream of weekly_active_users metric
-$ dbt ls --select +metric:* --resource-type source # list all source tables upstream of all metrics
+dbt build --select "+metric:weekly_active_users" # build all resources upstream of weekly_active_users metric
+dbt ls --select "+metric:*" --resource-type source # list all source tables upstream of all metrics
```
### The "result" method
@@ -255,10 +256,10 @@ $ dbt ls --select +metric:* --resource-type source # list all source tables
The `result` method is related to the `state` method described above and can be used to select resources based on their result status from a prior run. Note that one of the dbt commands [`run`, `test`, `build`, `seed`] must have been performed in order to create the result on which a result selector operates. You can use `result` selectors in conjunction with the `+` operator.
```bash
-$ dbt run --select result:error --state path/to/artifacts # run all models that generated errors on the prior invocation of dbt run
-$ dbt test --select result:fail --state path/to/artifacts # run all tests that failed on the prior invocation of dbt test
-$ dbt build --select 1+result:fail --state path/to/artifacts # run all the models associated with failed tests from the prior invocation of dbt build
-$ dbt seed --select result:error --state path/to/artifacts # run all seeds that generated errors on the prior invocation of dbt seed.
+dbt run --select "result:error" --state path/to/artifacts # run all models that generated errors on the prior invocation of dbt run
+dbt test --select "result:fail" --state path/to/artifacts # run all tests that failed on the prior invocation of dbt test
+dbt build --select "1+result:fail" --state path/to/artifacts # run all the models associated with failed tests from the prior invocation of dbt build
+dbt seed --select "result:error" --state path/to/artifacts # run all seeds that generated errors on the prior invocation of dbt seed.
```
### The "source_status" method
@@ -276,8 +277,8 @@ After issuing one of the above commands, you can reference the source freshness
```bash
# You can also set the DBT_ARTIFACT_STATE_PATH environment variable instead of the --state flag.
-$ dbt source freshness # must be run again to compare current to previous state
-$ dbt build --select source_status:fresher+ --state path/to/prod/artifacts
+dbt source freshness # must be run again to compare current to previous state
+dbt build --select "source_status:fresher+" --state path/to/prod/artifacts
```
@@ -286,8 +287,8 @@ $ dbt build --select source_status:fresher+ --state path/to/prod/artifacts
```bash
# You can also set the DBT_STATE environment variable instead of the --state flag.
-$ dbt source freshness # must be run again to compare current to previous state
-$ dbt build --select source_status:fresher+ --state path/to/prod/artifacts
+dbt source freshness # must be run again to compare current to previous state
+dbt build --select "source_status:fresher+" --state path/to/prod/artifacts
```
@@ -305,9 +306,9 @@ Supported in v1.5 or newer.
The `group` method is used to select models defined within a [group](/reference/resource-configs/group).
- ```bash
- dbt run --select group:finance # run all models that belong to the finance group.
- ```
+```bash
+dbt run --select "group:finance" # run all models that belong to the finance group.
+```
@@ -321,12 +322,12 @@ Supported in v1.5 or newer.
-The `access` method selects models based on their [access](/reference/resource-properties/access) property.
+The `access` method selects models based on their [access](/reference/resource-configs/access) property.
```bash
-dbt list --select access:public # list all public models
-dbt list --select access:private # list all private models
-dbt list --select access:protected # list all protected models
+dbt list --select "access:public" # list all public models
+dbt list --select "access:private" # list all private models
+dbt list --select "access:protected" # list all protected models
```
@@ -344,11 +345,26 @@ Supported in v1.5 or newer.
The `version` method selects [versioned models](/docs/collaborate/govern/model-versions) based on their [version identifier](/reference/resource-properties/versions) and [latest version](/reference/resource-properties/latest_version).
```bash
-dbt list --select version:latest # only 'latest' versions
-dbt list --select version:prerelease # versions newer than the 'latest' version
+dbt list --select "version:latest" # only 'latest' versions
+dbt list --select "version:prerelease" # versions newer than the 'latest' version
dbt list --select version:old # versions older than the 'latest' version
-dbt list --select version:none # models that are *not* versioned
+dbt list --select "version:none" # models that are *not* versioned
```
+
+### The "semantic_model" method
+
+Supported in v1.6 or newer.
+
+
+
+The `semantic_model` method selects [semantic models](/docs/build/semantic-models).
+
+```bash
+dbt list --select semantic_model:* # list all semantic models
+dbt list --select +semantic_model:orders # list your semantic model named "orders" and all upstream resources
+```
+
+
\ No newline at end of file
diff --git a/website/docs/reference/node-selection/putting-it-together.md b/website/docs/reference/node-selection/putting-it-together.md
index 8faf02e6cc9..48fc5188b32 100644
--- a/website/docs/reference/node-selection/putting-it-together.md
+++ b/website/docs/reference/node-selection/putting-it-together.md
@@ -4,16 +4,16 @@ title: "Putting it together"
```bash
- $ dbt run --select my_package.*+ # select all models in my_package and their children
- $ dbt run --select +some_model+ # select some_model and all parents and children
+dbt run --select "my_package.*+" # select all models in my_package and their children
+dbt run --select "+some_model+" # select some_model and all parents and children
- $ dbt run --select tag:nightly+ # select "nightly" models and all children
- $ dbt run --select +tag:nightly+ # select "nightly" models and all parents and children
+dbt run --select "tag:nightly+" # select "nightly" models and all children
+dbt run --select "+tag:nightly+" # select "nightly" models and all parents and children
- $ dbt run --select @source:snowplow # build all models that select from snowplow sources, plus their parents
+dbt run --select "@source:snowplow" # build all models that select from snowplow sources, plus their parents
- $ dbt test --select config.incremental_strategy:insert_overwrite,test_name:unique # execute all `unique` tests that select from models using the `insert_overwrite` incremental strategy
- ```
+dbt test --select "config.incremental_strategy:insert_overwrite,test_name:unique" # execute all `unique` tests that select from models using the `insert_overwrite` incremental strategy
+```
@@ -22,8 +22,8 @@ and feed exports, while _excluding_ the biggest incremental models (and one othe
```bash
- $ dbt run --select @source:snowplow,tag:nightly models/export --exclude package:snowplow,config.materialized:incremental export_performance_timing
- ```
+dbt run --select "@source:snowplow,tag:nightly models/export" --exclude "package:snowplow,config.materialized:incremental export_performance_timing"
+```
This command selects all models that:
diff --git a/website/docs/reference/node-selection/set-operators.md b/website/docs/reference/node-selection/set-operators.md
index 7d6b6c2411c..af399b9cad5 100644
--- a/website/docs/reference/node-selection/set-operators.md
+++ b/website/docs/reference/node-selection/set-operators.md
@@ -11,7 +11,7 @@ Run snowplow_sessions, all ancestors of snowplow_sessions, fct_orders, and all a
```bash
- $ dbt run --select +snowplow_sessions +fct_orders
+dbt run --select "+snowplow_sessions +fct_orders"
```
### Intersections
@@ -22,15 +22,15 @@ Run all the common ancestors of snowplow_sessions and fct_orders:
```bash
- $ dbt run --select +snowplow_sessions,+fct_orders
- ```
+dbt run --select "+snowplow_sessions,+fct_orders"
+```
Run all the common descendents of stg_invoices and stg_accounts:
```bash
- $ dbt run --select stg_invoices+,stg_accounts+
+dbt run --select "stg_invoices+,stg_accounts+"
```
@@ -38,5 +38,5 @@ Run models that are in the marts/finance subdirectory *and* tagged nightly:
```bash
- $ dbt run --select marts.finance,tag:nightly
- ```
+dbt run --select "marts.finance,tag:nightly"
+```
diff --git a/website/docs/reference/node-selection/state-comparison-caveats.md b/website/docs/reference/node-selection/state-comparison-caveats.md
index baeeb7e4c75..73947c80a66 100644
--- a/website/docs/reference/node-selection/state-comparison-caveats.md
+++ b/website/docs/reference/node-selection/state-comparison-caveats.md
@@ -27,8 +27,8 @@ The command `dbt test -s state:modified` will include both:
As long as you're adding or changing tests at the same time that you're adding or changing the resources (models, seeds, snapshots) they select from, all should work the way you expect with "simple" state selection:
```shell
-$ dbt run -s state:modified
-$ dbt test -s state:modified
+dbt run -s "state:modified"
+dbt test -s "state:modified"
```
This can get complicated, however. If you add a new test without modifying its underlying model, or add a test that selects from a new model and an old unmodified one, you may need to test a model without having first run it.
@@ -36,8 +36,8 @@ This can get complicated, however. If you add a new test without modifying its u
In v0.18.0, you needed to handle this by building the unmodified models needed for modified tests:
```shell
-$ dbt run -s state:modified @state:modified,1+test_type:data
-$ dbt test -s state:modified
+dbt run -s "state:modified @state:modified,1+test_type:data"
+dbt test -s "state:modified"
```
In v0.19.0, dbt added support for deferring upstream references when testing. If a test selects from a model that doesn't exist as a database object in your current environment, dbt will look to the other environment instead—the one defined in your state manifest. This enables you to use "simple" state selection without risk of query failure, but it may have some surprising consequences for tests with multiple parents. For instance, if you have a `relationships` test that depends on one modified model and one unmodified model, the test query will select from data "across" two different environments. If you limit or sample your data in development and CI, it may not make much sense to test for referential integrity, knowing there's a good chance of mismatch.
@@ -45,8 +45,8 @@ In v0.19.0, dbt added support for deferring upstream references when testing. If
If you're a frequent user of `relationships` tests or data tests, or frequently find yourself adding tests without modifying their underlying models, consider tweaking the selection criteria of your CI job. For instance:
```shell
-$ dbt run -s state:modified
-$ dbt test -s state:modified --exclude test_name:relationships
+dbt run -s "state:modified"
+dbt test -s "state:modified" --exclude "test_name:relationships"
```
### False positives
@@ -58,7 +58,7 @@ State comparison works by identifying discrepancies between two manifests. Thos
dbt will do its best to capture *only* changes that are the result of modifications made in development. In projects with intricate env-aware logic, dbt will err on the side of running too many models (i.e. false positives). Over the next several versions of dbt, we're working on:
- iterative improvements to dbt's built-in detective abilities
-- better options for more complex projects, in the form of more-specific subselectors (see [this issue](https://github.com/dbt-labs/dbt-core/issues/2704))
+- better options for more complex projects, in the form of more-specific sub-selectors (see [this issue](https://github.com/dbt-labs/dbt-core/issues/2704))
State comparison is now able to detect env-aware config in `dbt_project.yml`. For instance, this target-based config would register as a modification in v0.18.0, but in v0.19.0 it no longer will:
diff --git a/website/docs/reference/node-selection/syntax.md b/website/docs/reference/node-selection/syntax.md
index 7c165b0f4ff..bb2aeefd742 100644
--- a/website/docs/reference/node-selection/syntax.md
+++ b/website/docs/reference/node-selection/syntax.md
@@ -14,6 +14,7 @@ dbt's node selection syntax makes it possible to run only specific resources in
| [compile](/reference/commands/compile) | `--select`, `--exclude`, `--selector`, `--inline` |
| [freshness](/reference/commands/source) | `--select`, `--exclude`, `--selector` |
| [build](/reference/commands/build) | `--select`, `--exclude`, `--selector`, `--resource-type`, `--defer` |
+| [docs generate](/reference/commands/cmd-docs) | `--select`, `--exclude`, `--selector` |
:::info Nodes and resources
@@ -24,6 +25,8 @@ We use the terms
"
By default, `dbt run` executes _all_ of the models in the dependency graph; `dbt seed` creates all seeds, `dbt snapshot` performs every snapshot. The `--select` flag is used to specify a subset of nodes to execute.
+To follow [POSIX standards](https://pubs.opengroup.org/onlinepubs/9699919799/basedefs/V1_chap12.html) and make things easier to understand, we recommend CLI users use quotes when passing arguments to the `--select` or `--exclude` option (including single or multiple space-delimited, or comma-delimited arguments). Not using quotes might not work reliably on all operating systems, terminals, and user interfaces. For example, `dbt run --select "my_dbt_project_name"` runs all models in your project.
+
### How does selection work?
1. dbt gathers all the resources that are matched by one or more of the `--select` criteria, in the order of selection methods (e.g. `tag:`), then graph operators (e.g. `+`), then finally set operators ([unions](/reference/node-selection/set-operators#unions), [intersections](/reference/node-selection/set-operators#intersections), [exclusions](/reference/node-selection/exclude)).
@@ -51,28 +54,28 @@ Examples:
```bash
- $ dbt run --select my_dbt_project_name # runs all models in your project
- $ dbt run --select my_dbt_model # runs a specific model
- $ dbt run --select path.to.my.models # runs all models in a specific directory
- $ dbt run --select my_package.some_model # run a specific model in a specific package
- $ dbt run --select tag:nightly # run models with the "nightly" tag
- $ dbt run --select path/to/models # run models contained in path/to/models
- $ dbt run --select path/to/my_model.sql # run a specific model by its path
+dbt run --select "my_dbt_project_name" # runs all models in your project
+dbt run --select "my_dbt_model" # runs a specific model
+dbt run --select "path.to.my.models" # runs all models in a specific directory
+dbt run --select "my_package.some_model" # run a specific model in a specific package
+dbt run --select "tag:nightly" # run models with the "nightly" tag
+dbt run --select "path/to/models" # run models contained in path/to/models
+dbt run --select "path/to/my_model.sql" # run a specific model by its path
```
dbt supports a shorthand language for defining subsets of nodes. This language uses the characters `+`, `@`, `*`, and `,`.
```bash
- # multiple arguments can be provided to --select
- $ dbt run --select my_first_model my_second_model
+# multiple arguments can be provided to --select
+ dbt run --select "my_first_model my_second_model"
- # these arguments can be projects, models, directory paths, tags, or sources
- $ dbt run --select tag:nightly my_model finance.base.*
+# these arguments can be projects, models, directory paths, tags, or sources
+dbt run --select "tag:nightly my_model finance.base.*"
- # use methods and intersections for more complex selectors
- $ dbt run --select path:marts/finance,tag:nightly,config.materialized:table
- ```
+# use methods and intersections for more complex selectors
+dbt run --select "path:marts/finance,tag:nightly,config.materialized:table"
+```
As your selection logic gets more complex, and becomes unwieldly to type out as command-line arguments,
consider using a [yaml selector](/reference/node-selection/yaml-selectors). You can use a predefined definition with the `--selector` flag.
@@ -150,7 +153,7 @@ After issuing one of the above commands, you can reference the results by adding
```bash
# You can also set the DBT_ARTIFACT_STATE_PATH environment variable instead of the --state flag.
-$ dbt run --select result: --defer --state path/to/prod/artifacts
+dbt run --select "result: --defer --state path/to/prod/artifacts"
```
The available options depend on the resource (node) type:
@@ -169,7 +172,7 @@ The available options depend on the resource (node) type:
The state and result selectors can also be combined in a single invocation of dbt to capture errors from a previous run OR any new or modified models.
```bash
-$ dbt run --select result:+ state:modified+ --defer --state ./
+dbt run --select "result:+ state:modified+ --defer --state ./"
```
### Fresh rebuilds
@@ -183,7 +186,7 @@ As example:
```bash
# Command step order
dbt source freshness
-dbt build --select source_status:fresher+
+dbt build --select "source_status:fresher+"
```
@@ -202,6 +205,6 @@ After issuing one of the above commands, you can reference the source freshness
```bash
# You can also set the DBT_ARTIFACT_STATE_PATH environment variable instead of the --state flag.
-$ dbt source freshness # must be run again to compare current to previous state
-$ dbt build --select source_status:fresher+ --state path/to/prod/artifacts
+dbt source freshness # must be run again to compare current to previous state
+dbt build --select "source_status:fresher+" --state path/to/prod/artifacts
```
diff --git a/website/docs/reference/node-selection/test-selection-examples.md b/website/docs/reference/node-selection/test-selection-examples.md
index 52439d95d97..feb3898c230 100644
--- a/website/docs/reference/node-selection/test-selection-examples.md
+++ b/website/docs/reference/node-selection/test-selection-examples.md
@@ -19,14 +19,14 @@ Run generic tests only:
```bash
- $ dbt test --select test_type:generic
+ dbt test --select "test_type:generic"
```
Run singular tests only:
```bash
- $ dbt test --select test_type:singular
+ dbt test --select "test_type:singular"
```
In both cases, `test_type` checks a property of the test itself. These are forms of "direct" test selection.
@@ -87,8 +87,8 @@ By default, a test will run when ANY parent is selected; we call this "eager" in
In this mode, any test that depends on unbuilt resources will raise an error.
```shell
-$ dbt test --select orders
-$ dbt build --select orders
+dbt test --select "orders"
+dbt build --select "orders"
```
@@ -102,8 +102,10 @@ It will only include tests whose references are each within the selected nodes.
Put another way, it will prevent tests from running if one or more of its parents is unselected.
```shell
-$ dbt test --select orders --indirect-selection=cautious
-$ dbt build --select orders --indirect-selection=cautious
+
+dbt test --select "orders" --indirect-selection=cautious
+dbt build --select "orders" --indirect-selection=cautious
+
```
@@ -122,8 +124,8 @@ By default, a test will run when ANY parent is selected; we call this "eager" in
In this mode, any test that depends on unbuilt resources will raise an error.
```shell
-$ dbt test --select orders
-$ dbt build --select orders
+dbt test --select "orders"
+dbt build --select "orders"
```
@@ -137,8 +139,10 @@ It will only include tests whose references are each within the selected nodes.
Put another way, it will prevent tests from running if one or more of its parents is unselected.
```shell
-$ dbt test --select orders --indirect-selection=cautious
-$ dbt build --select orders --indirect-selection=cautious
+
+dbt test --select "orders" --indirect-selection=cautious
+dbt build --select "orders" --indirect-selection=cautious
+
```
@@ -152,8 +156,9 @@ It will only include tests whose references are each within the selected nodes (
This is useful in the same scenarios as "cautious", but also includes when a test depends on a model **and** a direct ancestor of that model (like confirming an aggregation has the same totals as its input).
```shell
-$ dbt test --select orders --indirect-selection=buildable
-$ dbt build --select orders --indirect-selection=buildable
+dbt test --select "orders" --indirect-selection=buildable
+dbt build --select "orders" --indirect-selection=buildable
+
```
@@ -172,8 +177,8 @@ By default, a test will run when ANY parent is selected; we call this "eager" in
In this mode, any test that depends on unbuilt resources will raise an error.
```shell
-$ dbt test --select orders
-$ dbt build --select orders
+dbt test --select "orders"
+dbt build --select "orders"
```
@@ -187,8 +192,9 @@ It will only include tests whose references are each within the selected nodes.
Put another way, it will prevent tests from running if one or more of its parents is unselected.
```shell
-$ dbt test --select orders --indirect-selection=cautious
-$ dbt build --select orders --indirect-selection=cautious
+dbt test --select "orders" --indirect-selection=cautious
+dbt build --select "orders" --indirect-selection=cautious
+
```
@@ -202,8 +208,8 @@ It will only include tests whose references are each within the selected nodes (
This is useful in the same scenarios as "cautious", but also includes when a test depends on a model **and** a direct ancestor of that model (like confirming an aggregation has the same totals as its input).
```shell
-$ dbt test --select orders --indirect-selection=buildable
-$ dbt build --select orders --indirect-selection=buildable
+dbt test --select "orders" --indirect-selection=buildable
+dbt build --select "orders" --indirect-selection=buildable
```
@@ -213,8 +219,10 @@ $ dbt build --select orders --indirect-selection=buildable
This mode will only include tests whose references are each within the selected nodes and will ignore all tests from attached nodes.
```shell
-$ dbt test --select orders --indirect-selection=empty
-$ dbt build --select orders --indirect-selection=empty
+
+dbt test --select "orders" --indirect-selection=empty
+dbt build --select "orders" --indirect-selection=empty
+
```
@@ -234,22 +242,25 @@ The following examples should feel somewhat familiar if you're used to executing
```bash
# Run tests on a model (indirect selection)
- $ dbt test --select customers
+ dbt test --select "customers"
+
+ # Run tests on two or more specific models (indirect selection)
+ dbt test --select "customers orders"
# Run tests on all models in the models/staging/jaffle_shop directory (indirect selection)
- $ dbt test --select staging.jaffle_shop
+ dbt test --select "staging.jaffle_shop"
# Run tests downstream of a model (note this will select those tests directly!)
- $ dbt test --select stg_customers+
+ dbt test --select "stg_customers+"
# Run tests upstream of a model (indirect selection)
- $ dbt test --select +stg_customers
+ dbt test --select "+stg_customers"
# Run tests on all models with a particular tag (direct + indirect)
- $ dbt test --select tag:my_model_tag
+ dbt test --select "tag:my_model_tag"
# Run tests on all models with a particular materialization (indirect selection)
- $ dbt test --select config.materialized:table
+ dbt test --select "config.materialized:table"
```
@@ -258,16 +269,20 @@ The following examples should feel somewhat familiar if you're used to executing
```bash
# tests on all sources
- $ dbt test --select source:*
+
+ dbt test --select "source:*"
# tests on one source
- $ dbt test --select source:jaffle_shop
+ dbt test --select "source:jaffle_shop"
+
+ # tests on two or more specific sources
+ dbt test --select "source:jaffle_shop source:raffle_bakery"
# tests on one source table
- $ dbt test --select source:jaffle_shop.customers
+ dbt test --select "source:jaffle_shop.customers"
# tests on everything _except_ sources
- $ dbt test --exclude source:*
+ dbt test --exclude "source:*"
```
### More complex selection
@@ -276,10 +291,12 @@ Through the combination of direct and indirect selection, there are many ways to
```bash
- $ dbt test --select assert_total_payment_amount_is_positive # directly select the test by name
- $ dbt test --select payments,test_type:singular # indirect selection, v1.2
- $ dbt test --select payments,test_type:data # indirect selection, v0.18.0
- $ dbt test --select payments --data # indirect selection, earlier versions
+
+ dbt test --select "assert_total_payment_amount_is_positive" # directly select the test by name
+ dbt test --select "payments,test_type:singular" # indirect selection, v1.2
+ dbt test --select "payments,test_type:data" # indirect selection, v0.18.0
+ dbt test --select "payments" --data # indirect selection, earlier versions
+
```
@@ -288,13 +305,14 @@ Through the combination of direct and indirect selection, there are many ways to
```bash
# Run tests on all models with a particular materialization
- $ dbt test --select config.materialized:table
+ dbt test --select "config.materialized:table"
# Run tests on all seeds, which use the 'seed' materialization
- $ dbt test --select config.materialized:seed
+ dbt test --select "config.materialized:seed"
# Run tests on all snapshots, which use the 'snapshot' materialization
- $ dbt test --select config.materialized:snapshot
+ dbt test --select "config.materialized:snapshot"
+
```
Note that this functionality may change in future versions of dbt.
@@ -312,8 +330,8 @@ models:
- name: orders
columns:
- name: order_id
- tests:
tags: [my_column_tag]
+ tests:
- unique
```
@@ -322,7 +340,8 @@ models:
```bash
- $ dbt test --select tag:my_column_tag
+ dbt test --select "tag:my_column_tag"
+
```
Currently, tests "inherit" tags applied to columns, sources, and source tables. They do _not_ inherit tags applied to models, seeds, or snapshots. In all likelihood, those tests would still be selected indirectly, because the tag selects its parent. This is a subtle distinction, and it may change in future versions of dbt.
@@ -350,5 +369,6 @@ models:
```bash
- $ dbt test --select tag:my_test_tag
+ dbt test --select "tag:my_test_tag"
+
```
diff --git a/website/docs/reference/node-selection/yaml-selectors.md b/website/docs/reference/node-selection/yaml-selectors.md
index 78342e32779..1e3f8d8d1e2 100644
--- a/website/docs/reference/node-selection/yaml-selectors.md
+++ b/website/docs/reference/node-selection/yaml-selectors.md
@@ -34,6 +34,7 @@ Each `definition` is comprised of one or more arguments, which can be one of the
Use the `union` and `intersection` operator-equivalent keywords to organize multiple arguments.
### CLI-style
+
```yml
definition:
'tag:nightly'
@@ -42,6 +43,7 @@ definition:
This simple syntax supports use of the `+`, `@`, and `*` [graph](/reference/node-selection/graph-operators) operators, but it does not support [set](/reference/node-selection/set-operators) operators or `exclude`.
### Key-value
+
```yml
definition:
tag: nightly
@@ -317,7 +319,7 @@ selectors:
Then in our job definition:
```bash
-$ dbt run --selector nightly_diet_snowplow
+dbt run --selector nightly_diet_snowplow
```
## Default
@@ -325,6 +327,7 @@ $ dbt run --selector nightly_diet_snowplow
Selectors may define a boolean `default` property. If a selector has `default: true`, dbt will use this selector's criteria when tasks do not define their own selection criteria.
Let's say we define a default selector that only selects resources defined in our root project:
+
```yml
selectors:
- name: root_project_only
@@ -338,16 +341,18 @@ selectors:
```
If I run an "unqualified" command, dbt will use the selection criteria defined in `root_project_only`—that is, dbt will only build / freshness check / generate compiled SQL for resources defined in my root project.
+
```
-$ dbt build
-$ dbt source freshness
-$ dbt docs generate
+dbt build
+dbt source freshness
+dbt docs generate
```
If I run a command that defines its own selection criteria (via `--select`, `--exclude`, or `--selector`), dbt will ignore the default selector and use the flag criteria instead. It will not try to combine the two.
-```
-$ dbt run --select model_a
-$ dbt run --exclude model_a
+
+```bash
+dbt run --select "model_a"
+dbt run --exclude model_a
```
Only one selector may set `default: true` for a given invocation; otherwise, dbt will return an error. You may use a Jinja expression to adjust the value of `default` depending on the environment, however:
diff --git a/website/docs/reference/programmatic-invocations.md b/website/docs/reference/programmatic-invocations.md
index 6afcd65c1bc..dfd5bae09e6 100644
--- a/website/docs/reference/programmatic-invocations.md
+++ b/website/docs/reference/programmatic-invocations.md
@@ -2,7 +2,7 @@
title: "Programmatic invocations"
---
-In v1.5, dbt-core added support for programmatic invocations. The intent is to expose the existing dbt CLI via a Python entry point, such that top-level commands are callable from within a Python script or application.
+In v1.5, dbt-core added support for programmatic invocations. The intent is to expose the existing dbt Core CLI via a Python entry point, such that top-level commands are callable from within a Python script or application.
The entry point is a `dbtRunner` class, which allows you to `invoke` the same commands as on the CLI.
diff --git a/website/docs/reference/references-overview.md b/website/docs/reference/references-overview.md
index 16afd01607c..91a228b6c3e 100644
--- a/website/docs/reference/references-overview.md
+++ b/website/docs/reference/references-overview.md
@@ -4,6 +4,8 @@ id: "references-overview"
sidebar_label: "About References"
description: "Connect dbt to any data platform in dbt Cloud or dbt Core, using a dedicated adapter plugin"
hide_table_of_contents: true
+pagination_next: null
+pagination_prev: null
---
The References section contains reference materials for developing with dbt, which includes dbt Cloud and dbt Core.
@@ -49,9 +51,27 @@ Learn how to add more configurations to your dbt project or adapter, use propert
icon="computer"/>
+
+
+
+
+
+
diff --git a/website/docs/reference/resource-configs/access.md b/website/docs/reference/resource-configs/access.md
new file mode 100644
index 00000000000..da50e48d2f0
--- /dev/null
+++ b/website/docs/reference/resource-configs/access.md
@@ -0,0 +1,97 @@
+---
+resource_types: [models]
+datatype: access
+---
+
+
+
+```yml
+version: 2
+
+models:
+ - name: model_name
+ access: private | protected | public
+```
+
+
+
+
+
+Access modifiers may be applied to models one-by-one in YAML properties. In v1.5 and v1.6, you are unable to configure `access` for multiple models at once. Upgrade to v1.7 for additional configuration options. A group or subfolder contains models with varying access levels, so when you designate a model with `access: public`, make sure you intend for this behavior.
+
+
+
+
+
+You can apply access modifiers in config files, including `the dbt_project.yml`, or to models one-by-one in YAML properties. Applying access configs to a subfolder modifies the default for all models in that subfolder, so make sure you intend for this behavior. When setting individual model access, a group or subfolder might contain a variety of access levels, so when you designate a model with `access: public` make sure you intend for this behavior.
+
+There are multiple approaches to configuring access:
+
+In the model configs of `dbt_project.yml``:
+
+```yaml
+models:
+ - name: my_public_model
+ access: public # Older method, still supported
+
+```
+Or (but not both)
+
+```yaml
+models:
+ - name: my_public_model
+ config:
+ access: public # newly supported in v1.7
+
+```
+
+In a subfolder:
+```yaml
+models:
+ my_project_name:
+ subfolder_name:
+ +group:
+ +access: private # sets default for all models in this subfolder
+```
+
+In the model.sql file:
+
+```sql
+-- models/my_public_model.sql
+
+{{ config(access = "public") }}
+
+select ...
+```
+
+
+
+## Definition
+The access level of the model you are declaring properties for.
+
+Some models (not all) are designed to be referenced through the [ref](/reference/dbt-jinja-functions/ref) function across [groups](/docs/build/groups).
+
+| Access | Referenceable by |
+|-----------|-------------------------------|
+| private | same group |
+| protected | same project/package |
+| public | any group, package or project |
+
+If you try to reference a model outside of its supported access, you will see an error:
+
+```shell
+dbt run -s marketing_model
+...
+dbt.exceptions.DbtReferenceError: Parsing Error
+ Node model.jaffle_shop.marketing_model attempted to reference node model.jaffle_shop.finance_model,
+ which is not allowed because the referenced node is private to the finance group.
+```
+
+## Default
+
+By default, all models are "protected." This means that other models in the same project can reference them.
+
+## Related docs
+
+* [Model Access](/docs/collaborate/govern/model-access#groups)
+* [Group configuration](/reference/resource-configs/group)
diff --git a/website/docs/reference/resource-configs/bigquery-configs.md b/website/docs/reference/resource-configs/bigquery-configs.md
index 89a750f47bd..ffbaa37c059 100644
--- a/website/docs/reference/resource-configs/bigquery-configs.md
+++ b/website/docs/reference/resource-configs/bigquery-configs.md
@@ -414,7 +414,7 @@ models:
columns:
- name: field
policy_tags:
- - 'projects//locations//taxonomies//policyTags/'
+ - 'projects//locations//taxonomies//policyTags/'
```
diff --git a/website/docs/reference/resource-configs/contract.md b/website/docs/reference/resource-configs/contract.md
index e8ea6d82287..59cc511890b 100644
--- a/website/docs/reference/resource-configs/contract.md
+++ b/website/docs/reference/resource-configs/contract.md
@@ -23,11 +23,34 @@ When the `contract` configuration is enforced, dbt will ensure that your model's
This is to ensure that the people querying your model downstream—both inside and outside dbt—have a predictable and consistent set of columns to use in their analyses. Even a subtle change in data type, such as from `boolean` (`true`/`false`) to `integer` (`0`/`1`), could cause queries to fail in surprising ways.
+
+
The `data_type` defined in your YAML file must match a data type your data platform recognizes. dbt does not do any type aliasing itself. If your data platform recognizes both `int` and `integer` as corresponding to the same type, then they will return a match.
-When dbt is comparing data types, it will not compare granular details such as size, precision, or scale. We don't think you should sweat the difference between `varchar(256)` and `varchar(257)`, because it doesn't really affect the experience of downstream queriers. If you need a more-precise assertion, it's always possible to accomplish by [writing or using a custom test](/guides/best-practices/writing-custom-generic-tests).
+
+
+
+
+dbt uses built-in type aliasing for the `data_type` defined in your YAML. For example, you can specify `string` in your contract, and on Postgres/Redshift, dbt will convert it to `text`. If dbt doesn't recognize the `data_type` name among its known aliases, it will pass it through as-is. This is enabled by default, but you can opt-out by setting `alias_types` to `false`.
+
+Example for disabling:
+
+```yml
+
+models:
+ - name: my_model
+ config:
+ contract:
+ enforced: true
+ alias_types: false # true by default
+
+```
+
+
+
+When dbt compares data types, it will not compare granular details such as size, precision, or scale. We don't think you should sweat the difference between `varchar(256)` and `varchar(257)`, because it doesn't really affect the experience of downstream queriers. You can accomplish a more-precise assertion by [writing or using a custom test](/guides/best-practices/writing-custom-generic-tests).
-That said, on certain data platforms, you will need to specify a varchar size or numeric scale if you do not want it to revert to the default. This is most relevant for the `numeric` type on Snowflake, which defaults to a precision of 38 and a scale of 0 (zero digits after the decimal, such as rounded to an integer). To avoid this implicit coercion, specify your `data_type` with a nonzero scale, like `numeric(38, 6)`.
+Note that you need to specify a varchar size or numeric scale, otherwise dbt relies on default values. For example, if a `numeric` type defaults to a precision of 38 and a scale of 0, then the numeric column stores 0 digits to the right of the decimal (it only stores whole numbers), which might cause it to fail contract enforcement. To avoid this implicit coercion, specify your `data_type` with a nonzero scale, like `numeric(38, 6)`. dbt Core 1.7 and higher provides a warning if you don't specify precision and scale when providing a numeric data type.
## Example
@@ -47,6 +70,8 @@ models:
- type: not_null
- name: customer_name
data_type: string
+ - name: non_integer
+ data_type: numeric(38,3)
```
diff --git a/website/docs/reference/resource-configs/delimiter.md b/website/docs/reference/resource-configs/delimiter.md
new file mode 100644
index 00000000000..58d6ba8344a
--- /dev/null
+++ b/website/docs/reference/resource-configs/delimiter.md
@@ -0,0 +1,126 @@
+---
+resource_types: [seeds]
+datatype:
+default_value: ","
+---
+
+## Definition
+
+You can use this optional seed configuration to customize how you separate values in a [seed](/docs/build/seeds) by providing the one-character string.
+
+* The delimiter defaults to a comma when not specified.
+* Explicitly set the `delimiter` configuration value if you want seed files to use a different delimiter, such as "|" or ";".
+
+:::info New in 1.7!
+
+Delimiter is new functionality available beginning with dbt Core v1.7.
+
+:::
+
+
+## Usage
+
+Specify a delimiter in your `dbt_project.yml` file to customize the global separator for all seed values:
+
+
+
+```yml
+seeds:
+ :
+ +delimiter: "|" # default project delimiter for seeds will be "|"
+ :
+ +delimiter: "," # delimiter for seeds in seed_subdirectory will be ","
+```
+
+
+
+
+Or use a custom delimiter to override the values for a specific seed:
+
+
+
+```yml
+version: 2
+
+seeds:
+ - name:
+ config:
+ delimiter: "|"
+```
+
+
+
+## Examples
+For a project with:
+
+* `name: jaffle_shop` in the `dbt_project.yml` file
+* `seed-paths: ["seeds"]` in the `dbt_project.yml` file
+
+### Use a custom delimiter to override global values
+
+You can set a default behavior for all seeds with an exception for one seed, `seed_a`, which uses a comma:
+
+
+
+```yml
+seeds:
+ jaffle_shop:
+ +delimiter: "|" # default delimiter for seeds in jaffle_shop project will be "|"
+ seed_a:
+ +delimiter: "," # delimiter for seed_a will be ","
+```
+
+
+
+Your corresponding seed files would be formatted like this:
+
+
+
+```text
+col_a|col_b|col_c
+1|2|3
+4|5|6
+...
+```
+
+
+
+
+
+```text
+name,id
+luna,1
+doug,2
+...
+```
+
+
+
+Or you can configure custom behavior for one seed. The `country_codes` uses the ";" delimiter:
+
+
+
+```yml
+version: 2
+
+seeds:
+ - name: country_codes
+ config:
+ delimiter: ";"
+```
+
+
+
+The `country_codes` seed file would be formatted like this:
+
+
+
+```text
+country_code;country_name
+US;United States
+CA;Canada
+GB;United Kingdom
+...
+```
+
+
diff --git a/website/docs/reference/resource-configs/enabled.md b/website/docs/reference/resource-configs/enabled.md
index b6d0961ee60..52045503088 100644
--- a/website/docs/reference/resource-configs/enabled.md
+++ b/website/docs/reference/resource-configs/enabled.md
@@ -15,6 +15,7 @@ default_value: true
{ label: 'Sources', value: 'sources', },
{ label: 'Metrics', value: 'metrics', },
{ label: 'Exposures', value: 'exposures', },
+ { label: 'Semantic models', value: 'semantic models', },
]
}>
@@ -250,10 +251,39 @@ exposures:
+
+
+
+
+Support for disabling semantic models has been added in dbt Core v1.7
+
+
+
+
+
+
+
+```yml
+semantic_models:
+ - name: semantic_people
+ model: ref('people')
+ config:
+ enabled: false
+
+```
+
+
+
+The `enabled` configuration can be nested under the `config` key.
+
+
+
+
+
## Definition
-An optional configuration for disabling models, seeds, snapshots, and tests.
+An optional configuration for disabling models, seeds, snapshots, tests, and semantic models.
* Default: true
diff --git a/website/docs/reference/resource-configs/group.md b/website/docs/reference/resource-configs/group.md
index dd73d99edff..7515d8c5789 100644
--- a/website/docs/reference/resource-configs/group.md
+++ b/website/docs/reference/resource-configs/group.md
@@ -16,6 +16,7 @@ This functionality is new in v1.5.
{ label: 'Tests', value: 'tests', },
{ label: 'Analyses', value: 'analyses', },
{ label: 'Metrics', value: 'metrics', },
+ { label: 'Semantic models', value: 'semantic models', },
]
}>
@@ -265,6 +266,43 @@ metrics:
+
+
+
+
+Support for grouping semantic models has been added in dbt Core v1.7.
+
+
+
+
+
+
+
+```yml
+semantic_models:
+ - name: model_name
+ group: finance
+
+```
+
+
+
+
+
+```yml
+semantic_models:
+ [](resource-path):
+ +group: finance
+```
+
+
+
+The `group` configuration can be nested under the `config` key.
+
+
+
+
+
## Definition
diff --git a/website/docs/reference/resource-configs/meta.md b/website/docs/reference/resource-configs/meta.md
index d24c5fbaee1..65c8b5f908e 100644
--- a/website/docs/reference/resource-configs/meta.md
+++ b/website/docs/reference/resource-configs/meta.md
@@ -14,6 +14,8 @@ default_value: {}
{ label: 'Tests', value: 'tests', },
{ label: 'Analyses', value: 'analyses', },
{ label: 'Macros', value: 'macros', },
+ { label: 'Exposures', value: 'exposures', },
+ { label: 'Semantic Models', value: 'semantic models', },
]
}>
@@ -172,6 +174,34 @@ exposures:
+
+
+
+
+Support for grouping semantic models was added in dbt Core v1.7
+
+
+
+
+
+
+
+```yml
+semantic_models:
+ - name: semantic_people
+ model: ref('people')
+ config:
+ meta: {}
+
+```
+The `meta` configuration can be nusted under the `config` key.
+
+
+
+
+
+
+
## Definition
@@ -248,3 +278,19 @@ select 1 as id
```
+
+### Assign owner in the dbt_project.yml as a config property
+
+
+
+```yml
+models:
+ jaffle_shop:
+ materialized: table
+ config:
+ meta:
+ owner: "@alice"
+```
+
+
+
diff --git a/website/docs/reference/resource-configs/starrocks-configs.md b/website/docs/reference/resource-configs/starrocks-configs.md
new file mode 100644
index 00000000000..093534515c6
--- /dev/null
+++ b/website/docs/reference/resource-configs/starrocks-configs.md
@@ -0,0 +1,116 @@
+---
+title: "Starrocks configurations"
+id: "starrocks-configs"
+description: "Starrocks Configurations - Read this in-depth guide to learn about configurations in dbt."
+---
+
+## Model Configuration
+
+A dbt model can be configured using the following syntax:
+
+
+
+
+
+
+
+```yaml
+models:
+ :
+ materialized: table // table or view or materialized_view
+ keys: ['id', 'name', 'some_date']
+ table_type: 'PRIMARY' // PRIMARY or DUPLICATE or UNIQUE
+ distributed_by: ['id']
+ buckets: 3 // default 10
+ partition_by: ['some_date']
+ partition_by_init: ["PARTITION p1 VALUES [('1971-01-01 00:00:00'), ('1991-01-01 00:00:00')),PARTITION p1972 VALUES [('1991-01-01 00:00:00'), ('1999-01-01 00:00:00'))"]
+ properties: [{"replication_num":"1", "in_memory": "true"}]
+ refresh_method: 'async' // only for materialized view default manual
+```
+
+
+
+
+
+
+
+```yaml
+models:
+ - name:
+ config:
+ materialized: table // table or view or materialized_view
+ keys: ['id', 'name', 'some_date']
+ table_type: 'PRIMARY' // PRIMARY or DUPLICATE or UNIQUE
+ distributed_by: ['id']
+ buckets: 3 // default 10
+ partition_by: ['some_date']
+ partition_by_init: ["PARTITION p1 VALUES [('1971-01-01 00:00:00'), ('1991-01-01 00:00:00')),PARTITION p1972 VALUES [('1991-01-01 00:00:00'), ('1999-01-01 00:00:00'))"]
+ properties: [{"replication_num":"1", "in_memory": "true"}]
+ refresh_method: 'async' // only for materialized view default manual
+```
+
+
+
+
+
+
+
+```jinja
+{{ config(
+ materialized = 'table',
+ keys=['id', 'name', 'some_date'],
+ table_type='PRIMARY',
+ distributed_by=['id'],
+ buckets=3,
+ partition_by=['some_date'],
+ ....
+) }}
+```
+
+
+
+
+### Configuration Description
+
+| Option | Description |
+|---------------------|----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------|
+| `materialized` | How the model will be materialized into Starrocks. Supports view, table, incremental, ephemeral, and materialized_view. |
+| `keys` | Which columns serve as keys. |
+| `table_type` | Table type, supported are PRIMARY or DUPLICATE or UNIQUE. |
+| `distributed_by` | Specifies the column of data distribution. If not specified, it defaults to random. |
+| `buckets` | The bucket number in one partition. If not specified, it will be automatically inferred. |
+| `partition_by` | The partition column list. |
+| `partition_by_init` | The partition rule or some real partitions item. |
+| `properties` | The table properties configuration of Starrocks. ([Starrocks table properties](https://docs.starrocks.io/en-us/latest/sql-reference/sql-statements/data-definition/CREATE_TABLE#properties)) |
+| `refresh_method` | How to refresh materialized views. |
+
+## Read From Catalog
+First you need to add this catalog to starrocks. The following is an example of hive.
+
+```sql
+CREATE EXTERNAL CATALOG `hive_catalog`
+PROPERTIES (
+ "hive.metastore.uris" = "thrift://127.0.0.1:8087",
+ "type"="hive"
+);
+```
+How to add other types of catalogs can be found in the documentation. [Catalog Overview](https://docs.starrocks.io/en-us/latest/data_source/catalog/catalog_overview) Then write the sources.yaml file.
+```yaml
+sources:
+ - name: external_example
+ schema: hive_catalog.hive_db
+ tables:
+ - name: hive_table_name
+```
+Finally, you might use below marco quote
+```jinja
+{{ source('external_example', 'hive_table_name') }}
+```
\ No newline at end of file
diff --git a/website/docs/reference/resource-configs/store_failures.md b/website/docs/reference/resource-configs/store_failures.md
index 3c965179211..6c71cdb9296 100644
--- a/website/docs/reference/resource-configs/store_failures.md
+++ b/website/docs/reference/resource-configs/store_failures.md
@@ -3,7 +3,7 @@ resource_types: [tests]
datatype: boolean
---
-The configured test(s) will store their failures when `dbt test --store-failures` is invoked.
+The configured test(s) will store their failures when `dbt test --store-failures` is invoked. If you set this configuration as `false` but [`store_failures_as`](/reference/resource-configs/store_failures_as) is configured, it will be overriden.
## Description
Optionally set a test to always or never store its failures in the database.
diff --git a/website/docs/reference/resource-configs/store_failures_as.md b/website/docs/reference/resource-configs/store_failures_as.md
new file mode 100644
index 00000000000..a9149360089
--- /dev/null
+++ b/website/docs/reference/resource-configs/store_failures_as.md
@@ -0,0 +1,76 @@
+---
+resource_types: [tests]
+id: "store_failures_as"
+---
+
+For the `test` resource type, `store_failures_as` is an optional config that specifies how test failures should be stored in the database. If [`store_failures`](/reference/resource-configs/store_failures) is also configured, `store_failures_as` takes precedence.
+
+The three supported values are:
+
+- `ephemeral` — nothing stored in the database (default)
+- `table` — test failures stored as a database table
+- `view` — test failures stored as a database view
+
+You can configure it in all the same places as `store_failures`, including singular tests (.sql files), generic tests (.yml files), and dbt_project.yml.
+
+### Examples
+
+#### Singular test
+
+[Singular test](https://docs.getdbt.com/docs/build/tests#singular-tests) in `tests/singular/check_something.sql` file
+
+```sql
+{{ config(store_failures_as="table") }}
+
+-- custom singular test
+select 1 as id
+where 1=0
+```
+
+#### Generic test
+
+[Generic tests](https://docs.getdbt.com/docs/build/tests#generic-tests) in `models/_models.yml` file
+
+```yaml
+models:
+ - name: my_model
+ columns:
+ - name: id
+ tests:
+ - not_null:
+ config:
+ store_failures_as: view
+ - unique:
+ config:
+ store_failures_as: ephemeral
+```
+
+#### Project level
+
+Config in `dbt_project.yml`
+
+```yaml
+name: "my_project"
+version: "1.0.0"
+config-version: 2
+profile: "sandcastle"
+
+tests:
+ my_project:
+ +store_failures_as: table
+ my_subfolder_1:
+ +store_failures_as: view
+ my_subfolder_2:
+ +store_failures_as: ephemeral
+```
+
+### "Clobbering" configs
+
+As with most other configurations, `store_failures_as` is "clobbered" when applied hierarchically. Whenever a more specific value is available, it will completely replace the less specific value.
+
+Additional resources:
+
+- [Test configurations](/reference/test-configs#related-documentation)
+- [Test-specific configurations](/reference/test-configs#test-specific-configurations)
+- [Configuring directories of models in dbt_project.yml](/reference/model-configs#configuring-directories-of-models-in-dbt_projectyml)
+- [Config inheritance](/reference/configs-and-properties#config-inheritance)
\ No newline at end of file
diff --git a/website/docs/reference/resource-configs/teradata-configs.md b/website/docs/reference/resource-configs/teradata-configs.md
index f0f4f1a6f3e..12a8929429d 100644
--- a/website/docs/reference/resource-configs/teradata-configs.md
+++ b/website/docs/reference/resource-configs/teradata-configs.md
@@ -35,14 +35,21 @@ id: "teradata-configs"
###
* `table_kind` - define the table kind. Legal values are `MULTISET` (default for ANSI transaction mode required by `dbt-teradata`) and `SET`, e.g.:
- ```yaml
- {{
- config(
- materialized="table",
- table_kind="SET"
- )
- }}
- ```
+ * in sql materialization definition file:
+ ```yaml
+ {{
+ config(
+ materialized="table",
+ table_kind="SET"
+ )
+ }}
+ ```
+ * in seed configuration:
+ ```yaml
+ seeds:
+ :
+ table_kind: "SET"
+ ```
For details, see [CREATE TABLE documentation](https://docs.teradata.com/r/76g1CuvvQlYBjb2WPIuk3g/B6Js16DRQVwPDjgJ8rz7hg).
* `table_option` - defines table options. The config supports multiple statements. The definition below uses the Teradata syntax definition to explain what statements are allowed. Square brackets `[]` denote optional parameters. The pipe symbol `|` separates statements. Use commas to combine multiple statements as shown in the examples below:
```
@@ -87,37 +94,51 @@ id: "teradata-configs"
```
Examples:
-
- :::info Separators between statements
- Note the commas that separate statements in `table_option` config.
- :::
-
- ```yaml
- {{
- config(
- materialized="table",
- table_option="NO FALLBACK"
- )
- }}
- ```
- ```yaml
- {{
- config(
- materialized="table",
- table_option="NO FALLBACK, NO JOURNAL"
- )
- }}
- ```
- ```yaml
- {{
- config(
- materialized="table",
- table_option="NO FALLBACK, NO JOURNAL, CHECKSUM = ON,
- NO MERGEBLOCKRATIO,
- WITH CONCURRENT ISOLATED LOADING FOR ALL"
- )
- }}
- ```
+ * in sql materialization definition file:
+ ```yaml
+ {{
+ config(
+ materialized="table",
+ table_option="NO FALLBACK"
+ )
+ }}
+ ```
+ ```yaml
+ {{
+ config(
+ materialized="table",
+ table_option="NO FALLBACK, NO JOURNAL"
+ )
+ }}
+ ```
+ ```yaml
+ {{
+ config(
+ materialized="table",
+ table_option="NO FALLBACK, NO JOURNAL, CHECKSUM = ON,
+ NO MERGEBLOCKRATIO,
+ WITH CONCURRENT ISOLATED LOADING FOR ALL"
+ )
+ }}
+ ```
+ * in seed configuration:
+ ```yaml
+ seeds:
+ :
+ table_option:"NO FALLBACK"
+ ```
+ ```yaml
+ seeds:
+ :
+ table_option:"NO FALLBACK, NO JOURNAL"
+ ```
+ ```yaml
+ seeds:
+ :
+ table_option: "NO FALLBACK, NO JOURNAL, CHECKSUM = ON,
+ NO MERGEBLOCKRATIO,
+ WITH CONCURRENT ISOLATED LOADING FOR ALL"
+ ```
For details, see [CREATE TABLE documentation](https://docs.teradata.com/r/76g1CuvvQlYBjb2WPIuk3g/B6Js16DRQVwPDjgJ8rz7hg).
@@ -160,46 +181,67 @@ id: "teradata-configs"
```
Examples:
-
- :::info Separators between statements
- Note, unlike with `table_option` statements, there are no commas between statements in `index` config.
- :::
-
- ```yaml
- {{
- config(
- materialized="table",
- index="UNIQUE PRIMARY INDEX ( GlobalID )"
- )
- }}
- ```
-
- ```yaml
- {{
- config(
- materialized="table",
- index="PRIMARY INDEX(id)
- PARTITION BY RANGE_N(create_date
- BETWEEN DATE '2020-01-01'
- AND DATE '2021-01-01'
- EACH INTERVAL '1' MONTH)"
- )
- }}
- ```
-
- ```yaml
- {{
- config(
- materialized="table",
- index="PRIMARY INDEX(id)
- PARTITION BY RANGE_N(create_date
- BETWEEN DATE '2020-01-01'
- AND DATE '2021-01-01'
- EACH INTERVAL '1' MONTH)
- INDEX index_attrA (attrA) WITH LOAD IDENTITY"
- )
- }}
- ```
+ * in sql materialization definition file:
+ ```yaml
+ {{
+ config(
+ materialized="table",
+ index="UNIQUE PRIMARY INDEX ( GlobalID )"
+ )
+ }}
+ ```
+ > :information_source: Note, unlike in `table_option`, there are no commas between index statements!
+ ```yaml
+ {{
+ config(
+ materialized="table",
+ index="PRIMARY INDEX(id)
+ PARTITION BY RANGE_N(create_date
+ BETWEEN DATE '2020-01-01'
+ AND DATE '2021-01-01'
+ EACH INTERVAL '1' MONTH)"
+ )
+ }}
+ ```
+ ```yaml
+ {{
+ config(
+ materialized="table",
+ index="PRIMARY INDEX(id)
+ PARTITION BY RANGE_N(create_date
+ BETWEEN DATE '2020-01-01'
+ AND DATE '2021-01-01'
+ EACH INTERVAL '1' MONTH)
+ INDEX index_attrA (attrA) WITH LOAD IDENTITY"
+ )
+ }}
+ ```
+ * in seed configuration:
+ ```yaml
+ seeds:
+ :
+ index: "UNIQUE PRIMARY INDEX ( GlobalID )"
+ ```
+ > :information_source: Note, unlike in `table_option`, there are no commas between index statements!
+ ```yaml
+ seeds:
+ :
+ index: "PRIMARY INDEX(id)
+ PARTITION BY RANGE_N(create_date
+ BETWEEN DATE '2020-01-01'
+ AND DATE '2021-01-01'
+ EACH INTERVAL '1' MONTH)"
+ ```
+ ```yaml
+ seeds:
+ :
+ index: "PRIMARY INDEX(id)
+ PARTITION BY RANGE_N(create_date
+ BETWEEN DATE '2020-01-01'
+ AND DATE '2021-01-01'
+ EACH INTERVAL '1' MONTH)
+ INDEX index_attrA (attrA) WITH LOAD IDENTITY"
+ ```
## Seeds
:::info Using seeds to load raw data
@@ -220,6 +262,35 @@ Loading CSVs using dbt's seed functionality is not performant for large files. C
+use_fastload: true
```
+#### Grants
+
+Grants are supported in dbt-teradata adapter with release version 1.2.0 and above. You can use grants to manage access to the datasets you're producing with dbt. To implement these permissions, define grants as resource configs on each model, seed, or snapshot. Define the default grants that apply to the entire project in your `dbt_project.yml`, and define model-specific grants within each model's SQL or YAML file.
+
+for e.g. :
+ models/schema.yml
+ ```yaml
+ models:
+ - name: model_name
+ config:
+ grants:
+ select: ['user_a', 'user_b']
+ ```
+
+Another e.g. for adding multiple grants:
+
+ ```yaml
+ models:
+ - name: model_name
+ config:
+ materialized: table
+ grants:
+ select: ["user_b"]
+ insert: ["user_c"]
+ ```
+> :information_source: `copy_grants` is not supported in Teradata.
+
+More on Grants can be found at https://docs.getdbt.com/reference/resource-configs/grants
+
## Common Teradata-specific tasks
* *collect statistics* - when a table is created or modified significantly, there might be a need to tell Teradata to collect statistics for the optimizer. It can be done using `COLLECT STATISTICS` command. You can perform this step using dbt's `post-hooks`, e.g.:
diff --git a/website/docs/reference/resource-properties/access.md b/website/docs/reference/resource-properties/access.md
deleted file mode 100644
index 42b9893ed7f..00000000000
--- a/website/docs/reference/resource-properties/access.md
+++ /dev/null
@@ -1,53 +0,0 @@
----
-resource_types: [models]
-datatype: access
-required: no
----
-
-:::info New functionality
-This functionality is new in v1.5.
-:::
-
-
-
-```yml
-version: 2
-
-models:
- - name: model_name
- access: private | protected | public
-```
-
-
-
-Access modifiers may be applied to models one-by-one in YAML properties. It is not currently possible to configure `access` for multiple models at once. A group or subfolder contains models with a variety of access levels, and designating a model with `access: public` should always be a conscious and intentional choice.
-
-## Definition
-The access level of the model you are declaring properties for.
-
-Some models (not all) are designed to be referenced through the [ref](/reference/dbt-jinja-functions/ref) function across [groups](/docs/build/groups).
-
-| Access | Referenceable by |
-|-----------|-------------------------------|
-| private | same group |
-| protected | same project/package |
-| public | any group, package or project |
-
-If you try to reference a model outside of its supported access, you will see an error:
-
-```shell
-dbt run -s marketing_model
-...
-dbt.exceptions.DbtReferenceError: Parsing Error
- Node model.jaffle_shop.marketing_model attempted to reference node model.jaffle_shop.finance_model,
- which is not allowed because the referenced node is private to the finance group.
-```
-
-## Default
-
-By default, all models are "protected." This means that other models in the same project can reference them.
-
-## Related docs
-
-* [Model Access](/docs/collaborate/govern/model-access#groups)
-* [Group configuration](/reference/resource-configs/group)
diff --git a/website/docs/reference/seed-configs.md b/website/docs/reference/seed-configs.md
index d74f414cbfe..429aa9444ae 100644
--- a/website/docs/reference/seed-configs.md
+++ b/website/docs/reference/seed-configs.md
@@ -23,6 +23,7 @@ seeds:
[](/reference/resource-configs/resource-path):
[+](/reference/resource-configs/plus-prefix)[quote_columns](/reference/resource-configs/quote_columns): true | false
[+](/reference/resource-configs/plus-prefix)[column_types](/reference/resource-configs/column_types): {column_name: datatype}
+ [+](/reference/resource-configs/plus-prefix)[delimiter](/reference/resource-configs/delimiter):
```
@@ -43,6 +44,7 @@ seeds:
config:
[quote_columns](/reference/resource-configs/quote_columns): true | false
[column_types](/reference/resource-configs/column_types): {column_name: datatype}
+ [delimiter](/reference/resource-configs/grants):
```
diff --git a/website/docs/reference/snowflake-permissions.md b/website/docs/reference/snowflake-permissions.md
deleted file mode 100644
index 6a469d12230..00000000000
--- a/website/docs/reference/snowflake-permissions.md
+++ /dev/null
@@ -1,25 +0,0 @@
----
-title: "Snowflake Permissions"
----
-
-## Example Snowflake permissions
-
-```
--- NOTE: warehouse_name, database_name, and role_name are placeholders!
--- Replace as-needed for your organization's naming convention!
-
-grant all on warehouse warehouse_name to role role_name;
-grant usage on database database_name to role role_name;
-grant create schema on database database_name to role role_name;
-grant usage on schema database.an_existing_schema to role role_name;
-grant create table on schema database.an_existing_schema to role role_name;
-grant create view on schema database.an_existing_schema to role role_name;
-grant usage on future schemas in database database_name to role role_name;
-grant monitor on future schemas in database database_name to role role_name;
-grant select on future tables in database database_name to role role_name;
-grant select on future views in database database_name to role role_name;
-grant usage on all schemas in database database_name to role role_name;
-grant monitor on all schemas in database database_name to role role_name;
-grant select on all tables in database database_name to role role_name;
-grant select on all views in database database_name to role role_name;
-```
diff --git a/website/docs/terms/materialization.md b/website/docs/terms/materialization.md
index fdeaaebfcc8..328076f1483 100644
--- a/website/docs/terms/materialization.md
+++ b/website/docs/terms/materialization.md
@@ -11,7 +11,7 @@ hoverSnippet: The exact Data Definition Language (DDL) that dbt will use when cr
:::important This page could use some love
-This term would benefit from additional depth and examples. Have knowledge to contribute? [Create a discussion in the docs.getdbt.com GitHub repository](https://github.com/dbt-labs/docs.getdbt.com/discussions) to begin the process of becoming a glossary contributor!
+This term would benefit from additional depth and examples. Have knowledge to contribute? [Create an issue in the docs.getdbt.com repository](https://github.com/dbt-labs/docs.getdbt.com/issues/new/choose) to begin the process of becoming a glossary contributor!
:::
The exact Data Definition Language (DDL) that dbt will use when creating the model’s equivalent in a . It's the manner in which the data is represented, and each of those options is defined either canonically (tables, views, incremental), or bespoke.
diff --git a/website/docs/terms/model.md b/website/docs/terms/model.md
new file mode 100644
index 00000000000..c589cc196a7
--- /dev/null
+++ b/website/docs/terms/model.md
@@ -0,0 +1,9 @@
+---
+id: model
+title: Model
+description: A model is an essential building block of the DAG
+displayText: model
+hoverSnippet: A model is an essential building block of the DAG
+---
+
+A model is an essential building block of the DAG that lives in a single file and contains logic that transforms data. This logic can be expressed as a SQL `select` statement or a Python dataframe operation. Models can be materialized in the warehouse in different ways — most of these materializations require models to be built in the warehouse.
\ No newline at end of file
diff --git a/website/docs/terms/table.md b/website/docs/terms/table.md
index 69fc2b3e6b6..cbe36ec1315 100644
--- a/website/docs/terms/table.md
+++ b/website/docs/terms/table.md
@@ -6,7 +6,7 @@ displayText: table
hoverSnippet: In simplest terms, a table is the direct storage of data in rows and columns. Think excel sheet with raw values in each of the cells.
---
:::important This page could use some love
-This term would benefit from additional depth and examples. Have knowledge to contribute? [Create a discussion in the docs.getdbt.com GitHub repository](https://github.com/dbt-labs/docs.getdbt.com/discussions) to begin the process of becoming a glossary contributor!
+This term would benefit from additional depth and examples. Have knowledge to contribute? [Create an issue in the docs.getdbt.com repository](https://github.com/dbt-labs/docs.getdbt.com/issues/new/choose) to begin the process of becoming a glossary contributor!
:::
In simplest terms, a table is the direct storage of data in rows and columns. Think excel sheet with raw values in each of the cells.
diff --git a/website/docs/terms/view.md b/website/docs/terms/view.md
index 5d9238256e0..90cd5d1f36f 100644
--- a/website/docs/terms/view.md
+++ b/website/docs/terms/view.md
@@ -6,7 +6,7 @@ displayText: view
hoverSnippet: A view (as opposed to a table) is a defined passthrough SQL query that can be run against a database (or data warehouse).
---
:::important This page could use some love
-This term would benefit from additional depth and examples. Have knowledge to contribute? [Create a discussion in the docs.getdbt.com GitHub repository](https://github.com/dbt-labs/docs.getdbt.com/discussions) to begin the process of becoming a glossary contributor!
+This term would benefit from additional depth and examples. Have knowledge to contribute? [Create an issue in the docs.getdbt.com repository](https://github.com/dbt-labs/docs.getdbt.com/issues/new/choose) to begin the process of becoming a glossary contributor!
:::
A view (as opposed to a ) is a defined passthrough SQL query that can be run against a database (or ). A view doesn’t store data, like a table does, but it defines the logic that you need to fetch the underlying data.
diff --git a/website/docusaurus.config.js b/website/docusaurus.config.js
index 0eae62ecec3..ce81d614c65 100644
--- a/website/docusaurus.config.js
+++ b/website/docusaurus.config.js
@@ -71,13 +71,13 @@ var siteSettings = {
announcementBar: {
id: "biweekly-demos",
content:
- "Register now for Coalesce 2023. The Analytics Engineering Conference!",
- backgroundColor: "#7444FD",
+ "Join our weekly demos and dbt Cloud in action!",
+ backgroundColor: "#047377",
textColor: "#fff",
isCloseable: true,
},
announcementBarActive: true,
- announcementBarLink: "https://coalesce.getdbt.com/",
+ announcementBarLink: "https://www.getdbt.com/resources/dbt-cloud-demos-with-experts?utm_source=docs&utm_medium=event&utm_campaign=q1-2024_cloud-demos-with-experts_awareness",
// Set community spotlight member on homepage
// This is the ID for a specific file under docs/community/spotlight
communitySpotlightMember: "faith-lierheimer",
diff --git a/website/sidebars.js b/website/sidebars.js
index 8b162f67af3..055dfed0e7d 100644
--- a/website/sidebars.js
+++ b/website/sidebars.js
@@ -7,6 +7,7 @@ const sidebarSettings = {
collapsed: true,
link: { type: "doc", id: "docs/supported-data-platforms" },
items: [
+ "docs/supported-data-platforms",
"docs/connect-adapters",
"docs/verified-adapters",
"docs/trusted-adapters",
@@ -17,12 +18,12 @@ const sidebarSettings = {
{
type: "category",
label: "About dbt Cloud",
+ link: { type: "doc", id: "docs/cloud/about-cloud/dbt-cloud-features" },
items: [
"docs/cloud/about-cloud/dbt-cloud-features",
"docs/cloud/about-cloud/architecture",
"docs/cloud/about-cloud/tenancy",
"docs/cloud/about-cloud/regions-ip-addresses",
- "docs/cloud/about-cloud/about-cloud-ide",
"docs/cloud/about-cloud/browsers",
],
}, // About dbt Cloud directory
@@ -35,6 +36,7 @@ const sidebarSettings = {
type: "category",
label: "Set up dbt",
collapsed: true,
+ link: { type: "doc", id: "docs/about-setup" },
items: [
"docs/about-setup",
"docs/environments-in-dbt",
@@ -42,12 +44,14 @@ const sidebarSettings = {
type: "category",
label: "dbt Cloud",
collapsed: true,
+ link: { type: "doc", id: "docs/cloud/about-cloud-setup" },
items: [
"docs/cloud/about-cloud-setup",
"docs/dbt-cloud-environments",
{
type: "category",
label: "Connect data platform",
+ link: { type: "doc", id: "docs/cloud/connect-data-platform/about-connections" },
items: [
"docs/cloud/connect-data-platform/about-connections",
"docs/cloud/connect-data-platform/connect-starburst-trino",
@@ -61,13 +65,15 @@ const sidebarSettings = {
{
type: "category",
label: "Manage access",
+ link: { type: "doc", id: "docs/cloud/manage-access/about-user-access" },
items: [
"docs/cloud/manage-access/about-user-access",
- "docs/cloud/manage-access/seats-and-users",
{
type: "category",
- label: "Permissions",
+ label: "User permissions and licenses",
+ link: { type: "doc", id: "docs/cloud/manage-access/seats-and-users" },
items: [
+ "docs/cloud/manage-access/seats-and-users",
"docs/cloud/manage-access/self-service-permissions",
"docs/cloud/manage-access/enterprise-permissions",
],
@@ -75,7 +81,8 @@ const sidebarSettings = {
{
type: "category",
- label: "Single sign-on",
+ label: "Single sign-on and Oauth",
+ link: { type: "doc", id: "docs/cloud/manage-access/sso-overview" },
items: [
"docs/cloud/manage-access/sso-overview",
"docs/cloud/manage-access/auth0-migration",
@@ -83,16 +90,11 @@ const sidebarSettings = {
"docs/cloud/manage-access/set-up-sso-okta",
"docs/cloud/manage-access/set-up-sso-google-workspace",
"docs/cloud/manage-access/set-up-sso-azure-active-directory",
- ],
- }, // SSO
- {
- type: "category",
- label: "OAuth with data platforms",
- items: [
"docs/cloud/manage-access/set-up-snowflake-oauth",
+ "docs/cloud/manage-access/set-up-databricks-oauth",
"docs/cloud/manage-access/set-up-bigquery-oauth",
],
- }, // oauth
+ }, // SSO
"docs/cloud/manage-access/audit-log",
],
}, // Manage access
@@ -100,38 +102,60 @@ const sidebarSettings = {
{
type: "category",
label: "Configure Git",
+ link: { type: "doc", id: "docs/cloud/git/git-configuration-in-dbt-cloud" },
items: [
+ "docs/cloud/git/git-configuration-in-dbt-cloud",
+ "docs/cloud/git/import-a-project-by-git-url",
"docs/cloud/git/connect-github",
"docs/cloud/git/connect-gitlab",
{
type: "category",
label: "Azure DevOps",
+ link: { type: "doc", id: "docs/cloud/git/connect-azure-devops" },
items: [
"docs/cloud/git/connect-azure-devops",
"docs/cloud/git/setup-azure",
"docs/cloud/git/authenticate-azure",
],
},
- "docs/cloud/git/import-a-project-by-git-url",
],
}, // Supported Git providers
{
type: "category",
- label: "Develop in the IDE",
- link: {
- type: "doc",
- id: "docs/cloud/dbt-cloud-ide/develop-in-the-cloud",
- },
+ label: "Develop in dbt Cloud",
+ link: { type: "doc", id: "docs/cloud/about-cloud-develop" },
items: [
- "docs/cloud/dbt-cloud-ide/ide-user-interface",
- "docs/cloud/dbt-cloud-ide/lint-format",
- "docs/cloud/dbt-cloud-ide/dbt-cloud-tips",
+ "docs/cloud/about-cloud-develop",
+ "docs/cloud/about-cloud-develop-defer",
+ {
+ type: "category",
+ label: "dbt Cloud CLI",
+ link: { type: "doc", id: "docs/cloud/cloud-cli-installation" },
+ items: [
+ "docs/cloud/cloud-cli-installation",
+ "docs/cloud/configure-cloud-cli",
+ ],
+ },
+ {
+ type: "category",
+ label: "dbt Cloud IDE",
+ link: { type: "doc", id: "docs/cloud/dbt-cloud-ide/develop-in-the-cloud" },
+ items: [
+ "docs/cloud/dbt-cloud-ide/develop-in-the-cloud",
+ "docs/cloud/dbt-cloud-ide/ide-user-interface",
+ "docs/cloud/dbt-cloud-ide/lint-format",
+ "docs/cloud/dbt-cloud-ide/dbt-cloud-tips",
+ ],
+ },
],
- }, // dbt Cloud IDE directory
+ }, // dbt Cloud develop directory
{
type: "category",
label: "Secure your tenant",
+ link: { type: "doc", id: "docs/cloud/secure/secure-your-tenant" },
items: [
+ "docs/cloud/secure/secure-your-tenant",
+ "docs/cloud/secure/ip-restrictions",
"docs/cloud/secure/about-privatelink",
"docs/cloud/secure/snowflake-privatelink",
"docs/cloud/secure/databricks-privatelink",
@@ -149,13 +173,15 @@ const sidebarSettings = {
collapsed: true,
link: { type: "doc", id: "docs/core/about-core-setup" },
items: [
- "docs/core/about-the-cli",
+ "docs/core/about-core-setup",
+ "docs/core/about-dbt-core",
"docs/core/dbt-core-environments",
{
type: "category",
label: "Install dbt",
link: { type: "doc", id: "docs/core/installation" },
items: [
+ "docs/core/installation",
"docs/core/homebrew-install",
"docs/core/pip-install",
"docs/core/docker-install",
@@ -170,6 +196,7 @@ const sidebarSettings = {
id: "docs/core/connect-data-platform/about-core-connections",
},
items: [
+ "docs/core/connect-data-platform/about-core-connections",
"docs/core/connect-data-platform/profiles.yml",
"docs/core/connect-data-platform/connection-profiles",
"docs/core/connect-data-platform/bigquery-setup",
@@ -212,6 +239,7 @@ const sidebarSettings = {
"docs/core/connect-data-platform/fal-setup",
"docs/core/connect-data-platform/decodable-setup",
"docs/core/connect-data-platform/upsolver-setup",
+ "docs/core/connect-data-platform/starrocks-setup",
],
},
],
@@ -224,16 +252,19 @@ const sidebarSettings = {
type: "category",
label: "Build dbt projects",
collapsed: true,
+ link: { type: "doc", id: "docs/build/projects" },
items: [
"docs/build/projects",
{
type: "category",
label: "Build your DAG",
collapsed: true,
+ link: { type: "doc", id: "docs/build/models" },
items: [
{
type: "category",
label: "Models",
+ link: { type: "doc", id: "docs/build/models" },
items: [
"docs/build/models",
"docs/build/sql-models",
@@ -257,38 +288,43 @@ const sidebarSettings = {
link: { type: "doc", id: "docs/build/build-metrics-intro" },
collapsed: true,
items: [
+ "docs/build/build-metrics-intro",
"docs/build/sl-getting-started",
{
type: "category",
label: "About MetricFlow",
link: { type: "doc", id: "docs/build/about-metricflow" },
items: [
+ "docs/build/about-metricflow",
"docs/build/join-logic",
"docs/build/validation",
+ "docs/build/saved-queries",
"docs/build/metricflow-time-spine",
- "docs/build/metricflow-cli",
- ]
+ "docs/build/metricflow-commands",
+ ],
},
{
type: "category",
label: "Semantic models",
link: { type: "doc", id: "docs/build/semantic-models" },
items: [
+ "docs/build/semantic-models",
"docs/build/dimensions",
"docs/build/entities",
- "docs/build/measures"
- ]
+ "docs/build/measures",
+ ],
},
{
type: "category",
label: "Metrics",
link: { type: "doc", id: "docs/build/metrics-overview" },
items: [
+ "docs/build/metrics-overview",
"docs/build/cumulative",
"docs/build/derived",
"docs/build/ratio",
"docs/build/simple",
- ]
+ ],
},
],
},
@@ -296,7 +332,9 @@ const sidebarSettings = {
type: "category",
label: "Enhance your models",
collapsed: true,
+ link: { type: "doc", id: "docs/build/enhance-your-models" },
items: [
+ "docs/build/enhance-your-models",
"docs/build/materializations",
"docs/build/incremental-models",
],
@@ -305,7 +343,9 @@ const sidebarSettings = {
type: "category",
label: "Enhance your code",
collapsed: true,
+ link: { type: "doc", id: "docs/build/enhance-your-code" },
items: [
+ "docs/build/enhance-your-code",
"docs/build/project-variables",
"docs/build/environment-variables",
"docs/build/packages",
@@ -316,7 +356,9 @@ const sidebarSettings = {
type: "category",
label: "Organize your outputs",
collapsed: true,
+ link: { type: "doc", id: "docs/build/organize-your-outputs" },
items: [
+ "docs/build/organize-your-outputs",
"docs/build/custom-schemas",
"docs/build/custom-databases",
"docs/build/custom-aliases",
@@ -333,6 +375,7 @@ const sidebarSettings = {
collapsed: true,
link: { type: "doc", id: "docs/deploy/deployments" },
items: [
+ "docs/deploy/deployments",
"docs/deploy/job-scheduler",
"docs/deploy/deploy-environments",
"docs/deploy/continuous-integration",
@@ -341,6 +384,7 @@ const sidebarSettings = {
label: "Jobs",
link: { type: "doc", id: "docs/deploy/jobs" },
items: [
+ "docs/deploy/jobs",
"docs/deploy/deploy-jobs",
"docs/deploy/ci-jobs",
"docs/deploy/job-commands",
@@ -351,7 +395,9 @@ const sidebarSettings = {
label: "Monitor jobs and alerts",
link: { type: "doc", id: "docs/deploy/monitor-jobs" },
items: [
+ "docs/deploy/monitor-jobs",
"docs/deploy/run-visibility",
+ "docs/deploy/retry-jobs",
"docs/deploy/job-notifications",
"docs/deploy/webhooks",
"docs/deploy/artifacts",
@@ -365,11 +411,14 @@ const sidebarSettings = {
{
type: "category",
label: "Collaborate with others",
+ link: { type: "doc", id: "docs/collaborate/collaborate-with-others" },
items: [
+ "docs/collaborate/collaborate-with-others",
"docs/collaborate/explore-projects",
{
type: "category",
label: "Git version control",
+ link: { type: "doc", id: "docs/collaborate/git-version-control" },
items: [
"docs/collaborate/git-version-control",
"docs/collaborate/git/version-control-basics",
@@ -381,6 +430,7 @@ const sidebarSettings = {
{
type: "category",
label: "Document your dbt projects",
+ link: { type: "doc", id: "docs/collaborate/documentation" },
items: [
"docs/collaborate/documentation",
"docs/collaborate/build-and-view-your-docs",
@@ -395,6 +445,7 @@ const sidebarSettings = {
id: "docs/collaborate/govern/about-model-governance",
},
items: [
+ "docs/collaborate/govern/about-model-governance",
"docs/collaborate/govern/model-access",
"docs/collaborate/govern/model-contracts",
"docs/collaborate/govern/model-versions",
@@ -406,24 +457,38 @@ const sidebarSettings = {
{
type: "category",
label: "Use the dbt Semantic Layer",
+ collapsed: true,
link: { type: "doc", id: "docs/use-dbt-semantic-layer/dbt-sl" },
items: [
+ "docs/use-dbt-semantic-layer/dbt-sl",
"docs/use-dbt-semantic-layer/quickstart-sl",
"docs/use-dbt-semantic-layer/setup-sl",
- "docs/use-dbt-semantic-layer/avail-sl-integrations",
"docs/use-dbt-semantic-layer/sl-architecture",
+ {
+ type: "category",
+ label: "Integrations",
+ link: { type: "doc", id: "docs/use-dbt-semantic-layer/avail-sl-integrations" },
+ items: [
+ "docs/use-dbt-semantic-layer/avail-sl-integrations",
+ "docs/use-dbt-semantic-layer/gsheets",
+ "docs/use-dbt-semantic-layer/tableau",
+ ],
+ },
],
},
{
type: "category",
label: "dbt Cloud APIs",
collapsed: true,
+ link: { type: "doc", id: "docs/dbt-cloud-apis/overview" },
items: [
"docs/dbt-cloud-apis/overview",
{
type: "category",
label: "Authentication",
+ link: { type: "doc", id: "docs/dbt-cloud-apis/authentication" },
items: [
+ "docs/dbt-cloud-apis/authentication",
"docs/dbt-cloud-apis/user-tokens",
"docs/dbt-cloud-apis/service-tokens",
],
@@ -433,6 +498,7 @@ const sidebarSettings = {
label: "Administrative API",
link: { type: "doc", id: "docs/dbt-cloud-apis/admin-cloud-api" },
items: [
+ "docs/dbt-cloud-apis/admin-cloud-api",
{
type: "link",
label: "API v2 (legacy docs)",
@@ -455,18 +521,25 @@ const sidebarSettings = {
label: "Discovery API",
link: { type: "doc", id: "docs/dbt-cloud-apis/discovery-api" },
items: [
+ "docs/dbt-cloud-apis/discovery-api",
"docs/dbt-cloud-apis/discovery-use-cases-and-examples",
"docs/dbt-cloud-apis/project-state",
"docs/dbt-cloud-apis/discovery-querying",
{
type: "category",
label: "Schema",
+ link: { type: "doc", id: "docs/dbt-cloud-apis/discovery-schema-environment" },
items: [
+ "docs/dbt-cloud-apis/discovery-schema-environment",
{
type: "category",
label: "Job",
- link: { type: "doc", id: "docs/dbt-cloud-apis/discovery-schema-job" },
+ link: {
+ type: "doc",
+ id: "docs/dbt-cloud-apis/discovery-schema-job",
+ },
items: [
+ "docs/dbt-cloud-apis/discovery-schema-job",
"docs/dbt-cloud-apis/discovery-schema-job-model",
"docs/dbt-cloud-apis/discovery-schema-job-models",
"docs/dbt-cloud-apis/discovery-schema-job-metric",
@@ -486,11 +559,6 @@ const sidebarSettings = {
],
},
{
- type: "category",
- label: "Environment",
- link: { type: "doc", id: "docs/dbt-cloud-apis/discovery-schema-environment" },
- items: [
- {
type: "category",
label: "Applied",
items: [
@@ -504,9 +572,7 @@ const sidebarSettings = {
// items: [
// // insert pages here
// ],
- // },
- ],
- },
+ // },
],
},
],
@@ -516,6 +582,7 @@ const sidebarSettings = {
label: "Semantic Layer APIs",
link: { type: "doc", id: "docs/dbt-cloud-apis/sl-api-overview" },
items: [
+ "docs/dbt-cloud-apis/sl-api-overview",
"docs/dbt-cloud-apis/sl-jdbc",
"docs/dbt-cloud-apis/sl-graphql",
"docs/dbt-cloud-apis/sl-manifest",
@@ -526,14 +593,33 @@ const sidebarSettings = {
{
type: "category",
label: "Available dbt versions",
+ link: { type: "doc", id: "docs/dbt-versions/core" },
items: [
"docs/dbt-versions/core",
"docs/dbt-versions/upgrade-core-in-cloud",
"docs/dbt-versions/product-lifecycles",
"docs/dbt-versions/experimental-features",
+ {
+ type: "category",
+ label: "dbt Core upgrade guides",
+ link: {
+ type: "generated-index",
+ title: "Version upgrade guides",
+ description:
+ "Learn what's new in the latest version of dbt Core.",
+ slug: "/docs/dbt-versions/core-upgrade",
+ },
+ items: [
+ {
+ type: "autogenerated",
+ dirName: "docs/dbt-versions/core-upgrade",
+ },
+ ],
+ },
{
type: "category",
label: "dbt Cloud Release Notes",
+ link: { type: "doc", id: "docs/dbt-versions/dbt-cloud-release-notes" },
items: [
"docs/dbt-versions/dbt-cloud-release-notes",
{
@@ -622,6 +708,7 @@ const sidebarSettings = {
"reference/resource-configs/fal-configs",
"reference/resource-configs/oracle-configs",
"reference/resource-configs/upsolver-configs",
+ "reference/resource-configs/starrocks-configs",
],
},
{
@@ -633,7 +720,6 @@ const sidebarSettings = {
type: "category",
label: "General properties",
items: [
- "reference/resource-properties/access",
"reference/resource-properties/columns",
"reference/resource-properties/config",
"reference/resource-properties/constraints",
@@ -650,6 +736,7 @@ const sidebarSettings = {
type: "category",
label: "General configs",
items: [
+ "reference/resource-configs/access",
"reference/resource-configs/alias",
"reference/resource-configs/database",
"reference/resource-configs/enabled",
@@ -684,6 +771,7 @@ const sidebarSettings = {
"reference/seed-properties",
"reference/seed-configs",
"reference/resource-configs/column_types",
+ "reference/resource-configs/delimiter",
"reference/resource-configs/quote_columns",
],
},
@@ -711,6 +799,7 @@ const sidebarSettings = {
"reference/resource-configs/limit",
"reference/resource-configs/severity",
"reference/resource-configs/store_failures",
+ "reference/resource-configs/store_failures_as",
"reference/resource-configs/where",
],
},
@@ -864,7 +953,13 @@ const sidebarSettings = {
{
type: "category",
label: "Database Permissions",
- items: ["reference/snowflake-permissions"],
+ items: [
+ "reference/database-permissions/about-database-permissions",
+ "reference/database-permissions/databricks-permissions",
+ "reference/database-permissions/postgres-permissions",
+ "reference/database-permissions/redshift-permissions",
+ "reference/database-permissions/snowflake-permissions",
+ ],
},
],
guides: [
@@ -928,7 +1023,19 @@ const sidebarSettings = {
},
{
type: "category",
- label: "Materializations best practices",
+ label: "How we build our dbt Mesh projects",
+ link: {
+ type: "doc",
+ id: "guides/best-practices/how-we-mesh/mesh-1-intro",
+ },
+ items: [
+ "guides/best-practices/how-we-mesh/mesh-2-structures",
+ "guides/best-practices/how-we-mesh/mesh-3-implementation",
+ ],
+ },
+ {
+ type: "category",
+ label: "Materialization best practices",
link: {
type: "doc",
id: "guides/best-practices/materializations/materializations-guide-1-guide-overview",
@@ -1025,20 +1132,17 @@ const sidebarSettings = {
{
type: "category",
label: "Versions",
- link: {
- type: "generated-index",
- title: "Version migration guides",
- description:
- "Learn how to upgrade to the latest version of dbt Core.",
- slug: "/guides/migration/versions",
- },
items: [
- {
- type: "autogenerated",
- dirName: "guides/migration/versions",
+ "docs/dbt-versions/core-upgrade/upgrading-to-v1.7",
+ "docs/dbt-versions/core-upgrade/upgrading-to-v1.6",
+ "docs/dbt-versions/core-upgrade/upgrading-to-v1.5",
+ "docs/dbt-versions/core-upgrade/upgrading-to-v1.4",
+ "docs/dbt-versions/core-upgrade/upgrading-to-v1.3",
+ "docs/dbt-versions/core-upgrade/upgrading-to-v1.2",
+ "docs/dbt-versions/core-upgrade/upgrading-to-v1.1",
+ "docs/dbt-versions/core-upgrade/upgrading-to-v1.0",
+ ],
},
- ],
- },
{
type: "category",
label: "Tools",
diff --git a/website/snippets/_adapters-trusted.md b/website/snippets/_adapters-trusted.md
index 10af0218e22..7747ce16dec 100644
--- a/website/snippets/_adapters-trusted.md
+++ b/website/snippets/_adapters-trusted.md
@@ -2,7 +2,19 @@
+
+
+
+
diff --git a/website/snippets/_adapters-verified.md b/website/snippets/_adapters-verified.md
index 7caf099b7d1..3cc1e800448 100644
--- a/website/snippets/_adapters-verified.md
+++ b/website/snippets/_adapters-verified.md
@@ -2,61 +2,60 @@