Merge pull request #101 from dpguthrie/feat/use-beta-endpoint

Update Metadata Interface

Author: dpguthrie

.github/workflows/build.yml

@@ -1,18 +1,24 @@
 name: Build
 
 on:
-  pull_request_target:
-    branches: [ main ]
+  pull_request:
+    branches:
+      - main
+    types:
+      - opened
+      - reopened
+      - synchronize
+      - ready_for_review
 
 jobs:
   build:
 
     runs-on: ubuntu-latest
 
     steps:
-    - uses: actions/checkout@v2
+    - uses: actions/checkout@v3
     - name: Set up Python 3.9
-      uses: actions/setup-python@v2
+      uses: actions/setup-python@v3
       with:
         python-version: "3.9"
     - name: Install dependencies

.isort.cfg

@@ -4,7 +4,6 @@ import_heading_future=future
 import_heading_local=local
 import_heading_stdlib=stdlib
 import_heading_thirdparty=third party
-known_third_party=sgqlc
 
 # These settings makes isort compatible with Black:
 # https://github.com/psf/black#how-black-wraps-lines

.pre-commit-config.yaml

@@ -19,7 +19,7 @@ repos:
   - id: mypy
     stages: [commit,push]
     name: mypy
-    entry: poetry run mypy --ignore-missing-imports
+    entry: poetry run mypy --ignore-missing-imports --follow-imports=skip --strict-optional
     language: system
     types: [python]
 - repo: https://github.com/pre-commit/pre-commit-hooks

CHANGELOG.md

@@ -1,5 +1,26 @@
 # Changelog
 
+## [0.9.0] - 2024-01-11
+
+### Removed
+
+- All of the methods in the `_MetadataClient` except for `query`.  The Discovery API no longer allows a user to specify every single field recursively, which is what the `sgqlc` package would do.
+
+### Added
+
+- An optional keyword argument `use_beta_endpoint` to the `dbtCloudClient` class.  This will default to `True`, which means that the Discovery API will use the beta endpoint at https://metadata.<host>/beta/graphql instead of https://metadata.<host>/graphql.  This contains both the stable API resources (environment, models, tests, etc.) but also contains things for performance, recommendations, and lineage.
+- Ability to automatically paginate requests for the Discovery API.  If pagination is required/desired, ensure that your query is properly created with an `$after` variable and all of the fields within the `pageInfo` field.
+
+### Updated
+
+- Loosen restrictions on Pydantic - ">=2.0,<3.0"
+
+## [0.8.1] - 2023-12-20
+
+### Updated
+
+- Pydantic version from 1.10.5 to 2.5.*
+
 ## [0.8.0] - 2023-12-04
 
 ### Added

README.md

@@ -83,24 +83,50 @@ run['data']
 run['status']
 ```
 
-Similarly, use the `metadata` property to retrieve information about certain resources within your project - the example below shows how to retrieve metadata from models related to the most recent run for a given `job_id`.
+Similarly, use the `metadata` property to retrieve information from the [Discovery API](https://docs.getdbt.com/docs/dbt-cloud-apis/discovery-api).
+Here's how you could retrieve all of the metrics for your project.
 
 ```python
 from dbtc import dbtCloudClient
 
 client = dbtCloudClient()
-
-job_id = 1
-
-models = client.metadata.get_models(job_id)
-
-# Models nested inside a couple keys
-models['data']['models']
-
-# This is a list
-models['data']['models'][0]
+query = '''
+query ($environmentId: BigInt!, $first: Int!) {
+  environment(id: $environmentId) {
+    definition {
+      metrics(first: $first) {
+        edges {
+          node {
+            name
+            description
+            type
+            formula
+            filter
+            tags
+            parents {
+              name
+              resourceType
+            }
+          }
+        }
+      }
+    }
+  }
+}
+'''
+variables = {'environmentId': 1, 'first': 500}
+data = client.metadata.query(query, variables)
+
+# Data will be in the edges key, which will be a list of nodes
+nodes = data['data']['definition']['metrics']['edges']
+for node in nodes:
+    # node is a dictionary
+    node_name = node['name']
+    ...
 ```
 
+If you're unfamiliar either with the Schema to query or even how to write a GraphQL query, I highly recommend going to the [dbt Cloud Discovery API playground](https://metadata.cloud.getdbt.com/beta/graphql).  You'll be able to interactively explore the Schema while watching it write a GraphQL query for you!
+
 ### CLI
 
 The CLI example below will map to the python cloud example above:

dbtc/_version.py

@@ -1 +1 @@
-__version__ = "0.8.0"
+__version__ = "0.9.0"