Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

feat(schematic): fds 1447: implement pagination (#2421) #2428

Merged
merged 1 commit into from
Jan 17, 2024

Conversation

andrewelamb
Copy link
Contributor

fixes fds-1447

For all endpints that return an array of items, there are now 2 endpoints:

  • A non-paginated version "*Array"
  • A paginated verison "*Page"

Pagainated versions implement a limited version of pagination:

  • Users can set their amount per page to any amount
  • Pagiantion does not use tokens, changes to the underlying data in between queries will have incorrect results
  • Assumes schematic methods are deterministic(don't result in a random oreder)
  • Gets entire result of schematic method and breaks this up over several queries
  • Non-paginated queries don't have a cap on results
  • Endpoints that return data about a synapse entity (as opposed to the etnity itself such a a manifest) have been renamed Metadata

* feat(schematic): integration testing (#2398)

* changed authenticication so that only endpoints that need it have it

* updated schematic

* add patch for access token

* schema endpoints no longer mockeed

* added tests for handle exceptions

* added integration tests

* marked synapse tests

* added error handling for bad schema urls

* fix error message

* add workflow for end to end testing

* fix some test results

* add unit mark

* add unit mark

* add workflow for testing with secrets

* rename file

* fix synapse test file when secrets file doesnt exists

* fix test workflows

* turned synapse ids into secrets in workflow

* turned synapse ids into secrets in workflow

* Update schematic-api-ci.yml

* Update schematic-api-ci.yml

* Update schematic-api-ci.yml

* Update schematic-api-ci.yml

* Update schematic-api-ci.yml

* Update schematic-api-ci.yml

* Update schematic-api-ci.yml

* Update schematic-api-ci.yml

* add paging, and split connected noeds into two endpoints

* paginated preoject datasets query

* paginated preoject datasets query

* paginated dataset files endpoint

* pagniate project manifests endpoint

* paginate get node dependencies

* paginate get node dependencies

* paginate node properties endpoint

* paginate validation rules endpoint

* paginate get projects endpoint

* unpaginate node properties and validation rules endpoints

* unpaginate node properties and validation rules endpoints
Copy link

@mialy-defelice mialy-defelice left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Previously reviewed, so just approving.

Copy link
Member

@tschaffter tschaffter left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Approved changes to the CI workflow (change to file format)

@mialy-defelice mialy-defelice self-requested a review January 17, 2024 17:24
@andrewelamb andrewelamb merged commit 6d5cd01 into main Jan 17, 2024
7 checks passed
@andrewelamb andrewelamb deleted the Schematic-API-Staging branch January 17, 2024 17:26
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants