-
Notifications
You must be signed in to change notification settings - Fork 7
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Only enable requester pays when necessary #400
Comments
mbaumann-broad
pushed a commit
that referenced
this issue
Jan 18, 2023
Only enable (Google) requester pays data access when the given DRS URIs require it and the platform TNU is running on supports it.
mbaumann-broad
pushed a commit
that referenced
this issue
Jan 18, 2023
Only enable (Google) requester pays data access when the given DRS URIs require it and the platform TNU is running on supports it.
mbaumann-broad
pushed a commit
that referenced
this issue
Jan 18, 2023
Only enable (Google) requester pays data access when the given DRS URIs require it and the platform TNU is running on supports it.
mbaumann-broad
pushed a commit
that referenced
this issue
Jan 19, 2023
Only enable (Google) requester pays data access when the given DRS URIs require it and the platform TNU is running on supports it.
mbaumann-broad
pushed a commit
that referenced
this issue
Jan 19, 2023
Only enable (Google) requester pays data access when the given DRS URIs require it and the platform TNU is running on supports it.
mbaumann-broad
pushed a commit
that referenced
this issue
Jan 19, 2023
Only enable (Google) requester pays data access when the given DRS URIs require it and the platform TNU is running on supports it.
mbaumann-broad
pushed a commit
that referenced
this issue
Jan 19, 2023
Only enable (Google) requester pays data access when the given DRS URIs require it and the platform TNU is running on supports it.
mbaumann-broad
pushed a commit
that referenced
this issue
Jan 20, 2023
Only enable (Google) requester pays data access when the given DRS URIs require it and the platform TNU is running on supports it.
mbaumann-broad
pushed a commit
that referenced
this issue
Jan 20, 2023
Only enable (Google) requester pays data access when the given DRS URIs require it and the platform TNU is running on supports it.
mbaumann-broad
pushed a commit
that referenced
this issue
Jan 24, 2023
Only enable (Google) requester pays data access when the given DRS URIs require it and the platform TNU is running on supports it.
mbaumann-broad
pushed a commit
that referenced
this issue
Jan 25, 2023
Only enable (Google) requester pays data access when the given DRS URIs require it and the platform TNU is running on supports it.
mbaumann-broad
added a commit
that referenced
this issue
Jan 25, 2023
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Objective
Only perform the operation to enable requester pays when it is currently necessary.
The only case where it is currently necessary is when accessing AnVIL data hosted by Gen3.
This case can be identified by DRS URIs starting with "drs://dg.ANV0".
In all other cases, enabling requester pays adds unnecessary delays and, in some cases, as when a Terra user has an external identity linkage to Kids First, spurious errors.
Background
Accessing DRS data hosted by Gen3 in requester pays buckets using Google cloud-native access APIs required that the Gen3 user client service account be granted the
serviceusage.services.use
permission on the Terra user's Google project.This privileged operation is performed by the Rawls API enableRequesterPaysForLinkedServiceAccounts.
Originally, this was needed for Terra DRS access to Gen3-hosted data for both Biodata Catalyst and AnVIL.
Terra DRS access to Gen3-hosted Biodata Catalyst data has since changed to using signed URLs, and enabling requester pays is no longer necessary for Biodata Catalyst.
Terra DRS access to Gen3-hosted data for NCI CRDC and Kids First has never been necessary.
When AnVIL data is fully hosted in Terra Data Repo (TDR), enabling requester pays will no longer be needed at all and should be disabled completely.
Acceptance Criteria
The text was updated successfully, but these errors were encountered: