Description
My use-case is local development and testing using fake-gcs-server. Auth is not required to use its GCS-compatible API.
However, Trino does not appear to have a way to not attempt authentication with Google's servers, even when setting the gcs.endpoint property.
If no auth-related properties (like gcs.json-key) are set, then GcsStorageFactory calls GoogleCredentials.getApplicationDefault().
However, this method throws an exception if no credentials are found:
Caused by: java.io.IOException: Your default credentials were not found. To set up Application Default Credentials for your environment, see https://cloud.google.com/docs/authentication/external/set-up-adc.
In contrast, if StorageOptions.Builder#setCredentials is simply never called, the underlying GCS library populates it by calling GoogleCredentials.getApplicationDefault() but ignoring any exceptions it throws.
I have a temporary workaround to just not call StorageOptions.Builder#setCredentials in GcsStorageFactory if gcs.endpoint is set:
diff --git a/lib/trino-filesystem-gcs/src/main/java/io/trino/filesystem/gcs/GcsStorageFactory.java b/lib/trino-filesystem-gcs/src/main/java/io/trino/filesystem/gcs/GcsStorageFactory.java
index f8ea12c452..b6b20537fd 100644
--- a/lib/trino-filesystem-gcs/src/main/java/io/trino/filesystem/gcs/GcsStorageFactory.java
+++ b/lib/trino-filesystem-gcs/src/main/java/io/trino/filesystem/gcs/GcsStorageFactory.java
@@ -93,6 +93,9 @@ public class GcsStorageFactory
credentials = GoogleCredentials.fromStream(inputStream).createScoped(DEFAULT_SCOPES);
}
}
+ else if (endpoint.isPresent()) {
+ credentials = null;
+ }
else {
credentials = jsonGoogleCredential.orElseGet(() -> {
try {
@@ -110,10 +113,13 @@ public class GcsStorageFactory
endpoint.ifPresent(storageOptionsBuilder::setHost);
+ if (credentials != null) {
+ storageOptionsBuilder.setCredentials(credentials);
+ }
+
// Note: without uniform strategy we cannot retry idempotent operations.
// The trino-filesystem api does not violate the conditions for idempotency, see https://cloud.google.com/storage/docs/retry-strategy#java for details.
return storageOptionsBuilder
- .setCredentials(credentials)
.setStorageRetryStrategy(getUniformStorageRetryStrategy())
.setRetrySettings(RetrySettings.newBuilder()
.setMaxAttempts(maxRetries + 1)
The three solutions I see are, in order of preference:
- where
GcsStorageFactorycurrently callsgetApplicationDefault(), just setcredentialsto null instead, and if it is indeed null, do not callStorageOptions.Builder#setCredentials, allowing the underlying library's behavior to apply - suppress exceptions thrown by
GoogleCredentials.getApplicationDefault()like the GCS client library itself does by default - add an additional config property to disable auth for GCS altogether, perhaps resulting in passing
NoCredentials.getInstance()as the credentials
I'm happy to open a PR for this.