Goth Deps Update

[snackfart]

Hey,

it seems to me that the current doc of goth(1.3) no longer documents the version used by waffle_gcs (1.1).
Therefore the setup and usage of waffle_gcs is a bit frustrating.

Regards

[almirsarajcic]

Have you been able to set it up successfully? I'd really appreciate a PR if you've been able to resolve the issue.

[snackfart]

I simply gave up, sry.

[snackfart]

But the main problem is that goth cant access the auth json

[davidspek]

This is a guide to update to Goth 1.3, I believe that will be useful here. https://hexdocs.pm/goth/upgrade_guide.html

[petermueller]

Has anyone taken a stab at this?

I'm making a POC for pigeon at the moment and if folks would like, I can take a look at either updating the lib and/or adding updated docs for waffle_gcs too. For context, I'd probably be doing this work anyway for ourselves.

[petermueller]

Good-ish news

As an update, I have an implementation that should work for public buckets, or private buckets that don't require signed urls. It's basically just defining a Token.Fetcher on a module that you pass in config.

# config.exs
config :waffle,
  token_fetcher: MyApp.Goth

# my_app_goth.ex
defmodule MyApp.Goth
  @impl Waffle.Storage.Google.Token.Fetcher
  def get_token(_scope), do: Goth.fetch!(__MODULE__).token
end

This presumes that MyApp.Goth was initialized in application.ex with the correct scopes and credentials source for the GCS buckets.

---

Bad(?) news

A problem can arise though when you're not using a "service account json" (from a file, or passed directly) as there's no "client_email" key, nor "private_key" in a Metadata-Server OAuth2 access_token, and GCS's URL signing (v2, what is in use here, and v4 that's the current approach) requires a private key.

This presents a conflict, as using the Metadata-Server's OAuth2 access_token is the suggested approach now for Compute Engine hosts. So if you try to use signed: true in your config, it'll fail in Waffle.Storage.Google.UrlV2.build_signed_url/3 at line 82 and even if set manually, would fail in sign_request trying to get the private key.

An approach to fixing this would be to implement a UrlV4 for the Waffle.Storage.Google.Url behaviour that utilizes GCP's IAM API and the its "signBlob" method which is luckily already available in the google_api_iam_credentials library

I'm conflicted on whether the "signBlob" approach is a good idea since a network round-trip to that API will be a lot higher than signing locally, and if you're displaying a number of files on the same page, that's a lot of network calls.

The main concern I have though is that the current implementation is relying on undocumented Goth modules now (Goth.Config) making it harder to separate my config per bucket. I don't want to have a private key json file of the same service account that controls all my buckets, if I could just have a private key json of a service account for the buckets that need signing. I don't know. I think this is probably more of a waffle config thing in general, it seems?

[ulissesalmeida]

Based on @petermueller comments, I created this PR that adds the first step to officially support Goth >= 1.3.

Thankfully, there are not many changes required when you're using the old service account json file. For URLV4 could be separated, PR/feature release.