Refresh expired access tokens

[studer-l]

Adds an additional flow to the existing middleware to refresh expired
tokens. The implementation handles multiple active grants. Expired
grants that failed to refresh are removed.

The refresh workflow is unconditionally activated. Since a token refresh
may occur for any request, access tokens are now always added to the
response's :session. This may break code which previously relied on the
:session only being set during the initial grant workflow. I do not
think this can be avoided.

If a refresh occurs, the tokens in (:session request) are left as-is,
the updated access tokens are accessibly via the existing
:oauth2/access-token key. This allows downstream handlers to observe
that a token refresh occurred.

There is a potential bug, where concurrent requests with expired tokens
may race. For example, consider a page containing a css and a js
resource. If a user's access token was to expire exactly as the
index.html finishes loading, their browser may concurrently fetch both
the css and js resources, triggering two concurrent token refresh
attempts with the same token, one of which may fail. I do not see a way to
address this without introducing considerable complexity.

I have added a timeout of 60 seconds to the refresh http request, which
means a slow oauth backend will cause users to become logged out. I
think this is more informative for users than hanging forever.

Fixes #40

[weavejester]

Thanks for the PR. This might take a while for me to get through. Please ensure that the commit contains only changes relevant to this feature, that all lines are within 80 characters, and that only public functions have docstrings.

[weavejester]

If I have time I'll do a more thorough review a little later, but these are my initial thoughts.

[weavejester]

It feels as if this function is attempting to do a lot, as we can see from it's length! I think we could split this up into three logical steps:

1. Collect the expired access tokens
2. Send them off via HTTP to be refreshed
3. Update the access token map

Only step 2 is different for sync/async. What we essentially want here is a way of running a bunch of async functions in parallel, and then to collect the results. That way we can have a function like:

(defn- refresh-expired-tokens
  ([profiles access-tokens]
    (pmap (partial refresh-token profiles)
          (expired-access-tokens access-tokens)))
  ([profiles access-tokens respond raise]
    (amap (partial refresh-token profiles)
          (expired-access-tokens access-tokens)
          respond raise)))

If map (and pmap) expect a function (f x), then amap would expect a function (f x respond raise).

I'm running out of review time so I don't have enough time to sketch out the implementation of amap, but if I get time later I'll go ahead and do so.

[studer-l]

Yes, you are right that this is a giant function; I usually refrain from writing novel combinators, but you are right that here I have an implementation of a ring-style-async-map that could be extracted.

I will give it a try later to see if I can simplify it.

I am not sure about the use of pmap, i do not think it is worth it to spawn a thread when in almost all cases only a single authentication provider is used.

[studer-l]

I gave it a try, but ultimately decided not to implement a generic async map; there are a handful complications which I feel are not worth the headache of generalization (error handling, preserving order); the most recent commit adds a less ambitious, specialized version that works for associative containers and assigns nil to a key on failure. This does greatly simplify refresh-all-tokens though, so I hope this addresses your concern 😄

[weavejester]

This docstring addition should be part of another commit ideally.

[weavejester]

What if there are no refreshed tokens?

[studer-l]

i assume the naming was too confusing here --- the refreshed-tokens contain both the newly refreshed tokens as well as the existing, non-expired tokens.

[studer-l]

Hi @weavejester thanks for the very prompt and thorough review 😄

I will see about doing some more work to refactor the giant function you mentioned, don't hesitate to let me know if anything else looks clunky 😅

[studer-l]

Yes, you are right that this is a giant function; I usually refrain from writing novel combinators, but you are right that here I have an implementation of a ring-style-async-map that could be extracted.

I will give it a try later to see if I can simplify it.

I am not sure about the use of pmap, i do not think it is worth it to spawn a thread when in almost all cases only a single authentication provider is used.

[studer-l]

i assume the naming was too confusing here --- the refreshed-tokens contain both the newly refreshed tokens as well as the existing, non-expired tokens.

[weavejester]

Thank you very much for the updates! I have some additional feedback, but we're getting there. One thing that does need to be taken account of is what happens if the handler doesn't return a session. In that case, we should take the session from the request instead.

[weavejester]

This assumes a :session key in the response map from the handler, which I don't believe is something you can rely on.

[ieugen]

hi @studer-l ,

I'm working on an app that uses oauth2 and I would need token refresh - since I have 5 min tokens.
Do you have time to finish this PR?
I'll see if I can implement and test it.
Please ping me when you have anything new pushed.

Thanks,
Eugen

[studer-l]

hi @studer-l ,

I'm working on an app that uses oauth2 and I would need token refresh - since I have 5 min tokens. Do you have time to finish this PR? I'll see if I can implement and test it. Please ping me when you have anything new pushed.

Thanks, Eugen

Hi @ieugen
Thank you for pinging me about this, sorry for letting this pull request linger; I should be able to find the time to finish this, no worries.

[ieugen]

Hi,

Anything I can do to help?
I've taken a break from this but I think I will get back on it in 2-3 weeks.

Eugen

[studer-l]

Hey @weavejester sorry to bother you with this again, I've gone through your comments again and I think they should all be addressed.

[weavejester]

Thanks for the updated changes. I think we're nearly there. I've made a few more suggestions in order to improve code clarity.

[weavejester]

In the synchronous map, the error catching function is in the map iteration function, but in async-map-values, the error catching function is built in.

My suggestion would be to make these two maps consistent. We assume that the mapping function, f, will not raise an exception:

(defn- async-map-values [f respond m]
  (let [total (count m)
        results (atom {})
        respond-when-done #(when (= (count %) total) (respond %))]
    (if (zero? total)
      (respond {})
      (doseq [[k v] m]
        (f v #(respond-when-done (swap! results assoc k %)))))

We then do the equivalent of "catching" the exception when calling it in refresh-all-tokens:

   (async-map-values
    (fn [[profile refresh-token] respond]
      (refresh-one-token profile refresh-token respond (fn [_] (respond nil)))
    (fn [refreshed-tokens]
      (respond (reduce update-tokens access-tokens refreshed-tokens)))
    (refresh-tasks profiles access-tokens))))

[studer-l]

Indeed your refactor is more elegant, but its behavior differs from what I initially wrote. Assume two tokens require refresh, one of which fails:

• When following your suggestion, refresh-all-tokens's continuation is called with nil. So no refresh occurs.
• In the version you reviewed, the results in async-map-values will accumulate both a nil and a new access token, and the user's session will receive the one updated access token. The failed token will be retried on subsequent requests.

Do you have a preferred behavior in this case?