Are user initiated aborts a network failure?

[igrigorik]

The user clicks on a blue link, which kicks off the navigation request. After some time delta, the user's patience runs out and they..

1. Click cancel and navigation is aborted (rendering of initial page hasn't been updated)
2. Hit the back key and navigation is aborted (they're looking at white screen and give up)

If the origin being navigated to has an associated NEL policy, would the origin see a report for either or both of these cases? Per our current definition in the spec, assuming the UA did not get a 2XX or 3XX response and user aborts the navigation.. I believe the answer is "yes", but want to sanity check both from spec and implementation perspective.

I guess the meta question here is: how do we treat user initiated aborts?

[chlily1]

I think the answer right now is "no" on the implementation side.

My understanding of how this currently works in Chromium is that a "successful" report is generated (but most likely not sent, depending on the success sampling rate) if valid headers have been received and the network transaction is in an idle state [1]. If not, then the network transaction is silently destroyed without generating a NEL report.

At the level at which we're generating NEL reports, I don't think there's a way to tell why exactly the transaction was cancelled, but we may be able to add code to cover more cases. (I may be misunderstanding how URLRequests/HttpNetworkTransactions work. @MattMenke2 is the expert here -- please correct me if I'm wrong.)

[1] https://cs.chromium.org/chromium/src/net/http/http_network_transaction.cc?l=131

[igrigorik]

Thanks for the pointer and confirming the behavior Lily.

@dcreager curious, have these scenarios come up in previous discussions?

An example use case I'm thinking of here is user clicking on a link that's taking way too long to load (reasons unknown) and manually aborting the navigation — e.g. DNS resolved, TCP handshake completed but the server is non responsive and user runs out of patience, at which point they initiate a manual abort. In this case, from a site owners perspective.. I would argue that's a failed navigation and unless NEL emits a report, they're blind to it.

[MattMenke2]

It looks to me like we may or may not generate a success report on cancellation, depending on state of the HttpNetworkTransaction.

If there's a pending body read, we don't generate a report. If there's no pending body read, we do generate a successful report. I assume there's generally a pending body read when a request is cancelled, unless we're actually stalled on something (auth challenge, client cert request, body IPC buffer is full, pending redirect not yet followed, etc).

[igrigorik]

@MattMenke2 what are the prerequisites for transitioning to "pending body read" state? Does this imply we received the response headers?

The types of failures I would be interested in here are...

• user cancelling during DNS resolve or TCP and TLS handshakes
• request headers sent but server is slow or unresponsive when user cancels
• response is streaming but not complete when user cancels

[MattMenke2]

It's only the case where we're streaming the body where we have a pending body read, so at that point we've read the final headers (after all redirects). Cancelling at earlier times will also not result in reporting error information, since we have a check in a destructor for having valid headers before putting together a report, if we haven't put one together already.

[MattMenke2]

Another fun case: Redirects. When we see a redirect, we follow the redirect, without trying to read the response body. If we're using H2/QUIC, we just cancel the stream we received the redirect response on. If it's HTTP/1.x, we try and drain the response body out of band, to try and reuse the socket. If draining the response body takes too long, we time out the connection. We can also get a network error while draining the body. In either case, we've already reported the HTTP headers, and followed the redirect. How should that fit into error reporting? Is it a success or a failure?

[igrigorik]

Matt, thanks that's really helpful. The behavior you outlined addresses most of the cases I highlighted..

• Slow DNS, TCP or TLS handshake -> user abort -> failure report is sent (no body read)
• Unresponsive server (no http headers received) -> user abort -> failure report is sent

It's possible that a server flushes headers and then trickles out the response (some CDN's do and promote this as an optimization feature), but generally in my experience it's (unfortunately) not common for servers to flush headers well ahead of the response... Well, except we do use that strategy for Search and other Google products.

Thinking out loud, would it be crazy to emit a success report IFF response body finished?

Re, redirects: that's a great question. For same origin redirect chain I could see a case for either each redirect being treated as separate navigation, or as a single 'transaction' — i.e. if any fails we emit an error report. For cross-origin redirects, it seems that we ought to treat each origin separately. E.g. if user navigates to foo.com which then redirects them to bar.com, both foo.com and bar.com have different NEL policies and we should respect them.

[MattMenke2]

Not sure we want to introduce a concept of origin at that layer of the stack. Another thing to think about is proxies. If we have an error looking up the domain name of the proxy to use, establishing a connection to the proxy, or a proxy returns a funky result to a CONNECT request (auth challenge, or something other than the response indicating it has established a tunnel), should we report that?

If there are multiple proxies in the proxy list, and the first fails, but the next in the list succeeds, should we report the initial failure? What if preconnects fail - there's no request in that case. What if we reject a pushed HTTP stream because we don't like it? (We reject pushed auth challenges, for instance). What if the server requests client auth? Is that an error? HTTP auth we seem to include in the error set, so I guess so?

There's such a thing as "expected" errors whenever a socket is reused. If we reuse a socket (Or use a preconnected but stale socket), we sometimes get a connection close even because the server timed out the socket while we were keeping it around for reuse. In this case, we automatically resend the request. Should we log the original request, which might be an expected error, as an error? We do this both on close and on reset, actually - neither of these may actually reflect an error on the part of the server, and if the retry succeeds, we never surface the error page to the user.

Another case: What if we see a redirect, and the request is aborted rather than following the redirect. In particuar, this can happen on CORS failures (Though the layer we're logging at doesn't know about CORS failures, and we can't really teach it about them).

Another issue: We normally isolate CORS and uncredentialed / non-CORS requests. Is there any reason to separate out network errors by this classification? This is mostly to protect the site the request is being made to from third party sites making the request, and I don't really see any attacks that combining the two sorts of requests could surface, but I'm not really an expert in that space.

[MattMenke2]

And actually answering your question:

"Thinking out loud, would it be crazy to emit a success report IFF response body finished?"

I think this is the full list of caveats to that (most of these already mentioned):

• HEAD and 304 responses have no bodies. I think we do try and read bodies for HEAD requests through the normal paths, and 304 responses go through the response body drainer, like redirects.
• We don't always read the bodies of auth challenges and redirects (We only read them if the user declines to provide auth credentials). I guess we consider these failures, either way.
• Some consumers just cancel requests after they see the headers rather than waiting for the body.

[chlily1]

Just landed https://crrev.com/c/1480195, so the behavior now is to generate an "abandoned" report if the request is cancelled without having previously generated either an error or success report.

Unfortunately these will all show up as "application" phase, regardless of what phase the network transaction was in when it was cancelled.

[igrigorik]

Just landed https://crrev.com/c/1480195, so the behavior now is to generate an "abandoned" report if the request is cancelled without having previously generated either an error or success report.

@chlily1 nice work! I think that resolves all the scenarios and edge cases @MattMenke2 highlighted above. However, it looks like it got reverted yesterday (doh), are you planning to take another run at it?

[chlily1]

are you planning to take another run at it?

Yes, once I fix a bug that it caused.

[igrigorik]

@chlily1 any updates on this one? Looks like the original CL is still reverted, or is there a new one I should follow?

[chlily1]

Woops, sorry for forgetting to update this thread. It was relanded as https://crrev.com/c/1491905.

[igrigorik]

\o/ ... awesome work, thanks Lily! Based on the branch point, we should see this rollout in M74 stable?

On a related note, do we have any WPT tests to verify this behavior? /cc @yoavweiss