Allow a path include list for url handling

[jwheare]

We have a PWA at https://www.irccloud.com but I'd only want that to handle URLs for the base domain root path at / and a sub path at /irc where our in-app urls are namespaced. I don't want plain HTML pages like /about or /faq to be handled by the PWA.

Is this something the proposal could allow for?

[tomayac]

This is possible, please see web app origin association file section of the article.

[jwheare]

I might be missing something, but that section seems to be about verifying ownership of different domains. I see the "paths" field but it's not clear if that's what I want from the documentation, or how I would use it to handle my case.

[tomayac]

Oh, I misread your message, assuming you were after the URL handling behavior, but (I think) you're actually asking about the service worker routing behavior. The new URLPattern can help, see this article.

[jwheare]

I think I am after the URL handling behaviour.

As I understand it, this spec is to designed to allow someone to click a link to our web app and have it open in an existing PWA, rather than a browser tab.

I want this behaviour, but only for certain paths.

[tomayac]

OK, so you are after the URL handling behavior after all :-) Let's ask @LuHuangMSFT: will a web app origin association file be honored if it's for the same origin? To make this concrete: example.com wants to handle links at example.com/handle, but ignore links at example.com/ignore. Will this work if they put up a web app origin association file that states these preferences in their own origin's example.com/.well-known/ folder?

[mandymsft]

will a web app origin association file be honored if it's for the same origin?

Yes, web app origin association file is used for all the origins (same or different) declared in the manifest url_handlers field. See this example association file of Lu's test app that declares /about as an excluded path, which will be handled by the browser.

[tomayac]

Thanks, @mandymsft. Going full circle then, turns out my initial response was right :-)

[jwheare]

OK, the documentation could be a lot clearer about this. At the moment it seems entirely geared towards cross domain concerns, not an obvious place to look. It's also odd and undesirable that you have to place your path includes in a separate file, rather than just define them inline with the url_handlers field in the main manifest, especially when you have no cross domain concerns.

[jwheare]

Thanks for the help.

[LuHuangMSFT]

@jwheare Appreciate the feedback. Sorry I was late to the conversation.
@tomayac, @mandymsft Thanks for your responses.

We should highlight in documentation that a WAOA file is needed even for the PWA's own origin. This is required because the app's scope may not be equal to the entire origin.

A developer may also want to make use of the path and exclude_paths features in the WAOA files to define a set of URLs for URL handling that's smaller than then app's scope. It may be possible to simply use the app's own scope as the boundary for URL handling and not have to provide a WAOA file - if there is no need for cross origin URL handling and no need to restrict the app scope for URL handling. Let us know if this is something you would support and use.

paths and exclude_paths were designed to be provided by the origins' WAOA and not by the manifest itself because we felt that the origins should control what paths to allow for URL handling. If paths and exclude_paths come from the web app manifest, they can be changed without the origin's knowledge and agreement. For consistency, we decided not to allow single-origin-root-scope apps to control this from the web app manifest.

[benfrancis]

I find this all a bit strange, since deep links are already meant to be a feature of Web App Manifest, it's just that no user agents implement that feature.

What's the rationale for using separate metadata files rather than just using the existing scope member for this purpose, and extending it if necessary?

There was extensive discussion around this topic for a number of years in w3c/manifest#114. Are the authors of this new proposal familiar with that prior work?

[jwheare]

@LuHuangMSFT

It may be possible to simply use the app's own scope as the boundary for URL handling and not have to provide a WAOA file - if there is no need for cross origin URL handling and no need to restrict the app scope for URL handling. Let us know if this is something you would support and use.

How is "the app's own scope" defined here? The start_url?

[jwheare]

Ah I see there's a scope field defined for manifests: https://w3c.github.io/manifest/#scope-member

Will have to look into this further, but if we can just rely on this to make sure only URLs that are in scope are handled then that would definitely work for us and seems the simplest option.

[LuHuangMSFT]

Responding to comments from @benfrancis here.

Summary up front: we proposed a model that requires a two-way handshake, similar to Android App Links, Windows Apps for Websites, iOS Universal Links, and unlike Android Deep Linking that requires no handshake. Allowing app developers the freedom to configure a different "scope" for URL handling behavior was intentional and we think unaffected by whether app scope can contain origins from different domains.

deep links are insufficient if apps want to handle URLs not within the app scope.

What's the rationale for using separate metadata files rather than just using the existing scope member for this purpose, and extending it if necessary?

Requiring WAOA files separate from the web app manifest has the purpose of validating the association with PWAs. This model is similar to that of Android App Links vs. Android Deep Links [1]. The same model is seen in Windows [2] and iOS [3]. In those examples, the information to allow or disallow paths is also provided by the metadata file and not the app itself.

Extending app scope with more expressive syntax and allowing multi-origin scope is useful in their own right but we found that developers often need to be able to configure a separate "scope" for URL handling. If app scope can be extended to include multiple origins from different domains, I think the origins themselves should play a part in validating the relationship. Not doing so means that unassociated 3rd party apps will be able to capture URLs with "deep linking" similar to Android.

[1] https://developer.android.com/training/app-links/verify-site-associations?authuser=0#the-difference
[2] https://docs.microsoft.com/en-us/windows/uwp/launch-resume/web-to-app-linking#associate-your-app-and-website-with-a-json-file
[3] https://developer.apple.com/library/archive/documentation/General/Conceptual/AppSearch/UniversalLinks.html

[LuHuangMSFT]

@jwheare Yes. If a scope member is not found within the web app manifest, app scope is derived from start_url. You can use both.

[LuHuangMSFT]

@benfrancis The proposed web-app-origin-association format could be useful for more than just URL handling. See: WICG/get-installed-related-apps#30

[benfrancis]

@LuHuangMSFT wrote:

similar to Android App Links, Windows Apps for Websites, iOS Universal Links

I am aware of these app linking technologies which divert web traffic to native apps, something which I would argue is harmful to the web but is unfortunately quite common. What I'm not sure about is why a similar technology is needed for web apps which should already be able to support deep linking using ordinary hyperlinks?

deep links are insufficient if apps want to handle URLs not within the app scope.

Web App manifest scope is indeed currently limited to a single origin, though there have been discussions in the past about ways to expand this for certain use cases. The original post regarding IRCCloud does not span multiple origins, however. It just has a navigation scope which can't currently be expressed by the current single path approach in Web App Manifest. In this case I'm not sure why something like WAOA files should be required?

Extending app scope with more expressive syntax and allowing multi-origin scope is useful in their own right but we found that developers often need to be able to configure a separate "scope" for URL handling.

I'm interested to know more about this. What is the difference between "URL handling" and "deep linking" here, and what are the use cases for two separate scopes?

If app scope can be extended to include multiple origins from different domains, I think the origins themselves should play a part in validating the relationship.
Not doing so means that unassociated 3rd party apps will be able to capture URLs with "deep linking" similar to Android.

I agree that in the case of capturing navigations across origins, something like that may be necessary.

There are other use cases for navigation scope spanning multiple origins though, such as a web app (e.g. mail.google.com) redirecting to another origin (e.g. accounts.google.com) for authentication, where the developer wants such navigations to stay inside the app context rather than open a separate browser window.

See w3c/manifest#114 (comment) for an example of how that might work with an expanded scope member of a web app manifest.

I agree that for certain use cases, a mechanism for establishing trust between origins may be necessary, but simple deep linking inside a web app within a single origin (like the IRCCloud case) should be possible with a Web App Manifest alone.

[tomayac]

We should highlight in documentation that a WAOA file is needed even for the PWA's own origin. This is required because the app's scope may not be equal to the entire origin.

Done, see GoogleChrome/web.dev#5929.

[LuHuangMSFT]

@benfrancis I have an update with some new information. Please take a look, thanks.

There is a new explainer here that proposes adding a scope_extensions web app manifest member:
https://github.com/WICG/manifest-incubations/blob/gh-pages/scope_extensions-explainer.md
scope_extensions more closely aligns with concept of extending app scope. It is intended to take the place of url_handlers and continue making use of web-app-origin-association. Origins will have to explicitly grant permissions for actions like "intercepting links" in web-app-origin-association. This way, the URL handling capable part of the app scope can differ from the app scope.

I agree that in the irccloud scenario (where the app is based in a single origin), it would be more developer-friendly if the app could opt in to single-origin-scope link handling behavior using the web app manifest alone. Do you think that should have a separate control in the manifest since scope_extensions is now more clearly a cross-origin concept?

What if url_handlers contains URL patterns to describe what URLs are allowed for URL handling but does not rely on web-app-origin-association. scope_extensions does depend on WAOA and still provides permission for URL handling but does not determine which URLs should be handled. This way a single-origin-scope app can use url_handlers for URLs within app scope without having to use scope_extensions or WAOA.

[alancutter]

Will have to look into this further, but if we can just rely on this to make sure only URLs that are in scope are handled then that would definitely work for us and seems the simplest option.

I'm interested in the outcome of this. If configuring app scope is sufficient for web developers then we don't need to add additional API surfaces here.

[benfrancis]

@LuHuangMSFT Thank you for pinging me regarding this proposal.

...

Edit: I've moved my feedback here WICG/manifest-incubations#40