Consider rc semver compatible for the linker

[mschuwalow]

Feature

Currently the linker only considers only non-rc versions to be backwards compat https://github.com/alexcrichton/wasmtime/blob/main/crates/environ/src/component/names.rs#L194.

This is quite annoying when creating an rc for a new minor versions that is intended to be backwards compat.

Example:
We are currently supporting version 1.1.x of our wit in the host. We want to start development of a new, backwards compat 1.2.0 release, optimally publishing the first draft as 1.2.0-rc1. The problem becomes that if we want to support both 1.1.x and 1.2.0-rc we need to implement all the host functions twice, otherwise some of our users would get linking errors.

The proposal would be to consider rc versions to be backwards compat if they do not increment the major component. It's probably right if rcs are not considered to be semver compatible with each other, as during development of an rc apis often change.
So

/// * `foo` => `None`
/// * `foo:bar/baz` => `None`
/// * `foo:bar/[email protected]` => `Some(foo:bar/baz@1)`
/// * `foo:bar/[email protected]` => `Some(foo:bar/[email protected])`
/// * `foo:bar/[email protected]` => `None`
/// * `foo:bar/[email protected]` => `None`

would become

/// * `foo` => `None`
/// * `foo:bar/baz` => `None`
/// * `foo:bar/[email protected]` => `Some(foo:bar/baz@1)`
/// * `foo:bar/[email protected]` => `Some(foo:bar/[email protected])`
/// * `foo:bar/[email protected]` => `None`
/// * `foo:bar/[email protected]` => `None`
/// * `foo:bar/[email protected]` => `Some(foo:bar/[email protected])`
/// * `foo:bar/[email protected]` => `None`
/// * `foo:bar/[email protected]` => `Some(foo:bar/baz@1)`

Benefit

Allow platform maintainers to more easily ship new versions of their api without duplicating a lot of code.

Implementation

From what I can tell only the alternate_lookup_key function would need to be adjusted to handle the new cases.
https://github.com/alexcrichton/wasmtime/blob/main/crates/environ/src/component/names.rs#L199

Alternatives

One option that comes to mind is bumping the minor version on every change, but that makes it very difficult to keep the wit version in sync with the version of the platform.

Another is not publishing the new version of the wit until it is fully ready, but that makes it difficult to test with users and iterate on the new version.

A third option is to implement both the rc and the old version, but I would like to avoid that duplication.

[alexcrichton]

Thanks for the report! This is something where technically the decision of what to do here should be within the scope of the component model itself rather than Wasmtime. That being said I don't actually see docs for this upstream yet (or I'm missing them...) so it's also not necessarily a perfect venue just yet.

My personal feeling is that it's best to not deviate too too much from semver and my impression is that "rc" (or prereleases in general) are generally assumed to have no compatibility within each other for added APIs. In that sense trying to automatically substitute one for the other loses the ability for a more seamless experience.

Is this perhaps something where the unstable features of WIT would help out? That way you could add APIs as unstable and then as they're iterated on they're gated by default and you can break the unstable APIs as much as you want as well. Then at the end a minor release could be done and the APIs marked stable.

[mschuwalow]

I would propose not making the rcs compatible with each other, just with the latest backwards compatible version.
I.e.
1.1.0-rc.1 would be backwards compat to 1.0.0
1.1.0-rc.2 would be backwards compat to 1.0.0 but not to 1.1.0-rc.1

I think this would also be in line with what the semver spec says semver/semver#111

---

Thanks for pointing out the unstable features, I will check whether these are usable for us!

[alexcrichton]

Oh interesting, and that makes sense! Implementing such semantics doesn't seem unreasonable to me, but the nuances of that property would be difficult to implement. For example in the recommended change to the doc comment you mentioned above it would consider two release candidates equivalent to each other. There'd have to be a different and/or more clever implementation to implement that rcs are not equivalent but they are equivalent to historical versions (and maybe future? that also throws a wrench into things given how these versions match...)