Support multiple package versions

[rvanasa]

We are currently working a new --override compiler flag which should make it possible for Mops to support conflicting major package versions as originally discussed in dfinity/motoko#3971.

The flag is inspired by the design in the above GitHub issue with some modifications for generality and backwards compatibility.

@ZenVoich, what are your thoughts on this? The motivating use case is being able to release the new Motoko base library as base@1.0.0 without breaking packages using the current version.

[ZenVoich]

LGTM!

Is it easier to implement --package + --override <dir> <from-name> <to-name> in moc/mops
Compared to just using optional value in --package --package <name> <pkg-src-dir> [dir]?

So

--package base .mops/base@1.0.0/src
--package base@0.13.0 .mops/base@0.13.0/src
--package vector .mops/vector@0.4.1/src

--override .mops/vector@0.4.1 base base@0.13.0

instead of

--package base .mops/base@1.0.0/src
--package vector .mops/vector@0.4.1/src
--package base .mops/base@0.13.0/src .mops/vector@0.4.1/src

[rvanasa]

Glad to hear that this approach works for you! The --override flag is preferable on the compiler end because it's more intuitive for several edge cases. For example, if we had two packages that pointed to the same path, these would be considered equivalent to each other since the package is loaded before compiling the top-level program. Likewise, using a local import outside of the src directory of a package would otherwise result in the file being considered part of the top level rather than within the package.

Specifying the path also makes it possible to include different package versions within subdirectories of a project, which could be a useful feature down the road.

[Gekctek]

I have run into issue of having different packages reference different version of others. are there any good workarounds right now

[ZenVoich]

I have run into issue of having different packages reference different version of others. are there any good workarounds right now

Could you share more details?

[Gekctek]

https://github.com/edjCase/liminal/tree/CborVersionConflict

Here is a branch with an example
I reference serde and cbor in my MOPS but serde references cbor 1.0.0 and my cbor reference is 2.0.0

When i run my tests i get

✖ test/RateLimiter-integration.test.mo
 ✖ 
 FAIL .mops/serde@3.2.2/src/CBOR/lib.mo:130:44: type error [M0096], expression of type
  Blob
cannot produce expected type
  {next : () -> ?Nat8}

because the serde seems like its using the new cbor 2 vs 1, and there was a breaking change

[ZenVoich]

There are 3 major versions of cbor in dep tree:

Warning! Conflicting versions of dependency "cbor"
  cbor 2.0.0 is dependency of liminal@0.1.0
  cbor 0.1.3 is dependency of serde@3.1.0
  cbor 0.1.3 is dependency of ic-certification@0.1.3
  cbor 1.0.0 is dependency of serde@3.2.2

for now you can pin cbor version to name cbor@2 or cbor@2.0.0

"cbor@2" = "2.0.0"

and you have to import it as

import CBOR "mo:cbor@2";

With --override implementation this case would be handled without the need for pinning...