Fix or replace AppVeyor bottleneck

[Gabriella439]

Right now AppVeyor is severely limiting the rate at which we can merge pull requests because each revision it builds takes ~30 minutes and it can only build one revision at a time.

One possible solution is to replace AppVeyor with Azure DevOps since the latter supports up to 10 concurrent builds free for open source projects

[jneira]

@Gabriel439 i am afraid that azure still doesn't support caching between builds so each one would take longer.
They are going to add support soon, though.

[EggBaconAndSpam]

AppVeyor doesn't seem to be doing any caching either... It's downloading everything again for each build. Switching to azure is probably not going to make things worse ;)

[ocharles]

What is AppVeyor used for?

[jneira]

@EggBaconAndSpam withouth caching it takes 40+ mins instead 15+!
@ocharles to make build+test+artifacts for windows

[jneira]

But i agree in change the ci to azure, you have unlimited build time and you can build using linux, mac and windows.
Recently hie started to use it succesfully: haskell/haskell-ide-engine#1238

[ocharles]

@ocharles to make build+test+artifacts for windows

Could this now be done using haskell.nix cross compilation to Windows on Hydra?

[joneshf]

Whoa, Nice!

[Gabriella439]

@ocharles: I took a look at that a while back. The main issue was that the Windows cross-compilation support doesn't appear work with the static Haskell executables work from NixOS/nixpkgs#43795 (which is Linux-specific)

[sjakobi]

Currently AppVeyor takes ~35 to 45 minutes per build. I believe much of that time is spent on re-building dependencies that I wish would be cached. I've tried to look around for projects with a working cache setup, but I couldn't find one. :(

[joneshf]

The appveyor builds for purty tend to hit the cache and run in 2-5 minutes. It's using dhall and purescript as dependencies, though a fairly small project overall. The appveyor.yml shows the caching: STACK_ROOT is wherever Appveyor puts it and the key is relative. That looks like the only stuff that's different. Could that be it?

It doesn't look like the dhall-haskell builds are even trying to cache things at the end. For reference, at the very end of the build, the cache is updated for purty. Is it possible there's something you have to turn on in the settings? It's been a long time since I set this up, so I don't remember all the steps I went through. More than happy to provide whatever help I can though, so let me know if there's anything else I can do.

[sjakobi]

Thanks for the help @joneshf! I'll try to copy a few things from purty then! :)

It doesn't look like the dhall-haskell builds are even trying to cache things at the end. For reference, at the very end of the build, the cache is updated for purty.

It seems to me that updating the cache may be limited to builds on master: This build does it, but that one doesn't.

[sjakobi]

After #1207 and #1214 AppVeyor builds typically take 10-11 minutes now.

The main problem seems to have been that we got the syntax for the cache rules wrong!

[Gabriella439]

shakes fist at YAML