chore: Project and workflow maintenance

[sethaxen]

This PR makes a number of improvements to the environments and workflows to modernize the package, including:

• Replace [extras] with a test-specific environment ([extras] will be deprecated in Julia v1.13, and this is now the recommended approach)
• Add a test workspace for Julia >= v1.12
• Add a fallback [sources] entry to the test environment for Julia v1.11
• Add a [sources] entry to the docs environment
• Replace CompatHelper with DependaBot for managing dependency compats. DependaBot is more convenient when the same dependency compat might need to be bumped for more than one environment. We could keep CompatHelper as well for now, at the cost of more redundant PRs.
• Split the downgrade workflow out from CI to its own workflow so that a failure in this workflow doesn't effect our project CI status. Because of the above changes to the test project, the test and main environments are now jointly resolved by the downgrade action, so unlike before, now we're actually testing at the lower bounds of our test deps. To make this work, Aqua's lower bound needed to be bumped.

[github-actions]

MCMCDiagnosticTools.jl documentation for PR #150 is available at:
https://TuringLang.github.io/MCMCDiagnosticTools.jl/previews/PR150/

[coveralls]

Pull Request Test Coverage Report for Build 22668728124

Details

• 0 of 0 changed or added relevant lines in 0 files are covered.
• No unchanged relevant lines lost coverage.
• Overall coverage remained the same at 96.793%

---

Totals
Change from base Build 21401332519:	0.0%
Covered Lines:	845
Relevant Lines:	873

---

💛 - Coveralls

[codecov]

Codecov Report

✅ All modified and coverable lines are covered by tests.
✅ Project coverage is 96.79%. Comparing base (ef1019a) to head (2ccc448).

Additional details and impacted files

@@           Coverage Diff           @@
##             main     #150   +/-   ##
=======================================
  Coverage   96.79%   96.79%           
=======================================
  Files          10       10           
  Lines         873      873           
=======================================
  Hits          845      845           
  Misses         28       28           

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

🚀 New features to boost your workflow:

• ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.

[devmotion]

Thank you! I just have a few questions:

Replace [extras] with a test-specific environment ([extras] will be deprecated in Julia v1.13, and this is now the recommended approach)

Does this have any impact on Julia < 1.13?

In my experience separate test environments were often less reliable/stable than [extras], probably due to how Pkg resolved these environments. It seems the use of [extras] will be deprecated in 1.13 and users are supposed to use workspaces instead - but those are not supported on Julia 1.10 IIRC.

This makes me wonder whether we should wait with the switch until all supported Julia versions support workspaces?

Add a test workspace for Julia >= v1.12
Add a fallback [sources] entry to the test environment for Julia v1.11
Add a [sources] entry to the docs environment

Also all of these are not supported on Julia 1.10 it seems?

Replace CompatHelper with DependaBot for managing dependency compats. DependaBot is more convenient when the same dependency compat might need to be bumped for more than one environment. We could keep CompatHelper as well for now, at the cost of more redundant PRs.

Have you used dependabot for Julia compat updates? It's still a beta version, and while I haven't used it myself I saw a few issues reported on the Julia Slack (Guillaume ran into a few it seems, eg https://julialang.slack.com/archives/C6A044SQH/p1765873070848149?thread_ts=1765873070.848149&cid=C6A044SQH and https://julialang.slack.com/archives/C6A044SQH/p1763719297004809?thread_ts=1763719297.004809&cid=C6A044SQH).

That makes me wonder whether we should wait a bit with switching to dependabot until these initial quirks are fixed?

Split the downgrade workflow out from CI to its own workflow so that a failure in this workflow doesn't effect our project CI status.

That's a very reasonable change IMO.

[sethaxen]

Replace [extras] with a test-specific environment ([extras] will be deprecated in Julia v1.13, and this is now the recommended approach)

Does this have any impact on Julia < 1.13?

No, older versions supported the test environment as well. Workspace and sources will be ignored, but the julia test action already executes the tests in a way that works for these older versions.

In my experience separate test environments were often less reliable/stable than [extras], probably due to how Pkg resolved these environments. It seems the use of [extras] will be deprecated in 1.13 and users are supposed to use workspaces instead - but those are not supported on Julia 1.10 IIRC.

I don't recall having issues with stability of test environments. My main complaint was that it was annoying and error-prone to keep test dep compats and project dep compats synchronized, and CompatHelper is not useful here since it doesn't merge PRs. That's why this PR switches to Dependabot, which updates for all projects in a single PR. That has its own downside (see below) but helps with this problem.

This makes me wonder whether we should wait with the switch until all supported Julia versions support workspaces?

I don't see a reason to. Workspaces are just ignored for older versions, so we don't lose anything by using it. The only possible issue here would be if a user on Julia v1.10 activated test and then ran include("test/runtests.jl") without first developing .. Then they'd run with the latest release instead of the local version of the package, but I don't see this being a serious problem. On the other hand, by switching like this, the downgrade resolver actually downgrades the test-specific dependencies to their lower bounds, which it didn't do beforehand.

Add a test workspace for Julia >= v1.12
Add a fallback [sources] entry to the test environment for Julia v1.11
Add a [sources] entry to the docs environment

Also all of these are not supported on Julia 1.10 it seems?

Correct, but again, they're just ignored for v1.10, so adding them doesn't break anything.

Have you used dependabot for Julia compat updates? It's still a beta version, and while I haven't used it myself I saw a few issues reported on the Julia Slack (Guillaume ran into a few it seems, eg https://julialang.slack.com/archives/C6A044SQH/p1765873070848149?thread_ts=1765873070.848149&cid=C6A044SQH and https://julialang.slack.com/archives/C6A044SQH/p1763719297004809?thread_ts=1763719297.004809&cid=C6A044SQH).

Yes I've been using it for a few projects and I'm quite happy with it. In particular, it updates all environments together, so if we have a dep like MLJModelInterface in test and docs, we get a single PR updating it in both. A downside here is that if more than 1 dependency has a new release on a given day, we get a single PR updating both dependencies, but this seems rare. Regarding the issues above, the first seems rare (and perhaps already diagnosed), and the second is already resolved.

That makes me wonder whether we should wait a bit with switching to dependabot until these initial quirks are fixed?

I think it's polished enough to already be of use. A compromise could be to keep CompatHelper in parallel with dependabot until it's out of beta (I'm doing this for some of my repos alkready).

I do see an issue here I'm going to investigate a bit more, since the resolution process used by the downgrade action seems to ignore the sources and workspace entries. So if we set an MCMCDiagnosticTools compat in test that is incompatible with the main project, we should get an error, but instead the main Project manifest actually uses a version of MCMCDiagnosticTools downloaded from General instead of the local version. That makes me concerned that in general it is never using the local version of the package. I'm trying to find the edges of this to open an issue, so for now this PR should remain open.

[sethaxen]

I do see an issue here I'm going to investigate a bit more, since the resolution process used by the downgrade action seems to ignore the sources and workspace entries. So if we set an MCMCDiagnosticTools compat in test that is incompatible with the main project, we should get an error, but instead the main Project manifest actually uses a version of MCMCDiagnosticTools downloaded from General instead of the local version. That makes me concerned that in general it is never using the local version of the package. I'm trying to find the edges of this to open an issue...

Issue here: julia-actions/julia-downgrade-compat#36

[sethaxen]

Now that dependabot support for the julia ecosystem is out of beta, I've removed the requirement of the beta ecosystem.

The downgrad-compat action now has some support for jointly resolving the test env with the main project env, but I uncovered some remaining problems while updating this PR (specifically, the 3rd issue in julia-actions/julia-downgrade-compat#38), so we should wait for that to be resolved before merging.