RabbitMQ 4.2.2 RPM missing from mirror

[der-eismann]

Describe the bug

Hey everyone, we wanted to upgrade our RabbitMQ cluster to 4.2.2, however we noticed that it's currently missing from the yum1.novemberain.com mirror. It's present on Cloudsmith though, so maybe the mirror has a bug?

Reproduction steps

1. Package exists on Cloudsmith:
   https://cloudsmith.io/~rabbitmq/repos/rabbitmq-server/packages/detail/rpm/rabbitmq-server/4.2.2-1.el8/a=noarch;d=el%252F9;t=binary/
2. Check mirror repomd.xml:
   https://yum1.novemberain.com/rabbitmq/el/9/noarch/repodata/repomd.xml
3. Browse to shown filelist.xml:
   https://yum1.novemberain.com/rabbitmq/el/9/noarch/repodata/c16934843e1cb6b30aff2171eba53858-filelists.xml.gz
4. See that there's no 4.2.2 package

Expected behavior

4.2.2 can be installed via RPM mirror

Additional context

No response

[michaelklishin]

@der-eismann Repository Update Frequency

[der-eismann]

every two to four weeks

That is.. a lot. I guess I'm gonna start using the upstream Cloudsmith again then, thanks. Never noticed that it takes so long until now.

[michaelklishin]

@der-eismann we explicitly ask people to NOT use upstream Cloudsmith because the whole issue comes down to Cloudsmith traffic quota being exhausted in two weeks and standard dnf syncing not working until the quota rolls over.

To the point that if there was a way to block Cloudsmith traffic for everyone except our infrastructure hosts on our open source project plan, I'd do that without batting an eyelid.

RPMs are distributed with every GitHub release, take them and put them into any repository you like.

[michaelklishin]

For Debian packages, Cloudsmith is no longer used for package syncing. It took a major repository migration because a new repo means a new signing key.

The options for yum*.rabbitmq.com are

1. A similar migration that I expect to cause even more repetitive questions and complaints
2. Relying on Cloudsmith for syncing, which means the quota runs out in less than two weeks after every release (despite our repeated asks for several years now to migrate away from "the upstream"), so that syncing becomes possible only after the quota rolls over; this is what we have chosen to do for now
3. Forcing everyone to use an alternative from our key corporate sponsor which, I assure you, would be the absolute worst open source user experience imaginable

@der-eismann now you have a complete picture. Don't be a part of the problem, do not switch to "upstream Cloudsmith". If you need next day upgrades, download the .rpm from GitHub releases and put them to your own repositories any way you like.

"Open source" means "the soure is available", it does not mean that free infrastructure comes with it.

[michaelklishin]

Apparently there is a way to block Cloudsmith traffic except for our mirrors: IP-level restrictions.

Then all users would have to (finally) switch to yum*.rabbitmq.com but the upside of doing that is substantial:

1. No more 2-4 week delays in syncing, syncing will always succeed because the Cloudsmith account traffic quota will never be exhaused
2. For dnf (yum) clients, yum*.rabbitmq.com will have a dramatically higher quota (24 TiB or so)
3. At least for some time, this approach will not require a repository migration

So I will proceed to doing just that, restricting direct Cloudsmith access and solving the sync lag problem that way.

In the meantime, 4.2.2 is now available on the mirrors because the quota rolled over very late last month).

[der-eismann]

Well that sounds amazing, thank you for investigating that.

"Open source" means "the soure is available", it does not mean that free infrastructure comes with it.

I know, I've heard that in the Erlang repo before. IMHO a billion dollar company like VMware/Broadcom should be able to at least provide a basic up-to-date package mirror, but of course that would mean a few Dollars less profit. You wouldn't push Container images 2-4 weeks after tagging the release either. But we don't have to discuss this here, I expect you to have a different opinion on that and I have to accept that.

[michaelklishin]

@der-eismann there is a repository that can be used as an alternative. Most users, once they learn about what they have to do in order to use it, would simply automate re-publishing of packages from GitHub releases to their own, say, Cloudsmith, Artifactory or PackageCloud. It's that… cumbersome is the absolute nicest way I can put it.

So our team maintains these mirrors however we can to avoid that worse case scenario where the only way to consume open source RabbitMQ is via those aforementioned corporate repos.

[michaelklishin]

Speaking of container images, the community OCI is updated within hours from a new RabbitMQ, Erlang, OpenSSL or rabbitmqadmin v2 release but the actual image on Docker Hub shows up a couple of days later. Why? Because that's how the Docker Hub infrastructure works, and no one is going to change that over a few GitHub complaints.

Those who cannot wait for N days or a few weeks for new patch releases should use binary artifacts from GitHub or build from source. No open source maintainer, corporate-sponsored or not, owes anyone globally distributed infrastructure for free just because some people "really want it".

[michaelklishin]

"Direct" Cloudsmith access will be cut off soon: #15200.

This change should allow us to sync yum*.rabbitmq.com very quickly after a new patch release comes out. The next release is planned for the 2nd half of January, so we will know by then.

[michaelklishin]

@der-eismann just to wrap up: RabbitMQ 4.2.3 has shipped yesterday (Pacific time) and the yum*.rabbitmq.com mirrors were updated the next morning, let's say, within 12 hours.

Thank you for starting this conversation and challenging the status quo :)

[der-eismann]

Highly appreciated, thank you very much!