Base mirror functionality broken after migration to AWS S3 blob storage

[alvsan09]

Base mirror functionality broken after migration to AWS S3 blob storage

Description

The initial implementation of the openvsx mirror mode was introduced and documented via issue #505. The solution relied on keeping the openvsx Azure Blob storage centralized so any mirror
could download the vsix packages from this central location.

Problem

Recently in November, the blob storage for openvsx has been migrated from Azure Blob to AWS S3, and blobs are now accessed via a centralized URL from Eclipse Foundation: https://
openvsx.eclipsecontent.org/

However:
• There are no instructions on how an openvsx mirror instance should be reconfigured to use the AWS storage
• The latest source code of the mirror does not reflect the changes needed to make it work as per the initial implementation or as a new alternative option
• The migration appears to have broken existing mirror functionality

Expected Behavior

Mirror instances should be able to download vsix packages from the centralized storage location and function as documented in the original implementation.

Actual Behavior

Mirror functionality is broken due to the storage migration, with no clear migration path or updated documentation.

Additional Information

• I missed the notification of this change and it caught me by surprise
• Could you provide a reference to the notification of this change?
• What communication channel should we subscribe to for future notifications about such breaking changes?

References

• Original mirror implementation: #505
• Previous Azure Blob endpoint: https://openvsxorg.blob.core.windows.net/
• New Eclipse Foundation endpoint: https://openvsx.eclipsecontent.org/

[netomi]

The existing mirror mode has been flawed in the sense that the mirror needs to know internals of how the actual file resources are stored. In the case of Azure that was not a problem as the final storage location can be derived from publicly known information, thus setting up a fake azure storage location works.

For AWS that is not possible. We use now a CDN front to access file resources on AWS S3, but the code for that is not merged yet. With that merged it should be possible again to configure your mirror so correctly serve files, though in my opinion thats not how a mirror should work, it should have no intrinsic knowledge about the storage location but rather use the information provided to him by the upstream instance.