Upcoming changes to prevent direct use of raw URLs

[CodeAlDente]

This is a follow-up to the earlier discussion about the excessive bandwidth usage and GitHub’s temporary content warning. That issue has been resolved. Below, I’ll explain what changes are planned to help prevent something like this from happening again. You can find the original discussion here: 🔗 Repository Now Limited by GitHub – Here’s Why

---

To prevent direct use of static raw URLs (like block.txt) and avoid the kind of excessive traffic we recently experienced, the blocklist will soon be renamed dynamically based on a random hash — for example: block-[HASH].txt. This ensures that any update results in a new file path, making it harder to hardcode.

To keep things compatible with automation, a new meta.json file will be introduced. It will provide the current filenames programmatically using a simple structure like:

{
  "blocklist": "block-[HASH].txt",
  "allowlist": "allow-[HASH].txt"
}

Applications can fetch this file and extract the correct filenames to update blocklists without needing to guess or hardcode URLs. At the same time, this adds a layer of indirection that helps mitigate the problems caused by unthrottled automated access.

Updates are typically expected once a day — so fetching or refreshing more frequently than that (e.g. every few hours or minutes) should be avoided to keep load manageable and prevent unnecessary bandwidth usage.

In addition, pass.txt will be renamed to allow.txt to better match the naming convention and make the file’s purpose more intuitive.

Lastly, I’m taking this opportunity to rename the default Git branch from master to main in line with current best practices and modern Git/GitHub conventions.

These changes are planned for later this week and automatic updates of the lists will also resume then.

References:

• https://github.com/Bon-Appetit/porn-domains/blob/main/docs/CHANGELOG.md#2025-06-17
• https://github.com/Bon-Appetit/porn-domains/blob/main/docs/CHANGELOG.md#2025-06-20

[CodeAlDente]

The default branch has been renamed from master to main. To update your local repository, run the following commands:

git branch -m master main
git fetch origin
git branch -u origin/main main
git remote set-head origin -a

[CodeAlDente]

The changes have been applied. An update will now be triggered to test if everything works.

[CodeAlDente]

Everything seems to be working now. I'll monitor the results over the next few days.

[CodeAlDente]

Everything looks good so far, closing this for now.