Repository Now Limited by GitHub – Here’s Why

[CodeAlDente]

Update on June 17, 2025: GitHub Content Warning Removed & Bandwidth Issue Resolved

Since GitHub has now removed the content warning on this repository and the bandwidth issue has been resolved by Adobe, I’ll go ahead and close this discussion.

That said, I’m currently working on a long-term solution to help prevent similar issues in the future. You can follow the ongoing discussion here: 🔗 Upcoming changes to prevent direct use of raw URLs

Thanks again to everyone for your support and input! 👏 🙏

---

Hi everyone,

Just a quick update for those using this repository.

GitHub has recently placed a content warning on this repo, labeling it as potentially containing "sexually explicit content." To be clear: this repository does not contain any explicit images or media — just a list of domain names meant for use in DNS-based filtering. The list is designed for privacy tools, parental controls, and network filtering — not to distribute or promote explicit content.

Despite this, GitHub has decided to restrict access unless users are logged in and explicitly choose to view the content. I’ve reached out to GitHub to ask for clarification, but so far, this is where things stand.

[georgestephanis]

I'm not sure if this means they've dropped the requirement, however it seems like the raw.githubusercontent.com urls to pull the block/pass lists are loading without authentication.

[CodeAlDente]

You're right — fortunately, the raw URLs were never affected by this. But search results might be. New users who aren’t logged in won’t see this repo when searching, which makes it harder to discover. Also, the warning message is misleading, since this repo (like many others) only provides domain lists, not any explicit content.

I’ve submitted an appeal to lift the restriction, and GitHub is currently reviewing it. Hopefully, they decide to drop it.

[murchelon]

I want to describe a strange symptom we're facing:
We have a SharePoint site, and there is a tool called Netskope that is emitting the message:
"We've blocked the URL xxxxx" — and the URL points to a file named block.txt in this repo.

We don’t believe the SharePoint site is calling this URL from anywhere. We're starting to investigate the issue, but since the repo/URL seems legitimate, I wanted to drop this message here to see if anyone knows something about it or anything related.

Thanks a lot!

[murchelon]

Hi, I totally agree with everything you said.

I believe there is something calling the URL, and then Netskope blocks it. I know for sure that there is no code or anything related involved — it's just a URL with lots of domains to block.
It's probably Netskope itself that's reaching out to this URL. I'll investigate further, and if I find anything new, I'll let you know.

Thanks a lot for the reply!

[CodeAlDente]

It looks like Netskope—or possibly a third-party service they're using—is filtering domains or URLs, and somewhere in that process, they’re referencing the block.txt file from this repo. Netskope supports filtering across multiple content categories, as explained here: https://docs.netskope.com/en/category-definitions/

That said, I still don’t fully understand why you're seeing the URL to the file being blocked on your end—especially if it’s the raw URL. As mentioned earlier, raw URLs (from raw.githubusercontent.com) are not affected by the GitHub content warning and should continue to work as normal.

My guess is that they’re checking the HTTP status of the repo page first to see if it still exists (i.e., returns a 200 OK) before accessing any files. Since the repo is now gated behind a login screen, that check returns a 302 Found (redirect to login), which could cause their logic to treat it as unavailable—even if the raw file is still accessible. Again, that’s just speculation on my part.

To be transparent, I haven’t used Netskope myself, so I can't say for sure whether there’s a setting to bypass this behavior or whether additional logging is available to help clarify what’s being blocked and why.

Since you're a Netskope user, you might want to look up the URL directly using their tool here: https://www.netskope.com/url-lookup

That’s about all I can share at the moment—hope it helps at least point you in the right direction.

[murchelon]

Hi,
We've found the issue! It wasn't my app or SharePoint that was reaching out to your URL.

Initially, it looked like it was my app or SharePoint because it was the first page people opened in their browser, so the request seemed to be triggered there. But in fact, it was Adobe Manager — a Chrome extension that does "Adobe stuff" — that was retrieving your URL.

Anyway, thanks a lot for your reply and help!

[CodeAlDente]

Thanks for the confirmation!

[seanwesbartholomew]

replying here to report a similar issue an confirm the solution was the same for me. I am developing a web app and kept seeing a reference to the raw github link to the block.txt file. the porn-domains in the url made me very suspicious initially haha, but came here and saw @murchelon post the solution that helped them, and checked if I had any adobe extensions. sure enough, I had the acrobat Chrome extension, and removing this extension did away with the error. SMH Adobe!

[clyons032]

Not sure how relevant this is for you but we've been tracking an issue for the last week or so that was tripping some of our content filters and the url for your block list was coming up all the time. After looking high and low i think we might have finally tracked it to the GenAIWebpageBlocklist for Adobe Reader plugin on Chrome. Here's the section of log that led us to here

C:\Users\++REDACTED++\AppData\Local\Google\Chrome\User Data\Profile 2\Extensions\efaidnbmnnnibpcajpcglclefindmkaj\25.5.4 .1_0\content_scripts\SidePanel\GenAIWebpageEligibilityService.js:18:

class GenAIWebpageEligibilityService {
    static THIRTY_DAYS = 2592e6;
    static isReadable = null;
    static shouldShow = null;

    static isEdgeBrowser() {
        return navigator.userAgentData
            ? navigator.userAgentData.brands.some((t => "Microsoft  Edge" === t.brand))
            : navigator.userAgent.includes("Edg/");
    }

    static async isPopupOrApp() {
        const t = await chrome.runtime.sendMessage({ type: "getWindowType" }),
            { isPopup: e, isApp: i } = t;
        return e || i;
    }

    static reset() {
        this.isReadable = null;
        this.shouldShow = null;
    }

    static trackDomain(t, e) {
        window.dcLocalStorage.getItem("allDomainTrack ing") &&
            chrome.runtime.sendMessage({
                main_op: "log-info",
                log: { message: t, domain: e }
            });
    }

    static async fetchExplicitBlockList(t) {
        const e = await fetch(t);
        return (await e.text())
            .split("\n")
            .map((t => t.trim()))
            .filter((t => t.length > 0));
    }

    static async getExplicitBlockList() {
        let t = [];
        const e = new CacheStore("explicit-blocklist");
        if (t = await e.getWithTTL("explicitBlockList"), t && t.length > 0) return t;
        try {
            t = await this.fetchExplicitBlockList("https://raw.githubusercontent.com/Bon-Appetit/porn-domains/refs/heads/master/block.txt");
        } catch (e) {
            chrome.runtime.sendMessage({
                main_op: "log-error",
                log: {
                    message: "Error fetching explicit block list from  GitHub",
                    error: e.toString()
                }
            });
            t = await this.fetchExplicitBlockList("https://acrobat.adobe.com/dc-hosted-extension/block.txt");
        }
        return t && t.length > 0 && await e.setWithTTL("explicitBlockList", t, this.THIRTY_DAYS), t;
    }

    static isSupportedLanguage(t) {
        return ["en-US", "en-GB"].includes(t) || t.startsWith("en");
    }

    static async _shouldShowTouchpoints() {
        if (this.isEdgeBrowser()) return !1;
        await initDcLocalStorage();
        if (!window.dcLocalStorage.getItem("sidePanelRegistered")) return !1;
        const t = chrome.runtime.getURL(""),
            e = window.location.href,
            i = new URL(e).hostname;
        try {
            if ((await this.getExplicitBlockList()).includes(i))
                return this.trackDomain("Page Disqualified - Webpage is in the explicit  blocklist", "re.dac.ted"), !1;
        } catch (t) {
            chrome.runtime.sendMessage({
                main_op: "log-error",
                log: {
                    message: "Error fetching  explicit block  list",
                    error: t.toString()
                }
            });
        }
        if (document.contentType && document.contentType.includes("application/pdf"))
            return this.trackDomain("Page Disqualified - Webpage is a PDF", i), !1;
        if (await this.isPopupOrApp())
            return this.trackDomain("Page Disqualified - Webpage is in a popup or app", i), !1;
        const a = window.dcLocalStorage.getItem("appLoc ale") || chrome.i18n.getMessage("@@ui_locale");
        if (!this.isSupportedLanguage(a))
            return this.trackDomain("Page  Disqualified - Unsupported language", i), !1;
        if (e.startsWith(t) || GENAI_WEBPAGE_BLOCKLIST.find((t => e.includes(t))))
            return this.trackDomain("Page Disqualified - Webpage is in the blocklist", i), !1;
        const s = window.dcLocalStorage.getItem("hideFab DomainList") || [];
        if ((window.dcLocalStorage.getItem("genaiWebpageBlockList") || []).includes(i))
            return this.trackDomain("Page Disqualified - Webpage is in the floodgate blocklist", i), !1;
        if (s.includes(i))
            return this.trackDomain("Page Disqualified - Webpage is in the user blocklist", i), !1;
        const o = await this.isCurrentWebPageReadable();
        return this.trackDomain(o ? "Page Qualified - Webpage is readable" : "Page Disqualified -  Webpage is not readable", i), o;
    }

    static async isCurrentWebPageReadable() {
        await initDcLocalStorage();
        if ("true" === window.dcL ocalStorage.getItem("bypassReadabilityEligibility")) return !0;
        if (null !== this.isReadable) return this.isReadable;
        const t = chrome.runtime.getURL("resources/SidePanel/sidePanelUtil.js"),
            { isProbablyFirefoxReaderable: e } = await import(t);
        return this.isReadable = await e(document, document.URL), this.isReadable;
    }

    static async shouldShowTouchpoints() {
        return null !== this.shouldShow || (this.shouldShow = await this._shouldShowTouchpoints()), this.shouldShow;
    }
}

[CodeAlDente]

Thank you for providing the logs and additional information.

I've formatted the code from your comment to improve readability and redacted personal info like usernames and paths where needed — just to keep things clean and avoid exposing anything sensitive.

The code you posted is from a Chrome extension (likely part of Adobe Acrobat), and what it does is pretty straightforward: it tries to fetch a domain blocklist from this repo (block.txt) and uses it to decide whether to show certain UI features, like a sidebar or assistant. If the domain of the current webpage matches an entry in the list, the page gets “disqualified,” meaning the feature won't appear.

There's no issue with the code itself — it's using caching, handles errors, and even falls back to a different URL if the GitHub fetch fails.

That said, I want to clarify a common point of confusion:

The raw GitHub URL (e.g., https://raw.githubusercontent.com/Bon-Appetit/porn-domains/refs/heads/master/block.txt) has always been accessible and is not affected by the restrictions GitHub added to the repo’s web view. Only the repo page itself (the normal GitHub site) requires login now and shows a content warning — the raw file links still work fine for automated tools like this one.

Interestingly, I checked the fallback URL (https://acrobat.adobe.com/dc-hosted-extension/block.txt) and it currently returns a 403 Forbidden, so if GitHub access fails, the backup won’t work either.

Hope that clears things up a bit — let me know if you find anything else!

[murchelon]

Not sure how relevant this is for you but we've been tracking an issue for the last week or so that was tripping some of our content filters and the url for your block list was coming up all the time. After looking high and low i think we might have finally tracked it to the GenAIWebpageBlocklist for Adobe Reader plugin on Chrome. Here's the section of log that led us to here

C:\Users++REDACTED++\AppData\Local\Google\Chrome\User Data\Profile 2\Extensions\efaidnbmnnnibpcajpcglclefindmkaj\25.5.4 .1_0\content_scripts\SidePanel\GenAIWebpageEligibilityService.js:18:

OMG, that's EXACTLY what we discovered! It was the GenAIWebpageEligibilityService.js. In the newer version of their extension, this doesn't happen anymore.

[seanwesbartholomew]

lol same here on everything mentioned. acrobat was the culprit for me.

[CodeAlDente]

Hi everyone,

I wanted to share a quick update on the situation. GitHub got back to me about a sudden spike in downloads of files from the repo that began in late May. They’ve flagged it under their “Excessive Bandwidth Use” policy and warned that if we can’t identify and address the cause, the repo might be throttled or even disabled.
I replied to GitHub explaining that it looks like an Adobe Chrome extension (Chrome Web Store: Adobe Acrobat: PDF edit, convert, sign tools) is programmatically fetching our raw block.txt URL at scale. I’ve asked them if they can share any specifics, like request metrics or IP ranges, to help narrow down what’s happening exactly. I also took the opportunity to question why the content warning is still in place, since our repo only contains plain domain lists and no explicit media.

At the same time, I reached out to Adobe’s Chrome extension support to let them know their extension appears to be hitting our GitHub raw URL heavily. I suggested they could bundle a local copy of the blocklist in the extension or implement better caching to avoid repeated requests, and I mentioned our upcoming license change (we plan to move future releases to a non-commercial, share-alike license, more details here).

I’m watching for replies from both GitHub and Adobe. As soon as I have more information, I’ll post another update here.

[ryanmathis]

Seeing a 404 in console isn’t out of the ordinary. Seeing a 404 with the word porn in it that points to some random dudes github project….well, lmao.

Crazy story though, you should post this somewhere once it’s resolved. Your console famous.

[CodeAlDente]

Just a quick follow-up: Adobe resolved the underlying cause in their extension update (v25.6.1.3), released on June 16, 2025 — specifically by removing the direct call to the block.txt raw URL.

[Morl99]

Seeing a 404 in console isn’t out of the ordinary. Seeing a 404 with the word porn in it that points to some random dudes github project….well, lmao.

Crazy story though, you should post this somewhere once it’s resolved. Your console famous.

I came here to pretty much write exactly that comment. Spot on!

[seanwesbartholomew]

for awareness, I'm relatively sure my acrobat plugin was up to date and still ran into the 404 today due to acrobat plugin

[CodeAlDente]

for awareness, I'm relatively sure my acrobat plugin was up to date and still ran into the 404 today due to acrobat plugin

That would be quite interesting to investigate. Could you provide the version and the extension used?

As far as I know, the reference was removed in version 25.6.1.3 on June 16, 2025. See below:

./content_scripts/SidePanel/GenAIWebpageEligibilityService.js

--- Version 25.6.1.1
+++ Version 25.6.1.3
@@ -18,6 +18,7 @@
    }
    static async fetchExplicitBlockList(t) {
        const e = await fetch(t);
+        if (!e.ok) throw new Error(`Failed to fetch block list from ${t}: ${e.statusText}`);
        return (await e.text())
            .split("\n")
            .map((t) => t.trim())
@@ -25,15 +26,14 @@
    }
    static async getExplicitBlockList() {
        let t = [];
-        const e = new CacheStore("explicit-blocklist");
-        if (((t = await e.getWithTTL("explicitBlockList")), t && t.length > 0)) return t;
        try {
-            t = await this.fetchExplicitBlockList("https://raw.githubusercontent.com/Bon-Appetit/porn-domains/refs/heads/master/block.txt");
-        } catch (e) {
-            chrome.runtime.sendMessage({ main_op: "log-error", log: { message: "Error fetching explicit block list from GitHub", error: e.toString() } }),
-                (t = await this.fetchExplicitBlockList("https://acrobat.adobe.com/dc-hosted-extension/block.txt"));
+            const e = new CacheStore("explicit-blocklist");
+            if (((t = await e.getWithTTL("explicitBlockList")), t && t.length > 0)) return t;
+            (t = await this.fetchExplicitBlockList("https://acrobat.adobe.com/dc-hosted-extension/block.txt")), t && t.length > 0 && (await e.setWithTTL("explicitBlockList", t, this.THIRTY_DAYS));
+        } catch (t) {
+            chrome.runtime.sendMessage({ main_op: "log-error", log: { message: "Error fetching explicit block list from Adobe", error: t.toString() } });
        }
-        return t && t.length > 0 && (await e.setWithTTL("explicitBlockList", t, this.THIRTY_DAYS)), t;
+        return t;
    }
    static isSupportedLanguage(t) {
        return ["en-US", "en-GB"].includes(t) || t.startsWith("en");
@@ -45,11 +45,6 @@
        const t = chrome.runtime.getURL(""),
            e = window.location.href,
            i = new URL(e).hostname;
-        try {
-            if ((await this.getExplicitBlockList()).includes(i)) return this.trackDomain("Page Disqualified - Webpage is in the explicit blocklist", "re.dac.ted"), !1;
-        } catch (t) {
-            chrome.runtime.sendMessage({ main_op: "log-error", log: { message: "Error fetching explicit block list", error: t.toString() } });
-        }
        if (document.contentType && document.contentType.includes("application/pdf")) return this.trackDomain("Page Disqualified - Webpage is a PDF", i), !1;
        if (await this.isPopupOrApp()) return this.trackDomain("Page Disqualified - Webpage is in a popup or app", i), !1;
        const a = window.dcLocalStorage.getItem("appLocale") || chrome.i18n.getMessage("@@ui_locale");