domains no longer containing porn

[dzek69]

i'm looking for something to implement into my app, so i quickly verified random domains, and choosing atrandom around 50% of domains are no longer porn

amateurofficesex.com is for sale
chopstick16.com is a chinese hospital webpage?
club-geheimtipp.ch returns 403
fotosdechicasguapas.com is for sale

it's nice you verify the dns, but maybe some dumb webpage content analysis would be nice as well?

catching non-200 responses and domains for sale should be quite easy and should help drastically reduce the list (which is good for performance when the list is actually used)

[CodeAlDente]

You are right. Not all of the domains listed here are porn domains (anymore). Of course additional checks could be run to analyze them more in detail. But there are some problems with it:

For sale detection:
There is no universal phrase / keyword to determine if a domain is for sale. There could be ads on the website which say "for sale" or something like that. It would then result in a false positive.

Fetching HTTP status code:
Webmasters could easily block the requesting IP address and further requests would then be denied, e.g. with 403 Forbidden. This would also result in a false positive.

[spirillen]

Maybe you should stick your head together here: funilrys/PyFunceble#255 where @keczuppp have asked the same question.

[CodeAlDente]

We now exclude outdated sources to minimize false positives as much as possible, see: #43.

Additionally, there is an ongoing discussion about not only verifying domain availability through DNS checks but also determining whether the associated nameservers belong to domain parking services, see: #47. This would allow us to detect parked domains or domains currently for sale. However, such domains may contain ads related to the domain name or previous content associated with adult material, which would classify them as websites with adult content regardless.