lists of domains other than .gov and .mil?

[freegovinfo]

I see the seed lists for .gov and .mil, but wondering about .org/.us/.com etc. Is there a way to generate those lists as well?

[jeffersonbailey]

Uh, those lists are old. From pre-crawl research. I hope to get the final (or final as of now) lists up soon.

That said, we have not been separating the lists by .org/.us/.com but instead by "basic" (i.e. http), FTP, and "social media." One could easily take the "basic" list and grep out the .org/.us/.com etc.

[edsu]

I guess they were never added? The scrapes of those sites would be pretty darn useful if they aren't somewhere else by now.

[jeffersonbailey]

We decided to wait until all the data was indexed. We are wrapping up ingesting the data from all the crawling partners now. That should finish this month. At that point we will have a "complete" copy of all EOT data and will be working with folks at the GSA to make (and hopefully maintain) a complete list.

[konklone]

At that point we will have a "complete" copy of all EOT data and will be working with folks at the GSA to make (and hopefully maintain) a complete list.

We're happy to help, just let us know!