curl -s -o /dev/null -w "%{http_code}" $URL will return something other than 200 if the page doesn't exist. That will enable programmatic checking so we don't have to actually look at every site (although it might be good to do that as a second step).