Filter out expired domains

[lynt-smitka]

Many domains from this list are expired and not exist now.

There are several methods to filter them out:

1. whois - extremely rate limited

2. DNS A - problem with registered but non-active domains

3. DNS SOA serial - there are 2 common methods how to generate serials: YYYYMMDD## or Unix Timestamp of change - if you ask DNS server for a non-existing domain, it will return SOA with the current/cached timestamp

Any other suggestions?

[jsmitka]

I guess that authoritative servers should return a NXDOMAIN on NS and/or SOA query for expired domains. Or is there any case when this check fails?

[spaze]

.cz domains might not be in the zone and yet still be registered (similar to 2. "DNS A"), so it's impossible to have a list of just non-expired domains.

And if we'd have a list of registered domains, then some people would love to have a list of domains with A records. Others a list of domains responding on HTTP port.

So I'm inclining to Let's have a list so people can do whatever they want with it, Although it doesn't fix this issue, because it won't filter out expired records :-)