DNS resolve issue of Google Public DNS for some gTLDs in Asia

Dick Tang
Dick Tang
Published in
3 min readSep 3, 2019
image credit

Update 10:30pm HKT: Google Team are aware of it. and seems it is better now.
Update 4/9: Further details from Public DNS issue : “This problem affected CentralNIC hosted gTLDs throughout the Asia-Pacific region”

Since 12noon HKT (UTC+8), Google Public DNS Server (namely, 8.8.8.8) fail to resolve domain with some new gTLDs including .xyz, .blog, .app, .icu from Hong Kong and Singapore.

For example, DNS result with SERVFAIL instead of proper result (like NXDOMAIN as the following example):

# host a.xyz 8.8.8.8
Using domain server:
Name: 8.8.8.8
Address: 8.8.8.8#53
Aliases:
Host a.xyz not found: 2(SERVFAIL)# host a.xyz 1.1.1.1
Using domain server:
Name: 1.1.1.1
Address: 1.1.1.1#53
Aliases:
Host a.xyz not found: 3(NXDOMAIN)

There are already discussion about this issue on Twitter (https://twitter.com/search?q=google%20public%20dns&src=typed_query&f=live)

After some investigation, some findings are as follows:

  • S̶i̶m̶i̶l̶a̶r̶ ̶i̶s̶s̶u̶e̶ ̶h̶a̶p̶p̶e̶n̶ ̶f̶o̶r̶ ̶m̶o̶s̶t̶ ̶(̶b̶u̶t̶ ̶n̶o̶t̶ ̶a̶l̶l̶)̶ ̶r̶e̶g̶i̶o̶n̶s̶,̶ ̶j̶u̶s̶t̶ ̶n̶o̶t̶ ̶a̶ ̶r̶e̶g̶i̶o̶n̶a̶l̶ ̶i̶s̶s̶u̶e̶
    (previous RIPE Atlas measurements were without Recursion Desired bit accidentally)
  • quite a lot of new gTLD seems also got impacted by this incident.
# curl http://data.iana.org/TLD/tlds-alpha-by-domain.txt | gxargs -i -P 8 host a.{} 8.8.8.8 | grep SERVFAIL | sortHost a.ADAC not found: 2(SERVFAIL)
Host a.ALLFINANZ not found: 2(SERVFAIL)
Host a.ART not found: 2(SERVFAIL)
Host a.BABY not found: 2(SERVFAIL)
Host a.BAR not found: 2(SERVFAIL)
Host a.BASKETBALL not found: 2(SERVFAIL)
Host a.BEST not found: 2(SERVFAIL)
Host a.BH not found: 2(SERVFAIL)
Host a.BLOG not found: 2(SERVFAIL)
Host a.BMW not found: 2(SERVFAIL)
Host a.CAM not found: 2(SERVFAIL)
Host a.CEO not found: 2(SERVFAIL)
Host a.COLLEGE not found: 2(SERVFAIL)
Host a.COOP not found: 2(SERVFAIL)
Host a.DELOITTE not found: 2(SERVFAIL)
Host a.DESI not found: 2(SERVFAIL)
Host a.DESIGN not found: 2(SERVFAIL)
Host a.DHL not found: 2(SERVFAIL)
Host a.DM not found: 2(SERVFAIL)
Host a.DVAG not found: 2(SERVFAIL)
Host a.ETISALAT not found: 2(SERVFAIL)
Host a.FANS not found: 2(SERVFAIL)
Host a.FEEDBACK not found: 2(SERVFAIL)
Host a.FO not found: 2(SERVFAIL)
Host a.FORUM not found: 2(SERVFAIL)
Host a.FRESENIUS not found: 2(SERVFAIL)
Host a.FRL not found: 2(SERVFAIL)
Host a.GD not found: 2(SERVFAIL)
Host a.GENT not found: 2(SERVFAIL)
Host a.HM not found: 2(SERVFAIL)
Host a.ICU not found: 2(SERVFAIL)
Host a.INK not found: 2(SERVFAIL)
Host a.KE not found: 2(SERVFAIL)
Host a.KFH not found: 2(SERVFAIL)
Host a.KPN not found: 2(SERVFAIL)
Host a.KRED not found: 2(SERVFAIL)
Host a.LA not found: 2(SERVFAIL)
Host a.LAW not found: 2(SERVFAIL)
Host a.LIDL not found: 2(SERVFAIL)
Host a.LOVE not found: 2(SERVFAIL)
Host a.LPL not found: 2(SERVFAIL)
Host a.LPLFINANCIAL not found: 2(SERVFAIL)
Host a.MINI not found: 2(SERVFAIL)
Host a.MONSTER not found: 2(SERVFAIL)
Host a.OBSERVER not found: 2(SERVFAIL)
Host a.OOO not found: 2(SERVFAIL)
Host a.PID not found: 2(SERVFAIL)
Host a.PN not found: 2(SERVFAIL)
Host a.POHL not found: 2(SERVFAIL)
Host a.REALTY not found: 2(SERVFAIL)
Host a.REIT not found: 2(SERVFAIL)
Host a.RENT not found: 2(SERVFAIL)
Host a.REST not found: 2(SERVFAIL)
Host a.RUGBY not found: 2(SERVFAIL)
Host a.SAARLAND not found: 2(SERVFAIL)
Host a.SCHWARZ not found: 2(SERVFAIL)
Host a.SFR not found: 2(SERVFAIL)
Host a.SH not found: 2(SERVFAIL)
Host a.SMART not found: 2(SERVFAIL)
Host a.STC not found: 2(SERVFAIL)
Host a.STCGROUP not found: 2(SERVFAIL)
Host a.STORAGE not found: 2(SERVFAIL)
Host a.THEATRE not found: 2(SERVFAIL)
Host a.TICKETS not found: 2(SERVFAIL)
Host a.TUI not found: 2(SERVFAIL)
Host a.VG not found: 2(SERVFAIL)
Host a.VIVA not found: 2(SERVFAIL)
Host a.WIKI not found: 2(SERVFAIL)
Host a.WME not found: 2(SERVFAIL)
Host a.XN--54B7FTA0CC not found: 2(SERVFAIL)
Host a.XN--MGBAAKC7DVF not found: 2(SERVFAIL)
Host a.XN--NGBE9E0A not found: 2(SERVFAIL)
Host a.XN--VERMGENSBERATER-CTB not found: 2(SERVFAIL)
Host a.XN--VERMGENSBERATUNG-PWB not found: 2(SERVFAIL)
Host a.XYZ not found: 2(SERVFAIL)
Host a.ZUERICH not found: 2(SERVFAIL)

Workaround

For the time being, users can switch to alternative nameserver like 1.1.1.1 from Cloudflare.

--

--

Dick Tang
Dick Tang

Director of DevOps Engineering @HK01 . Former @9GAG SRE Engineer . OSS advocator