DNS SERVFAIL in wpad.dat causes freeze up for 30 seconds

same issue as mentioned in bug 213290 and bug 203793 ?

Can you please provide a log-file as mentioned in http://www.mozilla.org/projects/netlib/http/http-debugging.html ?

And can you also post your proxy.pac file, it might contain a DNS-lookup on its own (using isResolvable, isInNet, dnsResolve). Hide your private information if necessary.

Created an attachment (id=274625)
HTTP debug log

Created an attachment (id=274626)
Auto proxy config file

Sanitized hosts/domains and removed repetitive lines (&&, || for additional hosts), but did not remove any unique function calls.

Sorry, I forgot to mention that I disabled IPv6 (network.dns.disableIPv6) and the problem still occurs. (FWIW, the same proxy.pac file worked fine in Firefox before 2.0.0.5 and I didn't have external DNS available then either.)

I could be wrong, but I think there's an error on the following line of your PAC file :

  return "PROXY 192.168.3.1:3128;";

The extra semicolon inside the string shouldn't be there. But it's a bit strange that this causes such a result. But I have seen other weird results with PAC files before.

But I don't see anything wrong in ProcessPACString / ExtractProxyInfo (in nsProtocolProxyService.cpp), I don't think it's a problem.

I removed the semicolon, reloaded the PAC file via Preferences, and the problem still occurs. Also completely restarted Firefox and still have the issue.

If there is anything else you need, let me know. Thanks for looking into this.

I've noticed this also. This is happening even if the proxy.pac consists of the following 3 lines :

 function FindProxyForURL(url, host)
 {
     MYPROXY = "PROXY 10.10.10.102:9191";
     return MYPROXY;
 }

However, I've noticed it only happens on the first or 'start/home' page. Subsequent pages do not perform DNS Lookups. You can confirm this by doing ipconfig /flushdns and then launching mozilla (set to point to the above proxy and then ipconfig /displaydns will show the home pages dns entry).. subsequent pages have the DNS occuring at the proxy where it is supposed to be.

User-Agent: Mozilla/5.0 (X11; U; Linux i686; en-US; rv:1.9.1b1) Gecko/20081007 Firefox/3.1b1
Build Identifier: Mozilla/5.0 (X11; U; Linux i686; en-US; rv:1.9.1b1) Gecko/20081007 Firefox/3.1b1

Firefox (3.0.1-4 and 3.1b1-4) all freeze on websites which have their DNS servers incorrectly setup which going through a proxy server.

Facebook does not have its DNS records correctly set for AAAA and therefore returns SERVFAIL but not instantly, but after 4-5 seconds. Firefox is trying to look up the domain's IPv6 address and hangs while it is waiting.

The bug is two-fold:

1) The JavaScript engine should not be looking up the domain in the first place since it is using a proxy and therefore that will look it up itself.

2) Firefox should not hang while doing DNS queries such that none of the other tabs are available and the window cannot be rendered.

This does not occur on the same network and version on Windows.

Utilising gdb I have found the issue seems to be in the library libxul when it calls pthread_wait_cond. But strace showed the lagging to occur during a DNS query.

We have managed to stop the browser freezing by disabling IPv6 lookups, but it should not be looking the IP up anyway.

Reproducible: Always

Steps to Reproduce:
1. Have IPv6 lookups turned on (network.dns.disableIPv6 = false)
2. Use a proxy server (preferably squid-based to resemble conditions)
3. Navigate to www.facebook.com (from uk)
4. Watch Firefox (3.0.1-3.0.4 and 3.1b1-4) freeze while it tries to get the non-existent AAAA records.
Actual Results:
The browser freezes and the window does not render. It also utilises 100% CPU.

Expected Results:
The browser should remain operable while the DNS query is executing.

The DNS part of the bug is related to 390304 I believe. Although, it still should not lock up the system when doing DNS queries in the JavaScript engine.

I have "auto-detect proxy settings" enabled which is essentially the same thing and can confirm that Firefox is making DNS requests even though HTTP is directed through the proxy (configured from wpad.dat, proxy.pad).

If I use fixed proxy settings (ip:port) in firefox and disable wpad.dat/proxy.pac completely firefox stops makeing DNS requests.

Tested on V 3.0.10 Windohz Vista.

FF 3.5 on Linux also has this problem. When using the option "Use system proxy settings" FF spews out many DNS queries; when setting a manual proxy the DNS is not queried any more...

Platform should be changed to "All".

Timo: if you're talking about the DNS prefetch request (not the regular ones) in an environment where a PAC file or WPAD is used, then it's bug 507578. Prefetch request are made independently from the PAC file (they're not HTTP request at all), but are indeed not sdend when using a manual proxy server. The same thing should be done for a PAC or WPAD too. Or maybe it should be an option, since it's still useful in some environments.

Is there a simple way to tell if the DNS queries I see are prefetch queries or otherwise? The DNS traffic occurred very frequently (every 30 seconds or so) with a stationary FF (did not click on anyting). From your description I guess they must be prefetch...
But to confirm this bug for Linux, what should I look for?

You can disable DNS prefetching by setting network.dns.disablePrefetch to false in about:config (you'll probabaly have to create the preference first).

You can see them in a Wireshark network trace : they're the DNS requests that are done after the page is loaded. But they're no occurring continuously as you describe, it's done only once. If you see continuously DNS requests, then it's might be coming from something else.

The page that generates the traffic is a page of a commercial product that has links to the website of the manufacturer, but my PC has no direct Internet connection. This page refreshes regularly (every 60 seconds)so that is the reason I see it so often.
However, I think that prefetching is still a bit shaky since it appears that for each element that is referenced outside the page a DNS query is done, even if it uses the same host. If the page reloads, all elements are queried again. Looks to me that prefetched URLs are not cached...

There is a DNS cache in Firefox (aside from what is potentially installed in your operating system, like nscd), but it's by default 3 minutes in Firefox 3.5 (1 minute before that).

To change it, go to about:config, create the preference network.dnsCacheExpiration, default is 180 seconds. There's also a total number of entries, network.dnsCacheEntries, default 400.

I guess I have to look into that further; it might be a different bug altogether (but linked to this one, since it disappears when I manually configure a proxy). My (standard) Firefox queries the DNS _each_ minute and tries to resolve the same hostname several times in one birst...
The query frequency corresponds to the refresh interval of the page and the number of times a single host is queried looks to match the number of links to it on the page (not sure though).
Would this happen if the cache should somehow be full? And if so, how do I see how many entries there currently are in the cache?

The lock up is bug 235853, almost certainly.