firefox spends lots of time hung

Created an attachment (id=124913)
the offending atuomatic configuration script.

Attached the automatic configuration script.

yikes. that is one bloated PAC file.

I would guess that you are dying in the javascript execution.

What exactly are you typing. For example, a hostname that does not exist
(nonexist.mozilla.org)?

Darin: is there a better way of isolating this?

If i type :
http://a
evertything goes ok. (it says correctly host not found)
if i type
a
evertything goes ok. (it says correctly host not found)
if i type
a.
The browser freezes.
Note that i removed the "automatically add "www" and ".com" setting.

I did some testing with my pac script.If i modify the script and remove the
IsInnet() part in the start it works as expected: (Host a. not found in a popup)
So it freezes on isinNet.

The bug stays: the PAC script should not freeze the entire browser, only the
current connections.

Later i found out this bug is NOT ALWAYS reproducable. It has to do with
something in out network that resolves (??DNS queries??) slowly. When i files
this bug it was always reproducable. Later (with the same mozilla installation)
mozilla only frooze for half a second.

yeah, the problem is that IsInNet function needs to synchronously resolve the
given hostname, so Mozilla ends up locking up the main thread (the thread which
handles browser UI) until the DNS query completes. Since DNS queries can have
relatively large timeouts (on the order of a minute), this can make the user
think the browser has completely locked up. Unfortunately, there is little we
can do about this problem, with the exception that it would be nice if
synchronous DNS queries could be canceled by the user. We could put up a modal
dialog with some sort of status message and a cancel button (if the DNS query
was taking more than X seconds). Such UI work isn't going to happen for
sometime unfortunately.

I don't know if we should do a lot to encourage the use of DNS-calling PAC
functions. Ari Luotonen, the designer of PAC, had said on occassion that he
wanted to end-of-life these functions because the failure conditions caused
excessive delays. Many PAC file writers do not understand DNS well enought to
understand the pitfalls. I saw one large bank call isResolvable() for every
request behind a internal-only DNS server. (If the hostname is not resolvable,
THEN go to the proxy server...)

Probably some better documentation on writing PAC files would be a more useful
solution.

hmm... i see your point, but since it is part of the PAC spec, and since it is
widely used, it would seem like a good idea to do whatever we can to make the
browser UI not hang. i don't think it would be too much work to implement, but
i also don't think it is critical enough to get high priority either. probably
this bug will just sit around for a while until someone has time to implement
it. as for the documentation approach, that should still happen. but, getting
information out to everyone is not exactly trivial either.

Duplicate of bug 174733
DNS: hangs on lookups and at various stages of making tcp connections

This bug shouldn't just be considered a straight duplicate of bug 174733. It is
true that both bugs stem from the same problem: very poor single-threaded DNS
design, locking up the entire browser during a DNS query. As others have
pointed out, this is hopefully being rewritten for 1.6.

I believe that this bug should remain open to address the PAC issue. If this
issue magically goes away when DNS is fixed in 1.6, that will be great, but this
bug should stay open until then.

User-Agent:
Build Identifier: Mozilla/5.0 (X11; U; Linux i686; en-US; rv:1.7b) Gecko/20040225

This bug has been split from #224447 as the symptoms are different. I will copy
relevant comments from #224447 into this bug report.

Reproducible: Always
Steps to Reproduce:
1. Go to Preferences, Advanced, Proxies
2. Select "Automatic Proxy Configuration URL"
3. Enter a URL to a valid PAC file
4. Click OK
5. Browse a website (particularly one with many offsite adds/images)

Actual Results:
DNS resolution causes the UI to hang

Expected Results:
DNS resolution should not cause the UI to hang (it doesn't when using either
"Direct connection to the Internet" or "Manual Proxy Configuration")

Copied from bug #224447

 ------- Additional Comment #6 From mike 2004-01-20 17:17 PST [reply] -------

Also experiencing this. Firebird 0.8+ MacOS X
Resolving Host takes about 5 seconds for any page with a new DNS. For example,
if i go to www.google.com, it will hang for a few secs at resolving host before
loading, but then anything i do within google will be snappy. But, say i click
a link from google to another site, it will hang for another few seconds until i
can get into that new site's DNS.

------- Additional Comment #7 From <email address hidden> 2004-02-17 05:38 PST
[reply] -------

This ought to be renamed to "Complete hang when DNS resolving/lookup"

I see this too, but not always (local site dns cacheing?) -- and experienced
this also when using Mozilla.

Note: IE does not hang -- that is, its menus etc still function, it can redraw
etc while the lookup occurs.

(Firefox 0.8, Win2K, via a proxy squid/2.5.STABLE3)

------- Additional Comment #9 From Stewart Jeacocke 2004-02-25 06:30 PST [reply]
-------

Mozilla/5.0 (X11; U; Linux i686; en-US; rv:1.7b) Gecko/20040224

As of 23/02/04 I'm still seeing this bug on the SeaMonkey CVS Tree. The UI
(every windows and tab) is blocked while name resolution is performed. This is a
real PITA particulary for sites which have many adds etc. For example it can
take over 30s for http://www.ebay.co.uk to load during which time the UI is
completely hung.

This bug seems to be due to two things
i) The UI is blocked during host resolution
AND
ii) Host resolution in mozilla is sometimes very slow, even when the dns names
have already been cached at a local DNS server. For example I used the "host"
command to resolve all the names required for www.ebay.co.uk and none of them
took longer than 0.05s to resolve

------- Additional Comment #10 From Stewart Jeacocke 2004-02-25 06:42 PST
[reply] -------

This bug is not specific to Firefox. Would it be a good idea to change "Product"
to something other than FireFox?

I've stuck some cout statements at the beginning and end of nsDNSService::
Resolve() [nsDNSService2.cpp] and can confirm that we are inside this method
when the UI hangs.

I'm willing to help track down the cause of this bug (as its making me have to
use Opera ;-) ) but I have no experience with the internals of mozilla so would
need guidance as to where to start looking.

------- Additional Comment #11 From Stewart Jeacocke 2004-02-25 12:56 PST
[reply] -------

It seems that this bug is triggered by something in my preferences. I moved my
.mozilla directory out of the way and let mozilla create a new one. This caused
the bug to disappear (at least on current CVS and mozilla 1.6).

When using my old .mozilla dir (the config the elicits this bug) the method
nsDNSService::Resolve() is being called lots, however when I let mozilla create
.mozilla from scratch nsDNSService::Resolve() is never called and is replaced by
calls to nsDNSService::AsyncResolve()

------- Additional Comment #12 From Stewart Jeacocke 2004-02-25 14:30 PST
[reply] -------

The offending preference is enabling Proxy Auto Config (PAC).

When I enable PAC (set to use a valid remote PAC file) the
nsDNSServic...

Confirming on Mozilla/5.0 (Windows; U; Windows NT 5.0; en-US; rv:1.6)
Gecko/20040206 Firefox/0.8

This is a dup of a bug that was WONTFIX'd about a year and a half ago. The
problem is that DNS resolution run in a PAC file is done in the UI thread,
whereas all normal DNS resolution is done in the network thread itself. I'll
look up the original bug; I seem to remember it being an issue of ugly
architectural issues.

It was bug 97605 (the actual resolution was invalid). The "workaround" is not
to use PAC scripts that use dnsResolve().

The PAC script in question doesn't use dnsResolve(). But it does use
isResolvable() which I guess may be the culprit. Here is the PAC function

function FindProxyForURL(url, host)
{
  //So the error message "no such host" will appear through the
  //normal Netscape box - less support queries :)
  if (!isResolvable(host))
    return "DIRECT";

  if (isPlainHostName(host) ||
      dnsDomainIs(host, ".lan") ||
      dnsDomainIs(host, ".jeacocke.org.uk"))
    return "DIRECT";
  else
    return "PROXY hadron:3128; DIRECT";
}

Suggest Keywords: hang nsenterprise verified1.5 verified1.6 clean-report

This happens on dnsDomainIs calls as well. Even simple calls such as:
  dnsDomainIs(host, ".domain.local")
will cause the UI to freeeze for several seconds. Source of the proxy.pac is
not an issue.

Test proxy.pac:
function FindProxyForURL(url, host)
{
  if (dnsDomainIs(host, ".domain.local"))
    return "DIRECT";
  else
    return "PROXY 127.0.0.1:8118; DIRECT";
}

Darin landed a re-write, effective in 1.6a, so 1.6 and later should perform
better in some regards.

There are still several ways bad PAC writting can slow down your system.

(In reply to comment #10)
> Darin landed a re-write, effective in 1.6a, so 1.6 and later should perform
> better in some regards.
>
> There are still several ways bad PAC writting can slow down your system.

This still happens with 1.6 and 1.7b: use of dnsDomainIs results in complete UI
freeze of 5-10 seconds (non updating windows and background, no response from
mozilla task). So whatever Darin changed, it unfortunatly did not affect this.

System test done on WinXP, 2,8 GHz, 512 MB no other apps running. Tested with
Mozilla 1.6 and 1.7b as well as Firebird 0.8. Removal ov dnsDomainIs(host,
".domain.local") in the "if" call removed the UI freeze problem.

fixed blocks, added depends.

isResolvable() calls dnsResolve()

http://lxr.mozilla.org/mozilla/source/netwerk/base/src/nsProxyAutoConfig.js#267

267 "function isResolvable(host) {\n" +
268 " var ip = dnsResolve(host);\n" +

Chase, is this a won't fix?

I'll leave that for someone from Networking to decide, but I think it's a likely
outcome. I'd really like to see it fixed, but last time it was discussed, it
seemed like an arch problem in the worst way.

Duncan:

Thinking about this further, PAC probably cannot send more than one DNS query at
a time (based on the nature of the script activity). The improvement from the
re-write might have been minimal.

However, I think this bug has started to drift. dnsDomainIs() should not cause
the types of delays that isInNet() would, dnsDomainIs() does some string
comparisons. I've started running my mozilla sessions w/ PAC files for
diagnostic testing, and I have not seen delays as you have described. If this
persists in a 1.7b or later build, lets open a new bug.

The reason I'm asking is that I'm about to roll up some network admin docs for
PAC, so if you mark this wontfix, I'll use that as the final state of the bug.

*** Bug 207652 has been marked as a duplicate of this bug. ***

Finally I've tracked down the same problem on my Linux box here. I'm using
Mozilla, Galeon and Epiphany, all suffer the same long delay. IT is quite good
demonstratable on EBay when loading large list of offered articles.

As soon as I remove the isInNet() call from the PAC everything works fine again.
With that call the UI most times (! but not always) freezes for up to nearly 1
minute.

As described below, this seems to come from a synchronous DNS lookup. I can't
understand why waiting for a new page to come blocks the whole UI? At least
since tabbed browsing is available, I thought every page is loaded and displayed
by a seperate thread. And this should eliminate this problem. Even if DNS lookup
is synchronous.

Even if "using isInNet()" is not good practice and the warning for an DNS lookup
for every URL is in the documentation, "isInNet()" is a very handy function and
should not be removed or discouraged. For _every_ URL and every img tag a DNS
lookup is necessary (probably by the Proxy-Server). So it is no problem to do a
DNS lookup by the browser too. BTW: are the results from isInNet() cached somewhere?

The problem is here for Kernel 2.4 and 2.6. (BTW: I've enabled IPv6). I've
installed the mozilla-browser 1.6-5 Debian package (and others).

> I thought every page is loaded and displayed by a seperate thread

the layout code is not threadsafe.

*** Bug 248545 has been marked as a duplicate of this bug. ***

*** Bug 233366 has been marked as a duplicate of this bug. ***

Are there any workarounds for this? (For example, changing a setting so that the
resolution will give up sooner)

This bug is killing me!!! Simply type an URL that won't resolve and all tabs are
locked permentatly. I need to kill process and start all over again. Easily
repeatable on XP running Firefox 1.0

There are some screwing things going on with DNS here, can we please get this
fixed before I'm forced to abandon this browser

Jason: I was reading DNS&BIND the other day for work, and I think some very
modern BIND resolvers allow system-level timeout settings.

The other question is: why are you using this in the PAC file. When I supported
PAC for Netscape, I found only one customer that did this for a good reason, and
that was predicated on a completely bad DNS subdomain strategy that they decided
to re-do, based on this bug.

Confirm this problem occurs in FF 1.0 release under both XP and Gentoo ith FF 1.0.

The same effect is not noticed in IE, although obviously with it not using tabs
that could just be me not noticing it due to the expectation of a crap UI.

In my case I am using the automagically detect my proxy setting, rather than the
PAC file. The result is the same in that the UI appears to die toally :(

benc in comment 11:

> The other question is: why are you using this in the PAC file.

VPN. Specifically, using isInNet(host,subnet,mask) to proxy only certain
address blocks, the inverse of "No Proxy for" that can't easily be accomplished
any other way.

To see the full effect of this bug, by the way, try browsing around
http://www.myspace.com/ for a while with a few tabs open. Ouch.

Firefox 1.0.4 on Linux, ditto for recent trunk builds.

*** Bug 267023 has been marked as a duplicate of this bug. ***

I find no trick in the script in attachment.
See bug:240759 (maybe not the right bug cannot find it there were 2 versions of
the patch :(
It is a DNS cache problem.
I bet there is a famine problem in the fifo algorythm and then even host
resolution is affected that should be why proxies are affected too.
Should I ask for bug:240759 to be reopened or should i fill a new bug report.
I can just try and unbug bug:240759 's patch.
The problem has occured since bug: have been closed and is apparent because of
stability of 1.7 braunch and last trunk of mozilla suite

*** Bug 298310 has been marked as a duplicate of this bug. ***

A bit surprised that bug 298310 has been marked as a duplicate of this bug - at
least my report was about browser hangs independent of whether it was a valid
DNS address or not - only by using IP's seemed to "dodge" the problem. But hey,
you are the guys plowing the source :-). Just wanted to comment that, if this
action perhaps was an error.

> independent of whether it was a valid DNS address or not

to find out whether it's valid, a DNS request has to be made, no? and that is
what hangs the browser.

User-Agent: Mozilla/5.0 (Windows; U; Windows NT 5.1; en-US; rv:1.7.12) Gecko/20050915 Firefox/1.0.7
Build Identifier: Mozilla/5.0 (Windows; U; Windows NT 5.1; en-US; rv:1.7.12) Gecko/20050915 Firefox/1.0.7

When typing nonexistent address to location bar Firefox 1.0.7 freezes. This
happened also with version 1.0.6.

Reproducible: Always

Steps to Reproduce:
1.When typing nonexistent address to location bar Firefox 1.0.7 freezes
2.
3.

Actual Results:
Firefox freezes.

Expected Results:
"page not found" or something like that.

Mozilla/5.0 (Windows; U; Windows NT 5.1; en-US; rv:1.9a1) Gecko/20050922
Firefox/1.6a1 ID:2005092205

WFM. Does it also happen in the safe mode?
http://kb.mozillazine.org/Safe_Mode_(Firefox)

(In reply to comment #0)
> User-Agent: Mozilla/5.0 (Windows; U; Windows NT 5.1; en-US; rv:1.7.12)
Gecko/20050915 Firefox/1.0.7
> Build Identifier: Mozilla/5.0 (Windows; U; Windows NT 5.1; en-US; rv:1.7.12)
Gecko/20050915 Firefox/1.0.7
>
> When typing nonexistent address to location bar Firefox 1.0.7 freezes. This
> happened also with version 1.0.6.
>
> Reproducible: Always
>
> Steps to Reproduce:
> 1.When typing nonexistent address to location bar Firefox 1.0.7 freezes
> 2.
> 3.
>
> Actual Results:
> Firefox freezes.
>
> Expected Results:
> "page not found" or something like that.

This also happens in safe mode.

Can those of you who experienced this bug or are still experiencing this bug, please comment about the behavior with FF 1.5? I did some work on FF 1.5 that should have improved things for the case where the DNS query is made for the hostname passed to FindProxyForURL. Thanks!

A resolution for this is crucial especially for enterprise usage where such a proxy configuration is much more common than at home. I urge you to look into this issue for the upcoming enterprise version of firefox (http://blog.mozilla.com/blog/2012/01/10/delivering-a-mozilla-firefox-extended-support-release/).

It would be a huge improvement for us in Firefox, in fact the biggest in years. Thanks!

(In reply to Benjamin Scherrer from comment #227)
> I urge you to look
> into this issue for the upcoming enterprise version of firefox
> (http://blog.mozilla.com/blog/2012/01/10/delivering-a-mozilla-firefox-
> extended-support-release/).
It's no longer upcoming, it has already been released.

Thanks, I missed the news. So any hopes for progress here? I really hoped it would make this release.

All, sorry for the lack of responses on this one. There have been some other higher priority tasks for various reasons, but this seems to be next on my list of bugs. I have other tasks as well, though, so I can't devote all my time to it. I hope to have a patch up sometime in the next month.

Well "hopefully next month" is pretty awesome compared to the (almost exactly) 8 years that this bug has been around...

Created attachment 594181
"[PAC] Defer proxy resolution for HTTP and HTTPS PAC to avoid blocking main thread during DNS resolution" [a-1.9.1b2=mbeltzner]

I rebased this patch and I'm running it through try to see what the state of the world is.

Created attachment 594233
Defer proxy resolution for HTTP and HTTPS PAC to avoid blocking main thread during DNS resolution.

Try run for 5fa326c23268 is complete.
Detailed breakdown of the results available here:
    https://tbpl.mozilla.org/?tree=Try&rev=5fa326c23268
Results (out of 99 total builds):
    exception: 4
    success: 39
    warnings: 16
    failure: 40
Builds (or logs if builds failed) available at:
http://<email address hidden>

Try run for 55a64e81467b is complete.
Detailed breakdown of the results available here:
    https://tbpl.mozilla.org/?tree=Try&rev=55a64e81467b
Results (out of 129 total builds):
    success: 114
    warnings: 15
Builds (or logs if builds failed) available at:
http://<email address hidden>
 Timed out after 06 hours without completing.

That try run was completely green, modulo a couple unrelated known oranges. Jason/Biesi (or Steve, for that matter), does this patch require another round of review? The rebasing I performed was mostly mechanical.

Comment on attachment 594233
Defer proxy resolution for HTTP and HTTPS PAC to avoid blocking main thread during DNS resolution.

Steve is going to look the patch over.

Jdm, thanks for cleaning up the patch and trying it out. Looks good to me - no issues.

http://hg.mozilla.org/integration/mozilla-inbound/rev/20240add1d35

Gentleman thank you. It looks like this bug really should be fixed now..? if it gets into next FF release, then it would deserve it's own release party. Is it not the oldest bug ever fixed?

It took Mozilla 8 (verbal: E-I-G-H-T!) years to fix it.

Fixing TNEF attachments in Thunderbird is another sad example for time-consuming bugs and hence is still not fixed.
Maybe fixing proxy resolution will get fixed soon, too?! :)

Many thanks for looking at this issue. Much appreciated

https://hg.mozilla.org/mozilla-central/rev/20240add1d35

*** Bug 425146 has been marked as a duplicate of this bug. ***

Is this fixed in firefox 12 ? It still hangs if i try to use PAC.

Per the Target Milestone field above, it is fixed for Firefox 13. You can verify that it is fixed with a current beta release.

However, as we discovered in bug 750445, this may not actually be fixed in FF 13 either. More investigation is probably required.

Yes, this is still broken. I filed bug 750445 instead of reopening this one, because the patch for this one stuck (and shouldn't be backed out: we just need to fix it). Let me know if that's the wrong approach bugzilla-administration-wise.

That's absolutely the best practice, don't let anyone tell you otherwise :)

Making everyone who's been following this bug for the past 8 years jump through hoops. Oh well.

Unfortunately it doesn't really make sense for us to optimize our Bugzilla practices for the casual bug watcher audience (Bugzilla really wasn't designed for that). Trying to track several different changes in a single bug just doesn't work very well.

I suspect that many people on the CC list have since lost interest in getting frequent updates, but those who want to track every step along the way can still CC themselves to the new bug.

And apologies for this bug not getting fixed--it was an accident in the rebasing of a patch. For all we knew this was fixed until we discovered otherwise last week. I'll be looking into it very soon.

No worries, we all moved to Google Chrome anyway.
But I think I'll stay at the CC list for a while: every time I get an email notification about this defect not being fixed, it reassures me that I made a good decision.

(In reply to Avner from comment #253)
> No worries, we all moved to Google Chrome anyway.

I have to confirm this - at least for my colleagues.

Mozilla is way too good (that video about "we believe in freedom" still makes me cry - just beautiful!) to act like this.
Come on guys, step up and get stuff like x84 builds in the release channel done; it's about time!

Remember: its not about the "under the hood", it's about what changes usability and recognizable changes to the user - everything else is just for geeks, not end-users.

And this is definitely true: Firefox is the better browser, because it respects user's privacy like no other. So get that "nerd touch" destroyed by moving faster.

I don't care what goes on under the hood, and I don't care how the proxy server is being resolved technically.
As an end-user, when my browser hangs I just kill it and move to a faster browser.

That's what I tried to say - thanks for summing it up! :)

Indeed. In fact several years ago it was exactly this bug that forced me to move to Chrome. I don't work in the huge company with complex proxy server configs anymore, yet.. I am on Chrome already. Keeping Firefox only for a couple of Chrome-incompatible sites and following this bug out of nostalgia.

Guys, I know this has been a source of frustration for you, but please show some courtesy to the 103 people CCed to this bug by not spamming them with your reasons for switching to Chrome.

This is my last comment, because I want to respect Ryans kind wish to show some courtesy:

I'm not on Chrome, I'm on Firefox (even Nightly x64 right now) - since Version 1.0, at work, too, even on Android and until it really brakes the basic rules, I guess.

So, finally: sorry to those 103 people for spamming. I did not intend to do so - I just wanted to kind of "resurrect" the spirit that once brought so many users from IE to the great and lovely fox.

I stick with Firefox, and I really appreciate how it has kept listening to its users and stuck with a clean, native user interface... but there's a couple of old horrible bugs like this that really need to get fixed, and I keep following them in the (apparently not quite vain) hope that it will eventually happen. The biggest one is the incomplete downloads bug, where the remote end closes the connection before the download is finished and there's no indication on this end that you only got 20MB of an expected 40MB. Is that another "forgotten" one? If the download manager would just leave the expected size visible after the download ends we'd at least have a chance of catching it when it happens.

backed out in bug 767005

Since the fix has been backed out, please consider reopening this BZ. Thanks!

(In reply to Laszlo Ersek from comment #262)
> Since the fix has been backed out, please consider reopening this BZ. Thanks!

it would just be a dup of the work being done in bug 769764, you can follow it over there.

*** This bug has been marked as a duplicate of bug 769764 ***