xmpp4moz doesn't read SRV DNS records

Bulk move of all Networking-Core (to be deleted component) bugs to new
Networking component.

Help wanted.

See also Internet Draft draft-andrews-http-srv-00

It takes time to check for SRV records. Be careful with features that
can degrade the user experience! I'd like to see a study of SRV's impact
on web-page load times.

It takes time when an HTTP request times out because the server is down or
overloaded as well! Any testing which attempts to assess impact on "user
experience" should take into account not only the "best case" scenarios but
also the load-balancing and failover benefits of SRV. We live, after all, in an
imperfect world...

Please guys, we need this improvement on our browser of choice.

SRV RRs are very useful; such enhancement is _innovation_. Just take a look at
the specs.

There is no good reason to not include this (other than allocating some
development resources).

mass move, v2.
qa to me.

Can we target this (1.0 or future?)

Some corrections to the previous remarks:
 - Browsing is improved: you'd like the web robust like email? SRV do that!
 - Goal of supporting SRV records is support load balanced & backed-up servers.
 - RFC 2052 is superseded by RFC 2782
 - Dealing with SRV records does not take more time, it uses one DNS lookup
 - People are working on library support to automate SRV records
 - Targeting it at v1.0 seems smart; being able to claim robustness is nice ;-)
All in more detail on http://dns.vanrein.org/srv (or send me email and ask)

I am taking ownership of this RFE. I have been looking for a programming
project for some time now and this one is very interesting to me. I'm thinking
that all protocols that Mozilla supports should use SRV records, i.e. HTTP,
NNTP, FTP, POP, IMAP, LDAP, SMTP, GOPHER (does anyone still use this?). I will
be creating an outline of the plan for comments.

Why was I copied on this bug when it was updated? the update was sent to

<email address hidden>? Why?

By the way, the RFC is more than six months old - draft RFCs more than
six months old are questionable - I'd contact the authors to check on their
current felings before writing anything.

Nick/others,

RFC 2052, the originally mentioned topic, is indeed over 6 months, but you may
also have noticed my earlier remark that it was replaced with RFC 2782. This one
is on the Standards Track, and STD 1 specifies its status, so rather than making
an RFC look suspicious, I'd like to see a quote from STD 1.
Allow me to demonstrate: it is listed under the heading of
        3.4. Proposed Standard Protocols
And that is a result of a specific action by the IESG [see RFC 2026, sec 4.1.1],
so it definately makes sense to implement this particular feature.

Hope this helps to take your worries away,

Rick van Rein.

This past weekend I was working on some "proof of concept" code to kind of hash
out how it is going to work and some things occured to me that maybe others can
comment on.

If this was a simple ftp client, such as ncftp, then when I initiate a
connection to domain.org I could do a simple DNS lookup of _ftp._tcp.domain.org
and plow through the list of SRV records, sorted by priority and picked at
random by weight within the same priority. When a connection failes, remove it
from the list and try again. Once a successful connection has been established
the data concerning SRV records could be thrown away.

The web, by contrast, is a completely different paradigm. Instead of having
one continuous connection throughout the life of the visit, a seperate
connection is used for each "transaction". And each transaction may be
comprised of multiple connections when a particular page contains multiple
objects.

This is where my experience with Cisco Local Directors, ArrowPoints and Central
Dispatch comes in and confuses the issue. These devices are keeping running
tabs on the services that they are watching. For example the Local Director
will mark a service as bad when it gets a failed connection. Even though the
connection would finally make it over to a good server the user experienced a
delay assuming the server was off the network and had to wait for TCP
timeouts. From that point on the LD would direct traffic to services it knew
were good. Every once in a while it would try the bad one just for yuks to see
if it had been fixed.

So assuming we don't want the user to get hung up on a down server every time
they traverse the service list we need to keep a running tab on sevices we've
tried in the past. So does this mean that once we have queried the SRV record
we keep it cached within mozilla until the TTL runs out? Or do we make actual
DNS queries every time and run through the same algorithm to choose a server?
Or we could keep a running list of servers we've connected to and how
successful we were at it in a seperate list. Then we could do seperate SRV
queries every time and compare what we get back to our running list. We would
then try a server marked as down only if we had no other alternatives.

That then brings up the issue of how long do we keep the information around
for? I suppose it is no different than a history, but it eats up memory.

> (...) For example the Local Director
> will mark a service as bad when it gets a failed connection. Even though the
> connection would finally make it over to a good server the user experienced a
> delay assuming the server was off the network and had to wait for TCP
> timeouts. From that point on the LD would direct traffic to services it knew
> were good. (...)

Only the connection which "detected" the failed service hangs?

> So assuming we don't want the user to get hung up on a down server every time
> they traverse the service list we need to keep a running tab on sevices we've
> tried in the past. (...)

I haven't understood why the user will always get hung up on a
failed service. The LD is not going to deliver newer connections
to a healthy server after one failure?

>> (...) For example the Local Director
>> will mark a service as bad when it gets a failed connection. Even
>> though the connection would finally make it over to a good server
>> the user experienced a delay assuming the server was off the network
>> and had to wait for TCP timeouts. From that point on the LD would
>> direct traffic to services it knew were good. (...)
>
> Only the connection which "detected" the failed service hangs?

Correct.

>> So assuming we don't want the user to get hung up on a down server
>> every time they traverse the service list we need to keep a running
>> tab on sevices we've tried in the past. (...)

> I haven't understood why the user will always get hung up on a
> failed service. The LD is not going to deliver newer connections
> to a healthy server after one failure?

That part I was describing was without LDs and so strictly inside of
the browser. If we requery DNS every time and have to run down the
list servers listed for the service and there is a down server, the
user will get hung up on that down server. It will eventually time
out and the browser will try the next server, but the user will have
to wait. Which is why the implementation might want to cache the
DNS data internal to the browser and keep track of which servers are
up and which ones are down.

Or something else I had thought of, but didn't want to mention
because I didn't like it, was to initiate connections to all servers
listed and use the one that responds first. Or if there are, for the
sake of argument, 20 objects on a particular page and connections
were opened to 4 differant servers, then each connection can serve 5
objects, assuming each server is weighted equally, through the use of
persistant connections.

> That part I was describing was without LDs and so strictly inside of
> the browser. If we requery DNS every time and have to run down the
> list servers listed for the service and there is a down server, the
> user will get hung up on that down server. It will eventually time
> out and the browser will try the next server, but the user will have
> to wait. Which is why the implementation might want to cache the
> DNS data internal to the browser and keep track of which servers are
> up and which ones are down.

So you're talking about timeout on requests for the TCP connections.

I'm now wondering whether such "healthy cache" breaks the SRV behavior
as specified by RFC2782. The thing explicitily states:

"for each address record found, try to connect to the (protocol,
address, service)."

While the cache seems a smart approach, it would be a shame to have
the awesome SRV feature not adhering to the proper standard. How
about to make the "healthy cache" an optional, user-tunable feature
and let it disabled by default?

Changing title because RFC2052bis was superseeded a long time ago by RFC2782

*** Bug 162798 has been marked as a duplicate of this bug. ***

I'm disappointed that support for SRV's hasn't made it into Mozilla after three
years. There are are very useful device and are now being used by a number of
services: Jabber, LDAP, Kerberos, Windows 2000, etc...

The main reason I want SRV RRs is so I can have multiple SSL Apache virtual
hosts with one IP. Name-based virtual hosts don't work with SSL, but port-based
virtual hosts would :-)

This document answers many of the questions asked above about how to go about
implimenting http with srv:
http://www.globecom.net/ietf/draft/draft-andrews-http-srv-01.html

It is technically expired, but it's the closest thing to a standard I could find.

I have built a working library that supports SRV records in a generic
way. This library is intended for simple applications that want to
"grab a good connection quickly" so it may be unsuitable for Mozilla,
but at least it is example code, open source so ready for copy/paste.

The library can be found on
  http://libsrv.sf.net
and it works on Linux, OpenBSD, FreeBSD, MacOS X.
Any Mozilla coders out there who would like a quick-n-easy way to
get popular? ;-)

There are lots of issues related to caching and DNS, but I don't think that
necessarily should be a bottleneck to adding this support.

What you probably need to do is look at implementing this with a pref, and then
checking it in as off by default. You should also carefully research any
dependent changes to existing code, because you have to sell this to the module
owners.

After some bake time to make sure the existence of the feature is not causing
regressions, you can have individual users turn it on and see how it works.

Don't worry about caching now, it seems like the problems described w/ the
network load distributors would be issues for normal DNS as well.

I'd like to see this implemented so that things like mail server configuration
can be more automatic.

Is David still working on this bug or should this bug's assignment be changed?

Just curious.

Also consider looking at DNS caching discussions from bug 109313

see also bug 173804 (RendezVous/ZeroConf support)

What's the status on this bug? Is anyone working on it?

It would be a a very useful feature, but the bug seems to be rather old.

Adam

As long as the RFC was 2052, it was an "Experimental" RFC. RFC 2782 is
Standards Track. This should mean that the priority is higher. I guess that I
would like to hear about what the possibilities of supporting this are.

I did a search on "RFC 2782 Support Library" in google and found
http://www.vovida.org/document/vocaldoc/util/srv_readme.txt. There seems to be
other support code that could be "borrowed" running around as well.

There is more, actually:
This is an lgpl library which can deal with SRV records withoud dependency on
bind - http://faerion.sourceforge.net/doc/rascal/

currently, it can only be downloaded using CVS, but author will release it as a
package in the near future.

the exact function is, possibly, this:
http://faerion.sourceforge.net/doc/rascal/rascal_connect_service.html

There is also ruliwrap:
http://weinholt.se/hacks/

ruliwrap is a wrapper for getaddrinfo() that uses libruli to seamlessly
add SRV record support to programs that use getaddrinfo(3).

*** Bug 293701 has been marked as a duplicate of this bug. ***

At the risk of triggering stupendous wrath, but maybe some real discussion, too,
I want to suggest a dastardly hack in lieu of the DNS SRV support envisioned in
this bug report. It seems clear that:

- DNS SRV records have been around for a long time -- even this bug has been
around for a long time, but nobody seems to be implementing it for HTTP, even
though may other, newer protocols have been supporting the DNS SRV records long
since. Even Microsoft can't seem to implement this thing.

- A careful look at the name resolution process might offer an explanation of
why the current SRV proposed standard is not getting implemented in the world of
the interactive web. No matter how you look at it, if you have to query for an
SRV record, and then query for an A record, you have two round trips through the
DNS. Even if everything is working perfectly, the name resolution process will
take (nominally) twice as long if you have to go through an SRV resolution step.
 You could imagine fixing the problem by allowing IP addresses in SRV records
(this is not what the present standard says, and there are in general good
reasons for that), but even then, it is hard to optimize the process of looking
for records that usually won't be there. Negative caching is a hard problem.
(In other words, you have added a full-scale, uncacheable DNS query to every
lousy URL you want to resolve. A web browser that needs to build a complex page
from several URLs would have a hard time presenting a reasonably responsive
experience to the user even if all servers that need to say "no SRV record here"
are configured properly and don't drop the unknown request on the floor.)

- Doing SRV lookups is hard from a programming perspective. It goes against the
grain of the facilities provided in the standard C library. You almost need to
provide an SRV lookup library, and the more libraries you depend on, the less
portable your result.

- Mozilla is the cutting edge of browser development. If we could find a better
way to do it and write an RFC and maybe get Microsoft on board if it was simple
enough, we would have taken care of 95% of browsers out there and the rest would
take care of themselves.

- DNS SRV is wonderful for load balancing, but it's overkill if you just want to
run on an alternate port. (If we can consider the space of TCP and UDP ports to
be part of the address space of an IPv4 network, instead of worrying about
exhausting a 32 bit space, we now have 48 bits, which is a fair amount of
breathing room for the time between now and when IPv6 happens.)

I'm going to propose a simple flag that could be inserted in a set of DNS A
records to indicate that the name describes a service instead of a host and
provide the port where the service can be connected to. It will have the
advantages that:

- It is less than a dozen lines to add (or remove) in only one place,
- It is ridiculously portable,
- It imposes negligible performance/bandwidth/processing penalty/overhead,
- It solves the problem of naming a service on an alternate port, and
- It is safe in that it will not be triggered unexpectedly (keep reading).

Disadvantages:
- I'm proposing it here for the fi...

Bugzilla is not a place for creating new RFCs. sorry. You should try NNTP.
also, You are not right, about need of several DNS requests in process of SRV
queries. DNS answer includes "additional" section which already holds the
required data.
for example:

$ host -a -t SRV _irc._tcp.forestnet.org
Trying "_irc._tcp.forestnet.org"
;; ->>HEADER<<- opcode: QUERY, status: NOERROR, id: 5356
;; flags: qr rd ra; QUERY: 1, ANSWER: 1, AUTHORITY: 2, ADDITIONAL: 4

;; QUESTION SECTION:
;_irc._tcp.forestnet.org. IN SRV

;; ANSWER SECTION:
_irc._tcp.forestnet.org. 3600 IN SRV 1 0 6667 irc.forestnet.org.

;; AUTHORITY SECTION:
forestnet.org. 3600 IN NS ns1.cwn.ru.
forestnet.org. 3600 IN NS ve208.msm.ru.

;; ADDITIONAL SECTION:
irc.forestnet.org. 3600 IN A 81.9.48.3
irc.forestnet.org. 3600 IN A 195.91.162.198
irc.forestnet.org. 3600 IN A 213.248.60.142
ve208.msm.ru. 3600 IN A 213.248.60.142

Received 190 bytes from 192.168.0.1#53 in 725 ms

portability is not a big question too. The libraries which were mentioned
earlier are VERY portable.

p.s. just to clear things up. Rascal is BSD-licensed now:
http://faerion.sourceforge.net/librascal.html.var

I have worked with SRV records before (they're used in SIP networks, see
RFC3263), and I can tell you that multiple round-trip is not an issue at all.
SRV and A records will be returned in 1 packet, if there's enough room (that's
where RFC2671 comes in handy).

Note that 127.0.0.1/8 was originally a routable network. I still have customers
that use them, both on private networks and as 'internal' addresses (only
routable in 1 host). But it becomes harder and harder to use them, because lots
of routers and IP-stacks are now hardcoding them (due to RFC3330).

Well, as I'm seeing it right now, we have portable libraries, and we have
emerging technologies like Coral Cache that stand to make excellent use of it,
but no one can use it because the browsers aren't embracing it. :(

Is there really anything left about this whole process that is left to debate,
or is it really a matter of lack of a coder wanting to work on it? If it is the
latter, then I think we're almost approaching the point of needing to place a
bounty on it. Truly (affoardable) redundant systems are just dying for this
technology. You can place a load balancer in your data center all day long, but
if your load balancers fail, you're down. If you can implement SRV, then you
fail to another system, and if configured correctly, you can even pull from
Coral Cache.

Anyway, thoughts? Should be be starting a bounty on this?

Here are some pertinent links:

http://www.ietf.org/rfc/rfc2782.txt

Implementation Examples (not limited to http)
http://www.voip-info.org/tiki-index.php?page=DNS%20SRV
http://www.cisco.com/univercd/cc/td/doc/product/voice/sipproxy/admingd/ver2_1/ddns.htm

I made a Mozillzine forum for people who want to continue non-development discussion:
  http://forums.mozillazine.org/viewtopic.php?p=1676455#1676455

In my opinion SRV RRs should be supported by Mozilla/Firefox/Thunderbird/etc. in
each case, no matter if there are performance questions or anything like this.
The reson is: RFC2782 is a proposed standard according to the IETF.

Are there any current developments regarding SRV RRs in Mozilla products?

(In reply to comment #35)
> In my opinion SRV RRs should be supported by Mozilla/Firefox/Thunderbird/etc. in
> each case, no matter if there are performance questions or anything like this.
> The reson is: RFC2782 is a proposed standard according to the IETF.
>
> Are there any current developments regarding SRV RRs in Mozilla products?

I don't think there is, however just for grins, some pseudo-code for someone
familiar enough with the zilla sources, here are a pair of methods to give a try:

presuming someone has typed in a plain url of the type http://www.domain.tld,
which of course presumes port 80:

Rather than simply attempting to connect to www.domain.tld on port 80, first
attempt to get _http._tcp.www.domain.tld. Respect whatever is returned
(although to be honest, I'm not sure what happens at this point. I really
should try this from telnet sometime to see exactly what occurs here).

Second would be to see if there is a dns library floating around out there that
is lightweight and can handle srv records. When the user hits enter or clicks
the 'go' button, do an actual dns resolution before proceeding, again presuming
_http._tcp.www.domain.tld. Respect what is returned.

I believe the crux of the matter is timeouts. How long do you timeout on trying
srv's? If you go the second route and do a full name resolution and you get
valid srv data back before then reverting to traditional name resolution, I'd
then proceed with the correct priority site. Should that time out, then we
would stray from tradidition a bit. Rather than giving the usual timeout
message, give a new error:

"The requested page could not be retrieved, however the site has specified n
additional backup pages. Would you like to try the next one now?"
[Please take me to the next one.] [No, Thanks though.]
Don't ask me this again.[x]

Just a couple of thoughts on how to deal with it. The user experience wouldn't
be damaged in any way. The srv record timeouts could be adjusted through
about:config, and if done correctly an extension writer could probably extend
things a bit and make them prettier.

I do not think that the retry logic has to be changed, since browsers should
already try all returned A records. SRV just adds ordering to that logic.

I can offer some SRV query code if anyone wants to adapt it mozilla, but it is
quite easy anyway.

*** Bug 319939 has been marked as a duplicate of this bug. ***

<email address hidden>: lgpl code isn't acceptable for mozilla, please don't suggest it.
<email address hidden>: if you're really interested, i'd suggest you try doing it, build mozilla, hop on irc and ask for help. offering sample code isn't ilkely to get this very far.

personally, this whole thing seems very annoying. i don't feel like reading the rfc, but i would like a basic specification that explains what to do when:
1. there's no SRV record
2. there are 20 alternates in the SRV record and all of them are bad.
3. general cases in between.

The spec should list the series of requests (dns, http, other) as well as wait points and expected time spent at each step. the cost (size) of each inbound and outbound request should be listed, and a comparisoon should be made against today's browsing behaviors.

the following browsing behaviors are available today:
1. host doesn't respond, user tries a different link to a different server
2. user tries google cache for link
3. user tries coral cache for link

(In reply to comment #39)
> personally, this whole thing seems very annoying. i don't feel like reading the
> rfc, but i would like a basic specification that explains what to do when:
> 1. there's no SRV record
> 2. there are 20 alternates in the SRV record and all of them are bad.
> 3. general cases in between.

To be frank, "seems very annoying" and "don't feel like reading the rfc" seems like a pretty poor attitude to take. If you don't want to take the time to form an educated opinion on the bug, no less perform any work on it, that's certainly your right, but if that's going to be the case, why not just leave it alone?

Anyway, to answer these three questions:

1) If there's no SRV record, everything works as it does now, you look up an address record and make the request.

2) This exact case is not addressed in RFC2782 or Mark Andrews' HTTP SRV draft. I would see two possibilities - either the normal "this request timed out" error is displayed immediately, or a fallback to the standard A record / port 80 behavior takes place. I think the former would be more "correct", as specification of an SRV record says "if you speak SRV, you should use this".

3) I'm not sure what you mean by general cases in between.

If there are any developers out there interested in implementing this functionality (to be contributed back to Mozilla, if it will be accepted) under contract, please contact me off-bug, as my company is interested in sponsoring this work.

Am I the only one who thinks that Mozilla's new centralized approach of client autoconfiguration is a poor idea?

https://wiki.mozilla.org/Thunderbird/ISPDB/Requirements
http://ispdb.mozillamessaging.com/
https://wiki.mozilla.org/MailServerList

It allows for unverified users to submit the information (which has already happened for our domain, but they mostly got it right.)

It is also kind of a pain to manage if you host lots of domains (we host 270 domains.)

It also makes mozilla.org a dependency, which leads to concerns about their reliability and longevity.

How can we ensure that this mozilla-centric model won't interfere with other clients that want to implement autoconfiguration. If we have to backfill this information for 270 domains, I don't want to do it again for each client that chooses to implement their own centralized database.

Coincidentally, the XMPP folks are struggling with the same problem. XMPP client SRV records work wonderfully, but they rely on the server using Start-TLS with a signed certificate that matches the virtual domain. This has been a complaint of mine for some time, because obtaining certificates for all of our 270 domains is the biggest obstacle to enabling all of the email domains within our chat service. They have recognized the shortfall in this methodology (no doubt Google Apps Talk had something to do with this) and are working on the following draft to solve the problem.

http://tools.ietf.org/html/draft-hildebrand-dna-00

(In reply to comment #36)
> I'm voting for the bug, but only on the condition it uses a standard mechanism
> such as the approaching finalized IETF proposal:
>
> http://tools.ietf.org/html/draft-daboo-srv-email

I do agree with you Chris, but you may look into bug 356104 and bug 14328, these are showstoppers. Necko doesn't have code right now to resolve SRV records.

> Am I the only one who thinks that Mozilla's new centralized approach of
> client autoconfiguration is a poor idea?

No, you're not. It wasn't intended for the use case you mention. Please see https://wiki.mozilla.org/Thunderbird:Autoconfiguration and bug 534722.

(In reply to comment #36)
> I don't see that it's necessary to wait for DNSSEC to implement that. DNSSEC
> will deploy when it deploys and we'll get the extra security bump when it shows
> up.

I'd agree that it's not necessary to wait for DNSSEC if all servers found in this manner require SSL/TLS (i.e., the connection is configured to fail if TLS can't be initiated or if the cert doesn't match).

> (i.e., the connection is configured to fail if TLS can't be initiated or if the cert doesn't match)

Requiring that certs match the domain will be a show stopper for services that host lots of domains. We are an EDU, and we host 270 domains. Just think about services that host thousands of domains!

Supposing it is practical for email service administrators to maintain hundreds or thousands of valid signed matching certificates for domains that they don't own - it isn't practical, IMHO - do we know if the major MTAs support the ability to present a different cert for the individual domain during the Start-TLS negotiation? If not, that will also be a show stopper.

Is it feasible to abandon plain old TLS? Outlook still seems to work better with port 465/TLS. Start-TLS would be a new requirement since the domain needs to be known before the server can present a matching cert.

Take a look at http://tools.ietf.org/html/draft-hildebrand-dna-00 for some ideas on other methods of asserting domain ownership without the need for DNSSEC.

> Please see https://wiki.mozilla.org/Thunderbird:Autoconfiguration and bug 534722.

Yes, step 2 is a more rational approach. I still think that step 3 is problematic just in its existence. I see that there is mailing list http://groups.google.com/group/ispdb so I'll voice my concerns there.

I'd also like to see it, specifically with XPCOM access for extension use and use in other protocols like XMPP...

The patch here is great, because it shows how to make arbitrary DNS queries, but it unfortunately limits the API to SRV and TXT. In bug 342242, we need MX, too. So, I filed bug 545866 for an IDL API to allow arbitrary DNS record types to be queried from within Mozilla. This bug could then use this API.

See bug 14328 for the work being done on extending DNS support to include the resolution of SRV records, which seems to me to be closely related to the work being discussed here.

Yes, I filed bug 545866 (arbitrary DNS queries in Mozilla) based on that and made it block this bug.

Narrowing bug summary to DNS SRV records, to differentiate from DNS MX, which I filed as bug 551519.

*** Bug 536915 has been marked as a duplicate of this bug. ***

draft-daboo-srv-email is now http://tools.ietf.org/html/rfc6186

*** Bug 675476 has been marked as a duplicate of this bug. ***

If this is implemented, I'd suggest it be in compliance with rfc6186.

I don't see DNSSEC as a requirement, but rather an extra.
There's really little harm that can be done on a DNSSEC-less scenario that can't be done right now. And even if the SRV records are on a DNS domains that implements DNSSEC, the actual IMAP/SMTP servers might be in DNSSEC-less domains.

Any updates on this issue? The improvements to user experience by implementing this are huge:

Now:
User need to know email, username, password, smtp server, imap server, smtp port, imap port.

With RFC6186
User needs to know email, password.

Just FYI:
RFC 6186 doesn't give username.
The following method gives username and a number of other parameters, and it works today, see
https://developer.mozilla.org/en/Thunderbird/Autoconfiguration

There are some severe downsides to the ISBDB:

 1) It's controlled by Mozilla, instead of each ISP.
 2) It won't work on LANs.
 3) It requires intervention on Behalf of mozilla to update information.
 4) Only thunderbird uses this information. If each email client had it's own non-standard mechanism, then ISPs would have a huge burden to update each of them with new information when they change their email server.

The other mechanism:
 1) Shares point (4).
 2) Is inheretly insecure (why HTTP instead of HTTPS?)

While I have nothing in particular AGAINST maintaining support for mozilla's in-house mechanism, I belive that supporting standardized mechanisms (especially RFCs) is ultimately important.

> 1) It's controlled by Mozilla, instead of each ISP.
> 2) It won't work on LANs.
> 3) It requires intervention on Behalf of mozilla to update information.

Not true for the ISP fetch.

> 2) Is inheretly insecure ... HTTP

So is DNS (at least without DNSSEC)

(In reply to Hugo Osvaldo Barrera from comment #53)
> 4) Only thunderbird uses this information. If each email client had it's
> own non-standard mechanism, then ISPs would have a huge burden to update
> each of them with new information when they change their email server.

Evolution uses it too, although it looks like they host their own copy of the database. Check out:
http://git.gnome.org/browse/evolution/tree/mail/e-mail-autoconfig.c
which uses a (sadly non-SSL) base url of:
http://api.gnome.org/evolution/autoconfig/1.1/

The upcoming Firefox OS Gaia e-mail client also uses the ISP database too.

Note: Neither this nor I believe Ben's comments are meant to suggest that we don't want to support DNS SRV. Unfortunately, it's still a non-trivial undertaking to get the support in the gecko platform that has not been prioritized on the platform side. For Firefox OS v2 we are hoping to get it prioritized.

(In reply to Ben Bucksch (:BenB) from comment #54)
> > 2) Is inheretly insecure ... HTTP
>
> So is DNS (at least without DNSSEC)

So rfc6186 provides secure autoconfiguration for domains that implement DNSSEC. There's no way to achieve secure autoconfiguration with this mechanism, since HTTP provides no security mechanism.

However, my strongest point is it's non-standardness, while rfc6186 is standard, and grants total control (and optional security) to the ISP.
Also, there's no reason both autoconfiguration mechanisms can't exist.

(In reply to Andrew Sutherland (:asuth) from comment #55)
> (In reply to Hugo Osvaldo Barrera from comment #53)
> > 4) Only thunderbird uses this information. If each email client had it's
> > own non-standard mechanism, then ISPs would have a huge burden to update
> > each of them with new information when they change their email server.
>
> Evolution uses it too, although it looks like they host their own copy of
> the database. Check out:
> http://git.gnome.org/browse/evolution/tree/mail/e-mail-autoconfig.c
> which uses a (sadly non-SSL) base url of:
> http://api.gnome.org/evolution/autoconfig/1.1/

Sadly, this is a perfect example of how rfc6186 is superior, since every email server's configuration is in it's owner's DNS server.

>
> The upcoming Firefox OS Gaia e-mail client also uses the ISP database too.
>
>
> Note: Neither this nor I believe Ben's comments are meant to suggest that we
> don't want to support DNS SRV. Unfortunately, it's still a non-trivial
> undertaking to get the support in the gecko platform that has not been
> prioritized on the platform side. For Firefox OS v2 we are hoping to get it
> prioritized.

Actually, there's an alternate fix for this issue; if Mozilla's ISPDB contains no information related to a particular domain, it could query it using rfc6186, and return that.

This means that Thunderbird would still [indirectly] resolve using rfc6186, with no actual changes to the client code.

Shouldn't this depend on bug 14328 instead of 545866?

With the DNS-SD support brought with Flyweb in Firefox, the lack of DNS support in Gecko should have been mitigated. The security implications of using DNS data do not seem worse than those of using the current HTTP based mechanism.

I would love to have this implemented in Thunderbird, as a sysadmin that would like to make it easier for our users to connect to our service.

Regarding the security context of this, I actually think it's a great improvement, because it means people are more likely to automatically set up encrypted mail through imap(S) than if they have to manually enter the details.

2. As pointed out the MiTM risk is no worse than the current http method, and the guessing method (which actually uses the wrong server for us)

3. The MITM risk is only at setup time. If the user doesn't use autoconfig, and ends up using unencrypted mail, they are at MiTM risk everywhere they go, every coffee shop wifi, etc. Since the MitM risk is only at setup time, it's extremely hard for an attacker to exploit, because they have to convince the user to delete and recreate the connection in order to be able to hijack the autoconfigure request.

4. Doesn't the current UX flow mean that the users password is only sent to the server *AFTER* the autoconfigure completes and the server details are shown to them? What is the problem here? Doesn't the user get a chance to verify the server details?

If you want to make it easy for users, you can already do that today with http instead of DNS SRV:
https://developer.mozilla.org/en-US/docs/Mozilla/Thunderbird/Autoconfiguration

(In reply to Ben Bucksch (:BenB) from comment #60)
> If you want to make it easy for users, you can already do that today with http

Literally "http://". Bug 986967 is about that and is a real security problem.

RFC 6186 was published in 2011, but it remained only an option. In January 2018, RFC 8314 was published: https://tools.ietf.org/html/rfc8314

This explicitly sais:

> User-configurable MUAs SHOULD support the use of [RFC6186] for account setup.

https://tools.ietf.org/html/rfc8314#section-5.1

So please, let's follow the RFC and provide this option in Thunderbird.

Thanks, and keep up the good work!

Although RFC8314 is on the standards track, it's more a "best practices" document than an actual protocol definition.

It's true that Thunderbird does not implement configuration via DNS SRV (this bug here).
There are several fundamental reasons:

1. Arbitrary DNS lookups - bug 545866 / bug 14328 - are not implemented by Gecko.
   I've spent a lot of time implementing it, over 8 years ago (April 2010),
   only for the patch to be ignored and not reviewed. Forgive me when I'm a little less excited today.

2. DNSSEC is very hard to administrate without shooting yourself in the foot, and consequently,
   it's not widely deployed on second level domains. APNIC says:
   "Unfortunately, our recent study showed that DNSSEC deployment is still very low
   (only ~1% of .com, .net and .org domains deploy DNSSEC)"
   https://blog.apnic.net/2017/12/06/dnssec-deployment-remains-low/
   On most domains, DNS is completely insecure. I don't see it gaining quickly, either.
   APNIC <https://stats.labs.apnic.net/dnssec/XA> actually shows declining deployment in the last year.
   Without DNSSEC, DNS SRV is no more secure than http. In that light, comment 61 is a little ironic.

3. DNS SRV does not give us some critical information we need to configure the accounts.
   Specifically, which form the username needs to be transmitted. Is it bbob, or <email address hidden>, or
   even something like EXAMPLE\\bbob? Get it wrong, and the configuration doesn't work.
   Sure, we can try them all. And risk authentication failures and - worst case - triggering
   3-failures-in-a-row-and-you're-out security mechanisms.
   We're back to guessing and trail&error.
   That defeats the entire point of getting an authoritative configuration that will definitely work.
   If we were to support DNS SRV, then only as a second-class, fallback, legacy format.

DNS SRV is not widely used now. Why trying to establish a mechanism that has known shortcomings?

I think people here want this, because they want a standards-based mechanism. I fully concur with that.
But I'd rather make a new standard than implement a bad standard that nobody else implements anyway.

On the up side: Thunderbird already implements most of what RFC8314 requests, including most SHOULDs. In fact, it seems RFC8314 took Thunderbird as reference implementation, and basically says "Everybody should do what Thunderbird does" ;-) .

The goal in RFC 8314 was to reasonably promote security/interoperability improvements without forcing a lot of software to be declared broken. It wasn't actually based on what Thunderbird does, but if that's the impression, that's good.

The IETF has backed SRV records in general and email+SRV in particular so the spec followed that trend on the auto-configuration front. If the various ad-hoc proposals for HTTP+XML email auto-configuration were to come up with a single compromise proposal and bring that to the IETF, I wouldn't expect major problems moving that to the standards track, but unless that's done, DNS SRV is the only standard for email account auto-configuration and Thunderbird has a standards-gap by not implementing it.

It's easier to add a dns field, than to configure via a web server
We will hope that the RFC will be applied

For the record (and potential inspiration), implementation of rfc6186 in kmail is done: https://github.com/k9mail/k-9/pull/4719

After 14 years this is still open. What I can't figure out is why not having DNSSEC with RFC 6186 is seen as such a big problem while the currently implemented [Autoconfiguration](https://wiki.mozilla.org/Thunderbird:Autoconfiguration) feature gets the information from plain HTTP URLs. Both methods are identically vulnerable to MitM.

Esa, I answered that 3 years ago in comment 63. I mentioned 3 separate reasons. Each reason *on its own* is a serious problem.

(In reply to Ben Bucksch (:BenB) from comment #69)
> Esa, I answered that 3 years ago in comment 63. I mentioned 3 separate reasons. Each reason *on its own* is a serious problem.

Thanks, Ben! Some good and valid arguments there. It's just that while the current approach might have some clear advantages over RFC 6186, it doesn't seem any better security wise – quite the opposite. Or does the Autoconfiguration perhaps support HTTPS and HSTS (RFC 6797) with preloading, which would make forcing TLS possible?

(In reply to Ben Bucksch (:BenB) from comment #63)
> 3. DNS SRV does not give us some critical information we need to configure the accounts. Specifically, which form the username needs to be transmitted. Is it bbob, or <email address hidden>, or even something like EXAMPLE\\bbob? Get it wrong, and the configuration
> doesn't work.
> Sure, we can try them all. And risk authentication failures and - worst
> case - triggering
> 3-failures-in-a-row-and-you're-out security mechanisms.
> We're back to guessing and trail&error.
> That defeats the entire point of getting an authoritative configuration
> that will definitely work.
> If we were to support DNS SRV, then only as a second-class, fallback,
> legacy format.

This question is already addressed by RFC 6186, see under Section 4:
> When a user identifier is required, MUAs MUST first use the full email address provided by the user, and if that results in an authentication failure, SHOULD fall back to using the "local-part" extracted from the email address. This is in line with the guidance outlined in Section 5. If both these user identifiers result in authentication failure, the MUA SHOULD prompt the user for a valid identifier.

According to Section 5, service providers SHOULD implement their service so that either the mail address or the "local-part" only is sufficient for authentication. So this should work for most cases.

I propose that for the username Thunderbird should do according to the RFC. If both variants fail, Thunderbird could then fallback to first try out the other auto configuration methods. If any other auto configuration method are configured, Thunderbird could use those and ignore the DNS SRV records. If only the DNS SRV are configured, it could ask the user. If such rare case happens, users should probably know which username they should use. If not, it is still better to only ask for the username instead of currently not implementing DNS SRV because it might not work perfectly and ask the user for the complete server configuration.

To me, it sounds like you want users to have a great experience with Thunderbird. However, I think users will now go to a mail provider which claims to support auto configuration according to common standard (referring to RFC 6186) and they will see that most mail applications support his mail provider out of box, but not Thunderbird, which in my opinion could lead that users might not want to use Thunderbird "because it does not just work".

Hello,

at what stage is the implementation of RFC6186? It seems like it still doesn't work. It would really make life easier for us ISPs.

bug 1852752 seems to be necessary before this can be done

(In reply to Selek Respa from comment #73)
> bug 1852752 seems to be necessary before this can be done