no IDN in nslookup and host

Personally, I think lack of IDN support is a security feature.

I'm no expert on the security issue with IDN but it is used on the Internet today and both nslookup and host is very good tools to diagnose problems in some cases. There ought to be some way to implement IDN into the tools without compromising security.

Yes, this would be really good. Ping is also affected:

<email address hidden>: /etc/ssh # ping räksmörgås.se
ping: unknown host räksmörgås.se

The plan is to include idn support in bind 9.5, as part of a separate binary.

!ping :) - dnsutils 1:9.7.3.dfsg-1ubuntu4.1

From the man page in ubuntu 11.10 oneiric:

IDN SUPPORT
       If dig has been built with IDN (internationalized domain name) support, it can accept and
       display non-ASCII domain names. dig appropriately converts character encoding of domain name
       before sending a request to DNS server or displaying a reply from the server. If you'd like to
       turn off the IDN support for some reason, defines the IDN_DISABLE environment variable. The IDN
       support is disabled if the variable is set when dig runs.

There must be support for idn domain names, but perhaps you forgot to enable it?

Hm, perhaps I'm wrong, should dig tool be used with idn?
Or should it be able to translate utf-8 characters to punycode directly?
It works with punycode entered through idn command line tool, e.g. dig `idn њњњ.срб`

ah.. I assume idnkit (which is not packaged) is required:

dighost.c:41:24: fatal error: idn/result.h: No such file or directory
compilation terminated.41:24: fatal error: idn/result.h: No such file or directory
compilation terminated.

Hi,

IDN is not optional on the internet today.

The IANA is publishing several IDN top level domains: http://www.iana.org/domains/root/db

As it currently stands, I am having a difficult time using Ubuntu as a professional internet operator at an ISP, because my only choice is hacks like this:

reaper@silverstar:~$ dig ns $(idn2 한국)

On CentOS 6.5, the query is simple:
jgs@whmdev:~$ dig ns 한국

The man page of dig has the same message for at least 2 years, staing it's a compile time option. Please compile it into the next version of this package.

Is there any reason not to build dig with IDN support?

8 years passed for such an issue. :(
May be somebody will fix it at last?

IDN support would be really nice to have. Unfortunately, not even Ubuntu 16.04 has it yet:

Ubuntu 16.04 (Xenial Xerus):

nornetpp@experiment:~$ host bjørvika.uio.nornet
Host bjørvika.uio.nornet not found: 3(NXDOMAIN)
nornetpp@experiment:~$ nslookup bjørvika.uio.nornet
Server: 10.1.1.1
Address: 10.1.1.1#53

** server can't find bj\195\184rvika.uio.nornet: NXDOMAIN

nornetpp@experiment:~$ ping bjørvika.uio.nornet
ping: unknown host bjørvika.uio.nornet
nornetpp@experiment:~$ ping6 bjørvika.uio.nornet
unknown host
nornetpp@experiment:~$ dig bjørvika.uio.nornet

; <<>> DiG 9.10.3-P4-Ubuntu <<>> bjørvika.uio.nornet
;; global options: +cmd
;; Got answer:
;; ->>HEADER<<- opcode: QUERY, status: NXDOMAIN, id: 61159
;; flags: qr aa rd ra; QUERY: 1, ANSWER: 0, AUTHORITY: 1, ADDITIONAL: 1

;; OPT PSEUDOSECTION:
; EDNS: version: 0, flags:; udp: 4096
;; QUESTION SECTION:
;bj\195\184rvika.uio.nornet. IN A

;; AUTHORITY SECTION:
uio.nornet. 3600 IN SOA blindern.uninett.uio.nornet. root.uio.nornet. 57669665 900 60 84600 3600

;; Query time: 1 msec
;; SERVER: 10.1.1.1#53(10.1.1.1)
;; WHEN: Thu May 12 13:56:07 CEST 2016
;; MSG SIZE rcvd: 107

On the other hand, Fedora Core 23 has is enabled by default. So, it should not be too difficult to support it in Ubuntu as well.

Fedora Core 23:

[root@queenstown ~]# host bjørvika.uio.nornet
bjørvika.uio.nornet is an alias for bjoervika.uninett.uio.nornet.
bjoervika.uninett.uio.nornet has address 10.1.2.100
bjoervika.uninett.uio.nornet has IPv6 address (address removed)
[root@queenstown ~]# nslookup bjørvika.uio.nornet
Server: 10.1.1.1
Address: 10.1.1.1#53

bjørvika.uio.nornet canonical name = bjoervika.uninett.uio.nornet.
Name: bjoervika.uninett.uio.nornet
Address: 10.1.2.100

[root@queenstown ~]# ping -c1 bjørvika.uio.nornet
PING bjoervika.uninett.uio.nornet (10.1.2.100) 56(84) bytes of data.
64 bytes from bjoervika.uninett.uio.nornet (10.1.2.100): icmp_seq=1 ttl=62 time=3.84 ms

--- bjoervika.uninett.uio.nornet ping statistics ---
1 packets transmitted, 1 received, 0% packet loss, time 0ms
rtt min/avg/max/mdev = 3.844/3.844/3.844/0.000 ms
[root@queenstown ~]# ping6 -c1 bjørvika.uio.nornet
PING bjørvika.uio.nornet(bjoervika.uninett.uio.nornet) 56 data bytes
64 bytes from bjoervika.uninett.uio.nornet: icmp_seq=1 ttl=62 time=5.67 ms

--- bjørvika.uio.nornet ping statistics ---
1 packets transmitted, 1 received, 0% packet loss, time 0ms
rtt min/avg/max/mdev = 5.673/5.673/5.673/0.000 ms
[root@queenstown ~]# dig bjørvika.uio.nornet any

; <<>> DiG 9.10.3-P4-RedHat-9.10.3-12.P4.fc23 <<>> bjørvika.uio.nornet any
;; global options: +cmd
;; Got answer:
;; ->>HEADER<<- opcode: QUERY, status: NOERROR, id: 62089
;; flags: qr aa rd ra; QUERY: 1, ANSWER: 1, AUTHORITY: 2, ADDITIONAL: 3

;; OPT PSEUDOSECTION:
; EDNS: version: 0, flags:; udp: 4096
;; QUESTION SECTION:
;bjørvika.uio.nornet. IN ANY

;; ANSWER SECTION:
bjørvika.uio.nornet. 86400 IN CNAME bjoervika.uninett.uio.nornet.

;; AUTHORITY SECTION:
uio.nornet. 86400 IN NS ns.simula.nornet.
uio.nornet. 86400 IN NS blindern.un...

Interestingly, Ubuntu 16.04's traceroute supports IDN, but neither tracepath nor traceroute6/tracepath6:

nornetpp@experiment:~$ traceroute bjørvika.uio.nornet
traceroute to bjørvika.uio.nornet (10.1.2.100), 30 hops max, 60 byte packets
 1 fornebu.uninett.simula.nornet (10.1.1.1) 0.947 ms 0.945 ms 0.942 ms
 2 uninett.simula.uninett.uio.nornet (192.168.28.246) 3.111 ms 3.114 ms 3.110 ms
 3 bjoervika.uninett.uio.nornet (10.1.2.100) 5.103 ms 5.153 ms 5.140 ms
nornetpp@experiment:~$ tracepath bjørvika.uio.nornet
gethostbyname2: Unknown host
nornetpp@experiment:~$ traceroute6 bjørvika.uio.nornet
traceroute: unknown host bjørvika.uio.nornet
nornetpp@experiment:~$ tracepath6 bjørvika.uio.nornet
getaddrinfo: Name or service not known

This should be changed to support for IDN in bind9 packages and iputils packages, especially dnsutils and iputils-ping packages. The IDN support nowadays are mandatory. Almost every major GNU/Linux distro currently supports IDN in ping/dig/nslookup/etc from the box.

Status changed to 'Confirmed' because the bug affects multiple users.

I also vote for this.

Seriously, what's the problem? IDN support was added long time ago to BIND9 and it's tools. Whether someone likes it or not, IDN is a standard (bad one, but still) and should be supported. Right now I have to use wrapper to make these tools IDN-aware.

iputils-ping in bionic and artful is linked with libidn:

andreas@nsnx:~$ dpkg -S /bin/ping
iputils-ping: /bin/ping

andreas@nsnx:~$ ldd /bin/ping|grep idn
 libidn.so.11 => /lib/x86_64-linux-gnu/libidn.so.11 (0x00007f21be770000)

andreas@nsnx:~$ ping -c 1 räksmörgås.se
PING räksmörgås.se (91.226.36.2) 56(84) bytes of data.
64 bytes from common33.iis.se (91.226.36.2): icmp_seq=1 ttl=46 time=250 ms

--- räksmörgås.se ping statistics ---
1 packets transmitted, 1 received, 0% packet loss, time 0ms
rtt min/avg/max/mdev = 250.386/250.386/250.386/0.000 ms

Confirming that's not the case in xenial.

ubuntu@xenial-ping:~$ ping bjørvika.uio.nornet
ping: unknown host bjørvika.uio.nornet
ubuntu@xenial-ping:~$ ldd /bin/ping
 linux-vdso.so.1 => (0x00007ffddd516000)
 libcap.so.2 => /lib/x86_64-linux-gnu/libcap.so.2 (0x00007f68883fe000)
 libc.so.6 => /lib/x86_64-linux-gnu/libc.so.6 (0x00007f6888034000)
 /lib64/ld-linux-x86-64.so.2 (0x00007f6888604000)

I'm gonna mark the iputils task as fix released then.

Why it is "fix released" while it's still affects Xenial? Xenial are LTS and supported until 04-2021 and it should be fixed in Xenial. Every other major distro has support for IDN for years. But for some reason Ubuntu maintainers are ignoring this issue.

When the bug task does not mention a specific ubuntu release, it means "whatever is devel at the time". Now that means bionic, where it's fixed and available, that's why the task was marked as fix released.

To have it fixed in Xenial and other stable releases, the procedure outlined in https://wiki.ubuntu.com/StableReleaseUpdates has to be followed.

And what about other tools like dig and host? How many more years will they lack IDN support?

By the way, for all who need a workaround, here's a simple script that uses idn tool (apt install idn) to translate any non-ascii parameters to punycode and pass them to supplied executable.

Example:
$ type host
host is aliased to `idnwrapper.sh host'
$ host почта.рф
xn--80a1acny.xn--p1ai has address 91.215.36.43
xn--80a1acny.xn--p1ai mail is handled by 20 78.108.81.1.

But Bionic are not released yet: https://wiki.ubuntu.com/Releases
And Artful are not LTS release type. Effectively meaning Xenial are latest LTS release up-to-date and still supported until April 2021.
Does by any chance this can be fixed in currently active LTS release, i.e. Xenial? I don't think that would require a lot of efforts to re-compile packages with IDN support and push the update. Why this can't be done with Xenial?

The bionic task is fix released in the sense that the fix is published in the official archive and available to users.

The one issue I can see wrt a fix for xenial or any other stable release is the introduction of a new dependency via the update, that is, libidn. But I've seen that done in the past, all it needs is a good justification in the SRU process.

Debian enabled building with libidn2: https://salsa.debian.org/dns-team/bind9/commit/3e72f1a0c5cd695b7bcdefa458869cf5566198bd

This will be fixed in cosmic in the next upload:
root@cosmic-bind9-merge-9114:~# dig @127.0.0.1 +idnout räksmörgås.se

; <<>> DiG 9.11.4-3ubuntu1~ppa2-Ubuntu <<>> @127.0.0.1 +idnout räksmörgås.se
; (1 server found)
;; global options: +cmd
;; Got answer:
;; ->>HEADER<<- opcode: QUERY, status: NOERROR, id: 6004
;; flags: qr rd ra ad; QUERY: 1, ANSWER: 1, AUTHORITY: 0, ADDITIONAL: 1

;; OPT PSEUDOSECTION:
; EDNS: version: 0, flags:; udp: 4096
; COOKIE: 52a808b0d2c6f79dc90477ff5b5f2a02ff53d4bb83e8cf22 (good)
;; QUESTION SECTION:
;räksmörgås.se. IN A

;; ANSWER SECTION:
räksmörgås.se. 60 IN A 91.226.36.2

;; Query time: 2976 msec
;; SERVER: 127.0.0.1#53(127.0.0.1)
;; WHEN: Mon Jul 30 15:08:50 UTC 2018
;; MSG SIZE rcvd: 94

root@cosmic-bind9-merge-9114:~# dig @127.0.0.1 +noidnout räksmörgås.se

; <<>> DiG 9.11.4-3ubuntu1~ppa2-Ubuntu <<>> @127.0.0.1 +noidnout räksmörgås.se
; (1 server found)
;; global options: +cmd
;; Got answer:
;; ->>HEADER<<- opcode: QUERY, status: NOERROR, id: 1448
;; flags: qr rd ra ad; QUERY: 1, ANSWER: 1, AUTHORITY: 0, ADDITIONAL: 1

;; OPT PSEUDOSECTION:
; EDNS: version: 0, flags:; udp: 4096
; COOKIE: 5e9a3cfb67f8482724d5069d5b5f2a070720eaf49861e73b (good)
;; QUESTION SECTION:
;xn--rksmrgs-5wao1o.se. IN A

;; ANSWER SECTION:
xn--rksmrgs-5wao1o.se. 55 IN A 91.226.36.2

;; Query time: 0 msec
;; SERVER: 127.0.0.1#53(127.0.0.1)
;; WHEN: Mon Jul 30 15:08:55 UTC 2018
;; MSG SIZE rcvd: 94

This is fixed in cosmic:
https://launchpad.net/ubuntu/+source/bind9/1:9.11.4+dfsg-3ubuntu1

Any chances the fix will arrive in xenial?