openbsd-compat/getrrsetbyname.c: answer buffer size too large for EDNS0 and glibc

[Hauke Lampe]

Hello.

I have an issue with SSHFP lookups using "VerifyHostKeyDNS=yes" and
"options edns0" in /etc/resolv.conf (glib >= 2.6).


getrrsetbyname() calls res_query() with a maximum buffer size of 65536.
The glibc resolver truncates this value to 16 bits, reducing the query's
advertised buffer size to 0.

BIND appears to ignore it while Unbound returns a server failure.

glibc's source suggests that it should retry the query without EDNS0 but
it does not. Maybe a timeout triggers earlier.

OpenSSH then logs "DNS lookup error: general failure" and continues.

I propose reducing ANSWER_BUFFER_SIZE to 65535. Of course, the
stub-resolver should probably catch this kind of problem, too.


openbsd-compat/getrrsetbyname.c:
[...]
#define ANSWER_BUFFER_SIZE 1024*64
[...]
getrrsetbyname()
[...]
        u_char answer[ANSWER_BUFFER_SIZE];
[...]
        length = res_query(hostname, (signed int) rdclass, (signed int)
rdtype,
            answer, sizeof(answer));
[...]

In the glibc stub-resolver, "sizeof(answer)" is eventually passed on as
"anslen" to __res_nopt() in resolv/res_mkquery.c:
[...]
        NS_PUT16(anslen & 0xffff, cp);  /* CLASS = UDP payload size */

and sent out to the recursor:

| IP 127.0.0.1.44138 > 127.0.0.1.53: 31454+ [1au] SSHFP?
orbit.attraktor.org. ar: . OPT UDPsize=0 (48)
| IP 127.0.0.1.53 > 127.0.0.1.44138: 31454 ServFail-| [0q] 0/0/0 (12)



Hauke.
-------------- next part --------------
A non-text attachment was scrubbed...
Name: getrrsetbyname-anslen.patch
Type: text/x-patch
Size: 346 bytes
Desc: not available
URL: <http://lists.mindrot.org/pipermail/openssh-unix-dev/attachments/20090629/ec982e49/attachment-0002.bin>
-------------- next part --------------
A non-text attachment was scrubbed...
Name: signature.asc
Type: application/pgp-signature
Size: 197 bytes
Desc: OpenPGP digital signature
URL: <http://lists.mindrot.org/pipermail/openssh-unix-dev/attachments/20090629/ec982e49/attachment-0003.bin>