Q: core dumped on keygen in Sol 2.6, ssh2.3.0p1, openssl-0.9.6 and zlib-1.1.3

[Markus Friedl]

FYI

>Path: news.uni-erlangen.de!uni-erlangen.de!newsfeeds.belnet.be!news.belnet.be!news.tele.dk!130.133.1.3!fu-berlin.de!server1.netnews.ja.net!news.gla.ac.uk!not-for-mail
>From: Will Partain <partain at dcs.gla.ac.uk>
>Newsgroups: comp.security.ssh
>Subject: Re: Q: core dumped on keygen in Sol 2.6, ssh2.3.0p1, openssl-0.9.6 and zlib-1.1.3
>Date: 21 Feb 2001 12:27:24 +0000
>Organization: University of Glasgow
>Lines: 52
>Message-ID: <uv7l2knrv7.fsf at slicker.dcs.gla.ac.uk>
>References: <9578vs$npb$1 at nnrp1.deja.com>
>NNTP-Posting-Host: slicker.dcs.gla.ac.uk
>Mime-Version: 1.0
>Content-Type: text/plain; charset=us-ascii
>X-Trace: singer.cent.gla.ac.uk 982758493 8576 130.209.242.51 (21 Feb 2001 12:28:13 GMT)
>X-Complaints-To: newsmaster at gla.ac.uk
>NNTP-Posting-Date: 21 Feb 2001 12:28:13 GMT
>User-Agent: Gnus/5.0807 (Gnus v5.8.7) XEmacs/21.1 (Channel Islands)
>Xref: news.uni-erlangen.de comp.security.ssh:19258

I, too, am seeing these symptoms (and I don't think it's
just the two of us :-).

I can further confirm that I see the same with 2.2.0p1 and
2.5.1p1.

Other facts: gcc-2.95.2, solaris2.6 reasonably well patched,
sun linker, openssl stuff being linked statically (from a .a
file).

No difference if openssl is compiled with -O, instead of -O3
-fomit-frame-pointer.

openssh is being compiled with -O2 -fstrict-aliasing.

In the case I've investigated with gdb, ssh-keygen dies the
first time it gets to RC4_set_key (openssl), called from
arc4random_stir.  It dies trying to write into the memory of
"static RC4_KEY rc4;" (openbsd-compat/bsd-arc4random.c).
If, just for sport, I *initialize* rc4 [with zeros] (i.e. it
ends up in the data section, not bss), then we sail past
this problem [but it dies a bit later in some rsa code].

Do these further details spark any thoughts?  (I suspect
something in the compiler-linker chain...)

Will

Philip J. Bondi <pjbondi at SystemDatabase.com> writes:

> Everything configures, configs, make, make test make install, but at
> the end of the "make install" for openssh, I get the following and
> keygen dumps core.  Any hints?
> 
> if [ -z "" ] ; then \
>         if [ -f "/usr/local/etc/ssh_host_key" ] ; then \
>                 echo "/usr/local/etc/ssh_host_key already exists,
> skipping." ; \
>         else \
>                 ./ssh-keygen -b 1024 -f /usr/local/etc/ssh_host_key -
> N "" ; \
>         fi ; \
>         if [ -f /usr/local/etc/ssh_host_dsa_key ] ; then \
>                 echo "/usr/local/etc/ssh_host_dsa_key already exists,
> skipping." ; \
>         else \
>                 ./ssh-keygen -d -f /usr/local/etc/ssh_host_dsa_key -
> N "" ; \
>         fi ; \
> fi ;
> *** Error code 138
> make: Fatal error: Command failed for target `host-key'