Debugging ssh-keygen dsa on Solaris8

[Scott Burch]

Hello,

>From the response to my original post regarding openssh, obviously the
only way to resolve the problem of getting openssh to work properly
under Solaris 8 with openssl 0.9.6b was to compile with debugging an
analyze to core file. Let me state that I am not a developer, but maybe
the following will help. I have compiled both openssh2.96p2 and openssl
0.9.6b with debugging using the Sun Forte Compiler update 2. I compiled
each piece of software without any additional options, the only thing I
did was change the --prefix. Below is some output from dbx: (The core I
am examining here comes from the make install...when it trys to create
the private/public dsa keys.)

(If there is anything else I can do to help let me know. The system is
5.8 Generic_108528-08 with the recommended patch cluster from July 11th.
This is an Ultra10 workstation) I also have the same problem using gcc
2.95.3 on Solaris 8 and Solaris 2.6.


Reading ssh-keygen
core file header read successfully
Reading ld.so.1
Reading libsocket.so.1
Reading libnsl.so.1
Reading libgen.so.1
Reading libc.so.1
Reading libdl.so.1
Reading libmp.so.2
Reading libc_psr.so.1
program terminated by signal BUS (invalid address alignment)
Current function is DSA_new_method (optimized)
  127   	ret->flags=ret->meth->flags;
(/opt/SUNWspro/bin/../WS6U2/bin/sparcv9/dbx)
where                           
=>[1] DSA_new_method(meth = ???) (optimized), at 0x4b6b8 (line ~127) in
"dsa_lib.c"
  [2] DSA_new() (optimized), at 0x4b580 (line ~86) in "dsa_lib.c"
  [3] DSA_generate_parameters(bits = ???, seed_in = ???, seed_len = ???,
counter_ret = ???, h_ret = ???, callback = ???, cb_arg = ???)
(optimized), at 0x4ace4 (line ~115) in "dsa_gen.c"
  [4] dsa_generate_private_key(bits = 1024U), line 560 in "key.c"
  [5] key_generate(type = 2, bits = 1024U), line 576 in "key.c"
  [6] main(ac = 7, av = 0xffbefc74), line 781 in "ssh-keygen.c"