[Bug 3665] New: publickey RSA signature unverified: error in libcrypto to RHEL9 sshd (with LEGACY crypto policy enabled)

Thu Feb 8 04:19:31 AEDT 2024

https://bugzilla.mindrot.org/show_bug.cgi?id=3665

            Bug ID: 3665
           Summary: publickey RSA signature unverified: error in libcrypto
                    to RHEL9 sshd (with LEGACY crypto policy enabled)
           Product: Portable OpenSSH
           Version: 8.7p1
          Hardware: ix86
                OS: Linux
            Status: NEW
          Severity: major
          Priority: P5
         Component: sshd
          Assignee: unassigned-bugs at mindrot.org
          Reporter: ianveach at gmail.com

I'll give a nutshell here, but a lot of details are at
https://github.com/openssl/openssl/issues/23513 when I thought it might
be OpenSSL's libcrypto that was the issue.  That might be a prettier
read.

I have what I think is a bug, but I'm not a coder and wouldn't know
100%. Seems like it though.

Summary: We're attempting to replace a RHEL7 server with a new RHEL9
server. We have certain client systems that connect to a RHEL7
destination server just fine, using RSA user keys. Those same clients
are having problems connecting to the RHEL9 system, using the same
client RSA keys and same client systems. note!: we've implemented a
crypto policy that allows SHA1 (which I understand is a common
explanation for this failure - so I believe we can rule that out).
Error we see:

sshd: debug3: mm_answer_keyverify: publickey RSA signature unverified:
error in libcrypto

I wouldn't know what I'm doing, but I'm guessing that during user key
signature verification (RSA), the server is potentially getting
confused about which RSA signing occurred on one side (perhaps mixing
up ssh-rsa and rsa-sha256-512)?

The relevant error in sshd log with context is:

[authpriv.debug] sshd: debug3: mm_request_send: entering, type 23       
[authpriv.debug] sshd: debug3: mm_sshkey_verify: entering [preauth]     
[authpriv.debug] sshd: debug3: mm_request_send: entering, type 24
[preauth]                                
[authpriv.debug] sshd: debug3: mm_sshkey_verify: waiting for
MONITOR_ANS_KEYVERIFY [preauth]               
[authpriv.debug] sshd: debug3: mm_request_receive_expect: entering,
type 25 [preauth]                      
[authpriv.debug] sshd: debug3: mm_request_receive: entering [preauth]   
[authpriv.debug] sshd: debug3: mm_request_receive: entering             
[authpriv.debug] sshd: debug3: monitor_read: checking request 24        
[authpriv.debug] sshd: debug3: mm_answer_keyverify: publickey RSA
signature unverified: error in libcrypto

The clients (for reference): All succeed to RHEL7 server.
- AIX7.2 system (success to RHEL9): using ssh, scp, sftp:
OpenSSH_8.1p1, OpenSSL 1.1.1v
- AIX7.2 system (fail to RHEL9): same system, but using curl's sftp:
curl 8.4.0 libcurl/8.4.0 OpenSSL/1.1.1v libssh2/1.10.0
- Workday (fail to RHEL9): no idea on OS, ssh/ssl libraries, etc. using
JScape SFTP according to logs; no access

The destination servers (where I think the issue is - RHEL9):
- working destination: RHEL7 : OpenSSH_7.4p1, OpenSSL 1.0.2k-fips
(works for all three above client tests, same keys and users)
- failing destination: RHEL9 : OpenSSH_8.7p1, OpenSSL 3.0.7 (sshd
default provided by Red Hat)

Other details worth mentioning:
- the RHEL9 server is using a crypto policy of LEGACY atm. LEGACY,
DEFAULT, and DEFAULT:SHA1 have all failed. I've verified sshd -T is
including all three rsa signing/types for all appropriate settings.
- the same curl+sftp failing command from AIX client is successful if I
use ED25519 user keys, so ruling out many connection issue reasons.
unfortunately I cannot dictate to Workday what user keys to use there.
- the user keys in question are 2048 SHA256, and I can verify the
pairing is correct (plus they work to RHEL7 server). so keys seem ok
unto themselves.
- tried generating new user keys (3072bit, RSA), also no success using
those.
- the RHEL7 and RHEL9 (destination) servers use NFS for home, so same
destination user, same authorized_keys file, etc.

More details of the connection: Because we run the AIX clients, I'm
concentrating on that; However, so far, the Workday client exhibits
similar log messages/fails:

For successes, we see this in the logs (aix7.2 ssh/sftp/scp to RHEL9):
note that it makes rsa-sha2-512 references

2024 Feb  2 13:33:27 RHEL9 [authpriv.debug] sshd: debug2:
userauth_pubkey: valid user USERREDACTED querying public key
rsa-sha2-512 PUBLICKEYREDACTED [preauth]
2024 Feb  2 13:33:27 RHEL9 [authpriv.debug] sshd: debug1:
userauth_pubkey: test pkalg rsa-sha2-512 pkblob RSA
SHA256:RSASIGNATUREREDACTED [preauth]
2024 Feb  2 13:33:27 RHEL9 [authpriv.debug] sshd: debug1:
/home/USERREDACTED/.ssh/authorized_keys:33: matching key found: RSA
SHA256:RSASIGNATUREREDACTED
2024 Feb  2 13:33:27 RHEL9 [authpriv.info] sshd: Accepted key RSA
SHA256:RSASIGNATUREREDACTED found at
/home/USERREDACTED/.ssh/authorized_keys:33
2024 Feb  2 13:33:27 RHEL9 [authpriv.debug] sshd: debug3:
mm_answer_keyallowed: publickey authentication test: RSA key is allowed
2024 Feb  2 13:33:27 RHEL9 [authpriv.debug] sshd: debug2:
userauth_pubkey: authenticated 0 pkalg rsa-sha2-512 [preauth]
2024 Feb  2 13:33:27 RHEL9 [authpriv.debug] sshd: debug2:
input_userauth_request: try method publickey [preauth]
2024 Feb  2 13:33:27 RHEL9 [authpriv.debug] sshd: debug2:
userauth_pubkey: valid user USERREDACTED attempting public key
rsa-sha2-512 PUBLICKEYREDACTED [preauth]
2024 Feb  2 13:33:27 RHEL9 [authpriv.debug] sshd: debug3:
userauth_pubkey: have rsa-sha2-512 signature for RSA
SHA256:RSASIGNATUREREDACTED [preauth]
2024 Feb  2 13:33:27 RHEL9 [authpriv.debug] sshd: debug1:
/home/USERREDACTED/.ssh/authorized_keys:33: matching key found: RSA
SHA256:RSASIGNATUREREDACTED
2024 Feb  2 13:33:27 RHEL9 [authpriv.info] sshd: Accepted key RSA
SHA256:RSASIGNATUREREDACTED found at
/home/USERREDACTED/.ssh/authorized_keys:33
2024 Feb  2 13:33:27 RHEL9 [authpriv.debug] sshd: debug3:
mm_answer_keyallowed: publickey authentication: RSA key is allowed
2024 Feb  2 13:33:27 RHEL9 [authpriv.debug] sshd: debug3:
mm_answer_keyverify: publickey RSA signature verified
2024 Feb  2 13:33:27 RHEL9 [authpriv.info] sshd: Accepted publickey for
USERREDACTED from AIXSERVER port 40437 ssh2: RSA
SHA256:RSASIGNATUREREDACTED
2024 Feb  2 13:33:27 RHEL9 [authpriv.debug] sshd: debug2:
userauth_pubkey: authenticated 1 pkalg rsa-sha2-512 [preauth]

For failures, we see this (aix7.2 curl+sftp to RHEL9): note that it
makes ssh-rsa references to the same keys used in the success above

2024 Feb  2 13:55:37 RHEL9 [authpriv.debug] sshd: debug2:
userauth_pubkey: valid user USERREDACTED querying public key ssh-rsa
PUBLICKEYREDACTED [preauth]
2024 Feb  2 13:55:37 RHEL9 [authpriv.debug] sshd: debug1:
userauth_pubkey: test pkalg ssh-rsa pkblob RSA
SHA256:RSASIGNATUREREDACTED [preauth]
2024 Feb  2 13:55:37 RHEL9 [authpriv.debug] sshd: debug1:
/home/USERREDACTED/.ssh/authorized_keys:33: matching key found: RSA
SHA256:RSASIGNATUREREDACTED
2024 Feb  2 13:55:37 RHEL9 [authpriv.info] sshd: Accepted key RSA
SHA256:RSASIGNATUREREDACTED found at
/home/USERREDACTED/.ssh/authorized_keys:33
2024 Feb  2 13:55:37 RHEL9 [authpriv.debug] sshd: debug3:
mm_answer_keyallowed: publickey authentication test: RSA key is allowed
2024 Feb  2 13:55:37 RHEL9 [authpriv.debug] sshd: debug2:
userauth_pubkey: authenticated 0 pkalg ssh-rsa [preauth]
2024 Feb  2 13:55:37 RHEL9 [authpriv.debug] sshd: debug2:
input_userauth_request: try method publickey [preauth]
2024 Feb  2 13:55:37 RHEL9 [authpriv.debug] sshd: debug2:
userauth_pubkey: valid user USERREDACTED attempting public key ssh-rsa
PUBLICKEYREDACTED [preauth]
2024 Feb  2 13:55:37 RHEL9 [authpriv.debug] sshd: debug3:
userauth_pubkey: have ssh-rsa signature for RSA
SHA256:RSASIGNATUREREDACTED [preauth]
2024 Feb  2 13:55:37 RHEL9 [authpriv.debug] sshd: debug1:
/home/USERREDACTED/.ssh/authorized_keys:33: matching key found: RSA
SHA256:RSASIGNATUREREDACTED
2024 Feb  2 13:55:37 RHEL9 [authpriv.info] sshd: Accepted key RSA
SHA256:RSASIGNATUREREDACTED found at
/home/USERREDACTED/.ssh/authorized_keys:33
2024 Feb  2 13:55:37 RHEL9 [authpriv.debug] sshd: debug3:
mm_answer_keyallowed: publickey authentication: RSA key is allowed
2024 Feb  2 13:55:37 RHEL9 [authpriv.debug] sshd: debug3:
mm_answer_keyverify: publickey RSA signature unverified: error in
libcrypto
2024 Feb  2 13:55:37 RHEL9 [authpriv.info] sshd: Failed publickey for
USERREDACTED from AIXSERVER port 40489 ssh2: RSA
SHA256:RSASIGNATUREREDACTED
2024 Feb  2 13:55:37 RHEL9 [authpriv.debug] sshd: debug2:
userauth_pubkey: authenticated 0 pkalg ssh-rsa [preauth]

And that is where I'm wondering if a bug exists. Does the client think
it signed with the old sha1 signature maybe (that's implied in what it
told the server: "have ssh-rsa signature")? Otherwise, why wouldn't
reference rsa-sha2-512 signature? And, with crypto policy allowances,
shouldn't that work? Or could it be doing the "transparent algorithm
upgrade" to e.g. rsa-sha2-512, and thus client rsa-sha1 sig != server
rsa-sha2-512?

Grasping at straws a bit here, I know, but our configuration is scary
wide open (ok, no DSS, but... :) ), and yet "error in libcrypto" using
user RSA keys. Thanks!!!

(was trying to be concise here, but happy to provide more complete
logs, tests, etc in attachments as requested)

-- 
You are receiving this mail because:
You are watching the assignee of the bug.