Fw: Problem/bug report for "bad decrypted len" error in OpenSSH

[Nils Larsch]

Stefan Hadjistoytchev wrote:
....
> If anyone wants to do a private key sign, and the key is located in a device
> or the Microsoft certificate store in which the private key cannot be
> accessed directly ( you cannot access the private key directly for
> encryption or decryption ) he must use Microsoft Crypto API. That exact
> Microsoft Crypto API method always returns 36 bytes instead of the 35 bytes

The length of the PKCS#1 DigestInfo should depend of the hash alg
and if you select CALG_SHA it should be 35.

> (OpenSSH standard).

That's not a OpenSSH issue, the format of the signature is specified in
PKCS#1.

> 
> A private key sign is the method by which both SSL and SSH uses to do
> authentication. What this really means is the host sends an authentication
> challenge to the host which is signed by the workstation (private key
> encrypt) and is then decrypted with the public key by the host.  Since only
> the owner of the private key can encrypt data which can be decrypted by the
> public key held by the host, the host feels he can trust the connection.
> 
> The method used within the crypto API is:
> 
> (1) Obtain the context for the private key using the public key contained
> within the
> certificate.
> (2) Create a hash of the challenge using the CALG_SSL3_SHAMD5 method.

As far as I know OpenSSH (v2) uses the CALG_SHA method (note: the
length of the CALG_SSL3_SHAMD5 method is indeed 36, but that's not
what we want here, but with this I don't really understand why it
worked without the length check).

> (3) Sign the hash
> (4) return to host
> 
> (Notes:
>     1. This all pertains only to SSH-2.  SSH-1 uses another method, and in
> fact cannot be done using private keys that cannot be accessed directly;

Using OpenSC I can use smartcards and ssh-1 :-)

Nils