mac_int() does not call HMAC_CTX_init()

[Robert Dugal]

Last year I sent the email below describing a bug in openssh 5.6p1.
I just noticed this bug still exists in 5.9p1.

Probably my earlier email was treated as spam because it was in HTML format. 


-- 
Robert Dugal	Team Lead SSL & PKI Group
Certicom Corp.	A Subsidiary of Research In Motion
                  4701 Tahoe Blvd., Building A
                  Mississauga, ON
                  L4W 0B5
 
rdugal at certicom.com
direct       	+1.289.261.4148
mobile      	+1.416.276.8062
main         	+1.905.507.4220
fax             	+1.905.507.4230
www.certicom.com





From: Robert Dugal 
Sent: Wednesday, September 29, 2010 1:02 PM
To: openssh-unix-dev at mindrot.org
Subject: bug in openssh - mac_int() does not call HMAC_CTX_init()

I have been doing some work with OpenSSH 5.6p1, attempting to use an alternative engine for crypto.
In mac.c, the function mac_init() calls HMAC_Init() without previously having called HMAC_CTX_init().
However, OpenSSL documentation states that HMAC_CTX_init() is mandatory.

http://www.openssl.org/docs/crypto/hmac.html

HMAC_CTX_init() initialises a HMAC_CTX before first use. It must be called. 
HMAC_CTX_init() must have been called before the first use of an HMAC_CTX in this function. N.B. HMAC_Init() had this undocumented behaviour in previous versions of OpenSSL - failure to switch to HMAC_Init_ex() in programs that expect it will cause them to stop working

While this appears to cause no issues with OpenSSH 5.6p1 using OpenSSL 1.0.0a and the default crypto engine,
it may cause problems (such as segfaults in my case) when attempting to use an alternative engine.
-- 
Robert Dugal                        Senior Software Developer
Certicom Corp.                      A Subsidiary of Research In Motion 
rdugal at certicom.com
direct        905.501.3848
fax             905.507.4230
www.certicom.com


---------------------------------------------------------------------
This transmission (including any attachments) may contain confidential information, privileged material (including material protected by the solicitor-client or other applicable privileges), or constitute non-public information. Any use of this information by anyone other than the intended recipient is prohibited. If you have received this transmission in error, please immediately reply to the sender and delete this information from your system. Use, dissemination, distribution, or reproduction of this transmission by unintended recipients is not authorized and may be unlawful.