OpenSSH FIPS 140-2 support using OpenSSL FIPS modules?

[Tomas Kuthan]

On 12/ 4/15 03:26 AM, security veteran wrote:
> Hi All:
>
> I tried to rebuild openssl with the FIPS modules, and then install the new
> openssl libs (lib crypto.so to be specific) on my Ubuntu 12.04 box.
>
> After that I noticed it seemed to break OpenSSH: I couldn't login to the
> box using ssh, and couldn't run the client command like ssh-keygen either.
>
> My questions are:
>
> 1. Does OpenSSH support FIPS mode?
>
> 2. Or does OpenSSH support with OpenSSL FIPS modules?
>
> 3. Is there a way to re-compile OpenSSH by turning on/off some flags to
> make it FIPS complaint?
>
> 4. Does the RedHat OpenSSH FIPS modules (
> http://csrc.nist.gov/groups/STM/cmvp/documents/140-1/140sp/140sp1791.pdf)
> also open sourced to the OpenSSH community?

Hi security veteran,

vanilla OpenSSH doesn't support running OpenSSL in FIPS-140 mode. Some 
downstream providers patch OpenSSH they deliver with their distributions 
with changes to enable FIPS-140 mode.

In general, an application that wants to run a FIPS-140 capable OpenSSL 
library in FIPS-140 mode, needs to call FIPS_mode_set() first. Otherwise 
it runs OpenSSL in default mode with non-FIPS algorithms available. From 
my experience this works, but is not FIPS-140 compliant.

User Guide for the OpenSSL FIPS Object Module v2.0 [1]:
> 2.6
> FIPS Mode of Operation
> Applications that utilize FIPS mode must call the FIPS_mode_set() function. After successful
> FIPS mode initialization, the non-FIPS algorithms will be disabled by default.
> The FIPS Object Module together with a compatible version of the OpenSSL product can be used
> in the generation of both FIPS mode and conventional applications. In this sense, the combination
> of the FIPS Object Module and the usual OpenSSL libraries constitutes a “FIPS capable API”, and
> provide both FIP approved algorithms and non-FIPS algorithms.

Vanilla OpenSSH obviously doesn't call FIPS_mode_set(). If switching 
underlying OpenSSL libcrypto to FIPS-140 capable instance precludes you 
from running ssh, most probably there is something wrong with the 
FIPS-140 capable OpenSSL you built.

Tomas

[1] https://openssl.org/docs/fips/UserGuide-2.0.pdf