OpenSSH-2.2.0 problem with ssh.com ssh-agent2

[Richard E. Silverman]

The OpenSSH and ssh.com agents use a completely different set of messages for
dealing with DSA keys, so I don't expect the OpenSSH client to be able to get
DSA keys from ssh-agent2.  However, if I'm running "ssh-agent2 -1", I expect
OpenSSH to be able to use RSA keys stored in ssh-agent2's agent1 compatibility
box.  And it does.  However, there's a problem.  If I'm running "ssh-agent2
-1", and try to use the OpenSSH client, protocol 2, with DSA authentication
enabled, I get this:

  Bad authentication reply message type: 102

...and the client exits.  Again, I wasn't expecting it to get any DSA keys
from the agent, but it should use the keys on disk, or fall through to
password authentication.  But it just dies.

What happens is that OpenSSH issues one of its own set of DSA-related agent
messages, which ssh-agent2 doesn't understand, so the agent sends back an
"agent failure" message.  This should be fine; the OpenSSH code looks for
that.  However, there are two "agent failure" messages in the ssh.com world:

#define SSH1_AGENT_FAILURE              5
#define SSH_AGENT_FAILURE               102

OpenSSH only checks for the first one, but the agent returns the second one.
OpenSSH doesn't recognize it, and dies.

Below is a proposed fix, as diffs to authfd.c and authfd.h.

--
  Richard Silverman
  slade at shore.net

================================================================================

*** authfd.h	Sun Oct  8 17:45:16 2000
--- authfd.h.orig	Tue Aug 22 20:46:24 2000
***************
*** 39,50 ****
  #define SSH2_AGENTC_REMOVE_IDENTITY		18
  #define SSH2_AGENTC_REMOVE_ALL_IDENTITIES	19
  
- /* messages used by the ssh.com ssh-agent2 */
- #define SSH_COM_AGENT_FAILURE                   102
- 
- /* macro to check for "agent failure" message */
- #define agent_failed(x) ((x == SSH_AGENT_FAILURE) || (x == SSH_COM_AGENT_FAILURE))
- 
  typedef struct {
  	int     fd;
  	Buffer  identities;
--- 39,44 ----


*** authfd.c	Sun Oct  8 17:45:28 2000
--- authfd.c.orig	Tue Aug 22 20:46:24 2000
***************
*** 223,229 ****
  
  	/* Get message type, and verify that we got a proper answer. */
  	type = buffer_get_char(&auth->identities);
! 	if (agent_failed(type)) {
  		return NULL;
  	} else if (type != code2) {
  		fatal("Bad authentication reply message type: %d", type);
--- 223,229 ----
  
  	/* Get message type, and verify that we got a proper answer. */
  	type = buffer_get_char(&auth->identities);
! 	if (type == SSH_AGENT_FAILURE) {
  		return NULL;
  	} else if (type != code2) {
  		fatal("Bad authentication reply message type: %d", type);
***************
*** 322,328 ****
  	}
  	type = buffer_get_char(&buffer);
  
! 	if (agent_failed(type)) {
  		log("Agent admitted failure to authenticate using the key.");
  	} else if (type != SSH_AGENT_RSA_RESPONSE) {
  		fatal("Bad authentication response: %d", type);
--- 322,328 ----
  	}
  	type = buffer_get_char(&buffer);
  
! 	if (type == SSH_AGENT_FAILURE) {
  		log("Agent admitted failure to authenticate using the key.");
  	} else if (type != SSH_AGENT_RSA_RESPONSE) {
  		fatal("Bad authentication response: %d", type);
***************
*** 366,372 ****
  		return -1;
  	}
  	type = buffer_get_char(&msg);
! 	if (agent_failed(type)) {
  		log("Agent admitted failure to sign using the key.");
  	} else if (type != SSH2_AGENT_SIGN_RESPONSE) {
  		fatal("Bad authentication response: %d", type);
--- 366,372 ----
  		return -1;
  	}
  	type = buffer_get_char(&msg);
! 	if (type == SSH_AGENT_FAILURE) {
  		log("Agent admitted failure to sign using the key.");
  	} else if (type != SSH2_AGENT_SIGN_RESPONSE) {
  		fatal("Bad authentication response: %d", type);
***************
*** 513,519 ****
  {
  	switch (type) {
  	case SSH_AGENT_FAILURE:
- 	case SSH_COM_AGENT_FAILURE:
  		log("SSH_AGENT_FAILURE");
  		return 0;
  	case SSH_AGENT_SUCCESS:
--- 513,518 ----