getgrouplist: groups list too small on AIX / GID greater or equal LONG_MAX

[Winkel,Ralf]

Dear SSH developers, 

Thank you for your valuable work!
I found the following issue with openssh Server on AIX:

Login to a sshd (OpenSSH_8.1p1, OpenSSL 1.0.2u  20 Dec 2019 (from the IBM download site)),
fails with "getgrouplist: groups list too small" 
if user is member in a group with GID >= 2147483647 (LONG_MAX)
and sshd_conf contains a "match group ..." statement. 
AIX version doesn't matter, tested on 7200-04-02-2028 and 7100-05-05-1937.

	...
	debug1: userauth-request for user winkel2 service ssh-connection method none [preauth]
	debug1: attempt 0 failures 0 [preauth]
	debug3: mm_getpwnamallow entering [preauth]
	debug3: mm_request_send entering: type 8 [preauth]
	debug3: mm_getpwnamallow: waiting for MONITOR_ANS_PWNAM [preauth]
	debug3: mm_request_receive_expect entering: type 9 [preauth]
	debug3: mm_request_receive entering [preauth]
	debug3: mm_request_receive entering
	debug3: monitor_read: checking request 8
	debug3: mm_answer_pwnamallow
	debug2: parse_server_config: config reprocess config len 456
	debug3: checking match for 'group SFTPONLY' user winkel2 host 10.1.2.3 addr 10.1.2.3 laddr 10.1.2.3 lport 23
	getgrouplist: groups list too small
	debug1: do_cleanup
	...

I assume that it happens because getgrouplist() in openssh-portable/openbsd-compat/port-aix.c 
returns -1 if it finds a GID >= LONG_MAX.  
The code in getgrouplist() in port-aix.c is stil the same in the actual openssh version. 
The issue doesn't happen with OpenSSH_7.5p1, OpenSSL 1.0.2t  10 Sep 2019 (also from the IBM download site).
I don't understand, why OpenSSH_7.5p1 is not affected even if it has the same code in getgrouplist().

Due to 
   https://www.ibm.com/docs/en/aix/7.2?topic=accounts-user-group-ids
   https://www.ibm.com/docs/en/aix/7.2?topic=files-limitsh-file
AIX permits UIDs and GIDs up to 4294967295 (ULONG_MAX).



May I suggest to replace 

	l = strtol(grp, NULL, 10);
	if (ngroups >= maxgroups || l == LONG_MIN || l == LONG_MAX) {

with 

	l = strtoll(grp, NULL, 10);
	if (ngroups >= maxgroups || l < 0 || l > UID_MAX) {

in the port-aix.c file.



Thank you very much
Ralf Winkel