Unable to build OpenSSH-9.8p1 with custom SSL

[radiatejava]

Hello team
I am trying to build OpenSSH 9.8p1 with my SSL installed at
/opt/customssl, but it fails. It fails at the configure cmd itself. It
does not seem to be honor --with-ssl-dir=/opt/customssl option. What
could be the issue here?

$ ./configure --with-pam --with-ssl-dir=/opt/customssl
<<some output removed>>
checking for openssl... /usr/bin/openssl
checking for openssl/opensslv.h... yes
checking OpenSSL header version... 1010119f (CustomSSL 1.1.1y.7.3.377)
checking for OpenSSL_version... yes
checking for OpenSSL_version_num... yes
checking OpenSSL library version... not found
configure: error: OpenSSL library not found.

When building OpenSSH 9.6p1:

$ ./configure --with-pam --with-ssl-dir=/opt/customssl
<<some output removed>>
checking for openssl... /opt/customssl/bin/openssl
checking openssl/opensslv.h usability... yes
checking openssl/opensslv.h presence... yes
checking for openssl/opensslv.h... yes
checking OpenSSL header version... 1010117f (CustomSSL 1.1.1w.7.2.555)
checking for OpenSSL_version... yes
checking for OpenSSL_version_num... yes