Two big flaws in the `configure` script -- have located them, cannot fix

[Gordon Steemson]

Hello all,

On 06/09/2023, Darren Tucker <dtucker at dtucker.net> wrote:
> On Thu, 7 Sept 2023 at 15:07, Gordon Steemson <gsteemso at gmail.com> wrote:
>> I'm trying to install OpenSSH and have run into two different things
>> that cause the `configure` script to come to a violent halt.  The
>> first is [...the zlib version check that's already been fixed, nothing to see here folks...]
>> The second problem is more serious.  While I can find no reason for
>> the code that checks the OpenSSL library version to fai[...]
>> I hope you folks have better luck investigating this than I've had.
>
> That's unlikely since you have not provided any details that might
> allow us to do so, including but not limited to: what the error
> message was,

Yes. My bad.  To start off -- the ./configure output proceeded
uneventfully until:

$ checking for openssl... /usr/bin/openssl
$ checking for openssl/opensslv.h... yes
$ checking OpenSSL header version... not found
$ configure: error: OpenSSL version header not found.

>From squinting at the `configure` script, this appears to mean that
the test program fell over somewhere in the brief span between
#include-ing the header file and trying to use the values declared
therein.  It all looks spelled correctly... no clues there.

> the OS/platform, compiler, OpenSSL implementation and version, configure options, CFLAGS/LDFLAGS, what the config.log contained.

I spent a while typing all of that out and then discovered it was all
duplicated in `config.log`; then I further discovered that most of it
is probably irrelevant, as the problem seems to be caused by the
configure script randomly inserting an '/opt/' into my openssl path,
then acting surprised when the crypto library isn't at the new,
nonexistent location.  There is no 'opt' component in any of the
environment variables, nor in any command-line arguments.

In more detail:  All of the software components involved are in the
Homebrew repository on my Mac, each of them located at
`/Users/gsteemso/Brewery/Cellar/{packagename}/{version}/`.  (I have
instead downloaded the OpenSSH package, and only the OpenSSH package,
directly from you guys as the Homebrew version may or may not be not
working properly -- I can't really tell.)

For whatever reason, the `configure` script -- which had thitherto
been happily chewing on my OpenSSL installation in
`/Users/gsteemso/Brewery/Cellar/openssl/1.1.1v/*` -- abruptly tried to
load `/Users/gsteemso/Brewery/opt/openssl/lib/libcrypto.1.1.dylib`,
and then complained when the library was not found at this fictitious
location.  (Note how the `Cellar/openssl/1.1.1v` section was abruptly
replaced by an `opt/openssl` out of nowhere.)  A text search of your
entire installation package found mention of hard-coded pathnames that
look like this in exactly one place -- inside the github configuration
files, which should not have even been under consideration as I do not
yet have a git installation (constructing one is in fact the entire
reason I am currently attempting to install OpenSSH).

The reason I am telling you this, even though the problem may or may
not in fact be with the OpenSSH configuration at all, is that the
attempt to load a crypto library from the wrong place appears to be
taking place in the beginning of the little test program that wants to
read OpenSSL's version-number declarations.  The error message
produced is thus wholly misleading, and without extensive testing
based on a single line in the config.log, I would not have discovered
even as little as I have.

At this point my hypothesis is that the chain of #include statements,
in the various OpenSSL headers, is accidentally doing something stupid
in a way that has not yet affected very many people.  Should I
determine that there is in fact a problem with the OpenSSH side beyond
the suboptimal `configure` messages, I will report back here; in the
meanwhile, I believe I have probably wasted enough of your time
already.

Thank you all for all that you do here,

Gordon Steemson

-- 
The world’s only gsteemso