Option to make IdentityFile act like other options

[shawn wilson]

I would like an option so that IdentityFile only allows one file path
and ignores others after one is set. I mainly make this work with:

Host a* b* c*
  IdentityFile foo

Host *
  IdentityFile bar

But for some reason the sorting is wrong w/ some hosts (or maybe it's
an actual random hash like lookup table now?) and I'm getting some:
Aug  2 17:56:16 host sshd[2278]: Failed publickey for swilson from
1.2.3.4 port 45057 ssh2: ED25519
SHA256:m7EFGJRMFAcMVakPCm+atVGmvwkoVM61jaNy7N+ZUSU

Which is kind of annoying (I don't want to ignore "Failed publickey"
messages but the noise is annoying).