ssh host keys on cloned virtual machines

[Thorsten Glaser]

On Mon, 27 Feb 2023, Nico Kadel-Garcia wrote:

>> > does any one of you have a best practice on renewing ssh host keys on cloned
>> > machines?
>>
>> Yes: not cloning machines.
>
>Good luck with *that*. Building VM's from media is a far, far too
>lengthy process for production deployment, especially for auto-scaling
>clusters.

(It’s “VMs”, no genitive apostrophe.)

What media? debootstrap + a local mirror = fast.
In fact, much faster than cloning, possibly large, filesystems,
unless you use CoW, which you don’t because then you’re overcommitting.

>> There’s too many things to take care of for these. The VM UUID in
[…]

>That's what the "sysprep" procedure is for when generating reference
>VM images, and "cloud-utils" for setting up new VMs from images, at

What guarantees you “sysprep” and “cloud-utils” find everything that
needs to be changed?

(I’m not sure where inode generation numbers are (still) a concern,
on what filesystems, anyway. They only come into play with NFS,
AFAIK, though, so that limits this issue. When they come into play,
however, they’re hard to change without doing a newfs(8)…)

>If people really feel the need for robust random number services,
>they've got other problems. I'd suggest they either apply an init
>script to reset whatever they feel they need on every reboot, or find

I think you’re downplaying a very real problem here, as an aside.

>The more host-by-host customization, admittedly the more billable
>powers and the more yourself personally into each and every stop. But
>it doesn't scale

Huh? Scripting that creation from scratch is a job done once that
scales very well. debootstrap is reasonably fast, installation of
additional packages can be fast as well (since it’s a new image,
use eatmydata or the dpkg option I’ve not yet remembered).

And, given the system’s all-new, I believe this is even more
reliable than cloning something customised, then trying to
adapt *that* to the new requirements.

>, and you will eventually be told to stop wasting your
>time if your manager is attentive to how much time you're burning on
>each deployment.

If I’ve scripted the image creation, it’s no more work than
a cloning approach.

>> This is even more true as every new machine tends to get just the
>> little bit of difference from the old ones that is easier to make
>> when not cloning (such as different filesystem layout, software).
>
>And *that* is one of the big reasons for virtualization based
>deployments, so people can stop caring about the physical subtleties.

?!?!?!

How does that translate into needing, say, 8 GiB HDD for some VMs but
32 GiB HDD for some others?

This has *NOTHING* to do with physical vs virtual platforms.

>predicted, nor was reverse DNS likely to work at all which was its own
>distinct burden for logging *on* those remote servers.

Maybe invest your time into fixing infrastructure then…

>> (Fun fact on the side, while doing admin stuff at $dayjob, I even
[…]

>You probably don't work on the same scale I've worked, or had to

No, not for that. If I had to do it at larger scale I would have
scripted it. I didn’t so it turned out to be cheaper, work-time-wise,
to do part of the steps by hand the few times I needed to do it.
I don’t admin stuff at work for others any more, so that point is
moot. But I did want to share this as an anecdote: when scaling very
small, the “stupid” solution may be better than a clever one.

>It's also a great way to stretch
>our billable hours with very familiar tasks which only you know how to
>do.

I don’t need to do that. Besides, I’m employed, not freelancing,
so I don’t even have to care about billable hours.

bye,
//mirabilos
-- 
Infrastrukturexperte • tarent solutions GmbH
Am Dickobskreuz 10, D-53121 Bonn • http://www.tarent.de/
Telephon +49 228 54881-393 • Fax: +49 228 54881-235
HRB AG Bonn 5168 • USt-ID (VAT): DE122264941
Geschäftsführer: Dr. Stefan Barth, Kai Ebenrett, Boris Esser, Alexander Steeg

                        ****************************************************
/⁀\ The UTF-8 Ribbon
╲ ╱ Campaign against      Mit dem tarent-Newsletter nichts mehr verpassen:
 ╳  HTML eMail! Also,     https://www.tarent.de/newsletter
╱ ╲ header encryption!
                        ****************************************************