How sshd spawns and reuses pids

[Zakaria]

Hi All,

I hope everyone is doing well. I have a question and I would appreciate 
if anyone can answer but first I want to thank so much all contributors 
for deliverig previous as well as the latest releases with its promising 
login session separation implementation having each session is spawned 
in its own process which I hope isnt what is causing my issue.

Before I write my question, it might help understanding if I provide the 
context first, and therefore I come from background of having negative 
experience with sshd that drove me to analyse every corner in linux 
system I can and try to understand what is happening and more write 
thousands of lines of code trying to resolve the issue yet to no avail I 
found answer to the issue which unfoled with question and I am not being 
able to rectify it and after so many months of distraught suspecting my 
server is being compromised, I thought now to reach to the developers of 
opensshd with hope they might be able to answer my question which I 
couldnt find as it would allow me to rectify the issue with itself.

I wrote an EDR application and part of it, has a functionality to keep 
track of logged in sessions, and report whenever new ssh session was 
started. I had this experience, after I terminate all sessions I 
started, sometimes hours later like today and other days later, and in 
random times, I get notification of new ssh session was started, and 
when I login to server to inspect, I find no login session and no 
hinting traces in netstat, top, secure and messages log, utmp dump etc. 
I noticed all of session with specific PIDs which is getting reported as 
was active for very short period of time, are identical to previous 
sessions I started and have already terminated.

After observing the server for months, I noticed I get report of new 
session report whenever I run sudo -i who -a command, given because it 
was only through sudo by root, for strange reasons, it activates latest 
ssh session which I have started and terminated it from previous session 
with identical PID and reuses to perform the sudo command.

But now I observed the logs in messages and secure file and monitored 
all services which may run any command with sudo and tested them, none 
report new session likewise sudo -i who -a and no application runs even 
this command itslef, and still till today in random times, I get report 
in server ssh session started via openssh 10, while its pid belongs to 
previous session I made 100% sure it was terminated, based on sudo 
experience, it looks like somewhere some commands gets executed in linux 
in suspicious times when I am not on my laptop, and it requires root and 
uses sudo, which cause to activate and reuses terminated ssh sessions, 
out of no where like sudo -i who -a did. If anyone want to confirm, run 
it continously with && who -a and see the iteration of different PIDs 
everytime gets printed in the latest terminated session at bottom of the 
output.

Now, I have no application nor service runs any command with sudo, which 
may cause the terminated session reuse and makes me wonder more which 
part of linux is responsible of such ssh login session? Anyone have any 
idea? Isnt my server is being looking like compromised? I really wanted 
to build the most secure email application in the world, and I have been 
working on for years still not being able to reach the confidence I can 
deliver it with this random ssh session gets activated by someone isn't 
myself, could it be my VPS hosting provider, runs some periodic and 
random maintenance and executes some sudo commands?

I believed that I developed very smart EDR application as part of one of 
my dreams hoping to be completed soon, as most secure email application 
yet this issue puzzles and doubts me and more than ever again I will 
appreciate if anyone have any idea about what is truly happening to 
share it.

Also, as part of EDR, I developed reverse shell, hidden and suspicious 
processes as well as suspicious connections and opened ports for 
processes which isnt expected to establish, also kernel modules 
tampering, detection features and in times when I receive the ssh 
session gets activated for few seconds, and itself based on sudo -i who 
-a experience again, I inferred, usually such what I hope a false 
positive can happen when sudo commands gets executed, my applicaion 
detect no suspicious activity beside for some MSs to 1 second duration 
root ssh session gets activated?

I hope its not another next level of sophisticated rootkit implanted in 
kernel or memory but, in short if anyone has any clarification, to prove 
my server isnt compromised and no one had an ability to escalate 
privileges and run root commands yet then why randomly I see active ssh 
session when no one truly is logged? Any answer, again I will appreciate 
it.

Lastly, here is my most relevant question, Does and How sshd spawns and 
reuses ssh session with previous pids? and how it spawns each session 
processes and allocate its PID?

Could it be something like other services which runs sudo commands 
actually yields enumeration through sshd pid range continously until it 
hits previous recent terminated session which activates for as long as 
it runs, and thus produce almost like to my tracking of ssh pids, pid 
collision effect?

I really doubted every conclusion I reached, and I hope if anyone can 
help me on reaching a resolve and understand what is happening and help 
me deliver my dream email application soon to the world, any suggestion 
of place to investigate in sshd configs, service, process or anywhere 
else in Linux e.g. log any attempt to start ssh session before it gets 
started and the code execution which started? Any recommendation to do 
this as part of sshd binary? E.g. I can apply some code tweaks on source 
before compiling?

With thanks.

Zak.