X-forwarding stops working after a while

[Yuri]

I connect from host into the guest VM with X-forwarding (ssh -X), so 
that programs launched in the VM will run on the host display.

X-forwarding works first, but after a while it stops working. Connection 
(from inside of the VM) on the corresponding port (6010 or similar) is 
immediately closed by the foreign host. Ssh connection itself is okay 
though. Reconnecting ssh helps, but later the same thing repeats.

Why would this be happening?

FreeBSD-10.2, both host and guest VM.

Yuri