[PATCH 0/5] ZSTD compression support for OpenSSH

[Sebastian Andrzej Siewior]

On 2020-09-08 09:34:15 [+0200], Sebastian Andrzej Siewior wrote:
> Regarding statistics, do you have anything specific in mind?

Another data point: I have a VM image of 8GiB. To copy it via `scp' over
a gbit link (Ryzen7 on the sending/compressing side):

uncompressed:
|vm.img  100% 8192MB 111.8MB/s   01:13
|Transferred: sent 8600429856, received 1510848 bytes, in 73.4 seconds
|Bytes per second: sent 117222196.9, received 20592.6

zstd:
|vm.img  100% 8192MB 149.7MB/s   00:54
|Transferred: sent 943612408, received 5181584 bytes, in 54.8 seconds
|Bytes per second: sent 17220903.6, received 94563.8
|debug1: compress outgoing: raw data 8594657653, compressed 937958643, factor 0.11
|debug1: compress incoming: raw data 1665529, compressed 2220303, factor 1.33

zlib:
|vm.img  100% 8192MB  46.0MB/s   02:57
|Transferred: sent 1438436560, received 1422144 bytes, in 178.0 seconds
|Bytes per second: sent 8080210.1, received 7988.7
|debug1: compress outgoing: raw data 1574650157, compressed 362009415, factor 0.23
|debug1: compress incoming: raw data 119367, compressed 56535, factor 0.47
 
In the uncompressed case the link was the limiting speed. With zstd's
compression the remote side was the limiting factor (mostlikely the
disk). With zlib's compression, the CPU was the limiting factor.

After compressing it with `xz' the file size was 848MiB. Sending tha
file resulted in:

uncompressed:
| vm.img.xz   100%  848MB 112.1MB/s   00:07
| Transferred: sent 890169768, received 138532 bytes, in 7.6 seconds
| Bytes per second: sent 116547054.4, received 18137.5
 
zstd:
| vm.img.xz   100%  848MB 112.1MB/s   00:07
| Transferred: sent 890388312, received 192832 bytes, in 7.6 seconds
| Bytes per second: sent 116563363.4, received 25244.2
| debug1: compress outgoing: raw data 889869293, compressed 889950452, factor 1.00
| debug1: compress incoming: raw data 62005, compressed 82269, factor 1.33
 
zlib:
| vm.img.xz   100%  848MB  35.2MB/s   00:24
| Transferred: sent 890667956, received 157768 bytes, in 24.2 seconds
| Bytes per second: sent 36815106.4, received 6521.2
| debug1: compress outgoing: raw data 890112824, compressed 889965916, factor 1.00
| debug1: compress incoming: raw data 70699, compressed 33689, factor 0.48

There is no visible overhead with zstd compression compared to
uncompressed. 

Sebastian