/dev/blog

The achievable speed of copying a file with OpenSSH (e.g. with scp) will depend on quite a few different factors such as CPU speed, CPU architecture, Network throughput, OpenSSH implementation, OS, hard drive speed, etc. But how much of a difference does choosing a different cipher algorithm make? And what’s the fastest OpenSSH cipher algorithm?

Turns out, there’s no simple answer to this question, since most of the factors that influence the transfer speed can be ruled out, but the results will at least depend on the hardware platform and OpenSSH version. There’s quite a few different benchmarks out there, e.g. for the Bifferbord, E5 xeon CPUs or different consumer grade CPUs and ARM processors. But since the results are so heavily platform dependent, it’s a good idea to run your own benchmark on the particular platform you are interested in. So here’s another data point for an Intel Xeon E5-2640 and OpenSSH 6.9p1 (OpenSSL 1.0.1k).

The test setup is quite similar to the one described at blog.famzah.net. The bash script used to produce the data is:

for cipher in aes128-ctr aes192-ctr aes256-ctr arcfour256 arcfour128 aes128-cbc 3des-cbc blowfish-cbc cast128-cbc aes192-cbc aes256-cbc arcfour
do
   echo "$cipher"
   for try in 1 2
   do
      scp -c "$cipher" /tank/fs/testfile.img root@localhost:/tank2/fs/
   done
done

The test file consists of 5GiB random data. Both the source and target file system are RAM backed to remove the influence of HDD read and write speeds. In addition to that, the test file is written to localhost to ensure that network speed, load and NIC drivers do not influence the test results.

SCP file transfer speed

The results clearly show, that the Xeon’s AES instruction set is used. Most modern x86 CPUs do come with this extension these days.

While this data clearly suggests, that AES encryption is the faster cipher OpenSSH cipher (if there is hardware support for it as in this case), copying large amounts of data with scp is not a particularly interesting use case. Sending big streams of data through a pipe into ssh, as you do when you send and receive ZFS snapshots over ssh, is a very common application. For benchmarking reasons, sending actual ZFS snapshots is not ideal, since ZFS takes some extra time to check the receiving file system (and its snapshots) before starting the sending process. So here’s an altered script that should tell us, what the fastest cipher for that particular use case is:

for cipher in aes128-ctr aes192-ctr aes256-ctr arcfour256 arcfour128 aes128-cbc 3des-cbc blowfish-cbc cast128-cbc aes192-cbc aes256-cbc arcfour
do
   echo "$cipher"
   for try in 1 2
   do
      cat /tank/fs/testfile.img | pv | ssh -c "$cipher" root@localhost "cat - > /dev/null"
   done
done

The only difference can be found in the highlighted line: Instead of using scp the file is now piped directly into ssh and discarded on the receiving side. Again, the 5GiB test file lives on a RAM backed file system and the transfer is done to localhost.

SSH piped file transfer speed