Real-world Tailscale iperf3 results between a VM and a bare metal Desktop on a switched network

We tested iperf3 performance using our network based on the following devices:

  • Desktop: Ubuntu 21.10 i7-6700 CPU @ 3.40 GHz, connected using 1Gbase-T to
  • Desktop switch: Mikrotik CSS610-8G-2S+IN connected using 10GBase-T multimode SFP+ module to:
  • Core switch: Mikrotik CRS309-1G-8S+IN, connected using 10GBase-T DAC cable to
  • Virtualization server: i5-6500 CPU @ 3.20GHz running XCP-NG 8.2.1
  • Virtual Machine: Ubuntu 20.04, 4 cores, 8GB RAM

Tailscale version was

1.24.1
  tailscale commit: 1a9302b1edba91d0f638e775faeaa0ce2a6a25f8
  other commit: 1331ed5836e1a0ab32b10e6ce8748e17ba2c7598
  go version: go1.18.1-ts710a0d8610

 

The network is completely switched, not routed and we took care that tailscale actually used the switched connection using tailscale ping.

Test 0: Direct connection over switched network

Desktop running iperf -s, VM running iperf -c 10.9.2.10:

Connecting to host 10.9.2.10, port 5201
[  5] local 10.9.2.103 port 52944 connected to 10.9.2.10 port 5201
[ ID] Interval           Transfer     Bitrate         Retr  Cwnd
[  5]   0.00-1.00   sec  94.7 MBytes   794 Mbits/sec  338    109 KBytes       
[  5]   1.00-2.00   sec  98.0 MBytes   822 Mbits/sec  353    148 KBytes       
[  5]   2.00-3.00   sec  96.6 MBytes   811 Mbits/sec  382    117 KBytes       
[  5]   3.00-4.00   sec   103 MBytes   862 Mbits/sec  334    116 KBytes       
[  5]   4.00-5.00   sec   101 MBytes   851 Mbits/sec  483    102 KBytes       
[  5]   5.00-6.00   sec   104 MBytes   874 Mbits/sec  503    126 KBytes       
[  5]   6.00-7.00   sec   105 MBytes   883 Mbits/sec  527    119 KBytes       
[  5]   7.00-8.00   sec   108 MBytes   906 Mbits/sec  451    105 KBytes       
[  5]   8.00-9.00   sec   108 MBytes   903 Mbits/sec  442    117 KBytes       
[  5]   9.00-10.00  sec   107 MBytes   900 Mbits/sec  461    123 KBytes       
- - - - - - - - - - - - - - - - - - - - - - - - -
[ ID] Interval           Transfer     Bitrate         Retr
[  5]   0.00-10.00  sec  1.00 GBytes   861 Mbits/sec  4274             sender
[  5]   0.00-10.00  sec  1.00 GBytes   860 Mbits/sec                  receiver

iperf Done.

VM running iperf -s, Desktop running iperf -c 10.9.2.103

Connecting to host 10.9.2.103, port 5201
[  5] local 10.9.2.10 port 42630 connected to 10.9.2.103 port 5201
[ ID] Interval           Transfer     Bitrate         Retr  Cwnd
[  5]   0.00-1.00   sec  88.5 MBytes   742 Mbits/sec    0    966 KBytes       
[  5]   1.00-2.00   sec  90.0 MBytes   755 Mbits/sec    0   1.12 MBytes       
[  5]   2.00-3.00   sec  87.5 MBytes   734 Mbits/sec   33    833 KBytes       
[  5]   3.00-4.00   sec  90.0 MBytes   755 Mbits/sec    0    833 KBytes       
[  5]   4.00-5.00   sec  88.8 MBytes   745 Mbits/sec    0   1.00 MBytes       
[  5]   5.00-6.00   sec  88.8 MBytes   744 Mbits/sec    0   1.00 MBytes       
[  5]   6.00-7.00   sec  87.5 MBytes   734 Mbits/sec    0   1.09 MBytes       
[  5]   7.00-8.00   sec  90.0 MBytes   755 Mbits/sec    0   1.09 MBytes       
[  5]   8.00-9.00   sec  90.0 MBytes   755 Mbits/sec    0   1.09 MBytes       
[  5]   9.00-10.00  sec  90.0 MBytes   755 Mbits/sec   13    863 KBytes       
- - - - - - - - - - - - - - - - - - - - - - - - -
[ ID] Interval           Transfer     Bitrate         Retr
[  5]   0.00-10.00  sec   891 MBytes   747 Mbits/sec   46             sender
[  5]   0.00-10.00  sec   888 MBytes   745 Mbits/sec                  receiver

iperf Done.

The direction where the VM hosts the iperf -s server i.e. sends the data shows a slight degradation of performance

Test 1: Desktop running iperf -s, VM running iperf -c

Connecting to host 100.64.0.2, port 5201
[  5] local 100.64.0.3 port 37466 connected to 100.64.0.2 port 5201
[ ID] Interval           Transfer     Bitrate         Retr  Cwnd
[  5]   0.00-1.00   sec  39.4 MBytes   330 Mbits/sec   62    149 KBytes       
[  5]   1.00-2.00   sec  45.8 MBytes   385 Mbits/sec   44    150 KBytes       
[  5]   2.00-3.00   sec  38.9 MBytes   326 Mbits/sec   97    122 KBytes       
[  5]   3.00-4.00   sec  47.9 MBytes   401 Mbits/sec    7    242 KBytes       
[  5]   4.00-5.00   sec  39.5 MBytes   332 Mbits/sec  118    110 KBytes       
[  5]   5.00-6.00   sec  46.6 MBytes   391 Mbits/sec   32    136 KBytes       
[  5]   6.00-7.00   sec  41.8 MBytes   351 Mbits/sec   42    159 KBytes       
[  5]   7.00-8.00   sec  44.3 MBytes   372 Mbits/sec   91    104 KBytes       
[  5]   8.00-9.00   sec  36.1 MBytes   303 Mbits/sec   72    133 KBytes       
[  5]   9.00-10.00  sec  41.5 MBytes   348 Mbits/sec   39    139 KBytes       
- - - - - - - - - - - - - - - - - - - - - - - - -
[ ID] Interval           Transfer     Bitrate         Retr
[  5]   0.00-10.00  sec   422 MBytes   354 Mbits/sec  604             sender
[  5]   0.00-10.00  sec   421 MBytes   353 Mbits/sec                  receiver

iperf Done.

Test 2: VM running iperf -s, Desktop running iperf -c

Connecting to host 100.64.0.3, port 5201
[  5] local 100.64.0.2 port 36744 connected to 100.64.0.3 port 5201
[ ID] Interval           Transfer     Bitrate         Retr  Cwnd
[  5]   0.00-1.00   sec  23.7 MBytes   199 Mbits/sec  104   89.9 KBytes       
[  5]   1.00-2.00   sec  23.6 MBytes   198 Mbits/sec   80   49.2 KBytes       
[  5]   2.00-3.00   sec  21.1 MBytes   177 Mbits/sec   59   54.0 KBytes       
[  5]   3.00-4.00   sec  23.6 MBytes   198 Mbits/sec   68   69.6 KBytes       
[  5]   4.00-5.00   sec  19.1 MBytes   160 Mbits/sec   77   48.0 KBytes       
[  5]   5.00-6.00   sec  25.3 MBytes   212 Mbits/sec   76   62.4 KBytes       
[  5]   6.00-7.00   sec  21.4 MBytes   179 Mbits/sec   50    107 KBytes       
[  5]   7.00-8.00   sec  25.6 MBytes   215 Mbits/sec   35    124 KBytes       
[  5]   8.00-9.00   sec  22.5 MBytes   188 Mbits/sec   71   48.0 KBytes       
[  5]   9.00-10.00  sec  25.0 MBytes   209 Mbits/sec   42   64.8 KBytes       
- - - - - - - - - - - - - - - - - - - - - - - - -
[ ID] Interval           Transfer     Bitrate         Retr
[  5]   0.00-10.00  sec   231 MBytes   194 Mbits/sec  662             sender
[  5]   0.00-10.01  sec   230 MBytes   193 Mbits/sec                  receiver

UDP tests

UDP tests were mostly similar to TCP tests (albeit slightly higher throughput at up to 400 Mbit/s), including the sensitivity to the direction of the connection.

Interpretation of the results

Tailscale has significant impact on network speeds and will not regularly be able to achieve near-Gigabit iperf3 speeds given typical setup with Desktop that are a couple of years old, and virtual machines. However, achieving a throughput of 200-400 Mbit/s is more than enough for most applications.

Interestingly, the speed is highly dependent on the direction of transfer between a less powerful VM and a more powerful Desktop, with a factor of x1.5 … x2 between the two directions. This might be attributed to the amount of computation required to encrypt or decrypt the data.