Setting up an OpenVPN router on the Pi is pretty straight forward but what about performance? How much performance do we lose by using the Raspberry Pi or the Banana Pi? I have been testing a few different models to see what the overall performance difference is. I also wanted to compare them against each other. OpenVPN is heavy on the CPU due to it's encryption, there are a lot of guides out there about turning the encryption of but why even use a VPN then? It all depends on what you use your VPN tunnel for and what kind of through put you actually need. In this test I have used all three main versions of the Raspberry Pi and a Banana Pi.
A good VPN gateway of any kind needs two things, CPU and bandwidth! Here the earlier models of the Raspberry is at a disadvantage, both when it comes to CPU and the network since they only have 100Mbit nics. The Banana Pi has the advantage of 1Gbit nic but lacks computing power same as the earlier Raspberry Pi models. It all comes down to what use case you need. In my implementation I want a default gateway on my network that can provide me with a "Swedish" internet connection for play channels from home. To be able to stream news and series that are only available to a Swedish ip address. So I don't really need the entire bandwidth of my internet connection. But at the same time performance is always nice!
The other disadvantage you have with a VPN tunnel is latency, or ping as the gamers call it. The time it takes for a package to go to a server and for you two receive the confirmation package back. This is very important for TCP based protocols that verifies all data transfers. For UDP, used for streaming, bandwidth is more important. So a VPN connection, even with a high latency, that have decent bandwidth can still perform well for streaming or torrents. You can read more about how to setup an OpenVPN gateway on the Pi.
How did I do the test
I used speedtest.net using the closest test server to my exit point and the same each time. I ran this on my laptop connected to my local LAN via cable using the Pi as the default gateway to access the internet. The reason why I did this was to simulate what performance I would actually get in the real implementation. I also installed speedtest-cli on the Pi to run a few tests but more on that later in this article. You can easily install it with:
sudo apt-get install speedtest-cli
And then just run it with:
I ran the test a few times and took the best "score" produced for each model. As you can see in the results there are pretty big margin of error in the tests them self's.
Internet connection, without any VPN:
Down: 180.38 Mb/s
Up: 12.18 Mb/s
Laptop running the OpenVPN client (Dell Precision M4800, 2.5Ghz 8 cores, 32Gb memory, Gbit-nic)
Down: 19.13 Mb/s
Raspberry Pi Model B (700Mhz single core, 512Mb memory, 100Mbit-nic)
Down: 9.02 Mb/s
Up: 3.13 Mb/s
Raspberry Pi 2 Model B (900Mhz quad core, 1Gb memory, 100Mbit-nic)
Down: 10.67 Mb/s
Up: 4.49 Mb/s
Raspberry Pi 3 Model B (1.2Ghz quad core, 1Gb memory, 100Mbit-nic)
Down: 12.16 Mb/s
Up: 3.59 Mb/s
Banana Pi M1 (1Ghz dual core, 1Gb memory, Gbit-nic)
Down: 8.99 Mb/s
Up: 2.81 Mb/s
I also ran a few tests with speedtest-cli on the Pi's them self's without the VPN running. Here I could see a huge difference with the Nic speed! The Raspberries leveled out between 35-45Mb/s while the Banana Pi got 170 Mb/s! So yes Nic speed does matter!
First conclusion is that my VPN provider sucks! It's not really a geo-location VPN but used for privacy back home, had the account since before that's why I still use it. I think this numbers would be very different with a faster VPN provider. You can however see the difference between the CPU speeds. If the VPN baseline from my laptop would have been higher I would have taken the time to install an Odroid C1 which has both the CPU horsepower as well as the Gbit nic. But I will defiantly revisit this when I have a better VPN to try with so I can floor out the CPU's on all these boxes.
When I started these tests my daily driver OpenVPN gateway was a Raspberry Pi Model B, it still is. The performance is enough for what I'm using it for and the difference in speed is to little for me to sacrifice a Raspberry Pi 2 or 3 that can be used for other projects at this time. But as I said I will revisit this one when I have better VPN to run on!