Cloudways WordPress Performance Value Review

Cloudways is a managed WordPress hosting provider that allows you to deploy your WordPress site onto several different platforms. These platforms are:

Cloudways home page
Cloudways!

If you are just starting with Cloudways it can be tough to figure out where your money is best spent! The prices are different for each platform and how they are sized isn’t them same either. This review gathers data using Kernl’s WordPress Load Testing tool and then massages that data into someone easy to digest.

Load & performance test your own WordPress site with Kernl’s WordPress Load Testing service!

What Was Tested?

We tested the following Cloudways setups:

  • Google Cloud – A “small” instance out of their Northern Virginia data center. $39.36 / month.
  • Amazon Web Services – A “small” instance out of their Northern Virginia data center. $36.51 / month.
  • Vultr – “2GB” plan out of their New Jersey data center. $23 / month
  • Linode – “2GB” plan from their New Jersey data center. $24 / month
  • Digital Ocean – “2GB” plan from one of the New York City data centers. $22 / month.

And we had the following load testing setup:

  • Content was an export of this blog
  • 1000 concurrent users
  • 30 minute duration
  • 3 users per second ramp up
  • Load generating machines were in the San Francisco #2 Digital Ocean data center.

Overall Cloudways Performance and Value

Each provider performed extremely well in our tests. No errors were reported and they all reached over 700 requests / s sustained traffic. However not all hosts are created equal.

AWSGCPLinodeVultrDigital Ocean
Costs / Month (cents)36513936240023002200
Total Requests13409671274018144230114557221445759
$ / 1000 req (cents)2.723.091.661.581.52

Given that each load test was exactly the same, you can see that some hosts performed better than others.

Cost per 1000 requests (in cents)

AWS and GCP are a bit more costly than Linode, Vultr, and Digital Ocean and they also performed worse than the lower cost providers.

Cloudways Provider Response Times

While cost per requests is a good metric we also care about the quality of those requests. Quality can be measured in a number of ways, but response time is often a “good enough” measure of how well a particular host performs. For the providers that you can get through Cloudways, the variance in request quality is extremely interesting.

The chart below shows the 99th percentile response time performance for each provider. It means that 99% of all requests during the load test finished at or below this time.

99th percentile performance

You can see that lower cost providers once again out-perform the higher cost AWS and GCP in pretty significant ways. The really interesting result here is that Vultr responded to 99% of requests in 240ms or less! Way to go Vultr!

Conclusions

If you aren’t sure which provider is the best value on Cloudways, from a raw cents per requests perspective you can’t go wrong with Linode, Vultr, and Digital Ocean. It’s the same story when you consider the quality (re: response time) of service: Vultr, Linode, and Digital Ocean are the clear winners here.

Full WordPress Load Test Results:

Load & performance test your own WordPress site with Kernl’s WordPress Load Testing service!

WordPress Plugin Performance Implications: Wordfence

In the world of WordPress there are a lot of different plugins you can install to extend its functionality. One of the most popular plugins is Wordfence, a security plugin developed by the fine folks over at Defiant.

Test the performance implications of your own WordPress plugins with Kernl WordPress Load Testing.

Most WordPress developers understand that the more plugins you add to your site the slower it goes, but exactly how much slower isn’t something that is often measured. More importantly, nobody has bothered to figure out the performance implications of installing many of the most popular plugins available to WordPress users today.

This all changes now with this series of blog posts exploring the performance implications of different WordPress plugins. In this series of posts we’ll test each plugin in isolation and then with caching enabled using Kernl’s WordPress load testing service. First up, is Wordfence.

Machine Setup

The test machine for these tests was a $5 Digital Ocean droplet in their SFO2 (San Francisco) data center. The machine has 1GB of RAM, 1 vCPU, and a 25GB hard disk.

The software installed on the machine is as follows:

  • Ubuntu 19.04 with all updates installed.
  • Nginx 1.16.1
  • PHP-FPM 7.3
  • MariaDB 10.3

For tests that required caching we used our favorite caching plugin, W3 Total Cache, with memcached as the data store.

The theme that was used was TwentyTwenty with no modifications and the content was an export of this blog.

All traffic was generated out of Digital Ocean’s NYC3 (New York City) data center.

Test Methodology

A series of 4 tests were run to test the performance implications of installing Wordfence. They were:

Each test was with 200 concurrent users for 1 hour.

Maximum Requests per Second

The first metric that we looked at was the maximum requests per second that the site was able to handle under each situation outlined above.

Wordfence - Maximum Requests per Second
Wordfence – Maximum Requests per Second

As you can see the difference between having Wordfence enabled and having Wordfence disabled is huge. The non-cached site with no plugins enabled handled a maximum of 26 requests per second. With Wordfence enabled it could only handle 12.

More interesting though was that with caching enabled (W3 Total Cache) the site could handle 165 requests per second, but only 67 requests per second with Wordfence enabled.

These results were so surprising that we ran the tests twice. The results were the same (within 1%-2%) each time.

First Error Occurrence

The next metric we looked at was when did the first error occur during our load tests.

Wordfence - First Error req/s
Wordfence – First Error req/s

Once again we see that adding Wordfence took a pretty serious toll on our performance. For the baseline test (no plugins, no cache) we see our first error at around 26 requests / second. In the Wordfence test with no cache we saw it at 12 requests / second.

With our caching enabled test, we never saw any errors when Wordfence wasn’t enabled. However with Wordfence enabled we saw our first error at 56 requests / second.

Average Response Time

Now that we’ve looked at server capacity metrics (max requests, first error), let’s take a look at how your end user experience changes with Wordfence enabled.

Wordfence - Avg. Response Time (milliseconds)
Wordfence – Avg. Response Time (milliseconds)

These results were fairly interesting and not at all what was expected. The average response time for the baseline test was around 4.6 seconds, while the average response with Wordfence enabled was about 2.5 seconds. So why might that be? Looking through the data it appears that the baseline test had far more successful requests and that with Wordfence enabled the requests seemed to fail faster. In short, baseline test had higher throughput but slower response time. The Wordfence test had lower throughput and faster response time.

Turning our attention to the caching scenarios, we can see that enabling Wordfence is extremely problematic at scale. With caching enabled, our baseline test had an average response time of 146ms. With caching + Wordfence that time ballooned over 10x to 1874ms.

99th Percentile Response Time

Our final metric was looking at the 99th percentile response times for our different scenarios. What does that mean? It answers the question “How long does it take for 99% of requests to finish?”. This intentionally leaves out the last 1% because those are often outliers.

Wordfence - 99th Percentile Response Time
Wordfence – 99th Percentile Response Time

As you can see above, the un-cached 99th percentile hovers right around 5s when Wordfence is enabled or disabled. Since this is the 99th percentile such a high response time isn’t too surprising in both scenarios.

The more interesting scenario is when caching is enabled. For WordPress with only a caching plugin running, the 99th percentile is 370ms. That means 99% of all requests finished in 370ms. With Wordfence also enabled (Wordfence + caching), that number jumped to 2400ms. That’s a ~7X increase.

Conclusions

From these tests we came to a few conclusions:

  1. Caching does not solve all of your performance problems. If your cache strategy relies on requests making it all the way through to WordPress, then you are very likely to still take a performance hit from other plugins. Things like Cloudflare, Varnish, Litespeed, or Nginx can help alleviate this problem.
  2. Running Wordfence is expensive from a performance standpoint. The data suggests that by simply enabling Wordfence you lose about 50% of your maximum capacity and can increase your response time between 2x-7x.
  3. Wordfence is still worth it for a lot of people. If you’ve operated a WordPress site for any length of time you know how often they get attacked. Wordfence does a great job of reducing attack surface area and making it hard for people to attack you.

Test the performance implications of your own WordPress plugins with Kernl WordPress Load Testing.

WordPress Database Performance Showdown: MySQL vs MariaDB vs Percona

WordPress requires that you use a MySQL compatible database for its database backend. It used to be that you could confidently choose MySQL and go on with life, but in 2019 the choice isn’t quite that simple. With the MySQL, MariaDB, and Percona as the attractive options, how do you know which to choose?

Choosing a database isn’t always about performance, but for the sake of this article it will be πŸ™‚

For this series of tests we tested database performance out-of-the-box with no special tweaks. It is quite possible that a professional database administrator could make each database run more performant, but most people hosting WordPress aren’t DBAs. With regards to caching, none was enabled. We wanted to test database performance, not cache performance.

MariaDB Logo

What was tested?

The WordPress host machine was a DigitalOcean CPU Optimized droplet with 16 vCPUs (dedicated hyper-threads) and 32 GB of RAM. This monster of a machine was chosen so that we could be certain that Nginx + PHP-FPM weren’t the cause of any bottlenecks. A minor tweak to the PHP-FPM config allowed for full use of all 16 vCPUs.

The database host machines were DigitalOcean CPU Optimized droplets with 4 vCPUs (dedicated hyper-threads) and 8GB of RAM. CPU Optimized droplets were chosen because we didn’t want our tests to be at the mercy of shared CPU resources.

Each deployment was in DigitalOcean’s SFO2 region with the WordPress server communicating with the database over the internal private network. The traffic producing nodes were deployed in DigitalOcean’s NYC3 data center and communicated via the public internet.

The software versions are as follows:

  • Percona Server for MySQL 8.0.16-7
  • MariaDB 10.4.8-GA
  • MySQL 8.0.18

What tests were run?

Want to test your site’s performance? Sign up for Kernl WordPress Load Testing.

For each database that was tested, we ran a load test with the following parameters:

  • 500 concurrent users
  • 2 req/s ramp up
  • 30 minute duration

The goal here wasn’t to bring the database to it’s knees but instead see how it performed under sustained heavy load, but not so heavy that it falls over.

Left: WordPress (Nginx + PHP-FPM), Right: MySQL

MariaDB WordPress Performance

During the MySQL acquisition of Oracle in 2009 there was a lot of concern amongst the core developers that Oracle would eventually close off MySQL to the world (similar to Oracle’s business model). Before that could take place, a GPL fork of MySQL was created named MariaDB.

MariaDB is open source and in active development. But how does it stand up to our WordPress load tests? Let’s find out.

First we’ll take a look at the requests and failures per second.

~379 req/s with periodic errors

As you can see from the results above the performance scales up very well over time eventually peaking at ~379 req/s. We see periodic database-related errors (the vCPUs on the database were almost completely saturated) but nothing too crazy. Next, let’s see how the response times look.

Median response time of ~190ms.

The median response time of WordPress when backed by MariaDB under heavy load stays remarkably consistent. You can see that the average is slowly creeping up by the end of the test, but nothing that would be noticeable by customers yet.

99% of request finish in 500ms

The response time distribution is a little more interesting than the median response time. 90% of all requests finish in under 300ms and 99% of all requests finish in less than 500ms. Overall, the performance of MariaDB out of the box with no configuration is quite good.

Percona WordPress Performance

Another open-source fork of MySQL, Percona was started in 2006 and has been steadily delivering value-added features and enterprise support on top of MySQL for more than 12 years. With all that accumulated experience, how does Percona measure up?

~320 requests/s, ~3 errors/s

Overall the Percona WordPress performance was pretty good. Not quite a good as MariaDB but nothing to scoff at. The one interesting thing was that the error rate was consistent across most of the test once the request volume passed ~200 requests/s. Lets see if the response time graph adds to the story.

~450ms Median Response Time

The data here is actually pretty interesting. The response time for Percona was comparable to MariaDB right up until the time it started getting consistent errors. After that, it more than doubled and stayed that way for the rest of the test.

99% of requests finished in ~550ms

The response time distribution tells similar story to the response time chart. The difference between the 50th percentile and the 99th percentile shows that performance was very consistent across the entire test, it just wasn’t as good as MariaDB.

MySQL WordPress Performance

Last but not least (well….) in our test is MySQL. It’s been around since 1994 and is now owned by Oracle. In it’s latest releases there has been lot of great features such as window functions and more JSON features to compete with PostgreSQL. So let’s see how it did.

295 requests/s, 3-4 errors/s

MySQL had a similar trajectory to Percona: It did pretty good across the board with a small but consistent series of errors after it started to go past 200 req/s. Also note that it never achieved higher than 300 req/s, which both MariaDB and Percona did.

500ms Median Response Time

The shape of the MySQL response time graph looks nearly identical to Percona, just about 50ms slower once the vCPUs started to get saturated. The detailed look in the response time distribution tells a similar story.

99% of requests finished in 875ms

The response time distribution is where you can see Percona and MySQL diverge a bit. At the 99th percentile MySQL is returning at 875ms, while Percona was more like 550ms. In general, the distribution matches what one would expect given how the response time graph looks

Conclusions

The out of the box numbers for MariaDB make it look like the clear winner here. And compared to both Percona and MySQL it is in the raw performance department. This isn’t to say that you couldn’t tune Percona or MySQL to out-perform MariaDB, but only that you get more performance with zero configuration changes.

Req / sError / sMed. Response Time99% Dist.
MariaDB379~1190ms500ms
Percona3203450ms550ms
MySQL2953500ms875ms

Load Testing Vultr’s New High Frequency Servers with WordPress

Run your own WordPress Load Tests with Kernl!

Back in June Vultr announced the general availability of their new “High Frequency” servers. Reading through the announcement I was intrigued by their claim of using “3+GHZ processors and blazing fast NVMe storage!” and immediately wondered what WordPress performance would look like versus their regular Cloud Compute offering.

What was tested?

To get a better idea of the performance characteristics of the Vultr High Frequency Compute servers, I ran several types of load tests with all Vultr servers sitting in their Silicon Valley data center:

  • 200 concurrent users from New York City (Digital Ocean NYC3)
  • 2000 concurrent users from New York City (Digital Ocean NYC3) & London (Digital Ocean LON1)

And I tested the following scenarios:

All servers with the exception of the “performance tuned” ones were built using the pre-built WordPress image that Vultr offers with no caching or performance tuning done. The “performance tuned” servers were created using Nginx, PHP-FPM, MariaDB, Memcached, and W3 Total Cache.

Results

Starting with the Vultr High Frequency boxes, lets break down performance.

$6 32GB NVMe

The $6 Vultr High Frequency(HF) machine performed well against the roughly equivalent $5 Vultr Cloud Compute(CC) instance. If we take a look at the requests/failures per second charts you can see that the HF machine out-performed the CC instance by quite a lot.

$6 32GB NVMe Vultr High Frequency Requests/Failures
$5 25GB SSD Vultr Cloud Compute Requests/Failures

As you can see the HF server was able to handle roughly twice as many requests per second as the CC server without any errors. Let’s check out the average and median response times as well as the response time distribution.

$62 32GB NVMe Vultr High Frequency Response Time
$6 32GB NVMe Vultr High Frequency Response Time
$62 32GB NVMe Vultr High Frequency Response Time Distribution
$6 32GB NVMe Vultr High Frequency Response Time Distribution

You can see that as the test progressed response times got steadily worse until leveling out at around 2.5s. The response time distribution shows that 99% of requests finished in 3s or under, with the 100th percentile outlier coming in at just under 6s. This is honestly pretty good performance for no tuning at all.

$5 25GB SSD Vultr Cloud Compute Response Time
$5 25GB SSD Vultr Cloud Compute Response Time
$5 25GB SSD Vultr Cloud Compute Response Time Distribution
$5 25GB SSD Vultr Cloud Compute Response Time Distribution

The response times for the $5 cloud compute instance were about 2 seconds worse than the high frequency instance, with the average coming in at around 4.5. The response time distribution was worse from a performance perspective, but better from a consistency perspective with the spread between the 50th percentile and the 100th percentile only being ~1.3s.

At a glance, it looks like the Vultr High Frequency server out-performs the cloud compute in a significant way for only $1 extra. However, our next tests show that it might not be so simple.

$24 128GB NVMe (Performance Tuned)

The next set of tests that were performed were 2000 concurrent users on a performance tuned setup. The traffic originated from a cluster of servers in both New York and London, with the host server being in Vultr’s Silicon Valley data center.

First, lets take a look at the requests per second handled by the $24 High Frequency server and the $20 Cloud Compute server.

$24 128GB NVMe Vultr High Frequency Requests

Now, 2000 concurrent users is a lot. I think that the HF machine did quite well overall, but I confess that I did expect a bit more out of it. It topped out at around 1250 requests per second, with errors hovering in the ~175 requests per second range. If left to run for another 30 minutes I think it would have reached closer to 1350 requests per second.

$20 80GB SSD Vultr Cloud Compute Requests

This is where things start to get interesting. The Vultr marketing team bills the high frequency machines as heads and shoulders above the regular cloud compute machines. Maybe it is for some workloads. But for this test it doesn’t seem much better at all. Looking at both graphs you can see that the HF machine topped out a bit higher, and had fewer errors, but not enough to warrant the extra cash (in my opinion). Lets see what sort of story the response times tell.

$24 128GB NVMe Vultr High Frequency Response Time
$24 128GB NVMe Vultr High Frequency Response Time Distribution

On the performance tuned box you can see that the requests that did complete successfully were all quite fast. The average response time was between 100ms and 200ms throughout the entire test. The response time distribution was solid as well, with 99% of requests finishing in under 500ms. Not bad for 2000 concurrent users.

Now let’s take a look at how the comparable cloud compute server did.

$20 80GB SSD Vultr Cloud Compute Response Time
$20 80GB SSD Vultr Cloud Compute Response Time Distribution

For the Vultr Cloud Compute instance response times also hovered between 100ms-200ms, although just a hair higher than the high frequency servers. The response time distribution was excellent as well, with 99% of requests coming in under 500ms. The main difference here is that there was a 100th percentile outlier here at the high frequency server didn’t have.

Thoughts & Conclusions

From what I can tell, the Vultr High Frequency servers are better than the Cloud Compute servers, but maybe only for certain types of workloads. You can see in the $5/$6 test that they easily out-performed the cloud compute instances, but in the more expensive $20/$24 test the results weren’t so cut and dry.

If we look at it from a requests per dollar standpoint, the cloud compute instance is actually a better value for this type of workload.

Requests per Dollar

I suggest running extensive load tests against the Vultr High Frequency machines before making any effort to switch over. They might be better for you. They might perform the same for more money.

In conclusion: Β―\_(ツ)_/Β―

Run your own WordPress Load Tests with Kernl!

What’s New With Kernl – June 2019

Hello everyone! Kernl got some pretty interesting updates this month, so let’s dive in!

Features

Average & Median Response Times for Load Tests – Kernl’s WordPress load testing services has always had this data available, we just never surfaced it in a way that was easy to consume. You’ll now see a new tab when you run a load test with information about average and median response times during the course of your test.

Progress Bar for Load Test Initialization – When you create a load test you will now see a progress bar that indicates how far in the process of instantiating your infrastructure we are. Prior to this update it was easy to think that the process had stalled out or broke.

Bug Fixes & Other

  • Load test site ownership verification would fail to certain types of HTML minification. This has been resolve.
  • Our internal analytics services has been refactors to be a singleton. This reduced each app server’s memory footprint by 5MB.
  • The public license validate endpoint now has a 10 second cache on it. This helps us deal with any sort of burst traffic from license validations.
  • Home page Javascript has been minified and concatenated into a single script to decrease load time.
  • All packages have been update on our servers.

Thats it for June! If you have any questions reach out to jack@kernl.us.

Digital Ocean 30 Hour WordPress Load Test for Reliability and Consistency

Perform your own 30 hour load tests with Kernl!

Over the past 5 months I’ve been writing a lot of different articles testing WordPress performance when under heavy load. One of the comments that I often receive is “Yes, but how reliable is the host over time?”. To determine that answer I made some changes to Kernl that would allow customers to do long duration tests against their providers with a steady load. Given my affinity for Digital Ocean, I figured that would be a great first host to test.

What Was Tested & How I Tested It

Digital Ocean has several data centers across the globe and I figured that I should test each of these data centers to see how reliable they were. For this test I ran a single load test against the following data centers for 30 hours with a 25 concurrent users:

  • New York City (NYC3)
  • Toronto (TOR1)
  • Bangalore (BLR1)
  • Frankfurt (FRA1)
  • London (LON1)
  • Singapore (SGP1)
  • Amsterdam (AMS3)

All requests were made from the Digital Ocean data center in San Francisco (SFO2). The target of each load test was a simple $5 / month droplet with the WordPress image from the Digital Ocean marketplace installed on it.

Results

The table below summarizes the results for all of the long duration load tests. Click the region to see more details of the load test.

Example Load Test Results Page
RegionRequestsFailuresFailure %Req/s Avg
NYC32.49M1390.005%23
TOR12.46M1260.005%22/23
BLR12.17M1150.005%20
FRA12.28M14050.06%21
LON12.33M13350.05%21
SGP12.29M980.004%21
AMS32.30M2280.009%21

As you can see from the results above Digital Ocean’s reliability is excellent across the entire testing period. Even the data centers with the highest error rate (Frankfurt, London) had an incredibly small error rate. I’m not going to add the response time distribution results here because they were uniformly excellent.

Anomalies

  • The Bangalore (BLR1) test averaged only 20 req / s. Even though the geographic distance is far, I expected the response times to go up but to have a similar throughput.
  • The Toronto (TOR1) load test averaged 22 req / s for 28 hours, then jumped up to 23 req / s for that last two hours of the test. Maybe a noisy neighbor went quiet?
  • Digital Ocean’s Frankfurt (FRA1) and London (LON1) data centers had an order of magnitude more errors than the other data centers I tested.

Conclusions

As a whole, Digital Ocean performed very well in all of their data centers with a moderate amount of sustained traffic to a WordPress instance. In the future I would like to try running all of these tests twice with a different origin for each test run. It’s also worth noting that I haven’t done this type of test on any other platform yet. I hope that as I test more providers I’ll find out whether or not Digital Ocean performed as well as I think it did.

Perform your own 30 hour load tests with Kernl!

What’s New With Kernl – April 2019

Lots of great stuff came down the pipeline this month at Kernl, so lets dive in!

Features / Bugs / Improvements

  • Share Load Tests – You can now share your load tests publicly! Just click the “share” button that shows up after your load test is completed.
  • Analytics Widget – If you scroll down past the list of plugin/theme versions in the product detail view you’ll now see a widget with analytics data (for Kernl Analytics subscribers).
  • Load Testing Line Graph Performance – For large load testing data sets the line graph performance has been improved by disabling some of the flashier features of Chart.js
  • Analytics page now attempts to show the analytics for the first product in your product list upon entry.
  • NODE_ENV was not set to “production” on our app servers. It is now.
  • The transition to a fully managed load balancer is now complete.

Blog Posts

Thats it for this month!

Introducing Shareable WordPress Load Test Results

For quite awhile now Kernl has had the ability to throw some serious load at your WordPress site, but never a great way to share the results. Today that changes with the introduction of load test result sharing!

πŸŽ‰

Why would I share my load test results?

The main use case for sharing your load test results is showing your clients that their new WordPress site can handle the traffic load that they expect. For someone like a YouTube or Twitter celebrity this can be a very real problem. Larger organizations also would enjoy this peace of mind.

How do I get started?

Easy! Just click the share button next to the load test that you want to share. You’ll be presented with your sharing URL that can be copy and pasted into Facebook, Twitter, Slack, or Email!

Can I see some examples?

Yes!

As always, feel free to reach out to Jack if you have any questions!

Vultr Cloud Compute -vs- Dedicated -vs- Bare Metal WordPress Performance

Load test your own WordPress site with Kernl! Getting started is free!

In the world of cloud computing there are a lot of different options to choose from. Normally you only need to choose how big your instance will be (2 vCPUs or 4, 2GB RAM or 6), but some cloud compute providers are upping their game and providing an even wider array of options and instance types for you to choose from.

Vultr has 3 different types of compute instances:

  • Cloud Compute – You get your own virtual server, but it is sharing hardware resources with lots of friends. Noisy neighbors can definitely be a problem.
  • Dedicated – Dedicated servers, but virtualized. I (think) it is possible to run in to noisy neighbor problems in this situation.
  • Bare Metal – Dedicated servers and hardware. No hypervisor and no noisy neighbors taking up your resources.

In this article we’re going to see how a very basic WordPress install performs on the different types of Vultr compute instances. We’ll do so using Kernl’s WordPress Load Testing service.

The Test

As per usual with Kernl load tests I imported this blog’s content into each load testing environment. The load test skews extremely read heavy. If you have a site that is write heavy or a mix you may see different results.

Each test was performed for 1 hour with 2000 concurrent users generating load from London and New York to Vultr’s data center in New Jersey.

Configuration

For this test I used Vultr’s pre-built WordPress image with no caching. A lot of readers might say “But you can get much better performance using X or Y!”, and they would be right! But I’m not testing Apache vs Nginx performance, or W3 Total Cache vs WP Rocket, I’m testing Vultr hardware under load in a real world scenario. I simply want to know at the end of this article if Vultr Cloud Compute, Dedicated, or Bare Metal is better for WordPress hosting.

Test 1: Vultr Cloud Compute $10 / Month

The first test I performed was against the $10 per month Vultr Cloud Compute offering. As expected of a $10/month VPS performance wasn’t awesome, but it also wasn’t terrible.

All the red of red land

As you can see, lots of failed requests and only maintaining throughput of 16 req/s. Not unexpected with a single core and 1 GB of RAM. After all, I was throwing 2000 concurrent requests per second at the server. The response time distribution was similarly bad.

Bad, but could be a lot worse.

Overall, the results for the $10 VPS were as expected. This isn’t really an apples to apples comparison (we’ll get to that later), but I wanted to give you an idea of what basic VPS instance performance looks like.

Test 2: Vultr Cloud Compute $80 / Month

With this test we’re starting to get closer to the cost of bare metal and dedicated instances. This server had 6 CPUs and 16GB of RAM. Considerably more robust than the $10 server.

Lots of red, but also blue!!!

This graph tells a much different story than the previous test. Performance peaked at 169 req/s and then leveled off at 100 req/s. We still saw a lot of errors, but once again this isn’t unexpected. Honestly if you started to get this much traffic you would likely start breaking up WordPress into its components (file system, PHP + Nginx, MySQL) and start scaling horizontally.

Much Lower Response Time Distribution

The response time distribution was much better for this server as well. The upper end was just as bad as the cheaper box, but the 90% and below ranges were pretty solid for the amount of traffic that was being received.

Test 3: Vultr Bare Metal $120 / Month

The Vultr Bare Metal server was the instance I was most excited about testing. I’ve always had a soft spot for hardware and getting access to a bare metal server is pretty cool. For $120 per month (on sale, price will rise to $300/month eventually) you get 8 CPUs and 32GB of RAM. This is a pretty serious server.

Oooh, 200 req/s.

Lots of blue on this graph but also the expected amount of red. You can see that throwing 2 more non-virtual CPUs and 2X the RAM made a pretty big difference. We peaked at 200 req/s and then leveled out at 125 req/s. For reference that is 17.2 million requests per day.

πŸ™

The lower end of the response time distribution was solid, but the upper end wasn’t great at all. With all of those errors it isn’t surprising that this is the case.

Test 4: Vultr Dedicated $120 / Month

I honestly had a tough time figuring out why Vultr priced the bare metal and dedicated instances so close to each other. Dedicated is clearly inferior (far fewer CPUs and RAM) so why would anyone choose it? Anyway, let’s take a look at the graph.

πŸ’©πŸ’©πŸ’©πŸ’©πŸ’©

This test peaked at 100 req/s and then leveled off at around 70. I really would expect a lot better performance for this sort of money.

Also πŸ’©, but not as much πŸ’©.

Response time distribution was similar to the other boxes. With all the failures it tends to skew pretty hard in the wrong direction. I’m sure that there is a use case for these dedicated Vultr instances, but it definitely isn’t hosting a WordPress site.

Conclusions

With all of this data it was pretty easy to graph which of these is the best value.

Value was calculated by taking the cost per month and dividing it by the maximum number of requests. Based on the performance we saw above the Vultr Cloud Compute instances seem like your best value for WordPress hosting. For WordPress hosting it looks like Vultr Bare Metal and Dedicated instances aren’t a great choice. As mentioned above, there are likely use cases where they are a good choice though (maybe workloads that require very consistent performance).

As with all of these tests, your mileage may vary! I highly recommend that you run load tests on any new host that you use to get an idea of what sort of performance you can expect.

Load test your own WordPress site with Kernl! Getting started is free!

$5 WordPress VPS Performance Showdown

Want to load test your own WordPress site? Sign up for Kernl now!

In the world of affordable WordPress hosting there is an array of different VPS providers to choose from. With so many choices how do you know who to choose? In addition to criteria such as ease of use and support, performance is a huge concern for most people deploying WordPress. In this article we’ll take a look at the performance of several different VPS providers in the $5 tier to see how they perform under load by using Kern’s WordPress Load Testing feature.

Who are we testing?

There are a lot of VPS providers out there providing machines in the $5 / month tier, so we’ve chosen 7 of the more popular providers to test against:

  • Digital Ocean (1GB RAM, 1vCPU)
  • Linode (Nanode 1GB, 1vCPU)
  • Vultr (1vCPU, 1GB RAM)
  • AWS Lightsail (1 GB RAM, 1 vCPU)
  • Hetzner (2vCPU, 4GB RAM)
  • Google Cloud (f1-micro: 1vCPU 600MB RAM)
  • Azure ($15 A0 1vCPU, 1GB RAM)

What tests will we run?

Using Kernl’s WordPress Load Testing feature we ran 2 different tests per provider:

  • No Cache – 200 concurrent users for 30 minutes. We used this test to see raw WordPress performance with no caching enabled.
  • Cached with W3 Total Cache + Memcached – 200 concurrent users for 30 minutes. We used this test to see what a more real-world scenario looks like. In general most people use some form of caching on their site.

VPS Setup

To make sure that our test setup was consistent across all VPS providers we followed this setup guide, where we ended up with the following versions of software:

With regards to regions, for every test we kept the VPS instances on the east coast of the United States with the exception of Hetzer where we had the VPS instance in Germany.

Results (Request & Failures)

First, let’s take a look at the request / second results across the different providers.

As you can see there is a wide spread of results depending on host. Honestly this wasn’t want I expected when I started. You’ll also notice that the Azure box cost $15/month. It was the closest I could get to finding a $5/month box in their interface (which I felt like I needed another degree in Computer Science to understand!).

So let’s visualize the data with no caching enabled.

We get lots of interesting results here. If you run a site where caching is difficult to do, your $5 will go much further depending on your host. Some notes:

  • Google Cloud and Azure performed TERRIBLE. I’m not sure why. Maybe it had to do with accessing the disk so frequently to load up PHP files? (but I expect that those were cached by PHP FPM or some other underlying process).
  • If you are in Europe, Hetzner is your friend. $5/month gets you 4x the ram and 2x the vCPUs are the next closest provider.
  • If you are in the US, AWS seems to be winning in this test but not by much. It feels like you would be fine going with Digital Ocean or Vultr.
  • Before making any real decisions on a host, I’d want to run 5 or so tests across different instances to make sure that there isn’t a lot of variance in my results. Noisy neighbors can often be a problem on VPS providers.

Now let’s take a look at a more realistic scenario where you have a caching plugin installed.

You’ll notice that there aren’t any error bars on this graph. Thats because each host was able to handle the load without having any. This isn’t too surprising since most of the requests would be served right out of memory via Memcached. Some notes on cached requests:

  • Once again, Google Cloud and Azure perform the worst out of any of these hosts. Given how highly regarded they are in the hosting ecosystem outside of WordPress I expected better performance.
  • All of the other providers posted impressive numbers (170 req/s – 180 req/s). At level I would probably choose whatever provider had the best support, user interface, and reliability.
  • I suspect that most of these boxes could handle a little bit more load before going under. If I increased this test by an order of magnitude (200 users to 2000 users) I think most of the providers would tap out before Hetzner does due to how much more RAM and CPU it has.

Results (Response Time Distribution)

While requests per second and failures per second are valuable metrics, in order to get a more holistic view of raw performance for these $5 VPS instances we need to look at response time distribution.

So how should you interpret this chart? For the percentage columns, each value is in milliseconds. If you look at the 99% column, you can see that 99% of Vultr requests returned in <= 3600ms. If you look at the 80% column for Vultr, you can see that 80% of Vultr requests returned in <= 3400ms. Let’s take a look at our un-cached results.

Some notes on this graph:

  • Any service that hit the 5000ms mark killed the request. I think that they likely would have gone far beyond that.
  • Once again, for $5 Hetzner is just crushing it. You simple can’t compete with 4GB RAM and 2vCPUs. Even with no caching their 99th percentile was under 2 seconds!
  • For our non-european readers, Digital Ocean, Vultr, and AWS seemed to perform the best. AWS remained remarkably consistent across the response time distribution range. This is a good thing.
  • Google Cloud… wtf? So 99% of your requests finished in <= 5 seconds, and then 90% of requests finished in <= 300ms? Something is fishy.

And now, let’s see how things change when caching is used.

As expected, most providers do very well when caching is used. Digital Ocean, Linode, Vultr, AWS, and Hetzner are all performing in the <= 200ms range (some lower!). It’s hard to decide who is better at this level due to latency due to geographic distance. The point is that you could choose any of those hosts and be OK when using a cache plugin. Once again, I’m struggling to figure out why someone would spend their $5 on Azure or Google Cloud.

Results (Total Requests)

Our final metric we need to look at before we can pass any judgement on our VPS providers is total requests. This in particular needs to be compared with response time distribution.


This graph does a great job explaining some of the discrepancies in the response time distribution data. When caching wasn’t enabled, Google Cloud and Azure barely even show up on the graph. More thoughts:

  • I’m 90% positive that the Google Cloud and Azure instances nearly stopped processing requests at some point. They were so overwhelmed that they just fell over. The data seems to support this.
  • Google Cloud and Azure are not the best place to spend $5. Even with caching I would be scared if there was ever a cache-miss.
  • Hetzner is the clear winner in un-cached data. Once again, this makes sense how much more machine you are getting for your money.
  • On the U.S. side of the ocean, AWS seems to win here, although not by much over Digital Ocean and Vultr. Once caching is taken into consideration they all perform roughly the same (accounting for latency between data centers and load generators).

Conclusions

If performance is all you care about for your $5 (and thats a big if), then choose Hetzner if you need a VPS in Europe. If you need a VPS in the US or elsewhere, choose AWS, Digital Ocean, or Vultr. Microsoft and Google are not great for $5.

Want to load test your own WordPress site? Sign up for Kernl now!