WordPress Plugin Performance Implications: Wordfence

In the world of WordPress there are a lot of different plugins you can install to extend its functionality. One of the most popular plugins is Wordfence, a security plugin developed by the fine folks over at Defiant.

Test the performance implications of your own WordPress plugins with Kernl WordPress Load Testing.

Most WordPress developers understand that the more plugins you add to your site the slower it goes, but exactly how much slower isn’t something that is often measured. More importantly, nobody has bothered to figure out the performance implications of installing many of the most popular plugins available to WordPress users today.

This all changes now with this series of blog posts exploring the performance implications of different WordPress plugins. In this series of posts we’ll test each plugin in isolation and then with caching enabled using Kernl’s WordPress load testing service. First up, is Wordfence.

Machine Setup

The test machine for these tests was a $5 Digital Ocean droplet in their SFO2 (San Francisco) data center. The machine has 1GB of RAM, 1 vCPU, and a 25GB hard disk.

The software installed on the machine is as follows:

  • Ubuntu 19.04 with all updates installed.
  • Nginx 1.16.1
  • PHP-FPM 7.3
  • MariaDB 10.3

For tests that required caching we used our favorite caching plugin, W3 Total Cache, with memcached as the data store.

The theme that was used was TwentyTwenty with no modifications and the content was an export of this blog.

All traffic was generated out of Digital Ocean’s NYC3 (New York City) data center.

Test Methodology

A series of 4 tests were run to test the performance implications of installing Wordfence. They were:

Each test was with 200 concurrent users for 1 hour.

Maximum Requests per Second

The first metric that we looked at was the maximum requests per second that the site was able to handle under each situation outlined above.

Wordfence - Maximum Requests per Second
Wordfence – Maximum Requests per Second

As you can see the difference between having Wordfence enabled and having Wordfence disabled is huge. The non-cached site with no plugins enabled handled a maximum of 26 requests per second. With Wordfence enabled it could only handle 12.

More interesting though was that with caching enabled (W3 Total Cache) the site could handle 165 requests per second, but only 67 requests per second with Wordfence enabled.

These results were so surprising that we ran the tests twice. The results were the same (within 1%-2%) each time.

First Error Occurrence

The next metric we looked at was when did the first error occur during our load tests.

Wordfence - First Error req/s
Wordfence – First Error req/s

Once again we see that adding Wordfence took a pretty serious toll on our performance. For the baseline test (no plugins, no cache) we see our first error at around 26 requests / second. In the Wordfence test with no cache we saw it at 12 requests / second.

With our caching enabled test, we never saw any errors when Wordfence wasn’t enabled. However with Wordfence enabled we saw our first error at 56 requests / second.

Average Response Time

Now that we’ve looked at server capacity metrics (max requests, first error), let’s take a look at how your end user experience changes with Wordfence enabled.

Wordfence - Avg. Response Time (milliseconds)
Wordfence – Avg. Response Time (milliseconds)

These results were fairly interesting and not at all what was expected. The average response time for the baseline test was around 4.6 seconds, while the average response with Wordfence enabled was about 2.5 seconds. So why might that be? Looking through the data it appears that the baseline test had far more successful requests and that with Wordfence enabled the requests seemed to fail faster. In short, baseline test had higher throughput but slower response time. The Wordfence test had lower throughput and faster response time.

Turning our attention to the caching scenarios, we can see that enabling Wordfence is extremely problematic at scale. With caching enabled, our baseline test had an average response time of 146ms. With caching + Wordfence that time ballooned over 10x to 1874ms.

99th Percentile Response Time

Our final metric was looking at the 99th percentile response times for our different scenarios. What does that mean? It answers the question “How long does it take for 99% of requests to finish?”. This intentionally leaves out the last 1% because those are often outliers.

Wordfence - 99th Percentile Response Time
Wordfence – 99th Percentile Response Time

As you can see above, the un-cached 99th percentile hovers right around 5s when Wordfence is enabled or disabled. Since this is the 99th percentile such a high response time isn’t too surprising in both scenarios.

The more interesting scenario is when caching is enabled. For WordPress with only a caching plugin running, the 99th percentile is 370ms. That means 99% of all requests finished in 370ms. With Wordfence also enabled (Wordfence + caching), that number jumped to 2400ms. That’s a ~7X increase.

Conclusions

From these tests we came to a few conclusions:

  1. Caching does not solve all of your performance problems. If your cache strategy relies on requests making it all the way through to WordPress, then you are very likely to still take a performance hit from other plugins. Things like Cloudflare, Varnish, Litespeed, or Nginx can help alleviate this problem.
  2. Running Wordfence is expensive from a performance standpoint. The data suggests that by simply enabling Wordfence you lose about 50% of your maximum capacity and can increase your response time between 2x-7x.
  3. Wordfence is still worth it for a lot of people. If you’ve operated a WordPress site for any length of time you know how often they get attacked. Wordfence does a great job of reducing attack surface area and making it hard for people to attack you.

Test the performance implications of your own WordPress plugins with Kernl WordPress Load Testing.

Introducing Team Management and the Kernl Unlimited Plan

One of the most requested features that Kernl gets is Team Management. Team management can take on a lot of different forms, but for Kernl’s purposes it is the ability to share access via the Kernl interface with multiple people with different sets of credentials.

Team Management

Starting today you can sign up for the Kernl Unlimited plan (see below) and start managing your team. Team management allows you to share your Kernl account with authorized users. When a user is part of your team, they have a slightly restricted view of Kernl but otherwise see the same things you do. What’s restricted?

  • Continuous Deployment – Adding connected Git repositories is the responsibility of the admin. Non-admin users don’t need to see this information.
  • Billing – Once again, this is an admin responsibility. The admin is able to change plans, add credit cards, etc.
  • Team Management – Only the admin can add and remove users from their account.

Aside from that everything is the same for admin and non-admin users.

Kernl Team Management

The Unlimited Plan & Pricing Changes

In addition to team management we are also reducing the number of plans that Kernl has. Prior to this change we had 5 different plans (solo, agency, enterprise, huge, massive) which seemed like a bit much. To simplify Kernl’s pricing structure and give our customers better options we went down to 3 plans:

  • Solo – This $9 per month plan is our base plan. Most customers start here and then grow into larger plans as their needs change.
  • Agency – The $39 agency plan is essentially our former “Enterprise” plan but with increased limits for plugins, themes, licenses, load testing, and feature flags.
  • Unlimited – This new $79 unlimited plan allows you to have unlimited plugins, themes, licenses, feature flags and very high limited on load testing (50K concurrent users). This plan also includes team management so you can more easily control access to your Kernl account.

As always, current Kernl customers are grandfathered into the plan that they are currently on.

Kernl Pricing

What’s Next?

Now that Kernl has a more robust upper-level offering we hope to start rolling more features into it. For example:

  • In the near future we’ll include Kernl Analytics with Unlimited plan.
  • We’re also planning on rolling out limited team management into the Agency plan.

What’s New With Kernl – October 2019

October held a lot of great work for Kernl! Lots of refactoring, test coverage increases, and a new UI for managing Git deployments. Let’s dive in!

Features

  • Git Continuous Deployment UI Changes – Kernl now has a new and improved interface for adding continuous deployment to your projects.
  • Installed Plugin List Filter – You can now easily filter the installed plugin data in Kernl Analytics. Prior to this change the best you could do is column sort.

Infrastructure, Bugs, & Other

  • Fetching BitBucket repos with more that 20 branches failed. This has been resolved.
  • The plugin/theme webhook build log didn’t handle the case where a user had just signed up and didn’t have any build requests yet.
  • Easy Digital Downloads license validation was failing on newer versions of EDD. Kernl has been updated to support the newer version API as well as the older API.
  • The Git integration has been refactored and had additional test coverage added.
  • Kernl was upgraded to Node 12.13.x. With this upgrade we get async stack traces and a non-trivial decrease in response time (~50ms).

Blog Posts

WordPress Database Performance Showdown: MySQL vs MariaDB vs Percona

WordPress requires that you use a MySQL compatible database for its database backend. It used to be that you could confidently choose MySQL and go on with life, but in 2019 the choice isn’t quite that simple. With the MySQL, MariaDB, and Percona as the attractive options, how do you know which to choose?

Choosing a database isn’t always about performance, but for the sake of this article it will be πŸ™‚

For this series of tests we tested database performance out-of-the-box with no special tweaks. It is quite possible that a professional database administrator could make each database run more performant, but most people hosting WordPress aren’t DBAs. With regards to caching, none was enabled. We wanted to test database performance, not cache performance.

MariaDB Logo

What was tested?

The WordPress host machine was a DigitalOcean CPU Optimized droplet with 16 vCPUs (dedicated hyper-threads) and 32 GB of RAM. This monster of a machine was chosen so that we could be certain that Nginx + PHP-FPM weren’t the cause of any bottlenecks. A minor tweak to the PHP-FPM config allowed for full use of all 16 vCPUs.

The database host machines were DigitalOcean CPU Optimized droplets with 4 vCPUs (dedicated hyper-threads) and 8GB of RAM. CPU Optimized droplets were chosen because we didn’t want our tests to be at the mercy of shared CPU resources.

Each deployment was in DigitalOcean’s SFO2 region with the WordPress server communicating with the database over the internal private network. The traffic producing nodes were deployed in DigitalOcean’s NYC3 data center and communicated via the public internet.

The software versions are as follows:

  • Percona Server for MySQL 8.0.16-7
  • MariaDB 10.4.8-GA
  • MySQL 8.0.18

What tests were run?

Want to test your site’s performance? Sign up for Kernl WordPress Load Testing.

For each database that was tested, we ran a load test with the following parameters:

  • 500 concurrent users
  • 2 req/s ramp up
  • 30 minute duration

The goal here wasn’t to bring the database to it’s knees but instead see how it performed under sustained heavy load, but not so heavy that it falls over.

Left: WordPress (Nginx + PHP-FPM), Right: MySQL

MariaDB WordPress Performance

During the MySQL acquisition of Oracle in 2009 there was a lot of concern amongst the core developers that Oracle would eventually close off MySQL to the world (similar to Oracle’s business model). Before that could take place, a GPL fork of MySQL was created named MariaDB.

MariaDB is open source and in active development. But how does it stand up to our WordPress load tests? Let’s find out.

First we’ll take a look at the requests and failures per second.

~379 req/s with periodic errors

As you can see from the results above the performance scales up very well over time eventually peaking at ~379 req/s. We see periodic database-related errors (the vCPUs on the database were almost completely saturated) but nothing too crazy. Next, let’s see how the response times look.

Median response time of ~190ms.

The median response time of WordPress when backed by MariaDB under heavy load stays remarkably consistent. You can see that the average is slowly creeping up by the end of the test, but nothing that would be noticeable by customers yet.

99% of request finish in 500ms

The response time distribution is a little more interesting than the median response time. 90% of all requests finish in under 300ms and 99% of all requests finish in less than 500ms. Overall, the performance of MariaDB out of the box with no configuration is quite good.

Percona WordPress Performance

Another open-source fork of MySQL, Percona was started in 2006 and has been steadily delivering value-added features and enterprise support on top of MySQL for more than 12 years. With all that accumulated experience, how does Percona measure up?

~320 requests/s, ~3 errors/s

Overall the Percona WordPress performance was pretty good. Not quite a good as MariaDB but nothing to scoff at. The one interesting thing was that the error rate was consistent across most of the test once the request volume passed ~200 requests/s. Lets see if the response time graph adds to the story.

~450ms Median Response Time

The data here is actually pretty interesting. The response time for Percona was comparable to MariaDB right up until the time it started getting consistent errors. After that, it more than doubled and stayed that way for the rest of the test.

99% of requests finished in ~550ms

The response time distribution tells similar story to the response time chart. The difference between the 50th percentile and the 99th percentile shows that performance was very consistent across the entire test, it just wasn’t as good as MariaDB.

MySQL WordPress Performance

Last but not least (well….) in our test is MySQL. It’s been around since 1994 and is now owned by Oracle. In it’s latest releases there has been lot of great features such as window functions and more JSON features to compete with PostgreSQL. So let’s see how it did.

295 requests/s, 3-4 errors/s

MySQL had a similar trajectory to Percona: It did pretty good across the board with a small but consistent series of errors after it started to go past 200 req/s. Also note that it never achieved higher than 300 req/s, which both MariaDB and Percona did.

500ms Median Response Time

The shape of the MySQL response time graph looks nearly identical to Percona, just about 50ms slower once the vCPUs started to get saturated. The detailed look in the response time distribution tells a similar story.

99% of requests finished in 875ms

The response time distribution is where you can see Percona and MySQL diverge a bit. At the 99th percentile MySQL is returning at 875ms, while Percona was more like 550ms. In general, the distribution matches what one would expect given how the response time graph looks

Conclusions

The out of the box numbers for MariaDB make it look like the clear winner here. And compared to both Percona and MySQL it is in the raw performance department. This isn’t to say that you couldn’t tune Percona or MySQL to out-perform MariaDB, but only that you get more performance with zero configuration changes.

Req / sError / sMed. Response Time99% Dist.
MariaDB379~1190ms500ms
Percona3203450ms550ms
MySQL2953500ms875ms

WordPress Performance on DigitalOcean Managed MySQL

Want to performance test your own WordPress installation? Try Kernl’s WordPress Load Testing service.

The database is the most important part of any WordPress site. If you aren’t adept at managing MySQL, it can be a serious risk for you and your clients. One way to de-risk this portion of your WordPress site is by going with a managed MySQL offering. There are ton of different options out there, but for this post we’re going to look at Digital Ocean’s Managed MySQL.

DigitalOcean DB SaaS background.

Why Use DigitalOcean Managed MySQL?

There are a lot of reasons to use Digital Ocean’s Managed MySQL (or some other offering) including:

  • Simple Setup – With a few simple clicks you can have a high quality MySQL cluster set up.
  • Horizontal Scalability – Site growing fast? You can spin up read-only nodes to help scale out read operations.
  • Automated Daily Backups – No need to set up your own backups. DigitalOcean takes care of it for you.
  • Automated Failover – If for some reason your primary database node fails, you will automatically fail over to your warm spare.
  • Security – MySQL best practices for security are automatically followed by DigitalOcean. In addition to that your database is isolated to your private network so that outside requests can’t access it by default.

Baseline Performance Test

To get things started let’s do a baseline performance test where the MySQL database is on the same box as the Nginx server. The server configuration is as follows:

  • 1GB RAM
  • 3 vCPU
  • Nginx
  • PHP-FPM 7.2
  • MariaDB 10.1
  • No caching

A test of 400 concurrent users was run using Kernl’s WordPress Load Testing service. Without caching the results are predictably bad, but that is to be expected.

Graph of requests per second for self-hosted MySQL with WordPress
Things fall apart at around ~70 req/sec

For completeness, let’s also take a look at the response time distribution.

Response time distribution for self hosted MySQL with WordPress
99% of requests finish in ~2 seconds

Given how many failing requests we had and zero caching, the ones that did manage to get through didn’t perform too poorly. 99% in 2 seconds or less.

DigitalOcean Managed MySQL

When setting up a managed MySQL database on DigitalOcean you get the option to select the underlying hardware that powers it. For this blog post we went with the minimum possible configuration (1GB RAM, 1vCPU).

Knowing that fewer resources were allocated, it was expected that performance would actually be a bit worse on the dedicated MySQL instance. These expectations were confirmed with the load test.

Graph of requests per second for DigitalOcean Managed MySQL with WordPress
Things go sideways at ~50 req/s.

All things considered, 50 req/s against a WordPress site with no caching isn’t all that bad. Once we hit that level, the database charts were showing 100% CPU saturation and thats when we started to see the error rate increase. Now let’s look at the response time distribution.

Response time distribution for managed MySQL with WordPress
99% of requests finish in less than 2.6 seconds

As expected performance is slightly worse here due to fewer resources on the dedicated MySQL machine, but not in a huge way.

Cost & Why Managed MySQL is Important

For our test we used the most basic configuration that DigitalOcean offers:

  • 1 GB RAM
  • 1 vCPU
  • 10 GB Hard Disk
  • No failover
  • Automatic backups

All of this and not having to manage MySQL for $15/month. Not too bad, but to really be getting your money’s worth (automatic failover) you need to spend more money.

  • 2 GB RAM
  • 1 vCPU
  • 25 GB Hard Disk
  • 1 Standby node
  • Automatic failover
  • Automatic backups

With automatic failover you start to reduce the risk to yourself and your customers a lot. However, this level of availability isn’t free πŸ™‚ DigitalOcean doesn’t support failover on their cheapest node, so you have to upgrade to the next level ($30/month). After that, the failover node costs you an additional $20/month. Now we’re looking at $50/month for a managed MySQL cluster with automated failover.

If you run a WordPress based business on DigitalOcean, $50/month buys you a lot of peace of mind. It’s also easy to scale up as traffic increases and the cost is competitive in the landscape of managed MySQL. If you happen to be an excellent DBA though, you can definitely manage your own cluster at a reduced monetary cost, but increased time cost.

Conclusions

Managing your own database cluster is probably fine if cost is a problem and you are good at it. In general though, if you can afford it we recommend using a managed MySQL host so you can push the complexity of operating a highly available MySQL cluster on to someone else.

Want to performance test your own WordPress installation? Try Kernl’s WordPress Load Testing service.

What’s New With Kernl – July & August 2019

I hope everyone has had a great summer (or winter if you are in the southern hemisphere)! Over the past 2 months we’ve gotten a lot great stuff done, so let’s dive in.

Features & Infrastructure

  • Kernl has upgraded from Mongo 3.x to Mongo 4.2 with WiredTiger. We get improved performance and the latest features with this change.
  • Our Redis instance has moved to DigitalOcean along with the rest of our infrastructure. Prior to this we were using managed host that lived outside the NYC3 data center. Response times decreased ~50ms or so with this change.
  • The high traffic plugin and theme update check endpoints had a round of performance tuning done. Resource consumption was lowered in meaningful way.
  • πŸ”₯πŸ”₯πŸ”₯Kernl Analytics Active PluginsπŸ”₯πŸ”₯πŸ”₯- Kernl Analytics will now track what plugins are most active across your install-base. You only need to be signed up for Kernl Analytics and use the latest plugin_update_check.php file to get this new feature.
  • Our MongoDB database has been moved to DigitalOcean NYC3. Prior to this we were hosting on Compose.io outside of the datacenter. Originally this decision was made because managing databases is tough, but the quality of hosting at Compose has gone done significantly in the past year. With this change we shaved ~150ms off of response times.
  • Some tweaks were made to our network firewalls to make them easier to manage. Thanks DigitalOcean!
πŸ”₯πŸ”₯πŸ”₯Kernl Analytics Active Plugins πŸ”₯πŸ”₯πŸ”₯

Bug Fixes

  • Load testing machines would fail to provision if the API call to DigitalOcean failed. This has been resolved.
  • The load testing master node would fail to start sometimes if secondary nodes failed to connect. The threshold for starting tests has been lowered so that this won’t happen anymore.
  • If a credit card expires and the invoice payment fails, the account isn’t marked as paid when a new card is added and a successful payment happens.
  • When switching between themes/plugins in Kernl Analytics the domain data wasn’t reloading with the new plugin/theme.
  • Thanks to a customer bug report and code snippet, the plugin_update_check.php no longer sends headers before the license check fails.

Blog Posts

That’s it for July and August!

Vultr Cloud Compute -vs- Dedicated -vs- Bare Metal WordPress Performance

Load test your own WordPress site with Kernl! Getting started is free!

In the world of cloud computing there are a lot of different options to choose from. Normally you only need to choose how big your instance will be (2 vCPUs or 4, 2GB RAM or 6), but some cloud compute providers are upping their game and providing an even wider array of options and instance types for you to choose from.

Vultr has 3 different types of compute instances:

  • Cloud Compute – You get your own virtual server, but it is sharing hardware resources with lots of friends. Noisy neighbors can definitely be a problem.
  • Dedicated – Dedicated servers, but virtualized. I (think) it is possible to run in to noisy neighbor problems in this situation.
  • Bare Metal – Dedicated servers and hardware. No hypervisor and no noisy neighbors taking up your resources.

In this article we’re going to see how a very basic WordPress install performs on the different types of Vultr compute instances. We’ll do so using Kernl’s WordPress Load Testing service.

The Test

As per usual with Kernl load tests I imported this blog’s content into each load testing environment. The load test skews extremely read heavy. If you have a site that is write heavy or a mix you may see different results.

Each test was performed for 1 hour with 2000 concurrent users generating load from London and New York to Vultr’s data center in New Jersey.

Configuration

For this test I used Vultr’s pre-built WordPress image with no caching. A lot of readers might say “But you can get much better performance using X or Y!”, and they would be right! But I’m not testing Apache vs Nginx performance, or W3 Total Cache vs WP Rocket, I’m testing Vultr hardware under load in a real world scenario. I simply want to know at the end of this article if Vultr Cloud Compute, Dedicated, or Bare Metal is better for WordPress hosting.

Test 1: Vultr Cloud Compute $10 / Month

The first test I performed was against the $10 per month Vultr Cloud Compute offering. As expected of a $10/month VPS performance wasn’t awesome, but it also wasn’t terrible.

All the red of red land

As you can see, lots of failed requests and only maintaining throughput of 16 req/s. Not unexpected with a single core and 1 GB of RAM. After all, I was throwing 2000 concurrent requests per second at the server. The response time distribution was similarly bad.

Bad, but could be a lot worse.

Overall, the results for the $10 VPS were as expected. This isn’t really an apples to apples comparison (we’ll get to that later), but I wanted to give you an idea of what basic VPS instance performance looks like.

Test 2: Vultr Cloud Compute $80 / Month

With this test we’re starting to get closer to the cost of bare metal and dedicated instances. This server had 6 CPUs and 16GB of RAM. Considerably more robust than the $10 server.

Lots of red, but also blue!!!

This graph tells a much different story than the previous test. Performance peaked at 169 req/s and then leveled off at 100 req/s. We still saw a lot of errors, but once again this isn’t unexpected. Honestly if you started to get this much traffic you would likely start breaking up WordPress into its components (file system, PHP + Nginx, MySQL) and start scaling horizontally.

Much Lower Response Time Distribution

The response time distribution was much better for this server as well. The upper end was just as bad as the cheaper box, but the 90% and below ranges were pretty solid for the amount of traffic that was being received.

Test 3: Vultr Bare Metal $120 / Month

The Vultr Bare Metal server was the instance I was most excited about testing. I’ve always had a soft spot for hardware and getting access to a bare metal server is pretty cool. For $120 per month (on sale, price will rise to $300/month eventually) you get 8 CPUs and 32GB of RAM. This is a pretty serious server.

Oooh, 200 req/s.

Lots of blue on this graph but also the expected amount of red. You can see that throwing 2 more non-virtual CPUs and 2X the RAM made a pretty big difference. We peaked at 200 req/s and then leveled out at 125 req/s. For reference that is 17.2 million requests per day.

πŸ™

The lower end of the response time distribution was solid, but the upper end wasn’t great at all. With all of those errors it isn’t surprising that this is the case.

Test 4: Vultr Dedicated $120 / Month

I honestly had a tough time figuring out why Vultr priced the bare metal and dedicated instances so close to each other. Dedicated is clearly inferior (far fewer CPUs and RAM) so why would anyone choose it? Anyway, let’s take a look at the graph.

πŸ’©πŸ’©πŸ’©πŸ’©πŸ’©

This test peaked at 100 req/s and then leveled off at around 70. I really would expect a lot better performance for this sort of money.

Also πŸ’©, but not as much πŸ’©.

Response time distribution was similar to the other boxes. With all the failures it tends to skew pretty hard in the wrong direction. I’m sure that there is a use case for these dedicated Vultr instances, but it definitely isn’t hosting a WordPress site.

Conclusions

With all of this data it was pretty easy to graph which of these is the best value.

Value was calculated by taking the cost per month and dividing it by the maximum number of requests. Based on the performance we saw above the Vultr Cloud Compute instances seem like your best value for WordPress hosting. For WordPress hosting it looks like Vultr Bare Metal and Dedicated instances aren’t a great choice. As mentioned above, there are likely use cases where they are a good choice though (maybe workloads that require very consistent performance).

As with all of these tests, your mileage may vary! I highly recommend that you run load tests on any new host that you use to get an idea of what sort of performance you can expect.

Load test your own WordPress site with Kernl! Getting started is free!

Measuring the Performance of WordPress on DigitalOcean Droplets

Earlier this year Kernl launched our WordPress Load Testing offering. Prior to the launch we had been doing a series of blog posts testing the performance of “managed” WordPress providers. In this blog post I’ll test the performance of Digital Ocean’s VPS solutions with a standard LEMP (Linux Nginx MySQL PHP-FPM) installation by scaling it vertically.

Want to measure your own WordPress performance under heavy load? Sign up for Kernl.

Server & WordPress Configuration

The Digital Ocean 1-click LEMP install is already configured to take full advantage of as many cores and as much RAM as is available. During the course of these tests I never needed to tweak any server settings. That doesn’t mean that more performance couldn’t have been extracted through configuration tweaks, but even with these settings we were able to gain useful information about WordPress performance on the different Digital Ocean droplet levels.

With regards to WordPress configuration… there isn’t any. No caching, CDNs, or compression. Just raw WordPress. Its easy to get high performance out of WordPress if you cache everything, but seeing what performance looks like in the worst-case scenario is far more interesting.

The Test

For this series of tests I imported the contents of my personal blog (Re-cycledair) to the target and then used Kernl to run the load tests. Due to the nature of my blog this test is very read heavy. While this isn’t realistic for everyone, for my personal blog the test is representative of the actual traffic that it receives.

The test itself looked like:

  • Concurrent users – 500
  • Ramp up – 2 users per second
  • Duration – 30 minutes
  • Request Rate – Each user makes 1 request every second
  • Droplet location – New York City
  • Load test generator location – Amsterdam

Results

As expected the amount of traffic that we could handle scaled linearly with the amount of hardware we were using (and cost).

Cost -vs- Performance

One interesting thing you’ll see in the graph is that in two places the requests per second actually trend down.

Data

Looking at the data above you can see that I tried Digital Ocean’s standard droplets as well as their CPU optimized droplets. The difference between them is cost and dedicated hyper-threads. From a cost perspective, you’re better off going with the standard droplet in the same price category. Personally I expected the CPU optimized droplet to perform better, but this might not be the best type of workload for it.

What About Caching?

Just for fun I tried out the one-click install of WordPress with LiteSpeed configured on a $5 / month droplet.

Things went really really really well.

πŸ”₯πŸ”₯πŸ”₯πŸ”₯πŸ”₯

Thats right. By the time the test was completed my little $5 / month droplet was receiving 1800 requests / second WITH NO ERRORS. For perspective, thats 4.6 billion requests per month.

Conclusions

The data does a good job speaking for itself here, but in general you should definitely stick to Digital Ocean’s standard droplets when running WordPress. Even without caching you can get really good performance out of the $40/month droplet.

Want to measure your own WordPress performance under heavy load? Sign up for Kernl.

Load Testing the ChemiCloud Managed WordPress Hosting Service

At the beginning of December Kernl launched a closed beta for our WordPress Load TestingΒ service. As part of the bug shakedown we’ve been spending some time load testing different managed WordPress hosting services. Some of previous tests include WordPress.com, CloudWays, and GoDaddy. For this test, we turned our sights on ChemiCloud.

How do we judge the platform?

Using Kernl’s load testing feature we run 3 different load tests against the target system.

  • The Baseline – This is a simple baseline load test that we use to verify that our configuration is correct and that the target can handle even minor traffic. It consists of 200 concurrent users, for 10 minutes, ramping up at 2 users / second, with traffic originating in San Francisco.
  • Sustained Traffic – The sustained traffic test mimics what traffic might look like for a read-heavy website with a lot of visitors. This load test consists of 2000 concurrent users, for 2 hours, ramping up at 2 users / second, with traffic originating from San Francisco.
  • Traffic Spike – This test is brutal. We use it mimic the sort of traffic that your WordPress site might experience if a link to it were shared by a Twitter or Instagram celebrity. The load test consists of 10,000 concurrent users, for 1 hour, ramping up at 10 users / second, with traffic originating from San Francisco.

All traffic for this test is generated out of Digital Ocean’s SFO2 data center.

What ChemiCloud plan was used?

ChemiCloud has several different tiers for managed WordPress hosting. We decided on the “Oxygen” plan. At a high level this seemed to align well with the hosting that we tested thus far.

ChemiCloud - Oxygen Plan
ChemiCloud – Oxygen Plan

Caveats

This load test is intentionally simple. It is read heavy. Many WordPress sites have this sort of traffic profile, but not all do. If you need to perform a WordPress load test with a different traffic profile Kernl supports this. Ideally we should also do multiple tests over time to make sure that this test wasn’t an outlier. Future load test articles will hopefully include this sort of rigor but for now this test can give you reasonable confidence in how you can expect ChemiCloud to perform under a read-heavy load.

The Baseline Test

200 concurrent users, 2 users / s ramp up, 10 minutes, SFO

As most of the hosting providers that we test do, ChemiCloud performed well on the baseline test. They settled in at right around 25 requests / second.

ChemiCloud - Requests
ChemiCloud – Requests

We did see a few failures towards the end of the test, but it appears that it was only a spike. Once the spike passed we didn’t see any more errors for the duration of the test.

ChemiCloud - Failures
ChemiCloud – Failures

The response time distribution for ChemiCloud was solid for this baseline test. 99% of requests finished in 550ms. If we go further down the distribution you can see that 95% of requests finished in ~250ms which is quite good. Even the 100% outlier still wasn’t that bad.

ChemiCloud - Response Time Distribution
ChemiCloud – Response Time Distribution

Sustained Traffic Test

2000 concurrent users, 2 users / s ramp up, 2 hours, SFO

For the sustained traffic test ChemiCloud did a great job serving requests while keeping response times down. As you can see from the graph below, the test settled in to right around 260 requests / second. The journey to that many users was smooth and there aren’t any surprises on the graph.

ChemiCloud - Requests
ChemiCloud – Requests

There were a few failures during the test period, but it appears that they were only a temporary blip. You can see that about half-way through the test we ran into ~32 failures. After that we didn’t see any more for awhile, and then we had one more before not seeing any again for the rest of the test. For some perspective, we performed 1,861,230 requests again ChemiCloud and only 33 failed. Thats a failure rate of 0.0017%! Nice work team ChemiCloud.

ChemiCloud - Failures
ChemiCloud – Failures

The response time distribution was pretty great for the sustained test as well. While there was an outlier at 100% (which is common), 99% of requests finished in under 400ms. Thats an effort worthy of praise with WordPress!

ChemiCloud - Response Time Distribution
ChemiCloud – Response Time Distribution

Traffic Spike Test

20000 concurrent users, 10 users / s ramp up, 1 hour, SFO

The traffic spike load test is brutal for any host. Nobody ever expects to see this kind of traffic out of nowhere so few are prepared for it. ChemiCloud handled the traffic rather well though. We eventually reached 1200 requests / second which is pretty impressive for a plan that costs $17.95 a month. There weren’t any surprises on the way up to that level of traffic, but as you’ll see we did start to see error rates increase.

ChemiCloud - Requests
ChemiCloud – Requests per Second

At about 15 minutes into the load test we started to see an uptick in failure rates. The rate of failure stayed consistent throughout the test after that. This is a fairly common pattern when hosts become overloaded with traffic. In general ChemiCloud performed well even with these failures. We sent 4,332,244 requests to ChemiCloud over an hour period and 134,893 failed. For this sort of load test a failure rate of 3.1% isn’t bad.

ChemiCloud - Failures
ChemiCloud – Failures

The most interesting graph from this load test was the response time distribution. You would expect to see a general degradation of response time performance as request failures increased but that wasn’t the case at all. Everything below the 99th percentile performed remarkably well considering the traffic we threw at it. 98% of requests finished in under 370ms. Great work!

ChemiCloud - Response Time Distribution
ChemiCloud – Response Time Distribution

Conclusions

ChemiCloud competes well with the other hosts that we’ve tested. They have a solid price-point and you get a lot of control over your WordPress environment. If you need a host that can handle some solid traffic spikes they are a good choice.

Want to be part of the Kernl WordPress Load Testing Beta? Sign up and then send an email to jack@kernl.us

What’s New With Kernl – November 2018

It has been a busy few months for Kernl. Lots of great work has gone into the WordPress load testing feature work as well as a few structural changes to increase reliability.

  • Cache moved to Redis – For as long as Kernl has existed our cache backend was powered by Memcached. We have now finished migrating to Redis hosted at Compose.io.
  • AngularJS Upgrade to 1.7.5 – Fairly straight-forward upgrade to Angular 1.7.5. We wanted to take advantage of performance improvements and few bug fixes.
  • WordPress Load TestingΒ – Over the past few months we’ve been cooking up something new. Imagine if you could easily test performance changes to you or your client’s WordPress installation? Or be able to tell your client with confidence how many customers at a time their site can support (and what their experience will be like!). What if you could do all this without writing a single line of code or spinning up your own testing infrastructure? We’re ready to start beta testing so send an email to jack@kernl.us if you would like to be a part of it.