EasyWP Performance Review

Back in 2019 I became aware of a new player in the managed WordPress hosting space: EasyWP by Namecheap. As a Namecheap customer (domain registrar) I was curious about their entry into this market and gave them a try. When I initially did performance tests, the platform was still very new and it showed. But now its October 2020 and the platform has matured. Let’s dig into the performance

How did we test EasyWP Performance?

To performance test EasyWP we first created an export of this blog and imported it into the WordPress instance that EasyWP stood up for us. After that we ran a series of tests:

  • Authenticated Browsing
    • 50 users, 100 minutes
    • 500 users, 100 minutes
  • Unauthenticated Browsing
    • 200 users, 100 minutes
    • 2000 users, 100 minutes

All tests data generators were distributed in the DigitalOcean data centers in New York City, London, and Singapore.

What do each of these test types mean?

  • Authenticated Browsing – The load test user will authenticate and then browse around the front end of the website. In most cases this causes no caching to happen.
  • Unauthenticated Browsing – The load test user browses around the front end of the website. No login is performed. In most cases this means that we’ll hit cached pages.

EasyWP Authenticated Browsing – 50 Users

The first test we ran was an authenticated browsing test with 50 concurrent users. We used this as our baseline, because we assumed that EasyWP would be able to handle it without any issues.

Graph of EasyWP Authenticated - 50 Users Requests per Second
EasyWP Authenticated – 50 Users Requests per Second

As you can see we settled in at around 40 req/s with few errors. Not bad for un-cached performance.

Graph of EasyWP Authenticated - 50 Users Response Time
EasyWP Authenticated – 50 Users Response Time

Next we look at the average and median response times. With a median just under 200ms, this seemed like fairly good performance for being authenticated and with traffic distributed around the globe.

Graph of EasyWP Authenticated - 50 Users Response Time Distribution
EasyWP Authenticated – 50 Users Response Time Distribution

Finally, we take a look at the response time distribution. With the exception of the 100% outlier, 99% of all requests finished in under 660ms. This is great performance for an authenticated test.

EasyWP Authenticated Browsing – 500 Users

Our next test was the same as the previous test, except we increased the number of concurrent users by an order of magnitude. At this level, I expect things to fail when running the authenticated scenario. WordPress has awful performance without caching and you would need some seriously beefy infrastructure to absorb this number of authenticated requests.

Graph of EasyWP Authenticated – 500 Users Requests per Second
EasyWP Authenticated – 500 Users Requests per Second

As you can see things start off pretty great before going right off the rails. I’m not sure what is happening in the background at EasyWP, but I suspect that the service was scaling up while this test was running, otherwise we would have never made it to > 250 successful requests per second.

Graph of EasyWP Authenticated – 500 Users Response Time
EasyWP Authenticated – 500 Users Response Time

After the initial shock from the traffic volume, the median response time settled in at around 200ms. The average is much higher though, meaning we have some pretty serious outliers. These will likely show up in the response time distribution graph.

Graph of EasyWP Authenticated – 500 Users Response Time Distribution
EasyWP Authenticated – 500 Users Response Time Distribution

Finally we take a look at the response time distribution. Not entirely terrible given the amount of traffic we were throwing at EasyWP. 99% of requests completed in ~6.4 seconds. The outliers here were BIG though which skewed the average response time data.

EasyWP Unauthenticated Browsing – 200 Users

Next we move on to unauthenticated front end browsing. This is the use case that most manged WordPress hosting platforms optimize for (because 99% of traffic falls into this category). Let’s see how EasyWP does.

Graph of EasyWP Unauthenticated – 200 Users Requests per Second
EasyWP Unauthenticated – 200 Users Requests per Second

As expected for the unauthenticated baseline scenario, EasyWP did well. We settled in at around ~160 requests per seconds with very few errors throughout the entire duration of the test.

Graph of EasyWP Unauthenticated – 200 Users Response Time
EasyWP Unauthenticated – 200 Users Response Time

The response time graph looks solid. The median stayed at right around 190ms which is to be expected on a test with load generators spread across the globe. The average is a bit higher, so we probably had some outliers. Lets look at the distribution to see.

Graph of EasyWP Unauthenticated – 200 Users Response Time Distribution
EasyWP Unauthenticated – 200 Users Response Time Distribution

99% of requests finished in 820ms, with the outlier at 100% being 12 seconds. I honestly expected the 99% number to be a bit lower, but it’s still in a fine range for a load test of this size.

EasyWP Unauthenticated Browsing – 2000 Users

Finally we take a look at what happens to performance on EasyWP when we increase the size of the load test by an order of magnitude. Some hosts handle this test fine, others struggle. So let’s get to it.

Graph of EasyWP Unauthenticated – 2000 Users Requests Per Second
EasyWP Unauthenticated – 2000 Users Requests Per Second

EasyWP performed well in this test (ignore the spike, that was a reporting problem). We settled in at around ~1400 requests per second with very few errors. If you do the math, that’s 120M requests per day.

Graph of EasyWP Unauthenticated – 2000 Users Response Time
EasyWP Unauthenticated – 2000 Users Response Time

The performance on the 2000 user test was very similar to the 200 user test, which is great for EasyWP. Normally an order of magnitude increase in users would see a decrease in performance, but EasyWP handled it well.

Graph of EasyWP Unauthenticated – 2000 Users Response Time Distribution
EasyWP Unauthenticated – 2000 Users Response Time Distribution

The response time distribution was also strikingly similar to the 200 user test. The only change is that the 100% outlier is a lot higher, which is to be expected when working at a higher scale.

EasyWP Performance Conclusions

In general the performance of EasyWP was solid, especially considering how cost-effective it is. Is it the fastest WordPress host that we’ve tested? No. But it is in the top 2-3 for managed hosts in the performance per dollar category. EasyWP is definitely worth giving a try.

Hummingbird Cache for WordPress Performance Review

The Hummingbird Cache plugin is one of many different caching plugins available in the WordPress ecosystem. Enabling it will increase your site’s performance significantly, but by how much? In this review we’re going to use Kernl’s WordPress Load Testing tool to push our Hummingbird Cache WordPress installation to it’s limits.

HummingBird cache pushed to limits

Test System Setup

As with most of our cache reviews, we used a pretty standard PHP-FPM + Nginx setup.

  • DigitalOcean $5/month 1vCPU 1GB RAM machine
  • Ubuntu 20.04
  • PHP (FPM) 7.4
  • Nginx 1.18
  • MariaDB 10.3
  • Content – For this test I imported the contents of my personal blog and used it for testing.

The test system was located in San Francisco, CA, USA. Load test virtual users were located in New York, NY, USA along with some of the high volume tests spreading virtual users around Europe.

How did we test Hummingbird Cache?

To test the Hummingbird WordPress caching plugin ran 3 different load tests with Kernl WordPress Load Testing.

  1. Baseline – This is a 200 concurrent user test for 60 minutes with no caching enabled.
  2. Cache Enabled – The same test as the baseline run, but with caching enabled. This is the “apples to apples” comparison.
  3. Cache++ – After the “apples to apples” comparison, we pumped up the concurrent users to 400 to see how well the plugin would respond.

Baseline Load Test

The baseline load test is just the bare WordPress setup with no plugins enabled and the base TwentyTwenty theme. As expected performance isn’t great but it isn’t terrible either.

Hummingbird Cache - Baseline Request Throughput
Hummingbird Cache – Baseline Request Throughput

You can see from the throughput chart that the base WordPress installation with no caching enabled settled in at around 34 requests/s. Not too shabby, but what was the quality of those requests?

Hummingbird Cache - Baseline Response Times
Hummingbird Cache – Baseline Response Times

The average and median response times tell a story steady degradation of the user experience before finally settling at just shy of 5 seconds. If I were a reader of that blog, I would be extremely turned off by waiting for 5 seconds just to have the page load start.

Hummingbird Cache - Baseline Response Time Distribution
Hummingbird Cache – Baseline Response Time Distribution

The response time distribution is pretty awful here. 50% of requests finished in under 5s, and 99% of requests finished in under 5.5s. In most load tests we like to see the P50 number be a lot lower than the P99 number. In a perfect world they’re both really low, but that doesn’t happen in most cases.

Cache Enabled Load Test

Our next test was the same as the baseline test, but with HummingBird cache enabled. We went with all the default options making no changes to the settings.

Hummingbird Cache – Cache Enabled Request Throughput
Hummingbird Cache – Cache Enabled Request Throughput

As expected of a caching plugin, throughput goes up a lot and settles in at around 175 requests/second with zero errors. This is a nearly 6x improvement in throughput. But what about the response times? How did this look to the end user?

Hummingbird Cache – Cache Enabled Response Times
Hummingbird Cache – Cache Enabled Response Times

The response time results are extremely promising. The average response time was around 95ms and the median was around 75ms. Most performance best-practices hope for your site to respond within 100ms, which this plugin easily accomplishes even under incredibly heavy load. Let’s break the response time numbers down further.

Hummingbird Cache – Cache Enabled Response Time Distribution
Hummingbird Cache – Cache Enabled Response Time Distribution

For 50% of our users, the response time was 75ms or less. For 99% of our users, response time was less than 160ms. These are great numbers and just what I would expect from a WordPress caching plugin.

Cache++ Load Test

Now that we’ve established that Hummingbird Cache does a great job under (somewhat) normal circumstances, lets see what happens if we double the traffic (400 concurrent users -vs- 200 concurrent users).

Hummingbird Cache++ – Cache Enabled Request Throughput
Hummingbird Cache++ – Cache Enabled Request Throughput

Event at 2X the number of users, we don’t see any errors and we see the throughput settling at about 325 requests per second. If you do the math, this is about 28 million requests a day. On a $5 box. Obviously this test is fairly naive, but it does show that the plugin can handle some serious traffic when needed.

Hummingbird Cache++ – Cache Enabled Response Times
Hummingbird Cache++ – Cache Enabled Response Times

The best part about this test is that even with incredible load the response time average and median are still below 180ms. Most users visiting a site would be extremely happy with response times in that range.

Hummingbird Cache++ – Cache Enabled Response Time Distribution
Hummingbird Cache++ – Cache Enabled Response Time Distribution

The response time distribution still tells a reasonable story. 50% of users see responses in 150ms or less and 99% see responses in 375ms or less. Solid performance from the Hummingbird Cache team.

Hummingbird Cache Conclusions

If you need a caching plugin for your site, Hummingbird Cache is a solid choice. It performs well, was easy to install, and was generally low friction. I found the user interface to be a little immature, but that doesn’t change the excellent performance we saw during our tests.

Want to run your own load tests? Sign up for Kernl!

Should I use Memcached or Redis for WordPress caching?

Choosing between Memcached or Redis for your WordPress cache is a tough decision. Not because they have vastly different performance profiles (they don’t), but because either choice is a good one depending on your needs. In this post we’re going to explore the differences between Redis and Memcached, how they perform for WordPress, and a lot of different non-performance things you should consider when making your choice.

What is Memcached?

memcached logo

Memcached is an open-source, high performance, distributed memory object caching system. What does that mean? It means you can store a bunch of strings in memory and access them really fast. From a WordPress perspective, it means that using a caching plugin like W3 Total Cache we can store the results of the complicated SQL queries that WordPress does in memory and have them available instantly.

What is Redis?

Redis Logo

Redis is an open source in-memory store that can be used as a cache or a message broker. It’s a bit different then Memcached because you get a lot more out of the box with it. For example, Redis has built in replication, transactions, disk persistence, and provides high availability and partitioning. All those features means that managing WordPress can be a little harder to do, but not much harder. Especially if you just need to use it as a cache.

Performance

Both Redis and Memcached have excellent performance. They’re both used by some of the largest websites in the world and are fully ingrained in the Fortune 500. Given that all things are not created equal, let’s see how they perform with a read-heavy WordPress site (this blog).

The Setup

The load tests are performed against the DigitalOcean WordPress Marketplace image with either Redis or Memcached installed alongside of it. The machines have 2 vCPUs, 2 GB RAM and live in DigitalOcean’s SFO2 (San Francisco) data center.

The load test configuration:

  • 500 concurrent users
  • 2 users / second ramp up
  • 45 minute test ( ran twice )
  • Traffic comes from Digital Ocean’s NYC3 data center.

The content of the load test is a copy of this blog.

Baseline Performance (No Cache)

The baseline performance for WordPress with no cache isn’t great.

baseline requests/failures
50 requests / second with LOTS of failures

The response time also isn’t great. A little over 2 seconds on average.

baseline response times
~2 seconds response time on average.

Redis Performance

Once we install Redis and configure W3 Total Cache to use it, the number of requests that we can handle increases substantially.

redis requests/failures
300 requests per second

The requests remain steady at around 300 per second and no failures are recorded. The response time also improves quite a bit.

redis response times
Average ~475ms response time

475ms isn’t bad at all. That’s 4 times faster response times then without any caching at all.

Memcached Performance

With Memcached installed and W3 Total Cache configured to use it, we see some excellent performance.

memcached requests/failures
425 requests per second

In this situation, Memcached performs even better then Redis with 425 req/s versus Redis’ 300 req/s. Response time improvements are similar.

memcached response times
115ms response time

The Memcached response time is almost 3 times faster than the Redis response time. In general, the results where Memcached is faster than Redis are surprising. In most benchmarks Redis is equal or faster than Memcached, so it’s likely a configuration problem.

Other Considerations

When deciding what cache to use with your WordPress setup, there are a few other considerations your should be looking at:

  • Ease of setup – As you can see from the performance results above, Memcached has better performance out of the box. Knowing what I know about Redis this is likely a configuration issue, but the fact that I could get that level of performance with no configuration from Memcached is a good data point.
  • 3rd Party Hosting – Do you really want to manage your own Redis or Memcached server? If you don’t, you’ll want to look at the landscape of 3rd party providers. Redis has a robust provider ecosystem. Memcached’s is a little less robust.
  • Persistence – Do you need your cache to survive a reboot? This is important if the cost of re-populating your cache is too high for your system. If you do need persistence, Redis is your best option.
  • High Availability If you need high availability of your caching cluster, Redis is the clear winner here. Memcached can be made to operate this way, but Redis has it baked in to the core of the application.

If you’d like to see the full results of the load testing runs on Kernl, see the links below.

WordPress Plugin Performance Implications: Wordfence

In the world of WordPress there are a lot of different plugins you can install to extend its functionality. One of the most popular plugins is Wordfence, a security plugin developed by the fine folks over at Defiant.

Test the performance implications of your own WordPress plugins with Kernl WordPress Load Testing.

Most WordPress developers understand that the more plugins you add to your site the slower it goes, but exactly how much slower isn’t something that is often measured. More importantly, nobody has bothered to figure out the performance implications of installing many of the most popular plugins available to WordPress users today.

This all changes now with this series of blog posts exploring the performance implications of different WordPress plugins. In this series of posts we’ll test each plugin in isolation and then with caching enabled using Kernl’s WordPress load testing service. First up, is Wordfence.

Machine Setup

The test machine for these tests was a $5 Digital Ocean droplet in their SFO2 (San Francisco) data center. The machine has 1GB of RAM, 1 vCPU, and a 25GB hard disk.

The software installed on the machine is as follows:

  • Ubuntu 19.04 with all updates installed.
  • Nginx 1.16.1
  • PHP-FPM 7.3
  • MariaDB 10.3

For tests that required caching we used our favorite caching plugin, W3 Total Cache, with memcached as the data store.

The theme that was used was TwentyTwenty with no modifications and the content was an export of this blog.

All traffic was generated out of Digital Ocean’s NYC3 (New York City) data center.

Test Methodology

A series of 4 tests were run to test the performance implications of installing Wordfence. They were:

Each test was with 200 concurrent users for 1 hour.

Maximum Requests per Second

The first metric that we looked at was the maximum requests per second that the site was able to handle under each situation outlined above.

Wordfence - Maximum Requests per Second
Wordfence – Maximum Requests per Second

As you can see the difference between having Wordfence enabled and having Wordfence disabled is huge. The non-cached site with no plugins enabled handled a maximum of 26 requests per second. With Wordfence enabled it could only handle 12.

More interesting though was that with caching enabled (W3 Total Cache) the site could handle 165 requests per second, but only 67 requests per second with Wordfence enabled.

These results were so surprising that we ran the tests twice. The results were the same (within 1%-2%) each time.

First Error Occurrence

The next metric we looked at was when did the first error occur during our load tests.

Wordfence - First Error req/s
Wordfence – First Error req/s

Once again we see that adding Wordfence took a pretty serious toll on our performance. For the baseline test (no plugins, no cache) we see our first error at around 26 requests / second. In the Wordfence test with no cache we saw it at 12 requests / second.

With our caching enabled test, we never saw any errors when Wordfence wasn’t enabled. However with Wordfence enabled we saw our first error at 56 requests / second.

Average Response Time

Now that we’ve looked at server capacity metrics (max requests, first error), let’s take a look at how your end user experience changes with Wordfence enabled.

Wordfence - Avg. Response Time (milliseconds)
Wordfence – Avg. Response Time (milliseconds)

These results were fairly interesting and not at all what was expected. The average response time for the baseline test was around 4.6 seconds, while the average response with Wordfence enabled was about 2.5 seconds. So why might that be? Looking through the data it appears that the baseline test had far more successful requests and that with Wordfence enabled the requests seemed to fail faster. In short, baseline test had higher throughput but slower response time. The Wordfence test had lower throughput and faster response time.

Turning our attention to the caching scenarios, we can see that enabling Wordfence is extremely problematic at scale. With caching enabled, our baseline test had an average response time of 146ms. With caching + Wordfence that time ballooned over 10x to 1874ms.

99th Percentile Response Time

Our final metric was looking at the 99th percentile response times for our different scenarios. What does that mean? It answers the question “How long does it take for 99% of requests to finish?”. This intentionally leaves out the last 1% because those are often outliers.

Wordfence - 99th Percentile Response Time
Wordfence – 99th Percentile Response Time

As you can see above, the un-cached 99th percentile hovers right around 5s when Wordfence is enabled or disabled. Since this is the 99th percentile such a high response time isn’t too surprising in both scenarios.

The more interesting scenario is when caching is enabled. For WordPress with only a caching plugin running, the 99th percentile is 370ms. That means 99% of all requests finished in 370ms. With Wordfence also enabled (Wordfence + caching), that number jumped to 2400ms. That’s a ~7X increase.

Conclusions

From these tests we came to a few conclusions:

  1. Caching does not solve all of your performance problems. If your cache strategy relies on requests making it all the way through to WordPress, then you are very likely to still take a performance hit from other plugins. Things like Cloudflare, Varnish, Litespeed, or Nginx can help alleviate this problem.
  2. Running Wordfence is expensive from a performance standpoint. The data suggests that by simply enabling Wordfence you lose about 50% of your maximum capacity and can increase your response time between 2x-7x.
  3. Wordfence is still worth it for a lot of people. If you’ve operated a WordPress site for any length of time you know how often they get attacked. Wordfence does a great job of reducing attack surface area and making it hard for people to attack you.

Test the performance implications of your own WordPress plugins with Kernl WordPress Load Testing.