Vultr WooCommerce Showdown – Cloud or High Frequency?

So you’ve decided to host your WooCommerce site on Vultr. Great! You could be hosting through Cloudways or direct, but you still need to make the choice: Vultr Cloud or Vultr High Frequency? How do you choose? The pricing is pretty close and it hasn’t always been clear which has the best performance characteristics in any given situation.

To help you decide if you should choose Vultr Cloud or Vultr High Frequency (HF) for WooCommerce, we’re going to run some relatively simple load tests that should expose the differences between the two.

(click here for the tldr; at the end of this article)

You can load test your own WooCommerce sites using RoboSwarm by Kernl.

What are we testing and how?

When doing performance load testing of WooCommerce its important to have consistency. To ensure consistency of our WooCommerce setup on Vultr we used the pre-configured WooCommerce install provided by Cloudways. And imported our products and setup from Cloud -> High Frequency.

The systems we tested were:

• Cloud
  • 2vCPU
  • 4GB RAM
  • 80GB SSD
• High Frequency
  • 2vCPU
  • 4GB RAM
  • 128GB SSD

The test setup for each system:

• Concurrent users: 100
• Ramp up: 1 user per second
• Duration: 60 minutes
• WooCommerce location: New Jersey
• Load generator location: San Francisco

And last but not least, the load test itself:

1. Visit /shop
2. Visit /product-a
3. Add product-a to cart
4. Visit /product-b
5. Add product-b to cart
6. Visit /cart
7. Visit /checkout
8. Start and finish checkout process
9. Order confirmation page
10. Repeat.

As far as WooCommerce specific items, the only thing of note is that for our payment method we chose “Cash on delivery”. We mentioned in another post that we don’t want to load test our payment provider because it would likely fall outside acceptable use in their terms of service.

Vultr Cloud WooCommerce Performance

We’ll start out by taking a look at the overall performance of WooCommerce.

Looking at the chart above you can see that performance remained fairly consistent for the first 30% of the test but then we started to see some resource saturation and things got a little (but not terribly!) spotty. We averaged around 30 req/s with spotty failures from time to time.

You can see from the chart above that response time slowly increased over time during this test but remained on average below 3 seconds. Not bad for 100 concurrent users.

The overall response time distribution is where numbers start to get interesting. 50% of requests finished in a little less than 3 seconds, but when you get towards the upper 10% of requests things start to go sideways.

90% of requests finished in less than 7 seconds, but 99% finished in 17 seconds. We’re starting to get well passed the point of good performance here.

Checkout Performance

The heart of on online store is the checkout process. If that checkout process isn’t quick, people will straight up abandon their cart and never return. With that in mind, lets take a look at the performance of the checkout action on WooCommerce.

The blue line is how many checkouts per second the store was able to process. The red line is how many checkouts per second resulted in an error. If we were to put actual dollars to these numbers:

• If each checkout was worth $100, you would be able to make $300 in sales per second.
• If each checkout was worth $100, you would lose $100 is sales per second.
• Over the course of 1 minute, due to failed checkouts you would lose a total of $6,000 in potential sales.

The response time distribution for successful checkouts was decent. 50% of all checkouts finished in about 300ms, 90% in less than 800ms, and 100% of requests in less than 1000ms.

Vultr High Frequency WooCommerce Performance

Vultr High Frequency instances are powered by “high clock speed CPUs” according to Vultr. Let’s see if that has any measurable impact on WooCommerce performance.

Looking at the chart above you can see that performance was up and down for most of the test, but on average request throughput was around 50/s. There were instances of high failure rates that would seem to ramp up, explode, and then disappear for awhile.

The average response time was starting to level out just shy of 1600ms towards the end of the test, and the median sat at right around 1200ms for most of the test. If you have a good memory, you’ll remember the numbers from the Vultr Cloud Compute test were almost twice as high for this.

The response time distribution is interesting for the high frequency test. 50% of all responses were returned in under 1200ms. 99% of all responses were under 9s. The p100 outlier is up at more than 40s.

Vultr High Frequency WooCommerce Checkout Performance

Now that we’ve looked at overall performance, lets dig in to the checkout-specific performance numbers.

As the test ramped up and before resource saturation started to become an issue, we were checking out at a rate of ~5 per second. Eventually things settled towards ~4 per second with ~1 failure per second. This is a +1 checkout/s increase over Vultr Cloud.

The response time distribution for checkout is really great on the Vultr High Frequency instance. The gap between the p50 (200ms) and the p100 (375ms) is only a 175ms which means that all customers are getting a reasonably solid experience during their checkout process. On Vultr Cloud the spread was closer to 700ms.

Analysis and Conclusions

Now that we’ve gone through both of the load tests, let’s compare the results.

	$	Total Requests	Fail %	p50	p95	p99
Vultr Cloud	$20	97156	6307	2.8s	11s	17s
Vultr High Frequency	$24	153765	19953	1.2s	5.6s	8.8s

As you can see, for $4 extra per month you basically get to cut your response time in half and increase your throughput by 1.5x. That seems like a great deal to me. The 6.4% request failure rate of cloud is twice as good as the 12.9% request failure rate of high frequency though.

Based on the chart above, I would probably do something like

• Use Vultr High Frequency for WooCommerce hosting.
• When resource contention starts to hit ~60% or so, either scale up (increase instance size) or scale out (increase number of instances).

I think you would get the best performance in that situation at the best cost.

Before we wrap up, let’s look at a checkout specific performance comparison.

	$	Checkouts	Failures	p50	p95	p99
Vultr Cloud	$20	8902	512	11s	20s	24s
Vultr High Frequency	$24	12597	1031	5.5s	10s	14s

The story of the checkout performance table and the overall performance table is that for $4 extra per month you get nearly twice the performance. Maybe not in all cases, but definitely in enough that it warrants the extra cost if performance is a big concern.

Looking at the data above, I try to put myself in the shoes of a customer. When I start a checkout, am I willing to wait 5s for it to go through? Probably, though it’s not ideal. 11s? We’re starting to get to the point where I think something is broken.

Overall, if I were professionally hosting a WooCommerce site, I would choose Vultr High Frequency over Vultr Cloud.

You can load test your own WooCommerce sites using RoboSwarm by Kernl.