As we enter the holiday season here in the US work on Kernl slows down. But that doesn’t mean we didn’t get some great stuff done this month. Let’s dive in!
Features, Bug Fixes, Infrastructure
Site Health Worker – In the beginning, Kernl’s WordPress Site Health service used a simple cron job to run TTFB and response time tests against sites. The cron job was starting to get overwhelmed so it was refactored to use a queue based pub/sub model. The benefit here is that we can scale site health workers horizontally in the future as usage goes up.
Push-to-build multi-commit, multi-branch bug – In the very early days of Kernl we made the naive assumption that multi-commit pushes would contain commits only from one branch. This caused an issue where building from pushes didn’t work in some instances. This has been resolved.
Wildcard domain validation for licenses – The license management tool now supports wildcard domain validation. (ex: *.kernl.us)
/v2/license updates – The /v2/license endpoints now return the type of item the license is attached to (plugin, theme, null).
That’s it for this month! I hope everyone has a great December.
Happy (almost) November! It was a very bug-fix and infrastructure heavy month for Kernl, so lets dive in!
Features, Bugs, & Infrastructure
Redis robust connection handling – We’ve done some resiliency work so that Redis failing will not cause any issues in Kernl. If Redis goes away, all traffic is handled un-cached. Once it comes back, we automatically re-connect. We will also timeout any calls to Redis after 50ms and assume it failed.
Stripe Checkout – Kernl has been using a deprecated version of Stripe’s Checkout.js for the past year or so. We finally migrated to using Stripe’s Checkout service which directs you to Stripe’s website for card adding, and then back to Kernl. Localization and card handling is a lot better this way, and you should have fewer issues adding different payment types to Kernl.
Composer Installation – You can now install the Kernl update checker via Composer. Go to your plugin or theme detail page and look for the new tab in the “Meta” section.
Node.js – All Kernl servers have been upgraded to 12.19.0. This new version of Node.js brings stability and security improvements with it.
RoboSwarm Email Bug – There was a regression where RoboSwarm was sending an email when Kernl customers signed up. RoboSwarm powers the Kernl WordPress load testing service.
Build pipeline – We use BitBucket’s Pipelines services to power the continuous integration of our code base. There was a test with intermittent failures that was causing this to fail in unusual ways.
Admin account bug – A bug was fixed where an admin could get their account into a bad state by adding themselves as a team member.
License management documentation update – We’ve corrected an omission in the license management API docs. The notes field was missing from the documentation.
Invoices – For the longest time Kernl generated it’s invoices via WKHTMLtoPDF. We’ve finally switched over to using the invoices that Stripe generates for us, removing a significant headache from our code base.
Server host name – In load balanced systems it can be hard to track down issues if you don’t know what origin server it came from. Kernl now returns the origin server host name in the response headers to make this easier.
Development environment upgrades – All of our build packages have been upgraded to latest versions.
Back in 2019 I became aware of a new player in the managed WordPress hosting space: EasyWP by Namecheap. As a Namecheap customer (domain registrar) I was curious about their entry into this market and gave them a try. When I initially did performance tests, the platform was still very new and it showed. But now its October 2020 and the platform has matured. Let’s dig into the performance
How did we test EasyWP Performance?
To performance test EasyWP we first created an export of this blog and imported it into the WordPress instance that EasyWP stood up for us. After that we ran a series of tests:
50 users, 100 minutes
500 users, 100 minutes
200 users, 100 minutes
2000 users, 100 minutes
All tests data generators were distributed in the DigitalOcean data centers in New York City, London, and Singapore.
What do each of these test types mean?
Authenticated Browsing – The load test user will authenticate and then browse around the front end of the website. In most cases this causes no caching to happen.
Unauthenticated Browsing – The load test user browses around the front end of the website. No login is performed. In most cases this means that we’ll hit cached pages.
EasyWP Authenticated Browsing – 50 Users
The first test we ran was an authenticated browsing test with 50 concurrent users. We used this as our baseline, because we assumed that EasyWP would be able to handle it without any issues.
As you can see we settled in at around 40 req/s with few errors. Not bad for un-cached performance.
Next we look at the average and median response times. With a median just under 200ms, this seemed like fairly good performance for being authenticated and with traffic distributed around the globe.
Finally, we take a look at the response time distribution. With the exception of the 100% outlier, 99% of all requests finished in under 660ms. This is great performance for an authenticated test.
EasyWP Authenticated Browsing – 500 Users
Our next test was the same as the previous test, except we increased the number of concurrent users by an order of magnitude. At this level, I expect things to fail when running the authenticated scenario. WordPress has awful performance without caching and you would need some seriously beefy infrastructure to absorb this number of authenticated requests.
As you can see things start off pretty great before going right off the rails. I’m not sure what is happening in the background at EasyWP, but I suspect that the service was scaling up while this test was running, otherwise we would have never made it to > 250 successful requests per second.
After the initial shock from the traffic volume, the median response time settled in at around 200ms. The average is much higher though, meaning we have some pretty serious outliers. These will likely show up in the response time distribution graph.
Finally we take a look at the response time distribution. Not entirely terrible given the amount of traffic we were throwing at EasyWP. 99% of requests completed in ~6.4 seconds. The outliers here were BIG though which skewed the average response time data.
EasyWP Unauthenticated Browsing – 200 Users
Next we move on to unauthenticated front end browsing. This is the use case that most manged WordPress hosting platforms optimize for (because 99% of traffic falls into this category). Let’s see how EasyWP does.
As expected for the unauthenticated baseline scenario, EasyWP did well. We settled in at around ~160 requests per seconds with very few errors throughout the entire duration of the test.
The response time graph looks solid. The median stayed at right around 190ms which is to be expected on a test with load generators spread across the globe. The average is a bit higher, so we probably had some outliers. Lets look at the distribution to see.
99% of requests finished in 820ms, with the outlier at 100% being 12 seconds. I honestly expected the 99% number to be a bit lower, but it’s still in a fine range for a load test of this size.
EasyWP Unauthenticated Browsing – 2000 Users
Finally we take a look at what happens to performance on EasyWP when we increase the size of the load test by an order of magnitude. Some hosts handle this test fine, others struggle. So let’s get to it.
EasyWP performed well in this test (ignore the spike, that was a reporting problem). We settled in at around ~1400 requests per second with very few errors. If you do the math, that’s 120M requests per day.
The performance on the 2000 user test was very similar to the 200 user test, which is great for EasyWP. Normally an order of magnitude increase in users would see a decrease in performance, but EasyWP handled it well.
The response time distribution was also strikingly similar to the 200 user test. The only change is that the 100% outlier is a lot higher, which is to be expected when working at a higher scale.
EasyWP Performance Conclusions
In general the performance of EasyWP was solid, especially considering how cost-effective it is. Is it the fastest WordPress host that we’ve tested? No. But it is in the top 2-3 for managed hosts in the performance per dollar category. EasyWP is definitely worth giving a try.
Hello! I hope that everyone had a great September. This wasn’t a big feature month for Kernl, but we did fix some bugs and make some minor enhancements. Let’s dive in!
Features, Bugs, and Other Improvements
Load Testing – The 100th percentile for “final” response time distributions wasn’t being recorded. The reporting position changed in the underlying software that runs the load test which caused this regression. This has been resolved.
Analytics Theme Tracking – For those of you who use Kernl Analytics, you can now track which themes your customers are using. We also released v2.0.2 of the Kernl Update Checker to support this change.
License Management Bug – A bug was reported where the domain restriction on licenses wasn’t being honored in all situations. This was resolved and a unit test was added for the regression.
Analytics Infrastructure – As Kernl grows, so do the needs of our analytics services. The analytics database host has been upgraded to 4GB of RAM and 80GB of disk space.
Version Upload Bug – In rare instances, the file upload to AWS S3 might fail when uploading a new plugin or theme version. In those cases, Kernl didn’t catch the failure and would still create a new plugin/theme version. We now verify the upload was successful before creating the new plugin/theme version entry in Kernl.
That’s it for this month! Have an excellent October!
When running a WordPress site, security should be first and foremost at the front of your mind. One of the best ways to prevent malicious users from accessing your /wp-admin area is enabling two-factor auth (2FA) for WordPress.
The first step to getting two-factor auth enabled for your WordPress site is to install Wordfence. To do that:
Log in WordPress and go to Plugins.
Click “Add New”.
Search for “Wordfence”
Install and activate Wordfence
Now that Wordfence is installed you’ll need to enable and configure two-factor auth. To do that click the “Wordfence” menu that appeared after activation and then go to the “Login Security” sub-menu.
Now you’ll need to choose two factor authentication. I personally use FreeOTP (iOS, Android). Once your authenticator application is installed, scan the code and create the authenticator entry on your phone. Be sure to download your recovery codes too, just in case your phone is bricked, lost, or otherwise unavailable.
Testing Two-Factor Auth Out
Now that two-factor authentication is enabled for your WordPress site, you need to try it out! Log out of your admin, and now when you log back in you’ll be presented with this screen:
Go to your authenticate app, press the entry for your site, enter the code, and log in!
Wordfence makes is really easy to set up 2FA on WordPress. With it being this easy, it’s hard to justify not having it. After all, even if someone manages to get your password they still won’t be able to log in unless the can compromise your phone too. If you’re worried about performance, this feature of Wordfence doesn’t effect your site’s performance in a meaningful way (which can not be said about Wordfence’s other features). All in all, the Wordfence team has done a great job making this level of security accessible to the wider WordPress community.
Need automatic updates for your premium plugins & themes? Check out Kernl.
Hello everyone! It’s been awhile since the last update message, but now that summer is nearly over in the northern hemisphere Kernl will again be receiving regular updates and enhancements.
This doesn’t mean that we weren’t busy though! Let’s dig in.
License Management Activated Domains – When viewing your license list, you can now see the domains that have activated the license. If you go into the license detail view, you can make changes to that list. This data will give you granular insight into who is using a specific license.
Kernl Update Checker Automatic Update Support – With the release of WordPress 5.5 we finally upgraded our update checker library. There are a few differences like it being multiple files now, and the instantiation code changing a bit, but the upgrade path is simple for those who want automatic updates.
Bug Fixes & Miscellaneous Changes
Package Upgrades (Analytics) – All supporting packages have been upgraded to their latest version for better performance and security.
Date Retention Bug (Analytics) – We weren’t cleaning up some tables we were supposed to be. This lead to holding onto some data for much longer than 365 days.
UX Improvements (Analytics) – The UX around selecting and comparing dates for Kernl Analytics was a little bit confusing. We made some changes that make it a lot easier to understand for a first-time customer.
Data collection issues(Load Testing) – There was an issue where load testing wasn’t collecting data from the master node after an upgrade to the underlying infrastructure. This was resolved.
UX Improvements(Load Testing) – Some load test templates are large and take awhile to return from the server. An indeterminate spinner was added here to let customers know that things are actually happening. The same situation was happening when long running, high volume tests load initially. There is a lot of data that takes some time to return from the server so an indeterminate spinner was added.
Infrastructure (Load Testing) – The Kernl WordPress Load Testing box was upgraded from 1vCPU+1GB RAM to 2vCPU+2GB RAM.
Meta Tag Parsing(Load Testing) – When you verify a site with Kernl, we look for a meta tag in your HTML. Initially we were attempting to find, capture, and validate this using regular expressions. As is tradition, we found this was a bad idea and switched to using an open-source library (Cheerio.js) instead.
All servers have had their packages upgraded to the latest available versions.
The Hummingbird Cache plugin is one of many different caching plugins available in the WordPress ecosystem. Enabling it will increase your site’s performance significantly, but by how much? In this review we’re going to use Kernl’s WordPress Load Testing tool to push our Hummingbird Cache WordPress installation to it’s limits.
Test System Setup
As with most of ourcachereviews, we used a pretty standard PHP-FPM + Nginx setup.
Content – For this test I imported the contents of my personal blog and used it for testing.
The test system was located in San Francisco, CA, USA. Load test virtual users were located in New York, NY, USA along with some of the high volume tests spreading virtual users around Europe.
How did we test Hummingbird Cache?
To test the Hummingbird WordPress caching plugin ran 3 different load tests with Kernl WordPress Load Testing.
Baseline – This is a 200 concurrent user test for 60 minutes with no caching enabled.
Cache Enabled – The same test as the baseline run, but with caching enabled. This is the “apples to apples” comparison.
Cache++ – After the “apples to apples” comparison, we pumped up the concurrent users to 400 to see how well the plugin would respond.
Baseline Load Test
The baseline load test is just the bare WordPress setup with no plugins enabled and the base TwentyTwenty theme. As expected performance isn’t great but it isn’t terrible either.
You can see from the throughput chart that the base WordPress installation with no caching enabled settled in at around 34 requests/s. Not too shabby, but what was the quality of those requests?
The average and median response times tell a story steady degradation of the user experience before finally settling at just shy of 5 seconds. If I were a reader of that blog, I would be extremely turned off by waiting for 5 seconds just to have the page load start.
The response time distribution is pretty awful here. 50% of requests finished in under 5s, and 99% of requests finished in under 5.5s. In most load tests we like to see the P50 number be a lot lower than the P99 number. In a perfect world they’re both really low, but that doesn’t happen in most cases.
Cache Enabled Load Test
Our next test was the same as the baseline test, but with HummingBird cache enabled. We went with all the default options making no changes to the settings.
As expected of a caching plugin, throughput goes up a lot and settles in at around 175 requests/second with zero errors. This is a nearly 6x improvement in throughput. But what about the response times? How did this look to the end user?
The response time results are extremely promising. The average response time was around 95ms and the median was around 75ms. Most performance best-practices hope for your site to respond within 100ms, which this plugin easily accomplishes even under incredibly heavy load. Let’s break the response time numbers down further.
For 50% of our users, the response time was 75ms or less. For 99% of our users, response time was less than 160ms. These are great numbers and just what I would expect from a WordPress caching plugin.
Cache++ Load Test
Now that we’ve established that Hummingbird Cache does a great job under (somewhat) normal circumstances, lets see what happens if we double the traffic (400 concurrent users -vs- 200 concurrent users).
Event at 2X the number of users, we don’t see any errors and we see the throughput settling at about 325 requests per second. If you do the math, this is about 28 million requests a day. On a $5 box. Obviously this test is fairly naive, but it does show that the plugin can handle some serious traffic when needed.
The best part about this test is that even with incredible load the response time average and median are still below 180ms. Most users visiting a site would be extremely happy with response times in that range.
The response time distribution still tells a reasonable story. 50% of users see responses in 150ms or less and 99% see responses in 375ms or less. Solid performance from the Hummingbird Cache team.
Hummingbird Cache Conclusions
If you need a caching plugin for your site, Hummingbird Cache is a solid choice. It performs well, was easy to install, and was generally low friction. I found the user interface to be a little immature, but that doesn’t change the excellent performance we saw during our tests.
If you spend time in the WordPress hosting space and few names constantly come up, and one of them is UpCloud. UpCloud is a European cloud provider with data centers all over the world that claims to have the world’s “fastest cloud servers”. In this review we’re going to take a look at how WordPress performs while hosted on UpCloud and also see how the reliability of the service is.
For our test of WordPress running on UpCloud, we tested 3 different hardware configurations.
1vCPU + 1GB RAM – This is the cheapest possible option for hosting WordPress on UpCloud at $5 / month.
2vCPU + 4GB RAM – At $20 / month, this option is a nice balance of processing power and memory. If I were to host a few sites on UpCloud with moderate amounts of traffic I would probably start here.
4vCPU + 8GB RAM – This machine was as expensive as I was willing to go for this review, coming in at $40 / month.
The software used in this test was the latest available from the included repositories with Ubuntu 20.04.
Ubuntu 20.04 LTS
What was tested?
For each test we ran on UpCloud we used Kernl’s WordPress Load Testing service to generate load against each virtual machine. The content for the load tests was a copy of this blog. The load generators for each test lived in DigitalOcean’s SFO2 data center and ran against UpCloud’s Chicago data center.
UpCloud WordPress Load Tests
We ran a total of 6 WordPress load tests on UpCloud’s servers. For each machine we tested we ran a test where nothing was cached and a test where everything was cached (using W3 Total Cache backed by Memcached). Each test was for 1000 concurrent users for 45 minutes.
1vCPU + 1GB RAM (No Cache)
For this configuration the requests per second peaked at around 41/s before failure rates started to increase. Once failures started in earnest, successful requests leveled off at around 11/s.
On the response time front, things weren’t great. As the request (and error) rate increased, the response times started to get pretty unwieldy.
A graph of median and average response times doesn’t always tell the whole story, so lets take a look at the response time distribution.
The response time distribution tells us that 50% of requests finished in under 5000ms, but that 99% finished in under 6000ms. Usually you want to see a large difference between 50th and 99th percentile. But here they’re awfully close, meaning that most request response times were pretty terrible.
1vCPU + 1GB RAM (Cached)
Now that we’ve looked at uncached performance, lets take a look at how this UpCloud server handled WordPress with caching enabled.
As you can see, performance is much better. We end up leveling off at around 436 request per second. The only issue here is that we still have quite a few failures. Certainly more failures than would be acceptable in a production situation.
Looking at response times you can see performance is a lot better, and much more in line with what you’d expect from a WordPress site that’s under heavy load. Average response time was a hair less than 1000ms, with the median response time (doesn’t include outliers) closer to 475ms. This is a bit less than an order of magnitude improve in performance just be turning on caching with W3 Total Cache.
So far everything looked great (with the exception of the request failures) for this UpCloud configuration with caching enabled. However if we take a look at the response time distribution you can see that all is not what it seems. 80% of requests returned in less than 600ms, but the upper 20% too anywhere between 3000ms and 12000ms. 20% of your customers waiting more than 3 seconds for the page to load isn’t awesome.
2vCPU + 4GB RAM (No Cache)
The next machine we tested was far better provisioned than the first machine with twice the CPUs and 4x the RAM.
As you can see request throughput peaked at around 50 per second, and then leveled off at 22 per second when the error rate elevated. I honestly expected the throughput to be better for this scenario given how much more hardware there was to work with.
Response times slowly increased as load increased on the server, with average response times ending up near 5000ms and the median response times closer to 5500ms. At that point, 500ms probably doesn’t matter that much.
The response time distribution for this test was predictably bad. The difference between the lower 50% and upper 50% is only 1000ms, which doesn’t give me warm fuzzy feelings. If anything, this is a great example of how much you need caching in a WordPress installation. Throwing twice as much hardware at it barely even makes a difference uncached.
2vCPU + 4GB RAM (Cached)
Now lets take a look at the same machine, but this time with caching enabled via W3 Total Cache and Memcached.
No we’re getting somewhere! Requests per second leveled out at around 600, which is actually quite high. We also didn’t start to see failures increase until around 300 per second, which is pretty amazing for a $20/month machine.
The response times for the cached version were pretty great until the failure started accumulating. Up until the failures started we were seeing response times right around the 60ms mark, which is excellent under heavy load. After that they leveled out at around 450ms, which still isn’t bad when you are serving ~600 requests/s.
The response time distribution wasn’t terrible here either considering all the failures we were seeing. 95% of requests finished in under 1000ms and 50% finished in under 500ms.
4vCPU + 8GB RAM (No Cache)
Our final test was with a fairly robust machine (by my standards anyway).
As you can see the results in an uncached situation were pretty solid until failures started in a major way. This machine was able to handle about 175 req/s uncached, and then leveled out at 215 req/s once the errors started. Obviously this number of errors isn’t great, but at the 175 req/s mark thats enough to serve 15 million requests a day.
As expected in an uncached situation the response times weren’t great, but they weren’t terrible either given the amount of load the system was under. Prior to elevated error rates we were seeing response times around 100ms. Once errors picked up they leveled off at 2500ms.
The response time distribution is what we would expect from an uncached WordPress installation under heavy load. 50% of requests finished in 2500ms and 99% of requests finished in 3250ms. That’s not all bad considering there wasn’t any caching.
4vCPU + 8GB RAM (Cached)
Our final load test was a cached version of the previous test. Given drastic improvements we saw in the other tests, the same sort of results were expected here.
Now that we have a more robust machine + caching enabled, you can see that the server was able to process quite a few requests concurrently before errors started. Even once errors started they stayed relatively low. Initial errors didn’t show up until around 500 req/s and eventually we leveled off at 850 req/s. Not bad for $40/month.
The response times were also excellent in the cached scenario. Before errors and failures we were seeing 65ms and after we were stilling coming in at around 100ms. Not too shabby for the intense load the server was under.
My favorite chart this time around is the response time distribution. 99% of all requests finished in under 300ms. With 1000 concurrent users.
UpCloud WordPress Reliability
An often overlooked metric when comparing VPS providers like UpCloud is reliability. To get some objective numbers on that, I ran a reliability test with Kernl for 30 consecutive hours. The test is low volume (25 concurrent users), but enough to make sure the server stays active and that we’ll notice is sometimes goes awry.
Over the course of 30 hours we saw one error spike that was quickly resolved. After that, no errors at all. Over the entire test we saw 400 requests fail, which is roughly 16 seconds of failures. I’ve done quite a few of these reliability tests with different hosts and this is pretty average.
UpCloud is solid choice for VPS hosting though their claim to be the “fastest” may not be completely substantiated (with WordPress at least). If I wanted to use a VPS host based in Europe they would definitely be near the top of my list. The only odd thing about them is their billing model where you add “credits” to your account versus just having a credit card of file. This is likely to prevent fraud, but doesn’t really make for a great user experience.
Hello everyone and welcome to the May 2020 edition of “What’s New with Kernl”! This month was relatively slow for us with little work on new features and more focus on bugs and refactoring. Let’s dive in.
We now have video tutorials for enabling Git deployments on your WordPress plugins and themes. You can see them on your Kernl dashboard, or in the Kernl documentation.
License Tab UX – The license tab in the plugin and theme edit interface was getting out of control. We collapsed the provider specific controls into an accordion to make it a little easier to digest.
Global CDN – Kernl now has the option to serve your updates via a global CDN powered by Vercel. If you have a geographically diverse customer base, this might be for you.
Analytics Date Selection Bug – There was an issue with date selection being wrong in some cases.
Mongoose Upgrade – Kernl used Node.js and MongoDB under the covers to deliver your updates. Our ORM is Mongoose and had been stuck on the 4.X version of it for a long time. We finally upgraded to the 5.x series and saw 40% reduction in queries to our database.
Analytics Database Maintenance – Removed 2 indexes that we didn’t need and dud a full vacuum on the largest and highest traffic table. Recovered 5Gb of hard disk space.
Kernl Blog Rebuild – Our blog is self-hosted and was running on some pretty ancient hardware/software. We rebuilt the machine to use Ubuntu 20.20 + Nginx & PHP-FPM.
Analytics Daily Aggregates Failure – For 2 days Kernl analytics daily aggregates were failing due to disk space space and memory issues (see Analytics Database Maintenance). This has been resolved.
Site Health SSL Certificate Expiration – We forgot to renew the WordPress Site Health SSL Certificate (oops!). This is now an automated process 🙂
That’s all for this month. I hope everyone has a great June!
When hosting your WordPress site on DigitalOcean, storing your static assets nearby makes a lot of sense. Since the advent of DigitalOcean Spaces in 2017 this has become a much less arduous process. In this post we’ll go through step-by-step how to host your media (and static css/js if you want) in a DigitalOcean Space.
Setting up a DigitalOcean Space
Before you get started you need to set up your DigitalOcean space. To do that, log in to DigitalOcean and click the “Create” dropdown. From there, choose the “Space” option.
Select your region, ignore the CDN (for now. You should do this later though), leave the file listing as restricted, and then pick a name. It can be anything, but something descriptive is usually best. In our case, we’re going to use kernl-blog-space.
Once your space is created, you need to make a few minor changes in it’s settings to allow for file uploading. Under CORS Configuration we need to add the domain where files are going to be uploaded from. In our case, we added:
Origin – http://wploadtest.xyz (this is my test domain)
Methods – I selected all of them, because I want the plugin that will be managing the files to be able to make all the modifications that it needs.
The other two options I left blank/default.
Finally, you’ll need access keys. To generate those, look in the left menu on DigitalOcean, go down to the “Account” section, and click “API”. Now scroll down and click “Generate New Key”, pick a name, then click the check mark. Copy the access and secret key somewhere. You’ll need it for later.
WordPress Configuration for DigitalOcean Spaces
The next step is to set up WordPress to send and server media from your DigitalOcean space. First up, you need to install the WP Offload Media Lite plugin from the WordPress.org repository.
Once installed and activated, go to Settings -> Offload Media Lite. Select DigitalOcean Spaces, choose whether to store keys in the database or not, add your keys, and then click “Next”.
The next step is selecting the region where your space is located and then copying the name of that space into the “bucket” field. Once you’ve done that click “Save Bucket Settings”.
Testing it Out
Now let’s test it out! Go to your media library and upload a few images. Once uploaded, click into the image detail view, and checkout out the source.
And just like that, we’re done! Images are now served from DigitalOcean spaces. A next step you should take it to configure to the CDN so your media is globally distributed, but this is a great start.