Kernl News Archives | Kernl.us Blog

What’s New With Kernl – May 2020

Hello everyone and welcome to the May 2020 edition of “What’s New with Kernl”! This month was relatively slow for us with little work on new features and more focus on bugs and refactoring. Let’s dive in.

We now have video tutorials for enabling Git deployments on your WordPress plugins and themes. You can see them on your Kernl dashboard, or in the Kernl documentation.
License Tab UX – The license tab in the plugin and theme edit interface was getting out of control. We collapsed the provider specific controls into an accordion to make it a little easier to digest.
Global CDN – Kernl now has the option to serve your updates via a global CDN powered by Vercel. If you have a geographically diverse customer base, this might be for you.
Analytics Date Selection Bug – There was an issue with date selection being wrong in some cases.
Mongoose Upgrade – Kernl used Node.js and MongoDB under the covers to deliver your updates. Our ORM is Mongoose and had been stuck on the 4.X version of it for a long time. We finally upgraded to the 5.x series and saw 40% reduction in queries to our database.
Analytics Database Maintenance – Removed 2 indexes that we didn’t need and dud a full vacuum on the largest and highest traffic table. Recovered 5Gb of hard disk space.
Kernl Blog Rebuild – Our blog is self-hosted and was running on some pretty ancient hardware/software. We rebuilt the machine to use Ubuntu 20.20 + Nginx & PHP-FPM.
Analytics Daily Aggregates Failure – For 2 days Kernl analytics daily aggregates were failing due to disk space space and memory issues (see Analytics Database Maintenance). This has been resolved.
Site Health SSL Certificate Expiration – We forgot to renew the WordPress Site Health SSL Certificate (oops!). This is now an automated process 🙂

That’s all for this month. I hope everyone has a great June!

What’s New With Kernl – April 2020

Kernl had another great month, with some bug fixes, blog entries, and a few new features.

Most importantly we’re announcing a limited beta of our new global update CDN! With our new global CDN powered by Vercel, every Kernl update request is cached at edge nodes around the world. This means <= 50ms response times for updates globally.

Reach out to jack@kernl.us if you’d like to give it a try!

Features, Bugs, and Performance

Docs – Added more descriptions to the “Understanding Graphs” portion of the Kernl documentation.
Analytics Date Selection – The flow for selecting dates and products in analytics has changed. You can now only select dates in the range that the selected product has data.
Examples – The Kernl API Examples repo has been updated to include a license management example.
WHMCS License Validation – Kernl now supports WHMCS license validation for plugins and themes. This integration works similarly to our other 3rd party license validation integrations.
Performance – We added a few database indexes to our load testing database. In situations where a lot of load tests existed performance was abysmal.
Tutorial – On the main dashboard page of Kernl there is now video tutorials for getting started with plugin and/or theme updates.
License Expiration Date Bug – There was a bug in Kernl license management where date editing didn’t work correctly when the saved date started with a leading zero.
License Domain Restriction Bug – An edge case existed where license validation would fail with a server error if domain restrictions are set but no domain was passed in to be validated.
Download Increment Bug – Incrementing download count was being handled in a very naive way. It was originally developed when Kernl wasn’t expected to reach the scale it has. This has been resolved by using the $inc operator in MongoDB instead of a find -> increment -> upsert.
Load Test Infrastructure Bar – The progress bar when you create a load test didn’t have a very good default state (it looked like nothing was happening). Now when below 3% complete, the bar will show 3% so you can actually see that things are progressing.
Site Health Edit – You can now edit your site health entries.
Envato License Validation Bug – Due to the deprecation of an API endpoint we were using, users of our Envato license validation need to regenerate their Envato access token that is saved with Kernl.
Envato License Tied to ID – You can now verify that an Envato license that is being used is tied to a specific plugin or theme.

Blog Posts

What’s New With Kernl – March 2020

With all the uncertainty in the world right now due to COVID-19, we are intentionally keeping things boring at Kernl. Bug fixes, new docs, and increased capacity are all we did this month.

Stay safe out there everyone!

Features, Bugs, Updates

NEW DOCUMENTATION! – After 5 years we finally moved to a real documentation generator! In the process of moving docs to the new system, all of the docs were updated and checked for accuracy. Check it out at https://docs.kernl.us.
Site Health – You can now change you wp-json root. Some people change this for security reasons. Site Health needs it for some of the more advanced functionality.
Site Health Timezone Bugs – Timezones are hard and our Site Health service wasn’t using them correctly.
Analytics VM Changes – The Kernl Analytics service was starting to get bogged down with increased traffic. It now has 2 vCPUs and 2GB of RAM (up from 1 vCPU and 1GB).
Analytics Bug – We were logging analytics data for customers that aren’t paying for it. This is no longer the case.
Plugin/Theme Git Status – The plugin and theme list pages were displaying inconsistent results for the Git status of plugins/themes. There was an error in our Mongo query that populated this data which has been resolved.
Load Testing Remain Resources – The remaining resources page can take a while to load sometimes because we have to call out to external services. An indeterminate loading spinner was added to let customers know that it’s still working.
System Updates – All operating system level packages have been upgraded to their latest available version.

new documentation by docsify — New documentation by Docsify

What’s New With Kernl – February 2020

I hope everyone had a great February! We didn’t too much feature development this month, but there was a lot of bug fixing and performance improvements, so let’s dive in!

Feature, Bugs, and Performance

Node.js – Kernl is now on Node.js 12.16.1. This release was all about security fixes.
Load Testing Machine Provisioning – We weren’t calculating the correct number of machines to provision on DigitalOcean. This lead to some serious over-provisioning when running load tests. This has been resolved, which means more customers can run more load tests at the same time.
Load Testing Secondary Node Behavior – Kernl uses Locust under the covers to run our WordPress Load Testing service. The Locust primary node has an argument called “–expect-slaves”. It tells Locust “Don’t start the load test until at this this many secondary nodes have connected.”. We weren’t calculating this number correctly which led to some weird behavior. This is now resolved so load tests should start in every situation now.
Easy Digital Downloads Domains – Kernl wasn’t passing the domain along to EDD. We now do this, which allows you to restrict updates to specific domains while using EDD.
Load Testing Snapshots – Kernl used to build up each load testing machine from the ground up every time a load test was started. We now start from a snapshot that gets us 50% of the way there. This has improved load test start times (especially on large tests) by an average of 30%.
GitHub Authorization Changes – The GitHub API is changing how it handles authorization headers. We’ve update Kernl to handle this change, so we’ll be good going forward when GitHub deprecates the old method.
High Traffic Endpoint Audit – We did an audit of our high-traffic API endpoints and cleaned some things up. Slight performance improvements were had (1%-2%), but mostly the improvements have been in code readability and comprehension.
GitLab Deployment Issues – In a recent release of GitLab they changed the required fields when asking for an access token via a refresh token. This broke all GitLab deployments for Kernl for a few days while we tracked down the issue. This has since been resolved.
Load Testing Unit & Integration Tests – When our load testing service was launched we weren’t sure if it was going to be successful. We’ve proven that it is a worthwhile feature, so now we’re focusing on reliability. We’re in the process of adding a suite of unit and integration tests around this functionality.

That’s it for this month! See you in March.

What’s New With Kernl – January 2020

January was a pretty great month for Kernl. We got a lot of bug fixes, some performance improvements, and even a new beta feature out the door. Let’s dive in!

WordPress Site Health Beta

This month we released a beta of our WordPress Site Health service. The goal of this service is to help you determine where performance problems are and why they are happening. The dashboard below gives you a high level view of how performance looks on each of your sites.

Once you click in to any site you see data for the last 7 days.

The data here helps you diagnose performance issues. Using the plugin changes panel you can tie performance issues back to adding/removing/updating of plugins. Google Lighthouse scores show trends on you site’s performance, usability, SEO and more over time.

If you’d like the join the beta, send an email to jack@kernl.us.

Bugs & Performance

Improved performance of plugin/theme list pages – The API calls to /api/v1/plugins and /api/v1/themes were incredibly inefficient. They hadn’t really been touched since Kernl’s inception and needed some love. After doing some query optimization and stripping out unnecessary data, the payload and response time were reduced by a factor of 10. The worst case example was response size going from ~750KB to ~75KB and response time from ~4000ms to ~250ms.
Server resource increases – Kernl’s main Node.js application servers have been running with 1 vCPU and 1GB of RAM for about the last 2 years. Lately we’ve seen some resource exhaustion and decided it was time to upgrade. The new app servers now run with 2 vCPUs and 2GB of RAM.
Server disk space / inode exhaustion – The process of building plugins and themes can use up a lot of space and file system resources. We weren’t doing a great job of cleaning those resources up periodically which could cause some performance issue. We now clean up all temporary files once a day which should prevent this from happening anymore.
Easy Digital Download License Validation Error – There was a bug in EDD license validation where the source system wouldn’t send back valid JSON. This would break license validation instead of handling the error gracefully.
Profile page autocomplete – If you had form auto-completion on it would sometimes cause the profile page to reset your password. We’ve disabled autocomplete on this form to resolve the issue.
Theme tiles not showing correct build status – During the course of our performance improvement work we noticed that theme tiles were not showing the correct build status. This has now been resolved.

What’s New With Kernl – December 2019

Welcome to the end of 2019! I hope that everyone has had as good a year as Kernl. Let’s dive in to the final update of 2019 to see what’s new.

Features, Bug Fixes, & Misc.

Improved License Management Search – License management now includes improved search functionality. The previous search functionality was flaky (at best) and not very discoverable. Search is now a first-class citizen, includes free-text search, and should greatly improve the overall usability of Kernl’s WordPress license management.
Load Testing Unit & Integration Tests – When we created the WordPress load testing service it was an experiment. Now that we have proved the viability of the service it’s time to work on stability and overall platform longevity. This month’s focus was on the authorization framework that our load testing service uses.
JS Bundle Size Reduction – Over the past year Kernl’s JS bundle size for our web app grew to over 2MB. We spent some time this month figuring out why and making changes to reduce it. In the end we were able to reduce the bundle size by over 50% down to 1.1MB.
Bug: Inconsistent Webhook & Deploy Key Behavior – After a few customer reported incidents with the automatic webhook and deploy key behavior, we discovered that Kernl wasn’t deleting local references to remote keys and hooks. If you have some issues with deploy keys or webhooks please contact us and we can help resolve the data inconsistency issues that this caused.
Node.js Upgrade to 12.14.0 – This month we upgrade all of our servers to use the latest LTS version of Node. This is includes stability and performance improvements.

That’s it for this year. See everyone in 2020!

What’s New With Kernl – November 2019

November was a great month for Kernl! After several years of trying, we finally launched team management and also made our pricing structure easier to understand. Let’s dive in!

Features

Team Management – With our new Agency and Unlimited plans you can now grant users access to your account! The Agency plan allows for 3 team members and unlimited allows for unlimited team members.
Agency & Unlimited Plans – We introduced an unlimited plan for Kernl which has no usage limits (with the exception of load testing, but those limits have been increased substantially) and includes team management and Kernl Analytics. We also updated our agency plan to more closely match the old enterprise plan. The new agency plan has much higher limits than the previous agency plan as well as access to team management.

Bug Fixes & Other

Increased Load Test Generator VM Size – We’ve increased the load test generator machines from 1vCPU to 2vCPUs to allow us to scale our load testing up to 50,000 concurrent users.
Repository Sync – When we changed how you connected to your Git repositories we overlooked the ability to sync them from that same location. We’ve added that ability back in.
Load Test List Page Performance – With some clever SQL querying and awesome Postgres built-in functions, we’ve decreased the average load time on this page by 32%.
Bug: Invalid license if using licenses but no versions present – An odd edge case was found where Kernl would say your license was invalid if you didn’t have any plugin/theme versions uploaded. This has been resolved.
Bug: IPv6 Issue on Load Generators – The virtual machines that we spin up the Digital Ocean Singapore data center were having issues communicating over IPv6. We disabled IPv6 for all load generators for the time being.
Load Testing Service Node.js Upgrade – The WordPress load testing service backend and workers have been upgraded from Node.js 10.x to Node.js 12.x. All packages were upgraded to their latest with this change.
Analytics Service Node.js Upgrade – The WordPress analytics service backend was upgraded from Node.js 10.x to Node.js 12.x. All packages were upgraded to their latest with this change.

Blog Posts

That’s it for this month! Have a great December!

Introducing Team Management and the Kernl Unlimited Plan

One of the most requested features that Kernl gets is Team Management. Team management can take on a lot of different forms, but for Kernl’s purposes it is the ability to share access via the Kernl interface with multiple people with different sets of credentials.

Team Management

Starting today you can sign up for the Kernl Unlimited plan (see below) and start managing your team. Team management allows you to share your Kernl account with authorized users. When a user is part of your team, they have a slightly restricted view of Kernl but otherwise see the same things you do. What’s restricted?

Continuous Deployment – Adding connected Git repositories is the responsibility of the admin. Non-admin users don’t need to see this information.
Billing – Once again, this is an admin responsibility. The admin is able to change plans, add credit cards, etc.
Team Management – Only the admin can add and remove users from their account.

Aside from that everything is the same for admin and non-admin users.

The Unlimited Plan & Pricing Changes

In addition to team management we are also reducing the number of plans that Kernl has. Prior to this change we had 5 different plans (solo, agency, enterprise, huge, massive) which seemed like a bit much. To simplify Kernl’s pricing structure and give our customers better options we went down to 3 plans:

Solo – This $9 per month plan is our base plan. Most customers start here and then grow into larger plans as their needs change.
Agency – The $39 agency plan is essentially our former “Enterprise” plan but with increased limits for plugins, themes, licenses, load testing, and feature flags.
Unlimited – This new $79 unlimited plan allows you to have unlimited plugins, themes, licenses, feature flags and very high limited on load testing (50K concurrent users). This plan also includes team management so you can more easily control access to your Kernl account.

As always, current Kernl customers are grandfathered into the plan that they are currently on.

What’s Next?

Now that Kernl has a more robust upper-level offering we hope to start rolling more features into it. For example:

In the near future we’ll include Kernl Analytics with Unlimited plan.
We’re also planning on rolling out limited team management into the Agency plan.

What’s New With Kernl – October 2019

October held a lot of great work for Kernl! Lots of refactoring, test coverage increases, and a new UI for managing Git deployments. Let’s dive in!

Features

Git Continuous Deployment UI Changes – Kernl now has a new and improved interface for adding continuous deployment to your projects.
Installed Plugin List Filter – You can now easily filter the installed plugin data in Kernl Analytics. Prior to this change the best you could do is column sort.

Infrastructure, Bugs, & Other

Fetching BitBucket repos with more that 20 branches failed. This has been resolved.
The plugin/theme webhook build log didn’t handle the case where a user had just signed up and didn’t have any build requests yet.
Easy Digital Downloads license validation was failing on newer versions of EDD. Kernl has been updated to support the newer version API as well as the older API.
The Git integration has been refactored and had additional test coverage added.
Kernl was upgraded to Node 12.13.x. With this upgrade we get async stack traces and a non-trivial decrease in response time (~50ms).

Blog Posts

What’s New With Kernl – September 2019

I hope that everyone has had a great September! There has been some interesting changes with Kernl this month, so let’s dive in and discuss them.

Features

Credit Card Add Changes – For the past 3 years Kernl has had a simple credit card form (card number, expiration, etc). In an effort to help reduce fraud we’ve migrated out card addition system to use Stripe’s Checkout.js. This gives us advanced fraud protection provided by Stripe and the added benefit that your card details never hit Kernl’s servers.
Continuous Deployment Setup – Kernl is in the middle of a big UI change for continuous deployment setup. The first step was making the ‘connect your Gitlab/GitHub/BitBucket account’ piece a lot prettier. We’ve now made it a 3 panel selection, with nice big logos and fewer ways to get confused. Check it out by going to the “Continuous Deployment” section in Kernl.
Repository Pruning – Kernl will now prune repositories from your repository list that you no longer have access to or no longer have connected with Kernl. The exception here is that we’ll keep a reference to repositories that you have connected to plugins or themes.
Invoice Payment Failure Notifications – We recently had an issue where a customer wasn’t notified that their payment failed (card had expired) 🙁 This should be resolved now as we’ve integrated with Stripe web hooks and immediately catch the event and send a message to the account owner. We’ve also added a notification preference so that you can silence these emails.

Other

All packages have been upgraded on all Kernl servers.
Marketing pages are now cached in Redis.
Fixed a customer reported bug in plugin_update_check.php that threw a warning in newer versions of PHP. Upgrade to version 1.2.2 to get the fix.
Feature flag setup wizard had weird behavior if the customer didn’t have any plugins or themes. You can now manually name your feature flag product in the wizard if you so choose.
Fixed a few bugs in the feature flag UI where adding/removing individual users from a flag would occasionally fail.
The buttons to manually manage deploy keys have been moved to the bottom of the continuous deployment page. They also come with a disclaimer now.

Blog Posts

Beta testing WordPress plugin features with Kernl WordPress Feature Flags