Multiple High Priority Security Vulnerabilities In OpenLiteSpeed

This morning we woke up to a security disclosure report in our inbox which detailed multiple high priority security issues in OpenLiteSpeed.

A brief overview of the issues are described in an article on thehackernews.com. More detailed information about them are on the paloaltonetworks website.

The vulnerabilities were disclosed and patched in October.

So, naturally, the first thing we did was check our older OLS servers to make sure the patches were applied.

Ummm…no they weren’t.

Trying to figure out why security patches that have been available for weeks were not applied took us down a rabbit hole.

What we found does not speak well of the OpenLiteSpeed development team.

Our Discoveries

The first thing we noticed is that the OLS repo did not tag the release containing the fixes as a security update.

So, the LINUX automatic security update process that would normally kick in did not apply the patches.

Drilling further, we looked at their releases page on github. There, we found a single release with the patches – 1.7.16.

There’s just one major problem with this – that was the same release as the May release.

Instead of creating a full new release, the top 3 levels of the version number remained the same AND the releases page did not show two different releases (eg: 1.7.16.1)

The only way to recognize that something was wonky was to compare the official release date and the date that github tagged the assets:

The only explanation for this is that the team tried to down-play the issues by hiding them in an existing release.

In fact, if you look at the version on an OLS server prior to October and a version that is installed today, OLS will report the same version number with different build dates.

Servers Installed Prior to October

Servers Installed Today

The SAME version number for old releases and new releases.

No Excuse

There is no excuse for this.

There is an established versioning and release process for security releases that most of the industry follow. This is so far outside the norm as to be embarassing.

As much as we would like to give the OLS team the benefit of the doubt and believe that there is a good reason that they did this, we’re hard-pressed to do so.

Because of this, there are likely thousands of OLS servers out there that are not updated and remain vulnerable. Many admins depend on the Linux background updates to run security updates. They will be blindsided by this.

(Keep in mind there are about 1.9 MILLION OpenLiteSpeed servers according to thehackernews.com)

A Rant

We’ve spent most of the latter half of this year working on adding support for OpenLiteSpeed to WPCloudDeploy.

To say that it has been a painful process is an understatement.

After working with NGINX for so long, we were constantly surprised at the edge cases we had to work around and the lack of consistency in configs and other aspects of OLS.

Now, given how this security update was handled, we cannot recommend OLS for mission-critical production applications.

For many companies, a web server vulnerability can easily grant hackers a doorway into the rest of the org.

Until we can rely on good disclosure and behavior that meets the minimum expected industry standard we really cannot recommend the webserver to anyone (regardless of if they’re using WPCloudDeploy or not) for mission critical applications. (Though, maybe, the ENTERPRISE version has better disclosure and support?)

(Update 10-14-2022: Lightspeed eventually published a post on their blog a few days after this article was published: Full Disclosure of CVE-2022-0072, CVE-2022-0073, and CVE-2022-0074 and What You Should Do. However, an email has still not been sent to customers.)

WPCD OLS Support

We will continue to build out our OLS support because it’s the number one feature that our customers are asking for.

But if someone were to ask us, point blank, which web server we recommend, it would be NGINX hands down.

It doesn’t matter if OLS is ‘faster’ or might have a better WordPress cache or great CDN integration. Security trumps all that.

We will be releasing WPCD 5.0 shortly but will continue to mark OLS as beta because of these types of issues.

How To Fix Your Servers

Ok, now that you’ve read all that, you probably want to know how to fix your servers. It’s actually pretty simple.

There are two ways to update:

• Run the following from the command line:

apt-get upgrade openlitespeed

• Or you can run all Linux updates – this you can do from the WPCD UPGRADES tab for a server.

We recommend you use the command line option though because if a non-security Linux update requires user input and you run it from inside WPCD then the upgrade will fail.

Backup your servers first, of course!

Wrapup

If it turns out that the OLS folks had a really really good reason for rolling out their security updates this way we’ll update this article and maybe retract some of our stronger statements above. We’re definitely a little salty about the amount of work we had to do to figure out all this – it really should have been a standard straight-forward security update. So, for now, we’ll be very wary of using OLS for our own mission critical applications.