Just me, e. ELyssaD™: SCADA Security: Welcome to the Patching Treadmill

As regular readers of this blog know, after Stuxnet, security researchers and hackers on the prowl for new targets to exploit shifted their efforts to critical industrial infrastructure.

Unfortunately, the Supervisory Control and Data Acquisition (SCADA) and Industrial Control Systems (ICS) applications they are now focusing on are sitting ducks.

Up until recently SCADA and ICS systems have been designed with reliability and safety in mind; security has been a minor consideration. Products that have never faced security tests are now under attack from sophisticated vulnerability discovery tools, and major control system security flaws are being continuously exposed.

SCADA/ICS applications are easy targets for security researchers and hackers.

Image Credit: Active Rain Caption

In recent years, we have seen a staggering growth in government security alerts for these systems, and have witnessed some of the most sophisticated cyber-attacks on record.

The US government’s ICS-Computer Emergency Response Team (ICS-CERT) tracks and publishes Security Advisories for known security vulnerabilities found in industrial products. In the entire decade prior to the discovery of Stuxnet (July 2010), ICS-CERT published 5 security advisories involving 3 vendors.

Compare this to 2011, when there were:

• 215 publicly disclosed vulnerabilities

• 104 security advisories

• 39 vendors involved in those security advisories

By late 2012, the total publically disclosed vulnerabilities topped 569.

And remember that these vulnerabilities are typically disclosed to the world prior to ICS vendors having patches available.

Furthermore, 40% of disclosed vulnerabilities include working attack code. This means that individuals can download exploit tools and run them against a target with little understanding of control systems or the consequences of their actions. And download and attack they do - ICS-CERT reported over 20,000 reports of unauthorized internet access to control systems in the last half of 2012.

Welcome to the Industrial Security Patching Treadmill!

Since security researchers, hackers, and issues have essentially migrated from the IT environment, it’s not surprising that we look to that world for a solution. There, security vulnerabilities are addressed by applying software patches - a constant cycle involving multiple patches over the life of a product. In fact, the typical IT computer needs patching (with a full reboot) at least once per week.

You don’t need to be a SCADA/ICS expert to realize that shutdowns of that frequency just aren’t feasible for critical infrastructure control systems.

On and on and on it goes... Image Credit: Learning to Fly

So How Many Patches Does a Control System Need?

One might argue that a control system requires fewer patches than an IT system…or that the software footprint is smaller, the code quality better… If so, then maybe the patching cycle could be synchronized with annual maintenance shutdowns. In this scenario, patching could be a workable solution to address software vulnerabilities.

To determine if this was an option, in 2008 I participated in the analysis of a U.S. refinery process control network (PCN). There were 85 computers on the refinery PCN, and a similar number of industrial controllers. Although we could only gather reliable data for 78 of the computers, we determined that they were running 272 distinct processes/applications.

A search of the National Vulnerability Database (NVD) found that 48 of these processes had one or more serious security vulnerabilities. Across the refinery PCN, there were 5,455 publically known vulnerabilities, an average of 70 per machine. An aggressive Windows OS patch program reduced this number by almost 50%, but that still left 2,284 published vulnerabilities remaining. Why? Because the applications involved did not have a means of automated patching.

Latent PLC Vulnerabilities Are Not All Disclosed at Once

And let’s not forget the latent vulnerabilities lurking in the control system. Academic research tells us that most commercial software contains 3 - 10 defects for every thousand lines of code (KLOC), and that 1% to 5% of these result in vulnerabilities. That works out to between 0.03 and 0.5 vulnerabilities per KLOC.

So what does that mean in real life?

Take Windows XP, for example…it contains about 40 million lines of code (40,000 KLOC). As of October 2012, about 1106 moderate or severe vulnerabilities have been listed in the NVD for Windows XP - that’s a Vulnerability/KLOC ratio of about 0.0276.

Whatever your experience using Windows XP, seems it’s on the low end of vulnerabilities and pretty good from a security point of view!

Looking back at the history of Windows XP vulnerabilities, we have to assume that SCADA/ICS vulnerabilities won’t all be disclosed at one time. We’ll likely see a relatively small number of disclosures in the first few years, as researchers begin to investigate the products in the industrial space. Then, after SCADA/ICS products have been exposed to widespread security scrutiny, a virtual avalanche of vulnerabilities may occur, resulting in the need to install control system patches on a weekly basis.

Prepare for an avalanche of vulnerabilities once SCADA/ICS products have been exposed to security scrutiny. Image Credit: The Alaska Avalanche Information Center

We Can’t Ignore SCADA/ICS Firmware

It’s also obvious that the firmware in PLC and DCS controllers will also have vulnerabilities and will require patching. Controllers typically contain between 1,000 KLOC and 5,000 KLOC of firmware. Based on the analysis used above, this means that each is likely to contain between 30 and 150 vulnerabilities. If the vulnerability disclosure curves are similar to those we’ve seen in the IT sector, we can expect a low number of patches in the immediate future, followed by an epidemic in a few years.

The above analysis clearly indicates that the frequency of patching needed to address future SCADA/ICS vulnerabilities in both controllers and computers is likely to exceed the tolerance of most SCADA/ICS operators for system shutdowns.

Tune in to next week’s blog and learn about the impact of patches, what happens when there are no patches, and why many SCADA/ICS customers simply don’t want to patch...

Do you think that patching is a workable solution for securing SCADA/ICS control systems? Do you have any patching success or horror stories to share? Let me know your thoughts.

Related Content to Download

Presentation - "Patching for Control Systems - A Broken Model?"

Download this presentation and learn about:

•    The challenges of patching for control systems

•    Vendor data on patching deployment rates on ICS products and what can be achieved in the    future

•    Compensating control-based solutions for security vulnerabilities

This document is vendor neutral and is ideal for serious consideration of the topic.

Related Links

•    Press Release: Belden Research Shows that Patching for Industrial Cyber Security is a Broken Model

•    ICS-CERT.US-CERT.gov, Webpage: The Industrial Control Systems Cyber Emergency Response Team
•    Automation.com, Webpage: Cyber Attacks on Industrial Systems Increasing Rapidly
•    National Vulnerability Database (NVD), Webpage: Database search page
•    Blog: SCADA Security Basics: Why are PLCS so Insecure?
•    Blog: S4 Security Symposium Takeaway: Time for a Revolution
•    Blog: Tofino provides an Alternative to Patching