1. Cyber-Physical Attack Recovery Procedures
Many industrial facilities can have downtime costs of more than one million dollars per day. Equipment replacement times can easily be months. SCADA (Supervisory Control and Data Acquisition ) systems, Building Control System (BCS), Industrial Control Systems (ICS) , and Utility Control Systems (UCS) are extremely complex systems that require engineers and maintenance personnel to learn a new language. Its not enough anymore to simply know what a centrifugal chiller does and how a boiler works. Today, building maintenance personnel also need to know how hackers can attack critical equipment and damage critical infrastructure. Why? Because most building equipment today is computer-controlled and SCADA protocols are insecure by design. That means:
Control systems protocols have little or no security safeguards
Migration to TCP/IP networks with its inherent vulnerabilities
Increased demand for remote diagnostics and maintenance access
Interconnection with enterprise networks
Legacy operating systems and applications with poor patching procedures
Little monitoring of control systems for evidence of hacker presence
Vendors not securing their product offerings adequately
The majority of PLCs are ordered with web services enabled and most users leave unused web servers active (not configured), with factory default passwords
Increased interest in control systems by foreign governments and terrorists
Evidence that nation-states have already penetrated control systems
Legacy industrial controls do not have the chip sets and processing capability to authenticate commands or identify malware
Historically, ordinary cyber-attacks were often undetectable and untraceable. A major cyber-physical attack , on the other hand is difficult to hide because building equipment is failing in real time. This is what happens if you want all the benefits of being connected. This book focuses on the damage hackers that can do to building equipment and how they can really ruin your day. I describe potential ways to stop a cyber-physical attack, and then how to restore equipment operation without doing further damage.
Most companies cannot function without a physical presencebe it an office or a manufacturing facility. Large companies tend to link their facilities together in the cloud, not realizing that an attack on one facility can quickly spread to many others. The Repository of Industrial Security Incidents (RISI) claims that the number of incidents directly affecting SCADA controls systems (accidental and malicious) has increased by 20 percent over the last ten years.
In 2005, the Zotob worm attacked 175 companies, including Caterpillar, General Electric, UPS, and Chrysler. At Chrysler, the Zotob worm shut down 13 assembly lines, idling 50,000 employees. Chrysler had professionally installed firewalls between the Internet and the companys network, but the worm was able to travel from plant to plant in seconds. Poor systems design and a failure to contain communications are other key problems. For example, upgrading the business system at the Hatch nuclear reactor zeroed out the control system database, causing the reactor safety system to interpret this as a drop in water cooling levels and triggering a shutdown that took two days to restore.
If your facilities controls systems are not segregated, its only a matter of time before they all go down. Hackers only need to find vulnerabilities at one site to take them all down. It goes without saying that it is a crime to commit a cyber-physical attack, so preservation of evidence is important, but in my world (facilities), it is more important to restore normal building operations quickly.
Purpose of the Recovery Procedures
The purpose of the Cyber-Physical Attack Recovery Procedures (hereinafter referred to as the Recovery Procedures) is to stop a cyber-physical attack and restore mission-critical processes (and the controls system) when an attack causes equipment to fail. I assume that you did not take my advice and your BCS is still connected to the Internet; it has modems, embedded web servers, and wireless routers; and vendors happily log in remotely to equipment (that I recommended you secure) or they regularly bring personal laptops to your mechanical rooms and plug into your BCS with little or no supervision. This book presents examples of cyber-physical attacks that hackers will attempt against your building equipment, so that your maintenance personnel understand the vulnerabilities and are prepared to respond to a cyber-physical incident.
Embedded web server
Web server software built into a building control system device that is provided to configure the device from a web browser. Also used remotely by equipment vendors to update software or troubleshoot problems.
This is an important point because when you ask a maintenance engineer if it is possible to disable the safety controls on a steam boiler so the fuel doesnt cut off automatically when the flame goes out or when the feedwater is stopped, the reply is usually: Sure, but why would you want to?
The Recovery Procedures Template in the Appendix contains generic forms to be customized for your specific building controls. They serve as the central repository for the information, tasks, and procedures that would be necessary to restore critical equipment functions. Having this information readily at hand is especially important because a cyber-physical attack on a building control system is engineered such that a prompt resumption of operations cannot be accomplished by employing normal daily operating procedures. This is not business as usual.
When a cyber-physical attack strikes, you wont be able to simply log on to a vendor website and download the latest drivers. Vendor websites will experience denial of service attacks to prevent you and others like you from recovering after the attacks. If you dont have hard copy of the network diagrams, equipment operation manuals, rescue CD-ROMs and original vendor software, you will be out of luck.
Why do I need Cyber-Physical Attack Recovery Procedures? Cant we just disconnect the Internet and turn the equipment back on?
First of all, when a cyber-physical attack is verified, I recommend you physically pull the computer's power plug from the wall rather than gracefully shutting down. Forensic data can be destroyed if the operating system executes a normal shutdown process. Also, avoid running any antivirus software after the fact as the antivirus scan changes critical file dates and impedes discovery and analysis of suspected malicious files and timelines. Secondly, a well-designed and coordinated cyber-physical attack will include cyber booby-traps intended to continue to cause damage when maintenance personnel attempt to restore normal equipment operation.
Cyber booby-trap
When a hacker embeds malware that is triggered by actions of the building maintenance staff. For example, the initial indication of a cyber-attack may be that the hacker turned off the water to a boiler. The maintenance personnel in the control room are unaware that the malware pumped all the water out of the boiler and turned up the heat. Once the boiler is superheated, the action of turning on the water triggers an explosion. The hacker needed the triggering action by building maintenance personnel to maximize the damage.