Vulnerability Inheritance in PLCs

K. Reid WightmanDirector, Digital Bond Labs

About Digital Bond Labs

• New division of Digital Bond• Focused on working with vendors

– Finding new vulns in products– Fixing bugs before they’re an issue– We find the kinds of bugs shown here

• I am biased– Strong advocate of Red Teaming products– …but honestly, hire any outside team (or build

your own)

Outline

• Vulnerabilities: What are they?• Inheritance - 3rd party libraries• Specific example – Deep Dive• Affected Japanese Vendor #1• Affected Japanese Vendor #2• Conclusion

Vulnerability

• Any mechanism which allows unauthorized changes to configuration or control

• Includes traditional bugs– Authentication bypass– Parsing error

• Includes ‘insecure by design’

Outline

• Vulnerabilities: What are they?• 3rd party libraries• Specific example – Deep Dive• Affected Japanese Vendor #1• Affected Japanese Vendor #2• Conclusion

3rd Party Libraries

• Any OS (vxWorks, GNU/Linux, Windows CE)

• Common embedded servers (e.g. GoAhead Webserver)

• CoDeSys

3rd Party Libraries

• Ownership is the issue• You ‘own’ any problems with 3rd party

software– If Vendor software depends upon library, end

user must have Vendor assurance that patch will work

– For embedded, end user needs Vendor to produce patch

• Ex: Windows CE may have bugs– Will vendor keep track of the bugs?– Will vendor produce a patch to firmware?– Will vendor inform customers of the patch?

Outline

• Vulnerabilities: What are they?• 3rd party libraries• ICS-Specific example – Deep Dive• Affected Japanese Vendor #1• Affected Japanese Vendor #2• Conclusion

CoDeSys

• Programmable Logic Controller (PLC) Runtime and Engineering Software

• Used by hundreds of ICS vendors• Primarily used in Europe, though some

inroads into USA, Japan

CoDeSys Basic Components

• Engineering Workstation (EWS) Software– Common GUI for programming PLCs– Supports LAD, STL, FBD, etc.– Communicates with PLC via plugins

• PLC Ladder Logic Runtime– Runs on vxWorks, WinCE, Linux, Nucleus, and

others– Executes ladder logic– Service for sending status and receiving

commands• OPC Server• Gateway (Optional) between EWS and PLC

CoDeSys Deep Dive

• CoDeSys V2 protocol analysis and tools released in 2012

• CoDeSys V3 protocol analysis done by DB Labs September 2014, internal tools developed – The protocol changed a bit from V2 (more

complex)– Identical vulnerabilities as V2

CoDeSys V2 – Flaws

• Unauthorized ladder logic upload• ‘Secret’ command line• Start/Stop process control• Directory traversal• Upload/Download arbitrary files• Rootkit deployment possible• Many systems internet-connected (see

joint paper with Eireann Leverett)

CoDeSys V3 – Flaws

• Unauthorized ladder logic upload• ‘Secret’ command line• Start/Stop process control• Directory traversal quite likely• Upload/Download arbitrary files• Rootkit deployment possible• Internet scan underway

CoDeSys Flaws Summary

• No Security• Original vendor (3S-Software) may have

assumed that no-one would learn protocol• Original vendor (3S-Software) likely never

had a Red Team look at components

Red Team

• If Internal, best if they are dedicated to security work (not the developers)

• Focus on looking for design problems early in development

• Focus on looking for implementation problems as development progresses

• Red Team should have spotted CoDeSys problems

• Likely that affected vendors have no Red Team

CoDeSys V2

• Simple Protocol: Start bytes (usually 0xbbbb or 0xcccc), length field, function code, arguments

• Uses TCP/1200 (some systems use TCP/2455 or custom port)

• Determined how protocol works through protocol analysis (quite easy)

CoDeSys V2

• 3S Software included “password protection” in V2– Password protection was not enforced by PLC

initially– 3S Software provided patch for PLC to fix that– To date, 0 vendors have applied patch to PLCs

• “PLC-enforced password protection interferes with 3S OPC Server” (heard from anonymous friend)

• ‘Patching’ insecure-by-design is too late in Development Lifecycle – Requires changing too much other software

CoDeSys V2 Lessons to Learn

• Apply Security Development Lifecycle to products– Have Red Team review components– Look at the design early and the

implementation often– Decide: Will we live with these vulnerabilities,

will we fix them, or will we stop using the component?

– Determine how you will monitor and patch future problems

• Catching this problem early saved the two vendors who fixed it a lot of trouble

CoDeSys V2 Lessons to Learn

• Component DESIGN problems are contamination– One component’s insecure-by-design causes

add-ons to expect that behavior– Now instead of one fix, you need at least two– The problem gets worse as more components

communicate• Fixing a design problem late is often

impossibleGreat example: Modbus

• It could have security added via reserved function code

• …but then every system on Earth would have to be changed

CoDeSys V3

• Protocol differs from v2• Can use UDP or TCP

– UDP limits attack surface somewhat, although we can get around this

– TCP has not been thoroughly tested on live PLCs, only our simulator…

CoDeSys V3 Notes

• No documentation on CoDeSys protocols is publicly available

• Example presented is based entirely on reverse engineering and protocol analysis

• Our terminology may not match 3S Software’s (debugging symbols removed from binaries, mostly)

• UDP/1740, TCP/11740

CoDeSys V3 First Packet M->Sc5 73 40 40 00 11 00 e6 00 1e 00 00 c3 00 01 016a d5 79 f4 e1 17 9d 84 00 40 1f 00 04 00 00 00

• Destination Address, Last Octet– E.g. 192.168.63.230, 230 == 0xE6

• Source Address, Last Octet– E.g. 192.168.63.30, 30 == 0x1E

• CRC– CRC is calculated based on 0xC3 (byte 13)

onward– CRC algorithm is selected via protocol (more on

this)• Random Nonce

CoDeSys V3 First Reply S->Mc5 f3 40 40 00 11 00 1e 00 e6 00 00 83 00 01 01 51 a7 eb 4c e1 17 9d 84 00 00 00 2c 94 24 00 00 01 04 62 de

• Session ID (2 bytes)

CoDeSys V3 Session Packet M->S

c5 73 40 40 00 11 00 e6 00 1e 00 00 01 81 00 2c37 02 00 00 fd 01 00 00 1c 00 00 00 97 fa b3 5ecd 55 00 0c 00 02 00 10 00 00 1a c5 00 00 00 0c81 01 88 00 11 84 80 00 00 00 45 f9• Session Identifier, 2 bytes• Master Sequence number

– Little Endian Unsigned Int– Starts with 1, increments with each request from master

• Slave Sequence number– Little Endian, increments with each response from slave

• CRC

…So CoDeSys V3 Is Vulnerable!

• ICS-CERT gets it wrong

• Anonymous friend: “ICS-CERT probably got this information from 3S-Software. They don’t verify vendor claims.”

…So CoDeSys V3 Is Vulnerable!

• 3S-Software gets it wrong

• Bad Info (‘affects V2’)• Only OEMs can download patch

CoDeSys V3 Lessons

• 3rd party library vendor analysis– Most will say, “We have no problem”– Ask to see 3rd party vendor’s SDL

documentation– “Trust But Verify” – Bring in Red Team and test

yourself

Outline

• Vulnerabilities: What are they?• 3rd party libraries• Specific example – Deep Dive• Affected Japanese Vendor #1• Affected Japanese Vendor #2• Conclusion

Vulnerable Vendor Highlight

Hitachi EHV+ Line– CoDeSys V3

Hitachi EHV+ Line: CoDeSys V3

• EHV-CPU 1025 tested• UDP only*• Command line disabled*

* These features might be enabled via updating config.cfg file on the PLC. No success yet.

Hitachi EHV+ Line: What They Did Right

Hitachi deserves some credit– EHV+ has no open TCP ports by default– …Only one open UDP port that we could tell

(CoDeSys V3)– This is much smaller attack surface than most

vendors

Hitachi EHV+ Line

• No mechanism seen for applying a patch• (Have not reverse engineered firmware,

though, so maybe there is a way)

Hitachi EHV+: Exploiting UDP

• No security on CoDeSys V3 Protocol, just SID– 0-65535– But PLC increments by 4, so only 16384 values

• No mechanism for adding password via CoDeSys Engineering Software

• ‘Exploit’ script capabilities tested:– Stop/Start CPU– Retrieve/Send Ladder Logic– Send New config.cfg– Change IP address

Attacking UDP

Attacking UDP

Attacking UDP

Attacking UDP

Outline

• Vulnerabilities: What are they?• 3rd party libraries• Specific example – Deep Dive• Affected Japanese Vendor #1• Affected Japanese Vendor #2• Conclusion

Vulnerable Vendor Highlight

Sanyo-Denki SanMotion C– CoDeSys V2– vxWorks debugger service– Backdoor accounts (FTP)

Sanyo-Denki SanMotion C

SanMotion CFesto CECX-X-M1

SanMotion CFesto CECX-X-M1Keba CP 232/Z

Festo/Sanyo/OEM?

• Festo and Sanyo seem to use same OEM• Identical firmware on all systems

– Also in use by Keba, Kuka, Trumpf, Haitian, Buehler, Duerr, Engel, and other vendors

– OEM probably Keba Automation CP 232/Z (Austria)

• Tons of software components in system– vxWorks– CoDeSys– CANOpen protocol stack– Etc

Festo/Sanyo/Etc Vulnerabilities

• CoDeSys V2– All V2 problems exist– First contacted Festo via ICS-CERT in early 2013– See ICS-CERT advisory ICSA-14-084-01, Festo

refused to fix any problems• vxWorks Debugging Backdoor CVE-2005-

3804– Unauthenticated read/write to all memory– Vulnerability was 5 years old at product release

• Backdoor FTP account• CANBus debugging protocol ports

Festo/Sanyo/Etc Vulnerabilities

• Of the 9 total companies affected, none of them spotted the security problems

• Likely that none of these companies do internal or external red teaming

• Unknown if any of these companies share vulnerability information (what about hardware defect information?)

Outline

• Vulnerabilities: What are they?• 3rd party libraries• Specific example – Deep Dive• Affected Japanese Vendor #1• Affected Japanese Vendor #2• Conclusion

Conclusions

• Identify security issues before product release

• Vendors should build list of 3rd party products and monitor them for vulnerabilities

• In Sanyo-Denki example, controller released April 2010 (5 years after vxWorks vulnerability widely known)

• In Hitachi example, EHV+ line released before v2 vulnerability was public, ~June-September 2010. Should have been noticed during internal security analysis

Conclusions 2

• “Plan to Patch”– Design an update mechanism– Secure by design (functional/interface security) – Genuine bugs are then easy to patch

• Design problems major problem for ICS– Ex: Schneider patches webserver directory

traversal quickly, but years later design problems persist

– Ex: 3S Software patched directory traversal bugs (Aaron Portnoy) quickly, took >1 year to patch CoDeSys design problem. Still 0 adoption of that patch!

Questions?

Reid Wightman

wightman@digitalbond.com@ReverseICS – Twitter