Test and Verification SolutionsHow to avoid the Internet ... · • Most IoT products have security...
Transcript of Test and Verification SolutionsHow to avoid the Internet ... · • Most IoT products have security...
Mar, 2017
Test and Verification SolutionsHow to avoid the Internet of
Insecure Things
Delivering Tailored Solutions for
Hardware Verification and Software Testing
Index
✓ Why IoT Testing
✓ IoT Challenges and Recommendations
✓ Standards Body and Value addition
Copyright T&VS Limited | Private & Confidential | Page 3
IoT headlines – lack of consumer trust
Copyright T&VS Limited | Private & Confidential | Page 4
IoT Solution
Copyright T&VS Limited | Private & Confidential | Page 5
Testing of IoT Stack
Application
Process & Platform
Analytics
Network & Connectivity
Gateways DevicesS
ecu
rity
Perf
orm
an
ce
Fu
ncti
on
al
Inte
gra
tio
n
Pro
toc
ol
UI
Copyright T&VS Limited | Private & Confidential | Page 6
Nest Smoke Alarm
▪ Product - Nest Protect smoke + CO alarm
▪ Desc - Industrial-grade smoke sensor, can be silenced from
your phone, tests itself automatically and lasts for up to 10years. It also tells you what's wrong and even alert your phone.
▪ Connectivity Requirements:• Wi-Fi connection
• Free Nest account
• Phone or tablet with iOS 8 or later, or Android 4 or later
Copyright T&VS Limited | Private & Confidential | Page 7
Example - IoT issues
▪ (1) Nest Protect smoke alarm fault in 2014. The alarm could bedeactivated by waving at the device putting it into sleep mode.• Fix – users had to disable wave gesture feature, and a patch was made available via wifi.
▪ (2) Nest home thermostat recent fault meant the heating woulddeactivate and could not be turned back on.• Fix – a manual reset or 9 step procedure.
▪ (3) Nest Cam and Dropcam frequent outages on service for live homestreaming – potential baby monitoring.• Fix – no fix yet, was a service outage on the live video streams.
▪ All could have been caught with better testing• Also – see crowd testing later
Copyright T&VS Limited | Private & Confidential | Page 8
Example of an encryption bug: Logjam
▪ Logjam attack against the TLS protocol.• The Logjam attack allows a man-in-the-middle attacker to downgrade vulnerable TLS
connections to 512-bit export-grade cryptography.
• Due to a flaw in the TLS protocol rather than an implementation vulnerability
• Attacks a Diffie-Hellman key exchange rather than an RSA key exchange
▪ Discovered May 2015• By a group of academics
• It had been present since 1990!
• Affected all modern web browsers
▪ The advice given• If you have a web or mail server
• You should disable support for export cipher suites and use a 2048-bit Diffie-Hellman group.
• If you use a browser…• Use the most recent version of your browser installed, and check for updates frequently.
• If you’re a sysadmin or developer …• Make sure any TLS libraries you use are up-to-date ...
▪ Advice for testing …• It is IMPOSSIBLE to find these bugs in testing
• Make sure you keep up with the relevant websites that report such issues
• And make sure you follow the advice
Protocol Vulnerable to Logjam
HTTPS — Top 1 Million Domains 8.4%
HTTPS — Browser Trusted Sites 3.4%
SMTP+StartTLS — IPv4 Address Space 14.8%
POP3S — IPv4 Address Space 8.9%
IMAPS — IPv4 Address Space 8.4%
Copyright T&VS Limited | Private & Confidential | Page 9
What is a DDoS on DNS Services?
▪ DDoS = Distributed Denial of Service
▪ DNS = Domain Name System: translates website names into Internet Protocol (IP) addresses, and locate resources on the Internet.
Copyright T&VS Limited | Private & Confidential | Page 10
Botnet DNS attack
▪ Hackers hijacked millions of IoT devices
▪ Sent vast amounts of junk traffic at DNS services operated by US company Dyn
▪ Popular websites inaccessible.
Two things are clear, however: the
freewheeling idiots of the Internet of Things
business need the fear of regulation put
into them – and so do network owners and
operators.
Copyright T&VS Limited | Private & Confidential | Page 11
Technical Details
▪ Mirai is malware that turns computer systems running Linux into remotely controlled “botnets” • It primarily targets online consumer devices such as remote cameras and home routers
▪ Mirai spreads by logging into devices using their default, factory-set passwords• Mirai takes over routers, CCTV cameras, digital video recorders, etc.
Zombie = IoT device
running Mirai malware
▪ Advice for testing• Again – need to know which version of the
software is used on these devices and track reports regarding known faults
• Add tests to check if the device can be used with default passwords (force user to change
Copyright T&VS Limited | Private & Confidential | Page 12
Why are IoT devices so vulnerable?
Quality
Assurance
Security
Connectivity
standards
• Nest home thermostat had a fault where the heating
would deactivate and not be turned back on
• Petnet smart pet feeder recent incident saw a third-party
server service failure, causing pet feeds to be missed.
• Most IoT products have security measures that are 10
years out of date
• HP: 70% of the IoT devices and sensors examined were
susceptible to the vulnerabilities in the OWASP IoT Top 10
Connected devices create an increased
level of intrusion, generating new types and
unprecedented quantities of data, raising
potential quality and security issues.
onem2mOpen Interconnection
ConsortiumWireless IoT forum
IETF ZigBee AllianceIndustrial Internet
Consortium
ITUAllSeen
GSMAAlliance
IEEE AllJoyn Thread
Copyright T&VS Limited | Private & Confidential | Page 13
Patching IoT devices – the device
▪ IoT Devices are “constrained” and typically have limitations :• ROM/Flash: Puts limits on software or firmware code size
• RAM: Puts limits on the size of state and buffers available to executing software or firmware
• Processing: Puts limits on the amount of computation that can be performed in a given period of time.
• Cryptography: May have limited or no cryptographic abilities
• Energy: Limitations on the energy it has available to perform processing.
• Temperature: Temperature range under which it can operate safely.
• Interface: Puts limits on accessibility or what can be updated (e.g. software, firmware or security keys)
• Determinism: Patching mechanism must therefore be to alert other system components that this node will be off-line for a period of time
• Monolithic code: Most constrained devices have limited memory protection. Hence isolation of code components and updates of modules must be addressed via enhanced system design.
Copyright T&VS Limited | Private & Confidential | Page 14
Patching IoT devices – the connection
▪ Patching also requires consideration of the network connection• Low achievable bitrate/throughput (including limits on duty cycle),
• High packet loss and high variability of packet loss (delivery rate),
• Highly asymmetric link characteristics,
• Severe penalties for using larger packets (e.g., high packet loss due to link-layer fragmentation),
• Limits on reachability over time (a substantial number of devices may power off at any point in time but periodically "wake up" and can communicate for brief periods of time), and
• Lack of (or severe constraints on) advanced services such as IP multicast.
Copyright T&VS Limited | Private & Confidential | Page 15
Implications for testing
▪ Patching MUST be tested in advance within realistic situations
▪ Device level testing• Can the device be taken offline for the duration of the patch?
• Is the patching tested under all expected network conditions?
• What happens if the network condition is lost in the middle?(NOTE: many mobile phones have been “bricked” because the network connection was lost in the middle of a patch)
▪ System level testing• What is the effect of off lining a device for patching?
• How do you protect against a malicious patch?
• Have malicious patch protections been tested?
Copyright T&VS Limited | Private & Confidential | Page 16
A view on IoT testing
Copyright T&VS Limited | Private & Confidential | Page 17
Testing challenges – mass interoperability
▪ Many Communication protocols:• Mobile Z-Wave • Wifi 6LowPAN• Bluetooth Thread • Zigbee NFC
▪ Simulate wide range of Networking conditions:• RF testing• cell handovers• low signal strength• protocol analysis• moving between 2G, 3G & LTE or wifi
▪ Test scenarios to consider:• Moving between networks
• Losing power on upgrade
• Low bandwidth
• Simulate signal loss (going through a tunnel)
• Patching the device
Copyright T&VS Limited | Private & Confidential | Page 18
Crowd testing of the IoT
▪ What is crowd testing?• Crowdsourced testing is an emerging trend in software testing which exploits the benefits,
effectiveness, and efficiency of crowdsourcing and the cloud platform.
• It differs from traditional testing methods in that the testing is carried out by a number of different testers from different places, and not by hired consultants and professionals.
• The software is put to test under diverse realistic platforms
• Crowdsource testing allows for remote usability testing because specific target groups can be recruited through the crowd
▪ Example – Smart metering• Recruit a crowd of testers who have smart meters
• Upload latest versions of software just to their smart meters
• Get their feedback
▪ Example – Nest products• Install Nest products in selected crowds of testers
Copyright T&VS Limited | Private & Confidential | Page 19
Communication protocols - scenarios
1 Device registers to network and data connection is successfully established
2 Verify the data transferred from device to IoT platform.
3 IoT device can transfer/move between network connection types (if applicable.)
4 Device Application “stores and forwards” data to minimise the number of network
connections made by the device.
5 IoT Device Application uses dynamic polling intervals.
6 Check IoT Device Application behaviour in situations when network
communication requests fail
7 Check IoT Device Application reports power failure
8 Check IoT Device Application’s use of “off-peak’ communication
9 Check behaviour of IoT Device Application when resetting the Communications
Module after any communication failures or error conditions
10 Upgrade testing – verify post upgrade the comms unit is functioning correctly
11 Check the IoT Communications Module does not send unsolicited messages
12 Check the IoT Communications Module sends only a AAAA DNS Query. IPV6
Copyright T&VS Limited | Private & Confidential | Page 20
Security testing – OWASP TOP 10
1.) Insecure web interface
2.) Insufficient authentication/authorization
3.) Insecure network services
4.) Lack of transport encryption
5.) Privacy concerns
6.) Insecure cloud interface
7.) Insecure mobile interface
8.) Insufficient security configurability
9.) Insecure software/firmware
10.) Poor physical security
Copyright T&VS Limited | Private & Confidential | Page 21
OWASP TOP first 5 scenarios:
1. Insecure web interface• Check weak passwords• Product setup prompts for username and password• SQL Injections and sharing credential without encryption over the network.
2. Insufficient authentication/authorization• Password Complexity and two factor authentication• Password recovery and insecure mechanism• Lack of Role definition for access.
3. Insecure network services• Insecure web-services are subjected to Buffer overflow and DoS attacks• Open Ports and exploitable UDP services
4. Lack of transport encryption• Unencrypted services• Poorly implemented SSL / TSL
5. Privacy concerns• Poor data protection• Unwanted personal data collection
https://www.owasp.org/images/7/71/Internet_of_Things_Top_Ten_2014-OWASP.pdf
Copyright T&VS Limited | Private & Confidential | Page 22
OWASP TOP last 5 scenarios:
6. Insecure cloud interface• Account Enumeration
• No Account Lockout
• Credentials Exposed in Network Traffic
7. Insecure mobile interface
8. Insufficient security configurability• Poorly defined User Role and Permissions
• Lack of Password Security
• Open Security monitoring and Logging
9. Insecure software/firmware• Encryption not used to fetch Updates
• Unencrypted Updates file
• Sensitive Information in firmware
10. Poor physical security• Access to USB ports
• Removal of Storage Media
https://www.owasp.org/images/7/71/Internet_of_Things_Top_Ten_2014-OWASP.pdf
Copyright T&VS Limited | Private & Confidential | Page 23
NMI - IoTSF
▪ September 2015 NMI launched the Internet of ThingsSecurity Foundation (IoTSF), a non-profit organisationestablished to drive security excellence.
▪ Collaborative, vendor-neutral and international initiative; theIoTSF is a dedicated expert resource to drive securityexcellence by sharing knowledge, best practice and advice.
▪ 5 working groups:• 1: Self-Certification Scheme
• 2: Connected Consumer Products
• 3: Patching Constrained Devices
• 4: Framework for Vulnerability Disclosure
• 5: IoT Security Landscape
Copyright T&VS Limited | Private & Confidential | Page 24
Standards bodies – building TRUST
▪ The BSI (British Standards Institute) attempts to build TRUST with consumers• Can we build standards that guarantee some level of confidence
▪ Do we need different levels of confidence?• Autonomous car vs. smoke detector vs. pet feeder
• In safety systems we start with a hazard analysis
• From that we can set an integrity level
• And that implies different levels of development practises
▪ The NMI prefers levels of sign off• Self- certification
• External certification
• Independent certification
• Full certification against industry standards
Copyright T&VS Limited | Private & Confidential | Page 25
How we are organized
Members
Plenary Group
Executive
Steering Board
Working Groups
Working Group 1: Self-Certification
Working Group 2: Connected Consumer /
Home
Working Group 3: Patching Constrained
devices
Working Group 4: Vulnerability Disclosure
Working Group 5: IoT Landscape
2016 Priority Working
Groups
Chaired
by:
Mike
Bartley,
T&VS
Copyright T&VS Limited | Private & Confidential | Page 26
Working Group 1: Self-
Certification Scheme
The objective of this working group is to determine appropriaterequirements for a low-cost, accessible and fit-for-purpose systemof self-certification in order to improve the quality andpervasiveness of security in IoT products.
26Confidential & copyright © IoTSF 2016
Self-certificate
Is this the way forward?
Copyright T&VS Limited | Private & Confidential | Page 27
Marking the IoT Supply Chain of Trust for Complex products
Device
Hardware
Sensor
Actuator
TPM
Comms
module
Firmware
Encryption
keys
Copyright T&VS Limited | Private & Confidential | Page 28
Complex supply chain
ODM –
Develops
and
makes
device
Software
developer
Software
developers
Software
developer
Chip
vendor
Software
developer
Comms
module
vendor
“Brand
Owner” –
markets
and
supports
service
Users
Softwa
re
develo
per
IP
vendor
Copyright T&VS Limited | Private & Confidential | Page 29
Trusted supply chain
ODM –
Develops
and
makes
device
Software
developer
Software
developer
s
Software
developer
Chip
vendor
Software
developer
Comms
module
vendor
“Brand
Owner” –
markets
and
supports
service
Users
Softwa
re
develo
per
IP
vendor
OTS
RTOS
= IoTSF stamp of approval
= not approved, requires separate
audit
Copyright T&VS Limited | Private & Confidential | Page 30
IoTSF members…
▪ Follow the security guidelines for the
relevant device class
▪ Complete WG1 questionnaire with all
questions answered
▪ Assemble evidence of conformance - think
“Technical Construction File”
▪ Are entitled to use the Foundation Trustmark
for the product (possibly subject to audit)
Copyright T&VS Limited | Private & Confidential | Page 31
Auditing…
▪ The customer takes self-certification for
granted
▪ The customer audits either as due-diligence
or when there is a breach
▪ Or a trusted third party (“T3P”) may audit as
due-diligence
▪ Or IoTSF may audit (possibly via T3P) and
license Trustmark usage
Copyright T&VS Limited | Private & Confidential | Page 32
IoT Kitemark Model
IoT Network connectivity – (1) IoT End 2 end security – (2)
Purpose ensure IoT solutions verified against a wide range of networking connection / connectivity protocols
ensure IoT solutions verified against a wide range of security conditions and scenarios
Standards • GSMA IoT connection efficiencyguidelines
• onem2m connection standards
• GSMA IoT security standards• Onem2m security standards• OWASP Internet of Things Top 10• Online Trust Alliance’s IoT Trust
Framework
Example scenarios
1.) minimize the number of network connections. 2.) cope with variances in network data speed and latency considering 3.) communication requests fail.4.) Communication retry mechanisms implemented verified.
1.) Authentication / authorisation eginterfaces disallows weak passwords.2.) Encryption model eg HTTPS.3.) Cloud interface has account lockout4.) Software / firmware. Eg Ensure alldevices operate with a minimalnumber of network ports active.
Copyright T&VS Limited | Private & Confidential | Page 33
Summary
▪ Increased regulation
▪ Focus on QA & security
▪ IoT ongoing maintenance
▪ IoT Kitemark model
▪ Rebuild consumer trust
Unless these issues are addressed the only winners in the IoT wild west will be the hackers.
Mar, 2017
Test and Verification Solutions
Thank you
Delivering Tailored Solutions for
Hardware Verification and Software Testing
“IoT Device Testing: can we provide
assurance in the new wild west?”
Gaurav MaheshwariPrashant Jain