Black Box Testing Methodology SANS.ppt Box Testing... · PDF fileWhy Black Box Test? !...

Click here to load reader

  • date post

    30-Jan-2018
  • Category

    Documents

  • view

    245
  • download

    0

Embed Size (px)

Transcript of Black Box Testing Methodology SANS.ppt Box Testing... · PDF fileWhy Black Box Test? !...

  • Copyright Red Tiger Security Do not print or distribute without consent.

    Black Box Testing Methodologies

    Joe Cummins, PCIP, OPST Jonathan Pollet, CISSP, CAP, PCIP January 24, 2011 SANS SCADA Webinar, SCADA Summit Series 2011

    welcome

  • Copyright Red Tiger Security Do not print or distribute without consent. 2

  • Copyright Red Tiger Security Do not print or distribute without consent.

    Outline

    Why Black Box Test?

    Layered approach

    Black Box vs. White Box

    Components of an Assessment

    Process

    Reports and metrics

  • Copyright Red Tiger Security Do not print or distribute without consent.

    Why Black Box testing?

    Know what you are putting out on the network

    How does a device respond to protocols it does not recognize?

    What happens when it gets a confusing message?

    are you sure?

  • Copyright Red Tiger Security Do not print or distribute without consent.

    Phased Approach to Device / Application Testing

    Protocol RFCs Proper communications

    Software DOS, Overflow, Etc Kernel

    Firmware Assembler

    Hardware Components Monitoring

    OS

    Firmware

    Hardware

    Applications Kernel

    Assembler

  • Copyright Red Tiger Security Do not print or distribute without consent.

    Layered Defence

    6. Embedded Device

    5. Communication Method

    4. Servers / Workstations

    3. DMZ

    2. Infrastructure

  • Copyright Red Tiger Security Do not print or distribute without consent.

    Software / Middleware

    Exceptions Failures Null Pointers Access Violations

    Memory Corruption Buffer Overflow Stack Overflow

  • Copyright Red Tiger Security Do not print or distribute without consent.

    Hardware

    Components NIC (wired, wireless) Ports

    Monitoring CPU Temperature Cycles Processes Stack

  • Copyright Red Tiger Security Do not print or distribute without consent.

    Tools of the Trade

  • Copyright Red Tiger Security Do not print or distribute without consent.

    Manual Code Review

    Automated tools Highlights errors / changes Known common application faults Verification of Syntax

    Viewers Import / Export Source Render Analyze

  • Copyright Red Tiger Security Do not print or distribute without consent.

    Analysis Engine

    Core Fuzzing Process Reliance on the Tools and plugins to generate proper data

    Manual Code Review Line by line review

    Blended Analysis

  • Copyright Red Tiger Security Do not print or distribute without consent.

    Blended Analysis

    Device Testing Methodology

    Combination of both aspects Code review + Fuzzing = closer examination

    Benefits of both forms of Analysis

  • Copyright Red Tiger Security Do not print or distribute without consent.

    Anatomy of the Analysis

    Model to Mayhem

  • Copyright Red Tiger Security Do not print or distribute without consent.

    White Box vs. Black Box Testing

    Delivery

    Application Implementation Protocol Specification

    Function

    Design Abstraction Dissection

    Analysis

    Code Review Input Testing

    Testing

    Verification Validation

  • Copyright Red Tiger Security Do not print or distribute without consent.

    Analysis Engine

    Final Deliverable

    Input Modules

    Protocol Template

    Target

    Seed File Session

    Assembler

    Sessions

    Collection Method

    EKG

    Outputs

    Core Fuzzing Process

  • Copyright Red Tiger Security Do not print or distribute without consent.

    Input Generation Methods (Invalid)

    Error Collection

    Isolated Element

    Invalid Data

  • Copyright Red Tiger Security Do not print or distribute without consent.

    Input Generation Methods (Valid)

    Valid Output

    Isolated Element

    Valid Data

  • Copyright Red Tiger Security Do not print or distribute without consent.

    Device EKG / ECG

    ICMP Echo Reply Config

    SNMP Status Agent Manager

    TCP HTTP (S) SSH (22) TELNET (23)

  • Copyright Red Tiger Security Do not print or distribute without consent.

    Device EKG / ECG

    ICMP ICMP Echo / Reply Dropped Config, Delayed Response, etc

    TCP Active Session, keep-alive, timeouts HTTPS, SSH, Telnet,

    SNMP Monitoring Statistics

  • Copyright Red Tiger Security Do not print or distribute without consent.

    Comparison and Contrast

    What does an error look like? How do you work with this information? What can be determined about the program / device? Can this lead to cascading errors?

    What can you do with an error? POC? Weaponization / Exploit Development

  • Copyright Red Tiger Security Do not print or distribute without consent.

    Exploit Weaponization (Stages)

    Staged Attack Binary

    Exploit Payload

    Vuln.

    Code

    Socket

    Packaged Exploit

    Exploit

  • Copyright Red Tiger Security Do not print or distribute without consent.

    Output Collection

    Comparison and contrast Characteristics of an error Scale of vulnerability

    Weaponization Malicious code Payloads Repeatable

    Hardware EKG Health of the device State of the device

  • Copyright Red Tiger Security Do not print or distribute without consent.

    Reports and Metrics

    Black box testing Report: Spreadsheet of tests and outputs, Tools used, Findings, Recommendations, Remediation steps,

    Include: Packet Captures (in pcap) for replay Screen captures Outputs for future analysis

  • Copyright Red Tiger Security Do not print or distribute without consent.

    Wrap-up

    Devices need to be tested Vendors continue to push product to market Consumers need to be aware of the hazards

    Small investment / Resilient Devices

    Testing is CRITICAL

    Does not need to be resource intensive Complex task, automated and facilitated Part of the internal Testbed

  • Copyright Red Tiger Security Do not print or distribute without consent.

    Contact info:

    Jonathan Pollet, CISSP, CAP, PCIP Red Tiger Security - USA office: +1.877.387.7733 web: www.redtigersecurity.com [email protected] Check out our Industry Forum and sign up for RSS feed:

    Forum: http://www.redtigersecurity.com/forum/

    Joe Cummins, PCIP, OPST Founder, Principal Consultant Red Tiger Security - Canada office: +1.877.387.7733 web: www.redtigersecurity.ca [email protected]

    25