IBM Security Guardium: Troubleshooting ‘No Traffic’ Issues

IBM Security Guardium: Troubleshooting ‘No Traffic’ IssuesIBM SECURITY SUPPORT OPEN MIC

NOTICE: BY PARTICIPATING IN THIS CALL, YOU GIVE YOUR

IRREVOCABLE CONSENT TO IBM TO RECORD ANY STATEMENTS THAT

YOU MAY MAKE DURING THE CALL, AS WELL AS TO IBM’S USE OF SUCH

RECORDING IN ANY AND ALL MEDIA, INCLUDING FOR VIDEO POSTINGS

ON YOUTUBE. IF YOU OBJECT, PLEASE DO NOT CONNECT TO THIS CALL.

July 18, 2017

To hear the WebEx audio, select an option in the

Audio Connection dialog or by access the

Communicate > Audio Connection menu option. To

ask a question by voice, you must either Call In or

have a microphone on your device.

You will not hear sound until the host opens the

audio line.

For more information, visit:

http://ibm.biz/WebExOverview_SupportOpenMic

2 IBM Security

GUARDIUM MASTER SKILLS BOOTCAMP – CAMBRIDGE – JULY, 2017

© 2017 – IBM CORPORATION

Panelists

Presenter

John Adams

Guardium Support

Moderator

Andrew McCarl

Knowledge Manager, IBM Security

3 IBM Security



Agenda

• Welcome and overview

• Where’s my traffic?

• Collector or STAP?

• Reports and Policy

• STAP and Connections

• ATAP and Local Traffic

• Open discussion

4 IBM Security



Dude, where’s my traffic??

5 IBM Security



Could be almost anything…

• Sniffer is down

• STAP not installed

• STAP process not running

• KTAP or WFP not running

• ATAP needed but not enabled or misconfigured

• Network issue

• Firewall blocking STAP ports

• Policy issue

• Report conditions or runtime parameters

• Sniffer parser issue

• Even aggregation issues! https://goo.gl/images/nUgMbi

6 IBM Security



It ain't what you don't know that gets you

into trouble. It's what you know for sure

that just ain't so.

- Mark Twain

7 IBM Security



My Favorite Test – Invalid Username!

• STAP or Collector?

• Local traffic, remote or both?

• Why?

Login exceptions always captured, regardless of policy

Can test the exact instance and exact node

Bogus username is easy to find in a report

Proves that traffic was captured OR….

Almost proves traffic was not captured

• How?

Add pre-defined report, “Failed Login Attempts” to GUI

Have DBA log into the instance from a remote TCP client with an invalid username like “TestRemote”

Repeat with a local / shmem connection and username “TestLocal”

Run “Failed Login Attempts” for NOW to NOW -15 MINUTE and see which exceptions were captured.

8 IBM Security




9 IBM Security




• Similar to STAP Verification feature, but… Verification has some caveats

May require special configuration / datasources

Only tests remote TCP traffic

Basic verification doesn’t let you choose the exact node

• Results !

• Knowing where to start saves you time !

Remote Local Troubleshoot

Yes Yes Reports, policy

No No STAP, KTAP/WFP, network, firewalls

Yes No ATAP (Oracle)

No Yes ATAP (Sybase) or Windows needs reboot

Local and Remote CapturedTroubleshooting Reports and Policy

11 IBM Security



Troubleshooting Reports and Policy

• Make a clone

• Simplify the conditions Eliminate tuples and DB_USER (just for testing!)

Fields like DB_USER and SOURCE_PROGRAM have place-holder values which get replaced later. This affects how policy is applied

Watch out for nested AND/OR conditions

• Clone groups and test a small number of members

• Check the Main Entity on your report Main entity determines what each row represents

• Cross-check with a different report Compare conditions

Compare main entity

• Verify by using the ‘Allow All’ policy Simple default policy with no rules

Don’t leave this on in Production, you’ll fill the Collector !

Nothing CapturedTroubleshooting STAP, KTAP, WFP, Network, Firewall

13 IBM Security



Troubleshooting the Sniffer

• Local Taps: Are all STAPs red or just some?

If any STAPs are yellow or green, Snif is running.

• From CLI:

Stop inspection-core

Start inspection-core

• Check the Buffer Usage Report

When was the last traffic received?

Was TID stable or changing? (sniffer crash)

• If the Sniffer restarts without errors but all STAPs are red, troubleshoot the firewall next.

14 IBM Security



Buffer Usage Report

15 IBM Security



Local Taps

16 IBM Security



Troubleshooting Firewall and Network Issues

• From the host, run traceroute to the collector IP

UNIX port 16016 or 16018 (TLS)

Windows port 9500 or 9501 (TLS)

Minimal hops, latency under 100ms

• Blocked?

Work with your firewall team. Ports need to be open both ways

• When in doubt, use STAP debug or a packet sniffer

• Review the ports Technote! Guardium v10.0/10.1/10.1.2 and v9.0/9.1/9.5 Open Ports

• STAP debug: IBM MustGather: Collecting data for Guardium STAP

http://www.ibm.com/support/docview.wss?uid=swg21973188

http://www-01.ibm.com/support/docview.wss?uid=swg21606592

17 IBM Security



Troubleshooting UNIX STAP and KTAP Issues

• Is STAP installed?

find / -n guard_tap.ini

Many (but not all) flavors check /etc/inittab for utap

• Is the STAP process running? ps –ef | grep –i tap

ls –lhtr <guardium dir>/modules/STAP/current

• How many copies of guard_tap.ini do you have? *.err? *.bak?

• What are the timestamps?

• Is KTAP running?

lsomd | grep –i ktap (Linux)

genkex | grep –i ktap (AIX)

modinfo | grep –i ktap (Solaris)

Should match your STAP version!

OK if you have two (old and current version)

• Check syslog

18 IBM Security



Troubleshooting Windows STAP and WFP Issues

• Is STAP installed? Windows Services

• Is the STAP process running? Windows Services

Event Log – Applications

• Is WFP running?

C:\Users\Administrator>sc query wfpmonitor

SERVICE_NAME: wfpmonitorTYPE : 1 KERNEL_DRIVERSTATE : 4 RUNNING

(STOPPABLE, NOT_PAUSABLE, IGNORES_SHUTDOWN)WIN32_EXIT_CODE : 0 (0x0)SERVICE_EXIT_CODE : 0 (0x0)CHECKPOINT : 0x0WAIT_HINT : 0x0

C:\Users\Administrator>sc query lhmonproxy[SC] EnumQueryServicesStatus:OpenService FAILED 1060:

The specified service does not exist as an installed service.

Only Remote / Only Local CapturedTroubleshooting ATAP

20 IBM Security



Simplified Architecture of Unix/Linux STAP

21 IBM Security



Troubleshooting ATAP Issues

• Was ATAP configured and activated?

Non-GIM: <install_dir>/guard_stap/guardctl list-active

GIM: <install_dir>/modules/ATAP/current/files/bin/guardctl list-active

…/guardctl activate

• Was the DB just upgraded?

Before upgrading the database ATAP must be deactivated. Otherwise, manual intervention will be required.

…/guardctl deactivate

• Did you reboot?

When in doubt, try it.

Needed for Windows STAP in some cases. (Not related to ATAP)

IBM Guardium - When to Restart, When to Reboot

• Did you authorize the database user?

… /guardctl is-user-authorized

… /guardctl authorize_user db2admin


22 IBM Security



Consider Using an Exit Library Instead

• Easier to maintain

• Performs the same function as ATAP.

• Leverage native capabilities of the database to capture traffic.

• Uses DBMS specific libraries which ship with STAP.

• Supports:

DB2

Teradata 16.10

Informix 12.10.xC6

• Documentation: Configuring DB2_EXIT to integrate with Guardium Unix STAP


When All Else Fails …

24 IBM Security



Must-Gather for PMRs

• support must_gather sniffer_issues

• STAP diag

IBM MustGather: Collecting data for Guardium STAP

• Invalid username test:

Captured remote? Local? Both?

One node missing? The whole cluster?

• PDF or CSV of the report where you see the issue.

• Description of the missing traffic.

• Any other troubleshooting you have already done.


25 IBM Security



Questions for the panel

Now is your opportunity to ask questions of our panelists.

To ask a question now:

Raise your hand by clicking Raise Hand. The Raise Hand icon appears next to your name in the

Attendees panel on the right in the WebEx Event. The host will announce your name and unmute your

line.

or

Type a question in the box below the Ask drop-down menu in the Q&A panel.

Select All Panelists from the Ask drop-down-menu.

Click Send. Your message is sent and appears in the Q&A panel.

To ask a question after this presentation:

You are encouraged to participate in the dW Answers forum:

<https://developer.ibm.com/answers/topics/TAG.html>

26 IBM Security



Where do you get more information?

Questions on this or other topics can be directed to the product forum:https://developer.ibm.com/answers/topics/guardium.html.

More articles you can review:

• IBM MustGather: Collecting data for Guardium STAPhttp://www-01.ibm.com/support/docview.wss?uid=swg21606592

• What to do if you receive Guardium “no traffic” alerthttp://www-01.ibm.com/support/docview.wss?uid=swg21699786

• IBM Knowledge Center: Predefined Alerts https://www.ibm.com/support/knowledgecenter/en/SSMPHH_10.1.0/com.ibm.guardium.doc.admin/adm/predefined_alerts.html

Useful links:

Get started with IBM Security Support

IBM Support Portal | Sign up for “My Notifications”

Follow us:

http://www.youtube.com/IBMSecuritySupport

http://twitter.com/AskIBMSecurity

https://www.facebook.com/pages/IBM-Security-Support/221766828033861



https://www.ibm.com/support/knowledgecenter/en/SSMPHH_10.1.0/com.ibm.guardium.doc.admin/adm/predefined_alerts.html


http://www.ibm.com/support/entry/portal/support

http://www-01.ibm.com/software/support/einfo.html

27 IBM Security



Disclaimer

Please Note:

IBM’s statements regarding its plans, directions, and intent are subject to change or withdrawal without notice at IBM’s sole discretion.

Information regarding potential future products is intended to outline our general product direction and it should not be relied on in making a purchasing decision.

The information mentioned regarding potential future products is not a commitment, promise, or legal obligation to deliver any material, code or functionality. Information about potential future products may not be incorporated into any contract. The development, release, and timing of any future features or functionality described for our products remains at our sole discretion.

ibm.com/security

securityintelligence.com

xforce.ibmcloud.com

@ibmsecurity

youtube/user/ibmsecuritysolutions

© Copyright IBM Corporation 2017. All rights reserved. The information contained in these materials is provided for informational purposes only, and is provided AS IS without warranty of any kind,

express or implied. Any statement of direction represents IBM's current intent, is subject to change or withdrawal, and represent only goals and objectives. IBM, the IBM logo, and other IBM products

and services are trademarks of the International Business Machines Corporation, in the United States, other countries or both. Other company, product, or service names may be trademarks or service

marks of others.

Statement of Good Security Practices: IT system security involves protecting systems and information through prevention, detection and response to improper access from within and outside your

enterprise. Improper access can result in information being altered, destroyed, misappropriated or misused or can result in damage to or misuse of your systems, including for use in attacks on others.

No IT system or product should be considered completely secure and no single product, service or security measure can be completely effective in preventing improper use or access. IBM systems,

products and services are designed to be part of a lawful, comprehensive security approach, which will necessarily involve additional operational procedures, and may require other systems, products

or services to be most effective. IBM does not warrant that any systems, products or services are immune from, or will make your enterprise immune from, the malicious or illegal conduct of any party.

FOLLOW US ON:

THANK YOU

www.SecurityLearningAcademy.com

IBM Security Guardium: Troubleshooting ‘No Traffic’ Issues

Documents

Transcript of IBM Security Guardium: Troubleshooting ‘No Traffic’ Issues