06:05:55 07.11.2011 INFO (Class1.java:41) User ‘ bfernan ’ Start method1

Logging Best Practices

Dubna 2012

Benedicto Fernandez

Software Engineer

CERN / GS-AIS

Only useful for debugging/development?

Contents

• Logging libraries

• Logging levels

• Where to log?

• Performance

• How to access?

• What to log?

What are we talking about?

First Logging: System.out.println

• Easy

• Quick

• No dependencies

• Redirect to file or null in Production– Log everything or nothing

System.out.println(“Start method1”);

ExampleStart method1

Before retrieve information from DB

After retrieve information from DB

Updating information

Before save information in the DB

After save information in the DB

Finish method1

Start method2

Before retrieve information from DB

Error retrieving information from DB: NullPointerException

Next evolution:Logging Libraries

• Log4j, Java Logging…log.info(“Start method1”);

• Provide– Time of the message– The class, the method, the line– …

• Print easily the stacktrace• Log to files, databases, system log

mechanisms

Example06:05:56 07.11.2011 INFO (Class1.java:211) Start method2

06:05:56 07.11.2011 INFO (Class1.java:220) Before retrieve information from DB

06:05:56 07.11.2011 INFO (Class1.java:243) Error retrieving information from DB: NullPointerException

java.lang.NullPointerException

at pkg.Class1.method2(Class1.java:225)

at pkg.Class1.methodN(Class1.java:458)

at pkg2.ClassZ.methodUpdate(ClassZ.java:458)

at …

at java.lang.Thread.run(Thread.java:595)

at …

Example06:05:56 07.11.2011 INFO (Class1.java:211) Start method206:05:56 07.11.2011 INFO (Class1.java:220) Before retrieve information from DB

at …

Logging levels (I)

• Log4j– Trace– Debug– Info– Warning– Error– Fatal

• Java Logging– Finest– Finer– Fine– Config– Info– Warning– Severe

06:05:56 07.11.2011 INFO (Class1.java:211) Parameter ‘age’ is not an Integer06:05:56 07.11.2011 INFO (Class1.java:211) Finish method2

06:05:56 07.11.2011 INFO (Class1.java:211) Start method2

06:05:56 07.11.2011 DEBUG (Class1.java:220) Before retrieve information from DB

06:05:56 07.11.2011 ERROR (Class1.java:243) Error retrieving information from DB: NullPointerException

at …

06:05:56 07.11.2011 INFO (Class1.java:211) Parameter ‘age’ is not an Integer

06:05:56 07.11.2011 INFO (Class1.java:211) Finish method2

06:05:56 07.11.2011 DEBUG (Class1.java:220) Before retrieve information from DB06:05:56 07.11.2011 ERROR (Class1.java:243) Error retrieving information from DB: NullPointerException

at …

06:05:56 07.11.2011 INFO (Class1.java:211) Parameter ‘age’ is not an Integer

06:05:56 07.11.2011 INFO (Class1.java:211) Finish method2

06:05:56 07.11.2011 DEBUG (Class1.java:220) Before retrieve information from DB

06:05:56 07.11.2011 ERROR (Class1.java:243) Error retrieving information from DB: NullPointerExceptionjava.lang.NullPointerException

at …

Logging levels (II)

log4j.logger.ch.cern.ais=DEBUG

log4j.logger.ch.cern.ais.package.Class1=TRACE

log4j.logger.org.springframework=ERROR

log4j.logger.org.springframework.ws=INFO

Where to log?

• Text files (Plain text, XML,…)• Binary files• Database• Syslog (Windows & Unix/Linux)• Network (TCP & UDP)

Where to log?

• Text files (Plain text, XML,…)• Binary files• Database• Syslog (Windows & Unix/Linux)• Network (TCP & UDP)

Always to a Text file (+ something else)?

Performance

• "hidden" cost of parameter construction logger.debug("Entry number: " + i + " is " +

String.valueOf(entry[i]));

Performance

• "hidden" cost of parameter construction logger.debug("Entry number: " + i + " is "

+ String.valueOf(entry[i]));

• Solutionif (logger.isDebugEnabled()) {

logger.debug("Entry number: " + i + " is "

+ String.valueOf(entry[i]));

Next evolution:Logging Libraries v2.0

• Simple Logging Facade (SLF4J)– Logback

• Parameterized messageslogger.debug("Entry number: {} is {}“, i

, String.valueOf(entry[i]));

Multiple users (I)

• Different users working simultaneously

• Log messages mixed for different users

• Difficult to trace user’s actions

Multiple users (I)

• Different users working simultaneously

• Log messages mixed for different users

• Difficult to trace user’s actions

Log User ID

Multiple users - Example06:05:55 07.11.2011 INFO (Class1.java:41) User=bfernan; message=‘Start method1’;06:05:55 07.11.2011 DEBUG (Class1.java:44) message=‘Before retrieve information from DB’;

06:05:55 07.11.2011 DEBUG (Class1.java:46) message=‘After retrieve information from DB’;

06:05:55 07.11.2011 INFO (Class1.java:50) message=‘Updating information’;

06:05:55 07.11.2011 DEBUG (Class1.java:94) message=‘Before save information in the DB’;

06:05:56 07.11.2011 DEBUG (Class1.java:96) message=‘After save information in the DB’;

06:05:56 07.11.2011 INFO (Class1.java:100) User=bfernan; message=‘Finish method1’;

06:05:56 07.11.2011 INFO (Class1.java:211) User=xxxxxxx; message=‘Start method2’;06:05:56 07.11.2011 DEBUG (Class1.java:220) message=‘Before retrieve information from DB’;

06:05:56 07.11.2011 ERROR (Class1.java:243) message=‘Error retrieving information from DB: NullPointerException’;

Multiple users (II)

• Mapped Diagnostic Contexts (MDC)public void serviceMethod1(String userName,…) {

String userName = getLoggedUserName();

MDC.put( “userName”, userName);

• Configuration…PatternLayout=%T User=%X{userName};message=‘%m’;%n

Multiple users (III)public void doFilter(ServletRequest request, ServletResponse response,

FilterChain chain) throws IOException, ServletException {

HttpServletRequest req = (HttpServletRequest) request; Principal principal = req.getUserPrincipal(); // Please note that we could have also used a cookie to // retrieve the user name

if (principal != null) { String username = principal.getName(); MDC.put(USER_KEY, username); }

try { chain.doFilter(request, response); } finally { MDC.remove(USER_KEY); } }

Multiple users (III)public void doFilter(ServletRequest request, ServletResponse response,

FilterChain chain) throws IOException, ServletException {

HttpServletRequest req = (HttpServletRequest) request; Principal principal = req.getUserPrincipal(); // Please note that we could have also used a cookie to // retrieve the user name

if (principal != null) { String username = principal.getName(); MDC.put(USER_KEY, username); }

try { chain.doFilter(request, response); } finally { MDC.remove(USER_KEY); } }

Multiple users - Example06:05:55 07.11.2011 INFO (Class1.java:41) User=bfernan; message=‘Start method1’;

06:05:55 07.11.2011 DEBUG (Class1.java:44) User=bfernan; message=‘Before retrieve information from DB’;

06:05:55 07.11.2011 DEBUG (Class1.java:46) User=bfernan; message=‘After retrieve information from DB’;

06:05:55 07.11.2011 INFO (Class1.java:50) User=bfernan; message=‘Updating information’;

06:05:55 07.11.2011 DEBUG (Class1.java:94) User=bfernan; message=‘Before save information in the DB’;

06:05:56 07.11.2011 DEBUG (Class1.java:96) User=bfernan; message=‘After retrieve information in the DB’;

06:05:56 07.11.2011 INFO (Class1.java:100) User=bfernan; message=‘Finish method1’;

06:05:56 07.11.2011 INFO (Class1.java:211) User=xxxxxxx; message=‘Start method2’;

06:05:56 07.11.2011 DEBUG (Class1.java:220) User=xxxxxxx; message=‘Before retrieve information from DB’;

06:05:56 07.11.2011 ERROR (Class1.java:243) User=xxxxxxx; message=‘Error retrieving information from DB: NullPointerException’;

Multiple Connections Same User (I)

• “Account” shared by 2+ end users

• Different – Session ID– Remote Address

Multiple Connections Same User (I)

• “Account” shared by 2+ end users

• Different – Session ID– Remote Address

Log Session ID

Log Remote IP

Multiple Connections Same User (I) - Example

06:05:55 07.11.2011 INFO (Class1.java:41) IP=192.168.2.99;Session=00000001; User=bfernan;message=‘Start method1’;

06:05:55 07.11.2011 DEBUG (Class1.java:44) IP=192.168.2.99;Session=00000001; User=bfernan;message=‘Before retrieve information from DB’;

06:05:55 07.11.2011 INFO (Class1.java:211) IP=61.82.23.9;Session=00000002; User=bfernan;message=‘Start method2’;

06:05:55 07.11.2011 DEBUG (Class1.java:220) IP=61.82.23.9;Session=00000002; User=bfernan;message=‘Before retrieve information from DB’;

06:05:55 07.11.2011 DEBUG (Class1.java:46) IP=192.168.2.99;Session=00000001; User=bfernan;message=‘After retrieve information from DB’;

06:05:55 07.11.2011 INFO (Class1.java:50) IP=192.168.2.99;Session=00000001; User=bfernan;message=‘Updating information’;

06:05:56 07.11.2011 DEBUG (Class1.java:94) IP=192.168.2.99;Session=00000001; User=bfernan;message=‘Before save information in the DB’;

06:05:56 07.11.2011 DEBUG (Class1.java:96) IP=192.168.2.99;Session=00000001; User=bfernan;message=‘After retrieve information in the DB’;

06:05:56 07.11.2011 ERROR (Class1.java:243) IP=61.82.23.9;Session=00000002; User=bfernan;message=‘Error retrieving information from DB: NullPointerException’;

06:05:56 07.11.2011 INFO (Class1.java:100) IP=192.168.2.99;Session=00000001; User=bfernan;message=‘Finish method1’;

Multiple Connections Same User (II)

• What about tabs?– Same Remote IP– Same Session ID

Multiple Connections Same User (II)

• What about tabs?– Same Remote IP– Same Session ID

Log Transaction ID

Multiple Connections Same User (I) - Example

06:05:55 07.11.2011 INFO (Class1.java:41) IP=192.168.2.99;Session=00000001; Transaction=00000001;User=bfernan;message=‘Start method1’;

06:05:55 07.11.2011 DEBUG (Class1.java:44) IP=192.168.2.99;Session=00000001; Transaction=00000001;User=bfernan;message=‘Before retrieve information from DB’;

06:05:55 07.11.2011 INFO (Class1.java:211) IP=192.168.2.99;Session=00000001; Transaction=00000002;User=bfernan;message=‘Start method2’;

06:05:55 07.11.2011 DEBUG (Class1.java:220) IP=192.168.2.99;Session=00000001; Transaction=00000002;User=bfernan;message=‘Before retrieve information from DB’;

06:05:55 07.11.2011 DEBUG (Class1.java:46) IP=192.168.2.99;Session=00000001; Transaction=00000001;User=bfernan;message=‘After retrieve information from DB’;

06:05:55 07.11.2011 INFO (Class1.java:50) IP=192.168.2.99;Session=00000001; Transaction=00000001;User=bfernan;message=‘Updating information’;

06:05:56 07.11.2011 DEBUG (Class1.java:94) IP=192.168.2.99;Session=00000001; Transaction=00000001;User=bfernan;message=‘Before save information in the DB’;

06:05:56 07.11.2011 DEBUG (Class1.java:96) IP=192.168.2.99;Session=00000001; Transaction=00000001;User=bfernan;message=‘After retrieve information in the DB’;

06:05:56 07.11.2011 ERROR (Class1.java:243) IP=192.168.2.99;Session=00000001; Transaction=00000002;User=bfernan;message=‘Error retrieving information from DB: NullPointerException’;

06:05:56 07.11.2011 INFO (Class1.java:100) IP=192.168.2.99;Session=00000001; Transaction=00000001;User=bfernan;message=‘Finish method1’;

How to access the information? v1.0

• View the file

– Human readable format

• System utils (grep command)

$ grep '.*ERROR.* User bfernan .*' log_cra-ws.log

– Easy for regext (Ex: key=value;)

$ grep ‘.*ERROR.*;User=bfernan;.*’ log_cra-ws.log

Cluster of Servers

• Application distributed several Servers

• 1 log file for each server

Which one contains the error?

How to access the information? v2.0

• Forward log messages to a Server– Log4j Server

• UI Tools– Apache Log4j Chainsaw– MindTree Insight

Analytic Tools

• Splunk– Indexer– Search Engine– “Pseudo” real-time– Alerts

Example

What to log?

• User requests (start & end)

• Service methods calls (start & end)

• Calls to external systems ( before & after)

• Errors

• Any relevant state

Conclusions (I)

• Log everything useful, not only debug

information

• Log to text files, human readable format

• Use a template parseable with regexp (Ex:

key=value;)

Conclusions (II)

• Use the appropriate level

• One single action per line

• Be careful with the performance

• Be ready to profit of it

Why not logging standards?

“A good job can save a day of work”

Спасибо!

Benedicto Fernandez

Software Engineer

CERN / GS-AIS

06:05:55 07.11.2011 INFO (Class1.java:41) User ‘ bfernan ’ Start method1

Documents

Transcript of 06:05:55 07.11.2011 INFO (Class1.java:41) User ‘ bfernan ’ Start method1

Class1 110812

Welcome class1!

Heuristic Method1 2

Dip Class1

Class1 Ppt

jp method1-3

JAVA CLASS1

Class1 Mod

Class1 FINAL

Moment Distribution Method1 [Compatibility Mode]

Gadi gilon crm the people 07.11.2011 final

Class1 2005

BouncingBallExperimentLab...Table1 ExperimentalMeasurements,RegularPingPongDrop Method1 Rebound1[ 1in] Method1 Rebound2[ 1in] Method2 Bounce1[ 0.2s] Method2 Bounce2[ 0.2s] Method3

Class1 Introduction

Fa262 class1

Chapter 4 - Eddy Current Method1

dp15 C# 07.11.2011

1. Overview of Viplan Method1

Virology Class1

Ops class1