Cashing in on logging and exception data
Transcript of Cashing in on logging and exception data
![Page 1: Cashing in on logging and exception data](https://reader035.fdocuments.in/reader035/viewer/2022062310/58ecb5791a28ab11568b4623/html5/thumbnails/1.jpg)
Cashing In on Logging and Exception Data
Jason TaylorCTO, Stackify
![Page 2: Cashing in on logging and exception data](https://reader035.fdocuments.in/reader035/viewer/2022062310/58ecb5791a28ab11568b4623/html5/thumbnails/2.jpg)
“Logging is hard”
-No one, ever.
![Page 3: Cashing in on logging and exception data](https://reader035.fdocuments.in/reader035/viewer/2022062310/58ecb5791a28ab11568b4623/html5/thumbnails/3.jpg)
3
Logging: not hard, but not perfect
The Easy• Libraries: log4net, nLog, Elmah,
Enterprise Library• Very easy to integrate and use• Great for debug
The Not Easy• Production log file access• Volume of data * Number of
servers• Rotation and retention• Flat and hard to query• Getting enough context• Proactively consuming /
monitoring
![Page 4: Cashing in on logging and exception data](https://reader035.fdocuments.in/reader035/viewer/2022062310/58ecb5791a28ab11568b4623/html5/thumbnails/4.jpg)
4
Today’s topicsLog All The Things
Building a “culture of logging” to always have relevant, contextual logs that don’t add overhead.
Make the most of your logging framework.
Work Smarter, Not Harder
Using proper tooling to consolidate & aggregate all of your logging to be available to everyone that needs it, quickly consume it, and proactively put it to use.
![Page 5: Cashing in on logging and exception data](https://reader035.fdocuments.in/reader035/viewer/2022062310/58ecb5791a28ab11568b4623/html5/thumbnails/5.jpg)
5
What things? • Timestamp• Thread ID• Transaction ID• Server• Environment• User Data• Post Data• Cookies• Querystring• Etc, etc
If you’re going to take the time to log something, log enough to give you a play-by-play picture of events.
When?• For every important business transaction
![Page 6: Cashing in on logging and exception data](https://reader035.fdocuments.in/reader035/viewer/2022062310/58ecb5791a28ab11568b4623/html5/thumbnails/6.jpg)
Start Simple – handle all exceptions
Pros ConsElmah -Easy
-Lots of storage possibilities-Web page to view errors-Limited email & notification possibilities
-Unhandled only-Advanced usage requires a lot of configuration.-If MVC, don’t forget to grab Elmah.MVC-Don’t leave your errors web page exposed in producton!
Global.asax (Application_Error)-Use the framework of your choice-Captures all exceptions-Fully customizable
-You are responsible for returning state back to the method that called (e.g. a controller action) or redirecting
Try/Catch blocks -Try/Catch/Finally generally perceived as a good pattern-You will catch 100% of the exceptions that you “catch”
-You will miss 100% of the exceptions that you don’t “catch” -Can be “error prone” if you are only catching a certain type that doesn’t happen-Time consuming to “retro fit” into an app
Exception Filters (MVC) -Like using global.asax, very customizable -You have to implement everything, including errors that happen globally, how to log, etc
![Page 7: Cashing in on logging and exception data](https://reader035.fdocuments.in/reader035/viewer/2022062310/58ecb5791a28ab11568b4623/html5/thumbnails/7.jpg)
Adding ContextTreat logs like “debug when you can’t attach”
• Use verbosity levels appropriately – allows you to “turn up” and “turn down” the amount of logging as necessary
• Identity key processes and functions that need to be audited and logged• Capture the key steps, measure success / failure• Capture contextual data that helps you solve an issue or answer a question (i.e. logged in
user, Http Request data)
Don’t forget about logging in your client code• Choose a library that is smart enough to bridge server
and client code while keeping context
![Page 8: Cashing in on logging and exception data](https://reader035.fdocuments.in/reader035/viewer/2022062310/58ecb5791a28ab11568b4623/html5/thumbnails/8.jpg)
8
Adding Context - Basic
Bad Slightly Better
![Page 9: Cashing in on logging and exception data](https://reader035.fdocuments.in/reader035/viewer/2022062310/58ecb5791a28ab11568b4623/html5/thumbnails/9.jpg)
9
Adding Context - Advanced• Logged the input parameters• Logged the object created in my
DbContext• Had to use a custom appender for
log4net that knows how to serialize the objects
Alternatively, would have to serialize to a string:
log.Debug(string.Format("Creating a foo: {0}",JsonConvert.SerializeObject(foo)));
![Page 10: Cashing in on logging and exception data](https://reader035.fdocuments.in/reader035/viewer/2022062310/58ecb5791a28ab11568b4623/html5/thumbnails/10.jpg)
Adding Context – Power User!!Wouldn’t it be nice to:• Always capture the user who generated the log or exception?• Always capture basic request details?
This would allow us to have deeper, more meaningful data to search through our logs.
![Page 11: Cashing in on logging and exception data](https://reader035.fdocuments.in/reader035/viewer/2022062310/58ecb5791a28ab11568b4623/html5/thumbnails/11.jpg)
Adding Context – Power User!!Context Properties (log4net)
<log4net> <root> <level value="ALL" /> <appender-ref ref="StackifyAppender" /></root><appender name="StackifyAppender" type="StackifyLib.log4net.StackifyAppender, StackifyLib.log4net"> <logicalThreadContextKeys>User,Request</logicalThreadContextKeys> </appender></log4net></configuration>
![Page 12: Cashing in on logging and exception data](https://reader035.fdocuments.in/reader035/viewer/2022062310/58ecb5791a28ab11568b4623/html5/thumbnails/12.jpg)
Adding Context – Power User!!void MvcApplication_AuthenticateRequest(object sender, EventArgs e) {
try { log4net.LogicalThreadContext.Properties["User"] = User; } catch (Exception ex) { log.Error(ex); } }
![Page 13: Cashing in on logging and exception data](https://reader035.fdocuments.in/reader035/viewer/2022062310/58ecb5791a28ab11568b4623/html5/thumbnails/13.jpg)
The Results: Rich Logs and Errors
![Page 14: Cashing in on logging and exception data](https://reader035.fdocuments.in/reader035/viewer/2022062310/58ecb5791a28ab11568b4623/html5/thumbnails/14.jpg)
Tips & Tricks• Use your verbosity levels• Make sure your logging framework is asynchronous• Log heavily is core / shared services & other high fan-in areas• Always log the “who”• Use patterns like AOP to write enty / exit logging once, apply
everywhere• Make it a part of your development culture (don’t just write the logs –
USE them)
![Page 15: Cashing in on logging and exception data](https://reader035.fdocuments.in/reader035/viewer/2022062310/58ecb5791a28ab11568b4623/html5/thumbnails/15.jpg)
Working SmarterAt Stackify, we set out to make our logging and error monitoring smarter…
…it became a core part of our product offering.
![Page 16: Cashing in on logging and exception data](https://reader035.fdocuments.in/reader035/viewer/2022062310/58ecb5791a28ab11568b4623/html5/thumbnails/16.jpg)
Working SmarterLogs & Errors Tell (Some of) the Story• Logs as “breadcrumb trail” are good for:
• Capturing errors in raw form • Capturing key events (depending on
logging level)• Piecing together the story (if you logged it) on
one server at a time
• Error monitoring is good for:• “After the fact” notification• Seeing just the error in question• Pointing you back at log files for more info
![Page 17: Cashing in on logging and exception data](https://reader035.fdocuments.in/reader035/viewer/2022062310/58ecb5791a28ab11568b4623/html5/thumbnails/17.jpg)
Working SmarterThe Trouble With Your Current Logs:• Working with all those log files is tedious and time consuming• No cross-server / cross-app correlation• No trends / telemetry• File retention & rotation• Intermixed with Exception details
![Page 18: Cashing in on logging and exception data](https://reader035.fdocuments.in/reader035/viewer/2022062310/58ecb5791a28ab11568b4623/html5/thumbnails/18.jpg)
Working SmarterThe Trouble With Your Errors:• Logged errors usually are no more than exception and stack trace• No cross-server/cross-app tracking for errors• Missing/hard to get request details• No trends / telemetry• Need intelligent notifications• In Error Monitoring apps,
disconnected from Logs!
![Page 19: Cashing in on logging and exception data](https://reader035.fdocuments.in/reader035/viewer/2022062310/58ecb5791a28ab11568b4623/html5/thumbnails/19.jpg)
19
Indexing:• Search across apps and servers w/o managing log files• Fully indexed “Context” details• Real-time log tailing• Narrow scope by time range
Telemetry• Visualize trends in logging and exceptions
• Across all levels of granularity• Contextual telemetry – for any app or server, by default
Alerts• Notify when new or “first chance” exceptions occur• Notify on error regression• Notify when error rates are abnormal• Notify on any log query we can think of
Engage• Provide access to those who can help solve issues• Give everyone the same picture, with no extra effort
Architecture• Single platform solution• Relate exceptions to relevant logs for maximum context• Contribute to picture of “application health”
![Page 20: Cashing in on logging and exception data](https://reader035.fdocuments.in/reader035/viewer/2022062310/58ecb5791a28ab11568b4623/html5/thumbnails/20.jpg)
Live demo
Or….
How good is the conference WI-FI?
![Page 21: Cashing in on logging and exception data](https://reader035.fdocuments.in/reader035/viewer/2022062310/58ecb5791a28ab11568b4623/html5/thumbnails/21.jpg)
Search! Quickly filter!
Log tailing!
![Page 22: Cashing in on logging and exception data](https://reader035.fdocuments.in/reader035/viewer/2022062310/58ecb5791a28ab11568b4623/html5/thumbnails/22.jpg)
Context!
Full error details and telemtryFull application health details
![Page 23: Cashing in on logging and exception data](https://reader035.fdocuments.in/reader035/viewer/2022062310/58ecb5791a28ab11568b4623/html5/thumbnails/23.jpg)
23
Full search
Error Rates
Error Details
![Page 24: Cashing in on logging and exception data](https://reader035.fdocuments.in/reader035/viewer/2022062310/58ecb5791a28ab11568b4623/html5/thumbnails/24.jpg)
24
![Page 25: Cashing in on logging and exception data](https://reader035.fdocuments.in/reader035/viewer/2022062310/58ecb5791a28ab11568b4623/html5/thumbnails/25.jpg)
25
![Page 26: Cashing in on logging and exception data](https://reader035.fdocuments.in/reader035/viewer/2022062310/58ecb5791a28ab11568b4623/html5/thumbnails/26.jpg)
Benefits of an integrated logging / error platform• Get real telemetry, know how well your app behaves from release to
release• In context to overall performance• Use in QA environments, look for new exceptions and abnormal rates
before it is released to prod• Proactively hunt down bugs & improve your codebase• Know about an issue before your customers tell you• Quicker, more efficient debugging via logs