Introduction to logging in django
-
Upload
sivasubramaniam-arunachalam -
Category
Technology
-
view
39 -
download
1
Transcript of Introduction to logging in django
Introduction to Logging in Django
Sivasubramaniam Arunachalam
July 20, 2013
@sivaa_in
http://www.meetup.com/BangPypers/events/125755952/
Agenda
• Introduction to Logging
• Python/Django Logging
• Few Best Practices
• Demo
Your Application misbehaved
Next, what the user will do?
https://twitter.com/naitikvyas/statuses/221096865574293504
https://twitter.com/vijaypadiyar/status/352439670732369921
https://twitter.com/PaulAppleyard/status/42294624353656832
The Users
• End Users
• System Administrators
• Developers
• Support Team
The Questions
• What
• When
• How Much
• How to Control
• How to Correlate
to Log
What to Log?It’s an Art
Exceptions
• No Managed Exceptions
• Global Exception Handler
Events
• Security Incidents
• Life Cycle
• Resource usage
States
• Retry
• Time-Outs
Exceptions/Events alone will not help much
Debug Traces
• User Input
• System/API Calls
• Method Invocation
And much more…
• HTTP Requests
• DB Calls
• Threads
Logging in Django
Logging in DjangoPython• Standard Module
• Multiple Destinations
• Custom Filtering Options
import logging
• Loggers
• Handlers
• Filters
• Formatters
Loggers
• Entry Point
• Unique Name
• Log Level
Log Level
• DEBUG
• INFO
• WARNING
• ERROR
• CRITICAL
Log Record• Log Level
• Logger Name
• Message– Error Code
– Stack Trace
• Others
– Path, Line #, args, func
Handlers
• Output Mechanism – File
– Screen/Console
– HTTP/Mail
– Queue
• Log Level
Default Handlers
• StreamHandler
• SocketHandler
• DatagramHandler
• SysLogHandler
• NTEventLogHandler
• MemoryHandler
• FileHandler– Default
– Rotating
– TimedRotating
– Watched
• NullHandler
• SMTPHandler
• HTTPHandler
Handler 02Logger ‘X’
Handler N
Handler 01
Handler E-MailPayment Logger
Handler File
Handler SMS
Handler E-MailPayment Logger
Handler File
Handler SMS
ERROR
Filters
• Conditional Filtering
• Changing Log Levels
• Chaining
Logger 02Filter ‘X’
Handler 01
Logger 01
Log Record will be Dropped, If
• Log Level is lower than– Logger’s Log Level
– Handler’s Log Level
• Filter Chain returns a False
Formatters
• The output text format
• Python Formatting String
http://docs.python.org/2/library/logging.html#logrecord-attributes
logging.method()• logger.log()
• logger.critical()
• logger.error()
• logger.warning()
• logger.info()
• logger.debug()
• logger.exception()
Django Extensions - Loggers
1. django
2. django.request
3. django.db.backends
django.request
• 4xx -> WARNING
• 5xx -> ERROR
• Contexts– HTTP Status Code (Response)
– Entire Request Object
django.db.backends
• SQLs -> DEBUG
• Only when DEBUG = True
• Contexts– SQL Statement
– Duration
– Parameters
Django Extensions – Handlers
• AdminEmailHandler
AdminEmailHandler
• E-Mails -> Site Admins
• Contexts– Request Object
– Stack Trace
• include_html = True/False– Full Trace/Local Vars/Settings
Django Extensions – Filters
• CallBackFilter
• RequireDebugFalse
• RequireDebugTrue
CallBackFilter
• Call Back Functions
• Called with Log Record
• From 1.4
RequireDebugFalse
• DEBUG = False
• From 1.4
RequireDebugTrue
• DEBUG = True
• From 1.5
settings.LOGGING
disable_existing_loggers
• False - Merge
• True - Ignore
• From 1.5
SENTRY
Best Practices
self.logger = logging.getLogger(type(self).__name__)
No Single Logger For All Classes
Project Level Formatters
Avoid “print” in development. Use “stdout” with a “StreamHandler”
Don’t Catch -> Log -> Re-raiseLet the caller do that
Something terribly wrong? Don’t use logging alone. Raise a Exception to the caller.
. Logging is NOT a way to deal with exceptions
Different Log File for each Environment
LOG_ROOT = '/var/log/<project>/<env>/'
Un-handled Exceptions should be stored separately
Sensitive information’s in Logs
Size of the Log Files
• Text Editor Problems
• Archive via Scripts
• Compress to Save Space
Only Integers1300604499194,4,192168001002,20600,1001,2,500000
Be aware of cluster nodesserver<id>_node<id>_<logname>.log
is_debug()
I/O Performance
Batch Logging
No Repeated Logs & Stack Traces
Infrastructure Failures
End Users -> No Stack Trace Please!
Centralized Logging
Thank [email protected]
bit.ly/sivasubramaniambit.ly/sivaa_in
References
http://css.dzone.com/articles/best-practices-python-logginghttp://architects.dzone.com/articles/deferred-logging-file-handlerhttp://architects.dzone.com/articles/high-performance-and-smarterhttp://java.dzone.com/articles/dark-art-logginghttp://architects.dzone.com/articles/think-ahead-think-logginghttp://www.shutupandship.com/2012/02/how-python-logging-module-works.htmlhttp://java.dzone.com/news/application-logging-what-when