Event Source API & Semantic Logging (Talk at NYC Code camp Sep 2014)
description
Transcript of Event Source API & Semantic Logging (Talk at NYC Code camp Sep 2014)
![Page 1: Event Source API & Semantic Logging (Talk at NYC Code camp Sep 2014)](https://reader033.fdocuments.in/reader033/viewer/2022052411/55786496d8b42a4c748b4fca/html5/thumbnails/1.jpg)
Muhammad Siddiqi
Event Source API &
Semantic Logging
@SiddiqiMuhammad
http://shujaat.net
![Page 2: Event Source API & Semantic Logging (Talk at NYC Code camp Sep 2014)](https://reader033.fdocuments.in/reader033/viewer/2022052411/55786496d8b42a4c748b4fca/html5/thumbnails/2.jpg)
MARQUEE SPONSOR
![Page 3: Event Source API & Semantic Logging (Talk at NYC Code camp Sep 2014)](https://reader033.fdocuments.in/reader033/viewer/2022052411/55786496d8b42a4c748b4fca/html5/thumbnails/3.jpg)
PLATINUM SPONSOR
![Page 4: Event Source API & Semantic Logging (Talk at NYC Code camp Sep 2014)](https://reader033.fdocuments.in/reader033/viewer/2022052411/55786496d8b42a4c748b4fca/html5/thumbnails/4.jpg)
PLATINUM SPONSOR
![Page 5: Event Source API & Semantic Logging (Talk at NYC Code camp Sep 2014)](https://reader033.fdocuments.in/reader033/viewer/2022052411/55786496d8b42a4c748b4fca/html5/thumbnails/5.jpg)
GOLD SPONSORS
![Page 6: Event Source API & Semantic Logging (Talk at NYC Code camp Sep 2014)](https://reader033.fdocuments.in/reader033/viewer/2022052411/55786496d8b42a4c748b4fca/html5/thumbnails/6.jpg)
SILVER SPONSORS
![Page 7: Event Source API & Semantic Logging (Talk at NYC Code camp Sep 2014)](https://reader033.fdocuments.in/reader033/viewer/2022052411/55786496d8b42a4c748b4fca/html5/thumbnails/7.jpg)
Ask the Logs!!!
176 [main] INFO examples.Sort - Populating an array of 2 elements in reverse order.225 [main] INFO examples.SortAlgo - Entered the sort method.262 [main] DEBUG SortAlgo.OUTER i=1 - Outer loop.276 [main] DEBUG SortAlgo.SWAP i=1 j=0 - Swapping intArray[0] = 1 and intArray[1] = 0290 [main] DEBUG SortAlgo.OUTER i=0 - Outer loop.304 [main] INFO SortAlgo.DUMP - Dump of integer array:317 [main] INFO SortAlgo.DUMP - Element [0] = 0331 [main] INFO SortAlgo.DUMP - Element [1] = 1343 [main] INFO examples.Sort - The next log statement should be an error message.346 [main] ERROR SortAlgo.DUMP - Tried to dump an uninitialized array.467 [main] INFO examples.Sort - Exiting main method.
When should we deploy?
When should we schedule maintenance?
What is the effect of our recent email advertising campaign?Should we support IE or Windows XP
anymore?
Why does application response gets slower between 6 – 11 PM?
Is the application flow as we expect? (I see a lot of items getting added to the cart but no orders, what is going on?)
![Page 8: Event Source API & Semantic Logging (Talk at NYC Code camp Sep 2014)](https://reader033.fdocuments.in/reader033/viewer/2022052411/55786496d8b42a4c748b4fca/html5/thumbnails/8.jpg)
We need…
Meaningful structured logging
Minimum impact on performance
![Page 9: Event Source API & Semantic Logging (Talk at NYC Code camp Sep 2014)](https://reader033.fdocuments.in/reader033/viewer/2022052411/55786496d8b42a4c748b4fca/html5/thumbnails/9.jpg)
Session Agenda?
-What is Semantic??
- Event Source API
- Semantic Logging Application Block
- Event Source & ETW Tools
- Event Source & Transfer Events
- Event Source & Windows Event Logs
![Page 10: Event Source API & Semantic Logging (Talk at NYC Code camp Sep 2014)](https://reader033.fdocuments.in/reader033/viewer/2022052411/55786496d8b42a4c748b4fca/html5/thumbnails/10.jpg)
Semantic [Webster]“Of or relating to meaning, especially meaning in language.”
-------Log What you mean!!! ------ What, Where & How??
Semantic Web { Tim Berners Lee – Scientific American 2001 }[ converting the unstructured / semi-structured data on web to structured one ]
[ a web of data that can be processed by machines ]
--- Creating logs which can be processed by machines ---
Other Names In the Industry- Structured Logging- Strongly typed events
Semantic Logging is to create logs with machine’s consumption in mind
![Page 11: Event Source API & Semantic Logging (Talk at NYC Code camp Sep 2014)](https://reader033.fdocuments.in/reader033/viewer/2022052411/55786496d8b42a4c748b4fca/html5/thumbnails/11.jpg)
Log4Net Exam
pleLog File
![Page 12: Event Source API & Semantic Logging (Talk at NYC Code camp Sep 2014)](https://reader033.fdocuments.in/reader033/viewer/2022052411/55786496d8b42a4c748b4fca/html5/thumbnails/12.jpg)
EventSource APIProvides the ability to create events for event tracing for Windows (ETW).
.net framework 4.5
![Page 13: Event Source API & Semantic Logging (Talk at NYC Code camp Sep 2014)](https://reader033.fdocuments.in/reader033/viewer/2022052411/55786496d8b42a4c748b4fca/html5/thumbnails/13.jpg)
EventSourceProvides the ability to create events for event tracing for Windows (ETW).
Helps with…
• Allows to separate the event authoring request from writing the event. + Simpler Logging site.
• Make it easier to work with ETW.
• Takes care of • generating the manifest for your events• converting your method parameters to event data.• XCOPY deployment now possible.
automatic
Performance – ETW based Structured Payload Cross-application
Benefits
![Page 14: Event Source API & Semantic Logging (Talk at NYC Code camp Sep 2014)](https://reader033.fdocuments.in/reader033/viewer/2022052411/55786496d8b42a4c748b4fca/html5/thumbnails/14.jpg)
EventSource & EventListener
Logging Call site
RFC4122
Listener ConsumerController
In-Proc Usage
Controller (Attachment with EventSource)
Consumer [ OnEventWritten() ]
![Page 15: Event Source API & Semantic Logging (Talk at NYC Code camp Sep 2014)](https://reader033.fdocuments.in/reader033/viewer/2022052411/55786496d8b42a4c748b4fca/html5/thumbnails/15.jpg)
EventSource’s [Event] MethodsLevel Keyword
Task & Opcodes Versioning
Max level of events to enable
VIWECL Very Intelligent & Wise Elfs Can Learn
Power of 2
Task oriented grouping of events
Task
OpcodesUsed in conjunction
Integer Value
Operation within a taskTask Opcodes
Versions cannot co-exist…duplicate Event Ids not allowed
![Page 16: Event Source API & Semantic Logging (Talk at NYC Code camp Sep 2014)](https://reader033.fdocuments.in/reader033/viewer/2022052411/55786496d8b42a4c748b4fca/html5/thumbnails/16.jpg)
Demo EventSource with attributes
![Page 17: Event Source API & Semantic Logging (Talk at NYC Code camp Sep 2014)](https://reader033.fdocuments.in/reader033/viewer/2022052411/55786496d8b42a4c748b4fca/html5/thumbnails/17.jpg)
Event Source Proxy
![Page 18: Event Source API & Semantic Logging (Talk at NYC Code camp Sep 2014)](https://reader033.fdocuments.in/reader033/viewer/2022052411/55786496d8b42a4c748b4fca/html5/thumbnails/18.jpg)
ETW : Event Tracing for Windows
![Page 19: Event Source API & Semantic Logging (Talk at NYC Code camp Sep 2014)](https://reader033.fdocuments.in/reader033/viewer/2022052411/55786496d8b42a4c748b4fca/html5/thumbnails/19.jpg)
Event Provider
Event Payload
.net framework ETW
Event Type
Method Parameters
Event Method
Event Source
Event Controller
Event Consumer
Event Listener
EventSource & ETW Mapping
![Page 20: Event Source API & Semantic Logging (Talk at NYC Code camp Sep 2014)](https://reader033.fdocuments.in/reader033/viewer/2022052411/55786496d8b42a4c748b4fca/html5/thumbnails/20.jpg)
Semantic Logging Application Block [SLAB]
- Provides- Customized Listener- Sinks- Unit Testing
Destination specific
Console
Flat File
Rolling Flat File
Database
Windows Azure Table Storage
- Can be consumed- In-Proc- Out-Proc
Semantic Logging Service
SLAB is about making your log talk!!! EventSource : Authoring LogsSLAB: Example for Logs Consumption
![Page 21: Event Source API & Semantic Logging (Talk at NYC Code camp Sep 2014)](https://reader033.fdocuments.in/reader033/viewer/2022052411/55786496d8b42a4c748b4fca/html5/thumbnails/21.jpg)
SLAB & Rx
ObservableEventListener : IObservable<EventEntry>
ConsoleLog
FlatFileLog
RollingFlatFileLog
SqlDatabaseLog
WindowsAzureTableLog
XLog
LogToX (extension methods
- creates subscription)
ConsoleSink
FlatFileSink
RollingFlatFileSink
SqlDatabaseSink
WindowsAzureTableSink
XSink
IObs
erve
r<T>
![Page 22: Event Source API & Semantic Logging (Talk at NYC Code camp Sep 2014)](https://reader033.fdocuments.in/reader033/viewer/2022052411/55786496d8b42a4c748b4fca/html5/thumbnails/22.jpg)
![Page 23: Event Source API & Semantic Logging (Talk at NYC Code camp Sep 2014)](https://reader033.fdocuments.in/reader033/viewer/2022052411/55786496d8b42a4c748b4fca/html5/thumbnails/23.jpg)
SLAB Nuget Packages
Semantic Logging Service Separate Download
![Page 24: Event Source API & Semantic Logging (Talk at NYC Code camp Sep 2014)](https://reader033.fdocuments.in/reader033/viewer/2022052411/55786496d8b42a4c748b4fca/html5/thumbnails/24.jpg)
Unit Testing Event Source
Why? EventSource.EventMethod[Parameters]
ETW Event Type
EventSourceAnalyzer
EventSourceInspect
• Error when the given EventSource is enabled by an EventListener?• Can event schema be requested from the EventSource?• Can all Event based methods be invoked in the specified EventSource
class?
Checks Performed
![Page 25: Event Source API & Semantic Logging (Talk at NYC Code camp Sep 2014)](https://reader033.fdocuments.in/reader033/viewer/2022052411/55786496d8b42a4c748b4fca/html5/thumbnails/25.jpg)
Event Text Formatting
- Flat File Sink- Rolling Flat File Sink- Console Sink
Supported Sinks
+ Custom Event Text Formatter
![Page 26: Event Source API & Semantic Logging (Talk at NYC Code camp Sep 2014)](https://reader033.fdocuments.in/reader033/viewer/2022052411/55786496d8b42a4c748b4fca/html5/thumbnails/26.jpg)
Semantic Logging Service
Can run as Console Application
Windows Service
Out-Proc Event Listener
Must run on the same machine as source
Xml based configurationsDynamically updated
No restart required
Available as a separate download
Increased fault tolerance for Application crashes
Can monitor Event Providers from multiple processes
![Page 27: Event Source API & Semantic Logging (Talk at NYC Code camp Sep 2014)](https://reader033.fdocuments.in/reader033/viewer/2022052411/55786496d8b42a4c748b4fca/html5/thumbnails/27.jpg)
SLAB Customizations
Custom Text formatters
Custom Sinks
Rx In-Proc filtering
Console color and level mapping
![Page 28: Event Source API & Semantic Logging (Talk at NYC Code camp Sep 2014)](https://reader033.fdocuments.in/reader033/viewer/2022052411/55786496d8b42a4c748b4fca/html5/thumbnails/28.jpg)
Application Buffer1
ETW Sessions
Semantic Logging Service
S3
S3 Sink
SLAB Custom Sinks
![Page 29: Event Source API & Semantic Logging (Talk at NYC Code camp Sep 2014)](https://reader033.fdocuments.in/reader033/viewer/2022052411/55786496d8b42a4c748b4fca/html5/thumbnails/29.jpg)
Demo Using Semantic Logging Service
with Windows Azure Table Storage Sink &
Sql Database Sink
![Page 30: Event Source API & Semantic Logging (Talk at NYC Code camp Sep 2014)](https://reader033.fdocuments.in/reader033/viewer/2022052411/55786496d8b42a4c748b4fca/html5/thumbnails/30.jpg)
ETW Tools
![Page 31: Event Source API & Semantic Logging (Talk at NYC Code camp Sep 2014)](https://reader033.fdocuments.in/reader033/viewer/2022052411/55786496d8b42a4c748b4fca/html5/thumbnails/31.jpg)
ETW Manifests
Installed[Static Registration]
Inline [EventSource][Dynamic Registration] Manifest based Provider
Classic Provider
TraceEvent Package
EventSource & ETW Tools
ETW Tools must incorporate support using Trace Event API
![Page 32: Event Source API & Semantic Logging (Talk at NYC Code camp Sep 2014)](https://reader033.fdocuments.in/reader033/viewer/2022052411/55786496d8b42a4c748b4fca/html5/thumbnails/32.jpg)
WPR
XPerf
PerfView
…..…..…..…..
ETL FilePerf Monitor
LogMan
WPA TraceRpt
![Page 33: Event Source API & Semantic Logging (Talk at NYC Code camp Sep 2014)](https://reader033.fdocuments.in/reader033/viewer/2022052411/55786496d8b42a4c748b4fca/html5/thumbnails/33.jpg)
Performance Monitor [Perfmon.exe]
![Page 34: Event Source API & Semantic Logging (Talk at NYC Code camp Sep 2014)](https://reader033.fdocuments.in/reader033/viewer/2022052411/55786496d8b42a4c748b4fca/html5/thumbnails/34.jpg)
PerfMonitor 2.01Not a next version of PerfMon but a different utility.Also called command line version of PerfViewBased on Trace Event Library
PerfMonitor listSources MyApp.exe PerfMonitor monitor MyApp.exe <args>
Single file deployment
Perfmonitor listSessions
![Page 35: Event Source API & Semantic Logging (Talk at NYC Code camp Sep 2014)](https://reader033.fdocuments.in/reader033/viewer/2022052411/55786496d8b42a4c748b4fca/html5/thumbnails/35.jpg)
PerfView
Based on Trace Event Library
Can run using Command Line
Shows Event Data on User Interface
Vance Morrison’s for WPA support
![Page 36: Event Source API & Semantic Logging (Talk at NYC Code camp Sep 2014)](https://reader033.fdocuments.in/reader033/viewer/2022052411/55786496d8b42a4c748b4fca/html5/thumbnails/36.jpg)
Windows Performance Analyzer
WPR
XPerf
PerfView
…..…..…..…..
ETL File
Collectors
![Page 37: Event Source API & Semantic Logging (Talk at NYC Code camp Sep 2014)](https://reader033.fdocuments.in/reader033/viewer/2022052411/55786496d8b42a4c748b4fca/html5/thumbnails/37.jpg)
Log Manager [LogMan]
![Page 38: Event Source API & Semantic Logging (Talk at NYC Code camp Sep 2014)](https://reader033.fdocuments.in/reader033/viewer/2022052411/55786496d8b42a4c748b4fca/html5/thumbnails/38.jpg)
TraceRpt
- Processing binary ETL files into human readable formats including XML / CSV
- Xml (default) or Html based report for generated events data
- Command line utility
![Page 39: Event Source API & Semantic Logging (Talk at NYC Code camp Sep 2014)](https://reader033.fdocuments.in/reader033/viewer/2022052411/55786496d8b42a4c748b4fca/html5/thumbnails/39.jpg)
Activity Id for Events [Transfer Event]Grouping related events from more than one component
Helps in understanding End-to-End scenarios
Supported in both Manifest based and Classic ETW provider
EventSource
Additional WriteEvent like methods
EventWrittenEventArgs EventListener.OnEventWritten()
.net framework 4.5.1
Correlation between calls and tiers. Can span applications
![Page 40: Event Source API & Semantic Logging (Talk at NYC Code camp Sep 2014)](https://reader033.fdocuments.in/reader033/viewer/2022052411/55786496d8b42a4c748b4fca/html5/thumbnails/40.jpg)
Semantic Logging Service
Application 1
Application 2
Application 3
ETW Infrastructure
Event Store
ActivityId = X, RelatedActivityId = A
ActivityId = X, RelatedActivityId = C
ActivityId = X, RelatedActivityId = B
![Page 41: Event Source API & Semantic Logging (Talk at NYC Code camp Sep 2014)](https://reader033.fdocuments.in/reader033/viewer/2022052411/55786496d8b42a4c748b4fca/html5/thumbnails/41.jpg)
EventSource & Windows Event Log
EventSource doesn’t support Windows Event Log by default
No Channel support!
Microsoft.Diagnostics.Tracing.EventSource (Standalone New definition)Provides
Compile time checks
Supports .net framework 4.0
![Page 42: Event Source API & Semantic Logging (Talk at NYC Code camp Sep 2014)](https://reader033.fdocuments.in/reader033/viewer/2022052411/55786496d8b42a4c748b4fca/html5/thumbnails/42.jpg)
Naming Event Source – Hyphenated Friendly names Consult IT
Consult IT[Channel]
Step I : Implementing EventSource
Step II : Compile the Project
Step III : Registering Provider
Using the Package…
![Page 43: Event Source API & Semantic Logging (Talk at NYC Code camp Sep 2014)](https://reader033.fdocuments.in/reader033/viewer/2022052411/55786496d8b42a4c748b4fca/html5/thumbnails/43.jpg)
Unified Enterprise Logging
![Page 44: Event Source API & Semantic Logging (Talk at NYC Code camp Sep 2014)](https://reader033.fdocuments.in/reader033/viewer/2022052411/55786496d8b42a4c748b4fca/html5/thumbnails/44.jpg)
Questions ???
http://shujaat.net
@SiddiqiMuhammad