Event Console Integration - checkmk.comTemplates in PHP (ノಠ益ಠ) ノ彡 No semantics, no...
Transcript of Event Console Integration - checkmk.comTemplates in PHP (ノಠ益ಠ) ノ彡 No semantics, no...
![Page 1: Event Console Integration - checkmk.comTemplates in PHP (ノಠ益ಠ) ノ彡 No semantics, no uniformity* No customizability** *except for checks sharing templates **unless you wrote](https://reader033.fdocuments.in/reader033/viewer/2022052803/5f1f5053e033bd366e390e48/html5/thumbnails/1.jpg)
03.05.2018, Sven PanneCheck_MK Conference #4
Event Console Integration
![Page 2: Event Console Integration - checkmk.comTemplates in PHP (ノಠ益ಠ) ノ彡 No semantics, no uniformity* No customizability** *except for checks sharing templates **unless you wrote](https://reader033.fdocuments.in/reader033/viewer/2022052803/5f1f5053e033bd366e390e48/html5/thumbnails/2.jpg)
Check_MK Conference #42
Two Kinds of Monitoring
Pushing/pulling states
“Classic” Check_MK
Pushing of messages
Independent of state
Event Console
Status-based Event-based
Both kinds are useful!
![Page 3: Event Console Integration - checkmk.comTemplates in PHP (ノಠ益ಠ) ノ彡 No semantics, no uniformity* No customizability** *except for checks sharing templates **unless you wrote](https://reader033.fdocuments.in/reader033/viewer/2022052803/5f1f5053e033bd366e390e48/html5/thumbnails/3.jpg)
Check_MK Conference #43
Design Considerations
High frequency of incoming
messages
Human-manageable amount of
actual events
Not a high-performance
syslog archive
![Page 4: Event Console Integration - checkmk.comTemplates in PHP (ノಠ益ಠ) ノ彡 No semantics, no uniformity* No customizability** *except for checks sharing templates **unless you wrote](https://reader033.fdocuments.in/reader033/viewer/2022052803/5f1f5053e033bd366e390e48/html5/thumbnails/4.jpg)
Check_MK Conference #44
TrapParser
syslogParser
RuleEngine
Actions
SNMP
syslogspool
Notifications
Shell Script
local
EC
Architectural Overview IBasic Pipeline
![Page 5: Event Console Integration - checkmk.comTemplates in PHP (ノಠ益ಠ) ノ彡 No semantics, no uniformity* No customizability** *except for checks sharing templates **unless you wrote](https://reader033.fdocuments.in/reader033/viewer/2022052803/5f1f5053e033bd366e390e48/html5/thumbnails/5.jpg)
Check_MK Conference #45
TrapParser
syslogParser
RuleEngine
Actions
SNMP
syslogspool
History Events
ViewsEventsHistoryDetails
Notifications
Shell Script
Monitoring CoreLivestatus
local
GUI
EC
Architectural Overview IIPersistence & GUI
![Page 6: Event Console Integration - checkmk.comTemplates in PHP (ノಠ益ಠ) ノ彡 No semantics, no uniformity* No customizability** *except for checks sharing templates **unless you wrote](https://reader033.fdocuments.in/reader033/viewer/2022052803/5f1f5053e033bd366e390e48/html5/thumbnails/6.jpg)
Check_MK Conference #46
TrapParser
syslogParser
RuleEngine
Actions
SNMP
syslogspool
History Events
Event Service
ViewsEventsHistoryDetails
Check
Notifications
Shell Script
Monitoring CoreLivestatus
local
GUI
EC
Architectural Overview IIIThere and Back Again
Logwatch
![Page 7: Event Console Integration - checkmk.comTemplates in PHP (ノಠ益ಠ) ノ彡 No semantics, no uniformity* No customizability** *except for checks sharing templates **unless you wrote](https://reader033.fdocuments.in/reader033/viewer/2022052803/5f1f5053e033bd366e390e48/html5/thumbnails/7.jpg)
Check_MK Conference #47
TrapParser
syslogParser
RuleEngine
Actions
SNMP
syslogspool
History Events
Event Service
ViewsEventsHistoryDetails
Check
Notifications
Shell Script
Monitoring CoreLivestatus
local
GUI
ECHosts,Down-times
Architectural Overview IVHost Information
Logwatch
![Page 8: Event Console Integration - checkmk.comTemplates in PHP (ノಠ益ಠ) ノ彡 No semantics, no uniformity* No customizability** *except for checks sharing templates **unless you wrote](https://reader033.fdocuments.in/reader033/viewer/2022052803/5f1f5053e033bd366e390e48/html5/thumbnails/8.jpg)
Check_MK Conference #48
Events are useful,even during host downtimes
Three chunks yet to be completed
Still generated, but marked specially
Automatically archived after downtime
Tactical overview considers them “handled”
Actions can be skipped
![Page 9: Event Console Integration - checkmk.comTemplates in PHP (ノಠ益ಠ) ノ彡 No semantics, no uniformity* No customizability** *except for checks sharing templates **unless you wrote](https://reader033.fdocuments.in/reader033/viewer/2022052803/5f1f5053e033bd366e390e48/html5/thumbnails/9.jpg)
Check_MK Conference #49
Further Improvements
Finer configuration of limits
More powerful rewriting
Export of rule packs via MKPs
![Page 10: Event Console Integration - checkmk.comTemplates in PHP (ノಠ益ಠ) ノ彡 No semantics, no uniformity* No customizability** *except for checks sharing templates **unless you wrote](https://reader033.fdocuments.in/reader033/viewer/2022052803/5f1f5053e033bd366e390e48/html5/thumbnails/10.jpg)
![Page 11: Event Console Integration - checkmk.comTemplates in PHP (ノಠ益ಠ) ノ彡 No semantics, no uniformity* No customizability** *except for checks sharing templates **unless you wrote](https://reader033.fdocuments.in/reader033/viewer/2022052803/5f1f5053e033bd366e390e48/html5/thumbnails/11.jpg)
03.05.2018, Tom BärwinkelCheck_MK Conference #4
Background jobs
![Page 12: Event Console Integration - checkmk.comTemplates in PHP (ノಠ益ಠ) ノ彡 No semantics, no uniformity* No customizability** *except for checks sharing templates **unless you wrote](https://reader033.fdocuments.in/reader033/viewer/2022052803/5f1f5053e033bd366e390e48/html5/thumbnails/12.jpg)
Check_MK Conference #412
Examples of long running processes
Host renamingBake AgentsCreate Report
![Page 13: Event Console Integration - checkmk.comTemplates in PHP (ノಠ益ಠ) ノ彡 No semantics, no uniformity* No customizability** *except for checks sharing templates **unless you wrote](https://reader033.fdocuments.in/reader033/viewer/2022052803/5f1f5053e033bd366e390e48/html5/thumbnails/13.jpg)
Check_MK Conference #413
Problem – Apache process timeouts
![Page 14: Event Console Integration - checkmk.comTemplates in PHP (ノಠ益ಠ) ノ彡 No semantics, no uniformity* No customizability** *except for checks sharing templates **unless you wrote](https://reader033.fdocuments.in/reader033/viewer/2022052803/5f1f5053e033bd366e390e48/html5/thumbnails/14.jpg)
Check_MK Conference #414
The usual workaround
locate files increase timeout restart
Let’s just increase the timeout
update
![Page 15: Event Console Integration - checkmk.comTemplates in PHP (ノಠ益ಠ) ノ彡 No semantics, no uniformity* No customizability** *except for checks sharing templates **unless you wrote](https://reader033.fdocuments.in/reader033/viewer/2022052803/5f1f5053e033bd366e390e48/html5/thumbnails/15.jpg)
Check_MK Conference #415
Still a problem - What’s the status?
+ +No userfeedback
Blockedprocesses
Apacherestarts
![Page 16: Event Console Integration - checkmk.comTemplates in PHP (ノಠ益ಠ) ノ彡 No semantics, no uniformity* No customizability** *except for checks sharing templates **unless you wrote](https://reader033.fdocuments.in/reader033/viewer/2022052803/5f1f5053e033bd366e390e48/html5/thumbnails/16.jpg)
Check_MK Conference #416
Solution - separation of concerns
CommonInterface
GUI
configuration start jobs show progress
Background jobs
do actual work send updates
![Page 17: Event Console Integration - checkmk.comTemplates in PHP (ノಠ益ಠ) ノ彡 No semantics, no uniformity* No customizability** *except for checks sharing templates **unless you wrote](https://reader033.fdocuments.in/reader033/viewer/2022052803/5f1f5053e033bd366e390e48/html5/thumbnails/17.jpg)
Check_MK Conference #417
How it looks in practice - Reports
![Page 18: Event Console Integration - checkmk.comTemplates in PHP (ノಠ益ಠ) ノ彡 No semantics, no uniformity* No customizability** *except for checks sharing templates **unless you wrote](https://reader033.fdocuments.in/reader033/viewer/2022052803/5f1f5053e033bd366e390e48/html5/thumbnails/18.jpg)
Check_MK Conference #418
How it looks in practice - Reports
![Page 19: Event Console Integration - checkmk.comTemplates in PHP (ノಠ益ಠ) ノ彡 No semantics, no uniformity* No customizability** *except for checks sharing templates **unless you wrote](https://reader033.fdocuments.in/reader033/viewer/2022052803/5f1f5053e033bd366e390e48/html5/thumbnails/19.jpg)
Check_MK Conference #419
The Background jobs overview
![Page 20: Event Console Integration - checkmk.comTemplates in PHP (ノಠ益ಠ) ノ彡 No semantics, no uniformity* No customizability** *except for checks sharing templates **unless you wrote](https://reader033.fdocuments.in/reader033/viewer/2022052803/5f1f5053e033bd366e390e48/html5/thumbnails/20.jpg)
Check_MK Conference #420
Special rules for critical jobs
Affected jobs: bake agents rename host
Only one job allowed Error page with details Require acknowledgement
![Page 21: Event Console Integration - checkmk.comTemplates in PHP (ノಠ益ಠ) ノ彡 No semantics, no uniformity* No customizability** *except for checks sharing templates **unless you wrote](https://reader033.fdocuments.in/reader033/viewer/2022052803/5f1f5053e033bd366e390e48/html5/thumbnails/21.jpg)
Upcoming background jobs
Download of agent output
Service discovery
SLA calculations
![Page 22: Event Console Integration - checkmk.comTemplates in PHP (ノಠ益ಠ) ノ彡 No semantics, no uniformity* No customizability** *except for checks sharing templates **unless you wrote](https://reader033.fdocuments.in/reader033/viewer/2022052803/5f1f5053e033bd366e390e48/html5/thumbnails/22.jpg)
![Page 23: Event Console Integration - checkmk.comTemplates in PHP (ノಠ益ಠ) ノ彡 No semantics, no uniformity* No customizability** *except for checks sharing templates **unless you wrote](https://reader033.fdocuments.in/reader033/viewer/2022052803/5f1f5053e033bd366e390e48/html5/thumbnails/23.jpg)
03.05.2018, Konstantin BüttnerCheck_MK Conference #4
Time-specific check parameters
![Page 24: Event Console Integration - checkmk.comTemplates in PHP (ノಠ益ಠ) ノ彡 No semantics, no uniformity* No customizability** *except for checks sharing templates **unless you wrote](https://reader033.fdocuments.in/reader033/viewer/2022052803/5f1f5053e033bd366e390e48/html5/thumbnails/24.jpg)
Check_MK Conference #424
Use case● Recurring events lead to conditions that would be abnormal during other times
● Eg. backup jobs etc.
![Page 25: Event Console Integration - checkmk.comTemplates in PHP (ノಠ益ಠ) ノ彡 No semantics, no uniformity* No customizability** *except for checks sharing templates **unless you wrote](https://reader033.fdocuments.in/reader033/viewer/2022052803/5f1f5053e033bd366e390e48/html5/thumbnails/25.jpg)
Check_MK Conference #425
Predictive Levels?
![Page 26: Event Console Integration - checkmk.comTemplates in PHP (ノಠ益ಠ) ノ彡 No semantics, no uniformity* No customizability** *except for checks sharing templates **unless you wrote](https://reader033.fdocuments.in/reader033/viewer/2022052803/5f1f5053e033bd366e390e48/html5/thumbnails/26.jpg)
Check_MK Conference #426
Solution – Timespecific parameters● Define parameters for time periods
● Generic solution – works for all check plugins
![Page 27: Event Console Integration - checkmk.comTemplates in PHP (ノಠ益ಠ) ノ彡 No semantics, no uniformity* No customizability** *except for checks sharing templates **unless you wrote](https://reader033.fdocuments.in/reader033/viewer/2022052803/5f1f5053e033bd366e390e48/html5/thumbnails/27.jpg)
Check_MK Conference #427
How does it work?
![Page 28: Event Console Integration - checkmk.comTemplates in PHP (ノಠ益ಠ) ノ彡 No semantics, no uniformity* No customizability** *except for checks sharing templates **unless you wrote](https://reader033.fdocuments.in/reader033/viewer/2022052803/5f1f5053e033bd366e390e48/html5/thumbnails/28.jpg)
Check_MK Conference #428
How does it work?
![Page 29: Event Console Integration - checkmk.comTemplates in PHP (ノಠ益ಠ) ノ彡 No semantics, no uniformity* No customizability** *except for checks sharing templates **unless you wrote](https://reader033.fdocuments.in/reader033/viewer/2022052803/5f1f5053e033bd366e390e48/html5/thumbnails/29.jpg)
Check_MK Conference #429
![Page 30: Event Console Integration - checkmk.comTemplates in PHP (ノಠ益ಠ) ノ彡 No semantics, no uniformity* No customizability** *except for checks sharing templates **unless you wrote](https://reader033.fdocuments.in/reader033/viewer/2022052803/5f1f5053e033bd366e390e48/html5/thumbnails/30.jpg)
![Page 31: Event Console Integration - checkmk.comTemplates in PHP (ノಠ益ಠ) ノ彡 No semantics, no uniformity* No customizability** *except for checks sharing templates **unless you wrote](https://reader033.fdocuments.in/reader033/viewer/2022052803/5f1f5053e033bd366e390e48/html5/thumbnails/31.jpg)
03.05.2018, Konstantin BüttnerCheck_MK Conference #4
Customizable graph layouts
![Page 32: Event Console Integration - checkmk.comTemplates in PHP (ノಠ益ಠ) ノ彡 No semantics, no uniformity* No customizability** *except for checks sharing templates **unless you wrote](https://reader033.fdocuments.in/reader033/viewer/2022052803/5f1f5053e033bd366e390e48/html5/thumbnails/32.jpg)
Check_MK Conference #432
Our objective in graphing
● Traditional graphing system are for customizing graphs
● But: day-to-day, we’d rather just have graphsTweaking graphs is fun, but time consuming
● Our objective: Make our data useful out of the box
![Page 33: Event Console Integration - checkmk.comTemplates in PHP (ノಠ益ಠ) ノ彡 No semantics, no uniformity* No customizability** *except for checks sharing templates **unless you wrote](https://reader033.fdocuments.in/reader033/viewer/2022052803/5f1f5053e033bd366e390e48/html5/thumbnails/33.jpg)
Check_MK Conference #433
A bit of history - PNP4Nagios
PNP graph In the Olden Days
Graphs were defined for each check type Templates in PHP ( ノ ಠ益 ಠ) ノ彡┻━┻ No semantics, no uniformity* No customizability**
*except for checks sharing templates**unless you wrote your own templates
![Page 34: Event Console Integration - checkmk.comTemplates in PHP (ノಠ益ಠ) ノ彡 No semantics, no uniformity* No customizability** *except for checks sharing templates **unless you wrote](https://reader033.fdocuments.in/reader033/viewer/2022052803/5f1f5053e033bd366e390e48/html5/thumbnails/34.jpg)
Check_MK Conference #434
A bit of history - Metrics
Metrics graph A new dawn
Most visible: New look Most important: Semantics Checks no longer have graphs… ...but values have semantics
→Automatic graphs→Custom graphs
![Page 35: Event Console Integration - checkmk.comTemplates in PHP (ノಠ益ಠ) ノ彡 No semantics, no uniformity* No customizability** *except for checks sharing templates **unless you wrote](https://reader033.fdocuments.in/reader033/viewer/2022052803/5f1f5053e033bd366e390e48/html5/thumbnails/35.jpg)
Check_MK Conference #435
Now: More customizability
● Unified options for dashlets, views, reports(Werk #5569)
![Page 36: Event Console Integration - checkmk.comTemplates in PHP (ノಠ益ಠ) ノ彡 No semantics, no uniformity* No customizability** *except for checks sharing templates **unless you wrote](https://reader033.fdocuments.in/reader033/viewer/2022052803/5f1f5053e033bd366e390e48/html5/thumbnails/36.jpg)
Check_MK Conference #436
![Page 37: Event Console Integration - checkmk.comTemplates in PHP (ノಠ益ಠ) ノ彡 No semantics, no uniformity* No customizability** *except for checks sharing templates **unless you wrote](https://reader033.fdocuments.in/reader033/viewer/2022052803/5f1f5053e033bd366e390e48/html5/thumbnails/37.jpg)
Check_MK Conference #437
![Page 38: Event Console Integration - checkmk.comTemplates in PHP (ノಠ益ಠ) ノ彡 No semantics, no uniformity* No customizability** *except for checks sharing templates **unless you wrote](https://reader033.fdocuments.in/reader033/viewer/2022052803/5f1f5053e033bd366e390e48/html5/thumbnails/38.jpg)
Check_MK Conference #438
Now: More customizability
● Unified options for dashlets, views, reports(Werk #5569)
● Vertical scaling and mirroring customizable(Werk #3920)
![Page 39: Event Console Integration - checkmk.comTemplates in PHP (ノಠ益ಠ) ノ彡 No semantics, no uniformity* No customizability** *except for checks sharing templates **unless you wrote](https://reader033.fdocuments.in/reader033/viewer/2022052803/5f1f5053e033bd366e390e48/html5/thumbnails/39.jpg)
Check_MK Conference #439
![Page 40: Event Console Integration - checkmk.comTemplates in PHP (ノಠ益ಠ) ノ彡 No semantics, no uniformity* No customizability** *except for checks sharing templates **unless you wrote](https://reader033.fdocuments.in/reader033/viewer/2022052803/5f1f5053e033bd366e390e48/html5/thumbnails/40.jpg)
Check_MK Conference #440
![Page 41: Event Console Integration - checkmk.comTemplates in PHP (ノಠ益ಠ) ノ彡 No semantics, no uniformity* No customizability** *except for checks sharing templates **unless you wrote](https://reader033.fdocuments.in/reader033/viewer/2022052803/5f1f5053e033bd366e390e48/html5/thumbnails/41.jpg)
![Page 42: Event Console Integration - checkmk.comTemplates in PHP (ノಠ益ಠ) ノ彡 No semantics, no uniformity* No customizability** *except for checks sharing templates **unless you wrote](https://reader033.fdocuments.in/reader033/viewer/2022052803/5f1f5053e033bd366e390e48/html5/thumbnails/42.jpg)
03.05.2018, Konstantin BüttnerCheck_MK Conference #4
Performance improvements
![Page 43: Event Console Integration - checkmk.comTemplates in PHP (ノಠ益ಠ) ノ彡 No semantics, no uniformity* No customizability** *except for checks sharing templates **unless you wrote](https://reader033.fdocuments.in/reader033/viewer/2022052803/5f1f5053e033bd366e390e48/html5/thumbnails/43.jpg)
Check_MK Conference #443
Why is a performance focus critical?
● The typical Check_MK setup keeps growing – more services, more sites, more users
● Better scalability makes new things possible
![Page 44: Event Console Integration - checkmk.comTemplates in PHP (ノಠ益ಠ) ノ彡 No semantics, no uniformity* No customizability** *except for checks sharing templates **unless you wrote](https://reader033.fdocuments.in/reader033/viewer/2022052803/5f1f5053e033bd366e390e48/html5/thumbnails/44.jpg)
How to make things go faster
Avoid unnecessary work
Optimize
Make things feel faster
![Page 45: Event Console Integration - checkmk.comTemplates in PHP (ノಠ益ಠ) ノ彡 No semantics, no uniformity* No customizability** *except for checks sharing templates **unless you wrote](https://reader033.fdocuments.in/reader033/viewer/2022052803/5f1f5053e033bd366e390e48/html5/thumbnails/45.jpg)
Check_MK Conference #445
Avoid unnecessary work - Tabs● A typical browser: 50 open tabs
● You don’t use them all… but they all make regular livestatus queries, though
● New: Only update active tabs/windows (Werk #4753)
● Benefit: No more wasted data transfers
![Page 46: Event Console Integration - checkmk.comTemplates in PHP (ノಠ益ಠ) ノ彡 No semantics, no uniformity* No customizability** *except for checks sharing templates **unless you wrote](https://reader033.fdocuments.in/reader033/viewer/2022052803/5f1f5053e033bd366e390e48/html5/thumbnails/46.jpg)
Check_MK Conference #446
Avoid unnecessary work – Emails● Graphs can make emails reasonably large
● Multiply that by the number of recipients ...
● What’s a mail server for? (Werk #4813)
● Benefit: Notification system becomes more scalable
![Page 47: Event Console Integration - checkmk.comTemplates in PHP (ノಠ益ಠ) ノ彡 No semantics, no uniformity* No customizability** *except for checks sharing templates **unless you wrote](https://reader033.fdocuments.in/reader033/viewer/2022052803/5f1f5053e033bd366e390e48/html5/thumbnails/47.jpg)
Check_MK Conference #447
Avoid unnecessary work – Sites
● Frequent scenario: multi-site setup
● For many users, only few sites are actually relevant
● Yet, GUI in the past contacted all sites for every user
● No more: Now configureable for each user (Werk #4921)
![Page 48: Event Console Integration - checkmk.comTemplates in PHP (ノಠ益ಠ) ノ彡 No semantics, no uniformity* No customizability** *except for checks sharing templates **unless you wrote](https://reader033.fdocuments.in/reader033/viewer/2022052803/5f1f5053e033bd366e390e48/html5/thumbnails/48.jpg)
Check_MK Conference #448
Optimize – BI● Large aggregations now compile faster (Werk #5142)
● Magic - “it’s the algorithm, stupid”
● Further reworking already planned
![Page 49: Event Console Integration - checkmk.comTemplates in PHP (ノಠ益ಠ) ノ彡 No semantics, no uniformity* No customizability** *except for checks sharing templates **unless you wrote](https://reader033.fdocuments.in/reader033/viewer/2022052803/5f1f5053e033bd366e390e48/html5/thumbnails/49.jpg)
Check_MK Conference #449
Optimize – liveproxyd● Previously: # The main loop of the daemon goes here
● One python process – GIL limits scalability
● Now what?
![Page 50: Event Console Integration - checkmk.comTemplates in PHP (ノಠ益ಠ) ノ彡 No semantics, no uniformity* No customizability** *except for checks sharing templates **unless you wrote](https://reader033.fdocuments.in/reader033/viewer/2022052803/5f1f5053e033bd366e390e48/html5/thumbnails/50.jpg)
Check_MK Conference #450
Optimize – liveproxyd● Now: One process per site, master process to manage● Process view:
● More resilient, more performant through use of multiple CPUs (cf. Werk #4901)
![Page 51: Event Console Integration - checkmk.comTemplates in PHP (ノಠ益ಠ) ノ彡 No semantics, no uniformity* No customizability** *except for checks sharing templates **unless you wrote](https://reader033.fdocuments.in/reader033/viewer/2022052803/5f1f5053e033bd366e390e48/html5/thumbnails/51.jpg)
Check_MK Conference #451
Make things feel faster - Graphs● Previously: View with many Graphs? Better wait for all the data
● Now: Load the view with placeholders, update graphs asynchronously
![Page 52: Event Console Integration - checkmk.comTemplates in PHP (ノಠ益ಠ) ノ彡 No semantics, no uniformity* No customizability** *except for checks sharing templates **unless you wrote](https://reader033.fdocuments.in/reader033/viewer/2022052803/5f1f5053e033bd366e390e48/html5/thumbnails/52.jpg)
Take-home message
Attention to detail pays off
Continuously improving
If you have lots of sites and lots of users, Werk #4921 may help you
![Page 53: Event Console Integration - checkmk.comTemplates in PHP (ノಠ益ಠ) ノ彡 No semantics, no uniformity* No customizability** *except for checks sharing templates **unless you wrote](https://reader033.fdocuments.in/reader033/viewer/2022052803/5f1f5053e033bd366e390e48/html5/thumbnails/53.jpg)