Alain Ganuchaud - Reporting Large Environment Zabbix Database
-
Upload
zabbix -
Category
Technology
-
view
898 -
download
4
Transcript of Alain Ganuchaud - Reporting Large Environment Zabbix Database
![Page 1: Alain Ganuchaud - Reporting Large Environment Zabbix Database](https://reader031.fdocuments.in/reader031/viewer/2022030312/58ed5d0f1a28abac1a8b468b/html5/thumbnails/1.jpg)
Reporting Large Environment Zabbix Database
2015 Sept
![Page 3: Alain Ganuchaud - Reporting Large Environment Zabbix Database](https://reader031.fdocuments.in/reader031/viewer/2022030312/58ed5d0f1a28abac1a8b468b/html5/thumbnails/3.jpg)
Content
1. Reporting Components2. Reporting Architecture for large environment3. Reports examples built with Jasper Suite
Page 3
![Page 4: Alain Ganuchaud - Reporting Large Environment Zabbix Database](https://reader031.fdocuments.in/reader031/viewer/2022030312/58ed5d0f1a28abac1a8b468b/html5/thumbnails/4.jpg)
Summary
Part 1Components
-> SQLs & Zabbix Database Model
-> APIs
-> Jasper Suite
Part 2ReportingArchitecture
-> Warehouse Database
-> Reporting Architecture
Part 3Reports
-> Zabbix Configuration Checker
-> Manager Dashboard
-> Yearly Trends
Page 4
![Page 5: Alain Ganuchaud - Reporting Large Environment Zabbix Database](https://reader031.fdocuments.in/reader031/viewer/2022030312/58ed5d0f1a28abac1a8b468b/html5/thumbnails/5.jpg)
Introduction
Page 5
-> What are we talking about ?
-> Large Environment
-> Problem
-> Example : Zabbix Configuration Checker
![Page 6: Alain Ganuchaud - Reporting Large Environment Zabbix Database](https://reader031.fdocuments.in/reader031/viewer/2022030312/58ed5d0f1a28abac1a8b468b/html5/thumbnails/6.jpg)
What are we talking about ?
Reporting→ Extract essential Zabbix information to make better decisions by viewing consolidated data→ Presentation layer must be sexy
Data WareHouse→ Consolidate Zabbix data with some enrichment (teams, cmdb infos, …)→ Read Only Database <> Zabbix Database
BI→ retrieve, analyze, transform and report data→ Jasper Suite (iReport & Jasper Server)
Page 6
![Page 7: Alain Ganuchaud - Reporting Large Environment Zabbix Database](https://reader031.fdocuments.in/reader031/viewer/2022030312/58ed5d0f1a28abac1a8b468b/html5/thumbnails/7.jpg)
What is a Large Environment ?
• More than 14 000 servers , 1 million items
• Some servers with over 10 000 items• Some servers with over 10 000 triggers• More than 100 Zabbix administrators
developping Application Templates
Page 7
![Page 8: Alain Ganuchaud - Reporting Large Environment Zabbix Database](https://reader031.fdocuments.in/reader031/viewer/2022030312/58ed5d0f1a28abac1a8b468b/html5/thumbnails/8.jpg)
The Problem
In such huge environment, Zabbix (through the frontend) gives back too much details and you loose High level visibility (In French, we say « The tree that hides the forest ».
Reporting can help in this area, unfortunately Zabbix has no reporting facility.→ You have to develop your own Reporting
Architecture to get a consolidated view on data.
Page 8
![Page 9: Alain Ganuchaud - Reporting Large Environment Zabbix Database](https://reader031.fdocuments.in/reader031/viewer/2022030312/58ed5d0f1a28abac1a8b468b/html5/thumbnails/9.jpg)
Example : Zabbix Configuration Checker
• In the following slides, I will take the example of Reports that are dispatched to the teams who are responsible for an application.• Those teams are in charge of Zabbix templates development, as they are not Zabbix experts, they do mistakes.•Zabbix configuration Checker help those teams to correct those templates by sending Reports about bad or poor Zabbix Templates configurations.
Page 9
![Page 10: Alain Ganuchaud - Reporting Large Environment Zabbix Database](https://reader031.fdocuments.in/reader031/viewer/2022030312/58ed5d0f1a28abac1a8b468b/html5/thumbnails/10.jpg)
Part 1Reporting Components
Page 10
-> Where is the information to report ?
-> Zabbix database schema
-> SQL vs API
-> BI : Jasper Suite
![Page 11: Alain Ganuchaud - Reporting Large Environment Zabbix Database](https://reader031.fdocuments.in/reader031/viewer/2022030312/58ed5d0f1a28abac1a8b468b/html5/thumbnails/11.jpg)
Where is the Information ?
In Zabbix Database• Easy … se lect hos t from hos ts where …In Cache
• Glurps … APIs ?
→ Normally Reporting Data are meta data, we do not care about details or data in cache !
Page 11
![Page 12: Alain Ganuchaud - Reporting Large Environment Zabbix Database](https://reader031.fdocuments.in/reader031/viewer/2022030312/58ed5d0f1a28abac1a8b468b/html5/thumbnails/12.jpg)
Zabbix Database schema
Page 12
Know the D
atabase schema
and follo
w the changes
![Page 13: Alain Ganuchaud - Reporting Large Environment Zabbix Database](https://reader031.fdocuments.in/reader031/viewer/2022030312/58ed5d0f1a28abac1a8b468b/html5/thumbnails/13.jpg)
Zabbix Database schema
Page 13
![Page 14: Alain Ganuchaud - Reporting Large Environment Zabbix Database](https://reader031.fdocuments.in/reader031/viewer/2022030312/58ed5d0f1a28abac1a8b468b/html5/thumbnails/14.jpg)
Zabbix Database schema
Page 14
![Page 15: Alain Ganuchaud - Reporting Large Environment Zabbix Database](https://reader031.fdocuments.in/reader031/viewer/2022030312/58ed5d0f1a28abac1a8b468b/html5/thumbnails/15.jpg)
Zabbix Database schema
Page 15
![Page 16: Alain Ganuchaud - Reporting Large Environment Zabbix Database](https://reader031.fdocuments.in/reader031/viewer/2022030312/58ed5d0f1a28abac1a8b468b/html5/thumbnails/16.jpg)
Zabbix Database schema
Page 16
Nota Bene : No trend.get API
![Page 17: Alain Ganuchaud - Reporting Large Environment Zabbix Database](https://reader031.fdocuments.in/reader031/viewer/2022030312/58ed5d0f1a28abac1a8b468b/html5/thumbnails/17.jpg)
Zabbix Database schema
Page 17
Download Zabbix Database Model with comments athttps://share.zabbix.com/databases/mysql/zabbix-database-modelVersions 1.8 , 2.0 , 2.2 & 2.4
![Page 18: Alain Ganuchaud - Reporting Large Environment Zabbix Database](https://reader031.fdocuments.in/reader031/viewer/2022030312/58ed5d0f1a28abac1a8b468b/html5/thumbnails/18.jpg)
APIs
{ "jsonrpc": "2.0", "method": "host.get", "params": { "output": "extend", "filter": { "host": [ "Zabbix server", "Linux server" ] } },
"auth": "038e1d7b1735c6a5436ee9eae095879e", "id": 1}
Page 18
Ignore the Zabbix
Database schema …
some APIs are m
issing
ie trend.get
![Page 19: Alain Ganuchaud - Reporting Large Environment Zabbix Database](https://reader031.fdocuments.in/reader031/viewer/2022030312/58ed5d0f1a28abac1a8b468b/html5/thumbnails/19.jpg)
API vs SQL
Normally API requests are safe and are compatible with Zabbix Database schema, but in Large environment they are slow … for example triggers table size could be about 1Gb.
That's why, most of the time I prefer SQL requests on a Zabbix replicated Database, thus there no IOs generated on Zabbix Production Platform.
It implies you know and follow Zabbix Database Schema changes.
Page 19
![Page 20: Alain Ganuchaud - Reporting Large Environment Zabbix Database](https://reader031.fdocuments.in/reader031/viewer/2022030312/58ed5d0f1a28abac1a8b468b/html5/thumbnails/20.jpg)
BI : Jasper Suite
Page 20
![Page 21: Alain Ganuchaud - Reporting Large Environment Zabbix Database](https://reader031.fdocuments.in/reader031/viewer/2022030312/58ed5d0f1a28abac1a8b468b/html5/thumbnails/21.jpg)
Part 2Reporting Architecture
Page 21
-> Monitoring Data WareHouse
-> iReport / JasperStudio
-> Jasper Server
![Page 22: Alain Ganuchaud - Reporting Large Environment Zabbix Database](https://reader031.fdocuments.in/reader031/viewer/2022030312/58ed5d0f1a28abac1a8b468b/html5/thumbnails/22.jpg)
Page 22
Building DataWareHouse
![Page 23: Alain Ganuchaud - Reporting Large Environment Zabbix Database](https://reader031.fdocuments.in/reader031/viewer/2022030312/58ed5d0f1a28abac1a8b468b/html5/thumbnails/23.jpg)
Page 23
Building DataWareHouseZabbix Configuration Checker
+-------------------------------------+| Tables_in_zabbix_warehouse +-------------------------------------+| agent_unavailable | bad_nomenclatures | distrib_by_environment | distrib_by_hostgroup | distrib_by_os | distrib_by_regularity | distrib_by_status | hosts_low_discovery_items | hosts_low_discovery_triggers | hosts_per_template | hosts_with_not_templated_items | hosts_with_numerous_events | hosts_with_numerous_events_type_log | hosts_with_old_triggers | hosts_with_unsupported | hosts_without_triggers | items_log_without_skip_parameter | teams | templates_with_low_disc_regularity | web_with_low_regularity ………………..
mysql> show create table distrib_by_regularity;+----------------------------+----------------------------------------------| Table | Create Table +----------------------------+--------------------------------------------- distrib_by_regularity CREATE TABLE `distrib_by_regularity` ( `team` varchar(64) COLLATE utf8_bin NOT NULL DEFAULT '', `number` int(11) NOT NULL DEFAULT '0', `regularity` int(11) NOT NULL DEFAULT '0') ENGINE=InnoDB DEFAULT CHARSET=utf8 COLLATE=utf8_bin
![Page 24: Alain Ganuchaud - Reporting Large Environment Zabbix Database](https://reader031.fdocuments.in/reader031/viewer/2022030312/58ed5d0f1a28abac1a8b468b/html5/thumbnails/24.jpg)
Page 24
Building DataWareHouseZabbix Configuration Checker
$query_distrib_by_regularity = "select substring(i.name,5,3) as TEAM, i.delay as REGULARITY, count(*) as NUMBER from items i INNER JOIN hosts h ON ( i.hostid = h.hostid ) where ….
![Page 25: Alain Ganuchaud - Reporting Large Environment Zabbix Database](https://reader031.fdocuments.in/reader031/viewer/2022030312/58ed5d0f1a28abac1a8b468b/html5/thumbnails/25.jpg)
Page 25
Enrich DataWareHouseZabbix Configuration Checker
![Page 26: Alain Ganuchaud - Reporting Large Environment Zabbix Database](https://reader031.fdocuments.in/reader031/viewer/2022030312/58ed5d0f1a28abac1a8b468b/html5/thumbnails/26.jpg)
Page 26
iReport / JasperStudio
![Page 27: Alain Ganuchaud - Reporting Large Environment Zabbix Database](https://reader031.fdocuments.in/reader031/viewer/2022030312/58ed5d0f1a28abac1a8b468b/html5/thumbnails/27.jpg)
Page 27
iReport / JasperStudioSubreports
![Page 28: Alain Ganuchaud - Reporting Large Environment Zabbix Database](https://reader031.fdocuments.in/reader031/viewer/2022030312/58ed5d0f1a28abac1a8b468b/html5/thumbnails/28.jpg)
Page 28
iReport / JasperStudioReports
![Page 29: Alain Ganuchaud - Reporting Large Environment Zabbix Database](https://reader031.fdocuments.in/reader031/viewer/2022030312/58ed5d0f1a28abac1a8b468b/html5/thumbnails/29.jpg)
Page 29
Jasper Server
![Page 30: Alain Ganuchaud - Reporting Large Environment Zabbix Database](https://reader031.fdocuments.in/reader031/viewer/2022030312/58ed5d0f1a28abac1a8b468b/html5/thumbnails/30.jpg)
Page 30
Jasper Server
![Page 31: Alain Ganuchaud - Reporting Large Environment Zabbix Database](https://reader031.fdocuments.in/reader031/viewer/2022030312/58ed5d0f1a28abac1a8b468b/html5/thumbnails/31.jpg)
Page 31
Final Reporting Architecture
![Page 32: Alain Ganuchaud - Reporting Large Environment Zabbix Database](https://reader031.fdocuments.in/reader031/viewer/2022030312/58ed5d0f1a28abac1a8b468b/html5/thumbnails/32.jpg)
Part 3Reports
Page 32
-> Zabbix Configuration Checker
-> Summary Report
-> Yearly Trends
![Page 33: Alain Ganuchaud - Reporting Large Environment Zabbix Database](https://reader031.fdocuments.in/reader031/viewer/2022030312/58ed5d0f1a28abac1a8b468b/html5/thumbnails/33.jpg)
Page 33
Zabbix Configuration Checker
![Page 34: Alain Ganuchaud - Reporting Large Environment Zabbix Database](https://reader031.fdocuments.in/reader031/viewer/2022030312/58ed5d0f1a28abac1a8b468b/html5/thumbnails/34.jpg)
Page 34
Zabbix Configuration Checker
![Page 35: Alain Ganuchaud - Reporting Large Environment Zabbix Database](https://reader031.fdocuments.in/reader031/viewer/2022030312/58ed5d0f1a28abac1a8b468b/html5/thumbnails/35.jpg)
Page 35
Zabbix Configuration Checker
![Page 36: Alain Ganuchaud - Reporting Large Environment Zabbix Database](https://reader031.fdocuments.in/reader031/viewer/2022030312/58ed5d0f1a28abac1a8b468b/html5/thumbnails/36.jpg)
Page 36
Summary Report
![Page 37: Alain Ganuchaud - Reporting Large Environment Zabbix Database](https://reader031.fdocuments.in/reader031/viewer/2022030312/58ed5d0f1a28abac1a8b468b/html5/thumbnails/37.jpg)
Page 37
Yearly Trends
![Page 38: Alain Ganuchaud - Reporting Large Environment Zabbix Database](https://reader031.fdocuments.in/reader031/viewer/2022030312/58ed5d0f1a28abac1a8b468b/html5/thumbnails/38.jpg)
Merci pour votre attention !
Alain GanuchaudCore IT [email protected]
Questions?
Page 38
Follow us on Twitter: @zabbix_fr @AlainGanuchaud