Zabbix 3.0 Training Certified Specialist - xiaotonghz.com.cn · Zabbix 3.0 Training Certified...

WWW.YOURCOMPANY.COM

Zabbix 3.0 Training

Certified Specialist

Day 2

The Enterprise class Monitoring Solution for Everyone

WWW.ZABBIX.COM

Advanced monitoring Visualization

Events andnotifications

Zabbixadministration

AGENDA

ZABBIX 3.0 Certified SpecialistWWW.ZABBIX.COM 2

Configuration management

Host maintenance

CONFIGURATIONMANAGEMENT

Template_MySQLQueries per second

MySQL_serverQueries per second

Average queries per second forlast 5 minutes exceeds 5000

{MySQL_server:qps.avg(300)}>5000

Average queries per second forlast 5 minutes exceeds 5000

{Template_MySQL:qps.avg(300)}>5000

Graph

LINKING TEMPLATE


Properties:

Name

Groups

Downstream hosts

Upstream templates

Macros

What is affected

Items

Triggers

Graphs

Applications

Templated screens

Discovery rules

Some item and trigger properties can be overridden!

TEMPLATE PROPERTIES AND CONTENTS


Change upstream, changes downstream

Multiple templates can be linked to single host

E-mail serverWeb server

Template_MySQL

Template_Apache Template_Postfix

Template_PgSQL

HOW TEMPLATES WORK


E-mail server 1

E-mail server 2

Template_E-mail server

Template_Apache

Template_PostfixTemplate_PgSQL

E-mail server 3

E-mail server 4

NESTED TEMPLATING


Visible in the list of hosts (first two levels)

Visible in the list of templates

TEMPLATE LINKAGE


Create a "Custom Templates" host group

Create a new "Template Basic" template in it

Copy all items and trigger from host to the template

Create three additional hosts for other trainee's Vms

Add them to "Training servers" host group

Link the template to them

PRACTICAL SETUP


AGENT-LESSCHECKS

Availability and performance of remote services Syntax

net.tcp.service[service,<ip>,<port>]net.tcp.service.perf[service,<ip>,<port>]

Examples

net.tcp.service[ftp]net.tcp.service[ssh,,1022]net.tcp.service.perf[http,,8080]

SERVICE CHECKS


Add an additional item in "Template Basic":"HTTP service availability"

Make sure that the item is receiving data and is shown in a human readable format

PRACTICAL SETUP


ICMPCHECKS

Uses fping Full path in server configuration file Correct suid/permission settings SELinux can prevent Zabbix from running fping

Uses fping defaults (depends on the platform)

Source IP settings

IPv6 supported by fping6 in most distributions

Supported items

icmpping[<target>,<packets>,<interval>,<size>,<timeout>]

icmppingloss[<target>,<packets>,<interval>,<size>,<timeout>]

icmppingsec[<target>,<packets>,<interval>,<size>,<timeout>,<mode>]

CONFIGURING ICMP CHECKS


Add simple item in "Template Basic":"ICMP lost packets"

Add:"Ping loss is too high on <host>" trigger

Use 5 as threshold

Make sure that the item receives data

Use the following command to simulate dropped packets and test the trigger (run it once!):

# iptables -A INPUT -m statistic --mode random --probability 0.1 -j DROP

PRACTICAL SETUP


AGGREGATECHECKS

No agent is required

Syntaxgroupfunc["Host group","Item key",itemfunc,timeperiod]Functions: grpavg, grpmax, grpmin, grpsumItem functions: avg, count, last, max, min, sum

Examples

grpsum["MySQL Servers","vfs.fs.size[/,total]",last,0]grpavg["MySQL Servers","system.cpu.load[,avg1]",last,0]grpavg["MySQL Servers",mysql.qps,avg,5m]grpavg[["Servers A","Servers B"],system.cpu.load,last,0]

AGGREGATE CHECKS


Create "Production cluster" dummy host which will represent your "production HA cluster"

Create "Template Aggregate Check" template

Add aggregate item in "Template Aggregate Check":"Average CPU load in cluster" which calculates average CPU load on all systems from "Training servers" host group

Link "Template Aggregate Check" to "Production cluster" host


PRACTICAL SETUP


CALCULATEDCHECKS

No agent is required Syntax

func(<key>|<hostname:key>,<parameter1>,<parameter2>,...) Examples

100*last("vfs.fs.size[/,free]",0)/last("vfs.fs.size[/,total]",0)

last("net.if.in[eth0,bytes]",0)+last("net.if.out[eth0,bytes]",0)

100*last("net.if.in[eth0,bytes]",0)/(last("net.if.in[eth0,bytes]",0)+last("net.if.out[eth0,bytes]",0))

CALCULATED CHECKS


Add calculated item in "Template Basic":"Total throughput on eth0" (sum of "Incoming traffic on eth0" and "Outgoing traffic on eth0")


PRACTICAL SETUP


SNMPMONITORING

Zabbix supports SNMP v 1, 2c and 3

Community, security level and port per item

User macros supported for SNMP community

SNMP polled by Zabbix pollers

Zabbix comes with number of pre-defined templates for SNMP

SNMP MONITORING


SNMP entries indexed dynamically, text index on another MIB branch Syntax

<data base OID>["index","<index base OID>","<exact match>"]

Example

IF-MIB::ifInOctets[index,IF-MIB::ifDescr,eth0]

DYNAMIC SNMP INDEXES


Configure SNMP protocol on your VM

Add SNMP interface on the host

Link “Template SNMP Device”


PRACTICAL SETUP


LOG FILEMONITORING

Examplelog["/var/log/syslog","error"]

LOG FILE ITEM


Parameter Definition

Type Zabbix agent (active)

Key log[file,<regexp>,<encoding>,<maxlines>,<mode>,<output>]

Value type Log

Update interval

usually 1 sec

MaxLinesPerSecond – configurable per agent and item

Log rotation support●Different key – logrt● "file" becomes regular expression (not path)●More resource intensive, agent has to re-read directory

contents with each check

Log time format● y, M, d, h, m, s – everything else works as a

placeholder●Numeric values only

ADDITIONAL DETAILS


Special history view Mark selected/other Hide/Show selected Add multiple log items, sorted by entry data

VIEWING LOG ITEMS


There is an error in the log:{host:log["/var/log/syslog"].str(ERROR)}=1

There are several errors in the log for last 3 minutes:{host:log["/var/log/syslog",ERROR].count(3m,ERROR,like)}>2

EXAMPLES


log[file,<regexp>,<encoding>,<maxlines>,<mode>,<output>]

Use output parameter in log and logrt items to extract the desired number

Use Numeric type of information to see graphs and write triggers easily

Example:log[/var/log/syslog,"Total processors activated: ([0-9]+)",,,,\1]

ZABBIX 3.0 Certified SpecialistWWW.ZABBIX.COM

LOGFILE CONTENT EXTRACTION

31

log[/path/to/the/file,large result buffer allocation.*Entries: ([0-9]+),,,,\1] → 5948

log[path,([0-9]+) message ([0-9]+),,,,\1] → 55log[path,([0-9]+) message ([0-9]+),,,,\1 and \2] → 55 and 33log[path,([0-9]+) message ([0-9]+),,,,we got \1 and \2] we →got 55 and 33

EXAMPLES


Fr Feb 07 2014 11:07:36.6690 */ Thread Id 1400 (GLEWF) large result bufferallocation - /Length: 437136/Entries: 5948/User: AUser/Form:CFG:ServiceLevelAgreement

Logging 55 message 33

Create errors.log file in /tmp

Add log monitoring item in "Template Basic":

"The errors.log monitoring"

Use "error" string as regular expression for the item

Write "error" strings to the file


PRACTICAL SETUP


INVENTORY

AUTO-COLLECTION

Item configured to put it data in inventory field

ITEM -> INVENTORY FIELD


Inventory must be set to automaticConfiguration Hosts <Host>→ →

Use "Administration General Other" to set default host → →inventory mode

HOST INVENTORY MODE


HOST INVENTORY VIEW


VISUALISATION

Multiple items in a single graph

Customised look

Multiple graph typesNormalStackedPieExploded pie

CUSTOM GRAPHS


Legend, working time, trigger line, percentile lines (left/right)

Automatic,fixed or referencedY axis min/max

Multiple itemson two axis withdifferent colors& draw styles

GRAPH PROPERTIES


"avg" is the default"max" or "min" in the legend

GRAPH ITEM FUNCTION


{{HOST.HOST}:ifAlias[1].last(0)}

Similarly to map labels, only avg(), last(), max() and min() functions with seconds as parameter are supported within this macro in graph names. Value mapping is supported as well.

MACROS IN GRAPH NAMES


MAPS, SCREENS AND SLIDE SHOWS


Configuration for the entities is available under Monitoring

All users in Zabbix (including non-admin users) can create network maps, screens and slide shows

43

SHARING OPTIONS


Maps, screens and slide shows can be private or public:– public ones can be accessed by all users– private ones can be accessed by its owner (creator) and all users the entity has been shared with– permissions are respected

44

Map properties● Name, dimensions● Background image● Icon highlighting● Element marking upon

change● Single problem expanding● Icon label type and location● Icon mapping● Trigger acknowledgment

status displaying● Min trigger severity● URLs

Map elements● Image● Trigger● Host● Host group (or elements)● Map

MAPS


Different icons for different states

Custom URLs & labels

Drag and drop editing

Icon aligning, macro expanding

Map links with labels and different styles

Macros in labels for elements and links

MAP DETAILS


ACKNOWLEDGE STATUS


None Highlight Mark Don't expand All

OK

Disabled

Problem

Recent problem

MAP OPTIONS EFFECT


Multiple triggers can be attached to a single link

Active trigger with highest severity will determine link style

MAP LINK STATUS INDICATORS


Regexps matched against specified inventory field

No match – default icon used

ICON MAPPING


Map hierarchy: submaps

Custom labels showing any useful information:{{HOST.HOST}:system.cpu.load.last(0)} or{Zabbix server:system.cpu.load.last(0)}

Make links same color as background, appear only for problem

Route links by using invisible icons

Use background images

MAP HINTS


Create a new map

Add all hosts to it

Create links between hosts

Add useful information to a host and link label (CPU load, In/Out traffic)

PRACTICAL SETUP


Works like a tableDrag and drop editingRefreshes elements in backgroundDynamic screen elementsMany different elements:

Simple, custom and graph prototypesItem valuesMapsActive problemsOther screensCustom URLsOverview page

...more

SCREENS


Slide showCycles through many screensAny orderingMultiplier refresh in monitoring

SLIDE SHOWS


EVENTS

Event sources: Triggers

On trigger status changeDepends on "Multiple PROBLEM events" option

Network discoveryIf service is discovered/lostIf device is discovered/lost

Auto-registrationWhen previously unknown active agent contacts Zabbix server

InternalItem in "not supported" stateLow-level discovery rule in "not supported" stateTrigger in "unknown" state

HOW ARE EVENTS GENERATED?


Historical viewStatus, duration, action overviewAccess to individual event detailsFilter, export to CSV

HISTORY OF EVENTS


Source and event detailsAction, command and event history

EVENT DETAILS


NOTIFICATIONS AND

ESCALATIONS

Notify usersExecute remote commandsRepeated notifications until problem is resolvedDelayed notifications and remote commandsDifferent messages for acknowledged and unacknowledged problemsEscalations to different user or user groupUnlimited number of escalation stepsProvide information with macros

FUNCTIONALITY


Ways to send out notifications

Email

Jabber

SMSModem support

Ez-Texting

Custom "alert" scripts

MEDIA TYPES


One or more user mediaUsually e-mail, phone number or other identifierActive based on severityActive based on time period

USER MEDIA


Name

Message

Conditions

Operations

TRIGGER ACTIONS


Flexible conditionsHost, host groupTrigger name, severity or valueApplication, time period

Type of calculation

ACTION CONDITIONS


Notifications depend on permissions

Send messageSingle user, groupSingle media, allDefault/custom messageCondition based on acknowledgment

Run remote commandScript on server or agentIPMISSH, telnetGlobal script

ACTION OPERATIONS - TRIGGERS


Operation

User

User group

Script

Message

Command

Zabbix Server

Remote Agent

IPMI

SSH/Telnet

Global script

Remote host

Remote Agent

Remote host

OPERATIONS


Use macros to include in notifications:Technical data like last gathered valueHost inventory details

Useful for integration with ticketing systems{TRIGGER.NSEVERITY}; full list in manual

The {ESC.HISTORY} macro includes full escalation history

USING MACROS


Based on stepsDefault interval can be overridden in operationsIf intervals collide, smallest interval wins

ESCALATIONS


Immediately e-mail administrator

10 minutes send SMS to admin

15 minutes open report at helpdesk system

30 minutes e-email management

1 hour send SMS to management

2 hours restart Apache

6 hours reboot the server

24 hours power cycle whole server room

ESCALATE IT


DETECTION OF

MISCONFIGURATION

Detection of problems in configurationNot supported itemsNot supported low level discovery rulesUnknown triggers

DETECTION OF MISCONFIGURATION


CUSTOMNOTIFICATIONS

On Zabbix server side, add new MediaType having type Script

Script must be located in a directory, specified by AlertScriptsPath directive in zabbix_server.conf

Command line parameters can be defined per each script{ALERT.SENDTO} – Send to{ALERT.SUBJECT} – Subject{ALERT.MESSAGE} – Body

Executed by Zabbix server asthe "zabbix" user

CUSTOM MEDIA TYPES


HOST AND GROUP

LEVEL

MAINTENANCE

Suppress data collection in the maintenance period by choosing maintenance mode

Suppress notifications during the maintenance period by adding action condition

Visually identified in Status of triggers, maps and dashboard

If problem happens during maintenance, Zabbix starts new escalation after maintenance ends

Maintenance does not stop escalations

MAINTENANCE FUNCTIONALITY


Periods must match!

MAINTENANCE IN ACTION


ZABBIX

ADMINISTRATION

GUIHousekeeperIcons / Images

MacrosValue mappingWorking time

Triggers severities and displaying options

GENERAL CONFIGURATION


REGULAR EXPRESSION BUILDER


AUTHENTICATION


Supported mechanisms:InternalLDAPHTTP: anything supported by Apache

Major detailsIs online/last accessLogin blocking:

After unsuccessful loginsSuperadmins can unblock

Access details:GUI accessAPI accessDebug modeStatus

USER LIST


Details:● Alias/username, name,

surname● Password resetting● Group membership● Frontend language and

theme● Auto-login and logout● Refresh● Rows per page● URL after login● Media

Users can set some ofthese in their profiles

USER DETAILS


USER MEDIA TAB


User group listShows user countShows individual members

User group propertiesMembersAll access is group based"Special" groups

USER GROUPS


Permissions can be assigned on user group level only

Three levels:Read-writeRead onlyDeny

PERMISSION ASSIGNING


Can't revoke access for Super Admins

Read/write overrides read only

PERMISSION SYSTEM


Available in Status of Triggers Dashboard Maps Overview Events

Redirects stderr

FRONTEND SCRIPTS


FRONTEND SCRIPTS


Parameter Definition

NameUnique name of the script (<submenu/submenu.../>script)

Type Script or IPMI command

Execute on Script will be executed on Zabbix server or agent

CommandsAll commands to be executed within the scriptSome macros are supported

User/host group User/host group that the script is available

Required host permission

Permission level for the host group (Read or Write)

Enable confirmation

Display a confirmation message before executing the script

Confirmation text

Custom confirmation text for the confirmation popup

Filter

AUDIT LOG


Details for some operations

Verify message contents

See status and errors

AUDIT ACTIONS


Amount of items per categoryShould have received data N seconds/minutes ago, but did notPer Proxy and Details view

QUEUE


QUESTIONS?

The Enterprise class Monitoring Solution for Everyone

WWW.ZABBIX.COM

TIME FOR A BREAK :)

Zabbix 3.0 Training Certified Specialist - xiaotonghz.com.cn · Zabbix 3.0 Training Certified...

Documents

Transcript of Zabbix 3.0 Training Certified Specialist - xiaotonghz.com.cn · Zabbix 3.0 Training Certified...