TIBCO Messaging - Apache Kafka Distribution - …...About this Product TIBCO® is proud to announce...
Transcript of TIBCO Messaging - Apache Kafka Distribution - …...About this Product TIBCO® is proud to announce...
TIBCO® Messaging - Apache Kafka Distribution- BridgeKafka MonitoringSoftware Release 2.0November 2018
Two-Second Advantage®
Important Information
SOME TIBCO SOFTWARE EMBEDS OR BUNDLES OTHER TIBCO SOFTWARE. USE OF SUCHEMBEDDED OR BUNDLED TIBCO SOFTWARE IS SOLELY TO ENABLE THE FUNCTIONALITY (ORPROVIDE LIMITED ADD-ON FUNCTIONALITY) OF THE LICENSED TIBCO SOFTWARE. THEEMBEDDED OR BUNDLED SOFTWARE IS NOT LICENSED TO BE USED OR ACCESSED BY ANYOTHER TIBCO SOFTWARE OR FOR ANY OTHER PURPOSE.
USE OF TIBCO SOFTWARE AND THIS DOCUMENT IS SUBJECT TO THE TERMS ANDCONDITIONS OF A LICENSE AGREEMENT FOUND IN EITHER A SEPARATELY EXECUTEDSOFTWARE LICENSE AGREEMENT, OR, IF THERE IS NO SUCH SEPARATE AGREEMENT, THECLICKWRAP END USER LICENSE AGREEMENT WHICH IS DISPLAYED DURING DOWNLOADOR INSTALLATION OF THE SOFTWARE (AND WHICH IS DUPLICATED IN THE LICENSE FILE)OR IF THERE IS NO SUCH SOFTWARE LICENSE AGREEMENT OR CLICKWRAP END USERLICENSE AGREEMENT, THE LICENSE(S) LOCATED IN THE “LICENSE” FILE(S) OF THESOFTWARE. USE OF THIS DOCUMENT IS SUBJECT TO THOSE TERMS AND CONDITIONS, ANDYOUR USE HEREOF SHALL CONSTITUTE ACCEPTANCE OF AND AN AGREEMENT TO BEBOUND BY THE SAME.
ANY SOFTWARE ITEM IDENTIFIED AS THIRD PARTY LIBRARY IS AVAILABLE UNDERSEPARATE SOFTWARE LICENSE TERMS AND IS NOT PART OF A TIBCO PRODUCT. AS SUCH,THESE SOFTWARE ITEMS ARE NOT COVERED BY THE TERMS OF YOUR AGREEMENT WITHTIBCO, INCLUDING ANY TERMS CONCERNING SUPPORT, MAINTENANCE, WARRANTIES,AND INDEMNITIES. DOWNLOAD AND USE OF THESE ITEMS IS SOLELY AT YOUR OWNDISCRETION AND SUBJECT TO THE LICENSE TERMS APPLICABLE TO THEM. BY PROCEEDINGTO DOWNLOAD, INSTALL OR USE ANY OF THESE ITEMS, YOU ACKNOWLEDGE THEFOREGOING DISTINCTIONS BETWEEN THESE ITEMS AND TIBCO PRODUCTS.
This document contains confidential information that is subject to U.S. and international copyright lawsand treaties. No part of this document may be reproduced in any form without the writtenauthorization of TIBCO Software Inc.
TIBCO, the TIBCO logo, Two-Second Advantage, TIB, Information Bus, TIBCO Messaging, FTL, andeFTL are either registered trademarks or trademarks of TIBCO Software Inc. in the United States and/orother countries.
Enterprise Java Beans (EJB), Java Platform Enterprise Edition (Java EE), Java 2 Platform EnterpriseEdition (J2EE), and all Java-based trademarks and logos are trademarks or registered trademarks ofOracle Corporation in the U.S. and other countries.
All other product and company names and marks mentioned in this document are the property of theirrespective owners and are mentioned for identification purposes only.
THIS SOFTWARE MAY BE AVAILABLE ON MULTIPLE OPERATING SYSTEMS. HOWEVER, NOTALL OPERATING SYSTEM PLATFORMS FOR A SPECIFIC SOFTWARE VERSION ARE RELEASEDAT THE SAME TIME. SEE THE README FILE FOR THE AVAILABILITY OF THIS SOFTWAREVERSION ON A SPECIFIC OPERATING SYSTEM PLATFORM.
THIS DOCUMENT IS PROVIDED “AS IS” WITHOUT WARRANTY OF ANY KIND, EITHER EXPRESSOR IMPLIED, INCLUDING, BUT NOT LIMITED TO, THE IMPLIED WARRANTIES OFMERCHANTABILITY, FITNESS FOR A PARTICULAR PURPOSE, OR NON-INFRINGEMENT.
THIS DOCUMENT COULD INCLUDE TECHNICAL INACCURACIES OR TYPOGRAPHICALERRORS. CHANGES ARE PERIODICALLY ADDED TO THE INFORMATION HEREIN; THESECHANGES WILL BE INCORPORATED IN NEW EDITIONS OF THIS DOCUMENT. TIBCOSOFTWARE INC. MAY MAKE IMPROVEMENTS AND/OR CHANGES IN THE PRODUCT(S)AND/OR THE PROGRAM(S) DESCRIBED IN THIS DOCUMENT AT ANY TIME.
THE CONTENTS OF THIS DOCUMENT MAY BE MODIFIED AND/OR QUALIFIED, DIRECTLY ORINDIRECTLY, BY OTHER DOCUMENTATION WHICH ACCOMPANIES THIS SOFTWARE,INCLUDING BUT NOT LIMITED TO ANY RELEASE NOTES AND "READ ME" FILES.
2
TIBCO® Messaging - Apache Kafka Distribution - Bridge Kafka Monitoring
Copyright © 2018 TIBCO Software Inc. ALL RIGHTS RESERVED.
TIBCO Software Inc. Confidential Information
3
TIBCO® Messaging - Apache Kafka Distribution - Bridge Kafka Monitoring
Contents
About this Product . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 5
TIBCO Documentation and Support Services . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 6
Apache Kafka Monitoring . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 7
Data Flow of Monitoring Data and Log Messages . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 8
Apache Kafka Monitoring Gateway . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 9
Configuring the Apache Kafka Monitoring Gateway . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 9
Using the Configuration File . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 10
Monitoring Gateway Command Line Reference (monitor-start.py) . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .10
Catalogs of Metrics . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .12
Database and Visualization . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 28
InfluxDB . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .28
Grafana and Dashboard Examples . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .28
Security of Monitoring Data . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 31
4
TIBCO® Messaging - Apache Kafka Distribution - Bridge Kafka Monitoring
About this Product
TIBCO® is proud to announce TIBCO® Messaging - Apache Kafka Distribution - Bridge.
This release is the latest in a long history of TIBCO products that leverage the power of InformationBus® technology to enable truly event-driven IT environments. To find out more about how TIBCO®
Messaging software and other TIBCO products are powered by TIB® technology, please visit us at www.tibco.com.
TIBCO Messaging - Apache Kafka Distribution - Bridge is part of TIBCO® Messaging software.
Product Editions
TIBCO Messaging software is available in a community edition and an enterprise edition.
TIBCO Messaging - Community Edition is ideal for getting started with TIBCO Messaging, forimplementing application projects (including proof of concept efforts), for testing, and for deployingapplications in a production environment. Although the community license limits the number ofproduction processes, you can easily upgrade to the enterprise edition as your use of TIBCO Messagingexpands.
The community edition is available free of charge, with the following limitations and exclusions:
● Users may run up to 100 application instances or 1000 web/mobile instances in a productionenvironment.
● Users do not have access to TIBCO Support, but you can use TIBCO Community as a resource(https://community.tibco.com).
● Users do not have access to the FTL Monitoring component of TIBCO Messaging - Apache KafkaDistribution - Bridge .
TIBCO Messaging - Enterprise Edition is ideal for all application development projects, and fordeploying and managing applications in an enterprise production environment. It includes all featurespresented in this documentation set, as well as access to TIBCO Support.
Bundling
The enterprise edition of TIBCO ActiveSpaces® uses the enterprise edition of TIBCO Messaging andincludes a license for it. The community edition of that related product is compatible with both theenterprise and community editions of TIBCO Messaging.
5
TIBCO® Messaging - Apache Kafka Distribution - Bridge Kafka Monitoring
TIBCO Documentation and Support Services
How to Access TIBCO Documentation
Documentation for TIBCO products is available on the TIBCO Product Documentation website, mainlyin HTML and PDF formats.
The TIBCO Product Documentation website is updated frequently and is more current than any otherdocumentation included with the product. To access the latest documentation, visit https://docs.tibco.com.
Product-Specific Documentation
Documentation for TIBCO products is not bundled with the software. Instead, it is available on theTIBCO Documentation site. To access the documentation web page for this product from a localsoftware installation, open the following file:
TIBCO_HOME/akd/bridge/2.0/product_info/TIB_msg-akd-bridge_2.0.0_docinfo.html
TIBCO_HOME is the top-level directory in which TIBCO products are installed.
● On Windows platforms, the default TIBCO_HOME is C:\tibco.
● On UNIX platforms, the default TIBCO_HOME is /opt/tibco.
The User's Guide in this documentation set is shared with the following products:● TIBCO® Messaging - Apache Kafka Distribution
● TIBCO® Messaging - Apache Kafka Distribution - Bridge
● TIBCO® Messaging - Apache Kafka Distribution - Core
How to Contact TIBCO Support
You can contact TIBCO Support in the following ways:
● For an overview of TIBCO Support, visit http://www.tibco.com/services/support.
● For accessing the Support Knowledge Base and getting personalized content about products you areinterested in, visit the TIBCO Support portal at https://support.tibco.com.
● For creating a Support case, you must have a valid maintenance or support contract with TIBCO.You also need a user name and password to log in to https://support.tibco.com. If you do not have auser name, you can request one by clicking Register on the website.
How to Join TIBCO Community
TIBCO Community is the official channel for TIBCO customers, partners, and employee subject matterexperts to share and access their collective experience. TIBCO Community offers access to Q&A forums,product wikis, and best practices. It also offers access to extensions, adapters, solution accelerators, andtools that extend and enable customers to gain full value from TIBCO products. In addition, users cansubmit and vote on feature requests from within the TIBCO Ideas Portal. For a free registration, go to https://community.tibco.com.
6
TIBCO® Messaging - Apache Kafka Distribution - Bridge Kafka Monitoring
Apache Kafka Monitoring
The tibkmongateway process polls a configured set of Apache Kafka Brokers and saves the polledinformation using InfluxDB for display and analysis using customizable Grafana-based dashboards.
For ease of configuration, a .csv (comma-separated value) file is used to configure the monitoringbehavior allowing easy configuration for each Apache Kafka Broker including host and port, securitysettings, and other broker-specific settings.
Scope
The TIBCO FTL library gathers monitoring data for the following entities:
● Apache Kafka Brokers
● Apache Kafka Clients
● Kafka-related Apache ZooKeeper information
Sampling Rate
Data is sampled from each configured Apache Kafka Broker every 60 seconds.
7
TIBCO® Messaging - Apache Kafka Distribution - Bridge Kafka Monitoring
Data Flow of Monitoring Data and Log Messages
The following diagram illustrates the flow of Apache Kafka monitoring data and log messages.
Explanation of the Diagram
● All monitoring data begins in the Kafka Brokers. The tibkmongateway process collects the full set ofavailable statistics from the Apache Kafka Brokers. It posts the collected statistics to an InfluxDBdatabase.
● InfluxDB is an open-source database. You can access the data directly from InfluxDB, or using othertools that manipulate and display the data, for example, Grafana.
Key to Shapes and Colors in the Diagram
EllipseFTL client process
RectangleOther executable program process
Rounded RectangleAPI, GUI, or HTTP request
BlueComponent of TIBCO software
RedYour Apache Kafka Brokers
GreenThird-party tool
8
TIBCO® Messaging - Apache Kafka Distribution - Bridge Kafka Monitoring
Apache Kafka Monitoring Gateway
The monitoring gateway polls all configured Kafka Brokers and retrieves all of the available statisticsevery 60 seconds.
Configuring the Apache Kafka Monitoring GatewayThe tibkmongateway is configured using a JSON file which is specified as part of the command lineparameters to the monitor-start.py script.
This file contains all of the information needed to access Apache Kafka statistics including:
● Broker addresses and ports
● Broker user names and passwords for access
● Specific Kafka statistics to collect
● InfluxDB database access statistic storage information
The JSON file itself is quite complex and repetitive so to simplify the configuration of tibkmongateway,a .csv (comma separated value) spreadsheet template is provided which can be used toprogrammatically generate the corresponding JSON configuration file.
As an administrator you edit this file directly (using any available spreadsheet applications or directlyusing a text editor if you are familiar with the CSV format), set key configuration parameters, save thefile (as a .csv), and specify this file on the command line of monitor-start.py. This will generate thecomplete JSON file using the information provided in the .csv.
The sample .csv file is named tibkmon_config.csv and is located in the monitoring/tibkmon_JSON_Generator directory.
The .csv file has several columns each representing a specific configuration setting which will be set inthe JSON file as needed in every location it is required:
ConfigurationSetting Description
Kafka BrokerJMX Host &Port
The network address and host from where a specific broker's statistics can beaccessed via JMX. This access must also be enabled using the Kafka Broker'sconfiguration.
Kafka BrokerJMX Username& Password
User authentication information for connecting to the Kafka Brokers.
Kafka ClusterID
A text string which will be added to all statistics collected from this specific KafkaBroker to uniquely identify the Kafka broker cluster that this Kafka Broker is partof in the InfluxDB database.
It is recommended that this ID be unique across all your Kafka clusters.
Kafka Broker ID A text string which will be added to all statistics collected from this specific KafkaBroker to uniquely identify information from it in the InfluxDB database.
It is recommended that this ID be unique across all your Kafka Brokers.
9
TIBCO® Messaging - Apache Kafka Distribution - Bridge Kafka Monitoring
ConfigurationSetting Description
Enable SSL Flag Use SSL to secure the communications between the Kafka Broker andtibkmongateway.
Valid values are TRUE or FALSE.
MonitoringGatewayNumber ofThread
The number of threads that tibkmongateway should use when accessing aspecific broker.
The default value is 2.
Increasing this value will reduce the amount of time needed to retrieve statisticsfrom a specific Kafka Broker and make those statistics more closely aligned interms of access time. Increasing this number places a higher load on the KafkaBroker while decreasing it forces statistics to be gathered serially rather thanconcurrently.
InfluxDBAccess URL,DatabaseName, UserName, &Password
The configuration settings for accessing the appropriate Influx DB instance anddatabase to store the statistics collected from the Kafka Brokers.
Using the Configuration File
The configuration file should have one line per Kafka broker that you want to monitor. Once all of thesettings have been specified, save the file with a .csv file extension. When running monitor_start.pythe configuration file will be processed to generate the corresponding JSON configuration, and starttibkmongatway process using the newly generated JSON configuration.
The JSON file may be manually modified without having to modify the corresponding (.csv based)configuration file. In that case, any subsequent invocations of monitor_start.py should reference themodified JSON file instead of the corresponding configuration file to avoid overwriting the previouslycreated JSON file.
Monitoring Gateway Command Line Reference (monitor-start.py)The monitoring gateway service (tibkmongateway) stores monitoring data and log messages in anInfluxDB database.
Location
The monitor-start.py file is located in the bin directory of the product installation tree.
It is started using the monitor-start Python script. The script can be started using one of the followingways:
● monitor-start.py
● python monitor-start.py
10
TIBCO® Messaging - Apache Kafka Distribution - Bridge Kafka Monitoring
Help
Parameter Description
--help
-h
Show the help message, listing all parameters.
-e
--exit-on-error
Exit on any broker polling error.
-p POLLINGINTERVAL
--polling-interval
Polling interval in seconds, default is 60.
-f CSVFILE
--config-file CSVFILE
The configuration (.csv) file with broker access informationfor all Kafka brokers. Brokers are polled sequentially.
You are required to use either this or the -j option.
--generateonly Specified in conjunction with -f, the script will use the .csvfile to generate a JSON configuration file in the outputdirectory, and then exit.
-o DIRECTORY
--output DIRECTORY
Specify a directory to place the output files in. The defaultdirectory is /var/tmp/ on Unix and c:\tmp on Windows.
-j JSONFILE
--json JSONFILE
Use a previously generated JSON file to load configurationinformation.
Either specify -j or -f, but not both.
11
TIBCO® Messaging - Apache Kafka Distribution - Bridge Kafka Monitoring
Catalogs of Metrics
The following statistics are automatically included in the generated JSON configuration file and arecollected from each Kafka Broker. For more details on any specific statistic, refer to the Apache Kafkadocumentation.
Metric Name Attributes InfluxDB Alias Notes
kafka.server.MessagesIn
PerSec
Count
MeanRate
OneMinuteRate
FiveMinuteRate
FifteenMinuteR
ate
MessagesInPerSec
kafka.server.BytesInPer
SecCount
MeanRate
OneMinuteRate
FiveMinuteRate
FifteenMinuteR
ate
BytesInPerSec
kafka.server.BytesOutPe
rSecCount
MeanRate
OneMinuteRate
FiveMinuteRate
FifteenMinuteR
ate
BytesOutPerSec
kafka.server.Replicatio
nBytesInPerSecCount
MeanRate
OneMinuteRate
FiveMinuteRate
FifteenMinuteR
ate
ReplicationBytesInPerSec
kafka.network.RequestsP
erSecCount
MeanRate
OneMinuteRate
FiveMinuteRate
FifteenMinuteR
ate
RequestsPerSec
12
TIBCO® Messaging - Apache Kafka Distribution - Bridge Kafka Monitoring
Metric Name Attributes InfluxDB Alias Notes
kafka.network.ErrorsPer
SecCount
MeanRate
OneMinuteRate
FiveMinuteRate
FifteenMinuteR
ate
ErrorsPerSec
kafka.network.RequestBy
tesCount
Min
Max
Mean
StdDev
50thPercentile
75thPercentile
95thPercentile
98thPercentile
99thPercentile
999thPercentil
e
RequestBytes request=*
kafka.network.Temporary
MemoryBytesCount
Min
Max
Mean
StdDev
50thPercentile
75thPercentile
95thPercentile
98thPercentile
99thPercentile
999thPercentil
e
TemporaryMemoryBytesProduce request=Pr
oduce
13
TIBCO® Messaging - Apache Kafka Distribution - Bridge Kafka Monitoring
Metric Name Attributes InfluxDB Alias Notes
kafka.network.Temporary
MemoryBytesCount
Min
Max
Mean
StdDev
50thPercentile
75thPercentile
95thPercentile
98thPercentile
99thPercentile
999thPercentil
e
TemporaryMemoryBytesFetch request=Fe
tch
kafka.network.MessageCo
nversionsTimeMsCount
Min
Max
Mean
StdDev
50thPercentile
75thPercentile
95thPercentile
98thPercentile
99thPercentile
999thPercentil
e
MessageConversionsTimeMsFet
ch
request=Fe
tch
14
TIBCO® Messaging - Apache Kafka Distribution - Bridge Kafka Monitoring
Metric Name Attributes InfluxDB Alias Notes
kafka.network.MessageCo
nversionsTimeMsCount
Min
Max
Mean
StdDev
50thPercentile
75thPercentile
95thPercentile
98thPercentile
99thPercentile
999thPercentil
e
MessageConversionsTimeMsPro
duce
request=Pr
oduce
kafka.server.ProduceMes
sageConversionsPerSecCount
MeanRate
OneMinuteRate
FiveMinuteRate
FifteenMinuteR
ate
ProduceMessageConversionsPe
rSec
kafka.server.FetchMessa
geConversionsPerSecCount
MeanRate
OneMinuteRate
FiveMinuteRate
FifteenMinuteR
ate
FetchMessageConversionsPerS
ec
kafka.server.Replicatio
nBytesOutPerSecCount
MeanRate
OneMinuteRate
FiveMinuteRate
FifteenMinuteR
ate
ReplicationBytesOutPerSec
kafka.server.UnderRepli
catedPartitions
Value UnderReplicatedPartitions
kafka.server.
UnderMinIsrPartitionCou
nt
Value UnderMinIsrPartitionCount
15
TIBCO® Messaging - Apache Kafka Distribution - Bridge Kafka Monitoring
Metric Name Attributes InfluxDB Alias Notes
kafka.server.
OfflineLogDirectoryCoun
t
Value OfflineLogDirectoryCount
kafka.controller.Active
ControllerCount
Value ActiveControllerCount
kafka.controller.Leader
ElectionRateAndTimeMsCount
MeanRate
OneMinuteRate
FiveMinuteRate
FifteenMinuteR
ate
Min
Max
Mean
StdDev
50thPercentile
75thPercentile
95thPercentile
98thPercentile
99thPercentile
999thPercentil
e
LeaderElectionRateAndTimeMs
kafka.controller.Unclea
nLeaderElectionsPerSecCount
MeanRate
OneMinuteRate
FiveMinuteRate
FifteenMinuteR
ate
UncleanLeaderElectionsPerSe
c
kafka.server.kafka.serv
er.PartitionCount
Value PartitionCount
kafka.server.LeaderCoun
t
Value LeaderCount
16
TIBCO® Messaging - Apache Kafka Distribution - Bridge Kafka Monitoring
Metric Name Attributes InfluxDB Alias Notes
kafka.server.IsrShrinks
PerSecCount
MeanRate
OneMinuteRate
FiveMinuteRate
FifteenMinuteR
ate
IsrShrinksPerSec
kafka.server.IsrExpands
PerSecCount
MeanRate
OneMinuteRate
FiveMinuteRate
FifteenMinuteR
ate
IsrExpandsPerSec
kafka.server.MaxLag Value MaxLag clientId=R
eplica
kafka.server.PurgatoryS
ize
Value FetchPurgatorySize delayedOpe
ration=Fet
ch
kafka.server.PurgatoryS
ize
Value ProducePurgatorySize delayedOpe
ration=Pro
duce
kafka.network.TotalTime
MsCount
Min
Max
Mean
StdDev
50thPercentile
75thPercentile
95thPercentile
98thPercentile
99thPercentile
999thPercentil
e
ProduceTotalTimeMS request=Pr
oduce
17
TIBCO® Messaging - Apache Kafka Distribution - Bridge Kafka Monitoring
Metric Name Attributes InfluxDB Alias Notes
kafka.network.TotalTime
MsCount
Min
Max
Mean
StdDev
50thPercentile
75thPercentile
95thPercentile
98thPercentile
99thPercentile
999thPercentil
e
FetchConsumerTotalTimeMS request=Fe
tchConsume
r
kafka.network.TotalTime
MsCount
Min
Max
Mean
StdDev
50thPercentile
75thPercentile
95thPercentile
98thPercentile
99thPercentile
999thPercentil
e
FetchFollowerTotalTimeMS request=Fe
tchFollowe
r
18
TIBCO® Messaging - Apache Kafka Distribution - Bridge Kafka Monitoring
Metric Name Attributes InfluxDB Alias Notes
kafka.network.RequestQu
eueTimeMsCount
Min
Max
Mean
StdDev
50thPercentile
75thPercentile
95thPercentile
98thPercentile
99thPercentile
999thPercentil
e
FetchFollowerRequestQueueTi
meMs
request=Fe
tchFollowe
r
kafka.network.RequestQu
eueTimeMsCount
Min
Max
Mean
StdDev
50thPercentile
75thPercentile
95thPercentile
98thPercentile
99thPercentile
999thPercentil
e
FetchConsumerRequestQueueTi
meMS
request=Fe
tchConsume
r
19
TIBCO® Messaging - Apache Kafka Distribution - Bridge Kafka Monitoring
Metric Name Attributes InfluxDB Alias Notes
kafka.network.RequestQu
eueTimeMsCount
Min
Max
Mean
StdDev
50thPercentile
75thPercentile
95thPercentile
98thPercentile
99thPercentile
999thPercentil
e
ProduceRequestQueueTimeMS request=Pr
oduce
kafka.log.LogFlushRateA
ndTimeMsMeanRate
OneMinuteRate
FiveMinuteRate
FifteenMinuteR
ate
Min
Max
Mean
StdDev
50thPercentile
75thPercentile
95thPercentile
98thPercentile
99thPercentile
999thPercentil
e
LogFlush
20
TIBCO® Messaging - Apache Kafka Distribution - Bridge Kafka Monitoring
Metric Name Attributes InfluxDB Alias Notes
kafka.network.LocalTime
MsCount
Min
Max
Mean
StdDev
50thPercentile
75thPercentile
95thPercentile
98thPercentile
99thPercentile
999thPercentil
e
ProduceLocalTimeMs request=Pr
oduce
kafka.network.LocalTime
MsCount
Min
Max
Mean
StdDev
50thPercentile
75thPercentile
95thPercentile
98thPercentile
99thPercentile
999thPercentil
e
FetchConsumerLocalTimeMs request=Fe
tchConsume
r
21
TIBCO® Messaging - Apache Kafka Distribution - Bridge Kafka Monitoring
Metric Name Attributes InfluxDB Alias Notes
kafka.network.LocalTime
MsCount
Min
Max
Mean
StdDev
50thPercentile
75thPercentile
95thPercentile
98thPercentile
99thPercentile
999thPercentil
e
FetchFollowerLocalTimeMs request=Fe
tchFollowe
r
kafka.network.RemoteTim
eMsCount
Min
Max
Mean
StdDev
50thPercentile
75thPercentile
95thPercentile
98thPercentile
99thPercentile
999thPercentil
e
ProduceRemoteTimeMs request=Pr
oduce
22
TIBCO® Messaging - Apache Kafka Distribution - Bridge Kafka Monitoring
Metric Name Attributes InfluxDB Alias Notes
kafka.network.RemoteTim
eMsCount
Min
Max
Mean
StdDev
50thPercentile
75thPercentile
95thPercentile
98thPercentile
99thPercentile
999thPercentil
e
FetchConsumerRemoteTimeMs request=Fe
tchConsume
r
kafka.network.RemoteTim
eMsCount
Min
Max
Mean
StdDev
50thPercentile
75thPercentile
95thPercentile
98thPercentile
99thPercentile
999thPercentil
e
FetchFollowerRemoteTimeMs request=Fe
tchFollowe
r
23
TIBCO® Messaging - Apache Kafka Distribution - Bridge Kafka Monitoring
Metric Name Attributes InfluxDB Alias Notes
kafka.network.ResponseS
endTimeMsCount
Min
Max
Mean
StdDev
50thPercentile
75thPercentile
95thPercentile
98thPercentile
99thPercentile
999thPercentil
e
FetchConsumerResponseSendTi
meMs
request=Fe
tchConsume
r
kafka.network.ResponseS
endTimeMsCount
Min
Max
Mean
StdDev
50thPercentile
75thPercentile
95thPercentile
98thPercentile
99thPercentile
999thPercentil
e
FetchFollowerResponseSendTi
meMs
request=Fe
tchFollowe
r
24
TIBCO® Messaging - Apache Kafka Distribution - Bridge Kafka Monitoring
Metric Name Attributes InfluxDB Alias Notes
kafka.network.ResponseQ
ueueTimeMsCount
Min
Max
Mean
StdDev
50thPercentile
75thPercentile
95thPercentile
98thPercentile
99thPercentile
999thPercentil
e
ProduceResponseQueueTimeMs request=Pr
oduce
kafka.network.ResponseQ
ueueTimeMsCount
Min
Max
Mean
StdDev
50thPercentile
75thPercentile
95thPercentile
98thPercentile
99thPercentile
999thPercentil
e
FetchConsumerResponseQueueT
imeMs
request=Fe
tchConsume
r
25
TIBCO® Messaging - Apache Kafka Distribution - Bridge Kafka Monitoring
Metric Name Attributes InfluxDB Alias Notes
kafka.network.ResponseQ
ueueTimeMsCount
Min
Max
Mean
StdDev
50thPercentile
75thPercentile
95thPercentile
98thPercentile
99thPercentile
999thPercentil
e
FetchFollowerResponseQueueT
imeMs
request=Fe
tchFollowe
r
kafka.network.NetworkPr
ocessorAvgIdlePercent
Value NetworkProcessorAvgIdlePerc
ent
kafka.server.RequestHan
dlerAvgIdlePercent
Count
MeanRate
OneMinuteRate
FiveMinuteRate
FifteenMinuteR
ate
RequestHandlerAvgIdlePercen
t
kafka.server.ProduceBWQ
uota
queue-size ProduceBWQuota type=Produ
ce
kafka.server.FetchBWQuo
ta
queue-size FetchBWQuota type=Fetch
kafka.server.RequestQuo
ta
queue-size RequestQuota
26
TIBCO® Messaging - Apache Kafka Distribution - Bridge Kafka Monitoring
Metric Name Attributes InfluxDB Alias Notes
kafka.server.ZooKeeperR
equestLatencyMsCount
Min
Max
Mean
StdDev
50thPercentile
75thPercentile
95thPercentile
98thPercentile
99thPercentile
999thPercentil
e
ZooKeeperRequestLatencyMs
java.lang.OperatingSyst
emProcessCpuTime
ProcessCpuLoad
SystemCpuLoad
SystemLoadAver
age
CommittedVirtu
alMemorySize
FreePhysicalMe
morySize
OS
java.lang.Memory HeapMemoryUsag
e
NonHeapMemoryU
sage
MemoryUsage
27
TIBCO® Messaging - Apache Kafka Distribution - Bridge Kafka Monitoring
Database and Visualization
You can use a variety of other third-party tools, such as InfluxDB and Grafana, to store, analyze, andvisualize monitoring data.
InfluxDBInfluxDB is an open source time series database.
InfluxDB and Docker
You can run InfluxDB processes in Docker containers.
For convenience, you can start the InfluxDB server using a utility script, which also starts themonitoring gateway and other monitoring components in the correct order.
Grafana and Dashboard ExamplesGrafana is an open source application to visualize large-scale measurement data. You can use Grafanato display monitoring data stored in a database.
Grafana can display dashboards, such as the following examples, in a web browser.
The installer adds the Apache Kafka monitoring dashboards as part of the installation procedure. Thedashboards include a top-level dashboard which displays statistics common to many massagingsystems as well as a set of Kafka-specific dashboards.
The messaging independent topmost level dashboard displays information from both Apache Kafkaand TIBCO FTL allowing high-level messaging statistics from both messaging systems to be monitoredfrom a single dashboard.
Grafana dashboards in the sample are tailored to display monitoring data grouped by similar ApacheKafka functionality.
When the monitoring start script starts the Grafana server, it automatically loads the Apache Kafkadashboards from monitoring/config/influxdb/dashboards.
You can also define your own dashboards. For more information, refer to the Grafana documentation.
Following are some sample dashboard images.
28
TIBCO® Messaging - Apache Kafka Distribution - Bridge Kafka Monitoring
Security of Monitoring Data
If your enterprise uses secure Apache Kafka brokers, then data is secure as it travels from client toApache Kafka broker, and on to other clients.
If your Apache Kafka brokers are configured to secure JMX access, any statistics-relatedcommunications will be secured as well.
However, TIBCO makes no assertions about the security of third-party components, such as InfluxDBor Grafana.
Monitoring Gateway
The monitoring gateway (tibkmongateway) supports SSL to secure its connection to each ApacheKafka broker. Administrators must supply appropriate security information when configuring thegateway.
InfluxDB
The monitoring gateway can secure its connection InfluxDB.
Grafana
Grafana can use a secure connection to query InfluxDB. Grafana can secure client connections frombrowsers using HTTPS.
31
TIBCO® Messaging - Apache Kafka Distribution - Bridge Kafka Monitoring