© Freudenberg-NOK General Partnership 1 FNST’s QAD Monitoring One part of the...
-
Upload
delphia-todd -
Category
Documents
-
view
220 -
download
1
Transcript of © Freudenberg-NOK General Partnership 1 FNST’s QAD Monitoring One part of the...
© Freudenberg-NOK General Partnership 1
FNST’s QAD Monitoring
One part of the monitoring/management initiative
for FNST/FST
© Freudenberg-NOK General Partnership 2
Agenda• Intro – About FNST• Background/History• QAD Implementation• Present State• Benefits Realized• Future Directions• Quick Review (screenshots)• Lessons Learned• Q&A
© Freudenberg-NOK General Partnership 3
25%
~23%
75%
Freudenberg-NOK Sealing Technologies Established 1989
Headquartered in Plymouth, Mich., USA(Privately Held)
Established 1849Headquartered in Weinheim, Germany
(Privately Held)
Established 1939Headquartered in Tokyo, Japan(Publically Traded on the NSS)
© Freudenberg-NOK General Partnership 4
Diverse and Established Partners
Radial Shaft Seals
O-Rings
Gaskets
Fluid Power Products
Diaphragms
Specialty Sealing Products
Privately Held
$1 Billion
1989
Radial Shaft Seals
O-Rings
Gaskets
Fluid Power Products
Diaphragms
Specialty Sealing Products
Vibration Control Devices
Radial Shaft Seals
O-Rings
Gaskets
Fluid Power Products
Oil and Gas Products
Diaphragms
Specialty Sealing Products
Vibration Control Devices
Medical Products
Products
Publicly HeldPrivately HeldOwnership
$6.5 Billion
1849
Weinheim, Germany
$4.5 Billion2008 Sales
1939Established
Tokyo, JapanPlymouth, MichiganHeadquarters
© Freudenberg-NOK General Partnership 5
Freudenberg Global locationsProductionSalesProduction and Sales
© Freudenberg-NOK General Partnership 6
Production Locations in the Americas
Cuautla, Mexico
Radial Shaft Seals and Custom Molded Rubber Products and Vibration Control Devices
Querétaro, MexicoO-rings
Santa Ana, CAO-rings
Elgin, IL
Simrit Sales Office
Morristown, INCustom Molded Rubber Products
Plymouth, MITechnology Center
Milan, OHHeavy Industry Lead Center, Imported Products
Tillsonburg, ONSilicone and Fluoro-silicone Products
Findlay, OH
PTFE Products
Cleveland, GAValve Stem Seals and Powertrain Seals
LaGrange, GAO-rings
Troy, OHPTFE Products
New Hampshire Facilities:
Northfield – Axle Seals
Bristol – Mechanical Face Seals, Bonded Piston Seals
Ashland – Aerospace Radial Shaft Seals Industrial Radial Shaft Seals
Manchester – Microcellular Polyurethane
Manchester – Gaskets
Shelbyville, INBoots & Bellows
Necedah, WIGaskets
Sao Paulo, BrazilRadial Shaft Seals
© Freudenberg-NOK General Partnership 7
Freudenberg-NOK Sealing Technologies Today
Responsible for sealing technology business in the Americas
Total sales in 2010: US$ 798.7 million
Workforce in 2010: 4,224
Locations in: U.S., Canada, Mexico,
Brazil, Malaysia, China
Sales exceeded market recovery in 13
of 15 principal market segments
Sites: 35 production, sales and
technical facilities
© Freudenberg Sealing Technologies GmbH & Co. KG 8
Agenda
• Background/History• QAD Implementation• Present State• Benefits Realized• Future Directions• Quick Review (screenshots)• Live
© Freudenberg Sealing Technologies GmbH & Co. KG 9
Background/History
HP OpenView implemented in 1994
Dedicated a great deal of effort to include Progress and QAD in our monitoring/management
Versions not kept current (budget constraints) to support our Linux environment
© Freudenberg Sealing Technologies GmbH & Co. KG 10
Agenda
• Background/History• QAD Implementation• Present State• Benefits Realized• Future Directions• Quick Review (screenshots)• Live
© Freudenberg Sealing Technologies GmbH & Co. KG 11
QAD Implementation
Contracted QAD Services.
“Canned” product, virtual Linux server (recover image and change IP), Icinga and Cacti for alerting and graphing.
“Base” configuration file created. Deployment simplified, copy base and change host info.
Scripts built for deploying:- Database monitors based on our “/node/info/databases” file (list of databases per
host with environment, used for many purposes including start-up).- Disk statistics.
- Graphs too.
© Freudenberg Sealing Technologies GmbH & Co. KG 12
Our Environment
Centralized 2010SE Host (co-lo in NC)
Supporting 30 sites from Canada to Mexico
Distribution Site Linux eB2 (our facility in OH)
Brazil Site Linux 2007
© Freudenberg Sealing Technologies GmbH & Co. KG 13
Our Monitoring Environment
Each host has a system “control” file containing all databases and environments
This allows easy scripting of status checks, up/down, and query from other applications and scripts to invoke
This file allowed us to automate our monitoring implementation
© Freudenberg Sealing Technologies GmbH & Co. KG 14
Database Control File
######################
/prod/prddbhe10/prddbhe10 DEFAULT hlpea PRODEA10 QAD_add-on
/prod/prddbqxo17/prddbqxo17 DEFAULT qxodb PRODEA10 QAD_add-on
#######################
/prod/prdcmbme10/prdcmbme10 DEFAULT mfgdb PRODEA10 Prod_QAD
/prod/prdcmbae10/prdcmbae10 DEFAULT admea PRODEA10
/prod/prdcmbfe10/prdcmbfe10 DEFAULT fngpea PRODEA10 FNGP_QAD_add-on
/prod/prdcmbte10/prdcmbte10 DEFAULT tpea PRODEA10 TailorPro
/prod/prdcmbev17/prdcmbev17 DEFAULT qxevents PRODEA10 Qxtend_events
#######################
## Corteco Mexico
#######################
/prod/prd73me10/prd73me10 DEFAULT mfg73 PRODEA10 Prod_QAD mexico
/prod/prd73ae10/prd73ae10 DEFAULT admea PRODEA10
/prod/prd73fe10/prd73fe10 DEFAULT fngpea PRODEA10 FNGP_QAD_add-on
/prod/prd73te10/prd73te10 DEFAULT tpea PRODEA10 TailorPro
/prod/prd73ev17/prd73ev17 DEFAULT qxevents PRODEA10 Qxtend_events
#######################################################
© Freudenberg Sealing Technologies GmbH & Co. KG 15
Site Monitoring
Ping a host in each supported site
Good method of determining how a site sees performance
© Freudenberg Sealing Technologies GmbH & Co. KG 16
Scripts for Automation of Monitoring Services
./build_server_cfg.sh [remote host] [DLC]
Builds a file with the Icinga components for Progress database monitors based on the control file. This generated file gets inserted into the host’s configuration file in Icinga.
Creates Icinga monitors for DB check, extents, log entries, last backup, buffer hits, read rates, and locks
./build_server_graph.sh [remote host] [DLC]
Queries for host ID, then creates Cacti graphs for each database in the control file
./build_server_disk.sh [remote host]
Ran a “df” on the remote host, builds Icinga space monitor for each local disk, insert into host’s configuration file.
© Freudenberg Sealing Technologies GmbH & Co. KG 17
Script to Build Configuration File
copy the template to [servername]-server.cfg
cp fnst-server.cfg.template [servername]-server.cfg
find and replace ral2lnxqad with the server name
edit the host IP
correct the webspeed entries
correct DLC if needed
build the databases section:
./build_server_cfg.sh [remote host] [DLC]
© Freudenberg Sealing Technologies GmbH & Co. KG 18
Agenda
• Intro – About FNST• Background/History• QAD Implementation• Present State• Benefits Realized• Future Directions• Quick Review (screenshots)• Lessons Learned• Q&A
© Freudenberg Sealing Technologies GmbH & Co. KG 19
Present State
All Linux hosts monitored (ping, OS)
All sites monitored (ping to a host in all sites)
All Linux QAD hosts monitored (Progress databases, NetUI performance)
© Freudenberg Sealing Technologies GmbH & Co. KG 20
Agenda
• Intro – About FNST• Background/History• QAD Implementation• Present State• Benefits Realized• Future Directions• Quick Review (screenshots)• Lessons Learned• Q&A
© Freudenberg Sealing Technologies GmbH & Co. KG 21
Benefits Realized
Network issues identified
“stair step” graph noticed, turns out cable disconnect at site
Qxtend performance issue
Alert on CPU utilization, worked w/ EA Team
Cache Hit Rate
Alert, increased “Big B”
DB Writes/Min
Worked w/ ERP Team, identified code that was not “domainized”
© Freudenberg Sealing Technologies GmbH & Co. KG 22
Agenda
• Intro – About FNST• Background/History• QAD Implementation• Present State• Benefits Realized• Future Directions• Quick Review (screenshots)• Lessons Learned• Q&A
© Freudenberg Sealing Technologies GmbH & Co. KG 23
Future Direction
Regular internal review.
Regular review with ERP Team.
Review what else would benefit us.
Remote point(s) for monitoring. NetUI testing for one.
With the implementation of Vmware in remote sites we may just deploy VM
Include HPUX (retire OpenView).
Visibility to others outside the Server Team.
Migrate additional legacy performance and monitoring scripts
Review how to play with SCOM
© Freudenberg Sealing Technologies GmbH & Co. KG 24
Agenda
• Intro – About FNST• Background/History• QAD Implementation• Present State• Benefits Realized• Future Directions• Quick Review (screenshots)• Lessons Learned• Q&A
© Freudenberg Sealing Technologies GmbH & Co. KG 25
Quick Review
Alerts
Graphs
© Freudenberg Sealing Technologies GmbH & Co. KG 26
Alerts
Graphical, Emails
© Freudenberg Sealing Technologies GmbH & Co. KG 27
Status/Alerts by Location/Service
© Freudenberg Sealing Technologies GmbH & Co. KG 28
Alerts by Service
© Freudenberg Sealing Technologies GmbH & Co. KG 29
Alerts by Host
© Freudenberg Sealing Technologies GmbH & Co. KG 30
Email Alerts
© Freudenberg Sealing Technologies GmbH & Co. KG 31
Graphs
Hourly (min avg),
Daily (5 min avg),
Weekly (30 min avg),
Monthly (2 hr avg),
Yearly (1 day avg)
Data points include:
Ping Response
Host Statistics
QAD Users
DB Reads
Time to Read 10K Records
Database Performance Statistics
NetUI View
© Freudenberg Sealing Technologies GmbH & Co. KG 32
Site Overview
We included (at least) one host in each of the sites accessing our central QAD servers
This gives us up/down and response times by site, assisting in troubleshooting issues
© Freudenberg Sealing Technologies GmbH & Co. KG 33
Site Overview
© Freudenberg Sealing Technologies GmbH & Co. KG 34
Ping Response Time
© Freudenberg Sealing Technologies GmbH & Co. KG 35
Statistics Specific to QAD Hosts
Processor statistics
QAD users
Database statistics
NetUI performance
© Freudenberg Sealing Technologies GmbH & Co. KG 36
Host Performance Statistics
© Freudenberg Sealing Technologies GmbH & Co. KG 37
QAD Users
© Freudenberg Sealing Technologies GmbH & Co. KG 38
QAD DB Reads
© Freudenberg Sealing Technologies GmbH & Co. KG 39
Sidecar DB Reads
© Freudenberg Sealing Technologies GmbH & Co. KG 40
Admin DB Reads
© Freudenberg Sealing Technologies GmbH & Co. KG 41
Time to Read 10,000 Records
© Freudenberg Sealing Technologies GmbH & Co. KG 42
tr_hist Records Added(good indicator of activity)
© Freudenberg Sealing Technologies GmbH & Co. KG 43
NetUI Response
© Freudenberg Sealing Technologies GmbH & Co. KG 44
Live
Point browser to http://ra2sqadmon
© Freudenberg Sealing Technologies GmbH & Co. KG 45
Questions?
Questions,
Suggestions,
Comments,