Continuous Control Monitoring: HP and Google’s Perspective

San Francisco Chapter

Continuous Control Monitoring:

HP and Google’s Perspective

Jessica Amezquita, Hewlett-Packard Company

Erik Jonte, Google, Inc

Brad Ames, Hewlett-Packard Company

Segment S13

September 17, 2007

2007 Fall Conference

Agenda

• Continuous Control Monitoring Premise

and Framework

• HP Walkthrough

• Google Walkthrough

• Summary of our Learnings

Build toward a Strategy

• Continuous Control Measurement (CCM) is a

monitoring and benchmarking approach adopted by

HP internal audit to see emerging risk across the

enterprise

• The CCM tools and methodology enable the

examiner and governance to shift from a historical

view to an ongoing strategic perspective

• Since risk and response to risk can be analyzed

remotely, HP is reducing time and intrusion in the

field by implementing the CCM tools and

methodology

Premise for Continuous Control Monitoring

• Uncertainty - Less comfort regarding how risk ismanaged results in more testing.

• Tolerance - Tolerance and control activities gotogether. Low tolerance for risk mean morecontrol processes which reduces testing.

• Response - CCM provides a way for auditors togain visibility to risk tolerance, response to riskand generates confidence.

• Interdependence - It all goes together. Not all ofthe controls in the environment need to betested to conclude on risk. When one control isstrengthened it will effect another.

Continuous Control Measurement Tools and Methodology

Accepted Assurance Frameworks

Alignment is the Key to Provide a

Portfolio View of RiskCompliance

• Configurable Controls

• Exception Data

Financial Process RisksApplication Risks

• Change Management

• Security

• Operations

IT Operations Risks

• Release & Config Mgt

• Identity Management

• Incident Management

OpenView SAP KPI Infinite

HP Walkthrough

• Jessica.Amezquita

• Jessica.amezquita@hp.com

• Internal Audit Manager

SAP KPI Tool Design

• Logging on

• How many instances are monitored?

~30 SAP Source SystemsKPI Application

Manual Data Trigger Request

Overview of SAP KPI Tool

SAP System IDs

What is HP Currently Monitoring?

• Change Management– Number of transports– Users with the ability to develop and migrate changes

to production

• Security– Number of users (active, locked, expired)– Password parameters– Privileged access (SAP_ALL, users with ability to

maintain customer credit terms)– Terminated employee check

• Operations– Number of users with the ability to

create/modify/delete jobs

Number of Users

Last / Current Month

Compare Multiple KPIs Across One

SAP System

Compare One KPI Across Multiple SAP

Systems

Types of Reports Available

• Current and last month detail

• Compare multiple KPIs across one SAP

system

• Compare one KPI across multiple SAP

systems

• Complete history report

SAP_ALL (Privileged Access)

• What does this KPI tell you about the

application environment?

• How does HP use this KPI?

• Limitations/Considerations

Active Users (USED) vs.

Privileged Users (SAP_ALL)

555555SAP_ALL

4,1824,1764,2004,2624,2924,230USED

Mar-07Feb-07Jan-07Dec-06Nov-06Oct-06KPI:

History for System: R00

SAP_ALL Comparison Across Similar

Applications

212333R01

(Europe)

555555

(North

America)

1210101099APL

(Asia Pacific)

Mar-07Feb-07Jan-07Dec-06Nov-06Oct-06System

History for KPI:SAP_ALL

Challenges

• Identifying KPIs

• Determining how to pull relevant data in a

timely manner

• Setting up the automatic pull

• Audit traditionalist may be reluctant to

change

• Without a defined methodology auditors

may not know how to use CCM tools and

apply approach

Considerations for Implementation

• Accuracy and completeness of data

• Let auditors develop/identify KPIs as they

• Involvement of external audit

• Training

Google Walkthrough

• Erik Jonte

• ejonte@google.com

• Google, Inc. IT Risk Manager

Background

• Minimal control testing automation

• Resource growth enabled investment in

automation

• Desire to reduce manual testing effort

Getting Started

• Large teams not necessary

• Focus on open source or free software

• Hire / cultivate individuals with knowledge /

desire to develop

• Identify low hanging fruit to build morale

The Problem

• Code Review and SOD Key Controls

• Very large population of changes

• Complex data structure

• Manual testing approach (sampling)

• Unclear success criteria for testing

Solution Requirements

• Speed – must exceed manual testing

approach

• Accuracy – must be able to detect all error

conditions

• Easy Interface – cumbersome solutions

are not used

• Acceptable to External Auditors

Integration Challenges

• Getting Data Access

• Normalizing multiple data sources

• Database Storage

Architecture

Technology Choices

• Perl

– Tons of text processing speed

– Many community-available modules

– Shallow learning curve

• MySQL

– Free

– Fast

– Easy

Request Interface

Sample Report

Solution Results

• Delivery to end-users

• Integration with WP management

• Audit of 100% of change population in

minutes

• Audit of metadata with 100% accuracy

• Ability to conclude with certainty

Sampling Vs Automation

• Can choose sampling or 100% sample –

applicability in different arenas

• Self-documenting

• Certainty over error rates

• Consistent conclusions across team

members

• “Automation Dividend”

Complication

• External auditors wanted comfort over

exceptions

• Segmentation of exceptions

• Defining criteria was entry into production

• Refine the notion of a true exception

• Used statistical approach to define

reasonable assurance

Future Benefits

• Trending Across Time

• Integration into “Audit Data Warehouse”

• Periodic or Continuous Review

Take Aways

• Identify data sources early on

• Strive for normalization of data

• Keep the end-user in mind

• Document requirements and design

• Have a strategy for integration with

external audit

In Summary

• Challenges

• Considerations for Implementation

• Opportunities

Challenges

• Deciding the measurements

• Determining how to pull relevant data in a timely

manner

• Setting up the automatic pull

• Dealing with the Audit traditionalist (who may be

reluctant to change)

• Following a different way – without a

corresponding methodology, auditors may not

fully benefit from the CCM tools.

Considerations for Implementation

• Expect auditors to identify KPIs as they audit

• Establish practices to ensure accuracy and

completeness of data

• Involve external audit

• Scale appropriately for success

• Develop audit methodology to accompany the

• Benchmarking focuses the examiner to consider

risk and changes to key controls in order to reduce

or eliminate inspection testing

• Benchmarking provides an opportunity to shift the

SOX effort from a checklist-adherence approach to

an ongoing risk-based view of risk benefiting

governance

Opportunities

By being able to constantly ‘watch’ systematic controls,examiners can more easily and confidently measure the

operating effectiveness of internal controls.

Questions and Collaboration

Jessica AmezquitaHewlett-Packard, Internal Audit ManagerJessica.amezquita@hp.com

Eric JonteGoogle, IT Risk Managerejonte@google.com

Brad AmesHewlett-Packard, Internal Audit Directorbrad.ames@hp.com

Continuous Control Monitoring: HP and Google’s Perspective

Documents

Transcript of Continuous Control Monitoring: HP and Google’s Perspective

Tips to achieve continuous integration/delivery using HP ALM, Jenkins, and Skytap

Google’s cloud strategy

Google’s MapReduce

Google’s Billion Dollar Eigenvector

Towards Continuous Delivery and DevOps at HP Software, Adam Spektor

The Future of Print - Insights for continuous product innovation - HP

Taming Google-Scale Continuous Testing · Google Inc., Mountain View, USA Email: fbaonn,sanjeevdhanda,esnickell,robsiemb,jmiccog@google.com Abstract—Growth in Google’s code size

HP StorageWorks Continuous Access EVA administrator guideh10032.HP StorageWorks Continuous Access EVA administrator guide 13 Bidirectional replication When an array contains both source

Google’s Cloud Office

hp StorageWorks Continuous Access EVA V1h10032. · Design Reference Guide hp StorageWorks Continuous Access EVA V1.1B Product Version: 1.1B Seventh Edition (July 2004) Part Number:

Google’s Charlotte Morton: Performance art

HP StorageWorks Continuous Access EVA - Guía de Operación

What is New: HP LoadRunner 12.02, HP Performance Center 12 ... · Noise Generator in LoadRunner 12.02 Continuous testing / redefine performance . Helping Dev/Test load test in Continuous

(PDF) HP Storage Works Continuous Acces EVA Design

Brochure HP DesignJet portfolio - phippsrepro.com · Brochure HP DesignJet portfolio 1. HP DesignJet large-format printing—continuous innovation 2010s HP launches the first web-connected

HP EVA Array using vCenter Site Recovery Manager: … · 3 HP StorageWorks Continuous Access EVA Continuous Access EVA is a feature of the Enterprise Virtual Array (EVA) that allows

Seminar on google’s chromecast technology

Google’s Problem in China

Continuous Training for Production ML in the …for continuous pipelines in the TensorFlow Extended (TFX) platform [1]. TFX enables Google’s engineers to reliably run ML in production

Dr. Google’s office never closes