Centralizing and Qualifying IT Business Data

Centralizing and QualifyingIT Business Data

Ensuring Success of IT Service Management Initiatives for Assets

IT Intelligence Delivered

A BLAZENT TECHNOLOGY WHITE PAPER

© 2008 - Blazent, Inc. All rights reserved 2

AbstractIT organizations (ITOs) striving to reach a greater level of maturity in managing their service offerings need to put analysis at the forefront of their success measurements. Regardless of the initiative they are building to define (business service management, performance management, compliance, ITIL, chargeback systems, etc.), the first phase is to assemble a comprehensive, updated and reliable data set of information that describes their IT service environment. Only then can they ensure that the quality of that data can be measured and analyzed with confidence across the business.

This whitepaper describes the different organizational and technological hurdles prominent on the road to unifying IT service management asset data. It also proposes that a well-conceived business intelligence (BI) approach can create an adaptable, powerful solution — and that data warehousing methodologies in particular address the ongoing data quality issues within federated CMDBs (configuration management databases). This paper also describes a necessary workflow for maintaining data quality across IT and corporate service silos within a data warehousing strategy.

The Need to Trust Data .............................................................................................. 3

Maintaining Consistency, Accuracy, and Accountability ............................................. 3

IT Service Management Initiatives Driving Maturity ................................................... 4

The Failure of Manual Asset Analysis ......................................................................... 4

Converge or Collide: Merging Disparate Data .......................................................... 5

Centralizing IT Business Data ..................................................................................... 7

Building an IT Intelligence Data Warehouse .............................................................. 7

Data Integration Approach (Initial Cleanse from Sources) ......................................... 8

Data Quality Workflow (Ongoing Improvement) ........................................................ 9

Quality Problems ........................................................................................................10

Measuring Success .....................................................................................................11

Summary .....................................................................................................................12

ConTenTs


The need to Trust Data“Data rich; information poor.” “Garbage in; garbage out.” There’s a reason why IT clichés like these hit home. For many dynamic, expansive IT organizations, they resonate with truth: If you don’t trust the data, you don’t trust the analysis it’s based on. Consequently, distrust for your data cripples your decision making ability.

Any meaningful analysis or reporting requires that the data being processed is available, accurate and timely, not merely anecdotal. Ensuring the best quality data provides the reliable basis for business decisions focused on budgeting, planning, optimizing, and renegotiating IT licenses and contracts. It’s also imperative for more mature initiatives such as business intelligence analysis that attempt to master compliance, IT Infrastructure Library (ITIL) based operating models and chargeback systems.

Yet as enterprises inevitably go through technological, organizational and directional changes, the chaos of data spreads. Certainty and confidence in the data is reduced.

Most organizations try to manually gather data from multiple sources and build analyses. That means critical, timely decisions come from inefficient, inconsistentprocesses based on inaccurate or incomplete information. Using this limited approach, it doesn’t matter how sound the approach of analysis itself might be, because the problem is in the data itself.

The most viable solution is a data quality framework based on tools and workflows that maximize integration of disparate data and that refine quality by consistently honing the accuracy of the data.

Maintaining Consistency, Accuracy, and AccountabilityIT data changes because business changes. Business decisions (mergers, acquisitions, spin-offs, downsizing, right-sizing, outsourcing, new service offerings, etc.) and technology decisions (patches, upgrades, deployments, contract renewals, consolidations, migrations, etc.) that affect IT operations are made with increasing regularity. Ensuring that IT continues running effectively, efficiently and with greater accountability requires being able to regularly assess and analyze the data that describes an IT environment.

The most daunting challenge in assembling this data is dealing with changing and dispersed data sources. Some piece of the puzzle may lie within the service management workflow, another within the ERP or HR systems or on a spreadsheet on someone’s laptop. Even more data points are likely not even accounted for. After all, as Defense Secretary Rumsfeld has said, “there are known unknowns” and “unknown unknowns.”

Yet in a reactive IT environment, aligning these diverse data sources to build a complete business perspective is too often shelved as a “nice to do,” not a “need to do.” That way of thinking, however, is also changing.

What are the consequences of not having quality data about IT?

• Toomanyortoofewsoft-warelicensesorleasedequipmentprocured

• IncreaseinTCO(includ-ingcapital,tax&main-tenance) due to poor IT assetlifecycledetermina-tion and needs assess-ment

• Misalignedinformationa-cross IT service silos lead-ingtooperationswithoutgovernance

Unknown unknowns

“Reports that say that somethinghasn’thappenedarealwaysinterestingtomebecause,asweknow,thereareknownknowns;therearethingsweknowweknow.Wealsoknowthereareknownunknowns;thatistosayweknowtherearesomethingswedonotknow.Buttherearealsounknownunknowns–theoneswedon’tknowwedon’tknow.”

Donald Rumsfeld,Feb. 12, 2002,Department of Defense


IT service Management Initiatives Driving MaturityIT is becoming the most expensive, vital part of the business infrastructure for manyorganizations, and as a result, its operations, resources, and assets need to be accounted for. That’s why initiatives like ITIL, IT service management, chargeback systems, etc. are gaining so much momentum. These initiatives finally give IT and business managers a lexicon and framework to describe and measure the challenges they are facing. IT service management requires that ITOs not only collect and track the data, but routinely analyze that data to shape policy and activities.

Every component of the IT service delivery and support management areas (assets, personnel, and processes) need to be rationalized for optimum utilization and for maximum effectiveness. Software licenses and leased equipment need to be in compliance with existing contracts. Increasingly, there is a mounting trend for IT executives to be tasked with justifying expenditures, yet many don’t even have the most basic evidence to support their case. All of this data exists in the nooks and crannies of IT and the only way to gain any business perspective is to integrate this information.

The Failure of Manual Asset AnalysisThe critical first step of any IT service management initiative involves building accurate asset data for business analysis. Traditionally, IT business analysts are tasked to make sense of this data. To produce reports that measure the effectiveness of IT assets and resources, they are responsible for:

Assembling numerous ITAM (IT asset management) reports from varied sources•Correlating with fi nancial/procurement/HR data•Building spreadsheets or another databases•Determining points of analysis (i.e. the correct business metrics for the questions •they are trying to answer)

If analysis is performed manually as most organizations are doing today, generating analysis can be time consuming and at risk of human error such as when complex linking of dissimilar information is involved. Analysts also have to manually determine sources, and fields and data for merging. The most frequently used tool to drive analysis is the spreadsheet, which basically severs the link to source data prohibiting repeatable, more timely analysis.

If data is part of one source and not another, there’s no accurate way to analyze missing information. And even if data collection is real-time, the analysis often isn’t. In addition, manual processes don’t offer and an effective means of measuring the quality of the analysis. It is rare that business decision makers can quickly reconcile its analysis to discover errors. To complicate matters further, if multiple analysts are building reports, they may come up with very different answers simply because they have different paths to analysis.

Themostfrequentlyusedtool to drive analysis is the spreadsheet,whichbasicallyseversthelinktosource-data prohibiting repeatable, moretimelyanalysis.


Converge or Collide: Merging Disparate DataDuring integration, data can converge or it can collide. A collision results in data sources that are simply mashed together without consideration for what data is actually required for analysis. As you can imagine, in wide and varied interdepartmental systems, the data can look like something after a car crash where parts that were spread out everywhere are scooped together. Even in a structured integration endeavor where the “end result” is carefully prescribed into a template, things can get messy.

Checks and Balances for Data QualityIntegrating multiple data sources actuallydrives the basis for data quality. Yielding aresult from a variety of inputs provides moreresolution, i.e. a more holistic picture of theenvironment.

The classic example is a group of blind menall feeling different parts of an elephant andcoming to different conclusions as to whatthe object is. Only when you consolidate theirviews is it possible to determine that theobject is an elephant. Likewise, the more viewsavailable, the better quality the picture.

IT business management data benefits from achecks-and-balances approach to reconciling.This cross-silo approach also emphasizesgreater transparency so disparate businessfunctions can align as needed, and it addsbusiness dimensions to the data which canprovide deeper analysis and correlation.

The key to gaining true business intelligence in regards to IT involves taking in referential information from existing IT data sources, integrating it with business data, and verifying it with discovery and utilization data. This methodology provides the comprehensive quality checks of comparing between what you think you have, what you actually have, and how it’s being used. The types of information needed to manage IT assets include:

IT Data• – This includes repositories from asset management repositories, confi guration management databases (CMDBs), help desk or service desk data, enterprise architectures, provisioning and deployment data, and virtually any data or reports associated with the operational aspects of IT.Business Data• – This includes information from procurement systems, contracts, human resource systems, locations and department breakdowns, and virtually any data or reports associated with the financial and planning aspects of IT – including manually populated spreadsheets.

A consolidated data quality approachprovidesamoreaccurate perspective on data, greater transparency of business functions, and the opportunity to gain deeper analysis of the envi-ronment.


Discovery and Utilization Data • – This includes data from network scans for machine assets and monitoring of software and hardware inventory and utilization. While this piece is deeper than is needed in some business analysis situations, it provides added resolution to data as to which assets are being used in a given environment. This allows analysts to determine how vital the assets are to analysis and if they can be factored out.

The Urge to ConvergeCollisions occur because the data itself is likely to take different formats or conventions. The procurement team might name a company “Acme” while the IT service desk calls it “Acme Corp.” The asset management team might also have misspelled the name in one instance of their database: “Amce,” while also spelled correctly in another instance.

The resulting analysis then becomes unusable, because if you were to run a report on all the vendors, you’d have multiple, un-aggregated versions of “Acme,” (for example, aligning licensing data or contract data with every incarnation of “Acme.”) Repeatedly articulating the “mutant” data, not to mention to reconciling records back and correct their source data, becomes a very manual, labor-intensive process. Consider as well that by the time the report has been cleaned up, the information relating to that record might have changed.

Software can help the convergence of the data, but most integration initiatives still require a trained eye to spot data quality errors. The ideal solution is an intelligent quality workflow: a process by which bad data is identified, its effects mitigated, its errors resolved and the corrections verified.

Analysis using faulty asset data can lead to confusing results.Eventhoughthedatacollectedmightbeac-curate, different string val-uesmayimpedeaccurateconclusions.Datathathasbeen cleansed and certi-fiedprovidesmorereliableanalysis.


Centralizing IT Business DataThe ideal data integration and quality solution is scalable and vendor-agnostic — the data model and integration processes support the widest array of data stores. This is particularly important to ITOs that are looking to build federated CMDBs, i.e. IT service management data built from multiple, possibly distributed sources.

One way to ensure robust integration of this very dissimilar data is to create a single data warehouse for IT business data. A data warehouse provides an ideal way to store and relate multidimensional data, which adapts to support the very kind IT business processes are faced with. The data warehouse provides a workspace for the data to be cleansed and reconciled, and can automatically load or receive data from multiple sources. An integrated data warehouse also eliminates the need for manual collection of this data and its subsequent analysis.

The data in the warehouse segues ideally to allow complex analysis by leveraging the principles and methodologies of business intelligence (BI). BI methodologies provide deep, on-going, repeatable analysis, drawing from multiple perspectives of data. Analysts are able to perform unplanned drill down examination and see relationships in information, as well as perform daily reporting needs. Essential to the success of a BI for IT initiative is to build the centralized data warehouse to include “all the data that matters,” and consistently qualifying it for use.

Building an IT Intelligence Data WarehouseBecause data sources vary widely, populating the data warehouse requires a wide set of techniques and tools to produce a warehouse that will perform optimally and support the diversity of information. The data warehouse can provide both a convergence of data, as well as a high-performance, refi ned look at data. In other words, data can be structured to optimally support various query methods used in analysis, reporting, decision support and control. This also provides the basis for a well-designed cleansing and normalizing strategy to ensure ongoing data quality.

As with solutions to any complex, multi-stage problem, a proven methodology minimizes risk and shortens the implementation window. Blazent provides a comprehensive approach for integrating IT business data while minimizing the time and resources required to establish a consolidated data warehouse. Blazent’s Data Quality approach consists of:

An initial “mass integration”• of IT management data to establish a current baseline for the data warehouse. Integration and cleansing at this stage is based on industry best practices and conventions (e.g. standardizing units of measure, merge priorities, text-to-numeral, etc.).A data quality workflow• for continuous certifi cation of changing, converging data. Your operations staff becomes intimately involved in the shaping of the data to fit your corporate conventions and standards.

Formoreinformationondatawarehousedesign,particularlyfortakingad-vantage of business intel-ligencemethodologies,theKimballGroupoffersexcel-lentprimeranddiscussionmaterials(www.kimball-group.com).

Formoreinformationondata quality approaches, MIT’sTotalDataQualityManagementprogrampro-vides the latest research on paths to achieving reliable data(web.mit.edu/tdqm).


Data Integration (Initial Cleanse from sources)The key to fast and effective implementation of IT management data integration is that Blazent leverages an ITO’s best practices with a data warehouse built from Blazent’s expertise.

Determine sources and build integration processes • – Including IT and business data stores, such as: CMDBs, Asset Management repositories, HR data, contract and procurement data, ERP systems, network scanned discovery information, monitoring agents, as well as manually entered data. This provides a defi nition of information that will drive the business perspective of IT. Uniquely vendor agnostic, Blazent also determines what the best methods are for integrating this data depending on the source, such as ETL (extract, transform and load), direct integration between applications, or if real time information is required to replicate and propagate content into the data warehouse, etc.

Determine the scalability and performance needs• – Data warehouses require significant attention to optimizing the data for various analytics and reporting objectives and positioning them to be scaled according to the needs of your business. Blazent architects its intelligence data warehouse for maximum fl exibility. In addition, Blazent provides the expertise to organize the data warehouse into report ready data marts that maximize performance by focusing on key IT process management questions.establish data rules based on business requirements and conventions – •Determines what data is truly meaningful to the analysis of the ITO’s business processes. Also performs preliminary cleansing of data to provide consistent fi elds, formats, and merge rules upon collection of information across all sources.Audit asset data to establish gold standard• – Once integration of data is complete, Blazent provides a consolidated view of IT information from which ITOs have a reconciliation path to referenced data and feed back to their support systems for accuracy across the enterprise. For example, correcting service desk data stores with data from ERP systems or vice versa. The consolidate data view also positions the data for ongoing refinement as it changes.

Blazent’sRecordSourceDetailidentifieshowaconsolidatedviewofITbusiness data is collected frommultiplesourcestodeterminethemostaccu-rateinformation.Thisdatacanbereconciledbackintosourcesystemstocompletethequalityloop.


Data Quality Workflow (ongoing Improvement)No software can completely automate the data quality process. That’s because all humanreadable data requires a certain amount of human intervention to determine its meaning. To continuously improve data quality in a changing environment, an intelligent workflow between the tools and role-players must be established.

Blazent streamlines processes to automate quality improvements wherever possible, minimize intervention required during error identifi cation and certification, and mitigate the impact of poor or incorrect data quality. With training and support, Blazent can have a cross-discipline IT team quickly acclimated to managing the daily quality tasks.

The Quality Team WorkflowThe quality workflow consists of four phases:

Identification• – Gain visibility into data problems with highlighted and actionable reports. Issues and errors are marked for cleansing.Mitigation• – Questionable data is withheld from analysis to reduce its impact until resolved and approved.Resolution• – Data or approach to integration is corrected manually at the source (e.g. service desk, license management systems, etc.) or within Blazent, or automatically transformed and remapped with data rules. Remapping ensures that future data is automatically corrected.Validation• – The data is verifi ed as corrected through a series of reports and checks.

Blazent’squalityworkflowoptimizestheprocessofidentifying,mitigating,resolving and validating data.Italsoprovidesadi-rectmethodforreconcilingsource data located in other servicesupportsystems.


Quality Team RolesThe quality team consists of:

Business Analysts• – Those who work with the data will identify issues and mark the record for change by the administrators. They also validate changes are made.system Administrators• – Receive requests to modify data. Depending on the source, data may need to be put “on hold” to prevent false analysis until it can be corrected and remapped.

Resolution ProcessThe goal of any resolution process is to mitigate impact, accelerate corrections to source data and eliminate or reduce future errors. Blazent provides the ability to mitigate the impact of bad data by putting it on hold until it can be corrected, reducing its effect on analysis.

Blazent also provides key administrators a “trouble ticket” approach to correcting source data. Analysts can recommend values, which are reviewed and acted on by the responsible administrator. This activity is then recorded within the system for historical review.

Administrators can then decide how to reconfi gure global fi lters and merge rules within Blazent to normalize data issues. For example data strings such as “Acme Corp,” “Acme Corporation,” “Acme Inc.,” and “Amce” all can all map to “Acme.” This can also be useful when dealing with spelling errors and data with improper formatting, where the source cannot be corrected immediately.

Blazentfeaturesa“troubleticket”styleworkflowtomanagedataquality,recon-cilesourcedata,andkeepa historical archive of data qualitydecisions.


Quality ProblemsA myriad of data quality problems can emerge when data is integrated from multiple, diverse sources. Some of the data quality problem types Blazent provides cleansing solutions for are:

synonyms As in the previous “Acme” example, synonyms are where different strings refer to essentially what is commonly identified as the same. This can add considerable noise to final reports that skew report clarity. Blazent can be confi gured to treat different incarnations of these strings as the same.

Corrections Typos, inconsistent naming conventions, etc. Again, this can add considerable noise to final analysis. Blazent allows global filtering to treat the data before it reaches an analysis stage.

Invalid Data Can occur when format or leading special characters undermine expected data. This is a clear use for mitigating data so that it remains “Under Review” so it affects the analysis until the source is corrected. If it’s determined that the data is merely inconsistent or incorrect, an adjustment at the ETL stage may be required. It should be also noted that Blazent has unit-of-measure converters to avoid any issues where inconsistent data is a result of the wrong unit of measure.

Missing/ Similar to invalid data, missing or incomplete data can undermineIncomplete analysis and should be mitigated to prevent erroneous reports. OneData way to investigate the cause of missing or incomplete data is to use

Blazent’s Record Source Detail to determine how the consolidated view was built and which sources are reporting improper data.

Incorrect Level Data is either summarized above the needed detail, or detailed of Data below the level of analysis. Too much detail is not useful and pre-

summarized data is problematic since cohesive warehouses are “aggregate consistent;” i.e. aggregations always add up, so insertion of higher level data can violate this. Blazent aggregates data that is too detailed during its staging or warehouse loading operations. If the data is not detailed enough, Blazent connectors and interfaces allow collection of more detailed data from source systems.

noisy Keys When a source system has “questionable” accuracy on key fields. For example, if serial number is being used as the key fi eld in merging Asset Management data with Purchasing data but is blank in the purchasing system 25% of the time. This is a common problem that arises when key fi elds for merging of data are not key fields in the source system and thus are not required entries/monitored/validated. As a result, the quality data in those fields is lower. The same process for missing/incorrect data can also be applied.


Measuring successThe ultimate measure of success of a data quality methodology is that the data leads to clearer, more accurate analysis and reporting to drive better business decisions. Some quantifiable metrics to focus on are:

Less invalid data• – leading to greater confi dence in analysis and decisionsLess confusion in end reports and analysis• – for greater communication across business unitsGreater accountability of staff• – due to added integration process transparencyRepeatable cleansing processes• – for faster, more predictable results

Beyond these organization-wide metrics, Blazent offers high-level views of data integrity through its Administrator Reports. The reports provide various checks across source data to identify quality issues.

summaryAs the goals of business and IT converge, so must the data that describes it. Centralizing that data and maintaining its accuracy requires a comprehensive understanding of the challenges in integrating and qualifying that data. Only then can businesses reach an understanding of their IT environments and perform analysis that will drive its maturity.

Blazent’s business intelligence approach of creating a data warehouse for IT businessmanagement data is developed with an acute methodology and industry leading vision and features that include:

Proven advanced data model•Vendor-agnostic source data acquisition•Reconciliation path interoperable with other data sources•Unique focus and expertise•Automating real-time collection and verifi cation•Seamless transition to actionable analytics and business intelligence•

For more information on Blazent’s data quality solution, visit the website at www.blazent.com or contact a sales representative directly at 650.286.5500.

Blazent, Inc.1820 Gateway Drive, Suite 200San Mateo, CA 94404TEL 650.286.5500FAX 650.286.1956www.blazent.com

Centralizing and Qualifying IT Business Data

Documents

Transcript of Centralizing and Qualifying IT Business Data