DC Business Intelligentsia January Meetup: Agile BI and Data Chaos

How does Agile prevent common BI problems?

January 2014 www.SynaptitudeConsulting.com 2

http://www.google.com/url?sa=i&rct=j&q=&esrc=s&source=images&cd=&cad=rja&docid=d9CrdPpeqzeMzM&tbnid=NRv0A0RCUePVuM:&ved=0CAUQjRw&url=http://themarketingrobot.com/future-business-intelligence&ei=qZvWUsqbDeLjsASotID4BA&bvm=bv.59378465,d.eW0&psig=AFQjCNEk4kcXKgKTRM-npHJ1940AqAB2xQ&ust=1389882205194735

Business Intelligence and The Enterprise


Reporting is useful at the department level.

Business Information is useful only across the Whole Business.

http://www.google.com/url?sa=i&rct=j&q=&esrc=s&source=images&cd=&cad=rja&docid=d9CrdPpeqzeMzM&tbnid=NRv0A0RCUePVuM:&ved=0CAUQjRw&url=http://www.informationbuilders.com/solutions/operational_bi&ei=IJzWUtBPkuWwBIb8grAM&bvm=bv.59378465,d.eW0&psig=AFQjCNEk4kcXKgKTRM-npHJ1940AqAB2xQ&ust=1389882205194735

Enterprise BI Architecture


http://www.google.com/url?sa=i&rct=j&q=&esrc=s&source=images&cd=&cad=rja&docid=qGBSMF4PYCjFdM&tbnid=LzlTu5kEC6zvKM:&ved=0CAUQjRw&url=http://www.highpoint-solutions.com/technologyServices/businessintelligencesolutions/mppplatforms.asp&ei=R5vWUteCHtLNsQTitYHICA&bvm=bv.59378465,d.eW0&psig=AFQjCNEk4kcXKgKTRM-npHJ1940AqAB2xQ&ust=1389882205194735

Diminishing Returns Increased costs associated with collecting, processing and disseminating data, relate to the increase in benefits. What is essential is to evaluate where the cut-off point is for optimum benefits at a realistic economic cost.


But to provide actionable intelligence across the enterprise don’t you need to capture all the data in the enterprise?


http://www.google.com/url?sa=i&rct=j&q=&esrc=s&source=images&cd=&cad=rja&docid=HjGbWzWCpSUUzM&tbnid=JX8DjBbrtM6F1M:&ved=0CAUQjRw&url=http://alekzdz.blogspot.com/&ei=fK7WUo3JMo2isASYlICQCg&bvm=bv.59378465,d.eW0&psig=AFQjCNGTYaZUAOTROeL_wzVAH0g8Uv3sZg&ust=1389887362932991

Traditional Approach to Building the Factory – The Data Warehouse

June 2012 www.SynaptitudeConsulting.com 7

Traditional Approach to Building the Factory


Traditional DW/BI methodology has a heavy emphasis on requirements gathering, because you want to pre-calculate/pre-model all the answers. You need to know all the questions that might get asked. But if you are working with new information, new data, new business dynamics – how do you know what you don’t know?

Understanding the quality of the data, and the effectiveness of controls, is critical to maintaining the underlying DW environment.

But I need information now! Building data warehouses takes years and $millions

But I have to deliver ROI on all this new gear and systems soon than that, and I need information to do so…..Can I just get some reports?

And maybe a dashboard or 2?

And maybe then some analytics/forecasting/predictive…


A simple 5-step process to avoiding data chaos while building the factory

1. Find your data Current systems, other capabilities (available data that is not

currently collected, or is being collected but not utilized)

2. Evaluate your data Understand how your data is actually behaving vs. how you

expected it to behave Assess data quality across stacks/domains

3. Start with some departmental initiatives 4. Transform your data into information

Turn data into information by mapping to the enterprise model (the data warehouse thing…)

5. Repeat periodically Your company doesn’t stand still, and neither does your data…


Agile BI

So, what to do – Agile BI Deliver actionable results in weeks vs. years

Discover how your data is behaving before you design your information factory


Because as we know, Agile is the cure for everything IT

BI Characteristics and Goals (Kenochan 2-

year study)


Data entry — accuracy

Data consolidation — consistency

Data aggregation — scope

Information targeting — fit

Information delivery — timeliness

Information analysis — analyzability

Common BI Issues (Kenochan)


20% of data has errors in it (accuracy) 50% of data is inconsistent (consistency) It typically takes 7 days to get data to the end user

(timeliness) It isn't possible to do a cross-database query on 70% of

company data (scope) 65% of the time, executives don't receive the data they need

(fit) 60% of the time, users can't do immediate online analysis

of data they receive (analyzability) 75% of new key information sources that surface on the

Web are not passed on to users within the year (agility)

Agile addresses BI Issues Data Entry - Processes need to re-evaluated and corrected to minimize data

entry errors. Data Consolidation - Often companies have multiple data stores and data is

scattered across multiple data stores. "Agility theory emphasizes auto-discovery of each new data source, and automated upgrade of metadata repositories to automatically accommodate the new information".

Data Aggregation - Is a process in which information from many data stores is pulled and displayed in a summary report. OLAP is a simple type of data aggregation tools which is commonly used.

Information Delivery - One of the key principal of Agile BI is to deliver the right data at the right time to the right individual. Historical data should also be maintained for comparing the current performance with the past.

Information Analysis - One of the largest benefits of Agile BI is in improving the decision-making of its users. Real Agile BI should focus on analysis tools that make an operational process or new product development better.[


(Kenochan) Agile BI Key Principals

http://en.m.wikipedia.org/wiki/Agile_Business_Intelligence#cite_note-What_Agile_Business_Intelligence_Really_Means-10

Agile Best Practices also address the Data Issues A program charter should be created, which will set the stakeholder

expectations on how Agile BI system will work. Start with the business information needs to provide context for scope. Iterations should be time-boxed. Stress on data discovery through the requirements and design phase. Use the Agile process of incremental and iterative development and

deployment. Validate the BI Architecture and get approval on the proof of concept. Data validation and verification should be completed for each development

iteration. Use flow charts or diagram to explain the BI process along with some

documentation. Any change that will be deployed to production should be thoroughly tested in

regression environment. Have a formal change control; this will minimize the risk as all changes have to

be approved before it goes into production.


(Larson)


(Kolko)

So, if Agile solves the BI Data Issues, why do we continually fail to deliver Quality, Actionable Data

ACROSS THE ENTERPRISE?

Enterprise BI vs. Departmental BI

Agile Limitations

Since Agile cannot deliver across the Enterprise, Agile BI cannot deliver across

the Enterprise


Enabling Enterprise Action vs. Departmental Intelligence – Some Observations for Discussion

Agile Development is fairly straightforward to execute at a departmental level

Therefore, Agile BI is fairly straightforward to execute at a department level

Generally, Agile Development falls apart at the Enterprise Level

Scaled Agile Framework

Generally, Agile BI falls apart at the Enterprise Level

Solution?


Torture the data, and it will tell you anything… January 2014 www.SynaptitudeConsulting.com 20

DC Business Intelligentsia January Meetup: Agile BI and Data Chaos

Technology

Transcript of DC Business Intelligentsia January Meetup: Agile BI and Data Chaos