ADVANCED SITE OPTIMIZATION
Transcript of ADVANCED SITE OPTIMIZATION
ADVANCED SITE OPTIMIZATION Second-Generation Digital Analytical Tools in Action Eric T. Peterson, Web Analytics Demystified, Inc.
Sponsored by
Advanced Site Optimization: Second‐Generation Digital Analytical Tools in Action
© 1999 ‐ 2010 Tealeaf Technology, Inc. All Rights Reserved.
TABLE OF CONTENTS
Executive Summary ........................................................................................................ 2
The Challenge with Digital Analysis Today............................................................. 3
Data Volumes Create Their Own Problems..................................................... 3
“Web Analytics” Applications Lack Analytical Capabilities ........................ 4
Analysis Staff Aren’t Trained for Analysis ...................................................... 5
Net Result: Insights Are Slow to Materialize................................................... 5
Advanced Site Optimization ........................................................................................ 6
Putting Data to Work for You (Not the Other Way Around)........................ 6
First Steps Towards the Coming Revolution ................................................... 6
Second‐Generation Digital Analysis Tools ............................................................... 7
Multivariate and A/B Testing............................................................................. 8
Analytics Intelligence .......................................................................................... 9
Tealeaf 8............................................................................................................... 10
Conclusions.................................................................................................................... 14
About the Author .......................................................................................................... 15
About Web Analytics Demystified ........................................................................... 15
About Tealeaf ................................................................................................................ 15
Advanced Site Optimization: Second‐Generation Digital Analytical Tools in Action
© 1999 ‐ 2010 Tealeaf Technology, Inc. All Rights Reserved. 2
EXECUTIVE SUMMARY
Given the increasing importance of connected channels to nearly all businesses and government
functions, it is no wonder that analysis of these channels is getting more attention. Digital analysis is the
practice of making decisions using digitally collected qualitative and quantitative data by leveraging
clickstream, feedback, and customer experience management solutions. Powered by business process and
performed by digital analysts, digital analysis is purported to have the ability to dramatically transform
business outcomes.
Except that it doesn’t.
Nearly a decade into the digital analysis sector’s growth, cracks in the foundation are beginning to show.
Business leaders who have committed significant resources still complain about getting too much data,
too few insights, and not nearly the return on investment promised.
At Web Analytics Demystified it is our belief that this situation arises from a “perfect storm” of
challenges relating to people, process, and technology. Widely available digital measurement platforms
are decreasingly able to keep up with the volume of data generated by mobile, social, and traditional
digital marketing and communication channels of every business. What’s more, while the volume of data
needing analysis continues to grow, the few existing dedicated resources are dragged down by aging
business and knowledge management processes.
Fortunately, a new breed of technologies is emerging — second‐generation digital analysis applications
— within the web analytics, customer experience management, and testing and optimization sectors.
These applications and platforms bridge the gap between basic reporting solutions and the type of “big
iron” business and customer intelligence solutions commonly found in the offline enterprise. By moving
beyond data for data’s sake into the realm of statistical models and data for analysis sake, these vendors’
customers are pushing the limits of what is possible in digital channels and creating a competitive
advantage in the process.
Advanced Site Optimization: Second‐Generation Digital Analytical Tools in Action
© 1999 ‐ 2010 Tealeaf Technology, Inc. All Rights Reserved. 3
THE CHALLENGE WITH DIGITAL ANALYSIS TODAY
The promise of digital analysis, using tools like web analytics, has always been about surfacing high‐
value insights driven by the tremendous volume of data produced through digital channels. These
insights would then be presented to management in the form of recommendations and, when enacted,
the visitor experience would be continually improved. This process would be repeated and, over time,
key measures of business success like revenue, profit, and satisfaction would show substantial gains.
Unfortunately, this rarely happens.
Most business leaders today are frustrated with the lack of information and recommendations flowing
from their analytics efforts, despite substantial investments in technology and consultants. According to a
2009 study from Aberdeen Research1, 28% of companies described as “best‐in‐class” admitted that they
still struggle to interpret data provided by their web analytics solution. Reasons for this struggle include
lack of insight into how to convert metrics into tangible actions, lack of resources, difficulty associated
with data extraction, and difficulty with interpretation. The net result, according to Aberdeen, is that
nearly one‐third of companies paying for web analytics do not believe they are getting the full value from
their investment.
This data is corroborated by findings from the Web Analytics Association’s Outlook 2010: Survey Report2
where nearly 70% of respondents indicated that a key initiative in 2010 would be “acting on data to
improve site performance,” followed by “making business decisions driven by analytics” (63.5%). The
same report cited that 39% of respondents were challenged by taking action on available data, 37.6%
struggled to make business decisions driven by analytics, and 37.2% struggled to gain executive
management awareness and support for web analytics efforts.
At Web Analytics Demystified we believe this unfortunate situation arises directly from a “perfect storm”
of challenges — too much data being fed into inadequate systems and run by undertrained resources.
DATA VOLUMES CREATE THEIR OWN PROBLEMS
The Internet is like no other source of data that has been analyzed by business owners. Whereas offline
efforts to understand customers are typically limited to “known” individuals or broad samples based on
demographics or psychographics, digital channels create an almost unmanageable source of data, one
that grow with every mouse movement, click, and interaction. Every single thing each one of us does
online is tracked somewhere by someone, usually with the goal of “optimizing interactions and driving
conversion.”
Problem is, for many sites having access to this much data creates issues rather than solving them.
Especially when dealing with multiple disparate systems such as email marketing systems, social
networks, ad networks, and web delivery platforms, the hoops that data managers need to jump through
to connect each individual’s interaction are arduous at best. The result is usually independent reporting
silos, each of which has a tendency to produce their own version of “the truth.”
1 http://www.aberdeen.com/Aberdeen‐Library/6048/RP‐web‐analytics‐marketing.aspx 2 http://www.webanalyticsassociation.org/resource/resmgr/docs_research_committee/waa‐outlook‐survey‐report‐20.pdf
Advanced Site Optimization: Second‐Generation Digital Analytical Tools in Action
© 1999 ‐ 2010 Tealeaf Technology, Inc. All Rights Reserved. 4
Highly motivated organizations have had some success developing integrated data models, often built on
top of costly data warehouses. Most businesses simply try and make do with their data silos, hoping that
analysis resources will reconcile the differences and provide a consistent and believable assessment of the
online opportunity using the technology they have at their disposal. Unfortunately, collecting and
keeping “data for data’s sake” nearly always fails to produce the type and level of insights most
companies expect.
“WEB ANALYTICS” APPLICATIONS LACK ANALYTICAL CAPABILITIES
Despite assurances to the contrary, the majority of technologies deployed for “web analytics” today do
very little analysis of data. While certainly improving, most are little more than glorified reporting
interfaces allowing for a small amount of customization and segmentation. The most powerful solutions
— advanced data visualization and segmentation suites like Omniture Insights, Coremetrics Explore, and
Unica’s NetInsights — do an amazing job of summarizing and packaging the data but stop short of
putting that data to use algorithmically and mathematically.
When pressed to explain why their “analytics” applications don’t do any real analysis — either by
leveraging statistical models, supporting predictive analytics, or simply providing confidence intervals
and quality of fit scores for segmented data — the answer inevitably comes to “we didn’t think our users
would understand that type of output.” While this may have been true in the past, and to some extent
remains true today, the result is a profound gap between what the systems could do for the business and
what they actually do.
This gap manifests in a variety of ways:
Instead of highlighting changes in rates and measures based on statistical anomalies designed to
call attention to unexpected changes, most systems simply present line graphs and require
analysts to export the data into Excel;
Instead of providing ongoing real‐time analysis — potentially across multiple dimensions of data
— that allows analysts and business leaders to respond in real‐time to emerging challenges, most
systems expect that the business can’t respond quickly and assume that data latency is
acceptable;
Instead of extrapolating from existing data to make even basic predictions about potential
changes — thereby enabling business owners to respond proactively, revise forecasts, and
otherwise be prepared — most systems focus on providing information about events that have
already occurred.
Expedia.com has long been known for their aggressive approach towards site testing and conversion
optimization. The company regularly conducts “Opportunity Cost Analysis” — essentially an analytic
that uses statistical modeling to differentiate the likelihood to convert visitors experiencing problems
versus those who do not. Using Tealeaf, Expedia focuses on specific steps in a business process flow and
develops different segments based on customer behaviors. For example — conversion rate for customers
that triggered form validation messages when entering personal information versus those who did not.
Expedia then quantifies the opportunity cost for customers not converting by applying an average
transaction value for similar types of transactions.
While this effort currently requires that Expedia export data from their customer experience management
environments for offline analysis, the value to the business is clear: one corrected problem returned an
Advanced Site Optimization: Second‐Generation Digital Analytical Tools in Action
© 1999 ‐ 2010 Tealeaf Technology, Inc. All Rights Reserved. 5
estimated $2 million to $3 million this year and, given that the problem had existed for some time, is
likely to lead to tens of millions of dollars in incremental revenue over time. By using this “Opportunity
Cost Analysis” analytic, Expedia is able to prioritize projects by the benefit to their business. This analytic
has also enabled them to manage staffing levels focused on site optimization versus other competing
internal projects, thereby ensuring that the business is working on projects with the greatest return.
ANALYSIS STAFF AREN’T TRAINED FOR ANALYSIS
Given the limitations of the majority of “web analytics” applications sold today, the response from the
business has been to apply human resources to fill the gaps. Problem is, these operators — “web
analysts” by designation and trade — are rarely trained as analysts in the classical sense, most entering
the field by accident rather than intent. Even worse, because vendors have done such a good job of
convincing their customers that “web analytics is easy” and “the technology practically runs itself,” too
few organizations have even searched for and hired web analyst resources, regardless of their experience
and education.
According to the Web Analytics Association’s Survey 2010 report, 47% of respondents indicated three
years or less experience in the field and 76% had six years or less experience in the field. Compared to
more traditional analyst roles in business intelligence and financial analysis, where training and
traditional education are firm requirements for employment, a somewhat disturbing pattern emerges:
business leaders are forced to assign increasingly high‐value analysis projects to immature technology
managed by potentially inexperienced human resources.
We don’t blame the analysts or the business for this situation — everyone is simply doing what they can
in a rapidly changing environment. Remember, five years ago there was no Facebook; three years ago
there was no Twitter; two years ago there was no iPhone; last year tablet computing was something that
had been tried and had failed. The problem is not as much the people, data, or technology; it is how the
technology leverages the data so that the people can do the job they’ve been asked to do.
NET RESULT: INSIGHTS ARE SLOW TO MATERIALIZE
The net result of the current state is that most businesses struggle to translate the volumes of digitally
collected data into meaningful insights, much less solid business recommendations. What’s worse is that
we at Web Analytics Demystified see the potential for a “perfect storm” in digital analytics: senior
leadership is increasingly asking for more advanced, more robust, and more valuable analysis, but data
volumes are increasing exponentially, in many cases without a commensurate increase in the capabilities
of most available data analysis applications.
Fortunately, there is hope on the horizon coming in the form of systems and methodologies designed to
accelerate time to insight and dramatically improve the value existing data provides. These systems do
more with the available data, ranging from rapid, statistically powered design iteration support to full‐
blown, model‐based analysis to more effectively surface insights.
Advanced Site Optimization: Second‐Generation Digital Analytical Tools in Action
© 1999 ‐ 2010 Tealeaf Technology, Inc. All Rights Reserved. 6
ADVANCED SITE OPTIMIZATION
As discussed in length in our SAS‐sponsored white paper “The Coming Revolution in Web Analytics3,” Web
Analytics Demystified believes we are on the cusp of a paradigm shift in how companies analyze and
value digitally collected data. Increasingly we see movement towards what we described as “second‐
generation” digital analysis tools — applications and platforms that allow for both greater depth of
exploration of existing data and begin to automate the surfacing of insights in ways that first‐generation
tools simply do not.
PUTTING DATA TO WORK FOR YOU (NOT THE OTHER WAY AROUND)
Fundamentally, advanced site optimization is about getting your data to work for you, not requiring you
to continue to work for your data. Again, the “status quo” in web analytics today is at worst what my
friend Dennis Mortensen of Yahoo! has coined “report surfing4” and at best, hypothesis‐driven research
that requires tremendous attention to detail to account for myriad nuances in data collection, data
processing, and systems management. In either situation the analyst tasked with converting data into
information, information into insights, and insights into recommendations is required to spend hours
sifting through data to ensure that A) a change is manifest and B) the change manifest is significant.
This is tremendously wasteful, especially in the context of the limited number of truly experienced digital
analysts working in the market today and the fact that the offline world has done a pretty good job of
leveraging data, statistical models, and predictive systems to focus analysis. In fact, when it comes to
“time to insights” — despite the common complaint about business and customer intelligence initiatives
— online analysts have a lot to learn from their offline counterparts.”
FIRST STEPS TOWARDS THE COMING REVOLUTION
The observation that most web analytics efforts today are not particularly efficient juxtaposed against the
demonstrable value of model‐based systems has long provided Web Analytics Demystified with only one
possible outcome: web analytics practitioners and digital marketers need be begin to adopt more robust
and capable tools, technologies, and processes. As we described in detail under the header of “How [web
analytics] Technologies Fail Us Today” in The Coming Revolution in Web Analytics, business owners,
marketers, and analysts alike need more powerful tools to augment their reactive reporting efforts today.
Referencing Tom Davenport and Jeanne Harris in Competing on Analytics, we agree that a sustainable
competitive advantage built around analytics will manifest only when the business has addressed both its
need for ongoing reporting and advanced analytics. In Figure 1, the technologies Davenport and Harris
propose to create this type of advantage are outlined:
3 http://www.webanalyticsdemystified.com/content/white‐papers.asp 4 http://visualrevenue.com/blog/2007/09/web‐analytics‐report‐surfing‐and‐how‐to.html
Advanced Site Optimization: Second‐Generation Digital Analytical Tools in Action
© 1999 ‐ 2010 Tealeaf Technology, Inc. All Rights Reserved. 7
Figure 1: Reporting and Analytics Tools and Capabilities, taken from Competing on Analytics (Davenport and Harris, 2007.)
The red line is our addition, indicating the break between first‐generation and second‐ and third‐
generation digital analytics tools. Everything below the line is essentially “web analytics as we know it
today” and everything above the line is “digital analytics as we will do it tomorrow.”
When we published this white paper, many in the community responded negatively, essentially
concerned that the technology and processes above the line were difficult to use, expensive, and
fundamentally outside their area of expertise. While this may be true, it is difficult to reconcile complaints
that are primarily personal against the needs of the broader business. The last concern, cost, is reasonable,
although in our research we discovered that most enterprise‐class companies actually have many of the
necessary technologies already in place. All that is required is to bridge the gap between “online
analysts” and “business analysts” within the business.
That said, we at Web Analytics Demystified pride ourselves on giving practical, real‐world advice and
guidance and, given this response, we sought examples in the market today of the type of second‐
generation tools and technologies likely to catalyze the paradigm shift we have described. Fortunately,
we found a good partner for this work in Tealeaf, this paper’s sponsor, who have recently released an
excellent example of technology that’s capable of putting the data to work for the business by leveraging
well‐designed statistical models applied to digitally collected data. We also found examples in Google
Analytics “Analytics Intelligence” feature and the increasingly adopted Multivariate and A/B testing
platforms provided by vendors like Google, Omniture and Autonomy.
SECOND‐GENERATION DIGITAL ANALYSIS TOOLS
Business owners wishing to get a head start on transforming their digital analytical efforts and leading
the revolution are in luck. The past year has seen two new second‐generation tools to complement
existing testing platforms, both taking good advantage of the available data to surface otherwise non‐
obvious insights and evidence of consumer “struggle” online.
Advanced Site Optimization: Second‐Generation Digital Analytical Tools in Action
© 1999 ‐ 2010 Tealeaf Technology, Inc. All Rights Reserved. 8
MULTIVARIATE AND A/B TESTING
As discussed in our paper on The Coming Revolution in Web Analytics, we believe that A/B and
Multivariate testing tools provide an excellent example of analytical systems leveraging digitally
collected data to focus business decisions. Without going into detail, these systems essentially power
controlled experiments, exposing a sample of site visitors to alternative messages in an attempt to
determine which message is most likely to drive the desired outcome.
The important thing in the context of second‐generation tools is that these systems rely entirely on
statistical modeling to determine the “winner” in an experiment. The model usually depends on both
participation and outcome data — for example, people being exposed to the experiment and then going
on to complete whatever measurable success event is being tracked. In this context, the results are not
presented as “this many people did X” but rather “with a 95% confidence the model finds that X is 10%
better than Y.”
Figure 2: Example results from a multivariate test in Google Website Optimizer showing the system’s prediction and confidence
that “Combination 2 ‐ Own a Business” will outperform the control group (47.8% improvement with 96.5% confidence)
Web Analytics Demystified’s point of view is that A/B and Multivariate testing systems are incredible
second‐generation digital measurement systems for three main reasons:
1. They shift expectations away from whole numbers and raw counts towards samples and
confidence intervals;
2. They are able to dramatically simplify reporting, focusing on “winners” and “losers” without
needing huge spreadsheets;
3. They explicitly point the business toward logical actions — for example, “widely deploy
combination number two and increase your conversion rate.”
While testing continues to gain significant traction within the enterprise, these systems are oftentimes
difficult to deploy and require a specific set of expertise in order to reap their full benefits. The same
cannot be said for the next newest second‐generation digital analytics tool provided by Google.
Advanced Site Optimization: Second‐Generation Digital Analytical Tools in Action
© 1999 ‐ 2010 Tealeaf Technology, Inc. All Rights Reserved. 9
ANALYTICS INTELLIGENCE
Google’s Google Analytics offering initially gained traction in the marketplace as an excellent entry‐level
tool, capable of measuring a reasonable number of common digital activities and available at the best‐
possible price — free. However, in the years following the application’s launch, the team at Google has
built‐out the product to create one of the most user‐friendly and useful clickstream measurement tools in
the web analytics industry. They currently set the bar for a variety of analytical needs, including custom
data collection, segmentation, and custom reporting.
In October 2009, Google took Google Analytics to an entirely new level by introducing second‐generation
reporting capabilities, an industry first, into the application with their Analytics Intelligence feature. Still
in beta, according to Google:
“[Analytics Intelligence is] the initial phase of an algorithmic driven Intelligence engine to Google
Analytics. Analytics Intelligence will provide automatic alerts of significant changes in the data
patterns of your site metrics and dimensions over daily, weekly and monthly periods.”
Figure 3: Example of Analytics Intelligence in Google Analytics. The bars at the bottom indicate the number of alerts generated
based on the alert sensitivity (a proxy for number of standard deviations)
While helping to identify the days that statistically significant changes are occurring is one thing, going
one step deeper and highlighting the basis for those changes is something else completely.
Advanced Site Optimization: Second‐Generation Digital Analytical Tools in Action
© 1999 ‐ 2010 Tealeaf Technology, Inc. All Rights Reserved. 10
Figure 4: Example of drill‐down in Analytics Intelligence in Google Analytics highlighting the dimensions of data prompting the
alert to be generated, the expected range and observed values, and the relative significance (gray bars).
While Google’s offering is an excellent example of second‐generation functionality, critics aptly point out
that Google’s use of statistics is somewhat rudimentary. More importantly, Google’s data is often delayed
by several hours or more, thus limiting the application’s real utility as a insight discovery engine. This
problem has been solved by this paper’s sponsor, Tealeaf, in their most recent offering — Tealeaf 8.
TEALEAF 8
Tealeaf has long been known for their enterprise‐class capabilities for monitoring visitor and customer
behavior in the web and mobile channels. Their customers — a list that includes Expedia.com, Best Buy,
Walmart.com, Comcast, Wells Fargo, and hundreds of other name brands across all sectors and
geographic segments — have long used the company’s applications to help identify the points where
visitors struggle and fail. Through integrations with popular clickstream, feedback, and customer
relationship management applications, Tealeaf has become an integral component in the enterprise site
optimization stack.
Because Tealeaf is physically installed in the network architecture for a site, the application is able to
provide true real‐time feedback on visitor behavior and has tremendous oversight into online
interactions. This capability imparts a variety of advantages to Tealeaf customers, including up‐to‐the‐
moment key performance indicators, rich segmentation capabilities and browser replay — the feature for
which the company has historically been known.
However, in the context of advanced site optimization, the company’s most recent application upgrade
(Tealeaf 8) takes automated discovery to an entirely new level. Similar to Google’s Analytics Intelligence,
Tealeaf leverages statistical models to evaluate whether changes found in the data are significant. Unlike
Google Analytics, Tealeaf 8 tracks customer behaviors within segments, in real‐time, and gives system
users almost unlimited drill‐down capabilities to quantify the actual revenue impact and explore the root
cause of observed changes.
Advanced Site Optimization: Second‐Generation Digital Analytical Tools in Action
© 1999 ‐ 2010 Tealeaf Technology, Inc. All Rights Reserved. 11
Figure 5: Example of Top Movers and Drivers dashboard provided in Tealeaf 8 illustrating how Tealeaf uses statistical modeling to
automatically uncover struggle and other customer behavior issues within specific customer segments, in real‐time.
While there has been a long‐standing debate over the value of “real‐time analytics” within the digital
measurement community, Web Analytics Demystified’s opinion has always been that there is a certain
class of businesses that require immediate reporting. High‐volume retail, financial services, travel and
transportation, and organizations that depend as much on customer support as sales (e.g., Comcast,
DirectTV, Sky) all have sound financial and satisfaction‐related drivers to uncover challenges and
opportunities as quickly as possible, especially in the social era.
Case in point: Matt Raines,Vice President of Technology at online retailer BlueFly, notes that because
BlueFly focuses on on‐trend and in‐season clothing, any service outage or inability to transact is
unacceptable. Recently, one of the company’s credit card processing partners made a seemingly harmless
change that stopped BlueFly’s ability to transact using a handful of credit cards. Because this change
occurred at midnight, and because customers were getting only generic errors, the problem was slow to
surface.
With Tealeaf 8 in place, BlueFly has the ability to surface obscure, high‐impact issues like this one before
they become significant points of struggle for their customers. According to Mr. Raines, “Tealeaf provides
my team the ability to identify, diagnose, and resolve problems like this before they become an issue
across the entire company. The real‐time, automated insights provided by Tealeaf are the difference
between a response measured in hours and one measured in days for me, and that is huge.”
Advanced Site Optimization: Second‐Generation Digital Analytical Tools in Action
© 1999 ‐ 2010 Tealeaf Technology, Inc. All Rights Reserved. 12
Figure 6: Example of revenue impact analysis and drill‐down to visual session replay using Tealeaf 8.
Another Tealeaf customer, ING DIRECT USA, the nation’s largest direct bank, has been using Tealeaf’s
analytical capabilities to quickly identify problems on the site. Once identified, analysts work to
understand the nature and severity of the challenge to better communicate within the organization. This
effort, powered by Tealeaf, reduces the amount of “uncertainty” when problems emerge, allowing ING
DIRECT USA to make the best decisions possible based on data and intuition, not one or the other.
“We rely heavily on Tealeaf for tracking critical processes on our sites,” says Ethan John, Manager at ING
DIRECT USA. “Because we are an agile development shop, real‐time monitoring of our critical online
business and customer processes gives us the ability to identify, size, and proactively respond to
customer needs. Output from the analytical capabilities in Tealeaf has become part of our daily scorecard,
and as a result Tealeaf‐driven insights have widespread acceptance within the business.”
Advanced Site Optimization: Second‐Generation Digital Analytical Tools in Action
© 1999 ‐ 2010 Tealeaf Technology, Inc. All Rights Reserved. 13
Figure 7: Example of Tealeaf 8 management dashboard providing a snapshot of key customer experience indicators.
Another Tealeaf customer, Expedia.com, identifies issues impacting conversion by making extensive use
of statistical regression to identify “Pops and Drops” of key indicators and customer behaviors generated
by Tealeaf. Given that the company has thousands of Tealeaf events they are actively tracking, there isn’t
an option to use basic dashboard or spreadsheet reporting. Instead they rely on a basic statistical analysis
of the exported data to mine for events that have changed more than expected day‐over‐day and week‐
over‐week.
Figure 8: Example of drill‐down into real‐time revenue opportunity loss analysis triggered by a significant deviation in number of
customers struggling to checkout detected by Tealeaf 8.
Advanced Site Optimization: Second‐Generation Digital Analytical Tools in Action
© 1999 ‐ 2010 Tealeaf Technology, Inc. All Rights Reserved. 14
“Using pops‐and‐drops serves as a valuable addition to the other analysis we do across Expedia.com,”
says Michael Gulmann, Senior Director of Global Site Conversion. “To have any level of efficiency at all
we need our data to work for us, not the other way around. By leveraging very basic statistical regression
— functionality that should be in every web site optimization package in my opinion — we are able to
quickly discover opportunities for optimization.” By leveraging this simple data reporting strategy, Mr.
Gulmann estimates that nearly $10 million has been either saved or added to Expedia.com’s FY ’10
revenue stream.
While Tealeaf 8 is not the company’s first product using statistical modeling to surface insights, in Web
Analytics Demystified’s opinion the recent upgrade is a significant improvement in how that information
is communicated. Even more crucial, Tealeaf is creating a new expectation of how enterprise‐class
analytics applications will leverage the data they collect to expedite the analysis process.
CONCLUSIONS
Given that the volume of information collected in digital channels is unlikely to decrease anytime soon,
we hope that readers of this paper can appreciate the necessity of looking for innovative ways to manage
data and surface insights. More importantly, we hope that readers appreciate the value second‐
generation web analytics are likely to provide. By streamlining the presentation of data based on even the
most simple data mining and modeling, these applications will give analysts and organizations a
potentially dramatic competitive advantage over companies that must continue to manually mine for
insights using first‐generation tools and technology.
While we still believe that digital channels will ultimately be measured and managed using even more
powerful third‐generation technologies that are already widely deployed across the enterprise, it is our
opinion that the adoption and use of testing platforms and real‐time monitoring services like Tealeaf 8 are
a critical step towards developing internal comfort and competencies with samples, statistical models,
and algorithmically derived insights. Those companies wishing to explore the potential impact second‐
generation technology might have on their businesses are encouraged to reach out to the vendors
described in this document or to Web Analytics Demystified directly for guidance and support.
At the end of the day, businesses are run on a combination of data, gut instinct, and sound judgment. Our
hope is that this white paper has clarified how Web Analytics Demystified sees the use of data evolving
over time to become more timely, relevant, and valuable as part of decision‐making processes. As
businesses increasingly depend on connected channels, adept analysis of data from these channels has
the potential to create competitive advantages that have never existed. Your business has an opportunity
to leverage those advantages to grow and thrive in your marketplace.
We welcome your feedback and comments.
Advanced Site Optimization: Second‐Generation Digital Analytical Tools in Action
© 1999 ‐ 2010 Tealeaf Technology, Inc. All Rights Reserved. 15
ABOUT THE AUTHOR
Eric T. Peterson, CEO and Principal Consultant at Web Analytics Demystified, has worked in web
analytics since the late 1990ʹs in a variety of roles including practitioner, consultant, and analyst for
several market‐leading companies. He is the author of three best‐selling books on the subject, Web
Analytics Demystified, Web Site Measurement Hacks, and The Big Book of Key Performance Indicators, as well as
one of the most popular web analytics bloggers at www.webanalyticsdemystified.com. Mr. Peterson has
committed much of his life to the betterment of the web analytics community, so much so that Jim Sterne,
President and co‐founder of the Web Analytics Association says ʺEricʹs leadership in the industry in
unparalleled, his devotion to the community is legendary, and his years of experience translate
immediately into strategic and tactical competitive advantage for everybody who works with him.ʺ
ABOUT WEB ANALYTICS DEMYSTIFIED
Web Analytics Demystified, founded in 2007 by internationally known author and former Jupiter
Research analyst Eric T. Peterson, provides objective strategic guidance to companies striving to realize
the full potential of their investment in web analytics. By bridging the gap between measurement
technology and business strategy, Web Analytics Demystified has provided guidance to hundreds of
companies around the world, including many of the best known retailers, financial services institutions,
and media properties on the Internet.
For more information on Eric T. Peterson and Web Analytics Demystified, please visit
www.webanalyticsdemystified.com, email [email protected], or
call (503) 282‐2601.
ABOUT TEALEAF
This whitepaper is sponsored by Tealeaf, the leading provider of online customer experience
management solutions. Tealeafʹs CEM solutions include both a customer behavior analysis suite and a
customer service optimization suite. For organizations that are making customer experience a top
priority, these solutions provide unprecedented enterprise‐wide visibility into every visitorʹs unique
online interactions for ongoing analysis and web site optimization. Online executive stakeholders from
ebusiness and IT to customer service and compliance are leveraging Tealeaf to build a customer
experience management competency across the organization. Founded in 1999, Tealeaf is headquartered
in San Francisco, California, and is privately held. For more information, visit www.tealeaf.com.