1

Cloud Analyticsusing cloud based services to analyse big data

Associate Professor David Parsons, Massey University, Auckland‘Into the Cloud’ Event, Massey University, 21st November 2013

Image: http://blog.qburst.com/2012/11/deploying-cloud-analytics/

2

Topics

1. Big Data

2. What is Data Analytics?

3. Cloud Analytics User Options

3

1. Big Data

4

The Fire Hose

• “The world's technological per-capita capacity to store information has roughly doubled every 40 months since the 1980s; as of 2012, every day 2.5 quintillion (2.5×1018) bytes of data were created.”

• Wikipedia

Image: http://www.portfolioimprint.com/portfolio-items/drinking-from-the-fire-hose/

5

Big + Uncertain

• Big– Social media– Internet traffic– Primary data (e.g. scientific)

• Uncertain– Semi-structured, federated– Sensor (e.g. weather, satellite)– Need an indication of probability

6

Uncertain Sensors

• The implications of uncertain sensor data can be serious:

• 1983 Soviet nuclear false alarm• “It was subsequently determined that the false

alarms were caused by a rare alignment of sunlight on high-altitude clouds and the satellites' Molniya orbits”

• http://en.wikipedia.org/wiki/1983_Soviet_nuclear_false_alarm_incident

7

Big Data Attributes

• According to Gartner, Big Data exhibits three attributes– Volume

• large enough to require special consideration

– Velocity• produced at high rate, ‘stale’ data is not

valuable

– Variety• multiple types of data from multiple sources

8

More Vs

• Some would add quality to quantity:– Value

• perceived or quantifiable benefit

– Veracity• where the correctness of the data can be

assessed

9

2. What is Data Analytics?

10

Data Analytics

“The extensive use of data, statistical and quantitative analysis, explanatory and predictive models, and fact based management to drive decisions and actions.”

– ‘Competing on Analytics: The New Science of Winning’, Davenport and Harris (2007, p. 7)

11

Not Just BI

• Business intelligence uses descriptive statistics

• Big Data analysis uses inferential statistics to handle the random nature of the data

12

Understanding Customers

• Use social media data, sensor data, Web search trends, weather forecasts

• Target can predict when a customer will expect a baby

Image: http://www.forbes.com/sites/kashmirhill/2012/02/16/how-target-figured-out-a-teen-girl-was-pregnant-before-her-father-did/

13

Business Processes

• HR - optimization of talent acquisition Moneyball style

• Using the same datathat is available to others, but analysingit differently

Image: http://en.wikipedia.org/wiki/File:Moneyballsbn.jpg

14

Personal Data

• Jawbone UP band – Collects data on calorie

consumption, activity levels and sleep patterns

– 60 years worth of sleep data every night

Image: https://jawbone.com/up/download

15

Improving Sports Performance

• Video analytics and sensor technology in sports equipment

• Social media conversations to monitor emotional wellbeing

• IBM SlamTracker tool for tennis tournaments

Image:http://www.wimbledon.com/en_GB/slamtracker/slamtracker.html?ts=1385525456612&ref=www.wimbledon.com/en_GB/scores/results/day21.html&syn=none&promo=nav

16

Security and Law Enforcement

• National Security Agency uses big data analytics to spy on everyone

• Others use big data techniques to:– Detect and prevent cyber attacks– Catch criminals– Detect fraudulent

transactions

Image: http://www.telegraph.co.uk/technology/internet-security/10409330/Cyber-attacks-are-the-greatest-threats-UK-businesses-face.html

17

Financial Trading

• In High-Frequency Trading (HFT) big data algorithms are used to make trading decisions

• Take into account signals from social media networks and news websites

• Not always correctly interpreted!• http://

peakoil.com/business/oil-traders-misread-tweet-oil-prices-spike

Image: http://3.bp.blogspot.com/-w-4ckUoLH2c/Tj-yJXC28MI/AAAAAAAAAXw/jjCpKYvdalY/s1600/stock+market+crash+arrow.jpg

18

3. Cloud Analytics User Options

19

Cloud Analytics for Me?

• What relevance does this have to smaller enterprises?

20

All Companies Have Big Data

• Internal:– Online businesses, large or small, will

collect Web logs and clickstream data• External, e.g:

– World Bank statistical data – Twitter data since March 2006– News and investment data services

21

Twitter

• How does Twitter make money?• http://www.bbc.co.uk/news/business-24397472

• Twitter sells its “firehose” (public data) of 500 million tweets each day

• Companies can learn detailed and specific information about their users

22

For SMEs

• Replace spreadsheets and business information silos with analytics

• Same knowledge footing as larger organizations for the first time

• “there are solutions that can allow a small business to perform simple data analytics on a terabyte of data for as low as $100” - Forbes

23

New Businesses

• FlightCaster predicts flight delays– Analyses historical flight data and factors

in real-time conditions• Google and Yahoo! were once small

companies that grew by understanding their data

- Forbes

Image: http://flightcaster.com/

24

Some Cloud Analytics Providers

• IBM Business Analytics in the Cloud• http://www-01.ibm.com/software/analytics/cloud/

• Oracle RightNow Analytics Cloud Service• http://

www.oracle.com/us/products/applications/rightnow/engage/analytics/overview/index.html

• HP Vertica Analytics Platform• http://www.vertica.com/the-analytics-platform/

• Windows Azure Business Analytics• http

://www.windowsazure.com/en-us/services/business-analytics/

• Jaspersoft (for Amazon)• https://www.jaspersoft.com/cloud-analytics

25

Google BigQuery

• Google’s data platform provides some ‘try it now’ features

• https://cloud.google.com/products/big-query

• BigQuery demo• https://demobigquery.appspot.com/

26

Links to Sources (1)

• Stefan Groschupf - Why Small Businesses Are Using ‘Big Data’. Forbes.

– http://www.forbes.com/2010/06/02/internet-data-analytics-technology-cloud-computing-10-datameer.html

• IBM - Cloud analytics lets small and medium-sized companies think smarter

– http://www.ibm.com/midmarket/be/en/att/pdf/Cloud_Analytics_element_61_REDM_02674_IBM_Inspire_magazine8_Belux_P52-54.pdf

27

Links to Sources (2)

• Bernard M - The Awesome Ways Big Data Is Used Today To Change Our World

– https://www.linkedin.com/today/post/article/20131113065157-64875646-the-awesome-ways-big-data-is-used-today-to-change-our-world

• New York Times - How Companies Learn your Secrets

– http://www.nytimes.com/2012/02/19/magazine/shopping-habits.html?pagewanted=1&_r=3&hp