Accelerate Digital London Technical Masterclass

Technical Master Class

17 July 2014

Accelerate Digital London

17.07.2014

Architecture for Tomorrow

[email protected]

• The Challenge

• Shortcomings of the current Enterprise IT

• Introduction of the "Digital Business Platform"

• Location of the Digital Business Platform in the Enterprise Architecture

• Digital Business Platform and Orchestration

© 2014 Apigee Confidential – All Rights Reserved 4

Partner Integration

Web site

Wearable Electronics

Gaming Consoles

Smart Appliances

Smart Sensors

Smart phones

Connected Cars

Tablets

KiosksAPI

API

Your Digital Assets

AP

I AP

I

Your Digital Assets

The changing demands for Enterprise IT

• The Challenge





© 2014 Apigee Confidential – All Rights Reserved

SOA implementation projects

• choose tools with care

• built well designed services

• designed contracts through solid engineering principles

Overall system-to-system communication across the company got

• decoupled to ease maintenance

• properly secured

• made reliable

One of the current standards of Enterprise IT: SOA

SoRData base

Back end

SOA Middleware

SOA Service

SOA Service

The created infrastructure is great for complex business logic and process integration e.g. with long running transactions and human interaction


Is your architecture fit for these challenges?

Partner Integration

Web site


Gaming Consoles

Smart Appliances

Smart Sensors

Smart phones

Connected Cars

Tablets

KiosksAPI

API

Your Digital Assets

AP

I AP

ISoR

Data base

Back end

SOA Service

SOA Service

?

??

??

??

??

?

SOA Middleware

© 2013 Apigee. Confidential – All Rights Reserved© 2013 Apigee. Confidential – All Rights Reserved

Different Focus

8

DeveloperUser API API Team BackendApp

Inside-Out approach

Integration

Transactional

Predictable usage

Outside-In approach

Consumption

Interactive

Unpredictable usage

External Internal


Aspect SOA / ESB Apps and APIs

Some Technical Aspects

Enable developers, either external or internal, to build nifty, compelling apps, and allow users to run them

High-latency, untrusted (Mobile wireless network)

Web technologies, REST, JSON, Javascript

Internet standards including oAuth and OpenID

Of primary importance

Enable Internal developers and systems to connect, while complying with IT department standards

Low-latency, trusted

SOAP, XML, proprietary

Internal mechanisms, WS-Security, LDAP

Limited use, secondary importance.

Core goal

Network

Technologies

Authentication and Authorization

Usage Analytics

• The Challenge






Backend

Speed up while preserving the existing invests

Digital Business Platform layer

SOA Middleware layer

System of RecordYe

ars

Mon

ths

IT Applications Partner Apps Mobile Clients "Internet of Things"

Day

s

Database


• Fast, light and stateless

• Quickly configurable, but at the same time extensible

• Improve response time (caching, paging, …)

• Self service onboarding for developers

• Comprehensive and holistic view on API usage and performance

• Support both vertical and horizontal scaling

• Deployable across regions, e.g. disaster recovery, geo load balancing

• …

Selected features of a Digital Business Platform


A Digital Business Platform – Logical View

Mobile clients / AppsProvide data or need access to digital assets

ESB/MQ

System of Record

System ofRecord

API RouterCalls / data needs to be routed to the ideal target

API ProxySome calls / data will be routed to an SOR

App Services Custom data for the apps

App metricsApps require metrics

API metricsAPIs require metrics

Data mining

Machine

Learning

Predictive

Analytics

Data Lake

InsightsAPIs driven by data

Developer PortalDevs need access


• To achieve fault tolerance, horizontal scalability and high performance (high throughput, low latency) a Digital Business Platform needs to be thoroughly designed

Design of a Digital Business Platform

API Router

API Proxy

Data Platform

API Router

API Proxy

API Router

API Proxy

runtime db

runtime db

…

…

load balancer

Metrics

Insights

Stateless components

to easily scale horizontally

Replicating NoSQL to be fault tolerant and to scale horizontally

(even across regions)

Geo location aware, round robin,monitoring API Router/Proxy availability

Scalable, Real time access,

central data platform formodeling

Dev Mgmt

• The Challenge






A Generic Digital Business Platform – Summary

API Router

API Proxy

App Services

App metrics

API metrics

Developer Portal

Insights

Developer Management

API Services

API Intelligence


The simplified new Integration Architecture

SOA Services

ExternalClients

Partner Clients

SOA Middle-ware (ESB)

Back end

API Services

Back end

Back end

Adapter

DeveloperManagement

APIIntelligence

SOAManage-

ment

DigitalBusinessPlatform

Integration / SoaArchitecture

API API

Internal Clients

Data Plat-form


OUTSIDE

DMZ

It can can be deployed on premise …

ExternalClients

Partner Clients

Internal Clients

DeveloperManagement

APIIntelligence

API Services

SOA Services


Back end

Back end

Back end

Adapter

SOAManage-

ment

API API

Data Plat-form

CORE


DMZ

… but for agility reasons cloud is best

SOA Services


Back end

Back end

Back end

Adapter

DeveloperManagement

APIIntelligence

SOAManage-

ment

API API

ExternalClients

Internal Clients

API Services

Partner ClientsOUTSIDE

CORE

Data Plat-form

• The Challenge






A composite service combines different services to deliver a value-added new service and can be categorized into different types:

1. User Experience centric Mashup

• Enhance UI

– by adding public cloud data (e.g. maps, stock info, weather info, ...)

– by combination of content from different sources (e.g. RSS, ATOM,…)

2. Intermediation / Customization

• Enhance services

– with public cloud data (e.g. maps, stock info, weather, ...) or partner data

– by payload/security mediation to homogenize the service use

3. Arbitration/Aggregation

• Unify services by combination in a read-only way or with data changes that are completely independent

• Route services to provide one access point onto several different services without knowing implementation details

4. Process Integration/Package

• Create new services by tight integration into business workflows

• Resource changes will most probably be transactional and depend on each other

Definitions

21

Unsafe with depen-dencies

Indepen-dently

consistent

Safe

Safe


Placement in the Integration Architecture

22

Type Exposure Layer / API Gateway

Implementation Layer / Integration

Architecture

1. User Experience centric Mashup Yes No

2. Intermediation/Customization

Yes Not advised

3. Aggregation/Arbitration

Yes Not advised

4. Process Integration No Yes

SOA Services


Back end

Back end

API

API Services

Back end

Back end

Adapter

DeveloperManagement

APIIntelligence

SOAManage-

ment

API API

Back end

API

DIGITALBUSINESSPLATFORM


• The need for higher speed and agility demands for a new layer / system in the Enterprise IT

• The Digital Business Platform optimizes access for a large amount of different devices and an increasing number of business partners from the outside of the Enterprise

• It has to be deployed in a fault tolerant, high available, horizontally scalable manner to be prepared for the load of the near future

• It complements existing capabilities in the architecture:

– Services exposed by Middleware / SOA

– (Big) Data platform for data analysis / data mining / machine learning

Summary

23

17 July 2014

APIs are Interaction, not just Integration

S


Agenda

25

InteractionExplosion of devices - User centric

DevelopersEmpowered developer - Usability is key

IntegrationUbiquitous - Cheap and easy - Integration Explosion

Interaction


Partner Integration

Web site


Gaming Consoles

Smart Appliances

Smart Sensors

Smart phones

Connected Cars

Tablets

KiosksAPI

API

Your Digital Assets

AP

I AP

I

27

Your Digital Assets

App and Partner Interaction take Centre Stage


Interaction Patterns Govern APIs

Backend Services

App Server

Web

Partner ESB

Mobile API A

API B

API C

API Tier


Interaction Patterns Govern APIs

Backend Services

App Server

Web

Partner ESB

Mobile API A

API B

API C

API Tier

Chattiness

Mash Ups and Aggregation

Data & FormatFeedback Loop

(email, push notification)

Security

Developers


Developers are the Key Actor

31

DeveloperUser API API Team BackendApp


This applies …

32

Jakob's Law of the Internet User Experience:

Users spend most of their time on other sites. This means that users prefer your site to work the same way as all the other sites they already know.

Developers spend most of their time developing against other APIs. This means that developers prefer your API to work the same way as all the other APIs they already know.

34

Application developers are raison d'être for APIs.

35

Be pragmatic.

For the benefit of application developers.

Design – intuitive and simpleConsumption – ease of use and relevancePackaging – developer and use case centricSecurity – simple and consistent


API Design & Pragmatic REST

37

• https://groups.google.com/forum/#!forum/api-craft• http://info.apigee.com/Portals/62317/docs/web%20api.pdf• http://www.youtube.com/user/apigee

https://groups.google.com/forum/%23!forum/api-craft

https://groups.google.com/forum/%23!forum/api-craft

http://info.apigee.com/Portals/62317/docs/web%20api.pdf

http://info.apigee.com/Portals/62317/docs/web%20api.pdf

http://www.youtube.com/user/apigee

http://www.youtube.com/user/apigee

APIs are Integration Technology


APIs – Easy Integration

39

Complex and VisibleCORBA, RMI, RPC

Simple and AbstractEAI, EJB, ESB

Simple and VisibleREST/HTTP

Integration is hard however System Impedance and

Remote Nature is obvious



40




Integration is easier System Impedance

and Remote Nature is hidden



41




Integration is easier System Impedance

and Remote Nature is hidden

Abstractions break!



42




Easy integration based on simple,

standardised principlesUsed

everywhere!


App

Backend

APIs are Everywhere

API


App

Front end

Backend

APIs are Everywhere

App

Backend

API

API


App

Front end

Backend

AppApp

APIs are Everywhere

API

API

APIAPI


Front end

Backend

AppApp

Service Service Service



Micro service architecture

AP

Is

AP

Is

APIAPI

API

API

APIs are Everywhere

APIs not an Integration Platform


• Every Developer is an Integration Developer– Bye bye centralized integration platform

– Hello point to point integration

• Integrations are use case centric– Bye bye canonical data model

– Bye bye one size fits all interaction model

What’s the consequence of ubiquitous APIs?


Front end

Backend

AppApp




Micro service architecture

AP

Is

AP

Is

APIAPI

API

API

The API Landscape will shift!

Requires• Making APIs consumable

and accessible• Adding governance


Add Visibility, Governance and Decoupling

Backend Services

App Server

Web apps

Social apps

ESB

Mobile apps

Persistence

Security

Orchestration

Analytics

All Apps API Tier


Wrap Up

51

InteractionExplosion of devices - User centric

API DesignEmpowered developer - Usability is key

IntegrationUbiquitous - Cheap and easy - Integration Explosion

Your Questions?

Thank you

17 July 2014

Data: Powering the Digital Economy

Pankaj Sodhi, Accenture UK/I Big Data Lead

@AccentureDigiUK

Every business is a Digital Business

Data is the lifeblood of the organisation

Most business leaders – CEO, CMO and CFOs now view data as among their most valuable assets

Copyright © 2014 Accenture All rights reserved. 57

Data Supply Chain

Data should flow easily and usefully through the entire organisation

80% of high performing companies are investing in analytics, yet only 40% day they are getting the value that they want

Application

Service Interface Layer

In-MemoryIMDB, CEP,

CacheIMDB, CEP Distributed Cache In-Memory DB

Streaming CEPGraph Stream

1 Pass Stream

Big Data Core

Query Engine

In-Memory Analytics

InteractiveBulk Transfer

Sources

Streaming

Organisations can choose from different data technology components to build the architecture needed to support data acceleration. These include big data platforms, complex event processing, ingestion, in-memory databases, cache clusters, and appliances.

Decision

Learning

Discovery

Memory

Perception

Optimization

Prediction

Causality

Tagging

Filtering

Raw Data

Actionable Insights

Data Ingestion Capabilities

Data Discovery & Visual Analytics Capabilities

Advanced Data Analytics Capabilities

Known Business Questions

Business Hypothesis

Data Raising Interesting Questions

Insight Generation

Patterns in the Data

Operational Analytics Opportunities

Accenture Data Discovery

Accenture’s Data Discovery Platform

Advanced VisualizationsGet deeper insights from analytics by using interactive best practice advanced visualizations that

enable drill-to-detail. For maximum effectiveness use these visualizations in combination with blended data and advanced analytic models.

Text & Unstructured

Segmentation/Clustering

Predictions & Simulations

Behavior

Networks and Relationships

Geospatial

Treemaps and scatter plots visualize aggregate and individuals size, proximity

and concentration of clusters

Parallel coordinates visualize patterns in multivariate discrete and continuous datasets with parameterized colors

Visualize location and geographic network data on multi-layered maps give insights

into event concentrations.

Parameterize time-series predictions for ‘What-if’ analysis. Confidence bands

visualize the models strength

Find relationship clusters and patterns with network diagrams are used chord charts for

hierarchical entity relationships

Text clouds combined with sentiment scores are powerful for generating insights from

social and unstructured data.

Thames Water

Copyright © 2013 Accenture All rights reserved.

http://www.google.co.uk/url?sa=i&rct=j&q=&esrc=s&frm=1&source=images&cd=&cad=rja&uact=8&docid=jUDzpfTQcyL4FM&tbnid=TtSzpAawAXj7uM:&ved=0CAUQjRw&url=http://www.gpsuk.com/news/1/63/gps-pipe-used-for-thames-water-newbury-pipeline.html&ei=HJ2VU52zCIaH0AWxx4CwCw&bvm=bv.68445247,d.ZWU&psig=AFQjCNHAXCpnMdzlJo-sPhr_zabC2jfKgw&ust=1402400401084647

@AccentureDigiUK

Join the Conversation

April 10, 2023

Apigee Insights

Who we are

Helps business quickly achieve digital acceleration, with scale, insights and agility

More API deployments run on Apigee than any other platform

Customers include 20% of the Fortune 100, 50% of the top global brands, 50% of the top retailers

Customer-driven product leadership with APIs and Big Data

Over 100k developers in the fast growing Apigee developer community

Products

Services

Knowledge

Apigee customers span multiple industries

66Updated 5/31/14

Agenda

Predictive Analytics on Big Data, enabling a new

generation of businesses

Customer Success

Apigee Insights

Enterprises need to operate differently to stay relevant

68

Omni-channel AgilityIndividualization

Individualized interactions

Relevant experiences

Accelerate decisions

Proactive response

Adaptive processes

Continual learning

Be where customers are

Holistic view

Contextual journey

Proliferation of Mobile and Digital

Accelerating Pace of Change

Evolving Customer Expectations

Predictive Analytics on Big Data key to operating differently

69

DeveloperUser API API Team Backend

Predictive InsightHadoop

Data Warehouse

AppAdaptive App

Data Scientist

Adaptive API

Existing Big Data investments lack key capabilities

70

Data Lake / Data Warehouse

• Big Data Infrastructure

• Lack specialized data structures to analyze fine-grain event data

• Lower precision resulting from predictive analytics on summarized, tabular data

• Difficult to incorporate signals from unstructured data in analysis

• Difficult to access insights in real-time

• Difficult to integrate predictive insights into apps and APIs

Business Intelligence

• BI Visualization & Reporting

Data Science • Statistical Analysis• Scientific Visualization

ChallengesCurrent Big Data Investments

71

In addition, many challenges to address

Best Practices

Global Deployment

Unstructured Data Processors

Real Time Processing

Continual Improvement

APIs & Automation

Sophisticated Machine Learning

Model Building

Monetization

Security

System Management

Big Data Structure

AdaptiveApps & APIs

Agenda

Predictive Analytics on Big Data, enabling a new generation of businesses

Customer Success

Apigee Insights

Business & Technical

UserRapidly realize value API Team

Apps

Developer

Develop more precise insights

Discover complex hidden patterns

Data Scientist

Apigee Insights: Enable adaptive apps and APIs

Hadoop Data Warehouse

Other

Developer & Business User Tools

Predictive & Descriptive Analytics

Graph & Sequence Processing (GRASP)

Business User Tools

Developer Tools

RESTful APIs

Graph Machine Learning

Modeling Workbench

Real Time Scoring

Data LoadersUnstructured Data

ProcessorTemporal Graph

Database

Apigee Insights: Enable adaptive apps and APIs

Business & Technical

UserAPI Team

Apps

Developer

Data Scientist

Hadoop Data Warehouse

Other

Predictive & Descriptive Analytics

Developer & Business User Tools

Graph & Sequence Processing (GRASP)

How do you detect hidden patterns and relationships?

75

Data Data

• How do you discover hidden patterns in

interactions across touch points & channels?

• How do you understand the customer journey

across siloed channels?

• How do you deliver a contextual journey vs.

channel optimized interactions?

• Unique big data structure (GRASP) purpose-built to analyze fine-grain time-aware interaction data – at scale

• Identify complex hidden patterns and relationships in customer behavior

• Dramatically improve ability to predict future actions

Use specialized big data structure to analyze customer journey

76

PROFILEConsumerID: U56Gender: MGeo: San FranciscoInterests: Bikes

PROFILEConsumerID: U60Gender: FInterests: NewsAge: 35-40

2 4

1 3 40

0 3

View product A

on web

Buy product A on mobile

Call customer service

Return Product A in store

3

2

4

1

0

Time

GRASP (Graph and Sequence Processing)

time-sequenced graph database on Hadoop

Fine-grain event and entity data

How do you improve prediction accuracy?

77

• How do you avoid losing precision when working

with sampled or summarized data?

• How do you detect signals hidden in

unstructured data such as text?

• How do you prevent prediction accuracy from

degrading over time?

Apply powerful machine learning on fine-grained event data

• Significantly improve prediction accuracy by applying powerful machine learning algorithms directly on fine-grained event data in GRASP

• Always deliver fresh, relevant interactions with a continual learning feedback loop

78

Rules Based

Approach

Traditional StatisticalAnalytics

Apigee Insights

Precision

Traditional Analytics Apigee Insights(Machine Learning on Big Data)

Static Model Dynamic Model

Focuses only on top attributes

Leverages both strong and weak signals

Batch only Batch and real time

Adapts to changing conditions

Easier to put into action

Significantly better results

Discover hidden signals in unstructured data

79

MBRS payment was rcvd the same day as

dunning took effect. Dunning was not posted until

after payment was processed. Final notice sent

out after payment was sent out as well. Payment was sent

out and postmarked before due date. The member

was termed correctly.

88%

45%

With Unstructured

Data

Without Unstructured

Data

Prediction Accuracy(Likelihood for member to complain)

Healthcare Payer Example

2x

• Unstructured data is difficult to analyze but holds valuable signals

• Improve prediction accuracy by extracting signals hidden in text

How do you realize value from data?

80

• How do you enable developers to build new

adaptive apps and APIs?

• How do you enable business users and analysts

to interact with predictive insights?

• How do you integrate predictive insights into

existing apps and systems?

Predictive

Insights

Business Value

Access insights in real-time

81

• In-memory real-time processor to enable fast access to insights

• Regularly updated to always provide fresh, relevant scores

Real-time access to insights using NoSQL

Propensity Free Shipping 10% Off Churn

User 1 0.72 0.68 0.33

User 2 0.56 0.23 0.55

User 3 0.32 0.45 0.67

User 4 0.20 0.32 0.18

User 5 0.44 0.69 0.22

Easily consume insights via APIs and visualization

82

Predictive insights

Descriptive insights

• Use RESTful APIs to build new adaptive apps and APIs or extend existing ones

• Enable developers and business users to act on predictive insights

Build new adaptive apps and APIs, extend existing ones, or use or extend visualization tools provided

Deliver individualized interactions across multiple channels

83

Direct Mail

Email

Web

Mobile

Outreach

Fine grained event data

Individualized interactions across multiple channels

Machine Learning on GRASP

Predictive Models

Targeting Recommendations

Churn Other Advanced

3

2

4

1

0

Propensity Free Shipping 10% Off Churn

User 1 0.72 0.68 0.33

User 2 0.56 0.23 0.55

User 3 0.32 0.45 0.67

User 4 0.20 0.32 0.18

User 5 0.44 0.69 0.22

0 View product page on web

1 Receive mobile offer

2 Call customer service

3 Agent note: dissatisfied

4 Return product in-store

5 Purchase Online

Customer Profile

Agenda

Predictive Analytics on Big Data, enabling a new generation of businesses

Customer Success

Apigee Insights

85

IBC: Proactive Customer Service with Predictive Analytics

“Apigee has helped us move from simply answering customers’ calls to proactively reaching out to our members before they have issues.”

Somesh Nigam, SVP and Chief Informatics Officer, Independence Blue Cross

• $1.2B Medicare Advantage business

• 50+% Reduction in complaints to Medicare

• Transformed from reactive to proactive customer service

• Identified root causes for dissatisfaction

Typical Use Cases

Proactive Churn Reduction

Reduce churn by predicting customers likely to be at-risk and proactively responding to them

Individualized Targeting & Recommendations

Improve customer retention, cross-sell, up-sell, loyalty, and acquisition

20% increase in profit

Individualized daily deals emails

Leading Online Services Company

100% increase in revenue & profit

Individualized website recommendations

Leading Shopping Site

164% lift in conversions

60% more clicks, 55% fewer emails

Recommended products - game launch

Leading Gaming Company

50% fewer complaints

6x more precise predictions

1 month in advance

Leading Healthcare Insurer

Apigee Insights

87

Deep Predictive Analytics Expertise

• Team led Big Data @ Yahoo!, IBM, and others

• Thought leaders in machine learning and big data

• Extensive Vertical expertise from client engagements

Accelerate Business Growth

• Increase revenue, profitability

and customer satisfaction

• Individualized targeting and

recommendations

• Proactive Churn Reduction

Develop Adaptive Apps & APIs

• GRASP on Hadoop

• Machine Learning on GRASP

• Unstructured & API data

• Real Time

• APIs and Automation

• Cloud or on-premiseApigee Insights

Thank you

Accelerate Digital London Technical Masterclass

Technology

Transcript of Accelerate Digital London Technical Masterclass