NVIDIA AI & ANALYTICS - Computer Sweden...NVIDIA AI & ANALYTICS 2 A NEW ERA OF COMPUTING PC INTERNET...

2017 April

NVIDIA AI & ANALYTICS

2

A NEW ERA OF COMPUTING

PC INTERNETWinTel, Yahoo!1 billion PC users

MOBILE-CLOUDiPhone, Amazon AWS2.5 billion mobile users

AI & IOTDeep Learning, GPU100s of billions of devices

1995 2005 2015

3

NVIDIA — “THE AI COMPUTING COMPANY”Pioneered GPU Computing | Founded 1993 | $7B | 9,500 Employees

COMPUTER GRAPHICSGPU COMPUTING ARTIFICIAL INTELLIGENCE

4

AI FOR EVERYONE

AI will Revolutionize Transportation AI will Revolutionize Healthcare AI will Revolutionize Society

5

AI COMPUTE: UNREASONABALY EFFECTIVE*

Image Classification, Object Detection, Localization, Action Recognition, Scene Understanding

Speech Recognition, Speech Translation, Natural Language Processing

Pedestrian Detection, Traffic Sign RecognitionBreast Cancer Cell Mitosis Detection, Volumetric Brain Image Segmentation

*Credited to Yann LeCun, Facebook AI Research & Center for Data Science, NYU

6

NVIDIA AI COMPUTING ECOSYSTEM

AI-powered Consumer Services

AI-as-a-Service AI for Enterprise >1,500 AI Startups

iQIYI JD.comGoogleFlickr

Amazon FacebookeBayBaidu

ShazamQihoo 360 Skype Sogou

Periscope PinterestNetflixMicrosoft

TwitterTencent Yandex Yelp

AI for Auto

7

AI TRAINED SELF-DRIVING CARS HIT THE ROAD

8

30 CORPORATIONS WORKING ON AUTONOMOUS VEHICLES

Source: https://www.cbinsights.com/blog/autonomous-driverless-vehicles-corporations-list/

https://www.cbinsights.com/blog/autonomous-driverless-vehicles-corporations-list/

9

SELF-DRIVING IS HARD

10

THE ROAD TO AUTO-PILOT CARS

Environment Model • Situation Awareness • Path Finding • Learning

11

AI IS THE SOLUTION TO SELF-DRIVING

Perception Reasoning

HD Map Mapping

Driving

AI Computing

12

DEEP LEARNING + DATA + GPU ACCELERATION

13

DEEP LEARNING is A new computing model

Training

Inferencing

Inferencing

TRAIN

DIGITS

TRAINTEST

DEPLOY

TENSOR RT

INFERENCE ENGINE

DATA CENTER

AUTOMOTIVEEMBEDDED

DGX

JETSON DPX2TESLA

14

FREE SPACE DETECTION

CAR 3D DETECTION

LANE DETECTION

ALL COMBINED

15NVIDIA CONFIDENTIAL. DO NOT DISTRIBUTE.

AI CO-PILOT

16

IT IS ALL BASED ON DATA

INCREASING DATA VARIETY

Search Marketing

Behavioral Targeting

Dynamic Funnels

User Generated Content

Mobile Web

SMS/MMS

Sentiment

HD Video

Speech To Text

Product/Service Logs

Social Network

Business Data Feeds

User Click Stream

Sensors Infotainment Systems

Wearable Devices

CyberSecurity Logs

ConnectedVehicles

Machine Data

IoT Data

Dynamic Pricing

Payment Record

Purchase Detail

Purchase Record

Support Contacts

Segmentation

Offer Details

Web Logs

Offer History

A/B Testing

BUSINESS PROCESS

PETABYTES

TERABYTES

GIG

ABYTES

EXABYTES

ZETTABYTES

Streaming Video

Natural Language Processing

WEB

DIGITAL

AI

17Source: Gartner, “Architecting the On-Demand Digital Business”; Drue Reeves, Kyle Hilgendorf, Kirk Knoernschild, August 16, 2016

18

AI & ANALYTICS USE CASES

AUTOMOTIVEAuto sensors reporting

location, problems

COMMUNICATIONSLocation-based advertising

CONSUMER PACKAGED GOODSSentiment analysis of what’s hot, problems

$

FINANCIAL SERVICESRisk & portfolio analysis

New products

EDUCATION & RESEARCHExperiment sensor analysis

HIGH TECHNOLOGY /

INDUSTRIAL MFG.Mfg. quality

Warranty analysis

LIFE SCIENCESClinical trials

MEDIA/ENTERTAINMENTViewers / advertising

effectiveness

ON-LINE SERVICES /

SOCIAL MEDIAPeople & career matching

HEALTH CAREPatient sensors, monitoring, EHRs

OIL & GASDrilling exploration sensor

analysis

RETAILConsumer sentiment

TRAVEL &

TRANSPORTATIONSensor analysis for

optimal traffic flows

UTILITIESSmart Meter analysis for network capacity,

LAW ENFORCEMENT

& DEFENSEThreat analysis - social media

monitoring, photo analysis

19

0,0

1,0

2,0

3,0

4,0

5,0

6,0

2008 2009 2010 2011 2012 2013 2014 2016

NVIDIA GPU x86 CPUTFLOPS

M2090

M1060

K20

K80

K40

Fast GPU+

Strong CPU

THE ADVANTAGES OF GPU-ACCELERATED DATA CENTER

P100

20

GPU ACCELERATION OVERCOMES THE CHALLENGES OF SLOW COMPUTE

ASK QUESTIONS YOU DON’T KNOW THE ANSWERS TO

Issuing iterative queries becomes wearisome

EXPLORE FURTHER

Analyst creativity is impaired

GO BEYOND WHAT’S BEING ASKED

Long response timeconstrains questions asked

21

WORKAROUNDS ARE NOT THE ANSWERS

EXPLORE THE OUTLIERS AND LONG-TAIL EVENTS

Pre-aggregation struggles at scale

RELY ON ACCURATE DATA

Scale out on CPU infrastructure has

tremendous hidden costs

SCALE WITH A ROI

Sampling misses the whole picture

$

22

NVIDIA ACCELERATED ANALYTICSGPUs in the Data Center

AI-ACCELERATEVISUALIZEANALYZE

23

ANALYZE

10-100x faster data processing with the adoption of GPU-accelerated databases

Applications simply run significantly faster

Accelerated Insight fordigital business

GPU-accelerated Data Center

24

VISUALIZE

Interactive visualization solutions

Dynamic and deeper correlation

NVIDIA GPUs allow users to visualize 100x more data with 40x less infrastructure with sub-second response time

Interactive data visualization for transformative business


25

AI-ACCELERATE

Accelerated Algorithm

100x computational power

Software writing software

Data hungry (DL derives Data Correlation w/ enough data)

GOAL IS KNOWLEDGEAI-Accelerated data for knowledge business


27

DGX — THE ESSENTIAL TOOL FOR AI-ACCELERATED ANALYTICS

100X MORE DATA IN MILLISECONDS

250 NODE AI & ANALYTICSSUPERCOMPUTER-IN-A-BOX

TIME TO INSIGHT 10-100X FASTER

28

ace: applied cognitive enginehttps://www.ravn.co.uk/products/applied-cognitive-engine/

https://www.ravn.co.uk/products/applied-cognitive-engine/

29

CLASSES OF BIG DATADL makes these data classes “a valuable asset, not a problem”

“BIG IOT” “BIG ANALYTICS”

Unstructured

data

IoT data from machines

DBStructured

data

algorithmKnowledge

query Answer

Unstructured

data

New data from network transaction

DBStructured

data

algorithmKnowledge

query Answer

Graph building approach

30

DEEP LEARNING IN PORTFOLIO OPTIMIZATION & HEDGING

Left: black line is real market performance; orange & grey lines are hedged performance w/ DL

https://arxiv.org/abs/1605.07230

Deep Learning autoencoders learn complex correlations in market behavior. Can be used for portfolio construction

31

Watson for CyberSecurityFeb 2017

Security analysts at IBM X-Force Command Centers are now using Watson. Credit: John Mottern/Feature Photo Service for IBM

32

MEET CLEO

• “make finances easy to understand”• “make better informed decisions about

financial futures”

• Read-only SaltEdge SW + NLP (AI)• via Facebook’s secure Messenger API

http://meetcleo.com

Kinetica – Accelerated Analytics

Developed to Identify Terroristic Threats in Real-Time

33

• Kinetica incubated as a massively parallel

computational engine for US Army INSCOM

• Ingests 50+ sources of streaming data

producing 200B new records per hour

• Incorporates geospatial and temporal data

• Real-time, actionable threat intelligence

• First high-performance database to leverage

the power of GPUs (deployed 2012)

Converge AI and BI For Smarter, Faster Insights

FEATURES

• Data scientists can deploy custom code and machine learning libraries in-database and make it available to business users. Available now with C++, Java, Python

• Any user-defined program can receive table data, do arbitrary computations, and save output to a global table in a distributed manner

• UDFs have direct access to CUDA APIs – compute-to-grid analytics for GPU-acceleration within Kinetica

BENEFITS

• Converge Machine Learning, Deep Learning, and BI

• Extensibility to deploy custom logic or 3rd-party libraries

• Democratize data science with business access

34

ORCHESTRATION LAYER WITH USER-DEFINED FUNCTIONS (UDFs)

Kinetica Use Cases in Financial Services

• Risk Management

• Portfolio Management & Optimization

• Algorithmic and High Frequency Trading

• Real-Time Compliance

Kinetica GPU Accelerated In-Memory Database working with Financial Services companies to address some of their toughest challenges including:

• Risk Management, TCA, Derivatives Pricing, Sentiment Analysis

• Regulatory - Dodd Frank, Volker, Best Execution, Basel, MiFID, CCAR, etc.

• Compliance - AML, KYC, Fraud Detection, Trading Compliance)

36

Fraud Analysis

Real-

Time

Fraud

Engines

ML

LIB

S

ON DEMAND SCALE OUT +

IN-DATABASE PROCESSING

Native

APIs

SQL

Fraud

SQL QUERY

FINANCE

APPLICATIONS

Real-Time Financial Data Streams

Key/Value lookups for pattern analysis.

Spatial analytics looking at time and

distance between swipes/ATM

transactions. Ability to score against

trained models.UDF Functions

BIDMach

ANALYTICS

MILLISECONDS

Deep Learning APIs for

pattern detection and

predictive analytics. SQL

queries for BI analysis.

37

AI-accelerated analytics is a massive opportunity

Data Scientist productivity is vital

NVIDIA is the choice for AI-accelerated analytics

DGX-1 is fast, instantly productive

NVIDIA DGX-1The Essential Tool for

Data Scientists

38

DGX for Deep Learning: www.nvidia.com/dgx1DGX for Accelerated Analytics: www.nvidia.com/analytics

NVIDIA PARTNER ANALYTICS SOLUTIONS- REFERENCE MATERIAL ( NOT PRESENTED)

40

KINETICAOVERVIEW AND USE CASE

41

ACCELERATED DATABASE SOLUTION

Built from the ground up to scale linearly, Kinetica's distributed, in-memory database simultaneously ingests, explores, and visualizes streaming data for truly real-time actionable intelligence. Kinetica leverages the power of NVIDIA GPUs to deliver results 100x faster and 10x more cost-effective than traditional databases.

Overview

Retail: Customer 360/customer sentiment, supply chain optimizationCorrelating data from point of sales (POS) systems, social media streams, weather forecasts, and even wearable devices. Better able to track inventory in real time, enabling efficient replenishment and avoiding out-of-stock situations

Powering High Performance “Analytics as a Service” Solution: Delivering customer-focused services by leveraging all available transactional data. Currently no ability for business user to do customized analytics; IT has to. Query response times taking 10s of minutes, some over 2 hours, thus limiting ability to analyze and use data

Fin services: Large scale risk aggregations and billion+ row joins in sub-second time (5TB+ tables choke on RDBMS joins and Hadoop is too slow). Also ideal for fraud and compliance use cases.

Ridesharing: View all passengers and drivers to monitor behavioral analytics. Watch for fast acceleration, sudden braking, too many U-turns, etc. to avoid risk/lawsuits of faulty drivers

Manufacturing: Live streaming analytics on component functionality to ensure safety (avoid failures) and validate warranty claims

Industry use cases

42

COLUMN THREECOLUMN TWOCOLUMN ONE

DELIVERING ACCELERATEDINSIGHTS TO CUSTOMERS

Problem: Need to improve end-to-

end business process performance

while concurrently reducing costs.

Solution: Tracking every truck and

piece of mail (>200K devices

emitting every minute) with

Kinetica on only 10 nodes.

Reallocating resources on the fly

based upon personnel,

environmental and seasonal data.

Impact: In 2015 alone, USPS

delivered 150 billion pieces of mail

while driving 70 million fewer

miles and saving 7 million gallons

of fuel.

Problem: Existing very expensive

systems couldn’t ingest and handle

all the data. Data silos between gas,

electric, distribution/ transportation,

fiber vs. land-based.

Solution: Kinetica consolidates feeds,

fuses analytics, full real-time visibility

on workforce, meters, and grids.

Geospatially visualizing smart grid

while ingesting real-time vector data

streaming from smart meters.

Impact: Striving for optimized energy

generation and uptime based on

fluctuating usage patterns and

unpredictable natural disasters.

Problem: Too many data silos,

couldn’t fuse multiple data feeds

to get real-time anomaly

detection.

Solution: Identifying anomalies in

real time, monitoring multiple

streams of global attack vectors,

finding security lapses, mining

system logs, and/or determining to

what degree attackers are

coordinating with each other.

Impact: Protecting its financial

clients against current and

emerging cyber threats.

43

ACCELERATED DATABASE IN ACTION

ARCHITECTURE PRODUCT VIEW

44

MAPDOVERVIEW AND USE CASE

45

ACCELERATED ANALYTICS SOLUTION

MapD is a next-generation database and visual analytics layer that harnesses the power of NVIDIA GPUs to explore multi-billion row datasets in milliseconds. By combining a purpose-built GPU database with a rich visualization layer, MapD is able to deliver immersive, instantaneous analytics on data sets previously considered too large to explore interactively.

Overview

Telco: Correlates call records with server performance data to spot problems in real time, plus build ad targeting profiles

Retail: Analyzed historical sales to assess geographic product demand for future inventory and store locations

Finance: Hedge fund analysis of local and regional economic trends related to their portfolio companies

AdTech: Assessing inventory availability by matching millions of audience members against active ad units

Industry use cases

46

COLUMN THREECOLUMN TWOCOLUMN ONE

DELIVERING ACCELERATED INTERACTIVE INSIGHTS TO CUSTOMERS

PROBLEM: Compute constrains forced sampling - missing critical insights

SOLUTION: Memory footprint + query speed saw EVERY device.

IMPACT: Found devices that were costing $10s M per year in configuration + sizing.

PROBLEM: Simple queries took tens of minutes to run, complex queries months.

SOLUTION: MapD made the process real time for every query.

IMPACT: Generating new business via better responsiveness + insight.

PROBLEM: Massive volume (30B records per month) precluded CPU solutions on cost + performance.

SOLUTION: HW + SW delivered price/performance for massive data.

IMPACT: Massive financial impact, expanding to additional use cases.

47

ACCELERATED ANALYTICS IN ACTION


48SQREAM AND NVIDIA CONFIDENTIAL. DO NOT DISTRIBUTE.

SQREAMOVERVIEW AND USE CASE


SQREAM’S TERABYTE SCALE GPU DATABASE

SQream’s next generation GPU database delivers the solutions required for today’s terabyte scale data needs. It quickly relieves Big Data and complex Analytics pains, while leveraging existing resources with NVIDIA’s GPUs.

With minimum cost, hardware and infrastructure changes SQream enables entities to easily ingest, store and analyze heavy analytical workloads in near real-time.

SQream delivers up to 100 times faster results than any other key market player, with scalability capabilities surpassing existing solutions by orders of magnitude.

The power of a full-rack database machine is condensed into a standard 2U server.

SQream can be used as an analytical database or as an accelerator to an existing data warehouse.

Overview

Finance Services

Telecom

Government

Retail

IoT

Cyber Security

Marketing/Ad Tech

Healthcare, FMCG & Genome Research

Industry use cases


PROBLEM

Revenue creation and enhancement was

severely limited because real-time

customer activity generated terabyte

scale data volumes and throughput

needs which could not be met

SOLUTION

SQream DB with NVIDIA K80s

RESULTS

18x speed improvement within one

week enabled new revenue and insight

PROBLEM

Customer churn was growing due to a

lack of ability to create a timely

understanding of call detail records

(CDRs). The existing system was not

able to adapt or scale to technology and

market changes

SOLUTION

NVIDIA K40 based SQream DB

RESULTS

Effort requiring 14 specialists was

shifted to 2 staff members with

standard SQL skills, while reducing cost

by an order of magnitude and increasing

performance by over 30%.

SQREAM’S NEXT GENERATION GPU DATABASE CREATES BUSINESS VALUE STEP-CHANGE

PROBLEM

Existing systems and technologies only

allowed for sub-one hour windows of

analysis with limited data. This resulted

in costs and risks connected to a higher

rate of false alarms and misdetections

SOLUTION

SQream DB with NVIDIA K80s

RESULTS

Analysis windows were increased to

years, with a much higher granularity of

data. Productivity was increased by an

order of magnitude and false positives

were reduced 12-fold, substantially

reducing risks and costs.

ENTERPRISE CYBERSECURITY

NATIONAL MOBILE OPERATOR

GLOBAL TELECOM


SQREAM DB AND NVIDIA’S GPUS BREAK THROUGH LEGACY CPU BOTTLENECKS

SQREAM DB’S ARCHITECTURE LEVERAGES INDUSTRY STANDARD INTERFACES

NVIDIA’S GPUS ENABLE A NEW PARADIGM

SQream DB server SQream DB Compiler

SQL parser

Optimizations

Parallel query

graph plan

GPU/CPU Runtime

Columnar storage

engine

GPU CUDA kernels for

physical operators

SQL Query Data set

Query queue

manager

connectors

JDBC ODBC ADO.NETPython Driver

CLI client

Filesystemext4/XFS/ZFS

SQream solution

powered by

NVIDIA GPUs

Classic CPU Approach Next Generation

Multicore/GPU Enabled

SQream DB server SQream DB Compiler

SQL parser

Optimizations

Parallel query

graph plan

GPU/CPU Runtime

Columnar storage

engine

GPU CUDA kernels for

physical operators

SQL Query Data set

Query queue

manager

connectors

JDBC ODBC ADO.NETPython Driver

CLI client

Filesystemext4/XFS/ZFS

52

GRAPHISTRYOVERVIEW AND USE CASE

53

INTERACTIVE VISUALIZATION SOLUTION

Graphistry is a graph visualization platform, powered by NVIDIA GPUs, that allows customers to interactively visualize millions of data points. Enabling customers to see 100x more in sub-seconds change the dynamic of how customers will interact with and derive insights from their data.

Overview

Fraud: looked at money laundering in the blockchain

Bio: scientists have looked at protein networks, protons, & brains

Threat research: tracing the origin of malware through nearby families

Social (email): engagement of customers

Survey data correlations: (“super venn diagrams”)

Industry use cases

54

DEA theft of Silk Road bitcoins SIEM attack escalation Dropbox external sharing logs

Datacenter outages ML: Feature correlation, NLPTwitter botnet deconstruction

DELIVERING UNPRECEDENTED DATA CORRELATIONS TO CUSTOMERS

55

VISUAL PLAYBOOK IN ACTION


56

BLAZEGRAPHOVERVIEW AND USE CASE

57

ACCELERATED GRAPH DATABASE SOLUTION

Blazegraph is ultra-scalable, high-performance GPU accelerated graph database with support for the Blueprints and RDF/SPARQL APIs, available in a range of versions that provide solutions to the challenge of scaling graphs. Blazegraph exploits the main-memory bandwidth advantages of NVIDIA GPUs to provide extreme scaling that is 100 times faster than CPU main memory-based approaches.

Overview

Finance: Fraud detection (Use pattern matching queries to detect fraudulent transactions in a fraction of the time required previously)

Security: Cyberdefense (Quickly find anomalous behavior in a graph of network traffic; Identify Vulnerabilities in Networks)

Research: Medicine and life sciences (Use pattern matching queries to detect fraudulent transactions in a fraction of the time required previously)

Industry use cases

58

DELIVERING ACCELERATED INTERACTIVE INSIGHTS TO CUSTOMERS

PROBLEM

Need to see and understand the

relationship and the format of all the

content with speed and at scale

SOLUTIONSBlazegraph High Availability (HA) Graph Database PlatformRDF / OWL Semantic Web LanguagesSPARQL Graph Query

RESULTS

200% Increase in time on site

30% increase in page opens

65% increase in video starts

PROBLEM

need to visualize network relationships,

- support new platforms rapidly, - no

single model – integrate data

SOLUTIONS

Use open standards

RDF and OWL chosen for information

model

Model entire domain

RESULTS

Rapid deployment

Response time (minutes)

Very few defects found

EMCYAHOO

59

GRAPH DATABASE IN ACTION


NVIDIA AI & ANALYTICS - Computer Sweden...NVIDIA AI & ANALYTICS 2 A NEW ERA OF COMPUTING PC INTERNET...

Documents

Transcript of NVIDIA AI & ANALYTICS - Computer Sweden...NVIDIA AI & ANALYTICS 2 A NEW ERA OF COMPUTING PC INTERNET...