Vital.AI Creating Intelligent Apps

54
Creating Intelligent Apps with Semantics & Big Data October 2, 2013 Marc C. Hadfield, Founder Email: [email protected] Thursday, October 3, 13

description

Vital.AI's Big Data App Platform allows creating data-driven apps rapidly.

Transcript of Vital.AI Creating Intelligent Apps

Page 1: Vital.AI Creating Intelligent Apps

C r e a t i n g I n t e l l i g e n t A p p sw i t h S e m a n t i c s & B i g D a t a

O c t o b e r 2 , 2 0 1 3

Marc C. Hadfield, FounderEmail: [email protected]

Thursday, October 3, 13

Page 2: Vital.AI Creating Intelligent Apps

<introduction>Marc HadfieldFounder of Vital.AI

Vital.AIBig Data Application Platform:

Semantics & Big Data in one platform,

combining multiple Data Analysis

techniques.

Thursday, October 3, 13

Page 3: Vital.AI Creating Intelligent Apps

Intelligent Applications

Development Processes

Business Value

Today:

Thursday, October 3, 13

Page 4: Vital.AI Creating Intelligent Apps

Intelligent Applications:

...learn from experience.

Thus, are experience-driven: data-driven.

“self-optimizing”

Thursday, October 3, 13

Page 5: Vital.AI Creating Intelligent Apps

Data-Driven Applications

Thursday, October 3, 13

Page 6: Vital.AI Creating Intelligent Apps

Moderni.st

Thursday, October 3, 13

Page 7: Vital.AI Creating Intelligent Apps

Moderni.st

Thursday, October 3, 13

Page 8: Vital.AI Creating Intelligent Apps

Personal Agent App

Thursday, October 3, 13

Page 9: Vital.AI Creating Intelligent Apps

Why are data-driven applications different?

Thursday, October 3, 13

Page 10: Vital.AI Creating Intelligent Apps

User Interface

Business Logic

Database

Traditional Applications

Why are data-driven applications different?

“sta%c”

Thursday, October 3, 13

Page 11: Vital.AI Creating Intelligent Apps

Why are data-driven applications different?

“dynamic”

User Interface

Business Logic

Events

Data-Driven Applications

Data

Analytics

Derived Behavior

Logic

Thursday, October 3, 13

Page 12: Vital.AI Creating Intelligent Apps

Data-Driven Applications use many different technologies, with many different skill sets:greater power, higher cost.

Thursday, October 3, 13

Page 13: Vital.AI Creating Intelligent Apps

Why do we want a data-driven application?

Thursday, October 3, 13

Page 14: Vital.AI Creating Intelligent Apps

What are the components of a data-driven application?

Data Collection

Dynamically Adjust Behaviors

Data Analysis / Feedback

Thursday, October 3, 13

Page 15: Vital.AI Creating Intelligent Apps

Implementing a Data-Driven Application.

Thursday, October 3, 13

Page 16: Vital.AI Creating Intelligent Apps

Implementation Goals:

Off the shelf?

Minimize Impedance Mismatch:Mobile/Web User Interface vs.

Semantic vs.Big Data vs.

...

Efficient Development Processes

Thursday, October 3, 13

Page 17: Vital.AI Creating Intelligent Apps

Vital.AI Platform Stack

Clean separation of responsibilities and skill sets.

Rapid Development of Data-Driven Apps.

VitalSigns: Ontology-based Data Model.

Thursday, October 3, 13

Page 18: Vital.AI Creating Intelligent Apps

Vital Prime

REST Interface

Collects Real-Time Events

Database Interfaces:HBase, Allegrograph, ...

Workflow Interface

Script Engine

In-Memory Analytics

User State Management

Thursday, October 3, 13

Page 19: Vital.AI Creating Intelligent Apps

Vital Flows

Implements Workflows:

Natural Language Processing

Run Predictive Analytics

Graph Analytics

Logical Inference

Many open-source components in a common workflow framework.

Thursday, October 3, 13

Page 20: Vital.AI Creating Intelligent Apps

Hadoop

Implements Big Data Analysis:

Machine Learning

Build Predictive Models

Thursday, October 3, 13

Page 21: Vital.AI Creating Intelligent Apps

Vital Core Ontology

Thursday, October 3, 13

Page 22: Vital.AI Creating Intelligent Apps

Vital Core Ontology

Vital Domain Ontology

Application Domain Ontology

Extending the Ontology

Thursday, October 3, 13

Page 23: Vital.AI Creating Intelligent Apps

Generating Data Bindings with VitalSigns:

Ontology VitalSigns

Groovy Bindings

Semantic Bindings

Hadoop Bindings

Prolog Bindings

Graph Bindings

HBase Bindings

JavaScript Bindings

Thursday, October 3, 13

Page 24: Vital.AI Creating Intelligent Apps

person123.name = "John"person123.worksFor.company456

<person123> <hasName> "John"<worksFor123> <hasSource> <person123><worksFor123> <hasDestination> <company456><worksFor123> <hasType> <worksFor>

person123, Node:type=Person, Node:hasName="John"worksFor123, Edge:type=worksFor, Edge:hasSource=person123, Edge:hasDestination=company456

Groovy

RDF

HBase

Data Representations

Thursday, October 3, 13

Page 25: Vital.AI Creating Intelligent Apps

Editing the Ontology

Thursday, October 3, 13

Page 26: Vital.AI Creating Intelligent Apps

Developing with the Ontology in UI, Hadoop, NLP, Scripts, ...

Node:Person Node:PersonEdge:hasFriend

Set<Friend> person123.getFriends()

“Best Practices” in Ontology Development.

Thursday, October 3, 13

Page 27: Vital.AI Creating Intelligent Apps

Data Analysis

Thursday, October 3, 13

Page 28: Vital.AI Creating Intelligent Apps

Using Natural Language Processing

Topic Categorization Extract Entities

Dialogue System Entity Normalization

Thursday, October 3, 13

Page 29: Vital.AI Creating Intelligent Apps

Using Graph Analytics

PageRank, Centrality, Interest Graph, ...

Thursday, October 3, 13

Page 30: Vital.AI Creating Intelligent Apps

Using Machine Learning

Implemented via Hadoop

Algorithms from Mahout

Build Predictive Models

Models used in Workflows

Data defined in Ontology

Classification, Clustering...

Thursday, October 3, 13

Page 31: Vital.AI Creating Intelligent Apps

Data Visualization - Cytoscape

Thursday, October 3, 13

Page 32: Vital.AI Creating Intelligent Apps

Data Management - Dashboard, Wiki

Thursday, October 3, 13

Page 33: Vital.AI Creating Intelligent Apps

Examples

Shopping Recommendation App

Personal Agent App

Content Recommendation App

Thursday, October 3, 13

Page 34: Vital.AI Creating Intelligent Apps

Content Recommendation

REST Interface

Vital Client

Vital Prime

Vital Flow Queue

Integrator NLP HadoopML Model

S3

Elastic Map/Reduce

Mahout

300M Users

100K Publishers

NLP

Machine Learning

Content + User Signals

Recommendations

Thursday, October 3, 13

Page 35: Vital.AI Creating Intelligent Apps

Personal Agent

Dialogue System

Thursday, October 3, 13

Page 36: Vital.AI Creating Intelligent Apps

Personal Agent

Interest Graph Recommendations

Thursday, October 3, 13

Page 37: Vital.AI Creating Intelligent Apps

Thursday, October 3, 13

Page 38: Vital.AI Creating Intelligent Apps

Thursday, October 3, 13

Page 39: Vital.AI Creating Intelligent Apps

Thursday, October 3, 13

Page 40: Vital.AI Creating Intelligent Apps

Thursday, October 3, 13

Page 41: Vital.AI Creating Intelligent Apps

Thursday, October 3, 13

Page 42: Vital.AI Creating Intelligent Apps

Thursday, October 3, 13

Page 43: Vital.AI Creating Intelligent Apps

Thursday, October 3, 13

Page 44: Vital.AI Creating Intelligent Apps

Thursday, October 3, 13

Page 45: Vital.AI Creating Intelligent Apps

Thursday, October 3, 13

Page 46: Vital.AI Creating Intelligent Apps

Thursday, October 3, 13

Page 47: Vital.AI Creating Intelligent Apps

Moderni.st

App Server

Vital Prime

IntegratorFlow

NLP Flow Logger Flow

Inference Flow

VITAL APIVitalSigns

Web Browser /Mobile Device

HDFS

Hbase Hadoop JobsMahout

Vital Flows

Flume

Allegrograph

Thursday, October 3, 13

Page 48: Vital.AI Creating Intelligent Apps

Creating an Intelligent App

Create User Interface.Use Vital Client, Send Signals

Create App Ontology, includingDomain Classes & Events/Signals

Define Workflows for Data Processing

Define Predictive Models, Build Models

Create scripts for App Functionality

Deploy Components

Thursday, October 3, 13

Page 49: Vital.AI Creating Intelligent Apps

Development Processes

Use the right tool for the right job,

& the right people for the right job.

Integrated Framework.

Coordinate on Ontology.

Eliminate Data Mis-Matches.

Clean code separation of “paradigms.”

Solve “Variety” Big Data challenge.

Thursday, October 3, 13

Page 50: Vital.AI Creating Intelligent Apps

Business Value

Intelligent App.

Efficient, Rapid Development.

Flexibility to learn, adapt, improve.

Greatly reduce risk.

Thursday, October 3, 13

Page 51: Vital.AI Creating Intelligent Apps

For more information, please contact:Marc C. Hadfield, FounderE: [email protected]: 917.463.4776

51

THANK YOU!

Thursday, October 3, 13

Page 52: Vital.AI Creating Intelligent Apps

Thursday, October 3, 13

Page 53: Vital.AI Creating Intelligent Apps

Thursday, October 3, 13

Page 54: Vital.AI Creating Intelligent Apps

Thursday, October 3, 13