Neo4j&Introduction Harnessing)the)Power)of)the)Graphfiles.meetup.com/18870621/MOHUG_Meetup.pdf ·...

23
Neo4j Introduction Harnessing the Power of the Graph Bill Carden – Enterprise AM Dave Fauth– Field SE

Transcript of Neo4j&Introduction Harnessing)the)Power)of)the)Graphfiles.meetup.com/18870621/MOHUG_Meetup.pdf ·...

Page 1: Neo4j&Introduction Harnessing)the)Power)of)the)Graphfiles.meetup.com/18870621/MOHUG_Meetup.pdf · Neo4j&Introduction Harnessing)the)Power)of)the)Graph Bill)Carden)–Enterprise)AM

Neo4j&Introduction

Harnessing)the)Power)of)the)Graph

Bill)Carden)– Enterprise)AM

Dave)Fauth– Field)SE

Page 2: Neo4j&Introduction Harnessing)the)Power)of)the)Graphfiles.meetup.com/18870621/MOHUG_Meetup.pdf · Neo4j&Introduction Harnessing)the)Power)of)the)Graph Bill)Carden)–Enterprise)AM

The&Whiteboard&Model&Is the&Physical&Model

Page 3: Neo4j&Introduction Harnessing)the)Power)of)the)Graphfiles.meetup.com/18870621/MOHUG_Meetup.pdf · Neo4j&Introduction Harnessing)the)Power)of)the)Graph Bill)Carden)–Enterprise)AM

CAR

name:)“Dan”born:)May)29,) 1970twitter:)“@dan”

name:)“Ann”born: Dec)5,)1975

since:)Jan)10,)2011

brand:)“Volvo”model:)“V70”

Property&Graph&Model

Nodes

• The)objects)in)the)graph• Can)have)nameIvalue)properties• Can)be)labeled

Relationships

• Relate)nodes by)type)and)direction• Can)have)nameIvalue properties

All-stored-and-managed-by-Neo4j-(no-external-engines-or-storage)

LOVES

LOVES

LIVES)WITHPERSON PERSON

Page 4: Neo4j&Introduction Harnessing)the)Power)of)the)Graphfiles.meetup.com/18870621/MOHUG_Meetup.pdf · Neo4j&Introduction Harnessing)the)Power)of)the)Graph Bill)Carden)–Enterprise)AM

Relational&Versus&Graph&Models

Relational&Model Graph&Model

KNOWSANDREAS

TOBIAS

MICA

DELIA

Person FriendPersonIFriend

ANDREASDELIA

TOBIAS

MICA

Page 5: Neo4j&Introduction Harnessing)the)Power)of)the)Graphfiles.meetup.com/18870621/MOHUG_Meetup.pdf · Neo4j&Introduction Harnessing)the)Power)of)the)Graph Bill)Carden)–Enterprise)AM

Key&Neo4j&Product&Features

Native&Graph&Storage

Ensures)data)consistency)and)performanceNative&Graph&Processing

Millions)of)hops)per)second,)in)real)time“Whiteboard&Friendly”&Data&Modeling

Model)data)as)it)naturally)occursHigh&Data&Integrity

Fully)ACID)transactions

Powerful,&Expressive&Query&Language

Requires)10x)to)100x)less)code)than)SQLScalability&and&High&Availability

Vertical)and)horizontal)scaling)optimized)for)graphsBuiltOin&ETL

Seamless)import)from)other)databasesIntegration

Drivers)and)APIs)for)popular)languages

MATCH

(A)

Page 6: Neo4j&Introduction Harnessing)the)Power)of)the)Graphfiles.meetup.com/18870621/MOHUG_Meetup.pdf · Neo4j&Introduction Harnessing)the)Power)of)the)Graph Bill)Carden)–Enterprise)AM

Neo4j-Clustering-Architecture&Optimized&for&Speed&&&Availability&at&Scale

6

Performance&Benefits

• No)network)hops)within)queries• RealBtime)operationswith)fast)and)consistent)response)times)

• Cache)sharding)spreads)cache)across)cluster)for)very)large)graphs

Clustering&Features

• MasterIslave)replication)with)master)reBelection)and)failover

• Each)instance)has)its)own)local)cache• Horizontal)scaling)&)disaster)recovery

Load&Balancer

Neo4jNeo4jNeo4j

Page 7: Neo4j&Introduction Harnessing)the)Power)of)the)Graphfiles.meetup.com/18870621/MOHUG_Meetup.pdf · Neo4j&Introduction Harnessing)the)Power)of)the)Graph Bill)Carden)–Enterprise)AM

Graph&Components&and&Cypher

Page 8: Neo4j&Introduction Harnessing)the)Power)of)the)Graphfiles.meetup.com/18870621/MOHUG_Meetup.pdf · Neo4j&Introduction Harnessing)the)Power)of)the)Graph Bill)Carden)–Enterprise)AM

Basic&Query:&Who&do&people&report&to?

MATCH) (:Employee{) firstName:“Steven”}) ) I[:REPORTS_TO]I>) (:Employee{) firstName:“Andrew”}) )

REPORTS_TOSteven Andrew

LABEL PROPERTY

NODE NODE

LABEL PROPERTY

Page 9: Neo4j&Introduction Harnessing)the)Power)of)the)Graphfiles.meetup.com/18870621/MOHUG_Meetup.pdf · Neo4j&Introduction Harnessing)the)Power)of)the)Graph Bill)Carden)–Enterprise)AM

Basic&Query:&Who&do&people&report&to?

MATCH(e:Employee)<-[:REPORTS_TO]-(sub:Employee)

RETURN*

Page 10: Neo4j&Introduction Harnessing)the)Power)of)the)Graphfiles.meetup.com/18870621/MOHUG_Meetup.pdf · Neo4j&Introduction Harnessing)the)Power)of)the)Graph Bill)Carden)–Enterprise)AM

MATCH&(sub)%[:REPORTS_TO*0..3]%>(boss),(report)%[:REPORTS_TO*1..3]%>(sub)

WHERE&boss.name@=@“John@Doe”RETURN sub.name AS Subordinate,@count(report)@AS Total

Express&Complex&Queries&Easily&with&Cypher

Find)all)direct)reports)and)how)many)people)they)manage,)up)to)3)levels)down

Cypher&Query

SQL&Query

Page 11: Neo4j&Introduction Harnessing)the)Power)of)the)Graphfiles.meetup.com/18870621/MOHUG_Meetup.pdf · Neo4j&Introduction Harnessing)the)Power)of)the)Graph Bill)Carden)–Enterprise)AM

MATCH&(c:City)4[*..2]4>(co:Country{name:'Canada'})RETURN&c.name;

Express&Complex&Queries&Easily&with&Cypher

Find)all)cities)in)Canada

Cypher&Query

MS&SQL&Query

WITH)AreasCTE AS()IIanchor) select,)start)with)the)country)of)Canada,)which)will)be)the)root)element) for)our)searchSELECT)AreaID,) AreaName,) ParentAreaID,) AreaTypeFROM)dbo.AreaWHERE)AreaName =)'Canada'UNION)ALLIIrecursive) select,)recursive) until)you)reach) a)leaf)(an)Area)which)is)not)a)parent)of)any)other)area)SELECT) a.AreaID,) a.AreaName,) a.ParentAreaID,) a.AreaTypeFROM)dbo.Area a)INNER) JOIN)AreasCTE s)ON)a.ParentAreaID =)s.AreaID))IINow,) you)will)have) all)Areas) in)Canada,)so)now)let's) filter)by)the)AreaType "City"SELECT) *) FROM)AreasCTEwhere)AreaType =)'City')

https://myadventuresincoding.wordpress.com/2014/05/02/sqlIserverIsimpleIrecursiveIqueryIexample/

Page 12: Neo4j&Introduction Harnessing)the)Power)of)the)Graphfiles.meetup.com/18870621/MOHUG_Meetup.pdf · Neo4j&Introduction Harnessing)the)Power)of)the)Graph Bill)Carden)–Enterprise)AM

Neo4j&for&Network&and&IT&Operations

Impact&analysis&and&network&planning

• Determine)impacts)of)node)failures,)outages)and)interruptions• Recommend)alternate)routes)around)affected)components

RootOcause&analysis

• Identify)root)causes)of)problems)and)trace)dependencies• Give)service)desks)visibility)into)IT)infrastructure)relationships

Routing&and&qualityOofOservice&mapping

• Find)the)best,)shortest)or)least)busy)path)to)resources• Identify)ideal)network)locations)to)introduce)new)services• Map)QoS from)segment)to)entire)net

IT&infrastructure&management

• Map)IT)services)to)physical)and)virtual)infrastructure)and)cost)centers

Page 13: Neo4j&Introduction Harnessing)the)Power)of)the)Graphfiles.meetup.com/18870621/MOHUG_Meetup.pdf · Neo4j&Introduction Harnessing)the)Power)of)the)Graph Bill)Carden)–Enterprise)AM

Feeding&the&World&with&Better&Food&Crops

Getting&to&this

Takes

• 7 Years)on)average• $M’s&in)development• 100’s&of)crossIbreeds• TBs of)discovery)data

Page 14: Neo4j&Introduction Harnessing)the)Power)of)the)Graphfiles.meetup.com/18870621/MOHUG_Meetup.pdf · Neo4j&Introduction Harnessing)the)Power)of)the)Graph Bill)Carden)–Enterprise)AM

Feeding&the&World&with&Better&Food&Crops

• Significant) volumes) of)data)stored) in)Exadata,)but) lineage/parental) queries)weren’t)able)to)respond) fast)enough

• Inefficiency) in)query)performance)limited) the) insights) their)tools)could)provide)scientists) to)enable) better)decisions

• No)ability) to)“recreate”) highIyield)crops)because) they)couldn’t)connect)the)generations) of)crossIbreeding

Challenges

Page 15: Neo4j&Introduction Harnessing)the)Power)of)the)Graphfiles.meetup.com/18870621/MOHUG_Meetup.pdf · Neo4j&Introduction Harnessing)the)Power)of)the)Graph Bill)Carden)–Enterprise)AM

Seed&Development&Graph

M F

M F

M

M F

F

M

F

Parental)Lineage

1st Generation

2nd Generation

3rd Generation

Nth)Generation

Measurements) in)all)generations) include)

observed) characteristics,)output) yield,)quality,)

breeding) behaviors,)genetic)sequencing

Page 16: Neo4j&Introduction Harnessing)the)Power)of)the)Graphfiles.meetup.com/18870621/MOHUG_Meetup.pdf · Neo4j&Introduction Harnessing)the)Power)of)the)Graph Bill)Carden)–Enterprise)AM

Neo4j&Enabling&Monsanto&to&Discover&Lineage

• Reduce) costs)on)verification) and)testing) by)knowing) lineage

• Reduce) time) to)market)with)new)seed) by)improving)discovery)process) and)reducing)guessing

• Connect)crop)performance)to)lineage,)enabling)better)YoY sales

Neo4j&Value

Page 17: Neo4j&Introduction Harnessing)the)Power)of)the)Graphfiles.meetup.com/18870621/MOHUG_Meetup.pdf · Neo4j&Introduction Harnessing)the)Power)of)the)Graph Bill)Carden)–Enterprise)AM

Fraud&Detection&Use&Case

Types&of&Fraud

• FirstIParty)Fraud• Synthetic)Identities)and)Fraud)Rings• Insurance)Fraud• Ecommerce)Fraud

Types&of&Analysis

• Traditional)Analysis• GraphIBased)Analysis

Fraud&Detection&and&Prevention

Common&Questions

Page 18: Neo4j&Introduction Harnessing)the)Power)of)the)Graphfiles.meetup.com/18870621/MOHUG_Meetup.pdf · Neo4j&Introduction Harnessing)the)Power)of)the)Graph Bill)Carden)–Enterprise)AM

Gartner’s&Layered&Fraud&Prevention&Approach(4)

(4))http://www.gartner.com/newsroom/id/1695014

Traditional&Fraud&Prevention

Analysis)of)users)and)their)endpoints

Analysis)ofnavigation)

behavior)and)suspect)patterns

Analysis)of)anomaly)

behavior)by)channel

Analysis)of)anomaly)behavior)correlated)

across)channels

Analysis)of)relationships)to)detect)

organized)crime)and)collusion

Layer&1

EndpointO

Centric

NavigationO

Centric

AccountO

Centric

CrossO

Channel

Entity&

Linking

Layer&2 Layer&3 Layer&4 Layer&5

DISCRETE&DATA&ANALYSIS CONNECTED& ANALYSIS

Page 19: Neo4j&Introduction Harnessing)the)Power)of)the)Graphfiles.meetup.com/18870621/MOHUG_Meetup.pdf · Neo4j&Introduction Harnessing)the)Power)of)the)Graph Bill)Carden)–Enterprise)AM

Pros

SimpleStops)rookies

Discrete&Data&Analysis

RevolvingDebt

INVESTIGATE

INVESTIGATE

Number) of)accounts

Cons

False)positivesFalse)negatives

Page 20: Neo4j&Introduction Harnessing)the)Power)of)the)Graphfiles.meetup.com/18870621/MOHUG_Meetup.pdf · Neo4j&Introduction Harnessing)the)Power)of)the)Graph Bill)Carden)–Enterprise)AM

Connected&Analysis

RevolvingDebt

Number) of)accounts

PROS

Detect)fraud)ringsFewer)false)negatives

Page 21: Neo4j&Introduction Harnessing)the)Power)of)the)Graphfiles.meetup.com/18870621/MOHUG_Meetup.pdf · Neo4j&Introduction Harnessing)the)Power)of)the)Graph Bill)Carden)–Enterprise)AM

Value

Effective)in)detecting)some)of)the)most)impactful)attacks,)even)from)organized)rings

Challenge

Extremely)difficult)with)traditional)technologies

For-example-a-tenBperson) fraud)bustBout) is)$1.5M,-assuming-100-false-identities-and-3-financial- instruments- per-identity,- each-with-a-$5K-credit-limit

Connected&Analysis&with&Neo4j

Page 22: Neo4j&Introduction Harnessing)the)Power)of)the)Graphfiles.meetup.com/18870621/MOHUG_Meetup.pdf · Neo4j&Introduction Harnessing)the)Power)of)the)Graph Bill)Carden)–Enterprise)AM

Demo

Page 23: Neo4j&Introduction Harnessing)the)Power)of)the)Graphfiles.meetup.com/18870621/MOHUG_Meetup.pdf · Neo4j&Introduction Harnessing)the)Power)of)the)Graph Bill)Carden)–Enterprise)AM

Thank)You

Bill)Carden

Enterprise)Account)Executive)[email protected]