Using Graph Databases For Insights Into Connected Data.
-
Upload
xebia-it-architects -
Category
Technology
-
view
115 -
download
1
description
Transcript of Using Graph Databases For Insights Into Connected Data.
![Page 1: Using Graph Databases For Insights Into Connected Data.](https://reader035.fdocuments.in/reader035/viewer/2022062617/54c6c77c4a79596b3d8b45a3/html5/thumbnails/1.jpg)
Xebia India 1
Using Graph Databases For Insights Into Connected Data
Gagan Agrawal
![Page 2: Using Graph Databases For Insights Into Connected Data.](https://reader035.fdocuments.in/reader035/viewer/2022062617/54c6c77c4a79596b3d8b45a3/html5/thumbnails/2.jpg)
Netherlands | USA | India | France | UK
SOFTWARE DEVELOPMENT DONE RIGHT
![Page 3: Using Graph Databases For Insights Into Connected Data.](https://reader035.fdocuments.in/reader035/viewer/2022062617/54c6c77c4a79596b3d8b45a3/html5/thumbnails/3.jpg)
Xebia India 3
Agenda
High level view of Graph Space Comparison with RDBMS and other NoSQL
stores Data Modeling Cypher : Graph Query Language Graph Database Internals Graphs In Real World
![Page 4: Using Graph Databases For Insights Into Connected Data.](https://reader035.fdocuments.in/reader035/viewer/2022062617/54c6c77c4a79596b3d8b45a3/html5/thumbnails/4.jpg)
Xebia India 4
What is a Graph?
![Page 5: Using Graph Databases For Insights Into Connected Data.](https://reader035.fdocuments.in/reader035/viewer/2022062617/54c6c77c4a79596b3d8b45a3/html5/thumbnails/5.jpg)
![Page 6: Using Graph Databases For Insights Into Connected Data.](https://reader035.fdocuments.in/reader035/viewer/2022062617/54c6c77c4a79596b3d8b45a3/html5/thumbnails/6.jpg)
Xebia India 6
What is a Graph? A collection of vertices and edges. Set of nodes and the relationships that connect
them. Graph Represents -
Entities as NODES The way those entities relate to the world as
RELATIONSHIP Allows to model all kind of scenarios
System of road Medical history Supply chain management Data Center
![Page 7: Using Graph Databases For Insights Into Connected Data.](https://reader035.fdocuments.in/reader035/viewer/2022062617/54c6c77c4a79596b3d8b45a3/html5/thumbnails/7.jpg)
![Page 8: Using Graph Databases For Insights Into Connected Data.](https://reader035.fdocuments.in/reader035/viewer/2022062617/54c6c77c4a79596b3d8b45a3/html5/thumbnails/8.jpg)
![Page 9: Using Graph Databases For Insights Into Connected Data.](https://reader035.fdocuments.in/reader035/viewer/2022062617/54c6c77c4a79596b3d8b45a3/html5/thumbnails/9.jpg)
Xebia India 9
High Level view of Graph Space Graph Databases - Technologies used primarily
for transactional online graph persistence – OLTP.
Graph Compute Engines - Tecnologies used primarily for offline graph analytics - OLAP.
![Page 10: Using Graph Databases For Insights Into Connected Data.](https://reader035.fdocuments.in/reader035/viewer/2022062617/54c6c77c4a79596b3d8b45a3/html5/thumbnails/10.jpg)
Xebia India 10
Graph Databases Online database management system with -
Create, Read, Update, Delete
methods that expose a graph data model. Built for use with transactional (OLTP) systems. Used for richly connected data. Querying is performed through traversals. Can perform millions of traversal steps per
second. Traversal step resembles a join in a RDBMS
![Page 11: Using Graph Databases For Insights Into Connected Data.](https://reader035.fdocuments.in/reader035/viewer/2022062617/54c6c77c4a79596b3d8b45a3/html5/thumbnails/11.jpg)
Xebia India 11
Graph Database Properties
The Underlying Storage : Native / Non-Native
The Processing Engine : Native / Non-Native
![Page 12: Using Graph Databases For Insights Into Connected Data.](https://reader035.fdocuments.in/reader035/viewer/2022062617/54c6c77c4a79596b3d8b45a3/html5/thumbnails/12.jpg)
Xebia India 12
Graph DB – The Underlying Storage Native Graph Storage – Optimized and designed
for storing and managing graphs.
Non-Native Graph Storage – Serialize the graph data into a relational database, an object oriented database, or some other general purpose data store.
![Page 13: Using Graph Databases For Insights Into Connected Data.](https://reader035.fdocuments.in/reader035/viewer/2022062617/54c6c77c4a79596b3d8b45a3/html5/thumbnails/13.jpg)
![Page 14: Using Graph Databases For Insights Into Connected Data.](https://reader035.fdocuments.in/reader035/viewer/2022062617/54c6c77c4a79596b3d8b45a3/html5/thumbnails/14.jpg)
Xebia India 14
Graph DB – The processing Engine
Index free adjacency – Connected Nodes physically point to each other in the database
![Page 15: Using Graph Databases For Insights Into Connected Data.](https://reader035.fdocuments.in/reader035/viewer/2022062617/54c6c77c4a79596b3d8b45a3/html5/thumbnails/15.jpg)
![Page 16: Using Graph Databases For Insights Into Connected Data.](https://reader035.fdocuments.in/reader035/viewer/2022062617/54c6c77c4a79596b3d8b45a3/html5/thumbnails/16.jpg)
![Page 17: Using Graph Databases For Insights Into Connected Data.](https://reader035.fdocuments.in/reader035/viewer/2022062617/54c6c77c4a79596b3d8b45a3/html5/thumbnails/17.jpg)
![Page 18: Using Graph Databases For Insights Into Connected Data.](https://reader035.fdocuments.in/reader035/viewer/2022062617/54c6c77c4a79596b3d8b45a3/html5/thumbnails/18.jpg)
Xebia India 18
Power of Graph Databases
Performance
Flexibility
Agility
![Page 19: Using Graph Databases For Insights Into Connected Data.](https://reader035.fdocuments.in/reader035/viewer/2022062617/54c6c77c4a79596b3d8b45a3/html5/thumbnails/19.jpg)
Xebia India 19
Comparison Relational Databases
NoSQL Databases
Graph Databases
![Page 20: Using Graph Databases For Insights Into Connected Data.](https://reader035.fdocuments.in/reader035/viewer/2022062617/54c6c77c4a79596b3d8b45a3/html5/thumbnails/20.jpg)
Xebia India 20
Relational Databases Lack Relationships Initially designed to codify paper forms and
tabular structures. Deal poorly with relationships. The rise in connectedness translates into
increased joins. Lower performance. Difficult to cater for changing business needs.
![Page 21: Using Graph Databases For Insights Into Connected Data.](https://reader035.fdocuments.in/reader035/viewer/2022062617/54c6c77c4a79596b3d8b45a3/html5/thumbnails/21.jpg)
![Page 22: Using Graph Databases For Insights Into Connected Data.](https://reader035.fdocuments.in/reader035/viewer/2022062617/54c6c77c4a79596b3d8b45a3/html5/thumbnails/22.jpg)
![Page 23: Using Graph Databases For Insights Into Connected Data.](https://reader035.fdocuments.in/reader035/viewer/2022062617/54c6c77c4a79596b3d8b45a3/html5/thumbnails/23.jpg)
Xebia India 23
NoSQL Databases also lack Relationships NOSQL Databases e.g key-value, document or
column oriented store sets of disconnected values/documents/columns.
Makes it difficult to use them for connected data and graphs.
One of the solution is to embed an aggregate's identifier inside the field belonging to another aggregate.
Effectively introducing foreign keys Requires joining aggregates at the application
level.
![Page 24: Using Graph Databases For Insights Into Connected Data.](https://reader035.fdocuments.in/reader035/viewer/2022062617/54c6c77c4a79596b3d8b45a3/html5/thumbnails/24.jpg)
Xebia India 24
NoSQL DB Relationships between aggregates aren't first
class citizens in the data model. Foreign aggregate "links" are not reflexive. Need to use some external compute infrastructure
e.g Hadoop for such processing. Do not maintain consistency of connected data. Do not support index-free adjacency.
![Page 25: Using Graph Databases For Insights Into Connected Data.](https://reader035.fdocuments.in/reader035/viewer/2022062617/54c6c77c4a79596b3d8b45a3/html5/thumbnails/25.jpg)
![Page 26: Using Graph Databases For Insights Into Connected Data.](https://reader035.fdocuments.in/reader035/viewer/2022062617/54c6c77c4a79596b3d8b45a3/html5/thumbnails/26.jpg)
![Page 27: Using Graph Databases For Insights Into Connected Data.](https://reader035.fdocuments.in/reader035/viewer/2022062617/54c6c77c4a79596b3d8b45a3/html5/thumbnails/27.jpg)
Xebia India 27
Graph DB Find friends-of-friends in a social network, to a
maximum depth of 5. Total records : 1,000,000 Each with approximately 50 friends
![Page 28: Using Graph Databases For Insights Into Connected Data.](https://reader035.fdocuments.in/reader035/viewer/2022062617/54c6c77c4a79596b3d8b45a3/html5/thumbnails/28.jpg)
![Page 29: Using Graph Databases For Insights Into Connected Data.](https://reader035.fdocuments.in/reader035/viewer/2022062617/54c6c77c4a79596b3d8b45a3/html5/thumbnails/29.jpg)
Xebia India 29
Data Modeling with Graph
![Page 30: Using Graph Databases For Insights Into Connected Data.](https://reader035.fdocuments.in/reader035/viewer/2022062617/54c6c77c4a79596b3d8b45a3/html5/thumbnails/30.jpg)
Xebia India 30
Data Modeling “Whiteboard” friendly
The typical whiteboard view of a problem is a GRAPH.
Sketch in our creative and analytical modes, maps closely to the data model inside the database.
![Page 31: Using Graph Databases For Insights Into Connected Data.](https://reader035.fdocuments.in/reader035/viewer/2022062617/54c6c77c4a79596b3d8b45a3/html5/thumbnails/31.jpg)
![Page 32: Using Graph Databases For Insights Into Connected Data.](https://reader035.fdocuments.in/reader035/viewer/2022062617/54c6c77c4a79596b3d8b45a3/html5/thumbnails/32.jpg)
Xebia India 32
Cypher : Graph Query Language Pattern-Matching Query Language Humane language Expressive Declarative : Say what you want, now how Borrows from well know query languages Aggregation, Ordering, Limit Update the Graph
![Page 33: Using Graph Databases For Insights Into Connected Data.](https://reader035.fdocuments.in/reader035/viewer/2022062617/54c6c77c4a79596b3d8b45a3/html5/thumbnails/33.jpg)
Xebia India 33
Cypher Cypher Representation : (c)-[:KNOWS]->(b)-[:KNOWS]->(a), (c)-[:KNOWS]-
>(a)
(c)-[:KNOWS]->(b)-[:KNOWS]->(a)<-[:KNOWS]-(c)
![Page 34: Using Graph Databases For Insights Into Connected Data.](https://reader035.fdocuments.in/reader035/viewer/2022062617/54c6c77c4a79596b3d8b45a3/html5/thumbnails/34.jpg)
Xebia India 34
Cypher
START c=node:user(name='Michael')MATCH (c)-[:KNOWS]->(b)-[:KNOWS]->(a), (c)-
[:KNOWS]->(a)RETURN a, b
![Page 35: Using Graph Databases For Insights Into Connected Data.](https://reader035.fdocuments.in/reader035/viewer/2022062617/54c6c77c4a79596b3d8b45a3/html5/thumbnails/35.jpg)
Xebia India 35
Other Cypher Clauses WHERE
Provides criteria for filtering pattern matching results.
CREATE and CREATE UNIQUE Create nodes and relationships
DELETE Removes nodes, relationships and properties
SET Sets property values
![Page 36: Using Graph Databases For Insights Into Connected Data.](https://reader035.fdocuments.in/reader035/viewer/2022062617/54c6c77c4a79596b3d8b45a3/html5/thumbnails/36.jpg)
Xebia India 36
Other Cypher Clauses FOREACH
Performs an updating action for graph element in a list.
UNION Merge results from two or more queries.
WITH Chains subsequent query parts and forward
results from one to the next. Similar to piping commands in UNIX.
![Page 37: Using Graph Databases For Insights Into Connected Data.](https://reader035.fdocuments.in/reader035/viewer/2022062617/54c6c77c4a79596b3d8b45a3/html5/thumbnails/37.jpg)
Xebia India 37
Comparison of Relational and Graph Modeling
![Page 38: Using Graph Databases For Insights Into Connected Data.](https://reader035.fdocuments.in/reader035/viewer/2022062617/54c6c77c4a79596b3d8b45a3/html5/thumbnails/38.jpg)
![Page 39: Using Graph Databases For Insights Into Connected Data.](https://reader035.fdocuments.in/reader035/viewer/2022062617/54c6c77c4a79596b3d8b45a3/html5/thumbnails/39.jpg)
![Page 40: Using Graph Databases For Insights Into Connected Data.](https://reader035.fdocuments.in/reader035/viewer/2022062617/54c6c77c4a79596b3d8b45a3/html5/thumbnails/40.jpg)
![Page 41: Using Graph Databases For Insights Into Connected Data.](https://reader035.fdocuments.in/reader035/viewer/2022062617/54c6c77c4a79596b3d8b45a3/html5/thumbnails/41.jpg)
![Page 42: Using Graph Databases For Insights Into Connected Data.](https://reader035.fdocuments.in/reader035/viewer/2022062617/54c6c77c4a79596b3d8b45a3/html5/thumbnails/42.jpg)
![Page 43: Using Graph Databases For Insights Into Connected Data.](https://reader035.fdocuments.in/reader035/viewer/2022062617/54c6c77c4a79596b3d8b45a3/html5/thumbnails/43.jpg)
Xebia India 43
Graph Database Internals
![Page 44: Using Graph Databases For Insights Into Connected Data.](https://reader035.fdocuments.in/reader035/viewer/2022062617/54c6c77c4a79596b3d8b45a3/html5/thumbnails/44.jpg)
Xebia India 44
Non Functional Characteristics
Transactions Fully ACID
Recoverability Availability Scalability
![Page 45: Using Graph Databases For Insights Into Connected Data.](https://reader035.fdocuments.in/reader035/viewer/2022062617/54c6c77c4a79596b3d8b45a3/html5/thumbnails/45.jpg)
Xebia India 45
Scalability Capacity (Graph Size)
Latency (Response Time)
Read and Write Throughput
![Page 46: Using Graph Databases For Insights Into Connected Data.](https://reader035.fdocuments.in/reader035/viewer/2022062617/54c6c77c4a79596b3d8b45a3/html5/thumbnails/46.jpg)
Xebia India 46
Capacity 1.9 Release of Neo4j can support single graphs
having 10s of billions of nodes, relationships and properties.
The Neo4j team has publicly expressed the intention to support 100B+ nodes/relationships/properties in a single graph.
![Page 47: Using Graph Databases For Insights Into Connected Data.](https://reader035.fdocuments.in/reader035/viewer/2022062617/54c6c77c4a79596b3d8b45a3/html5/thumbnails/47.jpg)
Xebia India 47
Latency RDBMS – more data in tables/indexes result in
longer join operations. Graph DB doesn't suffer the same latency
problem. Index is used to find starting node. Traversal uses a combination of pointer chasing
and pattern matching to search the data. Performance does not depend on total size of the
dataset. Depends only on the data being queried.
![Page 48: Using Graph Databases For Insights Into Connected Data.](https://reader035.fdocuments.in/reader035/viewer/2022062617/54c6c77c4a79596b3d8b45a3/html5/thumbnails/48.jpg)
Xebia India 48
Throughput Constant performance irrespective of graph size.
![Page 49: Using Graph Databases For Insights Into Connected Data.](https://reader035.fdocuments.in/reader035/viewer/2022062617/54c6c77c4a79596b3d8b45a3/html5/thumbnails/49.jpg)
Xebia India 49
Graphs in the Real World
![Page 50: Using Graph Databases For Insights Into Connected Data.](https://reader035.fdocuments.in/reader035/viewer/2022062617/54c6c77c4a79596b3d8b45a3/html5/thumbnails/50.jpg)
Xebia India 50
Common Use Cases Social Recommendations Geo Logistics Networks : for package routing, finding shortest
Path Financial Transaction Graphs : for fraud detection
Master Data Management Bioinformatics : Era7 to relate complex web of information
that includes genes, proteins and enzymes Authorization and Access Control : Adobe Creative
Cloud, Telenor
![Page 51: Using Graph Databases For Insights Into Connected Data.](https://reader035.fdocuments.in/reader035/viewer/2022062617/54c6c77c4a79596b3d8b45a3/html5/thumbnails/51.jpg)
![Page 52: Using Graph Databases For Insights Into Connected Data.](https://reader035.fdocuments.in/reader035/viewer/2022062617/54c6c77c4a79596b3d8b45a3/html5/thumbnails/52.jpg)
![Page 53: Using Graph Databases For Insights Into Connected Data.](https://reader035.fdocuments.in/reader035/viewer/2022062617/54c6c77c4a79596b3d8b45a3/html5/thumbnails/53.jpg)
Xebia India 53
Thank You
![Page 54: Using Graph Databases For Insights Into Connected Data.](https://reader035.fdocuments.in/reader035/viewer/2022062617/54c6c77c4a79596b3d8b45a3/html5/thumbnails/54.jpg)
BigData & Real Time Analytics
Services
Solutions- Big data warehousing- Scalable big data etl- High volume web analytics
Trainings- Cloudera Data Analyst /
Developer / Admin Training
Products- Divolte- Wearable Sensors
Visualization (Tableau)
Integration (Sqoop, Flume , Storm)
Hadoop Powered Solutions (Pig, Hive, Oozie, Hbase Impala) (Solr, Elastic Search)
Core Hadoop (HDFS, MapReduce,Zookeeper, Cloudera Distribution)
Analytics Framework (Mahout)
![Page 55: Using Graph Databases For Insights Into Connected Data.](https://reader035.fdocuments.in/reader035/viewer/2022062617/54c6c77c4a79596b3d8b45a3/html5/thumbnails/55.jpg)
Contact us @
Xebia [email protected]
Websiteswww.xebia.inwww.xebia.comwww.xebia.fr
Thought LeadershipHtto://xebee.xebia.inhttp://blog.xebia.comhttp://podcast.xebia.com