Neo4j Introduction (for Techies)
-
Upload
patrick-baumgartner -
Category
Technology
-
view
114 -
download
1
description
Transcript of Neo4j Introduction (for Techies)
Neo4j from the Command Line Perspective (Introduction)
JUG MK | Skopje | 02.10.2013
2
Speaker Profile
Patrick Baumgartner
• Senior Software Consultant | Partner @ Swiftmind
• Spring Framework, OSGi, Neo4j • Spring Trainer, Neo4j Fanboy, Agilista • Co-Author of „OSGi für Praktiker“ • Co-Organizer of Neo4j Meetup Zurich • @patbaumgartner
3
Swiftmind
Your experts for Enterprise Java Areas of expertise • Java EE • Spring Framework • OSGi • Agile Methodologies • Software Engineering Best Practices
Headquarter • Zürich, Schweiz • @swiftmind • http://www.swiftmind.com
4
Agenda
• Introduction to Graphs • Use cases • Neo4j • Querying • Import / Export • Tools und APIs • Polyglot Persistence • Q & A
5
New Demands on Data Access
• Structured and unstructured data
• Massive amounts of data • Inexpensive horizontal
scaling • Apps and data in the
cloud • Social network features • …
6
New Types of Data Stores
7
NoSQL
Data Volume
Data Complexity
Key-Value Stores
Column Family
Document Databases
Graph Databases
Relational Databases
90% of all use cases
8
Property Graph Model
• Graphs are made of – Vertices / Nodes – Edges / Relationships – Properties
Alice
Bob
Car
name:Bob age:63 sex:male
name:Alice age:21 sex:female
type:car vendor:tesla
type:OW
NS
type:DRIVES
9
Where is my graph? Use cases
10
Social Data
Alice Carol Luke
Bob
Sam
Pete KNOWS
KN
OW
S
KNOWS KNOWS
11
Is it the only use case? Social Graph
12
Spatial Data
1 2 5
3
13
8 ROAD
RO
AD
ROAD type:ROAD length:3km
type:Hotel name:Radisson lat:47.452549 long:8.564615
type:Hotel name:Widder lat:47.373517 long:8.539252
13
Social & Spatial Data
Alice Mat 5
Bob
13
8 ROAD
RO
AD
LIKES KNOWS
type:Hotel name:Radisson Blu lat:47.452549 long:8.564615
14
Financial Data
Alice Hotel Luke
Bank
ATM
Pete WITHDRAW
TRANSFER OWNS
15
Other graph use cases?
16
Railroad in Switzerland
17
Your Business Domain
Process
KPI
Device
Activity
Function
Service CONTAINS
DEP
END
S
18
Neo4j The Big Picture
19
Neo4j – Big Picture
• Most used graph data base – Robust & Mature: in production 24/7 since 2003 – Community: Ecosystem with Tools, Bindings,
Frameworks
• Licenses – AGPLv3 Community Edition – OpenSource – Advanced/Enterprise – for commercial applications
• Neo Technology – Development & Support – ~ 50 Persons / 6 Countries / 3 Continents
20
Features
• Object oriented, flexible network structure • ACID transactions • Horizontal scalable • Java API
– Java, Groovy, Scala, JRuby • Runtime
– Standalone Server – Embedded Database
• Disk based storage manager • Plugin and testing support
21
How do I find my data? Querying
22
Graph Querying
• How do I query the right nodes from the graph? – Tell me all friends of friends of friends who liked the
movie XYZ with more than 3 stars – ...
• Querying methods – Traversing with Neo4j API – Traversal descriptions – Graph algorithms – Index queries – Cypher queries
23
Neo4j API
Node bob = ...;for ((Relationship rel: bob.getRelationships(RelTypes.KNOWS)) { Node friend = rel.getEndNode(); System.out.println(friend.getProperty("name"));}
• Simple, but very low level API • To verbose for complex traversals
24
Traversal API
• Querying API for graphs • Describes paths trough the graph • Call back based • Fluent API • Several predefined constructs
• Neo4j implements the following graph algorithms: – A* – Dijkstra – pathsWithLength – shortestPath – allSimplePaths – allPaths
25
Traversal API – Constructs
26
Example: Traversal API
TraversalDescription directFriends = Traversal.description() .breadthFirst() .relationships(RelTypes.KNOWS) .evaluator(Evaluators.toDepth(1)) .evaluator(Evaluators.excludeStartPosition());
for(Path p : directFriends.traverse(bob)) {System.out.println(p.endNode().getProperty(„firstname“));
}
27
Index Queries
• Support for node and relationship indices
• Lucene as default implementation
node id property value 15 name Alice
15 yearOfBirth 1972
16 name Bob
46 name Carol
46 yearOfBirth 1983
28
Cypher Query Language
• Graph query language similar to SQL • Declarative language • Defines „What“ and not „How“ • Uses pattern matching • Support for
– Filters – Pagination
• Supports CRUD functionality
29
Cypher Query Language – Elemente
Element Description START Start element(s) (index- or id-lookup) MATCH Pattern to find nodes, describes paths
(a) –[knows]-> (b) WHERE Result filter (boolean expression) SKIP LIMIT Pagination RETURN Defines return elements ORDER BY Sorting by properties PARAMETERS Parameter-Map, die im Cypher mittels Key oder
Position verwendet werden kann CREATE Creates node nodes and relations SET Updates properties on nodes and relations DELETE Deletes unused nodes or relationships
30
Example: Cypher Query
String query = "START person=node(12) “+ “MATCH person-[:KNOWS]-friend “+ “RETURN friend“;
ExecutionEngine engine = new ExecutionEngine(graphDb);ExecutionResult result = engine.execute(query);
Iterator<Object> column = result.columnAs("friend");while(column.hasNext()) {Node friendNode = (Node)column.next();System.out.println(friendNode.getProperty(PROP_FIRSTNAME));
}
person friend KNOWS
31
Example: Cypher Create Query
CREATE (carol {name:“Carol“}) return carol; +------------------------+| carol |+------------------------+| Node[1]{name:“carol“} |+------------------------+
START carol=node(1) CREATE (bob {name:“Bob"}), (bob)-[:knows]->(carol) return carol;
START carol=node(1)
CREATE (apple {name:“Apple“,type:“Fruit“}), (carol)-[:owns {number: 5}]->(apple);
Carol Bob KNOWS
Carol
Carol Apple type: Fruit
OWNS number: 5
32
Shipping data Import / Export – Node Manipulation
33
Node Creation with Java
org.neo4j.graphdb.GraphDatabaseService graphDb = new GraphDatabaseFactory().newEmbeddedDatabase(DB_PATH);
Transaction tx = graphDb.beginTx();try {
Node peter = createNode(graphDb, “Alice");Node carol = createNode(graphDb, “Carol"); peter.createRelationshipTo(carol, RelTypes.KNOWS);..
tx.success();} finally {tx.finish();}public Node createNode(GraphDatabaseService graphDb, String firstname) {
Node person = graphDb.createNode();person.setProperty(„firstname“, firstname);return person;
}
Alice Carol KNOWS
34
NeoClipse
35
Spring Data Neo4j (SDN)
• POJOs mapped as nodes or relationships – type safe • Works directly Database, typically embedded mode • Data is fetched very fast and lazy • Stores everything within a @Transaction • Uses heavily AspectJ magic to enhance the POJOs • …
36
Spring Data Neo4j – Entity
_type_: com.example.Person name: "Alice" age: 42
@NodeEntity public class Person { private String name;
private int age;
// getters/setters…
}
Person alice = new Person();
alice.setName("Alice");
alice.setAge(42);
alice.persist();
Node
37
Spring Data Neo4j – NodeEntity
@NodeEntity public class Person { private String name; private int yearOfBirth; @RelatedTo(type = "KNOWS", direction = Direction.OUTGOING) private Set<Person> knownPersons; public void knows(Person p) { knownPersons.add(p); } public Set<Person> getFriends() { return knownPersons; } } Person alice = ...; alice.knows(bob); alice.knows(carol);
Alice Bob KNOWS
Carol
KNOWS
38
Spring Data Neo4j – Relationship
@RelationshipEntity public class Knows { private int sinceYear; public Knows since(int year) { this.sinceYear = year; return this; } } @NodeEntity public class Person { public Known knows(Person p) { return this.relateTo(p, Knows.class, "KNOWS"); } } Person alice = ...; Person bob ...; alice.knows(bob).since(2012);
Alice Bob KNOWS since: 2012
39
Spring Data Neo4j – Repository
public interface PersonRepository extends GraphRepository<Person> {
@Query("start person = {0} match ... return ...")
Iterable<Product> getOwnedServices(Person person);
Iterable<Person> findByName(String name);
Iterable<Person> findByNameLike(String name);
}
@Autowired
PersonRepository personRepository;
Person alice = personRepository.findByName("Alice");
40
Thinkerpop Blueprint
Thinkerpop Blueprint Extensions • GraphML Reader/Writer
Library • GraphSon Reader/Writer
Library • GML Reader/Writer Library • Java Universal Network/
Graph Framework
41
GraphML
• GraphML (http://graphml.graphdrawing.org/ ) – Standardized format – Tool support – Pure and simple XML
<graph id="G" edgedefault="directed"> <node id="n0"/> <node id="n1"/> <edge source="n0" target="n1"/>
</graph>
42
Neo4j Tools
• GraphML – Gremlin Plugin (no further development) – Thinkerpop Blueprint Extensions
• CSV
– BatchInserter – ParallelBatchInserter
43
Useful tools for your daily work Tools & APIs
44
Tools & APIs (2)
• Web console – Data overview – Querying and visual graph presentation – „Command line“
• Neo4j Shell – Command line – Sends UNIX-like Commands via RMI (cd, ls, pwd) – Cypher Queries Support
• REST API – JSON – Clients for Java, .NET, PHP, Ruby, …
• Neoclipse – Visual presentation of the graph – Filter options for partial graph presentation
45
What?! Your app uses only one DB? Polyglot Persistence
46
Polyglot Persistence Web shop application
User Sessions Shopping Cart Recommendations
Product Catalog Analytics Financial Data
Redis Riak Neo4j
MongoDB Casandra MySQL
47
Neo4j on Meetup.com
48
Q&A
49
Thank you!
Patrick Baumgartner, @patbaumgartner patrick.baumgartner [at] swiftmind [dot] com http://www.swiftmind.com @swiftmind