Neo4j from the Command Line Perspective (Introduction)

JUG MK | Skopje | 02.10.2013

2

Speaker Profile

Patrick Baumgartner

•  Senior Software Consultant | Partner @ Swiftmind

•  Spring Framework, OSGi, Neo4j •  Spring Trainer, Neo4j Fanboy, Agilista •  Co-Author of „OSGi für Praktiker“ •  Co-Organizer of Neo4j Meetup Zurich •  @patbaumgartner

3

Swiftmind

Your experts for Enterprise Java Areas of expertise •  Java EE •  Spring Framework •  OSGi •  Agile Methodologies •  Software Engineering Best Practices

Headquarter •  Zürich, Schweiz •  @swiftmind •  http://www.swiftmind.com

4

Agenda

•  Introduction to Graphs •  Use cases •  Neo4j •  Querying •  Import / Export •  Tools und APIs •  Polyglot Persistence •  Q & A

5

New Demands on Data Access

•  Structured and unstructured data

•  Massive amounts of data •  Inexpensive horizontal

scaling •  Apps and data in the

cloud •  Social network features •  …

6

New Types of Data Stores

7

NoSQL

Data Volume

Data Complexity

Key-Value Stores

Column Family

Document Databases

Graph Databases

Relational Databases

90% of all use cases

8

Property Graph Model

•  Graphs are made of –  Vertices / Nodes –  Edges / Relationships –  Properties

Alice

Bob

Car

name:Bob age:63 sex:male

name:Alice age:21 sex:female

type:car vendor:tesla

type:OW

NS

type:DRIVES

9

Where is my graph? Use cases

10

Social Data

Alice Carol Luke

Bob

Sam

Pete KNOWS

KN

OW

S

KNOWS KNOWS

11

Is it the only use case? Social Graph

12

Spatial Data

1 2 5

3

13

8 ROAD

RO

AD

ROAD type:ROAD length:3km

type:Hotel name:Radisson lat:47.452549 long:8.564615

type:Hotel name:Widder lat:47.373517 long:8.539252

13

Social & Spatial Data

Alice Mat 5

Bob

13

8 ROAD

RO

AD

LIKES KNOWS

type:Hotel name:Radisson Blu lat:47.452549 long:8.564615

14

Financial Data

Alice Hotel Luke

Bank

ATM

Pete WITHDRAW

TRANSFER OWNS

15

Other graph use cases?

16

Railroad in Switzerland

17

Your Business Domain

Process

KPI

Device

Activity

Function

Service CONTAINS

DEP

END

S

18

Neo4j The Big Picture

19

Neo4j – Big Picture

•  Most used graph data base –  Robust & Mature: in production 24/7 since 2003 –  Community: Ecosystem with Tools, Bindings,

Frameworks

•  Licenses –  AGPLv3 Community Edition – OpenSource –  Advanced/Enterprise – for commercial applications

•  Neo Technology –  Development & Support –  ~ 50 Persons / 6 Countries / 3 Continents

20

Features

•  Object oriented, flexible network structure •  ACID transactions •  Horizontal scalable •  Java API

–  Java, Groovy, Scala, JRuby •  Runtime

–  Standalone Server –  Embedded Database

•  Disk based storage manager •  Plugin and testing support

21

How do I find my data? Querying

22

Graph Querying

•  How do I query the right nodes from the graph? –  Tell me all friends of friends of friends who liked the

movie XYZ with more than 3 stars –  ...

•  Querying methods –  Traversing with Neo4j API –  Traversal descriptions –  Graph algorithms –  Index queries –  Cypher queries

23

Neo4j API

Node bob = ...;for ((Relationship rel: bob.getRelationships(RelTypes.KNOWS)) { Node friend = rel.getEndNode(); System.out.println(friend.getProperty("name"));}

•  Simple, but very low level API •  To verbose for complex traversals

24

Traversal API

•  Querying API for graphs •  Describes paths trough the graph •  Call back based •  Fluent API •  Several predefined constructs

•  Neo4j implements the following graph algorithms: –  A* –  Dijkstra –  pathsWithLength –  shortestPath –  allSimplePaths –  allPaths

25

Traversal API – Constructs

26

Example: Traversal API

TraversalDescription directFriends = Traversal.description() .breadthFirst() .relationships(RelTypes.KNOWS) .evaluator(Evaluators.toDepth(1)) .evaluator(Evaluators.excludeStartPosition());

for(Path p : directFriends.traverse(bob)) {System.out.println(p.endNode().getProperty(„firstname“));

}

27

Index Queries

•  Support for node and relationship indices

•  Lucene as default implementation

node id property value 15 name Alice

15 yearOfBirth 1972

16 name Bob

46 name Carol

46 yearOfBirth 1983

28

Cypher Query Language

•  Graph query language similar to SQL •  Declarative language •  Defines „What“ and not „How“ •  Uses pattern matching •  Support for

–  Filters –  Pagination

•  Supports CRUD functionality

29

Cypher Query Language – Elemente

Element Description START Start element(s) (index- or id-lookup) MATCH Pattern to find nodes, describes paths

(a) –[knows]-> (b) WHERE Result filter (boolean expression) SKIP LIMIT Pagination RETURN Defines return elements ORDER BY Sorting by properties PARAMETERS Parameter-Map, die im Cypher mittels Key oder

Position verwendet werden kann CREATE Creates node nodes and relations SET Updates properties on nodes and relations DELETE Deletes unused nodes or relationships

30

Example: Cypher Query

String query = "START person=node(12) “+ “MATCH person-[:KNOWS]-friend “+ “RETURN friend“;

ExecutionEngine engine = new ExecutionEngine(graphDb);ExecutionResult result = engine.execute(query);

Iterator<Object> column = result.columnAs("friend");while(column.hasNext()) {Node friendNode = (Node)column.next();System.out.println(friendNode.getProperty(PROP_FIRSTNAME));

}

person friend KNOWS

31

Example: Cypher Create Query

CREATE (carol {name:“Carol“}) return carol; +------------------------+| carol |+------------------------+| Node[1]{name:“carol“} |+------------------------+

START carol=node(1) CREATE (bob {name:“Bob"}), (bob)-[:knows]->(carol) return carol;

START carol=node(1)

CREATE (apple {name:“Apple“,type:“Fruit“}), (carol)-[:owns {number: 5}]->(apple);

Carol Bob KNOWS

Carol

Carol Apple type: Fruit

OWNS number: 5

32

Shipping data Import / Export – Node Manipulation

33

Node Creation with Java

org.neo4j.graphdb.GraphDatabaseService graphDb = new GraphDatabaseFactory().newEmbeddedDatabase(DB_PATH);

Transaction tx = graphDb.beginTx();try {

Node peter = createNode(graphDb, “Alice");Node carol = createNode(graphDb, “Carol"); peter.createRelationshipTo(carol, RelTypes.KNOWS);..

tx.success();} finally {tx.finish();}public Node createNode(GraphDatabaseService graphDb, String firstname) {

Node person = graphDb.createNode();person.setProperty(„firstname“, firstname);return person;

}

Alice Carol KNOWS

34

NeoClipse

35

Spring Data Neo4j (SDN)

•  POJOs mapped as nodes or relationships – type safe •  Works directly Database, typically embedded mode •  Data is fetched very fast and lazy •  Stores everything within a @Transaction •  Uses heavily AspectJ magic to enhance the POJOs •  …

36

Spring Data Neo4j – Entity

_type_: com.example.Person name: "Alice" age: 42

@NodeEntity public class Person { private String name;

private int age;

// getters/setters…

}

Person alice = new Person();

alice.setName("Alice");

alice.setAge(42);

alice.persist();

Node

37

Spring Data Neo4j – NodeEntity

@NodeEntity public class Person { private String name; private int yearOfBirth; @RelatedTo(type = "KNOWS", direction = Direction.OUTGOING) private Set<Person> knownPersons; public void knows(Person p) { knownPersons.add(p); } public Set<Person> getFriends() { return knownPersons; } } Person alice = ...; alice.knows(bob); alice.knows(carol);

Alice Bob KNOWS

Carol

KNOWS

38

Spring Data Neo4j – Relationship

@RelationshipEntity public class Knows { private int sinceYear; public Knows since(int year) { this.sinceYear = year; return this; } } @NodeEntity public class Person { public Known knows(Person p) { return this.relateTo(p, Knows.class, "KNOWS"); } } Person alice = ...; Person bob ...; alice.knows(bob).since(2012);

Alice Bob KNOWS since: 2012

39

Spring Data Neo4j – Repository

public interface PersonRepository extends GraphRepository<Person> {

@Query("start person = {0} match ... return ...")

Iterable<Product> getOwnedServices(Person person);

Iterable<Person> findByName(String name);

Iterable<Person> findByNameLike(String name);

}

@Autowired

PersonRepository personRepository;

Person alice = personRepository.findByName("Alice");

40

Thinkerpop Blueprint

Thinkerpop Blueprint Extensions •  GraphML Reader/Writer

Library •  GraphSon Reader/Writer

Library •  GML Reader/Writer Library •  Java Universal Network/

Graph Framework

41

GraphML

•  GraphML (http://graphml.graphdrawing.org/ ) –  Standardized format –  Tool support –  Pure and simple XML

<graph id="G" edgedefault="directed"> <node id="n0"/> <node id="n1"/> <edge source="n0" target="n1"/>

</graph>

42

Neo4j Tools

•  GraphML –  Gremlin Plugin (no further development) –  Thinkerpop Blueprint Extensions

•  CSV

–  BatchInserter –  ParallelBatchInserter

43

Useful tools for your daily work Tools & APIs

44

Tools & APIs (2)

•  Web console –  Data overview –  Querying and visual graph presentation –  „Command line“

•  Neo4j Shell –  Command line –  Sends UNIX-like Commands via RMI (cd, ls, pwd) –  Cypher Queries Support

•  REST API –  JSON –  Clients for Java, .NET, PHP, Ruby, …

•  Neoclipse –  Visual presentation of the graph –  Filter options for partial graph presentation

45

What?! Your app uses only one DB? Polyglot Persistence

46

Polyglot Persistence Web shop application

User Sessions Shopping Cart Recommendations

Product Catalog Analytics Financial Data

Redis Riak Neo4j

MongoDB Casandra MySQL

47

Neo4j on Meetup.com

48

Q&A

49

Thank you!

Patrick Baumgartner, @patbaumgartner patrick.baumgartner [at] swiftmind [dot] com http://www.swiftmind.com @swiftmind