Neo4j Introduction (for Techies)

Click here to load reader

  • date post

    26-Jan-2015
  • Category

    Technology

  • view

    112
  • download

    1

Embed Size (px)

description

The social graph of Facebook is the most popular application for a graph database. In addition, there are far more exciting applications, such as spatial data, financial trail, indexing, and others. If you combine different graphs, you are able to evaluate those together with the algorithms known from the graph theory. As a graph, a domain can often be easier and more natural designed. This talk introduces the topic of graph databases and shows how to implement mediated models with large, complex and highly connected data with Neo4j. Subsequently, topics like querying, indexing, import / export are considered as well.

Transcript of Neo4j Introduction (for Techies)

  • 1. Neo4j from the Command Line Perspective (Introduction) JUG MK | Skopje | 02.10.2013

2. 2 Speaker Profile Patrick Baumgartner Senior Software Consultant | Partner @ Swiftmind Spring Framework, OSGi, Neo4j Spring Trainer, Neo4j Fanboy, Agilista Co-Author of OSGi fr Praktiker Co-Organizer of Neo4j Meetup Zurich @patbaumgartner 3. 3 Swiftmind Your experts for Enterprise Java Areas of expertise Java EE Spring Framework OSGi Agile Methodologies Software Engineering Best Practices Headquarter Zrich, Schweiz @swiftmind http://www.swiftmind.com 4. 4 Agenda Introduction to Graphs Use cases Neo4j Querying Import / Export Tools und APIs Polyglot Persistence Q & A 5. 5 New Demands on Data Access Structured and unstructured data Massive amounts of data Inexpensive horizontal scaling Apps and data in the cloud Social network features 6. 6 New Types of Data Stores 7. 7 NoSQL Data Volume Data Complexity Key-Value Stores Column Family Document Databases Graph Databases Relational Databases 90% of all use cases 8. 8 Property Graph Model Graphs are made of Vertices / Nodes Edges / Relationships Properties Alice Bob Car name:Bob age:63 sex:male name:Alice age:21 sex:female type:car vendor:tesla type:OWNS type:DRIVES 9. 9 Where is my graph? Use cases 10. 10 Social Data Alice Carol Luke Bob Sam Pete KNOWS KNOWS KNOWS KNOWS 11. 11 Is it the only use case? Social Graph 12. 12 Spatial Data 1 2 5 3 13 8 ROAD ROAD ROAD type:ROADlength:3km type:Hotel name:Radisson lat:47.452549 long:8.564615 type:Hotel name:Widder lat:47.373517 long:8.539252 13. 13 Social & Spatial Data Alice Mat 5 Bob 13 8 ROAD ROAD LIKES KNOWS type:Hotel name:Radisson Blu lat:47.452549 long:8.564615 14. 14 Financial Data Alice Hotel Luke Bank ATM Pete WITHDRAW TRANSFER OWNS 15. 15 Other graph use cases? 16. 16 Railroad in Switzerland 17. 17 Your Business Domain Process KPI Device Activity Function Service CONTAINS DEPENDS 18. 18 Neo4j The Big Picture 19. 19 Neo4j Big Picture Most used graph data base Robust & Mature: in production 24/7 since 2003 Community: Ecosystem with Tools, Bindings, Frameworks Licenses AGPLv3 Community Edition OpenSource Advanced/Enterprise for commercial applications Neo Technology Development & Support ~ 50 Persons / 6 Countries / 3 Continents 20. 20 Features Object oriented, flexible network structure ACID transactions Horizontal scalable Java API Java, Groovy, Scala, JRuby Runtime Standalone Server Embedded Database Disk based storage manager Plugin and testing support 21. 21 How do I find my data? Querying 22. 22 Graph Querying How do I query the right nodes from the graph? Tell me all friends of friends of friends who liked the movie XYZ with more than 3 stars ... Querying methods Traversing with Neo4j API Traversal descriptions Graph algorithms Index queries Cypher queries 23. 23 Neo4j API Node bob = ...; for ((Relationship rel: bob.getRelationships(RelTypes.KNOWS)) { Node friend = rel.getEndNode(); System.out.println(friend.getProperty("name")); } Simple, but very low level API To verbose for complex traversals 24. 24 Traversal API Querying API for graphs Describes paths trough the graph Call back based Fluent API Several predefined constructs Neo4j implements the following graph algorithms: A* Dijkstra pathsWithLength shortestPath allSimplePaths allPaths 25. 25 Traversal API Constructs 26. 26 Example: Traversal API TraversalDescription directFriends = Traversal.description() .breadthFirst() .relationships(RelTypes.KNOWS) .evaluator(Evaluators.toDepth(1)) .evaluator(Evaluators.excludeStartPosition()); for(Path p : directFriends.traverse(bob)) { System.out.println(p.endNode().getProperty(firstname)); } 27. 27 Index Queries Support for node and relationship indices Lucene as default implementation node id property value 15 name Alice 15 yearOfBirth 1972 16 name Bob 46 name Carol 46 yearOfBirth 1983 28. 28 Cypher Query Language Graph query language similar to SQL Declarative language Defines What and not How Uses pattern matching Support for Filters Pagination Supports CRUD functionality 29. 29 Cypher Query Language Elemente Element Description START Start element(s) (index- or id-lookup) MATCH Pattern to find nodes, describes paths (a) [knows]-> (b) WHERE Result filter (boolean expression) SKIP LIMIT Pagination RETURN Defines return elements ORDER BY Sorting by properties PARAMETERS Parameter-Map, die im Cypher mittels Key oder Position verwendet werden kann CREATE Creates node nodes and relations SET Updates properties on nodes and relations DELETE Deletes unused nodes or relationships 30. 30 Example: Cypher Query String query = "START person=node(12) + MATCH person-[:KNOWS]-friend + RETURN friend; ExecutionEngine engine = new ExecutionEngine(graphDb); ExecutionResult result = engine.execute(query); Iterator column = result.columnAs("friend"); while(column.hasNext()) { Node friendNode = (Node)column.next(); System.out.println(friendNode.getProperty(PROP_FIRSTNAME)); } person friend KNOWS 31. 31 Example: Cypher Create Query CREATE (carol {name:Carol}) return carol; +------------------------+ | carol | +------------------------+ | Node[1]{name:carol} | +------------------------+ START carol=node(1) CREATE (bob {name:Bob"}), (bob)-[:knows]->(carol) return carol; START carol=node(1) CREATE (apple {name:Apple,type:Fruit}), (carol)-[:owns {number: 5}]->(apple); Carol Bob KNOWS Carol Carol Apple type: Fruit OWNS number: 5 32. 32 Shipping data Import / Export Node Manipulation 33. 33 Node Creation with Java org.neo4j.graphdb.GraphDatabaseService graphDb = new GraphDatabaseFactory().newEmbeddedDatabase(DB_PATH); Transaction tx = graphDb.beginTx(); try { Node peter = createNode(graphDb, Alice"); Node carol = createNode(graphDb, Carol"); peter.createRelationshipTo(carol, RelTypes.KNOWS); .. tx.success(); } finally {tx.finish();} public Node createNode(GraphDatabaseService graphDb, String firstname) { Node person = graphDb.createNode(); person.setProperty(firstname, firstname); return person; } Alice Carol KNOWS 34. 34 NeoClipse 35. 35 Spring Data Neo4j (SDN) POJOs mapped as nodes or relationships type safe Works directly Database, typically embedded mode Data is fetched very fast and lazy Stores everything within a @Transaction Uses heavily AspectJ magic to enhance the POJOs 36. 36 Spring Data Neo4j Entity _type_: com.example.Person name: "Alice" age: 42 @NodeEntity public class Person { private String name; private int age; // getters/setters } Person alice = new Person(); alice.setName("Alice"); alice.setAge(42); alice.persist(); Node 37. 37 Spring Data Neo4j NodeEntity @NodeEntity public class Person { private String name; private int yearOfBirth; @RelatedTo(type = "KNOWS", direction = Direction.OUTGOING) private Set knownPersons; public void knows(Person p) { knownPersons.add(p); } public Set getFriends() { return knownPersons; } } Person alice = ...; alice.knows(bob); alice.knows(carol); Alice Bob KNOWS Carol KNOWS 38. 38 Spring Data Neo4j Relationship @RelationshipEntity public class Knows { private int sinceYear; public Knows since(int year) { this.sinceYear = year; return this; } } @NodeEntity public class Person { public Known knows(Person p) { return this.relateTo(p, Knows.class, "KNOWS"); } } Person alice = ...; Person bob ...; alice.knows(bob).since(2012); Alice Bob KNOWS since: 2012 39. 39 Spring Data Neo4j Repository public interface PersonRepository extends GraphRepository { @Query("start person = {0} match ... return ...") Iterable getOwnedServices(Person person); Iterable findByName(String name); Iterable findByNameLike(String name); } @Autowired PersonRepository personRepository; Person alice = personRepository.findByName("Alice"); 40. 40 Thinkerpop Blueprint Thinkerpop Blueprint Extensions GraphML Reader/Writer Library GraphSon Reader/Writer Library GML Reader/Writer Library Java Universal Network/ Graph Framework 41. 41 GraphML GraphML (http://graphml.graphdrawing.org/ ) Standardized format Tool support Pure and simple XML 42. 42 Neo4j Tools GraphML Gremlin Plugin (no further development) Thinkerpop Blueprint Extensions CSV BatchInserter ParallelBatchInserter 43. 43 Useful tools for your daily work Tools & APIs 44. 44 Tools & APIs (2) Web console Data overview Querying and visual graph presentation Command line Neo4j Shell Command line Sends UNIX-like Commands via RMI (cd, ls, pwd) Cypher Queries Support REST API JSON Clients for Java, .NET, PHP, Ruby, Neoclipse Visual presentation of the graph Filter options for partial graph presentation 45. 45 What?! Your app uses only one DB? Polyglot Persistence 46. 46 Polyglot Persistence Web shop application User Sessions Shopping Cart Recommendations Product Catalog Analytics Financial Data Redis Riak Neo4j MongoDB Casandra MySQL 47. 47 Neo4j on Meetup.com 48. 48 Q&A 49. 49 Thank you! Patrick Baumgartner, @patbaumgartner patrick.baumgartner [at] swiftmind [dot] com http://www.swiftmind.com @swiftmind