Fosdem 2011 - A Common Graph Database Access Layer for .Net and Mono
-
Upload
achim-friedland -
Category
Technology
-
view
5.059 -
download
0
Transcript of Fosdem 2011 - A Common Graph Database Access Layer for .Net and Mono
![Page 1: Fosdem 2011 - A Common Graph Database Access Layer for .Net and Mono](https://reader035.fdocuments.in/reader035/viewer/2022062303/55512ac4b4c905325d8b49a8/html5/thumbnails/1.jpg)
A Common Graph Database Access Layer
for .NET and Mono
Achim Friedland <[email protected]>, Aperis GmbH FOSDEM 2011 - Mono devroom
![Page 2: Fosdem 2011 - A Common Graph Database Access Layer for .Net and Mono](https://reader035.fdocuments.in/reader035/viewer/2022062303/55512ac4b4c905325d8b49a8/html5/thumbnails/2.jpg)
Graphs
2
Any structure having vertices and edges...
![Page 3: Fosdem 2011 - A Common Graph Database Access Layer for .Net and Mono](https://reader035.fdocuments.in/reader035/viewer/2022062303/55512ac4b4c905325d8b49a8/html5/thumbnails/3.jpg)
...but different representations possible.
3
Adjacency matrix vs. Incidence matrix vs. Adjacency list vs. Edge list vs. Classes, Index-based vs. Index-free Adjacency, Dense vs. Sparse graphs, On-disc vs. In-memory graphs...
![Page 4: Fosdem 2011 - A Common Graph Database Access Layer for .Net and Mono](https://reader035.fdocuments.in/reader035/viewer/2022062303/55512ac4b4c905325d8b49a8/html5/thumbnails/4.jpg)
The problem...
• Different graph representations...• have different levels of expressivity• can be very application specific• hard to optimize a single one for every
use-case• Multiple APIs from different vendors
4
![Page 5: Fosdem 2011 - A Common Graph Database Access Layer for .Net and Mono](https://reader035.fdocuments.in/reader035/viewer/2022062303/55512ac4b4c905325d8b49a8/html5/thumbnails/5.jpg)
5
So, where to find a common API?
![Page 6: Fosdem 2011 - A Common Graph Database Access Layer for .Net and Mono](https://reader035.fdocuments.in/reader035/viewer/2022062303/55512ac4b4c905325d8b49a8/html5/thumbnails/6.jpg)
Graph Commons
• Edges are first-class citizens• Information gathering by traversing
Indices are less important than in relational DBs
• Index-free adjacency, so the cost of traversing an edge is in-depended from the size of the graph( ...at least when you ignore the GC ;)
6
![Page 7: Fosdem 2011 - A Common Graph Database Access Layer for .Net and Mono](https://reader035.fdocuments.in/reader035/viewer/2022062303/55512ac4b4c905325d8b49a8/html5/thumbnails/7.jpg)
The Property-Graph Model
• directed:• attributed:• edge-labeled:• multi-graph:
The most common graph model withinthe NoSQL GraphDB space
Each edge has a source and destination vertexVertices and edges carry key/value pairsThe label denotes the type of relationshipMultiple edges between any two vertices allowed
7
Id: 1name: Alice
age: 21
vertex properties
Id: 2name: Bob
age: 23since: 2009/09/21
edge properties
Friends
edge label
![Page 8: Fosdem 2011 - A Common Graph Database Access Layer for .Net and Mono](https://reader035.fdocuments.in/reader035/viewer/2022062303/55512ac4b4c905325d8b49a8/html5/thumbnails/8.jpg)
8
// Use a class-based in-memory graphvar graph = new InMemoryGraph();
var v1 = graph.AddVertex(new VertexId(1));var v2 = graph.AddVertex(new VertexId(2));v1.SetProperty("name", "Alice");v1.SetProperty("age" , 21);v2.SetProperty("name", "Bob");v2.SetProperty("age" , 23);
var e1 = graph.AddEdge(v1, v2, new EdgeId(1), "Friends");e1.SetProperty(“since”, ”2009/09/21”);
A Property Graph Model Interface for .NET and Mono
![Page 9: Fosdem 2011 - A Common Graph Database Access Layer for .Net and Mono](https://reader035.fdocuments.in/reader035/viewer/2022062303/55512ac4b4c905325d8b49a8/html5/thumbnails/9.jpg)
9
Or, if you prefer the Dynamic Language Runtime...
// Use a class-based in-memory graphvar graph = new InMemoryGraph();
var v1 = graph.AddVertex().AsDynamic();var v2 = graph.AddVertex().AsDynamic();v1.name = "Alice";v1.age = 21;v2.name = "Bob";v2.age = 23;
var e1 = graph.AddEdge(v1, v2, "Friends").AsDynamic();e1.since = ”2009/09/21”;
![Page 10: Fosdem 2011 - A Common Graph Database Access Layer for .Net and Mono](https://reader035.fdocuments.in/reader035/viewer/2022062303/55512ac4b4c905325d8b49a8/html5/thumbnails/10.jpg)
10
• About 70% of the current JAVA original• e.g. Transactions, Auto-Indexing still missing• Expect first release within the next 4-6 weeks
Current status...
![Page 11: Fosdem 2011 - A Common Graph Database Access Layer for .Net and Mono](https://reader035.fdocuments.in/reader035/viewer/2022062303/55512ac4b4c905325d8b49a8/html5/thumbnails/11.jpg)
11
• Different implementations for different graph representations
• Implementations for remote graphs(Rexster and Neo4J REST in separate projects)
• Extensions, e.g. for semantic graphs
The future...
![Page 12: Fosdem 2011 - A Common Graph Database Access Layer for .Net and Mono](https://reader035.fdocuments.in/reader035/viewer/2022062303/55512ac4b4c905325d8b49a8/html5/thumbnails/12.jpg)
12
Ok, but how to query a property-graph?
![Page 13: Fosdem 2011 - A Common Graph Database Access Layer for .Net and Mono](https://reader035.fdocuments.in/reader035/viewer/2022062303/55512ac4b4c905325d8b49a8/html5/thumbnails/13.jpg)
13
A data flow framework for property graph models
AbstractPipe<S, E>S E
: IEnumerator<E>, IEnumerable<E>
Source Elements Emitted Elements
![Page 14: Fosdem 2011 - A Common Graph Database Access Layer for .Net and Mono](https://reader035.fdocuments.in/reader035/viewer/2022062303/55512ac4b4c905325d8b49a8/html5/thumbnails/14.jpg)
14
public class IdentityPipe<S> : AbstractPipe<S, S> {
public override Boolean MoveNext() {
if (_InternalEnumerator == null) return false;
if (_InternalEnumerator.MoveNext()) { _CurrentElement = _InternalEnumerator.Current; return true; }
return false;
}
}
A simple pipe yielding all objects without modification
![Page 15: Fosdem 2011 - A Common Graph Database Access Layer for .Net and Mono](https://reader035.fdocuments.in/reader035/viewer/2022062303/55512ac4b4c905325d8b49a8/html5/thumbnails/15.jpg)
15
public class ToUpperPipe<String> : AbstractPipe<String, String> {
public override Boolean MoveNext() {
if (_InternalEnumerator == null) return false;
if (_InternalEnumerator.MoveNext()) { _CurrentElement = _InternalEnumerator.Current.ToUpper(); return true; }
return false;
}
}
A simple pipe yielding all strings turned to uppercase
![Page 16: Fosdem 2011 - A Common Graph Database Access Layer for .Net and Mono](https://reader035.fdocuments.in/reader035/viewer/2022062303/55512ac4b4c905325d8b49a8/html5/thumbnails/16.jpg)
16
public class GetNameProperty : AbstractPipe<IVertex, String> {
public override Boolean MoveNext() {
if (_InternalEnumerator == null) return false;
if (_InternalEnumerator.MoveNext()) { _CurrentElement = _IVertex.GetProperty(“name”); return true; }
return false;
}
}
A simple pipe yielding the vertex property “name”
![Page 17: Fosdem 2011 - A Common Graph Database Access Layer for .Net and Mono](https://reader035.fdocuments.in/reader035/viewer/2022062303/55512ac4b4c905325d8b49a8/html5/thumbnails/17.jpg)
17
Allow the creation of state or side effects within a pipe
ISideEffectPipe<in S, out E, out T>S ESource
ElementsEmitted
ElementsTSide Effect
![Page 18: Fosdem 2011 - A Common Graph Database Access Layer for .Net and Mono](https://reader035.fdocuments.in/reader035/viewer/2022062303/55512ac4b4c905325d8b49a8/html5/thumbnails/18.jpg)
18
public class CountPipe<S> : AbstractPipe<S,S>, ISideEffectPipe<S,S,Int64 { [...] public override Boolean MoveNext() {
if (_InternalEnumerator == null) return false;
if (_InternalEnumerator.MoveNext()) { _CurrentElement = _InternalEnumerator.Current; Interlocked.Increment(ref _Counter); return true; }
return false;
}
public Int64 SideEffect { get { return _Counter; }}
}
Count the total number of elements passed through the pipe
![Page 19: Fosdem 2011 - A Common Graph Database Access Layer for .Net and Mono](https://reader035.fdocuments.in/reader035/viewer/2022062303/55512ac4b4c905325d8b49a8/html5/thumbnails/19.jpg)
Pipeline<S, E>
19
pipe1<S,A> pipe2<B,C> pipe3<C,E>
SSource
Elements
EEmitted
Elements
Create complex pipes by combining pipes to pipelines
![Page 20: Fosdem 2011 - A Common Graph Database Access Layer for .Net and Mono](https://reader035.fdocuments.in/reader035/viewer/2022062303/55512ac4b4c905325d8b49a8/html5/thumbnails/20.jpg)
20
// Friends-of-a-friendvar pipe1 = new VertexEdgePipe(VertexEdgePipe.Step.OUT_EDGES);var pipe2 = new LabelFilterPipe("Friends", ComparisonFilter.EQUALS);var pipe3 = new EdgeVertexPipe(EdgeVertexPipe.Step.IN_VERTEX);var pipe4 = new VertexEdgePipe(VertexEdgePipe.Step.OUT_EDGES);var pipe5 = new LabelFilterPipe("Friends", ComparisonFilter.EQUALS);var pipe6 = new EdgeVertexPipe(EdgeVertexPipe.Step.IN_VERTEX);var pipe7 = new PropertyPipe("name");
var pipeline = new Pipeline(pipe1,pipe2,pipe3,pipe4,pipe5,pipe6,pipe7);pipeline.SetSource(new SingleEnumerator( graph.GetVertex(new VertexId(1))));
foreach (var _foaf in pipeline.Take(5)) { Console.WriteLine(_foaf);}
A data flow framework for property graph models
![Page 21: Fosdem 2011 - A Common Graph Database Access Layer for .Net and Mono](https://reader035.fdocuments.in/reader035/viewer/2022062303/55512ac4b4c905325d8b49a8/html5/thumbnails/21.jpg)
21
• Think of “LINQ for graphs”• ...but without the syntactic sugar. • Very Powerful, but also very “noisy” :(
A data flow framework for property graph models
![Page 22: Fosdem 2011 - A Common Graph Database Access Layer for .Net and Mono](https://reader035.fdocuments.in/reader035/viewer/2022062303/55512ac4b4c905325d8b49a8/html5/thumbnails/22.jpg)
22
Current status & the future
• About 95% of the current JAVA original• Expect first release with the next weeks
![Page 23: Fosdem 2011 - A Common Graph Database Access Layer for .Net and Mono](https://reader035.fdocuments.in/reader035/viewer/2022062303/55512ac4b4c905325d8b49a8/html5/thumbnails/23.jpg)
23
And what about ad hoc querying for exploring a property-graph?
![Page 24: Fosdem 2011 - A Common Graph Database Access Layer for .Net and Mono](https://reader035.fdocuments.in/reader035/viewer/2022062303/55512ac4b4c905325d8b49a8/html5/thumbnails/24.jpg)
24
• Ad Hoc querying a property graph• User-friendly wrapper around pipes• “perl for graphs” ;)
Ad hoc Query Language for graphs
![Page 25: Fosdem 2011 - A Common Graph Database Access Layer for .Net and Mono](https://reader035.fdocuments.in/reader035/viewer/2022062303/55512ac4b4c905325d8b49a8/html5/thumbnails/25.jpg)
25
// Friends-of-a-friendvar pipe1 = new VertexEdgePipe(VertexEdgePipe.Step.OUT_EDGES);var pipe2 = new LabelFilterPipe("Friends", ComparisonFilter.EQUALS);var pipe3 = new EdgeVertexPipe(EdgeVertexPipe.Step.IN_VERTEX);var pipe4 = new VertexEdgePipe(VertexEdgePipe.Step.OUT_EDGES);var pipe5 = new LabelFilterPipe("Friends", ComparisonFilter.EQUALS);var pipe6 = new EdgeVertexPipe(EdgeVertexPipe.Step.IN_VERTEX);var pipe7 = new PropertyPipe("name");
var pipeline = new Pipeline(pipe1,pipe2,pipe3,pipe4,pipe5,pipe6,pipe7);pipeline.SetSource(new SingleEnumerator( graph.GetVertex(new VertexId(1))));
g:id-v(1)/outE[@label='Friends']/inV/outE[@label='Friends']/inV/@name
Ad hoc Query Language for graphs
![Page 26: Fosdem 2011 - A Common Graph Database Access Layer for .Net and Mono](https://reader035.fdocuments.in/reader035/viewer/2022062303/55512ac4b4c905325d8b49a8/html5/thumbnails/26.jpg)
26
Current status & the future
• Reimplementation not yet started :(• Will probably be based on the DLR• Expect first release with the next 2-3 months
![Page 27: Fosdem 2011 - A Common Graph Database Access Layer for .Net and Mono](https://reader035.fdocuments.in/reader035/viewer/2022062303/55512ac4b4c905325d8b49a8/html5/thumbnails/27.jpg)
27
But...
• Do not expect Gremlin as the one-and-only query language for each use-case!
• More application specific Query Languages could be implemented using pipese.g. sones GQL, OrientDB SQL
![Page 28: Fosdem 2011 - A Common Graph Database Access Layer for .Net and Mono](https://reader035.fdocuments.in/reader035/viewer/2022062303/55512ac4b4c905325d8b49a8/html5/thumbnails/28.jpg)
28
How to access a remote graph?How to expose a graph via the network?
![Page 29: Fosdem 2011 - A Common Graph Database Access Layer for .Net and Mono](https://reader035.fdocuments.in/reader035/viewer/2022062303/55512ac4b4c905325d8b49a8/html5/thumbnails/29.jpg)
29
• Exposes your application-embedded Blueprints graph via HTTP/REST
• Vertices and edges are REST resources• The rexster client library allows you to access most
JAVA-based GraphDBsNeo4J, OrientDB available; DEX, InfiniteGraph coming soon...
A HTTP/REST interface forBlueprints Property Graph Models
![Page 30: Fosdem 2011 - A Common Graph Database Access Layer for .Net and Mono](https://reader035.fdocuments.in/reader035/viewer/2022062303/55512ac4b4c905325d8b49a8/html5/thumbnails/30.jpg)
30
Common CRUD Operations...
![Page 31: Fosdem 2011 - A Common Graph Database Access Layer for .Net and Mono](https://reader035.fdocuments.in/reader035/viewer/2022062303/55512ac4b4c905325d8b49a8/html5/thumbnails/31.jpg)
31
Common CRUD Operations...
![Page 32: Fosdem 2011 - A Common Graph Database Access Layer for .Net and Mono](https://reader035.fdocuments.in/reader035/viewer/2022062303/55512ac4b4c905325d8b49a8/html5/thumbnails/32.jpg)
32
Common CRUD Operations...
![Page 33: Fosdem 2011 - A Common Graph Database Access Layer for .Net and Mono](https://reader035.fdocuments.in/reader035/viewer/2022062303/55512ac4b4c905325d8b49a8/html5/thumbnails/33.jpg)
33
Default resource representation: JSON
curl -H Accept:application/json http://localhost:8182/graph1/vertices/1{ "version" : "0.1", "results" : { "_type" : "vertex", "_id" : "1", "name" : "Alice", "age" : 21 }, "query_time" : 0.014235 }
![Page 34: Fosdem 2011 - A Common Graph Database Access Layer for .Net and Mono](https://reader035.fdocuments.in/reader035/viewer/2022062303/55512ac4b4c905325d8b49a8/html5/thumbnails/34.jpg)
34
Current status
• About 60% of the current JAVA original• Expect first release with the next 3-4 weeks• Rexster Shell might take some more time...
![Page 35: Fosdem 2011 - A Common Graph Database Access Layer for .Net and Mono](https://reader035.fdocuments.in/reader035/viewer/2022062303/55512ac4b4c905325d8b49a8/html5/thumbnails/35.jpg)
35
Future Resource Representations
• Link-aware, self-describing hypermedia (see Neo4J)
• e.g. ATOM, XML + XLINK, RDFa
• Application specific protocols• GEXF for exporting a graph to GEPHI
• GraphML/GDF/... graph formats
• MediaRSS your photo graph traversal via Cooliris
![Page 36: Fosdem 2011 - A Common Graph Database Access Layer for .Net and Mono](https://reader035.fdocuments.in/reader035/viewer/2022062303/55512ac4b4c905325d8b49a8/html5/thumbnails/36.jpg)
36
The GraphDB Graph...
Neo4J for GIS
InfoGrid for WebApps In-Memory for Caching
OrientDB for Documents
OrientDB for Ad Hoc
ThinkerGraph & Gremlin for Ad Hoc
Neo4J for HA
InfiniteGraph for Clustering
![Page 37: Fosdem 2011 - A Common Graph Database Access Layer for .Net and Mono](https://reader035.fdocuments.in/reader035/viewer/2022062303/55512ac4b4c905325d8b49a8/html5/thumbnails/37.jpg)
37
FOSDEM 2011 GraphDB dinner!Meet us in front of Le Roy d’EspagneGrand Place 1, Brussels20:00 PM
Still hungry for more?
![Page 38: Fosdem 2011 - A Common Graph Database Access Layer for .Net and Mono](https://reader035.fdocuments.in/reader035/viewer/2022062303/55512ac4b4c905325d8b49a8/html5/thumbnails/38.jpg)
38
Questions?
http://www.graph-database.orghttp://www.twitter.com/graphdbs