DeveelDB in the Cloud€¦ · DeveelDB in the Cloud (CloudB) FOSDEM 2011 Mono Dev Room Bruxelles, 5...
Transcript of DeveelDB in the Cloud€¦ · DeveelDB in the Cloud (CloudB) FOSDEM 2011 Mono Dev Room Bruxelles, 5...
![Page 1: DeveelDB in the Cloud€¦ · DeveelDB in the Cloud (CloudB) FOSDEM 2011 Mono Dev Room Bruxelles, 5 February Presented by Antonello Provenzano (antonell@deveel.com)](https://reader034.fdocuments.in/reader034/viewer/2022051607/6039c58fd3bd5a2202175ae5/html5/thumbnails/1.jpg)
DeveelDB in the Cloud (CloudB)
FOSDEM 2011Mono Dev Room
Bruxelles, 5 February
Presented byAntonello Provenzano ([email protected])
![Page 2: DeveelDB in the Cloud€¦ · DeveelDB in the Cloud (CloudB) FOSDEM 2011 Mono Dev Room Bruxelles, 5 February Presented by Antonello Provenzano (antonell@deveel.com)](https://reader034.fdocuments.in/reader034/viewer/2022051607/6039c58fd3bd5a2202175ae5/html5/thumbnails/2.jpg)
The Presentation
1. Background
2. Solution
3. Architecture
4. Community
5. Future Developments
6. Questions
![Page 3: DeveelDB in the Cloud€¦ · DeveelDB in the Cloud (CloudB) FOSDEM 2011 Mono Dev Room Bruxelles, 5 February Presented by Antonello Provenzano (antonell@deveel.com)](https://reader034.fdocuments.in/reader034/viewer/2022051607/6039c58fd3bd5a2202175ae5/html5/thumbnails/3.jpg)
The Background.
![Page 4: DeveelDB in the Cloud€¦ · DeveelDB in the Cloud (CloudB) FOSDEM 2011 Mono Dev Room Bruxelles, 5 February Presented by Antonello Provenzano (antonell@deveel.com)](https://reader034.fdocuments.in/reader034/viewer/2022051607/6039c58fd3bd5a2202175ae5/html5/thumbnails/4.jpg)
The Main Players
● Relational Databases● Structured (data modeling)● Consistent (data integrity checks)● Standard Syntax (SQL – PL/SQL, T-SQL)
● NoSQL (Mongo, Couch, BigTable, Cassandra)● Fast (distributed computation)● Scalable (networked storage)● Open Protocols (HTTP, JSON, XML, etc.)
![Page 5: DeveelDB in the Cloud€¦ · DeveelDB in the Cloud (CloudB) FOSDEM 2011 Mono Dev Room Bruxelles, 5 February Presented by Antonello Provenzano (antonell@deveel.com)](https://reader034.fdocuments.in/reader034/viewer/2022051607/6039c58fd3bd5a2202175ae5/html5/thumbnails/5.jpg)
What Else?
● Parallel Databases● Distributed Storage● Distributed Computation
● Resource Sharing● Shared Memory● Shared Disks● Shared Nothing
![Page 6: DeveelDB in the Cloud€¦ · DeveelDB in the Cloud (CloudB) FOSDEM 2011 Mono Dev Room Bruxelles, 5 February Presented by Antonello Provenzano (antonell@deveel.com)](https://reader034.fdocuments.in/reader034/viewer/2022051607/6039c58fd3bd5a2202175ae5/html5/thumbnails/6.jpg)
The Solution.
![Page 7: DeveelDB in the Cloud€¦ · DeveelDB in the Cloud (CloudB) FOSDEM 2011 Mono Dev Room Bruxelles, 5 February Presented by Antonello Provenzano (antonell@deveel.com)](https://reader034.fdocuments.in/reader034/viewer/2022051607/6039c58fd3bd5a2202175ae5/html5/thumbnails/7.jpg)
Model Agnostic
● Pluggable● No default data model● Data Process API (Path Functions)
– Delegated data processing– Consensus on transaction commits
● Multiple Path Types– In the same network
![Page 8: DeveelDB in the Cloud€¦ · DeveelDB in the Cloud (CloudB) FOSDEM 2011 Mono Dev Room Bruxelles, 5 February Presented by Antonello Provenzano (antonell@deveel.com)](https://reader034.fdocuments.in/reader034/viewer/2022051607/6039c58fd3bd5a2202175ae5/html5/thumbnails/8.jpg)
Scalable
● Master Nodes● Handle the network● Entry point for clients
● Root Nodes● Handle Paths● Use and share blocks
● Block Nodes● Manage data storage and processing
![Page 9: DeveelDB in the Cloud€¦ · DeveelDB in the Cloud (CloudB) FOSDEM 2011 Mono Dev Room Bruxelles, 5 February Presented by Antonello Provenzano (antonell@deveel.com)](https://reader034.fdocuments.in/reader034/viewer/2022051607/6039c58fd3bd5a2202175ae5/html5/thumbnails/9.jpg)
A Simple Network
![Page 10: DeveelDB in the Cloud€¦ · DeveelDB in the Cloud (CloudB) FOSDEM 2011 Mono Dev Room Bruxelles, 5 February Presented by Antonello Provenzano (antonell@deveel.com)](https://reader034.fdocuments.in/reader034/viewer/2022051607/6039c58fd3bd5a2202175ae5/html5/thumbnails/10.jpg)
Paths
● Define the data model● Process data at commits
● Green light to data storage (consensus)● Proprietary implementation
● Share memory● Path roots handle block data portions
![Page 11: DeveelDB in the Cloud€¦ · DeveelDB in the Cloud (CloudB) FOSDEM 2011 Mono Dev Room Bruxelles, 5 February Presented by Antonello Provenzano (antonell@deveel.com)](https://reader034.fdocuments.in/reader034/viewer/2022051607/6039c58fd3bd5a2202175ae5/html5/thumbnails/11.jpg)
Paths (An Example)
![Page 12: DeveelDB in the Cloud€¦ · DeveelDB in the Cloud (CloudB) FOSDEM 2011 Mono Dev Room Bruxelles, 5 February Presented by Antonello Provenzano (antonell@deveel.com)](https://reader034.fdocuments.in/reader034/viewer/2022051607/6039c58fd3bd5a2202175ae5/html5/thumbnails/12.jpg)
Transactional
● Optimistic Concurrency Control● Mandatory● Consensus by Paths
● Isolated Snapshots● Historical Snapshots Accessible● Restoreable
● Versioned data● Each commit creates a new branch
![Page 13: DeveelDB in the Cloud€¦ · DeveelDB in the Cloud (CloudB) FOSDEM 2011 Mono Dev Room Bruxelles, 5 February Presented by Antonello Provenzano (antonell@deveel.com)](https://reader034.fdocuments.in/reader034/viewer/2022051607/6039c58fd3bd5a2202175ae5/html5/thumbnails/13.jpg)
Commit (Success)
![Page 14: DeveelDB in the Cloud€¦ · DeveelDB in the Cloud (CloudB) FOSDEM 2011 Mono Dev Room Bruxelles, 5 February Presented by Antonello Provenzano (antonell@deveel.com)](https://reader034.fdocuments.in/reader034/viewer/2022051607/6039c58fd3bd5a2202175ae5/html5/thumbnails/14.jpg)
Commit (Failed)
![Page 15: DeveelDB in the Cloud€¦ · DeveelDB in the Cloud (CloudB) FOSDEM 2011 Mono Dev Room Bruxelles, 5 February Presented by Antonello Provenzano (antonell@deveel.com)](https://reader034.fdocuments.in/reader034/viewer/2022051607/6039c58fd3bd5a2202175ae5/html5/thumbnails/15.jpg)
Version Access
![Page 16: DeveelDB in the Cloud€¦ · DeveelDB in the Cloud (CloudB) FOSDEM 2011 Mono Dev Room Bruxelles, 5 February Presented by Antonello Provenzano (antonell@deveel.com)](https://reader034.fdocuments.in/reader034/viewer/2022051607/6039c58fd3bd5a2202175ae5/html5/thumbnails/16.jpg)
Setup a Network (cadmin / TCP/IP)
cloudb> start role manager on 192.168.0.32:3500cloudb> start role root on 192.168.0.33:3500cloudb> start role block on 192.168.0.33:3500cloudb> start role block on 192.168.0.34:3500cloudb> start role block on 192.168.0.32:3550
Starting a CloudB Network
Starting a Path
cloudb> add base path testdb to 192.168.0.33:3500cloudb> add path Deveel.Data.BasePath to 192.168.0.33:3500
Starting a Node of the Networkantonello@deveel~: mono mnode -port 3500
![Page 17: DeveelDB in the Cloud€¦ · DeveelDB in the Cloud (CloudB) FOSDEM 2011 Mono Dev Room Bruxelles, 5 February Presented by Antonello Provenzano (antonell@deveel.com)](https://reader034.fdocuments.in/reader034/viewer/2022051607/6039c58fd3bd5a2202175ae5/html5/thumbnails/17.jpg)
Programmatic Access (CloudB)
![Page 18: DeveelDB in the Cloud€¦ · DeveelDB in the Cloud (CloudB) FOSDEM 2011 Mono Dev Room Bruxelles, 5 February Presented by Antonello Provenzano (antonell@deveel.com)](https://reader034.fdocuments.in/reader034/viewer/2022051607/6039c58fd3bd5a2202175ae5/html5/thumbnails/18.jpg)
Software Architecture
![Page 19: DeveelDB in the Cloud€¦ · DeveelDB in the Cloud (CloudB) FOSDEM 2011 Mono Dev Room Bruxelles, 5 February Presented by Antonello Provenzano (antonell@deveel.com)](https://reader034.fdocuments.in/reader034/viewer/2022051607/6039c58fd3bd5a2202175ae5/html5/thumbnails/19.jpg)
Modular & Flexible
● Kernel Library● Data Management● Network● Configuration, Diagnostics
● Path Libraries● CloudB|ase (string key/value, binary files)● SQL (experimental)● CloudbObj (planned)
![Page 20: DeveelDB in the Cloud€¦ · DeveelDB in the Cloud (CloudB) FOSDEM 2011 Mono Dev Room Bruxelles, 5 February Presented by Antonello Provenzano (antonell@deveel.com)](https://reader034.fdocuments.in/reader034/viewer/2022051607/6039c58fd3bd5a2202175ae5/html5/thumbnails/20.jpg)
DataFile Class
● Spans data across the network● Can be obtained by Transaction● Streamable
![Page 21: DeveelDB in the Cloud€¦ · DeveelDB in the Cloud (CloudB) FOSDEM 2011 Mono Dev Room Bruxelles, 5 February Presented by Antonello Provenzano (antonell@deveel.com)](https://reader034.fdocuments.in/reader034/viewer/2022051607/6039c58fd3bd5a2202175ae5/html5/thumbnails/21.jpg)
Component Stack
![Page 22: DeveelDB in the Cloud€¦ · DeveelDB in the Cloud (CloudB) FOSDEM 2011 Mono Dev Room Bruxelles, 5 February Presented by Antonello Provenzano (antonell@deveel.com)](https://reader034.fdocuments.in/reader034/viewer/2022051607/6039c58fd3bd5a2202175ae5/html5/thumbnails/22.jpg)
Applications
● mnode (machine node)● Standalone Console Server● Daemon (experimental)● Exposes a machine to the network
● cadmin (cloud administration tool)● Console Application (REPL)● Interacts with Admin Service of a Node
![Page 23: DeveelDB in the Cloud€¦ · DeveelDB in the Cloud (CloudB) FOSDEM 2011 Mono Dev Room Bruxelles, 5 February Presented by Antonello Provenzano (antonell@deveel.com)](https://reader034.fdocuments.in/reader034/viewer/2022051607/6039c58fd3bd5a2202175ae5/html5/thumbnails/23.jpg)
The Community
![Page 24: DeveelDB in the Cloud€¦ · DeveelDB in the Cloud (CloudB) FOSDEM 2011 Mono Dev Room Bruxelles, 5 February Presented by Antonello Provenzano (antonell@deveel.com)](https://reader034.fdocuments.in/reader034/viewer/2022051607/6039c58fd3bd5a2202175ae5/html5/thumbnails/24.jpg)
Mono
● Entirely Written in C# Language● Profile .NET 2.0● Compiled with Nant (isolated environment)● Core library only needs corlib
● Optimized For Mono Framework
![Page 25: DeveelDB in the Cloud€¦ · DeveelDB in the Cloud (CloudB) FOSDEM 2011 Mono Dev Room Bruxelles, 5 February Presented by Antonello Provenzano (antonell@deveel.com)](https://reader034.fdocuments.in/reader034/viewer/2022051607/6039c58fd3bd5a2202175ae5/html5/thumbnails/25.jpg)
Open Source
● Kernel Library● GNU LGPLv3
● Command-Line Tools● Apache Public License 2.0
![Page 26: DeveelDB in the Cloud€¦ · DeveelDB in the Cloud (CloudB) FOSDEM 2011 Mono Dev Room Bruxelles, 5 February Presented by Antonello Provenzano (antonell@deveel.com)](https://reader034.fdocuments.in/reader034/viewer/2022051607/6039c58fd3bd5a2202175ae5/html5/thumbnails/26.jpg)
The Future
![Page 27: DeveelDB in the Cloud€¦ · DeveelDB in the Cloud (CloudB) FOSDEM 2011 Mono Dev Room Bruxelles, 5 February Presented by Antonello Provenzano (antonell@deveel.com)](https://reader034.fdocuments.in/reader034/viewer/2022051607/6039c58fd3bd5a2202175ae5/html5/thumbnails/27.jpg)
Version 1.0
● Node Services (Admin, Master, Root, Block)● Configuration● Diagnostics● Communication Protocols (HTTP, TCP/IP)● Messaging Formats (XML, Binary, JSON)● CloudBase
![Page 28: DeveelDB in the Cloud€¦ · DeveelDB in the Cloud (CloudB) FOSDEM 2011 Mono Dev Room Bruxelles, 5 February Presented by Antonello Provenzano (antonell@deveel.com)](https://reader034.fdocuments.in/reader034/viewer/2022051607/6039c58fd3bd5a2202175ae5/html5/thumbnails/28.jpg)
Version 1.5
● Kernel● 128-bit Addresses● Multiple Roots per Path● Message Queue
● Path Libraries● Full SQL Support● OODB● Graph
![Page 29: DeveelDB in the Cloud€¦ · DeveelDB in the Cloud (CloudB) FOSDEM 2011 Mono Dev Room Bruxelles, 5 February Presented by Antonello Provenzano (antonell@deveel.com)](https://reader034.fdocuments.in/reader034/viewer/2022051607/6039c58fd3bd5a2202175ae5/html5/thumbnails/29.jpg)
Some References
Website
http://code.google.com/cloudb
http://db.deveel.com
http://antonello.provenza.no
![Page 30: DeveelDB in the Cloud€¦ · DeveelDB in the Cloud (CloudB) FOSDEM 2011 Mono Dev Room Bruxelles, 5 February Presented by Antonello Provenzano (antonell@deveel.com)](https://reader034.fdocuments.in/reader034/viewer/2022051607/6039c58fd3bd5a2202175ae5/html5/thumbnails/30.jpg)
Fin