NOSQL – Not ONLY SQL - StartsteiteNoSQL – Not Only SQL – Why? Current popular problems in the...
Transcript of NOSQL – Not ONLY SQL - StartsteiteNoSQL – Not Only SQL – Why? Current popular problems in the...
![Page 1: NOSQL – Not ONLY SQL - StartsteiteNoSQL – Not Only SQL – Why? Current popular problems in the 24/7 world –How change a database schema without downtime? –How to change/delete](https://reader034.fdocuments.in/reader034/viewer/2022042309/5ed6ddac2aa02a23022b0009/html5/thumbnails/1.jpg)
Seite 1 Gunther Pippèrr © 2010 http://www.pipperr.de
NO:SQL – NOT ONLY SQL
NOSQL –
Only a Hype ? Or the trend for the next future ?
DOAG-Regionaltreffen München/Südbayern Donnerstag, 17. Februar 2011 um 17:00 Uhr
![Page 2: NOSQL – Not ONLY SQL - StartsteiteNoSQL – Not Only SQL – Why? Current popular problems in the 24/7 world –How change a database schema without downtime? –How to change/delete](https://reader034.fdocuments.in/reader034/viewer/2022042309/5ed6ddac2aa02a23022b0009/html5/thumbnails/2.jpg)
Seite 2 Gunther Pippèrr © 2010 http://www.pipperr.de
Agenda
1 Why?
2 ACID versus BASE – Horizontal versus vertical scalability
3 CAP Theorem
4 NoSQL – most important ideas
5 Some NoSQL DB examples
6 Oracle NoSQL features
7 Summary and discussion
Timeline: ~ 40 min + 5min discussion
![Page 3: NOSQL – Not ONLY SQL - StartsteiteNoSQL – Not Only SQL – Why? Current popular problems in the 24/7 world –How change a database schema without downtime? –How to change/delete](https://reader034.fdocuments.in/reader034/viewer/2022042309/5ed6ddac2aa02a23022b0009/html5/thumbnails/3.jpg)
Seite 3 Gunther Pippèrr © 2010 http://www.pipperr.de
The Big Battle?
NO:SQL
![Page 4: NOSQL – Not ONLY SQL - StartsteiteNoSQL – Not Only SQL – Why? Current popular problems in the 24/7 world –How change a database schema without downtime? –How to change/delete](https://reader034.fdocuments.in/reader034/viewer/2022042309/5ed6ddac2aa02a23022b0009/html5/thumbnails/4.jpg)
Seite 4 Gunther Pippèrr © 2010 http://www.pipperr.de
NoSQL Market Forecast 2011-2015
See: http://www.marketresearchmedia.com/2010/11/11/nosql-market/
….. According to Market Research Media’s latest forecasts, the worldwide NoSQL market is expected to reach $1.8 Billion by 2015 at a CAGR of 32% between 2011 and 2015. NoSQL market will generate $5.6 Billion revenues over the period 2011 – 2015 …...
![Page 5: NOSQL – Not ONLY SQL - StartsteiteNoSQL – Not Only SQL – Why? Current popular problems in the 24/7 world –How change a database schema without downtime? –How to change/delete](https://reader034.fdocuments.in/reader034/viewer/2022042309/5ed6ddac2aa02a23022b0009/html5/thumbnails/5.jpg)
Seite 5 Gunther Pippèrr © 2010 http://www.pipperr.de
NoSQL – Not Only SQL – Why?
Current popular problems in the database world
– Excessive growth of data
– Performance and scalability
Increasing hardware costs (especially the increasing higher
requirements of storage hardware)!
Unbelievable high license cost for strictly required features
for commercial high-volume database like EE partioning,
inMemory TimesTen
Consequences:
NoSQL„i
I„m cheaper!
![Page 6: NOSQL – Not ONLY SQL - StartsteiteNoSQL – Not Only SQL – Why? Current popular problems in the 24/7 world –How change a database schema without downtime? –How to change/delete](https://reader034.fdocuments.in/reader034/viewer/2022042309/5ed6ddac2aa02a23022b0009/html5/thumbnails/6.jpg)
Seite 6 Gunther Pippèrr © 2010 http://www.pipperr.de
NoSQL – Not Only SQL – Why?
Current popular problems in the programming world
– Lack of experience with SQL databases
– Unsolved problem of easy object to relational mapping
– Complex test automation for SQL statements
– Mindset to SQL => old-fashioned, to declarative
Unbelievable complex persistence layers to unlink java from
the database
Thousands of ideas to avoid the usage of SQL
Consequences:
Relationali
Lean programming!
![Page 7: NOSQL – Not ONLY SQL - StartsteiteNoSQL – Not Only SQL – Why? Current popular problems in the 24/7 world –How change a database schema without downtime? –How to change/delete](https://reader034.fdocuments.in/reader034/viewer/2022042309/5ed6ddac2aa02a23022b0009/html5/thumbnails/7.jpg)
Seite 7 Gunther Pippèrr © 2010 http://www.pipperr.de
No:SQL?
Schema
Rolle Rechte
The "no:sql(east)" conference 2009 in Atlanta
![Page 8: NOSQL – Not ONLY SQL - StartsteiteNoSQL – Not Only SQL – Why? Current popular problems in the 24/7 world –How change a database schema without downtime? –How to change/delete](https://reader034.fdocuments.in/reader034/viewer/2022042309/5ed6ddac2aa02a23022b0009/html5/thumbnails/8.jpg)
Seite 8 Gunther Pippèrr © 2010 http://www.pipperr.de
NoSQL – Not Only SQL – Why?
Current popular problems in the distributed server
world
– How we can grantee unlimited scalability?
– How to a upgrade to a new software version?
Unbelievable high license and hardware costs …….
Consequences:
NOSQL„i
I„m cheaper!
![Page 9: NOSQL – Not ONLY SQL - StartsteiteNoSQL – Not Only SQL – Why? Current popular problems in the 24/7 world –How change a database schema without downtime? –How to change/delete](https://reader034.fdocuments.in/reader034/viewer/2022042309/5ed6ddac2aa02a23022b0009/html5/thumbnails/9.jpg)
Seite 9 Gunther Pippèrr © 2010 http://www.pipperr.de
NoSQL – Not Only SQL – Why?
Current popular problems in the 24/7 world
– How change a database schema without downtime?
– How to change/delete a huge amount of values at runtime?
Again unbelievable high license and hardware costs…….
Consequences:
NOSQL„i
I„m easyer!
![Page 10: NOSQL – Not ONLY SQL - StartsteiteNoSQL – Not Only SQL – Why? Current popular problems in the 24/7 world –How change a database schema without downtime? –How to change/delete](https://reader034.fdocuments.in/reader034/viewer/2022042309/5ed6ddac2aa02a23022b0009/html5/thumbnails/10.jpg)
Seite 10 Gunther Pippèrr © 2010 http://www.pipperr.de
NOSQL – Definition
A pure NoSQL DB‟s is:
– non relational
– schema free
– from the scratch distributed multi-node architecture
• Goal: Shared nothing cluster solutions
– easy replication mechanisms
– Simple API ( Not only SQL = NoSQL)
– No ACID consistency model
– Open Source
BUT ! The big disadvantage at the moment
=> mostly no easy language to search
=> Complex Joins? Not implemented
![Page 11: NOSQL – Not ONLY SQL - StartsteiteNoSQL – Not Only SQL – Why? Current popular problems in the 24/7 world –How change a database schema without downtime? –How to change/delete](https://reader034.fdocuments.in/reader034/viewer/2022042309/5ed6ddac2aa02a23022b0009/html5/thumbnails/11.jpg)
Seite 11 Gunther Pippèrr © 2010 http://www.pipperr.de
NOSQL – Polyglot Persistence
Fight for free Database Software
Use the right database for your problem
– Not only the default one!
More effort to evaluate the requirement and search
after the right solution for your problem
![Page 12: NOSQL – Not ONLY SQL - StartsteiteNoSQL – Not Only SQL – Why? Current popular problems in the 24/7 world –How change a database schema without downtime? –How to change/delete](https://reader034.fdocuments.in/reader034/viewer/2022042309/5ed6ddac2aa02a23022b0009/html5/thumbnails/12.jpg)
Seite 12 Gunther Pippèrr © 2010 http://www.pipperr.de
ACID versus BASE
Horizontal versus vertical scalability
![Page 13: NOSQL – Not ONLY SQL - StartsteiteNoSQL – Not Only SQL – Why? Current popular problems in the 24/7 world –How change a database schema without downtime? –How to change/delete](https://reader034.fdocuments.in/reader034/viewer/2022042309/5ed6ddac2aa02a23022b0009/html5/thumbnails/13.jpg)
Seite 13 Gunther Pippèrr © 2010 http://www.pipperr.de
Our Oracle religion - ACID
ACID are the transactional properties of database
management systems
– Atomicity
• All of the operations in the transaction will complete, or none will.
– Consistency
• The database will be in a consistent state when the transaction
begins and ends.
– Isolation
• The transaction will behave as if it is the only operation being
performed upon the database.
– Durability
• Upon completion of the transaction, the operation will not be
reversed.
![Page 14: NOSQL – Not ONLY SQL - StartsteiteNoSQL – Not Only SQL – Why? Current popular problems in the 24/7 world –How change a database schema without downtime? –How to change/delete](https://reader034.fdocuments.in/reader034/viewer/2022042309/5ed6ddac2aa02a23022b0009/html5/thumbnails/14.jpg)
Seite 14 Gunther Pippèrr © 2010 http://www.pipperr.de
Horizontal scalability
One huge central database
– Bigger hardware helps to solve the performance issues
A
B
C
One Database
n…
Z
F
G 3
2
1
Costs
Performance
Only scalable to one absolute value
ONE DATABASE ! n Storage cell machines n Instance n App. Server
![Page 15: NOSQL – Not ONLY SQL - StartsteiteNoSQL – Not Only SQL – Why? Current popular problems in the 24/7 world –How change a database schema without downtime? –How to change/delete](https://reader034.fdocuments.in/reader034/viewer/2022042309/5ed6ddac2aa02a23022b0009/html5/thumbnails/15.jpg)
Seite 15 Gunther Pippèrr © 2010 http://www.pipperr.de
Do we really need strong ACID this in every Situation?
Amazon
Ebay
IT Monitoring
syslog
E-Mail Archive
Data ware house
Toll collect
Online News
Spiegel.de
GPS Tracking
BUT?
traffic guidance systems
![Page 16: NOSQL – Not ONLY SQL - StartsteiteNoSQL – Not Only SQL – Why? Current popular problems in the 24/7 world –How change a database schema without downtime? –How to change/delete](https://reader034.fdocuments.in/reader034/viewer/2022042309/5ed6ddac2aa02a23022b0009/html5/thumbnails/16.jpg)
Seite 16 Gunther Pippèrr © 2010 http://www.pipperr.de
The persistency of the data in a system
All data is visible to all users Partly visible in the system
Now!
Time frame
17:00 16:00
![Page 17: NOSQL – Not ONLY SQL - StartsteiteNoSQL – Not Only SQL – Why? Current popular problems in the 24/7 world –How change a database schema without downtime? –How to change/delete](https://reader034.fdocuments.in/reader034/viewer/2022042309/5ed6ddac2aa02a23022b0009/html5/thumbnails/17.jpg)
Seite 17 Gunther Pippèrr © 2010 http://www.pipperr.de
Vertical scalability
Small components working together in parallel
– All Components are independent
Costs
Performance
nearly endlessly scalable
A
B
C
1. Database
n…
Z
G 3
2
1
N databases N Instance N App. Server
2. Database
N. Database
F
![Page 18: NOSQL – Not ONLY SQL - StartsteiteNoSQL – Not Only SQL – Why? Current popular problems in the 24/7 world –How change a database schema without downtime? –How to change/delete](https://reader034.fdocuments.in/reader034/viewer/2022042309/5ed6ddac2aa02a23022b0009/html5/thumbnails/18.jpg)
Seite 18 Gunther Pippèrr © 2010 http://www.pipperr.de
CAP Theorem
Multi node environments
![Page 19: NOSQL – Not ONLY SQL - StartsteiteNoSQL – Not Only SQL – Why? Current popular problems in the 24/7 world –How change a database schema without downtime? –How to change/delete](https://reader034.fdocuments.in/reader034/viewer/2022042309/5ed6ddac2aa02a23022b0009/html5/thumbnails/19.jpg)
Seite 19 Gunther Pippèrr © 2010 http://www.pipperr.de
CAP Theorem
Start situation - Two nodes
X | 0 A
X | 0 B
Requirements:
• Consistency
• Availability
• Partition Tolerance
Eric Brewer : ACM Symposium 2000
![Page 20: NOSQL – Not ONLY SQL - StartsteiteNoSQL – Not Only SQL – Why? Current popular problems in the 24/7 world –How change a database schema without downtime? –How to change/delete](https://reader034.fdocuments.in/reader034/viewer/2022042309/5ed6ddac2aa02a23022b0009/html5/thumbnails/20.jpg)
Seite 20 Gunther Pippèrr © 2010 http://www.pipperr.de
Brewer's CAP Theorem for a distributed model
Is it possible to solve all this requirements?
Consistency
Availability Partition Tolerance Every operation must terminate in an intended response.
The client perceives that a set of operations has occurred all at once.
Operations will complete, even if individual components are unavailable.
See: http://www.julianbrowne.com/article/viewer/brewers-cap-theorem
Only two of Consistency, Availability and Partition Tolerance can be guaranteed!
![Page 21: NOSQL – Not ONLY SQL - StartsteiteNoSQL – Not Only SQL – Why? Current popular problems in the 24/7 world –How change a database schema without downtime? –How to change/delete](https://reader034.fdocuments.in/reader034/viewer/2022042309/5ed6ddac2aa02a23022b0009/html5/thumbnails/21.jpg)
Seite 21 Gunther Pippèrr © 2010 http://www.pipperr.de
Theory of the CAP Theorem (1)
Sunny Day Scenario for a update
X | 0 A
X | 0 B
X= 1 X | 1 A
X | 1 B
X= 1
X | 1 A
X | 1 B X= 1
1 2 3
Write 1
Read 1
![Page 22: NOSQL – Not ONLY SQL - StartsteiteNoSQL – Not Only SQL – Why? Current popular problems in the 24/7 world –How change a database schema without downtime? –How to change/delete](https://reader034.fdocuments.in/reader034/viewer/2022042309/5ed6ddac2aa02a23022b0009/html5/thumbnails/22.jpg)
Seite 22 Gunther Pippèrr © 2010 http://www.pipperr.de
Theory of the CAP Theorem (2)
Error Scenario for a update
X | 0 A
X | 0 B
X= 1 X | 1 A
X | 0 B
X= 1
X | 1 A
X | 0 B X= 1
1 2 3
Write 1
Read 0
![Page 23: NOSQL – Not ONLY SQL - StartsteiteNoSQL – Not Only SQL – Why? Current popular problems in the 24/7 world –How change a database schema without downtime? –How to change/delete](https://reader034.fdocuments.in/reader034/viewer/2022042309/5ed6ddac2aa02a23022b0009/html5/thumbnails/23.jpg)
Seite 23 Gunther Pippèrr © 2010 http://www.pipperr.de
Theory of the CAP Theorem (3)
How we solve the problem?
– Drop Partition Tolerance
• Only Node survived
– Side effect => we drop Availability
– Drop Consistency
• Live with inconsistencie
![Page 24: NOSQL – Not ONLY SQL - StartsteiteNoSQL – Not Only SQL – Why? Current popular problems in the 24/7 world –How change a database schema without downtime? –How to change/delete](https://reader034.fdocuments.in/reader034/viewer/2022042309/5ed6ddac2aa02a23022b0009/html5/thumbnails/24.jpg)
Seite 24 Gunther Pippèrr © 2010 http://www.pipperr.de
A other consistency model - BASE
A latency tolerant alternative to ASCI
– Basically Available
– Soft state
– Eventual consistency
• Given a sufficiently long period of time, over which no updates are
sent, we can expect that during this period, all updates will,
eventually, propagate through the system and all the replicas will be
consistent
![Page 25: NOSQL – Not ONLY SQL - StartsteiteNoSQL – Not Only SQL – Why? Current popular problems in the 24/7 world –How change a database schema without downtime? –How to change/delete](https://reader034.fdocuments.in/reader034/viewer/2022042309/5ed6ddac2aa02a23022b0009/html5/thumbnails/25.jpg)
Seite 25 Gunther Pippèrr © 2010 http://www.pipperr.de
ACID vs. BASE
Strong consistency
Isolation
Focus on “commit”
Nested transactions
Availability?
Conservative
(pessimistic)
Difficult evolution
(e.g. schema)
Weak consistency
– stale data OK
Availability first
Best effort
Approximate answers OK
Aggressive (optimistic)
Simpler!
Faster
Easier evolution
ACID BASE
Our application
Source: http://www.cs.berkeley.edu/~brewer/cs262b-2004/PODC-keynote.pdf
Do the right thing at each application layer
![Page 26: NOSQL – Not ONLY SQL - StartsteiteNoSQL – Not Only SQL – Why? Current popular problems in the 24/7 world –How change a database schema without downtime? –How to change/delete](https://reader034.fdocuments.in/reader034/viewer/2022042309/5ed6ddac2aa02a23022b0009/html5/thumbnails/26.jpg)
Seite 26 Gunther Pippèrr © 2010 http://www.pipperr.de
NoSQL architecture
Some base ideas
![Page 27: NOSQL – Not ONLY SQL - StartsteiteNoSQL – Not Only SQL – Why? Current popular problems in the 24/7 world –How change a database schema without downtime? –How to change/delete](https://reader034.fdocuments.in/reader034/viewer/2022042309/5ed6ddac2aa02a23022b0009/html5/thumbnails/27.jpg)
Seite 27 Gunther Pippèrr © 2010 http://www.pipperr.de
Some basic ideas and concepts
Map and Reduce
Consistent Hashing
MVCC-Protocol
Vector Clocks
Paxos
Wide Column Stores
![Page 28: NOSQL – Not ONLY SQL - StartsteiteNoSQL – Not Only SQL – Why? Current popular problems in the 24/7 world –How change a database schema without downtime? –How to change/delete](https://reader034.fdocuments.in/reader034/viewer/2022042309/5ed6ddac2aa02a23022b0009/html5/thumbnails/28.jpg)
Seite 28 Gunther Pippèrr © 2010 http://www.pipperr.de
Map and Reduce
Base ideas from the functional programming
– List processing
– Programing model – two functions
• Processes input key/value pair
• Produces set of intermediate pairs
• Combines all intermediate values for a particular key
• Produces a set of merged output values (usually just one)
map (input) -> list(intermediate_value)
reduce (out_key, list(intermediate_value)) -> list(out_value)
Source:http://labs.google.com/papers/mapreduce-osdi04-slides
![Page 29: NOSQL – Not ONLY SQL - StartsteiteNoSQL – Not Only SQL – Why? Current popular problems in the 24/7 world –How change a database schema without downtime? –How to change/delete](https://reader034.fdocuments.in/reader034/viewer/2022042309/5ed6ddac2aa02a23022b0009/html5/thumbnails/29.jpg)
Seite 29 Gunther Pippèrr © 2010 http://www.pipperr.de
Map and Reduce
Example – Count of Tokens --data
token1 = 'BMW, AUDI,VW,BMW,AUDI,SKODA'
token2 = 'KIA, VW ,VW,BMW,AUDI,VW'
map (token) : f() {count word} => List[]
-- first Run for each token list
map( tocken1 ) => list1 [ ('BMW,1,1') , ('AUDI,1,1')
, ('VW,1') , ('SKODA,1') ]
map( tocken2 ) => list2 [ ('BMW,1') , ('AUDI,1')
, ('VW,1,1,1'), , ('KIA,1') ]
-- Intermediate values
map (list1+list2) => list3 [ ('BMW,1,1,1'), ('AUDI,1,1')
, ('VW,1,1,1,1'), ('SKODA,1') , ('KIA,1') ]
reduce : f() { ++ } => List []
-reduce to the result
reduce(list3) => result [ ('BMW,3') , ('AUDI,3')
, (VW,4) , ('SKODA,1') , ('KIA,1') ]
select company , count(*) from cars_in_stock group by company;
![Page 30: NOSQL – Not ONLY SQL - StartsteiteNoSQL – Not Only SQL – Why? Current popular problems in the 24/7 world –How change a database schema without downtime? –How to change/delete](https://reader034.fdocuments.in/reader034/viewer/2022042309/5ed6ddac2aa02a23022b0009/html5/thumbnails/30.jpg)
Seite 30 Gunther Pippèrr © 2010 http://www.pipperr.de
Map and Reduce
Very easy to implement in parallel
BMW,
AUDI,VW,
BMW,AUDI
,SKODA
KIA,VW,
VW,BMW,A
UDI,VW
KIA,VW
,VW,BMW,
AUDI,VW
KIA,VW
,VW,BMW,
AUDI,VW
MAP
MAP
MAP
MAP
Reduce
Reduce
Result
DATA Map Intermediate Reduce Result
![Page 31: NOSQL – Not ONLY SQL - StartsteiteNoSQL – Not Only SQL – Why? Current popular problems in the 24/7 world –How change a database schema without downtime? –How to change/delete](https://reader034.fdocuments.in/reader034/viewer/2022042309/5ed6ddac2aa02a23022b0009/html5/thumbnails/31.jpg)
Seite 31 Gunther Pippèrr © 2010 http://www.pipperr.de
Consistent Hashing
Consistent Hashing
– A hash function to find the data in the right slot
– The addition or removal of one slot does not significantly
change the mapping of keys to slots.
See: http://en.wikipedia.org/wiki/Consistent_hashing
Adress data
X | 25
X | 99
X | 60
X | 50
X | 26
Hash(data)
X | 1
X | 5
X | 4
X |3
X | 2
Slots Server
192.168.1.90
192.168.1.30
192.168.1.10
![Page 32: NOSQL – Not ONLY SQL - StartsteiteNoSQL – Not Only SQL – Why? Current popular problems in the 24/7 world –How change a database schema without downtime? –How to change/delete](https://reader034.fdocuments.in/reader034/viewer/2022042309/5ed6ddac2aa02a23022b0009/html5/thumbnails/32.jpg)
Seite 32 Gunther Pippèrr © 2010 http://www.pipperr.de
Consistent Hashing (1)
Distributed Servers hold the data
– Hash(server) => Range of the storage location
– hash(data) => storage location
– Example:
3
2
1 X | 25
X | 99
X | 60
X | 50
X | 26
1. hash(192.168.1.10) = 15 2. hash(192.168.1.30) = 30 3. hash(192.168.1.90) = 80
![Page 33: NOSQL – Not ONLY SQL - StartsteiteNoSQL – Not Only SQL – Why? Current popular problems in the 24/7 world –How change a database schema without downtime? –How to change/delete](https://reader034.fdocuments.in/reader034/viewer/2022042309/5ed6ddac2aa02a23022b0009/html5/thumbnails/33.jpg)
Seite 33 Gunther Pippèrr © 2010 http://www.pipperr.de
Consistent Hashing (2)
Distributed servers hold the data
– If one server leave the cluster, only his data moved to the
next server
– The addressing at self is not changing!
3
2
1 X | 25
X | 99
X | 60
X | 50
X | 26
Leave Cluster
Move to 1
![Page 34: NOSQL – Not ONLY SQL - StartsteiteNoSQL – Not Only SQL – Why? Current popular problems in the 24/7 world –How change a database schema without downtime? –How to change/delete](https://reader034.fdocuments.in/reader034/viewer/2022042309/5ed6ddac2aa02a23022b0009/html5/thumbnails/34.jpg)
Seite 34 Gunther Pippèrr © 2010 http://www.pipperr.de
MVCC-Protocol
Multiversion concurrency control
(abbreviated MCC or MVCC)
– provide concurrent access to the database
– a database will implement updates not by deleting an old
piece of data and overwriting it with a new one
– instead marking the old data as obsolete and adding the
newer "version."
– multiple versions where stored, but only one is the latest.
See: http://en.wikipedia.org/wiki/Multiversion_concurrency_control
![Page 35: NOSQL – Not ONLY SQL - StartsteiteNoSQL – Not Only SQL – Why? Current popular problems in the 24/7 world –How change a database schema without downtime? –How to change/delete](https://reader034.fdocuments.in/reader034/viewer/2022042309/5ed6ddac2aa02a23022b0009/html5/thumbnails/35.jpg)
Seite 35 Gunther Pippèrr © 2010 http://www.pipperr.de
Vector clock
algorithm for generating a partial ordering of events in
a distributed system and detecting causality violations.
Contain the state of the sending process's logical
clock.
See: http://en.wikipedia.org/wiki/Vector_clock
Sender ID Time Counter
Sender ID Time Counter
Sender ID Time Counter
![Page 36: NOSQL – Not ONLY SQL - StartsteiteNoSQL – Not Only SQL – Why? Current popular problems in the 24/7 world –How change a database schema without downtime? –How to change/delete](https://reader034.fdocuments.in/reader034/viewer/2022042309/5ed6ddac2aa02a23022b0009/html5/thumbnails/36.jpg)
Seite 36 Gunther Pippèrr © 2010 http://www.pipperr.de
Vector clock
Example
A
B
C C:1
C:1 B:1
200
200
C:1 A:1
300
C:2 B:1 A:0
200
C:1 A:2
300
C:1 B:0 A:2
300
Use the counter
stamp to identify the
status and solve
problem with this
information
![Page 37: NOSQL – Not ONLY SQL - StartsteiteNoSQL – Not Only SQL – Why? Current popular problems in the 24/7 world –How change a database schema without downtime? –How to change/delete](https://reader034.fdocuments.in/reader034/viewer/2022042309/5ed6ddac2aa02a23022b0009/html5/thumbnails/37.jpg)
Seite 37 Gunther Pippèrr © 2010 http://www.pipperr.de
Paxos
Paxos is a family of protocols for solving consensus in
a network of unreliable processors.
– Consensus is the process of agreeing on one result among a
group of participants.
See: http://en.wikipedia.org/wiki/Paxos_algorithm
Opposite to the ACID two phase Commit (2PC => all processes can hang until all commit)
![Page 38: NOSQL – Not ONLY SQL - StartsteiteNoSQL – Not Only SQL – Why? Current popular problems in the 24/7 world –How change a database schema without downtime? –How to change/delete](https://reader034.fdocuments.in/reader034/viewer/2022042309/5ed6ddac2aa02a23022b0009/html5/thumbnails/38.jpg)
Seite 38 Gunther Pippèrr © 2010 http://www.pipperr.de
Paxos
Roles in a consensus Model
proposer acceptor learner
Make a
proposal with
the value v
Accepts
a proposal
Learn chosen
proposal
quorum
![Page 39: NOSQL – Not ONLY SQL - StartsteiteNoSQL – Not Only SQL – Why? Current popular problems in the 24/7 world –How change a database schema without downtime? –How to change/delete](https://reader034.fdocuments.in/reader034/viewer/2022042309/5ed6ddac2aa02a23022b0009/html5/thumbnails/39.jpg)
Seite 39 Gunther Pippèrr © 2010 http://www.pipperr.de
Wide Column Stores
Opposite of the row oriented table structure (for
example in a oracle database)
Column oriented
Example: Wide Column stores
ID:NAME:CITY:SALARY
“1,2,3” : “Scott,hugo,tiger”: “Huston,Munic,New York” : “1000,4500,6000”
![Page 40: NOSQL – Not ONLY SQL - StartsteiteNoSQL – Not Only SQL – Why? Current popular problems in the 24/7 world –How change a database schema without downtime? –How to change/delete](https://reader034.fdocuments.in/reader034/viewer/2022042309/5ed6ddac2aa02a23022b0009/html5/thumbnails/40.jpg)
Seite 40 Gunther Pippèrr © 2010 http://www.pipperr.de
Examples NoSQL
Some Examples
![Page 41: NOSQL – Not ONLY SQL - StartsteiteNoSQL – Not Only SQL – Why? Current popular problems in the 24/7 world –How change a database schema without downtime? –How to change/delete](https://reader034.fdocuments.in/reader034/viewer/2022042309/5ed6ddac2aa02a23022b0009/html5/thumbnails/41.jpg)
Seite 41 Gunther Pippèrr © 2010 http://www.pipperr.de
NOSQL Categories – for each problem a solution
Document store
Graph
Key Store – Key/value store on disk
– Key/value cache in RAM
– Eventually‐consistent key‐value store
– Key-value stores implementing the Paxos algorithm
– Hierarchical key-value store
– Ordered key-value store
Multivalue databases
Object database
Tabular
Tuple store Source: http://en.wikipedia.org/wiki/NoSQL
![Page 42: NOSQL – Not ONLY SQL - StartsteiteNoSQL – Not Only SQL – Why? Current popular problems in the 24/7 world –How change a database schema without downtime? –How to change/delete](https://reader034.fdocuments.in/reader034/viewer/2022042309/5ed6ddac2aa02a23022b0009/html5/thumbnails/42.jpg)
Seite 42 Gunther Pippèrr © 2010 http://www.pipperr.de
Document store -CouchDB
Apache CouchDB “cluster of unreliable commodity hardware Data Base” – document-oriented database
– queried and indexed in a MapReduce fashion using JavaScript.
– incremental replication with bi-directional conflict detection and resolution.
Store documents – Example of a document:
Surname: Gunther
Adress : Munich
Hobbies: Oracle DB
http://couchdb.apache.org/
![Page 43: NOSQL – Not ONLY SQL - StartsteiteNoSQL – Not Only SQL – Why? Current popular problems in the 24/7 world –How change a database schema without downtime? –How to change/delete](https://reader034.fdocuments.in/reader034/viewer/2022042309/5ed6ddac2aa02a23022b0009/html5/thumbnails/43.jpg)
Seite 43 Gunther Pippèrr © 2010 http://www.pipperr.de
Graph database
Neo4J
– Neo4j is a graph database
– fully transactional database
– stores data structured as graphs.
http://neo4j.org/
![Page 44: NOSQL – Not ONLY SQL - StartsteiteNoSQL – Not Only SQL – Why? Current popular problems in the 24/7 world –How change a database schema without downtime? –How to change/delete](https://reader034.fdocuments.in/reader034/viewer/2022042309/5ed6ddac2aa02a23022b0009/html5/thumbnails/44.jpg)
Seite 44 Gunther Pippèrr © 2010 http://www.pipperr.de
Key-Value Database
Redis
– open source, advanced key-value store.
– It is often referred to as a data structure server since keys
can contain strings, hashes, lists, sets and sorted sets.
http://redis.io/
Key Value
Key Value
Key Value
Key Value
In Memory Database
![Page 45: NOSQL – Not ONLY SQL - StartsteiteNoSQL – Not Only SQL – Why? Current popular problems in the 24/7 world –How change a database schema without downtime? –How to change/delete](https://reader034.fdocuments.in/reader034/viewer/2022042309/5ed6ddac2aa02a23022b0009/html5/thumbnails/45.jpg)
Seite 45 Gunther Pippèrr © 2010 http://www.pipperr.de
ColumnFamily-based data model
Cassandra
– The Apache Cassandra Project develops a highly scalable
second-generation distributed database, bringing together
Dynamo's fully distributed design and Bigtable's
ColumnFamily-based data model.
– Implemented in Java
– Example: Facebook , Twitter
– Schema less
• Only entities (column families are defined at startup)
• Add column at runtime
http://cassandra.apache.org/
![Page 46: NOSQL – Not ONLY SQL - StartsteiteNoSQL – Not Only SQL – Why? Current popular problems in the 24/7 world –How change a database schema without downtime? –How to change/delete](https://reader034.fdocuments.in/reader034/viewer/2022042309/5ed6ddac2aa02a23022b0009/html5/thumbnails/46.jpg)
Seite 46 Gunther Pippèrr © 2010 http://www.pipperr.de
Oracle
?
![Page 47: NOSQL – Not ONLY SQL - StartsteiteNoSQL – Not Only SQL – Why? Current popular problems in the 24/7 world –How change a database schema without downtime? –How to change/delete](https://reader034.fdocuments.in/reader034/viewer/2022042309/5ed6ddac2aa02a23022b0009/html5/thumbnails/47.jpg)
Seite 47 Gunther Pippèrr © 2010 http://www.pipperr.de
Oracle Berkeley DB 11g
Oracle Berkeley DB 11g
– designed to store data as opaque byte arrays of data in
key/value pairs indexed in one of the available access
methods.
– Create, read, update and delete (CRUD) operations on these
key/value pairs is done using the BDB get/put or cursor
position-based APIs.
http://www.oracle.com/technetwork/database/berkeleydb/overview/index.html
![Page 48: NOSQL – Not ONLY SQL - StartsteiteNoSQL – Not Only SQL – Why? Current popular problems in the 24/7 world –How change a database schema without downtime? –How to change/delete](https://reader034.fdocuments.in/reader034/viewer/2022042309/5ed6ddac2aa02a23022b0009/html5/thumbnails/48.jpg)
Seite 48 Gunther Pippèrr © 2010 http://www.pipperr.de
Where we find this ideas in an Oracle RDBMS?
Map and Reduce
– ???
Consistent Hashing
– Hash partioning
MVCC-Protocol
– Default since V3
Vector Clocks
– May be in the inter process mechanism
Paxos
– ???
![Page 49: NOSQL – Not ONLY SQL - StartsteiteNoSQL – Not Only SQL – Why? Current popular problems in the 24/7 world –How change a database schema without downtime? –How to change/delete](https://reader034.fdocuments.in/reader034/viewer/2022042309/5ed6ddac2aa02a23022b0009/html5/thumbnails/49.jpg)
Seite 49 Gunther Pippèrr © 2010 http://www.pipperr.de
Oracle NoSQL Feature
Mostly some Storage Feature in the database
– Paritioning
– Clustering
– Bitmap Join Index
But where is the scalability?
– Cluster? Only Horizontal……
See: http://asktom.oracle.com/pls/apex/f?p=100:11:0::::P11_QUESTION_ID:2664632900346253817
![Page 50: NOSQL – Not ONLY SQL - StartsteiteNoSQL – Not Only SQL – Why? Current popular problems in the 24/7 world –How change a database schema without downtime? –How to change/delete](https://reader034.fdocuments.in/reader034/viewer/2022042309/5ed6ddac2aa02a23022b0009/html5/thumbnails/50.jpg)
Seite 50 Gunther Pippèrr © 2010 http://www.pipperr.de
Summary
Relational databases
=> ALL Features First Solutions
– Best for feature-rich solutions
– Proven technologies
NoSQL
=> Solve one problem First solution
– Solves the problem, for that the DB is designed
– In some situations=>
the best solution to solve the problem
Traditional applications
One Question
on mass data
![Page 51: NOSQL – Not ONLY SQL - StartsteiteNoSQL – Not Only SQL – Why? Current popular problems in the 24/7 world –How change a database schema without downtime? –How to change/delete](https://reader034.fdocuments.in/reader034/viewer/2022042309/5ed6ddac2aa02a23022b0009/html5/thumbnails/51.jpg)
Seite 51 Gunther Pippèrr © 2010 http://www.pipperr.de
Conclusion
From the technical view:
– If we use the technologies to solve real problems both,
strong ACID and BASE DB‟s will live together
– Trend of the future!
From the business view:
– Save education for Java developers
– Save license costs
– Fight against the arrogance of the established companies
– More a hype then a advance
![Page 52: NOSQL – Not ONLY SQL - StartsteiteNoSQL – Not Only SQL – Why? Current popular problems in the 24/7 world –How change a database schema without downtime? –How to change/delete](https://reader034.fdocuments.in/reader034/viewer/2022042309/5ed6ddac2aa02a23022b0009/html5/thumbnails/52.jpg)
Seite 52 Gunther Pippèrr © 2010 http://www.pipperr.de
Fragen
NO:SQL
Discussion
![Page 53: NOSQL – Not ONLY SQL - StartsteiteNoSQL – Not Only SQL – Why? Current popular problems in the 24/7 world –How change a database schema without downtime? –How to change/delete](https://reader034.fdocuments.in/reader034/viewer/2022042309/5ed6ddac2aa02a23022b0009/html5/thumbnails/53.jpg)
Seite 53 Gunther Pippèrr © 2010 http://www.pipperr.de
More Information and Sources
The NoSQL Websites:
– http://nosql-database.org/
CAP
– http://en.wikipedia.org/wiki/CAP_theorem
– http://www.julianbrowne.com/article/viewer/brewers-cap-
theorem
ACID versus Base
– http://www.cs.berkeley.edu/~brewer/cs262b-2004/PODC-
keynote.pdf
Map And Reduce
– http://labs.google.com/papers/mapreduce-osdi04-slides
![Page 54: NOSQL – Not ONLY SQL - StartsteiteNoSQL – Not Only SQL – Why? Current popular problems in the 24/7 world –How change a database schema without downtime? –How to change/delete](https://reader034.fdocuments.in/reader034/viewer/2022042309/5ed6ddac2aa02a23022b0009/html5/thumbnails/54.jpg)
Seite 54 Gunther Pippèrr © 2010 http://www.pipperr.de
More Information
Paxos
– http://informatik.unibas.ch/lehre/hs10/cs341/_Downloads/Wo
rkshop/Reports/2010-HS-DIS-P_Fath-PAXOS-Report.pdf
![Page 55: NOSQL – Not ONLY SQL - StartsteiteNoSQL – Not Only SQL – Why? Current popular problems in the 24/7 world –How change a database schema without downtime? –How to change/delete](https://reader034.fdocuments.in/reader034/viewer/2022042309/5ed6ddac2aa02a23022b0009/html5/thumbnails/55.jpg)
Seite 55 Gunther Pippèrr © 2010 http://www.pipperr.de
Books
NoSQL: Einstieg in die Welt nichtrelationaler Web 2.0
Datenbanken
– Stefan Edlich, Achim Friedland, Jens Hampe , Benjamin
Brauer
• Verlag: Hanser Fachbuchverlag (7. Oktober 2010)
• Sprache: Deutsch
• ISBN-10: 3446423559
![Page 56: NOSQL – Not ONLY SQL - StartsteiteNoSQL – Not Only SQL – Why? Current popular problems in the 24/7 world –How change a database schema without downtime? –How to change/delete](https://reader034.fdocuments.in/reader034/viewer/2022042309/5ed6ddac2aa02a23022b0009/html5/thumbnails/56.jpg)
Seite 56 Gunther Pippèrr © 2010 http://www.pipperr.de
Referent
Gunther Pippèrr Dipl. Ing. Technische Informatik (FH) >10 Jahre IT Beratungserfahrung Freiberuflich tätig
Schwerpunkte der Beratungstätigkeit
IT System Architekt und technische Projektleitung
Web-Technologien
Design und Implementierung von Datenbank-basierten Internet-/Intranet-Anwendungen
Entwurf und Umsetzung von IT Infrastrukturen zum Datenmanagement mit Oracle Basis Technologien
Training rund um die Oracle Datenbank
Beruflicher Hintergrund
Dipl. Ing. Technische Informatik (FH) Weingarten
Mehr als10 Jahre Erfahrung in komplexen IT Projekten zum Thema Datenhaltung/Datenmanagement
Freiberufliche Projektarbeit
8 Jahre Geschäftsführer
Consultant bei großen Datenbank Hersteller
Projekterfahrung
Architekt für ein System zur Massendaten-Erfassung für Monitoring in der Telekommunikation
Architekt und technische Projektverantwortung für ein Smart Metering Portal für das Erfassen von Energiezählerdaten und Asset Management
Architekt und Projektleitung , Datenbank Design und Umsetzung für die Auftragsverwaltung mit Steuerung von externen Mitarbeitern für den Sprachdienstleister von deutschen Technologiekonzern
Architekt und technische Projektverantwortung für IT Infrastrukturprojekte, z.B.:
- Zentrale Datenhaltung für Münchner Hotelgruppe mit über 25 Hotels weltweit,
- Messdaten Erfassung für russischen Kabelnetzbetreiber
- Redundante Cluster Datenbank Infrastrukturen für diverse größere Web Anwendungen wie Fondplattform und Versicherungsportale
CRM- und Ausschreibungsportal für großen Münchner Bauträger
Technology Consultant
Kontaktdaten: E-Mail: [email protected] – Mobil:+49- (0)17180656113 – Internet : www.pipperr.de