Distributed Databases, by Dr. Khalil1 DISTRIBUTED DATABASES Dr. Awad Khalil Computer Science...

Distributed Databases, by Dr. Khalil 1

DISTRIBUTEDDATABASES

Dr. Awad KhalilDr. Awad Khalil

Computer Science DepartmentComputer Science Department

AUCAUC

Distributed Databases, by Dr. Khalil

2

Content

DBMS ArchitecturesDBMS Architectures Functions of DDBMSFunctions of DDBMS Generic DDBMS ArchitectureGeneric DDBMS Architecture Data FragmentationData Fragmentation Data AllocationData Allocation Data ReplicationData Replication Query ProcessingQuery Processing Concurrency Control and RecoveryConcurrency Control and Recovery


3

Distributed Computing Systems

Distributed databases bring the advantage of distributed Distributed databases bring the advantage of distributed computing to the database management domain.computing to the database management domain.

A A Distributed Computing System consists of a number of consists of a number of processing elements, not necessarily homogeneous, that processing elements, not necessarily homogeneous, that are connected by a communications network and that are are connected by a communications network and that are cooperate in performing certain assigned tasks.cooperate in performing certain assigned tasks.

A A Distributed Database is a collection of multiple is a collection of multiple logically interrelated databases distributed over a logically interrelated databases distributed over a computer network.computer network.

A A Distributed Database management System (DDBMS) is is the software system that manages a distributed database the software system that manages a distributed database while making the distribution transparent to the user.while making the distribution transparent to the user.


4

Parallel Versus Distributed Technology

There are two main types of multiprocessor system There are two main types of multiprocessor system architecture that are commonplace:architecture that are commonplace: Shared memory (tightly coupled) architectureShared memory (tightly coupled) architecture: Multiple : Multiple

processors share disk storage and also share main memory.processors share disk storage and also share main memory. Shared disk (loosely coupled) architectureShared disk (loosely coupled) architecture: Multiple processors : Multiple processors

share disk storage but each has their own main memory.share disk storage but each has their own main memory.

DBMSs developed using the above types of DBMSs developed using the above types of architectures are termed architectures are termed Parallel DBMSs rather rather than DDBMS, since they utilize parallel processor than DDBMS, since they utilize parallel processor technology.technology.

In In shared nothing architectureshared nothing architecture, every processor has its , every processor has its own main and secondary (disk) memory, and the own main and secondary (disk) memory, and the processors communicate over a high-speed processors communicate over a high-speed interconnection network (bus or switch).interconnection network (bus or switch).


5

DBMS Architectures

Centralized Database The database is stored at a single computer site. A centralized DBMS can support multiple users, but the DBMS and the database themselves reside totally at a single computer site.

All system component are accessed remotely via terminals to it but all processing is done centrally.


6

DBMS Architectures

Shared Nothing Architecture Every processor has its own main and secondary (disk) memory, and the processors communicate over a high-speed interconnection network (bus or switch).


7

DBMS Architectures

LAN-based File Server Networked small computers (PCs) share data (and other resources). Each has its own software (such as DBMS). One PC is designated as the file server where the shared database is stored.


8

DBMS Architectures

Truly Distributed Database Architecture

The actual database and DBMS software are distributed over many sites, connected by a communications network.


9

System Architectures

Traditional two-tier Client-Server Architecture

Three-tier architecture


10

Traditional two-tier Client-Server Architecture

Data-intensive business Data-intensive business applications consist of four major applications consist of four major components: components:

the database, the database, the transaction logic, the transaction logic, the application logic, and the application logic, and the user interface.the user interface.

The traditional two-tier client-The traditional two-tier client-server architecture provides a server architecture provides a basic separation of tasks.basic separation of tasks.

The client (tier 1) is primarily The client (tier 1) is primarily responsible for the presentation of responsible for the presentation of data to the user, data to the user,

The server (tier 2) is primarily The server (tier 2) is primarily responsible for supplying data responsible for supplying data services to the client.services to the client.


11

Traditional two-tier Client-Server ArchitectureProblems !!

The need for enterprise scalability challenged the The need for enterprise scalability challenged the traditional two-tier client-server model. In the mid-traditional two-tier client-server model. In the mid-1990s, as applications became more complex and 1990s, as applications became more complex and potentially could be deployed to hundreds or thousands potentially could be deployed to hundreds or thousands of end-users, the client side presented two problems that of end-users, the client side presented two problems that prevented true scalability:prevented true scalability:

A “fat” client, requiring considerable resources on A “fat” client, requiring considerable resources on the client’s computer to run effectively. This includes the client’s computer to run effectively. This includes disk space, RAM, and CPU power.disk space, RAM, and CPU power.

A significant client-side administration overhead.A significant client-side administration overhead.


12

Three-tier Architecture

By 1995, a new variation of the By 1995, a new variation of the traditional two-tier client-server model traditional two-tier client-server model appeared to solve the problem of appeared to solve the problem of enterprise scalability. This new enterprise scalability. This new architecture proposed three layers, each architecture proposed three layers, each potentially running on a different potentially running on a different platform:platform: The user interface layer, which runs The user interface layer, which runs

on the end-user’s computer (the on the end-user’s computer (the clientclient).).

The business logic and data The business logic and data processing layer. This middle tier processing layer. This middle tier runs on a server and is often called runs on a server and is often called the the application serverapplication server..

A DBMS, which stores the data A DBMS, which stores the data required by the middle tier. This tier required by the middle tier. This tier may run on separate server called the may run on separate server called the database serverdatabase server..


13

Three-tier Architecture – Advantages !!

The “thin” client, which requires less expensive hardware.The “thin” client, which requires less expensive hardware.

By centralizing the business logic for many end-users into a By centralizing the business logic for many end-users into a single application server, application maintenance is centralized. single application server, application maintenance is centralized. This eliminates the concerns of software distribution that are This eliminates the concerns of software distribution that are problematic in the traditional two-tier client-server model.problematic in the traditional two-tier client-server model.

The added modularity makes it easier to modify or replace one The added modularity makes it easier to modify or replace one tier without affecting the other tiers.tier without affecting the other tiers.

Separating the core business logic from the database functions Separating the core business logic from the database functions makes it easier to implement load balancing.makes it easier to implement load balancing.

The three-tier architecture maps quite naturally to the Web The three-tier architecture maps quite naturally to the Web environment, with a Web browser acting as the “thin” client, and environment, with a Web browser acting as the “thin” client, and a Web server acting as the application server.a Web server acting as the application server.


14

Distributed Databases - Definition

A A distributed database:distributed database: is a collection of is a collection of data that belongs logically to the same system data that belongs logically to the same system but is physically spread over the sites of a but is physically spread over the sites of a computer network.computer network.


15

Distributed Databases - Functions Location Transparency: A user can submit a query that accesses distributed : A user can submit a query that accesses distributed

objects without having to know where objects are.objects without having to know where objects are.

Performance Transparency: A query can be submitted from any node in a : A query can be submitted from any node in a distributed DBMS and it will run with comparable performance.distributed DBMS and it will run with comparable performance.

Copy Transparency: The system supports the optional existence of multiple : The system supports the optional existence of multiple copies of database objects.copies of database objects.

Transaction Transparency: A user can run an arbitrary transaction that updates : A user can run an arbitrary transaction that updates data at any number of sites, and the transaction behaves exactly like a local data at any number of sites, and the transaction behaves exactly like a local transaction and atomicity is maintained.transaction and atomicity is maintained.

Fragmentation Transparency: A relation (class) can be divided into multiple sites : A relation (class) can be divided into multiple sites according to specified distribution criteria.according to specified distribution criteria.

Schema Change Transparency: A user who adds or deletes a database object : A user who adds or deletes a database object from a distributed database need make the change only once and does not need to from a distributed database need make the change only once and does not need to change the catalog at all sites.change the catalog at all sites.

Local DBMS Transparency: The distributed database system provides services : The distributed database system provides services without regard for what local systems are actually managing local data.without regard for what local systems are actually managing local data.


16

An Example


17

Additional required Functions The ability to access remote sites and transmits queries and data The ability to access remote sites and transmits queries and data

among the various sites via a communication network.among the various sites via a communication network.

The ability to keep track of the data distribution and replication in The ability to keep track of the data distribution and replication in the DDBMS.the DDBMS.

The ability to devise execution strategies for queries and The ability to devise execution strategies for queries and transactions that access data from more than one site.transactions that access data from more than one site.

The ability to decide on which copy of a replicated data item to The ability to decide on which copy of a replicated data item to access.access.

The ability to maintain the consistency of copies of a replicated The ability to maintain the consistency of copies of a replicated data item.data item.

The ability to recover from individual site crashes and from new The ability to recover from individual site crashes and from new types of failures such as the failure of a communication link.types of failures such as the failure of a communication link.


18

Potential Advantages

Distributed nature of some database applications.Distributed nature of some database applications.

Increase Increase reliabilityreliability and and availabilityavailability..

Reliability: Reliability: is broadly defined as the probability that a system is is broadly defined as the probability that a system is running (not down) at a certain time point.running (not down) at a certain time point.

Availability: Availability: is the probability that a system is continuously is the probability that a system is continuously available during a time interval.available during a time interval.

Allowing data sharing while maintaining some measure of local Allowing data sharing while maintaining some measure of local control.control.

Improved performance.Improved performance.

Modular growth.Modular growth.

Lower communication costsLower communication costs


19

Cost

Software cost and complexity.Software cost and complexity.

Processing overhead.Processing overhead.

Data integrity.Data integrity.

Slow response.Slow response.


20

Generic DDBMS Architecture

A DDBMS is required to perform the following A DDBMS is required to perform the following functions:functions:

Determine the location(s) from which to retrieve Determine the location(s) from which to retrieve requested data.requested data.

If necessary, translate the request at one node using If necessary, translate the request at one node using a local DBMS into proper request to another node a local DBMS into proper request to another node using a different DBMS and data model.using a different DBMS and data model.

Provide data management functions such as Provide data management functions such as security, concurrency and deadlock control, query security, concurrency and deadlock control, query optimization, and failure recovery.optimization, and failure recovery.


21

Generic DDBMS Architecture In a DDBMS environment each site has a local DBMS that manages the In a DDBMS environment each site has a local DBMS that manages the

database stored at that site. database stored at that site. Also, each site has a copy of the distributed DBMS and the associated Also, each site has a copy of the distributed DBMS and the associated

distributed data dictionary/directory (DD/D). distributed data dictionary/directory (DD/D). The distributed DD/D contains the location of all data in the network, as well The distributed DD/D contains the location of all data in the network, as well

as data definitions. as data definitions. Requests for data by users or application programs are first processed by the Requests for data by users or application programs are first processed by the

distributed DBMS, which determines whether the transaction is local or distributed DBMS, which determines whether the transaction is local or global. global.

A local transaction is one in which the required data are stored entirely at the A local transaction is one in which the required data are stored entirely at the local site. local site.

A global transaction requires reference to data at one or more non-local sites A global transaction requires reference to data at one or more non-local sites to satisfy the request. For local transactions, the DDBMS passes the request to to satisfy the request. For local transactions, the DDBMS passes the request to the local DBMS; for global transactions, the DDBMS routes the request to the local DBMS; for global transactions, the DDBMS routes the request to other sites as necessary. other sites as necessary.

The DDBMSs at the participating sites exchange messages as needed to co-The DDBMSs at the participating sites exchange messages as needed to co-ordinate the processing of the transaction until it is completed (or aborted if ordinate the processing of the transaction until it is completed (or aborted if necessary). This process may be quit complex.necessary). This process may be quit complex.


22


The generic DDBS architecture is based on the following components:The generic DDBS architecture is based on the following components:

DDBS Global Language:: Most DDBS have a global language for communication Most DDBS have a global language for communication between nodes, this includes; message protocols; languages and formats for specifying between nodes, this includes; message protocols; languages and formats for specifying queries; responses; and errors. Nodes need to translate to this language. Global language queries; responses; and errors. Nodes need to translate to this language. Global language consists of: Data Definition Language – defines the description of objects, and Data consists of: Data Definition Language – defines the description of objects, and Data Manipulation Language – manipulates queries. One global language therefore only one Manipulation Language – manipulates queries. One global language therefore only one kind of translator. Without this global language would need a translator to and from every kind of translator. Without this global language would need a translator to and from every other language in the system.other language in the system.

DDBS Nucleus:: is responsible for co-ordination and control. The nucleus has to verify is responsible for co-ordination and control. The nucleus has to verify the initiator of the query is authorized to access the distributed database, or part of it, and the initiator of the query is authorized to access the distributed database, or part of it, and check for bugs. It also has to perform query decomposition, as a logical unit (relation) check for bugs. It also has to perform query decomposition, as a logical unit (relation) may be distributed over several nodes using a distributed data dictionary to produce the may be distributed over several nodes using a distributed data dictionary to produce the sub-queries. Query optimization is also performed by the nucleus. The DDBS nucleus also sub-queries. Query optimization is also performed by the nucleus. The DDBS nucleus also has a has a message distribution modulemessage distribution module that is responsible for creating a DDBS message that is responsible for creating a DDBS message for each sub-query, and creating message log information for control purposes when sub-for each sub-query, and creating message log information for control purposes when sub-responses (responses to sub-queries) are received back. When all responses have been responses (responses to sub-queries) are received back. When all responses have been received or a time limit has been reached then the appropriate information will be passed received or a time limit has been reached then the appropriate information will be passed onto the onto the response composition moduleresponse composition module. Response composition module produces a . Response composition module produces a single global response. The nucleus may be completely centralized (global node). single global response. The nucleus may be completely centralized (global node). Alternatively, the functions of the global node are distributed amongst all the query nodes.Alternatively, the functions of the global node are distributed amongst all the query nodes.


23


DDBS InterfaceDDBS Interface:: is responsible for the mapping and translation is responsible for the mapping and translation software. The interfaces between the local DBMS and the global software. The interfaces between the local DBMS and the global DBMS are:DBMS are:

Query InterfaceQuery Interface:: It is the interface at the user/application end. Users It is the interface at the user/application end. Users express the query in external query language (SQL) which may be the express the query in external query language (SQL) which may be the language of the local DBMS or a special DDBS query language. This is language of the local DBMS or a special DDBS query language. This is translated by the external/global query translation module to the global translated by the external/global query translation module to the global language. language.

Data InterfaceData Interface:: Translates from global to local language (may be SQL). Translates from global to local language (may be SQL).

The query is processed in the normal way by DBMS and a local response The query is processed in the normal way by DBMS and a local response will be the result.will be the result.


24

Data Fragmentation Techniques

In a DDBS, decision must be made regarding which site should be used to store In a DDBS, decision must be made regarding which site should be used to store which portions of the database. The simplest which portions of the database. The simplest logical unitslogical units of a database that of a database that can be distributed are the can be distributed are the relationsrelations themselves. There are several themselves. There are several fragmentation techniques:fragmentation techniques:

Horizontal Fragmentation: A horizontal fragment of a relation is a : A horizontal fragment of a relation is a subset of the tuples in that relation. The tuples that belong to the horizontal subset of the tuples in that relation. The tuples that belong to the horizontal fragment are specified by a condition on one or more attributes of the relation. fragment are specified by a condition on one or more attributes of the relation. A set of horizontal fragments whose conditions C1, C2, ..., Cn include all the A set of horizontal fragments whose conditions C1, C2, ..., Cn include all the tuples in R - which is, every tuple in R satisfies (C1 OR C2, OR ... OR Cn) - is tuples in R - which is, every tuple in R satisfies (C1 OR C2, OR ... OR Cn) - is called a called a complete horizontal fragmentation.complete horizontal fragmentation. In many cases a complete In many cases a complete horizontal fragmentation is also horizontal fragmentation is also disjointdisjoint; that is, no tuple in R satisfies (Ci ; that is, no tuple in R satisfies (Ci And Cj) for any i <> j. To reconstruct the relation R from a complete horizontal And Cj) for any i <> j. To reconstruct the relation R from a complete horizontal fragmentation, we need to apply the fragmentation, we need to apply the UNIONUNION operation to the fragments. operation to the fragments.


25


Vertical Fragmentation: A vertical fragment of a relation keeps : A vertical fragment of a relation keeps only certain attributes of the relation. A set of vertical fragments only certain attributes of the relation. A set of vertical fragments whose projection lists L1, L2, ..., Ln include all the attributes in whose projection lists L1, L2, ..., Ln include all the attributes in R but share only the primary key attribute of R is called a R but share only the primary key attribute of R is called a complete vertical fragmentation complete vertical fragmentation of R. In this case the of R. In this case the projection lists satisfy the following two conditions:projection lists satisfy the following two conditions: L1 L1 L2 L2 ... ... Ln = ATTRS(R).Ln = ATTRS(R). Li Li Lj = PK(R) for any i <> j, where ATTRS(R) is the set Lj = PK(R) for any i <> j, where ATTRS(R) is the set

of attributes of R and PK(R) is the primary key of R.of attributes of R and PK(R) is the primary key of R.

To reconstruct the relation R from a complete vertical To reconstruct the relation R from a complete vertical fragmentation, we apply the OUTER UNION operation to the fragmentation, we apply the OUTER UNION operation to the fragments. We could also apply the FULL OUTER JOIN fragments. We could also apply the FULL OUTER JOIN operation and get the same result for a complete vertical operation and get the same result for a complete vertical fragmentation.fragmentation.


26


Mixed Fragmentation: : The horizontal and vertical The horizontal and vertical fragmentations can be intermixed to yield a fragmentations can be intermixed to yield a mixed mixed fragmentationfragmentation. In this case the original relation can . In this case the original relation can be reconstructed by applying UNION and OUTER be reconstructed by applying UNION and OUTER UNION (or OUTER JOIN) operations in the UNION (or OUTER JOIN) operations in the appropriate order.appropriate order.


27


In general, a In general, a fragment fragment of relation R can be specified by a of relation R can be specified by a SELECT-PROJECT combination of operations SELECT-PROJECT combination of operations LL((CC(R)). (R)).

If C = True and L <> ATTRS(R), we get a vertical fragment.If C = True and L <> ATTRS(R), we get a vertical fragment.

If C <> True and L = ATTRS(R), we get a horizontal If C <> True and L = ATTRS(R), we get a horizontal fragment.fragment.

If C <> True and L <> ATTRS(R), we get a mixed If C <> True and L <> ATTRS(R), we get a mixed fragment.fragment.

A relation can itself be considered a fragment with C=True A relation can itself be considered a fragment with C=True and L= ATTRS(R).and L= ATTRS(R).


28

Fragmentation & Allocation Schemas

A A fragmentation schema of a database is a definition of a database is a definition of a set of fragments that includes all attributes and of a set of fragments that includes all attributes and tuples in the database and satisfies the condition that tuples in the database and satisfies the condition that whole database can be reconstructed from the whole database can be reconstructed from the fragments by applying some sequence of OUTER fragments by applying some sequence of OUTER UNION (or OUTER JOIN) and UNION operations.UNION (or OUTER JOIN) and UNION operations.

An An allocation schema describes the allocation of describes the allocation of fragments to sites of the DDBS; hence, it is the fragments to sites of the DDBS; hence, it is the mapping that specifies for each fragment the site(s) at mapping that specifies for each fragment the site(s) at which it is stored. If a fragment is stored at more than which it is stored. If a fragment is stored at more than one site, it is said to be one site, it is said to be replicatedreplicated..


29

Data Replication & Allocation

Why replication ?Why replication ?Replication is useful in improving the Replication is useful in improving the

availability of data.availability of data.


30

Replication Levels Fully Replication: The most extreme case is replication of the whole : The most extreme case is replication of the whole

database at every site in the distributed system, thus creating a fully replicated database at every site in the distributed system, thus creating a fully replicated distributed database. This can improve availability remarkably because the distributed database. This can improve availability remarkably because the system can continue to operate as long as at least one site is up. It also system can continue to operate as long as at least one site is up. It also improves performance of retrieval for global queries, because the result of improves performance of retrieval for global queries, because the result of such a query can be obtained locally from any one site. The disadvantage of such a query can be obtained locally from any one site. The disadvantage of full replication is that it can slow down update operations drastically, since a full replication is that it can slow down update operations drastically, since a single logical update must be performed on every copy of the database to single logical update must be performed on every copy of the database to keep the copies consistent.keep the copies consistent.

No Replication: Each fragment is stored at exactly one site. In this case all : Each fragment is stored at exactly one site. In this case all fragments must be disjoint, except for the repetition of primary keys among fragments must be disjoint, except for the repetition of primary keys among vertical (or mixed) fragments.vertical (or mixed) fragments.

Partial Replication:: Some fragments ofSome fragments of the database may be replicated the database may be replicated whereas others are not. The number of copies of each fragment can range whereas others are not. The number of copies of each fragment can range from one up tofrom one up to the total number of sites in the distributedthe total number of sites in the distributed system.system. A A description of the replication of fragments is called adescription of the replication of fragments is called a replication schemareplication schema..


31

An ExampleAssumptions:Assumptions: The company has three computer sites - The company has three computer sites -

one for each current department.one for each current department. Site 1 is used by the company Site 1 is used by the company

headquarters and accesses all headquarters and accesses all employees and project information employees and project information regularly, in addition to keeping track regularly, in addition to keeping track of DEPENDENT information for of DEPENDENT information for insurance purposes.insurance purposes.

Sites 2 and 3 are for departments 5 and Sites 2 and 3 are for departments 5 and

4, respectively. At each of these sites 4, respectively. At each of these sites we expect frequent access to the we expect frequent access to the EMPLOYEE and PROJECT EMPLOYEE and PROJECT information for the employees who information for the employees who work in the department and the projects work in the department and the projects controlled by that department. Further, controlled by that department. Further, we assume that these sites mainly we assume that these sites mainly access the NAME, SSN, SALARY, and access the NAME, SSN, SALARY, and SUPERSSN attributes of EMPLOYEE.SUPERSSN attributes of EMPLOYEE.


32

Fragmentation Schema The whole database can be stored at site 1.The whole database can be stored at site 1.

To determine the fragments to be replicated at sites 2 and 3, we can first To determine the fragments to be replicated at sites 2 and 3, we can first horizontally fragment the EMPLOYEE, PROJECT, DEPARTMENT, and horizontally fragment the EMPLOYEE, PROJECT, DEPARTMENT, and DEPT-LOCATIONS relations by department number. We can then vertically DEPT-LOCATIONS relations by department number. We can then vertically fragment the resulting EMPLOYEE fragments to include only the attributes fragment the resulting EMPLOYEE fragments to include only the attributes {NAME, SSN, SALARY, SUPERSSN, DNO}. The resulting mixed {NAME, SSN, SALARY, SUPERSSN, DNO}. The resulting mixed fragments are referred to as EMPD5 and EMPD4, which include the fragments are referred to as EMPD5 and EMPD4, which include the EMPLOYEE tuples satisfying the conditions DNO = 5 and DNO = 4, EMPLOYEE tuples satisfying the conditions DNO = 5 and DNO = 4, respectively. The horizontal fragments of PROJECTS, DEPARTMENT, and respectively. The horizontal fragments of PROJECTS, DEPARTMENT, and DEPT_LOCATIONS are similarly fragmented by department number. All DEPT_LOCATIONS are similarly fragmented by department number. All these fragments - stored at sites 2 and 3 - are replicated because they are also these fragments - stored at sites 2 and 3 - are replicated because they are also stored at the headquarters site 1.stored at the headquarters site 1.

We must now fragment the WORKS_ON relation and decide which fragments We must now fragment the WORKS_ON relation and decide which fragments of WORKS_ON to store at sites 2 and 3. The problem is that no attribute of of WORKS_ON to store at sites 2 and 3. The problem is that no attribute of WORKS_ON directly indicates the department to which each tuple belongs. WORKS_ON directly indicates the department to which each tuple belongs. In fact, each tuple in WORKS_ON relates an employee e to a project p. We In fact, each tuple in WORKS_ON relates an employee e to a project p. We could fragment WORKS_ON based on the department d in which e works or could fragment WORKS_ON based on the department d in which e works or based on the department d that controls p.based on the department d that controls p.


33

Fragmentation Schema Complete and disjoint fragments of the Complete and disjoint fragments of the

WORKS_ON relation.WORKS_ON relation. (a) Fragments of WORKS_ON for (a) Fragments of WORKS_ON for

employees working in department 5 employees working in department 5 (C=ESSN IN (SELECT SSN FROM (C=ESSN IN (SELECT SSN FROM EMPLOYEE WHERE DNO=5)).EMPLOYEE WHERE DNO=5)).

(b) Fragments of WORKS_ON for (b) Fragments of WORKS_ON for employees working in department 4 employees working in department 4 (C=ESSN IN (SELECT SSN FROM (C=ESSN IN (SELECT SSN FROM EMPLOYEE WHERE DNO=4)).EMPLOYEE WHERE DNO=4)).

(c) Fragments of WORKS_ON for (c) Fragments of WORKS_ON for employees working in department 1 employees working in department 1 (C=ESSN IN (SELECT SSN FROM (C=ESSN IN (SELECT SSN FROM EMPLOYEE WHERE DNO=1)).EMPLOYEE WHERE DNO=1)).

The union of fragments G1, G2, and G3 The union of fragments G1, G2, and G3 gives all WORKS_ON tuples for gives all WORKS_ON tuples for employees who work for department 5.employees who work for department 5.

The union of fragments G4, G5, and G6 The union of fragments G4, G5, and G6 gives all WORKS_ON tuples for gives all WORKS_ON tuples for employees who work for department 4.employees who work for department 4.

The union of fragments G1, G4, and G7 The union of fragments G1, G4, and G7 gives all WORKS_ON tuples for projects gives all WORKS_ON tuples for projects controlled by department 5.controlled by department 5.

The union of fragments G2, G5, and G8 The union of fragments G2, G5, and G8 gives all WORKS_ON tuples for projects gives all WORKS_ON tuples for projects controlled by department 4.controlled by department 4.


34

Allocation Schema

Site 2

We choose to include all fragments that can be joined to We choose to include all fragments that can be joined to either an EMPLOYEE tuple or a PROJECT tuple at sites 2 either an EMPLOYEE tuple or a PROJECT tuple at sites 2 and 3. Hence, we place the union of fragments G1, G2, and 3. Hence, we place the union of fragments G1, G2, G3, G4, and G7 at site 2.G3, G4, and G7 at site 2.


35

Allocation Schema

Site 3 We choose the union of fragments G4, G5, G6, G2, and G8 We choose the union of fragments G4, G5, G6, G2, and G8

at site 3 (G2 and G4 are replicated at both sites).at site 3 (G2 and G4 are replicated at both sites).


36

Types of Distributed Database Systems Degree of Homogeneity

Homogeneous DDBMSHomogeneous DDBMS:: all all servers (or individual local servers (or individual local DBMSs) use identical software DBMSs) use identical software and all clients use identical and all clients use identical software.software.

Heterogeneous DDBMSHeterogeneous DDBMS:: Servers and clients use different Servers and clients use different software. In a heterogeneous software. In a heterogeneous multidatabase system, one server multidatabase system, one server may be a relational DBMS, may be a relational DBMS, another a network DBMS, and a another a network DBMS, and a third a hierarchical DBMS; in such third a hierarchical DBMS; in such a case it is necessary to have a a case it is necessary to have a canonical system language and to canonical system language and to include language translators in the include language translators in the client to translate subqueries from client to translate subqueries from the canonical language to the the canonical language to the language of each server.language of each server.

Degree of Local Autonomy No Local AutonomyNo Local Autonomy: all access to the : all access to the

DDBMS must be gained through a client. A DDBMS must be gained through a client. A DDBMS looks like a centralized DBMS to the DDBMS looks like a centralized DBMS to the user. A single conceptual schema exists, and all user. A single conceptual schema exists, and all access to the system is obtained through a client.access to the system is obtained through a client.

Local AutonomyLocal Autonomy:: direct access by local direct access by local transactions to a server is permitted. Such type transactions to a server is permitted. Such type of DDBMS is called of DDBMS is called federated DDBMSfederated DDBMS (or a (or a multidatabase systemmultidatabase system). In such a system, ). In such a system, each server is an independent and autonomous each server is an independent and autonomous centralized DBMS that has its own local users, centralized DBMS that has its own local users, local transactions, and DBA and hence has a local transactions, and DBA and hence has a very high degree of local autonomy. Each server very high degree of local autonomy. Each server can authorize access to particular portions of its can authorize access to particular portions of its database by specifying an database by specifying an export schemaexport schema, , which specifies the part of the database that may which specifies the part of the database that may be accessed by a certain class of nonlocal users. be accessed by a certain class of nonlocal users. A federated system is a hybrid between A federated system is a hybrid between distributed and centralized systems; it is a distributed and centralized systems; it is a centralized system for local autonomous users centralized system for local autonomous users and a distributed system for global users. and a distributed system for global users.


37

Types of Distributed Database Systems

Degree of Distribution Transparency

High Degree of Distribution High Degree of Distribution TransparencyTransparency: the user sees a single : the user sees a single integrated schema without any integrated schema without any information concerning information concerning fragmentation, replication, or fragmentation, replication, or distribution.distribution.

No Distribution TransparencyNo Distribution Transparency:: the the user sees all fragmentation, allocation, user sees all fragmentation, allocation, and replication. The user must refer to and replication. The user must refer to specific fragment copies at specific specific fragment copies at specific sites when formulating a query, by sites when formulating a query, by appending the site name before a appending the site name before a relation or fragment name. This is a relation or fragment name. This is a part of a complex problem of part of a complex problem of namingnaming in distributed systems. in distributed systems.


38

Query Processing in Distributed Databases

Data Transfer CostsData Transfer Costs In a distributed system, the transfer of data over the In a distributed system, the transfer of data over the

network is considered an important factor affecting the network is considered an important factor affecting the processing and optimization of a query.processing and optimization of a query.

This data includes intermediate files that are This data includes intermediate files that are transferred to other sites for further processing, as well transferred to other sites for further processing, as well as final result files that may have to be transferred to as final result files that may have to be transferred to the site where the query result is needed.the site where the query result is needed.

DDBMS query optimization algorithms consider the DDBMS query optimization algorithms consider the goal of reducing the goal of reducing the amount of data transferamount of data transfer as an as an optimization criterion in choosing a distributed query optimization criterion in choosing a distributed query execution strategy.execution strategy.


39

Example Suppose that the EMPLOYEE and DEPARTMENT relations Suppose that the EMPLOYEE and DEPARTMENT relations

are stored in site 1 and site 2, respectively and neither relation are stored in site 1 and site 2, respectively and neither relation is fragmented. The size of the EMPLOYEE relation is 100 * is fragmented. The size of the EMPLOYEE relation is 100 * 10,000 = 1,000,000 bytes, and the size of the DEPARTMENT 10,000 = 1,000,000 bytes, and the size of the DEPARTMENT relation is 35 * 100 = 3500 bytes.relation is 35 * 100 = 3500 bytes.


40

Example

Q1Q1 Consider that the query is Consider that the query is

submitted at a distinct site 3: submitted at a distinct site 3: “For each employee, retrieve “For each employee, retrieve the employee name and the the employee name and the name of the department for name of the department for which the employee works.” which the employee works.”

The result of this query will The result of this query will include 10,000 records, include 10,000 records, assuming that every employee assuming that every employee is related to a department. is related to a department. Suppose that each record in the Suppose that each record in the query results in 40 bytes long.query results in 40 bytes long.

If minimizing the amount of If minimizing the amount of data transfer is our data transfer is our optimization criterion, we optimization criterion, we should chooseshould choose strategy 3 strategy 3..

1. Transfer both the EMPLOYEE and the DEPARTMENT relations to the result site, and perform the join at site 3. In this case we need to transfer a total of 1,000,000 + 3500 = 1,003,500 bytes.

2. Transfer the EMPLOYEE relation to site 2, execute the join at site 2, and send the result to site 3. The size of the query result is 40 * 10,000 = 400,000 bytes, so we must transfer 400,000 + 1,000,000 = 1,400,000 bytes.

3. Transfer the DEPARTMENT relation to site 1, execute the join at site 1, and send the result to site 3. In this case we have to transfer 400,000 + 3500 = 403,500 bytes.


41

Example

Q2Q2 Consider that another Consider that another

query is submitted at site query is submitted at site 3: “For each department, 3: “For each department, retrieve the department retrieve the department name and the name of the name and the name of the department manager.” department manager.”

There are three simple There are three simple strategies for executing strategies for executing this distributed query.this distributed query.

Again, we would chooseAgain, we would choose strategy 3strategy 3..

1. Transfer both the EMPLOYEE and the DEPARTMENT relations to the result site, and perform the join at site 3. In this case we need to transfer a total of 1,000,000 + 3500 = 1,003,500 bytes.

2. Transfer the EMPLOYEE relation to site 2, execute the join at site 2, and send the result to site 3. The size of the query result is 40 * 100 = 4000 bytes, so we must transfer 4000 + 1,000,000 = 1,004,000 bytes.

3. Transfer the DEPARTMENT relation to site 1, execute the join at site 1, and send the result to site 3. In this case we have to transfer 4000 + 3500 = 7500 bytes.


42

Example

Q3Q3

Consider that the Consider that the result site is site 2; result site is site 2; then we have two then we have two simple strategies.simple strategies.

1. Transfer the EMPLOYEE relation to site 2, execute the query, and present the result to the user at site 2. Here, we need to transfer the same number of bytes (1,000,000) for both Q1 and Q2.

2. Transfer the DEPARTMENT relation to site 1, execute the query at site 1, and send the result back to site 2. In this case we must transfer 400,000 + 3500 = 403,500 bytes for Q1 and 4000 + 3500 = 7500 bytes for Q2.


43

Distributed Query Processing Using SemiJoin

The idea behind distributed query processing using the The idea behind distributed query processing using the semijoin operation is to reduce the number of tuples in a semijoin operation is to reduce the number of tuples in a relation before transferring it to another site.relation before transferring it to another site.

Intuitively, the idea is to send the joining column of one Intuitively, the idea is to send the joining column of one relation relation RR to the site where the other relation to the site where the other relation SS is is located; this column is then joined with located; this column is then joined with SS..

Following that, the join attributes, along with the Following that, the join attributes, along with the attributes required in the result, are projected out and attributes required in the result, are projected out and shipped back to the original site and joined with shipped back to the original site and joined with RR..

This can be quite an efficient solution to minimizing This can be quite an efficient solution to minimizing data transfer.data transfer.


44

Example

Q1 and Q2 Processing Using semijoin: Project the join attributes of DEPARTMENT at site 2, and transfer Project the join attributes of DEPARTMENT at site 2, and transfer

them to site 1. For Q1, we transfer F = them to site 1. For Q1, we transfer F = DNUMBER(DEPARTMENT), whose size is 4 * 100 = 400 DNUMBER(DEPARTMENT), whose size is 4 * 100 = 400 bytes, whereas for Q2, we transfer F’ = bytes, whereas for Q2, we transfer F’ = MGRSSN(DEPARTMENT), whose size is 9 * 100 = 900 bytes.MGRSSN(DEPARTMENT), whose size is 9 * 100 = 900 bytes.

Join the transferred file with the EMPLOYEE relation at site 1, Join the transferred file with the EMPLOYEE relation at site 1, and transfer the required attributes from the resulting file to site 2. and transfer the required attributes from the resulting file to site 2. For Q1, we transfer R = For Q1, we transfer R = <DNO, FNAME, LNAME> (F <DNO, FNAME, LNAME> (F DNUMBER=DNO EMPLOYEE) whose size is 34 * 10,000 = DNUMBER=DNO EMPLOYEE) whose size is 34 * 10,000 = 340,000 bytes, whereas for Q2, we transfer R’ = 340,000 bytes, whereas for Q2, we transfer R’ = <MGRSSN, <MGRSSN, FNAME, LNAME>(F’ FNAME, LNAME>(F’ MGRSSN=SSN EMPLOYEE), whose MGRSSN=SSN EMPLOYEE), whose size is 39 * 100 = 3900 bytes.size is 39 * 100 = 3900 bytes.

Execute the query by joining the transferred file R or R’ with Execute the query by joining the transferred file R or R’ with DEPARTMENT, and present the result to the user at site 2.DEPARTMENT, and present the result to the user at site 2.


45

Concurrency Control and Recovery in Distributed Databases Dealing with multiple copies of the data Dealing with multiple copies of the data

items.items.

Failure of individual sites.Failure of individual sites.

Failure of communication links.Failure of communication links.

Distributed commit.Distributed commit.

Distributed deadlock.Distributed deadlock.


46

Distributed Concurrency Control Based on a Distinguished Copy of a Data Item

The idea is to designate a particular copy of each The idea is to designate a particular copy of each data item as a data item as a distinguished copydistinguished copy. .

The locks for this data item are associated with The locks for this data item are associated with the distinguished copy, and all locking and the distinguished copy, and all locking and unlocking requests are sent to the site that unlocking requests are sent to the site that contains that copy. contains that copy.

A number of different techniques are based on A number of different techniques are based on this idea.this idea.


47

Techniques

Primary Site TechniquesPrimary Site Techniques.. In this In this method, all distinguished copies are method, all distinguished copies are kept at one site, called the kept at one site, called the primary primary sitesite. The primary site is designated to . The primary site is designated to be the coordinator site for all database be the coordinator site for all database items. Hence, all locks are kept at that items. Hence, all locks are kept at that site, and all requests for locking or site, and all requests for locking or unlocking are sent to that site.unlocking are sent to that site.

Advantage:Advantage: It is a simple extension of the It is a simple extension of the

centralized approach.centralized approach.

Disadvantages:Disadvantages: All locking requests are sent to a single All locking requests are sent to a single

site, possibly overloading that site and site, possibly overloading that site and causing a system bottleneck.causing a system bottleneck.

Failure of the primary site paralyzes the Failure of the primary site paralyzes the system, since all locking information is system, since all locking information is kept at that site. This can limit system kept at that site. This can limit system reliability and availability.reliability and availability.

Primary Site with Backup SitePrimary Site with Backup Site. This . This approach addresses the second approach addresses the second disadvantage of the primary site method by disadvantage of the primary site method by designating a second site to be a designating a second site to be a backup backup sitesite. All locking information is maintained . All locking information is maintained at both the primary and the backup sites. In at both the primary and the backup sites. In case of failure of the primary site, the case of failure of the primary site, the backup site can take over as primary site, backup site can take over as primary site, and a new back site can be chosen.and a new back site can be chosen.

Advantage:Advantage: This approach simplifies the process of This approach simplifies the process of

recovery from failure of the primary site, recovery from failure of the primary site, since the backup site takes over and since the backup site takes over and processing can resume after a new backup processing can resume after a new backup site is chosen and the lock status site is chosen and the lock status information is copied to that site.information is copied to that site.

Disadvantage:Disadvantage: It slows down the process of acquiring It slows down the process of acquiring

locks, because all lock requests and locks, because all lock requests and granting of locks must be recorded at both granting of locks must be recorded at both the primary and the backup sites before a the primary and the backup sites before a response is sent to the requesting response is sent to the requesting transaction.transaction.


48

TechniquesPrimary Copy TechniquePrimary Copy Technique. . This method attempts to This method attempts to

distribute the load of lock distribute the load of lock coordination among various coordination among various sites by having the sites by having the distinguished copies of different distinguished copies of different data items stored at different data items stored at different sites. sites.

Failure of one site affects any Failure of one site affects any transactions that are accessing transactions that are accessing locks on items whose primary locks on items whose primary copies reside at that sit, but copies reside at that sit, but other transactions are not other transactions are not affected. affected.

This method can also use This method can also use backup sites to enhance backup sites to enhance reliability and availability.reliability and availability.

Choosing a New Coordinator Site Choosing a New Coordinator Site

in Case of Failurein Case of Failure. . Whenever a coordinator site fails Whenever a coordinator site fails

in any of the preceding in any of the preceding techniques, the sites that are still techniques, the sites that are still running must choose a new running must choose a new coordinator. coordinator.

In the case of the primary site In the case of the primary site approach with no backup site, all approach with no backup site, all executing transactions must be executing transactions must be aborted and restarted, and the aborted and restarted, and the recovery process is quite tedious.recovery process is quite tedious.

For methods that use backup sites, For methods that use backup sites, transaction processing is transaction processing is suspended while the backup site is suspended while the backup site is designated as the new primary site designated as the new primary site and a new backup site is chosen and a new backup site is chosen and is sent copies of all the and is sent copies of all the locking information from the new locking information from the new primary site.primary site.


49

Distributed Concurrency Control Based on Voting

In the voting method, there is no distinguished copy; rather, a lock In the voting method, there is no distinguished copy; rather, a lock request is sent to all sites that include a copy of the data item. Each request is sent to all sites that include a copy of the data item. Each copy maintains its own lock and grant or deny the request for it. If a copy maintains its own lock and grant or deny the request for it. If a transaction that requests a lock is granted that lock by a transaction that requests a lock is granted that lock by a majoritymajority of of the copies, it holds the lock and informs all copies that it has been the copies, it holds the lock and informs all copies that it has been granted the lock. If a transaction does not receive a majority of votes granted the lock. If a transaction does not receive a majority of votes granting it a lock within a certain granting it a lock within a certain time-out periodtime-out period, it cancels its , it cancels its request and informs all sites of the cancellation.request and informs all sites of the cancellation.

The voting method is considered a truly distributed concurrency The voting method is considered a truly distributed concurrency control method, since the responsibility for a decision resides with control method, since the responsibility for a decision resides with all the sites involved. Simulation studies have shown that voting has all the sites involved. Simulation studies have shown that voting has higher message traffic among sites than do the distinguished copy higher message traffic among sites than do the distinguished copy methods. If the algorithm takes into account possible site failures methods. If the algorithm takes into account possible site failures during the voting process, it becomes extremely complex.during the voting process, it becomes extremely complex.


50

Distributed Recovery

The recovery process in distributed databases is quite involved. In some cases The recovery process in distributed databases is quite involved. In some cases it is quite difficult even to determine whether a site is down without it is quite difficult even to determine whether a site is down without exchanging numerous messages with other sites. For example, suppose that exchanging numerous messages with other sites. For example, suppose that site X sends a message to site Y and expects a response from Y but does not site X sends a message to site Y and expects a response from Y but does not receive it. There are several possible explanations:receive it. There are several possible explanations:

The message was not delivered to Y because of communication failure.The message was not delivered to Y because of communication failure.

Site Y is down and could not respond.Site Y is down and could not respond.

Site Y is running and sent a response, but the response was not delivered.Site Y is running and sent a response, but the response was not delivered.

Another problem with distributed recovery is distributed commit. When a Another problem with distributed recovery is distributed commit. When a transaction is updating data at several sites, it cannot commit until it is sure that transaction is updating data at several sites, it cannot commit until it is sure that the effect of the transaction on every site cannot be lost. This means that every the effect of the transaction on every site cannot be lost. This means that every site must first have recorded the local effects of the transactions permanently in site must first have recorded the local effects of the transactions permanently in

the local site log on disk. The the local site log on disk. The two-phase commit protocol is often used to protocol is often used to ensure the correctness of distributed commit.ensure the correctness of distributed commit.

Distributed Databases, by Dr. Khalil 51

Thank you

Distributed Databases, by Dr. Khalil1 DISTRIBUTED DATABASES Dr. Awad Khalil Computer Science...

Documents

Transcript of Distributed Databases, by Dr. Khalil1 DISTRIBUTED DATABASES Dr. Awad Khalil Computer Science...