Block 1 – Databases in context Evolution of data managementwilsonwlw2009.webs.com/S359-Block...

Block 1 – Databases in context

There are 5 Sections.

Section 1. Evolution of data managementSection 2. Data Management – why it matters Sect o . ata a age e t w y t atte sSection 3. Database management system conceptsSection 4 Developing database systemsSection 4. Developing database systemsSection 5. Conceptual data model

OUHK S359 Relational Databases: theory and practice T1 - 1

Evolution of data management

Basic definitions and descriptionData • It is a term given to many recordable facts• for example S359 65467592 Peter Kan• for example, S359, 65467592, Peter Kan• The representation of data can be chosen for specific

i diff t f t h d tpurposes, or in different formats such as date• We need something other than data value and

representation to be able to say what data means.



Basic definitions and descriptionInformation• It is said to be meaningful data. It is any kind of

knowledge that is exchangeable amongst peopleknowledge that is exchangeable amongst people, about things, facts, concepts in some context.

• for example Course Code is S359• for example, Course Code is S359 My mobile phone is 65465792My OUHK tutor is ‘Peter Kan’



Basic definitions and descriptionInformation• must be understood by the receiver =>

the shared tho ghts abo t the domain of disco rsethe shared thoughts about the domain of discourse (i.e. scope) are carried in the minds of the people involved in the exchangeinvolved in the exchange.

• ‘Peter Kan’ may refer to different person. We need to k hi h i i ( ) ( O )know which organization (scope) (e.g. OU, HKU) when we are talking about ‘Peter Kan’



Basic definitions and descriptionDatabase• A collection of structured, persistent, shared data.

Str ct red data:• Structured data: the data has a regular, defined, structure that supports

th i t t i t t ti d i l ti f d tthe consistent interpretation and manipulation of data. the structure also allows a user to specify a data

i i h ld b i l li drequirement in a way that could be consistently applied to the entire collection of data without having to consider every data item in the database


consider every data item in the database



Persistent data:• Persistent data: Data that will continue to be stored even when the

li ti ti d i th d t lapplications creating and using the data are no longer running.

I i ll d i hi fil It is usually stored within a computer file system on disk or some other long-term storage medium




Shared data:• Shared data: Data can be used by any user applications authorized

t dto do so



Basic definitions and descriptionDatabase Management System (DBMS)g y ( )• A collection of software and procedures that enable

the storage, manipulation and protection of a g , p pcollection of databases

Database EngineDatabase Engine• A software program within a Database Management

System that is responsible for accessing a databaseSystem that is responsible for accessing a database and manipulating data. It is also responsible for processing a query and ensuring that database content


processing a query and ensuring that database content is retrieved and altered to meet the requirements

Evolution of data managementModern Data Processing

Modern Data Processing File based approach File based approach Database approach

Question: What are advantages and disadvantages of using each approach?


Evolution of data managementModern Data Processing – File based approach

File contains many records.• Each record is a collection of data values, each in a ,

field.

Serial file• Records are stored one after another as they are

written, and they can be retrieved only by starting at the first record and reading sequentially.



Sequential file• Records are stored in a particular order, and they can p , y

be retrieved only by starting at the first record and reading sequentially.g q y

Indexed file• Indexes are involved in providing an access path to

records based on a value within each record, such as name.

• Records can be retrieved sequentially or randomly.


q y y


Advantages1. By comparing with historical data management, it y p g g ,

allows manipulation of structured data2 The processing system using file based approach is2. The processing system using file based approach is

relatively simple than that using database approach

Disadvantages See following pages



1. Limited Data Sharing/ Data isolation• with the traditional applications approach, each pp pp ,

application has its own private files

h i li l i f h d• there is little opportunity for users to share data outside of their own applications.

• Consequence: the same data may have to be entered several times in order to update files with duplicateseveral times in order to update files with duplicate data.



2. Uncontrolled data redundancy and inconsistence

• Each application has its own files It records same• Each application has its own files. It records same data item in multiple files.

• Disadvantages: a) waste storage spacea) waste storage space b) data redundancy ) d i ic) data inconsistency



3. Inflexibility / Difficulty in accessing information in new format

• Reports can be produced routinely and efficiently if they were anticipated in the original design of the y p g gsystem.

I fl ibl d il d f• Inflexible and cannot easily respond to requests for information in a new format that was not anticipated i h i i l d iin the original design.

• Needs to write new programs for new taskOUHK S359 Relational Databases: theory and practice T1 - 15

Needs to write new programs for new task.


4. Inflexibility in defining new constraints and change existing constraints

• Constraints (e.g. sex must be either ‘M’ or ‘F’; age must be above 18) are not associated with data item.)

• In the file approach, a constraint is expressed as part of the procedural code within a program combiningof the procedural code within a program, combining the maintenance of the constraint with the rest of the processing.processing.

(continued…)



4. Inflexibility in defining new constraints and change existing constraints

• Problems in understanding how a constraint is expressedexpressed.

• Inflexibility in changing existing or defining new y g g g gconstraints.



5. Problem in atomic update

T ti t b f ll l t d thi i• Transaction must be fully completed or nothing is done if failure occur.

• In the file approach, a failure occur in the midway of a transaction will leave the data in inconsistentof a transaction will leave the data in inconsistent state.



5. Problem in atomic update

F l• For example,Your bank account --> HK$100 --> OpenUIf a failure occur at the midway, what happen ?HK$100 is deducted from your account.HK$100 is deducted from your account.However, there is no change in the bank account

f O Uof OpenU.



6. Problem in concurrency control

• To improve efficiency multiple users may access• To improve efficiency, multiple users may access the same file concurrently => concurrent access

• Uncontrolled concurrent access to files (in the file approach) can lead to inconsistency.pp ) y

• For example, two people read a balance and update i h iit at the same time.



7. Problem in security

• Different people may be allowed to read different• Different people may be allowed to read different fields in the same file.

• In the file system, it is difficult to define and control which person is authorized to access what precord or what field.


Evolution of data managementModern Data Processing – Database approach



Separation of concernsModern Data Processing – Database approachSeparation of concerns

Physical storage, logical description and individual user requirements of database system can beuser requirements of database system can be considered independently provided that some mechanism exists to translate between them.mechanism exists to translate between them.



Advantages1. Centralized functionality leads to consistency in the y y

management of data, its storage, validation, security, processing and control.y, p g

2. Data can be shared reducing the need for redundant duplicationduplication

3. Reduction of unproductive maintenance incurred when the requirements of data and programs arewhen the requirements of data and programs are changed.



Advantages4. Separation of concerns allows a focused way of p y

considering data representations, requirements and processingp g

5. Separation of concerns simplifies the task for programmers developing user processesprogrammers developing user processes

6. Ad hoc querying becomes easily achievable because DBMS can translate a logical user query into theDBMS can translate a logical user query into the more complex physical requirements of underlying computer systems


computer systems


Disadvantages1. DBMS is a very complex system. Well-trained and y p y

experienced system managers and developers are required. q

2. Risks of failure and security breaches are often severe If the DBMS fails then it fails for all userssevere. If the DBMS fails, then it fails for all users and all user processes until the failure is corrected. A security breach potentially leaves all dataA security breach potentially leaves all data vulnerable to attack.



Disadvantages3. Hardware and software required to support modern q pp

DBMS can be expensive. This disadvantage has been reduced in recent years as the simultaneous yrise in processing power and data storage capabilities has put powerful computers onto most p p p pdesktops.


Evolution of data managementTypes of Database Systems

Three are more than 3 types of database systems:

Pre-relational systemPre relational system • Network database system• Hierarchical database system• Hierarchical database system

Relational system• Relational database system

P t l ti l tPost-relational system• Object-oriented database system


• XML database system

Evolution of data managementTypes of Database Systems - Network

• A network database consists of records and links. • Links are implemented by adding pointer fields to p y g p

records that are associated via a link. • Each record must have one pointer field for each linkEach record must have one pointer field for each link

with which it is associated.

• e.g. Total database system in 60’sIDMS (developed by Cullinane DB systems)( p y y )


Evolution of data managementTypes of Database Systems - Network

For example,

John …. …. 202-1-101 1000

Mary …. …. 202-1-133 3000

203-1-456 2000


Evolution of data managementTypes of Database Systems - Hierarchical

• A hierarchical database is very similar to network database.

• The only difference is that records are organized as collections of trees rather than as arbitrary graphs. y g p

• e.g. System 2000 • (developed by MRI Corp., acquired by Intel later)• e.g. IMS (developed by IBM)g ( p y )


Evolution of data managementTypes of Database Systems - Hierarchical

For example,

John MaryJohn …. …. Mary …. ….

202-1-133 1500 202-1-133 3000 203-1-456 2000202 1 133 1500 202 1 133 3000 203 1 456 2000


Evolution of data managementTypes of Database Systems - Relational

• A database in which you may imagine the data to be stored as a number of tables, and there are relationships between tables.

E h t bl i di id d i t l d• Each table is divided into columns and rows.

• Each table represents many occurrences of the sameEach table represents many occurrences of the same kind of information.


Evolution of data managementTypes of Database Systems - Relational

• All the values in any one column are the same data type (e.g. integer or character string)

• A table may impose a uniformity on the data for each occurrenceoccurrence.

• e g Oracle Sybase DB2 Informix Ingres• e.g. Oracle, Sybase, DB2, Informix, Ingres, …


Evolution of data managementTypes of Database Systems – Object oriented

• Objects/Packages with both data and behaviours• Encapsulationp

• -> Object-oriented DBMS (OODBMS)• -> Object-relational DBMS which supports object-

oriented features and mechanisms for internally represented objects within the relational environment.


Evolution of data managementTypes of Database Systems – XML

• XML (eXtensible Markup Language)• Use data tagging to indicate meaning of data within gg g g

free text document



• Standardization of XML schemas (collections of tags and their description) has been said to have enabled the e-Commerce and e-Business growth of the late 1990s and early 2000s.

• XML database systems grew out of the business needs to store and manipulate XML taggedneeds to store and manipulate XML-tagged documents. However, they lack formal data model.



• There is a lot of interest in the development of XML databases.

• Vendors are incorporating facilities for XML data management into existing DBMSs. g g

• Web community is developing standards for the representation storage and manipulation of XML-representation, storage and manipulation of XMLbased data.

• Further discussion in Block 5 of the course• Further discussion in Block 5 of the course.


Evolution of data managementSpecialist data processing systems

• Online Analytical Processing System (OLAP)Analytical Processingy gData WarehousingData MiningData Mining

• Geographic Information System (GIS)• Multimedia databases




Section 1. Evolution of data management Section 2. Data Management – why it mattersg y

Section 3. Database management system conceptsSection 4 Developing database systemsSection 4. Developing database systemsSection 5. Conceptual data model


Data ManagementCorporate information strategy

Corporate information strategy can be broken into I f ti S t St t• Information Systems Strategy

• Information Management Strategy• Information Technology Strategy



Information Systems Strategy• This strategy exploits what IT is required to support gy p q pp

business plans or create new business opportunities.• Will integrate information requirements and long-Will integrate information requirements and long

term overall business goals.



Information Management Strategy• This strategy determines how the IS strategy gy gy

developments are to be implemented within the organization.g

• It includes planning for change, organization and reorganization of corporate environmentreorganization of corporate environment.

• Will be responsible for the management structures that will control those departments that use andthat will control those departments that use and manage the technology and applications.



Information Technology Strategy• This strategy determines how best to achieve the gy

Information (IS) requirements within the Information Management (IM) structures. g ( )

• It forms a framework to support the analysis and design of the technology infrastructuredesign of the technology infrastructure.

• This will include computing purchase, data acquisition and management application authoringacquisition and management, application authoring, management and maintenance, networking and security implementation


security implementation.

Data ManagementCost of Data

Cost of data• System costs: development costs….System costs: development costs….• Origination costs: initial acquisition of data

C i ti t d li t• Communication costs: delivery costs• Execution costs: cost of effort involved for end

users in utilizing data, end user training, employee costs and possible additional equipment and software costs.


Data ManagementData Quality

Data Quality• Accuracy: whether the data values are accurateAccuracy: whether the data values are accurate• Completeness: whether data values are sufficient

for making decisionsfor making decisions.• Timeliness: whether data is available for user

ithi bl f tiwithin a reasonable of time• Relevance: whether the data is appropriate for use

by the right person or appropriately represented


Data ManagementData Quality

Data quality• Understandable: whether the meaning andUnderstandable: whether the meaning and

interpretation of data is clear to the users.• Trusted: whether adequate guarantees of security• Trusted: whether adequate guarantees of security,

privacy and ownership.




Section 1. Evolution of data managementSection 2. Data Management – why it matters g y

Section 3. Database management system conceptsSection 4 Developing database systemsSection 4. Developing database systemsSection 5. Conceptual data model


Database Management System


Database Management System Three-schema architecture


Database Management System Logical Schema

• It is the central component.

• It defines the logical properties of data in aIt defines the logical properties of data in a database, being concerned with a representation of data, and associated constraints, that is independent pof how it is stored.

• A database system has only one logical schema• A database system has only one logical schema, used by a DBMS to manage all the data in the databasedatabase

• “ What ” data is available in database


Database Management System Storage Schema

• It defines how a database is stored in files and accessed.However, a storage schema can only specify how data is to be stored if it is already defined by an associated logicalbe stored if it is already defined by an associated logical schema.

• Storage schema definitions must relate to those in a logicalStorage schema definitions must relate to those in a logical schema and the connection between two schemas is known as a mapping.

• A database system has only one storage schema, associated with only one logical schema, and is used by DBMS to d t i h d t i t d d d i th d t bdetermine how data is stored and accessed in the database.

• “ How ” data is stored


Database Management System External Schema

• It defines data for a user process, which the user process retrieves or updates by interaction with a DBMS. It is concerned with data as it appears outside the database, providing a kind of ‘window’ enabling access to the particular part of a database required by some user process.

• It can only specify how data is available to a user if th t d t h l d b d fi d bprocess if that data has already been defined by an

associated logical schema.


Database Management System External Schema

• External schema definitions must relate to those in a logical schema, and each external schema must have a mapping from its associated logical schema.

• For a relational database an external schema is• For a relational database, an external schema is defined as tables which contain data required by a user processuser process.

• A database system has more than one external yschema.


Database Management System Three-schema architecture

The three-schema architecture can provide the following two data independence:

1. Logical data independence2 Ph i l d i d d2. Physical data independence


Database Management System Logical data independence

Logical data independence

i h h l i l h h• It exists when a change to a logical schema has no impact on a user process.

• This is possible because each user process is dependent on an external schema and not directlydependent on an external schema and not directly on a logical schema.


Database Management System Logical data independence

Logical data independence

A h l i l h d ff h• A change to a logical schema does not affect the external schema if either it does not include the h d h h b k ichange data or the change can be taken into account

by a change to the mapping.

• It is important in enabling a database system to meet new requirements or extending the range ofmeet new requirements or extending the range of data in a database


Database Management System Physical data independence

Physical data independence• It exists when a change to a storage schema has noIt exists when a change to a storage schema has no

impact on a user process.

• This is possible because such a change does not even affect a logical schema, since the change can be taken into account by a change to the mapping for the storage schema.


Database Management System Physical data independence

Physical data independence• Thus the characteristics of files can be changed orThus the characteristics of files can be changed or

indexes to data can be added without affecting how user processes work.user processes work.


Database Management System

DBMS i ft th t id i f ti

Functions of DBMSDBMS is a software that can provide various functions

for management of database.F ti i l dFunctions include:

1. Data Definition (Data Definition Language, DDL)2 D t M i l ti (D t M i l ti L DML)2. Data Manipulation (Data Manipulation Language, DML)3. Transaction Support4 Concurrency Support (/ Concurrency Control)4. Concurrency Support (/ Concurrency Control) 5. Constraint Management6 Access Control6. Access Control7. Backup and Recovery8. Restructuring


g9. Reorganization

Database Management System Function of DBMS – Data Definition

• In response to a request from a user process, this function provides users with the capability for defining data and modifying data structure.

• Such requests includes a specification of some• Such requests includes a specification of some properties of data, for example, name and type of each column for all tableseach column for all tables.

• For example, p ,Create table (name char(30), sex char(1) )


Database Management System Function of DBMS – Data Manipulation

• This function provides the means to manipulate specific data in a database, which includes retrieving and updating data.

• The language used is call ‘Data manipulation g g planguage’.

• For example adding a new student record changingFor example, adding a new student record, changing residential address or deleting a record of a staff who has resigned.who has resigned.

• SQL: Insert into ….Update ….


Database Management System Function of DBMS – Transaction Support

• This function ensures that a sequence of updates to be executed not piecemeal but as a unit of work. i.e. atomic transaction => all or nothing are enforced.

• Two SQL command: Commit and Rollback

• For distributed transactions, 2-phase commit protocol is usedprotocol is used.


Database Management System Function of DBMS – Concurrency Support

• This function enables many user processes to access a database at the same time in a way that prevents them interfering with each other.

• For example lock a record first before a update is• For example, lock a record first before a update is performed.


Database Management System Function of DBMS – Constraint management

• In response to a request from a user process, this function provides users with the capability for defining constraints of data.

• Once a constraint has been defined DBMS will• Once a constraint has been defined, DBMS will ensure that data in a database never violates the constraint The constraint is enforcedconstraint. The constraint is enforced automatically.

• For example, sex must be ‘M’ or ‘F’; age must be between 10 and 20.


Database Management System Function of DBMS – Access Control

• This function involves defining users’ rights to access data in response

to a request that specifies in detail how data may be accessed by users

preventing users to access data without appropriate access rights

• For examples, only C.C, not tutor, can update your final

i i kexamination mark salesman is not allowed to access (read/write)

l i f i f llOUHK S359 Relational Databases: theory and practice T1 - 66

personal information of colleagues.

Database Management System Function of DBMS – Backup and Recovery

• This function restores a database to a usable state by using a backup copy whenever there may be loss and / or corruption of data due to hardware and software failures.

• It ensures that no completed transaction is lost, and so is essential to the persistence of dataso is essential to the persistence of data.


Database Management System Function of DBMS – Restructuring

• This function provides a mean to change the existing structure of database in order to meet changes in requirements.

• For example if one new column is added the• For example, if one new column is added, the logical schema and storage schema will be changedchanged


Database Management System Function of DBMS – Reorganisation

• This function provides a mean to change the existing storage schema.

• Reasons: 1 O i l i if f f1. Operational maintenance - if performance of

database falls below some acceptable standard. 2. Porting and Implementation maintenance - in

which DBMS and underlying computer system undergoes changes that require the storage implementation to be revised.


Database Management System Information system architectures

Some common distributed data processing architectures are:

• Client Server systems (Two-tier, Three-tier)• Client Multi-server systems• Distributed database systemDistributed database system• Replicated systems


Database Management System Client Server systems (Two-tier)

Client: User interface + Business application logic Server: Accepts and processes database requestsp p q


Database Management System Client Server systems (Three-tier)

Client: User interfaceApplication server: Business application logic pp pp gServer: Accepts and processes database requests


Database Management System Client Multi-server systems

Client: Accesses to several databases (managed by different DBMSs).


Database Management System Client Multi-server systems

Client: Needs to serially connect to multiple database servers and combine results.

This approach offers little of the management supportThis approach offers little of the management support that we require data as it fails to treat data collection as if it were a single database.


g

Database Management System Distributed database system

User processes do not have to know how data is distributed and so can express database requests as if the data was not distributed.



Local database has a local schema (logical, storage).

For distributed database there are two more schemasFor distributed database, there are two more schemas. Global schema: it describes all data accessible

i hi di ib d d bwithin distributed database Distributed schema: it defines where data is

located (i.e. in which local database the data is stored).



Distributed Database Management System (DDBMS) should

1. Present a single view of all data, since users should not have to worry where the data is actually storednot have to worry where the data is actually stored.

2. Support transactions across system boundaries and i i h i i f h di ib d d b Imaintain the integrity of the distributed database. It

must be able to recover from the failure of any i l i i i i iparticular computer participating in a transaction or

a failure in the communication network.



Distributed Database Management System (DDBMS) should( )

3. Provide security process that can protect the di t ib t d d t b f th i ddistributed database from unauthorized access.

4. Work the same way whether it uses mainframe or PC and independently of the characteristics of the interconnecting network.



Consideration of Distributed Database Design

• FragmentationFragmentation Horizontal fragmentation Vertical fragmentation Vertical fragmentation Combined fragmentation

• Replication Whether copies of data fragment are distributed toWhether copies of data fragment are distributed to other database servers so that each user process can perform all processing locally.


p p g y

Database Management System Replicated systems


Database Management System Replicated systems

Replication server: to receive or collect updates from• its local database and pass copies on to other p p

systems that require them;• remote systems via their replication servers andremote systems, via their replication servers and

apply them to the local database


Database Management System User interaction

There are different ways for users to interact with DBMS

1. Application Processes (i.e. programs with embedded SQL statements)embedded SQL statements)

2. Database tools for direct input of SQL statements p Q(e.g. Oracle SQL plus, Sybase ISQL )


~ END ~


Block 1 – Databases in context Evolution of data managementwilsonwlw2009.webs.com/S359-Block...

Documents

Transcript of Block 1 – Databases in context Evolution of data managementwilsonwlw2009.webs.com/S359-Block...