Lecture 01 - Intro to DBs

7/30/2019 Lecture 01 - Intro to DBs

1/46

1

.


2/46

22

... !


3/46

3

Database Prehistory

Data entry Storage and retrieval

Query processing Sorting


4/46

44

0

,

70- .

N N


5/46

55

0

0


6/46

66

:

1.

2.

3. /

4.

!

10

50

100

1000


7/46

7

File Management Systems

File is uninterpreted, unstructured collection of information

File operations: delete, catalog, create, rename, open,close, read, write, find,

Access methods: Algorithms to implement operations along

with internal file organization Examples: File of Customers, File of Students; Access

method: implementation of a set of operations on a file ofstudents or customers.


8/46

8

File Management System Problems

Data redundancy

Data Access: New request-new program

Data is not isolated from the access implementation

Concurrent program execution on the same file

Difficulties with security enforcement

Integrity issues


9/46

9

Concurrent Program Execution

Program1

What is the final value of the account AC?

Program2

AC=AC-50

AC=AC-100

AC #103 450


10/46

10

Security Problems

Allow access to the file only to the authorized personnel

Ability to restrict access to parts of the record

Ability to control operation usage by different users

Protection from unauthorized use

Protection from the derivation of unauthorized information


11/46

1111

:

,

:


12/46

1212


13/46

1313

( !),

: !!


14/46

14

What is a Database?

A set of information held in a computerOxford English Dictionary

One or more large structured sets ofpersistent data, usually associated withsoftware to update and query the data

Free On-Line Dictionary of Computing

A collection of data arranged for ease andspeed of search and retrieval

Dictionary.com


15/46

15

Databases

Web indexes

Library catalogues

Medical records

Bank accounts Stock control

Personnel systems

Product catalogues

Telephone directories

Train timetables

Airline bookings

Credit card details

Student records Customer histories

Stock market prices

Discussion boards

and so on


16/46

16

Database Systems

A database systemconsists of

Data (the database)

Software

Hardware

Users

We focus mainly onthe software

Database systemsallow users to

Store

Update

Retrieve

Organise

Protect

their data.


17/46

17

Database Users

End users

Use the databasesystem to achieve somegoal

Application developers Write software to allow

end users to interfacewith the databasesystem

DatabaseAdministrator (DBA)

Designs & manages thedatabase system

Database systemsprogrammer

Writes the databasesoftware itself


18/46

1818

, ,

,


19/46

1919

:

DDL (Data Defenition Language)

DML (Data Manipulation Language)

-


20/46

20

Database Management Systems (DBMSs)

Raw Resources (bare metal)

DBMS

Your Applications

Go HereDatabase abstractions

allow this interface to

be cleanly defined andthis allows applications

and data management

systems to be

implemented

separately.


21/46

21

What is a database system?

A database is a large, integratedcollection of data

A database contains a model of

something! A database management system

(DBMS) is a software system designed tostore, manage and facilitate access to thedatabase


22/46

22

What the DBMS does

Provides users with

Data definitionlanguage (DDL)

Data manipulation

language (DML) Data control language

(DCL)

Often these are all thesame language

DBMS provides

Persistence

Concurrency

Integrity

Security

Data independence

Data Dictionary

Describes the database

itself


23/46

23

Data Dictionary - Metadata

The dictionary orcatalog storesinformation about thedatabase itself

This is data aboutdata or metadata

Almost every aspectof the DBMS uses the

dictionary

The dictionary holds

Descriptions ofdatabase objects(tables, users, rules,

views, indexes,) Information about who

is using which data(locks)

Schemas and mappings

l f T d l b


24/46

24

Example of a Traditional Database

Application

Suppose we are building a system

to store the information about:

students courses

professors

who takes what, who teaches what


25/46

25

Can we do it without a DBMS ?

Sure we can! Start by storing the data infiles:

students.txt courses.txtprofessors.txt

Now write C or Java programs to implementspecific tasks


26/46

26

Doing it without a DBMS...

Enroll Mary Johnson in CSE444:

Read students.txt

Read courses.txt

Find&update the record Mary Johnson

Find&update the record CSE444Write students.txt

Write courses.txt

Write a C/Java program to do the following:


27/46

27

Problems without an DBMS...

System crashes:

What is the problem ?

Large data sets (say 50GB) Why is this a problem ?

Simultaneous access by many users Lock students.txt what is the problem ?

Read students.txtRead courses.txtFind&update the record Mary JohnsonFind&update the record CSE444Write students.txtWrite courses.txt

CRASH !


28/46

28

Enters a DBMS

Data files

Database server

(someone elses

C program)Applications

connection

(ODBC, JDBC)

Two tier system or client-server


29/46

29

Functionality of a DBMS

The programmer sees SQL, which has twocomponents:

Data Definition Language - DDL

Data Manipulation Language - DML query language

Behind the scenes the DBMS has:

Query engine Query optimizer

Storage management

Transaction Management (concurrency,

recovery)


30/46

30

Databases are a Rich Area for

Computer Science

Programming languages and softwareengineering (obviously)

Data structures and algorithms (obviously)

Logic, discrete maths, computation theory Some of todays most beautiful theoretical results are in

finite model theory--- an area derived directly fromdatabase theory

Systems problems: concurrency, operatingsystems, file organisation, networks, distributed

systems

Many of the concepts covered in this course are classical --- they form

the heart of the subject. But the field of databases is still evolving and

producing new and interesting research (hinted at in lectures 11 & 12).


31/46

31

Some systems to play with

1. mysql: www.mysql.org

Open source, quite powerful

2. PostgreSQL: www.postgresql.org Open source, powerful

3. MicrosoftAccess: Simple system, lots of nice GUI wrappers

4. Commercial systems: Oracle 10g (www.oracle.com)

SQL Server 2000 (www.microsoft.com/sql)

DB2 (www.ibm.com/db2)


32/46

32

Biology

Old way:

Wet lab chemistry

New way: Microarray

Search GenBank, Ensembl,GDB, SwissProt, Entrez using

BLAST, FASTA, GCG,EMBOSS


33/46

33

Astronomy

Old way:

Sign up for telescope time

New way: Sloan Digital Sky Survey

Systematically mapping of the entire sky

12 TB to date,

15 TB final in 2007


34/46

34

Oceanography

Old way:

Field work

Simplified Calculations

New way:

Finite Element Analysis

In situ sensors

CODAR


35/46

35

National Weather Service: Timeline 1849: Smithsonian Institution provides weather instruments to telegraph

operators

1900: Galveston Hurricane

1935: Long range forecasts; buoys

1955-1960: Computer forecasts scheduled regularly; weather satelliteTIROS I launched.

1979: AFOS Computer system is deployed, connecting all WeatherService forecast offices.

1988: Weather Service mobilizes local forecasting operation to assist infighting week-long wildfire in Yellowstone park

1990: NEXRAD Radar deployment project; a Cray supercomputerdeployed


36/46

36

National Weather Service

Data Collection Radar

Satellite

Forecasts

Bulletins Data Dissemination

Radio: aviation, marine, military channels

FTP, HTTP, email, RSS: public

Part of a UN sponsored Gobal network


37/46

37

National Weather Service: Network


38/46

38

The Gateway

NWS: Gateway

Public

Anonymous FTP

FTPMail

Family ofServices(Direct phone line)

http webservices

(XML/SOAP)

web form

emailftp

bulletins

RSS

radarsatellite

buoys

models

N ti l W th r S r i Pr d t


39/46

39

National Weather Service: Products

(1/2)

Computer Models

GRIB files from 10+ models from regional to global scale

Example:

SL.008001/ST.opnl/MT.ruc_CY.06/RD.20000622/PT.grid_DF.gr1/fh.0003x_tl.press

Facsimile/Images Text products derived from models

Special products in special formats

Text Products -

Warnings, outlooks, advisories, forecast, discussion

~100 different types

N tion l We ther Ser i e Prod ts


40/46

40

National Weather Service: Products

(2/2) Observed Data -

kept for 24 hours at least

observations from aviation, buoys, ships, balloons

special formats, but some have parsed them to XML

Radar Products -

Multicast by connecting a router directly to NWS as well as FTP SL.us008001/DF.of/DC.radar/DS.p19r1/SI.kfws/sn.0114

Satellite Products

Cloud Water Vapor, Cloud Liquid Water, Rain Rate, Sea IceConcentration, Sea Ice Age, Sea Ice Edge, Soil Moisture, SurfaceWind, Water Vapor over oceans, Surface Temperature, Snow Water

Content, Cloud Amount, and EDR Surface Type


41/46

41

National Weather Service: Radar

N i l W h S i F


42/46

42

National Weather Service: Forecasts

(1/3)

Several Climate Models:

Weather Research and Forecast (WRF)

Global Forecast System (GFS)

North American Mesoscale (NAM)

Nested Grid Model (NGM)

Specialized Models:

Fire Weather

Hurricane

Aviation

N i l W h S i F


43/46

43


(2/3)

National Digital Forecast Database

3 hr temporal resolution

5km spatial resolution

GRIB files, GIS map layers, data products

N i l W h S i F


44/46

44


(3/3)

Model Output Statistics (MOS)

Examples:

Max/Min Temperature Forecasts

Surface Temp / Dewpoint Forecasts

Opaque Cloud Amount

Probability of Precipitation

Severe weather probabilities

MOS products
http://www.nws.noaa.gov/mdl/forecast/graphics/MET/index.htmlhttp://www.nws.noaa.gov/mdl/forecast/graphics/MET/index.html


45/46

45

National Weather Service: Satellites

Geostationary Operational EnvironmentalSatellites

Variety of images and products
http://www.goes.noaa.gov/http://www.goes.noaa.gov/


46/46

46

National Weather Service: Summary

Domain?

Customers?

Architecture?

Interfaces?

Lecture 01 - Intro to DBs

Documents

Transcript of Lecture 01 - Intro to DBs