1 File Management The File Manager Interacting With File Manager File Organization Physical Storage...
-
Upload
shauna-lloyd -
Category
Documents
-
view
220 -
download
0
Transcript of 1 File Management The File Manager Interacting With File Manager File Organization Physical Storage...
1
File Management
• The File Manager
• Interacting With File Manager
• File Organization
• Physical Storage Allocation
• Data Compression
• Access Methods
• Levels in File Management System
• Access Control Verification Module
Fixed Length Contiguous
Records Storage
Non-contiguous
Storage
Variable Length
Records
Indexed
Storage
Sequential or Direct File Access
Powered by DeSiaMore
2
The File Manager
• File Manager controls every file in system which is a complex job.
• Efficiency depends on:– how system’s files are organized (sequential, direct, or
indexed sequential).– how they’re stored (contiguously, noncontiguously, or
indexed).– how each file’s records are structured (fixed-length or
variable-length).– how access to these files is controlled .
Powered by DeSiaMore
3
Responsibilities of File Manager
1. Track where each file is stored.
2. Determine where and how files will be stored.
– Efficiently use available storage space.
– Provide efficient access to files.
3. Allocate each file when a user has been cleared for access to it, then record its use.
4. Deallocate file when it is returned to storage.
– Communicate its availability to others waiting for it.
Powered by DeSiaMore
4
Important Definitions
• Field -- group of related bytes that can be identified by user with name, type, and size.
• Record -- group of related fields.
• File (flat file) -- group of related records that contains info used by specific application programs to generate reports.
• Database -- groups of related files that are interconnected at various levels to give flexible access to users.– Appears to File Manager to be a type of file.
Powered by DeSiaMore
5
Definitions - 2
• Program files contain instructions.
• Data files contain data.
• Directories -- listings of file names and their attributes.
• Every program and data file accessed by computer system, and every piece of computer software, is treated as a file.
• File Manager treats all files exactly same way as far as storage is concerned.
Powered by DeSiaMore
6
Interacting With File Manager
• Users communicates with File Manager via specific commands that may be either embedded in user’s program or submitted interactively by user.
• Embedded commands:
– OPEN & CLOSE pertain to availability of file for program invoking it.
– READ & WRITE are I/O commands.
– MODIFY – specialized WRITE command for existing data files that allows for appending/rewriting records.
Powered by DeSiaMore
7
Interactive Commands
• CREATE & DELETE -- deal with system’s knowledge of file.
• SAVE -- first time used, a file is actually created.
• OPEN NEW -- within a program indicates file must be created.
• OPEN…FOR OUTPUT -- creates file by making entry for it in directory & finding space for it in secondary storage.
• RENAME -- allows users to change name of existing file.
• COPY – allows user to make duplicate copies of existing files.
Powered by DeSiaMore
8
Commands Are Device-Independent
• Interface commands designed to be as simple as possible to use. – Lack detailed instructions to run device where file is stored.
– Device independent.
• To access a file, user doesn’t need to know its exact physical location on disk pack or storage medium.
• Each logical command broken down into sequence of low-level signals that – Trigger step-by-step actions performed by device.
– Supervise progress of operation by testing device’s status.
Powered by DeSiaMore
9
Typical Volume Configuration
• Each secondary storage unit (removable or non-removable) is considered a volume.
– Each volume can contain several files called multifile volumes.
– Some files are extremely large and are contained in several volumes called multivolume files.
• Generally, each volume in system is given name.
– File Manager writes name & other descriptive info on easy-to-access place on each unit.
Powered by DeSiaMore
10
Master File Directory (MFD)
• MFD stored immediately after volume descriptor– Lists names & characteristics of every file contained in volume. – File names refer to program files, data files, and/or system files. – Subdirectories, if supported.– Remainder of volume is used for file storage.
• Early OS supported only a single directory per volume. – Created by File Manager.– Contains names of files, usually organized in alphabetical, spatial,
or chronological order. – Simple to implement and maintain.– Some major disadvantages
Powered by DeSiaMore
11
Volume Descriptor
Creation Date Date when volume was created
Pointer to Directory Area Indicates first sector where directory is stored
Pointer to File Area Indicates first sector where file is stored
File System Code Used to detect volumes with incorrect formats
Volume Name User-allocated name
Powered by DeSiaMore
12
Some Major Disadvantages of Single Directory Per Volume
1. Takes long time to search for an individual file, especially if MFD was organized in an arbitrary order.
2. If user has many small files stored in volume, directory space fills before disk storage space fills. User told “disk full” when only directory full.
3. Users can’t create subdirectories to group related files. 4. Multiple users can’t safeguard files from other users
browsing file lists ‘cause entire directory listed on request. 5. Each program in entire directory needs unique name.
• E.g., Only 1 person using directory can name program PROG1.
Powered by DeSiaMore
13
About Subdirectories
• Semi-sophisticated File Managers create MFD for each volume with entries for files & subdirectories.
• Subdirectory created when user opens account to access computer.
– MFD entry flagged to indicate subdirectory with unique properties.
• Improvement from single directory scheme.
• Still can’t group files in a logical order to improve accessibility & efficiency of system.
Powered by DeSiaMore
14
Subdirectories Can Be Implemented As an Upside-down Tree
• Today’s File Managers allow users to create subdirectories so related files are grouped together. – Extension of previous two-level directory structure.
• Tree structures allow system to efficiently search individual directories due to fewer entries in each.
• Path to requested file may lead through several directories.
• When user wants to access specific file, file name is sent to File Manager. File Manager searches MFD for user's directory. Then searches user's directory & any subdirectories for requested file & location.
Powered by DeSiaMore
15
File Descriptor
Each file entry in every directory contains info describing file:1. File name—usually represented in ASCII code.2. File type—organization and usage that are dependent on system (e.g.,
Files and directories).3. File size—size is kept here for convenience.4. File location—identification of first physical block (or all blocks)
where file is stored.5. Date and time of creation.6. Owner.7. Protection information—access restrictions based on who is allowed
to access file and what type of access is allowed.8. Record size —its fixed size or its maximum size, depending on type
of record
Powered by DeSiaMore
16
File Names
• Absolute file name (complete file name) – long name that includes all path info.
• Relative file name – short name seen in directory listings.– Selected by user when file is created.– E.g., ACCOUNT ADDRESSES, TAXES 2001, or AUTOEXEC.
• Extension – 2-3 character name used to identify type of file or its contents. – Separated from relative name by a period.– E.g., CPP, BAS, BAT, COB, & EXE signal to system to use
specific compiler or program to run these files. – E.g., TXT, DOC, OUT, MIC, & KEY created by applications or
by users for own identification.
Powered by DeSiaMore
17
File Naming Conventions
• Can vary in length from 1 or more characters.
• Can include letters of alphabet & digits.
• Every OS has specific rules that affect length of relative name & types of characters allowed.
– E.g., MS-DOS allows 1-8 alphanumeric character names without spaces.
– More modern OS allow names with dozens of characters including spaces.
• Try to select descriptive relative names that readily identify file contents/purpose of file.
Powered by DeSiaMore
18
Base and Current Directories Used by File Manager to Locate Files
• File Manager selects base directory for user when interactive session begins.– All file operations requested by that user start here.
• Then, user selects subdirectory (current directory or working directory). – Thereafter, files presumed to be located in current directory.
• Whenever file accessed, user types in relative name & File Manager adds proper prefix.
• As long as users refer to files in working directory, can access them without entering complete name.
Powered by DeSiaMore
19
File Organization : Record Format
1. Fixed-length records – easiest to access directly.– Most common type & ideal for data files. – Record size critical (too small – truncation; too large – wastes
space).
2. Variable-length records -- difficult to access directly because hard to calculate exactly where record is located.– Don’t leave empty storage space & don’t truncate any characters.– Frequently used in files accessed sequentially (e.g,. text files,
program files) or files using index to access records. – File descriptor stores record format, how it’s blocked, & other
related info.
Powered by DeSiaMore
20
Physical File Organization
• Concerned with how records are arranged & characteristics of medium used to store it.
• On magnetic disks, files can be organized as:
1. Sequential
2. Direct
3. Indexed sequential.
Powered by DeSiaMore
21
Characteristics Considered When Selecting File Organization
• Volatility of data—frequency with which additions & deletions made.
• Activity of file—% records processed during a given run.
• Size of file.
• Response time—amount of time user is willing to wait before requested operation is completed.
Powered by DeSiaMore
22
Sequential Record Organization
• Easiest to implement because records are stored & retrieved serially, one after other.
• To speed process some optimization features may be built into system.
– E.g., select a key field from record & then sort records by that field before storing them.
– Aids search process.
– Complicates maintenance algorithms because original order must be preserved every time records added or deleted.
Powered by DeSiaMore
23
Direct Record Organization (Random Organization)
• Uses direct access files which can be implemented only on direct access storage devices.
• Give users flexibility of accessing any record in any order without having to begin search from beginning of file.
• Records are identified by their relative addresses (their addresses relative to beginning of file).
– Logical addresses computed when records are stored & again when records are retrieved.
– Use hashing algorithms.
Powered by DeSiaMore
24
Advantages of Direct Access Organization
• Fast access to records.
• Can be accessed sequentially by starting at first relative address & incrementing it by one to get to next record.
• Can be updated more quickly than sequential files because records quickly rewritten to original addresses after modifications.
• No need to preserve order of the records, so adding or deleting them takes very little time.
Powered by DeSiaMore
25
Collisions Are a Problem With Direct Access Organization
• Several records with unique keys may generate same logical address (collision).
• Program generates another logical address before presenting it to File Manager for storage.
• Colliding records stored in overflow area via links.
• File Manager handles physical allocation of space.
• Maximum file size established when created & eventually file is full or too many records are stored in overflow area.
• Programmer must reorganize & rewrite file.
Powered by DeSiaMore
26
Indexed Sequential Record Organization
• Combines best of sequential & direct access. • Created & maintained through Indexed Sequential Access
Method (ISAM) software package.• Doesn’t create collisions because it doesn’t use result of
hashing algorithm to generate a record’s address. – Uses info to generate index file through which records retrieved.
• Divides ordered sequential file into blocks of equal size. – Size determined by File Manager to take advantage of physical
storage devices & to optimize retrieval strategies.
• Each entry in index file contains highest record key & physical location of data block where this record, & records with smaller keys, are stored.
Powered by DeSiaMore
27
Indexed Sequential - 2
• To access any record in file, system begins by searching index file & then goes to physical location indicated at that entry.
• Overflow areas are spread throughout file
– Existing records can expand & new records are in close physical & logical sequence.
– Last-resort overflow area is located apart from main data area but is used only when the other overflow areas are completely filled.
• When retrieval time becomes too slow, file has to be reorganized..
• Allows both direct access to a few requested records & sequential access to many records for most dynamic files.
• A variation of indexed sequential files is B-tree.
Powered by DeSiaMore
28
Physical Storage Allocation
• File Manager must work with files not just as whole units but also as logical units or records.
• Records within file must have same format but can vary in length.
• Records are subdivided into fields.
– Structure usually managed by application programs, not OS.
• When we talk about file storage, we’re actually referring to record storage .
Powered by DeSiaMore
29
R1 R2 R3 R4 R5 R6
Block # R1 R2 R3 Block 1 Recs 2
R1 R1 R2 R2 Length Length
Block Block # R1 R1 R2 R2 1 Size Recs. Len. Len.
R1 # R2 # R3
(a) Unblocked, fixed-length records
(b) Blocked, fixed length records
(c) Unblocked, variable-length records
(d) Unblocked, variable-length records
(e) Blocked, variable-length records
Powered by DeSiaMore
30
Contiguous Storage
• Records stored one after other. • Any record can be found & read once starting address &
size are known, so directory is very streamlined. • Direct access easy – every part of file is stored in same
compact area.• Files can’t be expanded unless there’s empty space
available immediately following it. – Room for expansion must be provided when file is created.
• Fragmentation occurs (slivers of unused storage space).– Can compact & rearrange files. – Files can’t be accessed while compaction is taking place.
Powered by DeSiaMore
31
Noncontiguous Storage
• Allows files to use any storage space available on disk.
• File’s records are stored in a contiguous manner if enough empty space.
• Any remaining records, & all other additions to file, are stored in other sections of disk (extents). – Linked together with pointers. – Physical size of each extent is determined by OS (e.g.,
256 bytes).
Powered by DeSiaMore
32
Linking File Extents
1. Linking at storage level – each extent points to next one in sequence.
– Directory entry consists of file name, storage location of first extent, location of last extent, & total number of extents, not counting first.
2. Linking at directory level – each extent listed with its physical address, size, & pointer to next extent.
• A null pointer indicates that it's last one.
• Eliminate external storage fragmentation & need for compaction.
• Don’t support direct access because no easy way to determine exact location of specific record.
Powered by DeSiaMore
33
Indexed Storage
• Allows direct record access by bringing pointers linking every extent of that file into index block.
• Every file has its own index block (addresses of each disk sector that make up the file)
– Lists each entry in same order in which sectors linked .
• When a file is created, pointers in index block set to null.
• As each sector is filled, pointer set to appropriate sector address.
– Address is removed from empty space list & copied into its position in index block.
Powered by DeSiaMore
34
Indexed Storage - 2
• Supports both sequential & direct access.
• Doesn’t necessarily improve use of storage space because each file must have index block.
• For larger files with more entries, several levels of indexes can be generated.
– To find a desired record, File Manager accesses first index (highest level), which points to a second index (lower level), which points to an even lower level index & eventually to data record.
Powered by DeSiaMore
35
Data Compression
• Several techniques (3) used to save space in files. • System must be able to distinguish between compressed &
uncompressed data.• Trade-off: storage space gained, but processing time lost.
1. Records with repeated characters can be abbreviated. – E.g., fixed-length field with short name & many blank characters;
replaced with variable-length field & special code to indicate # blanks truncated.
ADAMSbbbbbbbbbb ADAMSb10 300000000 3#8
Powered by DeSiaMore
36
Data Compression: Repeated Terms
2. Repeated terms compressed by using symbols to represent each of most commonly used words in the database.
– E.g., in a university’s student database common words like student, course, teacher, classroom, grade, & department could each be represented with single character.
Powered by DeSiaMore
37
Data Compression : Front-end Compression
3. Front-end compression used for index compression.
– For example, student database where the students’ names are kept in alphabetical order could be compressed
Original list Compressed list
Smith, Betty Smith, Betty
Smith, Gino 7Gino
Smith, Donald 7Donald
Smithberger, John 5berger, John
Smithbren, Ali 6ren, Ali
Smithco, Rachel 5co, Rachel
Smither, Kevin 5er, Kevin
Smithers, Renny 7s, Renny
Snyder, Katherine 1nyder, Katherine
Powered by DeSiaMore
38
Access Methods
• Access methods dictated by a file’s organization
• Most flexibility is allowed with indexed sequential files and least with sequential.
– File organized in sequential fashion can support only sequential access to its records, & these records can be of fixed or variable length.
– File Manager uses the address of last byte read to access the next sequential record.
– Current byte address (CBA) must be updated every time a record is accessed.
Powered by DeSiaMore
39
Sequential Access
• For sequential access of fixed-length records, CBA updated by incrementing it by record length (RL), which is constant:
CBA = CBA + RL
• For sequential access of variable-length records, File Manager adds length of record (RLk) plus number of bytes
used to hold record length (N) to CBA.
CBA = CBA + N + RLk
Powered by DeSiaMore
40
Direct Access & Fixed-Length Records
• If file is organized in direct fashion, accessed easily in direct or sequential order if have fixed-length records.
• For direct access with fixed length records, CBA computed directly from record length & desired record number RN (info provided through READ command) minus one:
CBA=(RN–1) * RL
Powered by DeSiaMore
41
Direct Access & Variable-Length Records
• Virtually impossible to access a record directly because address of desired record can’t be easily computed.
• To access a record, File Manager must do sequential search through records.
– If File Manager saves address of last record accessed, can do half-sequential read through file. When next request arrives it could search forward from CBA.
– Or File Manager can keep table of record numbers & their CBAs. Search table for exact storage location of desired record.
• To avoid this problem, many systems force users to have files organized for fixed-length records if want direct access to records.
Powered by DeSiaMore
42
Access of Records in Indexed Sequential File
• Accessed either sequentially or directly,
• Either CBA computations apply but with one extra step.– Index file must be searched for pointer to block where data stored.
– Because index file is smaller, kept in main memory & quick search to locate block where desired record is located.
– Block retrieved from secondary storage & beginning byte address
of record calculated.
• In systems with several levels of indexing, index at each level must be searched before computing CBA. – Entry point to this type of data file is usually through index file.
Powered by DeSiaMore
43
Levels in a File Management System
• Efficient management of files can’t be separated from efficient management of devices that house them.
• A wide range of functions must be organized for I/O system to perform efficiently.
• Each level implemented by using structured & modular programming techniques, which also set up a hierarchy.
Basic File System
Access Control Module
Logical File System
Physical File System
Device Interface Module
Device
Powered by DeSiaMore
44
Basic File System
• Highest level module that passes info to logical file system, which notifies physical file system, which works with Device Manager.
• Activates access control verification module to verify that this user is permitted to perform this operation with this file.
Powered by DeSiaMore
45
Access Control Verification Module
• Any file can be shared.
• Saves space & allows for synchronization of data updates.
• Improves efficiency of system's resources, because if files are shared in main memory, I/O operations reduced.
• However, integrity of each file must be safeguarded
– Control over who is allowed to access file and what type of access is permitted.
– READ only, WRITE only, EXECUTE only, DELETE only, or some combination.
Powered by DeSiaMore
46
File Access Control Methods
• Each file management system has own file access control method.
1. Access control matrix
2. Access control lists Most
3. Capability lists Common Methods
4. Lockword control.
Powered by DeSiaMore
47
Access Control Matrix
• Intuitively appealing & easy to implement.
• Works well only for systems with few files & few users.
• In matrix each column identifies a user & each row identifies a file.
• Intersection of row & column has access rights for that user to that file. User 1 User 2 User 3 User 4 User 5
File 1 RWED R-E- ---- RWE- --E-
File 2 ---- R-E- R-E- --E- ----
File 3 ---- RWED ---- --E- ----
File 4 R-E- ---- ---- ---- RWED
File 5 ---- ---- ---- ---- RWED
R = Read Access
W = Write Access
E = Execute Access
D = Delete Access
- = Access Not Allowed
Powered by DeSiaMore
48
Access Control Lists
• Modification of access control matrix technique.
• Each file is entered in list & contains names of users allowed to access it & type of access permitted.
• To shorten list, only those who may use file are named; those denied any access are grouped under global heading such as WORLD.
• Or shorten by putting every user into a category:
– SYSTEM – system personnel with unlimited access to all files.
– OWNER – absolute control over all files created in own account.
– GROUP – all users belonging to appropriate group have access.
– WORLD – all other users in system; default access types given by File Manager.
Powered by DeSiaMore
49
Access Control List Example
File Access
File 1 USER1 (RWED), USER2 (R-E-), USER4 (RWE-),
USER5 (--E-), WORLD (----)
File 2 USER2(R-E-), USER3 (R-E-), USER4 (--E-), WORLD (-
---)
File 3 USER2(RWED), USER4 (--E-), WORLD (----)
File 4 USER1(R-E-), USER5(RWED), WORLD(----)
File 5 USER5(RWED), WORLD (----)
Powered by DeSiaMore
50
Capability Lists
• Lists every user and files to which each has access.
• Requires less storage space than an access control matrix.
• Easier to maintain than an access control list when users are added or deleted from system.
User Access
User1 File1 (RWED), File4 (R-E-)
User2 File1 (R-E-), File2 (R-E-), File3 (RWED)
User3 File2 (R-E-)
User4 File1 (RWE-), File2 (--E-), File3 (--E-)
User5 File1 (--E-), File4 (RWED), File5 (RWED)
Powered by DeSiaMore
51
Lockword Control
• Lockword is similar to a password but protects a single file. – When file created, owner protects it via lockword– Stored in directory but isn’t revealed with directory listing. – User must provide correct lockword to access protected file.
• Require smallest amount of storage for file protection. • Can be guessed by hackers or passed on to unauthorized
users. • Generally doesn’t control type of access to file.
– Anyone who knows lockword can read, write, execute, or delete file.
Powered by DeSiaMore
52
Terminology
• access control list• access control matrix• capability list• complete file name• current byte address (CBA)• current directory• data compression• data file• database• device independent• direct access files• direct record organization
• directory• extension• extents• file• file descriptor• fixed-length record• hashing algorithm• indexed sequential record
organization• key field• lockword• logical address
Powered by DeSiaMore
53
Terminology - 2
• logical address
• master file directory (MFD)
• relative address
• relative file name
• sequential record organization
• subdirectory
• variable-length record
• volume
• working directory
Powered by DeSiaMore