Lecture 12: File Management

Lecture 12: File ManagementLecture 12: File Management

10.2 Silberschatz, Galvin and Gagne ©2005Operating System Concepts – 7th Edition, Jan 1, 2005

File ManagementFile Management

provides the file abstraction for data storage guarantees data validity (most of the time) performance: throughput and response time enforces protection


File-System InterfaceFile-System Interface

File Concept Access Methods Directory Structure File-System Mounting File Sharing Protection


File ConceptFile Concept

Contiguous logical address space Persistent - stored on non-volatile memory (storage) Identified by (external) name

pathname in UNIX


File StructureFile Structure

None - sequence of words, bytes Simple record structure

Lines Fixed length Variable length

Complex Structures Formatted document Relocatable load file

Can simulate last two with first method by inserting appropriate control characters

Who decides: Operating system Program


File AttributesFile Attributes

Name – only information kept in human-readable form Identifier – unique tag (number) identifies file within file system Type – needed for systems that support different types Location – pointer to file location on device Size – current file size Protection – controls who can do reading, writing, executing Time, date, and user identification – data for protection, security,

and usage monitoring Information about files are kept in the directory structure, which is

maintained on the disk


File OperationsFile Operations

Create Write Read Reposition within file Delete Truncate Open(Fi) –open read/write session for Fi

File pointer: pointer to last read/write location, Close (Fi) – close read/write session


Access MethodsAccess Methods

Sequential Accessread nextwrite next resetno read after last write

(rewrite) Direct Access

read nwrite nposition to n

read nextwrite next

rewrite nn = relative block number


DirectoriesDirectories

Set of files Used to organize files Translates external file names to internal file names (file labels, i-

nodes, file control blocks) Treated as a file but with special operations


Operations Performed on DirectoryOperations Performed on Directory

Search for a file Create a file Delete a file List a directory Rename a file Traverse the file system


Organize the Directory (Logically) to ObtainOrganize the Directory (Logically) to Obtain

Efficiency – locating a file quickly Naming – convenient to users

Two users can have same name for different files The same file can have several different names

Grouping – logical grouping of files by properties, (e.g., all Java programs, all games, …)


Single-Level DirectorySingle-Level Directory

A single directory for all users

Naming problem

Grouping problem


Two-Level DirectoryTwo-Level Directory

Separate directory for each user

Path name Can have the same file name for different user Efficient searching No grouping capability


Tree-Structured DirectoriesTree-Structured Directories


Tree-Structured Directories (Cont)Tree-Structured Directories (Cont)

Efficient searching

Grouping Capability

Current directory (working directory) cd /spell/mail/prog type list


Tree-Structured Directories (Cont)Tree-Structured Directories (Cont)

Absolute or relative path name Creating a new file is done in current directory Delete a file

rm <file-name> Creating a new subdirectory is done in current directory

mkdir <dir-name>Example: if in current directory /mail

mkdir count

mail

prog copy prt exp count

Deleting “mail” deleting the entire subtree rooted by “mail”


Acyclic-Graph DirectoriesAcyclic-Graph Directories

Have shared subdirectories and files


Acyclic-Graph Directories (Cont.)Acyclic-Graph Directories (Cont.)

Two different names (aliasing)

If dict deletes list dangling pointer

Solutions: Backpointers, so we can delete all pointers

Variable size records a problem Backpointers using a daisy chain organization Entry-hold-count solution

New directory entry type Link – another name (pointer) to an existing file Resolve the link – follow pointer to locate the file


File System MountingFile System Mounting

A file system must be mounted before it can be accessed

A file system is mounted at a mount point (a directory)


(a) Existing. (b) Unmounted Partition(a) Existing. (b) Unmounted Partition


Mount PointMount Point


File SharingFile Sharing

Sharing of files on multi-user systems is desirable

Sharing may be done through a protection scheme

On distributed systems, files may be shared across a network

Network File System (NFS) is a common distributed file-sharing method


File Sharing – Multiple UsersFile Sharing – Multiple Users

User IDs identify users, allowing permissions and protections to be per-user

Group IDs allow users to be in groups, permitting group access rights


File Sharing – Remote File SystemsFile Sharing – Remote File Systems

Uses networking to allow file system access between systems Manually via programs like FTP Automatically, seamlessly using distributed file systems Semi automatically via the world wide web

Client-server model allows clients to mount remote file systems from servers Server can serve multiple clients Client and user-on-client identification is insecure or

complicated NFS is standard UNIX client-server file sharing protocol CIFS is standard Windows protocol Standard operating system file calls are translated into remote

calls


File Sharing – Failure ModesFile Sharing – Failure Modes

Remote file systems add new failure modes, due to network failure, server failure

Recovery from failure can involve state information about status of each remote request

Stateless protocols such as NFS include all information in each request, allowing easy recovery but less security


File Sharing – Consistency SemanticsFile Sharing – Consistency Semantics

Consistency semantics specify how multiple users are to access a shared file simultaneously Similar to Ch 7 process synchronization algorithms

Tend to be less complex due to disk I/O and network latency (for remote file systems

Andrew File System (AFS) implemented complex remote file sharing semantics

Unix file system (UFS) implements: Writes to an open file visible immediately to other users of

the same open file Sharing file pointer to allow multiple users to read and write

concurrently AFS has session semantics

Writes only visible to sessions starting after the file is closed


ProtectionProtection

File owner/creator should be able to control: what can be done by whom

Types of access Read Write Execute Append Delete List


Access Lists and GroupsAccess Lists and Groups

Mode of access: read, write, execute Three classes of users

RWXa) owner access 7 1 1 1RWXb) group access 6 1 1 0RWXc) public access 1 0 0 1

Ask manager to create a group (unique name), say G, and add some users to the group.

For a particular file (say game) or subdirectory, define an appropriate access.

owner group public

chmod 761 game

Attach a group to a file chgrp G game


Protection MechanismsProtection Mechanisms files are OS objects: unique names and a finite set of operations that

processes can perform on them protection domain is a set of {object,rights}, where right is the

permission to perform one of the operations at every instant in time, each process runs in some protection domain in Unix, a protection domain is {uid, gid} protection domain in Unix is switched when running a program with

SETUID/SETGID set or when the process enters the kernel mode by issuing a system call

how to store all the protection domains ?


Protection Mechanisms (cont’d)Protection Mechanisms (cont’d) Access Control List (ACL): associate with each object a list of all the protection domains that may access the object and how in Unix ACL is reduced to three protection domains: owner, group and others Capability List (C-list): associate with each process a list of objects that may be accessed along with the operations C-list implementation issues: where/how to store them (hardware, kernel, encrypted in user space) and how to revoke them


A Sample UNIX Directory ListingA Sample UNIX Directory Listing


Secondary Storage ManagementSecondary Storage Management

Space must be allocated to files Must keep track of the space available for allocation


PreallocationPreallocation

Need the maximum size for the file at the time of creation Difficult to reliably estimate the maximum potential size of the file Tend to overestimated file size to avoid running out of space


Methods of File Allocation (1)Methods of File Allocation (1)

Contiguous allocation Single set of blocks is allocated to a file at the time of creation A single entry in the file allocation table

Starting block and length of the file External fragmentation will occur



Chained allocation Allocation on basis of individual block Each block contains a pointer to the next block in the chain A single entry in the file allocation table

Starting block and length of file No external fragmentation Best for sequential files No accommodation for the principle of locality



Indexed allocation File allocation table contains a separate one-level index

for each file The index has one entry for each portion allocated to the

file The file allocation table contains block number for the

index


File AllocationFile Allocation contiguous: a contiguous set of blocks is allocated to a file at the time of

file creation

consolidation to improve locality

chained allocation: each block contains a pointer to the next one in the chain

good for sequential files file size must be known at the time of file creation external fragmentation

indexed allocation: good both for sequential and direct access (UNIX)


Free Space ManagementFree Space Management bitmap: one bit for each block on the disk

chained free portions: {pointer to the next one, length} index: treats free space as a file

good to find a contiguous group of free blocks small enough to be kept in memory


UNIX File SystemUNIX File System

Naming External/Internal names translation using directories

Lookup File blocks Disk blocks

Protection Free Space Management


File NamingFile Naming

External names (used by the application) Pathname: /usr/users/file1

Internal names (used by the OS kernel) I-node: file number/index on disk

superblock

File system on disk

I-node area( one I-node per file)

File-block area

01


DirectoriesDirectories

Files that store translation tables (external names to internal names)

usr 23

Root directory(always I-node 2)

users 41

usr

file1 87

usr users

/usr/users/file1 corresponds to I-node 87


File Content LookupFile Content Lookup

address table used to translate logical file blocks into disk blocks address table stored in the I-node

File withi-node 87

File System disk

0 1 2

45 65 85

address table456585


File ProtectionFile Protection

ACL with three protection domains (file owner, file owner group, others)

Access rights: read/write/execute Stored in the i-node


Free Space ManagementFree Space Management

Free I-nodes Marked as free on disk An array of 50 free i-nodes stored in the superblock

Free file blocks Stored as a list of 50- free block arrays First array stored in the superblock


In-Kernel File System Data StructuresIn-Kernel File System Data Structures

01 File system

on disk

OS Kernel

Buffer cache

I-node cachePer-OS Open File Table(offset in file, ptr to I-node)

Per-process Open File Table

PCBs

Application fd=open(pathname,mode); /* fd = index in Per-Proc OFT */for (..) read(fd,buf,size);close(fd);


File System Metadata Consistency ProblemFile System Metadata Consistency Problem file system uses the buffer cache for performance reasons two copies of a disk block (buffer cache, disk) -> consistency problem if the system crashes before all the modified blocks are written back to disk the problem is critical especially for the blocks that contain control information (meta-data): directory blocks, i-node, free-list Solution:

write through meta-data blocks (expensive) or order of write-backs is important ordinary file data blocks written back periodically (sync) utility programs for checking block and directory consistency after crash


File System MetadataFile System MetadataConsistency Problem: ExamplesConsistency Problem: Examples

Example 1: create a new file Two updates: (1) allocate a free I-node; (2) create an entry in the directory (1) and (2) must be write-through (expensive) or (1) must be written-back before (2) If (2) is written back first and a crash occurs before (1) is written back the directory structure is inconsistent and cannot be recovered

Example 2: write a new block to a file Two updates: (1) allocate a free block; (2) update the address table of the I-node (1) and (2) must be write-through or (1) must be written-back before (2) If (2) is written back first and a crash occurs before (1) is written back the I-node structure is inconsistent and cannot be recovered

Lecture 12: File Management

Documents

Transcript of Lecture 12: File Management