File Systems CNS 3060 Operating Systems. TopicsTopics Files Directories File system implementation...

File SystemsFile SystemsFile SystemsFile SystemsCNS 3060CNS 3060

OperatingSystems

TopicsTopicsTopicsTopics

Files Directories File system implementation Example file systems

OperatingSystems

Long-term Long-term Information StorageInformation Storage

Long-term Long-term Information StorageInformation Storage

Must store large amounts of data

Information stored must survive the termination of the process using it

Multiple processes must be able to access the information concurrently

Must store large amounts of data

Information stored must survive the termination of the process using it

Multiple processes must be able to access the information concurrently

OperatingSystems

File StructureFile StructureFile StructureFile Structure

Three kinds of files byte sequence (O/S doesn’t care what’s inside) record sequence (O/S understands record structure) tree (Database systems)

Three kinds of files byte sequence (O/S doesn’t care what’s inside) record sequence (O/S understands record structure) tree (Database systems)

OperatingSystems

o read all bytes/records from the beginningo cannot jump around, could rewind or back up. o convenient when medium was mag tape

Sequential AccessSequential AccessSequential AccessSequential Access

OperatingSystems

Random AccessRandom AccessRandom AccessRandom Access

o bytes/records read in any order – disk driveso essential for data base systemso read can be … - move file marker (seek), then read or … - read and then move file marker

o bytes/records read in any order – disk driveso essential for data base systemso read can be … - move file marker (seek), then read or … - read and then move file marker

OperatingSystems

Typical File AttributesTypical File AttributesTypical File AttributesTypical File AttributesOperatingSystems

File OperationsFile OperationsFile OperationsFile Operations1. Create2. Delete3. Open4. Close5. Read6. Write

1. Create2. Delete3. Open4. Close5. Read6. Write

7. Append8. Seek9. Get attributes10. Set Attributes11. Rename

7. Append8. Seek9. Get attributes10. Set Attributes11. Rename

OperatingSystems

Memory Mapped FilesMemory Mapped FilesMemory Mapped FilesMemory Mapped Files

In some cases, it is convenient to map a file into the address space of a running process. File access is then done by normal reads and writes of memory. The result is much faster and to some, much easier than actual writing to the file.

Memory mapping is done by changing the system’s internal tables so that the file becomes backing store (ala paging) for the memory region into which the file is mapped.

OperatingSystems

Hierarchical Directory Hierarchical Directory SystemsSystems

Hierarchical Directory Hierarchical Directory SystemsSystems

Root

A B C

A

C C

B

C

B

B B

C C

C

OperatingSystems

Path NamesPath NamesPath NamesPath Names

A UNIX directory tree

A UNIX directory tree

OperatingSystems

Directory OperationsDirectory OperationsDirectory OperationsDirectory Operations

1. Create2. Delete3. Opendir4. Closedir

1. Create2. Delete3. Opendir4. Closedir

5. Readdir6. Rename7. Link8. Unlink

5. Readdir6. Rename7. Link8. Unlink

OperatingSystems

A File System A File System ImplementationImplementationA File System A File System

ImplementationImplementation

A possible file system layoutWhen the system is started, the BIOS reads in and executes the Master Boot Record (MBR). The MBR locates the active partition and finds it’s boot block. The boot block loads the O/S.

A possible file system layoutWhen the system is started, the BIOS reads in and executes the Master Boot Record (MBR). The MBR locates the active partition and finds it’s boot block. The boot block loads the O/S.

sector 01 partition is marked as active

OS Loader

OperatingSystems

superblock i-node table data

The superblock contains informationabout the file system itself, for example,how big is the file system


This is an array of i-node structures. An i-nodeis identified by it’s position in the array.

I-NodesI-NodesI-NodesI-Nodes

When a file is opened, it’s i-node is loaded from disk into memory. Thus only a small amount of memory is required, and only while the file is opened.

I-nodes are fixed in size.

OperatingSystems


All directories and files are stored in the data areaof the file system. Everything in the file system is stored in “blocks ”. Blocks are fixed in size and representthe smallest unit of storage in the file system.

Creating a new file involves the following operations:


1. The O/S finds an unused i-node

47



1. The kernel finds an unused i-node2. The kernel stores file attributes in the i-node

attributes

47



1. The kernel finds an unused i-node2. The kernel stores file attributes in the i-node3. The kernel find free blocks and stores the file data

attributes

20

0

63

5

82

1

||||

||||

|

||||

||||

|

||||

||||

|

47



1. The kernel finds an unused i-node2. The kernel stores file attributes in the i-node3. The kernel find free blocks and stores the file data4. The kernel stores the block number in the i-node

attributes

20

0

63

5

82

1

||||

||||

|

||||

||||

|

||||

||||

|

200

635

821

47



1. The kernel finds an unused i-node2. The kernel stores file attributes in the i-node3. The kernel find free blocks and stores the file data4. The kernel stores the block number in the i-node5. The kernel adds an entry to the directory

attributes

20

0

63

5

82

1

||||

||||

|

||||

||||

|

||||

||||

|

200

635

821

47

47 myFile.txt

TerminologyTerminologyTerminologyTerminologyTrack

Read/writehead

SectorBlock or cluster

A block is the minimum unit of storage for data. Block sizes are defined by the O/S and the file system.

Internal Fragmentation: Occurs when all of a block is not used by a file.

External Fragmentation:Occurs when blocks usedto store a file are notcontiguous.

OperatingSystems

ProblemProblemProblemProblemTrack

Read/writehead

SectorBlock or cluster

A block is the minimum unit of storage for data. Block sizes are defined by the O/S and the file system.

OperatingSystems

Given that you need n blocks on the diskto hold the contents of a file, how do youallocate those blocks to the application?

1

23

4

Contiguous File Contiguous File AllocationAllocation


The simplest file allocation scheme is to take blocks sequentially from the disk, as they are needed for each file. This has two major advantages:

o It is simple to implement. You only need to keep track of the starting block and the number of blocks in the file.

o It is very efficient. Only one seek is required to read in the entire file. (a seek is the operation that moves the read/write head over the correct track.)

OperatingSystems

ProblemProblemProblemProblemTrack

Read/writehead

SectorBlock or clusterOperatingSystems

1

2

4

Create a file of 3 blocks


OperatingSystems




OperatingSystems





Now . . . Delete the 2nd file

What’s the problem with this design?

External Disk Fragmentation.


OperatingSystems

you have to have a filethat is 5 blocks or less

to fit here!

Linked List AllocationLinked List AllocationLinked List AllocationLinked List Allocation

+ Every block on disk can be used. Disk blocks can be anywhere.+ The directory only need store the address of the first block of the file.- Each block sacrifices the space required to store the pointer- Random access of blocks in the file is slow.

OperatingSystems

OperatingSystems

1

2 3

4

what if you want to randomly access this block?

Random Access

OperatingSystems

1

2 3

4

So ... you have to access this block,because it contains the pointer

to the next block.The directory only contains theaddress of the first block.

OperatingSystems

1

2 3

4

Now you need to access this blockto get the location of the 3rd block

. . . but this will more than likely involvea disk seek, i.e. move the disk head

A File Allocation Table (fat)A File Allocation Table (fat)A File Allocation Table (fat)A File Allocation Table (fat)

The fat table usually resides in a fixed location at the beginning of on the disk.

No space is taken up in the file for pointers.

OperatingSystems

OperatingSystems

1

2 3

4

Why cache the fat?

fat

move the disk head toread the first entry in

the fat

OperatingSystems

1

2 3

4

Why cache the fat?

fat

now move the disk head toread in the first block of

the file.

OperatingSystems

1

2 3

4

Why cache the fat?

fat

move the disk head backto read the next entry in

the fat

OperatingSystems

1

2 3

4

Why cache the fat?

fat

Move the disk head toread in the next block

in the file

When the fat is in cacheWhen the fat is in cacheWhen the fat is in cacheWhen the fat is in cache

+ Random access is easier – the chain is entirely in memory - The biggest disadvantage is that the FAT resides in memory assume a 20GB disk with 1024KB block-size. The FAT needs 20 million entries (60-80MB)

OperatingSystems

Unix uses i-nodes!

OperatingSystems

Directory ImplementationsDirectory ImplementationsDirectory ImplementationsDirectory Implementations

A simple Directory* File attributes stored in the directory* Disk address stored in the directory (first block)* Fixed size entries (so fixed length file names)

(MS/DOS & Windows3.x)

mail attributes

attributes

attributes

attributes

attributes

disk address

disk address

disk address

disk address

disk address

games

homework

music

photos

OperatingSystems

Directory ImplementationsDirectory ImplementationsDirectory ImplementationsDirectory Implementations

Each directory entry pointsto an i-node. File attributes are stored in the i-node.

(Unix)

mail address of i-node

games

homework

music

photos

address of i-node

address of i-node

address of i-node

address of i-node

Handling Long File Handling Long File NamesNames


Fragmentation Issues(take a directory entry out)Page Faults may occur(directory spans multiple pages)

OperatingSystems


Handling Long File Handling Long File NamesNamesOperating

Systems

File SharingFile Sharing

Unix allows different processes to share files …

The Process Table: every process has an entry in the process table includes all open file descriptors owned by the process - file descriptor flags - a pointer into the file tableprocess table entry

fd flags ptr

fd 0:fd 1:fd 2:

...

The File Table (per process) table of all open files - status flags for the file (read, write, append, etc) - the current file offset - a pointer to the i-node for this file

process table entry

fd flags ptr

fd 0:fd 1:fd 2:

...

file table entry

file status flags

current file offset

i-node pointer

The i-node Table one for each open file read from disk when the file is opened includes a pointer to the file’s i-node - file permissions - file owner - file size - device file is physically located on - pointers to the actual file blocks on disk - etc

process table entry

fd flags ptr

fd 0:fd 1:fd 2:

...

file table entry

file status flags

current file offset

i-node pointer

i-node

permissions

user & group ids

File size

Time stamps

. . .

Pointer to first disk block

process table entry

fd flags ptr

fd 0:fd 1:fd 2:

...

file table

file status flags

current file offset

i-node pointer

file status flags

current file offset

i-node pointer

i-node

i-node

A Single Process With Two Open FilesA Single Process With Two Open Files

permissions

user & group ids

File size

Time stamps

. . .


permissions

user & group ids

File size

Time stamps

. . .


process table entry

fd flags ptr

fd 0:fd 1:fd 2:

...

file table

file status flags

current file offset

v-node pointer

file table

file status flags

current file offset

v-node pointer

i-node

process table entry

fd flags ptr

fd 0:fd 1:fd 2:

...

Two Processes Sharing the Same FileTwo Processes Sharing the Same File

Note that as each process writes to the file, the file offset isupdated in the file table for that process, to reflect the number of bytes written. If this causes the file offset to exceed the filesize, the file size is updated in the files i-node.

permissions

user & group ids

File size

Time stamps

. . .


Sharing FilesSharing FilesSharing FilesSharing Files

This file appears in User B’s directory aswell as in User C’s directory.

This file appears in User B’s directory aswell as in User C’s directory.

In the simple directory case, both directoriesmust contain the block addresses for the file.

If user B adds to the file, the appended diskblocks will not show up in C’s directory.

This problem is solved using i-nodes, sinceeach directory entry only need point to thei-node.

OperatingSystems

B’s directory

i-node

file

C ’s directory

data

OperatingSystems

Deleting a Shared FileDeleting a Shared FileDeleting a Shared FileDeleting a Shared File

User C creates a file

i-node

User C’sdirectory

OperatingSystems

i-node owner = Ccount = 1


User B links to the shared file.

i-node

User C’sdirectory

OperatingSystems


User B’sdirectory

What do you do whenUser C deletes the file?

You can’t delete the file,Because the B’s Directory will point to an invalid i-node.


User B links to the shared file.

i-node

User C’sdirectory

OperatingSystems


User B’sdirectory

What do you do whenuser C deletes the file?

You can’t delete the file,because the B’s Directory will point to an invalid i-node.

All you can do is delete C’s directoryEntry and leave the file. This works, but may cause accounting problems.

Using what is called a symbolic link,B links to one of C’s files by creating anew file type called a Link. The Link filejust contains the path name of the fileit links to.

Using Symbolic LinksUsing Symbolic LinksUsing Symbolic LinksUsing Symbolic LinksOperating

Systems

User C’sdirectory

i-node type = file

User B’sdirectory

i-nodetype = link

datafile path file

Now, when C deletes the file,the i-node pointed to by B’sdirectory is valid. However,attempts to access the file will fail.

Using Symbolic LinksUsing Symbolic LinksUsing Symbolic LinksUsing Symbolic LinksOperating

Systems

User C’sdirectory

User B’sdirectory

i-nodetype = link

path file???

Disk Space Disk Space ManagementManagement

Block SizeBlock Size

Disk Space Disk Space ManagementManagement

Block SizeBlock Size

As we have seen, disk space is usually managed in fixed size blocks. The question arises, how big should the blocks be?

If the block is too large, then a lot of space is wasted.

If the block size is too small, then performance suffersbecause reading each block requires a rotational delayand a seek.

OperatingSystems

Determining Optimal Block Size

Determining Optimal Block Size

• Dark line (left hand scale) gives the data rate of a disk• Dotted line (right hand scale) gives disk space efficiency• Note that disk space utilization and data rate are in direct conflict!

• Dark line (left hand scale) gives the data rate of a disk• Dotted line (right hand scale) gives disk space efficiency• Note that disk space utilization and data rate are in direct conflict!

for 2KB files

wastedspace

data rate is almost completelydominated by seek time androtational delay of the disk.

Block Size

Managing Free Disk Managing Free Disk BlocksBlocks

Managing Free Disk Managing Free Disk BlocksBlocks

Linked Listbit map

n bytes to hold a disk block number 1 bit per blockFixed size

OperatingSystems

MS-DOS File SystemMS-DOS File SystemMS-DOS File SystemMS-DOS File SystemThe MS-DOS File System was patterned after

the CP/M File System

Directory entry – fixed 32 byte length

File Name

8

Ext

3

Attribute• read-only• archive• hidden• system file

bytes 1

Reserved

10

TimeDate

First block number in FAT

Size

2 2 2 4

Accurate to within 2 seconds

Pinned to 1980

OperatingSystems

FAT-12FAT-12FAT-12FAT-12

MS-DOS defines a block (called a cluster by Microsoft) as some multiple of 512 bytes.

FAT-12 blocks were 512 bytes. Each cluster is represented in the File Allocation table by a 12 bit number. Thus, the maximum partition size was about 2MB and the File location Table contained 4096 2-byte entries.

OperatingSystems

Fat-16Fat-16Fat-16Fat-16

As drives got bigger, the File Allocation Table structure had to change to accommodate the bigger sizes. In a FAT-16 table, each cluster was now represented by a full 16 bit entry in the file allocation table. Additional block sizes of 8 KB, 16KB, and 32 KB supported 2 GB partitions.

But what’s the problem with bigger block sizes?

OperatingSystems

Fat-32Fat-32

FAT-32 was introduced in the second release of Windows 98.FAT-32 represents each cluster in the file table with a 28-bitentry. The maximum partition size for a FAT-32 system is 2 Terabytes. A big advantage of FAT-32 is that for anequivalent partition size, FAT-32 blocks can be much smallerthan FAT-16 blocks. Thus, the file system is more efficient.

The downside is that the File Allocation Table is much bigger.For a 4KB block and a 2GB partition, the FAT takes up2 MB of RAM.

The Windows 98 File System

The Windows 98 File System

File Name

8

Ext

3

Attribute• read-only• archive• hidden• system file

bytes 1 10

TimeDate

Lower 16 bits offirst block number

Size

2 2 2 4

NT bit (for compatibility)Adds time accuracy to creation date/time - within 10ms

Last write

Creation date/time

Last access time

Upper 16 bits offirst block number

Long File NamesLong File NamesLong File NamesLong File Names

Windows 98 provides 2 file names for each file. * The standard DOS 8 + 3 file name * A long file name

The algorithm for creating an 8 + 3 file nameis to truncate the file name to 6 characters andadd ~1 to the name ( or ~2 if the name already exists).

OperatingSystems

Each long file name is stored in the directory, in front of thenormal directory entry for the file. Up to 13 Unicode charactersare stored in the following format. Multiple entries are used toProvide storage for the entire long file name.

Sequence

5 characters 0

Attribute byte0x0F

checksum

6 characters 0 2 chars

OperatingSystems

The Big Long Name

T h e B I g ~ 1

1 T h e B

A

A

A

NT

0

0

S

C

C

CreationTime

i g L o n

. . . low

0

0

g N

65 a m e

OperatingSystems

Windows NT File System

NTFS does not use sectors. Insteadit uses a cluster, which is some number(power of 2 ) sectors. This makes thefile system independent of sector size.

bootsector datamaster file table

systemfiles

free space bitmap

MFT Records

RecordHeader

attribute-value pairs

Attributeheader

data

Attributeheader

data

one of: standard info file name attribute list (location of additional mft records, if needed) volume name data . . .

File Systems CNS 3060 Operating Systems. TopicsTopics Files Directories File system implementation...

Documents

Transcript of File Systems CNS 3060 Operating Systems. TopicsTopics Files Directories File system implementation...