1

Implementation of Log Structured File System

Report by Abhinav Jha, Nithya K and Sushanth Kumar Reddy

This document contains the design of the log structured file system programming assignment. The

document will cover in the overall design of the system, features implemented for the phase 1 and the

features planned for the phase 2.

Overview

We have designed a layered structure for the log structured file system. The following layers are present

in the system.

Disk Layer: Does all the operations to write to the virtual disk

Log Layer: This layer treats the disk as a log and will do log read and log write operations.

Inode Layer: Abstracts the internal representation of the file. All operations needed on a file are

provided here and corresponding inodes are also modified.

Vnode layer: This layer is responsible for the file and directory abstraction. It is the external interface to

the LFS

Fuse: fuse layer intercepts messages from FUSE, operates on the LFS through the vnode layer and

returns expected values

Detailed design

Basic Log structure

The virtual disk created by mklfs has the following format. The log is divided into segments. Each

segment is of fixed size.

Disk

Log

Inode

Vnode

Fuse

2

superblock

Data

segment

Data

segment

Data

segment

Data

segment

Data

segment

Data

segment

Log is divided into fixed sized segments.

Super Block

The first segment of the log contains the super block which will have the system Meta data information.

The super block will contain

Checkpoint 1 Checkpoint 2

Super block structure

System Meta data contains all the system information like segment size, disk size, block size etc. We

have 2 checkpoints.

Each checkpoint has the following information

• Last good segment points to the last good segment in the disk of threaded segments.

• Inode version table contains the list of all inodes and their version. This is an optimization that

will help us avoid the need to scan through the inodes while doing a segment clean.

• Segment usage table helps us keep track of the usage of segments and help us find the next free

segment.

• Ifile inode contains the inode of the ifile.

Data Block: The data block contains the following information.

Segment summary

table

Data blocks

Prev

segment

Next

segment

System Meta data

Last good segment

Inode version table

Segment usage table

Ifile inode

3

The initial block contains the segment header which contains the segment summary table. The segment

summary table will contain information about the blocks that the segment contains. For each block it

will store inode number that the block is part of, version (currently unused), offset of the block in the

file. The segment header also contains pointers to next and previous segments.

Log layer data structures

Log layer treats the disk as collected of segments. All the operations done by log are in segments. The

log layer maintains three important data structures.

• Tail segment: Tail segment is always in memory where all the log writes are done.

• Segment read cache: This will contain few segments that are frequently accessed. When a log

read happens, the segment is checked in segment reach cache and then tail segment and if it’s

not present the segment is read from the disk. The replacement policy used is LRU.

• Check pointing cache: This contains the segment summary tables of the segments that are

recently written. The size of the check pointing cache is set by the check pointing interval. Once

the number of segments written crossed the checkpointing interval, checkpointing is triggered.

Check pointing will make sure segment usage table, inode version table, ifile inode and last good

segment number is update. In case of LFS release, checkpointing will also write the tail segment

to the disk.

Inode Layer data structures

The following data structures are used in the inode layer.

• Inode read cache: Frequently used inodes are cached in the inode read cache. This helps

prevent making frequent log read calls. The replacement policy used in LRU.

• Inode write cache: This will contain the inode that is most recently modified. Once the inode is

written to the log it is also copied to the read cache, so that read cache will contain the most

recent copy of the inode.

• Ifile Inode: The ifile inode is kept in memory. Whenever the ifile is changes, the ifile inode has to

be updated. This ifile is frequently written to the log layer which also maintains a copy of ifile

which will eventually be written to disk at the time of checkpointing.

• List of free inodes: A list of free inodes are maintained which the Vnode layer will make use of

while creating a file. It is currently unused in the phase 1 of the project.

Vnode layer data structures

The following data structure is used in Vnode layer.

• Path to inode number mapping: This will contain a list of recently referred file paths and their

inode number. This will reduce the number of calls made to the directory file. This is currently

disabled for the phase 1 of the project.

4

Feature list

Feature Phase 1 Phase 2

Mklfs Complete implementation done

in phase 1.

Log layer

Checkpoint Complete implementation done

in phase 1.

Segment

Cleaner

Segment cleaner utility functions

are planned for the phase 2.

Roll forward Roll forward utility functions are

planned for the phase 2.

Others Tail segment free blocks reusing

to be implemented in the phase

2.

Inode Layer

Direct block

pointers

Complete implementation is

done in phase 1.

Indirect block

points

Partial implementation is done. Finish remaining parts of the

implementation

Roll forward Roll forward utility functions are

planned for the phase 2.

Vnode Layer

Directory

structure

Currently only 1 directory i.e.

root implemented. The file

names are inode number.

Full directory structure

implementation planned for

phase 2.

Links Complete implementation

planned for phase 2.

Fuse Layer Functions

implemented

• lfs_getattr

• lfs_open

• lfs_read

• lfs_unlink

• lfs_write

• lfs_opendir

• lfs_readdir

• lfs_destroy

• lfs_create

• lfs_truncate.

• lfs_readlink

• lfs_mkdir

• lfs_rmdir

• lfs_release

• lfs_releasedir

• lfs_init

• lfs_statfs

NOTE: The current implementation for phase 1 just supports direct block pointers. So since the

directory file can contain only N no of blocks (N is direct block pointers). So the number of files that

you can create is limited by the no of direct block points. For example if N = 4, then root can contain

only 4 blocks. Each block can contain entries for 4 files, so for phase 1 max no of files that can be

created is 15-16.