CS: Introduction to Record Manipulation & Indexing

Records and Indexing 14-Sep-03 1© Katrin Becker

All Rights Reserved

Record Manipulation & Indexing

•records/fields•index placement; index management•manipulating fixed-length record files•re-using space in fixed-length files•varying length records:[VLR] adds; dels; mods;•free lists for VLR - placement strategies (first, best, worst)•varying length record maintenance


All Rights Reserved

Records in General

A record is:• An identifiable, describable data set• Often contains a sub-structure• Typically part of a larger structure

This definition also works for: files; fields; …


All Rights Reserved

Records and Fields

FILE containing records

RECORD containing fields

FIELD containing elements

FILE SYSTEM containing files


All Rights Reserved

Record Manipulation

• Operations on Records:– Searches– Additions– Deletions– Modifications


All Rights Reserved

Record Manipulation - SearchSequential Search

•While NOT done:– Position file pointer– Read record– Examine record to see if it’s the

one•Yes DONE•No CONTINUE


All Rights Reserved

Other Searches• What changes?

– Binary search:• We position the file pointer in a different

fashion (the rest is the same)

– Search with an index• We apply the search to the index and retrieve

the record only when located in the index


All Rights Reserved

Record Manipulation – Addition

New record gets added to the end.

• Insertion into middle of file is impractical.• If there is an index, then we also perform

an addition to the index (addition to the end of this list is infeasible – WHY? ).


All Rights Reserved

Addition with an Index - 1

1. New record gets added to the end.

RECORDS

INDEX


All Rights Reserved


2. Locate place where index entry needs to go

INDEX

RECORDS


All Rights Reserved


RECORDS

INDEX

3. Insert New Index entry (it’s a record too)


All Rights Reserved

Records vs. Index:Assertions & Questions

• Moving file records is more expensive than moving index records.

• Should index be IN record file or its own file? (How do we maintain it? )

• If IN file: should it be at the beginning, end, middle, distributed?

• What if we are able to hold the index in memory?

• What if we can’t?


All Rights Reserved

Record Manipulation - Deletion

• Locate record (Search)• Mark space as deleted• Remove index entry? (why or why

not)


All Rights Reserved

Deletion with an index - 1

RECORDS

INDEX1. Locate index entry


All Rights Reserved


RECORDS

INDEX1. Locate index entry

2. Locate record


All Rights Reserved


RECORDS

INDEX

3. Delete (mark) record


All Rights Reserved


RECORDS

INDEX

4. Delete (mark?) index entry


All Rights Reserved

Record Manipulation - Modification

• Locate record• Read record• Modify record• Re-write record (assuming fixed-

size records – what if the record is now a different size? [see later])


All Rights Reserved

File Behaviour – 1 start

Record count = 9


All Rights Reserved

File Behaviour – 2 add record

Record count = 10


All Rights Reserved

File Behaviour – 3 add record

Record count = 11


All Rights Reserved

File Behaviour – 4 delete

Record count = 10


All Rights Reserved


Record count = 9


All Rights Reserved

File Behaviour – 6 add

Record count = 10


All Rights Reserved


Record count = 11


All Rights Reserved


Record count = 12


All Rights Reserved


Record count = 11


All Rights Reserved


And so on…….

Record count = 10


All Rights Reserved

What’s happening to the file?• File grows – does not shrink (we get

fragmentation)• We end up covering more ground to do the

same job • Q: If we are doing random access, why does it matter?

• The file system has less space to use (the fragmentation is internal from the perspective of the file system).

• Worst case = EVERY record access ends up costing us a seek.


All Rights Reserved

Re-Using Space in the File [FLR]• When there is a deletion, locate

the last record in the file, end move to the free slot– Costs:

•Additional file access to locate (where will we remember where the last records is?) and retrieve last record.

•Records will loose locality faster than if we simply mark the slot. (Why do we care?)


All Rights Reserved

Re-Using Space – Way 2

• Make a list of places where records have been deleted.

• When doing addition, check for empty ‘slot’ before placing new record at end.

Q: What about the index?

• When doing deletion, add location of deleted record to ‘free-list’


All Rights Reserved

What does the Free-List look like?

RECORDS

INDEX

All we need is the location.Order is unimportant.


All Rights Reserved

How to decide which ‘slot’ to re-use?

• In FLR every slot will fit a new record.• We can just take the first one – Free-

List can then be maintained as a stack (which is easy).

• Do we keep Free-List information in the file?


All Rights Reserved

Indexing – What is it?

• Table-of-contents for a file (directory)

• Uses keys• Byte Offset (BO) vs Relative

Record Number (RRN)


All Rights Reserved

Primary Key Properties:

• Unique• Canonical• Data-less• Unchanging


All Rights Reserved

Indexing – How does it Look?• Must have:

– Key– Way to locate record

• It is itself a structure containing ‘records’ (each index entry is a record)

• It may be separate from the main data or in the same file.

• It may be copied into memory for manipulation and only updated infrequently; or the file copy may be maintained as well.

INDEX


All Rights Reserved

Indexing – File Ops?

• Tied to records:– If records added – new/update index

entry– If record deleted – ‘delete’ index entry– If record modified – maybe no change

to index; maybe update BO [byte offset]


All Rights Reserved

Fixed-length vs Varying Length• VLR provides greater flexibility.• VLR increases maintenance overhead.• VLR decreases wasted space. *• VLR makes index virtually essential.• VLR complicates Free-List

maintenance.

*may simply waste space in a different place or a different way.


All Rights Reserved

VLR Index

RECORDS

INDEX

• Requires:– Key– Byte offset– Record size? [optional]


All Rights Reserved

VLR Search Operation

RECORDS

INDEX

• Same as for FLR:1. Locate key in index2. Locate record in file

• Binary search still possible on index, but NOT on records alone.


All Rights Reserved

VLR Deletion Operation - 1

RECORDS

INDEX

Locate key


All Rights Reserved


RECORDS

INDEX

Locate record


All Rights Reserved


RECORDS

INDEX

Delete record


All Rights Reserved

VLR Addition Operation – 2c

RECORDS

INDEXWe will end up with left-

over unused (and probably unusable space).

We call this “First-Fit” (because we are using the first slot that we find that fits).

Free-List

New

Record


All Rights Reserved

VLR Addition Operation – 2g

RECORDS

INDEX

1. Insert record.Free-List

New

Record

2. Delete Free-List entry.

3. Update Index

Notice the index entry is sorted differently.

What’s the advantage to leaving ‘spaces’ in the index?


All Rights Reserved

Free-Lists

• May want to keep Free-List sorted.

• If the List is short it may not matter.

• Placement Strategies:– First Fit– Best Fit– Worst Fit

• It could be its own list or we could make the regular index serve double-duty.

CS: Introduction to Record Manipulation & Indexing

Technology

Transcript of CS: Introduction to Record Manipulation & Indexing