UNIX System Administration

Adding a Disk

Chapter 8

Disk Interfaces

• Proprietary interfaces– Big. Yellow. Different.

• SCSI– Small Computer Standardized Interface

• IDE– Integrated Drive Electronics

• Fibre Channel– NKOTB

(IDE) ATA helping if you gave me one….• IDE - The idea of incorporating the drive’s controller (NOT the bus

controller) onto the drive unit itself.– PC IDE is attached to main system bus via only buffering/isolation

components - there is no IDE interface per-se

• More properly referred to as ATA - Advanced Technology Attachment– Defined protocol for DISK DRIVES

– ATAPI - ATA Packet Interface - protocol encapsulated in ATA for other “IDE” devices (CD-? Drives, misc removable storage devices)

ATA Performance

• Limited to two devices per channel, two channels per system

• Cheap, fast, single-user storage– ATA protocols have less overhead than SCSI

– No error correction (until ATA-4)

– No behind-the-scenes management (e.g. integrity checks)

• Less extensible than SCSI– Very simple protocol will not support more than two drives per channel

– Drives are not multitasking

– ATA bus is not “shared” with neighbor device - each drive is master of its own domain while handling an I/O request

• Host CPU must manage drives– 60%-100% CPU time of a PIII might be spent on a large transfer

SCSI• SCSI - Small Computer Sstandardized Interface

– Began as Shugart Associates Standardized Interface in 1979, adopted as ANSI standard (SCSI-1) in 1986 with a name change

• Separate, intelligent I/O bus attached to main bus. All devices on bus are intelligent as well.

• Managing device - initiator - typically is the host interface [though this is not required]. Peripherals are targets.

SCSI too

• Initiator controls whether a SCSI device is connected to the bus (separate control lines facilitate initiator/target control communications)

• Multiple I/O requests may be issued against multiple targets

• Controllers typically use DMA (bus mastering) to offload the work of transferring data to memory from the host CPU

SCSI Buses• Single Ended (SE)

– Oldest variety– Most susceptible to noise due to low signal levels - required strict adherence to cable

specifications– Uses common ground for all signal lines

• Differential varieties– Developed to overcome SE limitations - noise immunity– Uses a + and - line set for each signal– High Voltage Differential (HVD) - 12 volts– Low Voltage Differential (LVD) - 5 volts

• Less expensive version of HVD• LVD devices will detect connection to a SE bus and fall back to SCSI-2 speeds

• All are incompatible• All require termination. SE - passive H/LVD - active

– Absorbs signal at ends of the bus to prevent reflections

Feeling a little...irregular? Add some fibre to your diet.

• Fibre Channel - Member of a class of high-speed, serial interfaces for disk storage (serial referring to the transmission order of data bytes)– SSA (Serial Storage Architecture) is IBM’s proprietary attempt at a serial

interface for storage. As always, Big Blue sticks to that P-word. Regardless, SSA is too little, too late. Revel in the irony.

• Fibre Channel media– Coaxial copper cable

– Twisted Pair copper cable

– Multimode or Singlemode fiber (required for speeds over 1Gb/s [100MB/s]) - two fibers required (Tx/Rx) in a P2P link configuration.

• Fibre channel protocols– Provide capability to support FC-specific protocols as well as SCSI, IP,

and IEEE 802.2 (ethernet)

– Supports ‘class of service’ - datagram, fractional speed, connectionless, etc.

FC Topo- and Termi- nology• Point-to-point

– exactly two devices. Exclusive media access.

• Arbitrated loop– up to 127 devices, chained in a loop. Shared media access.

• Fabric-switched– up to 224 devices in a cross-point switched configuration. Requires a

switch, provides exclusive media access.

Drive Terminology

• Sector - atomic unit of storage, typically 512B

• Block - abstracted sector free from track/surface designations

• Track - sectors in a single groove on one surface of a platter

• Cylinder - from a vertical perspective, set of tracks on all disk surfaces at “equal distance from the center”

• Partition - Logical grouping of a set of consecutive cylinders or blocks

• LUN - Logical Unit Number - SCSI construct that permits dividing a single target into up to 7 logical subdevices. All targets have LUN 0, devices like multi-tray CD changers and drive arrays utilize other LUN values as required.

Preparing your disk

• Connect the disk to the computer

• Create device files so disk can be accessed

• Format the disk ****

• Partition the disk

• Create filesystems on partitions

• Configure filesystem mounting

Let there be light!• Connect the disk

• Create device files to access the disk

– All devices have entries in the /dev structure

– Disks (and like storage media) have two:

• Character (or raw) - a “low-level” device used for maintenance interaction (fsck, etc) - /dev/rdsk

• Block - conceptually “higher-level” device used for filesystem interaction (backups, mounting, etc) - /dev/dsk

– Typically established by a reconfiguration sequence (though can be established “manually” under certain circumstances)

– All /dev entries for LUN 0 will be created initially, extra LUNs may require extra steps

• Format the disk ****– In terms of “low-level” format, disks are pre-formatted by the manufacturer. This

procedure should not be necessary, and should only be attempted with sufficient knowledge of the ramifications.

Click to add title• Partition the disk

– Actual method is O/S dependent

– Logically groups cylinders (or blocks) into a usable portion which can be referenced

– Partitions are associated with and referred to by a device entry• CwTxDySz

– C = controller (w) - number of SCSI controller in the system– T = target (x) - SCSI ID of target on bus. 0-7 or 0-15– D = LUN (y) - 0 always exists, typically only one LUN. 0-7– S = slice (z) - partition ID. 0-7

• Slice 2 is the “backup” slice - a special partition that refers to the entire disk, used for low-level disk manipulation

– Partitions should not overlap (but the customer is always right…)

– A disk’s label refers to its partition table map. Labeling the disk writes this map onto the disk to be used by the system for interfacing with that drive.

• Remember to partition for O/S, swap, and other file storage needs (e.g. home, log, application directories)

Click harder!

• Create a filesystem on new partitions

– Partitions (slices) need to be initialized to accept a filesystem.

– Actual procedure varies by O/S - typically the newfs command

• Requires the /dev/rdsk filename of the partition to initialize

• Writes out the required filesystem data structures to disk (superblocks, disk block maps, inode tables, etc)

NOT THAT HARD!

• Configure filesystem mounting– Procedure is (AGAIN) O/S dependent

– Filesystem must be mounted before can be accessible

– Add entries for automatic mounting• /etc/vfstab

– Mount points are directories on an existing filesystem (or / if this is the root disk), but once mounted, the new filesystem will overlay and hide anything contained in that mount point directory.

– You must mount in order of dependence if mounting a filesystem on a mounted filesystem other than /

When good disks go bad…• fsck - Filesystem check and repair

• Capable of repairing most soft errors

• Runs automatically on boot (per settings in /etc/vfstab)

• Required to be run against filesystems that were potentially made inconsistent by an improper shutdown - mount will check a flag that indicates the possible state of the filesystem and will usually refuse to accept inconsistent filesystems

• A fsck should only be done on an unmounted filesystem especially if changes will be made - O/S buffers will become inconsistent and the system may panic…

• Certain repair operations may be destructive (but would be required before disk could be mounted again)

• Running fsck on a journaled filesystem takes little time as the journal log clears up inconsistencies (fsck will flush the log, which is very small)

Solaris

• Connect the disk (power off, please) and set the SCSI Ids and place terminators (or IDE master/slave jumper)

• Power on and enter PROM mode– Issue probe-scsi (or probe-scsi-all) to verify disk presence

• Use probe-ide for IDE machines

– Issue boot -r for reconfig boot

• Use the format command to label (partition) the disk– Ensure your slices do not overlap - no direct indication would be provided of this

oversight (the overlap exception is slice 2 - the backup slice - don’t ever touch it). Zero-out any unused partitions to avoid accidental use. Don’t forget to issue the label command before exiting.

• Use newfs to initialize the filesystem• Mount the filesystem to verify• Add entries in /etc/vfstab as required - enable journaling, but be careful!!