Impact of Disk Alignment in Virtualized Environments
description
Transcript of Impact of Disk Alignment in Virtualized Environments
![Page 1: Impact of Disk Alignment in Virtualized Environments](https://reader035.fdocuments.in/reader035/viewer/2022062315/56815de7550346895dcc0d6e/html5/thumbnails/1.jpg)
Grant Cohoe
IMPACT OF DISK ALIGNMENT IN VIRTUALIZED ENVIRONMENTS
![Page 2: Impact of Disk Alignment in Virtualized Environments](https://reader035.fdocuments.in/reader035/viewer/2022062315/56815de7550346895dcc0d6e/html5/thumbnails/2.jpg)
WHY SHOULD YOU CARE?• Performance
• Misalignment causes more IO’s than you need
• Shared Storage issues
![Page 3: Impact of Disk Alignment in Virtualized Environments](https://reader035.fdocuments.in/reader035/viewer/2022062315/56815de7550346895dcc0d6e/html5/thumbnails/3.jpg)
UNDERSTAND YOUR STUFF• Hard Disk Geometry
• Sector Size (Logical & Physical)
• Operating System
• What does it want?
• What does it do by default?
• Sometimes silly things…
![Page 4: Impact of Disk Alignment in Virtualized Environments](https://reader035.fdocuments.in/reader035/viewer/2022062315/56815de7550346895dcc0d6e/html5/thumbnails/4.jpg)
LAYERS
Disk Geometry/Partitions
LVM
Host File System
VMDK Geometry/Partitions
VMFS
![Page 5: Impact of Disk Alignment in Virtualized Environments](https://reader035.fdocuments.in/reader035/viewer/2022062315/56815de7550346895dcc0d6e/html5/thumbnails/5.jpg)
DISK GEOMETRY/PARTITIONS
Disk Geometry/Partitions
LVM
Host File System
VMDK Geometry/Partitions
VMFS
![Page 6: Impact of Disk Alignment in Virtualized Environments](https://reader035.fdocuments.in/reader035/viewer/2022062315/56815de7550346895dcc0d6e/html5/thumbnails/6.jpg)
TERMINOLOGY• Sectors
• Units of disk storage
• Partition
• Logical group of sectors
• Track
• Ring of sectors on a single side of a platter
• Cylinder
• 3D track (all platters at one track location)
Disk Geometry/Partitions
LVM
Host File System
VMDK Geometry/Partitions
VMFS
![Page 7: Impact of Disk Alignment in Virtualized Environments](https://reader035.fdocuments.in/reader035/viewer/2022062315/56815de7550346895dcc0d6e/html5/thumbnails/7.jpg)
MASTER BOOT RECORD (MBR)• That thing that boots your OS
• First 512 bytes of the disk
• 440 bytes of bootloader
• 32 bytes of partition information
• 4 primary partitions - max size 2TB
512
STAR
T 440 (Boot loader) 32
Disk Geometry/Partitions
LVM
Host File System
VMDK Geometry/Partitions
VMFS
![Page 8: Impact of Disk Alignment in Virtualized Environments](https://reader035.fdocuments.in/reader035/viewer/2022062315/56815de7550346895dcc0d6e/html5/thumbnails/8.jpg)
MASTER BOOT RECORD (MBR)• DOS Compatibility
• Cannot span cylinders (because DOS was silly)
• Number of sectors per cylinder = 63
• 63 – 1 (MBR) = 62 sectors before first usable
• This is deprecatedMBR LBA-1 LBA-62 63
Disk Geometry/Partitions
LVM
Host File System
VMDK Geometry/Partitions
VMFS
![Page 9: Impact of Disk Alignment in Virtualized Environments](https://reader035.fdocuments.in/reader035/viewer/2022062315/56815de7550346895dcc0d6e/html5/thumbnails/9.jpg)
MASTER BOOT RECORD (MBR)• 1MB Alignment
• Align all partitions to 1MB
• 1MB = 1048576B / 512B sectors = 2048 (1st Sector)
• Improves performance
• Ensures compatibility for 4K “Advanced Format”
• This is new standard (Windows Vista)
MBR LBA-1 LBA-2047 2048
Disk Geometry/Partitions
LVM
Host File System
VMDK Geometry/Partitions
VMFS
![Page 10: Impact of Disk Alignment in Virtualized Environments](https://reader035.fdocuments.in/reader035/viewer/2022062315/56815de7550346895dcc0d6e/html5/thumbnails/10.jpg)
RESULTING DISK
• 512B MBR –
• Alignment Space –
• 1st Partition Starting Sector –
• This is good!
MBR 2048 2049 2050 2051 2052 2053 2054 2055 … 16777215
MBR
2048
Disk Geometry/Partitions
LVM
Host File System
VMDK Geometry/Partitions
VMFS
![Page 11: Impact of Disk Alignment in Virtualized Environments](https://reader035.fdocuments.in/reader035/viewer/2022062315/56815de7550346895dcc0d6e/html5/thumbnails/11.jpg)
LOGICAL VOLUME MANAGEMENT (LVM)
Disk Geometry/Partitions
LVM
Host File System
VMDK Geometry/Partitions
VMFS
![Page 12: Impact of Disk Alignment in Virtualized Environments](https://reader035.fdocuments.in/reader035/viewer/2022062315/56815de7550346895dcc0d6e/html5/thumbnails/12.jpg)
TERMINOLOGY• Physical Volume
• Container of data stored as a partition on disk
• Logical Volume
• Virtualized storage structure stored as data in a PV
• pe_start
• LV offset within a PV
Disk Geometry/Partitions
LVM
Host File System
VMDK Geometry/Partitions
VMFS
![Page 13: Impact of Disk Alignment in Virtualized Environments](https://reader035.fdocuments.in/reader035/viewer/2022062315/56815de7550346895dcc0d6e/html5/thumbnails/13.jpg)
LVM PHYSICAL VOLUMES (LVM PV)• pe_start specifies the start of LV data
• Very intelligent. Usually not a problem
• Needs to be aligned to your sectors!
Disk Geometry/Partitions
LVM
Host File System
VMDK Geometry/Partitions
VMFS
![Page 14: Impact of Disk Alignment in Virtualized Environments](https://reader035.fdocuments.in/reader035/viewer/2022062315/56815de7550346895dcc0d6e/html5/thumbnails/14.jpg)
LVM PHYSICAL VOLUMES (LVM PV)• Bad
• pe_start does not line up with a sector
• Going to hurt performance later
Disk Geometry/Partitions
LVM
Host File System
VMDK Geometry/Partitions
VMFS
MBR 2048 2049 2050 2051 2052 2053 2054 2055 2056 2057 2058 2059
Physical Volume
pe_start PV Data Region
![Page 15: Impact of Disk Alignment in Virtualized Environments](https://reader035.fdocuments.in/reader035/viewer/2022062315/56815de7550346895dcc0d6e/html5/thumbnails/15.jpg)
LVM PHYSICAL VOLUMES (LVM PV)• Good
• As long as pe_start is a multiple ofyour sector size (usually 512B)you’re good!
Disk Geometry/Partitions
LVM
Host File System
VMDK Geometry/Partitions
VMFS
MBR 2048 2049 2050 2051 2052 2053 2054 2055 2056 2057 2058 2059
Physical Volume
pe_start PV Data Region
![Page 16: Impact of Disk Alignment in Virtualized Environments](https://reader035.fdocuments.in/reader035/viewer/2022062315/56815de7550346895dcc0d6e/html5/thumbnails/16.jpg)
LVM PHYSICAL VOLUMES (LVM PV)• PE Size
• Physical Extent – LVM “block” size
• Usually default is fine
• Multiple of sector size (512)
Disk Geometry/Partitions
LVM
Host File System
VMDK Geometry/Partitions
VMFS
![Page 17: Impact of Disk Alignment in Virtualized Environments](https://reader035.fdocuments.in/reader035/viewer/2022062315/56815de7550346895dcc0d6e/html5/thumbnails/17.jpg)
RESULTING VOLUME
• LV starting point aligned (pe_start)
• PV aligned to sectors on disk
Disk Geometry/Partitions
LVM
Host File System
VMDK Geometry/Partitions
VMFS
MBR 2048 2049 2050 2051 2052 2053 2054 2055 2056 2057 2058 2059
Physical Volume
pe_start PV Data Region
Logical Volume
![Page 18: Impact of Disk Alignment in Virtualized Environments](https://reader035.fdocuments.in/reader035/viewer/2022062315/56815de7550346895dcc0d6e/html5/thumbnails/18.jpg)
HOST FILE SYSTEM
Disk Geometry/Partitions
LVM
Host File System
VMDK Geometry/Partitions
VMFS
![Page 19: Impact of Disk Alignment in Virtualized Environments](https://reader035.fdocuments.in/reader035/viewer/2022062315/56815de7550346895dcc0d6e/html5/thumbnails/19.jpg)
HOST FILE SYSTEM• Not much to do here
• RAID would be a different story…
• Ext is good at picking sane defaults
• Block size
• Smallest unit of data for the filesystem
Disk Geometry/Partitions
LVM
Host File System
VMDK Geometry/Partitions
VMFS
![Page 20: Impact of Disk Alignment in Virtualized Environments](https://reader035.fdocuments.in/reader035/viewer/2022062315/56815de7550346895dcc0d6e/html5/thumbnails/20.jpg)
RESULTING FILESYSTEM
MBR 2048 2049 2050 2051 2052 2053 2054 2055 2056 2057 2058 2059
Physical Volume
pe_start PV Data Region
Logical Volume
Filesystem
![Page 21: Impact of Disk Alignment in Virtualized Environments](https://reader035.fdocuments.in/reader035/viewer/2022062315/56815de7550346895dcc0d6e/html5/thumbnails/21.jpg)
VMDK GEOMETRY & PARTITIONS
Disk Geometry/Partitions
LVM
Host File System
VMDK Geometry/Partitions
VMFS
![Page 22: Impact of Disk Alignment in Virtualized Environments](https://reader035.fdocuments.in/reader035/viewer/2022062315/56815de7550346895dcc0d6e/html5/thumbnails/22.jpg)
VMDK GEOMETRY/PARTITIONS• Same principles as host disks
• DOS compatibility sucks
• 1MB alignment is good
• Performance impact is bigger
Disk Geometry/Partitions
LVM
Host File System
VMDK Geometry/Partitions
VMFS
MBR 2048 2049 2050 2051 2052 2053 2054 2055 2056 2057 2058 2059
![Page 23: Impact of Disk Alignment in Virtualized Environments](https://reader035.fdocuments.in/reader035/viewer/2022062315/56815de7550346895dcc0d6e/html5/thumbnails/23.jpg)
VM FILE SYSTEM
Disk Geometry/Partitions
LVM
Host File System
VMDK Geometry/Partitions
VMFS
![Page 24: Impact of Disk Alignment in Virtualized Environments](https://reader035.fdocuments.in/reader035/viewer/2022062315/56815de7550346895dcc0d6e/html5/thumbnails/24.jpg)
VM FILE SYSTEM• Don’t use RAID/LVM in VMs
• Unless you really need it for some reason
• Or if you did a P2V
Disk Geometry/Partitions
LVM
Host File System
VMDK Geometry/Partitions
VMFS
MBR 2048 2049 2050 2051 2052 2053 2054 2055 2056 2057 2058 2059
VM File System
![Page 25: Impact of Disk Alignment in Virtualized Environments](https://reader035.fdocuments.in/reader035/viewer/2022062315/56815de7550346895dcc0d6e/html5/thumbnails/25.jpg)
VM ALIGNMENT
Disk Geometry/Partitions
LVM
Host File System
VMDK Geometry/Partitions
VMFS
![Page 26: Impact of Disk Alignment in Virtualized Environments](https://reader035.fdocuments.in/reader035/viewer/2022062315/56815de7550346895dcc0d6e/html5/thumbnails/26.jpg)
PERFECTLY ALIGNED VM
Disk Geometry/Partitions
LVM
Host File System
VMDK Geometry/Partitions
VMFS
MBR 2048 2049 2050 2051 2052 2053
VM File System
MBR 2048 2049 2050 2051 2052 2053 2054 2055 2056 2057 2058 2059
Physical Volume
pe_start PV Data Region
Logical Volume
Filesystem
2054
![Page 27: Impact of Disk Alignment in Virtualized Environments](https://reader035.fdocuments.in/reader035/viewer/2022062315/56815de7550346895dcc0d6e/html5/thumbnails/27.jpg)
PERFECTLY ALIGNED VM
Disk Geometry/Partitions
LVM
Host File System
VMDK Geometry/Partitions
VMFS
4096
512 512 512 512 512 512 512 512
4096
1024
512 512 512 512 512 512 512 512
VM FS Block
VMDK Sectors
Host FS Block
LVM PE*
Host Disk Blocks
* PE shown as 1K for example
1024 1024 1024
![Page 28: Impact of Disk Alignment in Virtualized Environments](https://reader035.fdocuments.in/reader035/viewer/2022062315/56815de7550346895dcc0d6e/html5/thumbnails/28.jpg)
MISALIGNED VM
Disk Geometry/Partitions
LVM
Host File System
VMDK Geometry/Partitions
VMFS
4096
512 512 512 512 512 512 512 512
4096
512 512 512 512 512 512 512 512
4096
512 512 512 512 512 512 512 512
• VM disk image sits across two Host FS blocks, thus requiring more reads of the host disks to get all data
• 4096B of VM data requires 8192B of host disk data to read
1024 1024 1024 1024 1024 1024 1024 1024
![Page 29: Impact of Disk Alignment in Virtualized Environments](https://reader035.fdocuments.in/reader035/viewer/2022062315/56815de7550346895dcc0d6e/html5/thumbnails/29.jpg)
END GOAL
Disk Geometry/Partitions
LVM
Host File System
VMDK Geometry/Partitions
VMFS
MBR 2048 2049 2050 2051 2052 2053
VM File System
MBR 2048 2049 2050 2051 2052 2053 2054 2055 2056 2057 2058 2059
Physical Volume
pe_start PV Data Region
Logical Volume
Filesystem Filesystem Filesystem Filesystem Filesystem
![Page 31: Impact of Disk Alignment in Virtualized Environments](https://reader035.fdocuments.in/reader035/viewer/2022062315/56815de7550346895dcc0d6e/html5/thumbnails/31.jpg)
MODERN STUFFBONUS MATERIAL
![Page 32: Impact of Disk Alignment in Virtualized Environments](https://reader035.fdocuments.in/reader035/viewer/2022062315/56815de7550346895dcc0d6e/html5/thumbnails/32.jpg)
ADVANCED FORMAT DISKS• 4K Sectors
• Old:
• New:
• Much more efficient with todays data usage
• 512e Emulation Mode
• Lets old stuff still work with new disks
• Logical (OS):
• Physical (Disk):
64 65 66 67 68 69 70 71
8
64 65 66 67 68 69 70 71
8
![Page 33: Impact of Disk Alignment in Virtualized Environments](https://reader035.fdocuments.in/reader035/viewer/2022062315/56815de7550346895dcc0d6e/html5/thumbnails/33.jpg)
ADVANCED FORMAT DISKS & MBR• Regular disks (512 byte sectors)
• LBA-63
• Advanced Format (4K sectors) w/ e512
• LBA-63
• PROBLEM LATER ON
MBR 1 62 63 64 65 66 67 68 69 70 71 72 73 74 75
MBR 1 62 63 64 65 66 67 68 69 70 71 72 73 74 75
0 4K sectors 7 8 9
![Page 34: Impact of Disk Alignment in Virtualized Environments](https://reader035.fdocuments.in/reader035/viewer/2022062315/56815de7550346895dcc0d6e/html5/thumbnails/34.jpg)
GUID PARTITION TABLE (GPT)• That new thing that boots your OS
• First 17K of the disk
• Lots of stuff ------------------------------>
• On Disk
GPT Alignment Space 2048
![Page 35: Impact of Disk Alignment in Virtualized Environments](https://reader035.fdocuments.in/reader035/viewer/2022062315/56815de7550346895dcc0d6e/html5/thumbnails/35.jpg)
RAID IMPLICATIONS• If RAID volume misaligned, entire array is affected
• RAID in VMs is BAD!
![Page 36: Impact of Disk Alignment in Virtualized Environments](https://reader035.fdocuments.in/reader035/viewer/2022062315/56815de7550346895dcc0d6e/html5/thumbnails/36.jpg)
RAID TERMINOLOGY• Data Disk
• A disk that has real data (not parity)
• Stripe
• RAID unit of IO (“block”)
• Also called “Chunk”
• Stride
• Amount of data from a stripe before moving to next disk
• Stripe Width
• Length of a stripe
![Page 37: Impact of Disk Alignment in Virtualized Environments](https://reader035.fdocuments.in/reader035/viewer/2022062315/56815de7550346895dcc0d6e/html5/thumbnails/37.jpg)
RAID MATH• Constants
• DATA_DISKS = 3 (lets say this is RAID5 with 4 disks)
• BLOCK_SIZE = 4K (from the filesystem)
• CHUNK_SIZE = 512K
• Calculate Stride
• STRIDE = CHUNK_SIZE / BLOCK_SIZE = 128K
• Calculate Stripe Width
• STRIPE_WIDTH = STRIDE * DATA_DISKS = 384K
• What this means:
• One unit of RAID IO will write 128K to the first disk then move on to the next one
![Page 38: Impact of Disk Alignment in Virtualized Environments](https://reader035.fdocuments.in/reader035/viewer/2022062315/56815de7550346895dcc0d6e/html5/thumbnails/38.jpg)
REFERENCES• http://en.community.dell.com/techcenter/extras/w/wiki/2838.aspx
• http://www.pixelbeat.org/docs/disk/
• http://computer-forensics.sans.org/blog/2010/07/28/windows-7-mbr-advanced-format-drives-e512/
• http://en.wikipedia.org/wiki/Advanced_format