Introduction to the Microsoft exFAT File System

HTCIA 2014 Conf - Aug 26, 2014

HTCIA 2014 International ConfHyatt Lost Pines Resort, Austin TexasTuesday August 26, 2014 8:00am

Introduction to the Microsoft exFAT File System

Robert Shullich CPP, CISSP, CRISC, GSEC, GCFA, CEH, CHFI, CCFP-US

Agenda

• About me, the paper and the presentation• The need for a new generation of FAT• Digital Forensics Relevance • Exponents and Standards• exFAT Overview• Linux Development• Memory Cards & Flash Memory• exFAT File System Internals• Closing


EXFAT

About me, the paper and the presentationAbout MeAbout the PresentationAbout the SANS PaperA Gold StandardAnother Paper ReferenceDisclaimer


About Me

• I have been in the IT field for 40+ Years, and in InfoSec for over 20 Years

• I carry many IT and InfoSec certifications • This research was originally for a class

term project towards my D4CS MS degree• I then expanded that term paper into a

practical paper for my SANS “Gold” GCFA certification

• Links to the SANS paper and my blog are provided at the end of this presentation


About the Presentation

What I call the exFAT Road Show• The New York Forensics Computer Show 4/20/2010• Techno Security and Digital Investigations 6/7/2010• SANS What Works in Forensics and IR Summit 7/8/2010• HTCIA International Training Conference & Expo 9/20/2010• The New York Forensics Computer Show 4/19/2011

http://techchannel.att.com/play-video.cfm/2011/8/16/Conference-TV-Computer-Forensics-Show:-Introduction-to-exFAT

• NYC4SEC 6/11/2014• HTCIA International Training Conference & Expo 8/26/2014





About the SANS Paper

• Consider it “exFAT – the missing manual”• Very little published about exFAT today• Two current forensics books mention

exFAT:• Wiley - Mastering Windows Network Forensics and

Investigation• Sybex - EnCase Computer Forensics - The Official EnCE:

EnCase Certified Examiner

“For those seeking an in-depth understanding of the exFAT file system, you should read the SANS paper entitled “Reverse Engineering the Microsoft Extended FAT File System (exFAT) by Robert Shullich”


A Gold Standard• 2005 Book considered

the authority on different file systems

• The book’s Author developed the open-source TSK forensics tools (The Sleuth Kit) & Autopsy

• This year adding exFAT to TSK


Another Paper Reference


Disclaimer

• The released specification and implementation is Release 1.00 of exFAT

• The specification mentions additional features that were not implemented yet, but may at a future time/ Some of these are Windows CE holdovers

• Both may be presented today• Some directory entries will be skipped• Focus is Microsoft Desktop/Server implementation• Will talk about Flash/Solid State, but high level• For exFAT, tried to stay with the patent

terminology HTCIA 2014 Conf - Aug 26, 2014

EXFAT

The need for a new generationLegacy FATWhy do we need a new file system?Why do we need Faster I/O and Higher Capacity?Hi-definition movie recording MPEG-4. H.264


Legacy FAT• FAT 8

• 1977 Bill Gates and Marc McDonald• Floppy based

• FAT 12• 1980

• FAT 16• 1984 with release of PC/AT & MS DOS 3

• FAT 16B• 1987 Compaq DOS 3.31

• FAT 16X• 1995 PC DOS 7.0/Win 95 – LBA Addressing

• FAT 32• 1996 Windows 95 OSR2, 98, ME, MS DOS 7.1 – CHS Addressing

• FAT 32X• LBA Addressing


Why do we need a new file system?

• Current Limits Exhausted (Ran Out of Bits!)• Larger volumes (>2TB) (Scale to Larger Capacity)• Larger files sizes (>4GB)• Faster I/O

• (UHS-I: 104 MB/s - UHS-II: 312MB/s)• Removable Media• Flash/Solid State Media• Flexibility• Extensibility (Difficult to add new features)• NTFS Features without the overhead• Easier to implement FS in firmware


Why do we need Faster I/O and Higher Capacity?


http://www.cnet.com/news/what-is-4k-uhd-next-generation-resolution-explained/

Hi-def movie recording MPEG-4. H.264

2 GB 4 GB 8 GB 16 GB 32 GB

Fine mode(13Mbps/CBR)

20 min

40 min 80 min 160 min

320 min

Normal Mode(9Mbps/VBR)

30 min


480 min

Economy mode (6Mbps/VBR)

45 min


720 min


EXFAT

Digital Forensics RelevanceRelevance to Forensics StudyWhat happens when you have exFAT formatted media and no exFAT support?Forensics Challenges in 2009Forensics Challenges Today


Relevance to Forensics Study

• Digital Evidence Extraction• Finding the evidence• Including the hiding places• Validation• Completeness

• Daubert Expert Testimony• Need to know and understand file org• Establish Credibility

• New Media (SD Cards) will drive exFAT adoption, and the potential for CP investigations.

• Larger Media Capacity also driving exFAT adoption HTCIA 2014 Conf - Aug 26, 2014

Trust but Verify


What happens when you have exFAT formatted media and no exFAT support?


Forensics Challenges 2009

• In 2009, in regards to exFAT:• No tools (RAW)• No documentation or Training• No expertise• Evidence backlog


Forensics Challenges Today

Today• exFAT Misunderstood• Linux OS Support

• Tuxera drivers may help (Embedded)• FUSE and No-FUSE hacks• Most Distributions – No native support

• Mac OS Support (Nov 2010) OS/X 10.6.5+• Implementation Deviations, No Standards• Open Source Tools • Commercial Tools

• Encase (6.14.3 Dec 2009)• Encase (6.18.0.59) NIST Test March 2014• FTK (3.2 Oct 2010)• FTK (3.3) NIST Test April 2014

• Cross Vendor Compatibility


NIST Computer Forensics Tool Testing

• Cyber Fetch• AAFS-2013 Conference 02/21/2013• Deleted File Recovery Tool

Testing Results• One Summary Item:Support for ExFAT, ext3 &ext4 is sometimes lacking.


Test Results for Deleted File Recovery and Active File Listing

• 17 Basic Tests• March 12, 2014 – Encase 6.18.0.59

• MAC differed by 9 hours• April 3, 2014 – FTK 3.3.0.33124

• MAC differed by 4 hours• The exFAT partition and HFS+ created on OS/X

10.6• exFAT: ctime meta-data replaced with the

time of file deletion [I was unable to recreate]

• Vendor Tool or Apple Implementation?• Who Validates the Test? HTCIA 2014 Conf - Aug 26, 2014

Who Validates the Validator?Superman: Easy, Miss, I've got youLois Lane: You...you've got me, who's got you?


EXFAT

Exponents and StandardsBase 2 or 10?ExponentsInternational System of Units (SI) TableIEC 60027-2Reference StandardsEndianMicrosoft MathMore Math – exFATWinCE


Base 2 or 10?


Exponents


• 102 = 10 times 10 = 100• 103 = 10 times 10 times 10 = 1000 (1K)• 22 = 2 times 2 = 4• 29 = 2*2*2*2*2*2*2*2*2 = 512• 210 = 2*2*2*2*2*2*2*2*2*2 = 1024 (1K)• 212 = 2*2*2*2*2*2*2*2*2*2*2*2 = 4096

International System of Units (SI) Table

• File System in powers of 2

• Device characteristics in power of 10

Shorthand Longhand Nth Bytes

KiB Kibibyte 210 1024

MiB Mebibyte 220 1024 KiB

GiB Gibibyte 230 1024 MiB

TiB Tebibyte 240 1024 GiB

PiB Pebibyte 250 1024 TiB

EiB Exbibyte 260 1024 PiB

ZiB Zebibyte 270 1024 EiB

YiB Yobibyte 280 1024 ZiB


IEC 60027-2


Prefixes for binary multiples

Factor Name Symbol Origin Derivation

210 kibi Ki kilobinary: (210)1 kilo: (103)1

220 mebi Mi megabinary: (210)2 mega: (103)2

230 gibi Gi gigabinary: (210)3 giga: (103)3

240 tebi Ti terabinary: (210)4 tera: (103)4

250 pebi Pi petabinary: (210)5 peta: (103)5

260 exbi Ei exabinary: (210)6 exa: (103)6

Examples and comparisons with SI prefixes

one kibibit 1 Kibit = 210 bit = 1024 bit

one kilobit 1 kbit = 103 bit = 1000 bit

one mebibyte 1 MiB = 220 B = 1 048 576 B

one megabyte 1 MB = 106 B = 1 000 000 B

one gibibyte 1 GiB = 230 B = 1 073 741 824 B

one gigabyte 1 GB = 109 B = 1 000 000 000 B

http://physics.nist.gov/cuu/Units/binary.html

How far off are we?

When we say

but mean we're this far off

1 kilobyte 210 bytes 2.4%

1 megabyte 220 bytes 4.9%

1 gigabyte 230 bytes 7.4%

1 terabyte 240 bytes 10.0%

1 petabyte 250 bytes 12.6%

1 exabyte 260 bytes 15.3%


http://cnx.org/content/m13081/1.1/

Reference Standards

• Bits are numbered right to left76543210

• Decimal Offsets (zero based)• Little-Endian numbers• Unsigned numbers• Sectors vs. Clusters• Strings are 16 bit Unicode• Strings not Terminated


Endian

• Numbering order may vary based on processor type, is determined by the order the data bytes are read from the register.

• A 32 bit number is read as 4 8-bit bytes• If I have the number 0x11 22 33 44• Big-Endian will store it as:

0x 11 22 33 44• Little-Endian will store it as:

0x 44 33 22 11


Microsoft Math

KB184006 Limitations of FAT32 File System

The maximum possible number of clusters on a volume using the FAT32 file system is 268,435,445. With a maximum of 32 KB per cluster with space for the file allocation table (FAT), this equates to a maximum disk size of approximately 8 terabytes (TB).

512B Sectors in a 32 KB cluster = 64

228 (268,435,445) * 26 (64) * 29 (512) = 243 = 8,796,093,022,208

Size of FAT32 FS specified in BPB as sectors (32 bit number) HTCIA 2014 Conf - Aug 26, 2014

More Math, exFAT

KB955704 Description of the exFAT file system driver update

package

• Support for volumes that are larger than 32 GB, the theoretical maximum volume size for FAT32 in Windows XP• The theoretical maximum volume size is 64 ZB.• The recommended maximum volume size is 512 TB.

• Support for files that are larger than 4 GB, the theoretical maximum file size for FAT32 in Windows XP• The theoretical maximum file size is 64 ZB.• The recommended maximum file size is 512 TB.


WinCEVersion Released End of Support

1.0 November 18, 1996 December 31, 2001

2.0 September 29, 1997

2.11 September 30, 2002

2.12 September 30, 2005

3.0 June 15, 2000

4.X October 9, 2007

4.0 January 7, 2002 July 10, 2012

4.1 January 8, 2013

4.2 July 9, 2013

5.X August 2004 October 14, 2014

6.0 September 2006 April 10, 2018

7.0 March 2011 April 13, 2021

2013 June 2013 October 10, 2023


EXFAT

OverviewFeatures of exFAT 1.004K (4096) Sector SizeSupported Cluster SizesFeatures of exFAT 1.00 (cont’d)Future Features of exFATMBR Partition LimitationsAdvantages of exFATDisadvantages of exFATOS Support for exFATKey Dates for exFAT


Features of exFAT 1.00

• Maximum Volume Size (Increased Capacity)• Architectural ≈ 128 PiB (232-11 * 225)• Implementation = 512 TiB

• Sector sizes from 512 [SF] to 4096 bytes [AF]• Clusters sizes to 32MiB (225)• Subdirectories to 256MiB (Root not restricted)• Maximum files on volume ≈ 232

• Maximum File Size 16 EiB-1• Built for speed, less overhead than NTFS• Catches up with some NTFS features• Template-based metadata structures• On-disk storage of file Valid Data Length (VDL)• Speeds up storage allocation processes


4K (4096) Sector Size


Supported Cluster Sizes


Features of exFAT 1.00 (cont’d)

• OEM Parameters Sector for device dependent parameters

• 12 sector VBR, support of larger boot program• Up to 2,796,202 files per sub-subdirectory• File Names max to 255 Characters• 16-Bit Unicode File Names and Volume Labels• Optimized for Flash Memory

• Device Boundary Alignment• No FAT32 minimum cluster (65,525) restriction• No 8.3 file name support (only LFN)• UTC Timestamp Support

• Vista/Server 2008 SP2+, XP/Server 2003 with KB• Native in Windows 7, 8, 8.1, Server 2008 R2, 2012HTCIA 2014 Conf - Aug 26, 2014

Future Features of exFAT

• TexFAT (To be released later)Exists in Windows CETransaction Safe exFAT

• ACL (To be released later)Exists in Windows CE

• Compression & Encryption Support?Not announced, but would be easy to add


MBR Partition Limitations

• Microsoft File Systems are limited when stored in a MBR partition

• A partition is defined by a Master Boot Record

• A MBR uses a 4 byte value for number of sectors

• LBA as 32 bit # times 512 Sector limits to 2TiB

• To get the maximum volume size, exFAT cannot be created within a MBR partition, Need GPT GUID Partition, or Super floppy Mode

• ExFAT on GPT works on Mac


Advantages of exFAT

• Large volume, file and directory sizes• Handle growing capacities in media, increasing

capacity to >32 GB.• > 1000 files in a single directory.• Speeds up storage allocation processes.• Breaks file size 4 GB barrier.• Supports interoperability with future desktop OSs.• Provides an extensible format.• Large cluster sizes• Metadata integrity with checksums


Disadvantages of exFAT

• Not all Windows CE features implemented• No direct conversion to or from other FS• Cannot use CONVERT command to NTFS• No Floppy Support• Mostly a Microsoft Desktop and Server World

• No Support for Older MS systems (Pre-XP)• Support for other devices, surfacing

• No Information Sector “Hint”• Like all FAT – Finding Stuff is via brute force


OS Support for exFAT

• Windows XP & Server 2003• KB955704 (requires SP2 or SP3)

• Vista & Server 2008 SP1• Vista & Server 2008 SP2

• (Adds UTC timestamp support)• Windows 7/Server 2008 R2 and later:

• RTM• Mac OS/X 10.6.5 and later


Key Dates for exFAT• September 2006 – Windows CE 6.0 • March 2008 – Windows Vista Service Pack 1• January 2009 – Announcement at CES of SDXC specification• January 2009 – Windows XP Drivers Available• May 2009 – Windows Vista Service Pack 2• August 2009 – Tuxera Signs File System IP Agreement with

Microsoft• March 2009 – Pretec Releases first SDXC Cards • December 2009 – Microsoft (re)announces exFAT license

program for third-parties• December 2009 – SDXC laptops due soon • December 2009 – Diskinternals releases exFAT recovery

utility• December 2009 – Encase support


More Key Dates for exFAT

• December 2009 Sony, Canon & Sanyo License• January 2010 Funai License (LCD TV)• February 2010 Panasonic License• February 2010 Panasonic 64/48GB SDXC• February 2010 Sony Memory Stick XC• February 2010 SanDisk Ultra SDXC 64GB Card 3.0 Spec

$350• April 26, 2010 DCF Version 2.0 (Edition 2010)• June 1st 2010 Tuxera Releases Linux & Android exFAT drivers• June 3rd 2010 Kingston Releases Class 10 SDXC 64GB Card

60 MB/s read, 35 MB/s write.• October 11th, 2010 FTK 3.2 with exFAT support announced


More Key Dates

• Mar 16th 2011 Lexar Releases SDXC 128GB• May 3rd, 2011 e.solutions (Volkswagen)• Aug 8, 2012 Sharp for Android Smart Phones• Sep 18, 2012 RIM (Blackberry) Smartphones• Nov 7, 2012 Sharp, Sigma, NextoDi, Black Magic and

Atomos Global• Jan 16, 2013 BMW• April 30, 2014 PS4 V1.7 update – hidden new feature: exFAT


EXFAT

Linux DevelopmentFUSE ProjectSamsung (No-FUSE)


Linux Development

• Open Source community developing FUSE• FUSE – File System in User Space

• Samsung accidently leaks native exFAT implementation, dubbed NO-FUSE

• Samsung source code on GitHUB with GPL License

• Still legal issues because of patent protection


FUSE Project


Samsung (No-FUSE)


EXFAT

Memory Cards (Including SSD)Applications (IOT)exFAT Gone WildSD Card AssociationCompact FlashSDXC Storage CapabilitiesStandard vs. Non-StandardGeneral Flash NotesSD Card Notes


Applications (IOT)

• Camera (Still, Video)• Entertainment Systems (Home, Plane, Train, & Automobiles)• GPS, Navigation Systems• Smart Phones, Audio/MP3 players• Laptop, Monitor, Printers• Handheld Computers (Tablets, Netbooks, Mobile)• Smart TVs, Home Theaters• Automatic inflight infotainment systems• Game Consoles• Medical Devices• Measuring Equipment• Other Consumer Electronics


exFAT Gone Wild

• Adoption Rate ↑• Prevalence ↑• Media Prices ↓

Storage Media larger than 32GB is being shipped out of the factory door pre-formatted with the exFAT file system

NTFS, FAT32, and HFS+ are still used in some cases but to a lesser degree


SD Card Association

• New Memory Card SDXC• Consumer Appliances• Follows SDHC• Specification for 2TB

Maximum Capacity



http://anythingbutipod.com/2009/01/next-generation-sdxc-details/

Market for SD Cards to Reach $21.3 Billion by 2018

The SD technology is employed by over 400 brands across numerous product categories and over 8,000 models, making it the de-facto industry standard. SD memory cards have been able to meet the requirements of high-end consumer devices.

http://www.storagenewsletter.com/rubriques/market-reportsresearch/global-industry-analysts-sd-cards/


Compact Flash

• Small Market• Specification 5.0 (Feb 2010)• Specification 6.0 (Nov 2010)• 48-Bit Addressing• Max Size 144PB (Up from 137GB)• UltraDMA 7 (167MBytes/s)• FAT32 won’t do (2TB Limit)• SanDisk factory preformats 256GB CF using exFAT• Not Sure Where the file system support will go,

but expect that exFAT will also become a FS of choice for other media


SDXC Storage Capabilities

• From 32GB to 2TB on a card• Exclusively exFAT File System• 312 MB/s I/O Transfer (UHS-II)• Storage (examples)

4,000 RAW images (14mb file size/64GB)

136,000 fine-grade photos

100 HD movies

480 hours of HD recording

On a single 2TB SDXC card


Standard vs. Non-Standard

• SDXC is supposed to be exFAT• In computer, you can format as

anything• Many devices, will enforce standard• Formatting SD card with OS Format

has issues and differences• Don’t assume FS based on card type


http://www.google.com/url?sa=i&rct=j&q=&esrc=s&frm=1&source=images&cd=&cad=rja&uact=8&docid=dBFkgcWP9YshlM&tbnid=4WRB-Qrw2P1zkM:&ved=0CAUQjRw&url=http://www.libertynews.com/2013/01/all-journalists-assume-everything-and-avoid-facts/&ei=DPmQU9qxLIKRqgb5wYDQCA&psig=AFQjCNEVQWDPWdo5HQBU9hg1FRavkDWgrg&ust=1402096203753746

General Flash Notes

• Write Endurance (Program Erase Cycles)

• Write Cliff• Wear Leveling• Pages (Unit of a write)• Blocks (Unit of an erase)


SD Card Notes

• SDXC Maximum set at 2TB• Two FAT Partitions within MBR• “Protected Area” and “User Area” • WinHex – Partition Offset ≠ 0• VBR differences on format/factory• AU (Allocation Unit) same as Cluster Size• Max AU = 64MiB• RU (Recording Unit) 16KB+• FAT Write Cycle {FAT1/FAT2/DIR}• exFAT Write Cycle {FAT/ABM/DIR}


EXFAT

File System InternalsRegionsFATVBRDirectoriesVolume LabelAllocation Bit MapUP Case TableFile Directory Entry Sets


File System Integrity

• Version Verified• 4 Checksums

• VBR• UP-Case Table• Directory File Set entry• Directory GUID entry

• Critical Directory Entries• Other Checks and Balances• File System should NOT mount if failures• File System may mount R/O when dirty• Dirty flags in VBR, not in the FAT


Data Hide Alert!

• FAT32 max cluster 64KiB• exFAT max cluster 32MiB

This is an increase of 512 fold• Potential for massive slack space


Volume Space Layout

• The Main Boot Region• Contains main VBR

• The Backup Boot Region• Contains backup VBR

• The FAT Region• Contains FAT Table(s)

• The Data Region (Cluster Heap)• This is where data resides


VBR – Volume Boot Record

• Contains 12 sectors1 sector main boot sector

• Jump Code (3 bytes)• Must be Zero (53 bytes)• BPB (BIOS Parameter Block)• Boot Strap Code

8 sectors main extended boot sectors (MEBS)

1 sector OEM parms1 sector reserved1 sector VBR Checksum HTCIA 2014 Conf - Aug 26, 2014

Boot Parameter Block (BPB)

• OEM Label “EXFAT ”• Volume Length (64-bit) [sector]• FAT Location & Size [sector]• Heap Location & Size [sector, cluster]• Volume Serial Number• Location of Root Directory [cluster]• Volume Flags• Sector and Cluster Sizes [2-shift]• Percent in use• File System Revision (0x0010=1.00)


Sectors & Clusters

• A 2-Shift is a power of 2• Another name for exponent

• Sector size and sectors per cluster• Each stored in 1 byte• Theoretical maximum is 2255

• Sector Size Maximum 212

• Sectors per cluster is derived• Cluster Size Maximum is 225


Executable Boot Code

• First 3 bytes of Main Boot Sector• Jump Code• 0xEB7690

• Offset 120 size 390• Remainder of boot code

• Offset 510• End signature marker• 0xAA55 = “55AA”

• Offset 512• Unused if defined


More Bootable Code

• Up to 8 Main Extended Boot Sectors• FAT32 had 3 sector VBR with 1 MEBS• Entire sector can be used for boot code• Last 8 bytes of sector is marker• 0xAA550000 = “000055AA”

• Larger capacity for boot virus!


VBR Checksum Sector

• The 12th sector of the VBR• Repeating 4 byte checksum• Checksum of previous 11 sectors• Flags and Percent excluded

• These are volatile and change often

• Boot Sector Virus & Checksum


Offset 0 1 2 3 4 5 6 7 8 9 A B C D E F

00000000 C9 D0 18 8B C9 D0 18 8B C9 D0 18 8B C9 D0 18 8B ÉÐ.‹ÉÐ.‹ÉÐ.‹ÉÐ.‹00000010 C9 D0 18 8B C9 D0 18 8B C9 D0 18 8B C9 D0 18 8B ÉÐ.‹ÉÐ.‹ÉÐ.‹ÉÐ.‹00000020 C9 D0 18 8B C9 D0 18 8B C9 D0 18 8B C9 D0 18 8B ÉÐ.‹ÉÐ.‹ÉÐ.‹ÉÐ.‹00000030 C9 D0 18 8B C9 D0 18 8B C9 D0 18 8B C9 D0 18 8B ÉÐ.‹ÉÐ.‹ÉÐ.‹ÉÐ.‹00000040 C9 D0 18 8B C9 D0 18 8B C9 D0 18 8B C9 D0 18 8B ÉÐ.‹ÉÐ.‹ÉÐ.‹ÉÐ.‹

Lines 00000050 through 01BF repeated

000001C0 C9 D0 18 8B C9 D0 18 8B C9 D0 18 8B C9 D0 18 8B ÉÐ.‹ÉÐ.‹ÉÐ.‹ÉÐ.‹000001D0 C9 D0 18 8B C9 D0 18 8B C9 D0 18 8B C9 D0 18 8B ÉÐ.‹ÉÐ.‹ÉÐ.‹ÉÐ.‹000001E0 C9 D0 18 8B C9 D0 18 8B C9 D0 18 8B C9 D0 18 8B ÉÐ.‹ÉÐ.‹ÉÐ.‹ÉÐ.‹000001F0 C9 D0 18 8B C9 D0 18 8B C9 D0 18 8B C9 D0 18 8B ÉÐ.‹ÉÐ.‹ÉÐ.‹ÉÐ.‹

VBR Checksum Sector


FAT – File Allocation Table

• When it is used, same as legacy FAT• Not used when file contiguous• Never used for cluster allocation• FAT 32 has 32 bit cells, uses 28 bits (LBA-28)• exFAT has 32 bit cells, uses 32 bits (LBA-48)

• There is no 64 bit FAT• Maximum clusters is 232-11• With TexFAT – 2 FAT Tables (2 Bitmaps)• 1st Addressed by pointer in VBR, 2nd Immed

Follows• Size stored in VBR


Reserved Cluster Index Values

• 0x00000000 – No significant meaning• 0x00000001 – Not a valid cell value• 0xFFFFFFF6 – Largest Value• 0xFFFFFFF7 – Bad Block• 0xFFFFFFF8 – Media Descriptor

• Fixed Disk• 0xFFFFFFF9-0xFFFFFFFE – Not Defined• 0xFFFFFFFF – End of Cluster Chain (EOC)


FAT Table Example

Offset 0 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15

0000 F8 FF FF FF FF FF FF FF FF FF FF FF FF FF FF FF 0010 FF FF FF FF 00 00 00 00 00 00 00 00 00 00 00 000020 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 000040 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 000060 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 000080 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 0000A0 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 0000C0 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 0000E0 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 000100 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00

Media ReservedUP-Case TableAllocation Bit Map

Root Directory


Allocation Bitmap

• Keeps track of cluster allocation status• Zero – Free Cluster• One – Allocated Cluster

• 1 Byte = Tracking of 8 Clusters• Bit Zero – Byte Zero = Cluster 2

• Cluster 0 & Cluster 1 are not defined• Addressed by Directory Entry• With TexFAT – 2 of these (FAT Pairing)


Legacy FAT vs. exFAT Chains

• When deleting a file in a legacy FAT FS the cells are wiped out

• When deleting a file in the exFAT FS the cells are not touched, regardless whether there is data in the cell

• If a file is fragmented, and is deleted, then the FAT may be still have the chain intact

*Some exFAT implementations might do it the legacy way HTCIA 2014 Conf - Aug 26, 2014

Data Hide Alert!

• The Allocation Bitmap and the UP-Case Table are stored as files, and provide hiding space in the metadata

• These files are static, typically won’t move, and have slack space.

• Nothing prevents someone from moving these files elsewhere in the cluster heap, and actually making them larger


Directories in exFAT

• Root (VBR Pointer)• Contains certain critical entries• Almost unlimited in size

• Subdirectory (by File Entry)• Contains file sets• 256MiB Max size• No physical “.” or “..” entries

• Uses 16 Bit Unicode for strings• Every Entry 32 bytes in size• Entry 0x00 is end of directory• Has capabilities for user entries


Data Hide Alert!

• Manipulation of the Allocation Bitmap, and creation of user directory entries provides the capability of hiding file within the file system

• It may also be possible to hide data within the directory metadata itself


Entry Type

Type Field Offset (Bits)

Size (Bits)

In Use 7 1

Category 6 1

Importance 5 1

Code 0 5


Entry Type

• In Use: • 0 – Not in Use, 1- In Use

• Category: • 0 – Primary, 1 – Secondary

• Importance: • 0 – Critical, 1 – Benign

• Code: Identifies the entry


Volume Label Directory Entry

• 0x83 or 0x03 Entry• Primary Entry• Only resident in Root Directory• Contains the Volume Label• 16 bit Unicode• 0x03 means no volume label (Blank

Label)


Offset 0 1 2 3 4 5 6 7 8 9 A B C D E F

00000000 83 0A 65 00 78 00 46 00 41 00 54 00 2D 00 31 00 ƒ.e.x.F.A.T.-.1.00000010 32 00 38 00 4B 00 00 00 00 00 00 00 00 00 00 00 2.8.K...........

Volume Label Directory Entry

Type

Volume Name Length (10)

Volume Label (exFAT-128K)


Allocation Bitmap Directory Entry

• 0x81 Entry• Primary Entry• Only resident in Root Directory• Points to the Allocation Bitmap

• If TexFAT, then 2 of these• Flag bits says which FAT/Bitmap

• Cluster Address of Bitmap• Size of Bitmap• NO flag for INVALID FAT


Allocation Bitmap Directory Entry

Offset 0 1 2 3 4 5 6 7 8 9 A B C D E F

0000 81 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 0010 00 00 00 00 02 00 00 00 3F 00 00 00 00 00 00 00

Type Cluster Address (Cluster 2) Size (63 bytes)


UP-Case Table Directory Entry

• 0x82 Entry• Primary Entry• Only resident in Root Directory• File names are case insensitive• Used to fold file name• Table has a checksum (32 bits)


UP-Case Table Directory Entry

Offset 0 1 2 3 4 5 6 7 8 9 A B C D E F

0000 82 00 00 00 0D D3 19 E6 00 00 00 00 00 00 00 00 0010 00 00 00 00 03 00 00 00 CC 16 00 00 00 00 00 00

Type Cluster Address (3)

Length (0x16CC = 5,836)Table Checksum


File Directory Entry Set

• Used to define a file• May have 3 to 19 entries, or more• 1 Primary, many Secondary• Is considered an array

• Must be in order• Must be contiguous (no gaps)

• Entire Set has Checksum


File Directory Entry

• 0x85 or 0x05 Entry• Primary Entry• Set Checksum (16 bits)

• Not modified on file delete• Secondary Count

• # Secondary entries that follow• File Attributes• Timestamps


Timestamps & Time Zones

• 3 Timestamps (MAC)• 32 bit DOS Date/Time

• Local Machine Time• 10ms Offset (MC)• TZ Offset (MAC)

• 15 minute increments• 7 bit signed number• ±16 hours• Present with UTC support


Timestamp Accuracy

• FAT32 – Last Access – Date only• exFAT – Last Access – Date/Time• All DOS DATE/TIME Double Seconds• 10ms adds 0-1990 ms to time• 10ms only for Create/Modify


Timestamp EXFAT

CreationTimeStored in UTC if available, else in local time

10 millisecond granularity

LastAccessTimeStored in UTC if available, else in local time

2 second granularityChangeTime Not Supported

LastWriteTimeStored in UTC if available, else in local time

10 millisecond granularity

Timestamps


Timestamp Reliability

• Timestamps appear to be updated when the file is created or modified.

• Last Accessed Timestamp appear to be updated when file is created or modified.

• Last Accessed Timestamp appear NOT modified on file read.

• Forensics Implication on MAC time analysis


File Attributes

Attribute Offset Size Mask

Reserved2 6 10

Archive 5 1 0x20

Directory 4 1 0x10

Reserved1 3 1

System 2 1 0x04

Hidden 1 1 0x02

Read-Only 0 1 0x01


File Directory Entry

Offset 0 1 2 3 4 5 6 7 8 9 A B C D E F

0000 85 04 D4 92 20 00 00 00 44 62 86 3B F1 62 BA 3A 0010 44 62 86 3B A8 00 EC EC EC 00 00 00 00 00 00 00

Type # Secondary Entries

Set Checksum (0x92D4)

Attributes (0x0020 = Archive)

Create

Modified

TZ Offset CMA EC = GMT-5

Accessed

Create 10ms

Modified 10ms


Formatted File Directory Entry

Root Entry Type Read is: 85 Directory Entry RecordChecksum: 92D4Calculated Checksum is: 92D4 Size Directory Set (bytes): 160Secondary Count 004File Attributes: 0020 Archive Create Timestamp: 3B866244 12/06/2009 12:18:08Last Modified Timestamp: 3ABA62F1 05/26/2009 12:23:34Last Accessed Timestamp: 3B866244 12/06/2009 12:18:08 10 ms Offset Create A8 168 10 ms Offset Modified 00 0 Time Zone Create EC 236 Value of tz is: GMT -05:00 Time Zone Modified EC 236 Value of tz is: GMT -05:00 Time Zone Last Accessed EC 236 Value of tz is: GMT -05:00


Stream Extension Directory Entry

• 0xC0 or 0x40 Entry• Secondary Entry• Length of Name• Length of File (2 of them)• Cluster address of first data block• Name Search Hash value• Secondary Flag

• FAT Invalid• Allocation Possible


Stream Extension Directory Entry

Offset 0 1 2 3 4 5 6 7 8 9 A B C D E F

0000 C0 03 00 28 AD 3C 00 00 1F 46 1D 01 00 00 00 000010 00 00 00 00 05 00 00 00 1F 46 1D 01 00 00 00 00

Entry Flags (Alloc Possible/Fat Invalid)

Length of File Name (0x28= 40)

Name Hash (0x3CAD)

Cluster (5)

Data Length 0x011d461f = 18,695,711


Parameters for Samples

Bytes Per Sector: 2 to the 09 power is: 512Sectors Per Cluster: 2 to the 08 power is: 256Bytes per Cluster: 131072 (128K)


Formatted Stream Extension

Root Entry Type Read is: C0 Directory Entry Record, Stream ExtensionSecondary Flags: 03 Flag Bit 0: Allocation Possible Flag Bit 1: FAT Chain InvalidLength of UniCode Filename is: 40Name Hash Value is: AD3CStream Extension First Cluster 5Cluster 5 is AllocatedStream Extension Data Length 18695711 Bytes Slack: 83487 Clusters Used: 143Stream Extension Valid Data Length 18695711 Bytes Slack: 83487 Clusters Used: 143


File Name Extension Directory Entry

• 0xC1 or 0x41 Entry• Secondary Entry• Secondary Flags

• Allocation not possible• FAT Invalid

• 15 Characters (30 bytes) of Name• Name in 16 Bit Unicode• In order (FAT32 LFN was reversed)• Up to 17 max, total 255 character


File Name Extension Directory Entry

Offset 0 1 2 3 4 5 6 7 8 9 A B C D E F

0000 C1 00 62 00 75 00 73 00 69 00 6E 00 65 00 73 00 Á.b.u.s.i.n.e.s.0010 73 00 5F 00 6F 00 66 00 5F 00 73 00 65 00 63 00 s._.o.f._.s.e.c.

0000 C1 00 75 00 72 00 69 00 74 00 79 00 5F 00 5F 00 Á.u.r.i.t.y._._.0010 62 00 75 00 73 00 2D 00 31 00 30 00 35 00 2D 00 b.u.s.-.1.0.5.-.

0000 C1 00 33 00 32 00 6B 00 62 00 70 00 73 00 2E 00 Á.3.2.k.b.p.s...0010 6D 00 70 00 33 00 00 00 00 00 00 00 00 00 00 00 m.p.3...........

File Name = business_of_security__bus-105-32kbps.mp3


Significance of “not in use” flag

• 0x05, 0x40 & 0x41 Entries• “Not in use” may mean deleted

files• May also be reallocated rename

• Set Checksum not changed when entries marked “not in use”


EXFAT

ClosingProblems ObservedSummaryQ&AContact InformationReferences


Problems Observed

• Looking at Forum Posts• Google Dork on “exFAT”• People getting thrown into exFAT and Lost• Conversion between exFAT & Fat32/NTFS, How-to• Corruption between Windows and Mac• Should File Defragmentation be done?• Repartitioning• Timestamp differences, and incompatibilities• Vendor cross compatibility• Chkdsk not cleaning disk• Users want large files (>4GB) not Large Volumes


Summary

• exFAT is still a relatively new FS• Need for exFAT support in forensics tools ↑• Inconsistent Implementations of exFAT• Compatibility across OS needed• Tools & Utilities Need Improvement• Need to Tool Up


Q&A


Contact Information

• E-mail: [email protected]@mindspring.com

• Blog: rshullic.wordpress.com• Blog: shullich.blogspot.com• Linkedin: www.linkedin.com/in/RobertShullich• Twitter: rshullic

Credit Cookie


mailto:[email protected]

mailto:[email protected]

http://www.linkedin.com/in/RobertShullich

NTFS 232-1 Clusters

Cluster size NTFS Max Size 512 bytes 2,199,023,255,040 (2TB)

1024 bytes 4,398,046,510,080 (4TB)

2048 bytes 8,796,093,020,160 (8TB)

4096 bytes 17,592,186,040,320 (16TB) (Default)

8192 bytes 35,184,372,080,640 (32TB)

16384 bytes 70,368,744,161,280 (64TB)

32768 bytes 140,737,488,322,560 (128TB)

65536 bytes 281,474,976,654,120 (256TB) (Maximum)


ReFSResilient File System

Coming to a Windows System soonhttp://blogs.msdn.com/b/b8/archive/2012/01/16/building-the-next-generation-file-system-for-windows-refs.aspx


http://blogs.msdn.com/b/b8/archive/2012/01/16/building-the-next-generation-file-system-for-windows-refs.aspx



References

Sans Reading Room:http://www.sans.org/reading_room/whitepapers/forensics/rss/reverse_engineering_the_microsoft_exfat_file_system_33274

SANS Summit ExFAT Presentation:exFAT (Extended FAT) File System – Revealed &

DissectedJeff Hamm & Robert Shullich, July 2010

https://digital-forensics.sans.org/summit-archives/2010/10-exfat-ham.pdf


http://www.sans.org/reading_room/whitepapers/forensics/rss/reverse_engineering_the_microsoft_exfat_file_system_33274

http://www.sans.org/reading_room/whitepapers/forensics/rss/reverse_engineering_the_microsoft_exfat_file_system_33274



References

Microsoft Patent US8583708, “Extensible File System”

Retrieved June 9, 2014 fromhttps://www.google.com/patents/us8583708

Microsoft Patent US8321439, “Quick Filename Lookup Using Name Hash”. Retrieved 06/09/2014 from

https://www.google.com/patents/US8321439


https://www.google.com/patents/us8583708

https://www.google.com/patents/us8583708


References

Microsoft Patent US8606830, “Contiguous file allocation in an extensible file system” retrieved 06/09/2014 from

http://www.google.com/patents/US8606830

Microsoft Patent US8024383, “Fat directory structure for use in transaction safe file System” retrieved 06/09/2014 from



http://www.google.com/patents/US8606830


References

ExFAT overviewhttp://ntfs.com/exfat-overview.htm

Data Recovery Concept: Extended File System (exFAT)

http://www.active-undelete.com/xfat_overview.htm

CIPA Standard DC-009-2010 (DCF)http://www.cipa.jp/std/documents/e/DC-009-2010_E.pdf

CIPA Standard DC-008-2012 (Exif)http://www.cipa.jp/std/documents/e/DC-008-2012_E.pdf

Comparison of File Systemshttp://en.wikipedia.org/wiki/Comparison_of_file_systems


http://ntfs.com/exfat-overview.htm

http://www.active-undelete.com/xfat_overview.htm

http://www.cipa.jp/std/documents/e/DC-009-2010_E.pdf



http://en.wikipedia.org/wiki/Comparison_of_file_systems

http://en.wikipedia.org/wiki/Comparison_of_file_systems

References

The Extended FAT file system - Differentiating with FAT32 file system - Keshava Munegowda , Venkatraman S, Dr. G T Raju

http://events.linuxfoundation.org/images/stories/pdf/lceu11_munegowda_s.pdf

File System Functionality Comparisonhttp://msdn.microsoft.com/en-us/library/windows/desktop/ee681827(v=vs.85).aspx




http://msdn.microsoft.com/en-us/library/windows/desktop/ee681827(v=vs.85).aspx

http://msdn.microsoft.com/en-us/library/windows/desktop/ee681827(v=vs.85).aspx

Resume

http://jjcweb.jjay.cuny.edu/d4cs/faculty/Shullich Robert.pdf


http://jjcweb.jjay.cuny.edu/d4cs/faculty/Shullich%20Robert.pdf

http://jjcweb.jjay.cuny.edu/d4cs/faculty/Shullich%20Robert.pdf

Introduction to the Microsoft exFAT File System

Documents

Transcript of Introduction to the Microsoft exFAT File System