This presentation is to provide a technical understanding of the Microsoft Extended File Allocation Table (exFAT) for forensics examiners.

It will also provide general knowledge of exFAT.

1

One of the is to provide the background and history of the file system, and the relationship to the other FAT versions in the family.

2

3

D4CS stands for Digital Forensics & Cyber Security

FCM 760 Fall 2009

4

This presentation was given 5 times within the 14 months since the SANS paper was published.

It will now be given twice in 2014, with some updates

At the 4/19 Computer Forensics Show, the session was recorded and can be found on the AT&T Tech Channel

5

Both books were published in 2012, other books have been published, and may mention exFAT, but these two send the reader to the SANS paper for more detailed and in-depth information.

6

Brian Carrier’s book is considered by some as the gold standard and bible on explaining file systems.

It has become a little outdated since it is about 9 years old, no 2nd edition, and other file systems have surfaced since the book’s release.

Earlier in 2014 exFAT support was planned/announced for TSK

7

This is the Sleuth Kit Wiki, on this page it is noted that exFAT support was added to TSK, and one of the docs used as a guide during development was the SANS paper.

8

Microsoft published a patent that included the exFAT 1.00 specification.

This presentation and the paper attempt to stick to the terminology used in the patent/specification as close as possible.

Links to the patent and my paper will be on a later slide, and references to the paper will also be on my blog.

The presentation will focus on desktop/server version of exFAT which in 2014 is still Version 1 00 as is still released on Windows 8Version 1.00, as is still released on Windows 8

There are differences between implementations of other vendors, including incompatibilities, and the Windows phone has advances features and also uses compression.

The technology behind this SD card feature is called Content Protection for Recordable Media (CPRM)Recordable Media (CPRM).

Content Protection for Pre-recorded Media (CPPM),

9

10

http://www.webopedia.com/TERM/O/OSR_2.html

OEM Service Release 2 – Windows 95B

According to Wikipedia, there was a 8 bit FAT originally

Ray Duncan (1988). The MS-DOS Encyclopedia - version 1.0 through 3.2. Microsoft Press. ISBN 1-55615-049-0.

Although we talk about FAT12/16/32, there were many flavors as the FAT family evolved.

11

exFAT is specifically designed for Removable media, but can be used for fixed media as well.

NTFS is not recommended for removable media, especially because of the lazy write problem.

Faster I/O through less file system overhead

Limitation on how many times you can write to a single electronic gate (e g nand)Limitation on how many times you can write to a single electronic gate (e.g. nand)

FAT in general is simpler, so in the case of embedded systems, stick with a variation of FAT instead of implementing NTFS

12

SD = Standard Definition

HD = High Definition

Quad HD is 4 times Full HD (Double wide, Double Long)

Amount of data being recorded depends on many factors, including frames per second, size, color size, resolution, and compression rations.

Definitely will exceed the 4GB file size limitation.

13

Source: https://www.sdcard.org/consumers/cards

14

15

•You need to be able to locate the evidence, just in general

•This includes re-assembly when a file is fragmented

•Also includes recovery of deleted files

•You also need to know the hiding places where it can be hidden

•For example, unallocated space

•You need to validate what you found is correct, in order (proper assembly), and complete (no missing pieces)

•CP (Child Porn) when created uses cameras, and as camera memory gets cheaper, and moves to exFAT, relevant evidence is going to be on exFAT

16

Don’t be a money pressing a button, need to know what is under the covers

17

If the OS can’t recognize the file system, then it thinks the media is not formatted.

When this slide was built, it was 2010, on a Windows XP machine, that did not have the hotfix.

This example is Microsoft specific, and with XP being retired, and later systems getting the support, this situation should not occur often anymore on Microsoft Systems.

Back when this happened this message would lead one to believe that the mediaBack when this happened, this message would lead one to believe that the media was blank and unformatted.

18

Any evidence with exFAT would probably be pushed aside with the lack of tools, documentation and expertise to process it.

Tools that were available were raw acquisitions and then data carving

Data carving easier and automatic when the file is contiguous

19

Linux and Open Source is used a lot for examinations

Commercial tools are lacking, but picking up

Little documentation or publications on exFAT internals, 4 years later and SNAS paper appears the authoritative resounse.

exFAT Computer Compatibility

http://gopro.com/support/articles/exfat-computer-compatibility

We are not there yet

https://www.cyberfetch.org/sites/default/files/EnCase%20Forensic%20v6.18.0.59%20Test%20Report pdf0Test%20Report.pdf

Test Results for Deleted File Recovery and Active File Listing:

EnCase Forensic Version 6.18.0.59

This report was prepared for the Department of Homeland Security Science and

20

This report was prepared for the Department of Homeland Security Science and Technology Directorate Cyber Security Division by the Office of Law Enforcement Standards of the National Institute of Standards and Technology.

For additional information about the Cyber Security Division and ongoing projects, please visit www.cyber.st.dhs.gov.

http://www.cftt.nist.gov/presentations/AAFS-2013-Lyle-DFR.pptx

AAFS = American Academy of Forensic Sciences

http://www.cftt.nist.gov/presentations/AAFS-2013-Lyle-DFR.pdf

6 Vendors, not named in the presentation

21

Encase: http://www.cyberfetch.org/sites/default/files/EnCase Forensic v6.18.0.59 Test Report.pdf

FTK: http://www.cyberfetch.org/sites/default/files/FTK v3 3 0 33124 Test Report.pdf

Access to Test Images

Layout of test ImagesLayout of test Images

Each test case is repeated at least four times to characterize the tool’s behavior for different file system families. These include FAT, exFAT, NTFS and ext. The NTFS and exFAT images contain a single partition. The FAT and ext images each contain three partitions. Each partition has the same pattern of files created and deleted for a gi en test case The FAT and e t cases (three partitions) ha e three times asa given test case. The FAT and ext cases (three partitions) have three times as many files as the NTFS and exFAT cases (one partition). The FAT images contain a FAT-12, a FAT-16 and a FAT-32 partition.

The FAT partitions were created on a Windows Vista system. Some partitions marked as FAT-12 in the partition table, appear to have a FAT table that is actually FAT 16 (thi did t i ifi tl ff t t t lt ) Th NTFS i lFAT-16 (this did not significantly affect test results). The NTFS images were also created on a Microsoft Windows Vista system.

The ext partitions were created on a Fedora Linux system.

The exFAT partition and HFS+ partitions were created on a Mac running Snow Leopard, OSX Version 10.6. 22

23

24

When we use the term “Mega”, is it 1,000,000 (1006) 0r do we mean “Mega” 1,048,576 (220) ?

25

A quick note on exponents, since we will get our hands dirty with math

Some simple numbers should be like learning the times table in school

26

In some cases you might see ZB or ZIB, technically they are really different, but are close.

So when we say 1 kb of disk, they mean 1,000 bytes, but when we say 1 kb of memory, they mean 1024 bytes.

IEC 60027-2 A.2 and ISO/IEC 80000

http://physics nist gov/cuu/Units/binary htmlhttp://physics.nist.gov/cuu/Units/binary.html

27

Just another slide

It is suggested that in English, the first syllable of the name of the binary-multiple prefix should be pronounced in the same way as the first syllable of the name of the corresponding SI prefix, and that the second syllable should be pronounced as "bee."

28

Being off by 15% when talking about an "exabyte" means being off by about bytes, or 150 petabytes.

29

exFAT uses 16 bit Unicode strings

This is the terminology as used in the specifications leaked in the patent

When reading the paper, and as we discuss here, these are the ground rules in terminology

30

It is important to note that Pentium processers use the little-endian format, so numbers stored in the file system are stored in little-endian. This can be significant because you need to change the order of the bytes in order to read the values from a hex dump.

This could have issues with support of exFAT in other architectures, and could affect acquisitions.

The exFAT specification requires little endian.

31

This is how Microsoft does Math, and then everyone uses these numbers not knowing the full context

232 sectors * 29 bytes per sector (512B) = 241 = 2,199,023,255,552 (2TB)

http://support.microsoft.com/kb/184006

32

http://support.microsoft.com/kb/955704

Volume size of 64ZB is architecturally incorrect. Currently it cannot exceed 128PiB because:

1) With 232 clusters (32 bit fat indices) tracking clusters with a maximum of 225 in size = 257 = 128 PiB [32+25]

2) With LBA 48 as the maximum addressable block on the FS with a 512B physical2) With LBA-48 as the maximum addressable block on the FS, with a 512B physical sector, the file system supported would be 257 = 128 PiB [48+9]

Note that with #2, we could go further with native AF 4K sectors, however it is interesting how the numbers add up.

For file size, the current architecture uses 64 bit numbers for the length of file, based on that the maximum (theoretical) file size is really 264-1 = 16EiB

Now, since the architecture limits the filoe system to less than 128 PiB, and PIB is smaller than EiB, the maximum file size is almost the volume size minus overhead and metadata.and metadata.

33

http://en.wikipedia.org/wiki/Windows_CE

Microsoft Windows CE (now officially known as Windows Embedded Compactand previously also known as Windows Embedded CE

Small footprint, limited API

Windows XPE, XP Embedded – Different, uses desktop code but not all features

WinCE code is used to derive code for other embedded systems including the phone

http://support.microsoft.com/default.aspx?scid=kb;EN-US;Q166915p pp p ; ;Q

34

35

You never really see another sector size other than 512 bytes, but everyone just assumes that it is only 512 (in earlier versions of Windows NT, there were weird sector sizes)

The 4096 size is special to support a device that is used for paging and supports 4K pages. But with the standard format, you can’t adjust sector size

http://en.wikipedia.org/wiki/Advanced_Format – Advanced format is for any sector size > 520 bytes.

Clusters (or blocks) are 64KiB max in FAT32

However, for Windows 95 OSR2, 98 and ME, the FAT32 clusters had a maximum of 32KiB, so for backward compatibility it was recommended to not exceed 32KiB

ExFAT and FAT32 Root Directories not restricted in size, other than space available on the volume.

Max files on FAT32 volume ≈ 228 while exFAT ≈ 232 This is based on a maximum of one file per allocation unit (each cell in the FAT is one allocation unit)

36

one file per allocation unit (each cell in the FAT is one allocation unit)

Since exFAT supports empty files that don’t take up space in the cluster heap (length = 0 first cluster = 0) Max Files theoretically could be more

Ref: http://support.microsoft.com/kb/955704

This new format, called Advanced format, provides via hardware 4K physical sector size.

An OS/FS can either use emulation (512e) or native (4Kn)

exFAT will support 4K sized sectors.

37

This command (help format) was issued on a Windows 7 32-bit system.

This snippet is for the allocation unit size.

Most “supports” largest clusters for 512b sectors at 64K, including NTFS, FAT & FAT32. FAT & FAT32 appear to support a larger allocation units of 128K and 256K when sectors are > 512b (probably AF sectors).

Although in some cases 64K allocation units are supported, not all OS support it, and in some cases 64K+ not supported and must be a power of 2 thus 32Kand in some cases 64K not supported, and must be a power of 2, thus 32K.

Differences between Windows 95/98/ME and Windows NT4/2000/XP, 7 & 8

Even though a FAT32 could lead to 8TB (for 32K) and 16TB for 64K cluster sizes, when putting into a MBR, LBA is 32 bits, a block is 512b, so all file systems in a MBR is restricted to 2TB

Might get 8TB if AF (4K sectors) are used.

38

Microsoft in the KB for Windows XP support indicated a capacity to 64ZiB and a file size maximum to 64ZiB.

In reality, the file system can only support up to 128PiB, and the file size up to 16EiB.

Microsoft documentation indicates a maximum file system size as 512TiB

The recommended maximum volume size is 512 TBThe recommended maximum volume size is 512 TB.

http://support.microsoft.com/?kbid=955704

The volume size is limited by a 32-bit FAT and a 25-bit cluster size giving a 57-bit addressable volume size

The file size is limited by the 8-byte (64-bit) number that holds the file size.The file size is limited by the 8 byte (64 bit) number that holds the file size.

The volume label and file names are all 16 bit unicode

Filenames to a maximum of 255 characters

Subdirectory is max at 256MiB, Directory records are 32 bytes, and the smallest fileset is 3x32 = 96 bytes and assumes no ACL and a filename < 16 characters in length.

exFAT better optimized, reduce the “write” actions

For media that use OEM parameters this may be a method to convey device39

With TexFAT there will be 2 FATS and 2 BITMAPS, with exFAT 1.0 – which does not have TexFAT (Transactional FAT) support, there is ony 1 FAT and 1 BITMAP, where previous FAT versions had 2 FATs.

To be released later, but it is 6 years and we are still at VV.MM 01.00

TexFAT and ACL already exist in Windows CETexFAT and ACL already exist in Windows CE

40

Any FS is limited, even FAT32 and NTFS.

This is Windows only, we are not talking GUID Partition Table (GPT)

Although a MBR uses a 4 byte sector count, remember that the FS can be larger if you make the sectors larger (512 vs. 4096) and this causes a lot of confusion on how big a FS fits.

A FAT32 filesystem could reach 8 TiB in size (2**28 x 32K), but with 512 sector sizes a MBR can only support 2TB (with 4K size a MBR can support 16TiB)sizes, a MBR can only support 2TB. (with 4K size, a MBR can support 16TiB)

http://support.microsoft.com/kb/314463

http://techcosupport.com/press/maximum-size-of-a-fat-32-partition/

GUID Limits:

http://support.microsoft.com/kb/302873http://support.microsoft.com/kb/302873

http://msdn.microsoft.com/en-us/library/windows/hardware/dn640535(v=vs.85).aspx

A GPT GUID Partition uses a 64 bit number for the number of logical blocks

In theory, a GPT disk can be up to 2^64 logical blocks in length. Logical blocks are commonly 512 bytes in size.

41

This would be 264 * 29 = 273 which is 270 = 1 ZiB and 23 = 8 for ≈ 8Zib

maximum partition size of 264−1 sectors. For disks with 512-byte sectors, that would be 9.4 ZB (9.4 × 1021 bytes) or 8 ZiB−512 bytes (9,444,732,965,739,290,426,880 bytes or 18,446,744,073,709,551,615 (264−1) sectors × 512 (29) bytes per sector)

Windows would not format FAT32 beyond 32GB, it required using a FAT32 format on a different OS

Some Windows utilities did not work properly with volume spaces GT 32GB, but you can mount a device that was GT 32GB

Limitations of FAT32 File System: http://support.microsoft.com/kb/184006

SDXC predecessor (SDHC) had a max spec of 32GB. SDXC picks up from 32GB. (But starts around 48GB 32GB will till be SDHC for a while)(But starts around 48GB, 32GB will till be SDHC for a while)

4GB maximum file size barrier existed in both FAT and FAT32.

SD 4.0 Specification – 300MB/s I/O speeds

http://www.flashmemorysummit.com/English/Collaterals/Proceedings/2009/20090813_S204_Lin_Yee.pdf

Starting at 104 mega bytes per second, and later to 300 mega bytes per second

http://www.letsgodigital.org/en/20985/sdxc-cards/

Microsoft set limits on FAT32 volume size

In one argument, older utilities could not format the volume correctly or could not determine the proper size

42

In another argument, since the larger volumes had a much larger FAT, massive reads of the FAT would be required to find free space. For example, with a 32K cluster size and a 32GB media, the FAT would be about 4MB, and for a heavily used (low free space) volume there could be a lot of I/O to find free clusters. FAT32 limited this overhead by adding a hint of freespace using the FAT32 File System Information sector, but it was a hint and not always to be relied upon, just to point the software to where free cluster might be

3rd party file utilities may provide conversion to and from exFAT, but no Convert command, and current convert command doesn’t work even to change exFAT to NTFS or even FAT32.

Mostly a Microsoft Desktop and Server World – there is Linux, and MAC, Microsoft dominates

43

There are discussions of creation of exFAT on a Vista or Windows 7 machine that can’t be seen on Vista. This is usually a case of creating the media on a machine with exFAT support and then trying to read the media on a different machine without exFAT support. The common mistake is creation of the file system on removable media with a Vista SP1 (or higher machine) and trying to read it on a machine with Vista RTM.

44

45

The SDXC media will not be backward compatible\e-solutuions/volkswagon for in vehicle entertainment systems

DCF 2.0 – Design Rule for Camera File System

Camera and Imaging Products Association (CIPA) – DC-009-2010

Japan Electronics and Information Technology Industries Association (JEITA) CP3461BCP3461B

Exchangeable image file format (officially Exif, not EXIF according to JEIDA/JEITA/CIPA specifications) is a standard that specifies the formats for images, sound, and ancillary tags used by digital cameras (including smartphones), scanners and other systems handling image and sound files recorded by digital cameras The specification ses the follo ing e isting file formats ith the additioncameras. The specification uses the following existing file formats with the addition of specific metadata tags: JPEG Discrete cosine transform (DCT)

[1] for compressed image files, TIFF Rev. 6.0 (RGB or YCbCr) for uncompressed image files, and RIFF WAV for audio files (Linear PCM or ITU-T G.711 μ-Law PCM for uncompressed audio data, and IMA-ADPCM for compressed audio data).

[2] It is not supported in JPEG 2000, PNG, or GIF. (Source Wikipedia)

46

This is just a selection of some dates, not exhaustive.

Volkswagon and Audi licensed Tuxera drivers to get exFAT support.

http://www.thesixthaxis.com/2014/04/30/the-hidden-features-and-changes-of-playstation-4-firmware-1-70/

The 1.70 PS4 firmware update is rumored to have exFAT support

47

Tuxera

http://www.tuxera.com/products/tuxera-exfat-embedded/

Provides exFAT & NTFS to be integrated in embedded systems.

They do software development and integrate the support

They have a working relationship with MS, and have licensed exFAT. Their development then gets integrated for other companies.

However, there are developers working on their own to build and generate drives for Linux.

48

The Linux community is very hostile to Microsoft, complaining that a Microsoft standard was forced on them and Microsoft expects to get paid.

However, that is what they are stuck with.

Meanwhile the community is trying to build exFAT drivers for the various Linux distributions in order to support exFAT.

IN 2013 someone leaked the source code to Samsung’s exFAT kernal basedIN 2013 someone leaked the source code to Samsung s exFAT kernal based drivers. Eventually Samsung put those drivers under a GPL license and released them as open source.

Even through there is now a GPL license for Samsung’s exFAT implementation, I would expect that Microsoft will want their royalties and there may be legal issue ahead.

E er since the Tom Tom settlement companies are afraid of iolating Microsoft’sEver since the Tom Tom settlement, companies are afraid of violating Microsoft’s patent rights.

49

50

51

52

I use a CPAP machine, and it has a SD card to record my sleep patterns

These are the uses, many of these devices take USB, SD, CF, and even memory stick

Most of these are embedded systems, but produce or consume media that can/or was processed on desktop systems

53

SANDISK ships high capacity CF cards pre-formatted as exFAT

PNY 128GB Turbo USB flash

DigiStore SSD

Some Kingston Memory

Even some magnet disk media, example Western Digital

Pre-formatted file system is not usually specified in the product specs, so it is hard to determine unless you see discussions or go out and buy & test everything

In 2010 a 64GB SANDISK SDXC card was selling on Amazon for $350, in May 2014 I bought 2 of these at $49.50 each (with free tax & free shipping)

54

New Devices may accept SDXC, but older devices might not.

Mentioning memory cards such as camera cards is important because the target market for exFAT is removable storage.

Back in 2009 I believed that this was going to be a big driver towards exFAT adoption but flash memory in any type of memory card or even SSD drives mayadoption, but flash memory in any type of memory card, or even SSD drives may have contributed to the adoption rate and prevalence.

55

With Sony adopting the XC memory stick to exFAT, plus the SD market, is almost 90% of the market today.

http://anythingbutipod.com/2009/01/next-generation-sdxc-details/

Jan 8, 2009

56

July 2012

57

The 137GiB comes from LBA-28 addressing.

228 = 268,435,456

228 * 29 = 128GiB = 137GB = 137,438,953,472

144PB comes from LBA-48 addressing

248 = 281,474,976,710,656

248 * 29 = 128PiB = 144PB = 144,115,188,075,855,872

58

SD – up to 2GB – FAT

SDHC – 2GB to 32GB FAT32

SDXC – 32GB to 2TB exFAT

These are marketing hype and represent maximums

http://www.dpreview.com/news/2009/1/8/sdxc

http://www.computerworld.com/s/article/9125622/Memory_card_standard_could_provide_up_to_2TB_on_an_SD_card

The 64GB SDXC card, for instance, can store a full 16 hours of 1080p High-Definition footage (1920x1080 9Mbps H.264 AVC compression) or over 4000 RAW images (based on 14MB file size), ideal for situations that demand continuous burst-images (based on 14MB file size), ideal for situations that demand continuous burstmode shooting and non-stop video recording.

Source: http://dk.transcend-info.com/About/press/10044

The size of a photo will vary based on the camera resolution and the effectiveness of compression.

Transcend (from the statement above) indicates 4,000 Raw images based on 14MB file size. This was for a 64GB card, while the SD press releases were saying 4,000 images per 2TB card, something is off.

And in the case of just taking the Nikon D7100 DX, RAW images are really almost59

Nand gate wear and tear, less write, longer the memory may last

Although the SD association states that the official, standard and only file system for SDXC is exFAT, users will format the card using other file systems.

Some cameras may allow a SDXC card to be formatted as FAT32, but others will indicate that the SD Card is not formatted properly and ask to format it.

Yet, a user may format the SD Card in another file system, and use it in a non-compliant device such as a slot on the laptop where only the OS will read and writecompliant device, such as a slot on the laptop where only the OS will read and write the card.

60

Write Endurance (Program Erase Cycles)

Limited, maybe up to a million, writes

Writes require the storage area to be erased first, almost like an EPROM

Flash memory, nand and nor gates, should not be full formatted unless needed – do a quick format. Good for forensics because data is not erased

Degrags should not be done either, flash memory doesn’t have moving parts so and the extra writes wear down the solid state chipsthe extra writes wear down the solid state chips.

Writes need to be limited, the less writing the longer the memory will last and the better the performancebetter the performance

Write Cliff

Blocks are rotated and pre-erased, but if you run out of empty blocks, then you have to wait for a block to be erased before you can reuse it for a different set of values.

Wear Leveling

Methods of providing a block of gates, and rotating through the gates.

61

Why a 2TB limit when the CF has a 144PB limit?

Looks like SDXC uses a MBR partition to separate the protected area from the user data area, and that limits volume to 2TB

The follow-on the SDXC will need to use a GPT, a partition would probably still be required with a separate protected area.

CD does not have DRM/copy protection, so extra partition probably not required. Then issue is superfloppy or GPTThen issue is superfloppy or GPT

Format of a SDXC card puts standard boot code in the boot sector, while cards from the factory ha all those fields filled up with F4

Need to format card using SD Formatter utility and inside the camera – both cases to see what is going into the sector.

AU Sizes vary based on size of total volume.

62

63

Currently use exFAT 1.00, but if a later version of exFAT is in use, it will check the version # and not mount the FS unless it can suppoort it

Checksums protect against corruption and viruses

If there is a problem with critical directory entries, the FS should not mount.

The dirty flag used to be in the 2nd FAT index in FAT32

64

65

4 Regions defined on the volume

The FAT tables reside outside the cluster heap

Everything except the data region is measured in and addressed as sectors.

Data region is measured and addressed as blocks, blocks are called clusters

66

FAT and Cluster heap have their own offsets, which allow alignment, if needed to force these region on a designated boundary line

Might be needed in SD and other flash memory

The specification for exFAT says the # of FATs is either 1 or 2.

For legacy FAT it is recommended to be 2, could be 1, but could even be more although rarely seenalthough rarely seen.

Since there are offsets, I could build a 3rd, 4th or more FAT – just stick space between the 2nd FAT and the cluster heap start and have an area of slack space.

67

Details follow in the next slides

A mirror of the VBR follows, and is a backup VBR

In case the first gets corrupted

FAT32 had a mirror also, the mirror was at sector 6

68

When you take the volume length (64-bit) * 4K sector, that is 64+12 = 276.

270 = 1 ZiB, 26 = 64, thus based on this value, a file system of 64 ZiB. However, the current architecture specification cannot produce a file that big unless some parameter somewhere gets changed.

69

If there was no restriction, then the size of a cluster could be 4255

70

If the sector size is > 512 bytes, all space on the first sector of the VBR (Main Boot Sector) is not used (Only the first 512 bytes)

71

Unlike the first sector, the other 8 boot sectors can use the entire sector and the signature marker is moved to the last 8 bytes of the sector

72

If a virus modified the boot record, and doesn’t fix the checksum, there should be a mount failure

73

Repeats over and over again, 4 bytes = 32 bit checksum

Can be used to determine if the VBR was modified

3 bytes in the VBR are not calculated in the checksum

This sector does not have a signture

74

The BITMAP is used to track cluster allocation, and the FAT is only required for re-assembling the original file. If the original file is contiguous, then the FAT isn’t needed for THAT file. We will see later that a flag in the directory record is used to tell the FS whether the FAT should be used or ignored.

For Contiguous Cluster Allocation see Patent: US8606830B2 (Contiguous File Allocation in an Extensible File System)

For TexFAT see Patent: US7613738B2 (FAT Directory Structure for use in Transaction Safe File System)

For Extensible File System see Patent: US8583708B2 (Extensible File System)

75

Because there is no floppy support, there is only one possible media descriptor value

Cluster 0 and 1 are not defined, so 0 & 1 are not significant (Same as legacy FAT)

Since the FAT is no longer used for cluster allocation, 0 (zero) is no longer significant (used to mean the cluster was unused/free/unallocated)

FF..F9 thru FF..FE were also EOC (end of cluster chain) markers in FAT32, but are unused in exFATunused in exFAT.

Values 2 thru FF..F6 are cluster addresses.

The Cell’s content is called an index or indices

1st indices contains FFFFFFF8 (Media Descriptor)

2nd indices contains FFFFFFFF and is not used to hold dirty volume flags

76

77

The 3 main critical records: Allocation Bitmap, UP-Case Table, and Root Directory will use FAT chains.

The Root Directory can grow and since it is dynamic in its growth, most likely will fragment.

The UP-CASE Table and Allocation bitmap should be static and not grow or change, although theoretically they could probably be relocated and moved somewhere else on the volume.

However, in the SD standard, the allocation bitmap must be within the first 4MB of the cluster heap.

The locations (cluster addresses) of the 3 special metadata files may change, this is based on one formatting and in reality these files could eventually end up in any cluster.

These 3 special files (4 if TexFAT) will use chains even if unfragmented becauseThese 3 special files (4 if TexFAT) will use chains even if unfragmented because even though they appear in the cluster heap, they are not true user files, and are defined by special directory records and not file records to point to these special system files. These special files don’t have the INVALID FAT flag.

78

If there are 2 FATs in a TexFAT Transactional Safe exFAT environment, then each FAT is paired with a allocation bitmap

The allocation BITMAP is pointed to by a 0x81 entry.

To locate an empty cluster, each 512 byte sector would hold 4096 allocation bits (512 bytes x 8 bits)

More efficient but still require reading many sectors if the FS is large and the earlyMore efficient, but still require reading many sectors if the FS is large and the early part of the FS is allocated.

79

When files on legacy are fragmented, and deleted, the deletion wipes out the FAT chain because each FAT cell has to be zeroed out to indicate that the cell is no longer allocated.

Since allocation is moved from the FAT to the Allocation Bitmap, the FAT cells for the chain remain intact.

There is still the possibility of cells being overlaid by other file fragments after the delete.

This is not in the spec, buit this behavior has been observed.

80

If the files are made larger, then code can be hidden into those files

The SD Specification indicates that the Allocation Bitmap must be within the first 4MB of the file system.

81

This is an eye chart, but the idea is to show how to get to the bitmap.

You start at the VBR (BPB), go to the root directory, look up the 0x81 entry to get the cluster address, and then go into the BITMAP table.

The first byte of a directory record is the entry type, here we see x’83’, X’82’ and X’81’

82

We will see details of the directory entry construction later, including what we mean by an entry type.

If there are issues with the critical entries, then the file system should not mount.

In FAT the largest directory size is 221 Which equals 2,097,152

http://read.pudn.com/downloads77/ebook/294884/FAT32%20Spec%20(SDA%20Contribution) pdfntribution).pdf

In FAT, with a directory size of 221 and a director record size of 25 (32 bytes) the number of 32 character entries in the directory is 216 = 65,536

Keep in mind that the maximum number of files will be related to the FAT itself since not more than a single file can occupy a cluster.

In exFAT the largest directory size is 228 = 256MiB

exFAT directory size limit is 27 (128) times the size of the FAT limitation

83

Benign directory records

Fake secondary records

Zero length/zero cluster files

Phantom / orphan Files

84

The first byte of every directory entry is the “entry type” and describes the directory entry.

85

When a file set is not in use, it is usually (but not always) a deleted file

When a volume label is not in use, it means no volume label

In a file set, it could be caused by renaming a file with a longer file name.

Only files have secondary entries so far

Missing Benign entries usually won’t prevent the file system from being mounted.

0x80 is not defined.

In FAT32, file deletion is done by overlaying the first byte with 0x’E5’, or X’05” if the first byte of the filename is already an 0x’E5’.

For KANJI character set based names, the value 0x05 is stored in DIR_Name[0] - if required - to represent 0xE5.

86

Primary and Critical

In legacy FAT, the Volume label is in the Root Directory, and has an attribute

87

Since we use 16 bit unicode without string termination, we need the length of the volume label – in unicode characters.

88

Primary and Critical. If the FS can’t find the BITMAP table, it can’t mount the FS

Since there is no flag, this file will always have a FAT chain, even if it is one cluster, will always have a cluster chain ending with EOC

89

This was a small volume. 63 bytes can support maximum of 63x8 = 504 clusters.

90

Filenames are stored case insensitive, so when a search is done, the filenames are converted to upper case (folded). The UP-CASE table is used to convert the filename to all uppercase.

91

The UP-Case table is less than 6K – imagine if it was in a 32K cluster, now imagine if it was in a 32MB cluster, the amount of available slack space.

92

File Entry Set would have a File, Stream Extensions, and up to 17 File Name Extension for a total of 19.

Later, when a new exFAT version comes out, the ACL will be another secondary entry bringing this up to 20.

As more file secondary entries are added, let’s say one for encryption, this increases to a max of 255 secondaries.

93

Attributes and Timestamps in later slides

Checksum is across the Primary and all secondaries in the set.

94

Modified, Access, and Create.

Timestamps are NOT stored in this order, but MAC is a common acronym in the literature.

Timestamps are not one single field like NTFS which uses a 64 bit value. exFAT combines pieces to make a UTC value.

TZ offset is absent in Vista SP1, and does not appear in the exFAT 1.00 spec.

Note: By default, the creation time is tunneled if a file is deleted, and a file with the same name is created within 15 seconds.

(See KB172190 http://support.microsoft.com/kb/172190)

95

The standard DOS Date/Time, also used in the previous FAT versions, does not count to the second, but double seconds.

To get seconds, a 33 bit number would have been needed.

The OS doesn’t always update last access.

And even NTFS last access is disabled in some versions, can modify behavior with” fsutil behavior set disablelastaccess 0fsutil behavior set disablelastaccess 0

Not sure if FAT32/exFAT is relaiable

96

97

FAT and exFAT timestamp behavior varies, but is just not reliable as far as last accessed.

TSK research shows some differences between OS, so timestamp analysis could be very inconsistent

Even in later Windows releases, NTFS doesn’t even update the Last Accessed on READ for performance reasons but this behavior can be restored via a registry keyREAD for performance reasons, but this behavior can be restored via a registry key.

98

These are pretty much the same as previous FAT versions.

Since we have a separate volume label entry, there is no attribute for it, and since we don’t have 8.3 support, there is no LFN (Long File Name) attribute either because everything is LFN.

Reserved1, which is mask 0x08 was ATTR_VOLUME_ID (0x08) in legacy FAT

99

The update behavior on the 10ms Modified is also not predictable, sometimes it is just set to zero.

Note that the create time is really 3B866244 (reversed because of little-endian)

100

In order to validate the analysis in reverse engineering the FS, I had to write a C program to format the directory entries.

This is an example of the output.

All the timestamps are even because of the double seconds. But since the create is 168, this means that the create time was really 12:18:09.68

Secondary count is 4, meaning that this file set is 5 entries, 1 File, 1 Stream, and 3 filenamefilename.

101

There is 2 file lengths, one is supposed to be the physical file length and the other the amount of data actually written into the file so far (Valid Data Length - VDL)

These are two 64 bit length and are similar to the two lengths in NTFS in the $FILE_NAME 0x30 attribute.

Length of name is needed because there is no string termination, but the file name (max 255) may require multiple directory entries (we will see later).

This is where the FS indicates whether the FAT is used if the FAT Invalid flag is setThis is where the FS indicates whether the FAT is used, if the FAT Invalid flag is set, then the FAT is ignored.

In legacy FAT

The cluster number of the first cluster of the file is recorded in the directory entry associated with

the file. For zero-length files, the first cluster number in the associated directory entry is set to 0.

exFAT also supports a first cluster of zero if the length is zero.

One of the Lengths is called “DataLength” Field

102

One of the Lengths is called DataLength Field

The specification states: If the corresponding file directory entry describes a directory, then the valid value for this field is the entire size of the associated allocation, in bytes, which may be zero. Further, for directories, the maximum value for this field is 256MB. The other length field is called “ValidDataLength” field says that if this is for a directory, then this value must match the DataLength field.

103

Since these values can vary based on the format parameters, for reference this is what the samples in this presentation is using.

104

Another output from the C program.

Allocation possible indicates that the directory entry specifies a cluster address field

FAT invalid indicates that this file does not use the FAT

This file is 18MB and required 143 clusters to store the file.

As we said before, there are 3 filename entries (each holds 15 characters of the filename), and as we see above, the filename is 40 characters in length.

105

Allocation not possible indicates that there is no cluster address in the entry.

FAT Invalid has no meaning

106

Filename is 40 characters (80 bytes) and takes 3 entries to store it.

Notice that in Uni-Code the file name is stored in mixed case

107

When the entries are not in use, some may be overwritten, and some may not. This means that a complete set may not exist.

108

109

There are discussions of creation of exFAT on a Vista or Windows 7 machine that can’t be seen on Vista. This is usually a case of creating the media on a machine with exFAT support and then trying to read the media on a different machine without exFAT support. The common mistake is creation of the file system on removable media with a Vista SP1 (or higher machine) and trying to read it on a machine with Vista RTM.

Microsoft distributes a specification, each vendor writes their own drivers, so variations between vendors, causing compatibility issues are occurring

Users try for format drive on Windows system, drive is >32GB, the only options are: exFAT & NTFS, they format in exFAT and then find out their device doesn’t work in other places due to lack of exFAT support.

The drive of a user to get away from a device in FAT32 is the 4GB barrier.

110

New, but 8 years old, misunderstood

More forensics tools need exFAT support

Implementations across vendors are inconsistent and might not implement all features

Needs to be fixed before it gets worse

Even utilities for Disk Partition, Defragmentation, File Recovery, and commands like CHKDSK need exFAT supportCHKDSK need exFAT support

More evidence is going to show up in exFAT format, need to acquire the right tools and get experience

111

112

I need followers

113

Since NTFS has a smaller maximum cluster size (64K) 216 (29 * 27 = 216) while exFAT maximum is set to 225. Then the question is: What happends to NTFS?

114

http://www.snia.org/sites/default/files2/SDC2012/presentations/File_Systems/JRTipton_Next_Generaltion-3.pdf

115

My paper on exFAT and the Microsoft Patent that exposes the specification

116

117

I encountered these other sites that have information on exFAT.

I include them here to provide more information for the reader

118

119

120

121