Beruflich Dokumente
Kultur Dokumente
Are you limited by 128 bytes of EEPROM on your MCU or even the few kilobytes of flash in
your project? Instead of just downloading a library like Petit FAT File System Module and
following blindly a tutorial on how to customize it to your microcontroller and SD card, would
you like to really understand what you are doing, and maybe learn a bit about filesystems and
SPI in the process?
In this first part of my FAT and SD tutorial, we’ll take a SD card image, and create a simple C
program to interpret its contents. For this part, you don’t need any hardware at all, just a
computer with gcc (GNU C Compiler) or any other ANSI C compatible compiler installed.
I then launched HxD in Administrator mode to allow it to open physical disks. Note that you
don’t open just the one FAT16 partition (“H:”), but the whole removable disk (I circled the icon
you need to click first):
Because the partition data, file system and few test files only occupy only the very beginning
the 1 GB disk, I then proceeded to select only the first 1 MB (Edit > Select block… > Offset 0-
FFFFF) and copy-pasted that into a new file, which I then saved as test.img in my project
folder. If you don’t have a smallish SD/micro-SD card at hand, you can just grab the project
zip and use my test image.
Note: If you’re using Linux or a Mac, you can just use dd if=/dev/sda
of=~/test.img bs=1M count=1 to create the image (assuming the kernel
selected /dev/sda as the device for your card). You may need to add “sudo” in the
beginning.
#include <stdio.h>
#include <stdlib.h>
int main() {
FILE * in = fopen("test.img", "rb");
unsigned int i, start_sector, length_sectors;
fread(&start_sector, 4, 1, in);
fread(&length_sectors, 4, 1, in);
printf(" Relative LBA address %08X, %d sectors longn", start_sector, length_sectors);
}
fclose(in);
return 0;
}
Save this as read_mbr.c in the same folder that you saved test.img, and compile
with gcc read_mbr.c -o read_mbr.exe and run:
Pretty easy, huh! But reading every field manually gets cumbersome after a while, so we could
just define everything in a nice struct. Here’s the modified read_mbr2.c:
#include <stdio.h>
#include <stdlib.h>
typedef struct {
unsigned char first_byte;
unsigned char start_chs[3];
unsigned char partition_type;
unsigned char end_chs[3];
unsigned long start_sector;
unsigned long length_sectors;
} __attribute((packed)) PartitionTable;
int main() {
FILE * in = fopen("test.img", "rb");
int i;
PartitionTable pt[4];
fclose(in);
return 0;
}
Note that I printed out only the minimum needed information this time.
The __attribute((packed)) is needed so the compiler won’t align our data structure
fields to 32-bit or 64-bit boundaries (by default, it likes to pad structures with empty areas so
the processor can handle the data a bit faster) and make a mess. Also, I’m using a compiler
where sizeof(char) is 1, sizeof(short) is 2 and sizeof(long) is 4, adjust your
data types if you’re using a different system or compiler!
Reading and interpreting the boot sector
Based on the previous output, it is rather evident that the first partition is the one we want to
examine further. I now advise you to read the chapter “FAT and the Boot Sector”
from CompuPhase tutorial. Based on the tutorial, the boot sector for a FAT16 file system
should nicely fit into this structure:
typedef struct {
unsigned char jmp[3];
char oem[8];
unsigned short sector_size;
unsigned char sectors_per_cluster;
unsigned short reserved_sectors;
unsigned char number_of_fats;
unsigned short root_dir_entries;
unsigned short total_sectors_short; // if zero, later field is used
unsigned char media_descriptor;
unsigned short fat_size_sectors;
unsigned short sectors_per_track;
unsigned short number_of_heads;
unsigned long hidden_sectors;
unsigned long total_sectors_long;
To make the code a bit more robust, I used a for loop that stops when it finds the first partition
with compatible partition type (4, 6 or 14), so the partition entry is in pt[i]. To move inside
the image file, we’ll first use the fseek() command, then read the boot sector:
fseek(in, 512 * pt[i].start_sector, SEEK_SET);
fread(&bs, sizeof(Fat16BootSector), 1, in);
I added a few printout commands and saved the result into read_boot.c – you should
understand it easily when comparing with read_mbr2.c. The results of a test run can be seen
on the right.
PUBLISHED BY
Joonas Pihlajamaa
Coding since 1990 in Basic, C/C++, Perl, Java, PHP, Ruby and Python, to name a few. Also
interested in math, movies, anime, and the occasional slashdot now and then. Oh, and I also
have a real life, but lets not talk about it! View all posts by Joonas Pihlajamaa
Posted onApril 2, 2012AuthorJoonas PihlajamaaCategoriesElectronicsTagsdisk image, fat, fat16,file allocation table, hex
editor, hxd, sd, tutorial
4. BJsays:
April 27, 2012 at 08:51
Excellent tutorial, im looking forward to the 4th installment. You may need to check this page
though as I believe you have a typo “partition table should start at offset 0x1CE”, i think it is
0x1BE as in the example code.
Your teaching style is great and hope to see more.
Regards,
BJ
REPLY
1. jokkebksays:
April 27, 2012 at 09:44
10. Mikesays:
March 10, 2013 at 06:28
Interesting page, but I think your information is a little off. Not all SD card volumes will have a
partition. See this page for more thorough coverage: http://www.maverick-
os.dk/FileSystemFormats/FAT16_FileSystem.html
REPLY
1. jokkebksays:
March 10, 2013 at 11:03
Yes, SD cards can also be formatted in a different way, but all the cards I had followed the
structure I’m describing in the tutorial, so I skipped the more advanced details for simplicity. :)
Thanks for the link, it’s a useful addition to the post!
REPLY
11. rogersays:
June 7, 2013 at 00:39
hello, when i see the last sector that occupies the HEMLET.TXT (cluster 13), theres is some
finland text or something,which is at offset 0x7F600. Why could it be there??
Thanks a lot.
REPLY
1. jokkebksays:
June 9, 2013 at 14:36
Nice find. :) I did some testing with the disc image before deciding what to write there, and as
FAT doesn’t remove old data, just marks sectors free, I guess it was still there.
So if I had used zero-formatted image all unused data would be zeroes, but now there’s some
old data in unused area of the sector.
REPLY
12. SMsays:
June 20, 2013 at 03:03
I now advise you to read the chapter “FAT and the Boot Sector” from CompuPhase tutorial.
REPLY
1. Joonas Pihlajamaasays:
December 10, 2013 at 10:21
Thanks for pointing that out. I actually forgot http:// from the beginning so the link was just
broken. :)
REPLY
13. Cristisays:
August 18, 2013 at 15:06
I’m confused. Removable drives don’t have MBR, they only have a single PBR. Why do you
have a MBR on a removable drive?
REPLY
1. jokkebksays:
August 18, 2013 at 15:40
At least according to http://www.compuphase.com/mbr_fat.htmmany CF and SD cards are
arranged similarly as hard disks, and do contain a MBR. It’s been 18 months since I read more
about the subject and I’ve forgotten most of the details, but at least the SD cards I had did
follow the structure outlined in the tutorial. I think USB flash sticks and possibly some SD
cards could be simpler.
REPLY
1. Cristisays:
August 19, 2013 at 21:50
http://en.wikipedia.org/wiki/Master_boot_record
“MBRs are not present on non-partitioned media like floppies, superfloppies or other storage
devices configured to behave as such.”
I use a USB microSD card reader to format the microSD, that is basically seen as a
“Removable Disk”, and all “Removable Disks” that I have don’t hold a MBR. BOOTICE also
confirms this by having the button “Process MBR” grayed out and the hex editor because the
first sector starts with PBR.
Anyway, is still a great tutorial.
Thanks!
REPLY
1. jokkebksays:
August 20, 2013 at 10:01
So if you make a raw copy of the SD card, the first sector actually starts with stuff that is
described in “Reading and interpreting the boot sector”? Anyways, I’ll add a note to my MBR
section, that there are (micro)SD cards that don’t have it at all – I did encounter it when doing
research for the tutorial, but as all my SD cards had MBR, didn’t remember to mention it at all.
Thanks for the information!
REPLY
1. Cristisays:
August 20, 2013 at 10:32
14. Vladimirsays:
August 29, 2013 at 19:45
1. Joonas Pihlajamaasays:
August 29, 2013 at 21:17
No I think it is consistent: If the SD is formatted like a hard disk with many partitions, its first
sector (0) is the MBR, and partition table starts at offset 0x1BE. If there are no reserved sectors
between MBR and first partition, the first partition’s boot sector is next (sector 1), also 512
bytes.
So 0x1BE is a relative offset in sector 0 (MBR). Boot sector (sector 1 or further) has different
data at 0x1BE of that sector.
What may be confusing is the sector,offset method of addressing stuff instead of just linear
offset – if you’re viewing a disk image in hex editor, sector X offset 0x1BE is at X*0x200 +
0x1BE (for example 0x3BE for sector 1). As a car analogy, you could think a sector as a bus
with 512 seats. First bus (MBR) seat 0x1BE is the beginning of partition table, but there is a
seat 0x1BE in every bus after that, including the boot sector which is usually the second bus,
right after MBR. :)
REPLY
1. vladimirsays:
August 30, 2013 at 17:23
1. Joonas Pihlajamaasays:
September 2, 2013 at 09:44
Yeah, I think also that if not running as administrator, HxD doesn’t even offer the option to
read whole raw device, just the partitions.
REPLY
2. Filbysays:
June 13, 2014 at 17:16
It is the same with WinHex. You have to open the physical disk if you want to see *all* of the
disk. The logical disk will just show the current volume/partition.
REPLY
15. Sebastiansays:
December 3, 2013 at 05:57
im a noob trying here. I got the first exempel successfully compiled and it printed out the
Partition Tabel just fine. Im using the test.img you provided.
But when i printing out the boot sector it dosent match with the image you are showing. Jump
code and OEM code match, but after that it derails.
Is it my compiler somehow, or is the test.img diffrent from the test.img printed on the
read_boot.png image?
1. Joonas Pihlajamaasays:
December 9, 2013 at 23:12
The compiler might be the issue, e.g. different size for short or not respecting the __packed__
attribute (i.e. don’t pad fields to 32-bit boundaries). You could print out the field offsets with
something like:
Not quite sure about the casting but a substraction and proper cast should do the trick.
I used gcc from MinGW project myself, what platform and compiler you are using?
REPLY
1. Sebastiansays:
December 11, 2013 at 05:12
Yes, you are right. After some digging around i found this:
http://gcc.gnu.org/bugzilla/show_bug.cgi?id=52991
The last comment said there was a patch for this for some time now, but i dont know where i
can get it, or apply it or anything.
So i almost quite. But some fooling around i found some guy do this: #define PACKED
__attribute__ ((__packed__))
I got it to match the result you are showning. So im reading thru this tutorial now :)
REPLY
Really Nice Tutorial. I liked your iterative approach to develop projects. I will adapt that.
REPLY
17. Mironsays:
February 1, 2014 at 07:23
Good job! Exactly what I need! Thanks a lot for full description of this material!Good luck for
your posting!
REPLY
18. Srinivas Nistalasays:
August 16, 2014 at 01:24
19. Markussays:
September 3, 2014 at 16:20
20. serbansays:
October 28, 2014 at 11:31
Hey, first of all this is a great tutorial!! I’m reading it step by step and it’s very detailed…but I
have a question about modified date and time packed shorts in root directories entry. If second
is a value (between 0 and 31 – 5 bits) which represent a 2 second interval, any value of second
should generate a even number (0, 2, 4, 6, etc.). Why in your example the file README.TXT
has the time 20:31.27? 27 is not even…
REPLY
1. Joonas Pihlajamaasays:
October 28, 2014 at 17:00
Haha, sharp observation! I did wonder this myself, but I think the most obvious explanation is,
that I forgot to multiply the field value with 2 when printing it out. So it is actually 20:31:54…
:)
REPLY
1. serbansays:
October 29, 2014 at 10:42
I think that, too! I observed that because I have read the tutorial many times, just to understand
it clearly.
Hi, great light implementation of FAT. Quick question about casts from the fat16_buffer to
different stucts. Why you cast first void * and then struct *? Wouldn’t simply casting directly to
struct * be sufficient?
REPLY
1. Joonas Pihlajamaasays:
October 28, 2014 at 22:24
That is also an excellent question. There seems to be no reason for cast. Only one I can think
that some compiler I used warned about cast between incompatible types. Or then I just had a
dull moment when writing those lines. :)
REPLY
22. larrysays:
November 14, 2014 at 04:50
I used Tiny C Compiler for read_boot.c and get different results using your test.img file. Don’t
know what the problem is but it falls apart at sector_size reporting 8194 instead of 512 and
continues to give erroneous results from there
REPLY
1. Joonas Pihlajamaasays:
November 14, 2014 at 16:31
Most likely is that TCC has different way to specify a packed struct (i.e. no alignment), in
which case sizeof(structure) is larger and some fields are read into wrong place. I this is the
case, googling for “Tiny C compiler struct alignment” (or packing). Or just get gcc and make
some tests to determine what is different. Sizes of ints, shorts and longs is the most usual
reason.
REPLY
1. larrysays:
November 14, 2014 at 18:03
I downloaded gcc 4.9.2 and get the same results. I tried read_mbr.c compiled with both tcc and
gcc and get different results from your example. For example “Partition start in CHS” yields
02:04:00 instead of 00:04:02. the results I get are in the same order that is shown in HxD
starting at 0x1BF. The same happens in “Partition end in CHS” where i get 04:E4:C9 instead of
C9:E4:04. I don’t understand why the difference between the read_mbr.exe from the project
folder vs what I compile.
REPLY
1. Joonas Pihlajamaasays:
November 14, 2014 at 23:12
Hmm, sounds like a possible endianness problem – if you have 0x01, 0x02, 0x03, 0x04 in a file
and you read those into a dword, some systems will handle that as 0x01020304 and other
0x04030201. Although I cannot think of a hardware platform nowadays that would not
conform to the one Intel is using (as Macs are also on Intel platform). Weird. Well at least you
know the bytes themselves seem similar to what there’s supposed to be…
REPLY
2. larrysays:
November 14, 2014 at 23:27
1. Joonas Pihlajamaasays:
November 15, 2014 at 13:19
Ah, 64-bit compiler. I compiled mine on 32-bit version of the MinGW. It does return the same
values as you, first_byte = 0, start_chs = 1, length_sectors = 12, and the sizeof(PartitionTable)
= 16. Does that struct read OK?
What do you get for sizeof(Fat16BootSector)? I get 518 (weirdly enough, would’ve thought
512) and sector_size member offset is 12.
REPLY
1. larrysays:
November 15, 2014 at 19:01
I really like your tutorial and wish that when recompiling the C code things would be the same.
What I’m after is a light weight SD card data logger for Arduino instead of the will do every
possible combination for FAT12 16 and 32 cards that the SD.h arduino library performs at the
expense of using about 50% of the atmega328p program space.
I’m not at all a proficient C program but have learned a lot trying to debug what is happening.
Thanks for your time answering my issues.
2. larrysays:
November 16, 2014 at 06:17
I discovered that after a char or series of chars (oem[8] or printing 3 char results in a row) the
compiler skips the next location. When it gets to volume label it has skipped five locations thus
giving the output
“Volume label: [ME FAT16]”. Now I need to figure out how to fix this. Maybe try byte instead
of char.
2. Nhat Minhsays:
November 21, 2014 at 16:41
Hi Larry,
This my solution:
Minh
3. larrysays:
December 12, 2014 at 18:28
Hi Minh,
I solved the problem using MinGW32 g++ before seeing your solution. I tried your solution
using __attribute((gcc_struct, __packed__)) on my “offsetof_example.c” code and get a correct
result. I did need to add an extra underscore to attribute so that it would compile correctly.
__attribute((gcc_struct, __packed__)) not _attribute((gcc_struct, __packed__))
23. fabiansays:
November 19, 2014 at 03:02
Hey I have no problem with the EEPROM, but my problem is that I have a 512 RAM..is it
possible to adapt or to make FAT16 to use less RAM?
REPLY
1. Joonas Pihlajamaasays:
November 19, 2014 at 15:33
If you read the next part of the tutorial, you’ll notice that my AVR library has a memory
footprint of 51 bytes + some stack, so 512 should definitely be enough (128 bytes of SRAM
would work, too).
REPLY
24. larrysays:
December 12, 2014 at 18:06
typedef struct {
char c;
short s;
int i;
long l;
float f;
} __attribute((packed)) sfoo;
int main ()
{
printf (“offset of char c is %d size of %d\n”,offsetof(sfoo,c), sizeof(char));
printf (“offset of short s is %d size of %d\n”,offsetof(sfoo,s), sizeof(short));
printf (“offset of int i is %d size of %d\n”,offsetof(sfoo,i), sizeof(int));
printf (“offset of long l is %d size of %d\n”,offsetof(sfoo,l), sizeof(long));
printf (“offset of float f is %d size of %d\n”,offsetof(sfoo,f), sizeof(float));
}
with MinGW 32bit compiler I get offset of 2 between char and int and should be 1. Must be
adding a null byte after a char.
Installed MinGW32 and compiled using g++. Offset between char and int is now 1 byte. I am
now able to recompile all of the programs in the tutorial and get them to work.
Minh,
Thanks for your code. I haven’t tried it yet as I just saw it today will posting this comment.
REPLY
25. Jespersays:
December 23, 2014 at 00:28
Hi Guy,
}
ans prints : ” FILE: [SUBDIR . ]”
And It treats SUBDIR such as normal file.
Please help me explain at this point.
Thanks a lot,
Minh
REPLY
27. Adithyasays:
February 14, 2015 at 12:38
Hey,thanks a lot for the tutorial! I’m stuck at reading the partition data after creating an image
of my sd card. I’m getting bizarre partition type values viz 72,65,79,0D for the partition entries
0-3 respectively. Can you help me out? I’m using a 512MB sd card formatted with a FAT32
filesystem.
REPLY
28. Artsays:
February 18, 2015 at 20:36
Hey,Really thanks for the Tutorial. And many thanks for the comments which helped me a lot!
REPLY
29. Marcosays:
October 19, 2016 at 13:27
Wonderful tutorial :)
REPLY
Great Page! I’m in the process of trying to integrate an SD with FAT at the moment and I found
your page was extremely informative. I’m still working my way through all of the information.
But I wanted to say thanks!