How to read the MFT on NTFS

[Beans4You]

I read an article at http://www.pcguide.com/ref/hdd/file/ntfs/arch_MFT.htm on the MFT (Master File Table). I have been trying for some time to find a way to view the MFT, and have come up empty handed. The MSDN library states that you can type dir /a $mft to view the size of the file. I try this at the command line and get a file not found error. I would really like to be able to view the contents of this file. If you have any information on this topic I would greatly appreciate it.

The language of preference is VC++ 6.0. My ultimate goal is to read the MFT and write its contents to a flat file. Any help would be greatly appreciated.


Thanks

Eric Cashwell

[Paul Komski]

There's more than one file to read. The various metadata files can be viewed from a command line by typing "dir /ah filename" at the root of the boot drive (ex: dir /a:h $BOOT)

see:- The Evolution of NTFS: NTFS 1.1 (http://www.arstechnica.com/paedia/n/ntfs/ntfs4-2.html)

other info:- File - $MFT (0) (http://linux-ntfs.sourceforge.net/ntfs/files/mft.html)

[Beans4You]

Paul,

Thank you for the information and links. However, I still cannot use the dir command to view the files. Is there any thing that must be activated for me to use this function? (See attached gif screen shot)

Thanks

Eric Cashwell

[Paul Komski]

Hi. Hope this isn't a silly question, but, is your installation of Win2K on a FAT or NTFS partition?

[Beans4You]

I'm running Win2K on a NTFS partition. But I also tried to run the same command on a XP box which was running on an NTFS partition and I received the same results.

I downloaded a program called NTFSINFO.exe which displays your MFT info along with some other info. I downloaded this software at SysInternals (http://www.sysinternals.com/ntw2k/source/ntfsinfo.shtml). Now I don't know what exactly this is telling me, but it appears to be able to find the MFT metadata files.

Eric

[Paul Komski]

Can't play with the files in parallel because I'm installed on a FAT partition. They are "complicated" files but lie at the heart of the WinNT OS stability and security. I remember (and have since forgotten most of it) finding out quite a lot about them on a forensic pc site - but can I find it now; you guessed it; nope. Good Luck. :)

[Beans4You]

If you happen to find the site I would appreciate it if you remember me. I can't find very many people that know much about the MFT. - Or if they do they're not very helpful. Thanks for all your help.

Eric

[Paul Komski]

ixl aka Charles aka "The Boss" is likely to know, but is very busy these times. If I find good info I will post back. :p

[hawk7771us]

try here http://www.forensics-intl.com/define.html it may or may not help try here http://www.winntmag.com/Articles/Index.cfm?IssueID=27&ArticleID=3455 i hope it helps i do not know if it will im new at this game.

[Beans4You]

Thanks, the sites are full of great info about the MFT and gave me a glimps of what the tables should look like, but still no luck in how to read from them.

[hawk7771us]

http://www.google.com/search?hl=en&ie=UTF-8&oe=UTF-8&q=forensic++master+file+table&btnG=Google+Search this might be what your looking for there are a lot of sites here to look at.hope this helps

[videobruce]

How large should this MFT Zone be? On my drives they are 650MB and 1GB from converted FAT32 drives that were orginally less than 65MB.

[Beans4You]

Mine is ~50Mgb

[Paul Komski]

Are we talking about the size of the MFT or the MFT zone?

My current setup is:

NTFS Information Dump
Copyright (C) 1997 Mark Russinovich
http://www.ntinternals.com

Volume Size
-----------
Volume size : 4996 MB
Total sectors : 10233404
Total clusters : 1279175
Free clusters : 614645
Free space : 2400 MB (48% of drive)

Allocation Size
----------------
Bytes per sector : 512
Bytes per cluster : 4096
Bytes per MFT record : 1024
Clusters per MFT record: 0

MFT Information
---------------
MFT size : 25 MB (0% of drive)
MFT start cluster : 262144
MFT zone clusters : 268640 - 422048
MFT zone size : 599 MB (11% of drive)
MFT mirror start : 639587

Meta-Data files
---------------



You can read this info using NTFSinfo (12kb) from http://www.sysinternals.com/ntw2k/source/ntfsinfo.shtml

Unzip it, and place the ntfsinfo folder in the root of your ntfs partition (makes it easiest to find).

From the command prompt navigate to the ntfsinfo folder and type ntfsinfo x (where x is the drive letter of the relevant partition).

;)

he he - this is such an old thread, that I didn't see that beans had already used ntfsinfo!

also - I could never get that other dir command to see the file(s); but I think it maybe only works in WinNT

and - for those that don't know it already. It is easy to copy from the command prompt window. RClick and choose Mark. Highlight relevant text. Press Enter (this copies it to the clipboard). Go where you want it and Paste. ;)

[Beans4You]

Yes it is an old thread but my original question has never been answered.

I could never get the DIR command to work either.

As far as the size I think that it is telling me that mine is ~50Mgb and that, I believe, is the MFT File not the zone.

If you know of any way to read the MFT I am still anxious to know. I have heard that it's the fastest way to read the directory of a NTFS system drive.

[Paul Komski]

I keep thinking I'm getting near - and keep getting side-tracked!

Running
dir /t:a /a /s /o:d x: >> x:\meta.txt

... should (theoretically) create a text file in the x partition listing every file on it.

In XP it creates a listing of all files and directories (including System Volume Information) but does not list what is inside SVI !!

That's the angle I'm following-up now, since one isn't allowed access to that volume in windows in any case.

There must be some permissions to crack somewhere; perhaps as simple as logging-on as administrator and not me with admin rights.

But I'm off to bed.

:D

[videobruce]

MFT Zone.

Here is the Main drive:

Volume Size
-----------
Volume size : 8307 MB
Total sectors : 17012771
Total clusters : 17012771
Free clusters : 12424326
Free space : 6066 MB (73% of drive)

Allocation Size
----------------
Bytes per sector : 512
Bytes per cluster : 512
Bytes per MFT record : 1024
Clusters per MFT record: 2

MFT Information
---------------
MFT size : 19 MB (0% of drive)
MFT start cluster : 16646
MFT zone clusters : 39680 - 113568
MFT zone size : 36 MB (0% of drive)
MFT mirror start : 8776406

Now here is the backup drive with the mirror image of the main drive:


Volume Size
-----------
Volume size : 5726 MB
Total sectors : 11727386
Total clusters : 1465923
Free clusters : 1113051
Free space : 4347 MB (75% of drive)

Allocation Size
----------------
Bytes per sector : 512
Bytes per cluster : 4096
Bytes per MFT record : 1024
Clusters per MFT record: 0

MFT Information
---------------
MFT size : 6 MB (0% of drive)
MFT start cluster : 4
MFT zone clusters : 1632 - 183264
MFT zone size : 709 MB (12% of drive)
MFT mirror start : 732961


How's that?

[Beans4You]

videobruce:
I can get that info already. What I'm looking into is being able to get inside the MFT. I have several programs that I wrote with different algorithms to parse through the hard drive of a NTFS volume. However, there is no need for me to parse each and every folder recursively if I could just get my hands on the MFT.

Paul Komski:
That's not exactly what I'm looking for. What I would like to be able to find the MFT, open the database up, and just copy what is in it. I have actually seen flat files on the internet of how the MFT is organized and how it looks. There must be a way to get in it!

- Eric

[Paul Komski]

videobruce
There's one fundamental difference in the two setups, with one using a cluster size 8 times larger than the other. This would effect the different values in both installations. One or the other might be slighly more efficient depending on what your average file size is but not nearly as significant as with a FAT volume.

You may (or may not) have been given a choice at the formatting stage to choose the cluster size and a new default was chosen without you knowing or noticing.

As long as the MFT itself has'nt "grown" out of proportion, then the size of the MFTzone is essentially immaterial and is neither wasted space nor anything that would slow things down.

Beans4You
I would like to find out how to access it directly too but I'm coming round to the opinion that you need a third party utility to access/read/write to it (something like a hex editor) and that the dir commands, already referred to, cannot be made to work from either Win2K or XP's command prompts - however the attributes are played around with.

Guess it would be the same with the FATs on Win9x - even though we know that all the MFT entries are de facto files.

The SVI is a red-herring; its only the restore files for XP, which I hadn't noticed before.

[Beans4You]

Paul Komski:
Have you been able to find the physical location of these MFT files?

I know that people have been able to edit the MFT. I have seen defragmenters built for it and Norton's Ghost and Partition Magic both inform you that they are checking and reading the MFT.

- Eric

[Paul Komski]

ntfsinfo gives you the starting cluster - so that's where I would look to first.

[Paul Komski]

More info - but cant find that $MFT :D

fsutil fsinfo ntfsinfo X:
displays some mft info including its location (if X is ntfs of course).

140365 - Default Cluster Size for FAT and NTFS
http://support.microsoft.com/default.aspx?scid=kb;en-us;Q140365

JSI Tip 4936. How do I locate and correct disk space problems on NTFS volumes in Windows XP?
http://www.jsifaq.com/SUBJ/tip4900/rh4936.htm

JSI Tip 2709. Manage the size of the Master File Table Reserved Zone on Windows 2000.
http://www.jsifaq.com/subf/tip2700/rh2709.htm

[Beans4You]

When you run that command, do you get mft and mft2 information?

I do...

I'll look over your links as soon as I get the kids off to school. :)

- Eric

[Whyzman]

Paul, just wondering if you had the original command prompt correct. It appears that the : was not supposed to be part of it... Rather, dir /ah $mft

A bit late, but then again, I'm a slow learner...:rolleyes:

[Beans4You]

Whyzman:

I have seen it written both ways during my research.
Neither worked for me.

- Eric

[Whyzman]

Beans, what about this:

DIR $MFT /ASH

Here's the link:

http://www.c-ad.bnl.gov/kinyip/windows/MFT.html

[Paul Komski]

Beans Yes - both of them.

Whyzman :( Have tried it in so many different combinations. It doesn't work for me in XP or 2K.

eg:
=========================

J:\>dir $Boot /ASH
Volume in drive J is WINXP NTFS
Volume Serial Number is 38A6-1BFC

Directory of J:\

File Not Found

===========================

J:\>ATTRIB $Boot
File not found - $Boot

============================

J:\>dir $MFT /ASH
Volume in drive J is WINXP NTFS
Volume Serial Number is 38A6-1BFC

Directory of J:\

File Not Found
=================================

I'm just wondering if it maybe only works in NT and not in 2K/XP in order to protect the files.

[Whyzman]

The one article I read definitely cited NT as focal point...

If XP or Win2k I think it said to use the defrag utility to view the $MFT file...

[taikari]

I just stumbled upon this thread searching for a much more serious problem I'm having with MFT. But I'd just like to let you know, you can view the $MFT and all other metadata files with file recovery programs. So, you might want to start there. You need to know what the programs are doing in order to access the MFT, isn't that right?
I don't think any simple command line can do it. Hexedit sounds about right, plus some sort of very low-level data access reading capability.

Well, I don't know how far I'm off, but I DO know that I've written my metadata system files into flat files, and it SUCKS. Because I've basically removed them from that system area, and now even using recovery utils, there's no reference to my files.

I'm trying to figure out how to put my metadata system files BACK to the original location. Anybody?

[BTW, which recovery utils can read and write MFT files? iRecover seemed to do it for me]

[Budfred]

taikari,

Welcome to http://www.pcguide.com/ubb/pcgubb.gif

You might want to start your own thread on your question. This thread is almost a year old and long abandoned.....

[Paul Komski]

A number of us have been trying to get to grips with this for some time and the thread has been reinvigorated before so thank you for responding. If you had posted a new thread directly then no eMail notification would have been sent and I, for one, might have missed your input.

So although the thread is latent it certainly has not been abandoned. In fact, I have been waiting for a response from ixl to an eMail clarification on material in the pcguide before responding to this very thread!! Abandoned - Duh! Wouldn't that be a moderator's decision.

Hmm - indeed a hex editor might well be a way to proceed.

taikari from what you say you have somehow removed all the metadata files and not just $MFT; notably the $MFT Mirror. And thus presumably CHKDSK can't rebuild them. Don't know if it will help you but it would be interesting to know just how you wrote your flat files and how you managed to erase the data.

budfred I hope this post of yours wasn't just a wind up.

[Paul Komski]

Have spent a lot of time and effort over the past year or so trying to get my head properly around NTFS's MFT. In that process, a better understanding of the $MFTMirr file would have been helpful. The PCGuide (http://www.pcguide.com/ref/hdd/file/ntfs/arch.htm) describes it as a mirror of the first 16 records of the real MFT and SourceForge (http://linux-ntfs.sourceforge.net/ntfs/files/mftmirr.html) describes it as a copy of its first four records viz:- $MFT, $MFTMirr, $LogFile and $Volume. I eMailed ixl some time back from a PCGuide form and asking if he could clarify these different descriptions. Possibly the mail got lost en route, but I am none the wiser to date.

SourceForge also asks the questions. (1) Is this always the first four FILE records of the MFT, or has it a constant 4KB size? and (2) How large is it if the cluster size is greater than 4KB.

Now, the MFT consists of a series of RECORDS (which can also be called Inodes) starting with Inode 0 and named $MFT within its own $FILE_NAME attribute.

Each Inode-record (that is to say each MFT-metadata-file) is composed of attributes among which is a data attribute named $DATA. If the data is small enough then this is where it is kept - in totality; however if it cannot fit into the space allocated for the attribute it is referenced amd expanded onto another section of the HDD. This IMO is where the MFT is most often misunderstood, particularly by those who see it as some kind of super-FAT system.

Also, as I see it, this is where the problem of data recovery REALLY begins, since if all the data can be contained within the $DATA attribute (and in any case some of the data must be contained within it) then losing that record also loses some or all of the data it references. FATs are different in that their records only reference the data; they are just "virtual card indexes" for a "virtual library of books". Each Inode-record is also like a card index BUT onto which the first page of the book is actually appended; so losing the file loses that first page - and if it is a one page book, then the whole book is lost.

The MFT file called Inode-Record-0 (also called $MFT within its own name attribute) is variously described as a relational database, an index, etc. This file's database or index data, however constructed, and as with any other NTFS file, only has "the first page of its book" contained therein. The remainer of this "database-book" is stored elsewhere on the HDD, (within an empty buffer area to prevent/reduce it becoming fragmented), and where it will grow and contract as files and folders (which are also just NTFS files) are created and deleted in the partition.

Another area of confusion is that the Master File Table, the MFT, is not the same as the $MFT; the latter is just the first metadata file within the MFT; its first Inode. So when we want to "see" the MFT, do we mean see the MFT (being a sequence of metadata files) or see the $MFT (being a database of files stored on the partition). Well, probably the latter. But even then, assuming one can access the data in the first place from a hexeditor or whatever, one would also need to know how that database was constructed to be able to build it into a meaningful table (or tables and relationships - if it is a relational database). The starting position and the overall size of the MFT can be determined from methods outlined in previous posts or from utilities such as PQ's partition info.

So full circle back to $MFTMirr and Data Recovery. Knowledge of what its $DATA attribute contains, and how it is organised, is crucial to understanding how to recover NTFS Data. Perhaps professional data recovery organisations can recover stuff when the metadata files have gone or been corrupted, but it must be made very difficult when not only has the database of file records gone but a lot of the actual data too. If all the metafiles (the MFT) have gone then not only has the database gone but also the bitmaps of data on the HDD. In that case rebuilding the bitmaps and databases from scanning the partition, bit by literal bit, must be a very specialised area.

Anyone that can add clarity or understanding to this whole area would be warmly welcomed to contribute.

[taikari]

Reference to new thread: http://www.pcguide.com/vb/showthread.php?s=&postid=140887#post140887

Very interesting. Though, I still do believe that if I can replace the current $MFT, $LOGFILE and $BITMAP, then I may just be able to restore my files. Why? When I used the recovery prog, it showed these very files under a folder: ROOT. After I restored them, not knowing what it was, the reference to THOSE particular files disappeared. See what I mean? When I run a recovery on the same drive, I can see all the same files EXCEPT for the files under the ROOT folder (77GB of data). I've GOT to find a way to write these files intact as they are to the drive. As long as the recovery software can parse these files (when they are in their correct system location) (it wont parse flat files as far as I can see), then I do believe I can try recovery again. The worst answer would be bit/byte by bit/byte recovery. I can't imagine how garbled the output of so many files could be.

I keep reading different definitions of the $MFTMirr file: saying it reads the first 4, or 16 files, etc. It would make sense for it to reference the most important files which you listed already, though my $LOGFILE is 64MB, I wonder if it really does have a copy of this data. I'm trying hard to understand in a definitive manner why the recovery of the metafiles completely removed reference to 77GB of data, yet left the rest (previously, purposely deleted data) intact.

I've taken a look at the hex of the $MFT. All FF for several KB, then references to other metafiles. I also took a look at the current MFT files and compared them with what I've backed up. The only difference is the file size of the backed up $MFT is significantly smaller than the new $MFT 29KB vs 55KB . Also, the file id #'s are different. In hex, the new file starts with File0, and immediately begins referencing all the other MFT files while the old starts with a lot of FF hex. I need a way out of this maze! :(

[Paul Komski]

Using a Hex editor is a nice way to at least "visualise" the files. DiskProbe and WinHex both have their own advantages but both will do this and also allow you to see the absolute structures of the HDD - that is to say, in the main, the MBR and the Partition Boot Sectors. Haven't seen any FF at the start of any $MFTs - they all seem to start with File0.

The sizes you quote for your MFT and its flat backup are both way too small to possibly be correct. Whether they are damaged or just fragmented is another matter. The size should be in MB, though each MFT-Record will only take up 2 sectors, whatever the cluster size happens to be.

This also helps to understand the size of $MFTMirr, which is only 3 records long prior to NT5 (Win2K) but which is either 4 records long (8 sectors) or else it can be however many records will fit in the cluster if the cluster is larger than 8 sectors; thus with a cluster size of 16 sectors it will contain copies of the first 8 $MFT records.

Since the whole partition "disappeared" after the incomplete merge, it sounds like just the partition tables or the partition boot sector became corrupted. Either of these is relatively easy to edit and could be the way back, since if Windows can finally see the partition, then CHCKDSK might be able to repair it.