Slightly more advanced file recovery

[koffiejunkie]

So, my 32GB card somehow picked up some filesystem corruption. But koffiejunkie, didn't you buy the iPad to backup photos on the go? Well, yes, I did, and it worked brilliantly, until someone on a bus in Cambodia decided they needed it more than I do. Oh well.

I ran the Sandisk recovery software (the one that comes with the CF cards) and it worked well, except for files that were not stored in continuous fashion.

Short explanation for those who don't know how filesystems work: CF cards are formatted in the FAT32 filesystem. When you format a card, all that happens (in very oversimplified terms) is your camera creates a new File Allocation Table (FAT for short) that contains a few columns. One is for the start sector, one for the end sector, and one for the the file ID. There's more to it but not relevant here. When a file is written, the name is recorded, where it begins and where it ends. When your computer reads off the card, it looks up the file name, sees where on the card (which sector to which sector) the data is located, and reads that. With the FAT, the card is just one continuous blob of data.

When your card gets corrupted, what's really happening is the FAT gets messed up, so it's impossible to know where on the card the files are physically located. So what Sandisk Rescue Pro (and similar software) does, is read each sector, and checks if the data coming off it looks like the beginning of an image file, and it keeps reading until it finds the next sector that looks like the beginning of an image file.

So far so good. But what happens if you take a few pictures that don't come out right, you delete them right from the camera? You now have bits of data with empty bits in between. When you take your next picture, it will be written into one of those gaps. If the file is small enough to fit in the gap, all is good, and you'll be left with a smaller gap. If the file doesn't fit, the gap will be filled, and the remainder of the file will be written in the next gap, and the FAT will have multiple entries referring to each part of the file (file system fragmentation - why you have a defragger in Windows).

Now, the beginning of the second bit of data can be any random stuff, so it's not easy to know what the data is. Thus, when you run Sandisk recovery software (or any of the other ones), it will only restore the first chunk of a file written into one of those gaps, and you'll end up with a useless file.

Ok, that wasn't so short. Anyway, this is what I'm looking at. Most of my images were recoverable, but a few weren't. They weren't too serious, I don't remember any special ones that I don't see in my collection, so that's OK. But I did take a fair bit of video, and of the 30 or so clips, I could recover only one.

So I'm wondering if anyone knows of any software that either has a bit of intelligence to recognise secondary chunks of data and match them up, or at least store them out as separate files so I can go mix and match them manually? (and please don't just shout out your favourite recovery tool unless you know for sure it does this - I'm not going to risk corrupting my card further)

If all else fails I'll have to fire up the hex editor, but with 32GB to work with, that's a looooong term project!

Thanks

 

[ponder]

Dunno if it can do what you want but maybe have a look at photorec/testdisk docs.

 

[koffiejunkie]

I did try testdisk, and I am still experimenting with it. I haven't tried PhotoRec yet, but I might. Reading the documentation, it doesn't mention fragmented files except once when it only mentions recovering files if they are not fragmented. I am still looking at the advanced options though.

Thanks

 

[bwana]

Ouch, glad you've managed to recover at least most of your photos. Hope the ipad was insured too!

But what happens if you take a few pictures that don't come out right, you delete them right from the camera?

I've always had my own reasons for never deleting photos on the camera, I guess you do as well now.

 

[koffiejunkie]

Ouch, glad you've managed to recover at least most of your photos. Hope the ipad was insured too!

Sadly, I neglected to add it to my camera insurance before I left. I thought I had but when I looked through my sent e-mail, I couldn't find that particular mail. I do have travel insurance, but they only cover up to £200 per item, so at best I'll only get just under half the money back. Nevertheless, between the iPad and free wifi everywhere, I still spent/lost less than I usually do on data roaming, so I'm not to broken up about it. I'll definitely buy a new one - it was great to travel with!

 

[koffiejunkie]

I just realised I never updated you guys on this. I ended up being able to recover almost everything, with a combination of tools, and a little bit of luck.

A colleague at work bought a bunch of new Sandisk SD cards, which came with vouchers for version 5 of Sandisk Rescue Pro (which is really Rescue Pro by LC Technology International). Since had a bunch of the same, he gave me one, and since still had version 2 or therabouts, I thought what the hell, I'll give it a try.

I was able to recover almost all the pictures, and all the videos except one.

I also tried testdisk, which didn't do much. Photoreq at first failed to see CR2 files at all (it only seems to know about CRW files, but I don't see how that matters, since it can't be looking for file names/extensions anyway), and I found I had to update it to the latest build to correct that issue. After doing thiat, it still didn't pick up anything if I only selected CRW files (still no CR2 description) but leaving everything enabled and fiddling with the advanced settings eventually caused it to find my files. It found the exact same CR2 files as did Rescue Pro, and found the missing video file.

The downside with Photoreq is that, while it's able to stich together the fragmented video file bits, it doesn't appear to be able to tell where the end of the file is. So each video file ended up having a couple of hundred MB of random stuff appended at the end. This doesn't affect playback, since each video player I tried knows where the video stream ends, but it did add about 70 GB to my collection of 30 second clips. Fortunately, Rescue Pro didn't have this problem, and the one video file it didn't find that Photoreq did, turned out not to be a keeper anyway, so no harm done.

There was a couple of image files that I was not able to recover at all, with either tool. It simply look like the files themselves were damaged, or neither tool was able to find the remaining fragments, because all of them were much smaller than they should be. There are a couple I'm really gutted over - most my pictures of someone awesome I met randomly. Most of them are simply not there, and a few look kinda like this:

A few more are less important but nevertheless nice to have, but fortunately my friend who was travelling with me has the same images for the most part.

 

[bwana]

Glad to hear you were successful in recovering most of your images.