Missing or Corrupt Image Data File on HD Camcorder ??

[bhelms]

Hi all. I wasn't sure where to post this one so I thought I'd start here in "gadgets".

I have 2 Sony HD Handicams, an XR-500V (120GB HDD) and an XR-520V (240GB). To date they have both been essentially flawless. I have been backing them up to a 3TB external HDD with some regularity (drag and drop data over USB through my PC), but I'm not sure if those backed-up files would ever be "usable" as such, or could be pulled in by an editing SW of some type. I'll leave that discussion for another day however.

On Fri. I was doing another back-up of the 520V. It was running late and I couldn't finish it so I "cancelled" the transfer on the PC then backed out of the USB connection properly on the camera. But once out I got some kind of error msg that the IDF (Image Database File) had become "damaged". I powered-off then back on and the error still appeared. So I used the utility in the camera to "repair" that problem. Per the next message it needed about 90 seconds do to that, and the progress bar moved over about that duration. When the "repair" function was complete it reported that there were no inconsistencies. However when I now go to the View Image screen that allows me to play videos, it's blank as if none existed! Nothing I have tried to this point has restored that screen. I do see all the photos I have stored on the View Photos screen. When I reconnected the camera to my PC and ext. HDD, I was able to see a "normal" file structure on the camera's HDD and I was able to transfer all those files to the ext. HDD as I would have expected. I did however notice that the "MTS" (AVCHD image) files are now seemingly in a subfolder and not where I would expect them to be.

Does anyone here know how to reconstruct or replace the IDFs so I can again see the recorded videos appear in the camera? Any other suggestions?

TIA and BRgds...

 

[bhelms]

In the immortal words of Emily Litella (played by Gilda Radner on SNL)...."Never mind...!!"

I solved the problem! All it took was some file management using my PC and another "Repair Damaged IDF" effort on the camera's part. It turns out that somehow all the image files and all their supporting information including the IDF that are included in the "BDMV" folder had been “moved” on the camera’s HDD to another existing subfolder. I don't know how that happened, but it seemed to be before I started the backup on Fri. afternoon (??) based on the last file timestamps. I also don't know how I lucked-out that nothing had apparently been damaged, only "misplaced". I noticed all of this only after looking at the data backups I did overnight Fri. following the "crash" in more detail today. The folder structure of the 520V camera (the corrupted one) did not match that of the 500V, nor previous backups of itself.

So today I attached the camera to my PC again and used the file manager ("Windows Explorer") on the computer to “move” those files to where they belonged on the camera's HDD. Frankly, I was quite surprised the camera allowed me to do that - I thought it would be "read only". But that seemed to work as hoped! When I stopped the USB connection, the camera then recognized the Image Database File was “damaged” again. It took almost an hour to fix itself, but this time – voila! – Everything seems to work properly again! My files seem to be completely restored and a couple of spot checks are all looking OK.

I’ll be “basking in the glow” of that little accomplishment for several hours to come. I think I might just have myself a beer to celebrate…!!

Life is good (at the moment!) Lessons learned: Don't be afraid to poke around a bit in your camera's HDD as long as you know where everything belongs...and don't "F-up"...!!

 

[TheForce]

bhelms- There is a fundamental flaw in how you are managing your video files. This flaw will cause many yet undiscovered troubles with playback of your video you believe is safely backed up to your 3Tb archive drive. The reason for this is rather complex but simply put, Sony creates a container for the video files designated MTS This file may or may not be the complete video for the entire clip you shot. This is especially true for long video recordings. The files are referenced by number and to properly play them you may need MTS files 6,7, & 8 in order. Or maybe just 6. You can see that if the number system ( index) gets corrupted the playback will not work. If you wish to transfer the video you shot to an archive drive, format the archive drive to NTFS and then connect it to your computer and be sure it is actively mounted. Next install the Picture Motion Browser ( PMB) utility software that came with the camcorder. This is very important. The use of this software is necessary to not only transfer the video and audio MTS file(s) but also stitch them together in the right order so that one video recording will be stored as one video file on your archive drive. In addition, the audio and video is not exactly synced for playback in the MTS file. Longer programs recorded the worse the sync can be. PMB will correct this as well. Finally, the PMB software is necessary to reset the file number system in the camcorder so that additional recordings will not be paired with a corrupt index file. But the repair utility can fix this as you discovered. Sony added the repair utility in the camcorder in case the power goes out in the middle of a recording or transfer operation and the repair can get things back in alignment.

To many people who think they know computers, will try to relate to how the Sony camcorder manages the video and it is not the same. When you use windows file manager, you do not transfer and reset the complete package for computer playback. You just transferred computer files.

Besides using the PMB software, Sony Vegas Pro editor has a built in utility that will work too and there is a PS3 utility that also works to properly manage the video recordings on a Sony camcorder with digital storage.

One final note for PMB. Sony has many different versions of this software and you need to use the proper one for each model camcorder. Best to use the version that came with your camcorder.

There are other tricks too such as transfer from camcorder to the hard archive drive without the use of a computer, but I will pass on those for now.

Glad you got through the repair process OK but you still may have trouble with those MTS files in playback later on. At least what's left on your camcorder is saved and my guess uncorrupted. But if you find the files problematic, you can put them back into the camcorder in the right folder in it's drive, then execute the repair utility that will renumber everything and then use PMB to properly put them back to your archive.

 

[bhelms]

Tks, Don. I was hoping you'd weigh-in on this one. The fact the PMB will not recognize the files stored on the ext. HDD was a hint to me that there's more to this than a simple back-up. I was hoping that at least moving everything to the ext. HDD would give me a place to start in case the camera was lost/stolen/damaged. Then maybe I could "reload" the entire back-up to a new Sony camera's HDD and let the camera "fix" itself and restart from there? Maybe just a pipe dream. In the meantime, I need to uptick burning AVCHD disks so I don't lose everything! I figure my nearly-full 240GB drive will take about an entire 50-disk stack!

I do have the PMB version that came with my cameras installed in the same PC I used. I'll digest what you took the time to detail above (tks for that!) then perhaps IM a couple of questions to you. One basic ? for the moment - How do I format the ext. drive to NTFS? (I'll have to get another "virgin" drive for that as I have a lot of other stuff archived on the current 3TB drive as well.)

Tks again for your help...!

PS - Just connected my camera and PMB was apparently able to pull-up all the images. I have hardly used this SW so I'm very new to it. (To this point I have simply burned unedited videos with a Sony VDR-MC5 multifunction recorder. That has been essentially flawless.) It looks like I can move all the camera image files to an external back-up through PMB using the "Easy PC Back-up" function. I believe that's what you're discussing in your post. I just need to get that properly-formatted ext. HDD with enough capacity to do that...

 

[TheForce]

I'm not familiar with your camera model. Some Sony cameras can connect to an ext HDD to off load the video clips without PMB or computer. In this case you must let the camcorder format the drive so the folder structure is also set up properly. In you just want to format a drive to archive your video to on a computer, then use windows to format the drive like any other but be sure, in this case, to use NTFS so it will be able to handle the larger video files PMB will stitch together.

You can back up all the video and still photos to the drive with PMB or select individual ones. You should spend some time learning this software as it was designed to do what you need to do. It also makes things easier as it uses thumbnails too to recognize your video. PMB is similar to Apple's itunes in what is does.

Personally, I don't archive to any optical. I put everything on hard drives and catalog it in folders. Mostly I shoot to edit anyway so I make the base folder a project name, like Disney 5-2012 then let the PMB build folders with dates for all the clips. Then I add a folder for the edited projects script files. Anything and everything from music, voice overs, sound effects and other graphics would get put in it's own folder under the main one. When the project is done I may burn a BD-R of the finished edited production, but simply copy the entire folder of everything to a cheap slower hard drive in case I want to open it up in the future and do another project with the video. I've been using this form of file project management for literally thousands of TV shows and video projects for many years.
But if you do archive to optical, I suggest you add a BD-R burner as it will be less expensive.

 

[John Kotches]

Don:

How often do you check the archival disks, and what would be the financial consequences of losing the data? I know this is a business for you so I'm asking that from the worst case scenario.

 

[bhelms]

Tks again Don! I checked properties in the 2 ext. HDDs I have with me here, both WD MBEs. The older 750GB is formatted FAT32, but surprisingly the 3TB drive says it's formatted NTSF. It must have come that way. All I did was "plug and play" both; if there was any formatting then my 'puter took care of that at first connection on its own and there were no additional drivers required.

I wasn't suggesting that I wanted to archive to optical, but that's one way to make sure I don't lose anything completely. I don't know if there's any practical way to use those files later for editing.

I'm not aware that these Handicams can work directly with an external HDD, but they do allow dubbing to removable media. Time to dig out the manual again!

I'll get another WD drive to dedicate to PMB back-ups and start learning that SW. It seems to be quite capabable for a consumer-level experience and product. I'll also look into a BD burner.

Tks again...!