Corrupt R3D Files?

[Tanner Field]

Hey Everyone,
I was called in by a friend to take a look at some R3Ds that he is having problems with from a shoot last week. They seem to be an issue that I have never encountered before. I thought I would run it by you guys to see if you had any ideas as to how to salvage the files.

The Issue:
There are a total of 5 takes that have this issue. The clip looks to load fine into RCX. It will begin to play, but at a certain point the playback will freeze, but the cursor will continue to move. This locks up the program and you are not able to change clips in the viewer until you restart. If you close the program to restart it a crash report will pop up. It looks to happen at a certain point for each clips. However, during recording there was no report of dropped frames.
If you export the file it has had varied result. Once it said there was an error. Other times it will complete, but then if you look at the playback in QT it will play fine for a while.. Then it will hit a point where it begins to replay a 1-2 second part of the clip, and this will continue for the length of the QT file. I've tried all RED programs and had similar results.

The Details:
The files were recorded to RED 16GB CF cards on an MX RedOne. I was told by the data manager that the cards were dumped and verified using R3D Data Manager, however I can not verify this. There was a total of 40 rolls shot on the camera over the shoot and only 5 total clips seem to exhibit the issue. The troubled clips came from rolls that also had clips that work fine. All of the troubled takes have multiple R3Ds making up the take, as they are rather long shots. However, there are plenty of takes that are the same length or longer that work fine.

I'd appreciate any help that you could provide. I'd happily post the R3D files, but they are far too big to get up... Let me know if you have ever experienced or heard of this problem and happen to have a solution.
Thanks for any help!

Best,
Tanner

 

[joe hedge]

Let me know if you have ever experienced or heard of this problem and happen to have a solution.

Problem is: Bad on-set data management

Solution is: Good on-set data management

 

[Tanner Field]

Problem is: Bad on-set data management

Solution is: Good on-set data management

Helpful..

 

[Tim Sutherland]

Helpful..

Unfortunately his assessment seems pretty spot on to me. Bad cf reader possibly, but seems like there was a lack of visual verification going on. Do the QT proxies play?

Tim

 

[Paul Hazlett]

Tim nailed it. Bad CF reader corrupted the file. I had this happen and R3D manager caught the bad files before i wiped the card, re imported from the original from another card reader and all was fine.

If you are running a card reader hard sometimes they overheat and data gets corrupted.

 

[Tanner Field]

Thanks guys... That was my fear as well. The QT proxies suffer the same issue. I appreciate the confirmation. Off to relay the bad news.

 

[Dustin Cross]

Tanner,

If R3D Data Manager was used you should have the md5 checksums with the files. If you don't have the checksums they most likely turned them off to speed up R3D DM, which is stupid. If you do have checksums, then R3D DM failed.


Dusty

 

[Tanner Field]

Dusty,
Thank you for the information. I was given a separate drive with just the troubled files, but I'll take that information back to the producers and have them check the drives. Again, I really appreciate the support and the info.

Best,
Tanner

 

[Olivier Madar]

I have exactly the same issue than you, not on the compact flash but on the hard drive.
Usually, they do two time the file transfer, to have 2 master copy. This day, I don't know why, they did it only once. If you don't look very close, you don't see the problem when you are on the set. All seems fine, the quicktime plays. They did not either have the RedRocket mobile this day.

Now, when I try to conform to Prores, RedCine crash or everyfile are black. If i conform the file one by one, sometime, I get a prores with these kind of drop (green or white half screen).

Here is my question :
What is the more robust software able to conform and save what we can, even with drops ?

Is it more safe to use R3D software Data Manager or to do the copy twice, on 2 separate hard drive ? Or do you do both ?


Olivier

Hey Everyone,
I was called in by a friend to take a look at some R3Ds that he is having problems with from a shoot last week. They seem to be an issue that I have never encountered before. I thought I would run it by you guys to see if you had any ideas as to how to salvage the files.

The Issue:
There are a total of 5 takes that have this issue. The clip looks to load fine into RCX. It will begin to play, but at a certain point the playback will freeze, but the cursor will continue to move. This locks up the program and you are not able to change clips in the viewer until you restart. If you close the program to restart it a crash report will pop up. It looks to happen at a certain point for each clips. However, during recording there was no report of dropped frames.
If you export the file it has had varied result. Once it said there was an error. Other times it will complete, but then if you look at the playback in QT it will play fine for a while.. Then it will hit a point where it begins to replay a 1-2 second part of the clip, and this will continue for the length of the QT file. I've tried all RED programs and had similar results.

The Details:
The files were recorded to RED 16GB CF cards on an MX RedOne. I was told by the data manager that the cards were dumped and verified using R3D Data Manager, however I can not verify this. There was a total of 40 rolls shot on the camera over the shoot and only 5 total clips seem to exhibit the issue. The troubled clips came from rolls that also had clips that work fine. All of the troubled takes have multiple R3Ds making up the take, as they are rather long shots. However, there are plenty of takes that are the same length or longer that work fine.

I'd appreciate any help that you could provide. I'd happily post the R3D files, but they are far too big to get up... Let me know if you have ever experienced or heard of this problem and happen to have a solution.
Thanks for any help!

Best,
Tanner

 

[Jeremy G]

Prepare yourself for some fix-it time.

Prepare yourself for some fix-it time.

I got bit hard by this bug on my current Red project. Like you, it was some but not all of the clips. It turned out it was not only corrupt frames (which would turn any viewer green or grey or whatever) but missing (dropped) frames as well, which would hard crash any application I worked with.

With the help of the attentive Red support team, I was able to at least get around the corrupt and missing frames. After identifying all of the corrupt clips (by playing each clip end to end in RCX and waiting for either a crash or other series of freak outs) I pulled those clips out of the data set in to their own "Corrupt Files" folder to stay organized. Then, I used RCX on those segregated clips to render out a lores tiff sequence using redline (frame only mode) using the instructions the helpful people at Red gave me, make sure to turn off handles everywhere.

I would then take the image sequence in to Adobe Bridge to scroll through the myriad of frames to see which frames were bad. I used Bridge as I found this easiest on the eyes, you can use what finder or whatever you want. Bad frames showed up all black or with some digital garbage, but mostly black/blank.

I then viewed the image sequences in Motion's browser, which showed me which frames were actually missing, if any. I used Motion because it was easy and did what I need. Other applications might be able to identify missing frames out of image sequences as well. The Motion method was super easy and quick.

I then used RCX and the "R3D Trim" option to create new R3Ds around the bad/missing frames by frame number. Some of the clips I had to break up in to 5 new R3Ds to get around all the goofiness. It was a shame that the information was gone, but I was able to at least get to the offline edit to make proxy files. I put those fixed R3Ds back in to the good data set and left the corrupt R3Ds separated from the rest of the pack. The online process of one of the spots from that data set has been smooth so far through color correction. Getting all those files fixed/proxied took the better part of 5 days and nights. Prepare accordingly.

Contrary to what flies around this website, this is was not due to file mismanagement/handling, the files were bad on the CF cards themselves as I was able to have a few of them sent to me across the country from the rental house before they were sent back out on another gig. There is a good reason that Red CF cards are going the way of the dodo, I'd imagine. It is straight up silly not to have a write protect switch on a professional medium, but I am starting to digress.

Jeremy

 

[Stephen Strangways]

If you can, get the md5sums.txt files for the corrupt clips. They should have been made at the time the footage was downloaded from the cards with R3D Data Manager. From there, you'll be able to verify the files you have to see if they have been corrupted at some point. Even if the transfer from the CF cards worked perfectly, the transfer from there to a second hard drive could have corrupted the files. It has happened on shoots I've been on.

 

[Lesley Ibbotson]

Hi there, I have this exact same problem - drop frames and glitches all the way through my footage. I could not transcode it as it would just hit the first drop frame and either encode the rest as black with audio, or would crash Redcine-x pro. I took it to a post house who had a red rocket who could transcode it but with drop frames through out. It was shot on the red epic with the cards and not transferred off them using a data management programme (much to my disgust). They also deleted the cards before I could transcode it so I could not go back to them to see if the problem was on there or created on transfer off the cards. But my question is - does anyone know what can have caused this? Both the cameramen and the post house have not seen this before. The post house did mention that mobile phones can cause disturbance and we were shooting in a sports hall with lots of athletes on their phones - could this be it????????? Any thoughts appreciated.

Thanks

Lesley

 

[Gunleik Groven]

Haven't had issues with Epic SSDs...

This is 100% of the times I have been able to verify it a data management problem.
Dropped frames have occured to the disks, but then the reason for the framedropping allways have been obvious. Mostly vibrations of sorts.

It is interesting, though, that RED files seem more prone to datamanagement issues than most other files I have worked with.

Cheers!

G

 

[tomspark]

I have just been shooting a doco on an epic-x and have had corrupt files turn up. Half the frame is normal colour and the other half goes black and white with funky bright colour appearing over the top in an almost sensor like pattern. They are isolated corruptions but occur across the entire effected reels , meaning unless I can figure a solution the entire reels are unusuable. Some of the clips won't even show an image in red cine x pro, and the exported example frame was extracted from a tiff sequence export of the files I could read in Red Cine X Pro. For the clips that won't display in RCXpro the file number appears to identify the clip, but it simply has the notation 'offline' where the thumbnail would normally appear, so no clip deatails such as format etc.
I have not been able to verify if they were due to a copy error or an in camera one. Perhaps someone can help me figure out the problem, so I can hopefully avoid in future. These errors appeared fairly early in the shoot, but as we were on location in remote areas it was difficult to check all the footage as we went so as much as better data mgt would be great it wasn't an immediate option so would not have made any difference because we could not reshoot anyways. As such I only identified the problem on the effected reels when ingesting footage on return. I don't think it was a faulty SSD card as I then went on to use the exact same cards over and over again afterwards without encountering the corruption again. Any one seen this before?

I suspect its an in camera error, and my feeling is perhaps the card was not fully seated in the ssd slot as we spent quite a lot of time on poor roads, so vibration could have been a cause. Again I cannot verify. I was able to play back various shots from the card as I was shooting, that could then not be played back in red cine. At some stage on this real, but only once, an error message did come up and I had to restart the camera to proceed. It did however only happen once on one clip on the reel, yet the entire reel is effected. Unfortuneately I cannot remember the exact details of what the error message said, as I had no means of writing it down at the time, I just had to keep shooting, and my check was to see if the clip would playback and at the time it did, so I thought it just must be a bug or crash as computers sometimes do.
Cheers
Tom

 

[tomspark]

Anyone? I have come back to this as there is still no solution or response from red, and I would love to use this footage if in the edit. Corrupted files across entire reel - happened in 3 reels out of 60, still sitting there, unusable, not playing back with the odd frame looking like the picture above...