Our experience with corrupt R3D files

[George Fisher]

Having gone to hell and back with the RED MX workflow, and reading (almost) every post on this board, I thought I’d post what we went through and how we solved our problems, to help anyone out there going through the same struggle.

This was our first shoot with any RED camera, and the intention was to do a short film that did not cost too much, as a test run for some projects we have planned later in the year.

We were using CF cards (of which we had four), and reading those back onto a new Macintosh. We cycled the cards, 1 2 3 4 1 2 3 4 etc.

We were doing double-system sound, so sound had no issues.

We were copying the files from the CF cards using R3D Data Manager. R3D data manager does two reads of the data and checks for a difference in the checksum.


SIGNS OF PROBLEMS IN PRODUCTION

The first few copies went well, but then there was a bad copy (identified by R3D Data Manager), something had gone wrong. We tried a second copy and got the same problem. We also did a second copy from the card using Finder.

We had trouble viewing the video in REDCine-X. At this point things were confusing – some shots were viewable in REDCine-X, some were partially viewable, some weren’t at all, and we were getting confused.

We removed the problematic CF card from circulation.

We then got a second copy error from R3D Data Manager on ANOTHER card. This is when we realized we had a serious problem, and thank goodness, we could break for lunch early.

Our solution was to drop from 4.5K to 2K. By using less cards and having smaller files, and relying only on the cards that had worked, we figured we had a better chance of the shooting going well.

We had no additional problems during the shoot, other than some shots appearing not to work in REDCine-X (showing up black).

IN RETROSPECT, I have read a lot about CF technology and the problems we ran into, and here’s what I would have done differently and what I have discovered:

• Ideally we should have used a RED hard drive. A hard drive is meant for storing large quantities of data reliably, flash memory is used for storing small individual files (like photos), not 8 gb files
• We needed more than 4 cards, preferably enough not to have to cycle any
• We should have low level formatted each card before the shoot, so all bad sectors would be identified and marked as part of the FAT32 file system. FAT32 is NOT to blame for these problems.
• CF cards have the flash controller in the disk (that manages error detection etc.). I am assuming the cards from RED are good quality cards and use some degree of error checking but I do not know for sure. However, the point to take away from this is that different CF cards can have different levels of quality.
• We should have had multiple card readers. Because CF cards have the controller inside the card, the card reader is quite a dumb tool. It can overheat quite easily. If card reader heats up too much, it can transfer this heat to other cards used in the same reader.
• Flash media (CF cards in this case) is highly susceptible to static damage. If you walk around on carpets you’re going to damage your flash memory.

POST PRODUCTION WORKFLOW with corrupt files

Almost all of the 2K shots were fine. Almost every 4.5K file had something wrong with it. These problems were of two kinds:

TYPE A
Some R3D files would work a little of the time, but other times, show up green after imported in Final Cut Pro or black in RedCine-X 356.

• Once the shot went green or black in the software, that shot would stay green or black until the application was restarted.

• Sometimes these files would crash RedCine-X 356, RedAlert, or Foundry Storm.

TYPE B
Some R3D files would not play at all in anything, either crashing the application or not appearing. This footage could not be imported into FCP.


I arrived AFTER the rough cut had been completed and there was an urgent need to get these clips working because some scenes did not have enough coverage. 20% of our shots were corrupt.

A note about RED file structure: You will see multiple R3D video files in the same folder. This is because FAT32 file system on the CF cards has a file size limitation of around 2gb. Hence each shot has to be spread over multiple files. RedCine-X, RedAlert, RedLine all seem to be smart enough to know how to find file 002 if you point it to 001.


- Solving TYPE B -

Run through RedUndead command line tool (included in the RedCine-X package) and then proceed with TYPE A if the file is recovered, but still has corrupt frames.


- Solving TYPE A -

RedCine-X is clunky and highly automatic. It will be too clever, particularly trying to read corrupt clips and therefore crashing itself. I found that I could view the R3D files in RedAlert much more easily, and find out where the corrupt frames were located. However finding corrupt frames 1 by 1 would take too long. Also RedAlert had a yellow green tint to every frame (caused by RedAlert not supporting the color science of the RED MX camera)

Nothing seemed to be working. I could not export in RedCine-X, even if I knew what the corrupt frames were, because it would try to preload the file and then crash.

I then found out about RedLine. RedLine is a command line tool that allow you to perform all the behind the scenes operations that RedCine-X provides through its GUI interface, but from the command line. It is provided as part of the RedCine-X download package.

It turns out that RedCine-X has the ability to generate a script that will extract shots frame by frame into tiff files. It is then relatively straight forward to import these into FCP and export a videoclip. (You do this by setting the default import duration of still frames to 1 frame and then dragging all the clips onto the timeline, then exporting)

It is hard to figure out the RedCine-X interface. In the default layout there are tabs down the left hand side, and one is called EXPORT. Click the PLUS symbol, and create a new preset. The file format is TIFF. “Other” select “Use REDLine (Software Only)” and “Frame Only Mode”. Set a name for your preset. Press OK.

Do not load the clip you want to extract the frames from because it may crash. Instead, just drag it into a bin, and select the export option “Bin (all clips)” or “Bin (selected clips)”. Then press the EXPORT button and the tool will export the frames 1 by 1.

If this works for you, great!
If not:

It is possible to generate a script for REDLine to be run from the command line. To do this, you select “Generate REDLine Batch Script” in the export preset settings. When you export, a command line script will be generated that will tell RedLine to line-by-line extract frames from the R3D files. The file will have the extension .batch

In our case, the script DID NOT export all the frames. It would stop at the end of the first R3D file, so we’d be missing a significant portion of the shot even after exporting the TIFF files.

To solve this, we had to get more technical:

The generated batch script will contain a separate command line call to RedLine for every frame in the R3D file, where $number is the frame number and $input_file is the R3D file (001, 002, 003 it doesn’t matter)

/Applications/REDCINE-X_build_356.app/Contents/MacOS/REDLine --exportPreset \"George's Preset\" --useRSX 2 --noRocket -i $input_file --outDir $output_folder -s $number -e $number

It is important to maintain the exportPreset and useRSX 2 – without these arguments for RED MX shots, you’ll get the green and yellow tint color frame problem because it will assume the RED ONE color science. The export preset file also allows you to specify resizing of the image etc. in the comfort of the REDCine-X application.

... continued in next item in this thread ...

 

[George Fisher]

So I wrote a Perl script that exports every frame, and checks for the end of the frame sequence by capturing the STDERR output of RedLine and checking to see if the output file number is the same as the requested frame number. This is because RedLine will not tell you if you go beyond the end of the R3D file, it will just repeatedly output the last frame. I include this script here for people who KNOW PERL and KNOW UNIX. *** IT IS FOR REFERENCE ONLY. NOT FOR COMMERCIAL USE! IT DEPENDS ON WHAT VERSION OF REDLINE YOU HAVE! ***

#!/usr/bin/perl
# PERL script to extract frame by frame from R3D files into individual TIF files

$input_file_name = "/Volumes/Mrs P/2010-12-06 0137 Mrs P/Mrs P Transfer Done On Ext HD/Pack #1/A003_1204L1/A003_1204L1.RDM/A003_C002_12055Q.RDC/A003_C002_12055Q_001.R3D";

$output_folder_base_path = "/Users/leighannbellamy/Desktop/temp recovery data";

# EXAMPLE (NOTE, NO TRAILING SLASH IN OUTPUT FOLDER)
# ***** $input_file_name = "/Volumes/IOMEGA HDD/Mrs P Transfer Done On Ext HD/Pack #1/A003_1204L1/A003_1204L1.RDM/A003_C002_12055Q.RDC/A003_C002_12055Q_003.R3D";
# ***** $output_folder_base_path = "/Volumes/IOMEGA HDD/recovered/redline2";
# ***** $number_of_frames_to_export = 99999999;
# ***** $starting_frame_number = 3;


$number_of_frames_to_export = 99999999;
$starting_frame_number = 3;

# do not change anything below this line
#----------------------------------------------------------------------
$number = $starting_frame_number;
$count = 0;
$reached_end_of_file = 0;

# create output folder based on time, date, and filename
@input_file_strings = split(/\//, $input_file_name);
# get the second last part which is the folder name
$length_folders_input_file = scalar(@input_file_strings);
$red_disk_shot_id = $input_file_strings[$length_folders_input_file - 1];

my ($sec,$min,$hour,$mday,$mon,$year,$wday,$yday,$isdst) = localtime(time);
$year += 1900; ## $year contains no. of years since 1900, to add 1900 to make Y2K compliant
my @abbr = qw( Jan Feb Mar Apr May Jun Jul Aug Sep Oct Nov Dec );

$folder_string = "$red_disk_shot_id - $abbr[$mon] $mday $year at $hour" . "h $min" . "m $sec" . "s";

# append the output folder directory
$output_folder_base_path = $output_folder_base_path . "/" . $folder_string;

# input file and output file must escape all spaces
$input_file = $input_file_name;
$output_folder = $output_folder_base_path;
$input_file =~ s/ /\\ /g;
$output_folder =~ s/ /\\ /g;

print "Reading: $input_file_name\n";
print "Writing to: $output_folder_base_path\n\n";

# notes to self: this always looks at the 001 file and finds the other red files
while ($count < $number_of_frames_to_export && !$reached_end_of_file)
{
# resize is done from XML
$command = "/Applications/REDCINE-X_build_356.app/Contents/MacOS/REDLine --exportPreset \"George's Preset\" --useRSX 2 --noRocket -i $input_file --outDir $output_folder -s $number -e $number";

# capture stderr and stout
$result = `$command 2>&1`;

# loop through each line in the file
foreach (split(/\n/, $result)) {

# compare to unescaped base path (is this the output line containing the output file name?)
if ($_ =~ /$output_folder_base_path/)
{
# take apart the output file name
@file_name_deconstructed = split(/\./, $_);

$length = scalar(@file_name_deconstructed);

# get the frame number exported
$number_exported = $file_name_deconstructed[$length - 2];

print "Exported image $number_exported ($number)\n";

# if the requested frame is after the exported frame
if ($number > $number_exported)
{
print "EOF reached\n";
$reached_end_of_file = 1;
}
}
}

$number = $number + 1;
# count is the total number of frames exported
$count = $count + 1;

#print "\n\n";
}

print "FINISHED: $count frames exported.\n\n";

I would be very interested in your comments, suggestions and corrections regarding this experience. Clearly it won’t help everybody as we are not maintaining the ability to edit in RAW, but I hope this can help at least solve some confusion.

 

[Tim Sutherland]

I was on a show using CF cards for Steadicam only, but I noticed a way to stop bad CF transfers, which is what you had. If you dismount the card in OSX and the blue activity light on the reader stays on after the card is pulled out, you will probably have a bad transfer on that card. If you unplug the reader and repatch it before inserting the next card so the activity light is off, you will probably not have a good transfer. Although I say probably, I have never had a corrupt transfer doing this. I have had the activity light stay on on readers from multiple brands, so I don't know if it is a driver or hardware issue.

I would guess that this is why your first few cards were ok and then you started having problems that didn't go away.

Tim

 

[Manuel Wenger]

Dear George,

Sorry to hear about your hassle. I´d get in touch with RED Support because that sounds very uncommon. I had troubles similar to that over a year ago, and i turned out it was a bad CF Module in the camera.

Other than that i always check the copied media visually before the CF Card goes back to the camera (mainly using Clipfinder therefore). I use only disks which don´t get hot just warm (G-techologies G-mini Disks)
I do the copies from the CF Card one via a Lexar FW 800 Card Reader, and the second using a NEXTO DI 2700.
I never do more than one copy process at a time, and i carry a dozen of Firewire Cables arround with me.

This worked for me the last 1 1/2 years and i never had serious corrupt file problems.
I remember two events in the last year and they could get solved with the backup copy from the nexto. The Cause was a bad Firewire Cable.

I´m impressed about the level of effort you put in getting things going again, and for sharing your efforts with us. And i hope you give the camera another chance, cause this is by far not the way it normally works.

Best regards and good luck
Manuel

 

[Nick Gardner]

You had bad gear in the chain some where. The CF cards are practically bullet proof, and dead nuts reliable. I suspect bad CF reader, or bad CF module in the camera.

Nick

 

[Meryem Ersoz]

Did you try a different card reader? Sometimes these things can be simpler than they appear. I'd start there.

From your description, especially that things were better with less data throughput, from 4.5K to 2K sounds like your card reader was too slow. You need the fastest card reader available, especially with higher data throughput required by bigger files.

 

[Olivier Madar]

I had a big problem with big files transfer on a new macbookpro 17 on june, that could explain what happen too you. The big files were not R3D but it could be.
in about 20 or 30% of the time, big file transfert failed. I changed, hard drives, cable, esata card, and... the macbook pro itself. That never happens again. It seems that the new i7 macbook pro is sometimes too hot and that damaged the motherboard insidiously.

 

[ericyoung]

I was on a show using CF cards for Steadicam only, but I noticed a way to stop bad CF transfers, which is what you had. If you dismount the card in OSX and the blue activity light on the reader stays on after the card is pulled out, you will probably have a bad transfer on that card. If you unplug the reader and repatch it before inserting the next card so the activity light is off, you will probably not have a good transfer. Although I say probably, I have never had a corrupt transfer doing this. I have had the activity light stay on on readers from multiple brands, so I don't know if it is a driver or hardware issue.
...Tim

Hi Tim.

Did you mistype a "not" in your post? Ie did you mean "If you unplug the reader and repatch it before inserting the next card so the activity light is off, you will probably ... have a good transfer." rather than "...not have a good transfer."?

 

[ericyoung]

...

• Ideally we should have used a RED hard drive. A hard drive is meant for storing large quantities of data reliably, flash memory is used for storing small individual files (like photos), not 8 gb files
• We needed more than 4 cards, preferably enough not to have to cycle any
• We should have low level formatted each card before the shoot, so all bad sectors would be identified and marked as part of the FAT32 file system. FAT32 is NOT to blame for these problems.
• CF cards have the flash controller in the disk (that manages error detection etc.). I am assuming the cards from RED are good quality cards and use some degree of error checking but I do not know for sure. However, the point to take away from this is that different CF cards can have different levels of quality.
• We should have had multiple card readers. Because CF cards have the controller inside the card, the card reader is quite a dumb tool. It can overheat quite easily. If card reader heats up too much, it can transfer this heat to other cards used in the same reader.
• Flash media (CF cards in this case) is highly susceptible to static damage. If you walk around on carpets you’re going to damage your flash memory.

...

Hi George

Were these RED or third party 8GB cards? RED cards have been tested to be good for large digital motion files, not just photos.

CF card readers getting hot have been a cause of bad CF data transfers, as have bad firewire cables and some brands/models of CF card reader.

Static damage is always possible, but very hard to definitively prove as a cause.

 

[Scott Snare]

My experience with cf card failure is primarily with the readers. They get hot after 2 or 3 cards are read. I rotate 3 or 4 readers on set and keep an ice pack on top of the active reader. Keep the readers cool and your transfers will be smooth. What I notice when they get hot is the read time increases by 20 to 30 % while down loading.

 

[Tim Sutherland]

Hi Tim.

Did you mistype a "not" in your post? Ie did you mean "If you unplug the reader and repatch it before inserting the next card so the activity light is off, you will probably ... have a good transfer." rather than "...not have a good transfer."?

That's correct. Repatching solves the issue.

That's what I get for writing on my phone.

Tim