Welcome to our community

Be a part of something great, join today!

Log in
  • Hey all, just changed over the backend after 15 years I figured time to give it a bit of an update, its probably gonna be a bit weird for most of you and i am sure there is a few bugs to work out but it should kinda work the same as before... hopefully :)

Spotlight corrupting frames in R3D files?

Chris,

This is a Red One with 16GB CF cards? If so that is your problem. I thought this was SSD media.

16GB CF cards themselves heat up and change the data on the card. Checksum will say it is perfect, because the data on the card has been corrupted. I have seen that several times and have had other DITs see the same thing. Checksum says all is good, but there are corrupt frames in the footage on playback or transcode. It is not the reader that heats up it is the 16GB Red CF cards. The 8 GB cards do not have this problem and the Red Drives do not have this problem and the Red SSDs are AWESOME. Get an SSD reader on the Red One to solve this problem.


Dusty
 
That helps a lot. Thanks. However, we've been playing back the footage files and not getting green or black or crashes.

Either way, I'll be taking this into account on the next job. It's a good thing I spent a grand on Red CF cards.
 
Chris Santucci said:
Gotcha. Is there a way to verify the integrity of the R3D files other than by playing them?
Click to expand...

Only a visual check will do. If time permits, playing several seconds at the start, end, and middle (if the clip is long) as well as scrubbing back and forth will increase your chances of finding corruptions.

I can't speak to the Spotlight possibility, but I always look to the CF card reader first in these cases. If you're not using a RED Station CF Reader it's really a crapshoot. We've done tests with third party readers and seen highly inconsistent results even between same-model units. Even when copying the same clips from the same unit it could fail only one out of five, or one out of ten, times.
 
Then it's not really any different from backing up/copying any other footage from camera media in that you gotta visually verify the footage.

I may do an experiment with Spotlight to at least prove to myself whether it causes these issues or not once and for all.
 
I haven't run into any issues with spotlight and media files - I do highly doubt this could happen as I don't believe spotlight writes data to the actual files, it maintains its own database on each drive in a hidden folder. I have had bad CF card readers corrupt Canon raw files on import - but it was immediately noticeable and doesn't match the description from your DIT reviewing the copies.

Either way I'm posting because I'm curious as to what you find out.
 
Matthew B said:
I haven't run into any issues with spotlight and media files - I do highly doubt this could happen as I don't believe spotlight writes data to the actual files, it maintains its own database on each drive in a hidden folder.
Click to expand...


I read it somewhere and then went and tried to find the post(s) about Spotlight and could not locate it. I know our DIT forgot to disable Spotlight initially and was getting errors on copies to one drive (not the other) with the R3D data manager and when he shut Spotlight off, the errors stopped.

As it stands, I have no conclusive determination about our bad frames but I'd think it's either the CF card reader and/or the cards. The funny thing is, we did a 4 day job with the same gear a month prior to this and had zero issues.

We just completed another 4 day job but used the Red Drives only, and had zero issues.
 
The bad frames generally result in green. All green.

But here's one frame I was able to find that shows the odd pattern:

Screenshot2012-09-10at70256PM.png
 
Hi Chris,

That looks exactly like CF reader data corruption due to overheating. This is exactly the reason why we built our own CF readers because everything else out there is just not built to handle fast CF reading all day long.

I've never seen or heard of Spotlight corrupting R3D files. If you still think that's the case then I would start to look in to any Spotlight plugins you may have installed (that extend the capability of Spotlight to index files).

Best,
 
That's helpful, Rob. Thanks. Red should make it more clear that this can happen. I'll be sure and warn the new owner of my R1.
 
Tim: we are not aware of any issues with the SSD station. With SSD's we usually see this when there were copy issues somewhere else in the pipe (failing hardware, unreliable software, etc.)
 
Rob Lohman said:
Tim: we are not aware of any issues with the SSD station. With SSD's we usually see this when there were copy issues somewhere else in the pipe (failing hardware, unreliable software, etc.)
Click to expand...
Hi Rob, I had the same sort of corrupted images on my Epic using SSDs and a Red station SSD reader plugged directly into my mac book pro fw800 port. The transfer was using R3Data manager latest release version. My back up drive was plugged in via FW express card adapter. The issue only happen on one day of a two day shoot, without changing anything between the days. Any idea where the corruption came in the pipe?
thanks for your help Rob !
Cheers, Tim
 
Rob Lohman said:
Tim: we are not aware of any issues with the SSD station. With SSD's we usually see this when there were copy issues somewhere else in the pipe (failing hardware, unreliable software, etc.)
Click to expand...

I have had prior instances of SSDs with corrupt frames in the R3D, prior to ANY copying - at the time (1 yr ago) Deanan told me it was likely to have been spotlight or somthing similar corrupting the filesystem on mounting the SSD read/write. At that time Red watchdog didn't exist. Since its inception I've not had a repeat of the problem (Deanan mentioned at the time that Red were working on a solution, ie. watchdog )

Could these problems with SSDs be traced to users mounting read/write not read only, ( mounting on systems with no Red Watchdog installed? )
 
John: that's possible. We've seen cases where someone does not safely eject while the camera or computer is still writing to it, and then the file system gets corrupted.

Tim: that's hard to say, it may have been an unsafe unmount, and we've also seen an issue with an older version of that copy software.

Sometimes it can be a bad firewire cable or something too (even if it doesn't do it all the time). In your case I would've tried to copy it a bunch of times with different readers & cables to see if I could've pinpointed the problem.
 
Just to add the sum of my experience when encountering corrupt R3D files:

1 - The original RED CF cards are NOT recommended. CF cards are simply too slow to reliably write data quick enough in normal or extreme (hot) conditions over extended periods of time. This is not to say they don't work because they do. This is just to remind everyone that CF media has to work at 95% just to record a R3D. If it slips to 94% you get a corrupt frame. You are much more prone to corruption if recording to CF. And you will be similarly prone to corruption attempting to get your R3D's off the CF so now you've got twice the chance of getting a corrupt frame. When the temperature goes up so do your chances of corruption with CF. Could be a hot camera, hot CF card, hot CF reader, hot firewire cable, etc. If you can, upgrade to the RedMag SSD's. They are a million + times better than the CF media.

2 - Besides CF the other big culprit is the RedRocket card. It's highly doubtful that corrupt R3D frames were introduced downstream by copying or using your original set hard drive(s). It's much more likely that a failing RedRocket card is introducing corruption during playback or render.

3 - File verification and checksums are SLOW and can easily FAIL. As others have said, always open and scrub thru your R3D's directly with RedCine or Resolve etc. and double-check your files visually. If you do have a corrupt frame from recording or transferring you WILL see it. BTW: I recommend AGAINST using a RedRocket for scrubbing. You're better of setting debayer to 1/4 Rez.
 
Hi

On red-one, we have a routine of allways transcoding before formatting (in addition to checksum copy). A working transcode, is the thing that has given us a 100% result for quite a few years now.

Checksum copies is not veryfying what is actually in the file.

That said.
When we have had bad copies, usually the file itself hasn't been corrupted as suggested here. Actually "never" is the right word.
I know deanan HATED this suggestion, but I allways had a really slow el-cheapo usb cf reader handy, that didn't heat, and thus managed rhe copies none of our highspeed readers did.

Anyhow. Visual check of the r3d's + transcode of files prior to format has given us 0 lost frames on more than 400 recordingdays with 1-5 cameras i have been responsible for the last 2 years (prior to going epic)

Cheers

G
 
You must log in or register to reply here.
Back
Top