Welcome to our community

Be a part of something great, join today!

Log in
  • Hey all, just changed over the backend after 15 years I figured time to give it a bit of an update, its probably gonna be a bit weird for most of you and i am sure there is a few bugs to work out but it should kinda work the same as before... hopefully :)

Software that will detect broken/corrupt frames

M

Mick van Rossum NSC

Well-known member
Joined
Feb 1, 2007
Messages
386
Reaction score
0
Points
0
Website
www.mickvanrossum.com
Hi all,

Lately we had some corrupt frames in footage -apparantly most of the time this happens when offloading (we are using CF cards ) , but since this was detected too late and our back up copy had the same problems (cards were copied simultaneously which to 2 different drives which we will stop doing now as well, and we will continue copying drive by drive), I started to look for software that will detect on the backups if there are corrupt frames (green frames, half garbled images, double frames). Our Posthouse renders through Redrocket enabled Redcine X (that's were the failures came up) but apparantly not all broken frames are detected by RC-X. So I tried different programs with mixed results; I rendered to the lightest smallest Qt's (SD,1/8th resolution) which most of the time renders 8 minutes material in 8-16 minutes rendertime. (MBPro 2.66). Here are some results;

-RC-X (no rocket on my MacBook pro) does not give warnings (AFAIK) but renders out black after the corrupt frame
-PPro 5 (I tried the 30 day license version) warns with every failure and stops the renderproces ("error compiling movie, unknown error")
-Redrushes ; out of 5 corrupt clips 2 were stopped during the progress and those haltes QT's were unplayable.
-Clipfinder; I could not get it to render, failed instantly, but that has to do with my with my limited knowledge and experience. (sorry Hans, even after your personal demo last October).

So far PPro 5 gives me all the warnings I need without checking all material for single broken frames, but if there are other solutions/workarounds I'd love to hear them. Ofcourse this is the job for the DIT, but I could not resist testing myself.
 
Are you doing checksums on your data copies? That would be the first thing to do.

Before I ever erase camera shot media, I either wait for the Lab/Post house to confirm that there were no bad frames, or if there is not enough time, I watch every circled take in realtime before erasing so that I know my checksummed copies do not have corrupt frames. I really hate doing it this way, but it's the only way that I can feel comfortable erasing the negative.

Tim
 
Tim,

Checksums are done/OK, and don't give warnings. (R3D datamanager) The DIT checked teh material, but esecially with green frames in greenscreen, or double frames it is very hard to notice. BTW R3D datamanager renders all, without noticing errors.
 
Ideally, a DIT should always be visually scrubbing the footage, checking for errors, and noting them in his reports (it sounds like you are doing this, which is great!).

If the camera is really physically dropping frames, there is of course an indication on the camera itself that should be watched out for. However, often a particular card reader will malfunction and cause corruption on import - which is where the checksums come into play.

I had this problem every once in a while with build v.21. Occasionally there would be a green or black frame right at the start of the clip. However, since build v.30 came out, I haven't seen this occurring. Updating (or re-installing) your camera firmware might help.

Last but not least, sometimes the CF cards and drives need to be completely re-formatted from scratch. Format them into the "MS-DOS" file system using Disk Utility, and then format them again in the camera.

I hope this helps...

Best,

- Jordan
 
Mick van Rossum said:
Tim,

Checksums are done/OK, and don't give warnings. (R3D datamanager) The DIT checked teh material, but esecially with green frames in greenscreen, or double frames it is very hard to notice. BTW R3D datamanager renders all, without noticing errors.
Click to expand...

That's why I always make sure someone has watched every frame of the takes that are important before deleting something from the camera media. Scrubbing is okay but it won't find the issues you're talking about.

Tim
 
Tim Sutherland said:
That's why I always make sure someone has watched every frame of the takes that are important before deleting something from the camera media. Scrubbing is okay but it won't find the issues you're talking about.
Click to expand...

I think that's an excellent practice! They didn't screen dailies in ye olden days for nothing.

- JL
 
Tim Sutherland said:
That's why I always make sure someone has watched every frame of the takes that are important before deleting something from the camera media. Scrubbing is okay but it won't find the issues you're talking about.

Tim
Click to expand...

Hi Tim,

I completely agree this is best practice, but I'm surprised you are able get someone to do it.

Often it's all I can do to just do a quick scrub. Particularly on multi-camera shoots where you are doing on set one lights and transcodes. Even with Rocket and multiple MacPro's that just a lot of work.

There is just no way for me to watch every frame in real time from one camera let alone multiple cameras.

And getting another resource on set is generally a non starter with the director/producer.

Good for you.

Steve
 
Let me clear up a few small points:

The way R3D Data Manager works is to read the source material twice. So if there is an error reading the CF cards, it should catch most of these. R3D Data Manager will only report success if the checksums on BOTH reads match. This has been a feature since 2.x. This is different than just "using checksums". Other checksum methods just read the source and assume the source read was valid. R3D Data Manager does a double read on the source.

Because of this, copying to a single drive at a time or copying to two drive simultaneously will yield the same result. Since R3D Data Manager reads twice from the source, and both reads matched, one can deduct with reasonable certainty that all reads will read the same (i.e., the error was on the CF card, as written).

On another front, I have always advocated that proper data management requires three steps: 1. ensure that all files were copied, 2. ensure the data in all the files exactly matches, and 3. ensure that all the data in each file adds up to an image. R3D Data Manager will do #1 and #2, but there is no software that will do step #3 in any sort of automated fashion. The only way to do step #3 is for a human to do it manually, in real time. Ive written software that will detect some errors, but by far there is no way to detect all types of errors: codec errors, duplicate frames, missing frames, flash frames, missing pixles, horizontal or vertical black or white lines, etc.

Last point: checksums are only as good as the point from which they are implemented. I dont know how Red plans to implement their frame checksum feature, but it will only be good from that point forward. Say for example the frame checksum is a checksum of the wavelet encoded data - well then that assumes the codec properly encoded the data. If the frame checksum is a checksum of the RAW data, then that assumes there was no hardware error in the sensor or readout and transport of that data. Make no mistake, having frame checksums will be great, as they will remove (or identify) more points of failure, but it wont stop all failure. Even then, the only way will be for a human to manually watch the footage, in real time.
 
Dear Conrad,

Thank you very much for explaining your checksum methodology. It is most helpful! I'm a huge fan of R3D Data Manager and I hope you will keep up the excellent work. Though I didn't previously know the ins and outs of your checksum implementation, I did know this: your product is invaluable.

In almost all of my experiences, the root of any recording problem was that the camera needed it's firmware re-installed, or the CF cards / hard drives needed to be reformatted. Ideally, I hope to catch any such issues during test shoots at the camera prep, and these problems will already be solved before we ever get into the field.

Best,

- Jordan
 
Well, my thing is that there is no way to check for corrupt frames unless you watch the footage realtime , so no scrubbing. Apparantly the only way to detect these is by actually rendering these into another form, and that is where I tried all these programs for; so I don't have to wait till the posthouse has rendered out (for offline) the material. BTW I tried the "make quicktime" command in R3D Data Manager, and also here I don't get a warning when a frame is corrupt. Conrad, is there any difference why your program, like most of all already mentioned programs, renders out all quicktimes without warning, where by example Premiere Pro halts the renders and gives a warning ? Just curious. Thanks for the program, indeed a great invaluable tool.
 
A thought - render out as an image sequence, say 1/8th res TIFF with compression. Then sort the sequence in your OS's file browser by size. Any unusually small frames are suspect (because an all-single-colour frame is much easier to compress, and will therefore be much smaller).

Will not catch all problems, but will find black frames.
 
Charles Angus said:
A thought - render out as an image sequence, say 1/8th res TIFF with compression. Then sort the sequence in your OS's file browser by size. Any unusually small frames are suspect (because an all-single-colour frame is much easier to compress, and will therefore be much smaller).

Will not catch all problems, but will find black frames.
Click to expand...

Problem is:
If you come back with 3hours+ of data per day and have to deliver multiple formats for offline/online/rushes...

You cannot really start to render all that data before it is recorded...
And the days are still quite long.
 
Gunleik Groven said:
Problem is:
If you come back with 3hours+ of data per day and have to deliver multiple formats for offline/online/rushes...

You cannot really start to render all that data before it is recorded...
And the days are still quite long.
Click to expand...


I really don't understand why other DIT's are saying that this isn't red's fault. This is clearly red's fault and fanboy-ism has taken over. There is no way that I can watch a 3 camera show in realtime.

http://reduser.net/forum/showthread.php?t=51338 - please take a look at this thread. RED hasn't responded to helping us out.
 
use or rent a red ram till the ssd module is finished.
what do you expect?
Red to write a software plugin 3 months before its EOL.
i am not against it, i really would love it too.
but i dont see it happening, so i find ways to deal with it.

edit:
there are not that much situations where you can have dropped frames.
i even shot in helis with regular HDD red drives without dropped frames (before red rams were available)
if you know what you will shoot and think that special scenario can drop frames you find ways to work around it.
btw. there is always a scenario where you can be sure that the camera system of your choice will have problems. we certainly find ways to work around them or switch the camera system for that particular shot.
 
Cüneyt Kaya said:
use or rent a red ram till the ssd module is finished.
what do you expect?
Red to write a software plugin 3 months before its EOL.
i am not against it, i really would love it too.
but i dont see it happening, so i find ways to deal with it.

edit:
there are not that much situations where you can have dropped frames.
i even shot in helis with regular HDD red drives without dropped frames (before red rams were available)
if you know what you will shoot and think that special scenario can drop frames you find ways to work around it.
btw. there is always a scenario where you can be sure that the camera system of your choice will have problems. we certainly find ways to work around them or switch the camera system for that particular shot.
Click to expand...


What a defeatist attitude.
 
jamie, i can understand that you are frustated,
and i agree that no one is able to watch 3hrs of footage in realtime on set.

but you seem to be a little bit too emotional on this.
if you shoot with 35mm, HDcam, Alexa or whatever, all of them have different problems.
we shot a stunt 4 weeks ago with 4 Red MX, 35 mm High Speed, Canon 7Ds and a Sony.
All of them behave different in that special shooting setup.
we did test as good as we could by simulating the circumtances on prep for that particular day to find out what could happen and did find ways to work around them.
this shooting day did cost $$$$$$ and it would suck if the cams failed.
( this is what i was paid for, configure and make things work)

its not that you had dropped frames while shooting a 40 sec. close up using a tripod.
Did you test before the shooting day how the RED with your assec. behaves using MoCo?
 
plus:
on really important set ups we change mags after every take to controll immediatly if everything is fine.

my attitude is not defeatist.
you seem to have worked relying on laptop drives on an expensive setup.
its your fault imho to not having prepared enough, be it the AC, the configuration or the workflow for that day.
there is no perfect equipment, you have to find ways to make it work.
 
Cuneyt:

I prepped the camera @ a major rental house the day before the shoot. We tested all the drives, all the cf cards and all the cables. I don't need you telling me how to do my job.

What you are doing is taking the blame away from the company that needs to find a solution to this problem. The truth is, is that I did notice some weird playback with the dropped clips while scrubbing through them, but since I didn't see a green or grey flash, I thought the Rocket was having trouble, which it often does.

The problem here is that there are major inconstancies which can't be left up to human error or judgement. No where in the Redcine-x upgrade log does it say "dropped frames are now grey, or green, or rainbow" (atleast to my knowledge), so how should we know what to look for?

I don't understand how you can feel so confident about your success when I've done +100 RED shoots and rarely do we have one go smooth without issues. DIT's have enough to worry about, why are you saying it's ok for us to deal with this issue without a legitimate way to find the problem, and deal with it?

I was using Gtech 2TB drives with a MacPro. Downloading to my internal RAID. We were shooting 500GB of media/day. Shit happens. I understand that, but RED has had 15 build versions to come up with SOME kind of solution for this.

My clients are dropping RED for Alexa and F35 pretty rapidly... Must be a sign, right?
 
You must log in or register to reply here.
Back
Top