Protocol for Footage on Set

[Jochen Schmidt-Hambrock]

Maybe REDCINE needs a format option. Ian

I personally like the way Aperture handels this - although I always feel a little uneasy clicking "yes" to format and eject when I haven´t checked all the material yet.

Jochen

 

[tj williams]

mmost: Exactly I talked to my friend who has been working with the S2 equipment. you already knew!
I'm totally in agreement that built in work flow safety is critical to wide acceptance.

Seems to me that some of the folks here will also want to use LTO tapes as on set backup. Again if RED automates this in a way similiar and perhaps even easier to use than S2 it seems to me a good thing!

So: is this somewhat complete?

1. record to either single media, or if allowed to dual card or card/disk on camera.
2. download card to either dual lto or two seperate drives or redundant raid
3. RED cine automatically does check sum (or perhaps faster or better data check) and automatically gives a simple answer that data is copied correctly and checked and then and only then gives the option to reformat the card or disk mag.
4. RED cine reformats the disk with the same project settings.
5. media is installed onto camera and menu item flashes asking for reel number.
5. The cheap wage kid then delivers one to post and one to safe storage at a seperate location. (yea that is amazing when you think of it. (My son has been doing some 2nd AC on features and got a good laugh about this!)

Hi David:
The S2 system is pretty bulletproof as opposed to the RED light weight drives on-board camera or CF cards that can easily be damaged while being carried around. My point is not that, for every project this is right just that it will be good on high end projects, to be as reliable as S2. In fact I imagine S2 will introduce a some kind of S2 Light to work with the RED compressed and when the optional uncompressed output comes out they will also support that!

 

[I Bloom]

I hope this thread eventually leads to a consensus on how to deal with footage. I think people are getting lost in the technical specifics. The technology is helpful, but it still requires a human to make sure its working. And humans works better at these things when they have a good set of steps to check themselves.

4. RED cine reformats the disk and increments the reel number.

Seems like reel number should be incremented by the camera. Since otherwise you would need to keep all your cards in the order that you formatted them.

I actually think that reel numbers are less important in the digital world since each clip likely has a date and time of recording in metadata.

IBloom

 

[tj williams]

ibloom: other thread says longer to format in cam so good to format on set comp? Good amend to put reel number in camera, I wasn't thinking clearly about so many CF cards.
Poverty can do that to one....

The many posts about hiring good people carry across all set jobs. The plain truth is the budget for a data manager, will be similiar to the amount to hire a film loader..

I agree I'd like to see a consensual workflow come out of this thread. Here's maybe a beginning we can add to take away from or change to get a workflow which we can take to the bank.

1. record to either single media, or if allowed to dual card or card/disk on camera.
1A If using single recording, pull partially filled card or drive mag immediately after crucial shots for transfer.
2. download card to either dual LTO's, two separate drives, redundant raid. (S2 docking system.)
3. RED cine automatically does check sum (or perhaps faster or better data check) and automatically gives a simple answer that data is copied correctly and checked and then and only then gives the option to reformat the card or disk mag.
4. RED cine reformats the disk with the same project settings.
5. media is installed onto camera and menu item flashes asking for reel number.
6. Deliver one to post house.
7. Using a seperate delivery method deliver one to safe storage at a separate location from the post house.

 

[david farland]

In the risk adverse digital cinema world there are unique mobile, environments and just plain human error elements that don’t exist in the risk adverse financial markets that trade in billions of dollars each day and can’t afford to lose any of it.

However as data storage technology changes so quickly I’m sure digital cinema trails a little in this regard for a bunch of reasons, some valid…some just inertia and some just quirky 100 year old habits.
This area gets better, safer & cheaper every 12 months…..I hate seeing it shackled like the introduction of the automobile where people with flags worked out in front because others didn’t understand it.

I guess you need to redesign the model with these risk, time, environmental, human & cost factors taken into accounts.

I’m all for a list TJ, good work….but obviously each step will have its particular variances and qualifications.


Cheers,

 

[E.J. Sadler]

There are a few good reads for anyone who's concerned about stability of CF media -

http://news.bbc.co.uk/2/hi/technology/3939333.stm

http://news.digitaltrends.com/news/story/5140/blast_destroys_camera_flash_card_survives

http://digitaljournalist.org/issue0111/biggart_intro.htm


The vast majority of CF problems happen during handling - insertion/removal, static, etc. So if you're paranoid, having a static mat handy and powering off the camera is the way to go.

We use ChronoSync (http://www.econtechnologies.com/site/Pages/ChronoSync/chrono_overview.html) to copy data off of our CF cards/firestores, it will do a byte level file verification of the copy so you can skip the pre/post checksum. The copy is done to a mirrored raid, and then 'ChronoSynced' again to a secondary removable drive. The files are then burned to duplicate blu-ray discs,one of which goes to off-site storage.

We also don't re-use a CF or Firestore until all the stills/footage has been verified.

Once you have a workflow and stick to it, you'll wind up with a system that's far safer than film ever provided.


p.s. - Our still cameras write to CF and SD cards simultaneously, so we have a backup right out of the camera. So I agree it would be great to be able to send the data stream to two recording media for immediate back-ups

 

[Dan McCain]

I've been a first A.C. on many, many P2 shoots. The system I use is basic. When I get a "hot card" I pull the reprotect tab I put it in my left pocket (which is the same side as a powerbooks PCMIA slot). I then copy the card to 2 passport drives simultaneously. I open the folders to verify the size an physically make sure the files match. Then I pull the write protect tab and put in my right pocket. I only reach into the right pocket for new media, and never touch the left unless I am downloading. I have never screwed up once (and I've been on a lot of all night shoots where I am brain dead).

The beauty of the pocket system, is you can quickly shove the card into your pocket when you get busy. For example if you are in process of inserting the card into the powerbook and someone approaches you with a question you quickly put the card back into your left pocket so as not to get confused about what you were doing.

I have heard of people using ziplock bags and bins for organization but I like the pocket system because it is always right there.

I never hold a card in my hand, it either goes into one of the pockets or into the camera or computer. If you hold a card you can quickly forget if the card is hot or not.

 

[Chris Parker]

ha. mccainds. i love your system.

lots of good ideas here. but remember, there is only one negative on set. i think one CF Card, two hard drive copies, and MAYBE an LTO-3 tape would be as far as it ever needs to go.

 

[tj williams]

Hi David: Technology will change, maybe disk drives will be outmoded by years end? Maybe RED will allow simultaneous recording maybe it won't? flash cards will surely get larger, and thus more widely used. I'm trying here to incorporate everyones Ideas which we can all agree to to make a check list for the RED Dit. If you know about specific new tech. or gritty old habits which are no longer useful, I hope you will contribute your knowledge here! Perhaps this list will be to paranoid for some
It's easy to leave out steps you don't feel are necessary harder to invent your own additional steps. EjSadlers links above sure serve to point out that most of the time if we lose data it will be human error not technology failure. I think even nailing to a tree would come under human error!

1. record to either single media, or if allowed by RED Technology, to dual card or card/disk or dual disk mag. on camera.
2. Dit wears a grounding wrist strap and grounds camera during media removal. Possibly Dit asks for the camera to be powered
down during offloading? ChronoSync (http://www.econtechnologies.com/site..._overview.html) may be prefereable to check
sum. I strongly feel RED needs to provide the data check software as part of RED Cine to adapt a technology which they
stand behind.
2. If using single recording, pull partially filled card or drive mag immediately after crucial shots for transfer.
3. immediately pull the re-protect tab on CF media.
4. Place the recorded media into some particular bag or left pocket etc. where you always store the media until you get it out to
put into the onset backup storage.
(This is subject to the info. that handling is a majhor source of CF failure.)
4A Set computer and Esata or Firewire 800 raid are set up on an anit static matt.
5. download card to either dual LTO's, two separate drives, redundant raid. (S2 docking system.)
6. RED cine automatically does check sum (or perhaps faster or better data check) and automatically gives a simple answer that data is copied correctly and checked and then and only then gives the option to reformat the card or disk mag.
7. RED cine reformats the disk with the same project settings.
8. Assistant places the blank formatted media into some repeated storage bag or box and or perhaps keeps it in their right
pocket until it is reloaded into the camera (see () above)
9. media is installed onto camera and menu item flashes asking for reel number.
10. Deliver one to post house.
11. Using a seperate delivery method deliver one to safe storage at a separate location from the post house.

 

[I Bloom]

Hi David: Technology will change, maybe disk drives will be outmoded by years end? Maybe RED will allow simultaneous recording maybe it won't? flash cards will surely get larger, and thus more widely used. I'm trying here to incorporate everyones Ideas which we can all agree to to make a check list for the RED Dit. If you know about specific new tech. or gritty old habits which are no longer useful, I hope you will contribute your knowledge here! Perhaps this list will be to paranoid for some
It's easy to leave out steps you don't feel are necessary harder to invent your own additional steps.

1. record to either single media, or if allowed by RED Technology, to dual card or card/disk or dual disk mag. on camera.
2. Dit wears a grounding wrist strap and grounds camera during media removal. Possibly Dit asks for the camera to be powered
down during offloading? ChronoSync (http://www.econtechnologies.com/site..._overview.html) may be prefereable to check
sum. I strongly feel RED needs to provide the data check software as part of RED Cine to adapt a technology which they
stand behind.
2. If using single recording, pull partially filled card or drive mag immediately after crucial shots for transfer.
3. immediately pull the re-protect tab on CF media.
4. Place the recorded media into some particular bag or left pocket etc. where you always store the media until you get it out to
put into the onset backup storage.
(This is subject to the info. that handling is a majhor source of CF failure.)
4A Set computer and Esata or Firewire 800 raid are set up on an anit static matt.
5. download card to either dual LTO's, two separate drives, redundant raid. (S2 docking system.)
6. RED cine automatically does check sum (or perhaps faster or better data check) and automatically gives a simple answer that data is copied correctly and checked and then and only then gives the option to reformat the card or disk mag.
7. RED cine reformats the disk with the same project settings.
8. Assistant places the blank formatted media into some repeated storage bag or box and or perhaps keeps it in their right
pocket until it is reloaded into the camera (see () above)
9. media is installed onto camera and menu item flashes asking for reel number.
10. Deliver one to post house.
11. Using a seperate delivery method deliver one to safe storage at a separate location from the post house.

I'm going to add an idea, that will seem wierd but I think should be taken seriously: In addition to being numbered all your CF cards or drives should have kid stickers on them of various random things. Human memory is aided by association. If you see the cards with the horsey in the reader, you are less likely to confuse it with the other card you have with Mr. Yuck on it.

I'm at least going to do this. Maybe I'll start collecting stickers at concerts instead.

BTW I'm not convinced about grounding the camera, I think that's overkill.

Ian

 

[monolith]

I'm gonna add something funky to the mix here - timecode. Potentially as antiquated as reel numbers, except that in my P2 experience (which is also lowly, unpaid, abusive and extensive) I have found that TC is useful because it tells you that one clip goes before another clip, and that nothing was recorded in between.

Lining up the top and tails of your timecode for ingested clips in your NLE takes about 5 minutes - it can be difficult to concentrate on at the end of a shooting day, but always seemed to me the best way to ensure that ALL the footage is there at the end of the day.

When coupled with the P2 Store (which has a non-human verification that the data on the card and the Store are identical) I've only missed one card in the year and a half/60+ shoots that I've been using the P2 workflow. And that was in Jamaica. You get the idea..... (it was also before I'd discovered TC as my friend and ally)

Do with it what you will.


Ed

 

[david farland]

thanks steve.....nice work.
just a quick note (on break) for now.

As you've mentioned.....on critical shots you backup accordingly.
We do this every day when word processing. Say you were writing a novel in Word on a 80GB disk....
You wouldn't wait till you filled the disk before you backed it up!
When writing your piece you perform a risk evaluation on the content you just produced and backup accordingly.
You also evaluate the risks associated with your backup facilities and backup/distribute as necessary.
Single points of failure such a single disk/flash, single camera that houses all data, single location, single event, time of day (sunset) as you're alluded to, all present additional risks than require special mitigation.
If you don't have any redunancy (single flash card or housing) your backup regime would be in in the order of minutes whether that card/device can hold 4GB or 4TB.
So the choices associated with the data in my hand are about cost of recovery vs cost of backup.

Also ground strap is interesting. Normally i earth myself to the chassis before pulling cards and make sure i don't generate lots of static thereafter. i guess this is similiar to what your doing with the media. This may not be so effective if the system isn't grounded.

Cheers,

 

[Dave Weber]

LaCie's are diasters waiting to happen. Look at G-tech instead.

Funny you say this. I have a G-Drive down right now but all of my LaCie drives are running like champs (Knock on wood)

 

[Simon Baechler]

I think this thread is very important. A lot of people who ordered the camera haven't thought this problem out yet.

Reel or in this case card numbers are important throughout the workflow. Because it helps find media if the director wants to look at a particular shot. The camera assistant gives every card a number as he would do with a film mag. The script supervisor notes this number as well. And when the loader downloads he copies the media in a folder with this number. That way it will be quick to find a certain shot.
Maybe there should also be a track keeping of card-numbers like there is with film mags. In case there is a problem with the footage you know on which card this was recorded to. On the other hand if there was a problem then it's the card that was in your card reader when the error message appeared.

I like the idea of using LTOs. Maybe in combination with a RAID 5 that stays on the set and the LTO goes to the editor. It would be safer to ship a tape than to ship a harddrive.
Can you do multiple sessions on an LTO? You would not want to dump all of that days 100GB to the tape at the end of the day. Even if the loader doesn't get paid overtime

 

[Cail Young]

The camera assistant gives every card a number as he would do with a film mag. The script supervisor notes this number as well. And when the loader downloads he copies the media in a folder with this number. That way it will be quick to find a certain shot.
Maybe there should also be a track keeping of card-numbers like there is with film mags. In case there is a problem with the footage you know on which card this was recorded to.

It's important to keep the same distinction the film production workflow has between "mag" and "roll" - the "mag" here is the CF card itself, and is worth tracking in the event of faulty cards. The "roll" is a specific set of data/footage on a given mag until it's copied and made safe. This is why the camera sets an incremental roll number on format rather than a mag number. The mag number (ID - I've seen letters used) should never change.

 

[Drew Mylrea]

Should this be in the Workflow section?

 

[Jay A. Kelley]

For myself, I am looking at copying to two drives.

One of them will be a RAID 5 and the other a faster larger RAID.

Off-Set these will be backed up to a couple LTO-4 drives. One Master, and one work tape.

We'll see.. This will change a lot I have no doubt.
Jay

 

[Paul Leeming]

I went looking on the weekend for a simple two drive backup solution and found this in one of the shops here in Akihabara:

http://www.jpcentury.com/pro_con.aspx?id=P_00000005

Basically it looks like you use a USB connection from the host computer to write to two PATA or SATA HDDs simultaneously (in Mirror mode). It looks like it incorporates a verification function as well, plus it powers both drives from the included power cables.

For our on set workflow, having two 3.5" external HDDs wrapped in those silicone protection sleeves you can get for $5 should provide a great backup and handoff solution to the client without carrying around anything more than a CF capable laptop and this device.

It's priced at JPY 24,800 so I may pick one up to play with it and verify its abilities before committing to it for four cameras.

Any other solutions out there?

Paul

 

[tj williams]

trying to include what is new in the summary. any help appreciated!
TJ

1. mark each media mag (CF or Drop in disk/memory based mag.) with its own mag. number or symbol. This number or symbol is used to trace any recording failure to the specific piece of media used. This is usually called the "mag number" This along with the "reel number", or sometimes called "roll number" which increments each time media is replaced, should be marked at the head of the recording. They can either be put in as data or by shooting a slate at the head of the recording.

2. record to either single media, or if allowed by RED Technology, to dual card or card/disk or dual disk mag. on camera.

3. Dit wears a grounding wrist strap and grounds camera during media removal. Possibly Dit asks for the camera to be powered
down during off loading? ChronoSync (http://www.econtechnologies.com/site..._overview.html) may be preferable to check sum. I strongly feel RED needs to provide the data check software as part of RED Cine to adapt a technology which they
stand behind.

4. If using single recording, pull partially filled card or drive mag immediately after crucial shots for transfer.

5. immediately pull the re-protect tab on CF media.

6. Place the recorded media into some particular bag or left pocket etc. where you always store the media until you get it out to
put into the onset backup storage. (This is subject to the info. that handling is a major source of CF failure.also to avoid the possibility of confusion between the media just removed and the media which is ready, having been dubbed to be reused.)

7. Set computer and Esata or Firewire 800 raid are set up on an anit static matt.

8. download card to either dual LTO's, two separate drives, redundant raid. or perhaps some combination of slow raid/ fast raids/LTO's or seperate drives as seems good for the particular production. (S2 docking system. may be the best option at this time for uncompressed 4K as there is not yet a better system from RED)

9. RED cine automatically does check sum (or perhaps faster or better data check) and automatically gives a simple answer that data is copied correctly and checked and then and only then gives the option to reformat the card or disk mag.

10. RED cine reformats the disk with the same project settings.

11. Assistant places the blank formatted media into some repeated storage bag or box and or perhaps keeps it in their right
pocket until it is reloaded into the camera (see () above)

12. media is installed onto camera and (if RED will support this) A menu item flashes on the camera asking for the reel number.
the time code continues from the last media recording so there is not time code gap between reels. This will assure us that there is no reel missing on the backup, or that reels have not been copied to backup twice.

14. Deliver one to post house. *(when one of these must be shipped consider shipping LTO tape as it will probably survive better than disks in rough handling.)

15. Using a seperate delivery method deliver one to safe storage at a separate location from the post house.*

 

[Paul Leeming]

5. immediately pull the re-protect tab on CF media.

My CF cards don't have a protect tab or slider, only my SD cards do. Do Red CF cards have a slider?