Welcome to our community

Be a part of something great, join today!

Log in
  • Hey all, just changed over the backend after 15 years I figured time to give it a bit of an update, its probably gonna be a bit weird for most of you and i am sure there is a few bugs to work out but it should kinda work the same as before... hopefully :)

Workflow LTO6

Björn Benckert

Björn Benckert

Well-known member
Joined
Dec 5, 2007
Messages
7,423
Reaction score
74
Points
48
Age
52
Website
www.syndicate.se
I´m about to jump on a 10x45min episode 4k web series and I´m assigned to set up the workflow. I calculate for about 200Tb footage but want that to be rather scalable. So the idea is as follows.

DIT:
Transfer mags to his mobile raid 40tb.
Color time and transfer 2k proxies to editorial on transport HDD´s
Sends us 6tb thunderbolt HDD´s with r3d + proxies.

We do:
2 x LTO6 6.25Tb tapes from the DIT´s HDD´s. One for the secure vault and one to work from. (The HDD´s are later curried to the production company, and when they got those drives the DIT can erase from his raid if wanted, that way we got material in 3 places or more the whole time)

Then editorial sends us a XML and ref copy.

My idea is that We conform in resolve straight from the LTO6 tapes and do a r3d trim onto a large 8xHDD pegasus thunderbolt raid.

Then we work on that raid and backup each episode to LTO6 + HDD, 4k prores, DCP h.264 etc. and also resolve project and r3d trim folders. Simply one drive one tape per finished episode.

Now my question: Are you guy doing this? Using XML´s to extract r3d trims straight from the LTO tapes, does it work well, I hear people are doing it but I don't know if it works smooth and so far I got no LTO6 deck to test it on. Plan is to get a Mraid thunderbolt LTO6.
 
Björn Benckert said:
Thanks.

What I´m after I guess is to know if the LTO 6 tapes can be read like a normal drive. As I understand there is a protocol that makes the tape deck/tape show up just like an external drive. If anyone here could confirm that would be great.
Click to expand...

I'm not a pro at all on this, but I believe you want to use LTO as backup and not working. You'll see some really slow times if you're not doing a simple transfer. The deck can only perform a WRITE or a READ at any given time so you can't be reading and writing from the drive...this makes it a tough choice as a working drive. You'll definitely want to go to 2k or 1080 proxies (I'd say ProRes Proxy if you can) and edit off of just a few small drives. Reconnect later if you decide to do an online color. If I am wrong, please let me know!
 
Brendan_H_Banks said:
... The deck can only perform a WRITE or a READ at any given time so you can't be reading and writing from the drive...this makes it a tough choice as a working drive. ...
Click to expand...

But in Björn's workflow scenario, it would limited to READ only. Theoretically, it should work. I would have never suggested or envisioned this type of execution, but not a bad idea.
 
M.Halsell said:
But in Björn's workflow scenario, it would limited to READ only. Theoretically, it should work. I would have never suggested or envisioned this type of execution, but not a bad idea.
Click to expand...

Definitely true. Make sure to keep all your render/project/autosave files on a non-LTO drive....basically anything that requires read/write access. Who knows man, maybe you cracked the code!
 
My projected setup is to store on LTO (7) and offload to a RAID 0 setup for whatever segment of the movie I'm working on at the time. I'll transfer the edit when satisfied with that part and put it on another LTO cartridge as back-up edit. The ability to switch out LTO cartridges makes it easy to store and use because you can use partially filled cartridges to segment the edit.
 
I see the benefit of having LTO7 on set to make the backups. (I won't have a director erasing a Master just to put some stuff on it...) Is there any small LTO7 recorder wich could be used on set and small enough to take in the backpack?

Pat
 
Patrick Tresch said:
I see the benefit of having LTO7 on set to make the backups. (I won't have a director erasing a Master just to put some stuff on it...) Is there any small LTO7 recorder wich could be used on set and small enough to take in the backpack?

Pat
Click to expand...

All the LTO tape drives I've worked with are way too noisy to be used on set. If you have a long enough cable to put them next to the generator, maybe...
 
Brendan_H_Banks said:
Definitely true. Make sure to keep all your render/project/autosave files on a non-LTO drive....basically anything that requires read/write access. Who knows man, maybe you cracked the code!
Click to expand...

I did not crack it. I got this workflow briefly explained to me.

But as I understand and would love to have confirmed is if I can see the LTO6 6.25Tb written tapes as a normal folder on my mac, even if it´s slow and only have read permissions (I don't want read write, I only want write to it one time and then only read a small percentage from it when extracting the selected takes as r3d trims) There are a few nice things with this workflow.

In a way the tapes are far more secure then HDD´s simply because people do not have LTO6 decks. The tapes are also more durable over time than HDD´s. And they cost far less. A 10 pack of new 6.25TB tapes on eBay goes for 400USD that is over 60TB of storage which gives you a cost of 7 USD/Tb...

As I been told, but have not been able to get confirmed, you can set it up so that the tape shows up as a external drive, you can point at it with your XML and have resolve or your NLE to conform and make a r3d trim from the deck onto a faster read write HDD raid then you work from the trimmed files from that raid.

Then I will also do a backups onto LTO6 tapes, one per 45 episode would be enough to hold both 4k masters, folder with r3d trims, XML´s, resolve project, flame archive graphics etc. Simply a low cost tape archive per episode if my raid fail while I work.
 
Found it, theory confirmed. Will order it and test it out. apparently the solution is called LTFS, Linear Tape File System and works on LTO5 and 6.

"One of the exciting features available with the Quantum LTO-5 and LTO-6 tape drives is called the Linear Tape File System (LTFS), which allows you to use the tape almost as if it were a hard disk. You can drag and drop files from your server to the tape, see the list of saved files using a standard operating system directory (no backup software catalog needed), and use point and click to restore. To implement this feature, you simply need to download and install the LTFS software on your host machine, using the link at right.

LTFS is an open standard, interchangeable with other LTO-5 and LTO-6 brands, and is also cross-platform, so you can use the tape as you would a USB flash drive. Simply load an LTFS-formatted tape into your LTO-5 or LTO-6 drive, mount it into the file system, and it becomes visible as if it were a disk. Currently, the Quantum LTFS software supports Windows, Linux and Mac OS X."
 
Björn Benckert said:
I did not crack it. I got this workflow briefly explained to me.

But as I understand and would love to have confirmed is if I can see the LTO6 6.25Tb written tapes as a normal folder on my mac, even if it´s slow and only have read permissions (I don't want read write, I only want write to it one time and then only read a small percentage from it when extracting the selected takes as r3d trims) There are a few nice things with this workflow.

In a way the tapes are far more secure then HDD´s simply because people do not have LTO6 decks. The tapes are also more durable over time than HDD´s. And they cost far less. A 10 pack of new 6.25TB tapes on eBay goes for 400USD that is over 60TB of storage which gives you a cost of 7 USD/Tb...

As I been told, but have not been able to get confirmed, you can set it up so that the tape shows up as a external drive, you can point at it with your XML and have resolve or your NLE to conform and make a r3d trim from the deck onto a faster read write HDD raid then you work from the trimmed files from that raid.

Then I will also do a backups onto LTO6 tapes, one per 45 episode would be enough to hold both 4k masters, folder with r3d trims, XML´s, resolve project, flame archive graphics etc. Simply a low cost tape archive per episode if my raid fail while I work.
Click to expand...

Very solid. I've definitely seen programs that will mount the tape as a drive, especially on OSX. You will most likely be looking at a high program cost if one doesnt come included on your deck (seen prices from $300-1500usd). But even if that adds a few bucks to your per-TB cost, that's still insanely low. If you go this route and test it, I would love to hear your results.
 
LTFS works. I use HP Enterprise Open Store as the Free Open Source implementation.
 
Tape Drive

Tape Drive

Björn,

We appreciate braves souls like yourself going against convention, re thinking the unthinkable. Please circle back with the findings. I'm quite curious if Resolve will hang if it senses the data is not be feed fast enough
The linear nature of the tape record/read process requires the tape to shuffle back & forth until it find what it needs.
 
M.Halsell said:
Björn,

We appreciate braves souls like yourself going against convention, re thinking the unthinkable. Please circle back with the findings. I'm quite curious if Resolve will hang if it senses the data is not be feed fast enough
The linear nature of the tape record/read process requires the tape to shuffle back & forth until it find what it needs.
Click to expand...

Yes that´s what I was thinking as well. Like how can the NLE search for all clips in timeline from the given tape without becoming uber slow / come to a halt. But as I understand this is the practice when doing the conforms on some major productions.
 
Thomas Koch said:
LTFS works. I use HP Enterprise Open Store as the Free Open Source implementation.
Click to expand...

Cool.
So can you confirm that you can have a LTO filled with r3d´s and then with LTFS use a EDL / XML or what not in resolve and point at the tape and resolve can link to the clips? And then you are able to cash or render out the clips as r3d trim.

I understand that it´s far from possible to hit play and that it would really lag if jumping between clips etc.
 
Hans von Sonntag said:
I think that if (big if) the NLE can conform from the LTFS mounted tape drive one has to consolidate the shots in question on a normal HD for further handling.

Hans
Click to expand...

Thats the whole idea. r3d trim for sellected takes onto a pegasus 8 xHDD raid.
 
I wouldn't recommend doing it over LTFS since it's not indexed properly hence the drive will have a much tougher workload and the risk for it to crash is much larger.
I'm working on a project now, shot on ARRIRAW, which uses a LTO7-tape machine and the trick is to use Yoyotta LTFS and Yoyotta Conform since together they work to do exactly this. Yoyotta LTFS does a indexing of the raw-files, then when you've edited you import the AAF/EDL (they are working on XML-support) into Yoyotta Conform and it instantly takes out just the trimmed parts of the .r3d.


Look here for more info:
https://yoyotta.com/yoyottaid/conform.html
 
Vincent Larsson said:
I wouldn't recommend doing it over LTFS since it's not indexed properly hence the drive will have a much tougher workload and the risk for it to crash is much larger.
I'm working on a project now, shot on ARRIRAW, which uses a LTO7-tape machine and the trick is to use Yoyotta LTFS and Yoyotta Conform since together they work to do exactly this. Yoyotta LTFS does a indexing of the raw-files, then when you've edited you import the AAF/EDL (they are working on XML-support) into Yoyotta Conform and it instantly takes out just the trimmed parts of the .r3d.


Look here for more info:
https://yoyotta.com/yoyottaid/conform.html
Click to expand...

Possibly so. But arri raw 6tb and r3d 6tb is two completly different beasts for a index. Simply there is one arri file per frame where as red do one folder per clip with a few files in it. The index for red get considerably much smaller and easier to handle. And sure extracting the full length clips work it´s done since DTF2.

Im looking to get selects with little bit of tales. not whole clips
 
You must log in or register to reply here.
Back
Top