Welcome to our community

Be a part of something great, join today!

Log in
  • Hey all, just changed over the backend after 15 years I figured time to give it a bit of an update, its probably gonna be a bit weird for most of you and i am sure there is a few bugs to work out but it should kinda work the same as before... hopefully :)

DIT/Loader Cart

Nick,

Get the Atto H680. If you need three now, having eight available doesn't hurt. Atto makes a one SAS port (4 esata) card (H644), but the price is about the same as the H680. Get the H680.

There are a couple ways I save time with a fast raid.

First lets look at how R3Ddata manager and shotput work. They do one read and multiple writes. They can't read again until the write is finished. So they can only copy as fast as the slowest destination drive. If you have two drives that work at light speed and one slow old USB drive the entire copy will be at the speed of the slow old USB drive. If you have a slow source, like Red station connected via USB and you are writing to multiple super fast destination drives, you will be limited by the bandwidth of the USB source.

I wrote my own little app (posted on Reduser) that copies to the fast raid first and from the fast raid to all other drives. It checksums the source media and each of the destinations as soon as the copy is done and compares all the checksums to the source. The first place this speeds things up is as soon as the card is copied to the raid and checksumed, I can pull the card and send it back to camera. The card just has to get through one superfast copy/checksum and it is ready to go back to set. Since my raid is raid 6 I have lots of protection for that one copy. Raid 5 works great, but gets really scary when you loose a drive on set and you have no protection until the raid rebuilds several hours later.

Next my script is general faster than RDM or Shotput because one semi performing destination drive really kills them. And as drives fill up they slow down. With three fast drives and plenty of bus, RDM is slightly faster and I have not tested the new Shotput, but the old shotput was stupid slow. My app gets a speed advantage because while it is waiting for the slow drive to finish, it can start the checksum process on the rest of the drives. RDM has to wait for all the copies to finish before it starts checksum. So two drives can be sitting idle while waiting on the third drive to catch up so they can move to the next step.

Checksums can really be slowed down by a slow or shared bus, because multiple drives are being read at the same time. If each drive can read at 120MB/s, but all three are on a 100MB/s firewire bus, they will only read at 33MB/s each while reading at the same time. That is only a 20% performance hit for a single drive, but about a 65% performance on parallel operations.

Adding a fast raid to RDM or Shotput won't speed up your process. It will give you fast storage to work from without impacting downloads.

Hopefully that makes sense.


Dusty
 
Totally makes sense.

And while I have you here, I'm just gonna keep picking your brain. Is this the fan cable you were talking about? http://eshop.macsales.com/item/High...n=googlebase&gclid=CIGNlaTp2a0CFYbe4AodzTXBkg

As near as I can tell, that'll get me from the Atto card out to eSata drives. Not a fan of uncommon, semi-expensive cables. If a cable fails, you need to either have a backup already on hand, or you're SOL. Which means I would want to buy 2 or 3, even if we do only ever need 1 actually connected to the card. One of those "small costs add up quick" things.
 
That is the cable. That is the most common one I see, but there are a couple other places that make them in different lengths. I would get two just to be safe, but I have never had one go bad. $500 for an H680 and two fan cables will make your life so much easier, it is worth the cost.


Dusty
 
Dustin Cross said:
Nick,

Get the Atto H680. If you need three now, having eight available doesn't hurt. Atto makes a one SAS port (4 esata) card (H644), but the price is about the same as the H680. Get the H680.

There are a couple ways I save time with a fast raid.

First lets look at how R3Ddata manager and shotput work. They do one read and multiple writes. They can't read again until the write is finished. So they can only copy as fast as the slowest destination drive. If you have two drives that work at light speed and one slow old USB drive the entire copy will be at the speed of the slow old USB drive. If you have a slow source, like Red station connected via USB and you are writing to multiple super fast destination drives, you will be limited by the bandwidth of the USB source.

I wrote my own little app (posted on Reduser) that copies to the fast raid first and from the fast raid to all other drives. It checksums the source media and each of the destinations as soon as the copy is done and compares all the checksums to the source. The first place this speeds things up is as soon as the card is copied to the raid and checksumed, I can pull the card and send it back to camera. The card just has to get through one superfast copy/checksum and it is ready to go back to set. Since my raid is raid 6 I have lots of protection for that one copy. Raid 5 works great, but gets really scary when you loose a drive on set and you have no protection until the raid rebuilds several hours later.

Next my script is general faster than RDM or Shotput because one semi performing destination drive really kills them. And as drives fill up they slow down. With three fast drives and plenty of bus, RDM is slightly faster and I have not tested the new Shotput, but the old shotput was stupid slow. My app gets a speed advantage because while it is waiting for the slow drive to finish, it can start the checksum process on the rest of the drives. RDM has to wait for all the copies to finish before it starts checksum. So two drives can be sitting idle while waiting on the third drive to catch up so they can move to the next step.

Checksums can really be slowed down by a slow or shared bus, because multiple drives are being read at the same time. If each drive can read at 120MB/s, but all three are on a 100MB/s firewire bus, they will only read at 33MB/s each while reading at the same time. That is only a 20% performance hit for a single drive, but about a 65% performance on parallel operations.

Adding a fast raid to RDM or Shotput won't speed up your process. It will give you fast storage to work from without impacting downloads.

Hopefully that makes sense.


Dusty
Click to expand...


Did you ever get a chance to update that app of yours? I remember it had some issues depending on how you were copying and so on. I'd love to give it another try if you're willing to share.
 
Dusty,

Just wanted to say thanks for your help here -- this sort of information and problem solving work is invaluable.

Nice going!

P
 
Hey,

I've one question. I want to put our 2 Pegasus R4 Raids into a Pelicase, ready to start. That means, I open it and connect it to power and my Macbook and I can start to work.
But I don't know, how to secure the Raids into the case. I can put it into foam, but than the Raids don't have any ventilation, so that isnt an practicable option.
Have you an idea?

Greetings
Tino
 
Im glad to see this thread, because I think its important for DITs/Data managers to know their equipment well. What better way to know your equipment inside and out than to be the the person who owns/Built it. Lately there has been a disturbing trend of DITs loosing their cart rental to third party companies. The DIT should have a choice in their cart and computer equipment. Honestly computers are just computers and its their operators that make the difference (Granted you have to have the right tools for the job).

I recently built a new DIT/Data cart and the best advice i can give is to think about ergonomics in your design. You will spend a big chunk of your life working at this cart, small annoyances become big annoyances over time.
 
Ryan McGregor said:
Here's the cart I just had custom made this fall. Made locally in Winnipeg, Canada. I do rock a third monitor on an Ergotron arm on the right hand side that wasn't mounted yet when I took this photo.
Click to expand...

thats PIMPED as ive ever seen
 
Cory Schulthies said:
the best advice i can give is to think about ergonomics in your design. You will spend a big chunk of your life working at this cart, small annoyances become big annoyances over time.
Click to expand...

1000%!!!! That is why I do not like any of the commercial carts out there and built my own carts from experience working on location.

Move a cart with small rigid wheels across a football field! The wheels just dig into the ground.
Move a cart with the MacPro held to the side of the cart by velcro straps across up a across a bumpy parking lot. The Velco comes loose and the tower falls to the ground.
Spend a few days with your SDI monitor directly in front of your computer montor.
Two monitors on arms that easily move around sounds great until you have to move the cart and the monitors are all over that place.

Most of the carts I see are great on a stage, but pure evil once you get on location.


Dusty
 
One of the problems I've been having is that the size of a Magliner DIT cart is too big for a lot of locations I work in. We're always shooting in a small apartment or house and there's never really room for me, which means I'm often confined to be too far off-set (which I hate). So - I've been giving it some thought as to how I could make my footprint a lot smaller but still be able to have a full Mac Pro system with all the bells and whistles with me. On my flight back from NAB it finally dawned on me when the flight attendant ran over my feet with one of their service carts: they are small, lightweight and must be able to take a beating. So, I've picked up a couple of these and will be kitting it out in the next month or so. The plan is to fit a modified UPS in the bottom, a Mac Pro and a couple of Mac Minis, Wifi routers and storage in there. I probably will need to change the wheels, but I'm still undecided about that. I'm also going to fit a fold-out table onto the side of it - possibly with some sort of legs on it for extra support. Not sure what to do about monitors yet. I may fit those into the second cart I bought.

Thoughts?
 

Attachments

  • airlinecart.jpg
    airlinecart.jpg
    95.8 KB · Views: 0
You must log in or register to reply here.
Back
Top