Welcome to our community

Be a part of something great, join today!

Log in
  • Hey all, just changed over the backend after 15 years I figured time to give it a bit of an update, its probably gonna be a bit weird for most of you and i am sure there is a few bugs to work out but it should kinda work the same as before... hopefully :)

thinking about inexpensive storage solution - 10GB Ethernet & iSCSI

C

chrisvater

Member
Joined
Jun 24, 2007
Messages
7
Reaction score
0
Points
0
Age
50
Website
www.impactquality.com
10GbE Storage Accelerators and Ports have come down in price a whole lot. If you had a 10GbE server adapter and decent SATA 7200rpm drives in a good server, wouldn't you be talking about a really inexpensive and hot-swappable storage, online/offline editing and shared network solution for multiple workstations? Is it possible a smaller company like mine could have massive streamlined and central storage without breaking the bank in the near future? I haven't read about anyone using IP SAN for storage with their RED workflows on this site yet, so I am feeling like I could be way off; I am not super smart about this stuff, but from what I have seen, I could swear this type of 2k (or even 4k, why not?) data transmission is possible. Isn't it possible that this is how real time collaboration over office networks and even across long distances through high-speed internet becomes affordable to more of us? Or am I just delusional?
 
If you want 4K, you will need 900MB/sec :shifty:

Cheers

Pop
 
Pop said:
If you want 4K, you will need 900MB/sec :shifty:

Cheers

Pop
Click to expand...

That would mean you only get < 9 seconds to an 8GB flash card...

I really think the right figure is more like 900Mb/s, which should work well for a 10Gb/s fibre connection if the server is up to it. In real life, you can count on about 10% of the rated network speed being available for time critical applications.
 
Pop said:
If you want 4K, you will need 900MB/sec :shifty:
Click to expand...

4K RAW was quoted as 323MB/s - so RGB would logically be around triple that. However you may not need to transport the RGB if the client computer can handle the decode; all rather moot because there's nothing that will debayer and display 4K at 4K in real time. (Please prove me wrong! I want one!)
 
4k RAW at 24fps is 27Mbyte/sec wich equals 216Mbit/sec,
means 1620Mbyte for one minute wich equals 1,58 Gbyte
So a 8GB CF Card takes arround 4-5min of 24fps 4k wich was also stated somewhere else.
Hope this helps on data rate conversion ;)
 
I installed a TerraBlock T24 in a post house with Fiberchannel connections and it moved HD cam 1:1 at 120Mbs..When I get my Red I will be able to test it on this system.
 
I've talked about using a SAN before. I have one here... Mine isn't 10G based, only 1G. Under ideal conditions, I could send data around at rates over 100MB/s. A lot depends on your work load and work flow and how you structure your SAN. But they can be good for "live" backups and serving up your data to multiple workstations as well as easily expanding storage volumes that are accessible from all your connected systems.

As for data rates, this depends once again on your workflow. Uncompressed 4.5K RAW at the full 60fps would need about 1GB/s. But who is going to be doing that? Especially with the RAW port apparently cancelled now.

Even something in the 300~400MB/s range is very demanding. Even with a SAN, you will probably only have one or two specific nodes and workstations, even in a mid-sized facility, that would handle that sort of rate. Simply having 10G-E or quad (or more) channel Infiniband network topology isn't going to give you those transfer rates. You need the systems and storage hardware to put up the numbers too.

I think that 10G is going to become very common over the next couple years. It would make sense, IMO, for anyone doing RED production, even small boutiques or operations with only a few people. A SAN isn't necessarily a proper solution for everyone, but at least a 10G backbone and connection to a common server or storage system will keep network traffic from being a bottleneck in most situations for now. Even if individual workstations are connected at 1Gbps.
 
The cards to connect individual workstations through 10 GbE are down to $1,000.00 already. They have already tested and verified 10GbE moving data well over 3Gb/s, which would mean (not that smaller companies would want to) you could even quickly move RAW 4K with an optimized SAN using 10GbE and iSCSI. This is, of course, assuming you have very fast workstations optimized for this purpose. My company is already using these (minus the 10GbE cards) and future systems that will work well in this configuration are also affordable to a smaller company like mine. Essentially, I think what this will allow for is what high-end post houses and studios have paid 100's of thousands of $$$ for in the past.

Someone mentioned that the hard drives you use in this type of SAN setup would also need to be able to handle the speeds...does anyone know for sure that decent SATA 7200 rpm drives would not work with a 10GbE configuration?

They did, after all, send 1TB of data over 7,000 miles in less than one hour....i can only imagine how fast it would be from my computer to a server in the same office!
 
David Birdy said:
I installed a TerraBlock T24 in a post house with Fiberchannel connections and it moved HD cam 1:1 at 120Mbs..When I get my Red I will be able to test it on this system.
Click to expand...

I can highly recommend the Facilis Terrablock. Excellent product. Facilis has the best customer service I've ever dealt with in any industry. I've been using a Terrablock SAN for almost 2 years - love it. If you're considering a SAN, they should be at the top of your list. I don't have a Red One yet, but I believe that the Terrablock, properly set up (which is sooo easy), would handle just about any workflow you could throw at it.
 
Didn't realize 10G cards were at the $1K mark or less already. That was quick! That means they're almost ready for prime-time.

As for drive speeds, no single drive out there can sustain speeds of 10Gbps. But 7200rpm SATA drives will work in a sufficiently large RAID configuration. It won't be small and it won't be cheap. Some of the higher performance 10K and 15K rpm drives would be more useful, but a large RAID is still required and the cost per gigabyte really skyrockets with those faster spinning platters. Also, the RPM of a drive is only one factor dictating speed. The number of heads, platters, data density on those platters, read cycles, head tracking, etc.. all play a role.

But sustaining the full 10Gbps bandwidth with each storage system on the network isn't typically the primary goal, or even an objective. The real benefit of a 10G-E implementation is to remove bottlenecks due to network traffic. Bottlenecks and collisions on 1Gbps networks are becoming a common thing these days as file sizes grow and systems and storage can fully load those 1G data pipes.

BTW, those Facilis Terrablock products look pretty slick.
 
chrisvater said:
This is the type of advancement (under 100,000$$$!) that will lead to ALL theaters projecting digitally. I guarantee you, we are VERY CLOSE to it.
Click to expand...

Sony's SXRD 4K panels will drop down to "affordable" soon enough, too. It isn't projection that is the issue, it's direct-view screens for post.
 
A major issue from my perspective is the ability to finish a movie and directly send it to theaters for projection. Granted, without a 4k display, you will have to make compromises in your post workflow, but whether delivering a 2K or 4K film through digital files directly (without a filmout) I think this type of direct distribution is very important to a lot of the indie filmmakers who will be shooting with the RED ONE> you can compromise by editing at 1K (or 2K if your system allows) but storage is the main issue, while being able to directly view every pixel of resolution will be secondary for most of us. Unless, of course, you are doing advanced special effects or seriously tweaking the footage. In that case though, you will probably have enough of a budget to at least rent a 2K or 4K display for the final stage.

But back to storage solutions, from what was posted recently, it sounds like the 10GbE SAN solution (with affordable 7200rpm drives) would at least be adequate for an offline edit and conform to 2K (or 4K) without taking days and days or costing an arm and a leg, while also allowing more than one workstation to simultaneously work on the project.

If anyone is upgrading their SAN or implementing a 10GbE structure, I would love to hear more about what you are doing. I would not be setting up my own SAN for at least 6 months, and I will wait until the last minute to do so.
 
You must log in or register to reply here.
Back
Top