Welcome to our community

Be a part of something great, join today!

Register Log in
  • Hey all, just changed over the backend after 15 years I figured time to give it a bit of an update, its probably gonna be a bit weird for most of you and i am sure there is a few bugs to work out but it should kinda work the same as before... hopefully :)

Post-Production Storage Systems in 2018

D

Dennis S.

Well-known member
Joined
Aug 19, 2012
Messages
48
Reaction score
0
Points
0
Hey guys,

right now we are in the process of replacing our old storage system.
Its a 24bay FC 8Gb/s storage system, filled with 3TB harddrives. Attached to the main storage array is a 24bay JBOD via SAS, also with 24x3TB harddrives.
We export four Raid6 Sets out of this storage system, giving us rougly 27TB (usable) storage per raidset.
The storage server is directly attached via fibre channel to a 8Gb/s FC-HBA on our workstation.

As you guys are all aware the market of storage solutions is vast and while we are willing to invest, we can`t afford something like a TerraBlock or any other Turnkey Shared Storage solutions.
Going the hardcore DIY route is something i would like to avoid as well, even if its tempting money wise.
I am looking for something in between and i would like to have your opinions on what options i might consider or shouldnt overlook.

Given the fact that this is 2018 and we are talking 12TB per 3.5" HDD you can get a lot of capacity even out of smaller storage systems.
I know that among certain people the QNAP TS-1685 is pretty popular, given the relatively low price point and the capability of having a "shared storage server" without any proprietary magic.
I am a little bit concerned if a NAS server can compete with a block level access based storage server in terms of performance because you introduce additional overhead due to the SMB protocol for example.
Then again these things might not have such a big impact with SMB 3.0 and SMB direct but i am still looking for opinions i guess.

Nowadays we mostly use Resolve as our main NLE and we would benefit significantly from having a storage system which multiple workstations can access simultaneously.
Basically i am open for suggestions or ideas and i think this is a good place to ask cause many people here probably had (or still have) the same question a while ago and already found answers. :)

Thanks in advance for your help.

Dennis
 
the QNAP TS-1685 will work perfectly with .r3d footage and DaVinci Resolve ( as well as with FCP X, AVID Media Composer, Adobe Premiere and After Effects).
The TS-1685 is currently under $2800, and has 12 drive bays, so with even 10TB drives, you will get 100 TB after RAID 6. The 10G ports are built into the QNAP
Just get a small 10G switch (like a Netgear XS708T or QNAP QSW-1208-8C) and you are ready to go. Your computers should have 10G thunderbolt adaptors or 10G
NIC cards. For a PC, the QNAP QXG-10G1T or Sonnet PCIe card is under $100, and you will get over 1000 MB/sec in a x4 lane slot. For Macs, if you have a thunderbolt 3
Mac, you can get a Sonnet Solo 10G for $175, and QNAP is about to release one that is even cheaper. For Thunderbolt 2 Mac's, the Sonnet Twin 10G or the Promise SanLink2
is about $495 (sorry - nothing cheaper, and I don't recommend the Akitio Thunderbolt 2 buss powered 10G adaptor).
I install these all the time - they are great, and reliable - and CHEAP ! And if $2800 is too much money, QNAP has even cheaper options.

Bob Zelin
 
What kind of speeds and storage capacities(and in how many groups) are you aming for?
How many workstations do you want to attach?
Do you want a mix between spinning drive for the more cold storage and SSD's for hot storage?
Do the drives have to be hot-swapable?

What are your goals?
 
I avoid the redundancy of RAID and backup to LTO-6. I want to use LTO-8, but there is a lawsuit between Sony and Fuji that is screwing this up, but the tape backups offer no electronic parts that likely will fail or not boot up and 30+ years of reliable storage. They have an index on the front of the tape and mounts like a hard drive so you can retrieve only the files you need.
 
Brian Timmons said:
https://www.lumaforge.com

Heard nothing but good things and they are lower cost than many other options.
Also hear that the support is great and the product is very solid.
Click to expand...
Interesting product i will check it over over the next few days. Thank you.

Bob Zelin said:
the QNAP TS-1685 will work perfectly with .r3d footage and DaVinci Resolve ( as well as with FCP X, AVID Media Composer, Adobe Premiere and After Effects).
The TS-1685 is currently under $2800, and has 12 drive bays, so with even 10TB drives, you will get 100 TB after RAID 6. The 10G ports are built into the QNAP
Just get a small 10G switch (like a Netgear XS708T or QNAP QSW-1208-8C) and you are ready to go. Your computers should have 10G thunderbolt adaptors or 10G
NIC cards. For a PC, the QNAP QXG-10G1T or Sonnet PCIe card is under $100, and you will get over 1000 MB/sec in a x4 lane slot. For Macs, if you have a thunderbolt 3
Mac, you can get a Sonnet Solo 10G for $175, and QNAP is about to release one that is even cheaper. For Thunderbolt 2 Mac's, the Sonnet Twin 10G or the Promise SanLink2
is about $495 (sorry - nothing cheaper, and I don't recommend the Akitio Thunderbolt 2 buss powered 10G adaptor).
I install these all the time - they are great, and reliable - and CHEAP ! And if $2800 is too much money, QNAP has even cheaper options.
Click to expand...

Thank you for your help. Its good to know that you have experience with the TS-1685.
Do you have rough performance figures (read/write) of what i can roughly expect of a TS-1685 filled with 12TB harddrives and using the SSD cache option (with M.2 drives) in an SMB 3.0 environment? (using Win10 x64 clients with Intel 10Gbe NICs)
I know it depends on my factors but i just need to be able to figure out the rough performance metrics of the TS-1685. My biggest concern right now is the performance impact that SMB has on the r/w performance. (compared to block level access storage.)
Our footage is indeed mostly r3d. (5-8K) The NAS should be able to scale beyond the 12slots that the TS-1685 offers, but afaik QNAP offers a SAS expansion chassis for the device.

Misha Engel said:
What kind of speeds and storage capacities(and in how many groups) are you aming for?
How many workstations do you want to attach?
Do you want a mix between spinning drive for the more cold storage and SSD's for hot storage?
Do the drives have to be hot-swapable?

What are your goals?
Click to expand...

We plan for 2 workstations that need to access the storage server simultaneously, maybe a third one later on but thats is unlikely.
Speed should be as high as possible by still using regular 3.5" harddrives, cause even though SSDs dropped in price over that last few years our capacity demands are so high right now that we can`t afford doing this entirely flash based.
Goals are to get a much higher capacity and more r/w speeds out of a server, that is ideally smaller in size (<4HE) but even more important not as noisy as our current server, cause we were forced to put all of our storage servers into our office building a while ago. We have to work mainly with RED footage (Helium 8K S35) most of it will be 6-8K from the beginning of 2019. The rest is mostly drone footage (CinemaDNG). So we need s storage solutin that can deliver the R/W performance for that kind of footage.
As a starting point i would say at least 120TB of usable storage. Backups will be dealt with seperately.

Tom S said:
I avoid the redundancy of RAID and backup to LTO-6. I want to use LTO-8, but there is a lawsuit between Sony and Fuji that is screwing this up, but the tape backups offer no electronic parts that likely will fail or not boot up and 30+ years of reliable storage. They have an index on the front of the tape and mounts like a hard drive so you can retrieve only the files you need.
Click to expand...

Of course backups will be planned seperately. In case of an affordable solution like the TS-1635, we would probably just purchase two of them and put one of it offsite to backup everything to that machine.
Additionally we archive everything to LTO tapes (unfortunately still LTO5) but the first backup tier is storage based and the last pillar of defense against data loss is our lto library. ;)
We wouldn`t go as far and skip the redundancy that certain raid levels are offering cause drives will fail and in our case its more a question of downtime in case they do.
(restoring huge amount of data from lto5 tapes with roughly 120MB/s can cost a lot of time and therefor money)

Michael Lindsay said:
We have a editshare system here and have only good things to say.... this is our 2nd editshare because their support is 1st class!
Click to expand...

Editshare is probably out of our reach financially but i heard they offer editshare as a software package as well for 3rd party storage servers. (SaaS)
That sounds interesting cause it creates more options for people who trust and have good experience with editshare but want to rely on other storage hardware.
 
There's lots of good options out there. QNAP is very popular as is Lumaforge's Jellyfish (and other options) for small multiroom setups.

Some companies you can contact to compare prices and available options:

http://www.accusys.com
https://www.avid.com/products/avid-nexis-pro
http://www.facilis.com
http://www.lumaforge.com
https://opendrives.com
http://promax.com/shared-storage/
https://www.qnap.com/en-us/
http://www.studionetworksolutions.com/
https://www.synology.com/en-us

We're using several G-Speed XLs at the moment, but we've pretty much outgrown them and will have to go to shared storage sooner or later.
 
Bob Zelin said:
the QNAP TS-1685 will work perfectly with .r3d footage and DaVinci Resolve ( as well as with FCP X, AVID Media Composer, Adobe Premiere and After Effects).
The TS-1685 is currently under $2800, and has 12 drive bays, so with even 10TB drives, you will get 100 TB after RAID 6. The 10G ports are built into the QNAP
Just get a small 10G switch (like a Netgear XS708T or QNAP QSW-1208-8C) and you are ready to go. Your computers should have 10G thunderbolt adaptors or 10G
NIC cards. For a PC, the QNAP QXG-10G1T or Sonnet PCIe card is under $100, and you will get over 1000 MB/sec in a x4 lane slot. For Macs, if you have a thunderbolt 3
Mac, you can get a Sonnet Solo 10G for $175, and QNAP is about to release one that is even cheaper. For Thunderbolt 2 Mac's, the Sonnet Twin 10G or the Promise SanLink2
is about $495 (sorry - nothing cheaper, and I don't recommend the Akitio Thunderbolt 2 buss powered 10G adaptor).
I install these all the time - they are great, and reliable - and CHEAP ! And if $2800 is too much money, QNAP has even cheaper options.

Bob Zelin
Click to expand...

Again thank you Bob for your posts. $2700 gets you into the base TS-1685 with the 250w psu and 16gb of RAM. Is that enough for serving 5-8K footage to 3-4 for computers? Or should I look at more ram and watts at the $3600 range? I am pretty much solid on this unit after talking to others and seeing your posts. Just unsure of the configuration of the specs. Thanks!
 
Dennis S -
with a SMB connection to the QNAP (you MUST disable SMB signing on the Macs or you won't get these speeds) - with a thunderbolt 3 to 10G adaptor (like the Sonnet Solo 10G or an iMac Pro native 10G port)
you get 1000 MB/sec READ (based on AJA System Test 5K with 16 Gig test) - same with Blackmagic Speed Test. So can you play back 8K raw - YES. Multiple users - YES.
If you only have 2 users, you don't even need a switch. The TS-1685 comes for free with two 10Gbase-T ports.

For Josh - yes, the base model with the 250 psu and 16 Gig of RAM is all you need. The larger power supply is for installing an Nvidia GTX-1080ti if you are going to setup a VM in the QNAP for After Effects rendering.
For straight editing, you DO NOT NEED the fancier models. For 3 - 4 computers (which will require a small Netgear 10G switch) - 16 gig of RAM is plenty.
You can always contact me for more information ! And I don't sell a damn thing !

Bob Zelin
 
Marc said:
There's lots of good options out there. QNAP is very popular as is Lumaforge's Jellyfish (and other options) for small multiroom setups.

Some companies you can contact to compare prices and available options:

http://www.accusys.com
https://www.avid.com/products/avid-nexis-pro
http://www.facilis.com
http://www.lumaforge.com
https://opendrives.com
http://promax.com/shared-storage/
https://www.qnap.com/en-us/
http://www.studionetworksolutions.com/
https://www.synology.com/en-us

We're using several G-Speed XLs at the moment, but we've pretty much outgrown them and will have to go to shared storage sooner or later.
Click to expand...

thank you, will check those out too. :)

Bob said:
Dennis S -
with a SMB connection to the QNAP (you MUST disable SMB signing on the Macs or you won't get these speeds) - with a thunderbolt 3 to 10G adaptor (like the Sonnet Solo 10G or an iMac Pro native 10G port)
you get 1000 MB/sec READ (based on AJA System Test 5K with 16 Gig test) - same with Blackmagic Speed Test. So can you play back 8K raw - YES. Multiple users - YES.
If you only have 2 users, you don't even need a switch. The TS-1685 comes for free with two 10Gbase-T ports.

For Josh - yes, the base model with the 250 psu and 16 Gig of RAM is all you need. The larger power supply is for installing an Nvidia GTX-1080ti if you are going to setup a VM in the QNAP for After Effects rendering.
For straight editing, you DO NOT NEED the fancier models. For 3 - 4 computers (which will require a small Netgear 10G switch) - 16 gig of RAM is plenty.
You can always contact me for more information ! And I don't sell a damn thing !
Click to expand...

thank you for providing that information.
You gave an example earlier where you were talking about putting 12x10TB drives in the TS-1685 for 100TB storage in RAID6.
I am a little bit concerned of creating such a huge single RAID6 array. (single 12 drive array) I would assume the rebuild times in such a big array consisting of such big harddrives will take a long time and
the longer the rebuild time the more the raid array is exposed to the risk of data loss while the rebuild takes place. (in case 2 drives fail while it does or 2 drives went bad in the first place)
I was thinking about 2xRAID6 arrays each with 6x12TB hardrives in RAID6 and maybe even striping that array to a single RAID60. What are your thoughts on that if you have any?

I am considering the TS-1685 as a base model and maybe upgrading the RAM myself. Do you have any experience in going with regular DDR4 2133 modules instead of the recommended one from QNAP from Transcend?
What is your opinion about the SSD/M.2 SSD caching that you can make use of to further improve read/write performance. Do you think its unneccessary from your experience or do you recommend using it?

Again thank you.

Chad said:
45drives.com
Click to expand...

i heard of them, i think Linus from linustechtipps is using their prodcuts frequently afaik.
As far as i know they just deliver the hardware part, there is no software preinstalled without it, i might as well go DIY.
 
Chad Lancaster writes -
45drives.com

however, this is not what Chad Lancaster uses -
from the www.Akitio.com website

AKiTiO is the official storage solutions provider for The Southwest Film. As a gear sponsor for this project with its massive files from 4K and 6K footage, we recommended our fastest storage solutions with multiple drive bays and the lightning fast Thunderbolt™ 2 interface. This provides both the speed that is needed to move the data and the option to RAID the drives for extra performance or redundancy. As with all the important data, it's always important to backup and back it up again.

“This may all sound a bit extreme, but for a film its what we have to do to protect the footage. Remember that this also applies to your personal data. Don't put all your eggs in one basket. Again make sure you have multiple copies in multiple locations. Ask a friend to join together and give each other hard drive copies so that you both are backed up.”, states Chad Lancaster from The Southwest team.
 
I use AKiTiO enclosures on location but when you've got hundreds of terabytes of data you need something like 45drives that can hold a lot of drives and has server level of redundancy.

and Dennis yes 45drives comes with the software, it's not just the enclosure.
 
Hi Bob, We've interacted several times over the years and I've always found your advice to be solid. Likely like many others here my direct attached SAS Raids are getting a little long in the tooth and I'm considering the QNAP TS-1685. I've got near-line managed with another server and I archive on LTO, so I'm really just looking for fast edit storage and to free up some PCI lanes by moving to 10gig. Which begs the question how many spindles do I need or what combination of m2 cache / drives is required to get north of 1000 MBs? The sweet spot on price seems to be the 8 - 10 TB drives, but frankly I don't need the capacity of 12+ of those. The config would be raid six. But I suspect that moving to anything under 8 drives won't be fast enough. What do you think?
 
Hi Stephen -
the QNAP TS-1685 is great, and it's $2790 street price. 8TB Seagate Ironwolf Drives are now $250 and 10TB are $300. The 10G ports are included with the TS-1685. The TS-1677x is the same box but with
AMD CPU instead of Intel CPU. Both can do 8K all day long, with Resolve, FCP X, Premiere, and even AVID (if you run Indiestor Mimiq). If you do not want to spend the money for the TS-1685, and do not want to
buy 12 drives, the least expensive QNAP you can get is the TVS-873e. This is currently $1269, and has 8 drives (and yes, you need 8 spindles or you are wasting your time). You also must purchase an option
card for this which is the 10G card - it's the QNAP QXG-10G1T, and that is $89 bucks. You plug that into a cheap switch, like a small QNAP or Buffalo 10G switch, and now you can connect up to 7 clients. The small switches are between $500 - $580. Of course, you need thunderbolt to 10G adaptors on your Mac, and if you are running Windows 10 PC's, you need 10G PCIe cards. For the PC's, these 10G cards today are usually under $100 each. ASUS, QNAP and Sonnet makes them, and they are all fast.

Please let me know if there are any other questions I can answer for you. I do this every day.

Bob Zelin
 
I love this thread! I'm waiting for my QNAP TS-1685, ordered yesterday. Ive been struggling with 8K running off my USB3 caddy, lol. Thanks for the advice Bob.
 
Bob Zelin said:
Please let me know if there are any other questions I can answer for you. I do this every day.
Click to expand...

If you dont mind, i quote myself here and just repeat my questions in case you overlooked them. :)

Dennis said:
thank you for providing that information.
You gave an example earlier where you were talking about putting 12x10TB drives in the TS-1685 for 100TB storage in RAID6.
I am a little bit concerned of creating such a huge single RAID6 array. (single 12 drive array) I would assume the rebuild times in such a big array consisting of such big harddrives will take a long time and
the longer the rebuild time the more the raid array is exposed to the risk of data loss while the rebuild takes place. (in case 2 drives fail while it does or 2 drives went bad in the first place)
I was thinking about 2xRAID6 arrays each with 6x12TB hardrives in RAID6 and maybe even striping that array to a single RAID60. What are your thoughts on that if you have any?

I am considering the TS-1685 as a base model and maybe upgrading the RAM myself. Do you have any experience in going with regular DDR4 2133 modules instead of the recommended one from QNAP from Transcend?
What is your opinion about the SSD/M.2 SSD caching that you can make use of to further improve read/write performance. Do you think its unneccessary from your experience or do you recommend using it?

Again thank you.
Click to expand...

Thank you in advance for your help.
 
Hi Dennis -
you should not be afraid to create one large RAID 6 array. I have found from experience with multiple systems (more than just QNAP and Synology) that once the drives are "stable" (which you usually find out if you have bad ones within the first 48 hours) - you rarely see failures. And I am talking about YEARS later. I have seen numerous RAID arrays last 5 years without one single drive failure. Modern drives like WD RED Pro's and Seagate Ironwolf drives are very stable, and very reliable. It's not like the old days. If you want max speed (so you can easily edit with 8K RAW footage) - create one single static volume RAID 6, and you will be just fine, and get great speeds.
I unfortunately have no experience with M.2 SSD. I have lots of experience with SSD Caching, specifically with Samsung 850 and 860 EVO series SSD's in a QNAP. Bottom line - it does absolutely nothing for improved performance with video editing, and transcoding - specifically with programs like RedCine-X, Davinci Resolve, AVID Media Composer, Apple FCP X, Adobe Premiere, Adobe After Effects, Adobe Media Encoder, and Cinema 4D

You can certainly put together a TS-1685 or TS-1677x with just the SATA drives (as long as they are good quality 7200 RPM drives) and have no issues with 8K editing. What's "funny" is that while your 8K .r3d footage will play all day long, with MP4 footage from cameras like Go Pro's or DJI drones, you will STILL get stuttering playback. Which is why transcoding is so important. (at least in Premiere).

Bob Zelin
 
oh - I did not answer your question about rebuild times. A 12 TB drive takes about 8 - 12 hours to rebuild in an .ext4 format (QNAP). A BTRFS file system will take DAYS to rebuilt (G-Tech GRACK12 or Synology).
For Synology systems, you can choose an .ext4 format when you first setup the RAID. I have no idea of why Synology continues to use BTRFS. Even the Linux RedHat group advises now not to use BTRFS as a file system.

Bob Zelin
 
You must log in or register to reply here.
Back
Top