Welcome to our community

Be a part of something great, join today!

Log in
  • Hey all, just changed over the backend after 15 years I figured time to give it a bit of an update, its probably gonna be a bit weird for most of you and i am sure there is a few bugs to work out but it should kinda work the same as before... hopefully :)

Qnap raid system

we have a cheapass, crappy Dobro that is used for matadata backup, and autmagicly syncs with a mirrored Drobro offsite
dead slow, not a useable thing for much of anything beyond project file backups, and it was inherted so no choice on my part
but it does do the one thing it's asked to do, and does it reliably
and it's a very inexpensive pillow to aid sleeping peacefully
 
as for the backup system - remember, the QNAP is cheap, but the drives are not. If you have eight 10TB drives, you need eight 10TB drives to back that up.
It is impossible to backup 80 TB over the internet. You purchase your first QNAP, and then if you want to backup to a second QNAP, you buy a second QNAP (it can be a cheaper model that is not fast enough to edit with),
but you still need eight identical drives, which is the bulk of the cost. You can do the initial huge backup locally in your office, and then bring home the second QNAP, and do incremental backup's over the internet.
When you use a Cloud site - Amazon sends you a "Snowball" which is a 10G 50 TB drive. When you backup to Backblaze B2, Backblaze sends you a Synology. You transfer to it, mail it back to them, and they put it on their cloud
service. You can't just transfer 50 TB to any cloudsite, simply because NO ONE's internet access is that fast.
Systems from QNAP and Synology allow for scheduled backups. So when you get lazy, and you schedule your backup for 9pm every evening to the second device - it automatically happens.
I do this all the time now for people. I dont' see the point of paying for cloud service, when you can simply purchase your own second QNAP or Synology, and put it on the internet for remote access.

When Ben discusses the "8 SATA drive barrier" - SSD's cost MORE MONEY than SATA drives - so exactly what is the point. SSD's are super fast, but they are super small, and super expensive.

Bob Zelin
 
Could you theoretically use less drives in a backup machine as long as the total space it the same? For example an 8 bay qnap with 4tb drives and a 4 bay qnap with 8tb drives?
 
So if I only want sertain folders, not the whole 100 something terabytes to be back up over the cloud what qnap app is best to use?

Our internet speed is 1000Mb/s both ways and I can also upgrade that to 10k Mb/s so moving the stuff over is not really the problem. Just want a hassle free system that keeps a backup of certain project folders that is currently in use.
 
could you use less drives in the backup machine as long as it's the same total space as in the main system (like eight 4TB drives in the main system, and four 8TB drives in the backup system) -
yes but WHAT IS WRONG WITH YOU PEOPLE. You own RED cameras. You own insanely expensive prime lens packages. Each lens cost more than ALL THESE DRIVES put together. Isn't your data important to you (you know, the
stuff that you shot with your RED camera) ? Your media is what makes you money. If you lose your media, you are screwed.

As for Bjorn - you can't backup 100 TB to a cloud site. When you use Amazon S3, they send you a Snowball, which is a 50 TB drive with a 10G adaptor, and you backup to the Snowball, mail it back to Amazon, and they upload it to their server.
And exactly how are you going to download 100 TB from Amazon. When you use Backblaze B2, they ship you a Synology NAS, and you backup to the Synology, mail it back to them, and then they upload this to their servers locally.
Again - you can't upload 100 TB to a cloud site. 1000 Mb/sec upload is 125 MB/sec. That is the equivalent of a 1G connection. It will take you FOREVER. Backing up 100 TB of data is not "hassle free". That's a lot of data. You buy a redundant
system that can handle 100 TB (the same price as your main system), and you backup. Both QNAP and Synology offer free backup software that works within your facility. You backup your 60 TB (60 - 90 minutes per terabyte over a 10G local connection) and then you can backup a few hundred gigs each night as you increment your data to a remote location.

When you have 100 TB of data - I don't care if it's just on the local drive - you have to MANAGE that data, or it becomes a huge mess, and no one can find anything. Just like an old tape library. You just throw the tapes in a closet, and don't label them, and you will never find your footage.
YES - you can select certain folders of that 100 TB to backup.

Today, a 10TB Seagate Ironwolf drive costs $309 US. So 12 drives in a RAID 6 configuration will give you 100 TB of usable storage. That is $3708 for 12 drives. Want a backup - get another 12 drives. So you spend under $8000 for the drives.
A QNAP TS-1685 is $2800, and if you want a cheaper backup with the same capacity, the TS-1635ax is $1500 US.
Is that too expensive ? Do you know how much a Lumaforge with 100 TB costs ? Do you know how much an AVID Nexis with 100 TB costs ? Same with EditShare, DDP, GB Labs, Facilis, Promax.
Is only Apple and RED allowed to make a living ?

Bob Zelin
 
Possibly right so, can't send 100Tb over the web. But that's not really what I'm looking for.

What I would like is to know what qnap cloud app that automatically can upload a few folders from my 100tb NAS. Simply make sure that my ongoing projects is also stored some place else. Old projects are backed up on sepereate drives and so on. What worries me it what is currently in the pipe.

Simply if the server goes toast I want to be able to continue where we left off the day before. To do so I need a synced folder someplace outside the shop. I rather see that as something that is done automatically and that does not need to be manicured. Could be something simple like the last 5 or what ever choosen number of root folders touched on the server is simply mirrored to some remote cloud service.

That way if shop burns down I would only have to worry about setting it up again and the work can continue.

As it now I got everything in one place and such situation could possibly set me back months of several people's hard work which would not fly well with most projects that does hardly have even days to spare before delivery.

Normally we got sorce material on sepereate source discs / raids and then we extract only the trimmed used takes onto the qnap and do the VFX/finishing work from there so per project it's usually not more than a few terabyte per project and when done the project folder is also cloned back to the source raid/disk.
 
Björn, have you looked at Optware-NG, which provides a way (/opt/bin/ipkg) to install rsnapshot (into /opt/bin/rsnapshot)? My tech knowledge may be out of date, but rsnapshot can be a great way to sync together two servers at two different sites. Sync them together once, then move the offsite one offsite. You can tune how frequently to update and how frequently you want to capture and snapshot your changes. The defaults are quite sensible:

https://us.v-cdn.net/6025034/upload.../Daily incremental backups with rsnapshot.pdf

Note that the above document references an earlier world, before Optware became Optware-NG (which must be downloaded from a 3rd party repository).
 
Michael Tiemann said:
Björn, have you looked at Optware-NG, which provides a way (/opt/bin/ipkg) to install rsnapshot (into /opt/bin/rsnapshot)? My tech knowledge may be out of date, but rsnapshot can be a great way to sync together two servers at two different sites. Sync them together once, then move the offsite one offsite. You can tune how frequently to update and how frequently you want to capture and snapshot your changes. The defaults are quite sensible:

https://us.v-cdn.net/6025034/upload.../Daily incremental backups with rsnapshot.pdf

Note that the above document references an earlier world, before Optware became Optware-NG (which must be downloaded from a 3rd party repository).
Click to expand...

Thanks Michael, But I'm thinking this must be something thats available in one or many of the Apps that comes with Qnap, no?
There is several apps that deal with cloud things the difficulty I'm having is to figure out which one to use as the spec for them is not so well documented.
 
Misha Engel said:
Qnap has many ways to protect your data https://www.qnap.com/solution/data-backup/

and

https://www.qnap.com/en-us/how-to/tutorial/article/protect-qnap-nas-data-with-real-time-remote-replication-rtrr

The most cost effective and flexible is to have two NAS devices, 1 in the office (the fast one) and one at a remote location (not to far away for easy access when needed).
Sync them during idle over your fast internet connection.
Click to expand...

Thanks Misha, Yes guess thats what Im looking for. But prefereably I would like to have someone else cater for the remote node. I guess I can place a spare nas that I got at a "server hotel" or such or in my own cellar. But must be someone providing a secure, but price worthy option for this where I can pay per Tb / year or so, no?
 
Björn Benckert said:
Thanks Misha, Yes guess thats what Im looking for. But prefereably I would like to have someone else cater for the remote node. I guess I can place a spare nas that I got at a "server hotel" or such or in my own cellar. But must be someone providing a secure, but price worthy option for this where I can pay per Tb / year or so, no?
Click to expand...

1000 Mbit/s is pretty cheap where I live (The Netherlands) ~ € 40/month.
Energy is around € 0.20/kWh. (average power 100 Watt) > ~ € 15/month

QNAP TS-1635 (8GB ram)( Volume types: RAID 0/1/5/6/10/50/60, + hot spare, single, JBOD) ~ € 1,500 incl. vat
12x Seagate IronWolf Pro, 12TB ~ € 470 incl. vat > ~ € 5,640 in raid 6 + hot spare makes ~ 100 TB
UPS €200
additional cost ~ € 1.060
totals € 8.400 over 4 years (write-off) > € 175/month

total monthly cost €(40 + 15 + 175) ~ €230 when you can put it in your basement.

Pretty good value...
 
yes Bjorn -
QNAP gives you for free Hybrid Backup Sync. This will allow you to backup to a local drive, to a remote drive (another QNAP at your shop, or another QNAP in a remote location over the internet), and it allows to to backup or sync to a cloud service like Google Cloud, Drop Box, Amazon S3, and a slew of others. And yes, it's free, and yes, it can be scheduled. You select the folder that you want, tell it to backup or sync to your cloud site account (like dropbox) and it happens automatically. It's free. Of course, if free is not cheap enough, perhaps I can ask QNAP if they can pay you to use their free software.
No matter what - you are paying for cloud space. You can buy a cheap 2 bay QNAP, stick it at home, and do remote back up from your office to your home. The only "trick" in doing this, is if you have ports blocked on the router that your cable or internet service provider has given you. I face this challenge with almost every install that wants to do remote access. In my most recent install, my client has Frontier FIOS as their internet service provider. Even though I have port forwarded their Linksys router, the ports are still blocked - and of course, the only way for me to unblock these ports is to get cooperation from Frontier (the internet service provider). I have just asked my client to email me a copy of their last bill, so I can call Frontier, and fight it out with them. I have found that fighting with cable companies has become a regular aspect of my installations recently.

Bob Zelin
 
Yes, it is completely possible to use cloud as (preferably) second backup target, but it completely depends on the scenario and the bandwidth available.

I've got over 100 TB in the cloud. No problems.
 
DJ Meyer said:
Yes, it is completely possible to use cloud as (preferably) second backup target, but it completely depends on the scenario and the bandwidth available.

I've got over 100 TB in the cloud. No problems.
Click to expand...


Agree I got 1Gb connection both at the office and in my house and both can now actually be updated to 10Gb connections.
 
In the United States, I am not aware of ANYONE (not even the cable company) that has a 10GbE connection over a public ISP. Perhaps the US Military has this at some secret locations, but I don't know anyone that has a true 10G connection
via internet service providers. And I am talking about major public companies that can afford anything.
And I would be shocked to learn that services like Google Drive, Backblaze B2, Amazon S3 could even accept 10G speeds over the internet. Perhaps in your country. Not here.

Bob Zelin
 
Bob Zelin said:
In the United States, I am not aware of ANYONE (not even the cable company) that has a 10GbE connection over a public ISP. Perhaps the US Military has this at some secret locations, but I don't know anyone that has a true 10G connection
via internet service providers. And I am talking about major public companies that can afford anything.
And I would be shocked to learn that services like Google Drive, Backblaze B2, Amazon S3 could even accept 10G speeds over the internet. Perhaps in your country. Not here.

Bob Zelin
Click to expand...

Yes in comparisons with Sweden the US is a third world nation in this domain. I can get 10G for my cottage in the middle of the forest outside Stockholm for just a little more than my current 1G connection. Same at my office even though the homeuser contracts is less costly than the company ones.

https://www.bahnhof.se/press/press-...router-for-hemanvandare-snabbast-pa-marknaden

Banhof is pushing 10Gbit/s the most but Ownit and other providers are also delivering. From Banhofs page:

Three quick facts about Bahnhof 10 Gbit/s
> 10 Gbit/s = 10 000 Mbit/s
> 10 Gbit/s is available to those who have fiber installed in their house, those who have a dedicated fiber line to their apartment, and those who have a fiber connection through Skanova or Stokab.
> You will get the most out of your 10 Gbit/s service with the network cable Cat 6a.




Dont know why we got such high prestanda for internet around here, but no joke, lots of time its faster to move over the web than between two serves with 10GBe connections.
 
Björn Benckert said:
Yes in comparisons with Sweden the US is a third world nation in this domain. I can get 10G for my cottage in the middle of the forest outside Stockholm for just a little more than my current 1G connection. Same at my office even though the homeuser contracts is less costly than the company ones.

https://www.bahnhof.se/press/press-...router-for-hemanvandare-snabbast-pa-marknaden

Banhof is pushing 10Gbit/s the most but Ownit and other providers are also delivering. From Banhofs page:

Three quick facts about Bahnhof 10 Gbit/s
> 10 Gbit/s = 10 000 Mbit/s
> 10 Gbit/s is available to those who have fiber installed in their house, those who have a dedicated fiber line to their apartment, and those who have a fiber connection through Skanova or Stokab.
> You will get the most out of your 10 Gbit/s service with the network cable Cat 6a.




Dont know why we got such high prestanda for internet around here, but no joke, lots of time its faster to move over the web than between two serves with 10GBe connections.
Click to expand...

That is cheap, 500 sek/50€ a month.

So with a second NAS in you basement it could become something like this.

10.000 Mbit/s ~ € 50/month.
Energy is around € 0.20/kWh. (average power 100 Watt) > ~ € 15/month

QNAP TS-1635 (8GB ram)( Volume types: RAID 0/1/5/6/10/50/60, + hot spare, single, JBOD) ~ € 1,500 incl. vat
12x Seagate IronWolf Pro, 12TB ~ € 470 incl. vat > ~ € 5,640 in raid 6 + hot spare makes ~ 100 TB
UPS €200
additional cost ~ € 1.060
totals € 8.400 over 4 years (write-off) > € 175/month

total monthly cost €(50 + 15 + 175) ~ €240 when you can put it in your basement.

No cloud service provider can beat that 10 Gb/s and 100TB storage/backup
 
You must log in or register to reply here.
Back
Top