Backup and Tape device answers

[Tim Whitcomb]

(1) first off - for windah's set up there are tonnes of solutions out there - see your favourite I.T. storage company for those. I cant help you with that.

(2) - FOR standalone MACOSX archive solution.
Heres is the simple solution that works for Ultrium LTO4 SAS tape drive that works with a MAC PRO using MAC OSX based (hosted) archive applications on OSX 10.5.6 an also work on the brand new MAC PRO 2009 2.93 OCTO (16 x Virt/core jobbie - thats this one I'm using right now). No need to sweat it .. this works and is used by most:

• ATTOTECH EXPRESSAS HBAs PCIE - R380 or the H380 (external SFF-8088 multi-lane SAS connectors
• HP desktop external Ultrium LTO4 tape drives either the Full Height HP1840 of their Half Height 1760
• TolisGroups BRU-PE (Producers Edition) - MACOSX BASED archive software
• some Ultrium LTO4 data tapes (HP or SONY are great)
• an SFF-8088-to-SFF-8088 multilane sas cable (maybe you can get one with the tape drive).
  

This absolutely works for archiving creative/post work.

Thats as simple as it gets and works 100%.

I have exactly this configuration and BOTH of these tape drives and they work fine on my old 2007 MAC PRO 8core and now n this new monster 2009 2.93Ghx 8 x core Nehalen with the R380 EXPRESSSAS in PCIE slot 2 (x16).

THere is a great deal of inforation on these threads> PLease revew and draw your conclusion. Simply the support of other component vendors for MACOSX is NOT THERE. (i.e Quantum and LSILOGIC).

Looking in these threads will get opinions on other solutions that work and the differences in teh tape drives and maybe the software.

Skip the rest if you like...

I also have a QUANTUM HH SAS LTO4 ULtrium tape drive. I'm ready to dump in the local landfill. I Had to buy an LSI SAS3442E-R PCI SAS HBA card (windows support only) to get it to work on OSX. (Stupid me!).. so dont go this route.

See these links. More if you search other forums.

http://www.reduser.net/forum/showthread.php?t=21284
http://www.reduser.net/forum/showthread.php?t=24522
http://www.reduser.net/forum/showthread.php?t=17496
http://www.reduser.net/forum/showthread.php?t=7556 (very long and detailed thread)
http://www.reduser.net/forum/showthread.php?t=23545
http://www.dvxuser.com/V6/showthread.php?t=155693
http://www.dvxuser.com/V6/showthread.php?t=114778

NO need to sweat. Maybe you can see this solution at NAB somewhere.

if you are a non OSX platform user then there a lots more options for windahs's and linux using tried and true I.T. backup solutions for tape drives..

If you are on a budget and dont trust your archives to cheap spinning disks or scatchable optical disks then you might want look at the line of HP DAT tape drives. These work with USB and SATA(??) not sure of the later.. and you can run on other machines that dont need an HBA.

THere are SCSI an Fibre Channel versions of the tape drives requiring difference (obviously) HBA's..

I mention the above configuration because it is cost effective and works wonderfully.

LAstly there are also CLIENT SERVER verison of TOLISGROUPS BRU and also other I.T. based and expensive archive and backup solutions that supports mostly all OS platforms.. plenty form which to pick and choose.

HTH

w

Thanks, this is fantastic... much appreciated!

 

[warwickt]

Hello,
So no notebook eh... ok plan B

HP desktop external Ultrium LTO4 tape drivhttp://reduser.net/forum/images/style/editor/menupop.gifes either the Full Height HP1840 of their Half Height 1760

What is the difference between these two ? besides the size... :blink:

Does Bru-PE work on XP?

Can I run an LTO on my XP partition (bootcamp) on my macbook?

Lastly but not leastly, What is the best to hook up the above hp LTO4 to the PC - card and cable (and software if the bru-pe does not work on xp)

sorry for the questions - I am trying to set up something I can access on my PC and then later when I have a mac tower I can switch. (I am transitioning to mac - fed up with my pc and love my macbookpro - but meanwhile I have a lot of pc gear)

Hi Bryan, these are great questions and other on this forum will have more definitive answers with respect to windahs. I cant advise you much in the windah's side. For many, archiving on LTO ULtrium data TAPE drives you will need some SCSI, FC or SAS adapter, I dunno if such things exist for laptops via Express34 or ye olde PCMIA.

Anyway you will need some kind of tower for an HBA to service this.

So for windah's any old PCI-X (SCSI cards ok here?) or PCI-E for HBA's that support tape device pass through will work ok I suspect. Again consult the vendors of the windah's based tape backup software. There's many of these and google is your friend here Also Linux and Solaris too. SO it might pay you to get an elcheapo pc that takes PCI-X or PCI-E HBA's and go from there. Also its rumoured that ARECA might support tape device passthough.. we'll find out in a while. Just check will all the HBA vendors.

IT might also be worth to have a look for a discarded G5 POWERMAC and get into a SCSI LTO3 with a cheaper card.. but only if you are on a budget and really wanting to go totally to OSX like myself.Hmm... on second thought u might be wasting your money...

Another option (a bad one I think for me is the use of proprietary Quantum formatted ETHERNET LTO3A, DLT600A and their newer LTO4A .. NOTE THESE ARE NOT LTO ULTRIUM tape drives, THey have some workflow advantages in that you can connect to them over IP (IPoE), but aside from the fact that they are very very slow in comparison to the ULTRIUM lTO4 and the newer Ultrium LTO5s at the end of the year or 2010, you will be stuck with a proprietary format tape drive and proprietary interface over IP (LTO3A). MAny on this forum have had great success with the "A" type tape devices wi their proprietary encapsulation of objects under "MXF". I dont like them for my purposes.. as always do what is right or your business. Ayway then you CAN use them wi th a laptop.

I'd say with the new MACPRO 2009 announcements and the amazing results we are seeing now with RED workflows (RED ALERT) and transcoding to name a few so far, that there will be a load of older INTEL BASED clovertown MACPROs on the market soon as people see the real workflow benchmarks coming out. I just sold my 2007 MACPRO 8*core to a guy who is a switcher from windah's. (Shipped it out today). THese work really well for RED workflow and may also provide you with a good way to get into the OSX domain to use this tape archive workflow that is the subject of this thread, if you are on a budget.

Plan B for a laptop is that you might wanna look at those HP USB DAT tape drives.. these look cool and are portable in sense if u need that.

For windahs theres loads of I.T. based tape backup options from Symantecs to EMC and beyond... you just need tolook at the HBAs and the configs for all the data tape vemdors and you'll be set with many .apps as well.

On OSx like I posted earlier, you have no OSX support unless you go the ATTO and HP way. HP support the tape drives ON OSx. It is ruoured that HP will also have their firmware and tape drive performance tool implemented on OSX in a while. So THat means support on OSX.

ATTO already support OSX and so does Tolisgroup.

I believe Tim said in a previous post that Tolis dont provide a windah's based .app.. I'm sure he'll post soon on this.

Running in a partition on a laptop wont help you as there are no HBA's unless you can find one that fits in to a express34 slot I suppose.

Bootcamp from an internal DDM on a MACPRO works great if u want to do that under windahs for running your backuyp/restore I.T. sowftare and your tape drive. You have options then with Quantum , Tandberg, IBM, SUN and other HBAs such as LSI LOGIC..

ALso turning virtual wont help as the HOST OS doesn't see the HBAs .. at least on a MACPRO... so no good.

HP 1760 HH SAs LTO4 Ultriumtape drive and the HP 1840 Full Height LTO4 Ultrium tape drives:

• I have bothe these here above me on a shelf and connected to an an HBA as I need on the new MAC PRO 2009
• same speeds & feeds (performance)
• same content capacities and features
• different prices
• the Full Height HP 1840 is a better manufacture (??) as like cameras, bigger usually has less compromises than the smaller compacts devices... .
• both are very good and will work fine on OSX with ATTO and Tolisgroups BRU-PE
• one is more expensive than the other.- search out (www.digitape.com)
• my guess is that the HP 1840 Full height is much more industrial strength... but that is a pure guess

hth

w

 

[sublimefilms]

So no notebook eh... ok plan B

There are not SAS expresscard hbas, you could maybe put the atto card in a Magama ExpressBox - not sure if it would work in a 1x slot. But for the cost, minds well use a pc and backup over the network.

Lastly but not leastly, What is the best to hook up the above hp LTO4 to the PC - card and cable (and software if the bru-pe does not work on xp)

You can use the same Atto H380 under windows, and move it to the mac when you want to. I suggest NovaBackup or BackupExec as windows backup apps.
For best transfer rates, run server2008. There were issues with vista, but they might have been resolved with update by bow- we switcherd to server08.

 

[sublimefilms]

Atto vs. LSI

Atto vs. LSI

We have both LSI and atto sas controllers, the atto is the better choice. The LSI will work out of the box, no need to flash it on windows, but it has a primitive efi boot rom which causes issues when trying to use bootcamp on the same macpro, you have to use REFIT and its still a pain, and it only supports 1M blocks on the mac side (crummy apple written driver). The Atto H380 is cheap and works. The H680 (6gb) has no advantage, you can already run 8 lto drives at max speed easily off one H380, drive = 1Gbs, card has 8 3Gbs ports.

Use a better raid card for your disk storage.


Plan on having bootcamp and xp or server2003/8 installed on your backup machine if you want to upgrade the firmware of the LTO drive, no matter what brand.

 

[Tim Jones]

Hello,
So no notebook eh... ok plan B

HP desktop external Ultrium LTO4 tape drives either the Full Height HP1840 of their Half Height 1760

What is the difference between these two ? besides the size... :blink:

That's it.

Does Bru-PE work on XP?

Not at this time.

Can I run an LTO on my XP partition (bootcamp) on my macbook?

No - no connectivity.

Lastly but not leastly, What is the best to hook up the above hp LTO4 to the PC - card and cable (and software if the bru-pe does not work on xp)

Buy the HP LTO-4 / HBA bundle (includes the HP SAS HBA) and add the Backup Exec software. I know others have mentioned NovaBack as a very low-cost option, but I'm not familiar enough to recommend it.

sorry for the questions - I am trying to set up something I can access on my PC and then later when I have a mac tower I can switch. (I am transitioning to mac - fed up with my pc and love my macbookpro - but meanwhile I have a lot of pc gear)

All good questions and for the right reasons. By using the SAS sollution, you simply add the ATTO Tech SAS HBA to the Mac when you get it and you can swap the drive between the systems. SAS is hot-plug capable. For now, there's no crossplatform software option, but we are working towards both Linux and Windows versions of PE for Summer time release.

Tim

 

[Tim Jones]

This situation is changing

This situation is changing

Plan on having bootcamp and xp or server2003/8 installed on your backup machine if you want to upgrade the firmware of the LTO drive, no matter what brand.

That is changing with the HP drives. They are working on a Mac version of the L&TT tool - I'll probably get yelled at for mentioning that, but I wanted to reaffirm HP's dedication to the Mac community.

Tim

 

[Tim Jones]

If you are on a budget and don't trust your archives to cheap spinning disks or scratchable optical disks then you might want look at the line of HP DAT tape drives. These work with USB and SATA(??) not sure of the later.. and you can run on other machines that don't need an HBA.

Actually, HP's DAT offerings are available in USB and SCSI with the DAT160 also available in SAS. And, the USB version works great with BRU PE under 10.3.9 through that "new" version on everything from an old PPC iBook and Mac Mini to the new Nehalem-based Mac Pros. The only limitation with the DAT drives is the performance - 2.5MB/sec (9GB/hr) to 6MB/sec (21.6GB/hr) throughput (still faster than CD or DVD) - regardless of the connection technology.

Lastly there are also CLIENT SERVER version of TOLIS Group's BRU and also other I.T. based and expensive archive and backup solutions that supports mostly all OS platforms.. plenty form which to pick and choose.

And, BRU Server tapes are compatible with Producer's Edition and vice-versa. BRU Server can be run on Linux or LS X as the server and support every OS platform out there with the exception of older non-Linux Netware and Tru64 Unix (both very old platforms).

Tim

 

[Tim Jones]

There are not SAS expresscard hbas, you could maybe put the atto card in a Magama ExpressBox - not sure if it would work in a 1x slot.

Actually, the ATTO and LSI cards both need a minimum of a 4 lane slot (x4).

Tim

 

[sublimefilms]

Updated BRU Speedtests

Updated BRU Speedtests

An update to our previous speed test results in the other thread.

At first glance I did not realize that BRU LE and BRU PE are just RealBasic front ends to the same BRU application. LE and PE just put a different face on the same command line engine. Similar to how r3d-datamanager is frontend to md5 and redline.

There should not be any performance differences at all.

Seems that having BRU server demo previously installed caused some issue, installed LE and PE demos on a clean system and saw better results.

The short version: LE and PE are on par with windows apps, PE has no speed advantage over LE

NovaBackup (Server2008/bootcamp) - 2M block size (LTO4 max) = 115 MB/sec
BRU PE or LE (patched) (10.5.6) - Variable block size = 111 MB/sec
BackupExec (Server2008/bootcamp) - 2M block size (LTO4 max) = 110 MB/sec

Long version:

**BRU does not honor the block setting in the brutab, its always in variable block mode, blocksize=0. BAD!

** PE writes the tape catalog ALWAYS with a 64k buffer size (-b 64k), again ignoring the buffsize set for the tape in brutab. It seems like a VERY bad idea to write the catalog at one bufsize, when the archive on the tape is a different bufsize.

** The abufsize in brutab is wrongly set to 256k by default

** LE has a bug where the buffersize is hardcoded into the application, limiting you to 1M buffersize. So until they fix this bug you can just patch out the -b command line argument being passed to BRU in the BRU LE executable. This will stop the hard coding of the buffer size and let BRU use the setting you specify in the brutab settings file, like it is suppose to do.
(you also have to patch where its writes the catalog file as well)
...but I saw no improvement going from 1M to 2M buffersize from local copies, maybe slightly with 10Gbe network shares.

** Neither see network shares directly, just drag and drop files from shares on the desktop.

Here is good brutab setting for lto4 and large filesizes:

ntape0 devname=LTO4a \
size=0 \
bufsize=2M \
asbufsize=2M \
maxbufsize=2M \
minbufsize=1M \
blocksize=2M \

You can verify the brutab settings in terminal by:
bru -hh

If you change settings in LE/PE the brutab will get reset and will need to re-done, keep a backup copy.

With these settings we get:

111MB/sec (126.3MB/sec!!) 40% CPU on Write, 10% CPU on Verify

The SAME in LE and PE.

BRU is writing at the LTO4 max of 125MB/sec which is great, but because of the file size bloat of their format you get throughput of only 111MB/sec.

CPU use of 40% (out of 800%) is worse than I originally thought, but is adequate for archiving and doing a redline render at the same time (on a 8core). BRU LE/PE will show very little load, you need to look at BRU under All processes.

Encryption: Looks possible to do software encryption now by using the external software compressor setting in the brutab:
#+BRUZEXTERN=”/usr/bin/bzip2"

replace bzip2 with script that pipes stdio to your favorite encryption command line.

But its messy. And software encryption is not the way to go, in a film asset environment the main threat is opportunity, vault assistant or courier who thinks its cool to post a prerelease on rapidshare. I prefer to have it tied to hardware encryption built into the drive, not all drives support the encryption so less likely a person is able to have easy access. Plus with CPU at 40% already, adding software encryption will have too much of a performance impact. LTO spec is only 2 generations of read compatibility, so thinking you will be reading lto4 tapes beyond lto6 drives is wishful thinking - data assets need to get migrated to current formats, just as video assets do. I still vote for adding hardware encryption support to BRU, most win backup apps support it.

Retrospect8 beta currently is fixed buffer and block size so its too slow to be usable. We have provided them feedback, if they allowed 2M block size its probable it would be faster than BRU since their format does not increase data size 15%.

Another thing to look at is that Amanda now compiles properly under leopard. Amanda is a very flexible open source client/server backup app. It looks like it could be possible to add tape support to amanda using tolis tapetools. Amanda supports RAIT, so with 3 lto drives you could use tar and stripe parity information, gain double the speed, and raid-5 redundancy on tape. Tapetools does not mount a standard /dev tape device, and its taperead/write commands are very basic so it would require some work.


If you dont need or like the "session" backups, LE is a good, much cheaper alternative and you dont loose any speed with the settings manually configured.

 

[sublimefilms]

Transfer Times

Transfer Times

At 111MB/sec, thats about a 800GB written in 2 hours, add another 2hrs for verify, plus a few minutes for tape rewind - BEST CASE - Local Raid

Your source is your limitation, single modern sata drive will start off fine, but when full its going to trail off to maybe 60MB/sec - so maybe your backup write is now 3 hours, verify is the same 2hrs unless you do a full verify.

100GB of data on a single local esata drive - about 50 minutes for write/rewind/verify/rewind.

Firewire800 drive worse, max is 80MB/sec read.

10gbe share is 180MB/sec so no performance loss there. but most people will only have 1Gbe network shares, SMB maxs out at about 80MB/sec with jumbo frames.

 

[Jeff Kilgroe]

I'm still running the Quantum LTO-4 w/ LSI controller. It works just fine as long as you stick within the limitations of the crappy Apple driver (1M blocks, as warwickt says). I still like this drive, it's fast and available at a good price. I have had no troubles with mine. It's best to buy the kit that includes the drive, controller and cable. For some reason the Quantum drive does not play nice with other SAS controllers on the Mac, supposedly the issue is being worked on, but it's taking forever.

That said, if I were to do it over again, I would buy the HP 1760 or 18xx unit and use an ATTO controller. I will probably be doing this soon anyway as I'm looking to add some more external SAS connectivity and RAID support. I will probably be switching over to the HP 1760 desktop LTO-4 along with the ATTO R680 RAID controller. ...Probably won't do this until I buy another Mac Pro, maybe later this summer.

 

[sublimefilms]

Laptop SAS

Laptop SAS

Actually, the ATTO and LSI cards both need a minimum of a 4 lane slot (x4).

For proper throughput on 8 ports, but atto tech said they had not tried it in a 1x slot to see if it would function at all. You can use a 16x graphics card in the slot, 1 lane is more than enough for a 125MB/s lto transfer. Just a matter if the H380 is downward compatible or not.

Would be useful for field backups. Dont have a expressbox here to test.

 

[Tim Jones]

Warning - long (winded) response below

Warning - long (winded) response below

Before I respond to the OP's statements, let me reiterate that I've tried to keep this thread from turning into a thread about BRU, but I feel that I must respond to the statements that have been made. Sorry if this usurps the intention of the thread.

Tim

**BRU does not honor the block setting in the brutab, its always in variable block mode, blocksize=0. BAD!

Sorry, but that's GOOD. We set the tape drive into variable block mode so that it can make the best decision on the data being written. The blocksize you set in the application relates to the BRU I/O blocksize. Additionally, there is no "blocksize" value in the brutab.

** PE writes the tape catalog ALWAYS with a 64k blocksize (-b 64k), again ignoring the buffsize set for the tape in brutab. It seems like a VERY bad idea to write the catalog at one bufsize, when the archive on the tape is a different bufsize.

Again - not bad. By always writing the catalog at 64K, we can determine the blocksize used for the catalog's archive from the catalog. Since we know we can restore the catalog at 64K, the actual archive's blocksize can be determined in the event the blocksize was changed. This was done to better support media interchange between editor/producer and client. This means that the client doesn't need to worry about the tape's buffer size to restore. The catalog has the necessary data included on the tape.

** The abufsize in brutab is wrongly set to 256k by default

This has no bearing on PE. asbufsize is not even used under OS X.

** LE has a bug where the buffersize is hardcoded into the application, limiting you to 1M buffersize. So until they fix this bug...

It's neither a bug nor recommended. The way the BRU LE GUI interacts with the BRU engine is a design decision and should not be modified (hacked?).

Also, keep in mind that hacking, modifying, or reverse engineering any TOLIS Group binary components is a violation of the TOLIS Group License Agreement and the DMCA.

** Neither see network shares directly, just drag and drop files from shares on the desktop.

Because both are Desktop applications. If you need a network and client / server app, you should be looking at BRU Server.

Here is good brutab setting for lto4 and large filesizes:

ntape0 devname=LTO4a \
size=0 \
bufsize=2M \
asbufsize=2M \
maxbufsize=2M \
minbufsize=1M \
blocksize=2M \

The as, max and min values have no meaning on the Mac platform -they're not even used. There is no brutab setting for "blocksize", so that one is simply ignored.

You can verify the brutab settings in terminal by:
bru -hh

If you change settings in LE/PE the brutab will get reset and will need to re-done, keep a backup copy.

With these settings we get:

111MB/sec (126.3MB/sec!!) 40% CPU on Write, 10% CPU on Verify

The SAME in LE and PE.

Those settings will be overwritten by the GUIs the next time you modify your preferences. However, you can also run BRU from the command line and gain access to BRU's full suite of capabilities. The GUIs are provided for the 95th percentile of users who don't want to muck around in the application's internals and running from the CLI. For more information on the huge range of BRU settings, please see the BRU CLI manual at:

http://www.tolisgroup.com/docs/pdf/BRU_17.0_Manual.pdf​

BRU is writing at the LTO4 max of 125MB/sec which is great, but because of the file size bloat of their format you get throughput of only 111MB/sec.

That "bloat" is what makes BRU "BRU". As I mentioned in a previous posting, we are not recommending that you use something that's not right for your environment. If the differences in BRU's methodologies are not acceptable to you, please stay with what you're currently using. We'd rather you be satisfied with your existing solution than feel animosity towards us because you switched to a BRU solution. That's why we offer a 30 day demo on all of our products.

CPU use of 40% (out of 800%) is worse than I originally thought, but is adequate for archiving and doing a redline render at the same time (on a 8core). BRU LE/PE will show very little load, you need to look at BRU under All processes.

To see this in a Terminal, use:

ps -ax | grep -i bru

And even those numbers will vary during an archive run depending on what BRU is doing at a given sample moment.

Encryption: Looks possible to do software encryption now by using the external software compressor setting in the brutab:
#+BRUZEXTERN=”/usr/bin/bzip2"

replace bzip2 with script that pipes stdio to your favorite encryption command line.

Yes - you can do all sorts of things with the full BRU environment. As I mentioned above, the GUIs provide a subset of BRU functionality that is aimed at the 95th percentile user.

But its messy. And software encryption is not the way to go, in a film asset environment the main threat is opportunity, vault assistant or courier who thinks its cool to post a prerelease on rapidshare. I prefer to have it tied to hardware encryption built into the drive, not all drives support the encryption so less likely a person is able to have easy access. Plus with CPU at 40% already, adding software encryption will have too much of a performance impact.

Our upcoming encryption mechanism uses far less CPU than a externally shelled encryption engine. In fact, with encryption enabled we sustain 550MB/sec throughput on a G4 PPC at an additional 4% CPU. This low overhead is because our encryption algorithm is assembly code for both Intel and PPC platforms.

LTO spec is only 2 generations of read compatibility, so thinking you will be reading lto4 tapes beyond lto6 drives is wishful thinking

Actually, read is 3 generations - the current and its 2 predecessors. It's tape WRITE compatibility that's 2 generations. We can still read our LTO-2 test tapes on our LTO-4 drives and LTO-1 drives are still available.

From the LTrg FAQ:

28: What are the backward compatibility characteristics of the Ultrium format?

The LTO Ultrium compatibility is defined with two concepts demonstrating investment protection:
1) An Ultrium drive can read data from a cartridge in its own generation and two prior generations.
2) An Ultrium drive can write data to a cartridge in its own generation and to a cartridge from the immediate prior generation in the prior generation format.​

data assets need to get migrated to current formats, just as video assets do.

I agree with that totally, but not for the reason you mention above. Simply that it's smart to keep your assets refreshed. The tapecopy tool (part of the TTT suite) will happily perform a direct copy from any tape to any other tape - say AIT-1 to LTO4, so you don't even need to perform a restore.

I still vote for adding hardware encryption support to BRU, most win backup apps support it.

The issue with hardware encryption is that it is only available on an LTO-4 drive. With our upcoming software encryption, you get encrypted archives regardless of the tape or disk technology used.

Another thing to look at is that Amanda now compiles properly under leopard. Amanda is a very flexible open source client/server backup app. It looks like it could be possible to add tape support to amanda using tolis tapetools. Amanda supports RAIT, so with 3 lto drives you could use tar and stripe parity information, gain double the speed, and raid-5 redundancy on tape. Tapetools does not mount a standard /dev tape device, and its taperead/write commands are very basic so it would require some work.

We'd love to hear from anyone accomplishing this. As far as RAIT is concerned, what happens if you don't have 3 tape drives when you need to restore? And real-world tests (versus vendor claims) have shown that the number is more like 1.2x the speed of a standalone drive at 3x the cost.

If you dont need or like the "session" backups, LE is a good, much cheaper alternative and you dont loose any speed with the settings manually configured.

Almost correct, but you left out the loss of advanced library support, Doubler Archival Mode, and Separate Session Archival mode - all part of BRU Producer's Edition.

Tim

 

[sublimefilms]

that's GOOD. We set the tape drive into variable block mode

Ok, Mostly Bad. Variable block size decrease performance as the drive must choose a block size for each file, and having a tape full of varying block sizes increases the chance of problems on restore. The only benefit is that really small files take up less space on tape. How many audio/video files are less than 2megs? Who cares, tape is cheap.

Again - not bad. By always writing the catalog at 64K

Still seems bad. LE, server, and the command line do not follow this behavior, they are suppose to be compatible.

The way the BRU LE GUI interacts with the BRU engine is a design decision and should not be modified (hacked?).

Contacted tech support after initial slow performance, they said to increase the buffers in terminal, they would not be displayed in the gui but would be honored. When that did nothing, they said it was a bug. I was just proposing a temporary solution to what I was told was a bug. Seems a little silly for the application to ignore its own settings file.

asbufsize is not even used under OS X.
The as, max and min values have no meaning on the Mac platform -they're not even used. There is no brutab setting for "blocksize", so that one is simply ignored.

Since the settings are in the manual for the mac version, and seeing that bru (on a mac) recognizes the changed settings - including blocksize, you could see where someone might think they were doing what they were suppose to do.


ntape0 devname=LTO4a \
size=0 \ bufsize=1M \ asbufsize=1M \
maxbufsize=1M \ minbufsize=1M \
blocksize=1M \

bru -hh

**** brutab parameters for device: ntape0 ****
...
bufsize=1048576
asbufsize=1048576
maxbufsize=1048576
minbufsize=1048576
blocksize=1048576


ntape0 devname=LTO4a \
size=0 \ bufsize=2M \ asbufsize=2M \
maxbufsize=2M \ minbufsize=2M \
blocksize=2M \

bru -hh

**** brutab parameters for device: ntape0 ****
...
bufsize=2097152
asbufsize=2097152
maxbufsize=2097152
minbufsize=2097152
blocksize=2097152



Not attacking bru, just pointing out how it works and does not work.

When I set a SR deck to record 1080/444/23.976/440mbs I know thats exactly what it is recording. When I backup to a LTO tape, I'd like that same level of confidence. It has nothing to do with wanting to "muck" with the internals, its about protecting your assets.

 

[Tim Jones]

Ok, Mostly Bad. Variable block size decrease performance as the drive must choose a block size for each file, and having a tape full of varying block sizes increases the chance of problems on restore. The only benefit is that really small files take up less space on tape. How many audio/video files are less than 2megs? Who cares, tape is cheap.

Variable Block mode is faster than fixed block because the drive doesn't need to chop the incoming stream into appropriately sized pieces. If you ask any tape drive manufacturer, they'll tell you the same thing - use variable block mode when you can for the best performance. As far as the perceived increased problem potential - variable mode is just as stable as fixed and has been part of tape technology since 1988. Also, because we're now creating catalog entries at the static 64K BRU bufsize, we always know how the archive was created.

Still seems bad. LE, server, and the command line do not follow this behavior, they are suppose to be compatible.

In truth, LE and Server also use variable block mode and have from their very first versions. The only time we use a fixed block setting is with AIT1, VXA1 and VXA2 drives - simply because they don't support variable block mode. And then, we use a 64K fixed block on the drive and limit BRU bufsize setting to a multiple of 64K. As for using the CLI, we leave the preference settings completely up to the user to decide. A user at that level of technical experience will also understand what the settings do and why they should be defined a certain way. Again, our GUI-controlled versions are aimed at the major portion of users who don't want to have to deal with the internals and manual configuration options.

This is also why we've moved to the 64K BRU bufsize for catalogs under PE and will be transitioning LE and Server to that as a standard in their next major updates.

Contacted tech support after initial slow performance, they said to increase the buffers in terminal, they would not be displayed in the gui but would be honored. When that did nothing, they said it was a bug. I was just proposing a temporary solution to what I was told was a bug. Seems a little silly for the application to ignore its own settings file.

My apologies for implying something untoward may have occurred. I've since had a meeting with the support team and gotten their input on this. Unfortunately, it was a case of whisper-down-the-lane where one tech overheard and engineer talking about something else, and then sharing it with another tech that misinterpreted what was originally said. As I mentioned, it's not a bug, but rather a design decision to differentiate LE from the other versions.

As for the manual - there are two aspects to BRU and we don't try to hide them, but they're not completely compatible and I've spoken to the tech writer about updating the manuals to make that more clear.

Not attacking bru, just pointing out how it works and does not work.

When I set a SR deck to record 1080/444/23.976/440mbs I know thats exactly what it is recording. When I backup to a LTO tape, I'd like that same level of confidence. It has nothing to do with wanting to "muck" with the internals, its about protecting your assets.

Understood. Hopefully, the above further defines what you should expect and why we're doing things in this manner and further clarifies that variable block mode is quite safe in protecting your assets.

Tim

 

[Helmut Kobler]

What a great thread, this has been immensely helpful, so thanks to all parties that have contributed!

I have a question about the ATTO R380, any guidance much appreciated. I have a 2009 Mac Pro and I'm wondering if it's possible to use the ATTO in Raid 5 with a 4 drive enclosure (using only one of the ATTO's two ports for the RAID), and then use the second ATTO port to connect to an HP LTO4 drive.

Even better, I'm wondering if it's possible to get a magic splitter cable that lets me plug not only the LTO4 drive into the ATTO, but also an additional eSATA external hard drive (I would get one of those trayless hard drive docks where you can conveniently slip in a raw drive and it mounts immediately).

In other words, I'm trying to use one RAID card (the r380) to handle a 4 drive external raid, an LTO drive, and even a hot-swappable esata drive.

Any help appreciated, thanks much!

 

[Tim Jones]

Yes, yes, and Maybe

Yes, yes, and Maybe

What a great thread, this has been immensely helpful, so thanks to all parties that have contributed!

I have a question about the ATTO R380, any guidance much appreciated. I have a 2009 Mac Pro and I'm wondering if it's possible to use the ATTO in Raid 5 with a 4 drive enclosure (using only one of the ATTO's two ports for the RAID), and then use the second ATTO port to connect to an HP LTO4 drive.

Even better, I'm wondering if it's possible to get a magic splitter cable that lets me plug not only the LTO4 drive into the ATTO, but also an additional eSATA external hard drive (I would get one of those trayless hard drive docks where you can conveniently slip in a raw drive and it mounts immediately).

In other words, I'm trying to use one RAID card (the r380) to handle a 4 drive external raid, an LTO drive, and even a hot-swappable esata drive.

Any help appreciated, thanks much!

When working with that card, ATTO gives you 8 ports through the 2 SFF 8088 ports. And, with SAS, unlike parallel SCSI, each port is a separate channel and can readily support exactly what you're asking - technically.

However, you will need to locate an appropriate cable or breakout box (which can get expensive quickly). I don't know of anyone that makes an 8088 to 8088 AND eSATA cable.

If you do find a solution there, please share it with us.

Tim

 

[Helmut Kobler]

Thanks, Tim. I did a search, but came up with no trace of a 8088 eSATA combo cable. I might just have to go with an 8088 to 8088 cable, and an 8088 to eSata cable, and just unplug and plug them in on demand.

Thanks again....

 

[Shane Betts]

Sorry Tim, another Bru related question

I'd like to set up schedules on the server edition (Mac) to backup to two different sets, on alternative weeks. In other words, Set 1 will backup every night on week one, incrementally based on a once only full backup. That set will be then taken off-site and Set 2 installed into the library, again backing up incrementally based on a once only full backup.

But I can only find a way to backup every day, or every week, or every other day etc. How do I backup every day for a week, every second week without causing too much duplicate data to be written to any given set?

Thanks again for your help.

 

[Tim Jones]

Sorry Tim, another Bru related question

I'd like to set up schedules on the server edition (Mac) to backup to two different sets, on alternative weeks.
[snip]
How do I backup every day for a week, every second week without causing too much duplicate data to be written to any given set?

Thanks again for your help.

You would need to create multiple jobs and alternate the schedules. Check out the section in the Server manual on Example Scenarios. I believe there are some ideas that that will assist.

Tim