Backup and Tape device answers

[Matt Pope]

Matt,

• First and foremost, BRU will restore your data. However, the UI's we provide do depend on the catalog and tape information stored within the UI's operating environment - whether BRU PE, BRU LE, or BRU Server. In your case, it appears that elements of your catalog and tape information have become corrupt. Because the UI's view into your catalogs has become confused does not affect the validity of the data stored on your tapes, and BRU can always restore the data from those tapes.

• Second, the REDuser forums should not be used as a replacement for TOLIS Group Technical support and I don't want posts like this causing site management to disable my posting ability.

To restore your data, the first step that I would recommend is that you update your BRU PE installation to BRU PE 2.3.1. It's a free update for all registered BRU PE users - regardless of support status. Within the update, you will find the BRU PE Import Tool application that will allow you to import the catalogs off of your tapes very easily. Once you've imported these tapes, you should be able to restore your data as expected from within the UI. If, however, something is very broken for some reason, all that you need to do is contact our support team and they will help you restore the data.

I'll repeat - regardless of the status of your BRU PE UI's catalog environment, the data on your BRU tapes is intact and can be restored.

Thanks Tim, I understand about it not replacing tech support, but since they closed my ticket I wasn't sure where else to go for help. I did as you said and re-imported the tape using the Import Tool, but unfortunately it didn't change anything. Which is really odd since it seemed to import fine, and even showed the volume number and offset number just fine in the Import box when it concluded. But it still shows blanks for all of those when I try to restore from that newly imported catalog, and gets confused when I try to restore anyway. (Message says "Please insert Volume 0 of the tape labeled ""). When I enter the tape it just spits it back out and says "Incorrect Volume loaded. Please insert volume 0 of the tape set labeled "". It's archive ID is ."

I'll try to open a new ticket with support. Hopefully that'll go somewhere this time. Thanks!

 

[Tim Jones]

BRU Discussions

BRU Discussions

Tim, as a almost customer of BRU I think it is not your choice to tell us where to post and where not.
REDuser is a platform to post experience and personal opinions.

Hi Michael,

I'm not trying to dictate where or how we communicate, just trying to insure that we don't overrun an agnostic platform forum with product-specific threads. I'm most pleased to discuss BRU products here with all of you. However, maybe we should petition the REDuser management for a generic Archive / Backup SIG or even a BRU-specific topic area where we can discuss things like this without causing unwanted traffic for other users.

For me it has a lot of value to here the experience of Matt and it helps me to avoid problems by not having a support contract.

And we whole-heartedly agree that discussions are important and I thoroughly enjoy the discussions that occur here and on other forums. However, our annual support contract is only US$199 and includes unlimited support contact and all product upgrades during the contract period. Heck, our team members will even SSH or VNC into your system to assist if you so desire.

I am going to wire you the money for BRU this week.

We look forward to having you as a member of our family and working with you in your archival and deliverables needs.

 

[Tim Jones]

Matt - is this with 2.3.1? The guys will probably ask you for your environment so that we can see what's gotten crossed up. They'll fix it and get it back to you.

 

[Matt Pope]

Matt - is this with 2.3.1? The guys will probably ask you for your environment so that we can see what's gotten crossed up. They'll fix it and get it back to you.

Yep, 2.3.1. I've had the problem since the previous version but updated as soon as I got the email with the new release hoping it would clear things up.

 

[Tim Jones]

Yep, 2.3.1. I've had the problem since the previous version but updated as soon as I got the email with the new release hoping it would clear things up.

Hi Matt,

Open a new ticket and attach your environment by zipping up the /Library/Application Support/BRU PE folder. It's simplest to use Finder with Cmd-Shift-G and enter /Library/Application Support.

Once there, select the BRU PE folder and select "Compress BRU PE" from the popup menu. Attach that Zip file to the ticket and the guys will get it to me so that Andy and I can see what's happened on your system.

 

[Matt Pope]

Thanks Tim, done!

 

[Tim Jones]

It was a bug.

It was a bug.

Hi Matt,

It was an oddly quoted path. What you should have seen would have looked something like this (from the fixed version):

This is already fixed and I've got the build team working on a 2.3.2 update.

It is always a surprise :scared: as to the number of ways that Apple allows users to use special characters in file / path names :crazy:. Just when we think we have them all covered ...

 

[Matt Pope]

Good catch! And glad to be of service! Thanks for the assistance - that one had been driving me crazy for a few weeks. I appreciate the prompt reply and help!

 

[Michael Lindsay]

Hi Tim

We bought Bru PE a month or 2 ago. It was to replace retrospect (don't ask). However it was very slow and we just moved to LTFS which is blistering (exact same situaton). As a consequence of this I ditched Bru.

Is this new verson any faster? I love the idea of having suport but can't spend alot more time fiddling or spending more money as LTFS (fingers crossed) seems fine.

thanks

Michael

PS spec

Mac pro with H680 to a Rackmounted HP Storageworks LTO5 drive
We work from a SAN with a 10gig infrastructure easily maintaing 500MB+ sec

LTFS happliy backs up between 105-145MB/s we can do whole tapes in less than 4 hours
with Bru PU we were seeing about 10-15MB/s (Tollis suport where polite but basically said it was becuase we where using a network)

 

[Matt Pope]

Wanted to chime back in and say that Tim and the fine folks with Bru got me fixed up and running again in a flash. Found the bug that my problem was caused by and sent over a new .dmg that addressed the problem. Thanks for the fine support Tim!

 

[Ces Peynetti]

Hi Marcus and everyone - we're still tapping our toes impatiently waiting for the release of Thunderbolt connectivity products. It looks as if both ATTO Technology and Sonnet Tech are working on solutions for the new year, but we've still not seen the promised engineering samples. As soon as we get more concrete information on a release date and product capabilities, I'll be starting a new thread with product information.

I've always been too stubborn to do the Hard Drive route and not Tape for long term storage. Thankfully I've never run into any issues with HD failure, but I do fear it. This thread has been highly informative, so thank you Tim and everyone for your input.

At the moment, I don't have enormous amounts of data to store, which is one of the reasons I've never made the jump to tape, but starting this year I'll start running into a bigger need of storage, probably having to rethink my storage solutions by next year.

Now, the question I have is, do you think it's worth waiting a year to see how technology improves this year (such as the possibility of thunderbolt connectivity products) or should I look into investing on something right away. My budget is very limited so I believe if I invest on something now, it will take me a while to be able to upgrade it anytime soon. Thanks!

 

[zak forrest]

Are there any tape drives that hold more data than LTO3A and can also read my old LTO3A tapes ? That would be sweet if it existed.

If that is not an option, whats the least expensive (without being rubbish) tape backup desktop model drive I should get today that isn't out of date like the LTO3A

My quantum LTO3A drive was in the neighborhood of 7k, would love to not have to spend nearly that much on a new tape drive if I go that route again

 

[Tim Jones]

LTO-3A - not really an LTO tape drive...

LTO-3A - not really an LTO tape drive...

Are there any tape drives that hold more data than LTO3A and can also read my old LTO3A tapes?

Hi Zak,

Unfortunately, the -A series drives from Quantum (they also made a DLT and an LTO-4), use a proprietary format on the tapes and only another A-series LTO drive can read them. You might ask the Cache-A folks (that's where the "A" in Cache-A comes from) if their new units will read the old LTO-3A tapes since they're from the same base design.

Tim

 

[Björn Benckert]

My five cents on this matter.
Do not buy storage before you need it, exception is large nature catastropic events, that like in japan actually brought prices up. But that is the exception to the rule. Back in the days I remember buying 4 terabytes of fiber storage for about 800.000USD. That was 12 years ago... we named the array "Mother" as we figured that we would never be out of discspace ever again... then half a year later when "Mother" was filled the same amount of disc space cost about half the price... today it's down to fraction. Still buying high quality disc arrays have given me a lot less trouble than tapes.

We had DLT, DLT II and a few others. DLT was supposed to be the super safe solution.... well when more than half of our archives did not want to open.... we figured we had to buy into the bigger DLT II deck... that worked for quite a while, even though the tapes was super expansive compared to FW drives. Untaring the files took forever and just browsing the tapes was a nightmare. Then some older tapes stopped working, and then the deck itself stopped working.... ended up selling the tapes, they where quite a few.. 150 or so, on ebay for no money...

Right now we rely on Firewire or usb "no power supply" kind of discs sized to put to 1 tb. That have worked far better than any tape solution.

The worst investment I did was to get a Wooddoo D6 deck. uncompressed HD 8 bit. A single tape holding 120 minutes cost about 850USD. The deck itself cost about 100.000USD. It did not become a standard anywhere, people bought into the HDCAM deck's instead... so I sold it after having it around for about 5 years to some guy in Australia... I can tell you this much, he paid more for shipping than the actual deck... I could have bought a ferrari or something for that money...

So from the above I learned that tape decks does not get you any girls, and Hard drives is to me the only way to go and do not buy to many at a time since the prices and standards changes the whole time.

 

[Tom Goldberg]

Björn, I have to respectfully disagree.

Judging today's LTO tape drives based on early DLT experiences is like damning automobiles because a buggy whip was more reliable than a Model T (ok slight exaggeration, but you get the idea). With LTO being in its 5th generation and 3 more planned, being a consortium of IBM, HP and Quantum and with tapes available from Fuji, Maxell, TDK and Sony this is not the one source dead end that DLT was. They are way more reliable, supported, faster, cheaper, and better in every way.

And if you are relying on shelf-stored static hard disk drives, the odds are that in the not too distant future, one of these will not spin up. Not to say that you couldn't still recover that data, but when you look at the cost for a drive rescue, you will wish you'd used tape. This is not just my opinion reinforced by dozens of customers who've related horror stories - we commissioned an independent engineering firm to study these risks and publish a white paper that puts some science behind this claim - get it here

Just because you've been bit by early solutions that had no future is not enough reason to recommend that users put their long-term content on devices designed to work for the short-mid term.


BTW, Tim - as I've posted elsewhere on these forums: We had intended to read LTO-3A tapes, paid Quantum for the rights to do so, but found it would be way too much effort to implement. Our base design has nothing to do with them - we use only open standard tar and LTFS.

 

[Björn Benckert]

You might be right But I actually does not have that long term need for storing files. We do commercial work and it's not valid for more than a few years, during those years the material is spread over quite a few disks, TV stations etc. so when a disk pop I usually only have to call somebody to get a copy. We let clients handle their own source tapes so we only save finals and clean versions and never with any liability's attached. So for me it will be disc also in the future. I had about a few hundred broken tapes and not more than 20-30 bad disks during my 15 years in post...

 

[Bostjan Kravcar]

Ultrium 230 Tape Drive - Too fast for my PC ?

Ultrium 230 Tape Drive - Too fast for my PC ?

Greetings Tim and Red User members,

A first-time visitor on Red User forum, and I have this feeling, that I won't be all that popular after posting this writing. First off all, I sincerely apologize for the sheer length of this post. On top of that, my topic surely is one of the least popular (to put it mildly), on the forums of no-nonsense communities of real experts and IT professionals. Third, and that's probably my ultimate sin ... I have no association whatsoever with core topics discussed on Red User forums. A complete stranger, a black sheep I may say, desparately searching for a drop of water, a drip of wisdom in a wast desert, engulfed by a thick mirage of illusive hi-tech knowledge. I’m gonna fly into this storm anyway, and see if I can make it through the clouds …

Background:

For quite some time now, I've been contemplating the purchase of a mid-range standalone tape drive for personal use, mainly for long-term archiving of my photo slide scans, and additionally for safe-keeping most valuable graphic design work for future use. First of all, I should mention that I already use a backup setup – a small external enclosure with 10k SAS drives, teamed in several RAID 1 mirrored pairs. Additionally I make copies of most valuable work and photo scans on DVD drives. Tape drive and tapes are meant to be an extra security, with long-term archival properties in mind.

Goals:

Since this is a »fresh-start«, I am not chained to any specific type of tape storage technology, and thus I can choose more freely the most suitable one, matching my preferences and budget.

Here's the brief outline of my prerequisites, in no particular order of preference:

Tape capacity: 20GB min - 100GB max (native/uncompressed)
LVD/SE SCSI interface
Direct attachment, no server or network of any kind involved
Industry proven long-term reliability of mechanics, electronics and read/write heads
Easy basic maintenance (cleaning innards, manual unloading stuck tapes, lost tape leader simple repairs, etc.)
Low to moderate power consumption
Tape drive native and HW compression transfer rates, best suited for real-life capabilities of my computer (subsystem throughput, OS, choisen backup application, etc.) , to allow for 100% consistent streaming mode.

Arguments: This is a long-term investment, so my main concern is tape drive longevity. Avoiding bottlenecking and frequent re-positioning during writes is not an option, it’s a must !

"Meet the HP Ultrium 230 External Tape Drive:"

I recently came across a brilliant opportunity, to purchase a brand new HP SureStore Ultrium 230e tape drive, at the price that is almost too good to be true. But before I jump onto this wagon, I really need to identify most critical areas of potential issues. I'm hoping to receive some sound advices and recommendations.

Points of focus, after doing an extensive research, are the following:

1. Throughput & streaming:

A major source of my doubts and concerns, regarding the type of tape drive to purchase, is for being worried that my system will not be able keep up with the speed of “data conveyor belt” – feeding the tape drive fast enough to allow for consistent streaming mode.

If anyone cares to take a quick glance at my PC specs, and help me with a rough assessment of my system throughput projection (worst case scenario !), I may be a few steps closer to my final decision. I will merely focus on those system details, and intended tape drive usage, that I believe matter the most for the topic at hand (in my humble opinion);

- Workstation/Server board Tyan S2925 (ATX Form Factor)
- AMD Phenom X4 2.5MHz
- 8GB ECC RAM (I could designate 1GB just for extra tape drive cache)
- 3x PCI 32-bit/33MHz slots*
- One PCI slot occupied by Adaptec AHA-2940U2W Ultra-2 Wide SCSI card (PCI 32-bit, 80MB/s max)
- Ultra2 Wide segment** of Adaptec AHA-2940U2W reserved exclusively for tape drive
- Pair of 10K SAS hard drives in RAID_1, reserved exclusively for temporary holding storage of files to be archived. Attached to SAS HBA x8 PCIe
- Tape drive mainly designated for archiving large files (TIFF photo scans), but not exclusively so.
- Linux Fedora 15 distro


*Unfortunately I cannot make use of much faster, bandwidth generous PCI Express technology for tape drive SCSI controller, as those slots are already occupied by hard drive SAS controller and video card;

**Link to AHA-2940U2W Product Brief: http://download.adaptec.com/pdfs/aha2940u2w.pdf

In case it is rather obvious (to seasoned IT experts), based on these specs and »on paper« system performance, that my PC is a no match for Ultrium 230 LTO-1 streaming requirements, not even with HP Data Rate Matching feature at work (min 6MB/s at native/uncompressed rate), I would immensely appreciate if anyone could advise me on »down-scaling« scheme of my ambitions, in this particular order:

- HP SureStore Ultrium 215 HH
- DLT 7000/8000 (HP, Quantum or Sun flavour)
- Quantum/Benchmark DLT1 FH
- DLT 40/4000 (HP, Quantum or Sun flavour)


2. Performance Assessment & Diagnostic Tools:

I looked long and hard, initially browsing through HP Ultrium 230 documentation and list of available tools/utilities. Sadly, not one of them is supported in Linux Fedora, or any other desktop Linux distros for that matter.

- HP L&TT Tools
- HPTapePerf
- HPReadData
- HPCreateData

What are my other options ? Quantum’s xTalk is also available only for enterprise Linux distros. Can somebody recommend any well regarded open-source tools, with features and performance roughly comparable to HP's own tools ?

3. Ultrium 230 Firmware updates under Linux : How on earth … ?

Once again here, I bump into this mind boggling wall of Linux support .. or a lack thereof ! HP L&TT Tools, no doubt magnificent and dandy, are not available for Fedora (or Ubuntu, or Debian, or OpenSuse, or …. ). This may sound as a desparate attempt, an imulse thought spawn by frustration, but … will the Red Hat version of L&TT install and work in Fedora ? Do I feel lucky ?

It’s when I had begun asking myself these questions, that the first serious doubts about HP’s Ultrium surfaced up. To go for it anyway, and pray I won’t ever need a firmware update ? Sounds pretty unnerving, and not exactly a future-proof decision.

4. Switching HW compression ON/OFF: The manual way

HP Ultrium 230, being my No.1 choice, uses a firmware based automatic HW compression ON/OFF switching. By default compression setting is ON, and it’s turned off when the drive chooses to do so (uncompressible data, or for data already compressed). While this may look good on paper, and is perhaps useful for most backup/archiving scenarios, I am never too happy about any “auto-this-and-that” ingenuity, unless it can be switched on and off manually, at my free will and based on my own judgement (good or bad).

There seem to be further conlicts of interests, as HP warns users to - quote…”Make sure any software data compression is turned off”. On the other hand, some manuals/guides for Linux backup utilities caution in the opposite direction – to turn HW compression off, with a note that this issue is less critical with LTO drives (source: Amanda Wiki). Come to think again, I’d actually prefer to turn off HW compression on the drive permanently, and utilize either the uncompressed tape writes by default, or use the Linux compression tools whenever I see it fit.

Question:

Can I switch HW compression ON and OFF on Ultrium 230, using MTCOMPRESSION ioctl ?

If not, are there any advanced Linux backup utilities, that support switching off HW compression on Ultrium/LTO drives ?

5. Responsiveness/speed of backup utilities seems to be very common culprit for mediocre tape backup performance (inconsistent streaming). While I’d like to believe I should be just fine, using time proven Linux tape backup utilities such as tar and kDat, I’m also concerned due to comments and recommendations I read, especially HP’s own not to rely on tar and similar native Linux apps, supposedly because they are not fast enough for Ultrium 230/LTO-1.

If that’s the case, and assuming that HW components of my sistem are up to the task, I’ll need to carefully select a backup utility, with performance to match the available throughput scheme for streaming mode. Once again I’m facing the fact, that most advanced free open-source backup tools, sush as Amanda or Bacula for instance, seem to be designed for network/server/client backup schemes.

So what are my options, regarding a speedy backup tool for desktop Fedora box with direct attached tape drive ?


5. Essential things about tape drive backup/archiving, which I have acknowledged:

- During tape writing or tape restore tasks, no other application/process, being BUS/CPU/HD subsystem intensive, should be running in parallel on computer, in order to provide maximum transfer rates and improve chances for consistent streaming mode

- Block size 64KB, the larger the better.


This should be it. I am aware it is lengthy, but I can also assure you, first, that I worked hard to make it as brief as possible, and second, that I've done my homework as thoroughly as I thought I should have, before posting on this forum.

Thank you all for your time and effort, and your patience in particular !

Kind regards,

Bostjan

P.S.: Have mercy ...

 

[Stivan Widick]

Is it best to stick with a single tape manufacturer? Any recommendations on LTO-5 tape brands?

Also, on the Win7 x64 platform, is a Tandberg/ATTO solution as good and reliable as an HP/ATTO solution?

Thanks.

 

[Tim Jones]

Simply stated - for LTO-5, we recommend ONLY Fujifilm and HP brand media. Both are certified for both normal backup and single-shot, long term archival use. It is safe to mix brands.

We have seen other brands actually damage the heads of the drives when used in an archival situation because of the rough surface of the media.

Tim

 

[Tim Jones]

Question:

Can I switch HW compression ON and OFF on Ultrium 230, using MTCOMPRESSION ioctl ?

If not, are there any advanced Linux backup utilities, that support switching off HW compression on Ultrium/LTO drives ?

Yes, but why bother? The tape drive is VERY smart about this. Additionally, the compression used is a fully lossless algorithm and you do not suffer from the same issues you see in compressing audio and video. You always get 100% of what you put in back.

5. Responsiveness/speed of backup utilities seems to be very common culprit for mediocre tape backup performance (inconsistent streaming). While I’d like to believe I should be just fine, using time proven Linux tape backup utilities such as tar and kDat, I’m also concerned due to comments and recommendations I read, especially HP’s own not to rely on tar and similar native Linux apps, supposedly because they are not fast enough for Ultrium 230/LTO-1.

If that’s the case, and assuming that HW components of my sistem are up to the task, I’ll need to carefully select a backup utility, with performance to match the available throughput scheme for streaming mode. Once again I’m facing the fact, that most advanced free open-source backup tools, sush as Amanda or Bacula for instance, seem to be designed for network/server/client backup schemes.

So what are my options, regarding a speedy backup tool for desktop Fedora box with direct attached tape drive ?

The biggy here is to realize that the LTO-1 is ONLY capable of 15MB/sec. This is FAR less than even a 10 year old Linux system can deliver via SCSI-1 (40MB/sec max). As for performance, our BRU products have been running on Linux systems since kernel version 0.99PL12 (circa 1993), and we are still the fastest performing solution available. Plus your Linux-based BRU archives will be fully compatible with BRU on the Mac and Windows platforms with no additional work.

5. Essential things about tape drive backup/archiving, which I have acknowledged:

- During tape writing or tape restore tasks, no other application/process, being BUS/CPU/HD subsystem intensive, should be running in parallel on computer, in order to provide maximum transfer rates and improve chances for consistent streaming mode

Not necessary with BRU and most other tools because the drive is too slow for there to be an effect.

- Block size 64KB, the larger the better.

I would recommend 1MB - for BRU this is easy. for tar, you'll need to make sure that you're using the latest version of GNUtar and that it supports larger blocksizes. A tar command using 1MB would look like this (the -b argumanet is how many 512-byte blocks):

tar -cvf /dev/nst0 -b 2048 /stuff/to/backup​

But, be sure to write that on the tape label so that you know what you used. There's no way (other than trial and error) to determine a tar blocksize after the fact.

This should be it. I am aware it is lengthy, but I can also assure you, first, that I worked hard to make it as brief as possible, and second, that I've done my homework as thoroughly as I thought I should have, before posting on this forum.

Thank you all for your time and effort, and your patience in particular !

Kind regards,

Bostjan

P.S.: Have mercy ...

My pleasure - see, totally painless.

Tim