Welcome to our community

Be a part of something great, join today!

Register Log in
  • Hey all, just changed over the backend after 15 years I figured time to give it a bit of an update, its probably gonna be a bit weird for most of you and i am sure there is a few bugs to work out but it should kinda work the same as before... hopefully :)

Backup and Tape device answers

Thanks, Tim. I remembered seeing a post about the rough surface issue some time back, but I couldn't find the thread. Should one have a cleaning tape from both brands or are they essentially the same formulation?


Tim Jones said:
Simply stated - for LTO-5, we recommend ONLY Fujifilm and HP brand media. Both are certified for both normal backup and single-shot, long term archival use. It is safe to mix brands.

We have seen other brands actually damage the heads of the drives when used in an archival situation because of the rough surface of the media.

Tim
Click to expand...

Tim Jones said:
...the Tandberg / ATTO combination will work fine for you.
Click to expand...

Thank you!
 
Greetings Bostjan.
While Tim’s answers were very accurate and addressed a few of your questions, I thought I’d jump in here and address some of the other specific questions you had around the LTO specifications and in some cases the HP LTO hardware.

1.Throughput & stream

The SCSI card that you’ve referenced shows a Data Transfer rate of 80Mb/s. So without getting into the tech specs of your PC beyond the SCSI card and possible drives that you might be able to utilize, I think you have a few more options than the ones you’ve listed above.
Every generation of HP drives has electronics built into it to allow the drive to operate at lower speeds without causing a stream fail. This Data Rate Matching (DRM) allows the drive to monitor the incoming flow of data and to adjust our tape reel speeds based on an internal data buffer capacity.
The DRM system works by monitoring the amount of data in the drive's buffer 1000 times a second. In steady state (streaming) operations, the buffer is maintained at about half full. If the amount of data in the buffer starts to increase or decrease, the tape speed is adjusted, and as the density of the data written to tape always remains unchanged, the effective transfer rate of the drive is therefore changed. This brings the level of data in the buffer back to the 50% control point.
To maintain the maximum data transfer during a backup as the drive approaches the end of a wrap, the DRM system will empty the buffer. Whilst the drive stops writing to reverse the tape direction and the head assembly locks onto the next wrap, the buffer is allowed to fill. After the drive starts writing again the buffer level is once again maintained at the control point.
Now, one thing to note, If you are using a Gen 3 drive but have a piece of Gen 2 media loaded into it, the DRM numbers will be tied to the Gen 2 tape and not the Drive.

So, here are a few HP LTO Drives and their DRM numbers to allow you to see where you might leverage the transfer rates and to ensure that you setup a system that can constantly stream data.
HP LTO Gen 1HH: Capacity is 100Gb, Max transfer rate is 16Mb/s, the DRM range is 6.67-16Mb/s
HP LTO Gen 2HH: Capacity is 200Gb, Max transfer rate is 24Mb/s, the DRM range is 10-24Mb/s
HP LTO Gen 3HH: Capacity is 400Gb, Max transfer rate is 60Mb/s, the DRM range is 26-60Mb/s
HP LTO Gen 4HH: Capacity is 800Gb, Max transfer rate is 80Mb/s, the DRM range is 34.7-80Mb/s
HP LTO Gen 5HH (No PSCSI drive available. SAS and fiber connectivity only)
Capacity is 1.5Tb, Max transfer rate is 140Mb/s, the DRM range is 47-140Mb/s

2. Performance Assessment & Diagnostic Tools:

I looked long and hard, initially browsing through HP Ultrium 230 documentation and list of available tools/utilities. Sadly, not one of them is supported in Linux Fedora, or any other desktop Linux distros for that matter.

- HP L&TT Tools
- HPTapePerf
- HPReadData
- HPCreateData

What are my other options ? Quantum’s xTalk is also available only for enterprise Linux distros. Can somebody recommend any well regarded open-source tools, with features and performance roughly comparable to HP's own tools ?
HP does offer L&TT in many different flavors of Linux. I have yet to test a version that I was not able to find an HP build of L&TT that did not work. This allows you to do all the drive testing, drive upgrades, and drive monitoring that you should need. In cases where a user is looking for something more that L&TT doesn’t offer, we also have a a handful of Linux applications that are used on a daily basis in the R&D lab that we share on a case by case basis.
You can find the latest version of L&TT for free here: L&TT Downloads

3. Ultrium 230 Firmware updates under Linux : How on earth … ?

Once again here, I bump into this mind boggling wall of Linux support .. or a lack thereof ! HP L&TT Tools, no doubt magnificent and dandy, are not available for Fedora (or Ubuntu, or Debian, or OpenSuse, or …. ). This may sound as a desparate attempt, an imulse thought spawn by frustration, but … will the Red Hat version of L&TT install and work in Fedora ? Do I feel lucky ?

It’s when I had begun asking myself these questions, that the first serious doubts about HP’s Ultrium surfaced up. To go for it anyway, and pray I won’t ever need a firmware update ? Sounds pretty unnerving, and not exactly a future-proof decision.
L&TT is definitely the number one choice for doing things like firmware upgrades on the HP LTO drive. . If you choose not to utilize this solution, you could use the Linux firmware update executable that HP provides at a command line level.

4. Switching HW compression ON/OFF: The manual way

As Tim stated, there is no real reason to turn this off. The drive is intelligent enough about the type of data being written to know when it’s needed.

5. Responsiveness/speed of backup utilities seems to be very common culprit for mediocre tape backup performance (inconsistent streaming). While I’d like to believe I should be just fine, using time proven Linux tape backup utilities such as tar and kDat, I’m also concerned due to comments and recommendations I read, especially HP’s own not to rely on tar and similar native Linux apps, supposedly because they are not fast enough for Ultrium 230/LTO-1.

If that’s the case, and assuming that HW components of my sistem are up to the task, I’ll need to carefully select a backup utility, with performance to match the available throughput scheme for streaming mode. Once again I’m facing the fact, that most advanced free open-source backup tools, sush as Amanda or Bacula for instance, seem to be designed for network/server/client backup schemes.

So what are my options, regarding a speedy backup tool for desktop Fedora box with direct attached
There are many different options here for you to choose. As Tim stated, BRU is definitely one that has been successfully used on the LTO technology from its inception. Definitly one of the fastest that I have personally tested. I'd say that you should spend some time looking at options here and to look at some of the newer technology advances in LTO that is making it easier for someone to use tape. One such item would be LTFS. This is one of the latest buzz words in Tape. WWW.HP.COM/GO/LTFS

I hope this helps!
Brian

OEM Technical Account Manager
Hewlett Packard Storage Division
 
Last edited:
Tim, I have a question about LTO, and actually about BRU specifically. How does the CRC verification work? I'm familiar with md5 verification, which takes about twice as long as a plain copy because it reads the whole destination after writing. When writing data to an LTO tape, how is crc32 faster than md5?
Also, do the crc hashes get written to the tape somewhere in a text file that is user-readable? I presume that's how the tape can be verified later by a different computer.
 
In-stream CRCs versus after-the-fact MD5/SHA1 sums

In-stream CRCs versus after-the-fact MD5/SHA1 sums

How does the CRC verification work? I'm familiar with md5 verification, which takes about twice as long as a plain copy because it reads the whole destination after writing. When writing data to an LTO tape, how is crc32 faster than md5?
Click to expand...

It's not only that CRC32 is faster, it's that BRU actually generates the 32bit CRC internally within the data stream on a 2K basis. As BRU reads the data from your disk, it examines each 2K and generates the CRC for that 2K of data. We use assembly on all platforms for this routine and it costs very nearly 0. In fact, we've tested BRU's I/O with very high-speed systems where we can maintain over 2.6GB/sec including the CRC calculations. Additionally, because BRU is storing that CRC within the actual data block header for each block in the stream, it requires no additional work on your part and you can reverify the archive directly at that same 2K granularity level.

For a more detailed discussion, please check out this white paper on our site:

Reliable Verification

OTOH - after-the-fact tools like md5, md5sum, sha1sum, etc. all process the entire file after the fact. When you're dealing with multi-GB files as we deal with from the cameras, this means that the checksum routine must read the entire file before it can provide the checksum. Some routines are better than others, but they all are slow relative to processing the data in 2K chunks while it's being read from the filesystem. And, since you should be summing the file that's written onto the copy media and comparing it to the sam sum form the original media, that means 2 trips through the summing algorithm for each file.
 
I've read somewhere that to extend tape life, you should minimize eject/load cycles. Is that true? How many cycles are we talking about before it starts to be a problem? 100? 10000?
My instinct tells me it's not a good idea to leave a tape loaded in the drive while transporting on my cart in a truck. Is it better to eject at the end of each day?
 
Elliott Balsley said:
I've read somewhere that to extend tape life, you should minimize eject/load cycles. Is that true? How many cycles are we talking about before it starts to be a problem? 100? 10000?
My instinct tells me it's not a good idea to leave a tape loaded in the drive while transporting on my cart in a truck. Is it better to eject at the end of each day?
Click to expand...

That was true for older technologies, but LTO and modern DAT media can handle 10,000's of load unload cycles. We have LTO2 and LTO3 tapes that have seen more use that an average IT tape would see in 5 life cycles. While the drives also handle he tapes intelligently by disengaging the tape from the heads after periods of inactivity or during a power down, please do eject your tapes from the drive when you're not using them.
 
Any clues on Tolis Group discounts for we Reduser members?

Any clues on Tolis Group discounts for we Reduser members?

Hi guys.. Just a heads up that I heard a rumour that TOLIS Group ..(the BRU-PE blokes) are offering we fellow REDUser Forum members some really great discounts such as from $US250 to $US750 on tape and disk storage solutions.

is this true? Any clues on this? I'm up for some LTO4 replacements now.

Warwick
Hong Kong
 
Hey Tim, I have a couple new LTFS questions if you don't mind.

First of all, is there any problem using a Tandberg LTO-6 drive with HP drivers? Tandberg's drivers are old, and dont' support Yosemite. The HP StoreOpen GUI won't recognize the drive. The Tandbeg LTFS Manager app works, but it stalls upon mount. So I'm using the command-line ltfs for now.

When I write data to LTFS using Silverstack, the files are written in a seemingly random order. Example: C010, C011, C003, C002, C009, C008, C005, C012. I have no idea why. I thought this would cause shoe-shining during restore, but it doesn't. When I restore from tape using Finder, the files are transferred in the exact same order. Why? Is the ltfs driver smart enough to automatically arrange the files to be read as they were written on tape? I hope this works when I move the tape to a different computer, but I haven't been able to test that yet.

Sometimes write speed bounces around a lot, and averages about 70MBps. Read speed is always near full 160MBps. Not sure why.
 
You must log in or register to reply here.
Back
Top