Backup and Tape device answers

[Isaac Babcock]

Unfortunately, there's no type of notebook/laptop interface for SAS on a laptop. There are a couple of eSATA cards, but they won't work with SAS devices (even though SAS and SATA are usually combined). For now, your only option is a USB DAT drive from HP (DAT160 = 80GB/tape), but be aware that we're talking 6MB/sec.

H

Has anything changed on this front in the last couple months? I've been holding out on moving to LTO because I'm mobile and always working off a Macbook Pro. In the meantime I'm building up quite a hard drive cache, and getting nervous...

DAT seems painfully slow, but if things don't change in the near future, it may be a necessity.

thanks as always

 

[Tim Jones]

Hi Isaac,

Unfortunately, it hasn't changed and won't be changing. DAT is your only solution for the laptop situation for now. There have been a couple of Firewire to SCSI solutions out there (Ratoc Systems makes one currently), and you could try that, but I wouldn't go higher than an LTO-2 from a sanity stance :yikes: .

I guess we'll need to bring one of those into our labs and see what we can come up with.

[UPDATE] We have a unit heading for our labs. I'll be posting results with various tape drives on the Mac as soon as we get it.

Tim

 

[Jeff Kilgroe]

I believe that Sony has an AIT tape drive with an eSATA interface. Or at least they had announced one a year or so back. Which leads me to say, there is no reason why current desktop LTO models couldn't be offered with SATA interfaces.

I don't think there's enough of a market for anyone to make an ExpressCard SAS adapter. I think myself and 4 or 5 others in this thread would be the only ones to buy such a thing!

One option is to look at the tape solutions from Cache-A. They're just hitting the market now, but they have ethernet and SATA connectivity options. However, I haven't been able to get any confirmation or denial about their tape systems working as a standard tape unit (for use with BRU or other software).

Isaac, I know you're mobile much of the time. But what if you were to transfer those HDD backups to LTO at a desktop workstation every so often... Maybe every other month or so. Or have someone do it for you. I don't know what that would cost to have someone do it, but it's just a thought.

 

[Robert Kurhajetz]

This company is new to me...don't really know much about their offerings...just wondering if anyone on this thread has run across them?

www.powerfile.com

 

[Tim Jones]

Sony Electronics - RIP...

Sony Electronics - RIP...

I believe that Sony has an AIT tape drive with an eSATA interface. Or at least they had announced one a year or so back. Which leads me to say, there is no reason why current desktop LTO models couldn't be offered with SATA interfaces.

Unfortunately, Sony Electronics is no more. Qualstar has picked up the AIT reigns, but they're more interested in high-end stuff, so I don't expect to see anything in the realm outside of SCSI, SAS, abd Fibre-Channel. As for LTO and eSATA, it's as you say below - too low a potential customer base.

I don't think there's enough of a market for anyone to make an ExpressCard SAS adapter. I think myself and 4 or 5 others in this thread would be the only ones to buy such a thing!

Exactly. The economy of scale is too low. However, we just got wind of a new USB-2.0 to SCSI interface from RATOC and we already have one coming into our lab. Depending on how things are implemented, a USB-2.0 interface would be able to drive even an LTO-3 drive. I'll post more once we receive the promised unit.

One option is to look at the tape solutions from Cache-A. They're just hitting the market now, but they have ethernet and SATA connectivity options. However, I haven't been able to get any confirmation or denial about their tape systems working as a standard tape unit (for use with BRU or other software).

The answer to that is no - it's designed to look like a disk drive or FTP server instead of a standard tape device. Of course, BRU could easily use it as a disk destination under iSCSI, but you can buy a lot more than 800GB of disk for US$8K.

Isaac, I know you're mobile much of the time. But what if you were to transfer those HDD backups to LTO at a desktop workstation every so often... Maybe every other month or so. Or have someone do it for you. I don't know what that would cost to have someone do it, but it's just a thought.

Hang in there for a few more days and we may have a laptop answer with RATOC!

Tim

 

[Isaac Babcock]

Isaac, I know you're mobile much of the time. But what if you were to transfer those HDD backups to LTO at a desktop workstation every so often... Maybe every other month or so. Or have someone do it for you. I don't know what that would cost to have someone do it, but it's just a thought.

Jeff, thats a good idea, I think that makes the most sense in my current situation - I'll look into finding a place that can take my drives and back up to tape - for me, every few months would probably be sufficient.

Tim, thanks for all the info, and I'll be anxious to see what you come up with for laptops.

A desktop workstation is in my future someday, but just doesn't make that much sense right now (the yurt and solar are pretty taxed with just the Red, harddrives, laptop, etc) - But these are all good solutions, so thank you.

 

[Steen Dongo]

I've been trying to make the following combo work:

PowerMac G5 Dual 2.0 with PCI-X slots - OS 10.5.8 - 3GB RAM
IBM TS3100 Autoloader Tape-library LTO3 (22 slots) on ATTO UL4S SCSI card in slot3
United Digital 8 disk hardware RAID 5 on another ATTO UL4S SCSI card in slot 2
BRU Producer Edition 2.0.1 build 1.74

I can say one thing about this combo: It fails EVERY time

I've tried removing the Disk raid from the equation, tried reinstalling the OS and the software, but it doesn't change anything.

Anyone else have any experience like mine or a qualified guess to whatever could be wrong ?

 

[Evangelos Achillopoulos]

Tapes or Disks or SSD's for long term...

Tapes or Disks or SSD's for long term...

Sorry for the late post on this thread but I just noticed it...

Recently I started to research about long term media archiving... as everyone i also have the same problem to solve...

I will start with a small story...

Back in my university years, 1991 that was, I had design and build for the research lab of renewable energy sources, a wireless data logging system... a high speed 1/1000 of second wind speed meter...

In same point at 1993 I decided to back up everything PCB drawings, schematics, source code etc. in a QIC80 tape that it was a good and reliable system at the time... Everything was in MS-Dos and some at Windows 3...

Since 1997 my basement lab/office was not used anymore so my old computer was never worked again... My mother... as all mothers... was keeping the place covered with sheets and plastic covers to prevent dust to ruin my stuff...

In 2002 I bought a brand new Sony AIT-2 tape library system along with Arcserve... I did a backup of my 2Tb corporate SAN on it...

At 2004 i got a call from a Professor from the University that asked me if I had the drawings of my data logger system... so I tried to restore the QIC80 tapes... a nightmare... nothing was working, no drivers no software to restore it... at the end I managed and build a system that was able to restore it (old widows 3 etc)... data re... something error... no restore from tape at the end... I call my professor and tell him the news...

A year after in late 2005 he call's me again and ask me to try again... So I thought what the hell, I will go to the basement, and try to mount the disk from my 12 year's old PC... the funny thing was, it worked immediately! no fancy restoring software no driver's, no nothing just plug it to a legacy IDE port and have it on my XP system...

Recently I needed some drawings from my AIT backup... guess what I couldn't run the software... it took me five days to make it happen... at the end it was stack to a cartridge and again no luck I lost the data for the second time...

So in a period of 16 year's the tapes have make me feel bad twice and the disk was working... and still is...

The number one threats of tapes are:

1) Will the propriety software, that they need to operate, exist in the next twenty year's?
2) Will the company that build them still support (drivers, software for the OS that will be that time etc.) it in a reasonable cost? LTO-4 at 2029 will be a dinosaur...
3) Will the company even exist by then?
4) Both tapes and disks are based in magnetic field, so will they still have the data available in 20 year's?
5) Will the tape be able to operate after 20 year's of no threading?

In my case I didn't had any support for the QIC 80 after 11 years... no drivers no nothing... In the AIT front the same in just 7 years...! Sony doesn't build them any more so they are EOL...

In the case 16 year's old hard disk drive I just put it on a IDE legacy connector and worked immediately... because I didn't needed any proprietary support software to access the data... Everyone is supporting MS-Dos FAT...

So the conclusion is:

In 20 year's there will not be an OS-X 10.6 anymore it would be the (X) unknown OS... will the manufacturers of LTO4 still making drivers and software for the tapes?

What I know for sure is that, based of the 20+ years of the MS-DOS file system support in the todays OS's, there WILL BE support for the HFS or the NTFS file system and there WILL BE a legacy SATA port in a motherboard at 2029... because today I HAVE an ATA IDE legacy connector on my i7 motherboard... 20+ year's after...

So if a hard drive can spin, it will work...

But keep in mind that there are two types of hard disks in the market... the consumer type and the 24/7 type... only the second is good for the job...

Also there is a study from Stanford University that claims 5X longer life expectancy and data retention on 2.5" mobile HD's over 3.5" drives... thats 30 years life at least! because of the way the platers are mounted and how the heads are parked...

http://lockss.stanford.edu/locksswiki/files/ISandT2008.pdf

As for the SSD's if you want to keep them for long, don't expect to have the data after 10 years... there is the gate electron leakage problem... the chances are rising 2x to have the data after 10 years if you have write the SSD just once...

I thing a set of two mirrored 2.5" 500Gb disks in a Peli case is as safe as it can get... now if you make another set and put it in a different place it will be much better...

So simplicity is the aim when we need "long term"... so drivers, special software, proprietary compression, encryption and complex setups will be the enemy in 20 years from now... If we can't make an IBM autoloader (see the above post) to work today, imagine what will be the situation in 20 years if we try to put it to work to restore our data...

Tim, please tell me your opinion... Will you be here in 20 years to support us?

That's my two euro cents...

 

[sebastian toro]

Nice thoughts Evangelos.

I am wondering the same... the data is no longer easy for access, you will need help from the soft developers at that time.... and also if you need to recover in just two months but in another place/country you need to stay compatible with the type of soft you use when you write the tape.

Tim, please we need your response...

thanks in advance

 

[oleng tjioe]

Tim,

i don't want to bother you with PMs and emails...
Where can i find (would be great if) 24/7 support people who can answer questions related to BRU Server 2.0.x?
Thanks

 

[Evangelos Achillopoulos]

Nice thoughts Evangelos.

I am wondering the same... the data is no longer easy for access, you will need help from the soft developers at that time.... and also if you need to recover in just two months but in another place/country you need to stay compatible with the type of soft you use when you write the tape.

Tim, please we need your response...

thanks in advance

Not to be a misunderstood, we are just humans, we can't commit forever, its understood, but even Tim is not available in the forum any more...

Will LTO4 device drivers/software be available in 2029?

 

[Tim Jones]

Archival for the long haul

Archival for the long haul

Sorry for the late post on this thread but I just noticed it...

Recently I started to research about long term media archiving... as everyone i also have the same problem to solve...

[SNIP]

The number one threats of tapes are:

1) Will the propriety software, that they need to operate, exist in the next twenty year's?
2) Will the company that build them still support (drivers, software for the OS that will be that time etc.) it in a reasonable cost? LTO-4 at 2029 will be a dinosaur...
3) Will the company even exist by then?
4) Both tapes and disks are based in magnetic field, so will they still have the data available in 20 year's?
5) Will the tape be able to operate after 20 year's of no threading?

Let me respond to your bullets first:

1: For BRU - while there are no guarantees, we are planning to be around as long as data archival is a necessary and feasible operation. Additionally, we work with many organizations to provide source code in escrow for such an unexpected occurence as TOLIS Group being nuked by a hostile backup company.

2: BRU uses a device-neutral architecture. This is why we'll write to tape, disk, the cloud, CD/DVD, SSD, CompactFlash, and any other writable media.

Translation - no drivers or device dependencies. Whether you're talking Linux, FreeBSD, Windows, Solaris, A/IX, IRIX, HP-UX, or OS X, if you can attach it to the system and the OS recognizes it, BRU will utilize it. In fact, the code that accesses tape on OS X was written under 10.1, slightly updated in 10.2.5 and hasn't been changed since - and we support all tape devices on all interfaces - SCSI, SAS, Fibre-Channel, Firewire, and USB.

3: While we plan on it - see my note about code escrow above.

4: For disk, there are other things besides magnetic retentivity that need to be taken into account, but for tape, modern technologies such as LTO and newer DAT formats, life expectancy is 30 years on the low end of estimates and as much as 100+ years on the high end. Combine that with our support for archive migration between technologies and you'll see that BRU means long term archival and recovery are simply the standard rather than the exception.

5: I've recounted this in other threads, but we have a customer that just recovered a bunch of Amiga data that had been written using Amiga BRU to QIC 150 tapes in 1986/7 onto a Linux system using BRU 17.1 (our current release). These tapes had been sitting in a shoebox in his closet since he wrote them.

So simplicity is the aim when we need "long term"... so drivers, special software, proprietary compression, encryption and complex setups will be the enemy in 20 years from now... If we can't make an IBM autoloader (see the above post) to work today, imagine what will be the situation in 20 years if we try to put it to work to restore our data...

Tim, please tell me your opinion... Will you be here in 20 years to support us?

That's my two euro cents...

For longevity, BRU continues to move forward with new OS releases. We were the first commercial archival tool for Linux (1994 Kernel version 0.99PL12) and for BSD386 in 1995. We currently support 18 different OSes and even take the effort to support extinct beasts such as Ultrix, UniCOS, PtX, HP-UX 9, Tru64 Unix 4, and AT&T SYSV to make sure our customers' data is available. If OS X goes away tomorrow and OpenSolaris becomes the desktop OS of choice, you'll be able to restore your OS X data onto the OpenSolaris platform.

As far as tape is concerned, the number one issue is the software used, not the tape technology (as many disk-backup pundits would have you believe). If you had used BRU to write to your QIC80 (floppy tape) tapes and you still had a system that supported the floppy interface, a few retention commands would have been all that was needed to restore your data. In your case, the tape problems were related to your software's dependency on a given set of device drivers and particular OS version(s). Because of BRU's device agnostic design and true cross platform archive model, we don't suffer from those ills. As I mentioned above, your IBM library and tape drive will work fine with BRU on our supported platforms.

You are correct about the ease of Disk access, but you are also lucky that MS-DOS from that era is still supported. With Microsoft's patent battles, you actually stand more of a chance of the loss of a given filesystem's support than you are with tape and a product such as BRU. Just look at what Apple did with ZFS from the late Leopard releases and the Snow Leopard betas to the SL release - poof, it's gone.

Personally, I use disk for any data that I need short-term (less than 2 years) recovery for and tape for anything that we'll be shelving for more than 2 years. In either case, BRU supports both operations.

Clear as mud?

Tim

 

[Tim Jones]

Not to be a misunderstood, we are just humans, we can't commit forever, its understood, but even Tim is not available in the forum any more...

Actually, I am, just took a month off in Italy :biggrin5:.

Will LTO4 device drivers/software be available in 2029?

Depends on if you keep one on the shelf. However, there will be some technology available (tape has died 5 times since 1984 when I got started :smilewinkgrin: ). See my more in-depth response above.

Tim

 

[Tim Jones]

Tim,

i don't want to bother you with PMs and emails...
Where can i find (would be great if) 24/7 support people who can answer questions related to BRU Server 2.0.x?
Thanks

We offer a 24/7 support plan. Contact the sales team for details.

Tim

 

[Brian Gaither]

Larry Jordan, esteemed final cut guru has this to say about hard drives.




Link to his article
Hard Disk Warning! by Larry Jordan
http://www.larryjordan.biz/articles/lj_hard_disk_warning.html


EDIT
I know cost of LTO is an issue for a lot of us so I started a thread called
HP DAT as a backup alternative to LTO
http://reduser.net/forum/showthread.php?t=24988

anyone know where this thread is? It links to classifieds.

 

[Tim Jones]

Compatibility, devices, and the future

Compatibility, devices, and the future

Will LTO4 device drivers/software be available in 2029?

Let me take this answer a bit further -

All SCSI and SAS devices (and therefore devices that emulate SCSI on Firewire, F-C, USB, SATA, and ATAPI) are designed to adhere to a set of agreed-upon command and function specs defined by ANSI/INCITS's T10 committee (with which TOLIS Group is involved). What this means is that all tape drives are being designed and implemented to support the functional and command specifications as defined in the T10's SSC (SCSI Stream Command) specification. For Libraries, this is the SMC specification. To learn more, check out their website.

Many of you have read my posts about the fact that BRU's tape device code has not changed on OS X since 10.2.5 (Linux, Solaris, and FreeBSD since 1997). The reason behind this is because we write our code according to those same ANSI specifications. If you find a tape device that BRU does not play well with, I will stand behind a 99.9% guarantee that the drive is not fully compliant with the T10 SSC-3 spec or has a bug in it's firmware. The current version of BRU has identified problems in the last Overland NEO e series library, the Quantum SuperLoader 3 library, and the HP LTO-1 full height drives. All of these were resolved by new firmware from the device vendors, but required no changes to the BRU code.

The same logic should be followed by other software vendors, but this does not seem to be the case since device drivers always seem to be the root cause for many of their software issues (as Evangelos and others have found). This also explains why you need different device drivers for their software depending on who's device and what type is being used.

With that said, this is why I'm so positive that BRU will be able to deal with the new devices that appear as we move forward - so long as they adhere to the appropriate T10 spec.

Tim

 

[Tim Jones]

anyone know where this thread is? It links to classifieds.

Works for me...

Just for kicks, here's the link again:

Hard Disk Warning by Larry Jordan

HTH,
Tim

 

[Tim Jones]

Ooops - My Bad. I misread Brian's post. He was looking for the HP DAT thread...

As it doesn't show up at all, I'll start one (since I love DAT too!).

Tim

 

[Evangelos Achillopoulos]

Actually, I am, just took a month off in Italy
Tim

I'm glad that you are back... and I can have an expert answering these hard questions...

Don't get me wrong, I've being burned from tapes... so I have some questions that probably sound aggressive... you see in this archives we going to try to store fragments of our civilization for the years to come...

As I understand from your detailed answer, I must except your sincere reassurance that TOLIS group will exist in 20 years... what is really enlightening is that your source code is in escrow, of coarse you can understand the costs involved if I, as a simple user, need it in 30 years... but from nothing, something is much better.

So its good that the driver thing is almost solved (remains the bad firmware thing) and your software can work even with disk based systems.

Speaking of which, since we can use BRU for archive in disk drives...

A) can you please tell me your opinion on the study from Stanford university about the 2.5" drives long term data retention?

B) your opinion in SSD's long term retention?

C) And if a set of two 2.5" disks would be enough for 20 years + long term archival?

"Also there is a study from Stanford University that claims 5X longer life expectancy and data retention on 2.5" mobile HD's over 3.5" drives... thats 30 years life at least! because of the way the platers are mounted and how the heads are parked...

http://lockss.stanford.edu/locksswiki/files/ISandT2008.pdf

As for the SSD's if you want to keep them for long, don't expect to have the data after 10 years... there is the gate electron leakage problem... the chances are rising 2x to have the data after 10 years if you have write the SSD just once...

I thing a set of two mirrored 2.5" 500Gb disks in a Peli case is as safe as it can get... now if you make another set and put it in a different place it will be much better..."

 

[Tim Jones]

A) can you please tell me your opinion on the study from Stanford university about the 2.5" drives long term data retention?

I agree that the 2.5" drives are better, but the capacity points become an issue (640GB vs. 2TB).

B) your opinion in SSD's long term retention?

Anything that depends on an electrical signal to maintain state shouldn't be looked at for long term storage.

C) And if a set of two 2.5" disks would be enough for 20 years + long term archival?

"Also there is a study from Stanford University that claims 5X longer life expectancy and data retention on 2.5" mobile HD's over 3.5" drives... thats 30 years life at least! because of the way the platers are mounted and how the heads are parked...

http://lockss.stanford.edu/locksswik...ISandT2008.pdf

The issue here is interface longevity. However, if I were doing qhat you're doing and wanted to stay away from tape, I'd simply use commodity drive and copy the data to newer technology every 3 to 4 years. This serves 2 purposes - it keeps the bits fresh, and also keeps your technology on par with modern options. Anyone that simply puts data onto a disk of ANY sort and then stores it in hopes of 20 year recovery might as well make sure they have a good exit strategy when the disks refuse to spin up :smiley:

I thing a set of two mirrored 2.5" 500Gb disks in a Peli case is as safe as it can get... now if you make another set and put it in a different place it will be much better..."

Good option, but I'd still recommend refreshing the technology on a 3 to 5 year schedule.

Also, one thing that I've learned in 25 years in the sysadmin world, if it can be used as a filesystem, it will be. If you're going to use disk for archival, make sure that it's configured on your system so that it ABSOLUTELY CAN'T be used as normal storage.