Long-Term Storage of RED-One footage "other than hard drives" ??

[Tom.Wong]

What's a "green" drive?

.

it's type of western digital drive, there literally a green label on it, and they call it their "green" drive. they run cool, and quiet at large capacities, but aren't the fastest. affordable too.

 

[Elsie N]

What's a "green" drive?

.

George, they are drives that spin at from 5400 RPMs to 5900 RPMs. Most HDDs used in a computer are 7200 RPMs meaning that they can read and write data a little faster.

In my opinion, they are perfect for storage. I use them in conjunction with an eSATA docking station. You just insert the bare drive, turn it on, and it becomes this 2 TB thumb drive. I don't recommend the the USB 2.0 version docking station. A USB 3.0 version will serve you well for the time being.

The name "green" comes from the fact that they use less power and are therefore, supposedly more environmentally friendly. The so-called green drives are like a regular drive in every other way... except price. The green versions are usually about 2/3 or 3/4 of the price of a regular drive.

I had two, 2TB 5400 RPM drives setup internally (in my computer) as a RAID 0 once and they worked fine... probably as well as a single 7200 RPM drive or better.

EDIT: In re: 2, 2TB drive internal RAID 0... I think a 64 bit operating system is required to use drives over 2 TBs, which a RAID 0 of 2, 2TB drives would certainly be.

 

[George D.]

George, they are drives that spin at from 5400 RPMs to 5900 RPMs. Most HDDs used in a computer are 7200 RPMs meaning that they can read and write data a little faster.

In my opinion, they are perfect for storage. I use them in conjunction with an eSATA docking station. You just insert the bare drive, turn it on, and it becomes this 2 TB thumb drive. I don't recommend the the USB 2.0 version docking station. A USB 3.0 version will serve you well for the time being.

The name "green" comes from the fact that they use less power and are therefore, supposedly more environmentally friendly. The so-called green drives are like a regular drive in every other way... except price. The green versions are usually about 2/3 or 3/4 of the price of a regular drive.

I had two, 2TB 5400 RPM drives setup internally (in my computer) as a RAID 0 once and they worked fine... probably as well as a single 7200 RPM drive or better.

EDIT: In re: 2, 2TB drive internal RAID 0... I think a 64 bit operating system is required to use drives over 2 TBs, which a RAID 0 of 2, 2TB drives would certainly be.

OK, so I need an eStata docking station. I would rather be able to store the data on external hard drives so I can keep them in another location for safety.

I currently have external hard drives that hook to USB3.0. I was lead to believe that they were better than eStata. Which is true?

I can see that getting the RED-One is not the end of it, because it's data storage is another realm.

And, on and on it goes. Where it will end no one knows.

Wheeeeeeeeeeeeeeeeeew !!

 

[Elsie N]

OK, so I need an eStata docking station. I would rather be able to store the data on external hard drives so I can keep them in another location for safety.

I currently have external hard drives that hook to USB3.0. I was lead to believe that they were better than eStata. Which is true?

I can see that getting the RED-One is not the end of it, because it's data storage is another realm.

And, on and on it goes. Where it will end no one knows.

Wheeeeeeeeeeeeeeeeeew !!

eSATA is faster than USB 3.0. Still, for 4k footage USB 3.0 is probably enough bandwidth for checking footage on a stored HDD. Loading footage onto the drive in large amounts however, would be much faster through an eSATA port.

Offloading 5k footage... I think we definitely need at least the eSATA and hopefully for PC guys like myself, Light Peak (or whatever it's called now) in the near future.

 

[Seth Hampton]

We use the Rimage Disc Publishers and backup on BD-R's. The discs are cheap now $1.30 each or less. The Rimage unit also does disc spanning so you can load up to 300 BD-rs and it will serialize and print a label and fit as much info onto a disc and then keep spanning over the other discs. You can have it on your network so everyone has access to it. Great unit. And you dont worry about HDD's being idle.

My 2 cents

 

[George D.]

We use the Rimage Disc Publishers and backup on BD-R's. The discs are cheap now $1.30 each or less. The Rimage unit also does disc spanning so you can load up to 300 BD-rs and it will serialize and print a label and fit as much info onto a disc and then keep spanning over the other discs. You can have it on your network so everyone has access to it. Great unit. And you dont worry about HDD's being idle.

My 2 cents

This sounds more like it to me. Saving the data on external hard drives "AND" on Blu-ray discs seems to me to be much safer in the long run. It may take a lot of discs, but so what.

Would you be kind enough to be more specific as to this "Rimage" unit, and where I can get one, etc, etc.??

Sorry, I misunderstood. I just checked into this Rimage and it's way to expensive too. Why does everything have to be so f'ing expensive these days? Damn!!

Thanks.

 

[Johnny Friday]

George,
as others point out...if you are storing data for any length of time and putting it on the shelf or for a client....I'd hire out LTO---i did this in the past with my stock house. BUT now i keep all my data at my fingertips in an online capacity using four external 5 bay hot swappable, e-sata drive bays (that's a mouth full). Mine is not merely for long-term storage but i use and call up clips all the time. If i HAD the ability or wanted to INSURE myself 100% I'd LTO everything, but i have not nor will not go that route. I'm awaiting next solutions. But as mentioned i keep full HDD backups of all my data. I think im insured well.

Going this route is easy and relatively inexpensive:
5 bay drive:Hot swapable (I use the non-tray versions, so that i can just push in a drive and not screw it onto a mounting tray); you can go with 6 & 8 bay enclosures, but the most practical bang for your buck is 5 bay. One esta cable can handle a 5 bay enclosure. Anything more like 6 or 8 bay you then need to connect a 2nd e-sata cable. So, if you want MAX online capacity, then 5bay x 4 enclosures.
Sata Host card (goes in your PCIE expansion slot): i use the 4 port so i can connect four 5bay drives with 2tb on each drive for max online of 40TB storage

I currently keep two online at all times and other two are in backup mode. Soon I'll outgrow my 20tb and put a 3rd enclosure online, but i'm banking on 3tb or 4tb drives coming out by then. Actualy 3tb drives are out now, but a bit expensive.

this allows me to have all data online and what i would call stored. sure, i cannot shelf it for 5 years or more....but i use it all the time so i dont need to shelf it. Anyway, works for me and also you can set the bays up as raids for working with a large project. I also buy the enclosures with the port multipliers for faster access, but if using merely for storage, you don't need the port multiplier.

I have bought all my enclosures from Macgurus in Washington. Nice guys and they will talk to you on the phone and help you out with building a system. Also when e-sata 2 goes out....they will offer option of e-sata 3 or whatever is next--thunderbolt connectivity or something. So, you will likely be able to utilize the same enclosure when a faster pipeline is built.

Macgurus: http://www.macgurus.com/

 

[Seth Hampton]

I just checked into this Rimage and it's way to expensive too. Why does everything have to be so f'ing expensive these days? Damn!!

Thanks.

You dont have to get the "Top-O-The-Line" Rimage to get the job done. The Rimage 5400 has network abilities 150 disc capacity and 2 BD-R drives for around $12K. These machines are brilliant! Not to mention....the print quality is outstanding.

 

[rick youck]

though LTO is currently the best, and only good option for long term storage. "raw" hard drives make good sense with 2TB greens as cheap as $80

If you want something a little more rugged then a toaster style docking station, and don't want to build your own - try these:
http://www.wiebetech.com/products/RTX100.php

and store the drives in these:
http://www.hudzee.com/

and if in canada - hard drives here
http://www.memoryexpress.com

 

[Elsie N]

Sorry, I misunderstood. I just checked into this Rimage and it's way to expensive too. Why does everything have to be so f'ing expensive these days? Damn!!

Thanks.

Just buy a Blu Ray burner (about a hundred bucks) and take a little more time to get your Blu-ray backups. Think of time and money in balance. If you want less time (faster), put more money on that side. If you have less money, put more time (slower) on that side of the scale.

Oh, and a good eSATA docking station is about $40 or less.

 

[Alexander Ibrahim]

eSATA is faster than USB 3.0. Still, for 4k footage USB 3.0 is probably enough bandwidth for checking footage on a stored HDD. Loading footage onto the drive in large amounts however, would be much faster through an eSATA port.

Offloading 5k footage... I think we definitely need at least the eSATA and hopefully for PC guys like myself, Light Peak (or whatever it's called now) in the near future.

That's not quite right, mostly because of lack of specifics.

eSATA 6G is faster than USB 3, but its also not widely available. Only a very few drives have the interface.

There are a few i/o cards, and its built into a few MB's, but I've found no eSATA 6G enclosures ... none, zed, zero, zilch.

So, today, the answer is USB 3 is faster than eSATA 3.

Theoretically.

Single spinning disk hard drives don't saturate even the original SATA spec. A 4 drive RAID 5 with 7200RPM SATA 3G drives can sustain just under 300MBps ... so it fits in SATA 3G.

So, unless you have a RAID array, there is no practical difference between USB3 and SATA 3G. Both are only slightly faster than Firewire 800 with a single SATA spinning disk drive.

As for future planning for external storage, the word is Thunderbolt. That's what used to be called "Light Peak" and its shipping on MacBook Pro laptops. 10Gbps full duplex, plus a full bandwidth displayport. Faster than SATA 6G, faster than SAS, faster than even than most versions of Fiber Channel!

So fast, I probably won't be able to justify a new coffee machine.

I'm probably buying in a new 4 port eSATA RAID card, and a cheap eSATA enclosure or two- because Intel says that you can't retrofit Thunderbolt, its a MB chipset.

I think that's hogwash by the way. I see no engineering consideration stopping you from putting a Thunderbolt chipset on the graphics card.

Doing that would probably present you with a hit to Thunderbolt or graphics performance- possibly both. That's an acceptable engineering trade-off though. I'll accept 5Gbps duplex and slightly reduced graphics performance for the sake of a common interface and some forwards compatibility.

There are a lot of situations where that would work just fine. Especially in the case where you are adding a second card mostly for thunderbolt, or if the second card is a "light duty" card like a Resolve GUI card.

Of course when you see graphics cards with 6 MiniDisplayPort connectors ... they have bandwidth to burn and aren't sure what to do with it. Thunderbolt is a smarter answer. Give me three Thunderbolt ports instead of 6 MiniDisplay ports any day.

(A single MiniDisplayPort link requires 17Gbps. That's plenty of bandwidth- just use it differently.)

Since I have no need for a separate HBA, I may well buy two graphics cards if I need 6 displays.

With PCIe 3.0 on the horizon such a card makes even more sense

 

[Alexander Ibrahim]

Just buy a Blu Ray burner (about a hundred bucks) and take a little more time to get your Blu-ray backups. Think of time and money in balance. If you want less time (faster), put more money on that side. If you have less money, put more time (slower) on that side of the scale.

Oh, and a good eSATA docking station is about $40 or less.

Yeah, but the BluRay burner is 5x slower than LTO-4 or a two drive RAID -0 array. Plus you have to feed it disks way more often, so you can't let it run unattended. (LTO at least offers robotic loaders to solve this problem if need be.)

I think the BluRay solution is not tenable for anyone with a deadline.

It would have been FANTASTIC in 2004, and pretty good in 2007. If it had been more prevalent then, maybe today we'd have the promised 100+GB BD-RW disks at 40+MB/s that would make it work for more people as an archive solution for multi-terabyte data.

Even so, that much spanning of data is probably not a good idea for video data. So for our business, LTO and disk based archival are much better choices.

 

[Elsie N]

This sounds more like it to me. Saving the data on external hard drives "AND" on Blu-ray discs seems to me to be much safer in the long run. It may take a lot of discs, but so what.

... I just checked into this Rimage and it's way to expensive too. Why does everything have to be so f'ing expensive these days? Damn!!

Thanks.

Just buy a Blu Ray burner (about a hundred bucks) and take a little more time to get your Blu-ray backups. Think of time and money in balance. If you want less time (faster), put more money on that side. If you have less money, put more time (slower) on that side of the scale.

Yeah, but the BluRay burner is 5x slower than LTO-4 or a two drive RAID -0 array. Plus you have to feed it disks way more often, so you can't let it run unattended. (LTO at least offers robotic loaders to solve this problem if need be.)

It would have been FANTASTIC in 2004, and pretty good in 2007. If it had been more prevalent then, maybe today we'd have the promised 100+GB BD-RW disks at 40+MB/s that would make it work for more people as an archive solution for multi-terabyte data.

Even so, that much spanning of data is probably not a good idea for video data. So for our business, LTO and disk based archival are much better choices.

Understand you have preferences, but your choices are out of reach of the OP, financially. (And me too, for that matter.)

 

[Elsie N]

eSATA is faster than USB 3.0.

That's not quite right, mostly because of lack of specifics.

I'll leave that argument to our resident all-things-computer guru, Jeff Kilgroe, 'cause he's the one who originally posted that eSATA (3Gbs) is faster than USB 3.0.

 

[Rob Gardner]

archive

archive

For me (and probably for many producers) it is a complicated problem. We have dealt with format change over the years, transferring 1” tapes to digibeta and so on and recently transferring 28 hours of super 16 negative to HDcam tape and prores to upgrade it all to HD to extend its shelf life. All the SD stuff is basically worthless. And that is just the originating format issue.

With technology and prices changing so fast, I can’t see buying into what seems to me to be an expensive and complicated system (LTO) that also seems to be evolving in front of my eyes, and I have had problems in the past with backup software that became obsolete (we are talking about over decades) and backed up data becoming unreadable.

I am now parking everything on duplicate 2tb drives using a docking station and I’ll wait to see what happens before I invest in comprehensive system. Things are just changing too fast. Hard drives are cheap and getting cheaper and very simple to use—though it is clear they are not without risk. But in our business, risk is just part of the landscape. Everything fails sooner or later. That’s why we need duplicates and triplicates.

So I am taking the cowards way out and waiting to see what happens. Since my entire business has been running on hard drives for ever with a (comparatively) low failure rate, and since the archival data on hard drives can be organized with existing software, I’m just going to hold back and see what develops (though I am becoming concerned about the number of hard drives we seem to be piling up).

 

[HD Hildebrand]

Agree, I'm in the same boat. With the amount of material we (red-users) are consuming, this is a hole in the market that I hope someone fills soon. Be great if Red was the one to make this come to fruition.

For me (and probably for many producers) it is a complicated problem. We have dealt with format change over the years, transferring 1” tapes to digibeta and so on and recently transferring 28 hours of super 16 negative to HDcam tape and prores to upgrade it all to HD to extend its shelf life. All the SD stuff is basically worthless. And that is just the originating format issue.

With technology and prices changing so fast, I can’t see buying into what seems to me to be an expensive and complicated system (LTO) that also seems to be evolving in front of my eyes, and I have had problems in the past with backup software that became obsolete (we are talking about over decades) and backed up data becoming unreadable.

I am now parking everything on duplicate 2tb drives using a docking station and I’ll wait to see what happens before I invest in comprehensive system. Things are just changing too fast. Hard drives are cheap and getting cheaper and very simple to use—though it is clear they are not without risk. But in our business, risk is just part of the landscape. Everything fails sooner or later. That’s why we need duplicates and triplicates.

So I am taking the cowards way out and waiting to see what happens. Since my entire business has been running on hard drives for ever with a (comparatively) low failure rate, and since the archival data on hard drives can be organized with existing software, I’m just going to hold back and see what develops (though I am becoming concerned about the number of hard drives we seem to be piling up).

 

[Jeff Kilgroe]

That's because the technology is AMAZING.

No. The bogus claims are fantastical. The technology is underwhelming. Unfortunately, much of the hype surrounding holographic media is a product of its own time -- a time when many people still held careers in programming computers via punch cards and the hot new gadget coming onto the market for workstation-class computers was the 3.5" floppy.

You can basically shatter a holographic storage device and still read out the data.

That is 110% science fiction, dude.

data rate and density is theoretically limited by quantum effects, tehrmodynamics and the uncertainty principle.

I'm not sure what you're trying to say here. That applies to any form of storage media or any form of miniaturized technology, microcircuitry, etc.. I myself must obey the physical laws of our universe, even though I try to pretend that I don't have to.

One thing most people don't realize about optical systems is that many of these limits have already been reached. We can already construct data surfaces that are smaller than the incident angles (holographic perspectives) achievable by photon wavelengths.

The problem isn't the science, its the technology.

Funny. I was at a NASA/JPL thing a few years back when a well-known physicist took the podium to say the same thing about time travel and interstellar travel. It's not too difficult to make the science work out on paper. What is physically possible within the confines of our universe is still an entirely different matter.

Holography won't cross over with hard drives in performance per dollar until sometime after 2020.

If ever. Holography and most of these promises for storage or computing applications have been a decade away for the last 30+ years. Holography is not the future. The future is polymorphic electron-channeling circuitry. Already working in labs, the electron channeling is quite amazing. The polymorphic substrates are a sight to behold as well, combining them is also working, but it's still a ways off until we can create circuitry as complex as a microprocessor. That's a whole different discussion though. But this is the technology that will ultimately replace current lithographic manufacturing techniques and will allow us to construct microprocessors that can mutate as needed for the tasks/algorithms at hand.

With luck we can get holographic archival media before then- it only has to outperform LTO after all, and that's a much easier target than trying to replace hard drives. I think SSD has that in the bag based on performance.

Considering the current state of holographic media, I wouldn't hold my breath.

By 2020 we should have 2.5" spinning disk drives with 14TB for about $40.

Perhaps... If spinning media is still being developed in 2020. By the end of this year, there will only be three major hard drive manufacturers left -- Western Digital, Samsung and Kyocera.

Assuming that SSD follows the same curves, we should see a 14TB SSD for about $1500

Good chance of that. Intel/Hitachi already have 2.5" 4TB SSDs in their product roadmap for 2014~2015. Hitachi is handing over its HDD operations to Western Digital and transitioning all their storage development efforts over to solid state. The stakes are high as there are a lot of players in the SSD market right now and consumers are eager. You don't see it with current product offerings, but there is a huge race on in the industry -- everyone is scrambling to deliver the first 1TB 2.5" SATA-6Gbps SSD for under $1K, many hoping to do that by the end of this year. 22nm NAND processes are coming online now, the weak links in all of this is the host chipsets and the continued need for OS-level TRIM operations. Apple is reluctantly placing limited (and very specific) TRIM support into OSX 10.7 only because SSD makers can't seem to get past cell reset and clean issues.

 

[Jeff Kilgroe]

Back on track...

Other thoughts on storage here for small archival needs, individuals, etc.. First of all Blu-Ray is terrible. It makes absolutely zero sense in any way for a backup or archival media when any significant capacity is needed. For home users, maybe. But even then, it makes little sense... Many users will still need a dozen or more BD's just to archive their iTunes libraries. The cost per GB is very high, it's very slow and only stores 50GB on a dual-layer disc, which are a few times more expensive than a 25GB single-layer disc. Cost per GB of BD-R media is similar to that of DVD-R. In other words, not good. I have yet to hear or read any one single compelling argument to use Blu-Ray (or any other optical / holographic format) for data archival.

Hard drives are the price/performance leader up to a certain point. There comes a point, which I would have to figure, where LTO transitions to the cheaper technology. When comparing about 18 months ago, the magic number hovered somewhere between the 25~30TB mark. It's probably closer to 45TB of archived data today. For most people, that is still a shitload of data and for their needs, LTO can't win a pricing argument. For those who are backing up 8 to 16 TB per month, an LTO solution can save them a lot of money over hard drives.

Cost aside, LTO is the most rugged and accepted choice for data archival right now. Even for smaller archival needs where the LTO can't claim the edge on price, it's still a great option to consider for its other benefits.

Like I said a ways back in this thread, backup and archival are a strategy or a process, not a single technology. It makes sense to have redundant hard drive backups for online or near term storage. Then offload the important archival bits to more hard drives and/or LTO as needed.

Hiring out to a data service can make sense at times to allow others to manage your archives. Which is a service I provide to many clients. But It's also an option that can get very expensive, very fast. If you produce a significant amount of data on your own, you really should have your own data archival process in place and not have to rely on an outside service.

 

[Alexander Ibrahim]

No. The bogus claims are fantastical. The technology is underwhelming. Unfortunately, much of the hype surrounding holographic media is a product of its own time -- a time when many people still held careers in programming computers via punch cards and the hot new gadget coming onto the market for workstation-class computers was the 3.5" floppy.

The inPhase product was pretty cool. 300GB at 20MB/s would have been great if they released it on schedule. Remember this product was supposed to hit the market at about the same time as the HD-DVD/BluRay "war" started.

That was a working product. I used a few of their devices.

What killed it was logistics and manufacturing. I wish IBM or some other more experienced manufacturer had snapped them up back then.

That is 110% science fiction, dude.

A) I like science fiction
B) I've actually done this, so I'm going to disagree with the fiction part.

I will admit that I had "adult supervision" but of 2 GB I copied over to the device from my PowerBook we recovered 100% from a bunch of pieces on the floor where the researcher threw it down.

We only picked up the big bits, we just threw out the rest.

I'm not sure what you're trying to say here.

What I mean to say is what my physics professor used to say, "Physics is over, the rest is just engineering."

There are lots of structures and devices we can design trivially. Any first year physics student should be able to calculate the specifications and design for a cyclotron. I assure you none of them can build one yet.

Space elevator is another good example. When the idea emerged we didn't have the materials to do it, but we knew they could be made. Now we can make the materials ... but the planetary production is still in the grams, and we need kilotons of the stuff.

We know how to design holographic devices that are way better than anything you have seen in science fiction, and we know they are achievable within the bounds of physics.

We just can't build them. Tantalizingly, a lot of the parts are increasingly ordinary, but some are still out of reach.

One thing most people don't realize about optical systems is that many of these limits have already been reached. We can already construct data surfaces that are smaller than the incident angles (holographic perspectives) achievable by photon wavelengths.

For visible light.

Also... we don't need to be nearly as precise on read as we do on write. So long as the photon hits something we get data- statistics plays a part in reliably reading out data from these systems. (And I don't mean the "lies, damn lies, statistics!" sort of statistics.)

Funny. I was at a NASA/JPL thing a few years back when a well-known physicist took the podium to say the same thing about time travel and interstellar travel. It's not too difficult to make the science work out on paper. What is physically possible within the confines of our universe is still an entirely different matter.

I think you are misunderstanding. The complain is usually that our technology is insufficient.

For example, we have a very hard time making things with actual straight edges. At the nano-scale our best manufacturing techniques look like the results of delicate neurosurgery performed with a sledgehammer.

The complaint about a lot of really interesting things is that our science is insufficient.

For example it is trivial to travel faster than light, if we only had a small amount of negative energy.

Of course we have no clue what negative energy is now, nor even how to go about trying to understand it, much less "generate" any.

We also don't have a fundamental definition of time. No, seriously, we don't know what time is. Perhaps if we had a better definition I could not spend so much of whatever it is on RedUser

There is also a staggeringly clear parallel in how E-M and gravitation work ... but we don't know how to relate them in any practical sense- although generations of scientists (including Einstein and Hawking) had "unification" as a goal. (Its not just about tidying up the math!)

Of course, maybe we could understand gravity better if we actually knew what mass was? Just a guess. Of course, negative mass would help a lot of issues as well.

Oh wait ... we already know that negative mass is the same problem as negative energy. Wait ... what's energy?

If we solve any of those fundamental problems, our world will pretty suddenly be more science fiction than the science fiction we have today.

If ever. Holography and most of these promises for storage or computing applications have been a decade away for the last 30+ years. Holography is not the future. The future is polymorphic electron-channeling circuitry. Already working in labs, the electron channeling is quite amazing. The polymorphic substrates are a sight to behold as well, combining them is also working, but it's still a ways off until we can create circuitry as complex as a microprocessor. That's a whole different discussion though. But this is the technology that will ultimately replace current lithographic manufacturing techniques and will allow us to construct microprocessors that can mutate as needed for the tasks/algorithms at hand.

lol.

Look here is a solid rule of thumb:

When scientists say, "it will take 30 years to achieve this" they mean, "we have no frakking clue how to do this, but give us LOTS of time and funding and we're confident we'll figure something out."

Fusion and artificial intelligence are two other excellent examples.

That doesn't mean its impossible, just that its REALLY hard and scientists are "occasionally" overconfident.

For the record various versions of holographic storage have been working in the lab for most of the last 30 years. (I can attest to only about 20 years myself.)

Considering the current state of holographic media, I wouldn't hold my breath.

To be perfectly honest, I think that holographic media is at a point where it needs a lot of funding on engineering problems not science.

The problem is that for the most part the only people who understand holographic technology are scientists.

I don't know if we can push past the inPhase products in a meaningful way, at least not until we have a generation of engineers accustomed to working with photonics and optronics.

Oddly enough, Thunderbolt may be a key. Once its optical flavor rolls out, a generation is probably going to grow up using optical fiber the way most new engineers today grew up using USB to interface with things.

Now my group grew up using RS-232 and similar tech. There is a gulf between the facility of the "youngins" with serial digital communications. Oh I get serial digital of course, but I grew up with different sorts of signaling, and I tend to think fundamentally differently about how to design a device.

The flip side we have to prepare for, is that more and more engineers will gloss over the basics when they are REALLY learning to work- i.e. when they're fooling around in the garage after school. Its hard to make people get "primitive" when they get to university after they've had success with more advanced technology. The thing is you only learn certain fundamentals by dealing with their primitive versions.

OK, now that I've said ALL that ... I agree about not holding your breathe.

Perhaps... If spinning media is still being developed in 2020. By the end of this year, there will only be three major hard drive manufacturers left -- Western Digital, Samsung and Kyocera.

Its a sign of a maturing and commoditizing industry. Those companies know very well how to work in the margins being afforded to them today. That is harder for the IBM and Hitachi's ... they thrive on being able to turn R&D into products- and they need a higher margin to do that. So, companies like that are always chasing the "hard" problems.

HP used to be one of those companies, then something called a "Fiorina" happened.

Apple sort of is one of those companies ... but they innovate not in fundamental technology but rather in user interface and industrial design.

Good chance of that. Intel/Hitachi already have 2.5" 4TB SSDs in their product roadmap for 2014~2015. Hitachi is handing over its HDD operations to Western Digital and transitioning all their storage development efforts over to solid state. ... Apple is reluctantly placing limited (and very specific) TRIM support into OSX 10.7 only because SSD makers can't seem to get past cell reset and clean issues.

I agree with this part.

Its pretty exciting.

One thing I sort of lament in seeing the "research driven" companies leaving the spinning disk industry is that hard drives could be much better engineered for archival use. There is going to be an increasing desire to use spinning disk media for what I call "mid term" storage ... something you can plug in when you need it, and then leave on the shelf - you know exactly what we have been going on about here.

Then again, I suppose if Seagate and/or WD agree with that view, they can probably get a company like Dupont or BASF to help them develop the right lubricants and hardware refinements.

I think Apple's reluctance to integrate TRIM support is interesting, in that its a bit odd. It doesn't seem like a major engineering effort ... and they can always turn it on or off in software as needed- so why are they so skittish about implementing it? Curious.

Got any insight on that?

 

[Jeff Kilgroe]

I'm cutting this response pretty short... Taken too long to get back to it and we've gone off the deep end anyway...

The inPhase product was pretty cool. 300GB at 20MB/s would have been great if they released it on schedule. Remember this product was supposed to hit the market at about the same time as the HD-DVD/BluRay "war" started.

InPhase is (or was) located about 50 miles from me. I visited about 2 years ago, got to see the holographic discs and the bulky $16K drive. The technology did show a lot of promise, but had a lot of problems too. Marketing and market timing were two big problems, but hardly the primary concerns.

A) I like science fiction
B) I've actually done this, so I'm going to disagree with the fiction part.

I've seen similar demonstrations. Even done with tape and hard drive platters. It's not the holographic nature specifically that allows this. It's the design of their data storage pattern. At the expense of maximum capacity, additional redundant bits and parity data are encoded throughout. Roughly 15-20% of the entire data surface could theoretically fail and you could still recover the data.

What's funny is that holographic storage has been marketed, almost from its inception, as being capable of taking a single shard of broken media and salvaging every last bit of data from it. Which at the time the whole concept of holographic storage was invented, would have worked within the then-conceived methods. Essentially storing multiple redundant images of micro-film resolution binary data, from multiple perspectives or incident angles. Holograms would have been incorporated into a thin sheet of plastic or a cube/crystal of acrylic or silicate. At the time this was all being proposed, 10MB hard drives were awesome and 30MB were rumored to be coming sometime in the next year. a 2" acrylic cube could theoretically hold an entire GIGABYTE!!! A very bold claim for that time period.

I think Apple's reluctance to integrate TRIM support is interesting, in that its a bit odd. It doesn't seem like a major engineering effort ... and they can always turn it on or off in software as needed- so why are they so skittish about implementing it? Curious.

Got any insight on that?

Plain and simple, TRIM is not a valid solution to the problem at hand. It is a band-aid. The more you understand about Apple's design philosophy, the easier it is to see why they ignore certain technologies. USB3.0 is a perfect example. Why would Intel, the company who invented USB and dumped millions of $$ into perpetuating USB2.0, along with Apple, the company that firmly embraced and established USB as a standard when the PC world turned their noses up at it; both do an about-face and dump USB3.0 like a moldy potato? Sure, Thunderbolt was in the works, but there's more. USB3.0 is a terribly inefficient band-aid, an add-on that operates on top of two previous layers of obsolescence. Intel almost gave in and originally was going to roll USB3 into SandyBridge as there were some potential setbacks with ThunderBolt. But they got it worked out in time. The USB ship has sailed...

SSD TRIM is similar in some ways. It's commonly viewed in the PC world as this great and wonderful thing. Truth is, it's not. It's a band-aid, which allows the OS to intervene into FLASH memory reset and clear operations because SSD memory controllers are incapable of knowing how, where and why to reset cells within their own storage volume with the greatest efficiency. Reset and clean is both a time consuming and detrimental process with current NAND type memory. There is absolutely no reason why TRIM type functionality can't be more localized on the storage devices themselves and not an OS function. Even better, as NAND tech improves, cells can be reset and cleared just as fast as they can be read or written to and without detrimental effect. When that happens, TRIM will no longer be necessary. Much of Apple's resistance to things like USB3 and TRIM has to do with forward-thinking and trying to force their new-found industry leverage to make others innovate or try harder. It works more than people realize. Apple incorporating TRIM into the latest OSX beta seeds tells me that all their demands on their suppliers of creating SSDs that don't need TRIM for optimum performance are not working out so well. NAND FLASH is a technology that needs to be replaced with something better ASAP. Unfortunately, that something better has not materialized just yet...