SSD Suggestion

[Brian F Kobylarz]

You guys are seriously underestimating the amount of work it takes to get a single piece of media working properly on the camera as well as continual testing which each firmware upgrade. It takes like 6-9 months of ALOT of work to get things working and for a tiny market.

We're not talking is it just fast enough but wear patterns, write patterns, esd interactions, hotplug issues, block size variability, performance degradation, filesystem interactions, small vs big block performance, garbage collection and sector cleanup, as well as about 30 other parameters.

Geez Deanan, You make it sound so easy!

Seriously everyone, besides all the right "technology" reasons to use the right media, there is the PR side of the coin: If you are shooting for a client - and some frames drop (or worse, an entire take is trash), what does the client see and how will they react?
What do you think will be going through their mind - third party SSD's or a camera with RED Digital Cinema logos emblazoned.

Right or not, they will blame the first thing they see (probably you, then the camera). Picture yourself wimpering that "it might be the media"...
Think they are going to buy it?
Is it really worth risking your reputation?

If your name is worth that little, there are plenty of manufacturers building something just for you.

 

[Terry VerHaar]

This thread got me to thinking... I was considering buying a Prius or a Volt or a Leaf, but I may pass because none of them will let me substitute my AA rechargeable batteries for their very expensive (IMHO) proprietary battery systems. I think they are just trying to rip me off by making me buy into their overpriced infrastructure. :willy_nilly:

 

[Bruce Allen]

This thread got me to thinking... I was considering buying a Prius or a Volt or a Leaf, but I may pass because none of them will let me substitute my AA rechargeable batteries for their very expensive (IMHO) proprietary battery systems. I think they are just trying to rip me off by making me buy into their overpriced infrastructure. :willy_nilly:

Um, agree with you on the Epic side. However, for Scarlet, a better analogy would be:

I want an electric scooter that does 55mph but have to buy a battery designed to be guaranteed to work on a Tesla at 200+mph.

Scarlet's 55MB/sec max data rate of incompressible data could be handled by many SSDs, methinks. Epic is completely different - needs an absolute top-end SSD.

Personally, I hope this means RED plans to increase Scarlet's data rate with the Dragon upgrade but they don't want to promise anything so are keeping quiet about it.

This would explain why they are answering Scarlet SSD questions by saying many SSDs failed the test, without mentioning that the test was specific to Epic (and the SSDs would have passed a Scarlet test). If so, I encourage RED to continue to be as sneaky with their answers as possible! And cross fingers for 4K / 48fps on Dragon

Bruce Allen
www.boacinema.com

 

[Mike 'Fireman' Ross]

is there any reason to use 3rd party stuff on red besides its cheap? NO....
so RED please do not allow 3rd party media on cameras...People complaining about SSD prices should check Sony's SXS cards....

I really really don't like this attitude, but at the same time totally understand it.

'Allow'?

"no-one* gets to make decisions about what I am or am not 'allowed' to do with my own hardware. That's fundamental to the hacker ethic, and I'm a hacker, for some values of 'hacker'.

(and if you want to know what 'hacker' means, see: http://catb.org/~esr/faqs/hacker-howto.html#what_is )

On the one hand, RED don't, and shouldn't, have the power to allow or deny anyone the ability to use any storage media they chose. If Sandisk or Lexar decide to develop and sell RED-compatible SSD, they will do so. If I decide to develop and market alternative storage solutions, I will. It's down to the *professional* judgement of the *professional* doing the shooting.

On the other hand, RED *do* have the power to decline to endorse, refuse to support, or even positively disparage and speak against the use of, alternative storage media, if & when any such become available, and they very likely will do any or all of the above, for the perfectly good and valid business reasons that have been pointed out by several well-qualified contributors to this thread; people blame the camera, and mud sticks. Not every professional will exercise good judgement, not every storage manufacturer will manage to avoid producing a piece of crap. And as Deanan pointed out, it IS hard; if this tech was easy, everyone would be doing it and it would be cheap.

I was interested to note that Alexa used SXS cards; I've never laid hands on one so had no idea what way they jumped. Begs the questions of why RED didn't use SXS, and why Alexa didn't select RED SSD...

And I *still* think RED should revisit the idea of producing a SATA /LEMO-connector rear module to allow people to hook up their old RED DRIVEs & RAMs. You were going to. You canned it. Yes I know you love SSD. I know everyone hates spinning media; it's the past, for this kind of work anyway. But it should be cheap and simple to implement, and it would shut up a lot of people with less demanding applications and little budget for storage. Hell, some third party hacker could and probably should brew one up relatively quickly and cheaply.

And/Or build a 'RED RAM Mark 2' - maybe 4 x 256GB cheapish off the shelf SSDs, in a rear module, with a RAID controller. RAID 5 for data security if the performance is up to scratch. For those who want/need a 'lives on the camera all day' large capacity storage solution (that would be me; I have CF cards, but I still shoot 90% of my stuff on RED DRIVE).

Mike

 

[Mike 'Fireman' Ross]

Um, agree with you on the Epic side. However, for Scarlet, a better analogy would be:

I want an electric scooter that does 55mph but have to buy a battery designed to be guaranteed to work on a Tesla at 200+mph.

Scarlet's 55MB/sec max data rate of incompressible data could be handled by many SSDs, methinks. Epic is completely different - needs an absolute top-end SSD.

I kinda get it, but you don't fully get it.

It's not just about maximum sustained data rates, it's about *glitch and dropout-free sustained writing*. It doesn't matter if the drive can normally sustain 50, 250 or 500 MB/s for an hour at a time, if the firmware or basic drive design permits or mandates the occasional random or even regular brief drop in speed, or total dropout, for a second or two, for some kind of self-test or internal housekeeping; in almost any other application that would tolerable: 'device busy' is an acceptable response. In digital cinema with the unremitting real-time data coming in hot off the sensor during a take, you are DEAD if that happens.

It's way more complex than 55 vs. 250. Many drives that could easily sustain 55 would have misbehavior of the type I described above, or other reliability/repeatability issues.

Mike

 

[Cory Mitchell]

I think the heart of the problem, here, is that people see "SSD" and think that there are other manufacturers who make SSD, and are frustrated with the relative high expense of Red's SSDs compared to what's typically out there. I think it would be beneficial if people saw Red SSD's as proprietary. There might be other 1.8" SSD's out there, but they aren't really 'compatible.' In that way, it doesn't really work to compare it to motor oil.
I really think it's important to understand that even if your use is non-critical, and some error is acceptable, that there is more at risk here than your non-critical project. It's Red's reputation. And while you, or your immediate collaborators, might understand that the failure was your choice in media, other people won't. People are stubborn in that way. There are just too many dough heads out there who can't (won't?) understand otherwise.
In the end, and perhaps disappointing to people who want to cut costs, I really think this is where the motto "better safe, than sorry" should be the defining phrase for Red and camera media.

PS. no offense to all the dough heads out there.

 

[Ash Worth]

Seriously. Why do people think the cameras are so cheap? This is a pretty well known business model. Base model is cheap or free, accessories are expensive and proprietary. If someone had said you will be able to shoot 4k raw for $14k five years ago we would have laughed in their face. The sense of entitlement these days is astounding.

 

[Kemalettin Sert]

I kinda get it, but you don't fully get it.

It's not just about maximum sustained data rates, it's about *glitch and dropout-free sustained writing*. It doesn't matter if the drive can normally sustain 50, 250 or 500 MB/s for an hour at a time, if the firmware or basic drive design permits or mandates the occasional random or even regular brief drop in speed, or total dropout, for a second or two, for some kind of self-test or internal housekeeping; in almost any other application that would tolerable: 'device busy' is an acceptable response. In digital cinema with the unremitting real-time data coming in hot off the sensor during a take, you are DEAD if that happens.

It's way more complex than 55 vs. 250. Many drives that could easily sustain 55 would have misbehavior of the type I described above, or other reliability/repeatability issues.

Mike

mike im just saying i dont understand how people say 64GB SSD is not worth for 950$ think about it Red 16GB CF card is 300$ which is less secure than SSD and you get fastest media which is 64GB for 950$
so whats all this fuss about? tell me its not about money
people always wants lower price

 

[Richard Goodwin]

These drives, and the whole RED camera depend on real-time operations. Real-time does not mean that things happen in what a person perceives to be without delay. It has a specific meaning in engineering and that is that any operation will happen in a finite and predicable amount of time EVERY TIME. It also means that complex devices built on real-time operations depend on every part of the device working predictably in time. That is why there is a whole sub-branch of Electrical Engineering and Software Engineering devoted to real-time operations. It's the branch of engineering that powers commercial aircraft controls, medical instruments and disk brakes. RED cameras (in fact all digital cinema cameras) depend on real-time operations. While a dropped frame is not of the same danger level as a disk brake failure the Engineering of the systems depend on the same ideas.

What this means for RED cameras is that from when the light hits the sensor, the data is read from the sensor, processed by the onboard-eletronics and written to SSD; that entire chain of operations must happen in a very predictable way or not only will frames get dropped at the disk, but possibly the processors will get over-full or if the log-jam continues back the data from the sensor will not even get read in the first place. And, the more errors in any one part in a real time system then the more chance for complete breakdown where no footage gets recorded. There is are few if any commercial SSD drives with standard interfaces that would meet that standard. And the few that might would be build for very specific markets and likely have a price equal to or way beyond what RED charges for it's SSD. Drives designed for laptops and desktops would not come close to meeting real-time requirements.

This is why all the digital cinema manufactures have propriety FLASH storage based drives. They have to support real time operations. That requires using only high end FLASH chips, designs which can pipeline the data being recorded to the chips and specialized connectors which feed the data into the systems. As for having a standard connector to do that it seems an unfair requirement to force what is at best a 2nd generation technology (affordable 4K and up fully digital motion cameras) to have already reached a standard interface when a lot of the Engineering problems around digital cinema are just begin solved. To do that right would require a committee supported by a global standards body and that takes time. No third party manufacture worth their salt would put substantial resources behind an interface that was not backed by standards and clear licensing plans and methodology. It would be too risky.

Now, as for this idea that commercial "standard" drives are cheaper. Take a look at the pricing for server grade drives from IBM and HP that use standard connectors (SATA and SAS):

http://www.pcconnection.com/IPA/Sho...+4294966248&Sort=Availability&DefSort=Y&Term=

A 120gig SATA drive from HP lists at $2,278.30. Now with a discount that might come in close to the price of a RED 128gig SSD. This is representative of the 3rd party competitive market that various posters are talking about. These are drop in components that can be used interchangeably in server hardware. They meet the standards of relativity higher sustained throughput and continuous load over long periods of time. But, STILL they are not real time drives. IE they may or may not meet the requirements that RED, an airplane black box recorder, a medical data recorder, or other real-time system would need to work flawlessly.

So if you look at REAL MARKET NUMBERS, and add the requirements that a Digital Cinema Drive requires the facts say that RED is giving us a pretty fair deal.

 

[Mike 'Fireman' Ross]

These drives, and the whole RED camera depend on real-time operations. Real-time does not mean that things happen in what a person perceives to be without delay. It has a specific meaning in engineering and that is that any operation will happen in a finite and predicable amount of time EVERY TIME. It also means that complex devices built on real-time operations depend on every part of the device working predictably in time. That is why there is a whole sub-branch of Electrical Engineering and Software Engineering devoted to real-time operations. It's the branch of engineering that powers commercial aircraft controls, medical instruments and disk brakes. RED cameras (in fact all digital cinema cameras) depend on real-time operations. While a dropped frame is not of the same danger level as a disk brake failure the Engineering of the systems depend on the same ideas.

What this means for RED cameras is that from when the light hits the sensor, the data is read from the sensor, processed by the onboard-eletronics and written to SSD; that entire chain of operations must happen in a very predictable way or not only will frames get dropped at the disk, but possibly the processors will get over-full or if the log-jam continues back the data from the sensor will not even get read in the first place. And, the more errors in any one part in a real time system then the more chance for complete breakdown where no footage gets recorded. There is are few if any commercial SSD drives with standard interfaces that would meet that standard. And the few that might would be build for very specific markets and likely have a price equal to or way beyond what RED charges for it's SSD. Drives designed for laptops and desktops would not come close to meeting real-time requirements.

This is why all the digital cinema manufactures have propriety FLASH storage based drives. They have to support real time operations. That requires using only high end FLASH chips, designs which can pipeline the data being recorded to the chips and specialized connectors which feed the data into the systems. As for having a standard connector to do that it seems an unfair requirement to force what is at best a 2nd generation technology (affordable 4K and up fully digital motion cameras) to have already reached a standard interface when a lot of the Engineering problems around digital cinema are just begin solved. To do that right would require a committee supported by a global standards body and that takes time. No third party manufacture worth their salt would put substantial resources behind an interface that was not backed by standards and clear licensing plans and methodology. It would be too risky.

Now, as for this idea that commercial "standard" drives are cheaper. Take a look at the pricing for server grade drives from IBM and HP that use standard connectors (SATA and SAS):

http://www.pcconnection.com/IPA/Sho...+4294966248&Sort=Availability&DefSort=Y&Term=

A 120gig SATA drive from HP lists at $2,278.30. Now with a discount that might come in close to the price of a RED 128gig SSD. This is representative of the 3rd party competitive market that various posters are talking about. These are drop in components that can be used interchangeably in server hardware. They meet the standards of relativity higher sustained throughput and continuous load over long periods of time. But, STILL they are not real time drives. IE they may or may not meet the requirements that RED, an airplane black box recorder, a medical data recorder, or other real-time system would need to work flawlessly.

So if you look at REAL MARKET NUMBERS, and add the requirements that a Digital Cinema Drive requires the facts say that RED is giving us a pretty fair deal.

Richard,

As for real-time, you get it; you grok it in fullness. That's the point I was trying to get across in abbreviated layman's terms.

The key thing isn't the performance of the individual drive (of whatever type), it's the system as a whole - the pipeline.

RED have gone, exclusively, for the 'single drive' solution; an expensive custom drive & controller, made to the highest standards of performance and quality.

It's my opinion that this may constitute an error. But that isn't the only way to tackle the 'real time pipeline' problem. Consider the RED DRIVE: totally standard off-the-shelf spinning media, a standard RAID chip, and the gubbins to make it all work together and sit in a small pretty box. Result: $900 for 640GB.

There's a tenfold decrease in the price/capacity performance for REDMAG SSD: $950 for 64GB. That kind of quality custom engineering *costs*, especially in small volumes. Anyone who's been in the computer business, or the movie business, for more than five minutes knows this, or should do.

With the RED ONE, we had choices: CF cards, RED DRIVEs, and RED RAM. Each with their own upsides and downsides, cost vs. capacity vs. performance. You pays your money, you makes your choice. With the DSMC, RED give the appearance of having taken an 'our way or the highway' approach: the finest, highest performance SSD mags money can buy, or nothing. Aut imperator aut nihil. Even Jim himself has disavowed the notion of a cheaper lower spec SSD just for Scarlet, on the slightly spurious grounds they might get confused with EPIC SSD. Paint them scarlet instead of black ferchrisakes!

The other approach (the RED DRIVE approach) is to use cheap off the shelf components - drives and RAID controller - and put together a device that can, *in toto*, meet the real time performance requirements you so accurately describe. Sure, the EPIC needs more bandwidth than the RED ONE. But a $900 640GB device could be put together for the RED ONE two or three years ago(1); I find it hard to believe that a similar device at a similar cost/capacity price point couldn't be put together for EPIC and especially Scarlet in 2011-2012. People seem much interested in recording Prores to external devices; give them the option of recording raw 4K at comparable cost; RED DRIVE shows it can be done. Do it again with cheaper SSD instead of spinning media.

(1) and the very fact that RED *did* sell it for $900 shows me that there isn't any price gouging going on at RED when it comes to media; quite the contrary. Heck they've even been generous enough to offer a trade-in program. So let's put THAT argument to bed once and for all. And they probably lost money on warranty work related to spinning media failures, hence the aversion to spinning media and the lust for SSD!

Mike

 

[Richard Goodwin]

Richard,

The key thing isn't the performance of the individual drive (of whatever type), it's the system as a whole - the pipeline.

RED have gone, exclusively, for the 'single drive' solution; an expensive custom drive & controller, made to the highest standards of performance and quality.

It's my opinion that this may constitute an error. But that isn't the only way to tackle the 'real time pipeline' problem. Consider the RED DRIVE: totally standard off-the-shelf spinning media, a standard RAID chip, and the gubbins to make it all work together and sit in a small pretty box. Result: $900 for 640GB.

...
Mike

You are treating the SSD drive as if it were a single chip which can only be read/written to one location at a time. Any FLASH drive of substantial storage size is usually made of up multiple chips; reads/writes are multiplexed among those. I don't know exactly what RED does specifically, and while it is not RAID in the formal sense it is also not equivalent of a single drive head reading and writing to a single platter. Add to that multi-plexing to the elimination of moving heads, some level of buffering/caching etc and the fact that this is solid state and you get signification increases. (look at the photos of any flash 'drive' and you see numbers of chips on the drives.

Now the original RED drives are simply mirrored RAID so optimally you are getting double the write performance and that is it. You have only one process writing to the drives after all (writing the incoming digital frames). So, you are limited by the rotational speed of the platters. That currently maxes out at 15K RPM and that is it. It is unlikely to get much faster; maybe a bit, but flash storage and data pipelining, across many drives for larger systems are the techniques used to increase performance. But with two drives in RAID 0 configuration you have a max throughput for a single write stream that cannot be gone beyond. IE: you have a data rate ceiling that cannot be crossed with any method. (yes you can buffer in RAM but ultimately that is a limited buffer so the formula is the same).

While yes a $900 devices could be put together to support the RED One a few years ago the EPIC has even higher data needs, and future RED models only increase those data needs. While in theory RED could develop different tiers of SSD that would be a tall order for a company that is focused on image making (not storage). It would mean developing multiple levels of controllers and pipelining methods to support different data ceilings. And, you have to remember something else, lower cost media and controllers don't just have poorer sustained performance, they have higher error rates.

The other notion you need to look at is "off the shelf". RED is using off the shelf components at the core. Nothing against RED but they don't have the facility to develop a from scratch real time operating system or FLASH drive controller. If you look at any hard drive the amount of software loaded on there is in the Megabytes (yes megs). The software and algorithms to manage drives, hard drives or FLASH, is considerable and takes considerable expertise. That is why even the likes of HP or IBM generally is just repackaging drives and controllers from other companies. And, it is why there are so few companies in the drive business. You can fab chips, platters, and heads all day. But, to design controllers and control software is very difficult.

And, again if companies could develop cheap drives that could handle the sustained throughput and data rates a RED requires do you think they would not be doing it? FLASH/SSD is the only high growth area in terms of profits for storage companies. Companies with high amounts of data would buy them in huge quantities of they were affordable. But, they just are not yet. That is why Google, MicroSoft, etc all still use regular Hard Drives for the majority of their data centers. SSD's are reserved for specific areas needing high performance.

To ask a small company like RED, who's true focus is sensors and image processing to manage dual product lines for storage is just not realistic. The management of duplicate manufacturing would be expensive. The Engineering talent required to make truly custom controllers would be very expensive and is in such high demand that it would be very hard to hire for. Lower end components, be they spinning drives or cheaper FLASH, simply will not meet the requirements of current and future systems.

In short I think you and those who are asking RED to have a sub-focus on making slightly cheaper storage media are asking RED to divert energy away from their core business and expertise in order to provide a potential small savings that is likely not possible. Could it be that its time to examine the larger technical issues and market economics and just admit that RED really has little room to do anything else than it is doing on the the storage front?

 

[Bruce Allen]

I kinda get it, but you don't fully get it.

It's not just about maximum sustained data rates, it's about *glitch and dropout-free sustained writing*. It doesn't matter if the drive can normally sustain 50, 250 or 500 MB/s for an hour at a time, if the firmware or basic drive design permits or mandates the occasional random or even regular brief drop in speed, or total dropout, for a second or two, for some kind of self-test or internal housekeeping; in almost any other application that would tolerable: 'device busy' is an acceptable response. In digital cinema with the unremitting real-time data coming in hot off the sensor during a take, you are DEAD if that happens.

It's way more complex than 55 vs. 250. Many drives that could easily sustain 55 would have misbehavior of the type I described above, or other reliability/repeatability issues.

Mike

I am aware of the requirements of glitch-free, dropout-free writing, Mike.
But are you aware of how trivial it is to write 55MB/sec sustained? Not > 200MB/sec. That is on the bleeding edge.

55MB/sec is easy.

Sure the drive will have some brief drops in speed. That is what buffers are for.

With > 200 MB, that would present a problem even with buffering. Easy to get behind and never catch up.
With 55 MB/sec... ARE YOU JOKING??

Again, my hope is just that RED plans secretly to upgrade the specs, maybe with Dragon, and they want SSDs to be future-proof for that. That's cool, I support that.

But please don't try to argue that 55MB/sec write is a tough thing to sustain. I don't think that that is a very defensible position.

Bruce Allen
www.boacinema.com

 

[Deanan]

I am aware of the requirements of glitch-free, dropout-free writing, Mike.
But are you aware of how trivial it is to write 55MB/sec sustained? Not > 200MB/sec. That is on the bleeding edge.

55MB/sec is easy.

Sure the drive will have some brief drops in speed. That is what buffers are for.

With > 200 MB, that would present a problem even with buffering. Easy to get behind and never catch up.
With 55 MB/sec... ARE YOU JOKING??

Again, my hope is just that RED plans secretly to upgrade the specs, maybe with Dragon, and they want SSDs to be future-proof for that. That's cool, I support that.

But please don't try to argue that 55MB/sec write is a tough thing to sustain. I don't think that that is a very defensible position.

Bruce Allen
www.boacinema.com

Are you assuming that cameras have gigabytes of memory like a computer does to buffer?
That would be a very incorrect assumption.

 

[Dan Kanes]

maybe something to add in the future?

 

[Maxi Claudio]

The big pressure is up till know, we have Epic-M, X users needing expensive media. not a problem. Very shortly there will be a ton of Scarlet $10K users needing media. I can see quite a profit center for a Chinese company, to put out some 128gb SSDs for Scarlet for say $399 each. They will sell a ton.

 

[Bruce Allen]

Are you assuming that cameras have gigabytes of memory like a computer does to buffer?
That would be a very incorrect assumption.

Sorry, Deanan - maybe I am missing something vital. But:

Blackmagic HyperDeck Shuttle is a small $330 battery-powered device that records uncompressed 1080p 4:2:2 10bit to SSDs at up to 30fps.

The data rate of uncompressed 1080p 4:2:2 10 bit 30fps is approx 165MB per second.

They have tested and recommend the following drives:
OCZ 480GB Vertex 3 - $1100
OCZ 240GB Vertex 3 - $390
Crucial C300 256GB - $490

A $330 device can write 165MB per second video to a $490 256GB drive...

Scarlet is a $10,000 device that requires a $3,200 256GB drive in order to write 55MB per second?

Now I understand that for the most part, 1080p 4:2:2 10bit video is highly compressible. But I am sure Blackmagic tested it by feeding difficult to compress stuff (random color noise, etc) through the system.

I suspect that if you encapsulated REDCODE into YUV data and sent it out via the HD-SDI port, the Shuttle with the cheap (but certified) SSD would record it just fine?

Bruce Allen
www.boacinema.com

 

[Chris Jordan]

been asked before i'm sure but will the red drives work with the scarlet? Never used one so I don't know if the are fast enough.