Scarlet 3k Recording Costs

[Graham_C]

I find threads like this extremely interesting indeed,a new storage medium that has made a recently promised but yet to be released 64 gb card allready redundant,BEFORE it has been released.. WHAT!!
That`s crazy!

RED you are VERY excruciating,but in a wonderfull way,thank you..

 

[Ryan Farnes]

I too thank you RED.

 

[Cedric Akins]

Honestly I just hoped Red would have made a 2.5" drive module. I could use the SSDs I have as of now. I have four 32GB drives and only use them for external storage and not the raid I originally bought them for.

 

[Lee Saxon]

I think the 64GB CF cards might have been more relevant a few Scarlet release dates ago.

Now that the flash memory process shrink to 25nm is just around the bend, data density is going to go up significantly and $/GB should go down significantly (some are saying by half, but I doubt it).

That may mean a 128GB or higher card at a sane price is now a realistic possibility.

 

[Jesper Sichlau]

I wonder how this will affect the Fixed Scarlet-package deal. I can't imagine the price staying at the point it is now.

 

[Steve Wake]

The RED SSD mag will be able to handle 1-2 hours of 3k no problem.

RED, Please make the SSD mag module standard on Fixed Scarlet, or at least an option. I want Fixed to be my all-purpose camera, and since I sometimes shoot long 1hr-plus interviews with everyday folks (who speak in stories - not sound bites) I need the storage. I agree you should just fuhgeddabout CF.

SSD prices are dropping while write speed is increasing. I just bought two 64GB SSDs from OWC for 99.95 each on special. Read and write speeds are BOTH over 260MB/sec ! (The special ended already, but prices are still reduced). Prices for larger SSDs increase geometrically from there, but it just goes to show the potential for increased economy in this still-young technology.

So RED, request number two: sell or let others sell a magazine adapter that will take anyone's SSD and allow it to be plugged into your SSD module. (For anyone who missed it, RED found that the standard connectors on SSDs are not robust enough for a lifetime of insertion/removal cycles - a totally reasonable finding). I PROMISE I will not blame RED for any problems that arise from my decision to use someone else's SSD - just let me make that choice.

And while I'm making requests: #3 would be to make a weathersealed cover to flip over the SSD mag after it's inserted in the module.

 

[Mike Gross]

SSD prices are dropping while write speed is increasing. I just bought two 64GB SSDs from OWC for 99.95 each on special. Read and write speeds are BOTH over 260MB/sec !

I bought those too. They do horribly with any form of compressed data like h264, AVC, Redcode, mp3. Like abysmally slow. They're great for OS drives.

http://techreport.com/articles.x/18864/5

 

[Erich Ocean]

I bought those too. They do horribly with any form of compressed data like h264, AVC, Redcode, mp3. Like abysmally slow.

Disk hardware cannot tell the difference between "compressed data" and any other kind of data, so I find it incredible that you found the disks perform differently based on the bit-patterns being written to it. Can you provide more details on why you think your SSD is slow writing the bit-patterns found in compressed data?

 

[Aaron Newsome]

Disk hardware cannot tell the difference between "compressed data" and any other kind of data, so I find it incredible that you found the disks perform differently based on the bit-patterns being written to it. Can you provide more details on why you think your SSD is slow writing the bit-patterns found in compressed data?

Erich, I too have done a LOT of testing with SSD drives. Manufacturer claims mean absolutely nothing when it comes to writing compressed data like video. You are correct that standard disk drives don't care what data is being written but SSD are a whole different ball game. SSD controllers do things like wear leveling and write minimizing schemes to increase the life of the drive.

Do your own tests and you'll find this to be true. I'd love to see a $99 SSD that can sustain video data writes at 260mb/s. I'll say it doesn't exist. I only know of 1 ssd drive that can sustain video data writes at that speed at all.

Read Tom's hardware review of the ocz vertex 2 pro. They found the drive can sustain near e-sata limit on writes in their testing but when writing compressed data like video, cut the throughput in half or worse. That's going to be pretty common with SSD drives, especially ones that use that controller.

 

[Erich Ocean]

Read Tom's hardware review of the ocz vertex 2 pro. They found the drive can sustain near e-sata limit on writes in their testing but when writing compressed data like video, cut the throughput in half or worse. That's going to be pretty common with SSD drives, especially ones that use that controller.

Okay, I read the review and here's the money quote:

Because we are working with a compressed file, this could make the Vertex 2 Pro look slower than it could be in other circumstances. The SandForce SF-1500 uses some kind of compression to avoid using DRAM cache, and thus compressed files could actually slow the Vertex 2 Pro down. Regardless, it was still the fastest drive we tested here.

So basically, to save money, the drives are using less DRAM in their battery-backed cache. They're compressing incoming data on the fly before writing it to the DRAM cache. Already compressed data is difficult to compress further (and sometimes even gets bigger), so they're basically DRAM-buffer limited.

This is different than the actual SSD storage physically being "slower" on compressed data; it has nothing to do with that, or write leveling, or whatever -- it has to do with the DRAM battery backed buffer and the compression engine they're using to write to it effectively "throttling" write speeds on compressed data. In a nutshell, the solid state part can handle it, the rest of the system (controller, DRAM cache, etc.) is under-provisioned and only able to reach advertised speeds when they can cheat by compressing compressible files.

Good to know, and thanks to Mike and Aaron for making us aware of it.

 

[Subhadip Sen]

Back to the original topic, I don't think there's anything out there to suggest that Scarlet 2/3" Fixed will not ship with the CF Module as planned. I hope we will have plenty of options to select bitrates. Considering how well REDCODE 28 performs for 4K in its current iteration, I have no doubt that improved REDCODE will be just fine with lower bitrates (than 28) for 3K. Jim has mentioned previously (perhaps jokingly) that DSLR compression is more like REDCODE 2. 64GB CF cards are available today for as little as $200. I am not sure what kind of bitrates they can sustain, though. Sandisk Extreme 32 GB costs $160, and works just fine for REDCODE 42. Let's assume we use REDCODE 28 - the $160 card will safely record approximately 18 minutes.

Assuming 2.5:1 compression ratio for mathematically lossless, REDCODE 84 is lossless. Considering that, even REDCODE 42 sounds like overkill. Of course, this changes dramatically if the Scarlet records 16-bit and greater than 24 fps.

There are still a lot of unknowns, to be sure. It is also likely several Interchangeable users will opt for the SSD module instead. I just hope it is compatible with off-the-shelf SSDs.

As for EPIC, SSD is definitely going to be the new standard.

 

[Chuck Z]

I hope Red develops a new REDCODE terminology as follows:

REDCODE 100 is lossless.
REDCODE n is n% of mathematically lossless or a quality relative to lossless.

If REDCODE 50 is almost visually lossless and it's half the data rate of true mathematically lossless (REDCODE 100), users can get a good feel of at what data rate they need to shoot a project. If the subject matter is spatially and temporally complex, they can choose a higher level of quality (e.g. REDCODE 75) or go true lossless if they need to do so (REDCODE 100). If you're shooting a time lapse, you might be able to get away with REDCODE 25 or so. (There may be a new compression scheme altogether for the aforementioned given the new DSMC focus.)

Red needs to make this sort of technical science easier for most users to understand.

 

[Gerald Menclik]

So basically, to save money, the drives are using less DRAM in their battery-backed cache. They're compressing incoming data on the fly before writing it to the DRAM cache. Already compressed data is difficult to compress further (and sometimes even gets bigger), so they're basically DRAM-buffer limited.

This is different than the actual SSD storage physically being "slower" on compressed data; it has nothing to do with that, or write leveling, or whatever -- it has to do with the DRAM battery backed buffer and the compression engine they're using to write to it effectively "throttling" write speeds on compressed data. In a nutshell, the solid state part can handle it, the rest of the system (controller, DRAM cache, etc.) is under-provisioned and only able to reach advertised speeds when they can cheat by compressing compressible files.

Good to know, and thanks to Mike and Aaron for making us aware of it.

I'm not the tec guy, but isn't it true that ssd's lose their capacity each time you record something to it? Not in a big amount, but .....:undecided:

 

[PatrickW]

I'm not the tec guy, but isn't it true that ssd's lose their capacity each time you record something to it? Not in a big amount, but .....:undecided:

Here is an article that explains the basics of SSD technology quite well.
http://www.anandtech.com/show/2738/31

SSD have more memory chips in them that required (ie if you buy a 64Gig SSD, it really is 68Gig) which gives it some extra to "replace" broken parts. However because they have no moving parts, failure is rare, and will not be brought about catastrophically by being used in a bumpy environment.

 

[Stuart English]

REDCODE

REDCODE

I hope Red develops a new REDCODE terminology as follows:

REDCODE 100 is lossless.
REDCODE n is n% of mathematically lossless or a quality relative to lossless.

If REDCODE 50 is almost visually lossless and it's half the data rate of true mathematically lossless (REDCODE 100), users can get a good feel of at what data rate they need to shoot a project. If the subject matter is spatially and temporally complex, they can choose a higher level of quality (e.g. REDCODE 75) or go true lossless if they need to do so (REDCODE 100). If you're shooting a time lapse, you might be able to get away with REDCODE 25 or so. (There may be a new compression scheme altogether for the aforementioned given the new DSMC focus.)

Red needs to make this sort of technical science easier for most users to understand.

Don't worry too much about this. REDCODE is a variable bit rate codec. i.e. you are setting the peak data rate, not the actual data rate.

A simple rule is: the higher the RC value, the better the quality in complex scenes. What quality is acceptable to you is a matter of taste.

For me, on a RED ONE it's RC36, but for others it's RC28 or RC42 - it often depends on your delivery screen size.

 

[Steve Wake]

Here is an article that explains the basics of SSD technology quite well.
http://www.anandtech.com/show/2738/31

I'm not an engineer either (but I do teach science). SSD'd are a young technology and change is rapid. We're in a very steep part of the evolution curve. The DRAM cache mentioned in an earlier post is not even used in many current generation SSD's. Some manufacturers over-provision (build in "extra" GB's) drives by 7% or for enterprise-class drives 28%. Price per GB a little over a year ago was over $10/GB. Now it's about 2 to $4/GB.

What's true in that year-old article is that write speeds can vary dramatically depending on the match between SSD architecture and the application. What's much less true today - just a year later - is that we have to live with huge discrepancies between read and write speeds and that speed must deteriorate rapidly over time. Windows 7 even has a built in utility to clean "dirty" NAND and restore capacity. Everyone is developing and fighting for niches in this new stage (in evolution we would call this adaptive radiation).

All the more reason why I would want to be able to pop newer, better, and most likely cheaper SSDs into a RED magazine for use in their SSD module. I'm sure RED will spec SSDs for RED-branded magazines that are state of the art for use in RED. I'm also sure that others in this forum and elsewhere will be capable of suggesting off-the-shelf SSD's that have similar performance characteristics; or better, given a few month's time.

 

[David Rasberry]

There are quite a few new storage technologies being developed,
magneto-resistive ram, ram based on electron spin states, holographic cubes, nano ram.
Hopefully some of these will overcome the limitations of flash ram.
I like the holographic optical ram being developed for NASA. An SSD sized module could easily store Terabytes with potential read/write speeds in the 200Mb to 5Gb per second range.

http://esto.nasa.gov/conferences/estc-2002/Papers/A7P4(Chao).pdf