EPIC nightmare

[Aleksey Khruslov]

Hello! This is a long story.

We have 4 red epic-x and now 3 of them doesnt work properly and everything happened within 24 hours after firmware upgrade.


Now I will tell you all about it:
1) We had and issue with one of the RED TOUCH on saturday. Some strobes, lines and artifacts. We thought it was because of the hard weather condition - rainy and wet everywhere now in Moscow. We put all equipment and RED TOUCH to dry area for the night till next day.


2) On sunday everything was ok and we shoot with two cameras all day long.


3) On monday we got the same problems with RED TOUCH that we have had on suterday. Late evening we decide to install new firmware by the way - v4.0.18. We install it on two EPICS.


4) Tuesday - we see lines and artifacts on both RED TOUCH. After shooting we put all equpment to absolutely dry and safe place. Next day is day-off and we have to relax little bit.


5) Wednesday is hour day off. On late evening i decide to check the cameras before shooting day and i get shocked. All image on both RED TOUCH on both epics is affected with white noise and its impossible to use them as a monitor. I swap RED TOUCHES, unpack fresh spare RED LCD LEMO cable - nothing works. I pull absolutely 100% working RED TOUCH from another set and plug it on both affected EPICS and it doesnt work. I plug this 100% working RED TOUCH on our 3rd, working camera, and it works. I plug affected RED TOUCHes to our 3rd, working camera, and all i see is a full screen white noise. Also BOMB EVF which worked with those cameras with new firmware - all i see on all the cameras now is a colour squares, something like test chart.


We decide to swap DSMC 1.8" SSD SIDE MODULE from working camera to new one, with a new firmware. When we swap it - we get the same result, there is no image on all 3 RED TOUCHES which we tested.
But when we swap SSD MODULE back, our 3rd working camera become unworking too. I dont see picture with my RED TOUCH which worked few minutes ago! WTF???!!!


We checked HD-SDI - it works. Thanks god!!! We got handles so we can continue to shoot with only 5.6 hd-sdi tvlogic monitors on camera. The image is not affected. But wtf? What happened?! New firmware? Weather? Karma?


So what do i have now: 2 RED TOUCHES that show me white noise on all cameras, 1 BOMB EVF which show me test chart on all cameras, 3 RED EPICs which doesnt want to work with 100% work condition RED TOUCHES and BOMB EVF.


And i have two projects with 4 Epics with 50 shooting days more, till the middle of november.


And guess what? Im in damn RUSSIA! :_-)

Im already contacted tech support and send them logs from ssd's and have to wait a while.

Maybe you guys give some advice?

Thanks!

 

[Gunleik Groven]

Have you been in touch with RED in Moscow?

 

[Aleksey Khruslov]

Have you been in touch with RED in Moscow?

I will contact them on monday. But it looks like they dont have repair service for Red Epic anyway.

 

[Gunleik Groven]

But they may be able to help you out.

I think this is a situation anyone would like to avoid.

Best of luck!

 

[Mike Kushner]

This is really unfortunate and the chances are greater for mishap when hardware and software are released as a "work in progress" with constant firmware upgrades. I understand every company is racing with technology to compete with each other...but long testing periods should be required before new firmware is released.

 

[David Mullen ASC]

It begs the question whether once you begin production, should you stop doing upgrades.

 

[Guy Staley]

Guess what? I had a very similar thing happen after installing the 5.1.14 BETA firmware on my Scarlet last week! After initially installing the BETA firmware, everything worked brilliantly! I was so stoked with all the new enhancements and features. I spent about an hour just combing through the menus and basically just checking everything out. I was a happy camper! At that point I powered down the camera, removed the power supply and went to bed.

The next morning when I powered up the camera, the HDMI output was all garbled with pink lines running through the picture (see attached images), and essentially unusable as a monitor or for navigating menus. I tried several times rebooting the camera using different monitors and cables just to confirm if the problem was with the monitor/cable, or originating in the camera. In each instance I had the same garbled monitor output.

Of course my first instinct was to roll back to the previous Firmware v4.0.18. But unfortunately, when I inserted a REDMAG with the old firmware, the camera would not recognize my REDMAGS! I tried several times with different REDMAGS to no avail. After further attempts, the camera eventually did see the media, but claimed it wasn't a REDMAG! I attempted to reformat the SSDs but each attempt ended in failure (see attached images). I even tried doing a "secure format", which resulted in failure and a bricked SSD!!! Unrecognizable by either the camera OR RED Watchdog!

At this point I contacted RED support and, ultimately, sent them the camera log that was generated immediately after the BETA firmware install. That was last Thursday. Finally, on Wednesday, I heard back from RED and was informed that the camera log contained some errors and they wanted me to send the camera in for evaluation. Thank the stars this camera is still under warranty! The camera is now with RED in Irvine, and I don't suspect I'll hear back from them until next week. They told me 3-5 business days to turn it around.

Now, because of Jay's Kelley's recent thread, ( http://www.reduser.net/forum/showth...p?105665-NEW-Firmware-scare!!&highlight=scare ) I really didn't want to add fuel to flames, so I've been silent about this as my camera "symptoms" didn't really match any of the problems other's have been experiencing. I mean, I really don't know for certain that the BETA firmware is the cause of the problems (could be a coincidence), but after reading about your as well as some others issues after installing the new firmware, I'm starting to believe that the BETA firmware is at the heart of the problem. :frown5:

Serious bummer!

 

[Dominik Muench]

Aleksey did this happen on the latest release or beta build ?

I had a similar issue on a music video hire last thursday where the camera assistant reported problems with the LCD screen...then after a reboot the camera body died and the whole camera wouldn't boot up anymore and I had to send the camera in to RED.

I had the latest stable release on the camera... a few similar problems from other people coming in here on the forum and they are all on the latest beta it seems....I'm starting to wonder if my camera has been upgraded by the production to this beta before their shoot. Waiting for the LOG analysis from the RED technicians at the moment.

 

[D Fuller]

It begs the question whether once you begin production, should you stop doing upgrades.

I think this should not be a question at all. Changing builds during a production is kind of like changing film stocks in the middle of a production: you just shouldn't do it. It casts doubt on all of the testing you might have done for the project. Yes, its's true that the raw files can all be processed the same in post, but all the tools you use to judge exposure are subject to change, as is the behavior of the shutter and a host of other things you ned to count on. If you've tested and worked with a build, just use it until the project is finished.

 

[Marc Wielage]

I think this should not be a question at all. Changing builds during a production is kind of like changing film stocks in the middle of a production: you just shouldn't do it.

This is sadly true. I know of many editors, mixers, and colorists who will not change software until the new software build has been out for at least six months, just to confirm that they've gotten all the bugs out of it.

 

[Nick Fury]

This is sadly true. I know of many editors, mixers, and colorists who will not change software until the new software build has been out for at least six months, just to confirm that they've gotten all the bugs out of it.

Been that way since DOS 3.1...

Thank you very much

Fury

 

[Robert Berger]

Hello! This is a long story.

We have 4 red epic-x and now 3 of them doesnt work properly and everything happened within 24 hours after firmware upgrade.


Now I will tell you all about it:
1) We had and issue with one of the RED TOUCH on saturday. Some strobes, lines and artifacts. We thought it was because of the hard weather condition - rainy and wet everywhere now in Moscow. We put all equipment and RED TOUCH to dry area for the night till next day.


2) On sunday everything was ok and we shoot with two cameras all day long.


3) On monday we got the same problems with RED TOUCH that we have had on suterday. Late evening we decide to install new firmware by the way - v4.0.18. We install it on two EPICS.


4) Tuesday - we see lines and artifacts on both RED TOUCH. After shooting we put all equpment to absolutely dry and safe place. Next day is day-off and we have to relax little bit.


5) Wednesday is hour day off. On late evening i decide to check the cameras before shooting day and i get shocked. All image on both RED TOUCH on both epics is affected with white noise and its impossible to use them as a monitor. I swap RED TOUCHES, unpack fresh spare RED LCD LEMO cable - nothing works. I pull absolutely 100% working RED TOUCH from another set and plug it on both affected EPICS and it doesnt work. I plug this 100% working RED TOUCH on our 3rd, working camera, and it works. I plug affected RED TOUCHes to our 3rd, working camera, and all i see is a full screen white noise. Also BOMB EVF which worked with those cameras with new firmware - all i see on all the cameras now is a colour squares, something like test chart.


We decide to swap DSMC 1.8" SSD SIDE MODULE from working camera to new one, with a new firmware. When we swap it - we get the same result, there is no image on all 3 RED TOUCHES which we tested.
But when we swap SSD MODULE back, our 3rd working camera become unworking too. I dont see picture with my RED TOUCH which worked few minutes ago! WTF???!!!


We checked HD-SDI - it works. Thanks god!!! We got handles so we can continue to shoot with only 5.6 hd-sdi tvlogic monitors on camera. The image is not affected. But wtf? What happened?! New firmware? Weather? Karma?


So what do i have now: 2 RED TOUCHES that show me white noise on all cameras, 1 BOMB EVF which show me test chart on all cameras, 3 RED EPICs which doesnt want to work with 100% work condition RED TOUCHES and BOMB EVF.


And i have two projects with 4 Epics with 50 shooting days more, till the middle of november.


And guess what? Im in damn RUSSIA! :_-)

Im already contacted tech support and send them logs from ssd's and have to wait a while.

Maybe you guys give some advice?

Thanks!

Hi Aleksey, sorry to hear tha you are in trouble! Maybe you can try this. Go to your menual and look for the menu to find " new hardware" Let everything be connected to the Epic like you always have (LCD,+1, fans etc) maybe it will reset your Red LCD? Best of luck ! Robert

 

[Mark Toia]

It begs the question whether once you begin production, should you stop doing upgrades.

I hear you loud and clear...

I don't even use beta's anymore. Only the tried and tested builds.

 

[Jason Comparetto]

It begs the question whether once you begin production, should you stop doing upgrades.

I rarely do upgrades, and I would never do an upgrade during production or shortly before production either. Given RED's history, that is like playing russian roulette. Does anyone remember the early days of Red One where some builds were amazing but some were quite scary?

 

[Jason Comparetto]

This is sadly true. I know of many editors, mixers, and colorists who will not change software until the new software build has been out for at least six months, just to confirm that they've gotten all the bugs out of it.

I am one of those people. I value stability over having the "latest and greatest".

 

[Gunleik Groven]

I stayed on a build from last summer up until recently with no issues.

Maybe I'll just roll back.

 

[Patrick Tresch]

Does anyone remember the early days of Red One where some builds were amazing but some were quite scary?

Was it like build 14...? ;-)

Pat

 

[Gunleik Groven]

Was it like build 14...? ;-)

Pat

I know I loved 12...

Not sure if audio was enabled, but it was wonderful.

Crashed just under certain conditions, and gave the most lovely images I had seen.

16 was the turningpoint on many levels. But I was fond of the pre-16 builds.

 

[Aleksey Khruslov]

Guilty. Its my fault to upgarde it in such a manner. I will never do this again before the project ends...

Hi Aleksey, sorry to hear tha you are in trouble! Maybe you can try this. Go to your menual and look for the menu to find " new hardware" Let everything be connected to the Epic like you always have (LCD,+1, fans etc) maybe it will reset your Red LCD? Best of luck ! Robert

Thanks Robert! I will try this today. Never did this before.

 

[Michael Lindsay]

It begs the question whether once you begin production, should you stop doing upgrades.

It is an impotant question and my instant response is 'it not only begs but answers!'

but what if the production is frustrated by a problem that an upgrede claims to address? This happens and there is no easy answer...

I believe crews working with beta software has hurt Red's 'proffesional tool' equity in a way that is difficult to mend... I have done it myself for some killer feature now and then (and I really should know better)... I also have gone beta a couple of time just to try and solve a problem in a release build playing up.

I know of 2 'incidents' in the Uk comercial markets early on where production was burnt by camera faults that happneded on beta builds. I don't think these companies ever knowingly used Red camera again.

very stressful

M