HP Z840 or Z820

[Tom Hanmer]

As the title says, looking for some collective wisdom as I do not have much on this subject.

It is finally time for me to upgrade my 2007 HP XW8600. It has done me well but with Avid now going 4K the upgrade seems worthwhile. My primary editor is Avid and to keep things simple I would like to go with one of their qualified machines however I will also be using Redcine, After Effects and Resolve as well. For effects heavy projects I will also be using Premiere due to the simplicity in round tripping between there and AE.


My question is what are the benefits of the more expensive HP Z840 over the Z820? Will I see a significant difference going for the 840 and will it have a greater lifespan? Are there any major changes in computer technology on the horizon which would benefit me going with the cheaper machine now and upgrading sooner?


The machine that I have been advised to buy is:


HP Z840
Dual (2x) Xeon E5-2670v3, 2.3GHz 12 Core Processors
32GB (8x4GB) DDR4 2133MHz ECC Registered RAM
HP 512GB Solid State Drive - SATA 6Gb/s
HP Nvidia Quadro K5200 Professional Workstation Graphics (*K5200 is required for 4K / High Frame Rate)
DVDRW & 15-in-1 Media Card Reader
Windows 7 Pro 64 & Windows 8 Licenses (Windows 7 Installed)


If I want the machine to handle resolve happily I assume I should upgrade the RAM to 64 gb, should I also be thinking about adding another graphics card? I do have a rocket x.


The spec of the Z820 that has been suggested is:


HP Z820
2 x Xeon E5-2650v2 8 core 2.6 8 Core 2.60GHz,
32GB (8x4GB) RAM,
1TB SATA HDD,
Nvidia K4000,
SAS-BACKPNL,
Windows 7 Pro 64


I will be adding a thunderbolt 2 card to whichever machine I choose, are there any other additions or changes in spec that you would go for. As you can tell I try to keep my computers going for as long as possible.
Any advice on this would be great.
Thanks
Tom

 

[David Kuhn]

I've not yet spent a LOT of time reviewing the two platforms... but when the Z840 came out I did a quick review, my first inclination would be that the Z840 made sense when requirements exceeded what you could cram into a Z820. It didn't look like there was a big reason to go with an 840 if you weren't going to leverage the advantages it has over the 820. Though... if you like to expand hardware then it might be another reason (I tend to buy and use as is rather than upgrading). Felt like if the build was >$13k-$15k or so then the Z840 started to really look like the platform of choice - though it was really more about 'could the Z820 do the job' rather than a specific price point.

Z840 looks like a beast, looking forward to playing with one.

 

[PatrickFaith]

I really like the dual v3 10 core chip, and that is only in the 840 i think?

 

[Jeff Kilgroe]

The biggest, and only only real reason to go with the Z840 is the updated chipset and support for DDR4 memory. Otherwise the systems are nearly identical. DDR4 provides a noticeable advantage with faster memory throughput and this helps drive more powerful GPUs as well as facilitates better data transport within the system.

I'm not so sure I agree with either recommended system configuration. First of all, 64GB is an absolute must-have. End of discussion on that front. I would recommend going 128GB if you have more than 16 cores and you intend to truly use all of them.

As for cores, picking your CPUs is a balancing act. On the one hand, more cores becomes beneficial for certain applications like Resolve that can truly use and scale across them. On the other hand, most software does not scale well to many cores and even when they do, there is also an advantage to having the higher clock speeds available on CPUs with fewer cores. Avid is not as readily multithreaded as Resolve or Premiere and Resolve is noticeably better in that department vs. Premiere. For most, I would recommend getting the highest number of cores you can get within budget that also allows you to keep your base clock speed (not the turbo clock speed, which is faster) at 3GHz or higher. IMO, the 2.3GHz base clocked CPUs are detrimental to many video or image processing tasks, especially in applications that don't scale well beyond 12 cores (24 threads) or so. Now when turbo speeds kick in, and you're using fewer cores, the clock rate ramps up higher and this helps a lot. But you need to price out some options and see what else you can come up with. In my own testing, I've found dual 8-core and dual 10-core to be a very nice sweet spot. I have not tested dual 12-core, but dual 14-cores is amazing in Resolve and various 3D rendering apps that can use all those threads, while it becomes somewhat lacking in most other apps.

The K4000 GPU in the Z820 specs you listed is absolutely worthless. Do not buy that GPU. I'm not sure how finicky Avid is these days with GPU hardware, but most of us are seeing far better price/performance ratios in Resolve and Premiere with GeForce series cards. And this usually becomes an aftermarket option with the HP workstations -- so you would order with no GPU and install yourself, takes just a couple minutes. The K5200 is decent, but can't come close to the K6000 or upcoming K6200 or other cheaper cards like the GTX980 and similar. If you need full OpenGL functionality for Maya, XSI, SolidWorks, etc.. then you want the Quadro. If Avid is finicky and they really want you to have the Quadro, then you should probably get a Quadro. Otherwise, I would recommend going a different route. A secondary GPU is fine and dandy, but there are two problems there. First of all, the Z8xx workstations are notoriously bad for getting more than one powerful GPU installed --- they only give us 3 x 6-pin PCIe power connectors. So that translates to one 6-pin and one 8-pin, like you would use on a Quadro K6000 and then you have to pull external power or power from drive bays inside the system to supply a second GPU. Also need to make sure your software supports multiple GPUs. I'm not sure what Avid is providing here. I'm not a big Avid guy... Or at least I have worked with Avid software very little over the past few years. That said, even with apps that support multiple GPUs, and even more so when the majority of software does not, it often pays to spend a bit more on one really good GPU than to get a mediocre GPU now with thoughts of adding a second one later.

There are lots of drive bays in a Z820/840. But don't use the onboard LSI RAID host. It's OK, but the driver support and management tools are terrible, besides there are much better options out there. Get a nice Areca, ATTO, etc.. RAID PCIe card. The Thunderbolt - 2 card is handy too, but it only gives you a single port and you get optional video pass-thru where you connect one of the DisplayPort outputs on your GPU to the TB2 card for when you want to run a monitor off the end of your Thunderbolt chain. Silly really, but the option is there. The HP TB2 card works great for running the common TB2 RAID units from Areca and Promise. But I would go for an internal solution first since you'll have the drive bays. I also recommend the SSD for the OS and applications drive -- it' just makes everything a bit snappier.

I really like the dual v3 10 core chip, and that is only in the 840 i think?

V3 E5 Xeons are Z840 only. V1 and V2 E5 Xeons are Z820. You can get dual 10-core CPUs with either V2 or V3 Xeons, but the V3's in the Z840 give you the faster DDR4 memory.

 

[PatrickFaith]

Thanks Jeff, that is very interesting on the gpu cards, I have gotten confused on that. I mainly decode the r3d in nuke, and do a lot of opencl processing in nuke/Houdini/maya. I noticed that most opencl processing uses a single gpu, so what is the absolutely best opencl gpu irregardless of cost but works with the r3d library and the vendors listed?

 

[John Bellari]

Thanks Jeff, that is very interesting on the gpu cards, I have gotten confused on that. I mainly decode the r3d in nuke, and do a lot of opencl processing in nuke/Houdini/maya. I noticed that most opencl processing uses a single gpu, so what is the absolutely best opencl gpu irregardless of cost but works with the r3d library and the vendors listed?

In theory, could you designate a cheap / lesser $200 Quadro or Firepro GPU with 2GB VRAM simply for 4K 10bit display output duties with proper driver support, and meanwhile use a GTX 980 or R290X for compute in the same system... or is that a previous NVidia only thing when they were pushing dedicated compute cards without display output in combo with a lesser Quadro prior to the Titan releases?

 

[Jeff Kilgroe]

Best OpenCL card? That would currently be a Titan Black or Quadro K6000. The Quadro being more expensive and having more memory with a faster and more robust memory interface. The Titan being clocked faster in addition to the price advantage and often posting superior numbers in computational scenarios.

But hold up there a second. While the nVidia cards are the all around leaders for OpenCL performance, plus they offer CUDA support, there are some functions that ATI cards are better at. R290X cards are actually the performance leaders for certain types of matrix and hash computations -- like BitCoin mining -- if that's your thing. For what typically applies to stuff we see here in R3D land and the CGI or video/ film industry at large, CUDA support is still of great concern and nVidia mostly leads on the OpenCL front too.

Most GPU-accelerated applications don't support multiple GPUs. Resolve supports multiples and does it very well. REDCINE-X supports multiple GPUs, but on current hardware, the GPUs are usually not a bottleneck, so enabling more than one in your settings usually does little to nothing to speed things up. Adobe Premiere makes great use of your GPU, but still only supports a single GPU at this time for the Mercury Engine. Multiple GPUs are supported in CC2014 for rendering.

nVidia first provided a direct means in their CUDA API to allow apps to easily push GPU computational loads or OpenGL, OpenCL stuff off to secondary hardware that was there in addition to the primary display/GUI GPU. Now in apps like RC-X, Resolve, Premiere, etc.. we can select which GPUs we want that application to use.

In the OP's situation, if it is most advisable to go with the Quadro card for Avid, then he could do that and then add a next-generation Titan or something else down the road either as a replacement or as a dedicated computational GPU.

Personally, I'm not a fan of having a low-power GPU for display purposes. First of all, PCIe slots are precious and I don't want to waste them on under-powered hardware. The lower-spec'd GPUs typically perform as such, even for general GUI tasks, video playback, etc.. And many apps still don't allow us to choose which GPU to use. In the past with certain apps like Resolve, a dedicated GUI GPU was necessary, but it's not needed now and actually proves to be a hinderance for the reasons I just listed. The performance hit to also run GUI tasks on your computational GPU in Resolve is minimal. So I would recommend running one to three of the same powerful GPUs on a serious Resolve system.

One other thing to consider and this is a current limitation of both CUDA and OpenCL. When multiple GPUs are supported, memory sizing and thread/core count is held constant for all of them. So if you have a GPU with 1024 cores and 6GB RAM and another GPU with 640 cores and 4GB RAM, that bigger GPU gets "dumbed down" and will only be allowed to use the same number of threads (as they translate to cores) and 4GB of RAM. So in that situation, one would have to test whether it would be better to let both operate as dual 640-core/4GB GPUs, or if it would be better to just utilize all resources of the more powerful GPU.

And like most things around here, this all changes regularly and rapidly. Everything I just posted may not be true in the coming weeks or months...

 

[PatrickFaith]

In theory, could you designate a cheap / lesser $200 Quadro or Firepro GPU with 2GB VRAM simply for 4K 10bit display output duties with proper driver support, and meanwhile use a GTX 980 or R290X for compute in the same system... or is that a previous NVidia only thing when they were pushing dedicated compute cards without display output in combo with a lesser Quadro prior to the Titan releases?

A lot of these licenses run per machine, and cost between 2 to 8 thousand dollars per machine ... so it makes sense to just have fanatically uber machines.

Best OpenCL card? That would currently be a Titan Black or Quadro K6000. The Quadro being more expensive and having more memory with a faster and more robust memory interface. The Titan being clocked faster in addition to the price advantage and often posting superior numbers in computational scenarios.
...
And like most things around here, this all changes regularly and rapidly. Everything I just posted may not be true in the coming weeks or months...

From the OP's software list, basically avid & adobe, I think he can go with a more mellow card. For a more houdini/maya/nuke workstation I like your thinking on getting a single Q K6000, rather then trying to get a low end and high end card in the system (i have also had problems with that). Do you have any idea when the k6200 will be out? My timeframe for a z840 is late march, from what you said sounds like a dual 10 core, 128 gig ram, one k6000 would be a solid system. I'm debating moving to a 10G network, right now i'm cheating with a thunderbolt 2 network that works great on large files connected to T2 raid, but the z840 is making me lean to getting a 10g network though houdini is mainly just cpu and gpu processing.

Don't mean to derail the op's thread, hehe ... thanks for the info and signing out

EDIT: was looking up OpenCL support and if i am going to put down $4k on a gpu I want it to support OpenCL 2, but the amd w9100 looks more the way to go if i want a lot of OpenCL support (which btw is not a option on the hp z840 website). I also don't want to go custom build since i need a lot of software to work with it, so will probably hold off till a w9100 or comparable gpu is available as standard on the pc or mac. Hmmm ... the z840 does have the w7100 which looks pretty nice ... getting a dual one of those for a z840 could be very cool.

 

[Simon Dunne]

When speccing a machine like this, the one thing people forget and are not told until it's too late, is that the standard PSU on the Z820 does not have enough power connectors for all the kit people like us need to throw inside it. I had to order the larger PSU to get the job done (cuda cards etc), so worth looking it to that now, rather than later. Not sure if they've improved the options for power on the Z840 though.

 

[Jeff Kilgroe]

Even with the larger PSU for the Z8x0 workstation, it's still too weak for a lot of GPU configurations. And it's a proprietary form factor PSU, so that doesn't help much. The slot layout and other physical aspects of the internals of the system can further complicate serious GPU configurations. Yeah, it's a pain, and well-known amongst those who keep tabs on the workstation market. Can trip up a lot of new system buyers. At least HP will steer you to the bigger PSU when selecting certain CPU or GPU options on the configuration page. It gets people when they buy base configurations and try to upgrade later.

 

[Ryan Schroeder]

One other thing to consider and this is a current limitation of both CUDA and OpenCL. When multiple GPUs are supported, memory sizing and thread/core count is held constant for all of them. So if you have a GPU with 1024 cores and 6GB RAM and another GPU with 640 cores and 4GB RAM, that bigger GPU gets "dumbed down" and will only be allowed to use the same number of threads (as they translate to cores) and 4GB of RAM. So in that situation, one would have to test whether it would be better to let both operate as dual 640-core/4GB GPUs, or if it would be better to just utilize all resources of the more powerful GPU.

Jeff, what's your source on that? I had plans to use my Titan Black as my main GPU, but then throw in a GTX780 3GB that I had available as well, as a second GPU. If what you're saying is true, though, then I'll have to test if the combination is better than just the Titan... my guess would be barely! I primarily use them for Premiere and Resolve.

 

[PatrickFaith]

Random question on the z840, if I get it with 512 gig of ram, does that take a lot of watts? I have no idea how much watts ram takes nowadays, with the power supply being on the wimpy side (I was thinking about putting all the disk aside from the boot drives on external raid to lower the power load).

 

[Jeff Kilgroe]

Need to adjust that statement a bit as far as CUDA goes. Number of cores dose not seem to be dumbed down now, that was in the past. Memory sizing gets weird and when parallelizing between GPUs in CUDA, developers have the option of accessing each GPU individually and using all of its features, or grouping them. When grouping GPUs, then the software no longer has control over where CUDA instructions and supportive data are sent. In a group, memory allocations are matched, so you will most certainly run into a situation where you can't use all 6GB on your Titan card when combined with the 3GB 780 card. The one aspect that gets dumbed down is support CUDA abilities -- for example, the GTX780 does not have support for multipoint precision or FP64 calculations as does the better Quadro cards or the Titan cards. So if you use the 780 along with the Titan, then run an app that needs FP64 acceleration it will have to exclude the 780 from the CUDA set or it could use both cards and drop FP64 calculations from the list of supported functions.

Without pouring through all of it to find direct info or passages that definitively and simply state this, all the info is harbored at docs.nvidia.com/cuda

OpenCL still has cores (stream processor) and memory mapping restrictions. It's best to just match GPUs when it comes to OpenCL use.

 

[Jeff Kilgroe]

Random question on the z840, if I get it with 512 gig of ram, does that take a lot of watts? I have no idea how much watts ram takes nowadays, with the power supply being on the wimpy side (I was thinking about putting all the disk aside from the boot drives on external raid to lower the power load).

Hey I'm all for building kick-ass systems for sure, but what are you gonna do with 512GB of RAM? OTOH, it would make for a blazing fast immediate cache or scratch storage. And yet, I think I'd "suffer" with PCIe storage for that since It's going to cost you $11K to put 512GB in that system. Surprisingly, the power load isn't terribly huge, but you will need the 1125W 90% PSU to run all that RAM and you probably can't run 512GB plus a full storage compliment, especially with the high-density 8-bay option, combined with dual GPUs. I'd have to go look up numbers to tell you exactly how much power it would take, but the real issue here is the heat.

 

[PatrickFaith]

Hey I'm all for building kick-ass systems for sure, but what are you gonna do with 512GB of RAM? OTOH, it would make for a blazing fast immediate cache or scratch storage. And yet, I think I'd "suffer" with PCIe storage for that since It's going to cost you $11K to put 512GB in that system. Surprisingly, the power load isn't terribly huge, but you will need the 1125W 90% PSU to run all that RAM and you probably can't run 512GB plus a full storage compliment, especially with the high-density 8-bay option, combined with dual GPUs. I'd have to go look up numbers to tell you exactly how much power it would take, but the real issue here is the heat. It's going to generate a lot of heat because those new 32GB DDR4 LRDIMMs like to run hot!

I have 12k by 3k openexr files coming in, then am doing voxel & particles operations in Houdini ... get's really deadly quick. (12k files are nothing more then FF dragon with 2x anamorphic lens that are moved to "square" pixels for processing reasons)

 

[Tom Hanmer]

Patrick, David, Simon and Jeff
Many thanks for the replies to my question, your combined advice helps a lot. I asked an authorised Avid reseller to spec a machine for me and they have obviously only focused on Avid rather than looking at the bigger picture, it is great to be able to check their advice against those with a more rounded picture of the needs of the average Reduser.
Having cross referenced your information with the Avid configuration guidelines it seems as though my most balanced option at this point is to go with the Z840 with dual 10 core 3.1 ghz processors and 128 gb of ram although this amount of ram is not officially supported by Avid. There are only two cards supported by avid in the Z840, the K4200 I have immediately discounted on Jeff’s advice so will instead go with the K5200. Would it be possible to add in a GTX980 or a Titan black for use in Resolve?
In Jeff’s first response to my question he suggests using an Areca, ATTO raid PCIe card would this be for the internal drives or for an external raid. I currently use an Avid Video Raid SR which is directly connected by SAS to the rear of the XW8600. I would still need to use this but would be good to have an internal raid as well would the Areca or Atto enable this?
Will definitely go for the TB2 card as it will enable me to use the Maxx thunder raid mini and rocket x in the echo express 3 that I use for backup with my Macbook with the workstation, I will also be taking Simon’s advice and adding the larger PSU.
If anyone has any comments on this system it would be great to hear.
Thanks
Tom

 

[Ryan Schroeder]

Excellent to know. I'm primarily using CUDA for Resolve, and secondarily for Premiere. Would you know offhand if either of those group GPUs?

I'm not sure if this is the case, but I know that there was quite a while where Premiere would only use the first CUDA card. After that, no dice. So, say, if Premiere only used the first card - the Titan - and Resolve was smart enough to not group cards, it might be best to keep both cards - UNLESS - FP64 calc is something that Resolve can benefit from. All the things I don't know

 

[Mennovandijk]

The HPZ 840 has only 3 x 6pin power cables for graphics cards (At least mine does)
So if you want to add a Titan to the k5200 board (which uses 1x 6 pin cable) you need the correct cables. I think Titan cards use 2x 8 pin cables.
So be aware to ask for the right cables

Another thing is I don't know if you can add a GTX board next to a Quadro board which will cause driver issues?



gr
Menno

 

[Tom Hanmer]

Menno
Thanks for making me aware of that.
Tom

 

[PatrickFaith]

Excellent to know. I'm primarily using CUDA for Resolve, and secondarily for Premiere. Would you know offhand if either of those group GPUs?

I'm not sure if this is the case, but I know that there was quite a while where Premiere would only use the first CUDA card. After that, no dice. So, say, if Premiere only used the first card - the Titan - and Resolve was smart enough to not group cards, it might be best to keep both cards - UNLESS - FP64 calc is something that Resolve can benefit from. All the things I don't know

In my OpenCL 1.2 programming I've noticed a huge bottle neck through CPU memory between two cards, so you often only get a 10% increase in speed on the second card. It's also a bit harder to code across two cards, so you get more crashes. [opencl has this hierarchical shared memory model that is real fast until it hits the cpu memory] For more or less normal things, getting two 980's i think is the way to go, they scream. The new cards with are coming out support OpenCL 2.0 which has more shared memory operations, but i can't see the application vendors cutting over to OpenCL 2.0 till late this year. Also most of the gpu's out cannot support OpenCL 2.0, which then lowers the priority for the app vendors to upgrade their code to 2.0 (which btw isn't easy since it changes the memory model).

Me personally, i always run with two identical cards, and have one card for the screen and the other card for the application/opencl. I find a fast 6 core computer with two 980's will beat a system costing 5x times more - unless the high end system is perfectly built, tuned, with the proper application support.