New Dual Xeon 14 core build

[AndreasOberg]

Hi guys
This is what I have come up with so far:
- Fractal Design Define XL R2 Black Pearl Full Tower Performance XL-ATX Case with USB 3.0 w/o PSU
- Supermicro MBD-X10DAI-O Server Board, Intel C612, S 2011-3 x2, DDR4, SATA III 6Gb/s, PCIe 3.0, Gb LAN, E-ATX
- Intel Xeon E5-2697 V3, LGA 2011-3, Haswell, 14 Core, 2.6GHz Base, 3.6GHz Turbo, 9.6GT/s,
- 128GB (8x16GB) Crucial DDR4 Server Memory, PC4-17000 (2133), 288 Pins, ECC, Registered, CAS 15, 1.2V, CT4K16G4RFD4213
- 12GB Asus GTX TITAN X, PCIe 3.0 (x16), 7010MHz GDDR5, GPU 1000MHz, Boost 1075MHz, Cores 3072, 3x DP 1.2/HDMI2.0/DVI-I DL
-- is this the best titan brand? good if its not too loud.
- 3XS Only Corsair RM1000iW RMi Modular Gold Silent PSU
- maybe not strong enough the power supply?

- 960GB Samsung PM863 2.5" SSD, SATA III 6Gb/s, AHCI, 3-bit MLC V-NAND, 520MB/s Read, 475MB/s Write, 99k/18k IOPS
- 8TB HGST 0F23267 Ultrastar He8 Helium Platform Enterprise HDD, 3.5", SATA III - 6Gb/s, 7200, 128MB Cache, OEM
- 8 Port Areca ARC-1882i Dual Core PCIe 6G SAS RAID card w/2 SFF 8087

Any thoughts, anything I missed? I also looked at some of the Supermicro chassis, looks impressive.
/Andreas

 

[Eric Bowen]

Change the PSU to 1200W or 1300W. 1000W is to low for a Dual Xeon build with Titan X cards. Other than that it looks good. You dont need the Samsung Enterprise drives for the OS though. The 850 Pro series would be fine. The Enterprise are ideal for media drives especially in raid. Asus Geforce cards have the best cooling and compatibility at the moment for 4K and Displayport. The cooling difference is really the difference in metal/density used for the heatsinks. The Asus heatsinks are heavier and handle the thermal dissipation better so far.

 

[M.Halsell]

Dont forget the keyboard and mouse. Equally important.:wink5:

 

[luigivaltulini]

For me i change only case.. SuperChassis 747TG-R1400B-SQ is best...
Andy use only 1 GPU?

 

[AndreasOberg]

Change the PSU to 1200W or 1300W. 1000W is to low for a Dual Xeon build with Titan X cards. Other than that it looks good. You dont need the Samsung Enterprise drives for the OS though. The 850 Pro series would be fine. The Enterprise are ideal for media drives especially in raid. Asus Geforce cards have the best cooling and compatibility at the moment for 4K and Displayport. The cooling difference is really the difference in metal/density used for the heatsinks. The Asus heatsinks are heavier and handle the thermal dissipation better so far.

Great info! I have pumped up PSU. Any one special you would recommend Chieftain?
Good to know about Asus cooling, sounds like a good match.
I talked to Thinktank and they didn't recommend the the 850. Said they had a large number of returns for 850 and 840 so may be worth spending a few hundred pounds extra I'm thinking. They recommended - 800GB Micron M500DC 2.5" SATA 6.0Gb/s Solid State Drive.
/Andreas

 

[AndreasOberg]

For me i change only case.. SuperChassis 747TG-R1400B-SQ is best...
Andy use only 1 GPU?

Hi Luigi a pleasure to see you. Your compute building skill is legendary, haha
<SuperChassis 747TG-R1400B-SQ is best...
Yes I have looked at this chassis as well. Can you explain a bit more what you like with it. IT looks bigger and better built of course. Its a lot more expensive of course.

So far I only output in 2k, I don't have a 2k monitor. I will not be able to grade ok with 1 GPU? Even if I drop to half debayer? I thought I could always pop in another card later, but yes I do expect 1 will be a bit underpowered. Will even 2 GPUs be enough? One thing I would really like to be able to do is Scrub quickly in Red media. Always when I move around the timeline it stutters a lot.

Also regarding SSD. Its best to have a SATA one so I don't "waste" a PCI slot, right?
Thanks for the help
/Andreas

 

[Eric Bowen]

Great info! I have pumped up PSU. Any one special you would recommend Chieftain?
Good to know about Asus cooling, sounds like a good match.
I talked to Thinktank and they didn't recommend the the 850. Said they had a large number of returns for 850 and 840 so may be worth spending a few hundred pounds extra I'm thinking. They recommended - 800GB Micron M500DC 2.5" SATA 6.0Gb/s Solid State Drive.
/Andreas

We have had 1 850 Pro drive bad or fail since they released and that is the most common SSD we have sold since it's release. I believe the person you are talking to is confusing the 850 Evo with the Pro. The Evo drives use TLC NAND instead of the much better MLC NAND. TLC has 1/2 or 1/4 the endurance rating of MLC at best and even lower at lower grade. The Samsung 850 Pro drives have a 10 year warranty and 300TB endurance rating for the 512GB and 1TB. If you still are not comfortable with the 850 Pro then go with the original option you gave. The PM863 is the band new MLC SSD architecture while the M500DC is last years MLC. The performance on those M500DC units wont be what the current Samsung options have.

If you plan to add a 2nd Titan X GPU later then you want bare minimum 1300W PSU and we use the Antec 1300W which are fully modular. A 1500W would really be the better choice that with 8 drives in the system and 2x Titan X GPU's with a Dual Xeon

http://www.antec.com/product.php?id=706573&fid=358&lan=ec

Looking around it seems the Coolermaster 1500W we used is out of stock and maybe out of production. The Corsair PSU is available

 

[luigivaltulini]

Andy,
i have 2 o this mainboard and superchassis for my station Davinci.
1: Very well built, ventilation is designed to support HOT Gpu,Also buy 2 additional fans inside for a total of 4. Make a difference.
2. The motherboards supermicro are designated, for the towers supermicro. Attention on port JF1 !! Use caution if you want use another Tower,make sure that the plug is compatible.
3. The first PCI slot to the left, to be the most important, with this chassis are sure to have 2 PCI-e slot space.
4. The power 1400watt are very good, and I tried to keep up with 3 ati 7990 with dual 18 core xeon under strees in Davinci.All work without problem.
And yep is a bit expansive, but for long time you have a professional Tower

NB: this motherboard is very pretty, also allows overclocking. Remember that if you go to overclock miss the USB3, .. ehehe not panic is normal i hope in next bios relaise And Level 3 in bios is better for overclock
Pay attention to the length of the video card in first slot, because it could get above the slot sata, then you have to buy the cables with plugs to 90 degrees left and 90 degrees right.

Good luck with your beast

 

[AndreasOberg]

We have had 1 850 Pro drive bad or fail since they released and that is the most common SSD we have sold since it's release.
If you plan to add a 2nd Titan X GPU later then you want bare minimum 1300W PSU and we use the Antec 1300W which are fully modular. A 1500W would really be the better choice that with 8 drives in the system and 2x Titan X GPU's with a Dual Xeon
http://www.antec.com/product.php?id=706573&fid=358&lan=ec

Looking around it seems the Coolermaster 1500W we used is out of stock and maybe out of production. The Corsair PSU is available

Thanks a lot for the tip Eric! So Samsung 850 pro is your recommendation since its really cheap and large, right?
I will look into the PSU. I plan on getting 2 titans instead of one.
Cheers!
/Andreas

 

[AndreasOberg]

Andy,
i have 2 o this mainboard and superchassis for my station Davinci.
1: Very well built, ventilation is designed to support HOT Gpu,Also buy 2 additional fans inside for a total of 4. Make a difference.
2. The motherboards supermicro are designated, for the towers supermicro. Attention on port JF1 !! Use caution if you want use another Tower,make sure that the plug is compatible.
3. The first PCI slot to the left, to be the most important, with this chassis are sure to have 2 PCI-e slot space.
4. The power 1400watt are very good, and I tried to keep up with 3 ati 7990 with dual 18 core xeon under strees in Davinci.All work without problem.
And yep is a bit expansive, but for long time you have a professional Tower

NB: this motherboard is very pretty, also allows overclocking. Remember that if you go to overclock miss the USB3, .. ehehe not panic is normal i hope in next bios relaise And Level 3 in bios is better for overclock
Pay attention to the length of the video card in first slot, because it could get above the slot sata, then you have to buy the cables with plugs to 90 degrees left and 90 degrees right.

Good luck with your beast

Hiya Luigi. I'm still deciding about the chassis. One company I talk to use the Fractal design for all so its easier for them to service it.
What would you recommend for cooling the cpus?
I was looking at liquid cooling. Like this one for example:
http://www.scan.co.uk/products/cors...ti-cpu-liquid-cooler-with-corsair-link-for-in
Is it better than air cooling? Good if its not too loud.

I think I will get 2 titans, but then of course need to upgrade Davinci as well to Studio version.
Thanks again for he tips!
/Andreas

 

[Eric Bowen]

Thanks a lot for the tip Eric! So Samsung 850 pro is your recommendation since its really cheap and large, right?
I will look into the PSU. I plan on getting 2 titans instead of one.
Cheers!
/Andreas

The Samsung 850 Pro is more expensive than the 850 Evo but less than the PM863 series. If budget is less a concern then the PM863 has a 1400TB endurance rating versus the 300TB for the 850 Pro. It would be worth the money just not really required for the OS. Yes I would recommend atleast the 850 Pro and the PM863 if ideal configuration are the goals. You can re-purpose the PM863 as a cache or media drive later when the new Samsung Pro drives release in standard SSD package.

Keep in mind last I checked Corsiar uses Sesonic as their PSU ODM. I have never really been a fan of them because the productions are often sporadic with variable quality. We use Corsair PSU's here currently since they have a very quiet series that the cooling fan only spins up when temps require. They are ideal for audio. But we have had higher failure rate than Coolermaster with them. We used Coolermaster for close to 6 to 8 years exclusively. However Coolermaster often has issues keeping stock available in the channel consistently.

 

[AndreasOberg]

Hiya Eric,
Yes I think we can put in a bit extra for hte PM863, the difference is really not that big. Usually I will have some files on the operating system anyway. if there is space sometimes a cache for example.
Good to know about the Corsair. I do like that they are a bit more quiet.
/Andreas

 

[luigivaltulini]

Andy,
all original supermicro.
The motherboard manages the internal ventilation of the tower supermicro, in case of increase of the temperature increases ventilation, everything is automatically adjusted.
The problem of Fractal chassis is the power near to first GPu !!!
This does not happen in the tower supermicro, because everything already be on the top. THIS IS A BIG DIFFERENCE !!
and sorry Andy i have x10DAX not Dai (is same but with overclock ).

 

[AndreasOberg]

Andy,
all original supermicro.
The motherboard manages the internal ventilation of the tower supermicro, in case of increase of the temperature increases ventilation, everything is automatically adjusted.
The problem of Fractal chassis is the power near to first GPu !!!
This does not happen in the tower supermicro, because everything already be on the top. THIS IS A BIG DIFFERENCE !!
and sorry Andy i have x10DAX not Dai (is same but with overclock ).

Do you know how the sound volume is on the supermicro case compared with the Fractal? I talked to scan and they said that it was louder, but I wonder if that is really true, not sure he knew what he was talking about.

Would you recommend the x10DAX instead then. How much have you managed to overclock it? I guess every little matters since the base cores are not so fast.
I see that its almost twice as expensive at 570 pounds instead of 300.
Also what do you use for cooling the CPUs?

/Andreas

 

[Eric Bowen]

Hiya Eric,
Yes I think we can put in a bit extra for hte PM863, the difference is really not that big. Usually I will have some files on the operating system anyway. if there is space sometimes a cache for example.
Good to know about the Corsair. I do like that they are a bit more quiet.
/Andreas

Most dont know this but you want to avoid constant writing and deleting on your OS SSD's. Due to how SSD's use different firmware tricks to avoid the permanent erase write cycles that use endurance charges for the MLC, constant writing and deleting will slow the drive down allot. This is why many Users see their OS SSD's running half the speed in a period of time after they get the new system. The only way to reset an SSD to original factory state and speed is to wipe it. You can do that easily on a media or cache drive but not so on the OS. Now there are utilities I have client's use such as CCleaner that allow you to write 0's to free space on OS drive SSD's but that will use up a charge for all of those cells. It;s not something you want to use more than ever 4 to 6 months. Constant writing and deleting media files will slow the OS SSD well before that time if you do it daily or most days.

 

[AndreasOberg]

I see Eric, that is good to know. Maybe I should just have a smaller OS drive then and only really use it for the os. Do you have software on it as well? I usually have a lot of software installed.
/Andreas

 

[Eric Bowen]

Yes the OS drive would also include any applications. You can write docs, email and normal stuff to the OS drive without slowing it down since those are normally far smaller of a profile which allows the drive firmware to keep up with file deletion. Media files though are much bigger and often push beyond what the firmware can handle. Same with the media cache from Adobe or AE cache.

 

[AndreasOberg]

Ok. Sounds like i may as well get a smaller drive then. Maybe later add an sdd scratch drive but maybe the raid is better as scratch?
Cheers

 

[Eric Bowen]

The raid would be better for cache until you get an SSD drive for it. Leave the Windows Page file on the OS SSD.

 

[AndreasOberg]

The raid would be better for cache until you get an SSD drive for it. Leave the Windows Page file on the OS SSD.

Thanks will do!
Yes I guess for bigger files the Raid will be faster than the SSD. Not sure what the total speed will be but it should be snappy. Hope the Dual Xeons and Titans can deal with the red files in different programs.
I just tried Nuke at work and it wasn't so positive. I could barely run quarter debayer and they have 2x8 Core with 1 Geforce 980. Of course this machine will be much faster, but still.

Eric, do you think these coolers will fit: http://www.scan.co.uk/products/noctua-nh-u12dx-i4-high-performance-intel-xeon-cpu-cooler-120mm
I always liked the Noctua coolers because they are so silent and efficient.
Cheers
/Andreas