Welcome to our community

Be a part of something great, join today!

Log in
  • Hey all, just changed over the backend after 15 years I figured time to give it a bit of an update, its probably gonna be a bit weird for most of you and i am sure there is a few bugs to work out but it should kinda work the same as before... hopefully :)

New Workstation Build - i7 X99, Titan X, Overclocking...

The PC systems do not have to be loud to cool effectively. We use Noctua fans which are very quiet and handle the current heat generation of the X99 systems. They are also available open market.
 
At the risk of a moderate thread jack, I wanted to respond to Richard on the subject of a Mac/PC LAN setup for folks who prefer OS X and only want the PC side for specific tasks.

GigE will be a bottleneck for some operations, however, with some planning it shouldn't be a major issue. One strategy I use, is to make redundant copies of media assets in direct attached storage for both the Mac and PC. Having a back up copy is on the agenda anyway so it's not really extra effort. This way, the monster media files don't have to travel over the network as you go back and forth between Mac and PC operations, just the project files/XML/etc. As an added benefit, each OS can utilize it's native file system (HFS+, NTFS) for faster access.

10GigE is an option, but, IMHO, it's pricey overkill for a small setup. FWIW, I think ThunderBolt networking will be soon supplant GigE and 10GigE for many use cases. If anyone has gone that road with success please share your notes with the RedUser community. If TB2/3 HBAs for PCs with decent chipsets are widely available for under $100 within a year or two, and setup on both Windows and OS X is mostly automated, even independents should be able to create 20Gb/s LANs easily - then Bob's your Uncle.

Cheers - #19
 
PC Fan noise - my Noctua fans run quite quiet. In fact, I was amazed at how quiet my machine was sitting on the desk with two GPU's. If you run any TB Raids larger than 4 disks, then you likely already have a device that is louder. My BMD Ultrastudio, Areca 8050T2 and Sonnet Rack are all louder than my PC case, so ultimately I don't gain much with my nMP being so quiet. These fans run maybe $20 on Amazon or Newegg and are one of a few well balanced performance brands on market.

SM951 - Tried this in an enclosure and on a small ATX build with no issues. I've heard of a few boot issues early on, but machines seem to be taking to them well as long as you make sure your BIOS is setup correctly for your M.2 slot and you have a current gen M.2 slot to begin with for the increased speeds that the SM951 has. I think a lot of the negative flak comes from users expecting an easy 2GB/s from these new sticks and were surprised to see 1.5GB/s real world performance. They've been an option in certain MacBooks, Netbooks and Laptops for a couple months now.

Networking - 10GbE is cheap and really quite easy to integrate even in a small environment. I have the following setup:

- nMP with TB Chassis (SAS HBA, ESata, USB, etc), Areca 8050T2, BMD Ultrastudio, Promise Link2 10GbE
- PC with 10GbE, Decklink, Adaptec SAS Raid HBA, TB2 Card

The PC ingests footage to an internal 16 drive raid via SAS, ESata, USB, etc. That footage is then "online" via 10GbE to the nMP and can be NTFS (PC) or HFS+ (Mac) depending on your preferred workflow and all you need to implement is Paragon or MacDrive for compatibility. The nMP cuts the footage and networks with other Macs including my Laptop during various post stages for photo, graphics, VFX.

Once I'm done cutting, the PC begins doing the heavy lifting which isn't bottled due to the SAS storage and has room for plenty of GPU's, additional scratch volumes and so forth. After finishing, the project is passed along to the Areca as an "online archive" and eventually laid to tape as a spare backup. HFS+ with Paragon is pretty smooth and allows the 8050 Raid to be right at home with any Mac still in case an archived project needs to go mobile yet be read by a PC.

Cost:
Sanlink2 or Sonnet 10GbE for nMP - $500
Solarflare 10GbE for PC - $350 (Small Tree and Atto offer slightly faster/nicer cards at roughly $650-750)
Spool of Cat 6 or 7 - $50

Akitio and Sonnet both make single slot TB PCIe expanders for $300 or less than can house a 10GbE card should you prefer to go the PCIe route.
 
Blair/Mark: many thanks for the comments.
Jason, thanks for posting your workflow ... All very interesting and useful to know how others are doing it in a real environment.

cheers
Richard
 
Jeff Kilgroe said:
Heh. Less than 10 minutes later I have it working... <knock wood>

ASUS needs to seriously address their BIOS issues. I want to say it's complete rubbish, but they're still better than Gigabyte and I don't think anyone could pay me to use an ASRock or MSI motherboard right now. Would also help if UEFI in PC land wasn't so schizophrenic when trying to co-exist with legacy BIOS systems.

FWIW, the TB card is back in slot 2 where it should belong. However, I was able to get it to also work in slot 7. But only if slot 6 is unoccupied since the two slots are apparently linked for lane bandwidth.

Not sure why it's deciding to work now. I reset the BIOS to defaults and then made all the appropriate changes again and after a few boots it came up. Int19 has to be postponed and yeah, the above 4G decoding enabled...
Click to expand...

My setup is now up and running with Windows 8.1 Pro but the Thunderboltex II Dual is driving me crazy haha!

So for info it's with the Rampage V Extreme MB, my current slot configuration is:

1: GTX Titan X SC (PCIe 3.0 x16_1)
2: ---- (PCIe 3.0 x8_2)
3: Thunderbolt EX II Dual (PCIe 2.0 x4_1)
4: GTX Titan X SC (PCIe 3.0 x16/8_3)
5: ---- (PCIe 2.0 x1_1)
6: ---- (PCIe 3.0 x8_4)

I tried to put the TB EX II Dual in slot 2 and 3 but both doesn't seem to work (the Attached Thunderbolt Devices window is empty, I tried with the Pegasus R6 and Apple TB Display). It shows up in the BIOS and I can choose the slot. I tried "Int19 postponed" and "above 4G decoding enabled" as well. I installed Thunderbolt drivers from Intel website (version 2.0.4.54), it was more up-to-date than on Asus website. I didn't install the drivers that come with the CD since I don't have an optical drive, could it be the reason?

@Jeff: Do you have any ideas what could be wrong? Should I keep restarting?

Thank you guys in advance!

PS: I run latest BIOS version 1502
 
Last edited:
You'll never get the Thunderbolt EX/II Dual card to work -- the X99 boards do not currently support it! You must use the single-port EX/II card. ASUS is supposedly going to address this issue at some point, but I could not get any info out of them as to when or even if it definitely will happen. The card is recognized and everything seems to work, but you can't communicate with TB devices attached to it. Bios does not support the different multi-head split TB header on the card. You don't really lose anything by going to the single-port version as the dual port shares the 20Gbps bandwidth between the two TB2 ports. Primary reason to go with the dual port card is if you would like to drive dual TB monitors from the system, but like I said, not a possibility at the moment.
 
Jeff Kilgroe said:
You'll never get the Thunderbolt EX/II Dual card to work -- the X99 boards do not currently support it! You must use the single-port EX/II card. ASUS is supposedly going to address this issue at some point, but I could not get any info out of them as to when or even if it definitely will happen. The card is recognized and everything seems to work, but you can't communicate with TB devices attached to it. Bios does not support the different multi-head split TB header on the card. You don't really lose anything by going to the single-port version as the dual port shares the 20Gbps bandwidth between the two TB2 ports. Primary reason to go with the dual port card is if you would like to drive dual TB monitors from the system, but like I said, not a possibility at the moment.
Click to expand...

I made it working with the DUAL! Thanks to this guy: https://rog.asus.com/forum/showthre...page-V-Extreme-Requirements-How-To-Help/page5
 
Are you sure it fully works with the dual card? Seriously, it's not supported, not unless they quietly slipped in support for it with the last BIOS update on the Rampage V Extreme -- and I suppose that's possible. It is 100% not supported on the X99E-WS and the X99 Deluxe. At least not as of this post with the latest updates.
 
Jeff Kilgroe said:
Are you sure it fully works with the dual card? Seriously, it's not supported, not unless they quietly slipped in support for it with the last BIOS update on the Rampage V Extreme -- and I suppose that's possible. It is 100% not supported on the X99E-WS and the X99 Deluxe. At least not as of this post with the latest updates.
Click to expand...

I can see and use my drives (Pegasus R6) ~400MB/s Read/Write. I'm on the Rampage V Extreme Motherboard. Right before giving up it started working.
 
Jason Osterday said:
(snip)
Networking - 10GbE is cheap and really quite easy to integrate even in a small environment. I have the following setup:
- nMP with TB Chassis (SAS HBA, ESata, USB, etc), Areca 8050T2, BMD Ultrastudio, Promise Link2 10GbE
- PC with 10GbE, Decklink, Adaptec SAS Raid HBA, TB2 Card (snip)

Cost:
Sanlink2 or Sonnet 10GbE for nMP - $500
Solarflare 10GbE for PC - $350 (Small Tree and Atto offer slightly faster/nicer cards at roughly $650-750)
Spool of Cat 6 or 7 - $50
Akitio and Sonnet both make single slot TB PCIe expanders for $300 or less than can house a 10GbE card should you prefer to go the PCIe route.
Click to expand...

Based on these notes I get the impression that a GigE router is not required, so its a peer to peer setup? Does that mean its limited to 2 devices? So it's a grand worth of hardware, and assuming I can install PCIe cards and run cable myself, then it's a couple hundred bucks to hire someone to set up/de-bug the network and I'm rocking? Good to know, thanks Jason and Mark.

Cheers - #19
 
Octave Zangs said:
I can see and use my drives (Pegasus R6) ~400MB/s Read/Write. I'm on the Rampage V Extreme Motherboard. Right before giving up it started working.
Click to expand...

Do you know if thunderbolt networking works with that thunderbolt connector?
Can someone please test if they have a MacBook?

Windows and OS X work?
 
Jeff - were you able to get some benchmarks from your build? I just built my "Version" of your system and I would like to compare. (mine isn't quite as cool)
 
I've run some comparative tests with several of my own projects on this system and on others. And yes, some of the obligatory benchmarks for RC-X transcodes, storage I/O, etc.. I don't have those numbers handy at the moment... I don't place a lot of stock into the various benchmarking tools out there as I think most all of them do a piss-poor job of actually profiling a system or telling us exactly what we have. We know what the GPUs can do and there are countless tests and benchmarks out there for the CPUs as well. That said, if there is something specific you would like to see in order to compare numbers and notes, let me know.

I'm using my system for moderate editing and CG work, some Resolve. It's just an all-around nice workstation. Definitely have too much internal storage in this system with my 6TB HDDs chosen, but I guess I don't have to worry about running out of space. ;)

It's a good 15~20% faster for computational tasks than the 8-core Mac Pro cylinder due to the faster CPU clocking and faster RAM. It's not as powerful as my 20-core E5v2 system I predominantly use for Resolve. It's quite a bit faster than a 16-core E5v1 system, like my last Z820 I'm still running. However, on software that can utilize the additional cores, that Z820 system can still hold its own just fine and those systems can be picked up at steep discounts right now because people think they're outdated or something. Heh.

At times I think I could have served myself a bit better by using a 14-core Xeon CPU here with 128GB RAM. Perhaps that's the configuration I will build next. With that, my RAM would not feel as cramped for larger data work or 3D projects and the extra cores would be nice. However, I would miss out on some of the clock speed without the overclocking ability and the RAM would be operating almost 800MHz slower too. That makes a difference.
 
Octave Zangs said:
I can see and use my drives (Pegasus R6) ~400MB/s Read/Write. I'm on the Rampage V Extreme Motherboard. Right before giving up it started working.
Click to expand...

Took me a couple days to get an official answer, but yeah it seems the latest release or two of the Rampage V Extreme BIOS is updated to support the Dual card. :) Still no joy on the X99 Deluxe or the X99E-WS motherboards. Perhaps soon. I have an EX/II Dual card here as a spare or for when things get sorted out.

Which Pegasus R6 unit are you running and which RAID configuration? 400MB/s is a bit on the slow side unless you're running the 4-drive model or RAID-6, or have another TB device connected...
 
Jeff Kilgroe said:
Took me a couple days to get an official answer, but yeah it seems the latest release or two of the Rampage V Extreme BIOS is updated to support the Dual card. :) Still no joy on the X99 Deluxe or the X99E-WS motherboards. Perhaps soon. I have an EX/II Dual card here as a spare or for when things get sorted out.

Which Pegasus R6 unit are you running and which RAID configuration? 400MB/s is a bit on the slow side unless you're running the 4-drive model or RAID-6, or have another TB device connected...
Click to expand...

Oh good news, I was lucky haha! I change the Security level to Legacy Mode before that the drive would eject itself after few minutes. I was wondering if you think I could use my Apple Thunderbolt Display with this setup?

The Pegasus 2 R6 is a 6-drive model in RAID-5, I did the benchmark with CrystalDiskMark.

I'm so happy with the workstation, thanks Jeff for all your precious help! It's pretty stable at 4.4Ghz and 2800Mhz for the RAM. Final step is to add the RAID card and those Seagate drives (6x6TB), would you add more fans for it btw (I have the Corsair 750D)?

Last question does Davinci Resolve or Nuke benefit from the 2 Titan X or they only use one?

I'll post some pics soon! Thanks guys!
 
Davinci will scale across multiple GPU's significantly if you have the full version and not the light version. Blackmagic recommends you have atleast 2x video cards.
 
Jeff - Raja over at ASUS confirmed on the ROG forums that they have the Dual working on the R5E boards. He doesn't have an ETA for the other X99 (deluxe, WS, etc) ones though, as you said. I'm very interested in following what the TB3 will look like on Z170 boards (although with the price of DDR4 16GB sticks being what it is, and the limit being 4 cores, I don't see it as a viable replacement for my Z87/Z97 setups, and will likely stick X99 for now, which is annoying given these buggy AICs).
 
You must log in or register to reply here.
Back
Top