I just picked up a GTX 970 to add to this project and am contemplating where to place it in the computer. I have three PCIe slots I could place it in on my Asus Sabertooth Z97 board. The top two are PCIe 3.0 x16 and the bottom is PCIe 2.0 x16. The top slot has a GTX 980Ti in it and I was thinking about placing the new 970 in the bottom slot to give the cards a little more "breathing room" but I don't know how much of a difference placing it in a PCIe 2.0 slot makes.

Both cards are EVGA with the ACX 2.0 cooling which means they're dumping heat into the case, but I have a rather large case with really good airflow (Phanteks Enthoo Primo) so I don't know how much of a problem it will be with a second card. For reference, my 980Ti nearly always stays under 60C on GPUGrid with the default fan curve.

So it comes down to, is it better to place both cards in PCIe 3.0 slots and have them run warmer, or to skip a slot and have one card in a PCIe 2.0 slot?

Thanks for any advice.

PCIe 2 slot will show no noticeable difference.

PCIe 2 slot will show no noticeable difference.

Okay. Thanks for the reply!

I just picked up a GTX 970 to add to this project and am contemplating where to place it in the computer. I have three PCIe slots I could place it in on my Asus Sabertooth Z97 board. The top two are PCIe 3.0 x16 and the bottom is PCIe 2.0 x16.

The bottom slot is mechanically x16, but electrically it's only x4, so it's 8 times slower than the top slot.

The top slot has a GTX 980Ti in it and I was thinking about placing the new 970 in the bottom slot to give the cards a little more "breathing room" but I don't know how much of a difference placing it in a PCIe 2.0 slot makes.

It depends on the workunit type. For example the recent GERARD_A2AR workunits will be significantly slower in that slot, the others won't.

Both cards are EVGA with the ACX 2.0 cooling which means they're dumping heat into the case, but I have a rather large case with really good airflow (Phanteks Enthoo Primo) so I don't know how much of a problem it will be with a second card. For reference, my 980Ti nearly always stays under 60C on GPUGrid with the default fan curve.

For these cards a fan placed on the side plate blowing the hot air outwards is recommended.

So it comes down to, is it better to place both cards in PCIe 3.0 slots and have them run warmer, or to skip a slot and have one card in a PCIe 2.0 slot?

I would place it in the lowest slot to give breathing room for the 980Ti, regardless of the performance decrease it could bring. Moreover if you put the 2nd card in the 2nd PCIe3.0 slot, then both will run at only x8, reducing the PCIe bandwith for the GTX980Ti, which should be avoided.

Thanks, Retvari. I was kind of hoping you would see this as you always give good detailed replies.

I'll put it in the bottom slot and see how it performs.

Adding a fan to the side panel isn't an option on this case, unfortunately, but there are two bottom-mounted fans which direct cool air up toward the GPUs. Hopefully that won't send too much hot air up to the 980Ti from the 970. I'm doubtful that reversing the fans to create an exhaust on the bottom would provide much benefit.

I was just thinking, another option might be to swap some cards around between rigs. I have another machine with two 750Ti cards. I could mix them up so that one case has 980Ti + 750Ti and the other with 970 + 750Ti. I could then put the 970 in a PCIe 3.0 slot in the other machine and put the 750Ti in the second slot on both machines as they don't put out much heat. That way, all 4 cards would be in PCIe 3.0 slots and the heat management may not be as much of a concern.

I would put the GTX 750Ti to the PCIe 2.0 slot as well. Your hosts have OSes with WDDM, and the PCIe bandwidth limitation has lower impact on GPU performance than WDDM, so there's no point giving a mid-range card (like the GTX 750Ti) a PCIe3.0x8 slot and limit the airflow of a high-end card and limit its PCIe bandwidth to x8 at the same time.

So the CPU's 16 PCIe lanes don't divide up evenly between the 3.0 and 2.0 slots? I like your idea if that is the case.

The 970 won't arrive for a few days yet. I'll try that configuration with the 980Ti and the 970 occupying the top slot in each machine with a 750Ti in the bottom slot of each.

Thanks for the feedback.

So the CPU's 16 PCIe lanes don't divide up evenly between the 3.0 and 2.0 slots? I like your idea if that is the case.

No, the PCIe3.0x16 comes directly from the CPU, and divided between the two PCIe3.0 slots if necessary (when there is a GPU in the 2nd PCIe3.0 slot), while the PCIe2.0 comes from the "South Bridge" (Z97) chip.

So the CPU's 16 PCIe lanes don't divide up evenly between the 3.0 and 2.0 slots? I like your idea if that is the case.

No, the PCIe3.0x16 comes directly from the CPU, and divided between the two PCIe3.0 slots if necessary (when there is a GPU in the 2nd PCIe3.0 slot), while the PCIe2.0 comes from the "South Bridge" (Z97) chip.

Okay. Thank you for clearing that up. Always learning here.

The 970 won't arrive for a few days yet. I'll try that configuration with the 980Ti and the 970 occupying the top slot in each machine with a 750Ti in the bottom slot of each.

In support of this setup I've been looking at my app_config and trying to figure out how to adjust it such that the top card (980Ti/970) will run two tasks at once and the bottom card (750Ti) only run one task. Is this possible? I'm not seeing any tags on the BOINC Client Config guide that would apply to this.

The 970 won't arrive for a few days yet. I'll try that configuration with the 980Ti and the 970 occupying the top slot in each machine with a 750Ti in the bottom slot of each.

In support of this setup I've been looking at my app_config and trying to figure out how to adjust it such that the top card (980Ti/970) will run two tasks at once and the bottom card (750Ti) only run one task. Is this possible? I'm not seeing any tags on the BOINC Client Config guide that would apply to this.

You can't do it, with an app_config.xml file. I was in a similar situation to you a while back, where I had 3 GPUs, but only 1 beefy GPU could support 2-task-per-GPU. I had to just run 1-task-per-GPU, across the 3 GPUs, until I upgraded the GPUs again.

You *MIGHT* be able to do something strange, like setup GPU Exclusions and an app_info.xml file such that the beefy GPU runs one app at 2-tasks-per-GPU, while the non-beefy GPU runs another app at 1-task-per-GPU... but I don't recommend it, for 2 reasons. First, GPUGrid barely has tasks for 1 app, and it's likely a GPU would starve if you limited to a certain app type. Second, work fetch isn't really designed to handle this scenario. The client doesn't ask the host for work from a specific app type. Instead, work fetch treats all the GPUs as "NVIDIA" when fetching work, or deciding if work is needed.

Sorry you can't really do what you want. Like I said, I was the same way... until I upgraded my GPUs.

Thanks, Jacob. I had some of those same thoughts and figured it wouldn't work. Thanks for confirming my suspicion.

I'll likely have to go back to 1 at a time across all GPUs. That, or move my 750Ti cards to another project. How does a 970 compare to 2 750Tis on GPUGrid?

How does a 970 compare to 2 750Tis on GPUGrid?

Here's a great post, that describes GPU Grid performance (column 1), along with performance-per-watt (useful for those conserving electricity):
https://www.gpugrid.net/forum_thread.php?id=1150&nowrap=true#41294

In support of this setup I've been looking at my app_config and trying to figure out how to adjust it such that the top card (980Ti/970) will run two tasks at once and the bottom card (750Ti) only run one task. Is this possible? I'm not seeing any tags on the BOINC Client Config guide that would apply to this.

I can imagine only one way to achieve this: to install two separate BOINC managers in separate program & data folders, then set up the 1st to run two workunits on the GPUs and exclude the lesser GPU; then set up the 2nd to exclude the bigger GPU. I've never done it, but theoretically it could work.

In support of this setup I've been looking at my app_config and trying to figure out how to adjust it such that the top card (980Ti/970) will run two tasks at once and the bottom card (750Ti) only run one task. Is this possible? I'm not seeing any tags on the BOINC Client Config guide that would apply to this.

I can imagine only one way to achieve this: to install two separate BOINC managers in separate program & data folders, then set up the 1st to run two workunits on the GPUs and exclude the lesser GPU; then set up the 2nd to exclude the bigger GPU. I've never done it, but theoretically it could work.

That's another thought I had. Probably more than I really want to deal with. I'll set it up for 1x initially and see how it goes. Thanks, everyone.

Just an update on this to illustrate the difference between PCIe 2.0 and 3.0 on this project.

I swapped out my 2 750Tis for 2 960s. One of the 960s is in this host and is placed in the PCIe 2.0 slot with a 980Ti in the top PCIe 3.0 slot. A typical BestUmbrella Long Run task looks like this.

My other 960 is in this host and is the only GPU and so is placed in the top PCIe 3.0 slot. Here is a BestUmbrella task from this host.

The two cards are identical (same vendor, model, clock, etc.) and yet the card in the PCIe 2.0 slot is taking around 50% longer to complete a task as the card in the PCIe 3.0 slot.

At this point, I am considering moving the card in the 2.0 slot up into the other available 3.0 slot in that machine. This means both it and the 980Ti will then run in PCIe 3.0 x8, but I'm guessing that it will still be faster overall than 980Ti in PCIe 3.0 x16 with the 960 in PCIe 2.0. The other option is to move it to my machine which currently has a 970 in it and have those two share the PCIe 3.0 lanes.