Advanced search

Message boards : Graphics cards (GPUs) : Is it possible to run Multiple GPUs in one Tower?

Author Message
Profile AdamYusko
Send message
Joined: 29 Jun 12
Posts: 26
Credit: 21,540,800
RAC: 0
Level
Pro
Scientific publications
watwatwatwatwatwat
Message 28720 - Posted: 24 Feb 2013 | 0:26:02 UTC

I apologized if this has been covered before, but I did attempt to do a search. But I am wondering if it is possible to run multiple GPU's on BOINC in one system.

My set up

-650W PSU

-Intel i5 ivy bridge GPU ( No hyperthreading so only 4 cores for crunching),

-Asus P8B75-V Mother board has 1 PCI 3.0 x16 and one PCI 2.0 x16 @x4 (not quite sure what the last bit means unless it is just specifying its location in the tower).

I currently have a Passively cooled GT 640 GPU in there that is running nice.

I am wondering if it would be possible to add a GTX 680 into my system.


While I do have two systems I already learned that the passively cooled GPU does not fit into my older system, which has the lowest power GPU, and its fairly new so I do not want to not use it.

I am just wondering if there are weird driver considerations, and I run Ubuntu 12.04 and I do not know if that also has the issues where the computer will automatically disable any GPU that does not appear to be in use.

I am not looking to make the purchase immediately, rigth now I am just trying to scope out to see if it is possible.
____________

Dylan
Send message
Joined: 16 Jul 12
Posts: 98
Credit: 386,043,752
RAC: 0
Level
Asp
Scientific publications
watwatwatwatwatwatwat
Message 28723 - Posted: 24 Feb 2013 | 1:47:05 UTC - in response to Message 28720.
Last modified: 24 Feb 2013 | 1:49:26 UTC

I don't know about Ubuntu, but on Windows multiple GPU's can be used. For example, I have two GTX 670's that aren't in SLI mode, but both still run tasks for GPUGRID.

To tell BOINC to use more than one GPU, you have to add a line of text to a certain BOINC config file. The file is the cc_config.xml file. On Windows, it wasn't present already, so I had to make one, and then add the line of text.

Sorry to say, but I have forgotten what this text is, and am hoping someone else may help you here.

As for drivers, I'm pretty sure that the driver for the 640 works with the 680, as they are in the same series, however, this is coming from my experience with Windows and not with Ubuntu, as I only have Windows systems.

Hope this answers something for you.

Check out this thread for more information: Using two different GPUs.

http://www.gpugrid.net/forum_thread.php?id=3113#28722



Also, maybe this thread may help you: HOW TO - Install NVIDIA drivers on Linux 64 bit (generic)


http://www.gpugrid.net/forum_thread.php?id=266#1385

flashawk
Send message
Joined: 18 Jun 12
Posts: 297
Credit: 3,572,627,986
RAC: 0
Level
Arg
Scientific publications
watwatwatwatwatwatwatwatwatwatwatwat
Message 28728 - Posted: 24 Feb 2013 | 4:23:39 UTC - in response to Message 28720.

I apologized if this has been covered before, but I did attempt to do a search. But I am wondering if it is possible to run multiple GPU's on BOINC in one system.

My set up

-650W PSU

-Intel i5 ivy bridge GPU ( No hyperthreading so only 4 cores for crunching),

-Asus P8B75-V Mother board has 1 PCI 3.0 x16 and one PCI 2.0 x16 @x4 (not quite sure what the last bit means unless it is just specifying its location in the tower).

I currently have a Passively cooled GT 640 GPU in there that is running nice.

I am wondering if it would be possible to add a GTX 680 into my system.


While I do have two systems I already learned that the passively cooled GPU does not fit into my older system, which has the lowest power GPU, and its fairly new so I do not want to not use it.

I am just wondering if there are weird driver considerations, and I run Ubuntu 12.04 and I do not know if that also has the issues where the computer will automatically disable any GPU that does not appear to be in use.

I am not looking to make the purchase immediately, rigth now I am just trying to scope out to see if it is possible.


Yes you can, I have a GTX680 in the same box as a GTX560 and it is reported as 2 680's in my account. If I'm around, I like to have the 680 finish the jobs the 560 has been working on (as soon as it finishes it's own wu) makes things go much faster. You really don't want like cards in SLI mode as it can slow the work down considerably.

Profile AdamYusko
Send message
Joined: 29 Jun 12
Posts: 26
Credit: 21,540,800
RAC: 0
Level
Pro
Scientific publications
watwatwatwatwatwat
Message 28729 - Posted: 24 Feb 2013 | 4:29:13 UTC - in response to Message 28728.

Alright then my follow up question would be I assume I would want the 680 in the PCI 3.0 so it has the best transfer speeds, and to move the 640 to the PCI 2.0?

Though just to be sure Ubuntu is able to keep the drivers straight? Or they will work just fine on the same driver?

Also there is the question do I need to do anything special to keep one of the GPU's from shutting off every time a task finishes?
____________

flashawk
Send message
Joined: 18 Jun 12
Posts: 297
Credit: 3,572,627,986
RAC: 0
Level
Arg
Scientific publications
watwatwatwatwatwatwatwatwatwatwatwat
Message 28730 - Posted: 24 Feb 2013 | 4:39:22 UTC

You can use the same driver for both cards. GPU's require a CPU to feed them, it's set up that way through hardware and the motherboard BIOS, most will remove 1 CPU core from BOINC for every GPU their running, the video card will find and use the free core automatically. As far as GPU-GRID is concerned, it doesn't use enough bandwidth to saturate a PCIe 2.0 bus so either way would work.

Dylan
Send message
Joined: 16 Jul 12
Posts: 98
Credit: 386,043,752
RAC: 0
Level
Asp
Scientific publications
watwatwatwatwatwatwat
Message 28731 - Posted: 24 Feb 2013 | 4:50:42 UTC

I can answer your first and last questions.

Alright then my follow up question would be I assume I would want the 680 in the PCI 3.0 so it has the best transfer speeds, and to move the 640 to the PCI 2.0?


Yes, you would want to do this in situations not crunching related. The difference between PCI 3 and 2 would be noticeable on other GPU projects, but on GPUGRID the difference is very minimal, however, I wouldn't understand why you would want to put the more powerful GPU in a less powerful PCI slot, unless you ran into conflicts or something.



Also there is the question do I need to do anything special to keep one of the GPU's from shutting off every time a task finishes?



The problem with multiple GPU's is that you have to specify to BOINC that you have 2 GPU's and that you want them to be used. I'll look around for the text you have to put into the cc_config.xml to do so, but I don't remember it now.

Once you do this, GPUGRID will send you an additional task to work on, and a second GPUGRID task will run in addition to the first one. You don't have to worry about the second GPU shutting off, GPUGRID tasks will be constantly sent to it and it will crunch constantly, as long as BOINC is running, just like your GT 640 does. Even if it didn't do anything it would just be idle, not shut off.

With this second GPU, make sure you allow an additional core to be left open to power the GPU. I see you have a i5, so you make sure that the option "On multiprocessor systems, use at most __% of the processors" in the BOINC manager would be filled in with no higher than 50%, so BOINC won't use the CPU on other CPU tasks you may have running, and instead will leave 2 cores alone that the GPU's can pick up and use. It's better to sacrifice some of your CPU to power a monster like a GPU.


Hope this helped.

Dylan
Send message
Joined: 16 Jul 12
Posts: 98
Credit: 386,043,752
RAC: 0
Level
Asp
Scientific publications
watwatwatwatwatwatwat
Message 28733 - Posted: 24 Feb 2013 | 5:05:40 UTC

I found the cc_config.xml file, it was under Program Files - BOINC.

This was the code inside it that tell BOINC to use all GPU's.


<cc_config>
<options>
<use_all_gpus>1</use_all_gpus>
</options>
</cc_config>


Very simple, I guess.

ExtraTerrestrial Apes
Volunteer moderator
Volunteer tester
Avatar
Send message
Joined: 17 Aug 08
Posts: 2705
Credit: 1,311,122,549
RAC: 0
Level
Met
Scientific publications
watwatwatwatwatwatwatwatwatwatwatwatwatwatwatwatwatwatwatwatwatwatwatwatwat
Message 28737 - Posted: 24 Feb 2013 | 10:22:59 UTC

Yes, GTX 680 into the PCIe 3 16x and the GT640 into the other slot. PCIe 2 is about half as fast as PCIe 3, but your slot is only 4 lanes wide (that "4x"), whereas regular GPU slots use 16 lanes in parallel ("16x"). So that slot will be about 8 times slower than your primary one. Not a problem for the GT640 and GPU-Grid in general, but a high end card would suffer being placed there.

BTW: the GTX660Ti is much better value than the GTX680 (80% the performance for 60% the price, with a bit more overclocking headroom).

Linux can in general run several GPUs (and others already covered the BOINC configuration), but I can not promise you hassle-free installation on Ubuntu. Actually, when ever I want to do anything under Linux which goes further than launching Firefox and Openoffice it turns out to be quite a pain for me.. but apparently you already got so far as to say you're using it, so you've probably got a chance :D

MrS
____________
Scanning for our furry friends since Jan 2002

MarkJ
Volunteer moderator
Volunteer tester
Send message
Joined: 24 Dec 08
Posts: 738
Credit: 200,909,904
RAC: 0
Level
Leu
Scientific publications
watwatwatwatwatwatwatwatwatwatwatwatwatwatwatwatwatwatwatwatwat
Message 28739 - Posted: 24 Feb 2013 | 10:42:23 UTC

Just a point about BOINC. It will allow for multiple graphics cards fine. However it assumes they are all the same. BOINC doesn't actually crunch, all it does is schedule the tasks, so it will get a bit confused by having two different cards in the one machine from the point of view of estimated times jumping all over the place. Because of this most of us tend to try and either use the same sort of cards or restrict them to different projects. To do that you can use the <exclude_gpu> statements in the cc_config file. The 640 is not really suitable for GPUgrid work anyway so maybe best to use if for other projects.
____________
BOINC blog

Profile AdamYusko
Send message
Joined: 29 Jun 12
Posts: 26
Credit: 21,540,800
RAC: 0
Level
Pro
Scientific publications
watwatwatwatwatwat
Message 28748 - Posted: 24 Feb 2013 | 17:40:28 UTC

Thank you everyone who has responded so far. You have given me a lot to think about.

In response to MarkJ's last comment, I should say that this Tower is used almost exclusively to crunch Boinc Projects, so while the 640 is not the fastest pony at the race, it does make up for it at a certain level by being on nearly 24/7/365.

Now that you are mentioning this though, I realize it would be a headache to attempt to split tasks between GPU's by setting the 640 to crunch Short Runs, and the newer card to crunch Long Runs.

You have given me a bit to think about.
____________

ExtraTerrestrial Apes
Volunteer moderator
Volunteer tester
Avatar
Send message
Joined: 17 Aug 08
Posts: 2705
Credit: 1,311,122,549
RAC: 0
Level
Met
Scientific publications
watwatwatwatwatwatwatwatwatwatwatwatwatwatwatwatwatwatwatwatwatwatwatwatwat
Message 28749 - Posted: 24 Feb 2013 | 21:20:05 UTC - in response to Message 28748.

it would be a headache to attempt to split tasks between GPU's by setting the 640 to crunch Short Runs, and the newer card to crunch Long Runs.

You could set the GPUs up for different projects, so that the GT640 runs something with shorter WUs.

MrS
____________
Scanning for our furry friends since Jan 2002

Dylan
Send message
Joined: 16 Jul 12
Posts: 98
Credit: 386,043,752
RAC: 0
Level
Asp
Scientific publications
watwatwatwatwatwatwat
Message 28751 - Posted: 25 Feb 2013 | 0:38:32 UTC

If the GT 640 is running all the time, would it really be that bad running Long Runs on it instead of Short Runs to avoid the issue of splitting tasks?

Dagorath
Send message
Joined: 16 Mar 11
Posts: 509
Credit: 179,005,236
RAC: 0
Level
Ile
Scientific publications
watwatwatwatwatwatwatwatwatwatwatwatwatwat
Message 28754 - Posted: 25 Feb 2013 | 3:01:29 UTC - in response to Message 28748.

Now that you are mentioning this though, I realize it would be a headache to attempt to split tasks between GPU's by setting the 640 to crunch Short Runs, and the newer card to crunch Long Runs.


It might not be a headache after all. On Linux you can easily create 2 or more separate BOINC installations on 1 machine. Each installation has it's own unique ID in other words they will be seen by the server as different hosts. Give each host a unique location (home, work, school) in your website prefs and assign the one location to get long runs, the other location to get shortys.

The same idea is reported to work on Windows too but I've never been able to do it on Windows. I know it works for CPU tasks on Linux though, I've done it myself. If there is a way (there might not be, I don't know) to tell BOINC to use GPU x and not GPU y then it should work for GPU too. If you're interested in trying I can give more details. It's quite simple to create the 2 installations.

If you decide to forge ahead with this on your own then be aware that if you have BOINC installed as a daemon service (i.e. installed from repos) then you cannot easily install it again as a daemon service. It can be done but it's a pain. Instead use the Berkeley installer to create the second installation or, even easier, just copy the BOINC binaries from /usr/bin/ into ~/boinc/ and you're almost done.

____________
BOINC <<--- credit whores, pedants, alien hunters

Profile Beyond
Avatar
Send message
Joined: 23 Nov 08
Posts: 1112
Credit: 6,162,416,256
RAC: 0
Level
Tyr
Scientific publications
watwatwatwatwatwatwatwatwatwatwatwatwatwatwatwatwatwatwatwatwatwatwatwat
Message 28756 - Posted: 25 Feb 2013 | 7:52:25 UTC - in response to Message 28749.

it would be a headache to attempt to split tasks between GPU's by setting the 640 to crunch Short Runs, and the newer card to crunch Long Runs.

You could set the GPUs up for different projects, so that the GT640 runs something with shorter WUs.

Then you end up using exclusions which if used with app_infos or app_configs cause work fetch problems with every BOINC version since 7.0.2. Nasty bug, reported repeatedly long ago and never fixed.

MarkJ
Volunteer moderator
Volunteer tester
Send message
Joined: 24 Dec 08
Posts: 738
Credit: 200,909,904
RAC: 0
Level
Leu
Scientific publications
watwatwatwatwatwatwatwatwatwatwatwatwatwatwatwatwatwatwatwatwat
Message 28758 - Posted: 25 Feb 2013 | 10:32:54 UTC

You can use the exclude_gpu to restrict one card from long work units and another entry to do the opposite, so that way the faster card can run the longs and the 640 can run the short WU.. He'd need two exclude_gpu statements, one for each device.

You'd probably be better off doing this with the current alpha test BOINC client as it has some fixes around excluded devices and work-fetch issues. That is 7.0.52 at the moment.
____________
BOINC blog

Profile Beyond
Avatar
Send message
Joined: 23 Nov 08
Posts: 1112
Credit: 6,162,416,256
RAC: 0
Level
Tyr
Scientific publications
watwatwatwatwatwatwatwatwatwatwatwatwatwatwatwatwatwatwatwatwatwatwatwat
Message 28760 - Posted: 25 Feb 2013 | 13:22:24 UTC - in response to Message 28758.

You can use the exclude_gpu to restrict one card from long work units and another entry to do the opposite, so that way the faster card can run the longs and the 640 can run the short WU.. He'd need two exclude_gpu statements, one for each device.

Please post the appropriate exclude_gpu statements to do this for long and short WUs?

Rick A. Sponholz
Avatar
Send message
Joined: 20 Jan 09
Posts: 52
Credit: 2,518,707,115
RAC: 0
Level
Phe
Scientific publications
watwatwatwatwatwatwatwatwatwatwatwatwatwatwatwatwatwatwatwatwatwatwatwat
Message 28765 - Posted: 25 Feb 2013 | 17:49:06 UTC - in response to Message 28720.

Yes, you can run multiple graphics cards. I have three computers with multiple graphic card configurations. A computer with two (2) GTX590's, a computer with one (1) GTX590 AND one (1) GTX660Ti, also have a computer with one (1) GTX590 AND one (1) GTX550Ti. prior to my last upgrade, I had the same dual configuations with GTX295's, which were replaced by the GTX590's. I have been runing dual configuations for several years, with the only issue being heat build up during the summer. Regards, Rick
____________

ExtraTerrestrial Apes
Volunteer moderator
Volunteer tester
Avatar
Send message
Joined: 17 Aug 08
Posts: 2705
Credit: 1,311,122,549
RAC: 0
Level
Met
Scientific publications
watwatwatwatwatwatwatwatwatwatwatwatwatwatwatwatwatwatwatwatwatwatwatwatwat
Message 28768 - Posted: 25 Feb 2013 | 19:02:12 UTC - in response to Message 28751.

If the GT 640 is running all the time, would it really be that bad running Long Runs on it instead of Short Runs to avoid the issue of splitting tasks?

Mine with 1040 MHz stock core clock takes ~135 ks for a long run WU. That's 1.5 days - not nice and won't get the 1-day-return-bonus, but easily within the deadline.

MrS
____________
Scanning for our furry friends since Jan 2002

Profile AdamYusko
Send message
Joined: 29 Jun 12
Posts: 26
Credit: 21,540,800
RAC: 0
Level
Pro
Scientific publications
watwatwatwatwatwat
Message 28779 - Posted: 26 Feb 2013 | 0:52:37 UTC - in response to Message 28768.

If the GT 640 is running all the time, would it really be that bad running Long Runs on it instead of Short Runs to avoid the issue of splitting tasks?

Mine with 1040 MHz stock core clock takes ~135 ks for a long run WU. That's 1.5 days - not nice and won't get the 1-day-return-bonus, but easily within the deadline.

MrS



That brings up something else I was considering and if I decide to go further with it I will create another post. But I have a very nice case with good air flow, and I even have 4 120mm fan mounts that are unused. As my GT 640 is passively cooled, and stays far from running to hot currently, I think with a few extra fans, I can get a decent over clock on the GT 640, while still keeping it nice and cool. But yes I do not get the 24 hour bonus on most long runs, but I am more than happy for the 48 hour bonus, as honestly I am not that concerned with credit hunting, I just want to support the science.
____________

MarkJ
Volunteer moderator
Volunteer tester
Send message
Joined: 24 Dec 08
Posts: 738
Credit: 200,909,904
RAC: 0
Level
Leu
Scientific publications
watwatwatwatwatwatwatwatwatwatwatwatwatwatwatwatwatwatwatwatwat
Message 28790 - Posted: 26 Feb 2013 | 13:07:24 UTC - in response to Message 28760.
Last modified: 26 Feb 2013 | 13:08:03 UTC

You can use the exclude_gpu to restrict one card from long work units and another entry to do the opposite, so that way the faster card can run the longs and the 640 can run the short WU.. He'd need two exclude_gpu statements, one for each device.

Please post the appropriate exclude_gpu statements to do this for long and short WUs?


<exclude_gpu> <url>http://www.gpugrid.net/</url> <device_num>0</device_num> <type>NVIDIA</type> <app>acemd2</app> </exclude_gpu> <exclude_gpu> <url>http://www.gpugrid.net/</url> <device_num>1</device_num> <type>NVIDIA</type> <app>acemdlong</app> </exclude_gpu>


This says device 0 can't do acemd2 (short tasks) and device 1 can't do acemdlong.

He would need to get the device numbers from the BOINC start up messages as to which card is which in his system.

If you only have 1 sort of device in your system (ie only Nvidia or only ATI) it says in the doco that you can omit the <type> tag
____________
BOINC blog

Profile Beyond
Avatar
Send message
Joined: 23 Nov 08
Posts: 1112
Credit: 6,162,416,256
RAC: 0
Level
Tyr
Scientific publications
watwatwatwatwatwatwatwatwatwatwatwatwatwatwatwatwatwatwatwatwatwatwatwat
Message 28803 - Posted: 26 Feb 2013 | 19:34:08 UTC - in response to Message 28790.

Thanks Mark, an example always helps.

ExtraTerrestrial Apes
Volunteer moderator
Volunteer tester
Avatar
Send message
Joined: 17 Aug 08
Posts: 2705
Credit: 1,311,122,549
RAC: 0
Level
Met
Scientific publications
watwatwatwatwatwatwatwatwatwatwatwatwatwatwatwatwatwatwatwatwatwatwatwatwat
Message 28812 - Posted: 26 Feb 2013 | 20:44:58 UTC - in response to Message 28779.

Sure, GT640 is a very tame card by todays standards. Expect ~1050 MHz without touching voltage.

MrS
____________
Scanning for our furry friends since Jan 2002

Post to thread

Message boards : Graphics cards (GPUs) : Is it possible to run Multiple GPUs in one Tower?

//