In my opinion there is two possibilities:

- that owner of this maschine do not care about filling up GPU`s over whole (100%) time of work
- or that he have some special rules on server side (e.g. this user/Computer ID have possibility to DL and have `in stock` 2-3x CPU WU`s, what is btw my dream...).

---

We have at least one potential system out there that is going to have 6 cores that will be available for work.

Take a look at this one: Triple GTX295
I do hope I can keep this configuration...and am not forced to split it up again because I can't feed it 24/7...

Just to point it out clearly: currently GPU-Grid limits the number of concurrent WUs a Pc can have to the number of cores, i.e. 4 on "normal" quad and 8 on an i7. That's why you can have 6 WUs overall, Paul :)

As AiDec pointed out you'll run into trouble to keep 4 GPUs fed with a normal quad core, as there are always delays in up/download and scheduler contacts. Not to mention internet connection or server failures..

And the point of the current limitation to 1 WU per CPU core is to keep slow GPUs from being routinely overloaded with many WUs which they have to abort after 3 days. I suspect the plan was to eventually make BOINC smart enough to request proper amounts of GPU work. But these efforts have been.. not very successful, so far ;) (to use kind words..)

MrS

---

Scanning for our furry friends since Jan 2002

Just to point it out clearly: currently GPU-Grid limits the number of concurrent WUs a Pc can have to the number of cores, i.e. 4 on "normal" quad and 8 on an i7. That's why you can have 6 WUs overall, Paul :)

As AiDec pointed out you'll run into trouble to keep 4 GPUs fed with a normal quad core, as there are always delays in up/download and scheduler contacts. Not to mention internet connection or server failures..

And the point of the current limitation to 1 WU per CPU core is to keep slow GPUs from being routinely overloaded with many WUs which they have to abort after 3 days. I suspect the plan was to eventually make BOINC smart enough to request proper amounts of GPU work. But these efforts have been.. not very successful, so far ;) (to use kind words..)

MrS

Ok, I mis-understood ... or was not fully up to snuff on the details ... heck, I have only been here a couple weeks ... :)

So, I can consider getting a triplex of 295's for the i7 machine ... cool ...

The real count should be number of GPU cores plus one I would think rather than the number of CPU cores. But that is just me ...

As to the other comment, part of the problem is that the BOINC developers, like Dr. Anderson don't seem to be inclined to listen to users that much. This has been a continual problem with the BOINC System in that the three groups don't really interact that well SYSTEM WIDE ... this is not a slam against GPU Grid or any other project specifically ... but, in general, the communication between BOINC Developers, Users (participants) and the project staff is, ahem, poor at best ...

THAT said, GPU Grid at the moment is one of the more responsive AT THIS TIME ...

At one point in historical time Rosetta@Home was excellent ... six months later ... well ... it has never been the same ...

Anyway, with the three groups isolated from each other and no real good structures to facilitate communication ... well ... real issues never get addressed properly ...

---

Hi,
when we have finished with new applications, we will be testing a new BOINC feature which allows to send WU per GPU instead that per CPU. This way we will be fixing the limit to two/GPU.
So, this should not be a problem anymore in the near future. The feature is already in BOINC, it just needs to be tested.

gdf

The real count should be number of GPU cores plus one I would think rather than the number of CPU cores. But that is just me ...

Edit: never mind this post, GDFs answer above says enough.

Definitely, or even more if the server sees that results are returned very quickly. But BOINC has to know and report the number of GPUs reliably, which doesn't sound too hard but may not be the item of top priority.

MrS

---

Scanning for our furry friends since Jan 2002

Hi,
when we have finished with new applications, we will be testing a new BOINC feature which allows to send WU per GPU instead that per CPU. This way we will be fixing the limit to two/GPU.
So, this should not be a problem anymore in the near future. The feature is already in BOINC, it just needs to be tested.

gdf

Thanks for the answer ... almost as if we knew what we were doing ... :)

---

We are trying to build one. We would like to know what is the real power consumption to see if we can cope with a 1500W PSU.

gdf

http://www.extreme.outervision.com/psucalculator.jsp

9950 AMD Phenom X4 2.60Ghz,
2x DDR2 ram
4x NVIDIA GTX 295
2x SATA HDD
1x CD/DVD
2x 92mm Fans
4x 120mm Fans

At 100% load and adding 20% Capacitor Aging = 1429 Watts

110.9 Amps on the 12 volt rail though.

AFAIK there isn't a PSU readily available that can produce that much current on the 12 volt rail.

Some nice number, but sadly not useful. The GPUs won't run anywhere near their specified maximum power draw of ~300 W. This calculator has no idea how power the cards will draw under 100% GPU-Grid load. And generally I found most calculators gave vastly exaggerated numbers.. but I'd have to give this one the benefit of the doubt.

MrS

---

Scanning for our furry friends since Jan 2002

Seems very tempting to go dual 1500W power supplies.... Also, upgrading to Phenom II X4 CPU would actually reduce power consumption a little. Or forget the whole thing and build a new core i7 system w/ a dual power supply server chassis....

Dual 1.5 kW? If I look at the prices of these units i'm not tempted one little bit ;) And I'm sure you wouldn't need that much power, even for 4 GTX 295.
EDIT: just noted this post: 3 GTX 295 are fine with an Antec 850W.

MrS

---

Scanning for our furry friends since Jan 2002

4 GTX 295 would be awesome.. if only for the sake of it :D

Might want an i7 to keep them fed, though.

Interesting discussion about needing 8 CPU cores to feed 8 GPUs.

Leaving aside for a second the fact that *currently*, GPUGRID won't download 8 WUs unless you have 8 cores, the question is whether, or more accurately, by how much, having less than 8 CPU cores would slow down the GPUs.

After thinking about it for a while, I don't think 8 CPU cores are required. Here's why.

The argument was made that if one core is feeding two GPUs, and both GPUs need new work at the same time, one will have to wait for the other to be serviced by the CPU. That is true. Let's call such an event a 'collision'. When a collision occurs, a GPU sits idle. That's bad. But it's not an accurate description of what's actually happening inside the computer. Let me explain.

In a computer with, say, a Q6600 and 4x GTX295 (8 GPUs), the above example is simplifying the system from 4+8 to 1+2. While mathematically that arithmetic might be correct, it distorts (significantly) the metrics of how the system is going to perform.

Assume that it takes 1/8 of a CPU core to service the GPU (which is about right on my Q6600+GTX280 running Vista). In a 1 CPU + 2 GPU system, with a purely random distribution of when the GPUs need new work, you would expect a collision to occur approximately 1/8 of the time. That's a significant performance hit.

But let's look at what's happening on the real computer, which is 4+8, not 1+2. Each of the 8 GPUGRID tasks is NOT assigned to a specific CPU core. there's lots (probably a hundred or so) of tasks running on the computer, and all of them get swapped into the register set of an individual CPU core when needed. When that task is pre-empted by another task, its regsiter set is saved somewhere, and another task takes over that core. Since BOINC tasks run at lower priority than anything else, they get pre-empted almost continuously, whenever the computer needs to do anything else, such as servicing interrupts.

As a result, the BOINC tasks should be hopping around between the four cores quite a lot. The important thing is that each GPU task is not running permanently on one of the four cores in the CPU, it's running on whichever core happens to be free at that instant.

For a 1 core + 2 GPU system to have a collision, you merely need to have the second GPU need new work while the other GPU is in the process of receiving new work. There's a 1/8 chance of this.

But in the real computer, with 4 cores, in order for a collision to occur, a GPU has to need new work while *five* of the other 7 GPUs are also requesting new work. What are the odds of that? (Someone correct me if my math is wrong, it's been decades since I studied probability.) With 4 cores, up to 4 GPUs can request work at the same time with 0% probability of collision because all 4 can be serviced at once.

(note that I'm simplifying this somewhat...)

With the 5th GPU, what's the probability of a collision? In order for a collision to occur, all of the other GPUs would need to request new work at the same time. The odds of that happening are 1/(8^^4), or approximately 0.025%. That's higher than the 0.00% rate with 4 GPUs, but is certainly still an acceptable rate.

With the 6th GPU, the probability will rise. The chance of 4 of the other 5 GPUs needing servicing at the same time as the 6th GPU is (1+35)/(8^^5), which works out to 36/32768 or about 0.11%. Still pretty reasonable.

With the 7th GPU, the chance of 4 of the other 6 GPUs needing servicing at the same time is (1+42+5!*7^^2)/(8^^6). This evaluates to (1+42+120*49)/262144, or 5932/262144, or 2.26%.

With the 8th GPU, the chance of 4 of the other 7 GPUs being busy at the same time is (1+49+6!*7^^2+(6!/3!)*7^^3)/(8^^7), or (1+49+5040*49+120*343)/2097152, or (1+49+246960+4160)/2097152, or 251170/2097152, or 11.98%.

So, if you add up all the collision rates avnd average them out over all 8 GPUs, you end up with a grand total of 1.79%. Granted, there's a LOT of uncertainty and extrapolation in those calculations, but if correct, you would see less than a 2% degradation in performance by running on 4 cores instead of 8.

FYI, the 1-in-8 CPU utilization factor is based on my experience with a 2.4GHZ Q6600 running windows Vista. I understand that under Ubuntu the CPU utilization is much lower. In that case, the collision rate would drop exponentially, and 4 cores would be MORE than enough. Two (or even one) would probably surfice.

I think I read somewhere about an upcoming change to GPUGRID to change the "1 WU per CPU core" rule to "1 WU per GPU". Assuming my calculations are valid, once that change is made there's really no reason to need 8 CPU cores to run 8 GPUs.

Regards,
Mike

> "1 WU per CPU core"

We have tried to experiment with 1 WU per GPU, but there seem to be a bug in BOINC .

We will keep working on it when ready.

GDF

PS: yes 4 cores are more than enough for 8 GPUs.

Hi Mike,

good analysis! I won't bother thinking through the maths, I just agree with you that in case of 1/8th core/GPU the performance hit in a "4 cores 8 GPUs" config is going to be very small.

When I wrote my posts about this back then I didn't factor thread load balancing into account. However, around the 10th of January the win client 6.62 was not yet released (if I remember correctly) and the cpu utilization was more like 80% of one core per GPU. The new client was on the horizon, but I was not sure yet, how it would work out. Luckily I also wrote "If each GPU only needs about 1% of a core [as the Linux client used to do at that time] I can imagine that a quad is good enough for 8 GPUs." :)

MrS

---

Scanning for our furry friends since Jan 2002

After writing that long post, I was thinking about it some more, and realized that the impact would be even less than my calculations showed.

My analysis was based on the concept of the GPU being one big monolithic processor, and the whole GPU having to stop and wait for more data.

Perhaps the way the GPUGRID application is written, it works just like that. But that's not the way I would write it, and I suspect you didn't write it that way either.

A GTX280 (or 1/2 of a GTX 295) has 30 multiprocessors. In effect, it's a 30 core GPU. If your application loads data into each of the 30 cores independently, then when a collision occurs, only 1/30th of the GPU is actually blocked -- the other 29 cores keep on processing while that one core waits for more work.

If that's the case, then the 2% number my analysis showed is incorrect -- the actual number if 1/30th of that.

Mike

Yes i guess the application is setup in multi processing thats the power of these cards do much at once.
The speed comes from all these little cores calculating different parts

So i think the impact depends also if and when the cpu gets instructions that some part is done and a new part can be calculated since the cpu does the controlling
However these instructions really is little effort for the cpu's since it goes on doing other work till it gets another ready with this and that work.

I don't think that the Power Supply is going to be the only problem when trying to run 4 295's, space is going to be a problem too I would imagine as these things are big. What type of motherboard are you planning on using, and what type of case. These things generate lots of heat so cooling is definitely going to be a concern as well, particularly since rather than venting the heat out the back of the card and outside the case these card vent out the side directly into your case. While the cost in the summer will be high, you will be able to save money in the Winter as you could heat a small home with 4 295's running 100% 24/7.

My GTX 295's push hot air outside the case and suck it from inside. Four i would have to see. I think my sound cables are starting to melt being directly above the 295. The heat is extreme.
Also the 295 is exactly the same size as my 280. I think all the 200 series cards are majorly massive. Im putting everything into an E-ATX case(CM HAF 932) tomorrow as my old Antec 900 cant fit both a 200 series graphics card and a HDD at the same height.

I just had a thought on the WU cache issue, why not make it a user selectable value in preferences, say default 2/host at a time up to whatever the user believes his/her box can handle?

1. your motherboard is an motherboard with an AMD Chipset(https://www.megamobile.be/productinfo/103151/Moederborden/MSI_K9A2_Platinum_-4x_PCI_Express_x16,_AMD%C2%AE_CrossF/) that means it only has ATI's crossfire X so you use only the first gfx card lolzor

2. NVIDIA has only triple-way SLI not Quad-way SLI

conclusion: I think you're a kind of a show off that just wasted some money ;)