Richard, when GPU's reach ~79 to 80°C some sort of throttling kicks in and the boost usually drops. Whatever is happening on the GPU at this time it causes tasks to become unstable. Fortunately the app can now recover the WU's, but the individuals solution is to increase GPU fan speeds, reducing the temperatures, so GPU's don't reach 79C. It's better than starting and stopping tasks and as it increases GPU longevity, decreases heat and power usage its the generic thing to do when GPU crunching.

Wdethomas, the short WU's and the Beta's yield less credits/time than the long WU's. If you want good credits just crunch the long WU's.

There is an 8.15 (cuda60) app in the shot queue for Windows, and an 8.11 (cuda60) app in the Beta queue for Linux. I would suggest that the CUDA 6 apps (also 4.2 and 4.5) work for non Maxwell GPU's (Kepler cards and below). I don't know if there is any point in people who don't have Maxwell's running these WU's?

---

FAQ's

HOW TO:
- Opt out of Beta Tests
- Ask for Help

The Long Run 8.15 are generating only 54,300 points. I was getting 115,000 before. it seems that the driver 334 is affecting now the scoring.

It seems that the Titans are the ones generating these numbers . The Tesla are doing 115,000. Will keep an eye on this. Having the 4 cards on the same motherboard generates some drivers problems for me. Will be getting another motherboard to install the Teslas alone.

The recent Long tasks that you ran were shorter than previous tasks:

e4s8_e1s4f56-GIANNI_trypben3-0-1-RND7147_1 5285887 16 Mar 2014 | 0:06:56 UTC 16 Mar 2014 | 4:12:39 UTC Completed and validated 8,179.43 3,027.14 54,300.00 Long runs (8-12 hours on fastest card) v8.15 (cuda55)

867x-SANTI_MAR420cap310-26-32-RND0600_1 5285097 16 Mar 2014 | 0:49:34 UTC 16 Mar 2014 | 8:58:10 UTC Completed and validated 24,500.15 5,792.68 115,650.00 Long runs (8-12 hours on fastest card) v8.15 (cuda55)

They are a different type of WU (from a different researcher).
Gianni's tasks give more credit per day, 86400/8179 * 54300 = 573,605 credits/day.
The Santi_Mar WU's gave, 86400/24500 * 115650 = 407,843 credits/day

---

FAQ's

HOW TO:
- Opt out of Beta Tests
- Ask for Help

Richard, when GPU's reach ~79 to 80°C some sort of throttling kicks in and the boost usually drops. Whatever is happening on the GPU at this time it causes tasks to become unstable. Fortunately the app can now recover the WU's, but the individuals solution is to increase GPU fan speeds, reducing the temperatures, so GPU's don't reach 79C. It's better than starting and stopping tasks and as it increases GPU longevity, decreases heat and power usage its the generic thing to do when GPU crunching.

In my case, the solution is to preferentially crunch GPUGrid tasks on the (lower, cooler) Device 1. Device 0 crunches other projects.

I was doing that manually, and the unexpected Beta cuda60 arrived late in the evening and ran overnight on Device 0.

I've now set up a project exclusion for device 0 in cc_config.xml, so that shouldn't happen again. I also agree that there isn't much point in running the Beta tasks using a non-Maxwell card and last November's applications, so I've disabled Beta apps for the time being. Happy to have verified that the cuda60 app does run on pre-Maxwell hardware, and I'll watch out for re-deployment.

Most of the ACEMD beta version v8.15 (cuda55) I got failed immediate after starting.
http://www.gpugrid.net/results.php?userid=25200

I disabled these beta versions for now.

Alexander

Alexander, there was a CUDA6 Beta app (in the Beta queue) for Windows, but it's not there now.
At present there is a CUDA6 app in the short queue for Windows. Perhaps you want to see if you get any CUDA6 work there? I'm not sure if there is any, and any CUDA55 or CUDA42 tasks will fail on a Maxwell, but I guess it's worth a shot.

---

FAQ's

HOW TO:
- Opt out of Beta Tests
- Ask for Help

Alexander, there was a CUDA6 Beta app (in the Beta queue) for Windows, but it's not there now.
At present there is a CUDA6 app in the short queue for Windows. Perhaps you want to see if you get any CUDA6 work there? I'm not sure if there is any, and any CUDA55 or CUDA42 tasks will fail on a Maxwell, but I guess it's worth a shot.

Ah, yes and no.
I can remember a post that said with driver xxx I will get the cuda 6 wu's. My mistake, I thought, I will get ONLY cuda6 wu's.
Anyway, I changed my setup and enabled the short runs and the beta's. I had the long runs enabled before.
Let's see what happens.

Alexander

PS: while writing these lines the download of my first cuda6 wu started!

Many thanks Alexander:

818x-SANTI_MAR419cap310-29-84-RND6520_0 5288118 16 Mar 2014 | 14:01:45 UTC 16 Mar 2014 | 18:26:07 UTC Completed and validated 14,621.79 4,475.01 18,300.00 Short runs (2-3 hours on fastest card) v8.15 (cuda60)

Now we know it works, and it makes sense for the app to be in the short queue!

---

FAQ's

HOW TO:
- Opt out of Beta Tests
- Ask for Help

Many thanks Alexander:

818x-SANTI_MAR419cap310-29-84-RND6520_0 5288118 16 Mar 2014 | 14:01:45 UTC 16 Mar 2014 | 18:26:07 UTC Completed and validated 14,621.79 4,475.01 18,300.00 Short runs (2-3 hours on fastest card) v8.15 (cuda60)

Now we know it works, and it makes sense for the app to be in the short queue!

Glad I could help !
But the question remains; how can I make a setup to get the cuda6 wu's only? This should be a function of the server (as I understand it).

Another question - but first a statement.
There are discussions over @ Einstein about gpu load and what it means. For all of the volunteers here, @ Einstein it's very common to run two or more wu's together on one gpu. For my GTX 750ti two wu's are quite goot, resulting in 95% gpu load and < 60% TDP power consumption, according to GPU-Z 0.7.7

The cuda-version that is used @ Einstein is cuda32.

Running one of the beta wu's here I see 78% gpu load, 300MB dedicated mem, 39MB dynamic mem, 22% mem controller load and up to 75% TDP power load.

There are two questions now:
- with a lower gpu usage - how can the power load be higher?
- to get a higher gpu load ( I am a fan of >95% ) - how can I run 2 wu's together? CPU cores are available, this is not a limit. Is there a chance to get a choice via the settings menue like Einstein offers ( the GPU utilization factor) ?

Alexander

Alexander, a couple of questions for you:
Where you running Einstein on the iGPU (i7-3770) when running the GPUGRID WU?
- edit - Looks like you were; (Binary Radio Pulsar Search (Arecibo) v1.34 (opencl-intel_gpu))

How many CPU cores where you using to crunch CPU projects when you completed the CUDA6 WU?
If you have GPUZ, open it and select the Sensors Tab, what is the Memory Controller Load when running a GPUGrid WU?

I agree that it's better to run 2 GPUGrid WU's at a time on Einstein, but as you point out it's a CUDA 3.2 app (due to bugs) and at GPUGrid more recent CUDA versions are used (4.2, 5.5 and now 6.0).

The complexity of the program determines actual power draw and GPU usage, as does the amount of memory and the number of shaders you use... It's also the case that CPU apps use different amounts of power. I once found that a Q6600's power usage could vary by >30W depending on what Boinc apps its running (running 4 CPU apps).

When the power usage is higher you are doing more 'Work' (Applied Mechanics).

Running more that 1 WU has been exhaustively discussed. Basically, you would need a high end 3GB card and very poor performing tasks to make it worthwhile here. On my Windows system I'm getting 85 to 92% GPU usage for my GPUGrid tasks depending on the WU type. Both my GPU's are overclocked by ~10%, almost eliminating the WDDM issues.

---

FAQ's

HOW TO:
- Opt out of Beta Tests
- Ask for Help

Alexander, a couple of questions for you:
Where you running Einstein on the iGPU (i7-3770) when running the GPUGRID WU?
- edit - Looks like you were; (Binary Radio Pulsar Search (Arecibo) v1.34 (opencl-intel_gpu))

How many CPU cores where you using to crunch CPU projects when you completed the CUDA6 WU?
If you have GPUZ, open it and select the Sensors Tab, what is the Memory Controller Load when running a GPUGrid WU?

I agree that it's better to run 2 GPUGrid WU's at a time on Einstein, but as you point out it's a CUDA 3.2 app (due to bugs) and at GPUGrid more recent CUDA versions are used (4.2, 5.5 and now 6.0).

The complexity of the program determines actual power draw and GPU usage, as does the amount of memory and the number of shaders you use... It's also the case that CPU apps use different amounts of power. I once found that a Q6600's power usage could vary by >30W depending on what Boinc apps its running (running 4 CPU apps).

When the power usage is higher you are doing more 'Work' (Applied Mechanics).

Running more that 1 WU has been exhaustively discussed. Basically, you would need a high end 3GB card and very poor performing tasks to make it worthwhile here. On my Windows system I'm getting 85 to 92% GPU usage for my GPUGrid tasks depending on the WU type. Both my GPU's are overclocked by ~10%, almost eliminating the WDDM issues.

THX for the quick answer,

- yes, I do run igpu apps, two at a time. I'm aware that this reduces memory bandwith usable for the gpu apps, but not that much, that the gpu usage is < 80%. I leave one complete CPU free, so this is usually not a problem. Anyway, it is a beta version and the final version may make better use of the gpu. Let's delay that question.
- more than one wu - a 3GB card: GPU-Z reports 300MB used ram , this should not be the bottleneck unless GPU-Z has another understanding what used ram means. Mem controller load says 22%.

I have learned that it usually makes sense to run 2 or more wu's when the gpu usage is < 80% and makes little or no sense if usage is > 90%; the overhead eats the gain then.

I want to get a deeper understanding, what gpu usage means, how accurate that value is. Without knowing more about this I tend to assume a 95% gpu load equals a 95% power load, but reality shows that this assumption is wrong.

I posted my question in the techpowerup forum, hoping that a developer is reading it and finds time to answer my question.

Anyway, thanks for the time you take to answer my questions,

cheers
Alexander

Thanks for answering my questions.

22% is quite low (and a good thing)! The larger cache size might be the reason.
I suspect that GPUGrid WU performances are being reduced by ~20% (roughly) due to running the Einstein iGPU app, but you would have to suspend the Einstein app to find out what it is for you (and this will vary depending on your CPU). I've tested this to some extent.

At GPUGrid if the apps utilize >85% of the GPU you cannot benefit from running 2 tasks. I have tested this quite extensively. If apps operate at ~80% it's not worth doing either, somewhat due to error rates which are exponential when running 2 tasks.

Different GPUGRID WU's use different amounts of RAM; ~300MB, ~600MB and over 1GB have been common over the last year or two, and when you use more than 1.5GB things get complex for most cards (a shader to Memory ratio thing; with GK104 and 2GB).

---

FAQ's

HOW TO:
- Opt out of Beta Tests
- Ask for Help

I suspect that GPUGrid WU performances are being reduced by ~20% (roughly) due to running the Einstein iGPU app, but you would have to suspend the Einstein app to find out what it is for you (and this will vary depending on your CPU). I've tested this to some extent.

We will see the difference in ~ 4hrs

Cheers

Alexander

9.5% faster.

---

FAQ's

HOW TO:
- Opt out of Beta Tests
- Ask for Help

So what have we learnt about Maxwell, and what's the situation?

The performance is slightly below that of a GTX660, or half of a GTX Titan, but a GTX750Ti probably uses a lot less power (maybe around 45W). So it's a small to medium sized card but very efficient, and because it uses so little power it's not going to get too hot.
As it turns out, it's not bottlenecked by the stingy memory bandwidth. This could be down to the larger cache.

At present we know the CUDA6 app works on Windows for Maxwell GPU's, but those with Maxwell's are still getting CUDA 5.5 tasks, which all fail. This needs to be sorted.
Few people are going to buy a small card specifically to run short WU's so this begs the question, is the GTX750Ti fast enough to complete a Long task inside 24h? The answer is probably yes, at least for now and if you crunch 24/7.

So, who should buy a GTX570Ti?
There are 4 candidates:
People looking to replace a GTX650Ti (or lesser card),
People wanting to replace older cards (GTX400's say) with more efficient cards,
People who want to buy new entry to medium performing cards.
Those who want a desktop replacement system (laptop) - the same GPU is set to appear in the GTX800m range.

---

FAQ's

HOW TO:
- Opt out of Beta Tests
- Ask for Help

At present we know the CUDA6 app works on Windows for Maxwell GPU's, but those with Maxwell's are still getting CUDA 5.5 tasks, which all fail. This needs to be sorted.
Few people are going to buy a small card specifically to run short WU's so this begs the question, is the GTX750Ti fast enough to complete a Long task inside 24h? The answer is probably yes, at least for now and if you crunch 24/7.

I've tried to test that, but got cuda5.5 tasks only.
Answer will come later when cuda6 long runs are available.

Two short questions:
- where can I see that cuda 6 wu's are available (short / long runs)? After a very long download I had to kill 3 long runs because cuda5.5 will not run on my card.
- are admins aware of the fact, that the server and the web-pages are incredibly slow?

Alexander

Server Status Will tell you if work is available in a queue,

Tasks by application
application				unsent	in progress	avg runtime of last 100 results in h (min-max)	users in last 24h
Short runs (2-3 hours on fastest card) 	169 	1,520 		1.46 (0.04 - 9.46) 				652
ACEMD beta version 			0 	76 		1.20 (0.22 - 3.62) 				93
Long runs (8-12 hours on fastest card) 	21 	2,017 		5.26 (0.07 - 24.00) 				615
CPU only app 				1 	70 		0.15 (0.07 - 0.24) 				4

Apps Will tell you which queue has a CUDA 6.0 app,

Short runs (2-3 hours on fastest card)
Platform Version Installation time
Microsoft Windows (98 or later) running on an Intel x86-compatible CPU 8.15 (cuda42) 8 Dec 2013 | 15:32:01 UTC
Microsoft Windows (98 or later) running on an Intel x86-compatible CPU 8.15 (cuda55) 8 Dec 2013 | 15:32:04 UTC
Microsoft Windows (98 or later) running on an Intel x86-compatible CPU 8.15 (cuda60) 14 Mar 2014 | 22:23:53 UTC
Linux running on an AMD x86_64 or Intel EM64T CPU 8.00 (cuda42) 26 Aug 2013 | 10:49:37 UTC
Linux running on an AMD x86_64 or Intel EM64T CPU 8.00 (cuda55) 26 Aug 2013 | 10:49:36 UTC

ACEMD beta version
Platform Version Installation time
Microsoft Windows (98 or later) running on an Intel x86-compatible CPU 8.15 (cuda42) 14 Nov 2013 | 14:36:32 UTC
Microsoft Windows (98 or later) running on an Intel x86-compatible CPU 8.15 (cuda55) 14 Nov 2013 | 14:39:01 UTC
Linux running on an AMD x86_64 or Intel EM64T CPU 8.11 (cuda60) 15 Mar 2014 | 0:07:21 UTC
Linux running on an AMD x86_64 or Intel EM64T CPU 8.14 (cuda42) 20 Sep 2013 | 21:53:36 UTC
Linux running on an AMD x86_64 or Intel EM64T CPU 8.14 (cuda55) 20 Sep 2013 | 21:53:34 UTC

Long runs (8-12 hours on fastest card)
Platform Version Installation time
Microsoft Windows (98 or later) running on an Intel x86-compatible CPU 8.15 (cuda42) 23 Jan 2014 | 14:02:55 UTC
Microsoft Windows (98 or later) running on an Intel x86-compatible CPU 8.15 (cuda55) 23 Jan 2014 | 14:02:57 UTC
Linux running on an AMD x86_64 or Intel EM64T CPU 8.03 (cuda42) 30 Aug 2013 | 7:52:50 UTC
Linux running on an AMD x86_64 or Intel EM64T CPU 8.03 (cuda55) 30 Aug 2013 | 7:52:49 UTC

CPU only app
Platform Version Installation time
Microsoft Windows (98 or later) running on an Intel x86-compatible CPU 1.05 13 Mar 2014 | 21:53:20 UTC
Linux running on an AMD x86_64 or Intel EM64T CPU 1.05 13 Mar 2014 | 21:59:13 UTC

The problem is, just because there is a CUDA6 app in the short queue doesn't mean you will get a task that will use that app - you might get tasks that use the CUDA5.5 app.

I think the server is supposed to determine your GPU and appropriate apps according to the spec, but this doesn't work. For some reason people with a GTX650Ti seem to be getting lots of CUDA6 work, but those with a GTX750Ti are not. Whatever the system, it's not working.

Is there a manual way to force tasks to use the CUDA6 app?

---

FAQ's

HOW TO:
- Opt out of Beta Tests
- Ask for Help

https://dl.dropboxusercontent.com/u/50246791/gpugrid%20server%20status%20page.PNG
This is how my statuspage looks like, nothing to click on it.