This advantage is less than it was with the CUDA3.1 apps (it was around 33%)
It's too bad from the cruncher's perspective that nVidia doesn't make non-super-scalar GPUs anymore, but (as kind of compensation) the good news is that the CUDA4.2 can better utilize the super-scalar architecture than the CUDA3.1.

This discussion is difficult, because we're talking about the performance of a system consisting many parts, all of these parts continuously changing over time, and this change could alter (like it did in the past) their order of significance:
1. I/a The GPU
2. I/b The code running on the GPU
3. II - The computer (also a system consisting many parts)
4. II/a The operating system of the computer
5. II/b The optimization of the BOINC client for the hardware it's running on
6. II/c The hardware components of the computer (beside the GPU)

This is the actual order.
Except for item 2, the participants can optimize this system.
But item 2 is the fundamental of this optimization:
I've changed my Core 2 Quad systems to Core i7 systems to achieve better overall performance by eliminating the PCIe bandwith bottleneck which reduced the performance of the CUDA3.1 client. The CUDA 4.2 client is better in regards of this issue also, so no such change (read it as investment) is needed now from the participants. But you still have to spare a CPU thread per GPU (item 5)

Interesting. I did notice the large performance gain for GF106 cards with the CUDA 4.2 app. I didn't realize that the new app also improved the situation with PCIe. Performance is not too degraded now even for cards in an X4 slot. Thanks.

But you still have to spare a CPU thread per GPU (item 5)

This all is good information Retvari thanks.

One more question: 'keep one CPU core free', does that mean two on an i7 when HT is switched on?

---

Greetings from TJ

I think its went up from 66% to around 80%.

Then why would I still be calculating ~66% for a sample size of 3+2?

@TJ: I think he means logical cores.

MrS

---

Scanning for our furry friends since Jan 2002

You included a GTX660Ti which going by the theoretical TFlops is identical to the GTX670. We know it's memory bottle-necked as are other cards to varying extents in the GTX600 series - the Memory controller of my GTX660Ti's is 41%, my GTX660 is 31% and the 650TiBoost was ~27%. The theoretical TFlops is not that useful for comparisons. Even years ago we needed to use correction factors. Your sample is also skewed, even between your 580 and 570 the numbers don't add up. The theory says a 580 is 13% faster than a 570, but your sample has a 6% gap. There should be a 4% performance difference between a GTX480 and a GTX570, your sample has it at 9.7%. So the 660Ti isn't a 670, the 570 is probably OC'ed, and your 580 is from this system,
Coprocessors [7] NVIDIA GeForce GTX 580 (1535MB).

Following the move to CUDA 4.2 I remembered one of the researchers posted to say that the 66% shader utilization limit wasn't an issue any more.

---

FAQ's

HOW TO:
- Opt out of Beta Tests
- Ask for Help

Hello guys,

A quick question, I am installing my GTX660 currently.
Is it wise to update to the latest BOINC version 7.0.64?

I rad on other projects that this version has problems with Linux.

I run winVista ultimate x64 on the I7 with the new GTX660.

Thanks as always for the help!

---

Greetings from TJ

I'm using 7.0.27 on Ubuntu 13.04, with 304.88 drivers. New-ish OS, but mature apps and drivers - Working fine.

I don't know what version of Boinc you are presently using and can't see your system, but generally speaking, if it works, leave it be. If you need 7.0.64/65 for some other project then have a go.

---

FAQ's

HOW TO:
- Opt out of Beta Tests
- Ask for Help

I'm using 7.0.27 on Ubuntu 13.04, with 304.88 drivers. New-ish OS, but mature apps and drivers - Working fine.

I don't know what version of Boinc you are presently using and can't see your system, but generally speaking, if it works, leave it be. If you need 7.0.64/65 for some other project then have a go.

That is also my idea "if it works leave it alone".
I Installed the new GTX660 and all the software from the CD.
BOINC has a message that an app will not work.
Probably due to the 305.xx version of the nVidia drivers.
I am updating these now to 314.xx and see if that works.
I will leave BOINC than on 7.0.28, that I'm using on that PC.

---

Greetings from TJ

Is it wise to update to the latest BOINC version 7.0.64?
I run winVista ultimate x64 on the I7 with the new GTX660.

Yes. There are a huge number of bugfixes in 7.0.64. I'm running it on 11 machines and while not yet perfect, it's better than anything previous. BTW it's the recommended stable version currently:

http://boinc.berkeley.edu/download_all.php

More questions

I have now installed the GTX660 it is running some Milkyway to test as they are fast (approx. 7 minutes).
BOINC 7.0.28, nVidia driver 314.22 CUDA version 5.0, compute capacity 3.0.

Have installed EVGA precision and set GPU clock and fan speed to auto.
GPU Clock is running at 1110MHz, at times less. Should I set this lower?

Now in the BOINC tasks everything is "flashing" and going to other windows, e.g. browser, taskmanger, gives sometimes a white screen for a second.
That was not the case with the GTX285. Things seem to go slower.
How can I change this?

I will update to latest BOINC if Rosetta WU's are ready.

---

Greetings from TJ

[quote]I'm using 7.0.27 on Ubuntu 13.04, with 304.88 drivers. New-ish OS, but mature apps and drivers - Working fine.

That is also my idea "if it works leave it alone".
I Installed the new GTX660 and all the software from the CD.
BOINC has a message that an app will not work.
Probably due to the 305.xx version of the nVidia drivers.
I am updating these now to 314.xx and see if that works.
I will leave BOINC than on 7.0.28, that I'm using on that PC.

fyi, app_config.xml only works in 7.0.40 and higher, it is no related to Nvidia driver.

I whish I had listened to skgiven!
I am updating BOINC to 7.0.64 and get error message about a key, error 1402
that I don't have right for a key.
I don't know how to get the picture here I have it on the pc as a jpg.

But now is BOINC not working any more. New GTX660 and not to use.
Who can help me please to solve this?

---

Greetings from TJ

I am updating BOINC to 7.0.64 and get error message about a key, error 1402
that I don't have right for a key.

Googled the error, seems you have a permissions problem. It has nothing to do with 7.0.64. Read this, follow the instructions:

http://boinc.berkeley.edu/dev/forum_thread.php?id=3248#20925

I am updating BOINC to 7.0.64 and get error message about a key, error 1402
that I don't have right for a key.

Googled the error, seems you have a permissions problem. It has nothing to do with 7.0.64. Read this, follow the instructions:

http://boinc.berkeley.edu/dev/forum_thread.php?id=3248#20925

Thanks Beyond that was the trick.

Well all day busy with installing just a card. Booting my system takes 11 minutes?
Have run all sorts of diagnostics but seems something with PCAngel is the installer and repair utility from Windows via which the system builder installed it. Not funny.
Anyhow all is running now, an NOELIA on the new GTX660 seems to take 78 hours to complete? I'll see.
But system is slow, I have now used 5 cores to crunch Rosetta and one (0.737) for GPUGRID, so 2 left for other things and still slow. Even browsing. I have in FF all my project pages and tasks lists, and switching from one to the other and back is not always fast. The GPU load is around 50% and the GPU clock most at 1110MHz but sometimes lower. Even when typing this FF was not responding for a few seconds. At the moment I am not happy with my updates and tired. I go to sleep but hope to read some suggestions tomorrow from the experts out there.

Thanks for the help and input.

---

Greetings from TJ

A NOELIA WU should take 12 or 13h on a GTX660, 78h is way too long!
Are you going by the % complete and time taken, or the inaccurate remaining time?
50% GPU load is too low and suggests that the Noelia WU needs more CPU access, even though you have 2 free CPU's (perhaps FF is using them and some plug in is eating up a thread). What's the CPU kernel usage like? Is there heavy disk I/O.
11min to boot isn't normal for a desktop system.

---

FAQ's

HOW TO:
- Opt out of Beta Tests
- Ask for Help

A NOELIA WU should take 12 or 13h on a GTX660, 78h is way too long!
Are you going by the % complete and time taken, or the inaccurate remaining time?
50% GPU load is too low and suggests that the Noelia WU needs more CPU access, even though you have 2 free CPU's (perhaps FF is using them and some plug in is eating up a thread). What's the CPU kernel usage like? Is there heavy disk I/O.
11min to boot isn't normal for a desktop system.

Thanks skgiven, but you have to help me with the CPU kernel usage. Is that what the CPU is using? It's around 88%. According to process explorer I/O is very little, most of the time zero. From the 12GB mem, 4.5GB is used.
I don't know what is wrong with this system it is my only not Dell and having issues from the start.
NOELIA is 53% ready in 11:20 hours, estimated still 51:14 hours, but that is not right.
I use now a 550Ti and that is on a Dell quad with vista 32 bit and is running NOELIA's fine in approx 40 hours with a steady GPU load of 91%, monitoring with EVGA NV-Z.

The 660 is in the i7 with vista ultimate 64 bit and has varying GPU load of 5%, 43% up to 63% as I have seen as highest. GPU clock mostly at 1110MHz.

One thing I found is that last night there was 28GB free on my C-drive and now only 7.84 free. Don't know what happened yet??

Closing FF and checking if it was indeed killed, kept the CPU usage at 88%.

There is not a lot of software installed on this system and use it for crunching and some browsing now and then mostly all BOINC project related.

After yesterday's update to he GTX660, newer driver 314.22 and new BOINC 7.0.64 the system seems slower with building up the graphics. Blank screens for a second when switching and such. The remaining time in BOINC Manager is flickering all the time, not on my other 2 PC currently crunching.

Any ideas I can do?
Thanks

---

Greetings from TJ

You included a GTX660Ti which going by the theoretical TFlops is identical to the GTX670. We know it's memory bottle-necked as are other cards to varying extents in the GTX600 series - the Memory controller of my GTX660Ti's is 41%, my GTX660 is 31% and the 650TiBoost was ~27%. The theoretical TFlops is not that useful for comparisons. Even years ago we needed to use correction factors. Your sample is also skewed, even between your 580 and 570 the numbers don't add up. The theory says a 580 is 13% faster than a 570, but your sample has a 6% gap. There should be a 4% performance difference between a GTX480 and a GTX570, your sample has it at 9.7%. So the 660Ti isn't a 670, the 570 is probably OC'ed, and your 580 is from this system,
Coprocessors [7] NVIDIA GeForce GTX 580 (1535MB).

Following the move to CUDA 4.2 I remembered one of the researchers posted to say that the 66% shader utilization limit wasn't an issue any more.

By using the theoretical GFlops I actually deduce the current correction factor (between the included GPUs). And of course I can't know the real clock speed, that's why I'm not concerned about 5% differences - which is fine if I'm looking for the 33% elephant in the room. So let's add some more numbers:

runtime in ks | theoretical performance in TFlops | TFlops*runtime
GTX 580: 39.49 | 1.58 | 62.4
GTX 570: 41.92 | 1.40 | 58.7
GTX 480: 46.00 | 1.34 | 61.6
GTX690: 30.22 | 2.81 | 84.9
GTX 660Ti: 38.07 | 2.46 | 93.7
GTX660: 42.21 | 1.88 | 79.3
GTX 560Ti: 67.71 | 1.26 | 85.3

The difference between GTX690 and GTX660Ti might actually be attributed to the memory bus. The GTX660 shows exceptional performance (not unlike what we see in real world - it is the new bang-for-the-buck king here), but this ~5% advantage might easily be explained by an OC.

Overall I still see 2 distincly different classes: CC 2.0 cards around 60 ks*TFlops and super scalar cards around 90 ks*TFlops. However, if I exclude the GTX660Ti (due to memory constraints) and leave the GTX660 in (OC?) the average of these cards shifts downwards to 83 ks*TFlops. That's better than a pure 2/3 rule (in which case the average would have been 90), but still significantly less than what the "non-super" scalar cards achieve. This might well be the improvement you've been reading about and actually seems like a realistiv number to me.

MrS

---

Scanning for our furry friends since Jan 2002

A NOELIA WU should take 12 or 13h on a GTX660, 78h is way too long!
Are you going by the % complete and time taken, or the inaccurate remaining time?
50% GPU load is too low and suggests that the Noelia WU needs more CPU access, even though you have 2 free CPU's (perhaps FF is using them and some plug in is eating up a thread). What's the CPU kernel usage like? Is there heavy disk I/O.
11min to boot isn't normal for a desktop system.

Thanks skgiven, but you have to help me with the CPU kernel usage.

Task Manager, View, Show Kernel Times.

Is that what the CPU is using? It's around 88%

88% would be total CPU usage. Kernel usage is less (shown in red), and I just wanted to know if it was high or low (roughly). You are using 6 CPU cores which is 75% of the CPU and running a NOELIA WU (which Should use around half of a CPU), totaling 81%. The system is probably using ~3%, so something(s) else is using ~4% of your overall CPU (32% of a thread). I suspect the Noelia WU might not be using ~50% of a CPU core, and something else is using more than 4%. Could you check this; Boinc Manager (advanced view), tasks, click on the NOELIA WU and then Properties (to the left). CPU time should be ~half of elapsed time.

According to process explorer I/O is very little, most of the time zero. From the 12GB mem, 4.5GB is used.
I don't know what is wrong with this system it is my only not Dell and having issues from the start.
NOELIA is 53% ready in 11:20 hours, estimated still 51:14 hours, but that is not right.

Go by the 53% in 11.2h, rather than the estimated time remaining. That means the WU should take just over 21h to complete. You might see the estimated remaining time depreciate faster than the wall clock.

The 660 is in the i7 with vista ultimate 64 bit and has varying GPU load of 5%, 43% up to 63% as I have seen as highest. GPU clock mostly at 1110MHz.

Not sure what you mean by 5%? 43% to 63% means there is a bottleneck somewhere. Perhaps the CPU, PCIE freq, another app (system or more likely Rosetta), or something else...

One thing I found is that last night there was 28GB free on my C-drive and now only 7.84 free. Don't know what happened yet??

Rosetta - look at a tasks properties in Boinc.

Closing FF and checking if it was indeed killed, kept the CPU usage at 88%.

Rules out FF, but not add ons (Java, downloaders, scripts...)

After yesterday's update to he GTX660, newer driver 314.22 and new BOINC 7.0.64 the system seems slower with building up the graphics. Blank screens for a second when switching and such. The remaining time in BOINC Manager is flickering all the time, not on my other 2 PC currently crunching.

A new driver might be in order, but I would start by suspending Rosetta work and see if the GPU usage rises. If that doesn't improve things, do a restart with Rosetta WU's still suspended (the mini WU's do checkpoint).

For crunching, we recommend that the HDD has 10GB free space, with no more than 80% used. Not sure what type you have, but if it's a standard hard disk drive (not a Solid State Drive) and you have little space on it, that could be a problem, especially if running Rosetta WU's.

---

FAQ's

HOW TO:
- Opt out of Beta Tests
- Ask for Help

MrS., I did some similar calculations and comparisons myself (different cards though). Some even on my own systems (so those were as accurate as possible), but others largely unknown. I think there are two things at play here; the move to the CUDA 4.2 app inherently favored the CC2.1 GPU's (and GK's) over the CC2.0 cards, and the 2/3 issue might have some sort of a fix; in some circumstances there is no 2/3rds limitation. This might actually be app specific. The performance of the different apps presently in use even varies somewhat between the GK's (memory bandwidth/shader cache related), and might vary by OS/WU type.

Below is my GTX660Ti on W7, and a GTX660Ti on Linux (18% difference):

W7
I71R3-NATHAN_KIDc22_2-2-8-RND7308_0 4478415 139265 25 May 2013 | 4:01:52 UTC 25 May 2013 | 16:18:23 UTC Completed and validated 43,598.53 43,475.95 167,550.00 Long runs (8-12 hours on fastest card) v6.18 (cuda42)
I51R2-NATHAN_KIDc22_2-1-8-RND1700_0 4475974 139265 22 May 2013 | 18:19:57 UTC 23 May 2013 | 6:44:44 UTC Completed and validated 43,750.94 43,617.13 167,550.00 Long runs (8-12 hours on fastest card) v6.18 (cuda42)

Linux
I3R9-NATHAN_KIDc22_2-3-8-RND8249_0 4479099 25 May 2013 | 22:07:28 UTC 26 May 2013 | 10:51:12 UTC Completed and validated 36,793.87 36,662.43 167,550.00 Long runs (8-12 hours on fastest card) v6.18 (cuda42)
I84R9-NATHAN_KIDc22_2-2-8-RND1848_0 4478823 25 May 2013 | 11:51:33 UTC 26 May 2013 | 0:37:30 UTC Completed and validated 36,812.21 36,674.85 167,550.00 Long runs (8-12 hours on fastest card) v6.18 (cuda42)
I58R2-NATHAN_KIDc22_2-3-8-RND0613_0 4478419 25 May 2013 | 1:40:19 UTC 25 May 2013 | 14:23:32 UTC Completed and validated 36,848.42 36,708.87 167,550.00 Long runs (8-12 hours on fastest card) v6.18 (cuda42)

We also know that CPU usage can have a big influence on GPU performance, as can some CPU WU types with some hardware, and we know that PCIE bandwidth has some small influence, but hasn't been fully tested on recent WU's. So, lots of unknowns to contend with. Anyway, these results are closer to what I observed.

---

FAQ's

HOW TO:
- Opt out of Beta Tests
- Ask for Help

A NOELIA WU should take 12 or 13h on a GTX660, 78h is way too long!
Are you going by the % complete and time taken, or the inaccurate remaining time?
50% GPU load is too low and suggests that the Noelia WU needs more CPU access, even though you have 2 free CPU's (perhaps FF is using them and some plug in is eating up a thread). What's the CPU kernel usage like? Is there heavy disk I/O.
11min to boot isn't normal for a desktop system.

Thanks skgiven, but you have to help me with the CPU kernel usage.

Task Manager, View, Show Kernel Times.

Is that what the CPU is using? It's around 88%

88% would be total CPU usage. Kernel usage is less (shown in red), and I just wanted to know if it was high or low (roughly). You are using 6 CPU cores which is 75% of the CPU and running a NOELIA WU (which Should use around half of a CPU), totaling 81%. The system is probably using ~3%, so something(s) else is using ~4% of your overall CPU (32% of a thread). I suspect the Noelia WU might not be using ~50% of a CPU core, and something else is using more than 4%. Could you check this; Boinc Manager (advanced view), tasks, click on the NOELIA WU and then Properties (to the left). CPU time should be ~half of elapsed time.

The kernel is indeed low for 6 cores and little higher for 2 cores. Checking properties NOELIA WU is saying 15:08:53 Elapsed time and 11:15:07 CPU time. The WU is using 0.737 CPUs + 1 nVidia GPU.

According to process explorer I/O is very little, most of the time zero. From the 12GB mem, 4.5GB is used.
I don't know what is wrong with this system it is my only not Dell and having issues from the start.
NOELIA is 53% ready in 11:20 hours, estimated still 51:14 hours, but that is not right.

Go by the 53% in 11.2h, rather than the estimated time remaining. That means the WU should take just over 21h to complete. You might see the estimated remaining time depreciate faster than the wall clock.

That is coorect, BOINC estimations are roughly indeed.

The 660 is in the i7 with vista ultimate 64 bit and has varying GPU load of 5%, 43% up to 63% as I have seen as highest. GPU clock mostly at 1110MHz.

Not sure what you mean by 5%? 43% to 63% means there is a bottleneck somewhere. Perhaps the CPU, PCIE freq, another app (system or more likely Rosetta), or something else...

I mean that the load of the GPU is varying between 5 and 63% not constant 91% on my 550Ti in the other PC. How can I check the PCIE frequency?

One thing I found is that last night there was 28GB free on my C-drive and now only 7.84 free. Don't know what happened yet??

Rosetta - look at a tasks properties in Boinc.

Rosetta is indeed using some space but this is on the D-drive and there is room enough.

Closing FF and checking if it was indeed killed, kept the CPU usage at 88%.

Rules out FF, but not add ons (Java, downloaders, scripts...)

No downloaders active, other things I don't know.

After yesterday's update to he GTX660, newer driver 314.22 and new BOINC 7.0.64 the system seems slower with building up the graphics. Blank screens for a second when switching and such. The remaining time in BOINC Manager is flickering all the time, not on my other 2 PC currently crunching.

A new driver might be in order, but I would start by suspending Rosetta work and see if the GPU usage rises. If that doesn't improve things, do a restart with Rosetta WU's still suspended (the mini WU's do checkpoint).

I have suspended them and indeed GPU load increases, now varying between 50 en 66%, still low. Total CPU is now 28%

For crunching, we recommend that the HDD has 10GB free space, with no more than 80% used. Not sure what type you have, but if it's a standard hard disk drive (not a Solid State Drive) and you have little space on it, that could be a problem, especially if running Rosetta WU's.

BOINC and the BOINC DATA are on the D-drive with 825GB free. So I guess that is not the problem.

Perhaps I need to do a complete new install from the OS on this PC, perhaps Win7 buying? But that will take a lo of time and if I can avoid it with help from yours that would be great. Thanks

---

Greetings from TJ

Is it perhaps an idea to update the the latest nVidia driver 320.18?
Normally I don't use the latest from things, but in this case as my GTX660 is not perming as it should...

Thanks.

---

Greetings from TJ