Hi guys!
I've just set up a system with an NVIDIA GT240 GPU card and a dual core CPU. It's running Linux Ubuntu 10.04 64bit.

Nvidia driver is 195.36.24: a 6.12 app has been downloaded, which I think is fine for this cuda card, and crunching is going up of 0.1% every minute, so the whole workunit should take about 60,000 seconds. It seems in average.

I'd like to let you know a thing I've noticed, and to ask you a couple of questions.

First, my "report". LOL.
The gpugrid wu now being crunched wants 0.16 CPUs (however it doesn't go beyond 2%-3% load) and the GPU. The app runs with a niceness of 10 by default - whilst other CPU boinc apps (boincsimap, WCG...) run with a niceness of 19. I've found that 10 vs 19 is not enough: when the CPU is saturated, even by "idle" applications running at 19, the gpu almost stops working, its temperature falls and the gpugrid app becomes ways slower: many many times slower. Renicing gpugrid app to -1 has given it back its normal speed.
I have not tested any other values for now.

So my first question is: is there a simple way to tell boinc to set to -1 the niceness for gpugrid apps?

My second question is about overclocking the GPU. I know about the

Option "Coolbits" "1"

line in the Device section of /etc/X11/xorg.conf.
But it gives the chance to overclock only the GPU and the memory, while I happen to know that is the shader frequency that matters most. How could I rise it?

Thanks in advance for everything,
and bye.

If you free up a CPU core, use swan_sync=0, and report tasks immediately it should help a lot:

FAQ: Best configurations for GPUGRID

I don't know of a way to only overclock the shaders from within Linux.

From what I read the problem is that nice/renice settings tend to be lost on restart, but I read about a method that might stick over at WCG. Unfortunately I'm not a Linux expert and I cannot test it at the minute (don't even have a Linux system right now). Have a look and if you can work something out post it up, so others can benefit. This is worth a read.

If anyone has definite answers to these problems please post your methods.

Good luck,

If you free up a CPU core, use swan_sync=0, and report tasks immediately it should help a lot:

First af all, thanks a lot for your support! :)

Uhm... I think... No, I definitely do not want to waste 98% of a cpu thread (I have two cores without hyper threading), if I can have the exact same GPU efficiency through niceness adjustment (verified) while happily crunching two other CPU tasks that will be a tiny bit slower than usual.

I don't know of a way to only overclock the shaders from within Linux.

I suspected and feared it. :(
I'll keep on searching for a while, but I think I'm going to surrender.

From what I read the problem is that nice/renice settings tend to be lost on restart

Sure, but I've already put in /etc/rc.local the line:

renice -1 $(pidof acemd2_6.12_x86_64-pc-linux-gnu__cuda)

and in /etc/crontab the line:

*/5 * * * * root renice -1 $(pidof acemd2_6.12_x86_64-pc-linux-gnu__cuda) > /dev/null 2>&1

which is quite enough for me now.

Thanks again. :)
Bye.

Nope, that didn't work for me. I tried changing the niceness to -1 and then let rosetta@home run on all four cores on my i5 750, but rosetta@home effectively shut out the GPUGrid application (no meaningful work was being done by the GPU). This occurred even when the rosetta@home apps were running with a niceness of 19 and GPUGrid running with a niceness of -1.

Nope, that didn't work for me. I tried changing the niceness to -1 and then let rosetta@home run on all four cores on my i5 750, but rosetta@home effectively shut out the GPUGrid application (no meaningful work was being done by the GPU). This occurred even when the rosetta@home apps were running with a niceness of 19 and GPUGrid running with a niceness of -1.

Sorry about it. But you're dealing with a gtx570 (fine card!) and 6.13 app, aren't you? Maybe this makes the difference.

The niceness trick is actually working for me with boincsimap 5.10 and WCG (FightAIDS@Home 6.07 and Help Conquer Cancer 6.08) on the CPU side.

You said Rosetta... It works for me with Rosetta Mini 2.17 too.

However my next try, probably tomorrow, will be to test the newest 270.18 nvidia driver, and see what happens with 6.13 gpugrid (someone is getting fine results even with a GT240 and 6.13).

Bye.

When I use swan_sync=0 and free up a CPU core on my GT240’s they now improve performance by around 7.5% (Phenom II-940, compared to running 4 CPU tasks and not using swan_sync). It use to be higher but recent tasks seem less reliant on the CPU (the GPU task priority is set to below normal by the project, while the CPU task priority is less; low). I’m using the 6.12 app. The 6.13 app is substantially slower for the GT240 cards, and while that might have changed, I doubt it. I have not tested the 270 driver, as I don’t have a Linux platform, but partially because none of the 260.x drivers I previously tested offered any improvement for the 6.13app and caused some of my cards to drop their speed. I would be very reluctant to install Linux just to test the 270.18 Beta for a GT240, but let us know how you get on, should you chose to (I suggest you don’t if you are unsure of how to uninstall it and revert to your present driver).

CPU usage depends on the GPU:
If for example you have a GT240 and a Q6600, and it takes 20h to complete one GPUGrid task, using say 350sec of CPU time, then the total amount of CPU time you would need to support a GTX470 would be about 4 times that, as a GTX470 would do 4 similar tasks in 20h. It now appears more important to use swan_sync=0 and free up a CPU core/thread for high end GPUs, but less so for entry level GPU’s.

With high end CPU’s this is not too much of a problem, you just need 1 core/thread from between 6 and 12 cores/threads, and if you have a GTX570 or GTX580 the choice is clear. If I had one single system with a dual core CPU and en entry level GPU I think I would be more likely to try to crunch on both CPU cores, unless it caused the GPU tasks to extend beyond 24h.

To elucidate further, I am currently using Linux Mint 10 with kernel version 2.6.35-25-generic. My GTX 570 is using the latest stable Linux driver version, 260.19.36. And yes, I am using version 6.13 of the GPUGrid CUDA app. It certainly would be nice if the Rosetta app running on that fourth core would slow down a little bit just so that the GPUGrid app can get some CPU time in.

If that was my system I would be inclined to do some calculations.
How much quicker would your GTX570 be if you freed up one of those CPU cores, and how much faster would the other 3 CPU cores be when running Rosetta?
Elapsed Time - CPU Time

Hi, I seem to change the priority of the GPU through the GUI "System Monitor" in Ubuntu 10.10, each time you load a new task, ie between 5 to 8 hours (for my GTX295) is no problem or nuisance.

Personally I would comment that I use the system to move the two tasks of the GPU of the priority level "10 " starts GPUGRID default to "0" when I'm working normal with the PC and move it to "-10" in hours that do not use it anyway with the highest priority is not low on overall response on the computer (i7-930, 6GB RAM). Greetings.

---

http://stats.free-dc.org/cpidtagb.php?cpid=b4bdc04dfe39b1028b9c5d6fef3082b8&theme=9&cols=1

I don't know of a way to only overclock the shaders from within Linux.

Found this possible workaround (not yet tested): http://www.nvnews.net/vbulletin/showthread.php?t=158620

Bye.

Hi, I seem to change the priority of the GPU through the GUI "System Monitor" in Ubuntu 10.10, each time you load a new task, ie between 5 to 8 hours (for my GTX295) is no problem or nuisance.

Personally I would comment that I use the system to move the two tasks of the GPU of the priority level "10 " starts GPUGRID default to "0" when I'm working normal with the PC and move it to "-10" in hours that do not use it anyway with the highest priority is not low on overall response on the computer (i7-930, 6GB RAM). Greetings.

Same here, it's just every 28h +/-5h, depending on the WU, on my GT240.
All the stuff with swan_sync, freeing of a core or such doesn't change anything on this machine, it's just a smokescreen to pretend giving clues.
Changing the priority from 10 to 0 or even -3 increases the crunch speed big time, it's the only solution for my GT240 under Linux.

Fortunately Einstein now provides a reliable application for CUDA crunching under Linux, that does worthy science as well, so I manually stop Einstein every other day in the evening, DL a fresh GPUGrid-WU, set it manually to -3 and let it crunch for the next +/-28h, set GPUGrid to NNW asap, and set Einstein working again by hand once the GPUGrid is through.

Unfortunately sometimes Linux decides to set back the nice-factor to 10 during crunching, I don't know why and when, it looks unpredictable, and so I will loose precious crunching time because of the stubbornness of the app not to do what I want. I would very much appreciate a setting in my account or my BOINC (or my ubuntu, if there is a more permanent way of doing so outside System Monitor), that would keep the app at the desired nice level.

---

Gruesse vom Saenger

For questions about Boinc look in the BOINC-Wiki

I would very much appreciate a setting in my account or my BOINC (or my ubuntu, if there is a more permanent way of doing so outside System Monitor), that would keep the app at the desired nice level.

I have put in my /etc/crontab the line:

*/5 * * * * root renice -1 $(pidof acemd2_6.12_x86_64-pc-linux-gnu__cuda) > /dev/null 2>&1

So every 5 minutes a cronjob renices to -1 the task having name acemd2_6.12_x86_64-pc-linux-gnu__cuda (if it exists and if its niceness is different, otherwise it does nothing).

Modify the line according to your app name (6.13). You'll probably find the proper name by executing (as root!):

# ls /var/lib/boinc-client/projects/www.gpugrid.net/ |grep ace

You can also choose an other niceness, if -1 doesn't satisfy you. :)

HTH.
Bye.

I would very much appreciate a setting in my account or my BOINC (or my ubuntu, if there is a more permanent way of doing so outside System Monitor), that would keep the app at the desired nice level.

I have put in my /etc/crontab the line:

*/5 * * * * root renice -1 $(pidof acemd2_6.12_x86_64-pc-linux-gnu__cuda) > /dev/null 2>&1

It seems to work fine, thanks a lot

I still have to manually change between Einstein and GPUGrid, as otherwise I will not make the deadline here if BOINC switches between the apps, but that's nothing GPUGrid can do about (besides setting the deadline to the needed 48h), that's a BOINC problem.

---

Gruesse vom Saenger

For questions about Boinc look in the BOINC-Wiki

Saenger,

All the stuff with swan_sync, freeing of a core or such doesn't change anything on this machine, it's just a smokescreen to pretend giving clues.

Yet another misleading and disrespectful message!

Your 98.671 ms per step performance for a GIANNI_DHFR1000 is exceptionally poor for a GT240

Using the recommended config (even on vista; over 11% slower than Linux), using swan_sync and the 6.12 app, 22.424 ms per step

Lem Novantotto, thanks for the Linux tips.
That hardware overclock method should work for your GT240, as it was tested on a GT220. Several users at GPUGrid have performed hardware OC's for Linux in the past. If it’s any help I tend to leave the core clock at stock, the voltage at stock and only OC the shaders to around 1600MHz, usually 1599MHz (this is stable on 6 GT240 cards I presently use). Others can OC to over 1640, but it depends on the GPU.

Saenger,

All the stuff with swan_sync, freeing of a core or such doesn't change anything on this machine, it's just a smokescreen to pretend giving clues.

Yet another misleading and disrespectful message!

Your 98.671 ms per step performance for a GIANNI_DHFR1000 is exceptionally poor for a GT240

Using the recommended config (even on vista; over 11% slower than Linux), using swan_sync and the 6.12 app, 22.424 ms per step

Why do you ignore my messages?
I'm using this stupid swan_sync thingy, no f***ing use for it.
I've tried to "free a whole CPU" for it, the only effect was an idle CPU.

So don't talk to me about misleading and disrespectful!

---

Gruesse vom Saenger

For questions about Boinc look in the BOINC-Wiki

...and yet there is no mention of swan_sync in your task result details.

If it was in use the result details should say,
SWAN: Using synchronization method 0
and you would not have an idle CPU!

For example,
# Total amount of global memory: 497745920 bytes
# Number of multiprocessors: 12
# Number of cores: 96
SWAN: Using synchronization method 0
# Time per step (avg over 795000 steps): 22.424 ms
# Approximate elapsed time for entire WU: 44848.953 s
called boinc_finish

</stderr_txt>
]]>

Validate state Valid
Claimed credit 7491.18171296296
Granted credit 11236.7725694444
application version ACEMD2: GPU molecular dynamics v6.12 (cuda)

Your configuration is more suited to running Einstein and CPU tasks than GPUGrid tasks. So that is what you should do. What is the point in messing about every other day to run a different project, and at half efficiency or less?

...and yet there is no mention of swan_sync in your task result details.

I don't know how this stupid swan_sync stuff is supposed to work, it's your invention, not mine.

As I have posted in this post 66 days ago, and before that as well 82 days ago, and as I just tested again, my swan _sync is "0".

saenger@saenger-seiner-64:~$ echo $SWAN_SYNC
0

So if your precious swan_sync isn't working with my WU, as you claim, it's not my fault.

---

Gruesse vom Saenger

For questions about Boinc look in the BOINC-Wiki

I'm beginning to suspect that the reason swan_sync isn't working is because the environment variable is associated with the wrong user. GPUGrid tasks don't run at the user level. Rather, the user is boinc.

Using Linux is not easy, especially if you use several versions. As you know you can add export SWAN_SYNC=0 to your .bashrc file, but that is easier said than done, and depends on how/where you install Boinc. With 10.10 versions it is especially dificult; when I tried using it the repository only had the 260 driver, and some of the familiar commands did not work.
If you can't overclock or tune the fans properly and have niceness/swan_sync problems, the lure is not so strong - but this is down to lack of Linux knowledge/detailed instruction.

I installed BOINC using the package manager. Rather than adding

export SWAN_SYNC=0

to my .bashrc file, I added it to /etc/bash.bashrc instead. I changed it because when I looked back at all of the tasks I did, even though I set swan_sync in my .bashrc file, the SWAN synchronization message has never showed up in any of the tasks I have done. That tells me that the GPUGrid task is not taking up the set environment variable in .bashrc. Perhaps placing it in /etc/bash.bashrc would help.