GPUGRID and app_info

Message boards : Graphics cards (GPUs) : GPUGRID and app_info
Message board moderation

To post messages, you must log in.

Previous · 1 · 2

AuthorMessage
Profile Retvari Zoltan
Avatar

Send message
Joined: 20 Jan 09
Posts: 2380
Credit: 16,897,957,044
RAC: 0
Level
Trp
Scientific publications
watwatwatwatwatwatwatwatwatwatwatwatwatwatwatwatwatwatwatwatwatwatwatwatwatwat
Message 21644 - Posted: 8 Jul 2011, 20:24:47 UTC - in response to Message 21627.  

Let’s take the worst case scenario (theoretical case) when GPU load is 100%. If I'm running the 2nd WU concurrently, duration for both of them is twice longer then for single WU. Am I right?

You're right. Theoretically. :)

Let's come back to the real world. I finished one WU in 6hrs, so if I'm running 2 WUs it should take me 12 hrs to complete both of them. But remember - GPU load is less than 100%, so in reality it will be not 12hrs, but, let's say, 9-11hrs. All in all, I'm pretty much OK to be within 24hrs limit to get +50% bonus.

There is a prerequisite for this scenario: Your PC should not keep a spare pair of WU's in it's queue for too long, because if these two spare WUs are sitting in the queue for 12 hours, and then they are processed for another 12 hours they will just miss the 24 hours deadline. But if your queue is short, and a WU fails, your GPU will run only one WU (or in worst case: nothing) until a new one (or two) is downloaded.

That’s why IMHO it makes sense to go with app_info trick.

You forget about the reason behind low GPU utilization:
GPUGrid uses both the GPU and the CPU for processing a WU. The more the CPU usage of a WU is, the less its GPU usage is, because of the heavy data transmission on the PCIe bus. A GIANNI_KKFREE WU (82% GPU usage) uses 5.5 times more CPU than a TONI_AGGsoup WU (98% GPU usage). I learned from my experience, that I have to overclock a Core2Duo by 33% to get the performance (say 5-10% more GPU usage) of a Core i3 (its integrated PCIe controller is 33% faster than the X48 chipset's). I think the existence of a bus between the GPU and the CPU is the bottleneck, and this bus (ie. the PCIe) can be overloaded even by processing a single GIANNI_KKFREE-like WU. If this is right, there will be no significant rise in GPU usage by running two of them simultaneously (although there can be significant rise in GPU usage by running low and high GPU utilizing tasks at the same time, but maybe the lower GPU utilizing would be more slow than expected).
Don't get me wrong, I'm curious about this, but at the same time I'm very skeptical. Let's find out...
ID: 21644 · Rating: 0 · rate: Rate + / Rate - Report as offensive     Reply Quote
Profile Retvari Zoltan
Avatar

Send message
Joined: 20 Jan 09
Posts: 2380
Credit: 16,897,957,044
RAC: 0
Level
Trp
Scientific publications
watwatwatwatwatwatwatwatwatwatwatwatwatwatwatwatwatwatwatwatwatwatwatwatwatwat
Message 21645 - Posted: 8 Jul 2011, 20:29:44 UTC - in response to Message 21643.  

I did run using an app_info file for a week or more on a dual GPU setup, but that was over 6months ago. Others also did this. For a short time it was worth it, but that was before 6.14 and only when there were lots of less GPU utilizing tasks around (~50%).

Then maybe I'm wrong about low GPU utilizing tasks are overloading the PCIe bus, and in this case it's worth it to make an app_info.xml
I'm getting totally confused.
ID: 21645 · Rating: 0 · rate: Rate + / Rate - Report as offensive     Reply Quote
Profile skgiven
Volunteer moderator
Volunteer tester
Avatar

Send message
Joined: 23 Apr 09
Posts: 3968
Credit: 1,995,359,260
RAC: 0
Level
His
Scientific publications
watwatwatwatwatwatwatwatwatwatwatwatwatwatwatwatwatwatwatwatwatwatwatwatwatwatwatwat
Message 21646 - Posted: 8 Jul 2011, 20:45:50 UTC - in response to Message 21645.  
Last modified: 8 Jul 2011, 20:51:05 UTC

You have to remember this was six months ago and then SWAN_SYNC was much more important (a full CPU core/thread vs 2.5% of the total CPU; 16% of an i7 thread). A lot has changed and now running 2 tasks without SWAN_SYNC migh be different, but only so long as these tasks are relatively low GPU Utilizing tasks. There will be no benefit when just one task is a 95% utilizing task. At 85% who knows (things have changed so you would need to check). At 50% (and there is nothing near that) we would benefit, and I would be running on an app_info setup already.

PS. This is definately for Fermi Only, CC1.3 cards will fail if you try this!

--
Double checked and I did have the .exe in my old app_info file (might not make any difference in Windows).
ID: 21646 · Rating: 0 · rate: Rate + / Rate - Report as offensive     Reply Quote
Profile Retvari Zoltan
Avatar

Send message
Joined: 20 Jan 09
Posts: 2380
Credit: 16,897,957,044
RAC: 0
Level
Trp
Scientific publications
watwatwatwatwatwatwatwatwatwatwatwatwatwatwatwatwatwatwatwatwatwatwatwatwatwat
Message 21648 - Posted: 8 Jul 2011, 22:34:44 UTC - in response to Message 21646.  

Double checked and I did have the .exe in my old app_info file (might not make any difference in Windows).

The .exe has been omitted since version 6.14.
I know this because I'm using eFMer's Priority.
ID: 21648 · Rating: 0 · rate: Rate + / Rate - Report as offensive     Reply Quote
CTAPbIi

Send message
Joined: 29 Aug 09
Posts: 175
Credit: 259,509,919
RAC: 0
Level
Asn
Scientific publications
watwatwatwatwatwatwatwatwatwat
Message 21649 - Posted: 9 Jul 2011, 1:45:22 UTC - in response to Message 21644.  

Nice discussion. guys :-) Rite now I'm waiting WU to complete and then I'll try app_info - both with and w/o .exe

You're right. Theoretically. :)

That's good news:-)

There is a prerequisite for this scenario: Your PC should not keep a spare pair of WU's in it's queue for too long, because if these two spare WUs are sitting in the queue for 12 hours, and then they are processed for another 12 hours they will just miss the 24 hours deadline. But if your queue is short, and a WU fails, your GPU will run only one WU (or in worst case: nothing) until a new one (or two) is downloaded.

Anyway rite now I've got 0 tasks in queue, so I'm really worry about that.

You forget about the reason behind low GPU utilization:
GPUGrid uses both the GPU and the CPU for processing a WU. The more the CPU usage of a WU is, the less its GPU usage is, because of the heavy data transmission on the PCIe bus. A GIANNI_KKFREE WU (82% GPU usage) uses 5.5 times more CPU than a TONI_AGGsoup WU (98% GPU usage). I learned from my experience, that I have to overclock a Core2Duo by 33% to get the performance (say 5-10% more GPU usage) of a Core i3 (its integrated PCIe controller is 33% faster than the X48 chipset's). I think the existence of a bus between the GPU and the CPU is the bottleneck, and this bus (ie. the PCIe) can be overloaded even by processing a single GIANNI_KKFREE-like WU. If this is right, there will be no significant rise in GPU usage by running two of them simultaneously (although there can be significant rise in GPU usage by running low and high GPU utilizing tasks at the same time, but maybe the lower GPU utilizing would be more slow than expected).
Don't get me wrong, I'm curious about this, but at the same time I'm very skeptical. Let's find out...

That's a good point. I was wondering what the hell is going on: CPU and GPU usage is way under 100%, so what's the bottleneck? Now it's clear. If the reason for the is PCIe, then u r rite - nothing I can gain from that trick.

From other hand - PCIe 2.0 x16 is hell fast bus and I'n not really sure it is the problem. But anyways - let's wait for 20 minutes and we'll get the answer.
ID: 21649 · Rating: 0 · rate: Rate + / Rate - Report as offensive     Reply Quote
CTAPbIi

Send message
Joined: 29 Aug 09
Posts: 175
Credit: 259,509,919
RAC: 0
Level
Asn
Scientific publications
watwatwatwatwatwatwatwatwatwat
Message 21650 - Posted: 9 Jul 2011, 2:59:37 UTC
Last modified: 9 Jul 2011, 3:00:29 UTC

OK, I tried both version (with .exe and w/o .exe). They deleted everything (including .exe and all .dll) file from the BOINC folder, so smth wrong with app_info

Guys, don't get me wrong - I'm not complaining, I'm just reporting :-)
ID: 21650 · Rating: 0 · rate: Rate + / Rate - Report as offensive     Reply Quote
Profile skgiven
Volunteer moderator
Volunteer tester
Avatar

Send message
Joined: 23 Apr 09
Posts: 3968
Credit: 1,995,359,260
RAC: 0
Level
His
Scientific publications
watwatwatwatwatwatwatwatwatwatwatwatwatwatwatwatwatwatwatwatwatwatwatwatwatwatwatwat
Message 21653 - Posted: 9 Jul 2011, 11:10:54 UTC - in response to Message 21650.  
Last modified: 9 Jul 2011, 11:37:34 UTC

I don't know what the problem is but I guess the issue may be related to the fact that priority is now being controlled at the thread level rather than Process level.
ID: 21653 · Rating: 0 · rate: Rate + / Rate - Report as offensive     Reply Quote
Richard Haselgrove

Send message
Joined: 11 Jul 09
Posts: 1639
Credit: 10,159,968,649
RAC: 261
Level
Trp
Scientific publications
watwatwatwatwatwatwatwatwatwatwatwatwatwatwatwatwatwatwatwatwatwatwatwatwatwatwat
Message 21656 - Posted: 9 Jul 2011, 12:13:59 UTC - in response to Message 21653.  

I don't know what the problem is but I guess the issue may be related to the fact that priority is now being controlled at the thread level rather than Process level.

Eh?

App_info will work if both the structure and contents of the file are accurate. Just "guessing" the name of a file, library or DLL is never going to work - you need to apply some serious comprehension to the issue too. Then it works - and whatever an application does to control its own thread priority will work just the same, whether launched under app_info or otherwise.

skgiven posted an app_info for Windows yesterday. I haven't tested it, but it looks OK to me. Linux users can study it, see what the basic shape is, and adapt it to their own needs. You need:

<app>
The name GPUGrid uses internally to identify the application. 'Long' and 'normal' length tasks may use different app names.

<file_info>
One section for each executable, library, or other supporting file you're going to use. (You also need to have the actual files themselves, of course!)

<app_version>
A control structure which links the components together and tells BOINC how to use them - e.g. the <coproc><count> values which started this conversation.

Have a look at the app_info documentation. Note in particular the line at the bottom of that page:

Generally this should match the corresponding elements in a scheduler reply message (sched_reply_URL.xml)

If you are already running the project successfully in its normal, automatic download, mode you can read all the information you need to construct an app_info.xml file that will work on your own machine (including the urls for downloading the necessary files) from either sched_reply...xml or client_state.xml
ID: 21656 · Rating: 0 · rate: Rate + / Rate - Report as offensive     Reply Quote
Profile skgiven
Volunteer moderator
Volunteer tester
Avatar

Send message
Joined: 23 Apr 09
Posts: 3968
Credit: 1,995,359,260
RAC: 0
Level
His
Scientific publications
watwatwatwatwatwatwatwatwatwatwatwatwatwatwatwatwatwatwatwatwatwatwatwatwatwatwatwat
Message 21663 - Posted: 10 Jul 2011, 11:50:00 UTC - in response to Message 21656.  
Last modified: 10 Jul 2011, 13:43:40 UTC

The app name is wrong:

<app>
<name>acemd2</name>
</app>

should be,

<app>
<name>acemd_long</name>
</app>


Corrected app_info file:

<app_info>
<app>
<name>acemd_long</name>
</app>
<file_info>
<name>acemdlong_6.15_windows_intel86__cuda31.exe</name>
<executable/>
</file_info>
<file_info>
<name>cudart32_31_9.dll</name>
<executable/>
</file_info>
<file_info>
<name>cufft32_31_9.dll</name>
<executable/>
</file_info>
<file_info>
<name>tcl85.dll</name>
<executable/>
</file_info>
<app_version>
<app_name>acemd_long</app_name>
<version_num>615</version_num>
<avg_ncpus>0.025</avg_ncpus>
<max_ncpus>0.050</max_ncpus>
<flops>1089000000000</flops>
<plan_class>cuda31</plan_class>
<coproc>
<type>CUDA</type>
<count>0.5</count>
</coproc>
<gpu_ram>1280000000</gpu_ram>
<file_ref>
<file_name>acemdlong_6.15_windows_intel86__cuda31.exe</file_name>
<main_program/>
</file_ref>
<file_ref>
<file_name>cudart32_31_9.dll</file_name>
</file_ref>
<file_ref>
<file_name>cufft32_31_9.dll</file_name>
</file_ref>
<file_ref>
<file_name>tcl85.dll</file_name>
</file_ref>
</app_version>
</app_info>
ID: 21663 · Rating: 0 · rate: Rate + / Rate - Report as offensive     Reply Quote
Richard Haselgrove

Send message
Joined: 11 Jul 09
Posts: 1639
Credit: 10,159,968,649
RAC: 261
Level
Trp
Scientific publications
watwatwatwatwatwatwatwatwatwatwatwatwatwatwatwatwatwatwatwatwatwatwatwatwatwatwat
Message 21664 - Posted: 10 Jul 2011, 12:00:48 UTC - in response to Message 21663.  

If you're going to change it in one place, you have to change it in the other, too.

The app name is wrong:

<app>
<name>acemd2</name>
</app>

should be,

<app>
<name>acemd_long</name>
</app>


Corrected app_info file:

<app_info>
<app>
<name>acemd_long</name>
</app>
<file_info>
<name>acemdlong_6.15_windows_intel86__cuda31.exe</name>
<executable/>
</file_info>
<file_info>
<name>cudart32_31_9.dll</name>
<executable/>
</file_info>
<file_info>
<name>cufft32_31_9.dll</name>
<executable/>
</file_info>
<file_info>
<name>tcl85.dll</name>
<executable/>
</file_info>
<app_version>
<app_name>acemd_long</app_name>
<version_num>615</version_num>
<avg_ncpus>0.025</avg_ncpus>
<max_ncpus>0.050</max_ncpus>
<flops>1089000000000</flops>
<plan_class>cuda31</plan_class>
<coproc>
<type>CUDA</type>
<count>0.5</count>
</coproc>
<gpu_ram>1280000000</gpu_ram>
<file_ref>
<file_name>acemdlong_6.15_windows_intel86__cuda31.exe</file_name>
<main_program/>
</file_ref>
<file_ref>
<file_name>cudart32_31_9.dll</file_name>
</file_ref>
<file_ref>
<file_name>cufft32_31_9.dll</file_name>
</file_ref>
<file_ref>
<file_name>tcl85.dll</file_name>
</file_ref>
</app_version>
</app_info>

ID: 21664 · Rating: 0 · rate: Rate + / Rate - Report as offensive     Reply Quote
Profile skgiven
Volunteer moderator
Volunteer tester
Avatar

Send message
Joined: 23 Apr 09
Posts: 3968
Credit: 1,995,359,260
RAC: 0
Level
His
Scientific publications
watwatwatwatwatwatwatwatwatwatwatwatwatwatwatwatwatwatwatwatwatwatwatwatwatwatwatwat
Message 21665 - Posted: 10 Jul 2011, 14:07:05 UTC - in response to Message 21664.  

Thanks Richard.

If anyone tests this, post your findings.
ID: 21665 · Rating: 0 · rate: Rate + / Rate - Report as offensive     Reply Quote
CTAPbIi

Send message
Joined: 29 Aug 09
Posts: 175
Credit: 259,509,919
RAC: 0
Level
Asn
Scientific publications
watwatwatwatwatwatwatwatwatwat
Message 21670 - Posted: 11 Jul 2011, 11:16:16 UTC - in response to Message 21665.  

Thanks Richard.

If anyone tests this, post your findings.

I'll test it later 2day
ID: 21670 · Rating: 0 · rate: Rate + / Rate - Report as offensive     Reply Quote
CTAPbIi

Send message
Joined: 29 Aug 09
Posts: 175
Credit: 259,509,919
RAC: 0
Level
Asn
Scientific publications
watwatwatwatwatwatwatwatwatwat
Message 21673 - Posted: 13 Jul 2011, 23:55:26 UTC

same story - almost everything gone from www.gpugrid.net folder...
ID: 21673 · Rating: 0 · rate: Rate + / Rate - Report as offensive     Reply Quote
Profile Mad Matt

Send message
Joined: 29 Aug 09
Posts: 28
Credit: 101,584,171
RAC: 0
Level
Cys
Scientific publications
watwatwatwatwatwatwatwatwatwatwat
Message 21833 - Posted: 14 Aug 2011, 0:28:52 UTC

Probably too late, but maybe helpful for future tests:

Make a copy of all files needed for the app_info, e.g. cudart32_31_9.dll and so on, put them all into a folder with the app_info and copy all into your GPUGRID folder, overriding the existing files. Alternately you can get the files somewhere from the GPUGRID download server (sorry, but I forgot the link).

Not sure why, but BOINC does not recognize those files present even if they have the same file name.

Hope this helps.


ID: 21833 · Rating: 0 · rate: Rate + / Rate - Report as offensive     Reply Quote
Profile Mad Matt

Send message
Joined: 29 Aug 09
Posts: 28
Credit: 101,584,171
RAC: 0
Level
Cys
Scientific publications
watwatwatwatwatwatwatwatwatwatwat
Message 21834 - Posted: 14 Aug 2011, 9:43:30 UTC

Obviously it does not, I tried myself. But I also tried to translate a working app_info from 6.14 to 6.15 and it did not work either. :(
ID: 21834 · Rating: 0 · rate: Rate + / Rate - Report as offensive     Reply Quote
Previous · 1 · 2

Message boards : Graphics cards (GPUs) : GPUGRID and app_info

©2025 Universitat Pompeu Fabra