Message boards :
Graphics cards (GPUs) :
GPU Task Performance (vs. CPU core usage, app_config, multiple GPU tasks on 1 GPU, etc.)
Message board moderation
Previous · 1 · 2 · 3 · 4 · 5 · Next
| Author | Message |
|---|---|
|
Send message Joined: 21 Feb 09 Posts: 497 Credit: 700,690,702 RAC: 0 Level ![]() Scientific publications ![]() ![]() ![]() ![]() ![]() ![]() ![]() ![]() ![]() ![]() ![]() ![]() ![]() ![]() ![]() ![]() ![]() ![]() ![]() ![]()
|
Yesterday I retired my trusty ASUS GTX 460, replacing it with an ASUS GTX 660. I activated app_config.xml, restarted BOINC, then the 25% in-process NOELIA. Immediately another NOELIA appeared and both crunched together. The first WU finished this morning (18 hours run time, full bonus), but it was not until it had uploaded that another WU came down to replace it. So I lost more than an hour of crunching. Previously I had run with minimum work buffer set to 0.01, which gave me a new WU download about 10 minutes before the active WU finished. I just set the buffer to 2.00 days. No reaction, even after a few clicks on BOINC Update. Any thoughts? Thanks. It's early days but what I reckon is that the doom & gloom about lost bonuses when doing 2X is unfounded, at least for me. Both my current WUs will complete well within 24 hours. |
|
Send message Joined: 11 Oct 08 Posts: 1127 Credit: 1,901,927,545 RAC: 0 Level ![]() Scientific publications ![]() ![]() ![]() ![]() ![]() ![]() ![]() ![]() ![]() ![]() ![]() ![]() ![]() ![]() ![]() ![]() ![]() ![]() ![]() ![]() ![]() ![]() ![]() ![]() ![]() ![]() ![]() ![]() ![]()
|
Hmm... Do you have any GPU exclusions in your cc_config.xml file? Also, what version of BOINC are you using? Hopefully v7.0.64. Behavior I've noticed previously is that... if BOINC has a task that's running, it doesn't consider the GPU "idle", and so it only fetches work when the work left on *that task* is below the "Minimum work buffer". Assuming you want to keep the GPU busy with 2-tasks-at-1-time, I think you'd have to increase the "Minimum work buffer" to be larger than the time necessary to complete a task running 2-at-1-time (note: This is larger than the time necessary to complete a task running 1-at-a-time). But I noticed you said you increased it already, to 2 days! I would have thought 2 days would have been enough, but maybe it needs to be even higher? I kept my GTX 460 1GB in this machine, alongside my GTX 660 Ti, and so.... because the GTX 460 only has 1GB, I can't do 2-at-a-time unless I exclude that GPU. So I only do 1-at-a-time. |
|
Send message Joined: 21 Feb 09 Posts: 497 Credit: 700,690,702 RAC: 0 Level ![]() Scientific publications ![]() ![]() ![]() ![]() ![]() ![]() ![]() ![]() ![]() ![]() ![]() ![]() ![]() ![]() ![]() ![]() ![]() ![]() ![]() ![]()
|
Hmm... Do you have any GPU exclusions in your cc_config.xml file? BOINC 7.0.64 (x64), and here's my cc_config.xml file: <cc_config> <options> <report_results_immediately>1</report_results_immediately> </options> </cc_config> |
|
Send message Joined: 11 Oct 08 Posts: 1127 Credit: 1,901,927,545 RAC: 0 Level ![]() Scientific publications ![]() ![]() ![]() ![]() ![]() ![]() ![]() ![]() ![]() ![]() ![]() ![]() ![]() ![]() ![]() ![]() ![]() ![]() ![]() ![]() ![]() ![]() ![]() ![]() ![]() ![]() ![]() ![]() ![]()
|
You should take that report_results_immediately flag out of there -- BOINC has a feature where the project can request that now, and GPUGrid does request it, so... by you using it, you're only increasing server load on other projects that may not need results immediately. As far as your problem goes... Don't click update. BOINC evaluates whether it needs to fetch work every minute. So, I recommend increasing the Minimum buffer by a little (by 0.1 or 0.2), then waiting 2 minutes, to see if it got work. If it didn't, keep increasing it a little at a time, until it does what you want. Actually, you should be able to get it to fetch work if you increase the Minimum buffer to be a bit larger than the current "Remaining (estimated)" time of the current task, but again, you really should set it to be a bit higher than the largest "Remaining (estimated)" time possible for a given task, to ensure you always have 2 tasks to run. What other projects are you attached to? Are any of them GPU projects that get work? Are you sure you have network activity allowed? (BOINC -> Activity -> Network)? Are you sure you're setting the "Minimum work buffer"? |
BeyondSend message Joined: 23 Nov 08 Posts: 1112 Credit: 6,162,416,256 RAC: 0 Level ![]() Scientific publications ![]() ![]() ![]() ![]() ![]() ![]() ![]() ![]() ![]() ![]() ![]() ![]() ![]() ![]() ![]() ![]() ![]() ![]() ![]() ![]() ![]() ![]() ![]()
|
<cc_config> Personally I'd leave the report line in and try adding this one: <fetch_on_update>1</fetch_on_update> See if it makes a difference. |
|
Send message Joined: 21 Feb 09 Posts: 497 Credit: 700,690,702 RAC: 0 Level ![]() Scientific publications ![]() ![]() ![]() ![]() ![]() ![]() ![]() ![]() ![]() ![]() ![]() ![]() ![]() ![]() ![]() ![]() ![]() ![]() ![]() ![]()
|
Personally I'd leave the report line in and try adding this one: Did that, suspended GPUGRID, exited BOINC, restarted BOINC, restarted GPUGRID, issued the "Update" command a few times. Nada... In answer to Jacob's questions: I upped the minimum buffer several time. Nothing. In a fit of pique I set it to 10 days! Nothing. I am attached to POEM but it is set to get no work. BOINC -> Activity -> Network = activity always available. I confirm I'm setting the "Minimum work buffer", in Tools / Computing Preferences. |
BeyondSend message Joined: 23 Nov 08 Posts: 1112 Credit: 6,162,416,256 RAC: 0 Level ![]() Scientific publications ![]() ![]() ![]() ![]() ![]() ![]() ![]() ![]() ![]() ![]() ![]() ![]() ![]() ![]() ![]() ![]() ![]() ![]() ![]() ![]() ![]() ![]() ![]()
|
What does it say in the BOINC log when you manually update GPUGrid? |
|
Send message Joined: 21 Feb 09 Posts: 497 Credit: 700,690,702 RAC: 0 Level ![]() Scientific publications ![]() ![]() ![]() ![]() ![]() ![]() ![]() ![]() ![]() ![]() ![]() ![]() ![]() ![]() ![]() ![]() ![]() ![]() ![]() ![]()
|
What does it say in the BOINC log when you manually update GPUGrid? |
|
Send message Joined: 11 Oct 08 Posts: 1127 Credit: 1,901,927,545 RAC: 0 Level ![]() Scientific publications ![]() ![]() ![]() ![]() ![]() ![]() ![]() ![]() ![]() ![]() ![]() ![]() ![]() ![]() ![]() ![]() ![]() ![]() ![]() ![]() ![]() ![]() ![]() ![]() ![]() ![]() ![]() ![]() ![]()
|
Alright, let's try this: Set your cc_config to show work_fetch_debug, using something like this: --------------------------------------------------------------- <cc_config> <log_flags> <work_fetch_debug>1</work_fetch_debug> </log_flags> <options> </options> </cc_config> --------------------------------------------------------------- Then restart BOINC, and watch Event Viewer to see work fetch run. You'll see it say "[work_fetch] work fetch start" at the beginning of a run. In the Event Viewer, select all the lines from "Starting BOINC" all the way until the 2nd instance of "[work_fetch] work fetch start", so that we include all the messages from the entire first run. Then hit "Copy selected", and then paste here in a reply. I'm generally very good at reading these work fetch logs. |
|
Send message Joined: 11 Oct 08 Posts: 1127 Credit: 1,901,927,545 RAC: 0 Level ![]() Scientific publications ![]() ![]() ![]() ![]() ![]() ![]() ![]() ![]() ![]() ![]() ![]() ![]() ![]() ![]() ![]() ![]() ![]() ![]() ![]() ![]() ![]() ![]() ![]() ![]() ![]() ![]() ![]() ![]() ![]()
|
Oh, well there's your answer. If it says you've reached a limit on tasks in progress... BOINC did ask for work, but GPUGrid only lets you have so many tasks. I see (by clicking your name and looking at your computers) that you currently have 2 in progress; perhaps they only allow 2. Do you need more than 2? |
|
Send message Joined: 21 Feb 09 Posts: 497 Credit: 700,690,702 RAC: 0 Level ![]() Scientific publications ![]() ![]() ![]() ![]() ![]() ![]() ![]() ![]() ![]() ![]() ![]() ![]() ![]() ![]() ![]() ![]() ![]() ![]() ![]() ![]()
|
Oh, well there's your answer. If it says you've reached a limit on tasks in progress... BOINC did ask for work, but GPUGrid only lets you have so many tasks. I see (by clicking your name and looking at your computers) that you currently have 2 in progress; perhaps they only allow 2. Do you need more than 2? No! All I want is what I got in the 1X scenario; a new WU a few minutes before the active WU finishes [in the X2 scenario, before one of the two active WUs finishes]. |
|
Send message Joined: 11 Oct 08 Posts: 1127 Credit: 1,901,927,545 RAC: 0 Level ![]() Scientific publications ![]() ![]() ![]() ![]() ![]() ![]() ![]() ![]() ![]() ![]() ![]() ![]() ![]() ![]() ![]() ![]() ![]() ![]() ![]() ![]() ![]() ![]() ![]() ![]() ![]() ![]() ![]() ![]() ![]()
|
So, you DO need more than 2. You need a 3rd one to be downloaded, a couple minutes before either of the 2 running ones are done... right? :) I wonder if GPUGrid has things setup to only allow 2-tasks-per-GPU. (For reference, I've had 5 or 6 tasks in progress before, but my system has 3 GPUs). I'm not sure if I'll be able to help you test/confirm. To do that, I think I'd have to pull some GPUs out. But it sounds like your setup will probably be asking GPUGrid for work a lot, and not getting any, which is not ideal. |
|
Send message Joined: 21 Feb 09 Posts: 497 Credit: 700,690,702 RAC: 0 Level ![]() Scientific publications ![]() ![]() ![]() ![]() ![]() ![]() ![]() ![]() ![]() ![]() ![]() ![]() ![]() ![]() ![]() ![]() ![]() ![]() ![]() ![]()
|
Alright, let's try this: 25/05/2013 15:35:29 | | Starting BOINC client version 7.0.64 for windows_x86_64 25/05/2013 15:35:29 | | log flags: file_xfer, sched_ops, task, work_fetch_debug 25/05/2013 15:35:29 | | Libraries: libcurl/7.25.0 OpenSSL/1.0.1 zlib/1.2.6 25/05/2013 15:35:29 | | Data directory: C:\ProgramData\BOINC 25/05/2013 15:35:29 | | Running under account TOM 25/05/2013 15:35:29 | | Processor: 8 GenuineIntel Intel(R) Core(TM) i7 CPU 920 @ 2.67GHz [Family 6 Model 26 Stepping 5] 25/05/2013 15:35:29 | | Processor features: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss htt tm pni ssse3 cx16 sse4_1 sse4_2 popcnt syscall nx lm vmx tm2 pbe 25/05/2013 15:35:29 | | OS: Microsoft Windows 7: Home Premium x64 Edition, Service Pack 1, (06.01.7601.00) 25/05/2013 15:35:29 | | Memory: 5.99 GB physical, 11.98 GB virtual 25/05/2013 15:35:29 | | Disk: 465.76 GB total, 170.61 GB free 25/05/2013 15:35:29 | | Local time is UTC +2 hours 25/05/2013 15:35:29 | | VirtualBox version: 4.1.10 25/05/2013 15:35:29 | | CUDA: NVIDIA GPU 0: GeForce GTX 660 (driver version 320.18, CUDA version 5.50, compute capability 3.0, 2048MB, 1903MB available, 1982 GFLOPS peak) 25/05/2013 15:35:29 | | OpenCL: NVIDIA GPU 0: GeForce GTX 660 (driver version 320.18, device version OpenCL 1.1 CUDA, 2048MB, 1903MB available, 1982 GFLOPS peak) 25/05/2013 15:35:29 | GPUGRID | Found app_config.xml 25/05/2013 15:35:29 | Poem@Home | URL http://boinc.fzk.de/poem/; Computer ID 149338; resource share 75 25/05/2013 15:35:29 | GPUGRID | URL http://www.gpugrid.net/; Computer ID 151159; resource share 100 25/05/2013 15:35:29 | GPUGRID | General prefs: from GPUGRID (last modified 27-Apr-2013 11:10:03) 25/05/2013 15:35:29 | GPUGRID | Computer location: home 25/05/2013 15:35:29 | GPUGRID | General prefs: no separate prefs for home; using your defaults 25/05/2013 15:35:29 | | Reading preferences override file 25/05/2013 15:35:29 | | Preferences: 25/05/2013 15:35:29 | | max memory usage when active: 3067.49MB 25/05/2013 15:35:29 | | max memory usage when idle: 5521.48MB 25/05/2013 15:35:29 | | max disk usage: 10.00GB 25/05/2013 15:35:29 | | max CPUs used: 4 25/05/2013 15:35:29 | | (to change preferences, visit a project web site or select Preferences in the Manager) 25/05/2013 15:35:29 | | [work_fetch] Request work fetch: Prefs update 25/05/2013 15:35:29 | | [work_fetch] Request work fetch: Startup 25/05/2013 15:35:29 | | Not using a proxy 25/05/2013 15:35:29 | | [work_fetch] work fetch start 25/05/2013 15:35:29 | | [work_fetch] choose_project() for NVIDIA: buffer_low: yes; sim_excluded_instances 1 25/05/2013 15:35:29 | | [work_fetch] no eligible project for NVIDIA 25/05/2013 15:35:29 | | [work_fetch] choose_project() for CPU: buffer_low: yes; sim_excluded_instances 0 25/05/2013 15:35:29 | | [work_fetch] no eligible project for CPU 25/05/2013 15:35:29 | | [work_fetch] ------- start work fetch state ------- 25/05/2013 15:35:29 | | [work_fetch] target work buffer: 86400.00 + 0.00 sec 25/05/2013 15:35:29 | | [work_fetch] --- project states --- 25/05/2013 15:35:29 | Poem@Home | [work_fetch] REC 244.210 prio 0.000000 can't req work: suspended via Manager 25/05/2013 15:35:29 | GPUGRID | [work_fetch] REC 39539.005 prio -0.203740 can't req work: suspended via Manager 25/05/2013 15:35:29 | | [work_fetch] --- state for CPU --- 25/05/2013 15:35:29 | | [work_fetch] shortfall 345600.00 nidle 4.00 saturated 0.00 busy 0.00 25/05/2013 15:35:29 | Poem@Home | [work_fetch] fetch share 0.000 (blocked by prefs) 25/05/2013 15:35:29 | GPUGRID | [work_fetch] fetch share 0.000 (no apps) 25/05/2013 15:35:29 | | [work_fetch] --- state for NVIDIA --- 25/05/2013 15:35:29 | | [work_fetch] shortfall 86400.00 nidle 1.00 saturated 0.00 busy 0.00 25/05/2013 15:35:29 | Poem@Home | [work_fetch] fetch share 0.000 25/05/2013 15:35:29 | GPUGRID | [work_fetch] fetch share 0.000 25/05/2013 15:35:29 | | [work_fetch] ------- end work fetch state ------- 25/05/2013 15:35:29 | | [work_fetch] No project chosen for work fetch 25/05/2013 15:35:47 | GPUGRID | project resumed by user 25/05/2013 15:35:47 | | [work_fetch] Request work fetch: project resumed 25/05/2013 15:35:48 | GPUGRID | Restarting task 306px11x1-NOELIA_klebe_run2-2-3-RND3462_0 using acemdlong version 618 (cuda42) in slot 1 25/05/2013 15:35:48 | GPUGRID | Restarting task I2HDQ_6R14-SDOERR_2HDQd-1-4-RND2918_0 using acemdlong version 618 (cuda42) in slot 0 |
|
Send message Joined: 11 Oct 08 Posts: 1127 Credit: 1,901,927,545 RAC: 0 Level ![]() Scientific publications ![]() ![]() ![]() ![]() ![]() ![]() ![]() ![]() ![]() ![]() ![]() ![]() ![]() ![]() ![]() ![]() ![]() ![]() ![]() ![]() ![]() ![]() ![]() ![]() ![]() ![]() ![]() ![]() ![]()
|
If you read that work fetch log, you'll see that the reason it didn't fetch work there, was because "can't req work: suspended via Manager" Anyway, for your scenario, you do want a 3rd task. However, I have just confirmed (by pulling 2 of my 3 GPUs out of the system) that GPUGrid does only allow 2 in-progress tasks on a 1-GPU-system. That being said... when the "completed" task gets reported, if you have a sufficiently high minimum buffer at that point, then it might get a new task. So... try keeping your "Minimum work buffer" high enough (maybe set to 2 days), but then let it run long enough to complete a task. Don't do anything manual (don't click Update, don't suspend it, just let it go), and watch what it does. With a 2-day-buffer, does it automatically request and get a new task at the same time as reporting the completed one? Please report your findings. I also am doing this test, since I'm curious. |
|
Send message Joined: 21 Feb 09 Posts: 497 Credit: 700,690,702 RAC: 0 Level ![]() Scientific publications ![]() ![]() ![]() ![]() ![]() ![]() ![]() ![]() ![]() ![]() ![]() ![]() ![]() ![]() ![]() ![]() ![]() ![]() ![]() ![]()
|
If you read that work fetch log, you'll see that the reason it didn't fetch work there, was because "can't req work: suspended via Manager" Surely it's not just my scenario? Anyone running 2X wants a third WU before the WU that's about to complete actually completes. Yes? That being said... when the "completed" task gets reported, if you have a sufficiently high minimum buffer at that point, then it might get a new task. As I said this morning, I got a new WU when the completed task had been uploaded - reported - missing an hour+ of crunching. That was with a buffer of 0.01. My buffer is now at two days. If BOINC reports at WU-finished time, not just WU uploaded time, there's a chance I'll now see a new WU downloaded as soon as one finishes. The proof of the pudding comes in about 100 minutes, when my most-advanced WU completes. If, with a two day buffer, I still have to wait for a new WU until the upload is done, then I have to consider going back to 1X. |
|
Send message Joined: 11 Oct 08 Posts: 1127 Credit: 1,901,927,545 RAC: 0 Level ![]() Scientific publications ![]() ![]() ![]() ![]() ![]() ![]() ![]() ![]() ![]() ![]() ![]() ![]() ![]() ![]() ![]() ![]() ![]() ![]() ![]() ![]() ![]() ![]() ![]() ![]() ![]() ![]() ![]() ![]() ![]()
|
If you read that work fetch log, you'll see that the reason it didn't fetch work there, was because "can't req work: suspended via Manager" Correct. This scenario probably fits anyone trying to do 2-tasks-on-1-GPU. That being said... when the "completed" task gets reported, if you have a sufficiently high minimum buffer at that point, then it might get a new task. I hope it works out for you (and for me too!). But, even if it doesn't work, why would you have to go back to 1-at-a-time? Wouldn't you prefer "usually-2-at-a-time" instead of "always-1-at-a-time"? Just trying to understand your logic. Regards, Jacob |
|
Send message Joined: 21 Feb 09 Posts: 497 Credit: 700,690,702 RAC: 0 Level ![]() Scientific publications ![]() ![]() ![]() ![]() ![]() ![]() ![]() ![]() ![]() ![]() ![]() ![]() ![]() ![]() ![]() ![]() ![]() ![]() ![]() ![]()
|
I hope it works out for you (and for me too!). But, even if it doesn't work, why would you have to go back to 1-at-a-time? Wouldn't you prefer "usually-2-at-a-time" instead of "always-1-at-a-time"? Just trying to understand your logic. I've followed 2x on these fora this past month. Nice idea! But I'm no techie and I have not seen anything that nails down the performance improvements vs. 1X. I get the impression they are marginal. If whatever the performance improvement is, minus the lost crunch time waiting for a completed WU to upload, is significantly positive, fine. If not? ....... That's my logic. Cheers, Tom |
|
Send message Joined: 11 Oct 08 Posts: 1127 Credit: 1,901,927,545 RAC: 0 Level ![]() Scientific publications ![]() ![]() ![]() ![]() ![]() ![]() ![]() ![]() ![]() ![]() ![]() ![]() ![]() ![]() ![]() ![]() ![]() ![]() ![]() ![]() ![]() ![]() ![]() ![]() ![]() ![]() ![]() ![]() ![]()
|
If it is even only marginally better... I'd still think that "usually 2-at-a-time but always at least 1-at-a-time" would be always better than "always just 1-at-a-time". ie: Not having that 3rd task "ready" shouldn't make it any worse than running 1-at-a-time. :) |
|
Send message Joined: 21 Feb 09 Posts: 497 Credit: 700,690,702 RAC: 0 Level ![]() Scientific publications ![]() ![]() ![]() ![]() ![]() ![]() ![]() ![]() ![]() ![]() ![]() ![]() ![]() ![]() ![]() ![]() ![]() ![]() ![]() ![]()
|
Not having that 3rd task "ready" shouldn't make it any worse than running 1-at-a-time.:) I guess even you have yet to come to grips with measurable performance improvement.. |
|
Send message Joined: 11 Oct 08 Posts: 1127 Credit: 1,901,927,545 RAC: 0 Level ![]() Scientific publications ![]() ![]() ![]() ![]() ![]() ![]() ![]() ![]() ![]() ![]() ![]() ![]() ![]() ![]() ![]() ![]() ![]() ![]() ![]() ![]() ![]() ![]() ![]() ![]() ![]() ![]() ![]() ![]() ![]()
|
Not having that 3rd task "ready" shouldn't make it any worse than running 1-at-a-time.:) I have no idea what you meant by that. My testing has always showed increased GPU Load, which is the primary indicator of increased performance. Also, I have confirmed performance improvement (by looking at task run times), on World Community Grid, and POEM@Home, by running multiple tasks on a single GPU, for those projects. The only reason I don't have conclusive evidence for GPUGrid performance improvement is because I had to pause my testing (per Toni's request) for an unrelated problem.... and also because I had been running the GTX 460. For now, I'm testing running GPUGrid only on the GTX 660 Ti, for the purposes of testing your "3rd task" condition, and also now to test on getting that conclusive evidence. I'm hoping your results are positive. Since you too have increased GPU Load, I believe you will see increased task throughput. Regards, Jacob |
©2025 Universitat Pompeu Fabra