Message boards :
News :
*CXCL12_chalcone_umbrella* batch
Message board moderation
Previous · 1 · 2 · 3
Author | Message |
---|---|
Send message Joined: 16 Aug 08 Posts: 87 Credit: 1,248,879,715 RAC: 0 Level ![]() Scientific publications ![]() ![]() ![]() ![]() ![]() ![]() ![]() ![]() ![]() ![]() ![]() ![]() ![]() ![]() ![]() ![]() ![]() ![]() ![]() ![]() ![]() ![]() ![]() ![]() ![]() |
Yep, I have 3 GPUs, and all 6 of my tasks are completed with uploads stalled. I suppose this is a good test of GPU backup projects (they are working on Asteroids, Einstein, and SETI)... but I hope the GPUGrid admins get their upload/space issues resolved :) Don't forget Moo. I like it because the work units are short ( 20 minutes ) so my machine can get back to GPUgrid quicker. Though, that doesn't help when we go from 2-3 units/day to 2-3 days between units like I have seen lately. Oh, and no disc full messages for me. Just HTTP errors. Still seeing those even now. |
Send message Joined: 1 Jan 15 Posts: 1166 Credit: 12,260,898,501 RAC: 869 Level ![]() Scientific publications ![]() ![]() ![]() ![]() ![]() ![]() ![]() ![]() ![]() |
no more "Umbrella" WUs coming ? |
Send message Joined: 28 Mar 09 Posts: 490 Credit: 11,731,645,728 RAC: 47,738 Level ![]() Scientific publications ![]() ![]() ![]() ![]() ![]() ![]() ![]() ![]() ![]() ![]() ![]() ![]() ![]() ![]() ![]() ![]() ![]() ![]() ![]() ![]() ![]() ![]() ![]() ![]() ![]() ![]() ![]() ![]() |
no more "Umbrella" WUs coming ? I hope not. They were a pain! |
Send message Joined: 17 Feb 13 Posts: 181 Credit: 144,871,276 RAC: 0 Level ![]() Scientific publications ![]() ![]() ![]() ![]() ![]() ![]() ![]() ![]() ![]() ![]() ![]() ![]() ![]() |
Oh, I dunno: I processed 28 of these WUs successfully! no more "Umbrella" WUs coming ? |
![]() Send message Joined: 23 Apr 09 Posts: 3968 Credit: 1,995,359,260 RAC: 0 Level ![]() Scientific publications ![]() ![]() ![]() ![]() ![]() ![]() ![]() ![]() ![]() ![]() ![]() ![]() ![]() ![]() ![]() ![]() ![]() ![]() ![]() ![]() ![]() ![]() ![]() ![]() ![]() ![]() ![]() ![]() |
Got through 17 brollies without error but while the fastest took 8,187sec (2h 16min) the slowest took 13,463 sec (3h 44min) - the GPU downclocked - it 'thought' it wasn't using enough resources to justify remaining at a high frequency (boosters off). As I like a bit of a challenge I don't mind troubleshooting and tuning specifically for these tasks, forcing the clocks to remain high and might have run 2 tasks at a time if work continued to flow, but even doing this didn't yield great performances. Some of the performances look horrific, especially for those hampered by the WDDM overhead. I've seen a GTX970 on Linux finish in half the time of my 970 (the card that didn't downclock) and it looks like lots of devices down-clocked to barely functional levels - 28Ksec vs 4Ksec. The big issue was the output file size. That's what caused the server has no disc space errors and stopped people uploading results, until the batch was withdrawn and disk space freed up. Releasing as Beta might have been a better option. FAQ's HOW TO: - Opt out of Beta Tests - Ask for Help |
Send message Joined: 7 Jun 12 Posts: 112 Credit: 1,140,895,172 RAC: 0 Level ![]() Scientific publications ![]() ![]() ![]() ![]() ![]() ![]() ![]() ![]() ![]() ![]() ![]() |
All (71) · In progress (2) · Pending (0) · Valid (21) · Invalid (1) · Error (47) ....47vs71.. im crunch now only on old gtx 680 and laptop gtx 960 4gb. im dont check long time my stats and message board here, just im now say wow :-)) hope they fix it soon for new gf..-) |
Send message Joined: 25 Mar 12 Posts: 103 Credit: 14,948,929,771 RAC: 11,649 Level ![]() Scientific publications ![]() ![]() ![]() ![]() ![]() ![]() ![]() ![]() ![]() ![]() ![]() ![]() ![]() ![]() ![]() ![]() ![]() |
141 wus crunched, only few errors at the beginning due to incorrect upload figure in the WU, then smooth. Sure, I have a good internet connection. |
Send message Joined: 28 Mar 09 Posts: 490 Credit: 11,731,645,728 RAC: 47,738 Level ![]() Scientific publications ![]() ![]() ![]() ![]() ![]() ![]() ![]() ![]() ![]() ![]() ![]() ![]() ![]() ![]() ![]() ![]() ![]() ![]() ![]() ![]() ![]() ![]() ![]() ![]() ![]() ![]() ![]() ![]() |
I like these new GERARD_CXCL12_BestUmbrella WUs. They have good GPU usage and the output files were not too big. I successfully completed 3 so far. Good work! Can I have some more of them? |
Send message Joined: 26 Mar 14 Posts: 101 Credit: 0 RAC: 0 Level ![]() Scientific publications ![]() |
I am sorry for the Umbrella runs, they were somewhat experimental. However, the results look promising so far. I can try to tune some parameters for future releases like the file size (we may be able to reduce it without big impact on the analysis results) but unfortunately they will still be quite CPU consuming because of the aforementioned reasons. Could you post the main reasons why they were a "pain"? :) We can try to find a common solution. I am now sending plenty of classic long WU that should push your GPUs to the limit! :) However, I plan to launch more Umbrella short WU in the future, which to me seem ideal for old GPUs. I'll keep you posted! |
![]() ![]() Send message Joined: 20 Jan 09 Posts: 2380 Credit: 16,897,957,044 RAC: 1 Level ![]() Scientific publications ![]() ![]() ![]() ![]() ![]() ![]() ![]() ![]() ![]() ![]() ![]() ![]() ![]() ![]() ![]() ![]() ![]() ![]() ![]() ![]() ![]() ![]() ![]() ![]() ![]() ![]() |
Could you post the main reasons why they were a "pain"? :) We can try to find a common solution.1. High CPU usage -> low GPU usage -> need of 2 simultaneous short task per GPU -> need of 3 short task per GPU in the queue (presently the limit is 2 per GPU) 2. large output file combined with short runtimes -> upload congestion at the user & your server runs out of space |
![]() Send message Joined: 23 Apr 09 Posts: 3968 Credit: 1,995,359,260 RAC: 0 Level ![]() Scientific publications ![]() ![]() ![]() ![]() ![]() ![]() ![]() ![]() ![]() ![]() ![]() ![]() ![]() ![]() ![]() ![]() ![]() ![]() ![]() ![]() ![]() ![]() ![]() ![]() ![]() ![]() ![]() ![]() |
Also, the lack of testing, communication, advice on settings/setup, and GPU clocks dropping off to non-boost rates. Larger output files catch people out with contention, bandwidth limiting, peak hours throttling and possibly disk space and RAM for some. Not enough tasks to go around means we have to add other projects or run dry. FAQ's HOW TO: - Opt out of Beta Tests - Ask for Help |
Send message Joined: 28 Mar 09 Posts: 490 Credit: 11,731,645,728 RAC: 47,738 Level ![]() Scientific publications ![]() ![]() ![]() ![]() ![]() ![]() ![]() ![]() ![]() ![]() ![]() ![]() ![]() ![]() ![]() ![]() ![]() ![]() ![]() ![]() ![]() ![]() ![]() ![]() ![]() ![]() ![]() ![]() |
I am sorry for the Umbrella runs, they were somewhat experimental. However, the results look promising so far. I agree pretty much with what Retvari Zoltan and skgiven mentioned in their posts. Though, I don't think having 3 short tasks per GPU would have made much of a difference, (having 2 or 3 finished WUs per GPU not being able to upload, and not getting any new WUs, I still would have spent most of that Sunday crunching my back up project). What would have made a difference is not releasing "somewhat experimental" WUs on the weekends, when staff level is limited. Release them, during the week, when most everybody is at work, to deal with potential problems. |
Send message Joined: 1 Jan 15 Posts: 1166 Credit: 12,260,898,501 RAC: 869 Level ![]() Scientific publications ![]() ![]() ![]() ![]() ![]() ![]() ![]() ![]() ![]() |
[quote]need of 3 short task per GPU in the Queue (presently the limit is 2 per GPU) once these WUs are distributed again, it really would make a lot of sense to raise the limit from 2 to 3 per GPU ! |
Send message Joined: 5 Jan 09 Posts: 670 Credit: 2,498,095,550 RAC: 0 Level ![]() Scientific publications ![]() ![]() ![]() ![]() ![]() ![]() ![]() ![]() ![]() ![]() ![]() ![]() ![]() ![]() ![]() ![]() ![]() ![]() ![]() ![]() ![]() ![]() ![]() ![]() ![]() ![]() |
[quote]need of 3 short task per GPU in the Queue (presently the limit is 2 per GPU) I disagree it would mean even more units held up for 5 days by the hosts that never return a completed WU and those that error after a long period of time. We need different queues for the really fast cards such as 980ti, 980, Titan, 970, 780ti and a 2 day deadline. Mid cards could have a 3 day deadline. Slow cards could remain on 5 days, with an adjusted percentage of WU's allocated to each queue. We could also accelerate the drop of WU's available to hosts that don't return or consistently error. However as Gerard has said in another post he is already overun with work and probably does not have the time to do any of these things. |
Send message Joined: 28 Jul 12 Posts: 819 Credit: 1,591,285,971 RAC: 0 Level ![]() Scientific publications ![]() ![]() ![]() ![]() ![]() ![]() ![]() ![]() ![]() ![]() ![]() ![]() ![]() ![]() ![]() |
To offer a minority view, I was quite happy to get them. I don't normally run the shorts, but the science looks very interesting, and my GTX 750 Tis are not that good on the longs anymore. If they don't use much GPU power, but more CPU power, that is OK if that is what the calculations call for. You can't change the science or math just to heat up the cards more. Also, I have large enough upload bandwidth (4 Mbps) that I did not notice any problems there, or with memory, etc. But a warning as to all of this would undoubtedly be a good idea, since it may push many machines over the edge in one way or another.. |
Send message Joined: 1 Jan 15 Posts: 1166 Credit: 12,260,898,501 RAC: 869 Level ![]() Scientific publications ![]() ![]() ![]() ![]() ![]() ![]() ![]() ![]() ![]() |
However as Gerard has said in another post he is already overun with work and probably does not have the time to do any of these things. that's why we all should hope that the new students which where expected for January will finally come on bord. The amount of work which Gerard is doing, all by himself, is terrific! I guess, at some point he deserves rest and recreation :-) |
Send message Joined: 1 Jan 15 Posts: 1166 Credit: 12,260,898,501 RAC: 869 Level ![]() Scientific publications ![]() ![]() ![]() ![]() ![]() ![]() ![]() ![]() ![]() |
High CPU usage -> low GPU usage -> need of 2 simultaneous short task per GPU -> need of 3 short task per GPU in the queue the current short runs "Enamine_Umbrella" use some 50-60% of a high-end GPU. As already said above by one of our power crunchers: the Limit of 2 such WUs per GPU should be increased to 3, if not to 4. |
Send message Joined: 1 Jan 15 Posts: 1166 Credit: 12,260,898,501 RAC: 869 Level ![]() Scientific publications ![]() ![]() ![]() ![]() ![]() ![]() ![]() ![]() ![]() |
the current short runs "Enamine_Umbrella" use some 50-60% of a high-end GPU. any news on this? |
![]() ![]() Send message Joined: 20 Jan 09 Posts: 2380 Credit: 16,897,957,044 RAC: 1 Level ![]() Scientific publications ![]() ![]() ![]() ![]() ![]() ![]() ![]() ![]() ![]() ![]() ![]() ![]() ![]() ![]() ![]() ![]() ![]() ![]() ![]() ![]() ![]() ![]() ![]() ![]() ![]() ![]() |
the current short runs "Enamine_Umbrella" use some 50-60% of a high-end GPU. The recent workunits are not *that* problematic, so this is not that important right now. |
![]() Send message Joined: 23 Apr 09 Posts: 3968 Credit: 1,995,359,260 RAC: 0 Level ![]() Scientific publications ![]() ![]() ![]() ![]() ![]() ![]() ![]() ![]() ![]() ![]() ![]() ![]() ![]() ![]() ![]() ![]() ![]() ![]() ![]() ![]() ![]() ![]() ![]() ![]() ![]() ![]() ![]() ![]() |
There are no short runs at present, when there are short runs there's not always many WU's and the batches don't last as long - Server Status FAQ's HOW TO: - Opt out of Beta Tests - Ask for Help |
©2025 Universitat Pompeu Fabra