*CXCL12_chalcone_umbrella* batch

Message boards : News : *CXCL12_chalcone_umbrella* batch
Message board moderation

To post messages, you must log in.

Previous · 1 · 2 · 3

AuthorMessage
fractal

Send message
Joined: 16 Aug 08
Posts: 87
Credit: 1,248,879,715
RAC: 0
Level
Met
Scientific publications
watwatwatwatwatwatwatwatwatwatwatwatwatwatwatwatwatwatwatwatwatwatwatwatwat
Message 42959 - Posted: 8 Mar 2016, 17:08:46 UTC - in response to Message 42934.  

Yep, I have 3 GPUs, and all 6 of my tasks are completed with uploads stalled. I suppose this is a good test of GPU backup projects (they are working on Asteroids, Einstein, and SETI)... but I hope the GPUGrid admins get their upload/space issues resolved :)

Also, POEM@home and Milkyway@home

Emailed Gianni regarding the disks full/upload problem.

Don't forget Moo. I like it because the work units are short ( 20 minutes ) so my machine can get back to GPUgrid quicker. Though, that doesn't help when we go from 2-3 units/day to 2-3 days between units like I have seen lately.

Oh, and no disc full messages for me. Just HTTP errors. Still seeing those even now.
ID: 42959 · Rating: 0 · rate: Rate + / Rate - Report as offensive     Reply Quote
Erich56

Send message
Joined: 1 Jan 15
Posts: 1166
Credit: 12,260,898,501
RAC: 960
Level
Trp
Scientific publications
watwatwatwatwatwatwatwatwat
Message 42970 - Posted: 9 Mar 2016, 21:21:19 UTC

no more "Umbrella" WUs coming ?
ID: 42970 · Rating: 0 · rate: Rate + / Rate - Report as offensive     Reply Quote
Bedrich Hajek

Send message
Joined: 28 Mar 09
Posts: 490
Credit: 11,731,645,728
RAC: 52,725
Level
Trp
Scientific publications
watwatwatwatwatwatwatwatwatwatwatwatwatwatwatwatwatwatwatwatwatwatwatwatwatwatwatwat
Message 42971 - Posted: 9 Mar 2016, 23:19:21 UTC - in response to Message 42970.  

no more "Umbrella" WUs coming ?



I hope not. They were a pain!





ID: 42971 · Rating: 0 · rate: Rate + / Rate - Report as offensive     Reply Quote
John C MacAlister

Send message
Joined: 17 Feb 13
Posts: 181
Credit: 144,871,276
RAC: 0
Level
Cys
Scientific publications
watwatwatwatwatwatwatwatwatwatwatwatwat
Message 42972 - Posted: 10 Mar 2016, 5:25:31 UTC - in response to Message 42971.  

Oh, I dunno: I processed 28 of these WUs successfully!


no more "Umbrella" WUs coming ?



I hope not. They were a pain!






ID: 42972 · Rating: 0 · rate: Rate + / Rate - Report as offensive     Reply Quote
Profile skgiven
Volunteer moderator
Volunteer tester
Avatar

Send message
Joined: 23 Apr 09
Posts: 3968
Credit: 1,995,359,260
RAC: 0
Level
His
Scientific publications
watwatwatwatwatwatwatwatwatwatwatwatwatwatwatwatwatwatwatwatwatwatwatwatwatwatwatwat
Message 42974 - Posted: 10 Mar 2016, 9:23:54 UTC - in response to Message 42972.  
Last modified: 10 Mar 2016, 9:41:17 UTC

Got through 17 brollies without error but while the fastest took 8,187sec (2h 16min) the slowest took 13,463 sec (3h 44min) - the GPU downclocked - it 'thought' it wasn't using enough resources to justify remaining at a high frequency (boosters off).

As I like a bit of a challenge I don't mind troubleshooting and tuning specifically for these tasks, forcing the clocks to remain high and might have run 2 tasks at a time if work continued to flow, but even doing this didn't yield great performances.

Some of the performances look horrific, especially for those hampered by the WDDM overhead. I've seen a GTX970 on Linux finish in half the time of my 970 (the card that didn't downclock) and it looks like lots of devices down-clocked to barely functional levels - 28Ksec vs 4Ksec.

The big issue was the output file size. That's what caused the server has no disc space errors and stopped people uploading results, until the batch was withdrawn and disk space freed up.

Releasing as Beta might have been a better option.
FAQ's

HOW TO:
- Opt out of Beta Tests
- Ask for Help
ID: 42974 · Rating: 0 · rate: Rate + / Rate - Report as offensive     Reply Quote
Jozef J

Send message
Joined: 7 Jun 12
Posts: 112
Credit: 1,140,895,172
RAC: 0
Level
Met
Scientific publications
watwatwatwatwatwatwatwatwatwatwat
Message 42975 - Posted: 10 Mar 2016, 12:05:54 UTC

All (71) · In progress (2) · Pending (0) · Valid (21) · Invalid (1) · Error (47) ....47vs71.. im crunch now only on old gtx 680 and laptop gtx 960 4gb.
im dont check long time my stats and message board here, just im now say wow :-)) hope they fix it soon for new gf..-)
ID: 42975 · Rating: 0 · rate: Rate + / Rate - Report as offensive     Reply Quote
Trotador

Send message
Joined: 25 Mar 12
Posts: 103
Credit: 14,948,929,771
RAC: 12,866
Level
Trp
Scientific publications
watwatwatwatwatwatwatwatwatwatwatwatwatwatwatwatwat
Message 42982 - Posted: 11 Mar 2016, 19:40:00 UTC

141 wus crunched, only few errors at the beginning due to incorrect upload figure in the WU, then smooth.

Sure, I have a good internet connection.
ID: 42982 · Rating: 0 · rate: Rate + / Rate - Report as offensive     Reply Quote
Bedrich Hajek

Send message
Joined: 28 Mar 09
Posts: 490
Credit: 11,731,645,728
RAC: 52,725
Level
Trp
Scientific publications
watwatwatwatwatwatwatwatwatwatwatwatwatwatwatwatwatwatwatwatwatwatwatwatwatwatwatwat
Message 43006 - Posted: 15 Mar 2016, 0:00:01 UTC
Last modified: 15 Mar 2016, 0:00:20 UTC

I like these new GERARD_CXCL12_BestUmbrella WUs. They have good GPU usage and the output files were not too big. I successfully completed 3 so far.

Good work!

Can I have some more of them?
ID: 43006 · Rating: 0 · rate: Rate + / Rate - Report as offensive     Reply Quote
Gerard

Send message
Joined: 26 Mar 14
Posts: 101
Credit: 0
RAC: 0
Level

Scientific publications
wat
Message 43012 - Posted: 15 Mar 2016, 10:31:38 UTC - in response to Message 43006.  

I am sorry for the Umbrella runs, they were somewhat experimental. However, the results look promising so far.

I can try to tune some parameters for future releases like the file size (we may be able to reduce it without big impact on the analysis results) but unfortunately they will still be quite CPU consuming because of the aforementioned reasons. Could you post the main reasons why they were a "pain"? :) We can try to find a common solution.

I am now sending plenty of classic long WU that should push your GPUs to the limit! :) However, I plan to launch more Umbrella short WU in the future, which to me seem ideal for old GPUs.

I'll keep you posted!
ID: 43012 · Rating: 0 · rate: Rate + / Rate - Report as offensive     Reply Quote
Profile Retvari Zoltan
Avatar

Send message
Joined: 20 Jan 09
Posts: 2380
Credit: 16,897,957,044
RAC: 1
Level
Trp
Scientific publications
watwatwatwatwatwatwatwatwatwatwatwatwatwatwatwatwatwatwatwatwatwatwatwatwatwat
Message 43015 - Posted: 15 Mar 2016, 12:20:03 UTC - in response to Message 43012.  
Last modified: 15 Mar 2016, 12:20:22 UTC

Could you post the main reasons why they were a "pain"? :) We can try to find a common solution.
1. High CPU usage -> low GPU usage -> need of 2 simultaneous short task per GPU -> need of 3 short task per GPU in the queue
(presently the limit is 2 per GPU)
2. large output file combined with short runtimes -> upload congestion at the user & your server runs out of space
ID: 43015 · Rating: 0 · rate: Rate + / Rate - Report as offensive     Reply Quote
Profile skgiven
Volunteer moderator
Volunteer tester
Avatar

Send message
Joined: 23 Apr 09
Posts: 3968
Credit: 1,995,359,260
RAC: 0
Level
His
Scientific publications
watwatwatwatwatwatwatwatwatwatwatwatwatwatwatwatwatwatwatwatwatwatwatwatwatwatwatwat
Message 43018 - Posted: 15 Mar 2016, 20:13:27 UTC - in response to Message 43015.  

Also, the lack of testing, communication, advice on settings/setup, and GPU clocks dropping off to non-boost rates.
Larger output files catch people out with contention, bandwidth limiting, peak hours throttling and possibly disk space and RAM for some.
Not enough tasks to go around means we have to add other projects or run dry.
FAQ's

HOW TO:
- Opt out of Beta Tests
- Ask for Help
ID: 43018 · Rating: 0 · rate: Rate + / Rate - Report as offensive     Reply Quote
Bedrich Hajek

Send message
Joined: 28 Mar 09
Posts: 490
Credit: 11,731,645,728
RAC: 52,725
Level
Trp
Scientific publications
watwatwatwatwatwatwatwatwatwatwatwatwatwatwatwatwatwatwatwatwatwatwatwatwatwatwatwat
Message 43020 - Posted: 15 Mar 2016, 23:47:33 UTC - in response to Message 43012.  

I am sorry for the Umbrella runs, they were somewhat experimental. However, the results look promising so far.

I can try to tune some parameters for future releases like the file size (we may be able to reduce it without big impact on the analysis results) but unfortunately they will still be quite CPU consuming because of the aforementioned reasons. Could you post the main reasons why they were a "pain"? :) We can try to find a common solution.

I am now sending plenty of classic long WU that should push your GPUs to the limit! :) However, I plan to launch more Umbrella short WU in the future, which to me seem ideal for old GPUs.

I'll keep you posted!


I agree pretty much with what Retvari Zoltan and skgiven mentioned in their posts.

Though, I don't think having 3 short tasks per GPU would have made much of a difference, (having 2 or 3 finished WUs per GPU not being able to upload, and not getting any new WUs, I still would have spent most of that Sunday crunching my back up project).

What would have made a difference is not releasing "somewhat experimental" WUs on the weekends, when staff level is limited. Release them, during the week, when most everybody is at work, to deal with potential problems.




ID: 43020 · Rating: 0 · rate: Rate + / Rate - Report as offensive     Reply Quote
Erich56

Send message
Joined: 1 Jan 15
Posts: 1166
Credit: 12,260,898,501
RAC: 960
Level
Trp
Scientific publications
watwatwatwatwatwatwatwatwat
Message 43021 - Posted: 16 Mar 2016, 9:26:44 UTC - in response to Message 43015.  

[quote]need of 3 short task per GPU in the Queue (presently the limit is 2 per GPU)


once these WUs are distributed again, it really would make a lot of sense to raise the limit from 2 to 3 per GPU !

ID: 43021 · Rating: 0 · rate: Rate + / Rate - Report as offensive     Reply Quote
Betting Slip

Send message
Joined: 5 Jan 09
Posts: 670
Credit: 2,498,095,550
RAC: 0
Level
Phe
Scientific publications
watwatwatwatwatwatwatwatwatwatwatwatwatwatwatwatwatwatwatwatwatwatwatwatwatwat
Message 43022 - Posted: 16 Mar 2016, 9:46:58 UTC - in response to Message 43021.  
Last modified: 16 Mar 2016, 10:09:55 UTC

[quote]need of 3 short task per GPU in the Queue (presently the limit is 2 per GPU)


once these WUs are distributed again, it really would make a lot of sense to raise the limit from 2 to 3 per GPU !



I disagree it would mean even more units held up for 5 days by the hosts that never return a completed WU and those that error after a long period of time.

We need different queues for the really fast cards such as 980ti, 980, Titan, 970, 780ti and a 2 day deadline. Mid cards could have a 3 day deadline. Slow cards could remain on 5 days, with an adjusted percentage of WU's allocated to each queue.

We could also accelerate the drop of WU's available to hosts that don't return or consistently error.

However as Gerard has said in another post he is already overun with work and probably does not have the time to do any of these things.
ID: 43022 · Rating: 0 · rate: Rate + / Rate - Report as offensive     Reply Quote
Jim1348

Send message
Joined: 28 Jul 12
Posts: 819
Credit: 1,591,285,971
RAC: 0
Level
His
Scientific publications
watwatwatwatwatwatwatwatwatwatwatwatwatwatwat
Message 43024 - Posted: 16 Mar 2016, 13:38:58 UTC
Last modified: 16 Mar 2016, 13:39:26 UTC

To offer a minority view, I was quite happy to get them. I don't normally run the shorts, but the science looks very interesting, and my GTX 750 Tis are not that good on the longs anymore. If they don't use much GPU power, but more CPU power, that is OK if that is what the calculations call for. You can't change the science or math just to heat up the cards more. Also, I have large enough upload bandwidth (4 Mbps) that I did not notice any problems there, or with memory, etc. But a warning as to all of this would undoubtedly be a good idea, since it may push many machines over the edge in one way or another..
ID: 43024 · Rating: 0 · rate: Rate + / Rate - Report as offensive     Reply Quote
Erich56

Send message
Joined: 1 Jan 15
Posts: 1166
Credit: 12,260,898,501
RAC: 960
Level
Trp
Scientific publications
watwatwatwatwatwatwatwatwat
Message 43025 - Posted: 16 Mar 2016, 14:32:59 UTC - in response to Message 43022.  

However as Gerard has said in another post he is already overun with work and probably does not have the time to do any of these things.

that's why we all should hope that the new students which where expected for January will finally come on bord.

The amount of work which Gerard is doing, all by himself, is terrific! I guess, at some point he deserves rest and recreation :-)
ID: 43025 · Rating: 0 · rate: Rate + / Rate - Report as offensive     Reply Quote
Erich56

Send message
Joined: 1 Jan 15
Posts: 1166
Credit: 12,260,898,501
RAC: 960
Level
Trp
Scientific publications
watwatwatwatwatwatwatwatwat
Message 43072 - Posted: 24 Mar 2016, 8:41:06 UTC - in response to Message 43015.  

High CPU usage -> low GPU usage -> need of 2 simultaneous short task per GPU -> need of 3 short task per GPU in the queue
(presently the limit is 2 per GPU)


the current short runs "Enamine_Umbrella" use some 50-60% of a high-end GPU.
As already said above by one of our power crunchers: the Limit of 2 such WUs per GPU should be increased to 3, if not to 4.
ID: 43072 · Rating: 0 · rate: Rate + / Rate - Report as offensive     Reply Quote
Erich56

Send message
Joined: 1 Jan 15
Posts: 1166
Credit: 12,260,898,501
RAC: 960
Level
Trp
Scientific publications
watwatwatwatwatwatwatwatwat
Message 43143 - Posted: 4 Apr 2016, 4:50:55 UTC - in response to Message 43072.  

the current short runs "Enamine_Umbrella" use some 50-60% of a high-end GPU.
As already said above by one of our power crunchers: the limit of 2 such WUs per GPU should be increased to 3, if not to 4.

any news on this?
ID: 43143 · Rating: 0 · rate: Rate + / Rate - Report as offensive     Reply Quote
Profile Retvari Zoltan
Avatar

Send message
Joined: 20 Jan 09
Posts: 2380
Credit: 16,897,957,044
RAC: 1
Level
Trp
Scientific publications
watwatwatwatwatwatwatwatwatwatwatwatwatwatwatwatwatwatwatwatwatwatwatwatwatwat
Message 43148 - Posted: 4 Apr 2016, 23:07:04 UTC - in response to Message 43143.  

the current short runs "Enamine_Umbrella" use some 50-60% of a high-end GPU.
As already said above by one of our power crunchers: the limit of 2 such WUs per GPU should be increased to 3, if not to 4.

any news on this?

The recent workunits are not *that* problematic, so this is not that important right now.
ID: 43148 · Rating: 0 · rate: Rate + / Rate - Report as offensive     Reply Quote
Profile skgiven
Volunteer moderator
Volunteer tester
Avatar

Send message
Joined: 23 Apr 09
Posts: 3968
Credit: 1,995,359,260
RAC: 0
Level
His
Scientific publications
watwatwatwatwatwatwatwatwatwatwatwatwatwatwatwatwatwatwatwatwatwatwatwatwatwatwatwat
Message 43156 - Posted: 6 Apr 2016, 13:51:47 UTC - in response to Message 43148.  

There are no short runs at present, when there are short runs there's not always many WU's and the batches don't last as long - Server Status
FAQ's

HOW TO:
- Opt out of Beta Tests
- Ask for Help
ID: 43156 · Rating: 0 · rate: Rate + / Rate - Report as offensive     Reply Quote
Previous · 1 · 2 · 3

Message boards : News : *CXCL12_chalcone_umbrella* batch

©2025 Universitat Pompeu Fabra