NOELIAs are back!

Message boards : Number crunching : NOELIAs are back!
Message board moderation

To post messages, you must log in.

1 · 2 · 3 · 4 . . . 5 · Next

AuthorMessage
tomba

Send message
Joined: 21 Feb 09
Posts: 497
Credit: 700,690,702
RAC: 0
Level
Lys
Scientific publications
watwatwatwatwatwatwatwatwatwatwatwatwatwatwatwatwatwatwatwatwat
Message 29705 - Posted: 6 May 2013, 15:52:37 UTC

Just got one. Glad I have a 1GB GPU 'cause it's using 874MB of its memory!
ID: 29705 · Rating: 0 · rate: Rate + / Rate - Report as offensive     Reply Quote
Profile Stoneageman
Avatar

Send message
Joined: 25 May 09
Posts: 224
Credit: 34,057,374,498
RAC: 0
Level
Trp
Scientific publications
watwatwatwatwatwatwatwatwatwatwatwatwatwatwatwatwatwatwatwatwatwatwatwatwatwatwatwat
Message 29706 - Posted: 6 May 2013, 18:32:48 UTC

Have been getting many. None completed as yet, but so far no problems. Are these the same type as before?
ID: 29706 · Rating: 0 · rate: Rate + / Rate - Report as offensive     Reply Quote
GPUGRID

Send message
Joined: 12 Dec 11
Posts: 91
Credit: 2,730,095,033
RAC: 0
Level
Phe
Scientific publications
watwatwatwatwatwatwatwatwatwatwatwatwat
Message 29708 - Posted: 6 May 2013, 20:41:43 UTC

These new units are using really low CPU power. The gpus are way cooler aswell, and the processing times seems to be very high. Not using all the hardware.
ID: 29708 · Rating: 0 · rate: Rate + / Rate - Report as offensive     Reply Quote
Robert Gammon

Send message
Joined: 28 May 12
Posts: 63
Credit: 714,535,121
RAC: 0
Level
Lys
Scientific publications
watwatwatwatwatwatwatwatwatwatwatwatwatwatwatwatwatwat
Message 29709 - Posted: 6 May 2013, 22:00:51 UTC - in response to Message 29708.  

After over 4 hours execution with 12 hours estimated runtime to completion, I aborted one. The second one is showing the same issues. Estimated runtime to completion is rising with almost every second of execution.

GPU is hardly working, temps are lower than the CPU, that is unusual.

The process is using 30-35% of GPU memory (12% display, balance for computing)
ID: 29709 · Rating: 0 · rate: Rate + / Rate - Report as offensive     Reply Quote
Profile skgiven
Volunteer moderator
Volunteer tester
Avatar

Send message
Joined: 23 Apr 09
Posts: 3968
Credit: 1,995,359,260
RAC: 0
Level
His
Scientific publications
watwatwatwatwatwatwatwatwatwatwatwatwatwatwatwatwatwatwatwatwatwatwatwatwatwatwatwat
Message 29711 - Posted: 6 May 2013, 23:46:31 UTC - in response to Message 29709.  
Last modified: 6 May 2013, 23:50:10 UTC

I suspended my Nathan WU’s to have a look at these NOELIA_klebe WU:
    GPU power 91%
    temperature 67C
    GPU usage 94% (with CPU tasks suspended)
    Fan speed 77%
    Core clock 1202MHz

CPU usage is fairly low; looks like being ~44% of the GPUs’ runtime.
3% complete in just over 18min14sec. So estimated run time is 36,466sec (just over 10h on a GTX660Ti in a W7x64 system). That’s around twice as long as Nate’s recent WU’s.
When I allowed CPU tasks to run again (only 67% of the CPU in use overall) the GPU usage dropped to ~88%, and the power dipped to around 89%. With no CPU usage the CPU usage was a straight line. With 67% of the CPU in use the GPU usage is jagged. Even when I dropped the CPU to 50% in Boinc Manager the GPU usage graph remained jagged but ~92% GPU usage on average.
With CPU set to 25% the GPU usage line was reasonably straight with some downward spikes.

When I set the CPU usage to 100% the graph started going all over the place, initially dropping to 76%, spiked up to 91%, fluctuated more than before but mostly between 80% and 90% GPU usage. While I only saw a drop from 1202MHz to 1189MHz, I expect the GPU usage on other systems would fluctuate more depending on the CPU/GPU setups. So I think running at 100% CPU usage could well result in the GPU downclocking when running these WU’s.
I will let it run with only 1 CPU core being used, to get an accurate idea of an optimal settings performance on the 660Ti.


FAQ's

HOW TO:
- Opt out of Beta Tests
- Ask for Help
ID: 29711 · Rating: 0 · rate: Rate + / Rate - Report as offensive     Reply Quote
klepel

Send message
Joined: 23 Dec 09
Posts: 189
Credit: 4,798,881,008
RAC: 0
Level
Arg
Scientific publications
watwatwatwatwatwatwatwatwatwatwatwatwatwatwatwatwatwatwatwatwatwatwatwatwat
Message 29712 - Posted: 7 May 2013, 1:57:36 UTC

Just finished my first NOELIA_klebe workunit on my GTX 670 (W7)! Nothing unusual: runtime 34480.037 s, credit 127,800.00, which is by the way similar to NOELIA_PEPTGPRC: 29748.680 s, 113,250.00, last/oldest NOELIA WU I was able to find on my register (after that I only had NATHANs). About same credit/time ratio within NOELIAs and hey it is not all about credits, so no reason to abort just because NATHANs do better on credits, within the GPUGRID it will level out. So I am quite happy with these new NOELIAs, if the WUs don't start to crash down the road.
ID: 29712 · Rating: 0 · rate: Rate + / Rate - Report as offensive     Reply Quote
Robert Gammon

Send message
Joined: 28 May 12
Posts: 63
Credit: 714,535,121
RAC: 0
Level
Lys
Scientific publications
watwatwatwatwatwatwatwatwatwatwatwatwatwatwatwatwatwat
Message 29714 - Posted: 7 May 2013, 2:53:37 UTC - in response to Message 29712.  

I have a short run NOELIA klebe in process. on a 4 core AMD with 2 boincsimap tasks and wuprop running CPU use is low in comparison to NATHAN. CPU use is varying between 15% and 100% depending on which core, but the CPU is never saturated.

After 4 hours, it is 10.7% complete and BOINC 7.0.28 on Ubuntu 13.04 forecasts completion after another 9 hours 54 minutes. This forecast is rising with every few seconds of execution. Nvidia driver version is 319.17. All updates to Ubuntu are applied daily. This is on a GTX660Ti that processes NATHANs in 5 hours 15 minutes.

This wu will NOT finish before I have to go to work in 5 hours so I will report results when I get back home after work.

I did not want to abort the workunits, the Long Runs were working so slowly, I did not expect them to complete in less than 24 hours. This Short Run may take nearly that long to execute.

I personally hope that this batch of NOELIA klebe are exhausted soon.
ID: 29714 · Rating: 0 · rate: Rate + / Rate - Report as offensive     Reply Quote
Robert Gammon

Send message
Joined: 28 May 12
Posts: 63
Credit: 714,535,121
RAC: 0
Level
Lys
Scientific publications
watwatwatwatwatwatwatwatwatwatwatwatwatwatwatwatwatwat
Message 29715 - Posted: 7 May 2013, 7:33:18 UTC - in response to Message 29714.  

After 9 hours running. only 18% complete. Remaining time is still climbing.

I will come back to GPUGrid in a few days once these wu are gone. ACEMD for Linux cannot run these tasks correctly.
ID: 29715 · Rating: 0 · rate: Rate + / Rate - Report as offensive     Reply Quote
ExtraTerrestrial Apes
Volunteer moderator
Volunteer tester
Avatar

Send message
Joined: 17 Aug 08
Posts: 2705
Credit: 1,311,122,549
RAC: 0
Level
Met
Scientific publications
watwatwatwatwatwatwatwatwatwatwatwatwatwatwatwatwatwatwatwatwatwatwatwatwat
Message 29717 - Posted: 7 May 2013, 7:52:50 UTC

It might be better or even neccessary to for straight usage of one core, as the Nathans do. Freeing half a core for such a performance hit (on fully loaded systems) doesn't seem worth it.

MrS
Scanning for our furry friends since Jan 2002
ID: 29717 · Rating: 0 · rate: Rate + / Rate - Report as offensive     Reply Quote
Profile skgiven
Volunteer moderator
Volunteer tester
Avatar

Send message
Joined: 23 Apr 09
Posts: 3968
Credit: 1,995,359,260
RAC: 0
Level
His
Scientific publications
watwatwatwatwatwatwatwatwatwatwatwatwatwatwatwatwatwatwatwatwatwatwatwatwatwatwatwat
Message 29719 - Posted: 7 May 2013, 9:24:47 UTC - in response to Message 29717.  
Last modified: 7 May 2013, 10:25:35 UTC

It might be better or even neccessary to for straight usage of one core, as the Nathans do. Freeing half a core for such a performance hit (on fully loaded systems) doesn't seem worth it.

MrS

Agreed.

148px47x4-NOELIA_klebe_run-0-3-RND5398_0 4431950 6 May 2013 | 20:26:40 UTC 7 May 2013 | 9:18:47 UTC Completed and validated 36,452.79 16,142.69 127,800.00
I32R6-NATHAN_dhfr36_5-21-32-RND2187_1 4426158 5 May 2013 | 23:09:30 UTC 6 May 2013 | 8:22:25 UTC Completed and validated 18,944.86 18,944.86 70,800.00

I went into app_config and set the cpu_usage for GPUGrid to 1.0 to see what impact this had on running a NOELIA_klebe WU, with the CPU usage set in Boinc Manager to 75% (the most I typically use with 2 GPU's in the system):
GPU power 88%
temperature 65C
GPU usage ~90% still with some variation
Fan speed 74%
Core clock at 1189MHz (this dropped yesterday from 1202MHz and hasn’t risen yet).

There is still a 4% loss, going by GPU usage, but probably more if I ran an entire WU with these settings.

I also suspended and restarted Noelia’s task and closed, used snooze GPU and opened Boinc without problems.

My WU returned in just over 10h, as expected, but might have dipped below 10h if I hadn't been suspended and restarting...

-
With SWAN_SYNC set to 0 (and still using app_config) it didn’t make much difference:
GPU power 88% (no change)
temperature 65C (no change)
GPU usage ~90% still with some variation but very occasionally rising to 94% and staying at that for a few seconds.
Fan speed 74%
Core clock at 1189MHz (no change).
Actual progress was about the same. 1% after just over 6min.

Got a driver error when I exited Boinc. That suggests to me that the WU isn’t closing down gracefully. Maybe Boinc is to blame there.

Did a restart and removed app_config file, but left SWAN_SYNC in place. No change. Opened the init_data.xml file in the slot that the WU was running and it's still set to <ncpus>1.000000</ncpus>. So there's that theory confirmed. Once <ncpus> is set then that's it fixed for the duration of the run.
When I suspended the WU I got another driver restart.

-
Before I previously restarted I had set <max_concurrent>2</max_concurrent> in app_config (thinking this would limit the GPUGrids WU cache to 2). Then I removed the app_config file from the project directory. I aborted 2 WU's and restarted. I now have 4 WU's in the queue, but only 1 NVidia GPU. Either the number of WU's has been fixed in relation to time (1day, so unlikely given that the WU's are 10h each and Boinc is saying that) due to app_config, and you have to use the app_config always (or do a project restart to properly flush it out of the system), or it's in some way related to the GPU count and ignores the fact that the other GPU is an ATI. Either way having to do a project reset just to make configuration changes is far from ideal.
Anyway, if anyone is planning to do a project reset because they were using app_config, I suggest you choose no more tasks, finish any running WU's and abort any queued WU's, then do the project reset. The project doesn't resend tasks after a project reset, so by aborting the upstarted WU's they won't be in limbo until they time out and can be resent early (better for the project). If you don't they will appear as In Progress online for two weeks.
FAQ's

HOW TO:
- Opt out of Beta Tests
- Ask for Help
ID: 29719 · Rating: 0 · rate: Rate + / Rate - Report as offensive     Reply Quote
John C MacAlister

Send message
Joined: 17 Feb 13
Posts: 181
Credit: 144,871,276
RAC: 0
Level
Cys
Scientific publications
watwatwatwatwatwatwatwatwatwatwatwatwat
Message 29720 - Posted: 7 May 2013, 10:23:37 UTC
Last modified: 7 May 2013, 10:26:07 UTC

Hi, everyone:

I believe my system may not be running at optimal settings. Can someone please advise this non-technical user how to achieve maximum system usage?

Many thanks!

I get the following results with my AMD A10 5800K and GTX 650 Ti GPUs processing NOELIA tasks. I am also running three WCG SN2S tasks. I do not have an app_config file.

Device 0

temperature 46C
CPU usage 0.61
GPU usage ~70 -85% fluctuating
Fan speed 54%
Core clock at 979.8 MHz

Device 1

temperature 54C
CPU usage 0.61
GPU usage ~66 -92% fluctuating
Fan speed 46%
Core clock at 979.8 MHz

Running NATHAN tasks I was averaging 33,400 sec with 70,800 credits. The NOELIA tasks completed in about 67,700 sec, returning 127,800 credits.

John
ID: 29720 · Rating: 0 · rate: Rate + / Rate - Report as offensive     Reply Quote
Profile skgiven
Volunteer moderator
Volunteer tester
Avatar

Send message
Joined: 23 Apr 09
Posts: 3968
Credit: 1,995,359,260
RAC: 0
Level
His
Scientific publications
watwatwatwatwatwatwatwatwatwatwatwatwatwatwatwatwatwatwatwatwatwatwatwatwatwatwatwat
Message 29721 - Posted: 7 May 2013, 10:29:19 UTC - in response to Message 29720.  
Last modified: 7 May 2013, 10:30:08 UTC

By running less than 3 CPU tasks on your triple core CPU you should see some improvements in GPU usage (at least for these NOELIA WU's). If you don't run any CPU projects you will see the most improvement, but what you run is personal choice. I would probably run one CPU WU at a time with your setup, or just not use the CPU; your NVidia is vastly more powerful than your AMD CPU.
FAQ's

HOW TO:
- Opt out of Beta Tests
- Ask for Help
ID: 29721 · Rating: 0 · rate: Rate + / Rate - Report as offensive     Reply Quote
Profile dskagcommunity
Avatar

Send message
Joined: 28 Apr 11
Posts: 463
Credit: 958,266,958
RAC: 34
Level
Glu
Scientific publications
watwatwatwatwatwatwatwatwatwatwatwatwatwatwatwatwat
Message 29723 - Posted: 7 May 2013, 12:27:17 UTC
Last modified: 7 May 2013, 12:30:12 UTC

Still computing on the first one, but seems to run normal 12h on my 560ti 448 cores @ 98% gpu load and 2-6% pentium 4 load :)
DSKAG Austria Research Team: http://www.research.dskag.at



ID: 29723 · Rating: 0 · rate: Rate + / Rate - Report as offensive     Reply Quote
Profile Beyond
Avatar

Send message
Joined: 23 Nov 08
Posts: 1112
Credit: 6,162,416,256
RAC: 0
Level
Tyr
Scientific publications
watwatwatwatwatwatwatwatwatwatwatwatwatwatwatwatwatwatwatwatwatwatwatwat
Message 29725 - Posted: 7 May 2013, 12:53:01 UTC - in response to Message 29705.  
Last modified: 7 May 2013, 12:59:42 UTC

Just got one. Glad I have a 1GB GPU 'cause it's using 874MB of its memory!

Maybe that's why they're locking up my 4 GTX 460 768MB cards. Can Nathan and/or Toni perhaps help with designing these WUs? Please? Please?? Please???

Though they walk though the valley of the shadow of death, my GTX 460s fear no evil EXCEPT these crappy NOELIA WUs. Guess the 4 of them are off to other projects :-(
ID: 29725 · Rating: 0 · rate: Rate + / Rate - Report as offensive     Reply Quote
Profile skgiven
Volunteer moderator
Volunteer tester
Avatar

Send message
Joined: 23 Apr 09
Posts: 3968
Credit: 1,995,359,260
RAC: 0
Level
His
Scientific publications
watwatwatwatwatwatwatwatwatwatwatwatwatwatwatwatwatwatwatwatwatwatwatwatwatwatwatwat
Message 29729 - Posted: 7 May 2013, 14:11:08 UTC - in response to Message 29725.  

If WU's are going to use more than 750MB GDDR, some crunchers need an easy way to select what tasks they run, or better still the server would allocate WU's based on the amount of GDDR the cards have (might be problematic for people with multiple mixed series cards). No point sending out work that will never complete.

If you're only going to send such tasks in the Long queue then it would be useful if there were alternative tasks in the short queue, and that this was announced.

The main GPU's that would suffer from this problem are, GTX460, GTS450, GT440, and some entry level laptop GPU's. It doesn't really effect the GeForce 500 or 600 cards, only the 400 series. However these cards are CC2.1, just like others in the 400 and 500 series.

Consider Short, Long and Extra-Long queues, though that might impact on other plans.
FAQ's

HOW TO:
- Opt out of Beta Tests
- Ask for Help
ID: 29729 · Rating: 0 · rate: Rate + / Rate - Report as offensive     Reply Quote
GPUGRID

Send message
Joined: 12 Dec 11
Posts: 91
Credit: 2,730,095,033
RAC: 0
Level
Phe
Scientific publications
watwatwatwatwatwatwatwatwatwatwatwatwat
Message 29730 - Posted: 7 May 2013, 15:18:04 UTC

They start to crash on all machines. Lost like 50hr of processing since yesterday. Uncool to say the least.
ID: 29730 · Rating: 0 · rate: Rate + / Rate - Report as offensive     Reply Quote
tomba

Send message
Joined: 21 Feb 09
Posts: 497
Credit: 700,690,702
RAC: 0
Level
Lys
Scientific publications
watwatwatwatwatwatwatwatwatwatwatwatwatwatwatwatwatwatwatwatwat
Message 29731 - Posted: 7 May 2013, 16:36:56 UTC - in response to Message 29730.  

They start to crash on all machines. Lost like 50hr of processing since yesterday. Uncool to say the least.

There's quite a lot of doom & gloom over this latest batch of NOELIAs.
My own experience, on an oc'd GTX 460 1MB, is positive; one completed in under 18 hours with 127,800 credit, and one now running, almost six hours elapsed and 12+ hours remaining.

ID: 29731 · Rating: 0 · rate: Rate + / Rate - Report as offensive     Reply Quote
Profile Beyond
Avatar

Send message
Joined: 23 Nov 08
Posts: 1112
Credit: 6,162,416,256
RAC: 0
Level
Tyr
Scientific publications
watwatwatwatwatwatwatwatwatwatwatwatwatwatwatwatwatwatwatwatwatwatwatwat
Message 29732 - Posted: 7 May 2013, 16:48:58 UTC - in response to Message 29731.  
Last modified: 7 May 2013, 16:54:57 UTC

They start to crash on all machines. Lost like 50hr of processing since yesterday. Uncool to say the least.

There's quite a lot of doom & gloom over this latest batch of NOELIAs.
My own experience, on an oc'd GTX 460 1MB, is positive; one completed in under 18 hours with 127,800 credit, and one now running, almost six hours elapsed and 12+ hours remaining.

An experience of 1 is not much experience. Firehawk is having crashing problems on his very fast GPUs, sure a few completed there too. They don't seem to run at all on GPUs with under 1GB ram. Another user posted above that they will not run on Linux on his 660 Ti. Other people are also having crashes. Why are we inflicted with these WUs without testing or warning? It's ridiculous IMO. It's a waste of our resources and our money.
ID: 29732 · Rating: 0 · rate: Rate + / Rate - Report as offensive     Reply Quote
neilp62

Send message
Joined: 23 Nov 10
Posts: 14
Credit: 8,017,535,732
RAC: 0
Level
Tyr
Scientific publications
watwatwatwatwatwatwatwatwatwatwatwatwatwatwatwatwatwatwatwatwat
Message 29733 - Posted: 7 May 2013, 17:51:06 UTC - in response to Message 29732.  

They start to crash on all machines. Lost like 50hr of processing since yesterday. Uncool to say the least.

There's quite a lot of doom & gloom over this latest batch of NOELIAs.
My own experience, on an oc'd GTX 460 1MB, is positive; one completed in under 18 hours with 127,800 credit, and one now running, almost six hours elapsed and 12+ hours remaining.

An experience of 1 is not much experience. Firehawk is having crashing problems on his very fast GPUs, sure a few completed there too. They don't seem to run at all on GPUs with under 1GB ram. Another user posted above that they will not run on Linux on his 660 Ti. Other people are also having crashes. Why are we inflicted with these WUs without testing or warning? It's ridiculous IMO. It's a waste of our resources and our money.

I too have had a less-than-ideal experience with these new Noelia workunits. After weeks of successfully completing Nathans, my stable GTX 680 cruncher failed 17 Noelias in a row before I detected the issue and rebooted the box. I am still awaiting expiration of the WU limit timer to confirm whether this cruncher will even be able to complete one of these WUs. I concur with Beyond that this is a very unfortunate waste of resources that might have been avoided with some advance notification.
ID: 29733 · Rating: 0 · rate: Rate + / Rate - Report as offensive     Reply Quote
John C MacAlister

Send message
Joined: 17 Feb 13
Posts: 181
Credit: 144,871,276
RAC: 0
Level
Cys
Scientific publications
watwatwatwatwatwatwatwatwatwatwatwatwat
Message 29734 - Posted: 7 May 2013, 18:56:14 UTC

I have had two NOELIA tasks successfully complete with two more half way through....fingers crossed.
ID: 29734 · Rating: 0 · rate: Rate + / Rate - Report as offensive     Reply Quote
1 · 2 · 3 · 4 . . . 5 · Next

Message boards : Number crunching : NOELIAs are back!

©2025 Universitat Pompeu Fabra