NATHAN_FAX3 and FAX4 discussion

Message boards : Number crunching : NATHAN_FAX3 and FAX4 discussion
Message board moderation

To post messages, you must log in.

Previous · 1 · 2 · 3 · Next

AuthorMessage
Profile ritterm
Avatar

Send message
Joined: 31 Jul 09
Posts: 88
Credit: 244,413,897
RAC: 0
Level
Leu
Scientific publications
watwatwatwatwatwatwatwatwatwatwatwatwatwatwatwatwatwat
Message 23903 - Posted: 11 Mar 2012, 20:22:47 UTC - in response to Message 23902.  

I think one should get the bonus anyway whether you finish these behemoth tasks within 24 hours or not...

+1

ID: 23903 · Rating: 0 · rate: Rate + / Rate - Report as offensive     Reply Quote
JLConawayII

Send message
Joined: 31 May 10
Posts: 48
Credit: 28,893,779
RAC: 0
Level
Val
Scientific publications
watwatwatwatwatwatwatwatwatwatwatwatwat
Message 23906 - Posted: 11 Mar 2012, 21:47:34 UTC - in response to Message 23902.  
Last modified: 11 Mar 2012, 22:02:58 UTC


I think one should get the bonus anyway whether you finish these behemoth tasks within 24 hours or not.



Well, the posted estimate for long runs is 8-12 hours for the fastest cards. This was a gross overestimate until now, as my 260 would complete most of them in 8 hours, and some of the biggest ones would take 12-15 hours. Now it seems we've gone in the other direction, taking well over half a day to complete on 5xx series cards. If the size of the tasks is putting such a demand even on the most powerful cards, it seems reasonable that the bonus cutoffs would be extended a bit, maybe to 36/72 hours for 50%/25%, or something along those lines. Obviously you don't want to go crazy with bonus points, but it's something to consider if we're going to be modeling larger molecular systems that take considerably longer to complete.
ID: 23906 · Rating: 0 · rate: Rate + / Rate - Report as offensive     Reply Quote
rreit

Send message
Joined: 26 Nov 10
Posts: 1
Credit: 253,890,485
RAC: 0
Level
Asn
Scientific publications
watwatwatwatwatwatwatwatwatwatwatwatwatwatwatwatwatwat
Message 23908 - Posted: 12 Mar 2012, 1:53:18 UTC
Last modified: 12 Mar 2012, 1:55:02 UTC

I've had 2 NATHAN_FAX3 tasks complete. Both tasks were run with a dedicated core. One task was run without SWAN_SYNC being set, and the other task was run with SWAN_SYNC=0 (restarted BOINC after the change). The results were the same though.

NVIDIA GeForce GTX 570 (stock clock)
Windows 7 x64
Nvidia driver: 295.73
Core i7 920 @ 3.8Ghz

Dedicated core, no SWAN_SYNC
21 hrs. 8 min.

Dedicated core, SWAN_SYNC=0
21 hrs. 7 min.

So SWAN_SYNC didn't help at all for my setup.

I don't see why the GTX 570 takes 21 hours+ if the GTX 580 takes 12-13 hours. That seems like too large of a difference between the GTX 570 and GTX 580.
ID: 23908 · Rating: 0 · rate: Rate + / Rate - Report as offensive     Reply Quote
alephnull

Send message
Joined: 8 Jul 09
Posts: 13
Credit: 306,850,267
RAC: 0
Level
Asp
Scientific publications
watwatwatwatwatwatwatwatwatwatwatwatwat
Message 23909 - Posted: 12 Mar 2012, 3:45:18 UTC

there does seem to be some credit disparity for the FAX3 wu in terms of time vs credit rewarded but im not sure. fair enough, so long as the work keeps coming!

anyway, i definitely turned off long work units for my machines running some gtx275 cards. it was hurtin those poor, poor cards. the gtx500 series cards seem to be ok with them albeit they do take a while.

my question is this: i like running the older long wu (i.e. non FAX3 wu) on my gtx275s. they didnt have issues completing those in time. since there's no separation of long wu type on the preferences screen, would it be possible to add something like that so we can select to still run longs without FAX3? for example:

ACEMD standard
ACEMD beta
ACEMD for long runs (8-12 hours on fastest GPU)
ACEMD for long runs (8-12 hours on fastest GPU) & FAX3

or something to that effect? that way the slower cards can still get longs but not struggle with the FAX3s. just thought i'd ask. thanks for the consideration.

bob
ID: 23909 · Rating: 0 · rate: Rate + / Rate - Report as offensive     Reply Quote
MarkJ
Volunteer moderator
Volunteer tester

Send message
Joined: 24 Dec 08
Posts: 738
Credit: 200,909,904
RAC: 0
Level
Leu
Scientific publications
watwatwatwatwatwatwatwatwatwatwatwatwatwatwatwatwatwatwatwatwat
Message 23910 - Posted: 12 Mar 2012, 7:29:58 UTC

Perhaps we need a short, medium and long queues. All the current longs except FAX3 go to the medium queue. We'll need an option to select which type of work to allow as well. Deselect the long queue by default and put a note next to the long queue option to suggest GTX570 or better.

It may also be possible to limit the long queue to certain speed cards by checking the est flops and memory values returned in the scheduler request (assuming it does pass it across).
BOINC blog
ID: 23910 · Rating: 0 · rate: Rate + / Rate - Report as offensive     Reply Quote
Profile Damaraland

Send message
Joined: 7 Nov 09
Posts: 152
Credit: 16,181,924
RAC: 0
Level
Pro
Scientific publications
watwatwatwatwatwatwatwatwat
Message 23912 - Posted: 12 Mar 2012, 9:26:10 UTC - in response to Message 23910.  

Perhaps we need a short, medium and long queues. All the current longs except FAX3 go to the medium queue. We'll need an option to select which type of work to allow as well. Deselect the long queue by default and put a note next to the long queue option to suggest GTX570 or better.

I proposed something similar. Response: out of the question because maintenaince costs (it seems with the team they have they can only handle 3 queues.
It may also be possible to limit the long queue to certain speed cards by checking the est flops and memory values returned in the scheduler request (assuming it does pass it across).

I agree with this. I proposed it too. I think the program that distributes the task should me smarter. It wouldn't need to look in the Flops, I think the best would be that the server just look the computer configuration and determine if it's slow or fast.

HOW TO - Full installation Ubuntu 11.10
ID: 23912 · Rating: 0 · rate: Rate + / Rate - Report as offensive     Reply Quote
ich_eben

Send message
Joined: 31 Aug 10
Posts: 1
Credit: 9,517,287
RAC: 0
Level
Ser
Scientific publications
watwatwatwatwatwatwatwat
Message 23916 - Posted: 12 Mar 2012, 11:27:20 UTC - in response to Message 23893.  

Also, GPU usage is below 90% again, while the old Nathans finally got up to 97% on a Fermi (GTX 570).

You guys just don't get enough? WUs running for that long on a Fermi, ever thought how long they would run on a CPU? And finally paying these credits or zero if they fail?

I think one reason why this project has lost any measure in fair credits and useful WU sizes is the lack of a CPU app. Why isn't there any? Because it would demonstrate the monstrous demands you are posing on hardware and crunchers...


look over to primegrid and the Genefer World Record tasks.
I finished two tasks on my gtx 580 and both of them took easily over 75 hours.
Thats long and part of why i switched back for a bit gpugrid. (And to get the next batch here ;-))
ID: 23916 · Rating: 0 · rate: Rate + / Rate - Report as offensive     Reply Quote
JLConawayII

Send message
Joined: 31 May 10
Posts: 48
Credit: 28,893,779
RAC: 0
Level
Val
Scientific publications
watwatwatwatwatwatwatwatwatwatwatwatwat
Message 23919 - Posted: 12 Mar 2012, 18:48:22 UTC

Target acquired. FAX3!!

At the current rate of progress, it SHOULD take 37 hours on my GTX 260. We'll see how this estimate evolves as the WU progresses.
ID: 23919 · Rating: 0 · rate: Rate + / Rate - Report as offensive     Reply Quote
MD

Send message
Joined: 1 Apr 09
Posts: 4
Credit: 224,382,750
RAC: 0
Level
Leu
Scientific publications
watwatwatwatwatwatwatwatwatwatwatwatwatwatwatwatwatwatwatwat
Message 23920 - Posted: 12 Mar 2012, 21:22:53 UTC

41hrs in and still have 21 to go?!?!?!?! gts450, core 980, shader 1960, ram 2050 hooked upto a 9950be with slight overclock. had just switched over to long wu's, regular long wu's ran 1~3hrs longer than short ones till i got this fax3. miffed at the idea that im loseing the 24hr bonus plus losing credits from the wu's i could have ran waiting for this to finish.
ID: 23920 · Rating: 0 · rate: Rate + / Rate - Report as offensive     Reply Quote
[AF>Belgique] bill1170

Send message
Joined: 4 Jan 09
Posts: 13
Credit: 1,382,704,222
RAC: 0
Level
Met
Scientific publications
watwatwatwatwatwatwatwatwatwatwatwatwatwatwatwatwatwatwatwatwatwatwatwatwatwatwat
Message 23924 - Posted: 13 Mar 2012, 8:38:04 UTC - in response to Message 23920.  

34 hours with my GTX275 @633 1134 1521 (shaders slightly over-clocked).

A typical NATHAN was crunched in ~ 8 hours. Other tasks not more than 12 hours.

The GTX 275 was qualified for long queue work units, but is not any more. I'm downgrading to short queue. Unfortunately cobblestones will probably drop from 100K/day to 30K/day.

The drop in cobblestone is a little bit disappointing, but in the other hand the link newly established between our work and scientist's publications is exciting enough to compensate ;-)

Would be nice to build a table of GPU that are qualified for the long queue in it's new configuration.


ID: 23924 · Rating: 0 · rate: Rate + / Rate - Report as offensive     Reply Quote
Profile Retvari Zoltan
Avatar

Send message
Joined: 20 Jan 09
Posts: 2380
Credit: 16,897,957,044
RAC: 0
Level
Trp
Scientific publications
watwatwatwatwatwatwatwatwatwatwatwatwatwatwatwatwatwatwatwatwatwatwatwatwatwat
Message 23926 - Posted: 13 Mar 2012, 9:03:46 UTC - in response to Message 23924.  

Would be nice to build a table of GPU that are qualified for the long queue in it's new configuration.

We don't know their exact model numbers yet :)
ID: 23926 · Rating: 0 · rate: Rate + / Rate - Report as offensive     Reply Quote
Profile Damaraland

Send message
Joined: 7 Nov 09
Posts: 152
Credit: 16,181,924
RAC: 0
Level
Pro
Scientific publications
watwatwatwatwatwatwatwatwat
Message 23927 - Posted: 13 Mar 2012, 10:35:34 UTC - in response to Message 23926.  
Last modified: 13 Mar 2012, 10:41:10 UTC

SWAN_SYNC doesn't make any effect. In both cases I freed 2 cores out of 8. Linux 3.0.0-16-generic. i7-2600K CPU @ 3.40GHz

SWAN_SYNC=0
31,70 h GTX 560 Ti @1.76 GHz
36,35 h GTX 260 @1.41 GHz
Without SWAN_SYNC
31,64 h GTX 560 Ti @1.76 GHz
HOW TO - Full installation Ubuntu 11.10
ID: 23927 · Rating: 0 · rate: Rate + / Rate - Report as offensive     Reply Quote
Profile skgiven
Volunteer moderator
Volunteer tester
Avatar

Send message
Joined: 23 Apr 09
Posts: 3968
Credit: 1,995,359,260
RAC: 0
Level
His
Scientific publications
watwatwatwatwatwatwatwatwatwatwatwatwatwatwatwatwatwatwatwatwatwatwatwatwatwatwatwat
Message 23928 - Posted: 13 Mar 2012, 10:47:09 UTC - in response to Message 23927.  
Last modified: 14 Mar 2012, 10:56:59 UTC

For the new apps I have not tested the benefit of using SWAN_SYNC. I will start now for Windows, but it could do with being tested for Linux as well (especially). Performance differences may fluctuate by task type, so several task types would need to be looked at. Remember to restart for the changes to be applied, and to use capitals for Linux.

Both your comparison tasks ran without SWAN_SYNC=0.

If SWAN_SYNC was in use then the stderr output would include, "SWAN: Using synchronization method 0"
For example.

BTW, Why do you have your GTX260 in PCIE 0 and your GTX560Ti in PCIE 1?

- On WinX64 it looks like SWAN_SYNC is only increasing performance by around 3.5% (though I've only run one CB1 task without SWAN_SYNC on).
FAQ's

HOW TO:
- Opt out of Beta Tests
- Ask for Help
ID: 23928 · Rating: 0 · rate: Rate + / Rate - Report as offensive     Reply Quote
Profile dskagcommunity
Avatar

Send message
Joined: 28 Apr 11
Posts: 463
Credit: 979,266,958
RAC: 84,915
Level
Glu
Scientific publications
watwatwatwatwatwatwatwatwatwatwatwatwatwatwatwatwat
Message 23929 - Posted: 13 Mar 2012, 12:17:48 UTC
Last modified: 13 Mar 2012, 12:24:44 UTC

Pff O.o Over 5! times more computingtime but only ~2,8time more credits O.o Kick it over 100k and I´m back happy ^^ Little disapointing, i had fun with the "new" 285 for only 2 days :(

Unfortunaly the Mobile Connection itself needed additional 12 Hours for upload O.o Still in 125% Bonustime with it *lucky*


5094249 3254378 117426 11 Mar 2012 | 15:03:07 UTC 13 Mar 2012 | 8:21:35 UTC Fertig und Bestätigt 136,916.69 29,691.78 95,125.00 Long runs (8-12 hours on fastest card) v6.16 (cuda31)

5092622 3255475 117426 11 Mar 2012 | 8:13:44 UTC 11 Mar 2012 | 19:21:52 UTC Fertig und Bestätigt 24,709.95 1,202.67 35,811.00 Long runs (8-12 hours on fastest card) v6.16 (cuda31)
DSKAG Austria: http://www.dskag.at

ID: 23929 · Rating: 0 · rate: Rate + / Rate - Report as offensive     Reply Quote
alephnull

Send message
Joined: 8 Jul 09
Posts: 13
Credit: 306,850,267
RAC: 0
Level
Asp
Scientific publications
watwatwatwatwatwatwatwatwatwatwatwatwat
Message 23950 - Posted: 14 Mar 2012, 4:28:22 UTC

got a FAX4 wu just now. is there any significant difference between this and a FAX3?

im interested to see how long this will take to complete. im still only getting about 75-80% gpu use (for longs) out of my cards with swan_sync set and 1 free hyper threaded core per gpu on all machines with gtx500 series cards. when those machines run the shorts, they seem have higher gpu use but i havent observed carefully enough to say that with 100% conviction. just informational really, its been good enough up to this point.

the FAX series gpu use is about the same as described above for longs. will be interested to see how long this FAX4 wu takes if there is a difference to the FAX3s. anything in particular to look out for?

bob

ID: 23950 · Rating: 0 · rate: Rate + / Rate - Report as offensive     Reply Quote
JLConawayII

Send message
Joined: 31 May 10
Posts: 48
Credit: 28,893,779
RAC: 0
Level
Val
Scientific publications
watwatwatwatwatwatwatwatwatwatwatwatwat
Message 23967 - Posted: 15 Mar 2012, 0:24:59 UTC

My GTX260 completed the FAX3 WU in 36.5 hours. The FAX4 it's working on now looks like it will finish in around 22.5 hours. I think upload is still going to put it over the 24h limit, but anyone with a newer card should probably be okay now.
ID: 23967 · Rating: 0 · rate: Rate + / Rate - Report as offensive     Reply Quote
Profile Damaraland

Send message
Joined: 7 Nov 09
Posts: 152
Credit: 16,181,924
RAC: 0
Level
Pro
Scientific publications
watwatwatwatwatwatwatwatwat
Message 23968 - Posted: 15 Mar 2012, 7:40:16 UTC - in response to Message 23967.  

I think upload is still going to put it over the 24h limit, but anyone with a newer card should probably be okay now.

Still figuring out what I did wrong with SWAN_SYNC (it's off now)

Because upload didn't make 24h on GTX 260, but I did on GTX 560
GTX 260 22,7 h FAX4
GTX 560 Ti 19,82 h FAX4




HOW TO - Full installation Ubuntu 11.10
ID: 23968 · Rating: 0 · rate: Rate + / Rate - Report as offensive     Reply Quote
francescocmazza

Send message
Joined: 8 Oct 11
Posts: 2
Credit: 9,961,113
RAC: 0
Level
Ser
Scientific publications
watwatwatwatwatwatwatwatwatwat
Message 23970 - Posted: 15 Mar 2012, 11:59:16 UTC

Hi all,
I have a gtx 560 ti as well.
Damn, these nathan 3 are really huge. Have been working on one for 22 Hours and still have 13.30 to go. My card is neither overclocked nor underclocked. If you can, please try to reduce the size of the WU by 1/2, I am afraid that the reduction with nathan 4 are still going to take us still very long. Also the effective damage of one computational error becomes much greater with such huge wu. Unforuntunately there is no way for me to improve my stats, as my card has a bad manufacturer heatsink IMHO and runs at 80c. Is it normal btw?

Thank you and take care
Francesco
ID: 23970 · Rating: 0 · rate: Rate + / Rate - Report as offensive     Reply Quote
Profile skgiven
Volunteer moderator
Volunteer tester
Avatar

Send message
Joined: 23 Apr 09
Posts: 3968
Credit: 1,995,359,260
RAC: 0
Level
His
Scientific publications
watwatwatwatwatwatwatwatwatwatwatwatwatwatwatwatwatwatwatwatwatwatwatwatwatwatwatwat
Message 23971 - Posted: 15 Mar 2012, 12:24:38 UTC - in response to Message 23970.  

The researchers stated that they will actively review task sizes. If for example they see higher failure rates they will most likely make changes to reduce runtime. In the mean time if any crunchers don't like the duration or experiences failures, crunch some of the normal tasks. While credit will be lower you will get badges for contributing to different research papers ;)
FAQ's

HOW TO:
- Opt out of Beta Tests
- Ask for Help
ID: 23971 · Rating: 0 · rate: Rate + / Rate - Report as offensive     Reply Quote
DavidVR

Send message
Joined: 20 Jan 11
Posts: 6
Credit: 10,705,495
RAC: 0
Level
Pro
Scientific publications
watwatwatwatwatwatwatwatwatwat
Message 23972 - Posted: 15 Mar 2012, 17:26:02 UTC

On my GTX 260 it's looking like it's going to take about 50 hours to complete a NATHAN_FAX WU. Typically long runs would take 20 - 25 hours.
ID: 23972 · Rating: 0 · rate: Rate + / Rate - Report as offensive     Reply Quote
Previous · 1 · 2 · 3 · Next

Message boards : Number crunching : NATHAN_FAX3 and FAX4 discussion

©2026 Universitat Pompeu Fabra