Any changes to project since March30 ?

Message boards : Graphics cards (GPUs) : Any changes to project since March30 ?
Message board moderation

To post messages, you must log in.

1 · 2 · Next

AuthorMessage
The_Bad_Penguin

Send message
Joined: 16 Dec 07
Posts: 23
Credit: 9,286,325
RAC: 0
Level
Ser
Scientific publications
watwatwatwatwatwatwatwatwat
Message 8096 - Posted: 3 Apr 2009, 0:33:17 UTC

I may only have a 9600gso, but up until ~ March 30, that was good enough to complete ~ 1 wu /day.

Now all of the sudden, wu's are missing deadlines, and just runnning past 24 hours.

Whats happened in the last 3 days ?!
ID: 8096 · Rating: 0 · rate: Rate + / Rate - Report as offensive     Reply Quote
Profile Paul D. Buck

Send message
Joined: 9 Jun 08
Posts: 1050
Credit: 37,321,185
RAC: 0
Level
Val
Scientific publications
watwatwatwatwatwatwatwatwatwat
Message 8103 - Posted: 3 Apr 2009, 5:32:00 UTC

NOt sure,, I updated the onee system to 6.6.20 and now have elapsed time showing and some of my tasks on the GTX 295 are showing 16 to 21 hours and they have 2-3 hours to go ...

Not sure what is going on here ... but they also have Ignasi's name on a couple of the tasks so maybe he is trying nefarious deeds on me again ...

I mean, it is strange, most of the tasks that I have from that computer have 6 hours as their run time (or there abouts) so this is odd ...
ID: 8103 · Rating: 0 · rate: Rate + / Rate - Report as offensive     Reply Quote
MarkJ
Volunteer moderator
Volunteer tester

Send message
Joined: 24 Dec 08
Posts: 738
Credit: 200,909,904
RAC: 0
Level
Leu
Scientific publications
watwatwatwatwatwatwatwatwatwatwatwatwatwatwatwatwatwatwatwatwat
Message 8108 - Posted: 3 Apr 2009, 8:46:08 UTC
Last modified: 3 Apr 2009, 8:52:47 UTC

I've got a couple on my GTX260 cards that have been running all day and they haven't finished yet. BOINC is saying its elapsed time is 18 hours 47 mins. They normally take 6 hours.

This is one of the offending ones.

Its still in "initial" state as it hasn't completed at the time I write this. There is nothing listed on the task that would make it appear any different.
BOINC blog
ID: 8108 · Rating: 0 · rate: Rate + / Rate - Report as offensive     Reply Quote
Profile GDF
Volunteer moderator
Project administrator
Project developer
Project tester
Volunteer developer
Volunteer tester
Project scientist

Send message
Joined: 14 Mar 07
Posts: 1958
Credit: 629,356
RAC: 0
Level
Gly
Scientific publications
watwatwatwatwat
Message 8109 - Posted: 3 Apr 2009, 8:59:32 UTC - in response to Message 8108.  

There is nothing new on the workunits.

gdf
ID: 8109 · Rating: 0 · rate: Rate + / Rate - Report as offensive     Reply Quote
Profile Paul D. Buck

Send message
Joined: 9 Jun 08
Posts: 1050
Credit: 37,321,185
RAC: 0
Level
Val
Scientific publications
watwatwatwatwatwatwatwatwatwat
Message 8112 - Posted: 3 Apr 2009, 9:10:33 UTC
Last modified: 3 Apr 2009, 9:14:04 UTC

This was my first ... now the internal number says 6 hours, the elapsed time said over 20 ... sigh ... now I gotta try to figure out if it is an accounting bug in BOINC 6.2.20 or a serious issue. I have two tasks now listed as taking 25 plus hours ... and a third at 18:47 with 5:48 to go ...

I did just update my drivers to the latest version 182.5 and it looks like maybe the control panel won't GPF any longer when started on my GTX 295 cards (I think it got confused with multiple GPU "Cores" to work with ...

The bad news is the tasks are now listed as taking 40 hours (the newest started) the good news is that they seem to be running faster... which may be an illusion, but, the time is falling several seconds per second ...

So, I don't know ... too many unknown unknowns for me to completely sort through ...

Since I just installed the 6.6.20 today I may have run into a "bug" with the different ways of counting elapsed time and the tasks were suspended for most of the day and as such then ran up this huge "bill" of run time. The second task is coming due to end ...

The second one also lists as 6 hours internally ...

Hmmm, GDF says nothing new with the tasks ... sigh ... I hate mysteries ...

BUt, this is another that is likely going to bite us in the butt when the new version hits the streets and people upgrade. Another thing ETA will have to watch out for ...

{edit}
something sure as heck happened ... it looks like my system went from 15-16 tasks a day to 8-10 ...
ID: 8112 · Rating: 0 · rate: Rate + / Rate - Report as offensive     Reply Quote
MarkJ
Volunteer moderator
Volunteer tester

Send message
Joined: 24 Dec 08
Posts: 738
Credit: 200,909,904
RAC: 0
Level
Leu
Scientific publications
watwatwatwatwatwatwatwatwatwatwatwatwatwatwatwatwatwatwatwatwat
Message 8115 - Posted: 3 Apr 2009, 10:29:50 UTC - in response to Message 8112.  

This was my first ... now the internal number says 6 hours, the elapsed time said over 20 ... sigh ... now I gotta try to figure out if it is an accounting bug in BOINC 6.2.20 or a serious issue. I have two tasks now listed as taking 25 plus hours ... and a third at 18:47 with 5:48 to go ...

I did just update my drivers to the latest version 182.5 and it looks like maybe the control panel won't GPF any longer when started on my GTX 295 cards (I think it got confused with multiple GPU "Cores" to work with ...

The bad news is the tasks are now listed as taking 40 hours (the newest started) the good news is that they seem to be running faster... which may be an illusion, but, the time is falling several seconds per second ...

So, I don't know ... too many unknown unknowns for me to completely sort through ...

Since I just installed the 6.6.20 today I may have run into a "bug" with the different ways of counting elapsed time and the tasks were suspended for most of the day and as such then ran up this huge "bill" of run time. The second task is coming due to end ...

The second one also lists as 6 hours internally ...

Hmmm, GDF says nothing new with the tasks ... sigh ... I hate mysteries ...

BUt, this is another that is likely going to bite us in the butt when the new version hits the streets and people upgrade. Another thing ETA will have to watch out for ...

{edit}
something sure as heck happened ... it looks like my system went from 15-16 tasks a day to 8-10 ...


Well I know mine have been going all day because network times prevent internet access after 6:55am. It hasn't completed any other GPUgrid tasks, normally it would have 4 or 6 done by now.

I haven't updated my video card drivers (182.06) since I installed it 2 weeks ago and I haven't upgraded BOINC for a week (its 6.6.17).

I'll try shutting BOINC down in the first instance and see if the "to completion" time starts to move. If not will try a reboot and see if that helps.
BOINC blog
ID: 8115 · Rating: 0 · rate: Rate + / Rate - Report as offensive     Reply Quote
MarkJ
Volunteer moderator
Volunteer tester

Send message
Joined: 24 Dec 08
Posts: 738
Credit: 200,909,904
RAC: 0
Level
Leu
Scientific publications
watwatwatwatwatwatwatwatwatwatwatwatwatwatwatwatwatwatwatwatwat
Message 8116 - Posted: 3 Apr 2009, 10:42:28 UTC - in response to Message 8115.  

I'll try shutting BOINC down in the first instance and see if the "to completion" time starts to move. If not will try a reboot and see if that helps.


Restarting BOINC seems to have done the trick. Not sure if this is an app issue or a BOINC issue. Either way its moving fairly quickly now.
BOINC blog
ID: 8116 · Rating: 0 · rate: Rate + / Rate - Report as offensive     Reply Quote
MarkJ
Volunteer moderator
Volunteer tester

Send message
Joined: 24 Dec 08
Posts: 738
Credit: 200,909,904
RAC: 0
Level
Leu
Scientific publications
watwatwatwatwatwatwatwatwatwatwatwatwatwatwatwatwatwatwatwatwat
Message 8117 - Posted: 3 Apr 2009, 11:30:33 UTC - in response to Message 8108.  

I've got a couple on my GTX260 cards that have been running all day and they haven't finished yet. BOINC is saying its elapsed time is 18 hours 47 mins. They normally take 6 hours.

This is one of the offending ones.

Its still in "initial" state as it hasn't completed at the time I write this. There is nothing listed on the task that would make it appear any different.


Result uploaded and reported now. Interestingly it thinks its elapsed time is approx 23000 seconds (6.5 hours), when in fact it took over 20 hours.
BOINC blog
ID: 8117 · Rating: 0 · rate: Rate + / Rate - Report as offensive     Reply Quote
The_Bad_Penguin

Send message
Joined: 16 Dec 07
Posts: 23
Credit: 9,286,325
RAC: 0
Level
Ser
Scientific publications
watwatwatwatwatwatwatwatwat
Message 8123 - Posted: 3 Apr 2009, 13:35:59 UTC
Last modified: 3 Apr 2009, 13:38:07 UTC

I usually complete one wu per 24 hours.

Seems I haven't received ANY credit since ~ March 30.

Have two other wu's in queue, and very soon they're gonna already be past deadline.

Has to be more than a simple "elapsed time counter" problem, as even if counter was wrong, wu's would eventually finish up and be reported.

I have nothing in the past 4 days.

SOMETHING is going on.

I'll reboot, and see what happens, but I'm not holding my breath...
ID: 8123 · Rating: 0 · rate: Rate + / Rate - Report as offensive     Reply Quote
Profile Paul D. Buck

Send message
Joined: 9 Jun 08
Posts: 1050
Credit: 37,321,185
RAC: 0
Level
Val
Scientific publications
watwatwatwatwatwatwatwatwatwat
Message 8126 - Posted: 3 Apr 2009, 15:26:54 UTC

Its an infection I tell you ...

It looks like the tasks started last night are running at 6 hours elapsed time to completion ...

This is so strange ...
ID: 8126 · Rating: 0 · rate: Rate + / Rate - Report as offensive     Reply Quote
ExtraTerrestrial Apes
Volunteer moderator
Volunteer tester
Avatar

Send message
Joined: 17 Aug 08
Posts: 2705
Credit: 1,311,122,549
RAC: 0
Level
Met
Scientific publications
watwatwatwatwatwatwatwatwatwatwatwatwatwatwatwatwatwatwatwatwatwatwatwatwat
Message 8145 - Posted: 3 Apr 2009, 20:48:26 UTC

Oh dear.. too bad I can't watch my card closely any more, as it has been replaced by an irresistible offer from the red team and has moved to a *remote* machine now.

MrS
Scanning for our furry friends since Jan 2002
ID: 8145 · Rating: 0 · rate: Rate + / Rate - Report as offensive     Reply Quote
Profile Paul D. Buck

Send message
Joined: 9 Jun 08
Posts: 1050
Credit: 37,321,185
RAC: 0
Level
Val
Scientific publications
watwatwatwatwatwatwatwatwatwat
Message 8156 - Posted: 4 Apr 2009, 9:24:14 UTC

Well, that is too bad, but something is going on. I have 4 tasks in flight (as usual) two of which are running at 6 hours each and I have two listed at 12 (roughly) and they are only half way done.

I took a nap so I am not sure if I am seeing super long tasks, or, if 6.6.20 is suspending them because I am getting other tasks with deadlines that causes BOINC to suspend them and run other tasks (something that seems to be happening in VPU only tasks) ...

I took a big hit in production yesterday going from over 100K to about 90K so this is not my imagination.

The only question is if this is because I upgraded to 6.6.20 or something else ... or both ...

Since other have seen this on their systems ... I can't ascribe it all to 6.6.20 ...

The thing that really bothers me is that the internal accounting numbers from the tasks this morning showed 6 hours of run time ... but can I trust that number? That is the question ...
ID: 8156 · Rating: 0 · rate: Rate + / Rate - Report as offensive     Reply Quote
ExtraTerrestrial Apes
Volunteer moderator
Volunteer tester
Avatar

Send message
Joined: 17 Aug 08
Posts: 2705
Credit: 1,311,122,549
RAC: 0
Level
Met
Scientific publications
watwatwatwatwatwatwatwatwatwatwatwatwatwatwatwatwatwatwatwatwatwatwatwatwat
Message 8159 - Posted: 4 Apr 2009, 9:50:10 UTC - in response to Message 8156.  

Is the time per step showing anything unusual?

MrS
Scanning for our furry friends since Jan 2002
ID: 8159 · Rating: 0 · rate: Rate + / Rate - Report as offensive     Reply Quote
The_Bad_Penguin

Send message
Joined: 16 Dec 07
Posts: 23
Credit: 9,286,325
RAC: 0
Level
Ser
Scientific publications
watwatwatwatwatwatwatwatwat
Message 8169 - Posted: 4 Apr 2009, 13:33:58 UTC

I'm gonna be switching to F@H for the time being.

5 days and no wu's have finished, deadlines being missed, wu's being cancelled by server.

enough is enough.

call me when things get fixed.
ID: 8169 · Rating: 0 · rate: Rate + / Rate - Report as offensive     Reply Quote
MarkJ
Volunteer moderator
Volunteer tester

Send message
Joined: 24 Dec 08
Posts: 738
Credit: 200,909,904
RAC: 0
Level
Leu
Scientific publications
watwatwatwatwatwatwatwatwatwatwatwatwatwatwatwatwatwatwatwatwat
Message 8170 - Posted: 4 Apr 2009, 13:44:15 UTC - in response to Message 8169.  

I'm gonna be switching to F@H for the time being.

5 days and no wu's have finished, deadlines being missed, wu's being cancelled by server.

enough is enough.

call me when things get fixed.


I must admit I have been seeing a few "cancelled by server" work units and they are usually ones downloaded within the last 24 hours, so its not as if they'd reached the deadline.

Here is one.
Another
And a 3rd

They are on the machine with dual GTX260's so they normally get returned within 24 hours.
BOINC blog
ID: 8170 · Rating: 0 · rate: Rate + / Rate - Report as offensive     Reply Quote
ExtraTerrestrial Apes
Volunteer moderator
Volunteer tester
Avatar

Send message
Joined: 17 Aug 08
Posts: 2705
Credit: 1,311,122,549
RAC: 0
Level
Met
Scientific publications
watwatwatwatwatwatwatwatwatwatwatwatwatwatwatwatwatwatwatwatwatwatwatwatwat
Message 8176 - Posted: 4 Apr 2009, 15:23:07 UTC

The WUs are cancelled by the server because they're redundant (see here). They only get canceled if the result is not needed any more any if you didn't already start them. That's much better than wasting computation time, isn't it?

MrS
Scanning for our furry friends since Jan 2002
ID: 8176 · Rating: 0 · rate: Rate + / Rate - Report as offensive     Reply Quote
Profile Paul D. Buck

Send message
Joined: 9 Jun 08
Posts: 1050
Credit: 37,321,185
RAC: 0
Level
Val
Scientific publications
watwatwatwatwatwatwatwatwatwat
Message 8178 - Posted: 4 Apr 2009, 16:05:43 UTC - in response to Message 8159.  
Last modified: 4 Apr 2009, 16:09:29 UTC

Is the time per step showing anything unusual?

No...

35 MS per step ... I have the two latest coming up for turn in ... I will check those also ...

Since BOINC does not tell me which GPU it is running on, it is hard to know, but one of the tasks that is long running has the end tag "Ignasi", and history shows that many of those are "expirimental" and may cause side effects. So, what I was wondering if the one odd ball is slowing down the others.

{edit}
The current odd ball is the third of a triplet issued... the other two have returned the result already so I wonder if I am going to get cheated out on this one ...
ID: 8178 · Rating: 0 · rate: Rate + / Rate - Report as offensive     Reply Quote
Profile Paul D. Buck

Send message
Joined: 9 Jun 08
Posts: 1050
Credit: 37,321,185
RAC: 0
Level
Val
Scientific publications
watwatwatwatwatwatwatwatwatwat
Message 8182 - Posted: 4 Apr 2009, 19:41:24 UTC

Ok, my "bad" task just completed and on THIS one the time step sucked at 159 ms, and the time reflects the long run time at 79,645 seconds (21 hours) ...

Now, what is interesting is that the other task that is showing 22 hours seems to be speeding up and the time dropping like a rock. SO, was this task listed above causing problems with the OTHER task still running? Or is the other task, the "Ignasi" task the one with the problem?

SOMETHING is going on ... I did make the upgrade to 6.6.20 and the new drivers, but this is WEIRD ...
ID: 8182 · Rating: 0 · rate: Rate + / Rate - Report as offensive     Reply Quote
Profile GDF
Volunteer moderator
Project administrator
Project developer
Project tester
Volunteer developer
Volunteer tester
Project scientist

Send message
Joined: 14 Mar 07
Posts: 1958
Credit: 629,356
RAC: 0
Level
Gly
Scientific publications
watwatwatwatwat
Message 8183 - Posted: 4 Apr 2009, 19:59:20 UTC - in response to Message 8182.  

Could you downgrade BOINC/driver to check what's the problem?
Maybe, wait to see if the problem persists.

All workunits with the same name BBBB should last the same.
AAAA-BBBB-...

gdf
ID: 8183 · Rating: 0 · rate: Rate + / Rate - Report as offensive     Reply Quote
Profile Paul D. Buck

Send message
Joined: 9 Jun 08
Posts: 1050
Credit: 37,321,185
RAC: 0
Level
Val
Scientific publications
watwatwatwatwatwatwatwatwatwat
Message 8188 - Posted: 4 Apr 2009, 22:37:08 UTC - in response to Message 8183.  

Could you downgrade BOINC/driver to check what's the problem?
Maybe, wait to see if the problem persists.

All workunits with the same name BBBB should last the same.
AAAA-BBBB-...

gdf

Random in the sense that I can't find a common factor EXCEPT that while I am running slow tasks, one or more of those tasks end in "ignasi" which is why I brought that up. Usually when iggy makes new tasks he has the decency to put his name on them. What is puzzling is that not only the iggy task is slow, but a companion task is slow ... if two iggys are running at the same time, I had 4 slow tasks ...

Now I have NO iggy tasks on that machine and the 4 inflight seem to be on track for 6 hour run times. The oddest thing is that of all the tasks that seemed "troubled" only ONE of them seems to have recorded a bad time (150 MS plus step time) ...

If you look back at the last couple days production you can see I went from 15-16 per day to about 8 ... but when I llok at the tasks I can't see anything that out of place.

Which is why I also asked the question about trusting the internal run time number ... I think I listed tasks below where I *KNOW* the run time listed on the machine as 20+ hours and yet the run time logged was no where near that ...

Well, i will keep watching ... but I think you do have a problem ... and a task issued 3 times? Gotta be something going on there ...

Oh, at least the tasks complete successfully and validate, for what that is worth ...
ID: 8188 · Rating: 0 · rate: Rate + / Rate - Report as offensive     Reply Quote
1 · 2 · Next

Message boards : Graphics cards (GPUs) : Any changes to project since March30 ?

©2025 Universitat Pompeu Fabra