Duplicated work

Message boards : Number crunching : Duplicated work
Message board moderation

To post messages, you must log in.

Previous · 1 · 2

AuthorMessage
Profile Retvari Zoltan
Avatar

Send message
Joined: 20 Jan 09
Posts: 2380
Credit: 16,897,957,044
RAC: 0
Level
Trp
Scientific publications
watwatwatwatwatwatwatwatwatwatwatwatwatwatwatwatwatwatwatwatwatwatwatwatwatwat
Message 21456 - Posted: 15 Jun 2011, 8:54:23 UTC

At least long workunits should not be sent to slow hosts. They have no chance to return them even in 5 days. I'm still getting resent tasks, which are still being processed on GT 9500 or GT 9600 cards. I don't feel like warning these users one by one. I wrote about it in march.
ID: 21456 · Rating: 0 · rate: Rate + / Rate - Report as offensive     Reply Quote
Profile Carlesa25
Avatar

Send message
Joined: 13 Nov 10
Posts: 328
Credit: 72,619,453
RAC: 0
Level
Thr
Scientific publications
watwatwatwatwatwatwatwatwatwatwatwatwatwatwatwatwat
Message 21463 - Posted: 15 Jun 2011, 19:12:59 UTC - in response to Message 21456.  


Hi: Following the points made here, also estimated that if the project needs the results in two days makes no sense to confuse users with the story of five days and generate duplication.

It generates the five days that users with GPUs that need three or four days to complete a task (not learn it in two days) participate miserably losing time, money and enthusiasm to participate in these tasks.

A) Identifies that GPUs is installed, if it is not recommended simply not accept the participation in this project.

B) When the time came to execute the task, if it has not been completed, canceled automatically (Boinc can do it perfectly) and now if it is referred to another user.

When you cancel once or twice a task and you find out what the problem and project requirements in which you participate. Greetings.
ID: 21463 · Rating: 0 · rate: Rate + / Rate - Report as offensive     Reply Quote
Profile skgiven
Volunteer moderator
Volunteer tester
Avatar

Send message
Joined: 23 Apr 09
Posts: 3968
Credit: 1,995,359,260
RAC: 0
Level
His
Scientific publications
watwatwatwatwatwatwatwatwatwatwatwatwatwatwatwatwatwatwatwatwatwatwatwatwatwatwatwat
Message 21475 - Posted: 16 Jun 2011, 12:03:33 UTC - in response to Message 21463.  
Last modified: 16 Jun 2011, 23:40:55 UTC

Present GIANNI_KKFREE tasks now use less GPU (87% compared to 98%) and do not cause any Lag on either of my Fermi systems.

There was internal tests done prior to doing Beta tests (for those that run Betas). Some problems were identified and fixed. Then more Betas were run, and reported on by many users. It was not until the tasks were run by many users with many different systems that the extent of the lag was fully realized. The main concern was to see if the tasks were stable, rather than if each users system is fine with the new tasks. The feedback from the crunching contributors has prompted the modifications that have been implemented. Ergo the system works.

Most of the suggestions made in this thread had already been suggested, some time ago, along with others. Some things were looked at but not implemented. Other suggestions have been implemented, and the scientists came up with plenty of ideas of their own.

The project is in the main production phase; it spent a long time in development, now it's trying to get through some research, and just keep up with driver/cuda and GPU changes.

Please appreciate that GPUGrid is a small team with limited resources (especially manpower) and I know the team spent considerable time in the recent past trying to modify some systems unsuccessfully. This is a working project, and while there are problems it's not broken. It's very much down to the cruncher to accommodate this project; it's not a hook up and forget project (never was and never can be). You need to have an appropriate GPU, install the correct drivers, configure your system to crunch GPU tasks, use fan controlling software (hopefully), and modify things on occasions (SWAN_SYNC, Priorities, CPU usage).

While I did see some lag it was not as bad as others have described. I think it was (might still be?) impacting on some systems more than others. Perhaps W7 and Linux are on the whole more affected and perhaps the GTX460 is more affected than the CC2.0 cards. Lag might also depend whether or not Aero is installed/what flavor of Linux you are using, and on other hardware and setups (cores used for example), as well as what tasks are being crunched.

Post up what you are crunching and if you are experiencing any lag or not.
ID: 21475 · Rating: 0 · rate: Rate + / Rate - Report as offensive     Reply Quote
Profile GDF
Volunteer moderator
Project administrator
Project developer
Project tester
Volunteer developer
Volunteer tester
Project scientist

Send message
Joined: 14 Mar 07
Posts: 1958
Credit: 629,356
RAC: 0
Level
Gly
Scientific publications
watwatwatwatwat
Message 21477 - Posted: 16 Jun 2011, 12:42:10 UTC - in response to Message 21475.  

We have increased the duplication time from 2 to 3 days.

gdf
ID: 21477 · Rating: 0 · rate: Rate + / Rate - Report as offensive     Reply Quote
Profile Carlesa25
Avatar

Send message
Joined: 13 Nov 10
Posts: 328
Credit: 72,619,453
RAC: 0
Level
Thr
Scientific publications
watwatwatwatwatwatwatwatwatwatwatwatwatwatwatwatwat
Message 21583 - Posted: 3 Jul 2011, 13:56:05 UTC - in response to Message 21477.  

We have increased the duplication time from 2 to 3 days.
gdf


Hello: The increase of 2 to 3 days I think it was positive, not get so many duplicate tasks ... today.

The two tasks that I have received recently, had been implemented by others but within minutes one is over, so I was performing was just useless, I canceled the other and see ...

I sincerely believe that the best solición to this question is simply adapt the return time of the tasks to the maximum time allowed, two, three, etc ... days and nothing more. Greetings.
ID: 21583 · Rating: 0 · rate: Rate + / Rate - Report as offensive     Reply Quote
Profile Carlesa25
Avatar

Send message
Joined: 13 Nov 10
Posts: 328
Credit: 72,619,453
RAC: 0
Level
Thr
Scientific publications
watwatwatwatwatwatwatwatwatwatwatwatwatwatwatwatwat
Message 21657 - Posted: 9 Jul 2011, 21:07:36 UTC - in response to Message 21583.  

Hi: The two tasks that I am running right now are in the process by another user.

One of them has seven tasks in the process (with NVIDIA Quadro 4000) lowered on day 6 so have already been forwarded to as many users ... !!!

I know not much, I say so, but I keep insisting that it makes sense to keep two dates to complete the tasks, the famous 3 to 5 days. Greetings.
ID: 21657 · Rating: 0 · rate: Rate + / Rate - Report as offensive     Reply Quote
Profile Carlesa25
Avatar

Send message
Joined: 13 Nov 10
Posts: 328
Credit: 72,619,453
RAC: 0
Level
Thr
Scientific publications
watwatwatwatwatwatwatwatwatwatwatwatwatwatwatwatwat
Message 21680 - Posted: 16 Jul 2011, 16:10:54 UTC - in response to Message 21657.  

Hello: Every time I understand forwarding policies least half-finished tasks ... Now I get one that makes 24 hours sent to another user ... as 3 days or case. Greetings.

Task: http://www.gpugrid.net/workunit.php?wuid=2579334
ID: 21680 · Rating: 0 · rate: Rate + / Rate - Report as offensive     Reply Quote
Profile skgiven
Volunteer moderator
Volunteer tester
Avatar

Send message
Joined: 23 Apr 09
Posts: 3968
Credit: 1,995,359,260
RAC: 0
Level
His
Scientific publications
watwatwatwatwatwatwatwatwatwatwatwatwatwatwatwatwatwatwatwatwatwatwatwatwatwatwatwat
Message 21681 - Posted: 16 Jul 2011, 18:04:29 UTC - in response to Message 21680.  

Thought I already explained that situation?

Anyway,
The 1st resend was after 3days - the task was not returned, this is normal.
The 2nd resend occurred after that because the task failed (user aborted in this case) - this is also normal; a failure also triggers a resend, even if a resend is already in progress.
ID: 21681 · Rating: 0 · rate: Rate + / Rate - Report as offensive     Reply Quote
Profile Carlesa25
Avatar

Send message
Joined: 13 Nov 10
Posts: 328
Credit: 72,619,453
RAC: 0
Level
Thr
Scientific publications
watwatwatwatwatwatwatwatwatwatwatwatwatwatwatwatwat
Message 21682 - Posted: 16 Jul 2011, 18:19:51 UTC - in response to Message 21681.  
Last modified: 16 Jul 2011, 18:22:07 UTC

Thought I already explained that situation?

Anyway,
The 1st resend was after 3days - the task was not returned, this is normal.
The 2nd resend occurred after that because the task failed (user aborted in this case) - this is also normal; a failure also triggers a resend, even if a resend is already in progress.


Hello, explain whether this and I also understand, but continues with little sense, in my opinion.

The first user does not comply within 3 days and the task is forwarded, fine, but continues its execution. (sure he still believes that you have 5 days ...)

The second user receives the task and the performer is usually from about 24 hours ago

We return to the first user, which is now the task is aborted runs (you will notice that they have forwarded their task and makes no sense to stick with it ...?), well.

But I do not understand is that when you cancel the task by the first user to generate a new shipment of the same task, when it is running normally by another user and within 3 days.

If this does not duplicate work without any need and I will say it is. Greetings.
ID: 21682 · Rating: 0 · rate: Rate + / Rate - Report as offensive     Reply Quote
Profile skgiven
Volunteer moderator
Volunteer tester
Avatar

Send message
Joined: 23 Apr 09
Posts: 3968
Credit: 1,995,359,260
RAC: 0
Level
His
Scientific publications
watwatwatwatwatwatwatwatwatwatwatwatwatwatwatwatwatwatwatwatwatwatwatwatwatwatwatwat
Message 21684 - Posted: 16 Jul 2011, 18:56:12 UTC - in response to Message 21682.  

Perhaps this system needs to be revised, but the importance of returning the task increases following a failure. As to why the user aborted, who knows. Usually they just think it will take too long, but they might have a problem with it, which would make it more important to return the task. The server cannot know that the first resend was/is running smoothly. Should one of the resends finish before the other starts (when someone has their cache too high) the other resend will not start (assuming intervening communications).

Suggested revision:
First resends only to be sent to CC2.0 cards
Second resends to be sent internally
ID: 21684 · Rating: 0 · rate: Rate + / Rate - Report as offensive     Reply Quote
Dagorath

Send message
Joined: 16 Mar 11
Posts: 509
Credit: 179,005,236
RAC: 0
Level
Ile
Scientific publications
watwatwatwatwatwatwatwatwatwatwatwatwatwat
Message 21687 - Posted: 16 Jul 2011, 20:18:07 UTC - in response to Message 21684.  
Last modified: 16 Jul 2011, 20:25:00 UTC

The system certainly does need to be revised but let's keep it simple rather than make it more complicated. The suggestion to issue resends only to CC 2.0 hosts would require new code on the server side, I think.

The simple, easy to implement solution is to get rid of the double deadline that now exists. Having a 3 day deadline plus another 5 day deadline leads to waste as has been demonstrated numerous times in this thread. Have 1 deadline and 1 deadline only. If you think I'm asking to have the 3 day deadline increased then read the post again because that's NOT what I want.

BOINC server code already has an option to issue resends to fast, reliable hosts. The code to do that was tested and proven several years ago. Why not use that instead of writing new code to issue resends to CC 2.0 hosts?
ID: 21687 · Rating: 0 · rate: Rate + / Rate - Report as offensive     Reply Quote
Profile Carlesa25
Avatar

Send message
Joined: 13 Nov 10
Posts: 328
Credit: 72,619,453
RAC: 0
Level
Thr
Scientific publications
watwatwatwatwatwatwatwatwatwatwatwatwatwatwatwatwat
Message 21688 - Posted: 16 Jul 2011, 20:36:24 UTC - in response to Message 21684.  

Perhaps this system needs to be revised, but the importance of returning the task increases following a failure. As to why the user aborted, who knows. Usually they just think it will take too long, but they might have a problem with it, which would make it more important to return the task. The server cannot know that the first resend was/is running smoothly. Should one of the resends finish before the other starts (when someone has their cache too high) the other resend will not start (assuming intervening communications).

Suggested revision:
First resends only to be sent to CC2.0 cards
Second resends to be sent internally



Hi Allow me a suggestion on the topic:

1 - Leave only 3 days to complete a task, Amul duplication of 3 to 5
days. Avoid many forwards, duplicate tasks and less load to the server.

2 - The server knows when a task is running for being
forwarded and no problem as the task in the previous user will
automatically canceled prior to forwarding.

The references that I'm putting in these comments are only those that affect me personally, I know for sure, but I fear that the volume of duplicate tasks is very high. Greetings.
ID: 21688 · Rating: 0 · rate: Rate + / Rate - Report as offensive     Reply Quote
Profile Carlesa25
Avatar

Send message
Joined: 13 Nov 10
Posts: 328
Credit: 72,619,453
RAC: 0
Level
Thr
Scientific publications
watwatwatwatwatwatwatwatwatwatwatwatwatwatwatwatwat
Message 21716 - Posted: 21 Jul 2011, 18:10:45 UTC - in response to Message 21688.  
Last modified: 21 Jul 2011, 18:48:06 UTC

Hello: Following what I promised, I put a link to a new task duplicate.

http://www.gpugrid.net/workunit.php?wuid=2589619

If the term to complete a task was bound and 3 days, this circumstance would not occur.

It is a partner with a GT9600 that takes at least 44 hours (nonstop) to execute a task is working with the idea and hope to run a useful task, when in reality it is not, all tasks have doubled. Greetings.
ID: 21716 · Rating: 0 · rate: Rate + / Rate - Report as offensive     Reply Quote
Profile Carlesa25
Avatar

Send message
Joined: 13 Nov 10
Posts: 328
Credit: 72,619,453
RAC: 0
Level
Thr
Scientific publications
watwatwatwatwatwatwatwatwatwatwatwatwatwatwatwatwat
Message 21890 - Posted: 26 Aug 2011, 14:36:56 UTC - in response to Message 21716.  


Hello: Again is coming more duplicate tasks

The last four tasks (including the two that I am performing at this time) are underway by other users that have exceeded initial three days.

Honestly is a topic that I will never tire of repeating and claiming for the loss of effort (and money) it means ... regrettable. Greetings.
ID: 21890 · Rating: 0 · rate: Rate + / Rate - Report as offensive     Reply Quote
Profile Carlesa25
Avatar

Send message
Joined: 13 Nov 10
Posts: 328
Credit: 72,619,453
RAC: 0
Level
Thr
Scientific publications
watwatwatwatwatwatwatwatwatwatwatwatwatwatwatwatwat
Message 21894 - Posted: 27 Aug 2011, 14:44:33 UTC - in response to Message 21890.  


Hello: We are the same, the two tasks I have just received are also duplicated.

This time one has been completed by someone else soon after I start the same task (one hour), have been canceled and proceeded to ask a new one (it is not published ... thank goodness).

I keep arguing, it seems that without success, that if the program needs the results in - three days - this is the real deadline to complete the task and nothing more ... is simple ... no? Greetings.
ID: 21894 · Rating: 0 · rate: Rate + / Rate - Report as offensive     Reply Quote
Profile GDF
Volunteer moderator
Project administrator
Project developer
Project tester
Volunteer developer
Volunteer tester
Project scientist

Send message
Joined: 14 Mar 07
Posts: 1958
Credit: 629,356
RAC: 0
Level
Gly
Scientific publications
watwatwatwatwat
Message 21974 - Posted: 7 Sep 2011, 14:12:01 UTC - in response to Message 21894.  
Last modified: 7 Sep 2011, 14:23:32 UTC

I have just removed the duplication mechanism as announced in another thread.
It will take few days before all duplicated workunits disappear.
Deadline for now stays at 5 days.

gdf
ID: 21974 · Rating: 0 · rate: Rate + / Rate - Report as offensive     Reply Quote
Profile Saenger
Avatar

Send message
Joined: 20 Jul 08
Posts: 134
Credit: 23,657,183
RAC: 0
Level
Pro
Scientific publications
watwatwatwatwatwat
Message 21975 - Posted: 7 Sep 2011, 15:37:30 UTC - in response to Message 21974.  

I have just removed the duplication mechanism as announced in another thread.
It will take few days before all duplicated workunits disappear.
Deadline for now stays at 5 days.

gdf

Thank you very much!

So now it's like this:
less than 24h (except PYRT): bonus of 100%
less than 48h: bonus of 50%
before deadline: normal credits, no redundant results
after deadline: new result sent.

(BTW: What's the 100%-time for PYRT?)
Gruesse vom Saenger

For questions about Boinc look in the BOINC-Wiki
ID: 21975 · Rating: 0 · rate: Rate + / Rate - Report as offensive     Reply Quote
Profile GDF
Volunteer moderator
Project administrator
Project developer
Project tester
Volunteer developer
Volunteer tester
Project scientist

Send message
Joined: 14 Mar 07
Posts: 1958
Credit: 629,356
RAC: 0
Level
Gly
Scientific publications
watwatwatwatwat
Message 21976 - Posted: 7 Sep 2011, 16:04:27 UTC - in response to Message 21975.  

I have just removed the duplication mechanism as announced in another thread.
It will take few days before all duplicated workunits disappear.
Deadline for now stays at 5 days.

gdf

Thank you very much!

So now it's like this:
less than 24h (except PYRT): bonus of 100%
less than 48h: bonus of 50%
before deadline: normal credits, no redundant results
after deadline: new result sent.

(BTW: What's the 100%-time for PYRT?)



I still have to talk with Ignasi about PYRT because we just came back from Berlin. But it should be valid for all workunits.

gdf
ID: 21976 · Rating: 0 · rate: Rate + / Rate - Report as offensive     Reply Quote
Profile Carlesa25
Avatar

Send message
Joined: 13 Nov 10
Posts: 328
Credit: 72,619,453
RAC: 0
Level
Thr
Scientific publications
watwatwatwatwatwatwatwatwatwatwatwatwatwatwatwatwat
Message 21977 - Posted: 7 Sep 2011, 16:35:32 UTC - in response to Message 21974.  

Hello. Thanks for the change.
ID: 21977 · Rating: 0 · rate: Rate + / Rate - Report as offensive     Reply Quote
Previous · 1 · 2

Message boards : Number crunching : Duplicated work

©2025 Universitat Pompeu Fabra