Experimental Python tasks (beta) - task description

Message boards : News : Experimental Python tasks (beta) - task description
Message board moderation

To post messages, you must log in.

Previous · 1 . . . 42 · 43 · 44 · 45 · 46 · 47 · 48 . . . 50 · Next

AuthorMessage
Pop Piasa
Avatar

Send message
Joined: 8 Aug 19
Posts: 252
Credit: 458,054,251
RAC: 0
Level
Gln
Scientific publications
watwat
Message 59845 - Posted: 31 Jan 2023, 4:13:08 UTC

Is there any way to reduce the estimated remaining time showing in the manager on these?

I'm seeing 20+ days left when there is really 10 hours and I can't download the next task until the previous is well past 90%. That's around an hour or so in advance as they are finishing in 9-12 hrs. on my hosts.

Setting my task que longer only gets me the server message:
Tasks won't finish in time: BOINC runs 100.0% of the time; computation is enabled 99.9% of that


It appears that the server sets completion times based on the average among completed WU run times. Seeing that Pythons misreport the run time (which must be equal to or greater than the CPU time) it is logical that the estimated future completion times would reflect the inflated CPU time figures.

Is there a local manager config fix for that, anyone? Multi gratis
ID: 59845 · Rating: 0 · rate: Rate + / Rate - Report as offensive     Reply Quote
abouh

Send message
Joined: 31 May 21
Posts: 200
Credit: 0
RAC: 0
Level

Scientific publications
wat
Message 59846 - Posted: 31 Jan 2023, 8:47:42 UTC - in response to Message 59843.  
Last modified: 31 Jan 2023, 8:51:36 UTC

Hello Keith,

I also think for some reason your machine ran the old run.py file. Maybe the error was in the server side and the old script was provided by some reason, but I went and checked the files in task e00004a01419-ABOU_rnd_ppod_expand_demos29_exp1-0-1-RND6419_2 and the error should not be present.

Also, as I mentioned in a previous post I added and extra log message to check if a GPU is detected

sys.stderr.write(f"Detected GPUs: {gpus_detected}\n")


Which is only printed with the new runs but not in the one you shared. For example, in this one:

https://www.gpugrid.net/result.php?resultid=33273691
ID: 59846 · Rating: 0 · rate: Rate + / Rate - Report as offensive     Reply Quote
KAMasud

Send message
Joined: 27 Jul 11
Posts: 138
Credit: 539,953,398
RAC: 0
Level
Lys
Scientific publications
watwat
Message 59847 - Posted: 31 Jan 2023, 13:05:00 UTC - in response to Message 59845.  

Is there any way to reduce the estimated remaining time showing in the manager on these?

I'm seeing 20+ days left when there is really 10 hours and I can't download the next task until the previous is well past 90%. That's around an hour or so in advance as they are finishing in 9-12 hrs. on my hosts.

Setting my task que longer only gets me the server message:
Tasks won't finish in time: BOINC runs 100.0% of the time; computation is enabled 99.9% of that


It appears that the server sets completion times based on the average among completed WU run times. Seeing that Pythons misreport the run time (which must be equal to or greater than the CPU time) it is logical that the estimated future completion times would reflect the inflated CPU time figures.

Is there a local manager config fix for that, anyone? Multi gratis

___________________

I will agree with Pop. The same thing is going on, on my machine.
ID: 59847 · Rating: 0 · rate: Rate + / Rate - Report as offensive     Reply Quote
Ian&Steve C.

Send message
Joined: 21 Feb 20
Posts: 1116
Credit: 40,839,470,595
RAC: 5,269
Level
Trp
Scientific publications
wat
Message 59848 - Posted: 31 Jan 2023, 15:03:25 UTC

abouh, are you planning to release another large batch of the new tasks?
ID: 59848 · Rating: 0 · rate: Rate + / Rate - Report as offensive     Reply Quote
abouh

Send message
Joined: 31 May 21
Posts: 200
Credit: 0
RAC: 0
Level

Scientific publications
wat
Message 59849 - Posted: 31 Jan 2023, 16:51:19 UTC - in response to Message 59848.  

Yes! The experiment I am currently running has a population of 1000 (so it maintains the number of submitted tasks to 1000 by sending a task every time 1 ends until a certain global goal is reached)

I am planning to start another 1000 agent experiment, probably tomorrow.


ID: 59849 · Rating: 0 · rate: Rate + / Rate - Report as offensive     Reply Quote
abouh

Send message
Joined: 31 May 21
Posts: 200
Credit: 0
RAC: 0
Level

Scientific publications
wat
Message 59850 - Posted: 31 Jan 2023, 16:54:30 UTC - in response to Message 59847.  

Unfortunately I think the best reference is the progress %. I don't know if that is of much help to calculate at what time a task will end, but the progress increase should be constant as long as the machine load is also constant throughout the task.
ID: 59850 · Rating: 0 · rate: Rate + / Rate - Report as offensive     Reply Quote
Pop Piasa
Avatar

Send message
Joined: 8 Aug 19
Posts: 252
Credit: 458,054,251
RAC: 0
Level
Gln
Scientific publications
watwat
Message 59851 - Posted: 31 Jan 2023, 20:21:26 UTC - in response to Message 59849.  
Last modified: 31 Jan 2023, 20:21:52 UTC

Thanks abouh,

When there is a steady flow of tasks from Grosso the window of time a host needs to secure a new WU and maintain constant production shrinks drastically.

There is no need then to try to download the next task until the present task is almost finished, so the estimate of time remaining becomes of little concern to me.
ID: 59851 · Rating: 0 · rate: Rate + / Rate - Report as offensive     Reply Quote
Richard Haselgrove

Send message
Joined: 11 Jul 09
Posts: 1639
Credit: 10,159,968,649
RAC: 351
Level
Trp
Scientific publications
watwatwatwatwatwatwatwatwatwatwatwatwatwatwatwatwatwatwatwatwatwatwatwatwatwatwat
Message 59852 - Posted: 31 Jan 2023, 22:21:08 UTC - in response to Message 59850.  

Unfortunately I think the best reference is the progress %. I don't know if that is of much help to calculate at what time a task will end, but the progress increase should be constant as long as the machine load is also constant throughout the task.

And that can be easily utilised by setting

fraction_done_exact
if set, base estimates of remaining time solely on the fraction done reported by the app.

in app_config.xml. It wobbles a bit at the beginning, but soon settles down.
ID: 59852 · Rating: 0 · rate: Rate + / Rate - Report as offensive     Reply Quote
KAMasud

Send message
Joined: 27 Jul 11
Posts: 138
Credit: 539,953,398
RAC: 0
Level
Lys
Scientific publications
watwat
Message 59853 - Posted: 1 Feb 2023, 1:22:03 UTC - in response to Message 59852.  
Last modified: 1 Feb 2023, 1:24:36 UTC

Unfortunately I think the best reference is the progress %. I don't know if that is of much help to calculate when a task will end, but the progress increase should be constant as long as the machine load is also constant throughout the task.

And that can be easily utilised by setting

fraction_done_exact
if set, base estimates of remaining time solely on the fraction done reported by the app.

in app_config.xml. It wobbles a bit at the beginning but soon settles down.

______________

Richard, could you point me in the direction of app_config.xml. Where is it? Second, are we not playing around a bit too much? No other project requires us to play. Unless we are up to mischief and try to run multiple WU's at the same time and when they start crashing, blame others.
ID: 59853 · Rating: 0 · rate: Rate + / Rate - Report as offensive     Reply Quote
Richard Haselgrove

Send message
Joined: 11 Jul 09
Posts: 1639
Credit: 10,159,968,649
RAC: 351
Level
Trp
Scientific publications
watwatwatwatwatwatwatwatwatwatwatwatwatwatwatwatwatwatwatwatwatwatwatwatwatwatwat
Message 59854 - Posted: 1 Feb 2023, 8:18:36 UTC - in response to Message 59853.  

ID: 59854 · Rating: 0 · rate: Rate + / Rate - Report as offensive     Reply Quote
KAMasud

Send message
Joined: 27 Jul 11
Posts: 138
Credit: 539,953,398
RAC: 0
Level
Lys
Scientific publications
watwat
Message 59855 - Posted: 1 Feb 2023, 11:45:10 UTC - in response to Message 59854.  

It's documented in the User manual, specifically at:

https://boinc.berkeley.edu/wiki/Client_configuration#Project-level_configuration

_____________
That I can find but not on my computer unless it is hidden.
ID: 59855 · Rating: 0 · rate: Rate + / Rate - Report as offensive     Reply Quote
Erich56

Send message
Joined: 1 Jan 15
Posts: 1166
Credit: 12,260,898,501
RAC: 1
Level
Trp
Scientific publications
watwatwatwatwatwatwatwatwat
Message 59856 - Posted: 1 Feb 2023, 13:50:54 UTC - in response to Message 59855.  
Last modified: 1 Feb 2023, 14:20:11 UTC

_____________
That I can find but not on my computer unless it is hidden.

the app_config.xml is not there automatically, that's why you won't find it.
You need to write it yourself e.g. by the Editor or Notepad, than save it as "app_config.xml" in the GPUGRID project folder within the BOINC folder (contained in the ProgramData folder).

In order to put the app_config.xml into effekt, after having done the above mentioned steps, you need to open the "Options" tab in the BOINC manager and push once "read config files".
ID: 59856 · Rating: 0 · rate: Rate + / Rate - Report as offensive     Reply Quote
KAMasud

Send message
Joined: 27 Jul 11
Posts: 138
Credit: 539,953,398
RAC: 0
Level
Lys
Scientific publications
watwat
Message 59857 - Posted: 1 Feb 2023, 17:01:53 UTC - in response to Message 59856.  

_____________
That I can find but not on my computer unless it is hidden.

the app_config.xml is not there automatically, that's why you won't find it.
You need to write it yourself e.g. by the Editor or Notepad, than save it as "app_config.xml" in the GPUGRID project folder within the BOINC folder (contained in the ProgramData folder).

In order to put the app_config.xml into effekt, after having done the above mentioned steps, you need to open the "Options" tab in the BOINC manager and push once "read config files".

_______________________________
Is this Boinc in only one place?
OS(C)
Program files
Boinc
Locale
Skins
Boinc
boinc_logo_black
Boinccmd
Boincmgr
Boincscr
boincsvcctrl
boinctray
ca-bundle
COPYING
COPYRIGHT
liberationSans-Regular.
This is all I can find in the Boinc folder. no GPUGrid folder or any other project folder.
Unless, Boinc is in two places like in the old days.
ID: 59857 · Rating: 0 · rate: Rate + / Rate - Report as offensive     Reply Quote
Pop Piasa
Avatar

Send message
Joined: 8 Aug 19
Posts: 252
Credit: 458,054,251
RAC: 0
Level
Gln
Scientific publications
watwat
Message 59858 - Posted: 1 Feb 2023, 18:09:47 UTC - in response to Message 59854.  

Thank you Richard Hazelgrove,

It's documented in the User manual, specifically at:

https://boinc.berkeley.edu/wiki/Client_configuration#Project-level_configuration


I was unaware of that info site. Will cure my ignorance.
Many thanks.
"Together we crunch
To check out a hunch
And wish all our credit
Could just buy us lunch"


Piasa Tribe - Illini Nation
ID: 59858 · Rating: 0 · rate: Rate + / Rate - Report as offensive     Reply Quote
Erich56

Send message
Joined: 1 Jan 15
Posts: 1166
Credit: 12,260,898,501
RAC: 1
Level
Trp
Scientific publications
watwatwatwatwatwatwatwatwat
Message 59859 - Posted: 1 Feb 2023, 18:30:55 UTC - in response to Message 59857.  


_______________________________
Is this Boinc in only one place?
OS(C)
Program files
Boinc
.,..
This is all I can find in the Boinc folder. no GPUGrid folder or any other project folder.
Unless, Boinc is in two places like in the old days.

You are in the wrong folder.
BOINC still is in two places.
Your have to navigate to C:/ProgramData/BOINC/projects/GPUGRID
ID: 59859 · Rating: 0 · rate: Rate + / Rate - Report as offensive     Reply Quote
Pop Piasa
Avatar

Send message
Joined: 8 Aug 19
Posts: 252
Credit: 458,054,251
RAC: 0
Level
Gln
Scientific publications
watwat
Message 59860 - Posted: 2 Feb 2023, 1:25:37 UTC - in response to Message 59857.  
Last modified: 2 Feb 2023, 1:43:45 UTC

Hi KAMasud,

The folder mentioned by Erich56 here...
Your have to navigate to C:/ProgramData/BOINC/projects/GPUGRID


...is a hidden folder in windows so you must choose to show hidden folders in the file preferences to access it. (just in case you might not know that and can't see it in the program manager)

[Edit] The target folder for the appconfig.xml file is actually
C:\ProgramData\BOINC\projects\www.gpugrid.net
on my hosts

Hope that helped.
ID: 59860 · Rating: 0 · rate: Rate + / Rate - Report as offensive     Reply Quote
Erich56

Send message
Joined: 1 Jan 15
Posts: 1166
Credit: 12,260,898,501
RAC: 1
Level
Trp
Scientific publications
watwatwatwatwatwatwatwatwat
Message 59862 - Posted: 2 Feb 2023, 4:43:19 UTC - in response to Message 59860.  

[Edit] The target folder for the appconfig.xml file is actually
C:\ProgramData\BOINC\projects\www.gpugrid.net

yes, this is true. Sorry for the confusion.

However, on my system (Windows 10 Pro) this folder is NOT a hidden folder.
ID: 59862 · Rating: 0 · rate: Rate + / Rate - Report as offensive     Reply Quote
KAMasud

Send message
Joined: 27 Jul 11
Posts: 138
Credit: 539,953,398
RAC: 0
Level
Lys
Scientific publications
watwat
Message 59863 - Posted: 2 Feb 2023, 6:09:36 UTC - in response to Message 59860.  

Hi KAMasud,

The folder mentioned by Erich56 here...
Your have to navigate to C:/ProgramData/BOINC/projects/GPUGRID


...is a hidden folder in windows so you must choose to show hidden folders in the file preferences to access it. (just in case you might not know that and can't see it in the program manager)

[Edit] The target folder for the appconfig.xml file is actually
C:\ProgramData\BOINC\projects\www.gpugrid.net
on my hosts

Hope that helped.

____________________

Thank you, Pop. After this update from Microsoft, Windows has become ____. Needs Administrator privileges for everything. Even though it is my private computer. Yesterday, I marked show hidden folders it promptly hid them back. Today I unhid them and told it "stay", good doggy. I found the second folder in which I found the projects folders. Thank you everyone.
Fat32 was better in some ways. I have done what you all wanted me to do but years ago. Maybe two decades ago.
Erich, Richard, thank you.
Apple products are user repair unfriendly. Laptops are becoming repair unfriendly and now, Microsoft, is going the same way.
ID: 59863 · Rating: 0 · rate: Rate + / Rate - Report as offensive     Reply Quote
Profile Retvari Zoltan
Avatar

Send message
Joined: 20 Jan 09
Posts: 2380
Credit: 16,897,957,044
RAC: 0
Level
Trp
Scientific publications
watwatwatwatwatwatwatwatwatwatwatwatwatwatwatwatwatwatwatwatwatwatwatwatwatwat
Message 59900 - Posted: 11 Feb 2023, 9:52:14 UTC

I've been away from GPUGrid for a while...
Is there a way to control the number of spawned threads?
I've tried to modify the line:
        <setenv>NTHREADS=$NTHREADS</setenv>
in linux_job.###########.xml file to
        <setenv>NTHREADS=8</setenv>
but it made no difference.
The task was started with the original NTHREADS setting.
Is it the reason for no change in the number of spawned threads, or I should modify something else?
ID: 59900 · Rating: 0 · rate: Rate + / Rate - Report as offensive     Reply Quote
Ian&Steve C.

Send message
Joined: 21 Feb 20
Posts: 1116
Credit: 40,839,470,595
RAC: 5,269
Level
Trp
Scientific publications
wat
Message 59901 - Posted: 11 Feb 2023, 13:28:59 UTC - in response to Message 59900.  

I've been away from GPUGrid for a while...
Is there a way to control the number of spawned threads?
I've tried to modify the line:
        <setenv>NTHREADS=$NTHREADS</setenv>
in linux_job.###########.xml file to
        <setenv>NTHREADS=8</setenv>
but it made no difference.
The task was started with the original NTHREADS setting.
Is it the reason for no change in the number of spawned threads, or I should modify something else?


there is no reason to do this anymore. they already fixed the overused CPU issue. it's now capped at 4x CPU threads and hard coded in the run.py script. but that is in addition to the 32 threads for the agents. there is no way to reduce that unless abouh wanted to use less agents, but i don't think he does at this time.

if you want to run python tasks, you need to account for this and just tell BOINC to reserve some extra CPU resources by setting a larger value for the cpu_usage in app_config. i use values between 8-10. but you can experiment with what you are happy with. on my python dedicated system, I stop all other CPU projects as that gives the best performance.
ID: 59901 · Rating: 0 · rate: Rate + / Rate - Report as offensive     Reply Quote
Previous · 1 . . . 42 · 43 · 44 · 45 · 46 · 47 · 48 . . . 50 · Next

Message boards : News : Experimental Python tasks (beta) - task description

©2025 Universitat Pompeu Fabra