CPU jobs on Linux

Message boards : News : CPU jobs on Linux
Message board moderation

To post messages, you must log in.

Previous · 1 · 2 · 3 · 4 · 5 . . . 9 · Next

AuthorMessage
Trotador

Send message
Joined: 25 Mar 12
Posts: 103
Credit: 14,948,929,771
RAC: 14
Level
Trp
Scientific publications
watwatwatwatwatwatwatwatwatwatwatwatwatwatwatwatwat
Message 48857 - Posted: 7 Feb 2018, 6:26:25 UTC

No higher credits for the tasks I've crunched yesterday and today.

I stop and will try to understand what is happening.
ID: 48857 · Rating: 0 · rate: Rate + / Rate - Report as offensive     Reply Quote
NUCCpod_NAPTIMELABS_01

Send message
Joined: 18 Aug 17
Posts: 6
Credit: 174,440,173
RAC: 0
Level
Ile
Scientific publications
watwat
Message 48858 - Posted: 7 Feb 2018, 6:33:41 UTC - in response to Message 48837.  

I've also been having trouble with work units erroring out in this way. I have over 150 cpu cores spread out across a variety of machines, all under linux. Just a few moments ago I attempted to attach to GPUGRID only to have computational error after computational error. I hope this is fixed shortly as I would love to have GPUGRID as one of my default projects.
ID: 48858 · Rating: 0 · rate: Rate + / Rate - Report as offensive     Reply Quote
Keith Myers
Avatar

Send message
Joined: 13 Dec 17
Posts: 1419
Credit: 9,119,446,190
RAC: 731
Level
Tyr
Scientific publications
watwatwatwatwat
Message 48859 - Posted: 7 Feb 2018, 7:46:25 UTC

Yes, credit is doing something very strange. I got 111 credits for 3292 seconds of cpu time.

klepel got 1362 credits for the same time.

Run time 3,262.28
CPU time 12,834.34
Validate state Valid
Credit 1,361.79

Task 13118994
ID: 48859 · Rating: 0 · rate: Rate + / Rate - Report as offensive     Reply Quote
kain

Send message
Joined: 3 Sep 14
Posts: 152
Credit: 918,557,369
RAC: 23
Level
Glu
Scientific publications
watwatwatwatwatwatwatwatwatwatwatwat
Message 48862 - Posted: 7 Feb 2018, 12:23:54 UTC - in response to Message 48847.  


I used 4 cores to generate 110 credits for 54 minutes of compute time.

I can use one core to generate 108 credits for 60 minutes of compute time for SETI CPU tasks.

No reason to run these tasks still for me.


Are you seriously comparing SETI to GPUGRID?!
ID: 48862 · Rating: 0 · rate: Rate + / Rate - Report as offensive     Reply Quote
mmonnin

Send message
Joined: 2 Jul 16
Posts: 338
Credit: 7,987,341,558
RAC: 213
Level
Tyr
Scientific publications
watwatwatwatwat
Message 48863 - Posted: 7 Feb 2018, 12:53:02 UTC - in response to Message 48862.  


I used 4 cores to generate 110 credits for 54 minutes of compute time.

I can use one core to generate 108 credits for 60 minutes of compute time for SETI CPU tasks.

No reason to run these tasks still for me.


Are you seriously comparing SETI to GPUGRID?!


CreditNew is CreditNew.
ID: 48863 · Rating: 0 · rate: Rate + / Rate - Report as offensive     Reply Quote
PappaLitto

Send message
Joined: 21 Mar 16
Posts: 513
Credit: 4,673,458,277
RAC: 0
Level
Arg
Scientific publications
watwatwatwatwatwatwatwat
Message 48864 - Posted: 7 Feb 2018, 13:33:40 UTC - in response to Message 48863.  


I used 4 cores to generate 110 credits for 54 minutes of compute time.

I can use one core to generate 108 credits for 60 minutes of compute time for SETI CPU tasks.

No reason to run these tasks still for me.


Are you seriously comparing SETI to GPUGRID?!


CreditNew is CreditNew.

You cannot compare one project's credit to another.
ID: 48864 · Rating: 0 · rate: Rate + / Rate - Report as offensive     Reply Quote
computezrmle

Send message
Joined: 10 Jun 13
Posts: 9
Credit: 295,692,471
RAC: 0
Level
Asn
Scientific publications
watwatwatwatwat
Message 48866 - Posted: 7 Feb 2018, 16:54:10 UTC - in response to Message 48864.  

You cannot compare one project's credit to another.

At least it should be comparable as described in the BOINC documentation.
See: http://boinc.berkeley.edu/trac/wiki/CreditNew#Cross-projectversionnormalization
ID: 48866 · Rating: 0 · rate: Rate + / Rate - Report as offensive     Reply Quote
Keith Myers
Avatar

Send message
Joined: 13 Dec 17
Posts: 1419
Credit: 9,119,446,190
RAC: 731
Level
Tyr
Scientific publications
watwatwatwatwat
Message 48867 - Posted: 7 Feb 2018, 17:01:28 UTC - in response to Message 48864.  


I used 4 cores to generate 110 credits for 54 minutes of compute time.

I can use one core to generate 108 credits for 60 minutes of compute time for SETI CPU tasks.

No reason to run these tasks still for me.


Are you seriously comparing SETI to GPUGRID?!


CreditNew is CreditNew.

You cannot compare one project's credit to another.

One of the stated objectives of CreditNew it to make credit the same across all projects for the same amount of cobblestones used to compute.
ID: 48867 · Rating: 0 · rate: Rate + / Rate - Report as offensive     Reply Quote
mmonnin

Send message
Joined: 2 Jul 16
Posts: 338
Credit: 7,987,341,558
RAC: 213
Level
Tyr
Scientific publications
watwatwatwatwat
Message 48872 - Posted: 7 Feb 2018, 18:35:25 UTC

Thus my comment about CreditNew. CPU projects that have higher or lower then typical RAC are most likely using something else besides CreditNew like a fixed credit or another algorithm.
ID: 48872 · Rating: 0 · rate: Rate + / Rate - Report as offensive     Reply Quote
Keith Myers
Avatar

Send message
Joined: 13 Dec 17
Posts: 1419
Credit: 9,119,446,190
RAC: 731
Level
Tyr
Scientific publications
watwatwatwatwat
Message 48873 - Posted: 7 Feb 2018, 18:46:31 UTC - in response to Message 48872.  

Yes I understood your post and sentiment. My post was directed at the other poster's incredulous comment.

This project itself utilize both mechanisms. CreditNew for cpu tasks and fixed credit awards for gpu tasks.

As far as I have been able to find, that is unique among projects. Usually it is either/or not both.
ID: 48873 · Rating: 0 · rate: Rate + / Rate - Report as offensive     Reply Quote
mmonnin

Send message
Joined: 2 Jul 16
Posts: 338
Credit: 7,987,341,558
RAC: 213
Level
Tyr
Scientific publications
watwatwatwatwat
Message 48876 - Posted: 7 Feb 2018, 22:11:38 UTC - in response to Message 48873.  

Yes I understood your post and sentiment. My post was directed at the other poster's incredulous comment.

This project itself utilize both mechanisms. CreditNew for cpu tasks and fixed credit awards for gpu tasks.

As far as I have been able to find, that is unique among projects. Usually it is either/or not both.


I wasn't referencing you as I didn't quote you. ;)

I agree.
ID: 48876 · Rating: 0 · rate: Rate + / Rate - Report as offensive     Reply Quote
Profile Michael H.W. Weber

Send message
Joined: 9 Feb 16
Posts: 78
Credit: 656,229,684
RAC: 0
Level
Lys
Scientific publications
watwatwatwatwatwatwatwat
Message 48893 - Posted: 10 Feb 2018, 15:13:42 UTC
Last modified: 10 Feb 2018, 15:18:45 UTC

Would you please list the QC project progress on the server status page as well:

http://www.gpugrid.net/server_status.php

Thanks.

Another issue is that your app does not dynamically allocate CPU cores according to the BOINC settings. Instead it claims all physically present cores. That is a major problem when trying to run computations on the GPU as well because to do so, the GPU project automatically (or I manually) reserve(s) one CPU core per GPU task.
Example: When the BOINC manager is set to use 7 of the 8 cores to do CPU computations, your CPU client grabs all 8 cores (or since recently 2x 4 cores).
That is not acceptable.
Please fix this to attract more people to donate CPU cycles to your project.

Michael.
President of Rechenkraft.net - Germany's first and largest distributed computing organization.
ID: 48893 · Rating: 0 · rate: Rate + / Rate - Report as offensive     Reply Quote
Profile SMTB1963
Avatar

Send message
Joined: 27 Jun 10
Posts: 38
Credit: 524,420,921
RAC: 0
Level
Lys
Scientific publications
watwatwatwatwatwatwatwatwatwatwat
Message 48894 - Posted: 10 Feb 2018, 17:48:45 UTC - in response to Message 48893.  

Another issue is that your app does not dynamically allocate CPU cores according to the BOINC settings. Instead it claims all physically present cores.


I'm seeing this behavior as well. On my Ryzen 1700X system with 2 GPUs, these WUs basically take over all CPUs and throw the GPUs into "Waiting to run".

I suppose one could set max_concurrent in an app_config.xml to fix this...what would be the proper app name to use?
ID: 48894 · Rating: 0 · rate: Rate + / Rate - Report as offensive     Reply Quote
Keith Myers
Avatar

Send message
Joined: 13 Dec 17
Posts: 1419
Credit: 9,119,446,190
RAC: 731
Level
Tyr
Scientific publications
watwatwatwatwat
Message 48895 - Posted: 10 Feb 2018, 17:54:26 UTC - in response to Message 48894.  

QC is the proper app name. This is how I limit QC to 2 threads per task.

<app>
<name>QC</name>
<max_concurrent>1</max_concurrent>
</app>
<app_version>
<app_name>QC</app_name>
<plan_class>mt</plan_class>
<avg_ncpus>2.000000</avg_ncpus>
<cmdline>--nthreads 2</cmdline>


I also just run 1 task at a time to avoid the starting two tasks at the same time flaw in the application.
ID: 48895 · Rating: 0 · rate: Rate + / Rate - Report as offensive     Reply Quote
[VENETO] boboviz

Send message
Joined: 10 Sep 10
Posts: 164
Credit: 388,132
RAC: 0
Level

Scientific publications
wat
Message 48988 - Posted: 19 Feb 2018, 8:24:05 UTC

An error on a wu (task 17040682):

<core_client_version>7.8.3</core_client_version>
<![CDATA[
<message>
process exited with code 195 (0xc3, -61)</message>
<stderr_txt>
An HTTP error occurred when trying to retrieve this URL.
HTTP errors are often intermittent, and a simple retry will get you on your way.
ConnectionError(MaxRetryError("HTTPSConnectionPool(host='repo.continuum.io', port=443): Max retries exceeded with url: /pkgs/main/linux-64/repodata.json.bz2 (Caused by NewConnectionError('<urllib3.connection.VerifiedHTTPSConnection object at 0x7fc2a3f56860>: Failed to establish a new connection: [Errno -3] Temporary failure in name resolution',))",),)


Traceback (most recent call last):
File "pre_script.py", line 13, in <module>
raise Exception("Error installing h5py")
Exception: Error installing h5py
08:49:09 (1252): $PROJECT_DIR/miniconda/bin/python exited; CPU time 0.469942
08:49:09 (1252): app exit status: 0x1
08:49:09 (1252): called boinc_finish(195)
ID: 48988 · Rating: 0 · rate: Rate + / Rate - Report as offensive     Reply Quote
Profile Logan Carr

Send message
Joined: 12 Aug 15
Posts: 240
Credit: 64,069,811
RAC: 0
Level
Thr
Scientific publications
watwatwatwat
Message 49092 - Posted: 24 Feb 2018, 0:21:06 UTC - in response to Message 48820.  

Hi, we need more CPUs on Linux to run QM simulations. Anybody can help?


okay, I'm going to install Linux right now on my computer and it should be ready tonight or early tomorrow.
Cruncher/Learner in progress.
ID: 49092 · Rating: 0 · rate: Rate + / Rate - Report as offensive     Reply Quote
biodoc

Send message
Joined: 26 Aug 08
Posts: 183
Credit: 10,085,929,375
RAC: 0
Level
Trp
Scientific publications
watwatwatwatwatwatwatwatwatwatwatwatwatwatwatwatwatwatwatwatwatwat
Message 49094 - Posted: 24 Feb 2018, 12:04:32 UTC - in response to Message 48988.  


Traceback (most recent call last):
File "pre_script.py", line 13, in <module>
raise Exception("Error installing h5py")
Exception: Error installing h5py
08:49:09 (1252): $PROJECT_DIR/miniconda/bin/python exited; CPU time 0.469942
08:49:09 (1252): app exit status: 0x1
08:49:09 (1252): called boinc_finish(195)


If you are running the latest distros from Ubuntu or Mint you may need to install the python-support package.

wget http://launchpadlibrarian.net/109052632/python-support_1.0.15_all.deb

sudo dpkg -i python-support_1.0.15_all.deb
ID: 49094 · Rating: 0 · rate: Rate + / Rate - Report as offensive     Reply Quote
kain

Send message
Joined: 3 Sep 14
Posts: 152
Credit: 918,557,369
RAC: 23
Level
Glu
Scientific publications
watwatwatwatwatwatwatwatwatwatwatwat
Message 49096 - Posted: 24 Feb 2018, 15:57:29 UTC

One of my CPUs is crunching linux jobs, I will add two more next week.
ID: 49096 · Rating: 0 · rate: Rate + / Rate - Report as offensive     Reply Quote
kain

Send message
Joined: 3 Sep 14
Posts: 152
Credit: 918,557,369
RAC: 23
Level
Glu
Scientific publications
watwatwatwatwatwatwatwatwatwatwatwat
Message 49164 - Posted: 16 Mar 2018, 11:53:15 UTC

I have added three. Athlon 5350, I3 3240, I3 6100. Not the best ones but doing their job. I can add more but I would like to know how many WUs can we expect for the Linux QM app?

BTW ATM just 92 users are crunching QM, that's sad :(
ID: 49164 · Rating: 0 · rate: Rate + / Rate - Report as offensive     Reply Quote
Jim1348

Send message
Joined: 28 Jul 12
Posts: 819
Credit: 1,591,285,971
RAC: 0
Level
His
Scientific publications
watwatwatwatwatwatwatwatwatwatwatwatwatwatwat
Message 49165 - Posted: 16 Mar 2018, 13:05:14 UTC - in response to Message 49164.  
Last modified: 16 Mar 2018, 13:09:04 UTC

I have added another machine, and now have four on it (2 i7-3770, 1 i7-4790 and 1 Ryzen 1700), with two to four cores per machine allocated via the resource share. The main problem in running them is that when two or more start up at the same time, they error out. That happens mainly during reboots, but otherwise they never start up at the same time. I leave my machines running 24/7, so I don't reboot very often.

And to minimize the problem, you can run with the default 4 cores per work unit and only 4 cores (or less) per machine on average, so that they usually don't start more than one work unit at a time anyway. In that way, it is a manageable problem for me, though it would be best if they fix it. I am sure more people would then be willing to run it.

Also a Windows version would help of course, and for that they had better be seriously thinking about VirtualBox.
ID: 49165 · Rating: 0 · rate: Rate + / Rate - Report as offensive     Reply Quote
Previous · 1 · 2 · 3 · 4 · 5 . . . 9 · Next

Message boards : News : CPU jobs on Linux

©2025 Universitat Pompeu Fabra