Experimental Python tasks (beta) - task description

Message boards : News : Experimental Python tasks (beta) - task description
Message board moderation

To post messages, you must log in.

Previous · 1 . . . 45 · 46 · 47 · 48 · 49 · 50 · Next

AuthorMessage
abouh

Send message
Joined: 31 May 21
Posts: 200
Credit: 0
RAC: 0
Level

Scientific publications
wat
Message 59950 - Posted: 17 Feb 2023, 17:22:15 UTC - in response to Message 59949.  

Yes over the weekend I will review the results of the 2 experiments that just finished and start new ones. The idea is to continue like until now. With two populations of 1000 agents (task) each.
ID: 59950 · Rating: 0 · rate: Rate + / Rate - Report as offensive     Reply Quote
Richard Haselgrove

Send message
Joined: 11 Jul 09
Posts: 1639
Credit: 10,159,968,649
RAC: 428
Level
Trp
Scientific publications
watwatwatwatwatwatwatwatwatwatwatwatwatwatwatwatwatwatwatwatwatwatwatwatwatwatwat
Message 59951 - Posted: 17 Feb 2023, 19:11:13 UTC

And thanks from me, too. That went very smoothly, and allocation of v4.03 hasn't been disturbed. Another resend has arrived for processing when this one finishes, without manual intervention.
ID: 59951 · Rating: 0 · rate: Rate + / Rate - Report as offensive     Reply Quote
Aurum
Avatar

Send message
Joined: 12 Jul 17
Posts: 404
Credit: 17,408,899,587
RAC: 0
Level
Trp
Scientific publications
watwatwat
Message 59952 - Posted: 18 Feb 2023, 15:26:40 UTC - in response to Message 59933.  


==> WARNING: A newer version of conda exists. <==
  current version: 4.8.3
  latest version: 23.1.0

Please update conda by running

    $ conda update -n base -c defaults conda

Does anyone know if I need to install Miniconda and/or Anaconda to satisfy this error message?
E.g.: https://conda.io/projects/conda/en/latest/user-guide/install/linux.html
My Linux Mint Synaptic Package Manager can't find any program containing "conda."
Maybe this is just something for the server-side staff but then why post an error message to confuse crunchers?
ID: 59952 · Rating: 0 · rate: Rate + / Rate - Report as offensive     Reply Quote
Richard Haselgrove

Send message
Joined: 11 Jul 09
Posts: 1639
Credit: 10,159,968,649
RAC: 428
Level
Trp
Scientific publications
watwatwatwatwatwatwatwatwatwatwatwatwatwatwatwatwatwatwatwatwatwatwatwatwatwatwat
Message 59953 - Posted: 18 Feb 2023, 15:46:54 UTC - in response to Message 59952.  

Maybe this is just something for the server-side staff but then why post an error message to confuse crunchers?

It's not an error, it's simply a warning - information, if you like.

The project supply the conda package (which is why Mint doesn't know about it), and they're obviously happy with the version they're using. You don't need to do anything.
ID: 59953 · Rating: 0 · rate: Rate + / Rate - Report as offensive     Reply Quote
Ian&Steve C.

Send message
Joined: 21 Feb 20
Posts: 1116
Credit: 40,839,470,595
RAC: 6,423
Level
Trp
Scientific publications
wat
Message 59954 - Posted: 18 Feb 2023, 15:53:02 UTC - in response to Message 59952.  
Last modified: 18 Feb 2023, 15:57:14 UTC


==> WARNING: A newer version of conda exists. <==
  current version: 4.8.3
  latest version: 23.1.0

Please update conda by running

    $ conda update -n base -c defaults conda

Does anyone know if I need to install Miniconda and/or Anaconda to satisfy this error message?
E.g.: https://conda.io/projects/conda/en/latest/user-guide/install/linux.html
My Linux Mint Synaptic Package Manager can't find any program containing "conda."
Maybe this is just something for the server-side staff but then why post an error message to confuse crunchers?


even if you installed it, it wouldnt change anything and you'd get the same warning message. as Richard wrote, these tasks use its own environment. they do not use your locally installed conda at all. which is why they work on systems that do not have conda installed at all. this is all by design to avoid any version conflicts or dependencies on the local system. it has been this way from the beginning.

additionally, this message was only present when trying to run the old/incompatible 4.01 app. you do not get that message from the correct 4.03 app. 4.01 was re-published by accident and is an app version about 1.5 years old. it is not compatible with the design/structure/requirements of how these tasks function today. the project admins have removed this version so you wont see this problem again.
ID: 59954 · Rating: 0 · rate: Rate + / Rate - Report as offensive     Reply Quote
Erich56

Send message
Joined: 1 Jan 15
Posts: 1166
Credit: 12,260,898,501
RAC: 1
Level
Trp
Scientific publications
watwatwatwatwatwatwatwatwat
Message 59955 - Posted: 19 Feb 2023, 6:59:35 UTC

what catches my eye:

the Pythons which I got downloaded within the past 2 days seem to use a lot less system memory than the ones before.
Has Abouh made any changes to this effect?
ID: 59955 · Rating: 0 · rate: Rate + / Rate - Report as offensive     Reply Quote
KAMasud

Send message
Joined: 27 Jul 11
Posts: 138
Credit: 539,953,398
RAC: 0
Level
Lys
Scientific publications
watwat
Message 59956 - Posted: 19 Feb 2023, 17:49:40 UTC

❸ Name Abouh Meaning
Exceptional qualities that make this name special are negotiation skills and an amazing sense of tact. When developed these two assets will help you get what you want and achieve all goals.

Cooperation is a key aspect of your life!

Since you are far less successful in life if you do not find a level of unity with others. Most problems that people find too tricky to solve are often no match against your ingenuity.

Harmony in your surrounding is a key to happiness and feelings of relaxation. Having friends or family fight affects you greatly in a negative sense. That is why you have the reputation of being a peacemaker(only out of necessity).

At your best you become very kind hearted, charming and full of positive energy. An amazing person to spend time with.
ID: 59956 · Rating: 0 · rate: Rate + / Rate - Report as offensive     Reply Quote
feri

Send message
Joined: 31 Mar 20
Posts: 2
Credit: 139,952,008
RAC: 0
Level
Cys
Scientific publications
wat
Message 59957 - Posted: 19 Feb 2023, 19:56:47 UTC

hi all,
...i used to contribute to GPUgrid with 1 gtx1080ti since 2020, 4core nonHT cpu
since the mostly GPU based acemd tasks i see a lot has changed regarding the effective HW requirements,
so i m wondering what is a somewhat optimal HW setup with the current python tasks, or what is potentialy the biggest current bottleneck?
-CPU core/thread #?
-RAM size?
-RAM speed vs latency preference?
-SSD speed
...i concur ECC RAM is very needed for long runtimes/nonstop usage.
___________
Frank from Slovakia, EU
ID: 59957 · Rating: 0 · rate: Rate + / Rate - Report as offensive     Reply Quote
Keith Myers
Avatar

Send message
Joined: 13 Dec 17
Posts: 1419
Credit: 9,119,446,190
RAC: 891
Level
Tyr
Scientific publications
watwatwatwatwat
Message 59958 - Posted: 19 Feb 2023, 21:49:01 UTC - in response to Message 59957.  

For Python, you don't need to worry about anything other than having 8-10 cpu cores to support the task.

Enough system memory, at least 16GB.

Enough virtual swap space, at least 50GB.

Enough VRAM on the gpu, at least 4-6GB.
ID: 59958 · Rating: 0 · rate: Rate + / Rate - Report as offensive     Reply Quote
Pop Piasa
Avatar

Send message
Joined: 8 Aug 19
Posts: 252
Credit: 458,054,251
RAC: 0
Level
Gln
Scientific publications
watwat
Message 59959 - Posted: 21 Feb 2023, 1:25:10 UTC - in response to Message 59957.  

feri, If I might add to Keith Myers' excellent synopsis, the speed at which these tasks run appears more dependent upon CPU ability than GPU ability. You might want to consider that if you are thinking about assembling a host dedicated to running pythons and you maybe have an old GTX 1060 6GB or something else with sufficient VRAM (GTX1650) laying around.

ID: 59959 · Rating: 0 · rate: Rate + / Rate - Report as offensive     Reply Quote
feri

Send message
Joined: 31 Mar 20
Posts: 2
Credit: 139,952,008
RAC: 0
Level
Cys
Scientific publications
wat
Message 59962 - Posted: 22 Feb 2023, 22:12:09 UTC - in response to Message 59959.  

..a friend of mine actualy has a gtx1060 6GB laying around
thanks for the insights
ID: 59962 · Rating: 0 · rate: Rate + / Rate - Report as offensive     Reply Quote
Keith Myers
Avatar

Send message
Joined: 13 Dec 17
Posts: 1419
Credit: 9,119,446,190
RAC: 891
Level
Tyr
Scientific publications
watwatwatwatwat
Message 59963 - Posted: 22 Feb 2023, 23:35:29 UTC - in response to Message 59962.  

Look at Richard Haselgrove's results with 6GB GTX 1060's
ID: 59963 · Rating: 0 · rate: Rate + / Rate - Report as offensive     Reply Quote
Richard Haselgrove

Send message
Joined: 11 Jul 09
Posts: 1639
Credit: 10,159,968,649
RAC: 428
Level
Trp
Scientific publications
watwatwatwatwatwatwatwatwatwatwatwatwatwatwatwatwatwatwatwatwatwatwatwatwatwatwat
Message 59966 - Posted: 23 Feb 2023, 7:44:19 UTC - in response to Message 59963.  

Look at Richard Haselgrove's results with 6GB GTX 1060's

They're GTX 1660s, but 6 GB is right. They run fine on a setting of 3 CPUs + 1 GPU - a bit over 8 hours for the current jobs.
ID: 59966 · Rating: 0 · rate: Rate + / Rate - Report as offensive     Reply Quote
Relles

Send message
Joined: 1 Nov 17
Posts: 2
Credit: 29,189,111
RAC: 0
Level
Val
Scientific publications
wat
Message 59982 - Posted: 25 Feb 2023, 13:44:03 UTC
Last modified: 25 Feb 2023, 13:45:07 UTC

I've noticed that on the same computer (with dual boot), tasks finish almost twice as fast on Ubuntu compared to Windows. I've tried running tasks on Linux only a few days ago and did so on Windows before.
Has there been any recent change or do tasks just compute more efficiently on Linux?
ID: 59982 · Rating: 0 · rate: Rate + / Rate - Report as offensive     Reply Quote
Ian&Steve C.

Send message
Joined: 21 Feb 20
Posts: 1116
Credit: 40,839,470,595
RAC: 6,423
Level
Trp
Scientific publications
wat
Message 59983 - Posted: 25 Feb 2023, 15:25:47 UTC - in response to Message 59982.  

They have always been faster on Linux
ID: 59983 · Rating: 0 · rate: Rate + / Rate - Report as offensive     Reply Quote
Erich56

Send message
Joined: 1 Jan 15
Posts: 1166
Credit: 12,260,898,501
RAC: 1
Level
Trp
Scientific publications
watwatwatwatwatwatwatwatwat
Message 59984 - Posted: 25 Feb 2023, 15:29:24 UTC - in response to Message 59982.  

... tasks finish almost twice as fast on Ubuntu compared to Windows. I've tried running tasks on Linux only a few days ago and did so on Windows before.


They have always been faster on Linux


that's correct. What surprises me though is that tasks finish almost twice as fast. I don't think that this was true before, was it?

ID: 59984 · Rating: 0 · rate: Rate + / Rate - Report as offensive     Reply Quote
Relles

Send message
Joined: 1 Nov 17
Posts: 2
Credit: 29,189,111
RAC: 0
Level
Val
Scientific publications
wat
Message 59985 - Posted: 25 Feb 2023, 18:02:38 UTC - in response to Message 59984.  

Close to 10 hours are needed on Windows and almost six on Linux. I also find the difference striking, that's why I asked
ID: 59985 · Rating: 0 · rate: Rate + / Rate - Report as offensive     Reply Quote
STARBASEn
Avatar

Send message
Joined: 17 Feb 09
Posts: 91
Credit: 1,603,303,394
RAC: 0
Level
His
Scientific publications
watwatwatwatwatwatwatwatwatwat
Message 59995 - Posted: 26 Feb 2023, 16:40:50 UTC

Anyone having problems getting the ATM tasks to upload? I have 4 completed jobs on 3 machines trying to upload and have not been able to make contact for nearly a day now. Two tasks on one machine making that device unable to get any more work.
ID: 59995 · Rating: 0 · rate: Rate + / Rate - Report as offensive     Reply Quote
Greger

Send message
Joined: 6 Jan 15
Posts: 76
Credit: 25,499,534,331
RAC: 0
Level
Trp
Scientific publications
watwatwatwatwatwatwatwatwatwatwat
Message 59996 - Posted: 27 Feb 2023, 19:00:24 UTC - in response to Message 59995.  

Got several ATM stuck in upload there is now 2 days left to deadline.
ID: 59996 · Rating: 0 · rate: Rate + / Rate - Report as offensive     Reply Quote
Pop Piasa
Avatar

Send message
Joined: 8 Aug 19
Posts: 252
Credit: 458,054,251
RAC: 0
Level
Gln
Scientific publications
watwat
Message 59997 - Posted: 28 Feb 2023, 2:07:03 UTC - in response to Message 59995.  

I've been watching the ATMs on the linux hosts (since they won't run on my windoze machines) to find a siderr file of a finished WU to study the linux (while I try to learn it).
I haven't found one. Only 'in process'; most show previous failures which vary from host to host. I'd be interested to see one completed if anybody can post a link.
Thanks.
ID: 59997 · Rating: 0 · rate: Rate + / Rate - Report as offensive     Reply Quote
Previous · 1 . . . 45 · 46 · 47 · 48 · 49 · 50 · Next

Message boards : News : Experimental Python tasks (beta) - task description

©2025 Universitat Pompeu Fabra