Task 38577260

Name wu_7600f5f8-GIANNI_GPROTO7-0-1-RND9691_0
Workunit 31542813
Created 24 Sep 2025, 20:22:45 UTC
Sent 24 Sep 2025, 20:22:51 UTC
Report deadline 29 Sep 2025, 20:22:51 UTC
Received 24 Sep 2025, 22:30:37 UTC
Server state Over
Outcome Computation error
Client state Compute error
Exit status 195 (0x000000C3) EXIT_CHILD_FAILED
Computer ID 644258
Run time 3 min 43 sec
CPU time 2 min 7 sec
Validate state Invalid
Credit 0.00
Device peak FLOPS 19,492.84 GFLOPS
Application version LLM: LLMs for chemistry v1.00 (cuda124L)
x86_64-pc-linux-gnu
Peak working set size 2.18 GB
Peak swap size 32.81 GB
Peak disk usage 8.40 GB

Stderr output

<core_client_version>8.3.0</core_client_version>
<![CDATA[
<message>
process exited with code 195 (0xc3, -61)</message>
<stderr_txt>
2025-09-24 22:26:26 (2129): wrapper (8.1.26018): starting
2025-09-24 22:26:49 (2129): wrapper: running bin/python (bin/conda-unpack)
2025-09-24 22:26:49 (2129): wrapper: created child process 2131
2025-09-24 22:26:50 (2129): bin/python exited; CPU time 0.299929
2025-09-24 22:26:50 (2129): wrapper: running bin/tar (xjvf input.tar.bz2)
2025-09-24 22:26:50 (2129): wrapper: created child process 2137
2025-09-24 22:26:51 (2129): bin/tar exited; CPU time 0.018005
2025-09-24 22:26:51 (2129): wrapper: running bin/bash (run.sh)
2025-09-24 22:26:51 (2129): wrapper: created child process 2142
+ echo 'Setup environment'
+ source bin/activate
++ _conda_pack_activate
++ local _CONDA_SHELL_FLAVOR
++ '[' -n x ']'
++ _CONDA_SHELL_FLAVOR=bash
++ local script_dir
++ case "$_CONDA_SHELL_FLAVOR" in
+++ dirname bin/activate
++ script_dir=bin
+++ cd bin
+++ pwd
++ local full_path_script_dir=/workspace/BOINC/slots/3/bin
+++ dirname /workspace/BOINC/slots/3/bin
++ local full_path_env=/workspace/BOINC/slots/3
+++ basename /workspace/BOINC/slots/3
++ local env_name=3
++ '[' -n '' ']'
++ export CONDA_PREFIX=/workspace/BOINC/slots/3
++ CONDA_PREFIX=/workspace/BOINC/slots/3
++ export _CONDA_PACK_OLD_PS1=
++ _CONDA_PACK_OLD_PS1=
++ PATH=/workspace/BOINC/slots/3/bin:/usr/local/bin:/usr/local/sbin:/usr/bin:/usr/sbin:/bin:/sbin:.
++ PS1='(3) '
++ case "$_CONDA_SHELL_FLAVOR" in
++ hash -r
++ local _script_dir=/workspace/BOINC/slots/3/etc/conda/activate.d
++ '[' -d /workspace/BOINC/slots/3/etc/conda/activate.d ']'
+ export PATH=/workspace/BOINC/slots/3:/workspace/BOINC/slots/3/bin:/usr/local/bin:/usr/local/sbin:/usr/bin:/usr/sbin:/bin:/sbin:.
+ PATH=/workspace/BOINC/slots/3:/workspace/BOINC/slots/3/bin:/usr/local/bin:/usr/local/sbin:/usr/bin:/usr/sbin:/bin:/sbin:.
+ echo 'Create a temporary directory'
+ export TMP=/workspace/BOINC/slots/3/tmp
+ TMP=/workspace/BOINC/slots/3/tmp
+ mkdir -p /workspace/BOINC/slots/3/tmp
+ which python
+ pip install main_generation-0.1.0-py3-none-any.whl -v --no-deps
WARNING: Running pip as the 'root' user can result in broken permissions and conflicting behaviour with the system package manager, possibly rendering your system unusable. It is recommended to use a virtual environment instead: https://pip.pypa.io/warnings/venv. Use the --root-user-action option if you know what you are doing and want to suppress this warning.
+ export CUDA_VISIBLE_DEVICES=0
+ CUDA_VISIBLE_DEVICES=0
+ export HF_HOME=../.cache
+ HF_HOME=../.cache
+ export VLLM_ASSETS_CACHE=../.cache
+ VLLM_ASSETS_CACHE=../.cache
+ export VLLM_CACHE_ROOT=../.cache
+ VLLM_CACHE_ROOT=../.cache
+ echo RUNNING
+ pythonbinary=/workspace/BOINC/slots/3/lib/python3.12/site-packages/aiengine/main_generation.pyc
+ python /workspace/BOINC/slots/3/lib/python3.12/site-packages/aiengine/main_generation.pyc --conf conf.yaml

Generating train split: 0 examples [00:00, ? examples/s]
Generating train split: 2500 examples [00:00, 1037885.78 examples/s]

Loading safetensors checkpoint shards:   0% Completed | 0/2 [00:00<?, ?it/s]

Loading safetensors checkpoint shards:  50% Completed | 1/2 [00:00<00:00,  2.07it/s]

Loading safetensors checkpoint shards: 100% Completed | 2/2 [00:00<00:00,  2.14it/s]

Loading safetensors checkpoint shards: 100% Completed | 2/2 [00:00<00:00,  2.13it/s]


Loading safetensors checkpoint shards:   0% Completed | 0/2 [00:00<?, ?it/s]

Loading safetensors checkpoint shards:  50% Completed | 1/2 [00:00<00:00,  1.79it/s]

Loading safetensors checkpoint shards: 100% Completed | 2/2 [00:01<00:00,  1.82it/s]

Loading safetensors checkpoint shards: 100% Completed | 2/2 [00:01<00:00,  1.82it/s]

run.sh: line 26:  2154 Killed                  python ${pythonbinary} --conf conf.yaml
2025-09-24 22:30:08 (2129): bin/bash exited; CPU time 17.429030
2025-09-24 22:30:08 (2129): app exit status: 0x89
2025-09-24 22:30:08 (2129): called boinc_finish(195)

</stderr_txt>
]]>


©2025 Universitat Pompeu Fabra