Task 33797888
| Name | inputs_ace_pch_ms_gc_filt_af05_index_21228_to_21231-SFARR_PYSCF_ace_pch_ms_gc_filt_af05_v2-0-1-RND9644_6 |
| Workunit | 27677468 |
| Created | 2 Feb 2024, 5:25:24 UTC |
| Sent | 2 Feb 2024, 5:47:01 UTC |
| Report deadline | 7 Feb 2024, 5:47:01 UTC |
| Received | 2 Feb 2024, 18:26:58 UTC |
| Server state | Over |
| Outcome | Computation error |
| Client state | Compute error |
| Exit status | 195 (0x000000C3) EXIT_CHILD_FAILED |
| Computer ID | 614850 |
| Run time | 31 sec |
| CPU time | |
| Validate state | Invalid |
| Credit | 0.00 |
| Device peak FLOPS | 21,736.38 GFLOPS |
| Application version | Quantum chemistry calculations on GPU v1.04 (cuda1121) x86_64-pc-linux-gnu |
| Peak working set size | 0.00 MB |
| Peak swap size | 0.00 MB |
| Peak disk usage | 0.00 MB |
Stderr output
<core_client_version>7.20.5</core_client_version>
<![CDATA[
<message>
process exited with code 195 (0xc3, -61)</message>
<stderr_txt>
22:49:36 (78698): wrapper (7.7.26016): starting
22:49:57 (78698): wrapper (7.7.26016): starting
22:49:57 (78698): wrapper: running bin/python (bin/conda-unpack)
22:49:58 (78698): bin/python exited; CPU time 0.112121
22:49:58 (78698): wrapper: running bin/tar (xjvf input.tar.bz2)
11:25:12 (19048): wrapper (7.7.26016): starting
11:25:34 (19048): wrapper (7.7.26016): starting
11:25:34 (19048): wrapper: running bin/tar (xjvf input.tar.bz2)
11:25:35 (19048): bin/tar exited; CPU time 0.001592
11:25:35 (19048): wrapper: running bin/bash (run.sh)
+ echo 'Setup environment'
+ source bin/activate
++ _conda_pack_activate
++ local _CONDA_SHELL_FLAVOR
++ '[' -n x ']'
++ _CONDA_SHELL_FLAVOR=bash
++ local script_dir
++ case "$_CONDA_SHELL_FLAVOR" in
+++ dirname bin/activate
++ script_dir=bin
+++ cd bin
+++ pwd
++ local full_path_script_dir=/var/lib/boinc-client/slots/0/bin
+++ dirname /var/lib/boinc-client/slots/0/bin
++ local full_path_env=/var/lib/boinc-client/slots/0
+++ basename /var/lib/boinc-client/slots/0
++ local env_name=0
++ '[' -n '' ']'
++ export CONDA_PREFIX=/var/lib/boinc-client/slots/0
++ CONDA_PREFIX=/var/lib/boinc-client/slots/0
++ export _CONDA_PACK_OLD_PS1=
++ _CONDA_PACK_OLD_PS1=
++ PATH=/var/lib/boinc-client/slots/0/bin:/usr/local/bin:/usr/local/sbin:/usr/bin:/usr/sbin:/bin:/sbin:.
++ PS1='(0) '
++ case "$_CONDA_SHELL_FLAVOR" in
++ hash -r
++ local _script_dir=/var/lib/boinc-client/slots/0/etc/conda/activate.d
++ '[' -d /var/lib/boinc-client/slots/0/etc/conda/activate.d ']'
+ export PATH=/var/lib/boinc-client/slots/0:/var/lib/boinc-client/slots/0/bin:/usr/local/bin:/usr/local/sbin:/usr/bin:/usr/sbin:/bin:/sbin:.
+ PATH=/var/lib/boinc-client/slots/0:/var/lib/boinc-client/slots/0/bin:/usr/local/bin:/usr/local/sbin:/usr/bin:/usr/sbin:/bin:/sbin:.
+ echo 'Create a temporary directory'
+ export TMP=/var/lib/boinc-client/slots/0/tmp
+ TMP=/var/lib/boinc-client/slots/0/tmp
+ mkdir -p /var/lib/boinc-client/slots/0/tmp
+ export OMP_NUM_THREADS=1
+ OMP_NUM_THREADS=1
+ export CUDA_VISIBLE_DEVICES=0
+ CUDA_VISIBLE_DEVICES=0
+ export CUPY_CUDA_LIB_PATH=/var/lib/boinc-client/slots/0/cupy
+ CUPY_CUDA_LIB_PATH=/var/lib/boinc-client/slots/0/cupy
+ echo 'Running PySCF'
+ python compute_dft.py
/var/lib/boinc-client/slots/0/lib/python3.11/site-packages/gpu4pyscf/lib/cutensor.py:174: UserWarning: using cupy as the tensor contraction engine.
warnings.warn(f'using {contract_engine} as the tensor contraction engine.')
/var/lib/boinc-client/slots/0/lib/python3.11/site-packages/pyscf/dft/libxc.py:771: UserWarning: Since PySCF-2.3, B3LYP (and B3P86) are changed to the VWN-RPA variant, corresponding to the original definition by Stephens et al. (issue 1480) and the same as the B3LYP functional in Gaussian. To restore the VWN5 definition, you can put the setting "B3LYP_WITH_VWN5 = True" in pyscf_conf.py
warnings.warn('Since PySCF-2.3, B3LYP (and B3P86) are changed to the VWN-RPA variant, '
/var/lib/boinc-client/slots/0/lib/python3.11/site-packages/pyscf/gto/mole.py:1280: UserWarning: Function mol.dumps drops attribute charge because it is not JSON-serializable
warnings.warn(msg)
Traceback (most recent call last):
File "/var/lib/boinc-client/slots/0/lib/python3.11/site-packages/pyscf/lib/misc.py", line 1094, in __exit__
handler.result()
File "/var/lib/boinc-client/slots/0/lib/python3.11/concurrent/futures/_base.py", line 456, in result
return self.__get_result()
^^^^^^^^^^^^^^^^^^^
File "/var/lib/boinc-client/slots/0/lib/python3.11/concurrent/futures/_base.py", line 401, in __get_result
raise self._exception
File "/var/lib/boinc-client/slots/0/lib/python3.11/concurrent/futures/thread.py", line 58, in run
result = self.fn(*self.args, **self.kwargs)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/var/lib/boinc-client/slots/0/lib/python3.11/site-packages/gpu4pyscf/df/df_jk.py", line 43, in build_df
mf.with_df.build()
File "/var/lib/boinc-client/slots/0/lib/python3.11/site-packages/gpu4pyscf/df/df.py", line 102, in build
self._cderi = cholesky_eri_gpu(intopt, mol, auxmol, self.cd_low, omega=omega)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/var/lib/boinc-client/slots/0/lib/python3.11/site-packages/gpu4pyscf/df/df.py", line 254, in cholesky_eri_gpu
int3c2e.get_int3c2e_slice(intopt, cp_ij_id, cp_kl_id, out=ints_slices[k0:k1], omega=omega)
File "/var/lib/boinc-client/slots/0/lib/python3.11/site-packages/gpu4pyscf/df/int3c2e.py", line 1261, in get_int3c2e_slice
int3c_blk = cart2sph(int3c_blk, axis=0, ang=lk, out=out)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/var/lib/boinc-client/slots/0/lib/python3.11/site-packages/gpu4pyscf/lib/cupy_helper.py", line 333, in cart2sph
t_sph = contract('min,ip->mpn', t_cart, c2s, out=out)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/var/lib/boinc-client/slots/0/lib/python3.11/site-packages/gpu4pyscf/lib/cutensor.py", line 179, in contract
out[:] = alpha*einsum(pattern, a, b) + beta*out
^^^^^^^^^^^^^^^^^^^^^
File "/var/lib/boinc-client/slots/0/lib/python3.11/site-packages/cupy/linalg/_einsum.py", line 676, in einsum
arr_out, sub_out = reduced_binary_einsum(
^^^^^^^^^^^^^^^^^^^^^^
File "/var/lib/boinc-client/slots/0/lib/python3.11/site-packages/cupy/linalg/_einsum.py", line 418, in reduced_binary_einsum
tmp1, shapes1 = _flatten_transpose(arr1, [bs1, cs1, ts1])
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/var/lib/boinc-client/slots/0/lib/python3.11/site-packages/cupy/linalg/_einsum.py", line 298, in _flatten_transpose
a.transpose(transpose_axes).reshape(
File "cupy/_core/core.pyx", line 752, in cupy._core.core._ndarray_base.reshape
File "cupy/_core/_routines_manipulation.pyx", line 81, in cupy._core._routines_manipulation._ndarray_reshape
File "cupy/_core/_routines_manipulation.pyx", line 357, in cupy._core._routines_manipulation._reshape
File "cupy/_core/core.pyx", line 611, in cupy._core.core._ndarray_base.copy
File "cupy/_core/core.pyx", line 570, in cupy._core.core._ndarray_base.astype
File "cupy/_core/core.pyx", line 132, in cupy._core.core.ndarray.__new__
File "cupy/_core/core.pyx", line 220, in cupy._core.core._ndarray_base._init
File "cupy/cuda/memory.pyx", line 740, in cupy.cuda.memory.alloc
File "cupy/cuda/memory.pyx", line 1426, in cupy.cuda.memory.MemoryPool.malloc
File "cupy/cuda/memory.pyx", line 1447, in cupy.cuda.memory.MemoryPool.malloc
File "cupy/cuda/memory.pyx", line 1118, in cupy.cuda.memory.SingleDeviceMemoryPool.malloc
File "cupy/cuda/memory.pyx", line 1139, in cupy.cuda.memory.SingleDeviceMemoryPool._malloc
File "cupy/cuda/memory.pyx", line 1384, in cupy.cuda.memory.SingleDeviceMemoryPool._try_malloc
File "cupy/cuda/memory.pyx", line 1387, in cupy.cuda.memory.SingleDeviceMemoryPool._try_malloc
cupy.cuda.memory.OutOfMemoryError: Out of memory allocating 61,663,744 bytes (allocated so far: 337,081,856 bytes, limit set to: 7,503,976,857 bytes).
During handling of the above exception, another exception occurred:
Traceback (most recent call last):
File "/var/lib/boinc-client/slots/0/compute_dft.py", line 125, in <module>
e,f,dip,q = compute_gpu(mol)
^^^^^^^^^^^^^^^^
File "/var/lib/boinc-client/slots/0/compute_dft.py", line 32, in compute_gpu
e_dft = mf.kernel() # compute total energy
^^^^^^^^^^^
File "<string>", line 2, in kernel
File "/var/lib/boinc-client/slots/0/lib/python3.11/site-packages/gpu4pyscf/scf/hf.py", line 586, in scf
_kernel(self, self.conv_tol, self.conv_tol_grad,
File "/var/lib/boinc-client/slots/0/lib/python3.11/site-packages/gpu4pyscf/scf/hf.py", line 393, in _kernel
mf.init_workflow(dm0=dm)
File "/var/lib/boinc-client/slots/0/lib/python3.11/site-packages/gpu4pyscf/df/df_jk.py", line 56, in init_workflow
with lib.call_in_background(build_df) as build:
File "/var/lib/boinc-client/slots/0/lib/python3.11/site-packages/pyscf/lib/misc.py", line 1096, in __exit__
raise ThreadRuntimeError('Error on thread %s:\n%s' % (self, e))
pyscf.lib.misc.ThreadRuntimeError: Error on thread <pyscf.lib.misc.call_in_background object at 0x7f9ee05604d0>:
Out of memory allocating 61,663,744 bytes (allocated so far: 337,081,856 bytes, limit set to: 7,503,976,857 bytes).
11:25:41 (19048): bin/bash exited; CPU time 4.328599
11:25:41 (19048): app exit status: 0x1
11:25:41 (19048): called boinc_finish(195)
</stderr_txt>
]]>
©2025 Universitat Pompeu Fabra