Task 33787763

Name inputs_ace_pch_ms_gc_filt_af05_index_24248_to_24251-SFARR_PYSCF_ace_pch_ms_gc_filt_af05_v2-0-1-RND9352_0
Workunit 27678227
Created 1 Feb 2024, 14:32:36 UTC
Sent 2 Feb 2024, 9:29:05 UTC
Report deadline 7 Feb 2024, 9:29:05 UTC
Received 2 Feb 2024, 9:31:56 UTC
Server state Over
Outcome Computation error
Client state Compute error
Exit status 195 (0x000000C3) EXIT_CHILD_FAILED
Computer ID 593676
Run time 44 sec
CPU time 8 sec
Validate state Invalid
Credit 0.00
Device peak FLOPS 2,143.96 GFLOPS
Application version Quantum chemistry calculations on GPU v1.04 (cuda1121)
x86_64-pc-linux-gnu
Peak working set size 0.00 MB
Peak swap size 0.00 MB
Peak disk usage 0.00 MB

Stderr output

<core_client_version>7.9.3</core_client_version>
<![CDATA[
<message>
process exited with code 195 (0xc3, -61)</message>
<stderr_txt>
01:29:22 (16636): wrapper (7.7.26016): starting
01:29:46 (16636): wrapper (7.7.26016): starting
01:29:46 (16636): wrapper: running bin/python (bin/conda-unpack)
01:29:47 (16636): bin/python exited; CPU time 0.133434
01:29:47 (16636): wrapper: running bin/tar (xjvf input.tar.bz2)
01:29:48 (16636): bin/tar exited; CPU time 0.003273
01:29:48 (16636): wrapper: running bin/bash (run.sh)
+ echo 'Setup environment'
+ source bin/activate
++ _conda_pack_activate
++ local _CONDA_SHELL_FLAVOR
++ '[' -n x ']'
++ _CONDA_SHELL_FLAVOR=bash
++ local script_dir
++ case "$_CONDA_SHELL_FLAVOR" in
+++ dirname bin/activate
++ script_dir=bin
+++ cd bin
+++ pwd
++ local full_path_script_dir=/var/lib/boinc-client/slots/8/bin
+++ dirname /var/lib/boinc-client/slots/8/bin
++ local full_path_env=/var/lib/boinc-client/slots/8
+++ basename /var/lib/boinc-client/slots/8
++ local env_name=8
++ '[' -n '' ']'
++ export CONDA_PREFIX=/var/lib/boinc-client/slots/8
++ CONDA_PREFIX=/var/lib/boinc-client/slots/8
++ export _CONDA_PACK_OLD_PS1=
++ _CONDA_PACK_OLD_PS1=
++ PATH=/var/lib/boinc-client/slots/8/bin:/usr/local/bin:/usr/local/sbin:/usr/bin:/usr/sbin:/bin:/sbin:.
++ PS1='(8) '
++ case "$_CONDA_SHELL_FLAVOR" in
++ hash -r
++ local _script_dir=/var/lib/boinc-client/slots/8/etc/conda/activate.d
++ '[' -d /var/lib/boinc-client/slots/8/etc/conda/activate.d ']'
+ export PATH=/var/lib/boinc-client/slots/8:/var/lib/boinc-client/slots/8/bin:/usr/local/bin:/usr/local/sbin:/usr/bin:/usr/sbin:/bin:/sbin:.
+ PATH=/var/lib/boinc-client/slots/8:/var/lib/boinc-client/slots/8/bin:/usr/local/bin:/usr/local/sbin:/usr/bin:/usr/sbin:/bin:/sbin:.
+ echo 'Create a temporary directory'
+ export TMP=/var/lib/boinc-client/slots/8/tmp
+ TMP=/var/lib/boinc-client/slots/8/tmp
+ mkdir -p /var/lib/boinc-client/slots/8/tmp
+ export OMP_NUM_THREADS=1
+ OMP_NUM_THREADS=1
+ export CUDA_VISIBLE_DEVICES=0
+ CUDA_VISIBLE_DEVICES=0
+ export CUPY_CUDA_LIB_PATH=/var/lib/boinc-client/slots/8/cupy
+ CUPY_CUDA_LIB_PATH=/var/lib/boinc-client/slots/8/cupy
+ echo 'Running PySCF'
+ python compute_dft.py
/var/lib/boinc-client/slots/8/lib/python3.11/site-packages/gpu4pyscf/lib/cutensor.py:174: UserWarning: using cupy as the tensor contraction engine.
  warnings.warn(f'using {contract_engine} as the tensor contraction engine.')
/var/lib/boinc-client/slots/8/lib/python3.11/site-packages/pyscf/dft/libxc.py:771: UserWarning: Since PySCF-2.3, B3LYP (and B3P86) are changed to the VWN-RPA variant, corresponding to the original definition by Stephens et al. (issue 1480) and the same as the B3LYP functional in Gaussian. To restore the VWN5 definition, you can put the setting "B3LYP_WITH_VWN5 = True" in pyscf_conf.py
  warnings.warn('Since PySCF-2.3, B3LYP (and B3P86) are changed to the VWN-RPA variant, '
/var/lib/boinc-client/slots/8/lib/python3.11/site-packages/pyscf/gto/mole.py:1280: UserWarning: Function mol.dumps drops attribute charge because it is not JSON-serializable
  warnings.warn(msg)
Traceback (most recent call last):
  File "/var/lib/boinc-client/slots/8/lib/python3.11/site-packages/pyscf/lib/misc.py", line 1094, in __exit__
    handler.result()
  File "/var/lib/boinc-client/slots/8/lib/python3.11/concurrent/futures/_base.py", line 456, in result
    return self.__get_result()
           ^^^^^^^^^^^^^^^^^^^
  File "/var/lib/boinc-client/slots/8/lib/python3.11/concurrent/futures/_base.py", line 401, in __get_result
    raise self._exception
  File "/var/lib/boinc-client/slots/8/lib/python3.11/concurrent/futures/thread.py", line 58, in run
    result = self.fn(*self.args, **self.kwargs)
             ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/var/lib/boinc-client/slots/8/lib/python3.11/site-packages/gpu4pyscf/df/df_jk.py", line 52, in build_df
    rsh_df.build(omega=omega)
  File "/var/lib/boinc-client/slots/8/lib/python3.11/site-packages/gpu4pyscf/df/df.py", line 102, in build
    self._cderi = cholesky_eri_gpu(intopt, mol, auxmol, self.cd_low, omega=omega)
                  ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/var/lib/boinc-client/slots/8/lib/python3.11/site-packages/gpu4pyscf/df/df.py", line 257, in cholesky_eri_gpu
    if li>1: ints_slices = cart2sph(ints_slices, axis=2, ang=li)
                           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/var/lib/boinc-client/slots/8/lib/python3.11/site-packages/gpu4pyscf/lib/cupy_helper.py", line 333, in cart2sph
    t_sph = contract('min,ip->mpn', t_cart, c2s, out=out)
            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/var/lib/boinc-client/slots/8/lib/python3.11/site-packages/gpu4pyscf/lib/cutensor.py", line 177, in contract
    return cupy.asarray(einsum(pattern, a, b), order='C')
                        ^^^^^^^^^^^^^^^^^^^^^
  File "/var/lib/boinc-client/slots/8/lib/python3.11/site-packages/cupy/linalg/_einsum.py", line 676, in einsum
    arr_out, sub_out = reduced_binary_einsum(
                       ^^^^^^^^^^^^^^^^^^^^^^
  File "/var/lib/boinc-client/slots/8/lib/python3.11/site-packages/cupy/linalg/_einsum.py", line 421, in reduced_binary_einsum
    arr_out = cupy.matmul(tmp0, tmp1).reshape(shapes_out)
              ^^^^^^^^^^^^^^^^^^^^^^^
  File "/var/lib/boinc-client/slots/8/lib/python3.11/site-packages/cupy/_core/_gufuncs.py", line 683, in __call__
    self._apply_func_to_inputs(
  File "/var/lib/boinc-client/slots/8/lib/python3.11/site-packages/cupy/_core/_gufuncs.py", line 397, in _apply_func_to_inputs
    func(*args, out=outs)
  File "cupy/_core/_routines_linalg.pyx", line 814, in cupy._core._routines_linalg.matmul
  File "cupy/_core/_routines_linalg.pyx", line 896, in cupy._core._routines_linalg.matmul
  File "cupy/_core/core.pyx", line 2704, in cupy._core.core.ascontiguousarray
  File "cupy/_core/core.pyx", line 132, in cupy._core.core.ndarray.__new__
  File "cupy/_core/core.pyx", line 220, in cupy._core.core._ndarray_base._init
  File "cupy/cuda/memory.pyx", line 740, in cupy.cuda.memory.alloc
  File "cupy/cuda/memory.pyx", line 1426, in cupy.cuda.memory.MemoryPool.malloc
  File "cupy/cuda/memory.pyx", line 1447, in cupy.cuda.memory.MemoryPool.malloc
  File "cupy/cuda/memory.pyx", line 1118, in cupy.cuda.memory.SingleDeviceMemoryPool.malloc
  File "cupy/cuda/memory.pyx", line 1139, in cupy.cuda.memory.SingleDeviceMemoryPool._malloc
  File "cupy/cuda/memory.pyx", line 1384, in cupy.cuda.memory.SingleDeviceMemoryPool._try_malloc
  File "cupy/cuda/memory.pyx", line 1387, in cupy.cuda.memory.SingleDeviceMemoryPool._try_malloc
cupy.cuda.memory.OutOfMemoryError: Out of memory allocating 321,753,600 bytes (allocated so far: 2,961,228,288 bytes, limit set to: 3,811,442,688 bytes).

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "/var/lib/boinc-client/slots/8/compute_dft.py", line 125, in <module>
    e,f,dip,q = compute_gpu(mol)
                ^^^^^^^^^^^^^^^^
  File "/var/lib/boinc-client/slots/8/compute_dft.py", line 32, in compute_gpu
    e_dft = mf.kernel()  # compute total energy
            ^^^^^^^^^^^
  File "<string>", line 2, in kernel
  File "/var/lib/boinc-client/slots/8/lib/python3.11/site-packages/gpu4pyscf/scf/hf.py", line 586, in scf
    _kernel(self, self.conv_tol, self.conv_tol_grad,
  File "/var/lib/boinc-client/slots/8/lib/python3.11/site-packages/gpu4pyscf/scf/hf.py", line 393, in _kernel
    mf.init_workflow(dm0=dm)
  File "/var/lib/boinc-client/slots/8/lib/python3.11/site-packages/gpu4pyscf/df/df_jk.py", line 56, in init_workflow
    with lib.call_in_background(build_df) as build:
  File "/var/lib/boinc-client/slots/8/lib/python3.11/site-packages/pyscf/lib/misc.py", line 1096, in __exit__
    raise ThreadRuntimeError('Error on thread %s:\n%s' % (self, e))
pyscf.lib.misc.ThreadRuntimeError: Error on thread <pyscf.lib.misc.call_in_background object at 0x7f4d7e798850>:
Out of memory allocating 321,753,600 bytes (allocated so far: 2,961,228,288 bytes, limit set to: 3,811,442,688 bytes).
01:30:05 (16636): bin/bash exited; CPU time 14.029601
01:30:05 (16636): app exit status: 0x1
01:30:05 (16636): called boinc_finish(195)

</stderr_txt>
]]>


©2025 Universitat Pompeu Fabra