Name | wu_fdedaf99-GIANNI_GPROTO7-0-1-RND8054_0 |
Workunit | 31542528 |
Created | 24 Sep 2025, 12:58:43 UTC |
Sent | 24 Sep 2025, 12:59:47 UTC |
Report deadline | 29 Sep 2025, 12:59:47 UTC |
Received | 24 Sep 2025, 13:06:38 UTC |
Server state | Over |
Outcome | Computation error |
Client state | Compute error |
Exit status | 195 (0x000000C3) EXIT_CHILD_FAILED |
Computer ID | 624436 |
Run time | 2 min 55 sec |
CPU time | 41 sec |
Validate state | Invalid |
Credit | 0.00 |
Device peak FLOPS | 37,475.62 GFLOPS |
Application version | LLM: LLMs for chemistry v1.01 (cuda124L) windows_x86_64 |
Peak working set size | 4.89 GB |
Peak swap size | 19.97 GB |
Peak disk usage | 11.47 GB |
<core_client_version>8.0.4</core_client_version> <![CDATA[ <message> (unknown error) (0) - exit code 195 (0xc3)</message> <stderr_txt> 15:03:45 (18368): wrapper (7.9.26016): starting 15:03:45 (18368): wrapper: running Library/usr/bin/tar.exe (xjvf input.tar.bz2) tasks.json run.bat conf.yaml main_generation-0.1.0-py3-none-any.whl run.sh 15:03:46 (18368): Library/usr/bin/tar.exe exited; CPU time 0.000000 15:03:46 (18368): wrapper: running C:/Windows/system32/cmd.exe (/c call Scripts\activate.bat && Scripts\conda-unpack.exe && run.bat) Generating train split: 0 examples [00:00, ? examples/s] Generating train split: 2500 examples [00:00, 248742.97 examples/s] [W924 15:04:29.000000000 socket.cpp:759] [c10d] The client socket has failed to connect to [Avalon.v.cablecom.net]:50950 (system error: 10049 - Die angeforderte Adresse ist in diesem Kontext ungültig.). Loading safetensors checkpoint shards: 0% Completed | 0/2 [00:00<?, ?it/s] Loading safetensors checkpoint shards: 50% Completed | 1/2 [00:02<00:02, 2.21s/it] Loading safetensors checkpoint shards: 100% Completed | 2/2 [00:04<00:00, 2.19s/it] Loading safetensors checkpoint shards: 100% Completed | 2/2 [00:04<00:00, 2.19s/it] Loading safetensors checkpoint shards: 0% Completed | 0/2 [00:00<?, ?it/s] Loading safetensors checkpoint shards: 50% Completed | 1/2 [00:02<00:02, 2.86s/it] Loading safetensors checkpoint shards: 100% Completed | 2/2 [00:06<00:00, 3.04s/it] Loading safetensors checkpoint shards: 100% Completed | 2/2 [00:06<00:00, 3.01s/it] C:\actions-runner\_work\pytorch\pytorch\pytorch\aten\src\ATen\native\cuda\ScatterGatherKernel.cu:145: block: [807,0,0], thread: [5,0,0] Assertion `idx_dim >= 0 && idx_dim < index_size && "index out of bounds"` failed. C:\actions-runner\_work\pytorch\pytorch\pytorch\aten\src\ATen\native\cuda\ScatterGatherKernel.cu:145: block: [527,0,0], thread: [89,0,0] Assertion `idx_dim >= 0 && idx_dim < index_size && "index out of bounds"` failed. C:\actions-runner\_work\pytorch\pytorch\pytorch\aten\src\ATen\native\cuda\ScatterGatherKernel.cu:145: block: [2017,0,0], thread: [34,0,0] Assertion `idx_dim >= 0 && idx_dim < index_size && "index out of bounds"` failed. C:\actions-runner\_work\pytorch\pytorch\pytorch\aten\src\ATen\native\cuda\ScatterGatherKernel.cu:145: block: [581,0,0], thread: [74,0,0] Assertion `idx_dim >= 0 && idx_dim < index_size && "index out of bounds"` failed. C:\actions-runner\_work\pytorch\pytorch\pytorch\aten\src\ATen\native\cuda\ScatterGatherKernel.cu:145: block: [2200,0,0], thread: [85,0,0] Assertion `idx_dim >= 0 && idx_dim < index_size && "index out of bounds"` failed. C:\actions-runner\_work\pytorch\pytorch\pytorch\aten\src\ATen\native\cuda\ScatterGatherKernel.cu:145: block: [1819,0,0], thread: [99,0,0] Assertion `idx_dim >= 0 && idx_dim < index_size && "index out of bounds"` failed. C:\actions-runner\_work\pytorch\pytorch\pytorch\aten\src\ATen\native\cuda\ScatterGatherKernel.cu:145: block: [1303,0,0], thread: [116,0,0] Assertion `idx_dim >= 0 && idx_dim < index_size && "index out of bounds"` failed. C:\actions-runner\_work\pytorch\pytorch\pytorch\aten\src\ATen\native\cuda\ScatterGatherKernel.cu:145: block: [2811,0,0], thread: [73,0,0] Assertion `idx_dim >= 0 && idx_dim < index_size && "index out of bounds"` failed. C:\actions-runner\_work\pytorch\pytorch\pytorch\aten\src\ATen\native\cuda\ScatterGatherKernel.cu:145: block: [2812,0,0], thread: [105,0,0] Assertion `idx_dim >= 0 && idx_dim < index_size && "index out of bounds"` failed. C:\actions-runner\_work\pytorch\pytorch\pytorch\aten\src\ATen\native\cuda\ScatterGatherKernel.cu:145: block: [946,0,0], thread: [123,0,0] Assertion `idx_dim >= 0 && idx_dim < index_size && "index out of bounds"` failed. C:\actions-runner\_work\pytorch\pytorch\pytorch\aten\src\ATen\native\cuda\ScatterGatherKernel.cu:145: block: [683,0,0], thread: [127,0,0] Assertion `idx_dim >= 0 && idx_dim < index_size && "index out of bounds"` failed. C:\actions-runner\_work\pytorch\pytorch\pytorch\aten\src\ATen\native\cuda\ScatterGatherKernel.cu:145: block: [280,0,0], thread: [5,0,0] Assertion `idx_dim >= 0 && idx_dim < index_size && "index out of bounds"` failed. C:\actions-runner\_work\pytorch\pytorch\pytorch\aten\src\ATen\native\cuda\ScatterGatherKernel.cu:145: block: [987,0,0], thread: [84,0,0] Assertion `idx_dim >= 0 && idx_dim < index_size && "index out of bounds"` failed. C:\actions-runner\_work\pytorch\pytorch\pytorch\aten\src\ATen\native\cuda\ScatterGatherKernel.cu:145: block: [2880,0,0], thread: [56,0,0] Assertion `idx_dim >= 0 && idx_dim < index_size && "index out of bounds"` failed. C:\actions-runner\_work\pytorch\pytorch\pytorch\aten\src\ATen\native\cuda\ScatterGatherKernel.cu:145: block: [5856,0,0], thread: [121,0,0] Assertion `idx_dim >= 0 && idx_dim < index_size && "index out of bounds"` failed. C:\actions-runner\_work\pytorch\pytorch\pytorch\aten\src\ATen\native\cuda\ScatterGatherKernel.cu:145: block: [2342,0,0], thread: [68,0,0] Assertion `idx_dim >= 0 && idx_dim < index_size && "index out of bounds"` failed. C:\actions-runner\_work\pytorch\pytorch\pytorch\aten\src\ATen\native\cuda\ScatterGatherKernel.cu:145: block: [2046,0,0], thread: [81,0,0] Assertion `idx_dim >= 0 && idx_dim < index_size && "index out of bounds"` failed. C:\actions-runner\_work\pytorch\pytorch\pytorch\aten\src\ATen\native\cuda\ScatterGatherKernel.cu:145: block: [4453,0,0], thread: [43,0,0] Assertion `idx_dim >= 0 && idx_dim < index_size && "index out of bounds"` failed. C:\actions-runner\_work\pytorch\pytorch\pytorch\aten\src\ATen\native\cuda\ScatterGatherKernel.cu:145: block: [7019,0,0], thread: [59,0,0] Assertion `idx_dim >= 0 && idx_dim < index_size && "index out of bounds"` failed. C:\actions-runner\_work\pytorch\pytorch\pytorch\aten\src\ATen\native\cuda\ScatterGatherKernel.cu:145: block: [7783,0,0], thread: [112,0,0] Assertion `idx_dim >= 0 && idx_dim < index_size && "index out of bounds"` failed. C:\actions-runner\_work\pytorch\pytorch\pytorch\aten\src\ATen\native\cuda\ScatterGatherKernel.cu:145: block: [9954,0,0], thread: [9,0,0] Assertion `idx_dim >= 0 && idx_dim < index_size && "index out of bounds"` failed. C:\actions-runner\_work\pytorch\pytorch\pytorch\aten\src\ATen\native\cuda\ScatterGatherKernel.cu:145: block: [7544,0,0], thread: [113,0,0] Assertion `idx_dim >= 0 && idx_dim < index_size && "index out of bounds"` failed. C:\actions-runner\_work\pytorch\pytorch\pytorch\aten\src\ATen\native\cuda\ScatterGatherKernel.cu:145: block: [5954,0,0], thread: [26,0,0] Assertion `idx_dim >= 0 && idx_dim < index_size && "index out of bounds"` failed. C:\actions-runner\_work\pytorch\pytorch\pytorch\aten\src\ATen\native\cuda\ScatterGatherKernel.cu:145: block: [6802,0,0], thread: [58,0,0] Assertion `idx_dim >= 0 && idx_dim < index_size && "index out of bounds"` failed. C:\actions-runner\_work\pytorch\pytorch\pytorch\aten\src\ATen\native\cuda\ScatterGatherKernel.cu:145: block: [6889,0,0], thread: [48,0,0] Assertion `idx_dim >= 0 && idx_dim < index_size && "index out of bounds"` failed. C:\actions-runner\_work\pytorch\pytorch\pytorch\aten\src\ATen\native\cuda\ScatterGatherKernel.cu:145: block: [7558,0,0], thread: [88,0,0] Assertion `idx_dim >= 0 && idx_dim < index_size && "index out of bounds"` failed. C:\actions-runner\_work\pytorch\pytorch\pytorch\aten\src\ATen\native\cuda\ScatterGatherKernel.cu:145: block: [10001,0,0], thread: [14,0,0] Assertion `idx_dim >= 0 && idx_dim < index_size && "index out of bounds"` failed. C:\actions-runner\_work\pytorch\pytorch\pytorch\aten\src\ATen\native\cuda\ScatterGatherKernel.cu:145: block: [14054,0,0], thread: [99,0,0] Assertion `idx_dim >= 0 && idx_dim < index_size && "index out of bounds"` failed. C:\actions-runner\_work\pytorch\pytorch\pytorch\aten\src\ATen\native\cuda\ScatterGatherKernel.cu:145: block: [42577,0,0], thread: [125,0,0] Assertion `idx_dim >= 0 && idx_dim < index_size && "index out of bounds"` failed. C:\actions-runner\_work\pytorch\pytorch\pytorch\aten\src\ATen\native\cuda\ScatterGatherKernel.cu:145: block: [42711,0,0], thread: [94,0,0] Assertion `idx_dim >= 0 && idx_dim < index_size && "index out of bounds"` failed. C:\actions-runner\_work\pytorch\pytorch\pytorch\aten\src\ATen\native\cuda\ScatterGatherKernel.cu:145: block: [42767,0,0], thread: [79,0,0] Assertion `idx_dim >= 0 && idx_dim < index_size && "index out of bounds"` failed. C:\actions-runner\_work\pytorch\pytorch\pytorch\aten\src\ATen\native\cuda\ScatterGatherKernel.cu:145: block: [42805,0,0], thread: [94,0,0] Assertion `idx_dim >= 0 && idx_dim < index_size && "index out of bounds"` failed. C:\actions-runner\_work\pytorch\pytorch\pytorch\aten\src\ATen\native\cuda\ScatterGatherKernel.cu:145: block: [42829,0,0], thread: [44,0,0] Assertion `idx_dim >= 0 && idx_dim < index_size && "index out of bounds"` failed. C:\actions-runner\_work\pytorch\pytorch\pytorch\aten\src\ATen\native\cuda\ScatterGatherKernel.cu:145: block: [42933,0,0], thread: [69,0,0] Assertion `idx_dim >= 0 && idx_dim < index_size && "index out of bounds"` failed. C:\actions-runner\_work\pytorch\pytorch\pytorch\aten\src\ATen\native\cuda\ScatterGatherKernel.cu:145: block: [43545,0,0], thread: [82,0,0] Assertion `idx_dim >= 0 && idx_dim < index_size && "index out of bounds"` failed. C:\actions-runner\_work\pytorch\pytorch\pytorch\aten\src\ATen\native\cuda\ScatterGatherKernel.cu:145: block: [43566,0,0], thread: [5,0,0] Assertion `idx_dim >= 0 && idx_dim < index_size && "index out of bounds"` failed. C:\actions-runner\_work\pytorch\pytorch\pytorch\aten\src\ATen\native\cuda\ScatterGatherKernel.cu:145: block: [43613,0,0], thread: [36,0,0] Assertion `idx_dim >= 0 && idx_dim < index_size && "index out of bounds"` failed. C:\actions-runner\_work\pytorch\pytorch\pytorch\aten\src\ATen\native\cuda\ScatterGatherKernel.cu:145: block: [43785,0,0], thread: [12,0,0] Assertion `idx_dim >= 0 && idx_dim < index_size && "index out of bounds"` failed. C:\actions-runner\_work\pytorch\pytorch\pytorch\aten\src\ATen\native\cuda\ScatterGatherKernel.cu:145: block: [43819,0,0], thread: [4,0,0] Assertion `idx_dim >= 0 && idx_dim < index_size && "index out of bounds"` failed. C:\actions-runner\_work\pytorch\pytorch\pytorch\aten\src\ATen\native\cuda\ScatterGatherKernel.cu:145: block: [43861,0,0], thread: [65,0,0] Assertion `idx_dim >= 0 && idx_dim < index_size && "index out of bounds"` failed. C:\actions-runner\_work\pytorch\pytorch\pytorch\aten\src\ATen\native\cuda\ScatterGatherKernel.cu:145: block: [44044,0,0], thread: [120,0,0] Assertion `idx_dim >= 0 && idx_dim < index_size && "index out of bounds"` failed. C:\actions-runner\_work\pytorch\pytorch\pytorch\aten\src\ATen\native\cuda\ScatterGatherKernel.cu:145: block: [44130,0,0], thread: [89,0,0] Assertion `idx_dim >= 0 && idx_dim < index_size && "index out of bounds"` failed. C:\actions-runner\_work\pytorch\pytorch\pytorch\aten\src\ATen\native\cuda\ScatterGatherKernel.cu:145: block: [44241,0,0], thread: [49,0,0] Assertion `idx_dim >= 0 && idx_dim < index_size && "index out of bounds"` failed. C:\actions-runner\_work\pytorch\pytorch\pytorch\aten\src\ATen\native\cuda\ScatterGatherKernel.cu:145: block: [44349,0,0], thread: [119,0,0] Assertion `idx_dim >= 0 && idx_dim < index_size && "index out of bounds"` failed. C:\actions-runner\_work\pytorch\pytorch\pytorch\aten\src\ATen\native\cuda\ScatterGatherKernel.cu:145: block: [44363,0,0], thread: [101,0,0] Assertion `idx_dim >= 0 && idx_dim < index_size && "index out of bounds"` failed. C:\actions-runner\_work\pytorch\pytorch\pytorch\aten\src\ATen\native\cuda\ScatterGatherKernel.cu:145: block: [44388,0,0], thread: [47,0,0] Assertion `idx_dim >= 0 && idx_dim < index_size && "index out of bounds"` failed. C:\actions-runner\_work\pytorch\pytorch\pytorch\aten\src\ATen\native\cuda\ScatterGatherKernel.cu:145: block: [44640,0,0], thread: [78,0,0] Assertion `idx_dim >= 0 && idx_dim < index_size && "index out of bounds"` failed. C:\actions-runner\_work\pytorch\pytorch\pytorch\aten\src\ATen\native\cuda\ScatterGatherKernel.cu:145: block: [44698,0,0], thread: [17,0,0] Assertion `idx_dim >= 0 && idx_dim < index_size && "index out of bounds"` failed. C:\actions-runner\_work\pytorch\pytorch\pytorch\aten\src\ATen\native\cuda\ScatterGatherKernel.cu:145: block: [44776,0,0], thread: [93,0,0] Assertion `idx_dim >= 0 && idx_dim < index_size && "index out of bounds"` failed. C:\actions-runner\_work\pytorch\pytorch\pytorch\aten\src\ATen\native\cuda\ScatterGatherKernel.cu:145: block: [45171,0,0], thread: [6,0,0] Assertion `idx_dim >= 0 && idx_dim < index_size && "index out of bounds"` failed. C:\actions-runner\_work\pytorch\pytorch\pytorch\aten\src\ATen\native\cuda\ScatterGatherKernel.cu:145: block: [45419,0,0], thread: [54,0,0] Assertion `idx_dim >= 0 && idx_dim < index_size && "index out of bounds"` failed. C:\actions-runner\_work\pytorch\pytorch\pytorch\aten\src\ATen\native\cuda\ScatterGatherKernel.cu:145: block: [45395,0,0], thread: [41,0,0] Assertion `idx_dim >= 0 && idx_dim < index_size && "index out of bounds"` failed. C:\actions-runner\_work\pytorch\pytorch\pytorch\aten\src\ATen\native\cuda\ScatterGatherKernel.cu:145: block: [45448,0,0], thread: [89,0,0] Assertion `idx_dim >= 0 && idx_dim < index_size && "index out of bounds"` failed. C:\actions-runner\_work\pytorch\pytorch\pytorch\aten\src\ATen\native\cuda\ScatterGatherKernel.cu:145: block: [45534,0,0], thread: [3,0,0] Assertion `idx_dim >= 0 && idx_dim < index_size && "index out of bounds"` failed. C:\actions-runner\_work\pytorch\pytorch\pytorch\aten\src\ATen\native\cuda\ScatterGatherKernel.cu:145: block: [45587,0,0], thread: [41,0,0] Assertion `idx_dim >= 0 && idx_dim < index_size && "index out of bounds"` failed. C:\actions-runner\_work\pytorch\pytorch\pytorch\aten\src\ATen\native\cuda\ScatterGatherKernel.cu:145: block: [46098,0,0], thread: [35,0,0] Assertion `idx_dim >= 0 && idx_dim < index_size && "index out of bounds"` failed. C:\actions-runner\_work\pytorch\pytorch\pytorch\aten\src\ATen\native\cuda\ScatterGatherKernel.cu:145: block: [46109,0,0], thread: [64,0,0] Assertion `idx_dim >= 0 && idx_dim < index_size && "index out of bounds"` failed. C:\actions-runner\_work\pytorch\pytorch\pytorch\aten\src\ATen\native\cuda\ScatterGatherKernel.cu:145: block: [46238,0,0], thread: [36,0,0] Assertion `idx_dim >= 0 && idx_dim < index_size && "index out of bounds"` failed. C:\actions-runner\_work\pytorch\pytorch\pytorch\aten\src\ATen\native\cuda\ScatterGatherKernel.cu:145: block: [46502,0,0], thread: [109,0,0] Assertion `idx_dim >= 0 && idx_dim < index_size && "index out of bounds"` failed. C:\actions-runner\_work\pytorch\pytorch\pytorch\aten\src\ATen\native\cuda\ScatterGatherKernel.cu:145: block: [46556,0,0], thread: [126,0,0] Assertion `idx_dim >= 0 && idx_dim < index_size && "index out of bounds"` failed. C:\actions-runner\_work\pytorch\pytorch\pytorch\aten\src\ATen\native\cuda\ScatterGatherKernel.cu:145: block: [46628,0,0], thread: [78,0,0] Assertion `idx_dim >= 0 && idx_dim < index_size && "index out of bounds"` failed. C:\actions-runner\_work\pytorch\pytorch\pytorch\aten\src\ATen\native\cuda\ScatterGatherKernel.cu:145: block: [46691,0,0], thread: [4,0,0] Assertion `idx_dim >= 0 && idx_dim < index_size && "index out of bounds"` failed. C:\actions-runner\_work\pytorch\pytorch\pytorch\aten\src\ATen\native\cuda\ScatterGatherKernel.cu:145: block: [46791,0,0], thread: [107,0,0] Assertion `idx_dim >= 0 && idx_dim < index_size && "index out of bounds"` failed. C:\actions-runner\_work\pytorch\pytorch\pytorch\aten\src\ATen\native\cuda\ScatterGatherKernel.cu:145: block: [46854,0,0], thread: [22,0,0] Assertion `idx_dim >= 0 && idx_dim < index_size && "index out of bounds"` failed. C:\actions-runner\_work\pytorch\pytorch\pytorch\aten\src\ATen\native\cuda\ScatterGatherKernel.cu:145: block: [46999,0,0], thread: [18,0,0] Assertion `idx_dim >= 0 && idx_dim < index_size && "index out of bounds"` failed. C:\actions-runner\_work\pytorch\pytorch\pytorch\aten\src\ATen\native\cuda\ScatterGatherKernel.cu:145: block: [47014,0,0], thread: [36,0,0] Assertion `idx_dim >= 0 && idx_dim < index_size && "index out of bounds"` failed. C:\actions-runner\_work\pytorch\pytorch\pytorch\aten\src\ATen\native\cuda\ScatterGatherKernel.cu:145: block: [47150,0,0], thread: [109,0,0] Assertion `idx_dim >= 0 && idx_dim < index_size && "index out of bounds"` failed. C:\actions-runner\_work\pytorch\pytorch\pytorch\aten\src\ATen\native\cuda\ScatterGatherKernel.cu:145: block: [47546,0,0], thread: [116,0,0] Assertion `idx_dim >= 0 && idx_dim < index_size && "index out of bounds"` failed. C:\actions-runner\_work\pytorch\pytorch\pytorch\aten\src\ATen\native\cuda\ScatterGatherKernel.cu:145: block: [47696,0,0], thread: [46,0,0] Assertion `idx_dim >= 0 && idx_dim < index_size && "index out of bounds"` failed. C:\actions-runner\_work\pytorch\pytorch\pytorch\aten\src\ATen\native\cuda\ScatterGatherKernel.cu:145: block: [47798,0,0], thread: [77,0,0] Assertion `idx_dim >= 0 && idx_dim < index_size && "index out of bounds"` failed. C:\actions-runner\_work\pytorch\pytorch\pytorch\aten\src\ATen\native\cuda\ScatterGatherKernel.cu:145: block: [47913,0,0], thread: [54,0,0] Assertion `idx_dim >= 0 && idx_dim < index_size && "index out of bounds"` failed. C:\actions-runner\_work\pytorch\pytorch\pytorch\aten\src\ATen\native\cuda\ScatterGatherKernel.cu:145: block: [47930,0,0], thread: [56,0,0] Assertion `idx_dim >= 0 && idx_dim < index_size && "index out of bounds"` failed. C:\actions-runner\_work\pytorch\pytorch\pytorch\aten\src\ATen\native\cuda\ScatterGatherKernel.cu:145: block: [48076,0,0], thread: [100,0,0] Assertion `idx_dim >= 0 && idx_dim < index_size && "index out of bounds"` failed. C:\actions-runner\_work\pytorch\pytorch\pytorch\aten\src\ATen\native\cuda\ScatterGatherKernel.cu:145: block: [48159,0,0], thread: [9,0,0] Assertion `idx_dim >= 0 && idx_dim < index_size && "index out of bounds"` failed. C:\actions-runner\_work\pytorch\pytorch\pytorch\aten\src\ATen\native\cuda\ScatterGatherKernel.cu:145: block: [48208,0,0], thread: [12,0,0] Assertion `idx_dim >= 0 && idx_dim < index_size && "index out of bounds"` failed. C:\actions-runner\_work\pytorch\pytorch\pytorch\aten\src\ATen\native\cuda\ScatterGatherKernel.cu:145: block: [48376,0,0], thread: [104,0,0] Assertion `idx_dim >= 0 && idx_dim < index_size && "index out of bounds"` failed. C:\actions-runner\_work\pytorch\pytorch\pytorch\aten\src\ATen\native\cuda\ScatterGatherKernel.cu:145: block: [48781,0,0], thread: [73,0,0] Assertion `idx_dim >= 0 && idx_dim < index_size && "index out of bounds"` failed. C:\actions-runner\_work\pytorch\pytorch\pytorch\aten\src\ATen\native\cuda\ScatterGatherKernel.cu:145: block: [48942,0,0], thread: [82,0,0] Assertion `idx_dim >= 0 && idx_dim < index_size && "index out of bounds"` failed. C:\actions-runner\_work\pytorch\pytorch\pytorch\aten\src\ATen\native\cuda\ScatterGatherKernel.cu:145: block: [48985,0,0], thread: [5,0,0] Assertion `idx_dim >= 0 && idx_dim < index_size && "index out of bounds"` failed. C:\actions-runner\_work\pytorch\pytorch\pytorch\aten\src\ATen\native\cuda\ScatterGatherKernel.cu:145: block: [49050,0,0], thread: [118,0,0] Assertion `idx_dim >= 0 && idx_dim < index_size && "index out of bounds"` failed. C:\actions-runner\_work\pytorch\pytorch\pytorch\aten\src\ATen\native\cuda\ScatterGatherKernel.cu:145: block: [49095,0,0], thread: [124,0,0] Assertion `idx_dim >= 0 && idx_dim < index_size && "index out of bounds"` failed. C:\actions-runner\_work\pytorch\pytorch\pytorch\aten\src\ATen\native\cuda\ScatterGatherKernel.cu:145: block: [49251,0,0], thread: [37,0,0] Assertion `idx_dim >= 0 && idx_dim < index_size && "index out of bounds"` failed. C:\actions-runner\_work\pytorch\pytorch\pytorch\aten\src\ATen\native\cuda\ScatterGatherKernel.cu:145: block: [42433,0,0], thread: [55,0,0] Assertion `idx_dim >= 0 && idx_dim < index_size && "index out of bounds"` failed. C:\actions-runner\_work\pytorch\pytorch\pytorch\aten\src\ATen\native\cuda\ScatterGatherKernel.cu:145: block: [49313,0,0], thread: [42,0,0] Assertion `idx_dim >= 0 && idx_dim < index_size && "index out of bounds"` failed. C:\actions-runner\_work\pytorch\pytorch\pytorch\aten\src\ATen\native\cuda\ScatterGatherKernel.cu:145: block: [49354,0,0], thread: [31,0,0] Assertion `idx_dim >= 0 && idx_dim < index_size && "index out of bounds"` failed. C:\actions-runner\_work\pytorch\pytorch\pytorch\aten\src\ATen\native\cuda\ScatterGatherKernel.cu:145: block: [49401,0,0], thread: [115,0,0] Assertion `idx_dim >= 0 && idx_dim < index_size && "index out of bounds"` failed. C:\actions-runner\_work\pytorch\pytorch\pytorch\aten\src\ATen\native\cuda\ScatterGatherKernel.cu:145: block: [50035,0,0], thread: [120,0,0] Assertion `idx_dim >= 0 && idx_dim < index_size && "index out of bounds"` failed. C:\actions-runner\_work\pytorch\pytorch\pytorch\aten\src\ATen\native\cuda\ScatterGatherKernel.cu:145: block: [50123,0,0], thread: [55,0,0] Assertion `idx_dim >= 0 && idx_dim < index_size && "index out of bounds"` failed. C:\actions-runner\_work\pytorch\pytorch\pytorch\aten\src\ATen\native\cuda\ScatterGatherKernel.cu:145: block: [50168,0,0], thread: [39,0,0] Assertion `idx_dim >= 0 && idx_dim < index_size && "index out of bounds"` failed. C:\actions-runner\_work\pytorch\pytorch\pytorch\aten\src\ATen\native\cuda\ScatterGatherKernel.cu:145: block: [50194,0,0], thread: [126,0,0] Assertion `idx_dim >= 0 && idx_dim < index_size && "index out of bounds"` failed. C:\actions-runner\_work\pytorch\pytorch\pytorch\aten\src\ATen\native\cuda\ScatterGatherKernel.cu:145: block: [50329,0,0], thread: [64,0,0] Assertion `idx_dim >= 0 && idx_dim < index_size && "index out of bounds"` failed. C:\actions-runner\_work\pytorch\pytorch\pytorch\aten\src\ATen\native\cuda\ScatterGatherKernel.cu:145: block: [50389,0,0], thread: [116,0,0] Assertion `idx_dim >= 0 && idx_dim < index_size && "index out of bounds"` failed. C:\actions-runner\_work\pytorch\pytorch\pytorch\aten\src\ATen\native\cuda\ScatterGatherKernel.cu:145: block: [50631,0,0], thread: [65,0,0] Assertion `idx_dim >= 0 && idx_dim < index_size && "index out of bounds"` failed. C:\actions-runner\_work\pytorch\pytorch\pytorch\aten\src\ATen\native\cuda\ScatterGatherKernel.cu:145: block: [50700,0,0], thread: [37,0,0] Assertion `idx_dim >= 0 && idx_dim < index_size && "index out of bounds"` failed. C:\actions-runner\_work\pytorch\pytorch\pytorch\aten\src\ATen\native\cuda\ScatterGatherKernel.cu:145: block: [50738,0,0], thread: [37,0,0] Assertion `idx_dim >= 0 && idx_dim < index_size && "index out of bounds"` failed. C:\actions-runner\_work\pytorch\pytorch\pytorch\aten\src\ATen\native\cuda\ScatterGatherKernel.cu:145: block: [50883,0,0], thread: [125,0,0] Assertion `idx_dim >= 0 && idx_dim < index_size && "index out of bounds"` failed. C:\actions-runner\_work\pytorch\pytorch\pytorch\aten\src\ATen\native\cuda\ScatterGatherKernel.cu:145: block: [50913,0,0], thread: [30,0,0] Assertion `idx_dim >= 0 && idx_dim < index_size && "index out of bounds"` failed. C:\actions-runner\_work\pytorch\pytorch\pytorch\aten\src\ATen\native\cuda\ScatterGatherKernel.cu:145: block: [50947,0,0], thread: [71,0,0] Assertion `idx_dim >= 0 && idx_dim < index_size && "index out of bounds"` failed. C:\actions-runner\_work\pytorch\pytorch\pytorch\aten\src\ATen\native\cuda\ScatterGatherKernel.cu:145: block: [51129,0,0], thread: [45,0,0] Assertion `idx_dim >= 0 && idx_dim < index_size && "index out of bounds"` failed. C:\actions-runner\_work\pytorch\pytorch\pytorch\aten\src\ATen\native\cuda\ScatterGatherKernel.cu:145: block: [51195,0,0], thread: [49,0,0] Assertion `idx_dim >= 0 && idx_dim < index_size && "index out of bounds"` failed. C:\actions-runner\_work\pytorch\pytorch\pytorch\aten\src\ATen\native\cuda\ScatterGatherKernel.cu:145: block: [51218,0,0], thread: [36,0,0] Assertion `idx_dim >= 0 && idx_dim < index_size && "index out of bounds"` failed. C:\actions-runner\_work\pytorch\pytorch\pytorch\aten\src\ATen\native\cuda\ScatterGatherKernel.cu:145: block: [51710,0,0], thread: [41,0,0] Assertion `idx_dim >= 0 && idx_dim < index_size && "index out of bounds"` failed. [rank0]: Traceback (most recent call last): [rank0]: File "wheel_contents/aiengine/main_generation.py", line 87, in <module> [rank0]: File "wheel_contents/aiengine/model.py", line 36, in __init__ [rank0]: File "C:\ProgramData\BOINC\slots\0\Lib\site-packages\vllm\utils.py", line 1096, in inner [rank0]: return fn(*args, **kwargs) [rank0]: ^^^^^^^^^^^^^^^^^^^ [rank0]: File "C:\ProgramData\BOINC\slots\0\Lib\site-packages\vllm\entrypoints\llm.py", line 243, in __init__ [rank0]: self.llm_engine = LLMEngine.from_engine_args( [rank0]: ^^^^^^^^^^^^^^^^^^^^^^^^^^^ [rank0]: File "C:\ProgramData\BOINC\slots\0\Lib\site-packages\vllm\engine\llm_engine.py", line 521, in from_engine_args [rank0]: return engine_cls.from_vllm_config( [rank0]: ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ [rank0]: File "C:\ProgramData\BOINC\slots\0\Lib\site-packages\vllm\engine\llm_engine.py", line 497, in from_vllm_config [rank0]: return cls( [rank0]: ^^^^ [rank0]: File "C:\ProgramData\BOINC\slots\0\Lib\site-packages\vllm\engine\llm_engine.py", line 284, in __init__ [rank0]: self._initialize_kv_caches() [rank0]: File "C:\ProgramData\BOINC\slots\0\Lib\site-packages\vllm\engine\llm_engine.py", line 433, in _initialize_kv_caches [rank0]: self.model_executor.determine_num_available_blocks()) [rank0]: ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ [rank0]: File "C:\ProgramData\BOINC\slots\0\Lib\site-packages\vllm\executor\executor_base.py", line 103, in determine_num_available_blocks [rank0]: results = self.collective_rpc("determine_num_available_blocks") [rank0]: ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ [rank0]: File "C:\ProgramData\BOINC\slots\0\Lib\site-packages\vllm\executor\uniproc_executor.py", line 56, in collective_rpc [rank0]: answer = run_method(self.driver_worker, method, args, kwargs) [rank0]: ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ [rank0]: File "C:\ProgramData\BOINC\slots\0\Lib\site-packages\vllm\utils.py", line 2359, in run_method [rank0]: return func(*args, **kwargs) [rank0]: ^^^^^^^^^^^^^^^^^^^^^ [rank0]: File "C:\ProgramData\BOINC\slots\0\Lib\site-packages\torch\utils\_contextlib.py", line 116, in decorate_context [rank0]: return func(*args, **kwargs) [rank0]: ^^^^^^^^^^^^^^^^^^^^^ [rank0]: File "C:\ProgramData\BOINC\slots\0\Lib\site-packages\vllm\worker\worker.py", line 230, in determine_num_available_blocks [rank0]: self.model_runner.profile_run() [rank0]: File "C:\ProgramData\BOINC\slots\0\Lib\site-packages\torch\utils\_contextlib.py", line 116, in decorate_context [rank0]: return func(*args, **kwargs) [rank0]: ^^^^^^^^^^^^^^^^^^^^^ [rank0]: File "C:\ProgramData\BOINC\slots\0\Lib\site-packages\vllm\worker\model_runner.py", line 1243, in profile_run [rank0]: self._dummy_run(max_num_batched_tokens, max_num_seqs) [rank0]: File "C:\ProgramData\BOINC\slots\0\Lib\site-packages\vllm\worker\model_runner.py", line 1369, in _dummy_run [rank0]: self.execute_model(model_input, kv_caches, intermediate_tensors) [rank0]: File "C:\ProgramData\BOINC\slots\0\Lib\site-packages\torch\utils\_contextlib.py", line 116, in decorate_context [rank0]: return func(*args, **kwargs) [rank0]: ^^^^^^^^^^^^^^^^^^^^^ [rank0]: File "C:\ProgramData\BOINC\slots\0\Lib\site-packages\vllm\worker\model_runner.py", line 1826, in execute_model [rank0]: output: SamplerOutput = self.model.sample( [rank0]: ^^^^^^^^^^^^^^^^^^ [rank0]: File "C:\ProgramData\BOINC\slots\0\Lib\site-packages\vllm\model_executor\models\qwen2.py", line 480, in sample [rank0]: next_tokens = self.sampler(logits, sampling_metadata) [rank0]: ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ [rank0]: File "C:\ProgramData\BOINC\slots\0\Lib\site-packages\torch\nn\modules\module.py", line 1739, in _wrapped_call_impl [rank0]: return self._call_impl(*args, **kwargs) [rank0]: ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ [rank0]: File "C:\ProgramData\BOINC\slots\0\Lib\site-packages\torch\nn\modules\module.py", line 1750, in _call_impl [rank0]: return forward_call(*args, **kwargs) [rank0]: ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ [rank0]: File "C:\ProgramData\BOINC\slots\0\Lib\site-packages\vllm\model_executor\layers\sampler.py", line 287, in forward [rank0]: maybe_deferred_sample_results, maybe_sampled_tokens_tensor = _sample( [rank0]: ^^^^^^^^ [rank0]: File "C:\ProgramData\BOINC\slots\0\Lib\site-packages\vllm\model_executor\layers\sampler.py", line 775, in _sample [rank0]: return _sample_with_torch( [rank0]: ^^^^^^^^^^^^^^^^^^^ [rank0]: File "C:\ProgramData\BOINC\slots\0\Lib\site-packages\vllm\model_executor\layers\sampler.py", line 719, in _sample_with_torch [rank0]: probs[long_sample_indices], [rank0]: ~~~~~^^^^^^^^^^^^^^^^^^^^^ [rank0]: RuntimeError: CUDA error: device-side assert triggered [rank0]: CUDA kernel errors might be asynchronously reported at some other API call, so the stacktrace below might be incorrect. [rank0]: For debugging consider passing CUDA_LAUNCH_BLOCKING=1 [rank0]: Compile with `TORCH_USE_CUDA_DSA` to enable device-side assertions. 15:04:53 (18368): C:/Windows/system32/cmd.exe exited; CPU time 41.375000 15:04:53 (18368): app exit status: 0x16 15:04:53 (18368): called boinc_finish(195) 0 bytes in 0 Free Blocks. 536 bytes in 8 Normal Blocks. 1144 bytes in 1 CRT Blocks. 0 bytes in 0 Ignore Blocks. 0 bytes in 0 Client Blocks. Largest number used: 0 bytes. Total allocations: 712501 bytes. Dumping objects -> {1601251} normal block at 0x0000000000615560, 48 bytes long. Data: <PATH=C:\ProgramD> 50 41 54 48 3D 43 3A 5C 50 72 6F 67 72 61 6D 44 {1601240} normal block at 0x0000000000615F70, 48 bytes long. Data: <HOME=C:\ProgramD> 48 4F 4D 45 3D 43 3A 5C 50 72 6F 67 72 61 6D 44 {1601229} normal block at 0x0000000000615D40, 48 bytes long. Data: <TMP=C:\ProgramDa> 54 4D 50 3D 43 3A 5C 50 72 6F 67 72 61 6D 44 61 {1601218} normal block at 0x0000000000616050, 48 bytes long. Data: <TEMP=C:\ProgramD> 54 45 4D 50 3D 43 3A 5C 50 72 6F 67 72 61 6D 44 {1601207} normal block at 0x00000000005EF710, 48 bytes long. Data: <TMPDIR=C:\Progra> 54 4D 50 44 49 52 3D 43 3A 5C 50 72 6F 67 72 61 {1601176} normal block at 0x00000000024DE120, 64 bytes long. Data: <PATH=C:\ProgramD> 50 41 54 48 3D 43 3A 5C 50 72 6F 67 72 61 6D 44 {1601165} normal block at 0x0000000000616940, 140 bytes long. Data: <<project_prefere> 3C 70 72 6F 6A 65 63 74 5F 70 72 65 66 65 72 65 ..\api\boinc_api.cpp(309) : {1601162} normal block at 0x00000000005E6E30, 8 bytes long. Data: < > 00 00 1A 00 00 00 00 00 {1600516} normal block at 0x0000000000616870, 140 bytes long. Data: <<project_prefere> 3C 70 72 6F 6A 65 63 74 5F 70 72 65 66 65 72 65 {1599898} normal block at 0x00000000005E6CA0, 8 bytes long. Data: <à Z > E0 02 5A 02 00 00 00 00 ..\zip\boinc_zip.cpp(122) : {296} normal block at 0x00000000005EF9F0, 260 bytes long. Data: < > 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 {281} normal block at 0x00000000005DC660, 80 bytes long. Data: </c call Scripts\> 2F 63 20 63 61 6C 6C 20 53 63 72 69 70 74 73 5C {280} normal block at 0x00000000005F1160, 16 bytes long. Data: < _ > 08 20 5F 00 00 00 00 00 00 00 00 00 00 00 00 00 {279} normal block at 0x00000000005F1110, 16 bytes long. Data: <à _ > E0 1F 5F 00 00 00 00 00 00 00 00 00 00 00 00 00 {278} normal block at 0x00000000005F10C0, 16 bytes long. Data: <¸ _ > B8 1F 5F 00 00 00 00 00 00 00 00 00 00 00 00 00 {277} normal block at 0x00000000005F0F30, 16 bytes long. Data: < _ > 90 1F 5F 00 00 00 00 00 00 00 00 00 00 00 00 00 {276} normal block at 0x00000000005F19D0, 16 bytes long. Data: <h _ > 68 1F 5F 00 00 00 00 00 00 00 00 00 00 00 00 00 {275} normal block at 0x00000000005F1070, 16 bytes long. Data: <@ _ > 40 1F 5F 00 00 00 00 00 00 00 00 00 00 00 00 00 {274} normal block at 0x00000000005EF2B0, 48 bytes long. Data: <ComSpec=C:\Windo> 43 6F 6D 53 70 65 63 3D 43 3A 5C 57 69 6E 64 6F {273} normal block at 0x00000000005F18E0, 16 bytes long. Data: < _ > 08 0A 5F 00 00 00 00 00 00 00 00 00 00 00 00 00 {272} normal block at 0x00000000005EA7B0, 32 bytes long. Data: <SystemRoot=C:\Wi> 53 79 73 74 65 6D 52 6F 6F 74 3D 43 3A 5C 57 69 {271} normal block at 0x00000000005F0F80, 16 bytes long. Data: <à _ > E0 09 5F 00 00 00 00 00 00 00 00 00 00 00 00 00 {269} normal block at 0x00000000005F1430, 16 bytes long. Data: <¸ _ > B8 09 5F 00 00 00 00 00 00 00 00 00 00 00 00 00 {268} normal block at 0x00000000005F1890, 16 bytes long. Data: < _ > 90 09 5F 00 00 00 00 00 00 00 00 00 00 00 00 00 {267} normal block at 0x00000000005F0EE0, 16 bytes long. Data: <h _ > 68 09 5F 00 00 00 00 00 00 00 00 00 00 00 00 00 {266} normal block at 0x00000000005F1980, 16 bytes long. Data: <@ _ > 40 09 5F 00 00 00 00 00 00 00 00 00 00 00 00 00 {265} normal block at 0x00000000005F0CB0, 16 bytes long. Data: < _ > 18 09 5F 00 00 00 00 00 00 00 00 00 00 00 00 00 {264} normal block at 0x00000000005E9CD0, 32 bytes long. Data: <CUDA_DEVICE=0 PU> 43 55 44 41 5F 44 45 56 49 43 45 3D 30 00 50 55 {263} normal block at 0x00000000005F0C60, 16 bytes long. Data: <ð _ > F0 08 5F 00 00 00 00 00 00 00 00 00 00 00 00 00 {262} normal block at 0x00000000005F08F0, 320 bytes long. Data: <` _ М^ > 60 0C 5F 00 00 00 00 00 D0 9C 5E 00 00 00 00 00 {261} normal block at 0x00000000005F15C0, 16 bytes long. Data: < _ > 20 1F 5F 00 00 00 00 00 00 00 00 00 00 00 00 00 {260} normal block at 0x00000000005F17A0, 16 bytes long. Data: <ø _ > F8 1E 5F 00 00 00 00 00 00 00 00 00 00 00 00 00 {259} normal block at 0x00000000005EA3F0, 32 bytes long. Data: <C:/Windows/syste> 43 3A 2F 57 69 6E 64 6F 77 73 2F 73 79 73 74 65 {258} normal block at 0x00000000005F1020, 16 bytes long. Data: <Ð _ > D0 1E 5F 00 00 00 00 00 00 00 00 00 00 00 00 00 {257} normal block at 0x00000000005EA210, 32 bytes long. Data: <xjvf input.tar.b> 78 6A 76 66 20 69 6E 70 75 74 2E 74 61 72 2E 62 {256} normal block at 0x00000000005F0E40, 16 bytes long. Data: < _ > 18 1E 5F 00 00 00 00 00 00 00 00 00 00 00 00 00 {255} normal block at 0x00000000005F0DA0, 16 bytes long. Data: <ð _ > F0 1D 5F 00 00 00 00 00 00 00 00 00 00 00 00 00 {254} normal block at 0x00000000005F0D50, 16 bytes long. Data: <È _ > C8 1D 5F 00 00 00 00 00 00 00 00 00 00 00 00 00 {253} normal block at 0x00000000005F1570, 16 bytes long. Data: <  _ > A0 1D 5F 00 00 00 00 00 00 00 00 00 00 00 00 00 {252} normal block at 0x00000000005F11B0, 16 bytes long. Data: <x _ > 78 1D 5F 00 00 00 00 00 00 00 00 00 00 00 00 00 {251} normal block at 0x00000000005F1660, 16 bytes long. Data: <P _ > 50 1D 5F 00 00 00 00 00 00 00 00 00 00 00 00 00 {249} normal block at 0x00000000005F1930, 16 bytes long. Data: <Àõ^ > C0 F5 5E 00 00 00 00 00 00 00 00 00 00 00 00 00 {248} normal block at 0x00000000005EF5C0, 40 bytes long. Data: <0 _ áM > 30 19 5F 00 00 00 00 00 20 E1 4D 02 00 00 00 00 {247} normal block at 0x00000000005F0BC0, 16 bytes long. Data: <0 _ > 30 1D 5F 00 00 00 00 00 00 00 00 00 00 00 00 00 {246} normal block at 0x00000000005F1A20, 16 bytes long. Data: < _ > 08 1D 5F 00 00 00 00 00 00 00 00 00 00 00 00 00 {245} normal block at 0x00000000005EA690, 32 bytes long. Data: <Library/usr/bin/> 4C 69 62 72 61 72 79 2F 75 73 72 2F 62 69 6E 2F {244} normal block at 0x00000000005F1A70, 16 bytes long. Data: <à _ > E0 1C 5F 00 00 00 00 00 00 00 00 00 00 00 00 00 {243} normal block at 0x00000000005F1CE0, 992 bytes long. Data: <p _ ¦^ > 70 1A 5F 00 00 00 00 00 90 A6 5E 00 00 00 00 00 {87} normal block at 0x00000000005EA270, 32 bytes long. Data: <windows_x86_64__> 77 69 6E 64 6F 77 73 5F 78 38 36 5F 36 34 5F 5F {86} normal block at 0x00000000005E6B10, 16 bytes long. Data: <àô^ > E0 F4 5E 00 00 00 00 00 00 00 00 00 00 00 00 00 {85} normal block at 0x00000000005EF4E0, 40 bytes long. Data: < k^ p¢^ > 10 6B 5E 00 00 00 00 00 70 A2 5E 00 00 00 00 00 {64} normal block at 0x00000000005E6750, 16 bytes long. Data: < ê_õ÷ > 80 EA 5F F5 F7 7F 00 00 00 00 00 00 00 00 00 00 {63} normal block at 0x00000000005E6700, 16 bytes long. Data: <@é_õ÷ > 40 E9 5F F5 F7 7F 00 00 00 00 00 00 00 00 00 00 {62} normal block at 0x00000000005E6D40, 16 bytes long. Data: <øW\õ÷ > F8 57 5C F5 F7 7F 00 00 00 00 00 00 00 00 00 00 {61} normal block at 0x00000000005E6F70, 16 bytes long. Data: <ØW\õ÷ > D8 57 5C F5 F7 7F 00 00 00 00 00 00 00 00 00 00 {60} normal block at 0x00000000005E65C0, 16 bytes long. Data: <P \õ÷ > 50 04 5C F5 F7 7F 00 00 00 00 00 00 00 00 00 00 {59} normal block at 0x00000000005E6070, 16 bytes long. Data: <0 \õ÷ > 30 04 5C F5 F7 7F 00 00 00 00 00 00 00 00 00 00 {58} normal block at 0x00000000005E6E80, 16 bytes long. Data: <à \õ÷ > E0 02 5C F5 F7 7F 00 00 00 00 00 00 00 00 00 00 {57} normal block at 0x00000000005E6340, 16 bytes long. Data: < \õ÷ > 10 04 5C F5 F7 7F 00 00 00 00 00 00 00 00 00 00 {56} normal block at 0x00000000005E62F0, 16 bytes long. Data: <p \õ÷ > 70 04 5C F5 F7 7F 00 00 00 00 00 00 00 00 00 00 {55} normal block at 0x00000000005E60C0, 16 bytes long. Data: < ÀZõ÷ > 18 C0 5A F5 F7 7F 00 00 00 00 00 00 00 00 00 00 Object dump complete. </stderr_txt> ]]>
©2025 Universitat Pompeu Fabra