run7_replica47-NOELIA_sh2fragment_fixed-1-4-RND0785_0

So here's the deal. Currently the WU has been running for over 19 hours on my 680. I have checked both GPUz and EVGA Precision and the GPU is functioning properly. The WU is currently using 1.5GB of VRAM.

Currently it is at 88% and still progressing. I looked at the WU to see if I was not the first one to obtain this WU, and this is the fourth host it was sent two. Two timed out, and one aborted at 86,000 sec.

I looked at the stderr output and this is what I found:


Unhandled Exception Detected...

- Unhandled Exception Record -
Reason: Breakpoint Encountered (0x80000003) at address 0x74A2280C

Engaging BOINC Windows Runtime Debugger...



********************


BOINC Windows Runtime Debugger Version 6.7.0


Dump Timestamp : 08/01/12 06:44:00
Install Directory :
Data Directory : C:\ProgramData\BOINC
Project Symstore :
LoadLibraryA( C:\ProgramData\BOINC\dbghelp.dll ): GetLastError = 126
Loaded Library : dbghelp.dll
LoadLibraryA( C:\ProgramData\BOINC\symsrv.dll ): GetLastError = 126
LoadLibraryA( symsrv.dll ): GetLastError = 126
LoadLibraryA( C:\ProgramData\BOINC\srcsrv.dll ): GetLastError = 126
LoadLibraryA( srcsrv.dll ): GetLastError = 126
LoadLibraryA( C:\ProgramData\BOINC\version.dll ): GetLastError = 126
Loaded Library : version.dll
Debugger Engine : 4.0.5.0
Symbol Search Path: C:\ProgramData\BOINC\slots\1;C:\ProgramData\BOINC\projects\www.gpugrid.net;srv*C:\ProgramData\BOINC\projects\www.gpugrid.netsymbols*http://msdl.microsoft.com/download/symbols;srv*C:\ProgramData\BOINC\projects\www.gpugrid.netsymbols*http://boinc.berkeley.edu/symstore


ModLoad: 00400000 00354000 C:\ProgramData\BOINC\projects\www.gpugrid.net\acemd.2562.cuda42 (-nosymbols- Symbols Loaded)
Linked PDB Filename :

ModLoad: 76f50000 00180000 C:\Windows\SysWOW64\ntdll.dll (6.1.7601.17725) (-exported- Symbols Loaded)
Linked PDB Filename : wntdll.pdb
File Version : 6.1.7600.16385 (win7_rtm.090713-1255)
Company Name : Microsoft Corporation
Product Name : Microsoft® Windows® Operating System
Product Version : 6.1.7600.16385

ModLoad: 753c0000 00110000 C:\Windows\syswow64\kernel32.dll (6.1.7601.17651) (-exported- Symbols Loaded)
Linked PDB Filename : wkernel32.pdb
File Version : 6.1.7600.16385 (win7_rtm.090713-1255)
Company Name : Microsoft Corporation
Product Name : Microsoft® Windows® Operating System
Product Version : 6.1.7600.16385

ModLoad: 74a10000 00046000 C:\Windows\syswow64\KERNELBASE.dll (6.1.7601.17651) (-exported- Symbols Loaded)
Linked PDB Filename : wkernelbase.pdb
File Version : 6.1.7600.16385 (win7_rtm.090713-1255)
Company Name : Microsoft Corporation
Product Name : Microsoft® Windows® Operating System
Product Version : 6.1.7600.16385

ModLoad: 10000000 01c83000 C:\ProgramData\BOINC\projects\www.gpugrid.net\cufft32_42_9.dll (6.14.11.4020) (-exported- Symbols Loaded)
Linked PDB Filename :
File Version : 6,14,11,4020
Company Name : NVIDIA Corporation
Product Name : NVIDIA Windows XP CUDA 4.2.9 FFT Library
Product Version : 6,14,11,4020

ModLoad: 00220000 00073000 C:\ProgramData\BOINC\projects\www.gpugrid.net\cudart32_42_9.dll (6.14.11.4020) (-exported- Symbols Loaded)
Linked PDB Filename :
File Version : 6,14,11,4020
Company Name : NVIDIA Corporation
Product Name : NVIDIA CUDA 4.2.9 Runtime
Product Version : 6,14,11,4020

ModLoad: 009f0000 005f9000 C:\Windows\system32\nvcuda.dll (8.17.13.142) (-exported- Symbols Loaded)
Linked PDB Filename :
File Version : 8.17.13.0142
Company Name : NVIDIA Corporation
Product Name : NVIDIA CUDA 4.2.1 driver
Product Version : 8.17.13.0142

ModLoad: 74d70000 00100000 C:\Windows\syswow64\USER32.dll (6.1.7601.17514) (-exported- Symbols Loaded)
Linked PDB Filename : wuser32.pdb
File Version : 6.1.7601.17514 (win7sp1_rtm.101119-1850)
Company Name : Microsoft Corporation
Product Name : Microsoft® Windows® Operating System
Product Version : 6.1.7601.17514

ModLoad: 74690000 00090000 C:\Windows\syswow64\GDI32.dll (6.1.7601.17514) (-exported- Symbols Loaded)
Linked PDB Filename : wgdi32.pdb
File Version : 6.1.7601.17514 (win7sp1_rtm.101119-1850)
Company Name : Microsoft Corporation
Product Name : Microsoft® Windows® Operating System
Product Version : 6.1.7601.17514

ModLoad: 74770000 0000a000 C:\Windows\syswow64\LPK.dll (6.1.7600.16385) (-exported- Symbols Loaded)
Linked PDB Filename : wlpk.pdb
File Version : 6.1.7600.16385 (win7_rtm.090713-1255)
Company Name : Microsoft Corporation
Product Name : Microsoft® Windows® Operating System
Product Version : 6.1.7600.16385

ModLoad: 74810000 0009d000 C:\Windows\syswow64\USP10.dll (1.626.7601.17514) (-exported- Symbols Loaded)
Linked PDB Filename : usp10.pdb
File Version : 1.0626.7601.17514 (win7sp1_rtm.101119-1850)
Company Name : Microsoft Corporation
Product Name : Microsoft(R) Uniscribe Unicode script processor
Product Version : 1.0626.7601.17514

ModLoad: 74960000 000ac000 C:\Windows\syswow64\msvcrt.dll (7.0.7601.17744) (-exported- Symbols Loaded)
Linked PDB Filename : msvcrt.pdb
File Version : 7.0.7601.17744 (win7sp1_gdr.111215-1535)
Company Name : Microsoft Corporation
Product Name : Microsoft® Windows® Operating System
Product Version : 7.0.7601.17744

ModLoad: 76450000 000a0000 C:\Windows\syswow64\ADVAPI32.dll (6.1.7601.17514) (-exported- Symbols Loaded)
Linked PDB Filename : advapi32.pdb
File Version : 6.1.7600.16385 (win7_rtm.090713-1255)
Company Name : Microsoft Corporation
Product Name : Microsoft® Windows® Operating System
Product Version : 6.1.7600.16385

ModLoad: 748b0000 00019000 C:\Windows\SysWOW64\sechost.dll (6.1.7600.16385) (-exported- Symbols Loaded)
Linked PDB Filename : sechost.pdb
File Version : 6.1.7600.16385 (win7_rtm.090713-1255)
Company Name : Microsoft Corporation
Product Name : Microsoft® Windows® Operating System
Product Version : 6.1.7600.16385

ModLoad: 74ed0000 000f0000 C:\Windows\syswow64\RPCRT4.dll (6.1.7601.17514) (-exported- Symbols Loaded)
Linked PDB Filename : wrpcrt4.pdb
File Version : 6.1.7600.16385 (win7_rtm.090713-1255)
Company Name : Microsoft Corporation
Product Name : Microsoft® Windows® Operating System
Product Version : 6.1.7600.16385

ModLoad: 74630000 00060000 C:\Windows\syswow64\SspiCli.dll (6.1.7601.17856) (-exported- Symbols Loaded)
Linked PDB Filename : wsspicli.pdb
File Version : 6.1.7601.17856 (win7sp1_gdr.120601-1505)
Company Name : Microsoft Corporation
Product Name : Microsoft® Windows® Operating System
Product Version : 6.1.7601.17856

ModLoad: 74620000 0000c000 C:\Windows\syswow64\CRYPTBASE.dll (6.1.7600.16385) (-exported- Symbols Loaded)
Linked PDB Filename : cryptbase.pdb
File Version : 6.1.7600.16385 (win7_rtm.090713-1255)
Company Name : Microsoft Corporation
Product Name : Microsoft® Windows® Operating System
Product Version : 6.1.7600.16385

ModLoad: 75140000 0019d000 C:\Windows\syswow64\SETUPAPI.dll (6.1.7601.17514) (-exported- Symbols Loaded)
Linked PDB Filename : setupapi.pdb
File Version : 6.1.7600.16385 (win7_rtm.090713-1255)
Company Name : Microsoft Corporation
Product Name : Microsoft® Windows® Operating System
Product Version : 6.1.7600.16385

ModLoad: 74ea0000 00027000 C:\Windows\syswow64\CFGMGR32.dll (6.1.7601.17621) (-exported- Symbols Loaded)
Linked PDB Filename : cfgmgr32.pdb
File Version : 6.1.7601.17621 (win7sp1_gdr.110523-2108)
Company Name : Microsoft Corporation
Product Name : Microsoft® Windows® Operating System
Product Version : 6.1.7601.17621

Get Product Name Failed.
ModLoad: 755f0000 0008f000 C:\Windows\syswow64\OLEAUT32.dll (6.1.7601.17676) (-exported- Symbols Loaded)
Linked PDB Filename : oleaut32.pdb
File Version : 6.1.7601.17676
Company Name : Microsoft Corporation
Product Name :
Product Version : 6.1.7601.17676

ModLoad: 76530000 0015c000 C:\Windows\syswow64\ole32.dll (6.1.7601.17514) (-exported- Symbols Loaded)
Linked PDB Filename : ole32.pdb
File Version : 6.1.7600.16385 (win7_rtm.090713-1255)
Company Name : Microsoft Corporation
Product Name : Microsoft® Windows® Operating System
Product Version : 6.1.7600.16385

ModLoad: 74e70000 00012000 C:\Windows\syswow64\DEVOBJ.dll (6.1.7601.17621) (-exported- Symbols Loaded)
Linked PDB Filename : devobj.pdb
File Version : 6.1.7601.17621 (win7sp1_gdr.110523-2108)
Company Name : Microsoft Corporation
Product Name : Microsoft® Windows® Operating System
Product Version : 6.1.7601.17621

ModLoad: 75680000 00c4a000 C:\Windows\syswow64\SHELL32.dll (6.1.7601.17859) (-exported- Symbols Loaded)
Linked PDB Filename : shell32.pdb
File Version : 6.1.7601.17514 (win7sp1_rtm.101119-1850)
Company Name : Microsoft Corporation
Product Name : Microsoft® Windows® Operating System
Product Version : 6.1.7601.17514

ModLoad: 754e0000 00057000 C:\Windows\syswow64\SHLWAPI.dll (6.1.7601.17514) (-exported- Symbols Loaded)
Linked PDB Filename : shlwapi.pdb
File Version : 6.1.7600.16385 (win7_rtm.090713-1255)
Company Name : Microsoft Corporation
Product Name : Microsoft® Windows® Operating System
Product Version : 6.1.7600.16385

ModLoad: 764f0000 00035000 C:\Windows\syswow64\WS2_32.dll (6.1.7601.17514) (-exported- Symbols Loaded)
Linked PDB Filename : ws2_32.pdb
File Version : 6.1.7600.16385 (win7_rtm.090713-1255)
Company Name : Microsoft Corporation
Product Name : Microsoft® Windows® Operating System
Product Version : 6.1.7600.16385

ModLoad: 754d0000 00006000 C:\Windows\syswow64\NSI.dll (6.1.7600.16385) (-exported- Symbols Loaded)
Linked PDB Filename : nsi.pdb
File Version : 6.1.7600.16385 (win7_rtm.090713-1255)
Company Name : Microsoft Corporation
Product Name : Microsoft® Windows® Operating System
Product Version : 6.1.7600.16385

ModLoad: 00760000 000c9000 C:\ProgramData\BOINC\projects\www.gpugrid.net\tcl85.dll (8.5.2.2) (-exported- Symbols Loaded)
Linked PDB Filename :
File Version : 8.5.2
Company Name : ActiveState Corporation
Product Name : Tcl 8.5 for Windows
Product Version : 8.5.2

ModLoad: 763f0000 00060000 C:\Windows\system32\IMM32.DLL (6.1.7601.17514) (-exported- Symbols Loaded)
Linked PDB Filename : wimm32.pdb
File Version : 6.1.7601.17514 (win7sp1_rtm.101119-1850)
Company Name : Microsoft Corporation
Product Name : Microsoft® Windows® Operating System
Product Version : 6.1.7601.17514

ModLoad: 752e0000 000cc000 C:\Windows\syswow64\MSCTF.dll (6.1.7600.16385) (-exported- Symbols Loaded)
Linked PDB Filename : msctf.pdb
File Version : 6.1.7600.16385 (win7_rtm.090713-1255)
Company Name : Microsoft Corporation
Product Name : Microsoft® Windows® Operating System
Product Version : 6.1.7600.16385

ModLoad: 72e60000 00265000 C:\Windows\system32\nvapi.dll (8.17.13.142) (-exported- Symbols Loaded)
Linked PDB Filename : c:\dvs\p4\build\sw\rel\gpu_drv\r300\r301_07\drivers\nvapi\gpu\_out\win7_wow64_release\nvapi.pdb
File Version : 8.17.13.0142
Company Name : NVIDIA Corporation
Product Name : NVIDIA Windows drivers
Product Version : 8.17.13.0142

ModLoad: 734c0000 00009000 C:\Windows\system32\VERSION.dll (6.1.7600.16385) (-exported- Symbols Loaded)
Linked PDB Filename : version.pdb
File Version : 6.1.7600.16385 (win7_rtm.090713-1255)
Company Name : Microsoft Corporation
Product Name : Microsoft® Windows® Operating System
Product Version : 6.1.7600.16385

ModLoad: 74c20000 0002d000 C:\Windows\syswow64\WINTRUST.dll (6.1.7601.17787) (-exported- Symbols Loaded)
Linked PDB Filename : wintrust.pdb
File Version : 6.1.7601.17787 (win7sp1_gdr.120229-1502)
Company Name : Microsoft Corporation
Product Name : Microsoft® Windows® Operating System
Product Version : 6.1.7601.17787

ModLoad: 74c50000 0011e000 C:\Windows\syswow64\CRYPT32.dll (6.1.7601.17827) (-exported- Symbols Loaded)
Linked PDB Filename : crypt32.pdb
File Version : 6.1.7601.17827 (win7sp1_gdr.120423-1504)
Company Name : Microsoft Corporation
Product Name : Microsoft® Windows® Operating System
Product Version : 6.1.7601.17827

ModLoad: 76f20000 0000c000 C:\Windows\syswow64\MSASN1.dll (6.1.7601.17514) (-exported- Symbols Loaded)
Linked PDB Filename : msasn1.pdb
File Version : 6.1.7601.17514 (win7sp1_rtm.101119-1850)
Company Name : Microsoft Corporation
Product Name : Microsoft® Windows® Operating System
Product Version : 6.1.7601.17514

ModLoad: 71e40000 000eb000 C:\Windows\system32\dbghelp.dll (6.1.7601.17514) (-exported- Symbols Loaded)
Linked PDB Filename : dbghelp.pdb
File Version : 6.1.7601.17514 (win7sp1_rtm.101119-1850)
Company Name : Microsoft Corporation
Product Name : Microsoft® Windows® Operating System
Product Version : 6.1.7601.17514



*** Dump of the Process Statistics: ***

- I/O Operations Counters -
Read: 0, Write: 0, Other 0

- I/O Transfers Counters -
Read: 0, Write: 0, Other 0

- Paged Pool Usage -
QuotaPagedPoolUsage: 0, QuotaPeakPagedPoolUsage: 0
QuotaNonPagedPoolUsage: 0, QuotaPeakNonPagedPoolUsage: 0

- Virtual Memory Usage -
VirtualSize: 0, PeakVirtualSize: 0

- Pagefile Usage -
PagefileUsage: 0, PeakPagefileUsage: 0

- Working Set Size -
WorkingSetSize: 0, PeakWorkingSetSize: 0, PageFaultCount: 0

*** Dump of thread ID 3536 (state: Initialized): ***

- Information -
Status: Base Priority: Normal, Priority: Normal, , Kernel Time: 0.000000, User Time: 0.000000, Wait Time: 0.000000

- Unhandled Exception Record -
Reason: Breakpoint Encountered (0x80000003) at address 0x74A2280C

- Registers -
eax=00000000 ebx=00000000 ecx=006f9afa edx=049b613c esi=00000001 edi=00000000
eip=74a2280c esp=049bfb74 ebp=049bff94
cs=0023 ss=002b ds=002b es=002b fs=0053 gs=002b efl=00000246

- Callstack -
ChildEBP RetAddr Args to Child
049bff94 76f89ef2 00000000 739ee5f2 00000000 00000000 KERNELBASE!DebugBreak+0x0
049bffd4 76f89ec5 0045a640 00000000 00000000 00000000 ntdll!RtlInitializeExceptionChain+0x0
049bffec 00000000 0045a640 00000000 00000000 07742e99 ntdll!RtlInitializeExceptionChain+0x0

*** Dump of thread ID 3312 (state: Initialized): ***

- Information -
Status: Base Priority: Normal, Priority: Normal, , Kernel Time: 0.000000, User Time: 0.000000, Wait Time: 0.000000

- Registers -
eax=00000000 ebx=00000000 ecx=00000000 edx=00000000 esi=00000188 edi=00000000
eip=76f6f8b1 esp=00185b48 ebp=00185bb4
cs=0023 ss=002b ds=002b es=002b fs=0053 gs=002b efl=00000246

- Callstack -
ChildEBP RetAddr Args to Child
00185bb4 753d1194 00000188 ffffffff 00000000 02fabfd8 ntdll!ZwWaitForSingleObject+0x0
00185bcc 753d1148 00000188 ffffffff 00000000 00185bf0 kernel32!WaitForSingleObjectEx+0x0
00185be0 00de2ab6 00000188 ffffffff 00187010 00a3e356 kernel32!WaitForSingleObject+0x0
00185bf0 00a3e356 00186ffc ffffffff 00187048 00187048 nvcuda!clGetExtensionFunctionAddress+0x0
00187010 00a41412 00187048 032c12b8 0311fcf0 03117ba0 nvcuda!cuD3D11CtxCreate+0x0
00187030 00a2dfcb 00000002 00000002 00000000 04efb400 nvcuda!cuD3D11CtxCreate+0x0
00188428 00a159c5 0311fcf0 ffffff9c 00188490 00188494 nvcuda!cuD3D11CtxCreate+0x0
0018844c 009fb960 03021e78 00000000 001884cc 001884d0 nvcuda!cuD3D11CtxCreate+0x0
001884b8 00410272 03021e78 00188fa8 000027d8 0043449b nvcuda!cuStreamSynchronize+0x0
001884c8 0043449b 032c12b8 00000006 00000001 00188fa8 acemd.2562!+0x0
00188514 00435017 00000001 00188fa8 00000000 007ea502 acemd.2562!+0x0
00000000 00000000 00000000 00000000 00000000 00000000 acemd.2562!+0x0

*** Dump of thread ID 3508 (state: Initialized): ***

- Information -
Status: Base Priority: Normal, Priority: Normal, , Kernel Time: 0.000000, User Time: 0.000000, Wait Time: 0.000000

- Registers -
eax=00000000 ebx=0369fd24 ecx=00000000 edx=00000000 esi=00000004 edi=0369fd44
eip=76f7013d esp=0369fcd4 ebp=0369fd70
cs=0023 ss=002b ds=002b es=002b fs=0053 gs=002b efl=00000246

- Callstack -
ChildEBP RetAddr Args to Child
0369fd70 753d1a2c 0369fd24 0369fd98 00000000 00000038 ntdll!ZwWaitForMultipleObjects+0x0
0369fdb8 753d4208 00000004 7efde000 00000000 00000038 kernel32!WaitForMultipleObjectsEx+0x0
0369fdd4 00de2be2 00000004 0369fdf0 00000000 00000038 kernel32!WaitForMultipleObjects+0x0
0369fef0 00a49266 02fabf48 00000004 0369ff2c 00000038 nvcuda!clGetExtensionFunctionAddress+0x0
0369ff30 00de348c 027fbe70 00000000 00000000 0311d018 nvcuda!cuD3D11CtxCreate+0x0
0369ff48 00e37e49 0311cfe8 e041d67a 00000000 00000000 nvcuda!clGetExtensionFunctionAddress+0x0
0369ff80 00e37eee 00000000 753d339a 0311d018 0369ffd4 nvcuda!clGetExtensionFunctionAddress+0x0
0369ff94 76f89ef2 0311d018 746ce5f2 00000000 00000000 nvcuda!clGetExtensionFunctionAddress+0x0
0369ffd4 76f89ec5 00e37e6f 0311d018 00000000 00000000 ntdll!RtlInitializeExceptionChain+0x0
0369ffec 00000000 00e37e6f 0311d018 00000000 00000000 ntdll!RtlInitializeExceptionChain+0x0

*** Dump of thread ID 3516 (state: Initialized): ***

- Information -
Status: Base Priority: Normal, Priority: Normal, , Kernel Time: 0.000000, User Time: 0.000000, Wait Time: 0.000000

- Registers -
eax=00000000 ebx=00000000 ecx=00000000 edx=00000000 esi=000000c4 edi=00000000
eip=76f6f8b1 esp=0379fe60 ebp=0379fecc
cs=0023 ss=002b ds=002b es=002b fs=0053 gs=002b efl=00000246

- Callstack -
ChildEBP RetAddr Args to Child
0379fecc 753d1194 000000c4 ffffffff 00000000 00000000 ntdll!ZwWaitForSingleObject+0x0
0379fee4 753d1148 000000c4 ffffffff 00000000 0379ff08 kernel32!WaitForSingleObjectEx+0x0
0379fef8 00de2ab6 000000c4 ffffffff 0379ff30 00ab2388 kernel32!WaitForSingleObject+0x0
0379ff08 00ab2388 02fccfa4 ffffffff 02fccfb8 009f0000 nvcuda!clGetExtensionFunctionAddress+0x0
0379ff30 00de348c 02fccf50 00000000 00000000 0311d018 nvcuda!cuD3D11CtxCreate+0x0
0379ff48 00e37e49 02fccfb8 e051d67a 00000000 00000000 nvcuda!clGetExtensionFunctionAddress+0x0
0379ff80 00e37eee 00000000 753d339a 0311d018 0379ffd4 nvcuda!clGetExtensionFunctionAddress+0x0
0379ff94 76f89ef2 0311d018 747ce5f2 00000000 00000000 nvcuda!clGetExtensionFunctionAddress+0x0
0379ffd4 76f89ec5 00e37e6f 0311d018 00000000 00000000 ntdll!RtlInitializeExceptionChain+0x0
0379ffec 00000000 00e37e6f 0311d018 00000000 00412e74 ntdll!RtlInitializeExceptionChain+0x0


*** Debug Message Dump ****


*** Foreground Window Data ***
Window Name :
Window Class :
Window Process ID: 0
Window Thread ID : 0


I figure I might as well let it finish. Was gone last night, otherwise I may have cancelled it if I saw it was taking this long. So, is this a bad WU?

Comments?

I had a NOELIA wu with similar strange behavior. I restarted my host because I thought that my GPU is downclocked. After it booted up, I noticed that the progress indicator of this task has decreased significantly (I can't recall it exactly, but it was around 10% less). Then it finished without any further problems, but it had similar error messages in the stderr output file like yours.