# Using CUDA device 0
Cuda error in file 'deviceQuery.cu' in line 59 : out of memory.

http://www.ps3grid.net/result.php?resultid=51269
http://www.ps3grid.net/result.php?resultid=50907
http://www.ps3grid.net/result.php?resultid=50789
http://www.ps3grid.net/result.php?resultid=50903

i know how to get this error :(
these wu where (re)started by boinc when i was playing tf2. the fact is that nvidia driver can't allocate enough memory for the cuda application when there is a heavy gfx game running :(

i think that boinc should take a look at the video memory before starting this kind of tasks.

what OS, BOINC version, and driver version do you have? Also, what type of video card are you running?

what OS, BOINC version, and driver version do you have? Also, what type of video card are you running?

I had an 8800GT running just fine but had to re-install Vista Ultimate from scratch (HD foo-faa).

New HD.
Installed the Cuda drivers (Vista 64) just like before. Installed 6.3.10 just like before.

Now I get this message with the w/u's.

Cuda error in file 'deviceQuery.cu" Line 59 Initialization error

Any ideas????

Today i got the same message as below.

But i don't change anything on this computer, just the drivers (177.92).

I don't use this computer, i'm at work!

Boinc isn't install with service option and secure mode.
6.3.10, XP x64, 177.92

So what's happen?

Jim PROFIT

I've the same on 3 WU on my P5bv / 4Gb DDR2-8500 with :
XP32 / 6.3.10 / driver 177.92 on NVIDIA 8600GTS 256Mb ---> G84
I sugest to see in the file deviceQuery.cu at ligne 59, what king of parameters (memory size, ... )it used and put then im the error message to give more informations to debug.

After i load se cuba SDK to see what look like the file. I think the deviceQuery.cu is to be reWritten completly.

I sugest :

// Program main
int
main( int argc, char** argv)
{
int deviceCount=0, dev=0;

CUDA_SAFE_CALL(cudaGetDeviceCount(&deviceCount));

if (deviceCount == 0)
printf("There is no device supporting CUDA\n");
else
{
if (deviceCount == 1)
printf("There is 1 device supporting CUDA\n");
else
printf("There are %d devices supporting CUDA\n", deviceCount);

cudaDeviceProp deviceProp[deviceCount];

for (dev = 0; dev < deviceCount; ++dev) {

CUDA_SAFE_CALL(cudaGetDeviceProperties(&deviceProp[dev], dev));

if (dev == 0) { // see CudaRefenreceManuel 1.1.1 cudaGetDeviceCount
if (deviceProp[dev].major == 9999 && deviceProp[dev].minor == 9999)
printf("The device is no supporting CUDA.\n");
}

printf("\n Device %d: \"%s\"\n", dev, deviceProp[dev].name);
printf(" Major revision number: %d\n", deviceProp[dev].major);
printf(" Minor revision number: %d\n", deviceProp[dev].minor);
printf(" Total amount of global memory: %u bytes\n", deviceProp[dev].totalGlobalMem);
#if CUDART_VERSION >= 2000
printf(" Number of multiprocessors: %d\n", deviceProp[dev].multiProcessorCount);
printf(" Number of cores: %d\n", 8 * deviceProp[dev].multiProcessorCount);
#endif
printf(" Total amount of constant memory: %u bytes\n", deviceProp[dev].totalConstMem);
printf(" Total amount of shared memory per block: %u bytes\n", deviceProp[dev].sharedMemPerBlock);
printf(" Total number of registers available per block:%d\n", deviceProp[dev].regsPerBlock);
printf(" Warp size: %d\n", deviceProp[dev].warpSize);
printf(" Maximum number of threads per block: %d\n", deviceProp[dev].maxThreadsPerBlock);

printf(" Maximum sizes of each dimension of a block: %d x %d x %d\n",
deviceProp[dev].maxThreadsDim[0],
deviceProp[dev].maxThreadsDim[1],
deviceProp[dev].maxThreadsDim[2]);

printf(" Maximum sizes of each dimension of a grid: %d x %d x %d\n",
deviceProp[dev].maxGridSize[0],
deviceProp[dev].maxGridSize[1],
deviceProp[dev].maxGridSize[2]);

printf(" Maximum memory pitch: %u bytes\n", deviceProp[dev].memPitch);
printf(" Texture alignment: %u bytes\n", deviceProp[dev].textureAlignment);
printf(" Clock rate: %.2f GHz\n", deviceProp[dev].clockRate * 1e-6f);
#if CUDART_VERSION >= 2000
printf(" Concurrent copy and execution: %s\n", deviceProp[dev].deviceOverlap ? "Yes" : "No");
#endif
}
}
printf("\nTest PASSED\n");
CUT_EXIT(argc, argv);
}

Are you running any other applications that might be taking GPU memory? (Games or other 3D applications for example.)

MJH

A member of our Team Planet3DNow also reports errored Wus with similar problems

<core_client_version>6.3.10</core_client_version>
<![CDATA[
<message>
Unzul?ssige Funktion. (0x1) - exit code 1 (0x1)
</message>
<stderr_txt>
# Using CUDA device 0
Cuda error in file 'deviceQuery.cu' in line 59 : unknown error.

</stderr_txt>
]]>

I've been out of town for a few days. I piled up a few errors with a bunch of 6.43 and 1 - 6.45 apps:

http://www.ps3grid.net/results.php?hostid=8288

All with the same error message...

<core_client_version>6.3.10</core_client_version>
<![CDATA[
<message>
Incorrect function. (0x1) - exit code 1 (0x1)
</message>
<stderr_txt>
# Using CUDA device 0
Cuda error in file 'deviceQuery.cu' in line 59 : out of memory.

Win X64 pro
177.84 driver
8800GT No OC on GPU
Boinc 6.3.10


I am over my quota for today. But I have rebooted the box. I don't game on this box, and run 1 other Boinc Project alongside PS3Grid.

____________


Consciousness: That annoying time between naps......

Experience is a wonderful thing: it enables you to recognize a mistake every time you repeat it.

Ok, I picked up a new wu, and it is running 00:08 seconds at 18% finished....

____________


Consciousness: That annoying time between naps......

Experience is a wonderful thing: it enables you to recognize a mistake every time you repeat it.

This just started happening to my machine (XP64, 6gb, 6.3.10, 177.92, GTX 280). It is a dedicated cruncher. It successfully crunched 4 tasks under 6.45, then 7 of these errors in a row. I am rebooting to see if that helps.

____________
Reno, NV
Team: SETI.USA

I will have to wait a day, to see if rebooting helped. The failures maxed out my daily quota.

____________
Reno, NV
Team: SETI.USA

After the reboot. The wu finished with no errors. I'll keep an eye on the memory usage....

http://www.ps3grid.net/result.php?resultid=58599

But the run time increased from ~54,000 to 65,000 seconds.

____________


Consciousness: That annoying time between naps......

Experience is a wonderful thing: it enables you to recognize a mistake every time you repeat it.

This just started happening to my machine (XP64, 6gb, 6.3.10, 177.92, GTX 280). It is a dedicated cruncher. It successfully crunched 4 tasks under 6.45, then 7 of these errors in a row. I am rebooting to see if that helps.

This just started happening to my machine (XP64, 6gb, 6.3.10, 177.92, GTX 280). It is a dedicated cruncher. It successfully crunched 4 tasks under 6.45, then 7 of these errors in a row. I am rebooting to see if that helps.

Rebooting seems to have solved the problem, at least for now. I'll post if it happens again.

4 days later, it happened again. 8 tasks errored out in a row. I will reboot again.

I had the same "Cuda error in file 'deviceQuery.cu' in line 59 : out of memory" this morning on 4 or 5 Wu's in a row, I rebooted & the next Wu started running ok.

What it is if you have Time ticking down in the Manager Window for every second it ticks down the Memory consummation increases for the Manager. Setting the Manager Window to the Transfer Tab when not viewing the Wu's or something else stops the increase in consummation & actually after some time the Memory usage went down ...

My winx64 box has burped this '...out of memory error..', with the associated corrupt wu's, twice since I have first posted. I do not see any hugh increase in memory usage before or after the problem. I am only looking in 'Task Manager'.

using 6.3.10

Should I be looking else where?

____________


Consciousness: That annoying time between naps......

Experience is a wonderful thing: it enables you to recognize a mistake every time you repeat it.

I noticed this problem only on XP-64 and 2K3-64.

But on Vista, i don't have this problem!

Jim PROFIT

What do you mean by "manager window?" Tasks or messages maybe?

MrS

It happened again. This is after upgrading to 6.3.11. So that was not a fix.

Edit: I'm not saying that 6.3.11 was supposed to fix it. Just that it doesn't.
____________
Reno, NV
Team: SETI.USA

I guess the next thing is to 'shut down' Boincmgr in Taskmanager, and see what happens....

____________


Consciousness: That annoying time between naps......

Experience is a wonderful thing: it enables you to recognize a mistake every time you repeat it.

FWIW, I have not seen boincmgr memory usage increase. Not with .10 or .11.
____________
Reno, NV
Team: SETI.USA

I just looked @ 1 Box & the boincmgr.exe was using 1,100,777 of Memeory, I shut Boinc Down & restarted BOINC again and the boincmgr.exe was only using 10,584 of Memeory so it must increase in some cases ... This isn't the first time I've seen it that high either ...

[size=15]
I never saw boincmgr using more than 12-13k of memory. But something is messing up my wu's.....
____________


Consciousness: That annoying time between naps......

Experience is a wonderful thing: it enables you to recognize a mistake every time you repeat it.

As far as I understood the excessive memory usage of BOINC manager only affects 64 Bit system. I don't know which ones exactly, but 32 bit versions are certainly not affected.

And this is probably not directly related to the error, which is referring to GPU memory and not to system memory.

MrS

All my Systems are 64-Bit so that might explain why I'm seeing more than some people are ...

I too am using win 64 bit. And started getting memory errors a little ways back. I have closed boincmgr and the wu's are fine (so far).

I just opened boincmgr this morning, and it only showed ~3k of memory usage, but I closed it out (end process) again anyway. I will let it run like this for a few days, or until it errors out a wu again.

Win x64
177.84 driver
2 gig ram
boinc 6.3.10
8800gt (no oc)

____________


Consciousness: That annoying time between naps......

Experience is a wonderful thing: it enables you to recognize a mistake every time you repeat it.

Seems that closing Boincmgr in Taskmanager did not stop my computer memory errors. It ran for a few days and today I got the same memory error.

____________


Consciousness: That annoying time between naps......

Experience is a wonderful thing: it enables you to recognize a mistake every time you repeat it.

Seems that closing Boincmgr in Taskmanager did not stop my computer memory errors. It ran for a few days and today I got the same memory error.

I'm still getting aborted wu's with the 'out of memory' error. It seems to happen after 2-3 days, reboot, wait 2-3 days, etc...

http://www.ps3grid.net/results.php?hostid=8288

Win XP 64
6.3.14 (upgraded a week ago)
177.84 driver


____________


Consciousness: That annoying time between naps......

Experience is a wonderful thing: it enables you to recognize a mistake every time you repeat it.

Same error here... http://www.ps3grid.net/results.php?hostid=15279

WinXP64
178.08
boinc 6.3.14

It was running fine at first try but when I remote connect (boincview or boinc manager from diff pc) things gone wrong. http://www.ps3grid.net/result.php?resultid=83224

any ideas?

trying 6.3.10 now and waiting for my qouta to reset.

has anyone with vista ran into this? I know that vista has redone the graphics API, and it also allocates some of the system memory as addressable to the video card, thus increasing the work space the video apps has available. The graphics drivers also doesn't have legacy code in them, making them more efficient. I wonder if these changes are the reason vista doesn't run into the same problems as XP?

This also happened to me with 6.3.10...

I had hoped that .14 would solve that issue.

____________


Consciousness: That annoying time between naps......

Experience is a wonderful thing: it enables you to recognize a mistake every time you repeat it.

I'm getting this again...It seems to happen after 2-3 days

Going to try vista.

Maybe the developers can take a look if there are some memory leaked.

Today I got 9 WUs crashed on XP Pro 64 with my GTX260²:

Cuda error in file 'deviceQuery.cu' in line 59 : out of memory.

Now I get no new WUs for today, also not after reboot, detach and re-attach.

Any hint to avoid this situation?
____________

Today I got 9 WUs crashed on XP Pro 64 with my GTX260²:

Cuda error in file 'deviceQuery.cu' in line 59 : out of memory.

Now I get no new WUs for today, also not after reboot, detach and re-attach.

Any hint to avoid this situation?

... run 3DMark 2006 (should need some mem) after boot, note the score and rerun after you got this error. If the scores are identical ...

try upgrading DirectX9 to the latest August 2008 build:
http://www.microsoft.com/downloads/details.aspx?familyid=2DA43D38-DB71-4C1B-BC6A-9B6652CD92A3&displaylang=en

I've used Rivatuner instead of 3DBenbch06 to check the used memory of the graphic card under XP 64 bit. There is a plugin named Vidmem.dll in the installation of the Rivatuner you can use. After a fresh reboot the card use 70.13 MB of Video RAM. After 12 hours runtime the card use 138.26 MB. I had quit the BOINC manager and restart him, the card used then 206.32 MB. I checked again and made a second restart of the BOINC manager and the card used 274.82 MB. You can provoke the memory fault with repeated restart of the BOINC manager, then you surely get the message:

Cuda error in file 'deviceQuery.cu' in line 59 : out of memory

after some restarts or latest after 3 or 4 days of running time.

My tip for XP 64 user: Restart your XP 64 often while using CUDA or change to Vista 64 or Linux, they don't have this problem.


____________

I've used Rivatuner instead of 3DBenbch06 to check the used memory of the graphic card under XP 64 bit. There is a plugin named Vidmem.dll in the installation of the Rivatuner you can use. After a fresh reboot the card use 70.13 MB of Video RAM. After 12 hours runtime the card use 138.26 MB. I had quit the BOINC manager and restart him, the card used then 206.32 MB. I checked again and made a second restart of the BOINC manager and the card used 274.82 MB. You can provoke the memory fault with repeated restart of the BOINC manager, then you surely get the message:

Cuda error in file 'deviceQuery.cu' in line 59 : out of memory

after some restarts or latest after 3 or 4 days of running time.

My tip for XP 64 user: Restart your XP 64 often while using CUDA or change to Vista 64 or Linux, they don't have this problem.

I have been rebooting my machine daily, to try to avoid this problem. It seemed to help for a while. It didn't happen for a couple of weeks anyway. Then last night, it happened again. It's with 6.3.19. Now I have to wait 24 hours until I can get more tasks to crunch.
____________
Reno, NV
Team: SETI.USA

I have been rebooting my machine daily, to try to avoid this problem. It seemed to help for a while. It didn't happen for a couple of weeks anyway. Then last night, it happened again. It's with 6.3.19. Now I have to wait 24 hours until I can get more tasks to crunch.

Well, i got hit hard again by these errors. About a week ago I reported them in another thread and was directed here. After reading suggestions here, I rebooted the offending machine and it started to play nicely, until today. During that period I also upgraded BOINC client to 6.3.21, which is finnaly assigning correct number of tasks to CPU and GPU cores. Next I'll try upgrading graphics drivers to see if that sorts this problem.

Greetings,
____________

I got hit by memory errors again on 2 wu's.

6.3.21
XP 64
177.84 driver

<core_client_version>6.3.21</core_client_version>
<![CDATA[
<message>
Incorrect function. (0x1) - exit code 1 (0x1)
</message>
<stderr_txt>
# Using CUDA device 0
Cuda error in file 'deviceQuery.cu' in line 59 : out of memory.

I performed a re-boot. Hope that fixes it for a while..

EDIT: After the re-boot, I recieved 2 new tasks, and 1 is running fine now...

____________


Consciousness: That annoying time between naps......

Experience is a wonderful thing: it enables you to recognize a mistake every time you repeat it.

Sorry, now I see already a thread about this.

I have a GTX280/9800GT (no SLI/not overclocked) and I am using 6.3.21 client with XP Pro 32bit and had this second times in a few days. Before I was using 6.3.19 and never had this problem!
My system is not running 24/7, only crunching when I am at home. Normally it takes me 2-3 days to finish.
It seems there is no solution for now?


<core_client_version>6.3.21</core_client_version>
<![CDATA[
<message>
Unzul�ssige Funktion. (0x1) - exit code 1 (0x1)
</message>
<stderr_txt>
# Using CUDA device 1
# Device 0: "GeForce GTX 280"
# Clock rate: 1404000 kilohertz
# Device 1: "GeForce 9800 GT"
# Clock rate: 1620000 kilohertz
MDIO ERROR: cannot open file "restart.coor"
# Using CUDA device 0
# Device 0: "GeForce GTX 280"
# Clock rate: 1404000 kilohertz
# Device 1: "GeForce 9800 GT"
# Clock rate: 1620000 kilohertz
Cuda error: Kernel [reduce4_kernel] failed in file 'reduction.cu' in line 143 : unspecified launch failure.


<core_client_version>6.3.21</core_client_version>
<![CDATA[
<message>
Unzul�ssige Funktion. (0x1) - exit code 1 (0x1)
</message>
<stderr_txt>
# Using CUDA device 0
# Device 0: "GeForce GTX 280"
# Clock rate: 1404000 kilohertz
# Device 1: "GeForce 9800 GT"
# Clock rate: 1620000 kilohertz
MDIO ERROR: cannot open file "restart.coor"
Cuda error: Kernel [angle_kernel] failed in file 'bonded.cu' in line 547 : unspecified launch failure.
____________

Sorry! But is there anyone who had this error or knows about it??
____________

Hi,

I have a systems running with GTX 9800+ 512MB. Running with Vista64 and BOINC 6.3.21...

I have this one since Oct 28th:

<core_client_version>6.3.21</core_client_version>
<![CDATA[
<message>
Unzul�ssige Funktion. (0x1) - exit code 1 (0x1)
</message>
<stderr_txt>
# Using CUDA device 0
Cuda error in file 'deviceQuery.cu' in line 59 : initialization error.

</stderr_txt>
]]>

I have updated to the latest nvidia driver (178.08) BOINC to the latest Version and yesterday a bigger PSU (550W).

The error is still the same. No probs before with BOINC 6.3.19 and appl. 6.42.

What I have tested is restarting BOINC before getting one WU (settings to get only 0,05 days work, don't end in the 4WU per day limit...) and in most of the cases the WU is done.

This could only be a temporary solution!

Has somebody an idea about this?




____________

You can have a look an it :)

http://www.ps3grid.net/results.php?hostid=9664

The successful results are by baby-sitting GPUGrid.

This is really not all that much different than what I posted in the 6.3.21 thread and never got a response on....that is my Linux 8.04 Q9550 9800GT will not d/l work under 6.3.21. I have to revert back to 6.3.19 get work and then re-install 6.3.21 to run. This is not a permanent solution. I also get major screen flicker in the message tab.I get the normal out of work message in 6.3.21...only cell proccesor work available.

Sorry ETA the similarity is having to babysit Boinc.

This comp is just BOINC farm. There is no ANY other software, just drivers and BOINC.

Win XP x64
178.24
6.3.19
6.48
As always!

2GB RAM + 280GTX(1GB)

It`s crunching 24/7. No any changes in last week. ANY. Everything was fine until yesterday evening.

Today, right before I waked up:

<core_client_version>6.3.19</core_client_version>
<![CDATA[
<message>
The system cannot find the path specified. (0x3) - exit code 3 (0x3)
</message>
<stderr_txt>
# Using CUDA device 0
# Device 0: "GeForce GTX 280"
# Clock rate: 1296000 kilohertz
Cuda error in file '..\cuda/cutil.h' in line 298 : out of memory.
Memory usage: host: bytes device: bytes
Assertion failed: 0, file ..\cuda/cutil.h, line 298

OR

<core_client_version>6.3.19</core_client_version>
<![CDATA[
<message>
Incorrect function. (0x1) - exit code 1 (0x1)
</message>
<stderr_txt>
# Using CUDA device 0
Cuda error in file 'deviceQuery.cu' in line 59 : out of memory.

</stderr_txt>
]]>


This application has requested the Runtime to terminate it in an unusual way.
Please contact the application's support team for more information.

</stderr_txt>
]]>



Last 15 tasks... !!! :/.

http://www.gpugrid.net/results.php?hostid=15721

And now my daily quota is ofc 4 :/. Dunno why, haven`t changed ANYTHING in comp (hardware or software). I could understand if I`ll do something with that, but this comp was crunching correct hole night... And I`ll repeat: it`s just an BOINC farm. Then pls don`t write about SLi, different drivers or anything like that... This comp was crunching correct last weeks.

Why it happened? Any1 knows?
____________

this appears to be a problem with win64 here are the latest fails with 6.3.21 and app 6.52

<core_client_version>6.3.21</core_client_version>
<![CDATA[
<message>
Incorrect function. (0x1) - exit code 1 (0x1)
</message>
<stderr_txt>
# Using CUDA device 0
Cuda error in file 'deviceQuery.cu' in line 59 : out of memory.

</stderr_txt>
]]>

<core_client_version>6.3.21</core_client_version>
<![CDATA[
<message>
The system cannot find the path specified. (0x3) - exit code 3 (0x3)
</message>
<stderr_txt>
# Using CUDA device 3
# Device 0: "GeForce 8800 GT"
# Clock rate: 1512000 kilohertz
# Number of multiprocessors: 14
# Number of cores: 112
# Device 1: "GeForce 8800 GT"
# Clock rate: 1512000 kilohertz
# Number of multiprocessors: 14
# Number of cores: 112
# Device 2: "GeForce 8800 GT"
# Clock rate: 1512000 kilohertz
# Number of multiprocessors: 14
# Number of cores: 112
# Device 3: "GeForce 8800 GT"
# Clock rate: 1512000 kilohertz
# Number of multiprocessors: 14
# Number of cores: 112
MDIO ERROR: cannot open file "restart.coor"
# Using CUDA device 3
# Device 0: "GeForce 8800 GT"
# Clock rate: 1512000 kilohertz
# Number of multiprocessors: 14
# Number of cores: 112
# Device 1: "GeForce 8800 GT"
# Clock rate: 1512000 kilohertz
# Number of multiprocessors: 14
# Number of cores: 112
# Device 2: "GeForce 8800 GT"
# Clock rate: 1512000 kilohertz
# Number of multiprocessors: 14
# Number of cores: 112
# Device 3: "GeForce 8800 GT"
# Clock rate: 1512000 kilohertz
# Number of multiprocessors: 14
# Number of cores: 112
# Using CUDA device 3
# Device 0: "GeForce 8800 GT"
# Clock rate: 1512000 kilohertz
# Number of multiprocessors: 14
# Number of cores: 112
# Device 1: "GeForce 8800 GT"
# Clock rate: 1512000 kilohertz
# Number of multiprocessors: 14
# Number of cores: 112
# Device 2: "GeForce 8800 GT"
# Clock rate: 1512000 kilohertz
# Number of multiprocessors: 14
# Number of cores: 112
# Device 3: "GeForce 8800 GT"
# Clock rate: 1512000 kilohertz
# Number of multiprocessors: 14
# Number of cores: 112
Cuda error in file '..\cuda/cutil.h' in line 298 : out of memory.
Memory usage: host: bytes device: bytes
Assertion failed: 0, file ..\cuda/cutil.h, line 298

This application has requested the Runtime to terminate it in an unusual way.
Please contact the application's support team for more information.

</stderr_txt>
]]>

<core_client_version>6.3.21</core_client_version>
<![CDATA[
<message>
The system cannot find the path specified. (0x3) - exit code 3 (0x3)
</message>
<stderr_txt>
# Using CUDA device 1
# Device 0: "GeForce 8800 GT"
# Clock rate: 1512000 kilohertz
# Number of multiprocessors: 14
# Number of cores: 112
# Device 1: "GeForce 8800 GT"
# Clock rate: 1512000 kilohertz
# Number of multiprocessors: 14
# Number of cores: 112
# Device 2: "GeForce 8800 GT"
# Clock rate: 1512000 kilohertz
# Number of multiprocessors: 14
# Number of cores: 112
# Device 3: "GeForce 8800 GT"
# Clock rate: 1512000 kilohertz
# Number of multiprocessors: 14
# Number of cores: 112
MDIO ERROR: cannot open file "restart.coor"
# Using CUDA device 1
# Device 0: "GeForce 8800 GT"
# Clock rate: 1512000 kilohertz
# Number of multiprocessors: 14
# Number of cores: 112
# Device 1: "GeForce 8800 GT"
# Clock rate: 1512000 kilohertz
# Number of multiprocessors: 14
# Number of cores: 112
# Device 2: "GeForce 8800 GT"
# Clock rate: 1512000 kilohertz
# Number of multiprocessors: 14
# Number of cores: 112
# Device 3: "GeForce 8800 GT"
# Clock rate: 1512000 kilohertz
# Number of multiprocessors: 14
# Number of cores: 112
# Using CUDA device 1
# Device 0: "GeForce 8800 GT"
# Clock rate: 1512000 kilohertz
# Number of multiprocessors: 14
# Number of cores: 112
# Device 1: "GeForce 8800 GT"
# Clock rate: 1512000 kilohertz
# Number of multiprocessors: 14
# Number of cores: 112
# Device 2: "GeForce 8800 GT"
# Clock rate: 1512000 kilohertz
# Number of multiprocessors: 14
# Number of cores: 112
# Device 3: "GeForce 8800 GT"
# Clock rate: 1512000 kilohertz
# Number of multiprocessors: 14
# Number of cores: 112
Cuda error in file '..\cuda/cutil.h' in line 298 : out of memory.
Memory usage: host: bytes device: bytes
Assertion failed: 0, file ..\cuda/cutil.h, line 298

This application has requested the Runtime to terminate it in an unusual way.
Please contact the application's support team for more information.

</stderr_txt>
]]>

<core_client_version>6.3.21</core_client_version>
<![CDATA[
<message>
The system cannot find the path specified. (0x3) - exit code 3 (0x3)
</message>
<stderr_txt>
# Using CUDA device 2
# Device 0: "GeForce 8800 GT"
# Clock rate: 1512000 kilohertz
# Number of multiprocessors: 14
# Number of cores: 112
# Device 1: "GeForce 8800 GT"
# Clock rate: 1512000 kilohertz
# Number of multiprocessors: 14
# Number of cores: 112
# Device 2: "GeForce 8800 GT"
# Clock rate: 1512000 kilohertz
# Number of multiprocessors: 14
# Number of cores: 112
# Device 3: "GeForce 8800 GT"
# Clock rate: 1512000 kilohertz
# Number of multiprocessors: 14
# Number of cores: 112
MDIO ERROR: cannot open file "restart.coor"
# Using CUDA device 2
# Device 0: "GeForce 8800 GT"
# Clock rate: 1512000 kilohertz
# Number of multiprocessors: 14
# Number of cores: 112
# Device 1: "GeForce 8800 GT"
# Clock rate: 1512000 kilohertz
# Number of multiprocessors: 14
# Number of cores: 112
# Device 2: "GeForce 8800 GT"
# Clock rate: 1512000 kilohertz
# Number of multiprocessors: 14
# Number of cores: 112
# Device 3: "GeForce 8800 GT"
# Clock rate: 1512000 kilohertz
# Number of multiprocessors: 14
# Number of cores: 112
# Using CUDA device 2
# Device 0: "GeForce 8800 GT"
# Clock rate: 1512000 kilohertz
# Number of multiprocessors: 14
# Number of cores: 112
# Device 1: "GeForce 8800 GT"
# Clock rate: 1512000 kilohertz
# Number of multiprocessors: 14
# Number of cores: 112
# Device 2: "GeForce 8800 GT"
# Clock rate: 1512000 kilohertz
# Number of multiprocessors: 14
# Number of cores: 112
# Device 3: "GeForce 8800 GT"
# Clock rate: 1512000 kilohertz
# Number of multiprocessors: 14
# Number of cores: 112
Cuda error in file '..\cuda/cutil.h' in line 298 : out of memory.
Memory usage: host: bytes device: bytes
Assertion failed: 0, file ..\cuda/cutil.h, line 298

This application has requested the Runtime to terminate it in an unusual way.
Please contact the application's support team for more information.

</stderr_txt>
]]>

<core_client_version>6.3.21</core_client_version>
<![CDATA[
<message>
Incorrect function. (0x1) - exit code 1 (0x1)
</message>
<stderr_txt>
# Using CUDA device 0
# Device 0: "GeForce 8800 GT"
# Clock rate: 1512000 kilohertz
# Number of multiprocessors: 14
# Number of cores: 112
# Device 1: "GeForce 8800 GT"
# Clock rate: 1512000 kilohertz
# Number of multiprocessors: 14
# Number of cores: 112
# Device 2: "GeForce 8800 GT"
# Clock rate: 1512000 kilohertz
# Number of multiprocessors: 14
# Number of cores: 112
# Device 3: "GeForce 8800 GT"
# Clock rate: 1512000 kilohertz
# Number of multiprocessors: 14
# Number of cores: 112
MDIO ERROR: cannot open file "restart.coor"
# Using CUDA device 0
# Device 0: "GeForce 8800 GT"
# Clock rate: 1512000 kilohertz
# Number of multiprocessors: 14
# Number of cores: 112
# Device 1: "GeForce 8800 GT"
# Clock rate: 1512000 kilohertz
# Number of multiprocessors: 14
# Number of cores: 112
# Device 2: "GeForce 8800 GT"
# Clock rate: 1512000 kilohertz
# Number of multiprocessors: 14
# Number of cores: 112
# Device 3: "GeForce 8800 GT"
# Clock rate: 1512000 kilohertz
# Number of multiprocessors: 14
# Number of cores: 112
# Using CUDA device 0
Cuda error in file 'deviceQuery.cu' in line 59 : out of memory.

</stderr_txt>
]]>

I guess if anyone can say anything helpful regarding your problem it would be GDF. The problem appeared with 6.48-WUs, so it's not related to the new application. Edit: oh.. did you reboot?

Unrelated: I've seen your other host uses 6.3.19 and occasionally gets this error that BOINC quit (likely on a file transfer). I don't have this error any more since I swtiched to 6.3.21. Give it a try, it seems very well-behaved.

MrS

@Marodeur

- can you upgrade to 178.24, which is the latest one?
- 6.42 .. do you mean 6.45? It's been weeks since we switched from 6.45 to 6.48

It's a slowdown because of CPU <1 on Windows. The speed of my GTX 260 on Vista 64 also went down from 33 ms/step to 47 ms/step only because of CPU <1...
Speed on Linux 64 is the same like before.
I already reported that in an email to GDF 4 days ago, but got no reply... :?
____________

pixelicious.at - my little photoblog

It seems to me that my problem is fixed now :)

Two new workunits were started without an intervention from me.
The last driver update to 178.24 was the winner.

Great help, thx again!

It's a slowdown because of CPU <1 on Windows. The speed of my GTX 260 on Vista 64 also went down from 33 ms/step to 47 ms/step only because of CPU <1...
Speed on Linux 64 is the same like before.
I already reported that in an email to GDF 4 days ago, but got no reply... :?

I have been rebooting my machine daily, to try to avoid this problem. It seemed to help for a while. It didn't happen for a couple of weeks anyway. Then last night, it happened again. It's with 6.3.19. Now I have to wait 24 hours until I can get more tasks to crunch.

Me too....another day lost crunching toast, er...another day lost toasting crunchy bagels, hmmmm...


WinXP Pro64
6.3.21
177.84
Is there a solution on the horizon??

____________


Consciousness: That annoying time between naps......

Experience is a wonderful thing: it enables you to recognize a mistake every time you repeat it.

And again, this time with 6.3.21. And another day of crunching lost.

So, nvidia said that they are aware of a leaking problem, but could never reproduce the problem.
Can anyone affected by this issue specify:

OS: (WIN XP, Vista, Linux,etc)
Bit: 32 or 64
Driver: Nvidia driver version
or any other information useful to them to debug it

So, nvidia said that they are aware of a leaking problem, but could never reproduce the problem.
Can anyone affected by this issue specify:

OS: (WIN XP, Vista, Linux,etc)
Bit: 32 or 64
Driver: Nvidia driver version
or any other information useful to them to debug it

Yes. It's here only present on XP 64 bit with BOINC 64 bit. Every WU needs approximately 70 MB of the video memory. This problem is still present since the beginning of the use of his system for PrimeGrid/GPUGrid. I've never seen this problem on my XP 32 bit or Vista 64 bit PCs. When I forget to reboot my XP 64 bit PC every 2 daysw, they crash every WU from the third day on, when I have not further 70,2 MB free for the start of a new WU.

I have this problem with any CUDA enabled driver on my GTX260 card and before on the 8800GT card, which I had replaced.

I have had no issues with out of memory on 64bit server 2008 and windows 7 running the 64bit BOINC. I have used the latest drivers (even beta) soon after they come out, same with the BOINC client. I am currently running driver 180.42 beta with BOINC 6.3.21. Tomorrow I will upgrade both the driver and the client to the latest. I have left server 2008 running for a month straight with no reboot, and still no issues. I also have 4GB ram, DXdiag says that approx 2,298MB is allocated for video use. I have a 8800 GTS 512mb card.

I have riva tuner installed, but I dont know how to see how much memory is being used for an application.

All my GPU Capable Box's are 64-Bit Windows XP-Pro, I made it a Ritual several weeks ago to Stop BOINC and Re-Boot the Box every 1.5 Days. If I wait 2 Days I get stung by the out of memeory error once in awhile so 1.5 days is best for me anyway.

Okay, I'm changing 1 Box over now to 32-Bit but I just Re-Booted a few hours ago so it may take at least 2-3 days before I know or not ...

PS: I don't know what the problem is but the 32-Bit GPU Client refuses to install, I tried where I wanted it installed to & tried where it wanted to install to numerous times. It looks like it's installing but theres nothing there where it should be, I'm lucky I didn't lose the whole Directory as many times as I tried.

I tried on another box & it was a no go too ...

I'm asking about it over in the Boinc Dev Forum now to see what the deal is on that ...

Hi PoorBoy,

I had an old 32 bit Boinc (5.10.X) on my box. So I tried an upgrade to 6.3.21 (32 bit), and it werked.

I just finished loading a GPU task and some other Boinc wu's. I performed a re-boot, and will let it run on it's own for a few days....

WinXP pro x64
6.3.21 (32 bit)
8800GT
177.84 driver

I'm using the existing x64 video driver...as the 32 bit driver will not install due to the OS...

____________


Consciousness: That annoying time between naps......

Experience is a wonderful thing: it enables you to recognize a mistake every time you repeat it.

Hi Bender, I did finally just get a 32-Bit Client to install on 1 Box but it was v5.10.45 which isn't a Cuda Version, strange thing is though it's running 3 GPU Wu's as I type this and none of the regular Wu's, it's only a 8800GT OC on that Box & not a 8800GTx3 OC ... hahaha ... Weird ???

I'm going to shut BOINC down now on that box & see if it will install the Proper 32-Bit Cuda Client v6.3.21 ... Laterz

PS: Got the 32-Bit Version of 6.3.21 to install in the Directory's I wanted the files in so it's running okay now so now we have 2 people trying the 32-Bit Client out on a 64-Bit OS & checking to see if they get the Memory Error or not. 3 GPU Wu's got Toasted in the Process though but I got new Wu's. They probably Erred out because they ran under v5.10.45 for a few Minutes, I didn't think to Suspend them thinking they wouldn't even start up with that version of client.

I Rebooted & will let the Box run for 4-5 Days like that, if it doesn't get a Memory Error by then it Probably won't. I can let it run longer if need be as long as I'm running the regular Wu's I'm running now because they run just as fast with a 32-Bit Client as they do with a 64-Bit Client it looks like ...

LOL.. 5.x doesn't know anything about CUDA, but it seems it's still able to run the code, because you have the CUDA drivers. Interesting :D

@PoorBoy-Edit: to my knowledge ABC is the only project which really benefits from 64 Bit mode. Quite logical, since basically all other projects use floating point instead of integer, where most calculations are double precision anyway (i.e. 64 Bit external, 80 Bit internal).MrS

TankMaster, the problem seems to affect WinXP 64 but not Vista/7/Server2008. I think they use different drivers. Regarding RivaTuner and mem usage:

"Install RivaTuner, click on this strange "button with a triangle" next to the line which tells you the physical details of your card, then search for the hardware monitoring in the list of symbols, which pops up and if it asks, tell it to "handle the plugins automatically". This should enable the vid mem module.. at least it did for me :)"

Just ask if you need any further help.

MrS

All I see is core/shader/mem speeds, core/abient tems, fan RPM/dudty cycle, and supply voltage. No mem usage.

All I see is core/shader/mem speeds, core/abient tems, fan RPM/dudty cycle, and supply voltage. No mem usage.

Then you have to customize that yourself like I had to do, too.
In the hardware monitoring, click on the "setup"-button in the lower right corner.
Here, go to "plugins" in the lower left corner.
Then search for "vidmem.dll" and give it a hook.
When you click OK now, three more monitorings should appear.

All I see is core/shader/mem speeds, core/abient tems, fan RPM/dudty cycle, and supply voltage. No mem usage.

Then you have to customize that yourself like I had to do, too.
In the hardware monitoring, click on the "setup"-button in the lower right corner.
Here, go to "plugins" in the lower left corner.
Then search for "vidmem.dll" and give it a hook.
When you click OK now, three more monitorings should appear.

- Videomemory usage monitoring is not available under Vista due to Vista videomemory virtualization.