monitor suspend/resume bug in 295/296 drivers

Message boards : News : monitor suspend/resume bug in 295/296 drivers
Message board moderation

To post messages, you must log in.

1 · 2 · 3 · Next

AuthorMessage
Toni
Volunteer moderator
Project administrator
Project developer
Project tester
Project scientist

Send message
Joined: 9 Dec 08
Posts: 1006
Credit: 5,068,599
RAC: 0
Level
Ser
Scientific publications
watwatwatwat
Message 23636 - Posted: 24 Feb 2012, 21:49:03 UTC
Last modified: 25 Feb 2012, 9:55:52 UTC

There are some reports of bugs concerning the latest NVIDIA drivers (failures when monitor goes to sleep). GPUGRID may not be immune to the bug. If it occurs to you, either

* rollback to previous drivers
* or configure the monitor so that it does not turn off
ID: 23636 · Rating: 0 · rate: Rate + / Rate - Report as offensive     Reply Quote
Jacob Klein

Send message
Joined: 11 Oct 08
Posts: 1127
Credit: 1,901,927,545
RAC: 0
Level
His
Scientific publications
watwatwatwatwatwatwatwatwatwatwatwatwatwatwatwatwatwatwatwatwatwatwatwatwatwatwatwatwatwat
Message 23637 - Posted: 24 Feb 2012, 22:26:00 UTC

The following driver sets are bugged for me:
- 295.73 WHQL
- 295.51 Beta

The last driver set that worked for me was:
- 290.53 Beta

For me, the bug affects all 3 of my GPU projects, most of the time making tasks error out immediately:
- GPUGRID.net
- Einstein@Home
- SETI@Home
ID: 23637 · Rating: 0 · rate: Rate + / Rate - Report as offensive     Reply Quote
Profile Michael Goetz
Avatar

Send message
Joined: 2 Mar 09
Posts: 124
Credit: 124,873,744
RAC: 39
Level
Cys
Scientific publications
watwatwatwatwatwatwatwat
Message 23638 - Posted: 24 Feb 2012, 23:12:59 UTC - in response to Message 23636.  

There are some reports of bugs concerning the latest NVIDIA drivers (failures when monitor goes to sleep). GPUGRID may be immune to the bug, but if it occurs to you, rollback to previous drivers.


I wrote the BOINC version of the GeneferCUDA app over at PrimeGrid, and the diagnostics it's spitting out indicate that the CUDA subsystem is completely unavailable when the 295 drivers put a monitor into sleep mode. As far as I can tell, no CUDA program at all, from any project, or even non-BOINC CUDA programs, will be able to work under these circumstances.

I don't know yet which platforms it affects (Windows/Linux/Mac), and I don't know if OpenCL is affected, but I'd be very surprised if the GPUGRID apps worked.

We're advising people to either use an earlier driver, or make sure they've configured their system to never turn the monitors off.
Want to find one of the largest known primes? Try PrimeGrid. Or help cure disease at WCG.

ID: 23638 · Rating: 0 · rate: Rate + / Rate - Report as offensive     Reply Quote
jjwhalen

Send message
Joined: 23 Nov 09
Posts: 29
Credit: 17,591,899
RAC: 0
Level
Pro
Scientific publications
watwatwatwatwatwatwatwatwatwatwatwatwatwatwatwat
Message 23640 - Posted: 25 Feb 2012, 1:05:51 UTC

Does anyone know if nVIDIA is aware of/working on this issue?
ID: 23640 · Rating: 0 · rate: Rate + / Rate - Report as offensive     Reply Quote
Jacob Klein

Send message
Joined: 11 Oct 08
Posts: 1127
Credit: 1,901,927,545
RAC: 0
Level
His
Scientific publications
watwatwatwatwatwatwatwatwatwatwatwatwatwatwatwatwatwatwatwatwatwatwatwatwatwatwatwatwatwat
Message 23642 - Posted: 25 Feb 2012, 5:16:44 UTC - in response to Message 23640.  
Last modified: 25 Feb 2012, 5:20:23 UTC

nVidia has been informed, but there has been no response.

Claggy reported the issue formally to nVidia, on 2/1/2012, using Ref 120201-000013, as posted here:
http://forums.nvidia.com/index.php?showtopic=223426&view=findpost&p=1374585

I reported the issue in their 295.51 Beta drivers thread, on 2/16/2012, here:
http://forums.nvidia.com/index.php?showtopic=221985&view=findpost&p=1370579

I also reported the issue in their 295.73 WHQL drivers thread, on 2/24/2012, here:
http://forums.nvidia.com/index.php?showtopic=223426&view=findpost&p=1374645

We have not heard any response as of yet.
If you know of a more appropriate way to inform them of the problem, or get them to fix it, you're welcome to try it.
ID: 23642 · Rating: 0 · rate: Rate + / Rate - Report as offensive     Reply Quote
Toni
Volunteer moderator
Project administrator
Project developer
Project tester
Project scientist

Send message
Joined: 9 Dec 08
Posts: 1006
Credit: 5,068,599
RAC: 0
Level
Ser
Scientific publications
watwatwatwat
Message 23643 - Posted: 25 Feb 2012, 9:51:09 UTC - in response to Message 23642.  
Last modified: 25 Feb 2012, 9:58:55 UTC

Thanks, Michael and Jacob, for the details.
ID: 23643 · Rating: 0 · rate: Rate + / Rate - Report as offensive     Reply Quote
MarkJ
Volunteer moderator
Volunteer tester

Send message
Joined: 24 Dec 08
Posts: 738
Credit: 200,909,904
RAC: 0
Level
Leu
Scientific publications
watwatwatwatwatwatwatwatwatwatwatwatwatwatwatwatwatwatwatwatwat
Message 23645 - Posted: 25 Feb 2012, 10:48:26 UTC

The message threads over at SETI seem to indicate its the windows driver that has the issue. It has been reported by people using a DVI connected monitor, not sure if a VGA connected monitor also has the problems. It depends on the card and if they are using a DVI to VGA adaptor.
BOINC blog
ID: 23645 · Rating: 0 · rate: Rate + / Rate - Report as offensive     Reply Quote
coldFuSion

Send message
Joined: 22 May 10
Posts: 20
Credit: 85,355,427
RAC: 0
Level
Thr
Scientific publications
watwatwatwatwatwatwatwatwatwat
Message 23681 - Posted: 28 Feb 2012, 1:31:35 UTC - in response to Message 23645.  

The message threads over at SETI seem to indicate its the windows driver that has the issue. It has been reported by people using a DVI connected monitor, not sure if a VGA connected monitor also has the problems. It depends on the card and if they are using a DVI to VGA adaptor.


I use the HDMI connector on my GTX 580's and the issue affected me using both 295.51 beta and 295.73 WHQL drivers.

I have configured power settings to never turn off the monitor and have since completed 4 tasks in a row successfully.
ID: 23681 · Rating: 0 · rate: Rate + / Rate - Report as offensive     Reply Quote
BDDave
Avatar

Send message
Joined: 29 Jul 10
Posts: 8
Credit: 457,945,955
RAC: 0
Level
Gln
Scientific publications
watwatwatwatwatwatwatwatwatwatwatwatwatwatwatwatwatwatwatwatwatwatwatwat
Message 23682 - Posted: 28 Feb 2012, 3:45:32 UTC - in response to Message 23636.  

I’ve rolled back to previous drivers thanks. 3 days of all error on milkyway, SETI, GPUGRID and Einstein. What mess!
Get crunchin!
BDDave

ID: 23682 · Rating: 0 · rate: Rate + / Rate - Report as offensive     Reply Quote
Jacob Klein

Send message
Joined: 11 Oct 08
Posts: 1127
Credit: 1,901,927,545
RAC: 0
Level
His
Scientific publications
watwatwatwatwatwatwatwatwatwatwatwatwatwatwatwatwatwatwatwatwatwatwatwatwatwatwatwatwatwat
Message 23798 - Posted: 6 Mar 2012, 13:21:38 UTC - in response to Message 23642.  
Last modified: 6 Mar 2012, 13:22:55 UTC

I thought I'd chime in with some more information.

If you want to use 295.73 WHQL or 295.51 Beta without CUDA failures:
The workaround for the bug is that you must set the Windows Power Options to "Turn off the display: Never". You may still use a screen saver, and you may still physically turn the monitor off, but you must not let the software power the monitor down... according to my testing.

Also, for anyone trying to reproduce the problem, I have found that the problem occurs when Windows powers off the monitor first, and then BOINC tries to start or resume a CUDA task while the monitor is off. This means that, if you try to reproduce it using tasks that are already running before Windows powers down the monitor, those tasks will not fail. But any tasks that try to start or resume, while the monitor is off, will fail... according to my testing.

Finally, the best news yet, I have been contacted privately by an nVidia employee, who was having trouble recreating the problem. I assisted him, and he can now repro on demand now (it's easiest to repro with Einstein@Home, and he didn't know that the monitor has to power down before BOINC begins CUDA processing), and he will be presenting information to the developers.

I am now going to run 295.73 WHQL with a "Blank" screensaver and "Turn off the display: Never", and try to remember to physically turn the monitor off if I get up for an extended period of time.

Regards,
Jacob Klein
ID: 23798 · Rating: 0 · rate: Rate + / Rate - Report as offensive     Reply Quote
Profile skgiven
Volunteer moderator
Volunteer tester
Avatar

Send message
Joined: 23 Apr 09
Posts: 3968
Credit: 1,995,359,260
RAC: 0
Level
His
Scientific publications
watwatwatwatwatwatwatwatwatwatwatwatwatwatwatwatwatwatwatwatwatwatwatwatwatwatwatwat
Message 23806 - Posted: 6 Mar 2012, 16:44:10 UTC - in response to Message 23798.  

Thanks Jacob, I amended a post in the FAQ - Best configurations for GPUGRID thread to reflect your findings.

Good work, should help many.
FAQ's

HOW TO:
- Opt out of Beta Tests
- Ask for Help
ID: 23806 · Rating: 0 · rate: Rate + / Rate - Report as offensive     Reply Quote
kenlo

Send message
Joined: 25 Jan 11
Posts: 1
Credit: 1,346,771,532
RAC: 243,102
Level
Met
Scientific publications
watwatwatwatwatwatwatwatwatwatwatwatwatwatwatwatwatwatwat
Message 23931 - Posted: 13 Mar 2012, 12:59:46 UTC

I did a rollback to the 285.62 driver and still no work, what do i do now?
ID: 23931 · Rating: 0 · rate: Rate + / Rate - Report as offensive     Reply Quote
Jacob Klein

Send message
Joined: 11 Oct 08
Posts: 1127
Credit: 1,901,927,545
RAC: 0
Level
His
Scientific publications
watwatwatwatwatwatwatwatwatwatwatwatwatwatwatwatwatwatwatwatwatwatwatwatwatwatwatwatwatwat
Message 23948 - Posted: 14 Mar 2012, 2:12:28 UTC - in response to Message 23931.  

What does "still no work" mean?
ID: 23948 · Rating: 0 · rate: Rate + / Rate - Report as offensive     Reply Quote
MarkJ
Volunteer moderator
Volunteer tester

Send message
Joined: 24 Dec 08
Posts: 738
Credit: 200,909,904
RAC: 0
Level
Leu
Scientific publications
watwatwatwatwatwatwatwatwatwatwatwatwatwatwatwatwatwatwatwatwat
Message 23954 - Posted: 14 Mar 2012, 9:55:19 UTC

There is a 296.10 WHQL driver out. According to the SETI guys it still has the sleep mode bug.
BOINC blog
ID: 23954 · Rating: 0 · rate: Rate + / Rate - Report as offensive     Reply Quote
Toni
Volunteer moderator
Project administrator
Project developer
Project tester
Project scientist

Send message
Joined: 9 Dec 08
Posts: 1006
Credit: 5,068,599
RAC: 0
Level
Ser
Scientific publications
watwatwatwat
Message 23956 - Posted: 14 Mar 2012, 11:44:12 UTC - in response to Message 23954.  

Did not see anything CUDA-related in the changelog.
ID: 23956 · Rating: 0 · rate: Rate + / Rate - Report as offensive     Reply Quote
Richard Haselgrove

Send message
Joined: 11 Jul 09
Posts: 1639
Credit: 10,053,468,649
RAC: 1,308,024
Level
Trp
Scientific publications
watwatwatwatwatwatwatwatwatwatwatwatwatwatwatwatwatwatwatwatwatwatwatwatwatwatwat
Message 23963 - Posted: 14 Mar 2012, 20:25:39 UTC - in response to Message 23956.  

Did not see anything CUDA-related in the changelog.

We couldn't see anything either, though we had a good chuckle over some of them.

A new bug ticket has been raised by a SETI developer and acknowledged by a named NVidia staffer.

Einstein are also now in active engagement with NVidia:
http://einstein.phys.uwm.edu/forum_thread.php?id=9307&nowrap=true#116397
ID: 23963 · Rating: 0 · rate: Rate + / Rate - Report as offensive     Reply Quote
JLConawayII

Send message
Joined: 31 May 10
Posts: 48
Credit: 28,893,779
RAC: 0
Level
Val
Scientific publications
watwatwatwatwatwatwatwatwatwatwatwatwat
Message 23964 - Posted: 14 Mar 2012, 20:45:06 UTC
Last modified: 14 Mar 2012, 20:45:23 UTC

The 266.58 are the last drivers that seem to be problem-free, no downclocking bug and obviously no sleep mode bug. AFAIK they support everything up through the GTX 580. Unless you have a game or other software that requires the newer drivers, I would suggest rolling back to those. You will have to do a clean install though and be absolutely sure that no Nvidia software remains on your system before installing them. Otherwise certain core files will remain and you might still get the same issues.
ID: 23964 · Rating: 0 · rate: Rate + / Rate - Report as offensive     Reply Quote
Profile nenym

Send message
Joined: 31 Mar 09
Posts: 137
Credit: 1,429,587,071
RAC: 177,579
Level
Met
Scientific publications
watwatwatwatwatwatwatwatwatwatwatwatwatwatwatwatwatwatwatwatwatwatwatwatwat
Message 23965 - Posted: 14 Mar 2012, 20:55:40 UTC - in response to Message 23964.  

266.58 doesn't work well on Ubuntu with Albert&Einstein and DistrRTgen tasks.
ID: 23965 · Rating: 0 · rate: Rate + / Rate - Report as offensive     Reply Quote
Profile Matman

Send message
Joined: 3 Oct 10
Posts: 2
Credit: 34,005,977
RAC: 0
Level
Val
Scientific publications
watwatwatwatwatwatwatwatwatwatwatwatwatwatwatwat
Message 23998 - Posted: 16 Mar 2012, 20:28:40 UTC

1 am running 296.10 NVidia (WHQL) drivers. Screen saver is set never to turn monitor off or "sleep" system. GPUGRID tasks yield computation errors immediately. SETI and Einstein are functioning without errors. So what's up?
ID: 23998 · Rating: 0 · rate: Rate + / Rate - Report as offensive     Reply Quote
Profile skgiven
Volunteer moderator
Volunteer tester
Avatar

Send message
Joined: 23 Apr 09
Posts: 3968
Credit: 1,995,359,260
RAC: 0
Level
His
Scientific publications
watwatwatwatwatwatwatwatwatwatwatwatwatwatwatwatwatwatwatwatwatwatwatwatwatwatwatwat
Message 24001 - Posted: 16 Mar 2012, 22:41:08 UTC - in response to Message 23998.  

Again, avoid using 295 and 296 drivers.
296.10 fixed nothing and like 295 has been reported as causing errors on several GPU projects. 296.17 is just for Win8 Preview. So no point updating to that either!
FAQ's

HOW TO:
- Opt out of Beta Tests
- Ask for Help
ID: 24001 · Rating: 0 · rate: Rate + / Rate - Report as offensive     Reply Quote
1 · 2 · 3 · Next

Message boards : News : monitor suspend/resume bug in 295/296 drivers

©2025 Universitat Pompeu Fabra