This isn't funny --- ok, so it is funny, but only because nothing burned-up:

I've now caught two computers, both Windows, both running Precision X, one running 3x 560 and one 3x 560Ti, resetting my manually set 100% fan speeds back to "auto" but only on ONE card. Really weird. Really strange. In both cases it was the middle (read: hottest) card (Device 1).

Yeah. That happens when a WU fails on my machines too. But since it didn´t restart automatically a new WU, nothing burns. But it will loose all your precision X presets and waste the processing so far. Upseting mode on.

Edit: will say again: can we (ok you, the project guys) change the wus? They aren´t good and are upseting users.

I can confirm, the following:
GTX 570, Nvidea 311.06: 56x2-NOELIA_1MG_RUN1-0-2-RND1781_0 success with 71,288.79 s runtime.
GTX 670, Nvidea 311.06: 97x3-NOELIA_1MG_RUN-0-2-RND9119_0 success with 38,157.74 s runtime.
NOELIA_klebe tasks run on all three computers: GTX650 TI (2GB), GTX 570 and GTX 670 without mayor hickups, exept two failed early on the GTX570, since then no problem.

However I noticed that this NOELIA 1MG and klebe tasks need around 900 to 1350 MB of GPU Memory.

These newer NOELIA klebe tasks seem to be taking longer and longer to finish. The old NOELIAs were 9-10hours. Then it went to 12-13 hours. This latest one is going to be in the 15-16 hour range using 750MB of memory on an MSI660TI PE.

HELP! Nathan, where are you?

He's hiding in the Short queue :)

These newer NOELIA klebe tasks seem to be taking longer and longer to finish. The old NOELIAs were 9-10hours. Then it went to 12-13 hours. This latest one is going to be in the 15-16 hour range using 750MB of memory on an MSI660TI PE.

Don't take it personally, the present Looooong NOELIA WU's don't like anyone.

---

FAQ's

HOW TO:
- Opt out of Beta Tests
- Ask for Help

HELP! Nathan, where are you?

He's on vacation, I see that down clocking my cards a little has helped reduce my error rate. There's only a finite amount of wu's here, we got to bite the bullet and chug through the weekend, I think Nathan will be back on Monday. He should be able to sort things out.

HELP! Nathan, where are you?

He's on vacation, I see that down clocking my cards a little has helped reduce my error rate. There's only a finite amount of wu's here, we got to bite the bullet and chug through the weekend, I think Nathan will be back on Monday. He should be able to sort things out.

If I have read previous posts from Nathan correctly, every scientist does her or his own WU's with different functionality, thus Nathan would not interfere (at least much).
I have my clocks still high and the Noelia's that do not error within the first minutes will finish, but take (a lot) more time. I call them ELR's, exceptional long run's. Even on fast cards (770/690) they took long. I don't mind running them.

---

Greetings from TJ

Nathan helps Noelia all the time, I guess you weren't following about a year ago. On my 680's they take 7.5 hours to 8 hours, that's not long to me. I didn't know you had a GTX770 or 690, how long have you had those? My 770 is identical to my 680's time wise, memory speed doesn't seem to make that much difference.

Guys, not all of us have high-end to super-high-end cards crunching! My 650Ti was able to chew all long WUs within 24h up until these latest NOELIAs (xMG_RUN) appeared, which take me more than 40h!! Not only is the credit low, the risk of losing too much work becomes greater!

What I can't understand is, why do NOELIAs have to consistently be so problematic? Can't they do some debugging and optimizations? If not, why don't they create another queue just for them - named ELR as TJ suggested - so people with the ultra capable cards can chew them and the rest of us continue as usual?

Of course, I know the answer: that queue would be taken up by very few people. Well, that's no excuse for force-feeding ALL of us with hard-to-digest WUs!

---

Nathan helps Noelia all the time, I guess you weren't following about a year ago. On my 680's they take 7.5 hours to 8 hours, that's not long to me. I didn't know you had a GTX770 or 690, how long have you had those? My 770 is identical to my 680's time wise, memory speed doesn't seem to make that much difference.

I haven't, but a 770 is on the way, but one can look at rigs that have them and compare that with Nathan's. That's what I did and saw they took longer.

---

Greetings from TJ

I must admit that I think Vagelis has a point. Being around at this project for a while, I see that only the best (and thus expensive) hardware can cope with the Wu's lately. My GTX285 (a former workhorse) can no longer be used, as it will need 2.5-3 days to finish. The 550Ti is taking almost two days, so is waiting for retirement as well.

I just also figured out that my second 660 sits in a MOBO that has PCIe 1.1 only, so that is (I think) the reason that it takes more than a day to finish.
So staying with this project means that I (we) must invest heavily on hardware.

---

Greetings from TJ

It hasn't just been Noelia's wu's,

http://www.gpugrid.net/forum_thread.php?id=3116#26648

I have watched very carefully all the issues and problems and when I chose components and built my 4 rigs, I went for horse power rather than "bang for the buck", I didn't go all out high end but fairly powerful computers for this precise reason.

The work units are going to progress and become larger, that's inevitable and as newer versions of the Cuda app is released, that will contribute to the wu's becoming larger still. It's going to take time for Noelia to get up to speed, we've gotten spoiled with Nathan because he's very good at this. Every time this happens, someone gets really upset and demands separate queues or that they get pulled and reworked and I know it can be very frustrating when everything is going smooth and then the bottom drops out.

You think I don't mind when a Noelia crashes and takes out a CPDN model that had over 300 hours crunching? It's very frustrating and it doesn't matter what kind of card I have, there's some people that have had to quit all together because the wu's were too big and I've been trying to help someone out by sending him my old cards. I know how you guys feel.

The best "bang for buck" cards were the mid-range cards, at least until the arrival of the GTX770, when the prices started to fall across the range. The GTX 670 might now be the best "bang for buck" card, or not far off (but it depends on the price and they change regularly even in the same country).

You need to make sure you get a 2GB model however! The faster algorithms require more memory and if you don't have the extra memory some WU's fail. There are 1GB and 2GB models of the GTX 650, GTX 650Ti. Anything above that is 2GB or more, with the exception of one OEM GTX 660 which is 1.5GB. Some of the lesser cards such as the GTX 645, GT 645 and GT 640 Rev. 2 are all 1GB only (so not recommended).

Again, Nate has work in the short queue!
If your runs are taking 40h &/or failing, setup a profile to get short tasks for that system. You might find fewer failures and quick returns quite refreshing, despite the credit difference. BTW, it might be the case that some work in the short queue is for a different project and will eventually result in its own publication badge (happened in the past).

In my opinion, some of Noelia's WU's are failing due to a CUDA bug that seems to occasionally raise it's head, but it's better to continue developing than just to give up.

---

FAQ's

HOW TO:
- Opt out of Beta Tests
- Ask for Help

My experience with last Noelia's in my Linux Ubuntu 12.04 with Nvidia drivers 304.43 and two EVGA SC 660GTI cards is satisfactory.

Only once I had what seemed to be a driver crash due probably to be disabling one of the cards during the hottest hours of the day (hot summer here in Madrid) what implies restarting my 6.10.58 boinc manager after changing the cc_config.xml.

Noelia's Klebe take around 10,3 hours and Noelias 1/7MG around 11 hours.

I just have one crunching system with a single mid-range GPU, I'm no mega-cruncher like some of you guys. I may as well work only on the short queue, until these NOELIAs disappear.

The whole thing does leave a bad taste in my mouth, though. My 650Ti on Ubuntu 12.04 crunched NATHAN, NOELIA_klebe, SDOERR WUs in ~18h, it's not like I was on the edge of being obsolete. These specific NOELIAs just KO'ed my card, and I'm pretty sure the vast majority of crunchers here.

I do understand the increasing complexity of the models and the increasing processing power available as time passes, these WUs though seem like a very aggressive step forward. IMHO, the researchers must take into account not only their research goals, but also the average (not the high-end) cruncher's crunching power.

---

You think I don't mind when a Noelia crashes and takes out a CPDN model that had over 300 hours crunching?

Interesting. My CPDN work is done on a different PC than the ones that do GPUGrid, and it looks like I will be keeping it that way. But I haven't really noticed that a Noelia crash takes out anything else (yet) on Win7 64-bit.

IMHO, the researchers must take into account not only their research goals, but also the average (not the high-end) cruncher's crunching power.

They do. That's why there are two queues here at GPUGrid.

IMHO, the researchers must take into account not only their research goals, but also the average (not the high-end) cruncher's crunching power.

They do. That's why there are two queues here at GPUGrid.

I don't want work units to crash, but what I really want is for my cards to be used efficiently. Some projects work too hard to be backward-compatible with older cards that you don't get the full value of your investment in a new card. At that point, I start looking for other projects.

You think I don't mind when a Noelia crashes and takes out a CPDN model that had over 300 hours crunching?

Interesting. My CPDN work is done on a different PC than the ones that do GPUGrid, and it looks like I will be keeping it that way. But I haven't really noticed that a Noelia crash takes out anything else (yet) on Win7 64-bit.

I lost 4 models one day, it was the dreaded "ACEMD.2865P.exe*32 Encountered an error and needs to close", the CPDN models ranged from 328 hours, 256 hours, 198 hours and 73 hours (I wrote them down). It's only happened twice, the other time only got 1 model.

Unfortunately I can empathize with you all too well.
I would prefer these WU's were in a separate queue; Short queue, Long queue, Crashy the WU queue ;)
To be fair I've had 13 Noelia WU's finish and only 2 fail (both within a few minutes, which is a lot better than after 10h). That said I did edit the registry to try to prevent failures.

---

FAQ's

HOW TO:
- Opt out of Beta Tests
- Ask for Help