Message boards :
Graphics cards (GPUs) :
New Gianni tasks take loooong time... a warning (8-12-16)
Message board moderation
Previous · 1 · 2 · 3 · 4 · Next
| Author | Message |
|---|---|
|
Send message Joined: 29 Apr 16 Posts: 5 Credit: 79,699,134 RAC: 0 Level ![]() Scientific publications ![]() ![]()
|
Just finished a Gianni that took almost two full days on a 960. http://www.gpugrid.net/result.php?resultid=15235099 Took a long time, but it worked and didn't fail. |
caffeineyellow5Send message Joined: 30 Jul 14 Posts: 225 Credit: 2,658,976,345 RAC: 0 Level ![]() Scientific publications ![]() ![]() ![]() ![]() ![]() ![]() ![]()
|
Just finished a Gianni that took almost two full days on a 960. It looks like 38 hours. Good job! When I added that one had errored and there was a high error rate I was not taking into account the error rate being higher when they first release because that is all they have is the fast errors and not the ones that actually can complete yet. Zoltan pointed that out to me above. But as also mentioned, they are fragile, so any power glitch or anything has the potential to cause an error. I have errored out 2 so far, but that isn't even the majority of my errors recently. But when they do error, they cause the system to fail and need a reboot and also affect others running if they are on the same card or system. So being more fragile, I have clocked all the cards on my most problematic system down to zero overclocking above the factory boost and am hoping that helps. I had it that way for 2 days and turned it back up today and went 2 days without error. lol That will slow them down a bit, but they are already going to be over 24 hours, so what is an extra hour to 28-30 anyway? Either way, I don't think the "error rate" on these is an issue UNLESS you have one. At that point, one is too much. The time is the issue and why I put out the warning. |
Retvari ZoltanSend message Joined: 20 Jan 09 Posts: 2380 Credit: 16,897,957,044 RAC: 0 Level ![]() Scientific publications ![]() ![]() ![]() ![]() ![]() ![]() ![]() ![]() ![]() ![]() ![]() ![]() ![]() ![]() ![]() ![]() ![]() ![]() ![]() ![]() ![]() ![]() ![]() ![]() ![]()
|
Yes, but this host have a GTX 980, while the others has GTX 980 Ti's...15h 6m 47s (54.407s) 980Ti/XP the shortest is on an i7-4930K CPU @ 3.40GHz, and the one similar in length to the short one is on an i3-4160 CPU @ 3.60GHz. Is there a difference in settings, usage of other processes, or whatever else that is different between the i7-4930K and the i3-4160 that would make the 3.6Ghz slightly slower than the 3.4Ghz one both on 980TIs (like pcie speed on the mobo, etc)?The i7-4930K is running at 4.4GHz, and 5 CPU tasks are running simultaneously, while the on the i3-4160 no CPU tasks are running. But this not a clean comparison, as I've booted the i3-4160 to Windows 10 to update it to version 1607, and this task was running under Windows 10 for a short period. You can see it in the task's stderr output, as there are different driver versions present. |
|
Send message Joined: 28 Mar 09 Posts: 490 Credit: 11,731,645,728 RAC: 51 Level ![]() Scientific publications ![]() ![]() ![]() ![]() ![]() ![]() ![]() ![]() ![]() ![]() ![]() ![]() ![]() ![]() ![]() ![]() ![]() ![]() ![]() ![]() ![]() ![]() ![]() ![]() ![]() ![]() ![]()
|
Just finished a Gianni that took almost two full days on a 960. The one thing, I noticed is your CPU time is lot lower the the run time: Run time 136,638.91 CPU time 20,414.34 Which indicates to me that you are not using the SWAN_SYNC 1, which can reduce your run time. Click on the link below, the instructions to set this up, are at the bottom of the post: http://www.gpugrid.net/forum_thread.php?id=4346&nowrap=true#44111 |
BeyondSend message Joined: 23 Nov 08 Posts: 1112 Credit: 6,162,416,256 RAC: 0 Level ![]() Scientific publications ![]() ![]() ![]() ![]() ![]() ![]() ![]() ![]() ![]() ![]() ![]() ![]() ![]() ![]() ![]() ![]() ![]() ![]() ![]() ![]() ![]() ![]() ![]()
|
Obviously too late for any bonuses But that is including the 25% bonus. The credit may be ok for fast cards but it's poor for everyone else. On top of that there's more than double the chance of a failure due to power failure/BSD and no completion at all. |
Logan CarrSend message Joined: 12 Aug 15 Posts: 240 Credit: 64,069,811 RAC: 0 Level ![]() Scientific publications ![]() ![]() ![]()
|
Does anyone know how long it takes for projects with these kind of problems to be fixed? (the gianni project) I've been casually lurking and see a lot of people having problems with the gianni project. I have to say I'm at 20 hours with windows xp, 90% gpu usage, and I still have quite a bit to go on the gianni... (only at 53% complete) Thanks. p.s. I'll still post the results once it's done. Cruncher/Learner in progress. |
|
Send message Joined: 5 Jan 09 Posts: 670 Credit: 2,498,095,550 RAC: 0 Level ![]() Scientific publications ![]() ![]() ![]() ![]() ![]() ![]() ![]() ![]() ![]() ![]() ![]() ![]() ![]() ![]() ![]() ![]() ![]() ![]() ![]() ![]() ![]() ![]() ![]() ![]() ![]()
|
Does anyone know how long it takes for projects with these kind of problems to be fixed? (the gianni project) There is no problem Logan just some complaining about length of time to complete and failures due to excessive over clocking probably. |
BeyondSend message Joined: 23 Nov 08 Posts: 1112 Credit: 6,162,416,256 RAC: 0 Level ![]() Scientific publications ![]() ![]() ![]() ![]() ![]() ![]() ![]() ![]() ![]() ![]() ![]() ![]() ![]() ![]() ![]() ![]() ![]() ![]() ![]() ![]() ![]() ![]() ![]()
|
failures due to excessive over clocking probably. The failures have nothing at all to do with overclocking. They're due to an app that can't recover from outages such as power failures. |
|
Send message Joined: 5 Jan 09 Posts: 670 Credit: 2,498,095,550 RAC: 0 Level ![]() Scientific publications ![]() ![]() ![]() ![]() ![]() ![]() ![]() ![]() ![]() ![]() ![]() ![]() ![]() ![]() ![]() ![]() ![]() ![]() ![]() ![]() ![]() ![]() ![]() ![]() ![]()
|
failures due to excessive over clocking probably. There is no proof of that. However, even if it was the case, how many power outages do you have? |
caffeineyellow5Send message Joined: 30 Jul 14 Posts: 225 Credit: 2,658,976,345 RAC: 0 Level ![]() Scientific publications ![]() ![]() ![]() ![]() ![]() ![]() ![]()
|
Zoltan, that's why I asked. Slower CPU and GPU would make the it significantly slower on tasks. Also OS changes may affect things too. Bedrich, I have had issues with swan_sync on every system I have tried it on. I slows all the processes to a point that it makes the system unusable. Most of the systems I access remotely with Teamviewer and I am not sure if remoting in is affected by the setting or if it is a program or setting I have on all the systems, but I have chosen not to use it. While I was using it for that short time a few tasks completed and did not show improvement and in fact were slower across them all. I was not willing to experiment or investigate at that time and just gave up. My memory being what it is, I can only conclude that the problems were worse than the potential benefit for me to not take on the challenge. I like challenges when it comes to PCs usually. Beyond, I am not sure what you are saying, but a 20% bonus would be on the less than 24 hour ones that award 527,100, not the ones over 24 that award 439,250. And if a Gerard or Adria took the same amount of time you would get around 200,000, so there is more awarded for these longer units. Logan, I am not sure if this length issue is considered a problem. The error rate may actually be one though. I think it usually takes one of the forum volunteer moderators to contact someone on the inside to get an issue resolved, which is one reason why we have them to help us and the project and let the scientists and students keep their time on the work. I ask the mods now, if you haven't already, please contact someone about the error issue with these and inquire about shortening the units as well for the sake of our cards and times, or take Beyond's idea of adding a new level of maybe "Very Long Tasks" for new tasks created for the series 10 NVIDIA cards. After I posted the comment about the error rate possibly not being an accurate length of time to tell if they are erroring out more or not I had 4 error out on me across 2 different systems all GIANNI totaling almost 45.75 hours of work before they errored out. Maybe I am noticing it more and brought undue attention to it too early or maybe there is something to it, but would like some feedback as well on if there is a potential issue, if so can it be corrected, and possibly has it already been corrected and we are just erroring out the old broken ones. Thanks. |
BeyondSend message Joined: 23 Nov 08 Posts: 1112 Credit: 6,162,416,256 RAC: 0 Level ![]() Scientific publications ![]() ![]() ![]() ![]() ![]() ![]() ![]() ![]() ![]() ![]() ![]() ![]() ![]() ![]() ![]() ![]() ![]() ![]() ![]() ![]() ![]() ![]() ![]()
|
failures due to excessive over clocking probably. Frequent but usually only for a few seconds. Long enough to wreak havoc with computers. You should be thankful that you live in an area that's more reliable. The proof is that there's about a 50% failure rate when this happens. Zoltan has posted about the problem too. If you won't believe anyone else, maybe you'll believe him. BTW, other than some factory OCs, none of my cards are OCed. In fact some are down-clocked. |
BeyondSend message Joined: 23 Nov 08 Posts: 1112 Credit: 6,162,416,256 RAC: 0 Level ![]() Scientific publications ![]() ![]() ![]() ![]() ![]() ![]() ![]() ![]() ![]() ![]() ![]() ![]() ![]() ![]() ![]() ![]() ![]() ![]() ![]() ![]() ![]() ![]() ![]()
|
Beyond, I am not sure what you are saying, but a 20% bonus would be on the less than 24 hour ones that award 527,100, not the ones over 24 that award 439,250. And if a Gerard or Adria took the same amount of time you would get around 200,000, so there is more awarded for these longer units. As I understand it there's a 50% bonus for completing a WU in under 24 hours (including UL/DL time) and a 25% bonus for under 48 hours. So for instance a 200 credit base rate unit would get 250 credits if completed in 47 hours and 300 credits in 23 hours. Someone please clue me in if I'm mistaken. I ask the mods now, if you haven't already, please contact someone about the error issue with these and inquire about shortening the units as well for the sake of our cards and times, or take Beyond's idea of adding a new level of maybe "Very Long Tasks" for new tasks created for the series 10 NVIDIA cards. After I posted the comment about the error rate possibly not being an accurate length of time to tell if they are erroring out more or not I had 4 error out on me across 2 different systems all GIANNI totaling almost 45.75 hours of work before they errored out. Sorry to hear. It's no fun having large amounts of GPU time wasted. Hopefully the admins will improve the next app's fault tolerance, add a separate queue for super long WUs and also find a way to lower WU the error rate. The larger the WUs become, the more important it is to address these issues. Good for the project and good for their volunteers. |
|
Send message Joined: 29 Apr 16 Posts: 5 Credit: 79,699,134 RAC: 0 Level ![]() Scientific publications ![]() ![]()
|
Will that influence other projects I'm running? I got a 4 core i5, with 3 cores (that is 75%) running CPU WCG tasks and the last one remaining for GPUGRID and POEM@Home (when no GPUGRID are available). |
Logan CarrSend message Joined: 12 Aug 15 Posts: 240 Credit: 64,069,811 RAC: 0 Level ![]() Scientific publications ![]() ![]() ![]()
|
Alright all, thanks for clearing some things up for me. Here's my results: http://www.gpugrid.net/result.php?resultid=15236421 Took about 1 day and 14 hours, but hey, I got a decent amount of credit for how long it took. Hope the result helps someone Cheers, LC Cruncher/Learner in progress. |
BeyondSend message Joined: 23 Nov 08 Posts: 1112 Credit: 6,162,416,256 RAC: 0 Level ![]() Scientific publications ![]() ![]() ![]() ![]() ![]() ![]() ![]() ![]() ![]() ![]() ![]() ![]() ![]() ![]() ![]() ![]() ![]() ![]() ![]() ![]() ![]() ![]() ![]()
|
Took about 1 day and 14 hours, but hey, I got a decent amount of credit for how long it took Not so much. Here's your last GERARD_FXCXCL12RX: Time: 54,279.73 - 53,837.44 - Credits: 267,900.00 Here's the GIANNI_D3C36bCHL: Time: 137,043.38 - 136,562.60 - Credits: 351,400.00 2.5x the time, 1.3x the credits. Add to that: 2.5x the chance for failure due to many unforeseen factors. |
|
Send message Joined: 28 Mar 09 Posts: 490 Credit: 11,731,645,728 RAC: 51 Level ![]() Scientific publications ![]() ![]() ![]() ![]() ![]() ![]() ![]() ![]() ![]() ![]() ![]() ![]() ![]() ![]() ![]() ![]() ![]() ![]() ![]() ![]() ![]() ![]() ![]() ![]() ![]() ![]() ![]()
|
Will that influence other projects I'm running? You would be better off having 2 cores crunching your CPU project, one core supporting your GPU and one core free to run the operating system. |
|
Send message Joined: 5 Jan 09 Posts: 670 Credit: 2,498,095,550 RAC: 0 Level ![]() Scientific publications ![]() ![]() ![]() ![]() ![]() ![]() ![]() ![]() ![]() ![]() ![]() ![]() ![]() ![]() ![]() ![]() ![]() ![]() ![]() ![]() ![]() ![]() ![]() ![]() ![]()
|
failures due to excessive over clocking probably. I am sorry for your power outages, thought that the USA was beyond such things. In this part of the UK we count power outages in YEARS although there was a 2 day one last December due to flooding of a substation which is the longest power outage in my 64 year history, guess we're just lucky. |
BeyondSend message Joined: 23 Nov 08 Posts: 1112 Credit: 6,162,416,256 RAC: 0 Level ![]() Scientific publications ![]() ![]() ![]() ![]() ![]() ![]() ![]() ![]() ![]() ![]() ![]() ![]() ![]() ![]() ![]() ![]() ![]() ![]() ![]() ![]() ![]() ![]() ![]()
|
I am sorry for your power outages, thought that the USA was beyond such things. In this part of the UK we count power outages in YEARS although there was a 2 day one last December due to flooding of a substation which is the longest power outage in my 64 year history, guess we're just lucky. Thanks. Even though some (most likely mentally challenged) claim climate change to be a myth, we've been having crazy storms and frequent torrential downpours (another one just today). Goes great with the neighborhood underground power lines. Animal species previously unknown here have been steadily moving in from the south. Actually the most frequent reason for outages seems to be lightning strikes on the further out above ground lines. It's improved from a couple years ago when there used to be a few seconds outage almost every day at 7am. If you think the USA power grid is suspect, you should get a load of our abysmal internet service (except in big cities and where Google has graced the population). The horrible broadband speeds makes doing GPUGrid even more challenging. Yeah, greedy monopolies are great... :-( I'm crossing my fingers as my next door neighbor is having a new sewer system installed. Last time that happened a ways down the block the idiot contractors cut though the power and phone lines even though they were marked on the ground with bright neon orange paint. Took 3 days to get it fixed. |
|
Send message Joined: 28 Jul 12 Posts: 819 Credit: 1,591,285,971 RAC: 0 Level ![]() Scientific publications ![]() ![]() ![]() ![]() ![]() ![]() ![]() ![]() ![]() ![]() ![]() ![]() ![]() ![]()
|
Actually the most frequent reason for outages seems to be lightning strikes on the further out above ground lines. It's improved from a couple years ago when there used to be a few seconds outage almost every day at 7am. I was forced to start using uninterruptible power supplies when I went with ramdisks and large write caches a few years ago. But the UPS also take care of the brief (less than a second) power glitches we get here in the spring and summer due to switching loads around and lightning strikes. Otherwise, the power is very reliable where I am, but that varies a lot in the U.S. And our power company is now implementing a smart grid for automatically routing around downed power lines, to help isolate the problem. I once had an expert on buried telephone lines tell me that they are just as susceptible to lighting strikes as the overhead lines, since the lighting has no problem finding the best conductor anyplace. However, optical fiber cables have largely solved that problem for the Internet, and it is good where I am, but that varies a lot too. The U.S. is a big country; Europeans don't always realize how different it is from one section to another. (Americans don't always realize it either.) Global Warming will force a lot of investment in infrastructure upgrades though, assuming the affected areas still want access and power, etc. |
|
Send message Joined: 26 Feb 12 Posts: 184 Credit: 222,376,233 RAC: 0 Level ![]() Scientific publications ![]() ![]() ![]() ![]() ![]() ![]() ![]() ![]() ![]() ![]() ![]()
|
A quality UPS would solve that issue if you could do it. We have momentary glitches and surges where I live also. I bit the bullet and put UPSs on all 8 of my DC machines 1 at a time. Even put 1 on my fridge after a surge took out a $600 control board but that's another story. |
©2025 Universitat Pompeu Fabra