Why are Downloads So Slow

Message boards : Server and website : Why are Downloads So Slow
Message board moderation

To post messages, you must log in.

Previous · 1 · 2 · 3 · 4 · Next

AuthorMessage
Profile Retvari Zoltan
Avatar

Send message
Joined: 20 Jan 09
Posts: 2380
Credit: 16,897,957,044
RAC: 0
Level
Trp
Scientific publications
watwatwatwatwatwatwatwatwatwatwatwatwatwatwatwatwatwatwatwatwatwatwatwatwatwat
Message 45465 - Posted: 1 Dec 2016, 0:07:17 UTC - in response to Message 45427.  

The server status page is still loads halfway, then stops for a while, then completes.
ID: 45465 · Rating: 0 · rate: Rate + / Rate - Report as offensive     Reply Quote
Toni
Volunteer moderator
Project administrator
Project developer
Project tester
Project scientist

Send message
Joined: 9 Dec 08
Posts: 1006
Credit: 5,068,599
RAC: 0
Level
Ser
Scientific publications
watwatwatwat
Message 45522 - Posted: 2 Dec 2016, 8:49:05 UTC - in response to Message 45465.  

We tweaked some web and db parameters, let's see if anything improves.
ID: 45522 · Rating: 0 · rate: Rate + / Rate - Report as offensive     Reply Quote
Profile Beyond
Avatar

Send message
Joined: 23 Nov 08
Posts: 1112
Credit: 6,162,416,256
RAC: 0
Level
Tyr
Scientific publications
watwatwatwatwatwatwatwatwatwatwatwatwatwatwatwatwatwatwatwatwatwatwatwat
Message 45557 - Posted: 3 Dec 2016, 3:27:39 UTC - in response to Message 45465.  

The server status page is still loads halfway, then stops for a while, then completes.

Toni (re: Zoltan's post), the tweaks didn't help (sadly). Also, downloading WUs is the worst it's ever been.
ID: 45557 · Rating: 0 · rate: Rate + / Rate - Report as offensive     Reply Quote
Toni
Volunteer moderator
Project administrator
Project developer
Project tester
Project scientist

Send message
Joined: 9 Dec 08
Posts: 1006
Credit: 5,068,599
RAC: 0
Level
Ser
Scientific publications
watwatwatwat
Message 45574 - Posted: 3 Dec 2016, 18:52:30 UTC - in response to Message 45557.  

Yes, those download speeds are hard to explain. It has become a long thread in the news. I am inclined to think that they are suffering from the same problem as before, probably not related to the server itself, now aggravated by the increased traffic due to 1 day of stop.

T
ID: 45574 · Rating: 0 · rate: Rate + / Rate - Report as offensive     Reply Quote
Erich56

Send message
Joined: 1 Jan 15
Posts: 1171
Credit: 12,662,148,501
RAC: 1,014,572
Level
Trp
Scientific publications
watwatwatwatwatwatwatwatwat
Message 45578 - Posted: 3 Dec 2016, 19:29:35 UTC - in response to Message 45465.  

The server status page is still loads halfway, then stops for a while, then completes.

any solution for this to be expected soon?
ID: 45578 · Rating: 0 · rate: Rate + / Rate - Report as offensive     Reply Quote
WPrion

Send message
Joined: 30 Apr 13
Posts: 109
Credit: 3,977,737,860
RAC: 575,436
Level
Arg
Scientific publications
watwatwatwatwatwatwatwatwatwatwatwatwatwatwat
Message 45587 - Posted: 4 Dec 2016, 14:33:21 UTC
Last modified: 4 Dec 2016, 14:58:32 UTC

I've read many posts on this issue but I'm sure I've missed a lot, so my apologies in advance if this is a dumb or repeat question.

Is the slow download issue experienced from within the GPUGRID local area network? I mean, don't the GPUGRID folks have some PCs/Workstations running BOINC/GPUGRID to crunch WUs? Are any of these PCs/Workstations connected directly to the same local area network of GPUGRID servers? If so, what do they experience re: download hangs??

Win
ID: 45587 · Rating: 0 · rate: Rate + / Rate - Report as offensive     Reply Quote
Jim1348

Send message
Joined: 28 Jul 12
Posts: 819
Credit: 1,591,285,971
RAC: 0
Level
His
Scientific publications
watwatwatwatwatwatwatwatwatwatwatwatwatwatwat
Message 45589 - Posted: 4 Dec 2016, 16:02:10 UTC - in response to Message 45587.  
Last modified: 4 Dec 2016, 16:14:51 UTC

Is the slow download issue experienced from within the GPUGRID local area network?

The problem is not seen within Spain, but appears to be with their connections to the outside world, as Richard Haselgrove reports see #45570).
http://www.gpugrid.net/forum_thread.php?id=4442&nowrap=true#45570
ID: 45589 · Rating: 0 · rate: Rate + / Rate - Report as offensive     Reply Quote
klepel

Send message
Joined: 23 Dec 09
Posts: 189
Credit: 4,813,881,008
RAC: 42,053
Level
Arg
Scientific publications
watwatwatwatwatwatwatwatwatwatwatwatwatwatwatwatwatwatwatwatwatwatwatwatwat
Message 45597 - Posted: 5 Dec 2016, 15:15:37 UTC

Just like to inform, the problem with stalled downloads still exists for me after the server up-grade.
ID: 45597 · Rating: 0 · rate: Rate + / Rate - Report as offensive     Reply Quote
Profile skgiven
Volunteer moderator
Volunteer tester
Avatar

Send message
Joined: 23 Apr 09
Posts: 3968
Credit: 1,995,359,260
RAC: 0
Level
His
Scientific publications
watwatwatwatwatwatwatwatwatwatwatwatwatwatwatwatwatwatwatwatwatwatwatwatwatwatwatwat
Message 45606 - Posted: 6 Dec 2016, 20:12:56 UTC - in response to Message 45574.  

Yes, those download speeds are hard to explain. It has become a long thread in the news. I am inclined to think that they are suffering from the same problem as before, probably not related to the server itself, now aggravated by the increased traffic due to 1 day of stop.

T


Toni, did you upgrade the NAS storage as well, or just the server?
FAQ's

HOW TO:
- Opt out of Beta Tests
- Ask for Help
ID: 45606 · Rating: 0 · rate: Rate + / Rate - Report as offensive     Reply Quote
Profile Beyond
Avatar

Send message
Joined: 23 Nov 08
Posts: 1112
Credit: 6,162,416,256
RAC: 0
Level
Tyr
Scientific publications
watwatwatwatwatwatwatwatwatwatwatwatwatwatwatwatwatwatwatwatwatwatwatwat
Message 45609 - Posted: 7 Dec 2016, 7:40:52 UTC - in response to Message 45574.  

Yes, those download speeds are hard to explain. It has become a long thread in the news. I am inclined to think that they are suffering from the same problem as before, probably not related to the server itself, now aggravated by the increased traffic due to 1 day of stop.

T

Now there's hardly any work moving up or down and the DL stalling is worse than its ever been.
ID: 45609 · Rating: 0 · rate: Rate + / Rate - Report as offensive     Reply Quote
Rich

Send message
Joined: 27 Nov 14
Posts: 1
Credit: 859,512,671
RAC: 0
Level
Glu
Scientific publications
watwatwatwatwatwatwatwatwatwat
Message 45623 - Posted: 8 Dec 2016, 17:32:07 UTC

I haven't had any problems uploading the results to the server but downloads are the worst I have ever seen. Almost everyone is stalling, like the packet has been lost and it sits there waiting to time out. Most of the time it takes 3 or more retries before completing it. While trying to disable and then re-enable the network communications it took over a dozen times before one file was downloaded. Then the next file the same thing. I understand not having work units to process but the downloading problem, well that is another story.
ID: 45623 · Rating: 0 · rate: Rate + / Rate - Report as offensive     Reply Quote
Profile caffeineyellow5
Avatar

Send message
Joined: 30 Jul 14
Posts: 225
Credit: 2,658,976,345
RAC: 0
Level
Phe
Scientific publications
watwatwatwatwatwatwatwat
Message 45632 - Posted: 8 Dec 2016, 21:51:57 UTC

I think there is value looking into the "inside Spain vs outside Spain" angle. Many of these traceroutes have an increase in ping time starting when the connection gets from some ISP or backbone to the Spanish ISP or backbone. Also the user who reports no issues at all is the one inside Spain. I would be interested in seeing if the internal network computers running alpha, beta, regular, or testing units are seeing any issues like we are seeing. I would also be interested in seeing if any of the students or members of staff (etc) have PCs at home in Spain that could test the "inside Spain" theory for us outside of the local network. And perhaps it is the ISP within Spain that is just not good at the handoff to other ISPs through the backbones or some filtering method being used on the Spanish ISP networks for packet capture and analyzation. I would think that would more prevalent in places like Turkey, China, and Pakistan where it is a known issue that the government is tracking all the traffic in and out of the country to censor it, but maybe Spain has something in place that does something for some reason having this effect on our connections to the project???

I also do see different sounding reports from users within the same country, but on different ISPs or areas of that country. Some Comcast users in remote areas seeing increased issues than Comcast users in populated ares, but still reporting the issues are only with this project. I would wonder if some of those with the worst download and upload issues are also getting a greater number of failed tasks due to files that had dropped packets and seemed complete, but had some issue that caused the failures.

I will try the traceroute and downloading that file in both Firefox and BitCommet to see if there is any difference, failures, or interruptive delays.

One added thing I will add as an observation though is the increased reports of tasks that are listed for machines that do not have those tasks. I have one that has a delivery date of today on it to a computer that does not have that task on it. http://www.gpugrid.net/workunit.php?wuid=12186647 I wonder if this and the connection issue is also related, as a "tried to send" issue that one side (server) thinks it was sent and received while the other side (client) knows it got no such WU?
1 Corinthians 9:16 "For though I preach the gospel, I have nothing to glory of: for necessity is laid upon me; yea, woe is unto me, if I preach not the gospel!"
Ephesians 6:18-20, please ;-)
http://tbc-pa.org
ID: 45632 · Rating: 0 · rate: Rate + / Rate - Report as offensive     Reply Quote
CodeRedDewd

Send message
Joined: 11 Nov 09
Posts: 27
Credit: 4,925,174
RAC: 0
Level
Ala
Scientific publications
watwatwatwatwat
Message 45647 - Posted: 10 Dec 2016, 2:44:28 UTC

Everyone is saying it's the speed, but it's not my case in the US. I download fairly quickly....maybe 300KBps, then it totally stops after a few seconds. It's so bad that I can't even keep the work going. The downloads get exponentially longer retries then project backoff.

Another question... Why can I connect to a speed test website in Barcelona to check my speed and download several Megs? Hmmm
ID: 45647 · Rating: 0 · rate: Rate + / Rate - Report as offensive     Reply Quote
CodeRedDewd

Send message
Joined: 11 Nov 09
Posts: 27
Credit: 4,925,174
RAC: 0
Level
Ala
Scientific publications
watwatwatwatwat
Message 45648 - Posted: 10 Dec 2016, 2:52:25 UTC - in response to Message 45632.  

I think there is value looking into the "inside Spain vs outside Spain" angle. Many of these traceroutes have an increase in ping time starting when the connection gets from some ISP or backbone to the Spanish ISP or backbone. Also the user who reports no issues at all is the one inside Spain. I would be interested in seeing if the internal network computers running alpha, beta, regular, or testing units are seeing any issues like we are seeing. I would also be interested in seeing if any of the students or members of staff (etc) have PCs at home in Spain that could test the "inside Spain" theory for us outside of the local network. And perhaps it is the ISP within Spain that is just not good at the handoff to other ISPs through the backbones or some filtering method being used on the Spanish ISP networks for packet capture and analyzation. I would think that would more prevalent in places like Turkey, China, and Pakistan where it is a known issue that the government is tracking all the traffic in and out of the country to censor it, but maybe Spain has something in place that does something for some reason having this effect on our connections to the project???

I also do see different sounding reports from users within the same country, but on different ISPs or areas of that country. Some Comcast users in remote areas seeing increased issues than Comcast users in populated ares, but still reporting the issues are only with this project. I would wonder if some of those with the worst download and upload issues are also getting a greater number of failed tasks due to files that had dropped packets and seemed complete, but had some issue that caused the failures.

I will try the traceroute and downloading that file in both Firefox and BitCommet to see if there is any difference, failures, or interruptive delays.

One added thing I will add as an observation though is the increased reports of tasks that are listed for machines that do not have those tasks. I have one that has a delivery date of today on it to a computer that does not have that task on it. http://www.gpugrid.net/workunit.php?wuid=12186647 I wonder if this and the connection issue is also related, as a "tried to send" issue that one side (server) thinks it was sent and received while the other side (client) knows it got no such WU?



Why is it that I can connect to the Speedtest.net server in Barcelona, a few of them, and get 8.7Mbps that takes several seconds to measure, so it must be a larger file than most I would get from the project server. My ping is 222ms. The test seems to go well without interruption. Is this testing anything relevant?
ID: 45648 · Rating: 0 · rate: Rate + / Rate - Report as offensive     Reply Quote
Profile Beyond
Avatar

Send message
Joined: 23 Nov 08
Posts: 1112
Credit: 6,162,416,256
RAC: 0
Level
Tyr
Scientific publications
watwatwatwatwatwatwatwatwatwatwatwatwatwatwatwatwatwatwatwatwatwatwatwat
Message 45654 - Posted: 10 Dec 2016, 15:57:07 UTC - in response to Message 45647.  
Last modified: 10 Dec 2016, 16:03:43 UTC

Everyone is saying it's the speed, but it's not my case in the US. I download fairly quickly....maybe 300KBps, then it totally stops after a few seconds. It's so bad that I can't even keep the work going. The downloads get exponentially longer retries then project backoff.

Not sure who everyone is, but if they're saying that it's the speed they're wrong. It's not the speed. Personally I think the server is misconfigured. It's interesting that the stalling got much worse when the new server was installed. Hint, hint...

The 3rd post in this thread has some tips that will help a little by reducing the timeout between retries and the transfer timeout:

https://www.gpugrid.net/forum_thread.php?id=4373&nowrap=true#44374

Also, for those who have been saying it's due to high traffic volume. It's not. Even when there's hardly any work available (so barely any UL/DL going on) the stalling is just as bad when we do happen to receive a WU. And again: uploads are not affected, only downloads.
ID: 45654 · Rating: 0 · rate: Rate + / Rate - Report as offensive     Reply Quote
WPrion

Send message
Joined: 30 Apr 13
Posts: 109
Credit: 3,977,737,860
RAC: 575,436
Level
Arg
Scientific publications
watwatwatwatwatwatwatwatwatwatwatwatwatwatwat
Message 45762 - Posted: 16 Dec 2016, 22:39:35 UTC

Jeebus! I have a download that is as of this writing stuck for 4.5 hours - and still clocking!! That's a record for me. Usually the stuck downloads clear out in not more than 1.5-2 hours.

Win
ID: 45762 · Rating: 0 · rate: Rate + / Rate - Report as offensive     Reply Quote
Profile caffeineyellow5
Avatar

Send message
Joined: 30 Jul 14
Posts: 225
Credit: 2,658,976,345
RAC: 0
Level
Phe
Scientific publications
watwatwatwatwatwatwatwat
Message 45764 - Posted: 16 Dec 2016, 23:25:48 UTC - in response to Message 45762.  

I have a download that is as of this writing stuck for 4.5 hours - and still clocking!! That's a record for me. Usually the stuck downloads clear out in not more than 1.5-2 hours.

In the line
<http_transfer_timeout></http_transfer_timeout>
inside the cc_config.xml file which is located (on a Win7/8/10 machine) in the C:\ProgramData\BOINC folder, change the value to 60. That will decrease the amount of time before it determines a timeout, then will try a new connection continuing from the same point in the file. That SHOULD help some of the time length you are seeing there.
1 Corinthians 9:16 "For though I preach the gospel, I have nothing to glory of: for necessity is laid upon me; yea, woe is unto me, if I preach not the gospel!"
Ephesians 6:18-20, please ;-)
http://tbc-pa.org
ID: 45764 · Rating: 0 · rate: Rate + / Rate - Report as offensive     Reply Quote
WPrion

Send message
Joined: 30 Apr 13
Posts: 109
Credit: 3,977,737,860
RAC: 575,436
Level
Arg
Scientific publications
watwatwatwatwatwatwatwatwatwatwatwatwatwatwat
Message 45770 - Posted: 17 Dec 2016, 3:42:48 UTC - in response to Message 45764.  
Last modified: 17 Dec 2016, 3:45:32 UTC


In the line
<http_transfer_timeout></http_transfer_timeout>
inside the cc_config.xml file which is located (on a Win7/8/10 machine) in the C:\ProgramData\BOINC folder, change the value to 60.


I already have that parameter and it is set to 45. Would 60 be an improvement?

While the first hang was in progress, a second WU came in and downloaded "normally", around an hour or so. The slow WU finished downloading after about 5 hours.

OBTW - I now have something I haven't seen in a while - two long runs running concurrently ;-) Yaaaa!!

Thanks
ID: 45770 · Rating: 0 · rate: Rate + / Rate - Report as offensive     Reply Quote
Profile caffeineyellow5
Avatar

Send message
Joined: 30 Jul 14
Posts: 225
Credit: 2,658,976,345
RAC: 0
Level
Phe
Scientific publications
watwatwatwatwatwatwatwat
Message 45774 - Posted: 17 Dec 2016, 11:00:32 UTC - in response to Message 45770.  

I already have that parameter and it is set to 45. Would 60 be an improvement?

From what I have read here, yes, it would be. Apparently anything under 60 has results that are rather unpredictable to counter-productive. 60 seconds is apparently the lowest amount you can enter that will actually be consistently accurate at its set amount.
a second WU came in and downloaded "normally", around an hour or so.

I am not sure I am reading that correctly. Are you saying that to download a WU it normally takes you an hour just to download it? My downloads are usually less than 5 minutes for a WU and that is with a 60 second hang or two on a file or two. Sometimes it can be as long as 7-10 minutes to download if there are several 60 second hangs. Uploads can take at most 20 minutes, but are usually around 10-12 minutes. If a download of a single WU took an hour for me, I would think there was something severely wrong with something.

When the download is actually moving, what speed registers? I get between 300 and 1000 KBps. I find that the faster it registers, the more likely it is to hang. I once did an experiment by self regulating the speed at which BOINC can download to 250 KBps and it didn't hang on any WU downloads at all for that whole week. For the larger files, it was better to have them not hang, but for the smaller files, it ended up taking longer overall for the whole WU. But for an hour to download, that is quite over the top excessive according to my results in good and in bad connection times. So I hope I am reading it incorrectly and you are just saying that after an hour of the other one hanging excessively, this new one downloaded fast.

Let us know. You may have other issues yet to be dealt with or maybe you are just on a dial up modem and we are expecting broadband results from everyone in this day and age. ???
ID: 45774 · Rating: 0 · rate: Rate + / Rate - Report as offensive     Reply Quote
WPrion

Send message
Joined: 30 Apr 13
Posts: 109
Credit: 3,977,737,860
RAC: 575,436
Level
Arg
Scientific publications
watwatwatwatwatwatwatwatwatwatwatwatwatwatwat
Message 45775 - Posted: 17 Dec 2016, 13:08:24 UTC - in response to Message 45774.  

Sometimes they download as you described. Sometimes one file downloads continuously but very slowly - a few bytes at a time - for around an hour. Yesterday's 5 hour dl was a one-off.
ID: 45775 · Rating: 0 · rate: Rate + / Rate - Report as offensive     Reply Quote
Previous · 1 · 2 · 3 · 4 · Next

Message boards : Server and website : Why are Downloads So Slow

©2026 Universitat Pompeu Fabra