Why are Downloads So Slow

Message boards : Server and website : Why are Downloads So Slow
Message board moderation

To post messages, you must log in.

Previous · 1 · 2 · 3 · 4 · Next

AuthorMessage
Jesse Viviano

Send message
Joined: 9 Jan 15
Posts: 8
Credit: 122,350,512
RAC: 0
Level
Cys
Scientific publications
watwatwatwatwatwat
Message 45782 - Posted: 18 Dec 2016, 4:00:01 UTC

Has anyone tried increasing the network interface controller's number of receive buffers to the maximum? I am located in the USA, and I noticed that downloads from LHC@home in Switzerland to me sped up from kilobytes per second to near instant downloads when I bumped up the number of transmit and receive buffers from their default numbers to my NIC's maximum of 2048 buffers for both transmit and receive. Also, access to this website (which is in Spain) seemed vastly improved. Could it be that the default number of receive buffers your NIC is set to use be insufficient for efficient long distance communication over the Internet? It could be that our NICs are dropping packets due to not having enough buffers which forces timeouts and retransmissions, and increasing the number of buffers mitigates the problem. See http://www.intel.com/content/www/us/en/support/network-and-i-o/ethernet-products/000005593.html for some documentation on Intel NICs regarding this setting.
ID: 45782 · Rating: 0 · rate: Rate + / Rate - Report as offensive     Reply Quote
Profile skgiven
Volunteer moderator
Volunteer tester
Avatar

Send message
Joined: 23 Apr 09
Posts: 3968
Credit: 1,995,359,260
RAC: 0
Level
His
Scientific publications
watwatwatwatwatwatwatwatwatwatwatwatwatwatwatwatwatwatwatwatwatwatwatwatwatwatwatwat
Message 45783 - Posted: 18 Dec 2016, 12:46:24 UTC - in response to Message 45782.  

Given the limited info I have to go on, this download issue is an internal network problem at their end but probably beyond the control of the research group. It's probably not the server (replaced). More likely to be a mapped IP to mac address on a L2 switch or router, bad pointer record or something obtuse like insufficient memory in a router, bandwidth caps or just old backbone kit.
Boinc missed an opportunity to integrate with/replicate distributed sharing many years ago, but a lot of good suggestions were never implemented. I wouldn't mind another UK GPUGrid cruncher being able to download an encrypted GPUGrid file from my system, or even for some files to backup to my system. 5 or 10 unpaid people developing in their spare time for dozens of new projects/year and probably over a million users doesn't make much sense to me.
FAQ's

HOW TO:
- Opt out of Beta Tests
- Ask for Help
ID: 45783 · Rating: 0 · rate: Rate + / Rate - Report as offensive     Reply Quote
Jesse Viviano

Send message
Joined: 9 Jan 15
Posts: 8
Credit: 122,350,512
RAC: 0
Level
Cys
Scientific publications
watwatwatwatwatwat
Message 45787 - Posted: 18 Dec 2016, 19:05:55 UTC - in response to Message 45782.  

In my previous message, I should have written "Have any of the users tried increasing their network interface controllers' number of receive buffers to the maximum number allowed by their drivers?" instead of "Has anyone tried increasing the network interface controller's number of receive buffers to the maximum?".
ID: 45787 · Rating: 0 · rate: Rate + / Rate - Report as offensive     Reply Quote
Profile caffeineyellow5
Avatar

Send message
Joined: 30 Jul 14
Posts: 225
Credit: 2,658,976,345
RAC: 0
Level
Phe
Scientific publications
watwatwatwatwatwatwatwat
Message 45791 - Posted: 19 Dec 2016, 6:53:06 UTC - in response to Message 45787.  

In my previous message, I should have written "Have any of the users tried increasing their network interface controllers' number of receive buffers to the maximum number allowed by their drivers?" instead of "Has anyone tried increasing the network interface controller's number of receive buffers to the maximum?".

I changed the Transmit buffers and the Receive Buffers on the laptop to see if it makes a difference. Worth a try.
ID: 45791 · Rating: 0 · rate: Rate + / Rate - Report as offensive     Reply Quote
Profile caffeineyellow5
Avatar

Send message
Joined: 30 Jul 14
Posts: 225
Credit: 2,658,976,345
RAC: 0
Level
Phe
Scientific publications
watwatwatwatwatwatwatwat
Message 45793 - Posted: 19 Dec 2016, 19:55:58 UTC - in response to Message 45791.  

I just got a Gerrard Long on my laptop and watched it download completely in less than 30 seconds! It is running now. Hmmmmmmm.......
ID: 45793 · Rating: 0 · rate: Rate + / Rate - Report as offensive     Reply Quote
Stefan
Project administrator
Project developer
Project tester
Project scientist

Send message
Joined: 5 Mar 13
Posts: 348
Credit: 0
RAC: 0
Level

Scientific publications
wat
Message 45809 - Posted: 21 Dec 2016, 9:57:57 UTC

Don't worry, it's not just you. This issue is driving us all nuts. We have a publication which depends on us providing a stable webserver and it's not even possible to get that... I have no clue what's going on but it's beyong my control.
ID: 45809 · Rating: 0 · rate: Rate + / Rate - Report as offensive     Reply Quote
Profile Zarck

Send message
Joined: 16 Aug 08
Posts: 145
Credit: 328,473,995
RAC: 0
Level
Asp
Scientific publications
watwatwatwatwatwatwatwatwatwatwatwatwatwatwatwatwatwatwatwatwat
Message 45828 - Posted: 22 Dec 2016, 0:56:53 UTC

ID: 45828 · Rating: 0 · rate: Rate + / Rate - Report as offensive     Reply Quote
Peter Ingham

Send message
Joined: 24 Jun 09
Posts: 2
Credit: 59,497,281
RAC: 0
Level
Thr
Scientific publications
watwatwatwatwatwatwat
Message 45847 - Posted: 22 Dec 2016, 21:43:23 UTC - in response to Message 44995.  

I also had errors with the stated command line. Boinc 7.6.33 x64

Got it to work with:

"C:\Program Files\BOINC\boincmgr.exe" /b "--pers_retry_delay_max 60"
ID: 45847 · Rating: 0 · rate: Rate + / Rate - Report as offensive     Reply Quote
Peter Ingham

Send message
Joined: 24 Jun 09
Posts: 2
Credit: 59,497,281
RAC: 0
Level
Thr
Scientific publications
watwatwatwatwatwatwat
Message 45849 - Posted: 22 Dec 2016, 21:49:31 UTC - in response to Message 45809.  
Last modified: 22 Dec 2016, 22:24:52 UTC

Over the last few days, this has got REALLY bad for me.

One download was at about 1MB after 2 days (default parameters).

Not seeing any packet loss with a ping test.

Coming from New Zealand, I have high latency (350ms) which might be contributing if the cause of the problem in that the throughput is initially perceived as too low and triggering the timeout condition.

Hangup is usually occurring after 2-3 seconds of transfer activity.

I have now made changes to the suggested parameters (even more aggressive than suggested - 30sec timeout, 60 sec max retry) and now getting approx 2KB per second, average.

Another guess as to the cause: TCP "Slow start" or flow control parameters??

As my requests come from a static IP address, this may be helpful in diagnosing the problem.
ID: 45849 · Rating: 0 · rate: Rate + / Rate - Report as offensive     Reply Quote
Profile Beyond
Avatar

Send message
Joined: 23 Nov 08
Posts: 1112
Credit: 6,162,416,256
RAC: 0
Level
Tyr
Scientific publications
watwatwatwatwatwatwatwatwatwatwatwatwatwatwatwatwatwatwatwatwatwatwatwat
Message 45870 - Posted: 23 Dec 2016, 17:15:43 UTC - in response to Message 45847.  

I also had errors with the stated command line. Boinc 7.6.33 x64

Got it to work with:

"C:\Program Files\BOINC\boincmgr.exe" /b "--pers_retry_delay_max 60"

That's because you were trying to start boincmgr.exe instead of boinc.exe. In your case the Windows shortcut would contain:

"C:\Program Files\BOINC\boinc.exe" --pers_retry_delay_max 60
ID: 45870 · Rating: 0 · rate: Rate + / Rate - Report as offensive     Reply Quote
Tomas Brada

Send message
Joined: 3 Nov 15
Posts: 38
Credit: 6,768,093
RAC: 0
Level
Ser
Scientific publications
wat
Message 45889 - Posted: 24 Dec 2016, 10:21:27 UTC

There are too many threads about this problem that it is hard to decide where to post, but here are most replies.

The downloads are slow even here on telecom.sk . The trick with suspending and resuming network helps: download speed approaches my wifi speed but in few seconds drops to 10kBs^-1.
I noticed in Wiresharsk that the TCP connection is degraded to data-ack. There are tons of duplicate Acks. Incoming packet size is rock stable at 1420 bytes.

What amazes me that the files are transferred without compression! The files are plain text with decimal encoded numbers and lot's of whitespace (visible in wireshark). Why not use gzip/deflate on it?

Here is my traceroute. The last duplicated lines look very suspicious to me.

traceroute to 84.89.134.145 (84.89.134.145), 30 hops max, 60 byte packets
 1  192.168.0.1 (192.168.0.1)  2.416 ms  3.636 ms  4.878 ms
 2  st-static-bckb-189.213-81-234.telecom.sk (213.81.234.189)  7.483 ms  10.608 ms  11.073 ms
 3  st-static-bckb-165.213-81-237.telecom.sk (213.81.237.165)  17.225 ms  17.328 ms  17.434 ms
 4  st-static-srk97.87-197-196.telecom.sk (87.197.196.97)  23.263 ms  28.461 ms  30.906 ms
 5  80.156.160.98 (80.156.160.98)  30.904 ms  31.256 ms  31.588 ms
 6  ae-1-3101.bar1.Madrid2.Level3.net (4.69.210.222)  72.930 ms  56.966 ms  53.664 ms
 7  * ae-1-3101.bar1.Madrid2.Level3.net (4.69.210.222)  57.905 ms  61.126 ms
 8  213.242.113.78 (213.242.113.78)  58.332 ms  57.116 ms  59.481 ms
 9  TELMAD.AE4.uv.rt1.val.red.rediris.es (130.206.245.89)  61.419 ms  63.012 ms  63.987 ms
10  anella-val1-router.red.rediris.es (130.206.211.70)  70.758 ms  71.479 ms  74.027 ms
11  * * *
12  grosso.upf.edu (84.89.134.145)  67.483 ms  65.117 ms  67.104 ms
13  grosso.upf.edu (84.89.134.145)  68.139 ms  69.536 ms  68.875 ms
14  grosso.upf.edu (84.89.134.145)  69.612 ms !X  67.994 ms !X  71.975 ms !X


And traceoute to another host on your net:
traceroute to 84.89.134.149 (84.89.134.149), 30 hops max, 60 byte packets
 1  192.168.0.1 (192.168.0.1)  2.383 ms  2.414 ms  2.758 ms
 2  st-static-bckb-189.213-81-234.telecom.sk (213.81.234.189)  7.282 ms  8.930 ms  9.332 ms
 3  st-static-bckb-165.213-81-237.telecom.sk (213.81.237.165)  12.443 ms  12.806 ms  13.816 ms
 4  st-static-srk22.87-197-198.telecom.sk (87.197.198.22)  21.475 ms  21.942 ms  22.798 ms
 5  80.156.160.98 (80.156.160.98)  20.297 ms  20.648 ms  22.916 ms
 6  ae-1-3101.bar1.Madrid2.Level3.net (4.69.210.222)  68.372 ms  57.487 ms  53.777 ms
 7  ae-1-3101.bar1.Madrid2.Level3.net (4.69.210.222)  56.167 ms  57.161 ms  57.497 ms
 8  213.242.113.78 (213.242.113.78)  54.067 ms  55.685 ms  58.480 ms
 9  TELMAD.AE4.uv.rt1.val.red.rediris.es (130.206.245.89)  62.530 ms  62.889 ms  63.361 ms
10  anella-val1-router.red.rediris.es (130.206.211.70)  70.073 ms  76.882 ms  77.247 ms
11  * * *
12  capablanca.upf.edu (84.89.134.149)  59.068 ms  59.149 ms  59.735 ms
13  capablanca.upf.edu (84.89.134.149)  62.916 ms  59.670 ms  60.177 ms
14  capablanca.upf.edu (84.89.134.149)  62.880 ms !X  64.946 ms !X  65.670 ms !X



ID: 45889 · Rating: 0 · rate: Rate + / Rate - Report as offensive     Reply Quote
Profile bcavnaugh

Send message
Joined: 8 Nov 13
Posts: 56
Credit: 1,002,640,163
RAC: 0
Level
Met
Scientific publications
watwatwatwatwatwatwatwatwatwatwatwat
Message 45896 - Posted: 24 Dec 2016, 17:13:58 UTC
Last modified: 24 Dec 2016, 17:25:46 UTC

Can we pay for a DVD with 100 Tasks ready to load on a few computers?

It now takes over an hour for one task to download.
Turning off and On the network works for on 1%.
After 30 minutes going on and off I did get it downloaded.

Crunching@EVGA The Number One Team in the BOINC Community. Folding@EVGA The Number One Team in the Folding@Home Community.
ID: 45896 · Rating: 0 · rate: Rate + / Rate - Report as offensive     Reply Quote
Killersocke

Send message
Joined: 18 Oct 13
Posts: 53
Credit: 406,647,419
RAC: 0
Level
Gln
Scientific publications
watwatwatwatwatwatwatwatwatwatwatwatwatwatwat
Message 45905 - Posted: 25 Dec 2016, 1:31:52 UTC
Last modified: 25 Dec 2016, 1:33:04 UTC

Dear Masters of Desaster,

I would like to crunch your WU's

But i will not waiting hours of hours for a Error-Free Download.

So i will make a breake until the problem is solved.

Merry Chrismas and a happy new Year @all
ID: 45905 · Rating: 0 · rate: Rate + / Rate - Report as offensive     Reply Quote
Michael

Send message
Joined: 29 Apr 16
Posts: 5
Credit: 79,699,134
RAC: 0
Level
Thr
Scientific publications
watwatwat
Message 45910 - Posted: 25 Dec 2016, 12:08:43 UTC

Same problem here, coming from Australia. It's not about speed, it's about connection being dropped. It downloads fast for about 1 or 2 seconds, and then stops.

I had to babysit a download now suspending and resuming the network activity a billion times until it downloaded. Setting for no more tasks - I'll do this one and let me GPU do something else. Or maybe nothing at all, it's pretty hot this time of year Down Under.

This has nothing to do with the users - this is purely a server or ISP configuration problem on GPUGRID's side.
ID: 45910 · Rating: 0 · rate: Rate + / Rate - Report as offensive     Reply Quote
Profile caffeineyellow5
Avatar

Send message
Joined: 30 Jul 14
Posts: 225
Credit: 2,658,976,345
RAC: 0
Level
Phe
Scientific publications
watwatwatwatwatwatwatwat
Message 45925 - Posted: 26 Dec 2016, 2:38:58 UTC - in response to Message 45910.  
Last modified: 26 Dec 2016, 2:43:19 UTC

This has nothing to do with the users - this is purely a server or ISP configuration problem on GPUGRID's side.

Some things on the user end have helped. Like on the Intel cards, changing the Receive Buffers and Transmit Buffers to max helps get more before it does that timeout. (I cannot figure out how to do this to my Wireless Broadcom 802.11ac Network Adapter, so my main system is still having the issue harder than my laptop now.) That are other things that help like changing the BOINC timeout setting to 60 seconds helps. But yes, the main issue is some misconfiguration at the lab, at the school, or at the ISP, or maybe even at the backbone going into Spain. There is a user inside Spain that says there is no problems for his downloads, but not much follow-up has been done asking that user to test different things for confirmation.
ID: 45925 · Rating: 0 · rate: Rate + / Rate - Report as offensive     Reply Quote
Bedrich Hajek

Send message
Joined: 28 Mar 09
Posts: 490
Credit: 11,850,145,728
RAC: 301,281
Level
Trp
Scientific publications
watwatwatwatwatwatwatwatwatwatwatwatwatwatwatwatwatwatwatwatwatwatwatwatwatwatwatwat
Message 45926 - Posted: 26 Dec 2016, 4:18:08 UTC

Right now, not only am I getting slow WUs downloads on both computers, but the project web pages are loading slowly, and I have 2 ghost WU:

http://www.gpugrid.net/workunit.php?wuid=12226979

http://www.gpugrid.net/workunit.php?wuid=12226371



ID: 45926 · Rating: 0 · rate: Rate + / Rate - Report as offensive     Reply Quote
Profile Zarck

Send message
Joined: 16 Aug 08
Posts: 145
Credit: 328,473,995
RAC: 0
Level
Asp
Scientific publications
watwatwatwatwatwatwatwatwatwatwatwatwatwatwatwatwatwatwatwatwat
Message 45930 - Posted: 26 Dec 2016, 18:02:10 UTC
Last modified: 26 Dec 2016, 18:03:16 UTC

Router Security problems ?

http://kb.netgear.com/000036386/CVE-2016-582384

@+
*_*
ID: 45930 · Rating: 0 · rate: Rate + / Rate - Report as offensive     Reply Quote
[CSF] Aleksey Belkov

Send message
Joined: 26 Dec 13
Posts: 87
Credit: 1,292,358,731
RAC: 0
Level
Met
Scientific publications
watwatwatwatwatwatwatwatwatwatwatwatwatwat
Message 45931 - Posted: 27 Dec 2016, 0:17:07 UTC

Maybe this dump of traffic will help your network staff in determining the problem segment of network.
I'm not an expert on TCP, but judging by the repeated transmission of the same data segments, packets acknowledging receipt of the data simply do not reach the server.
http://forum.rosesguild.com/get.php?id=lmcs9XfG9Kf-6FLcKqMe92
ID: 45931 · Rating: 0 · rate: Rate + / Rate - Report as offensive     Reply Quote
Alan

Send message
Joined: 22 Oct 16
Posts: 1
Credit: 16,204,425
RAC: 0
Level
Pro
Scientific publications
watwatwatwat
Message 45933 - Posted: 27 Dec 2016, 5:30:55 UTC

Hello.The server downloads remain very slow using Boinc on windows, Get a few seconds of fast download then download stops and need to retry download.
Thank you
ID: 45933 · Rating: 0 · rate: Rate + / Rate - Report as offensive     Reply Quote
Profile MJH

Send message
Joined: 12 Nov 07
Posts: 696
Credit: 27,266,655
RAC: 0
Level
Val
Scientific publications
watwat
Message 45975 - Posted: 30 Dec 2016, 10:26:36 UTC

I think we have this problem solved now.

Matt
ID: 45975 · Rating: 0 · rate: Rate + / Rate - Report as offensive     Reply Quote
Previous · 1 · 2 · 3 · 4 · Next

Message boards : Server and website : Why are Downloads So Slow

©2026 Universitat Pompeu Fabra