Problems uploading completed work units

Message boards : Server and website : Problems uploading completed work units
Message board moderation

To post messages, you must log in.

Previous · 1 · 2

AuthorMessage
Profile skgiven
Volunteer moderator
Volunteer tester
Avatar

Send message
Joined: 23 Apr 09
Posts: 3968
Credit: 1,995,359,260
RAC: 0
Level
His
Scientific publications
watwatwatwatwatwatwatwatwatwatwatwatwatwatwatwatwatwatwatwatwatwatwatwatwatwatwatwat
Message 22596 - Posted: 27 Nov 2011, 10:38:23 UTC - in response to Message 22594.  
Last modified: 27 Nov 2011, 10:43:15 UTC

I think the problem is most likely to do with router/ISP settings; timeout, bandwidth restrictions or contention, which may change during different times of the day. Could you run a speed test, and a Ping Test between your location and Barcalona, and post the resulting images?
FAQ's

HOW TO:
- Opt out of Beta Tests
- Ask for Help
ID: 22596 · Rating: 0 · rate: Rate + / Rate - Report as offensive     Reply Quote
Toni
Volunteer moderator
Project administrator
Project developer
Project tester
Project scientist

Send message
Joined: 9 Dec 08
Posts: 1006
Credit: 5,068,599
RAC: 0
Level
Ser
Scientific publications
watwatwatwat
Message 22597 - Posted: 27 Nov 2011, 11:13:22 UTC - in response to Message 22596.  
Last modified: 27 Nov 2011, 11:24:21 UTC

Many users, especially those with mobile, are blessed by all kinds of "transparent" proxies and packet shaping courtesy of their ISPs.
ID: 22597 · Rating: 0 · rate: Rate + / Rate - Report as offensive     Reply Quote
wiyosaya

Send message
Joined: 22 Nov 09
Posts: 114
Credit: 589,114,683
RAC: 0
Level
Lys
Scientific publications
watwatwatwatwatwatwatwatwatwatwatwatwatwatwatwatwatwatwatwat
Message 22725 - Posted: 17 Dec 2011, 19:45:51 UTC - in response to Message 22596.  

I think the problem is most likely to do with router/ISP settings; timeout, bandwidth restrictions or contention, which may change during different times of the day. Could you run a speed test, and a Ping Test between your location and Barcalona, and post the resulting images?

Using Barcelona as the server, the Ping is 137 ms, download is 9.39 Mbps., upload is 0.75 Mbps. Upload has always seemed slow to GPUGrid; seems that the upload speeds confirm this, and that it is not as a result of GPUGrid.

With Pingtest and Barcelona, Packet loss is 0%, Ping is 143 ms, Jitter is 5 ms. overall grade is "B"

I am going to add the HTTP 1.0 setting to my config file. The problem is still happening after the server upgrade, so that would seem to at least somewhat rule out the server. It took 3 times to upload the _1 file for this result. Perhaps there is a transparent proxy somewhere that is interfering.

To me, the perplexing thing about this this is why is it always only the _1 file which is relatively small in size as compared to at least one of the other result files. Is there something special about this file? Answering these questions about the _1 files may give some clues.

As to the DNS cache, I regularly reboot this machine. It is not on all the time, and I have done nothing to persist the cache to file - if persisting the DNS cache to file is even something that can be done on a non-server version of Windows. In addition, I run a local DNS on my firewall that is on a separate machine that is also regularly rebooted. By regularly I mean that neither machine is usually up more than a few hours - except when I run GPUGrid. Since I set up the DNS on the firewall, I know that it does not persist its cache to file.

I appreciate your help and your patience.


ID: 22725 · Rating: 0 · rate: Rate + / Rate - Report as offensive     Reply Quote
Toni
Volunteer moderator
Project administrator
Project developer
Project tester
Project scientist

Send message
Joined: 9 Dec 08
Posts: 1006
Credit: 5,068,599
RAC: 0
Level
Ser
Scientific publications
watwatwatwat
Message 22727 - Posted: 17 Dec 2011, 20:43:02 UTC - in response to Message 22725.  
Last modified: 17 Dec 2011, 21:04:54 UTC

Hi, some random notes, which unfortunately, I'm afraid, are not going to help:

* Slow upload (eg your rate) is typical of current ADSLs, so the value you see is not strange
* Errors on _1 file is not easy to explain. File _2, for example, should be very similar in size and header.


My guess is that the ISP have put some proxy or, more likely, a packet-dependent filter. For example, they may be mis-recognizing some bytes in the file as a who-knowns-what that they want to filter.

The underlying problem in the broader scheme of things is that, nowadays, ISP tend to see uploads as suspicious by default. After all, internet works like a TV: why should one be uploading stuff? :)

By the way, you may try to search the internet for other complaints by users of the same ISP. I think there are even services to check for net-neutrality. Could you try http://broadband.mpi-sws.org/transparency/ (and/or see how you ISP fares in their list) and post the results?
ID: 22727 · Rating: 0 · rate: Rate + / Rate - Report as offensive     Reply Quote
wiyosaya

Send message
Joined: 22 Nov 09
Posts: 114
Credit: 589,114,683
RAC: 0
Level
Lys
Scientific publications
watwatwatwatwatwatwatwatwatwatwatwatwatwatwatwatwatwatwatwat
Message 22739 - Posted: 19 Dec 2011, 15:49:19 UTC - in response to Message 22727.  

Hi, some random notes, which unfortunately, I'm afraid, are not going to help:

* Slow upload (eg your rate) is typical of current ADSLs, so the value you see is not strange
* Errors on _1 file is not easy to explain. File _2, for example, should be very similar in size and header.


My guess is that the ISP have put some proxy or, more likely, a packet-dependent filter. For example, they may be mis-recognizing some bytes in the file as a who-knowns-what that they want to filter.

The underlying problem in the broader scheme of things is that, nowadays, ISP tend to see uploads as suspicious by default. After all, internet works like a TV: why should one be uploading stuff? :)

By the way, you may try to search the internet for other complaints by users of the same ISP. I think there are even services to check for net-neutrality. Could you try http://broadband.mpi-sws.org/transparency/ (and/or see how you ISP fares in their list) and post the results?

I don't have ADSL, I have cable. Since my upload rates are fine elsewhere, there is something about the path to Barcelona that is likely the problem.

I'm backing out the HTTP 1.0 change. The problem with the _1 file still exists even with that setting. However, as soon as I put that in, uploads for World Community Grid start failing regularly. Without the HTTP 1.0 flag, the WCG uploads are fine.

I'll check the net neutrality site when I get a chance.

Thanks.

ID: 22739 · Rating: 0 · rate: Rate + / Rate - Report as offensive     Reply Quote
Toni
Volunteer moderator
Project administrator
Project developer
Project tester
Project scientist

Send message
Joined: 9 Dec 08
Posts: 1006
Credit: 5,068,599
RAC: 0
Level
Ser
Scientific publications
watwatwatwat
Message 22783 - Posted: 21 Dec 2011, 1:32:15 UTC - in response to Message 22739.  
Last modified: 21 Dec 2011, 1:33:07 UTC

Are recent WUs uploaded smoothly or the problem persists?
ID: 22783 · Rating: 0 · rate: Rate + / Rate - Report as offensive     Reply Quote
wiyosaya

Send message
Joined: 22 Nov 09
Posts: 114
Credit: 589,114,683
RAC: 0
Level
Lys
Scientific publications
watwatwatwatwatwatwatwatwatwatwatwatwatwatwatwatwatwatwatwat
Message 22788 - Posted: 21 Dec 2011, 15:20:53 UTC - in response to Message 22783.  
Last modified: 21 Dec 2011, 15:21:38 UTC

Unfortunately, the problem persists.

I have not yet had a chance to try the net neutrality links you posted previously. I may have a chance to try those tonight, however, if not tonight, I will definitely get a chance this weekend.
ID: 22788 · Rating: 0 · rate: Rate + / Rate - Report as offensive     Reply Quote
Profile robertmiles

Send message
Joined: 16 Apr 09
Posts: 503
Credit: 769,991,668
RAC: 0
Level
Glu
Scientific publications
watwatwatwatwatwatwatwatwatwatwatwatwatwatwatwatwatwatwatwatwatwatwatwatwat
Message 22797 - Posted: 22 Dec 2011, 15:32:21 UTC - in response to Message 22351.  

From FAQ - Best configurations for GPUGRID:
    For Vista and Win7 create the file in this folder, C:\ProgramData\BOINC

    Add the following lines:
      <cc_config>
      <options>
      <report_results_immediately>1</report_results_immediately>
      <http_transfer_timeout>900</http_transfer_timeout>
      </options>
      </cc_config>


Boinc has to be closed then opened again for the changes to take effect, reading does just that, reads them but does not implement the changes.

PS. The cc_config.xml file is not there by default in the Windows versions of Boinc, however it is there by default in Linux versions, and comes with a list of options and log flags.



Is there a way to apply that only to GPUGRID under Vista and Windows 7? Some of the other BOINC projects my computers are participating want to reduce their server loads by having the workunits reported in batches instead of immediately after they finish.
ID: 22797 · Rating: 0 · rate: Rate + / Rate - Report as offensive     Reply Quote
Profile skgiven
Volunteer moderator
Volunteer tester
Avatar

Send message
Joined: 23 Apr 09
Posts: 3968
Credit: 1,995,359,260
RAC: 0
Level
His
Scientific publications
watwatwatwatwatwatwatwatwatwatwatwatwatwatwatwatwatwatwatwatwatwatwatwatwatwatwatwat
Message 22799 - Posted: 22 Dec 2011, 16:40:01 UTC - in response to Message 22797.  

The developers expressed an interested in applying the report_results_immediately option as a project default.

As it's presently only usable as a cc_config option, it would apply to all projects.

Boinc is developing towards improving individual project controls, so perhaps within a couple of months we will start to see this materialize.
I think a feature-rich 7.x client is not too far off.
FAQ's

HOW TO:
- Opt out of Beta Tests
- Ask for Help
ID: 22799 · Rating: 0 · rate: Rate + / Rate - Report as offensive     Reply Quote
wiyosaya

Send message
Joined: 22 Nov 09
Posts: 114
Credit: 589,114,683
RAC: 0
Level
Lys
Scientific publications
watwatwatwatwatwatwatwatwatwatwatwatwatwatwatwatwatwatwatwat
Message 22841 - Posted: 27 Dec 2011, 3:59:15 UTC - in response to Message 22727.  


...
By the way, you may try to search the internet for other complaints by users of the same ISP. I think there are even services to check for net-neutrality. Could you try http://broadband.mpi-sws.org/transparency/ (and/or see how you ISP fares in their list) and post the results?

I've run most of the tests at this site, and each one indicates that my ISP is not shaping traffic.

Interestingly enough, I've been running WU's over the holiday, and I am now noticing that the problem is "no longer limited" to the _1 files. It now seems to be happening randomly with the other files, too. I've never seen that behavior before. It has always been the _1 files only.

ID: 22841 · Rating: 0 · rate: Rate + / Rate - Report as offensive     Reply Quote
wiyosaya

Send message
Joined: 22 Nov 09
Posts: 114
Credit: 589,114,683
RAC: 0
Level
Lys
Scientific publications
watwatwatwatwatwatwatwatwatwatwatwatwatwatwatwatwatwatwatwat
Message 22868 - Posted: 31 Dec 2011, 14:19:30 UTC - in response to Message 22841.  

One note on my last post. Every wu that I have run over the holiday has experienced this problem. Except for one wu that failed on other files, all have repeatedly failed on the _1 file. Note that all along, the file at least partially uploads, then fails. Sometimes, it partially uploads, fails, uploads some more, fails, and the pattern repeats until the file finally uploads. In my opinion, this could be as a result of the same kind of problem that the other project mentioned in this thread experienced where there was a file hanging around on the server.

With the way that BOINC operates, increasing times before retry up to 10 or more hours, completed WUs could experience a substantial delay before the results are reported. Worst case, I could see completed WUs missing the deadline - though this has not yet happened to me. With the time sensitivity of this project, I think this problem is particularly annoying.

I think I have demonstrated that this is not on my end, that my connection is reasonable, and that my ISP is not filtering. In other words, I think that I have done all I can do at this point. Perhaps it is time to take a look at the server end. I'm not sure anything can be done, but it might be worth a look.
ID: 22868 · Rating: 0 · rate: Rate + / Rate - Report as offensive     Reply Quote
Profile robertmiles

Send message
Joined: 16 Apr 09
Posts: 503
Credit: 769,991,668
RAC: 0
Level
Glu
Scientific publications
watwatwatwatwatwatwatwatwatwatwatwatwatwatwatwatwatwatwatwatwatwatwatwatwat
Message 22959 - Posted: 13 Jan 2012, 6:45:48 UTC - in response to Message 22868.  
Last modified: 13 Jan 2012, 6:53:42 UTC

I've read that the transatlantic internet connections are significantly slower than the highest speed on-land internet connections.

Wiyosaya, you're in the US, so won't that slow down any connections you make to GPUGRID?

I've seen some other BOINC projects use enough output files to limit the maximum size of these files to about 4 MB, and not show that problem, so how practical would it be for the developers to add a step for splitting the largest output files into multiple, easily recombined pieces, and then test if that helps get good uploads from users on other continents?
ID: 22959 · Rating: 0 · rate: Rate + / Rate - Report as offensive     Reply Quote
Previous · 1 · 2

Message boards : Server and website : Problems uploading completed work units

©2026 Universitat Pompeu Fabra