I have a few large uploads that seem to not be able to upload. in both cases it is the large _9 file that will not go through. each time it tries, it gets a "transient http error" and then immediately backs off.

there is no problem connecting to the project, and more tasks will download, and the smaller tasks upload just fine.

is there a problem on the project side that's preventing the large files from uploading? do you have a limit in place? the affected are 509.74MB and 512.24MB respectively.
____________

just finished another task on the same system, _9 file size is 491MB, and uploaded just fine, while attempting the trouble files >500MB at the same time still returned the transient http error.

definitely seems to be some kind of file size limit on the project side.

would be great to get this looked at so I can report these results and not waste ~30+hours of crunch time.
____________

We've been round this cycle before - quite recently, in fact.

The limit is set in the task specification sent by the project, and applied by the client:

<file>
<name>e3s506_e1s419p0f1179-ADRIA_New_KIXcMyb_HIP_AdaptiveBandit-1-2-RND7999_2_9</name>
<nbytes>0.000000</nbytes>
<max_nbytes>1024000000.000000</max_nbytes>
<status>0</status>
<upload_url>https://www.gpugrid.net/PS3GRID_cgi/file_upload_handler</upload_url>
</file>

no, this is quite different. I've very familiar with the old issue, this is new.

the problem before was related to the file size limit in the client_state file. when this was exceeded, it caused a straight up computation error when attempting to upload. that is NOT happening here. the file upload attempts, and the project responds with a "transient http error" as reported by BOINC. and the file stays stuck in the transfers indefinitely. I've checked the client state already and confirmed that these files are listed with a max upload limit of <max_nbytes>1024000000.000000</max_nbytes>

OK, different. You should be able to see the actual size of the file in question using 'properties' in your equivalent of File Browser, or CLI equivalent. Post that where Toni can see it. It may be quickest for him to compress the file (if it isn't already) - that should buy some time until the next bout of inflation.

Edit - I have two tasks due to upload overnight (UK time). I'll watch out for them in ten hours or so, when I get up.

Edit 2 - you can usually get more detail on "transient http error" by setting the 'http debug' Event Log flag - including the exact wording of the response from the file upload handler.

e5s86_e1s499p0f1068-ADRIA_New_KIXcMyb_HIP_AdaptiveBandit-1-2-RND5913_5_9:
BOINC reported file size: 515.24 MB
Linux reported file size: 540.3 MB

e5s136_e3s165p0f813-ADRIA_New_KIXcMyb_HIP_AdaptiveBandit-1-2-RND0460_5_9
BOINC reported file size: 509.74 MB
Linux reported file size: 534.5 MB

likely differences in the way it's calculated.

I don't think you'll see an issue unless the _9 file ends up being >500MB as reported by BOINC. I had several tasks ~490MB go through fine.

____________

set the http_debug flag but good god it spams so many messages to the log that it nearly brings boincmgr to it's knees on a fast system.

looks like my hypothesis was correct. server side limit rejecting the transfer because it's too big.

likely differences in the way it's calculated.

I don't think you'll see an issue unless the _9 file ends up being >500MB as reported by BOINC. I had several tasks ~490MB go through fine.

Received header from server: HTTP/1.1 413 Request Entity Too Large

definitely seems to be some kind of file size limit on the project side.

<http_1_0>0</http_1_0>

<http_1_0>1</http_1_0>

definitely seems to be some kind of file size limit on the project side.

As an emergency bypass, you could try to open your /etc/boinc-client/cc_config.xml file as Administrator, and edit in <options> section the line

<http_1_0>0</http_1_0>

to

<http_1_0>1</http_1_0>

Re-read config files in BOINC Manager and retry the upload.
Whether it works or not, return cc_config.xml to original values and post your experiences, please.

Then, on server side there is the same size limitation or worse for HTTP V1.0 than for HTTP V1.1.
No more ideas than waiting for the problem to be addresed from server side...

I appreciate the effort :)

But Toni is aware of the problem so hopefully he or someone else at the project can find the setting that needs adjustment and make the change.
____________

I offered him some documentation covering Apache and Nginx - looked easy enough...

Not sure if they manage their own server or rely on their institution's tech support team.

Should be solved.

I offered him some documentation covering Apache and Nginx - looked easy enough...

Not sure if they manage their own server or rely on their institution's tech support team.

I don't think i got the memo...

I got four new tasks on Monday, created and issued around 10:00 UTC. Three finished overnight, and the fourth is uploading as I type.

The troublesome _9 file is shown as 524608164 bytes - 524.6 MB according to Linux (1000 convention), 500.31 MB according to BOINC (1024 convention). So in the danger zone, either way. But it's uploaded and reported - I think we're out of the woods on this one.

Just for fun, I tried gzip on the file: the resulting 'compressed' file was 524.7 MB - slightly bigger. So nothing to be gained there - we're already as efficient as we could hope.

What was the size in MB shown on the upload page?

My 2 trouble files still won’t upload, despite Toni claiming that it has been fixed.
____________

still getting the same error message:

What was the size in MB shown on the upload page?

Thanks, I was just double checking since I never caught a file so close to the 500MB mark (BOINC method). ~500MB was a guess at where the limit was. maybe it's 505? who knows.

all i know is that these two files still will not upload. reporting the same HTTP file size error as before.

e5s86_e1s499p0f1068-ADRIA_New_KIXcMyb_HIP_AdaptiveBandit-1-2-RND5913_5_9:
BOINC reported file size: 515.24 MB
Linux reported file size: 540.3 MB

e5s136_e3s165p0f813-ADRIA_New_KIXcMyb_HIP_AdaptiveBandit-1-2-RND0460_5_9
BOINC reported file size: 509.74 MB
Linux reported file size: 534.5 MB

in 0.5-1.0 day they will hit their 5-day deadlines, and be needlessly resent to other hosts. I have the results, but no way to get them back to the project.
____________

since this was never really fixed it seems, one of my tasks has expired, and been resent to all hosts which errored out immediately. now it's stale with a "too many errors" message.

https://www.gpugrid.net/workunit.php?wuid=27077250

should I bother continuing to try to upload my results? you can have them. just fix the upload issue.

the other task will suffer the same fate in about 12hrs.
____________

I had a look at that workunit. Six libboost failures, one cuda compiler failure, and something that looks like a driver/detection failure. That gives us an order of priority for which bug to fix next.

I've just picked up a resend from WU 27077036. A similar story: one sporadic error (overclocking?), two libboost - and a timeout.

Look at that timeout: host 528201. Oh, Mr. Kevvy, where art thou? 156 libboost errors? You can fix that...

I'll try to get up early on Saturday morning and catch the metrics on my copy of the task - assuming it breaks the file size limit when run on a GTX 1660 super, as it appears to have done on the RTX 2080 Ti.

Edit - I've sent him a PM, but if he's not reading the forums, he may not see the notification. Ian - maybe you could put a note for other users in your team forum, too? It'll save the project both time and money.

I’ve sent him several PMs.

I agree that the libboost issue should be fixed first.
____________

well since the 2nd task got resent (to SevicEnginIC), I decided to just cut my losses, and I aborted the transfer.

to my surprise, both tasks received credit. what? lol. so you don't even need that 500MB file? would have been nice if someone directly said, just abort the transfer and it'll be fine, but I was trying to prevent wasteful recomputation.

maybe the "fix" was to somehow prevent this file from getting so big in the first place, and they didnt change their file size limit on the server at all, which would explain why my files still wouldnt go through.

https://www.gpugrid.net/workunit.php?wuid=27077250
https://www.gpugrid.net/workunit.php?wuid=27077283
____________

Right, just this of your "never uploading" tasks was catched by a GTX 1650 SUPER GPU of mine.
My estimates are that it be finished in a total processing time of 232830 seconds, at about 10 UTC on this next Sunday.
I'm curious to see whether upload behavior is the same.
I'll try to be aware to suspend Network activity in BOIC Manager, to be able to document the exact files size...
And if upload can't progress, I'll follow your same pathway...
Thank you for being always so generous in sharing experiences!

The previously mentioned task finished today at predicted time.
Total processing time: 233077 seconds. Very near to the 232830 seconds predicted when progress was 45,12%
I suspended network activity, and when the task finished, 5 files were set in upload pending state.
I made a backup of all of them from original /var/lib/boinc-client/projects/www.gpugrid.net folder to another one.
Their names and sizes:
e5s86_e1s499p0f1068-ADRIA_New_KIXcMyb_HIP_AdaptiveBandit-1-2-RND5913_6_0 100.068 bytes
e5s86_e1s499p0f1068-ADRIA_New_KIXcMyb_HIP_AdaptiveBandit-1-2-RND5913_6_1 2.307.196 bytes
e5s86_e1s499p0f1068-ADRIA_New_KIXcMyb_HIP_AdaptiveBandit-1-2-RND5913_6_2 2.307.196 bytes
e5s86_e1s499p0f1068-ADRIA_New_KIXcMyb_HIP_AdaptiveBandit-1-2-RND5913_6_9 540.268.472 bytes
e5s86_e1s499p0f1068-ADRIA_New_KIXcMyb_HIP_AdaptiveBandit-1-2-RND5913_6_10 274 bytes

After resuming transfers, all files but _9 one were uploaded.
This file failed to upload exactly the same way reported by Ian&Steve C.

Sun 11 Jul 2021 10:58:47 WEST | GPUGRID | Started upload of e5s86_e1s499p0f1068-ADRIA_New_KIXcMyb_HIP_AdaptiveBandit-1-2-RND5913_6_9
.
Sun 11 Jul 2021 10:58:48 WEST | GPUGRID | [http] [ID#19070] Received header from server: HTTP/1.1 413 Request Entity Too Large
Sun 11 Jul 2021 10:58:48 WEST | GPUGRID | [http] [ID#19070] Received header from server: Date: Sun, 11 Jul 2021 09:58:48 GMT
Sun 11 Jul 2021 10:58:48 WEST | GPUGRID | [http] [ID#19070] Received header from server: Server: Apache/2.4.6 (CentOS) OpenSSL/1.0.1e-fips mod_auth_gssapi/1.3.1 mod_auth_kerb/5.4 mod_fcgid/2.3.9 PHP/5.4.16 mod_wsgi/3.4 Python/2.7.5
Thu 01 Jan 1970 00:00:00 WET | GPUGRID |
Sun 11 Jul 2021 10:58:47 WEST | GPUGRID | [http] [ID#19070] Info: Closing connection 8537