Could somebody please attend to this? I've posted full details of this before, at message 57397.

That was from a Linux machine: I've now got this on a Windows machine.

06/10/2021 08:10:27 | GPUGRID | Started upload of e2s61_e1s44p0f535-ADRIA_AdB_KIXCMYB_HIP-0-2-RND3591_2_9
06/10/2021 08:10:28 | GPUGRID | [http] [ID#15096] Received header from server: Server: Apache/2.4.6 (CentOS) OpenSSL/1.0.1e-fips mod_auth_gssapi/1.3.1 mod_auth_kerb/5.4 mod_fcgid/2.3.9 PHP/5.4.16 mod_wsgi/3.4 Python/2.7.5
06/10/2021 08:10:29 | GPUGRID | [http] [ID#15096] Sent header to server: Content-Length: 535846469
06/10/2021 08:10:29 | GPUGRID | [http] [ID#15096] Received header from server: HTTP/1.1 413 Request Entity Too Large

(WU 27079900)

Obviously, the server doesn't care what operating system is trying to contact it. But it does show that the combination of new applications and new data is capable of producing output files larger than the current server configuration is able to handle.

That's a waste of our resources, and may deprive the project of research results.

Apache HTTP/1.1 413 Request Entity Too Large
Could somebody please attend to this?

+1

I've also been affected once by the same problem, commented at Message #57194

See my reply here: https://www.gpugrid.net/forum_thread.php?id=5239&nowrap=true#57187

When this happened to me, i ended up just aborting the transfer. To my surprise I still got credit for it. But the resend that went to ServicEnginIC exhibited the same issue, and when he aborted the transfer he got an error. I can’t explain the difference in behavior but that’s what happened.

---

See my reply here: https://www.gpugrid.net/forum_thread.php?id=5239&nowrap=true#57187

When this happened to me, i ended up just aborting the transfer. To my surprise I still got credit for it. But the resend that went to ServicEnginIC exhibited the same issue, and when he aborted the transfer he got an error. I can’t explain the difference in behavior but that’s what happened.

some bug/idiosyncrasy on your system ?

PS. 5 successful and currently running #6 and #7 cuda101 tasks on Ampere.

... cuda101 tasks on Ampere.

I am really wondering how come

See my reply here: https://www.gpugrid.net/forum_thread.php?id=5239&nowrap=true#57187

When this happened to me, i ended up just aborting the transfer. To my surprise I still got credit for it. But the resend that went to ServicEnginIC exhibited the same issue, and when he aborted the transfer he got an error. I can’t explain the difference in behavior but that’s what happened.

some bug/idiosyncrasy on your system ?

could be, or on Servic's system. no evidence either way. just observed behavior from two systems running the same task with different results.

---

... cuda101 tasks on Ampere.

I am really wondering how come

technically, one could make a copy of the working 1121 app (it's in the project folder), rename it to the same name as the not-working cuda101 app. and replace the file. in this case, you will have two identical apps, under different names, and BOINC will treat them separately as if they were different (and mark tasks as such). you might have to set the <dont_check_file_sizes>1</dont_check_file_sizes> flag to off in the cc_config.xml file so it doesn't get detected/replaced by the project.

at the very least this could be a workaround for people who are being affected by the wrong app being sent and causing errors. I've done this on other projects before. I haven't tried on GPUGRID so there could be a complication with their wrapper setup. tasks destined for the cuda101 app would pull up the replaced version which is really the cuda1121 app under the hood and it should work.

this is all off-topic for the issue brought up in this thread though.

---

Hi Ian&Steve C.,

I have it done.
In the project folder I found two zip files with a bin folder:
conda-pack.zip.1d5c404efc7b1e2955ff7117efdcb358 new package with nvrtc64_112_0.dll and other files
conda-pack.zip.aeb48fd13371f930209a8d253488c86a old package with nvrtc64_102_0.dll and other files

This bin folder represents acemd3 with many DLLs. I have renamed the old package to *.old and copied the new package under the name of the old package.
It seems to work:
https://www.gpugrid.net/workunit.php?wuid=27081868 ending in about 11:30 h.

---

glad it's working as I theorized :)

---

technically, one could make a copy of the working 1121 app (it's in the project folder), rename it to the same name as the not-working cuda101 app. and replace the file. in this case, you will have two identical apps, under different names, and BOINC will treat them separately as if they were different (and mark tasks as such). you might have to set the <dont_check_file_sizes>1</dont_check_file_sizes> flag to off in the cc_config.xml file so it doesn't get detected/replaced by the project.

I have now done this, but due to lack of new WUs, it cannot be testet :-(

Thanks everyone for keeping the Apache problem at the top of this message board for admin to notice! ;-)

technically, one could make a copy of the working 1121 app (it's in the project folder), rename it to the same name as the not-working cuda101 app. and replace the file. in this case, you will have two identical apps, under different names, and BOINC will treat them separately as if they were different (and mark tasks as such). you might have to set the <dont_check_file_sizes>1</dont_check_file_sizes> flag to off in the cc_config.xml file so it doesn't get detected/replaced by the project.

I have now done this, but due to lack of new WUs, it cannot be testet :-(

I got downloaded a WU with cuda101, it startet and has been running for several minutes now. So far, the steps as described above seem to be successful :-)
Remains just to hope that after about 14 hours, the WU will deliver a valid result.

Apache HTTP/1.1 413 Request Entity Too Large

Thanks everyone for keeping the Apache problem at the top of this message board for admin to notice! ;-)

Regarding the original topic, I see that your mentioned task e2s61_e1s44p0f535-ADRIA_AdB_KIXCMYB_HIP-0-2-RND3591_2 is still stalled trying to upload unsuccessfully.
It will be reaching its deadline tomorrow on 7:39:06 UTC
After that, a new task e2s61_e1s44p0f535-ADRIA_AdB_KIXCMYB_HIP-0-2-RND3591_3 hanging from WU #27079900 will be generated and sent to another user.
And so on, wasting processing time at every new user... unless the problem is solved on server side, or maximum allowed number of tasks is reached, whatever comes first.

Earlier reference at Message #57321

Yes, I can see it uploading on the screen in front of me, and the deadline is 08:39:06 local (British time - UTC+1).

And there it might stay, until 90 days after the first attempt to upload the _9 result file. A dilemma might arise if that machine accumulates seven more refuseniks, because at that point no further work will be requested from this project. I'll cross that bridge when I come to it.

At least, the WU will stay live for a fair time for credit (and science) purposes - unless somebody shortens the resend interval by aborting or crashing their copy.

[http] [ID#116] Sent header to server: Content-Length: 540810854
[http] [ID#116] Sent header to server: Content-Type: application/x-www-form-urlencoded
[http] [ID#116] Sent header to server: Expect: 100-continue
[http] [ID#116] Sent header to server:
[http] [ID#116] Received header from server: HTTP/1.1 413 Request Entity Too Large
[http] [ID#116] Received header from server: Date: Sun, 10 Oct 2021 07:14:13 GMT
[http] [ID#116] Received header from server: Server: Apache/2.4.6 (CentOS) OpenSSL/1.0.1e-fips mod_auth_gssapi/1.3.1 mod_auth_kerb/5.4 mod_fcgid/2.3.9 PHP/5.4.16 mod_wsgi/3.4 Python/2.7.5
[http] [ID#116] Received header from server: Connection: close
[http] [ID#116] Received header from server: Content-Type: text/html; charset=iso-8859-1
[http] [ID#116] Received header from server:
[http_xfer] [ID#116] HTTP: wrote 360 bytes
[http_xfer] [ID#116] HTTP: wrote 527 bytes
[http] [ID#116] Info:  Closing connection 252
[http] [ID#116] Info:  TLSv1.2 (OUT), TLS alert, Client hello (1):
[file_xfer] http op done; retval -224 (permanent HTTP error)
[file_xfer] file transfer status -224 (permanent HTTP error)
Backing off 04:15:23 on upload of e2s67_e1s44p0f1240-ADRIA_AdB_KIXCMYB_HIP-0-2-RND1963_3_9
work fetch suspended by user

I put my hosts back to FAH until it gets fixed. There are many ways to fix it on the project's side.

I probed the actual cutoff point of the transfer length.
I thought that it is exactly 512 MiB (536.870.912 bytes), but it's less, even if I reduce the file lenght by the transfer overhead (482 bytes) I receive the "Request Entity Too Large" error.
It's also less than 536.000.000 bytes:

[http] [ID#122] Sent header to server: Content-Length: 536000482
[http] [ID#122] Sent header to server: Content-Type: application/x-www-form-urlencoded
[http] [ID#122] Sent header to server: Expect: 100-continue
[http] [ID#122] Sent header to server:
[http] [ID#122] Received header from server: HTTP/1.1 413 Request Entity Too Large

So I tried 530.000.000 bytes, and it's uploading:

[http] [ID#123] Sent header to server: Content-Length: 530000482
[http] [ID#123] Sent header to server: Content-Type: application/x-www-form-urlencoded
[http] [ID#123] Sent header to server: Expect: 100-continue
[http] [ID#123] Sent header to server:
[http] [ID#123] Received header from server: HTTP/1.1 100 Continue

I think my result will be invalid, as the result file is truncated.

---------- EDIT ----------
The upload stucked at 98% (505.45MB).

[http] [ID#123] Info:  We are completely uploaded and fine
[http] [ID#123] Received header from server: HTTP/1.1 200 OK
[http] [ID#123] Received header from server: Date: Sun, 10 Oct 2021 07:40:22 GMT
[http] [ID#123] Received header from server: Server: Apache/2.4.6 (CentOS) OpenSSL/1.0.1e-fips mod_auth_gssapi/1.3.1 mod_auth_kerb/5.4 mod_fcgid/2.3.9 PHP/5.4.16 mod_wsgi/3.4 Python/2.7.5
[http] [ID#123] Received header from server: Transfer-Encoding: chunked
[http] [ID#123] Received header from server: Content-Type: text/plain; charset=UTF-8
[http] [ID#123] Received header from server:
[http_xfer] [ID#123] HTTP: wrote 135 bytes
[http] [ID#123] Info:  Connection #260 to host www.gpugrid.net left intact
[file_xfer] http op done; retval 0 (Success)
[error] Error reported by file upload server: EOF on socket read : asked for 16382, got 9536
[file_xfer] parsing upload response: <data_server_reply>    <status>1</status>    <message>EOF on socket read : asked for 16382, got 9536</message></data_server_reply>
[file_xfer] parsing status: -127
[file_xfer] file transfer status -127 (transient upload error)
Temporarily failed upload of e2s67_e1s44p0f1240-ADRIA_AdB_KIXCMYB_HIP-0-2-RND1963_3_9: transient upload error
Backing off 03:20:09 on upload of e2s67_e1s44p0f1240-ADRIA_AdB_KIXCMYB_HIP-0-2-RND1963_3_9

The first thing to do is to establish communication between the research teams and the server administration. The research team's assumption about what the server can handle is clearly false - one side or the other has to make the next move.

And another one: WU 27079900. This one's 535,845,928 bytes

Edit: and I've done two copies of it, on hosts 43404 (Windows) and 132158 (Linux). Projects that use validation wouldn't normally allow that, would they?

And another one: WU 27079900. This one's 535,845,928 bytes

Edit: and I've done two copies of it, on hosts 43404 (Windows) and 132158 (Linux). Projects that use validation wouldn't normally allow that, would they?

I would have thought that was down to whether one_result_per_host_per_wu or one_result_per_user_per_wu had been specified ?

This WU only wants one valid result, if the server says it is valid nothing else matters does it ?

If I see I have one of these tasks that are too big I will abort it. Can't see the point in wasting time on the off-chance that it just might upload, especially when it can be easily fixed by the powers that be.

Regarding the original topic, I see that your mentioned task e2s61_e1s44p0f535-ADRIA_AdB_KIXCMYB_HIP-0-2-RND3591_2 is still stalled trying to upload unsuccessfully.

I was feeling bored, so I tried Retvari Zoltan's method on it.

I tried a size which turned out to be 532,721,637 bytes, roughly half way between his attempts. [I was using a line-based editor, so no point in trying for round numbers]. And - and I think this is crucial - I changed the value stored in client_state.xml to the same value. Just look for an _9 file file with lots of upload attempts - there won't be many around. Just change the nbytes value.

And it uploaded, and the task reported. And, not surprisingly, it was declared invalid with a Validate error.

Which is as it should be. Don't try this at home, kiddies!