Just BACE tasks affected. I've now aborted 14 tasks and half were credited.

really strange, isn't it? What's the criterion for granting credit or not granting credit ???

I now aborted two such BACE tasks which could not upload.
For one I got credit, for the other one it said "upload failure" - real junk :-((( 15 hours on a RTX3070 just for NOTHING :-(((

I just aborted my too large upload and got some credit for it. Missed the 50% bonus because of holding onto it for too long.

Wish I had known that aborting a task may give you credits anyway.

Now better now since we are likely to keep running into this situation because they will never fix the Apache misconfiguration.

When I attempt to Abort two of these WUs, nothing seems to happen at all. Both tasks still show "Uploading" and the Transfers page still shows them at 0%. I have tried "Retry Now" on the Transfers page several times (each) to no avail. Should I instead "Abort Transfer"?

Yes, you want to go to the Transfers page, select the tasks that are in upload backoff and "Abort Transfer"

Way to go GPUgrid. Sure are on a roll lately with the complete utter mess ups.

during last night, two of my machines downloaded and startet "Bace" tasks (first letter in upper case, the following ones in lower case).
In contrast to the failing ones before which were calles "BACE" (all upper case).

Did someone get the "Bace" before and did they succeed, or was their upload file also too large?

The question for me now is: should I abort them?

during last night, two of my machines downloaded and startet "Bace" tasks (first letter in upper case, the following ones in lower case).
In contrast to the failing ones before which were calles "BACE" (all upper case).

Did someone get the "Bace" before and did they succeed, or was their upload file also too large?

The question for me now is: should I abort them?

https://www.gpugrid.net/workunit.php?wuid=27494001

Bace Unit was successful. See above link. If you run them, watch the elapsed time and progress rate, and you will know in a few minutes, how they will go.

during last night, two of my machines downloaded and startet "Bace" tasks (first letter in upper case, the following ones in lower case).
In contrast to the failing ones before which were calles "BACE" (all upper case).

Did someone get the "Bace" before and did they succeed, or was their upload file also too large?

The question for me now is: should I abort them?

https://www.gpugrid.net/workunit.php?wuid=27494001

Bace Unit was successful. See above link. If you run them, watch the elapsed time and progress rate, and you will know in a few minutes, how they will go.

See examples:

This one is running well and should finish ok in a few hours. I am running two units simultaneously:

https://www.gpugrid.net/workunit.php?wuid=27494007


This one was running long and I aborted:

https://www.gpugrid.net/workunit.php?wuid=27492188


This one was probably good and I shouldn't have aborted it:

https://www.gpugrid.net/workunit.php?wuid=27494031


In the BOINC manager, highlight the unit and click the properties button on the left, and its progress rate will tell you whether its good or not.

OK, confirmed - it is still the Apache problem.

Tue 09 May 2023 15:06:15 BST | GPUGRID | [http] [ID#15383] Received header from server: HTTP/1.1 413 Request Entity Too Large
Tue 09 May 2023 15:06:15 BST | GPUGRID | [http] [ID#15383] Received header from server: Date: Tue, 09 May 2023 14:06:15 GMT
Tue 09 May 2023 15:06:15 BST | GPUGRID | [http] [ID#15383] Received header from server: Server: Apache/2.4.6 (CentOS) OpenSSL/1.0.2k-fips mod_auth_gssapi/1.5.1 mod_auth_kerb/5.4 mod_fcgid/2.3.9 PHP/5.4.16 mod_wsgi/3.4 Python/2.7.5

File (the larger of two) is 754.1 MB (Linux decimal), 719.15 MB (Boinc binary).

At this end, we have two choices:

1) Abort the data transfer, as Ian suggests.
2) Wait 90 days for somebody to find the key to the server closet.

Quico?

That's weird. I'll get a look.
But this shouldn't happen so cancel the BACE (uppercase) runs. I'll have a look on how to do it from here.
With the last implentation there shouldn't be such related file-size issues.

All bad/bug jobs should be cancelled by now.

This seems like the clearest web advice:

https://www.keycdn.com/support/413-request-entity-too-large

OK, confirmed - it is still the Apache problem.

Tue 09 May 2023 15:06:15 BST | GPUGRID | [http] [ID#15383] Received header from server: HTTP/1.1 413 Request Entity Too Large
Tue 09 May 2023 15:06:15 BST | GPUGRID | [http] [ID#15383] Received header from server: Date: Tue, 09 May 2023 14:06:15 GMT
Tue 09 May 2023 15:06:15 BST | GPUGRID | [http] [ID#15383] Received header from server: Server: Apache/2.4.6 (CentOS) OpenSSL/1.0.2k-fips mod_auth_gssapi/1.5.1 mod_auth_kerb/5.4 mod_fcgid/2.3.9 PHP/5.4.16 mod_wsgi/3.4 Python/2.7.5

File (the larger of two) is 754.1 MB (Linux decimal), 719.15 MB (Boinc binary).

At this end, we have two choices:

1) Abort the data transfer, as Ian suggests.
2) Wait 90 days for somebody to find the key to the server closet.

Quico?

That's weird. I'll get a look.
But this shouldn't happen so cancel the BACE (uppercase) runs. I'll have a look on how to do it from here.
With the last implentation there shouldn't be such related file-size issues.

All bad/bug jobs should be cancelled by now.

this is not something new from this project. this has been a recurring issue from time to time. seems to pop up about every year or so whenever the result files get so large for one reason or another. so don't feel bad if you are unable to find the setting to fix the file size limit. no one else from the project has been able to for the last several years.

why are the result files so large? 500+MB. that's the root cause of the issue. do you need the data in these files? if not, why are they being created?

---

This files hold the results from the last run, i.e. sample 1 to 70 to start the next run with sample 71 to 140. There are the checkpoint data.

OK, confirmed - it is still the Apache problem.

Tue 09 May 2023 15:06:15 BST | GPUGRID | [http] [ID#15383] Received header from server: HTTP/1.1 413 Request Entity Too Large
Tue 09 May 2023 15:06:15 BST | GPUGRID | [http] [ID#15383] Received header from server: Date: Tue, 09 May 2023 14:06:15 GMT
Tue 09 May 2023 15:06:15 BST | GPUGRID | [http] [ID#15383] Received header from server: Server: Apache/2.4.6 (CentOS) OpenSSL/1.0.2k-fips mod_auth_gssapi/1.5.1 mod_auth_kerb/5.4 mod_fcgid/2.3.9 PHP/5.4.16 mod_wsgi/3.4 Python/2.7.5

File (the larger of two) is 754.1 MB (Linux decimal), 719.15 MB (Boinc binary).

At this end, we have two choices:

1) Abort the data transfer, as Ian suggests.
2) Wait 90 days for somebody to find the key to the server closet.

Quico?

That's weird. I'll get a look.
But this shouldn't happen so cancel the BACE (uppercase) runs. I'll have a look on how to do it from here.
With the last implentation there shouldn't be such related file-size issues.

All bad/bug jobs should be cancelled by now.

this is not something new from this project. this has been a recurring issue from time to time. seems to pop up about every year or so whenever the result files get so large for one reason or another. so don't feel bad if you are unable to find the setting to fix the file size limit. no one else from the project has been able to for the last several years.

why are the result files so large? 500+MB. that's the root cause of the issue. do you need the data in these files? if not, why are they being created?

The heavy files are the .dcd which technically I don't really need to obtain to perform the final free energy calculation but it's necessary in case something weird is happening and we want to revisit those frames. .dcd files contains the information and coordinates of all the system atoms but uncompressed. Since there are other trajectory formats, such as .xtc, that compress this data resulting in much lower filesizes we asked to implement the fileformat into OpenMM. As far as I know this has been implemented in our lab but needs the final approval of the "higher-ups" to get it running and then modify ATM to process trajectory files with .xtc.

Nevertheless, this shouldn't have happened (it run OK in other instances with BACE) and apologise for this.

I resolved my issue by spoofing client_state.xml - I said the over-size file had completed uploading, and that the task was ready to report. The server accepted it as valid.

"ValueError: Energy is NaN" is back quite often :-(

Looks like the Progress bar has stopped working again, all quickly pegged at 100%.

I noticed that too. It was working for a while, then today's work it's back to 100% almost immediately.

Remind yourselves of my explanation at message 60315.

I just notice that on one of my machines 2 "BACE" tasks are being processed, plus a third one is in waiting position.

Will the same problem happen as before - upload file too large? So should I better delete these 3 BACE ?

Edit: also on another machine a BACE is running right now

Why? I am uploading.
task 33495789