https://blog.hubspot.com/website/413-request-entity-too-large?

Thanks :)
____________

We are experiencing multiple issues, e.g. hitting an undocumented upload size limit, and others. Please be patient.

Is this project still going? I rarely get work units anymore.

https://blog.hubspot.com/website/413-request-entity-too-large?

+1 can't upload 27079276

i'm getting peer certificates cannot be authticated with given CA certificates.
____________

Also seeing message:
9/30/2021 10:55:06 AM | GPUGRID | Scheduler request failed: Peer certificate cannot be authenticated with given CA certificates

That message jogged a memory.
Could it be another license expiry like last year about this time?
____________
"Together we crunch
To check out a hunch
And wish all our credit
Could just buy us lunch"

Piasa Tribe - Illini Nation

I doubt it - the licence expiry related to the science application (ACEMD), and the Peer certificate relates to https internet traffic.

But I've just dealt with a Peer certificate error reported for two other projects, on the BOINC web site. So something might be brewing.

Lo and behold, I've got

Nope. Not that issue. I already pinged Toni about the license renewal.
He said this year it is not his responsibility.

Went back through my BOINC logs and my GPUGrid scheduler connections have been working fine and normal.

Wonder what is different with your two hosts?

1) This problem only started happening today - I'll try to find a start time.
2) It only affects Windows hosts - Linux is not affected.

Hourly contact at 14:21 today was good, failed at 15:22 - times are British, UTC+1

I do see the same issue on my Windows host as well. Just happened a few hours ago.

Edit: Adding the project on a Win machine via the BOINC manager also temporarily fails.

If the project is using LetsEncrypt, then apparently there's some fairly widespread issues with that at the moment.

Turned out that an expired https certificate was the culprit. Workarounds as suggested on the main BOINC or WUprop forum worked fine and behavior is back to normal for me.

For Windows:
- make a backup from "C:\Program Files\BOINC\ca-bundle.crt"
- open it as administrator in an editor
it is a text file
- delete "DST Root CA X3" until "-----END CERTIFICATE-----"
- restart boinc
the needed "ISRG Root X1" is already contained


see https://boinc.berkeley.edu/forum_thread.php?id=14413
and https://letsencrypt.org/certificates/
____________

I just tried this on one of my hosts and the upload started up right away.

I copied the edited ca-bundle file to another system and that seems to work too.

Need to go through the rest of them and copy the updated file now.

Yes - and the Windows version on BOINC wasn't ready to handle it. The secondary problem was BOINC's continued use of an old version of OpenSSL, long deprecated.

There's some hope of an emergency release of a replacement Windows client within (possibly) 24 hours. I've also been kicking some butt at GitHub, and I've extracted a sort-of promise that there will be an improved version of BOINC across all platforms by mid-December.

I can't make a copy of the .crt / security file let alone edit it as a test file. OF course I have all the permissions.

Have not been able to get tasks for 4 days

____________

Bibi, thanks for the workaround. Works like a charm.

For Windows:
- make a backup from "C:\Program Files\BOINC\ca-bundle.crt"
- open it as administrator in an editor
it is a text file
- delete "DST Root CA X3" until "-----END CERTIFICATE-----"
- restart boinc
the needed "ISRG Root X1" is already contained


see https://boinc.berkeley.edu/forum_thread.php?id=14413
and https://letsencrypt.org/certificates/

Thanks bibi
Solved my problem immediately!
Candido
____________

For Windows:
- make a backup from "C:\Program Files\BOINC\ca-bundle.crt"
- open it as administrator in an editor
it is a text file
- delete "DST Root CA X3" until "-----END CERTIFICATE-----"
- restart boinc
the needed "ISRG Root X1" is already contained

Fixing the ca-bundle.crt file is certainly a cure for the current problem.

1) An alternative way of fixing it is to download a whole new file from the BOINC workround thread.
2) Make sure that you place the replacement in the correct place for your system. The location C:\Program Files\BOINC\ca-bundle.crt is BOINC's default on Windows, but you had the choice during installation to put it anywhere else you wanted.

Fixing the ca-bundle.crt file is certainly a cure for the current problem.

1) An alternative way of fixing it is to download a whole new file from the BOINC workround thread.
2) Make sure that you place the replacement in the correct place for your system. The location C:\Program Files\BOINC\ca-bundle.crt is BOINC's default on Windows, but you had the choice during installation to put it anywhere else you wanted.

OK, it seems to be a local problem specific to your machine, rather than the general problem affecting all Windows users.

You could get more detailed information by enabling 'http_debug' logging in the Event Log. Set that in BOINC Manager Options -> Event Log options: update the project: turn off the option again. (It produces a lot of output, so unwise to leave it running continuously)

See if you can see the cause of the problem, or post the output relating to GPUGrid here and we can take a look.

I've had to remove the project and re-attach even after replacing the crt file. Others have had to do the same.

I've had to remove the project and re-attach even after replacing the crt file. Others have had to do the same.

You could get more detailed information by enabling 'http_debug' logging in the Event Log. Set that in BOINC Manager Options -> Event Log options: update the project: turn off the option again. (It produces a lot of output, so unwise to leave it running continuously)

You could get more detailed information by enabling 'http_debug' logging in the Event Log. Set that in BOINC Manager Options -> Event Log options: update the project: turn off the option again. (It produces a lot of output, so unwise to leave it running continuously)

See if you can see the cause of the problem, or post the output relating to GPUGrid here and we can take a look.

Yup, that's the one.

14.10.2021 15.15.40 | GPUGRID | [http] [ID#1] Info: SSL certificate problem: unable to get local issuer certificate
14.10.2021 15.15.40 | GPUGRID | [http] HTTP error: Peer certificate cannot be authenticated with given CA certificates

The certificates you're offering (from BOINC) can't get it together with the certificates being offered by GPUGrid.

The quickest solution is to give BOINC a better set of certificates to offer. Edit the bundle, use somebody else's hacked version, or use the newer version available on the BOINC message board.

The 'local problem' I suggested earlier might simply be that some versions of Windows, or some local installations, cache frequently-used files and don't notice that the original source has changed.

You'll have to work through that one on your own system, I'm afraid - the simple file replacement worked on all my machines, without any further action.

Steps I'd suggest, in order:
1) Update the file
2) Check that the file has really updated - not been blocked by an anti-virus program.
3) Restart BOINC
4) Restart the computer
5) Detach from GPUGrid and re-attach (it worked for mmonnin, but I can't see why)

For Windows:
- make a backup from "C:\Program Files\BOINC\ca-bundle.crt"
- open it as administrator in an editor
it is a text file
- delete "DST Root CA X3" until "-----END CERTIFICATE-----"
- restart boinc
the needed "ISRG Root X1" is already contained

This didn't help. Last succesfull contact from a Windows host with project servers was on Sep 30. Linux hosts work fine.

Have anyone tried BOINC manager v7.16.20?
Is this version contain the updated certificates?

---- EDIT ----
I did.
This version has the updated certificate file.

Thanks for the update Retvari! I manually edited and replaced all the certificates on my Win systems earlier. When I have time I will start testing v7.16.20.

As always you can get actual CA bundle at curl project site:

https://curl.se/docs/caextract.html

Or use direct url to latest CA bundle:

https://curl.se/ca/cacert.pem

Steps I'd suggest, in order:
1) Update the file
2) Check that the file has really updated - not been blocked by an anti-virus program.
3) Restart BOINC
4) Restart the computer
5) Detach from GPUGrid and re-attach (it worked for mmonnin, but I can't see why)

Have anyone tried BOINC manager v7.16.20?
Is this version contain the updated certificates?

---- EDIT ----
I did.
This version has the updated certificate file.

Glad you got it sorted in the end.

Just for the record, BOINC v7.16.20 is an update for the whole BOINC package - Manager, client, and all the twiddly bits that glue it all together. It will be the client update that solved this particular problem - the Manager wasn't implicated this time.

Version 7.16.20 seems to be working well on the Windows clients.

For Windows:
- make a backup from "C:\Program Files\BOINC\ca-bundle.crt"
- open it as administrator in an editor
it is a text file
- delete "DST Root CA X3" until "-----END CERTIFICATE-----"
- restart boinc
the needed "ISRG Root X1" is already contained


see https://boinc.berkeley.edu/forum_thread.php?id=14413
and https://letsencrypt.org/certificates/

Just update the entire BOINC package. Don't bother with the certificate file edit when the entire new package just works. BOINC version 7.16.20

https://boinc.berkeley.edu/download_all.php

There is nothing wrong with any of the projects if you just fix the BOINC client in the first place.

My client is up-to-date, but I haven't gotten any jobs from GPUGrid in a very, very long time. I've tried resetting the project, removing and re-adding it, etc. with no luck. In my GPUGrid settings here on the website, I'm enabled to get any task from any project type.

My other projects are getting tasks. Rosetta stopped, but that's because it apparently wants VirtualBox now, but I don't want to install that version of BOINC w/ VB.

OS: Windows 10 64-bit
BOINC: 7.16.20 (x64)

Are there just no projects right now other than the Python beta? Is said beta Linux only?

Are there just no projects right now other than the Python beta? Is said beta Linux only?

That's fine. At least I know it's not just me. :)

https://blog.hubspot.com/website/413-request-entity-too-large?

And another:

Thu 30 Sep 2021 17:11:04 BST | GPUGRID | Started upload of e2s116_e1s44p0f981-ADRIA_AdB_KIXCMYB_HIP-0-2-RND2716_2_9
Thu 30 Sep 2021 17:11:05 BST | GPUGRID | [http] [ID#7007] Received header from server: Server: Apache/2.4.6 (CentOS) OpenSSL/1.0.1e-fips mod_auth_gssapi/1.3.1 mod_auth_kerb/5.4 mod_fcgid/2.3.9 PHP/5.4.16 mod_wsgi/3.4 Python/2.7.5
Thu 30 Sep 2021 17:11:05 BST | GPUGRID | [http] [ID#7007] Sent header to server: Content-Length: 540810054
Thu 30 Sep 2021 17:11:06 BST | GPUGRID | [http] [ID#7007] Received header from server: HTTP/1.1 413 Request Entity Too Large

Task 32645986

You'll need to re-configure Apache.

NET::ERR_CERT_DATE_INVALID
Subject: www.ps3grid.net
Issuer: R3
Expires on: 2021. nov. 27.
Current date: 2021. nov. 28.
I can reach the forum only through http.
All of my hosts are set to https, they can't reach the servers because the certificate has expired.

Is there a way to make boinc overlook certificate problems? S

NET::ERR_CERT_DATE_INVALID
Subject: www.ps3grid.net
Issuer: R3
Expires on: 2021. nov. 27.
Current date: 2021. nov. 28.
I can reach the forum only through http.
All of my hosts are set to https, they can't reach the servers because the certificate has expired.

GPUGRID | Scheduler request failed: SSL peer certificate or SSH remote key was not OK

Please fix it as soon as possible.
It could take days, or even weeks, so I've suspended GPUGrid on my hosts.

It also prevents the upload of completed work:

28/11/2021 08:51:47 | GPUGRID | [http] [ID#27114] Info: TLSv1.2 (OUT), TLS alert, certificate expired (557):
28/11/2021 08:51:47 | GPUGRID | [http] [ID#27114] Info: SSL certificate problem: certificate has expired
28/11/2021 08:51:47 | GPUGRID | [http] HTTP error: SSL peer certificate or SSH remote key was not OK
28/11/2021 08:51:48 | GPUGRID | Temporarily failed upload of e1s741_0-ADRIA_BanditGPCR_APJ_b0-0-1-RND2532_5_10: transient HTTP error
28/11/2021 08:51:48 | GPUGRID | Backing off 00:07:19 on upload of e1s741_0-ADRIA_BanditGPCR_APJ_b0-0-1-RND2532_5_10

Post here when it is fixed
I have subscribed.

I have managed (slowly, carefully) to upload and report completed tasks, by editing the urls in client_state.xml

And I have also managed (even more slowly, even more carefully, and with an error on the first attempt) to download and run new work.

But it's not for the faint-hearted: please restore normal service as soon as possible!

I have managed (slowly, carefully) to upload and report completed tasks, by editing the urls in client_state.xml

Richard wrote:

And I have also managed (even more slowly, even more carefully, and with an error on the first attempt) to download and run new work.

But it's not for the faint-hearted: please restore normal service as soon as possible!

I have managed (slowly, carefully) to upload and report completed tasks, by editing the urls in client_state.xml

I have managed (slowly, carefully) to upload and report completed tasks, by editing the urls in client_state.xml

And I have also managed (even more slowly, even more carefully, and with an error on the first attempt) to download and run new work.

But it's not for the faint-hearted: please restore normal service as soon as possible!

Since then, I've posted full details of the basic technique in message 57912, and possible reasons for the failure to report in message 57930.

Both are in the Stuck on 'uploading' thread, Server and website area. Read the whole thread for context.

Just tried the trick but it did not work. Can't get past 24 hour backoff.

deleted

deleted

The project is back ok now.
Try a reset, or re-attaching and it should pick up, as it did for me just now.
I had a 'fun' few hours forcing a download of 100Mb- but only got to 60% before the plug was pulled server side..
I see that the previous Cuda 1101 zip file is no longer attached -only Cuda101, as I had wondered if this was a factor -apart from file size and expired certs,but probably not.
However the larger Cuda file is still only downloading at ~8KBps, so the unit won't be running for quite a while..

Just try a normal update, or a retry on any stalled transfers, before going any further.

A full project reset shouldn't be needed, and all those extra transfers will just slow down the project recovery for everyone.

Having worked through your excellent instructions Richard, I finally succeeded in getting the pending uploads through. Forgot to set NNW and thus the manager requested 2 new tasks. However, I didn't bother to fix the new work download issue. Had 2 pending downloads that I aborted. This morning when all was fixed, I repeatedly got the scheduler request completed – downloads stalled message and thus reattached the project. Took less than 1.5 min to download all necessary project files and now I am back to business as usual. Only this approach seemed to solve the above annoying scheduler message.

Site is back to normal. No need for funny business or resets.
No more Unsecure message when browsing.
Uploads/Downloads work as intended.

there was a problem with the certificate renewal. Now is fine.

Just try a normal update, or a retry on any stalled transfers, before going any further.

A full project reset shouldn't be needed, and all those extra transfers will just slow down the project recovery for everyone.

Thanks for the kind words. Trying to solve these little problems goes some way to keeping those little grey cells in working order.

I did - many years ago - try to put forward the concept of 'technical moderators' as a specialist position within BOINC: people with technical knowledge who could bridge the gap between the mass of volunteers and the project scientists or administrators. There's a need for people who can decipher [5-year old voice on] Mummeeeee - it's not working! [5-year old voice off] and turn it into a technical description of what needs to be tweaked.

The idea never took off (the project side couldn't see the need), but I've gone on trying to live the dream.

Richard... and because you try there will be a small quite, wonderful place in heaven for you.

Bill F

Ta. Continuing on our theme of 'things the volunteers have noticed, and project admin might like to take a look at...'

We are now running ACEMD3 v2.19, deployed on 10 Nov 2021. The first tasks had a data error, but we've now been running ADRIA_BanditGPCR tasks successfully since Friday 26 November.

The apps come in two flavours, cuda 101 and cuda 1121. I'll let the owners of Ampere cards pursue their own private grief, but I'm worried about the rest of us.

The machines I run here all have GTX 1660 series cards - all modern and efficient, and fast enough to complete these tasks in under 24 hours. Five cards have returned four tasks each, and all twenty have validated.

All machines have tried both cuda101 and cuda1121, and on four out of five cuda1121 is clearly faster than cuda101. The fifth is a bit ambiguous.

That means that BOINC should be moving towards issuing cuda1121 preferentially. In fact, it should have reached that point by now - we must have completed well over 100 tasks globally since this version was launched.

But all four of my 'clear advantage' machines are currently running cuda101, and only the ambiguous one is trying cuda1121 again. That's the wrong way round.

Why? Looking at the details for each of our computers, there's a link for "Application details: Show". That brings up the history for that computer, running each application that its tried.

The crucial lines here are 'Number of tasks completed' and 'Average processing rate'. Once 'Number of tasks completed' reaches 11, the server should compare the APRs and preferentially assign the fastest app for new work, when there's a choice.

But my hosts are showing zero tasks completed, despite the 'Consecutive valid tasks' count being filled in. If the project-global count of completed tasks (which we can't inspect directly) is also not filling in properly, that would explain the bias towards cuda101. But I can't explain why the stats aren't being recorded properly.

Well, I don't know how it happened, but after writing all that yesterday...

Today's rotation has brought me a clean sweep of cuda1121 tasks across all five machines. Coincidence, or a tweak to the server? No way of knowing externally, but it's good news both for the project (the science will be done more quickly) and for the volunteers.

there was a problem with the certificate renewal. Now is fine.

How about the problem that the server does not accept result file sizes above approx. 500 MB and instead of crediting them appropriately throws these in the bin? Has this issue been solved as well?

How about the problem that the server does not accept result file sizes above approx. 500 MB and instead of crediting them appropriately throws these in the bin? Has this issue been solved as well?

Kind of.
The present workunits are much shorter, thus their result file is much shorter (~270MB) as well.