can't upload results. file size too big being blocked?

Message boards : Number crunching : can't upload results. file size too big being blocked?
Message board moderation

To post messages, you must log in.

Previous · 1 · 2

AuthorMessage
Ian&Steve C.

Send message
Joined: 21 Feb 20
Posts: 1116
Credit: 40,839,470,595
RAC: 6,423
Level
Trp
Scientific publications
wat
Message 57168 - Posted: 7 Jul 2021, 13:11:58 UTC

still getting the same error message:
Wed 07 Jul 2021 09:03:01 AM EDT | GPUGRID | [http] [ID#13807] Received header from server: HTTP/1.1 413 Request Entity Too Large



it would be nice to be able to upload these before they expire and get resent.
ID: 57168 · Rating: 0 · rate: Rate + / Rate - Report as offensive     Reply Quote
Richard Haselgrove

Send message
Joined: 11 Jul 09
Posts: 1639
Credit: 10,159,968,649
RAC: 428
Level
Trp
Scientific publications
watwatwatwatwatwatwatwatwatwatwatwatwatwatwatwatwatwatwatwatwatwatwatwatwatwatwat
Message 57169 - Posted: 7 Jul 2021, 14:33:32 UTC - in response to Message 57167.  

What was the size in MB shown on the upload page?

These were the precise details of the file I saw this morning:


BOINC uses the binary notation (MB = 1024*1024 bytes), and reported it as 500.3 MB.

See explanation at https://en.wikipedia.org/wiki/Byte#Multiple-byte_units
ID: 57169 · Rating: 0 · rate: Rate + / Rate - Report as offensive     Reply Quote
Ian&Steve C.

Send message
Joined: 21 Feb 20
Posts: 1116
Credit: 40,839,470,595
RAC: 6,423
Level
Trp
Scientific publications
wat
Message 57171 - Posted: 7 Jul 2021, 15:40:45 UTC - in response to Message 57169.  
Last modified: 7 Jul 2021, 15:43:38 UTC

Thanks, I was just double checking since I never caught a file so close to the 500MB mark (BOINC method). ~500MB was a guess at where the limit was. maybe it's 505? who knows.

all i know is that these two files still will not upload. reporting the same HTTP file size error as before.

e5s86_e1s499p0f1068-ADRIA_New_KIXcMyb_HIP_AdaptiveBandit-1-2-RND5913_5_9:
BOINC reported file size: 515.24 MB
Linux reported file size: 540.3 MB

e5s136_e3s165p0f813-ADRIA_New_KIXcMyb_HIP_AdaptiveBandit-1-2-RND0460_5_9
BOINC reported file size: 509.74 MB
Linux reported file size: 534.5 MB

in 0.5-1.0 day they will hit their 5-day deadlines, and be needlessly resent to other hosts. I have the results, but no way to get them back to the project.
ID: 57171 · Rating: 0 · rate: Rate + / Rate - Report as offensive     Reply Quote
Ian&Steve C.

Send message
Joined: 21 Feb 20
Posts: 1116
Credit: 40,839,470,595
RAC: 6,423
Level
Trp
Scientific publications
wat
Message 57174 - Posted: 8 Jul 2021, 3:51:19 UTC - in response to Message 57171.  

since this was never really fixed it seems, one of my tasks has expired, and been resent to all hosts which errored out immediately. now it's stale with a "too many errors" message.

https://www.gpugrid.net/workunit.php?wuid=27077250

should I bother continuing to try to upload my results? you can have them. just fix the upload issue.

the other task will suffer the same fate in about 12hrs.
ID: 57174 · Rating: 0 · rate: Rate + / Rate - Report as offensive     Reply Quote
Richard Haselgrove

Send message
Joined: 11 Jul 09
Posts: 1639
Credit: 10,159,968,649
RAC: 428
Level
Trp
Scientific publications
watwatwatwatwatwatwatwatwatwatwatwatwatwatwatwatwatwatwatwatwatwatwatwatwatwatwat
Message 57177 - Posted: 8 Jul 2021, 9:41:35 UTC - in response to Message 57174.  
Last modified: 8 Jul 2021, 10:27:56 UTC

I had a look at that workunit. Six libboost failures, one cuda compiler failure, and something that looks like a driver/detection failure. That gives us an order of priority for which bug to fix next.

I've just picked up a resend from WU 27077036. A similar story: one sporadic error (overclocking?), two libboost - and a timeout.

Look at that timeout: host 528201. Oh, Mr. Kevvy, where art thou? 156 libboost errors? You can fix that...

I'll try to get up early on Saturday morning and catch the metrics on my copy of the task - assuming it breaks the file size limit when run on a GTX 1660 super, as it appears to have done on the RTX 2080 Ti.

Edit - I've sent him a PM, but if he's not reading the forums, he may not see the notification. Ian - maybe you could put a note for other users in your team forum, too? It'll save the project both time and money.
ID: 57177 · Rating: 0 · rate: Rate + / Rate - Report as offensive     Reply Quote
Ian&Steve C.

Send message
Joined: 21 Feb 20
Posts: 1116
Credit: 40,839,470,595
RAC: 6,423
Level
Trp
Scientific publications
wat
Message 57178 - Posted: 8 Jul 2021, 11:36:45 UTC - in response to Message 57177.  

I’ve sent him several PMs.

I agree that the libboost issue should be fixed first.
ID: 57178 · Rating: 0 · rate: Rate + / Rate - Report as offensive     Reply Quote
Ian&Steve C.

Send message
Joined: 21 Feb 20
Posts: 1116
Credit: 40,839,470,595
RAC: 6,423
Level
Trp
Scientific publications
wat
Message 57187 - Posted: 8 Jul 2021, 18:36:15 UTC - in response to Message 57178.  
Last modified: 8 Jul 2021, 18:36:52 UTC

well since the 2nd task got resent (to SevicEnginIC), I decided to just cut my losses, and I aborted the transfer.

to my surprise, both tasks received credit. what? lol. so you don't even need that 500MB file? would have been nice if someone directly said, just abort the transfer and it'll be fine, but I was trying to prevent wasteful recomputation.

maybe the "fix" was to somehow prevent this file from getting so big in the first place, and they didnt change their file size limit on the server at all, which would explain why my files still wouldnt go through.

https://www.gpugrid.net/workunit.php?wuid=27077250
https://www.gpugrid.net/workunit.php?wuid=27077283
ID: 57187 · Rating: 0 · rate: Rate + / Rate - Report as offensive     Reply Quote
Profile ServicEnginIC
Avatar

Send message
Joined: 24 Sep 10
Posts: 592
Credit: 11,972,186,510
RAC: 1,447
Level
Trp
Scientific publications
watwatwatwatwatwatwatwatwatwatwatwatwatwatwatwatwatwatwatwatwatwatwatwat
Message 57191 - Posted: 9 Jul 2021, 22:48:24 UTC - in response to Message 57187.  

Right, just this of your "never uploading" tasks was catched by a GTX 1650 SUPER GPU of mine.
My estimates are that it be finished in a total processing time of 232830 seconds, at about 10 UTC on this next Sunday.
I'm curious to see whether upload behavior is the same.
I'll try to be aware to suspend Network activity in BOIC Manager, to be able to document the exact files size...
And if upload can't progress, I'll follow your same pathway...
Thank you for being always so generous in sharing experiences!
ID: 57191 · Rating: 0 · rate: Rate + / Rate - Report as offensive     Reply Quote
Profile ServicEnginIC
Avatar

Send message
Joined: 24 Sep 10
Posts: 592
Credit: 11,972,186,510
RAC: 1,447
Level
Trp
Scientific publications
watwatwatwatwatwatwatwatwatwatwatwatwatwatwatwatwatwatwatwatwatwatwatwat
Message 57194 - Posted: 11 Jul 2021, 11:58:18 UTC
Last modified: 11 Jul 2021, 12:10:11 UTC

The previously mentioned task finished today at predicted time.
Total processing time: 233077 seconds. Very near to the 232830 seconds predicted when progress was 45,12%
I suspended network activity, and when the task finished, 5 files were set in upload pending state.
I made a backup of all of them from original /var/lib/boinc-client/projects/www.gpugrid.net folder to another one.
Their names and sizes:
e5s86_e1s499p0f1068-ADRIA_New_KIXcMyb_HIP_AdaptiveBandit-1-2-RND5913_6_0 100.068 bytes
e5s86_e1s499p0f1068-ADRIA_New_KIXcMyb_HIP_AdaptiveBandit-1-2-RND5913_6_1 2.307.196 bytes
e5s86_e1s499p0f1068-ADRIA_New_KIXcMyb_HIP_AdaptiveBandit-1-2-RND5913_6_2 2.307.196 bytes
e5s86_e1s499p0f1068-ADRIA_New_KIXcMyb_HIP_AdaptiveBandit-1-2-RND5913_6_9 540.268.472 bytes
e5s86_e1s499p0f1068-ADRIA_New_KIXcMyb_HIP_AdaptiveBandit-1-2-RND5913_6_10 274 bytes

After resuming transfers, all files but _9 one were uploaded.
This file failed to upload exactly the same way reported by Ian&Steve C.

Sun 11 Jul 2021 10:58:47 WEST | GPUGRID | Started upload of e5s86_e1s499p0f1068-ADRIA_New_KIXcMyb_HIP_AdaptiveBandit-1-2-RND5913_6_9
.
Sun 11 Jul 2021 10:58:48 WEST | GPUGRID | [http] [ID#19070] Received header from server: HTTP/1.1 413 Request Entity Too Large
Sun 11 Jul 2021 10:58:48 WEST | GPUGRID | [http] [ID#19070] Received header from server: Date: Sun, 11 Jul 2021 09:58:48 GMT
Sun 11 Jul 2021 10:58:48 WEST | GPUGRID | [http] [ID#19070] Received header from server: Server: Apache/2.4.6 (CentOS) OpenSSL/1.0.1e-fips mod_auth_gssapi/1.3.1 mod_auth_kerb/5.4 mod_fcgid/2.3.9 PHP/5.4.16 mod_wsgi/3.4 Python/2.7.5
Thu 01 Jan 1970 00:00:00 WET | GPUGRID |
Sun 11 Jul 2021 10:58:47 WEST | GPUGRID | [http] [ID#19070] Info: Closing connection 8537

Then, I aborted the transfer for this file, and the result was a failed task due to "Upload failed" reason :-(
e5s86_e1s499p0f1068-ADRIA_New_KIXcMyb_HIP_AdaptiveBandit-1-2-RND5913_6
I agree this statement, since upload wasn't successful...
My Grandmother usually said to me: "Don't play with fire" :-)

I keep a copy of all the resulting files for this task, included the _9 one, in a safe folder.
I don't publish them in public, because I don't know if it could reveal some confidential data for Gpugrid project.
If them could be of interest, somebody from project is free to contact me in private for arranging some alternative way to upload.
ID: 57194 · Rating: 0 · rate: Rate + / Rate - Report as offensive     Reply Quote
Previous · 1 · 2

Message boards : Number crunching : can't upload results. file size too big being blocked?

©2025 Universitat Pompeu Fabra