huge result files
Message boards :
Graphics cards (GPUs) :
huge result files
Message board moderation
| Author | Message |
|---|---|
|
Send message Joined: 18 Sep 08 Posts: 65 Credit: 3,037,414 RAC: 0 Level  Scientific publications   
|
those huge result files are causing trouble - probably server bandwidth is too limited to cope with them. and you'll have to pay some extra credits for upload bandwidth used.. |
 (_KoDAk_) Send message Joined: 18 Oct 08 Posts: 43 Credit: 6,924,807 RAC: 0 Level  Scientific publications   
|
yes 25 mb upload it's too big 
|
|
Send message Joined: 18 Sep 08 Posts: 65 Credit: 3,037,414 RAC: 0 Level Scientific publications
|
yes 25 mb upload it's too big 25 is lean - i've heard of 50++.. |
|
Lazarus-uk Send message Joined: 16 Nov 08 Posts: 29 Credit: 122,821,515 RAC: 0 Level  Scientific publications
|
Yes, I just had my 2nd extra long WU (13-14hrs on OCed GTX260). Each one had a single file of 53MB, plus the usual normal files. It took me 25 minutes to upload, however, a team mate has been uploading one of his for well over an hour now @4kb/s. Mark |
 Send message Joined: 17 Aug 08 Posts: 2705 Credit: 1,311,122,549 RAC: 0 Level  Scientific publications |
The WUs which accidently have double the size also seem to have double the output file size. This doesn't change much for the user, though: within the same time you can have one 50MB file or 2 25 MB files. MrS Scanning for our furry friends since Jan 2002 |
|
Send message Joined: 18 Sep 08 Posts: 65 Credit: 3,037,414 RAC: 0 Level Scientific publications
|
The WUs which accidently have double the size also seem to have double the output file size. This doesn't change much for the user, though: within the same time you can have one 50MB file or 2 25 MB files. that's no true - i'm currently uploading a "normal" one, and the largest file alone shows 28.86 MB. |
|
Paul D. Buck Send message Joined: 9 Jun 08 Posts: 1050 Credit: 37,321,185 RAC: 0 Level  Scientific publications 
|
If memory serves several of my 13 hour tasks were nearly 60M in size. Sometimes having two 25M files is more desirable than one 60 if for no other reason that if there is a problem the retry does not cost so much. |
|
Send message Joined: 17 Aug 08 Posts: 2705 Credit: 1,311,122,549 RAC: 0 Level Scientific publications |
that's no true - i'm currently uploading a "normal" one, and the largest file alone shows 28.86 MB. Does it make the 50 MB uploads any worse when you have the choice between 1 times 50 MB or 2 times 28.86 MB? @Paul: true, but then it has been said several times that the extra large WUs were accidents not intended to be repeated. So I don't think it's worth the time to worry about them, as the overall transfer volume is still comparable. Regarding a possible general complaint, that 25 - 30 MB upload is too much for a normal WU: I guess this is something inherent to the project. Remember: GPU-Grid is here to work as a supercomputer with a capability not seen before. Its quite logical to apply such power to large systems, which could not be crunched before.. so there will be many atoms and the positions of all of them have to be used as starting points for the next iteration, i.e. they have to be sent back. I don't know the actual data structure, but the uploads have been rather large from the beginning on. Could also be that the newer files contain debug more information or more frequent result savepoints. I.e. ~800000 time steps are cruched per WU and it could be that previously all atom positions were saved each 100th iteration, whereas now it may be every 20th iteration. But that's just speculation on my part. MrS Scanning for our furry friends since Jan 2002 |
©2025 Universitat Pompeu Fabra