Python apps for GPU hosts needs [a whole lot] more disk space.

Message boards : Graphics cards (GPUs) : Python apps for GPU hosts needs [a whole lot] more disk space.
Message board moderation

To post messages, you must log in.

AuthorMessage
Jason M Sullivan

Send message
Joined: 1 Dec 21
Posts: 1
Credit: 11,546,345
RAC: 0
Level
Pro
Scientific publications
wat
Message 59121 - Posted: 17 Aug 2022, 14:56:24 UTC

I've gotten this notice a few times over the last few months...

Python apps for GPU hosts needs 27668.09MB more disk space. You currently have 5710.52 MB available and it needs 33378.60 MB.


A little over 30Gig? That seems like an awful lot of disk to be using. I know "disk is cheap" and whatnot, but are the datasets truly that big? Anyone know why it needs so much space?
ID: 59121 · Rating: 0 · rate: Rate + / Rate - Report as offensive     Reply Quote
Keith Myers
Avatar

Send message
Joined: 13 Dec 17
Posts: 1419
Credit: 9,119,446,190
RAC: 891
Level
Tyr
Scientific publications
watwatwatwatwat
Message 59122 - Posted: 17 Aug 2022, 16:29:33 UTC - in response to Message 59121.  

I've gotten this notice a few times over the last few months...

Python apps for GPU hosts needs 27668.09MB more disk space. You currently have 5710.52 MB available and it needs 33378.60 MB.


A little over 30Gig? That seems like an awful lot of disk to be using. I know "disk is cheap" and whatnot, but are the datasets truly that big? Anyone know why it needs so much space?

Because it DOES need that much space. The tasks do reinforcement learning using Pytorch and that framework infrastructure needs that much disk space for the examples.
ID: 59122 · Rating: 0 · rate: Rate + / Rate - Report as offensive     Reply Quote
Jim1348

Send message
Joined: 28 Jul 12
Posts: 819
Credit: 1,591,285,971
RAC: 0
Level
His
Scientific publications
watwatwatwatwatwatwatwatwatwatwatwatwatwatwat
Message 59123 - Posted: 17 Aug 2022, 16:34:00 UTC - in response to Message 59121.  
Last modified: 17 Aug 2022, 16:35:06 UTC

On Ubuntu 20.04.4, my www.gpugrid.net project directory is 3.7GB, and the slot it is running in shows 9.1GB. So that is certainly less than 30GB.

But my SSD is 250GB, and I have 170GB remaining, so I have not hit the minimum space requirements yet.

(I don't know what the maximum it needs is; it could be a lot more.)
ID: 59123 · Rating: 0 · rate: Rate + / Rate - Report as offensive     Reply Quote
[CSF] Aleksey Belkov

Send message
Joined: 26 Dec 13
Posts: 86
Credit: 1,292,358,731
RAC: 0
Level
Met
Scientific publications
watwatwatwatwatwatwatwatwatwatwatwatwatwat
Message 59134 - Posted: 19 Aug 2022, 10:04:14 UTC - in response to Message 59123.  

and the slot it is running in shows 9.1GB. So that is certainly less than 30GB.

Probably WUs may vary in their volume and/or occupied volume may grow during the execution of the task.
My current WU consumes almost 16 GiB(slot).

Although of course it was interesting to know exactly how the values of the required space are calculated.
Based on the fact that different values appear in such messages, this is some kind of automated algorithm, and not manually set values for WUs.

ID: 59134 · Rating: 0 · rate: Rate + / Rate - Report as offensive     Reply Quote
ptgst3Kfg7MDZf8ShnWCBMMnhX7

Send message
Joined: 7 May 20
Posts: 2
Credit: 507,838,725
RAC: 4
Level
Lys
Scientific publications
wat
Message 59180 - Posted: 1 Sep 2022, 4:03:30 UTC
Last modified: 1 Sep 2022, 4:09:37 UTC

deleted my post - sorry wrong thread! :)
ID: 59180 · Rating: 0 · rate: Rate + / Rate - Report as offensive     Reply Quote

Message boards : Graphics cards (GPUs) : Python apps for GPU hosts needs [a whole lot] more disk space.

©2025 Universitat Pompeu Fabra