Server is out of disk space

Message boards : Number crunching : Server is out of disk space
Message board moderation

To post messages, you must log in.

Previous · 1 · 2 · 3 · Next

AuthorMessage
doug

Send message
Joined: 2 Dec 24
Posts: 3
Credit: 32,120,373
RAC: 13
Level
Val
Scientific publications
wat
Message 62019 - Posted: 14 Dec 2024, 14:41:18 UTC

Hi,

I'm also getting the out of disk space message:

12/14/2024 9:35:39 AM | GPUGRID | [error] Error reported by file upload server: Server is out of disk space

12/14/2024 9:35:39 AM | GPUGRID | Temporarily failed upload of

The times are EST.

Doug
ID: 62019 · Rating: 0 · rate: Rate + / Rate - Report as offensive     Reply Quote
Profile dskagcommunity
Avatar

Send message
Joined: 28 Apr 11
Posts: 463
Credit: 958,266,958
RAC: 41
Level
Glu
Scientific publications
watwatwatwatwatwatwatwatwatwatwatwatwatwatwatwatwat
Message 62020 - Posted: 14 Dec 2024, 16:38:38 UTC - in response to Message 62018.  

On other projects this is very rarely i must say and im crunching 24/7 since 2011 ^^ But i dont complain too much, just happy this project has WUs again for just ParttimecrunchingPCs.
DSKAG Austria Research Team: http://www.research.dskag.at



ID: 62020 · Rating: 0 · rate: Rate + / Rate - Report as offensive     Reply Quote
mikey

Send message
Joined: 2 Jan 09
Posts: 303
Credit: 7,321,800,090
RAC: 330
Level
Tyr
Scientific publications
watwatwatwatwatwatwatwatwatwatwatwatwatwatwatwat
Message 62021 - Posted: 14 Dec 2024, 16:47:48 UTC - in response to Message 62020.  

On other projects this is very rarely i must say and im crunching 24/7 since 2011 ^^ But i dont complain too much, just happy this project has WUs again for just ParttimecrunchingPCs.


The problem probably is the fact that the credits are sooo high and the tasks are sooo available that more and more people who wants TONS of credits per day are coming here for them. It also comes down to that as opposed to 6 months ago when several more projects were around to spread the load gpugrid is now one of the few gpu projects around so like the rest is being overwhelmed. I would guess that in time they will figure out how to move things thru the system more quickly and get bigger drivers too and things will be alot better.
ID: 62021 · Rating: 0 · rate: Rate + / Rate - Report as offensive     Reply Quote
Greg _BE

Send message
Joined: 30 Jun 14
Posts: 153
Credit: 129,654,684
RAC: 0
Level
Cys
Scientific publications
watwatwatwatwatwat
Message 62023 - Posted: 14 Dec 2024, 18:28:53 UTC

Probably won't see anything done about it (disk space) until Monday if we are lucky.
ID: 62023 · Rating: 0 · rate: Rate + / Rate - Report as offensive     Reply Quote
KeithBriggs

Send message
Joined: 29 Aug 24
Posts: 71
Credit: 3,321,790,989
RAC: 1,408
Level
Arg
Scientific publications
wat
Message 62027 - Posted: 14 Dec 2024, 20:32:18 UTC - in response to Message 62023.  

Probably won't see anything done about it (disk space) until Monday if we are lucky.


+1

I started getting errors while computing (EWC) as soon as I started running a script to get them uploaded. There was already a lot of EWC which might contribute to the server overload in the first place. It seems odd that a script would be able to impact a task during what I call the initial cpu-only stage.
ID: 62027 · Rating: 0 · rate: Rate + / Rate - Report as offensive     Reply Quote
KeithBriggs

Send message
Joined: 29 Aug 24
Posts: 71
Credit: 3,321,790,989
RAC: 1,408
Level
Arg
Scientific publications
wat
Message 62028 - Posted: 14 Dec 2024, 20:51:44 UTC - in response to Message 62027.  

Richard Haselgrove identified the issue on the other thread: Corrupt compression likely due to server space but it's also a compression issue.
ID: 62028 · Rating: 0 · rate: Rate + / Rate - Report as offensive     Reply Quote
Richard Haselgrove

Send message
Joined: 11 Jul 09
Posts: 1639
Credit: 10,159,968,649
RAC: 428
Level
Trp
Scientific publications
watwatwatwatwatwatwatwatwatwatwatwatwatwatwatwatwatwatwatwatwatwatwatwatwatwatwat
Message 62029 - Posted: 14 Dec 2024, 21:08:35 UTC

Some files have started to trickle up, but others are reaching 100% and then sticking again. But it's a sign they may be trying to move some of the results to alternative storage. It'll take time, though.
ID: 62029 · Rating: 0 · rate: Rate + / Rate - Report as offensive     Reply Quote
Robertobit

Send message
Joined: 29 Mar 20
Posts: 4
Credit: 782,780,180
RAC: 16
Level
Glu
Scientific publications
wat
Message 62030 - Posted: 14 Dec 2024, 22:00:33 UTC

Hi I have the same issue | [error] Error reported by file upload server: Server is out of disk space
ID: 62030 · Rating: 0 · rate: Rate + / Rate - Report as offensive     Reply Quote
Robertobit

Send message
Joined: 29 Mar 20
Posts: 4
Credit: 782,780,180
RAC: 16
Level
Glu
Scientific publications
wat
Message 62031 - Posted: 14 Dec 2024, 22:01:20 UTC - in response to Message 61548.  

|me too [error] Error reported by file upload server: Server is out of disk space
ID: 62031 · Rating: 0 · rate: Rate + / Rate - Report as offensive     Reply Quote
Profile Khali

Send message
Joined: 13 Jan 14
Posts: 9
Credit: 390,232,404
RAC: 0
Level
Asp
Scientific publications
watwatwatwatwatwatwatwatwatwatwat
Message 62032 - Posted: 14 Dec 2024, 22:13:37 UTC

Just adding in my two cents here. I have files stuck in project back off as well.

Funny how this always seems to happen on a weekend and nothing can be done until Monday. Oh well, at least it wasn't Christmas week where every one is away for the holidays for up to a weeks time.
ID: 62032 · Rating: 0 · rate: Rate + / Rate - Report as offensive     Reply Quote
Erich56

Send message
Joined: 1 Jan 15
Posts: 1166
Credit: 12,260,898,501
RAC: 1
Level
Trp
Scientific publications
watwatwatwatwatwatwatwatwat
Message 62034 - Posted: 15 Dec 2024, 7:40:07 UTC - in response to Message 62032.  

Funny how this always seems to happen on a weekend and nothing can be done until Monday.

I am wondering anyway how much or not they are interested in keeping their own project working properly, given that obviously everyone leaves on Friday afternoon and no one keeps an eye on what's happening until the following week :-(
ID: 62034 · Rating: 0 · rate: Rate + / Rate - Report as offensive     Reply Quote
Greg _BE

Send message
Joined: 30 Jun 14
Posts: 153
Credit: 129,654,684
RAC: 0
Level
Cys
Scientific publications
watwatwatwatwatwat
Message 62036 - Posted: 15 Dec 2024, 11:58:23 UTC - in response to Message 62034.  
Last modified: 15 Dec 2024, 12:05:59 UTC

Funny how this always seems to happen on a weekend and nothing can be done until Monday.

I am wondering anyway how much or not they are interested in keeping their own project working properly, given that obviously everyone leaves on Friday afternoon and no one keeps an eye on what's happening until the following week :-(


EU project (Barcelona). M-F only. Not paid for weekends. They are a donation based program with a bit of government funding, so there is no money for weekend work. So if something goes SOL after hours then it waits until the next working day. Actually I don't think any BOINC project has a 24/7 monitor. Rosetta in the early days did, but all the projects I am on now don't.
ID: 62036 · Rating: 0 · rate: Rate + / Rate - Report as offensive     Reply Quote
Erich56

Send message
Joined: 1 Jan 15
Posts: 1166
Credit: 12,260,898,501
RAC: 1
Level
Trp
Scientific publications
watwatwatwatwatwatwatwatwat
Message 62037 - Posted: 15 Dec 2024, 12:00:44 UTC - in response to Message 62036.  

Funny how this always seems to happen on a weekend and nothing can be done until Monday.

I am wondering anyway how much or not they are interested in keeping their own project working properly, given that obviously everyone leaves on Friday afternoon and no one keeps an eye on what's happening until the following week :-(


EU project (Barcelona). M-F only. Not paid for weekends. And if they are a donation based projects (from the looks of their one page) then there is no money for weekend work. So if something goes SOL after hours then it waits until the next working day.

:-( :-( :-(
ID: 62037 · Rating: 0 · rate: Rate + / Rate - Report as offensive     Reply Quote
zioriga

Send message
Joined: 30 Oct 08
Posts: 47
Credit: 669,991,028
RAC: 206
Level
Lys
Scientific publications
watwatwatwatwatwatwatwatwatwatwatwatwatwatwatwatwatwatwatwatwatwat
Message 62038 - Posted: 15 Dec 2024, 12:41:51 UTC

And credits are less rewarded, 'cause of this delay
ID: 62038 · Rating: 0 · rate: Rate + / Rate - Report as offensive     Reply Quote
Richard Haselgrove

Send message
Joined: 11 Jul 09
Posts: 1639
Credit: 10,159,968,649
RAC: 428
Level
Trp
Scientific publications
watwatwatwatwatwatwatwatwatwatwatwatwatwatwatwatwatwatwatwatwatwatwatwatwatwatwat
Message 62045 - Posted: 16 Dec 2024, 10:11:52 UTC

Phew. All mine have uploaded, reported, and had replacement ATMML tasks issued.
ID: 62045 · Rating: 0 · rate: Rate + / Rate - Report as offensive     Reply Quote
roundup

Send message
Joined: 11 May 10
Posts: 68
Credit: 12,293,491,875
RAC: 3,176
Level
Trp
Scientific publications
watwatwatwatwatwatwatwatwatwatwatwatwatwatwatwatwat
Message 62070 - Posted: 23 Dec 2024, 6:41:50 UTC

And again:


Mo 23 Dez 2024 07:26:13 CET | GPUGRID | [error] Error reported by file upload server: Server is out of disk space
Mo 23 Dez 2024 07:26:13 CET | GPUGRID | [error] Error reported by file upload server: can't open file /home/ps3grid/projects/PS3GRID/upload/1e4/1i5xA00_300_3-ANTONIOM_MDCATH300r1si-13-50-RND8245_0_10: No space left on device


ID: 62070 · Rating: 0 · rate: Rate + / Rate - Report as offensive     Reply Quote
Vince

Send message
Joined: 27 Aug 22
Posts: 1
Credit: 667,195,495
RAC: 0
Level
Lys
Scientific publications
wat
Message 62071 - Posted: 23 Dec 2024, 9:23:33 UTC

Server space full no uploads possible
A totally unacceptable situation
This happening far too often and will have me looking at alternative
projects like moowrap and or primegrid for my dedicated GPU work
ID: 62071 · Rating: 0 · rate: Rate + / Rate - Report as offensive     Reply Quote
roundup

Send message
Joined: 11 May 10
Posts: 68
Credit: 12,293,491,875
RAC: 3,176
Level
Trp
Scientific publications
watwatwatwatwatwatwatwatwatwatwatwatwatwatwatwatwat
Message 62072 - Posted: 23 Dec 2024, 12:14:55 UTC - in response to Message 62071.  


A totally unacceptable situation
This happening far too often and will have me looking at alternative
projects like moowrap and or primegrid for my dedicated GPU work


The focus is on the science behind our projects, not on WU's regular supply. Since 2023, the supply has been much more regular than in many previous years. That has really improved.
Also, most of us understand that there are no unlimited resources for most european research institutes to buy or scale hardware.
Basically, it's always a good idea to have a backup project that steps in with resource share = 0 whenever there's no work for the main project.
ID: 62072 · Rating: 0 · rate: Rate + / Rate - Report as offensive     Reply Quote
Richard Haselgrove

Send message
Joined: 11 Jul 09
Posts: 1639
Credit: 10,159,968,649
RAC: 428
Level
Trp
Scientific publications
watwatwatwatwatwatwatwatwatwatwatwatwatwatwatwatwatwatwatwatwatwatwatwatwatwatwat
Message 62073 - Posted: 23 Dec 2024, 13:32:40 UTC - in response to Message 62072.  

Yes, but ...

The science the project is interested in is contained in the result files we return to the project: until they have been accepted, no (observable) science has happened. A regular supply of tasks implies, inevitably, a regular flow of attempted uploads: the project staff should be able to monitor the inflow, and if necessary regulate the work flow to what is manageable and affordable.

Having said that, two machines have uploaded completed ATMML tasks this lunchtime, and downloaded replacements without bothering my backup project.
ID: 62073 · Rating: 0 · rate: Rate + / Rate - Report as offensive     Reply Quote
Freewill

Send message
Joined: 18 Mar 10
Posts: 28
Credit: 41,810,583,419
RAC: 13,276
Level
Trp
Scientific publications
watwatwatwatwat
Message 62074 - Posted: 23 Dec 2024, 15:54:43 UTC - in response to Message 62073.  

On the project's Discord server, one of the admins (giadefa) has posted that "by the end of January we will move to a new server with more disk space." So, that change should address this issue. For now, they seem to be watching as the immediate issue has been resolve. I have been able to upload my completed tasks recently.
ID: 62074 · Rating: 0 · rate: Rate + / Rate - Report as offensive     Reply Quote
Previous · 1 · 2 · 3 · Next

Message boards : Number crunching : Server is out of disk space

©2025 Universitat Pompeu Fabra