Advanced search

Message boards : GPUGRID CAFE : Cost saving using GPUGRID instead of supercomputer

Author Message
VictordeHollander
Send message
Joined: 10 Apr 12
Posts: 6
Credit: 81,085,749
RAC: 0
Level
Thr
Scientific publications
watwatwatwatwatwatwatwat
Message 25730 - Posted: 16 Jun 2012 | 12:56:37 UTC
Last modified: 16 Jun 2012 | 12:59:07 UTC

Let's say we want to build a supercomputer with the same computational throughput as GPUGRID.

How much would it cost to build?
How much would it cost to keep it running? (electricity bills, etc)

Let's say we base our calculations on a throughput of 400,000 GFlops (400 PFLOPs).
And electricity costs are $0.10 for 1 KWh.

Go ahead! (please motivate your answer)


---------------------------------------------

Here's my calculation:
A GTX570 gives a theoretical output of 1400 GFLOPs
400,000/1400 = ~285 GTX570 required!

285/3 = 95 Motherbords, CPUs, Memorysticks, PSUs(3 GPUs on each motherbord)

95x Mid-segment Motherboards (95 x USD 150 )
95x Quad core CPUs (95 x USD 150 )
95x 4GB DDR3 mem (95 x USD 30 )
95x 1000W PSU (95 x USD 100 )
285x GTX570 (285 x USD 250 )
Gives a whopping total of USD 112,100 (not taking into account the costs to house and cool all this hardware)

The running costs for this beast:
700W for each system gives a total of 66,500W
66,500 x 1.5 (due to cooling and efficiency losses) = ~100KW

24 hours: 100KW x 24 x USD 0.10 = USD 240
a week: 240 * 7 = USD 1,680
a month: 240 * 30 = USD 7,200
a year: 240 * 365 = USD 87,600 (almost the costs of buying the hardware!!!!!)

VictordeHollander
Send message
Joined: 10 Apr 12
Posts: 6
Credit: 81,085,749
RAC: 0
Level
Thr
Scientific publications
watwatwatwatwatwatwatwat
Message 25838 - Posted: 23 Jun 2012 | 8:52:03 UTC - in response to Message 25730.


Let's say we base our calculations on a throughput of 400,000 GFlops (400 PFLOPs).

Somehow I can't edit my original post.
But 400,000 GFLOPs are 400 TFLOPs of course, not PFLOPs.



Profile skgiven
Volunteer moderator
Volunteer tester
Avatar
Send message
Joined: 23 Apr 09
Posts: 3968
Credit: 1,995,359,260
RAC: 0
Level
His
Scientific publications
watwatwatwatwatwatwatwatwatwatwatwatwatwatwatwatwatwatwatwatwatwatwatwatwatwatwatwat
Message 25839 - Posted: 23 Jun 2012 | 17:10:24 UTC - in response to Message 25838.

Somehow I can't edit my original post.

You can only edit your post for 1h.
____________
FAQ's

HOW TO:
- Opt out of Beta Tests
- Ask for Help

Profile dskagcommunity
Avatar
Send message
Joined: 28 Apr 11
Posts: 456
Credit: 817,865,789
RAC: 0
Level
Glu
Scientific publications
watwatwatwatwatwatwatwatwatwatwatwatwatwatwatwatwat
Message 26772 - Posted: 2 Sep 2012 | 15:18:14 UTC

And with the power in this hour (serverstats) we are powering an supercomputer that would fit into a great Place 37 on the Top 500 List of Supercomputers :)
____________
DSKAG Austria Research Team: http://www.research.dskag.at



ronny
Send message
Joined: 20 Sep 12
Posts: 17
Credit: 19,131,325
RAC: 0
Level
Pro
Scientific publications
watwat
Message 26994 - Posted: 24 Sep 2012 | 22:06:48 UTC

4x690 in 18 computers would be half the electricity cost. 400W power use per 690 card (measured), 200W for CPU and SSD, so 1800W per computer, divided by XFX 850+850+550 PSUs or a hugely expensive but more compact one (I say XFX because they have all the wattage on just one rail, and is guaranteed to give the said amount of power, so they're very easy to interconnect).

18x XFX 550w 100 USD
36x XFX 850W 140 USD
18x Corsair 2133MHz 2x4GB 80 USD
18x Corsair SSD 120gb 100 USD
18x ASUS Maximus V Extreme 500 USD
18x Intel i7-2700K 500 USD
72x gtx 690 72000 USD
100 000 USD put together.

If however these were placed in cold places, they could use the computers as the main source of heat (homes, offices, garages etc), and thus cost practically nothing to run.

Profile skgiven
Volunteer moderator
Volunteer tester
Avatar
Send message
Joined: 23 Apr 09
Posts: 3968
Credit: 1,995,359,260
RAC: 0
Level
His
Scientific publications
watwatwatwatwatwatwatwatwatwatwatwatwatwatwatwatwatwatwatwatwatwatwatwatwatwatwatwat
Message 27220 - Posted: 4 Nov 2012 | 18:44:47 UTC - in response to Message 26994.
Last modified: 5 Nov 2012 | 11:00:01 UTC

A GTX690 has a TDP of 300W. Here it's likely to be using around 225W while running (Not 400W)! I doubt it would use more than 250W.
You can't just go sticking a dual GPU into any motherboard and expect the same PCIE bandwidth. Especially for the 4th PCIE slot; very few boards offer four 16X PCIE2.0 lanes. Be specific, state the board and CPU.

You could use two GTX690's in 3rd generation Intel based systems, and you could use i3's which would mean more systems but less outlay and different running costs,

So, theoretical system:
i3-3220 £92 (35W)
Dual PCIE3 x16 motherboard (Z68A-GD80 (G3)) £165 (35W)
Corsair 4GB (2 x 2GB) DDR3 £20 (5W)
60GB SATA6 SSD £43 (5W)
Case + fans £45 (20W)
PSU £85
2x GTX690 £1500 (450W)

Total £1950, 550W for each 11240 GFLOPS system.
So, 36 systems would be required.

Purchase cost would be £70,200

Running cost would be, 16.2KW/h. @15pence per KW is £58.32/day

Annual running cost would be £21,287.
Total, just under £92,000.

System 2, and a more common DC system, albeit theoretical:
i3-3220 £92 (35W)
Dual PCIE3 x16 motherboard (Z68A-GD80 (G3)) £165 (35W)
Corsair 4GB (2 x 2GB) DDR3 £20 (5W)
60GB SATA6 SSD £43 (5W)
Case + fans £45 (20W)
PSU £85
2x GTX660Ti's (10% FOC) £500 (250W)

Total £1950, 350W for each 5411 GFLOPS system.
So, 74 'distributed' systems would be required.

Purchase cost would be £66,970

Running cost would be, 16.2KW/h. @15pence per KW is £93.24/day

Annual running cost would be £34,032.
Total, just under £101,000.
____________
FAQ's

HOW TO:
- Opt out of Beta Tests
- Ask for Help

Post to thread

Message boards : GPUGRID CAFE : Cost saving using GPUGRID instead of supercomputer

//