Incredible amount of compute power compared to Turing!
38Tflops single 32bit precision and 30Tflops for 3080. 20Tflops on 3070.

New arch details forthcoming. Plenty of websites with current info.

New samsung? 8nm die lithography with 28 billion transistors (3090) die size unknown currently.
Previous 12nm is TSMC "ffn".
Memory clocks are faster than Turing.
PCIe4.0 Ampere Boost should be around 2ghz as previous PCIe3.0 Pascal and Turing did routinely.

Will GRUGRID be ready for this newest generation?

I'll be an early adapter. Looking at couple 3080 since 3090 offers 1800 more cores for 700$ premium.
Purchase new pci4.0 mb and +12 core CPU with the savings.

I didn't see any publication of the actual memory clocks. Just the bandwidths. Will wait and see what the actual specs and test results are once the actual cards are in the hands of testers. Just because the cards will be great pixel pushers for ray-tracing games doesn't mean they will produce the commensurate compute improvements.

And to fully utilize the new architectural differences between Ampere and Turing/Pascal means we will need new applications.

Integer32 performamce same as fp32 and int32 Turing 1:1?

On gaming: the hardest pixel pushing 4k ray tracing is currently 3rd iteration of Metro game.

Certainly others are crisp too. Another game or demo 8k or video will showcase ampere chops. I have 4k now. 8k monitor right around the corner for mainstream purchase. 2020 Bleeding edge.
I was surprised at 10k or 8k <5k 3070 cores. Never mind ampere tensors or fp64 performance.

And to fully utilize the new architectural differences between Ampere and Turing/Pascal means we will need new applications.

The trick will be to use all the new hardware in the best parallelization of the current and future searches.

If you are a game designer, you have new SDK's for gaming. But is there going to be a compute SDK right after release?

Best scenario would be yes, and even better would be some new automatic profilers for compute loads. That way you could just input the current source code and the profiler would spit out the new optimized code for the new hardware resources in
Ampere. Then look at the generated code and iterate another revision that is better and faster. Rinse and repeat.

What concerns me: will Turing quality control issues crop up on Ampere?
Out of 6 turing GPUs I purchased 4 died. Gigabyte 2070 in a day. Zotac 2060 lasted 5 months. evga 2080 in 2 months with another evga 2080 enduring 2 years.

All had 3 year warranty. I sold all the warranty replacements since I didnt want anymore Turings due to their high death rate.
Had Pascal 1080 gpu knelt after 27 months. My Evga 1070 still holding on after 4 years 24/7 running it would be retired if the Turings lasted. And no Maxwell's quit they just retired.

I expect that the CUDA cores could be used for crunching will be only the half of that stated in the name of this thread.
Similarly to the GF116 architecture, where only the 2/3rd of the cores could be used for crunching (due to the dispatch unit/CUDA core (4/6) ratio):


I think the relative performance in computing compared to the RTX 2080Ti will be the following:

I think the CUDa core counts might be a bit of marketing magic. They claimed in the article that the Ampere cores can do 2 operations per clock, which is effectively doubling the work done, on a single physical core. We’ll see how it shakes out for compute work.

I’ll probably grab a 3070 when they are released to compare to my 2080ti and existing 2070s.

I’m very interested in performance per watt of the new cards. Don’t forget that all the new cards, while performance is looking great, they are seeing a healthy increase in power consumption as well. The 2080ti was a 250W card, the 2080 was a 215W card, the 2070 was a 175W card. Now these new cards are 350W/320W/220W for 3090/3080/3070 respectively.

If the 3070 performs comparably to a 2080ti that’s about a 12% power efficiency boost, which is believable.

For CUDA compute work, the 2070 was as fast or a little faster than a 1080ti at much less power draw (250W vs 175W). So I could see the same thing happening again for 3070 vs 2080ti.
____________

I expect that the CUDA cores could be used for crunching will be only the half of that stated in the name of this thread.

Did some more reading and I think I got a better grasp on the whole CUDA core count issue and relative performance.

What’s being left to the footnotes is that comment that Jenson made in the announcement “two instructions per clock”. What he didn’t say was how the SM partitioning worked to allow this to happen, and that the doubling affects FP32 calculations only. They have a new datapath design where each SM partition can handle either 32x FP32 (which is double Turing) OR 16FP32 and INT32.

For graphics loads, it’s mostly FP32 so this works to their advantage, but for INT32 workloads, you won’t have this doubling effect, and the performance will be closer to Turing, with the normal generational efficiency boost.

A full size Ampere GPU core (GA100) is 128SMs with 64 CUDA cores per SM for a highest possible CUDA core count of 8192. The 3090, 3080, 3070 are not full size Ampere cores. Given that the marketing number of the 3090 is “10496” cores, we can surmise that it’s really 5248 CUDA cores with 82 SMs. This is still more SMs and cores than the 2080ti has (68 SMs/ 4352 Cores). However, the 3070 with its “5888” cores really has 2944 cores.

So it will depend on your workload. If you’re only living in the bubble of gaming, then yeah enjoy that 2x boost. But for other non-FP32 loads (like more purely computational loads), you’re going to likely only see normal generational improvements with Ampere over Turing.
____________

Did some more reading and I think I got a better grasp on the whole CUDA core count issue and relative performance.

What’s being left to the footnotes is that comment that Jenson made in the announcement “two instructions per clock”. What he didn’t say was how the SM partitioning worked to allow this to happen, and that the doubling affects FP32 calculations only. They have a new datapath design where each SM partition can handle either 32x FP32 (which is double Turing) OR 16FP32 and INT32.

For graphics loads, it’s mostly FP32 so this works to their advantage, but for INT32 workloads, you won’t have this doubling effect, and the performance will be closer to Turing, with the normal generational efficiency boost.

A full size Ampere GPU core (GA100) is 128SMs with 64 CUDA cores per SM for a highest possible CUDA core count of 8192. The 3090, 3080, 3070 are not full size Ampere cores. Given that the marketing number of the 3090 is “10496” cores, we can surmise that it’s really 5248 CUDA cores with 82 SMs. This is still more SMs and cores than the 2080ti has (68 SMs/ 4352 Cores). However, the 3070 with its “5888” cores really has 2944 cores.

So it will depend on your workload. If you’re only living in the bubble of gaming, then yeah enjoy that 2x boost. But for other non-FP32 loads (like more purely computational loads), you’re going to likely only see normal generational improvements with Ampere over Turing.

Well, I'm optimistic :)

https://wccftech.com/nvidia-geforce-rtx-3080-flagship-2x-faster-than-rtx-2080-in-opencl-cuda-benchmarks/

Hopefully, the Amperes will be showing up on the Passmark GPU Direct Compute ratings soon.

https://www.videocardbenchmark.net/directCompute.html

https://www.tomshardware.com/features/nvidia-ampere-architecture-deep-dive

Real fp32 cores confirmed. 64 fp32 per sm with 32 cores for fp32 only with remaining 32 core being concurrent int32 or
fp32. See Ampere slides. Compute benchmarks released (tomorrow?) will show how well new fp32 design performs.

Ampere Integer32 performance 50-66% of floating 32 depending on code efficiency.

Consumer Ampere ga102/104 (fp64) double precision (2 per sm) now 1/64 ratio of fp32. Turing has 1/32 (4 fp64 per sm).

https://www.tomshardware.com/news/nvidia-geforce-rtx-3080-review

Tom's will reveal their compute benchmarks soon and show detailed power profiles.

Definitely an upgrade if own a 2080ti.

Are you purchasing an Ampere? If so which? For me 3080 and 3070 look good.
RTX 3090 the ultra halo card with an awful per core cost compared to 3080.

Note: GTX 3080 founders edition real power consumption similar to overclocked non founders 2080ti - demanding 330 watts.

Curious to what the non founders RTX 3090 pulls - my guess is ~400W overclocked.

Edit: noticed I made mistake in my previous post - meant to write Ampere has 128 fp32 cores per sm.

So far all the cards are power limited. Almost no overclocking potential. Maybe 2-3%. Doubt the 3090 cards will be any different.

So far all the cards are power limited. Almost no overclocking potential. Maybe 2-3%. Doubt the 3090 cards will be any different.

Until the compute apps get recompiled for CUDA 11.1 and the new PTX library, none are going to show the potential from using the dormant extra FP32 pipeline in the architecture.

https://www.techpowerup.com/272591/rtx-3080-crash-to-desktop-problems-likely-connected-to-aib-designed-capacitor-choice

https://www.techpowerup.com/272591/rtx-3080-crash-to-desktop-problems-likely-connected-to-aib-designed-capacitor-choice

Plain sailing from here?

Plain sailing from here?

No, not at all. No compatible apps yet.

Don't know if this has a lot of relevance for those of you currently considering to purchase or already having bought a RTX 30xx series card, but over at F@H they have recently been rolling out CUDA support on their GPU cores and were able to drastically increase the average speed of NVIDIA cards.

They ran an analysis on the efficiency gain of the cards produced by the CUDA support and included in one of their analysis an RTX 3080.
https://foldingathome.org/2020/09/28/foldingathome-gets-cuda-support/

I can't post pictures here, so just take a look at the third graph posted on this site. From what I can tell, the RTX 3080 achieves an improvement of 10-15% over a RTX 2080 Ti in this one particular GPU task they benchmarked the cards to. This might be interesting for benchmarking the performance here as well. Thought this might be of interest to some of you.

We don't have access to any card yet.

1.4.1.6. Improved FP32 throughput

Devices of compute capability 8.6 have 2x more FP32 operations per cycle per SM than devices of compute capability 8.0. While a binary compiled for 8.0 will run as is on 8.6, it is recommended to compile explicitly for 8.6 to benefit from the increased FP32 throughput.

Don't know if this has a lot of relevance for those of you currently considering to purchase or already having bought a RTX 30xx series card, but over at F@H they have recently been rolling out CUDA support on their GPU cores and were able to drastically increase the average speed of NVIDIA cards.

They ran an analysis on the efficiency gain of the cards produced by the CUDA support and included in one of their analysis an RTX 3080.
https://foldingathome.org/2020/09/28/foldingathome-gets-cuda-support/

I can't post pictures here, so just take a look at the third graph posted on this site. From what I can tell, the RTX 3080 achieves an improvement of 10-15% over a RTX 2080 Ti in this one particular GPU task they benchmarked the cards to. This might be interesting for benchmarking the performance here as well. Thought this might be of interest to some of you.

Don't know if this has a lot of relevance for those of you currently considering to purchase or already having bought a RTX 30xx series card, but over at F@H they have recently been rolling out CUDA support on their GPU cores and were able to drastically increase the average speed of NVIDIA cards.

They ran an analysis on the efficiency gain of the cards produced by the CUDA support and included in one of their analysis an RTX 3080.
https://foldingathome.org/2020/09/28/foldingathome-gets-cuda-support/

I can't post pictures here, so just take a look at the third graph posted on this site.

From what I can tell, the RTX 3080 achieves an improvement of 10-15% over a RTX 2080 Ti in this one particular GPU task they benchmarked the cards to. This might be interesting for benchmarking the performance here as well. Thought this might be of interest to some of you.

but every CUDA core has two FP32 units in the Ampere architecture. Actually the INT32 part (existing in previous generations too) of the CUDA cores has been extended to be able to handle FP32 calculations. It seems that these "extra" FP32 units can't be utilized by the scientific applications I'm interested in.

Just meant this as a pointer. Don't know what data they included in their analysis in the sense of how many GPUs were flowing in this data aggregation. Honestly, I am in way over my head with all this technical talk about GPU application porting, wrappers, code recompiling, CUDA libraries etc., even though I am trying to read up on it online as much as I can to get more proficient with those terms.

However, as F@H is currently arguably the largest GPU platform/colection of (at least consumer-grade) GPU cards, I would reckon that the numbers shown here are as representative as they can get due to those values being averaged out over a bunch of various card makes for each gen. So I'd argue that this value is a solid benchmark. At least for 1) now, 2) without further optimisation for the Ampere infrastructure and 3) for this specific task at F@H. I'd suggest to ask in their forums about the data that was used to derive those values and what those task requirements have been to induce valuable information for a possible RTX 30xx series performance on GPUGRID.

Sigh.... I wish I had the problem of figuring out how much performance is set to be gained of a 3080 over a 2080 Ti. For the moment I am stuck with the 750 Ti (fourth last)...

And thanks for including this graph here Zoltán!

Good luck with your endeavour!

this is something we can't know until GPUGRID recompiles the app for sm_86 with CUDA 11.1.

it will also depend on what kinds of operations GPUGRID is doing. if they are mostly INT, then maybe not much improvement, but if they are FP32 heavy, we can expect to see a big improvement.

it's just something that we need to wait for the app for. as I mentioned in my previous posts, you can't rely on that benchmark very strongly.

there are performance inconsistencies already between what it shows and what you can see here at GPUGRID (specifically as it relates to the performance between the V100 and the 2080ti)

was the F@h app compiled with SM_86? I couldn't find that info. as referenced in my previous comment from the nvidia documentation, that might be needed to get the full benefit from the double FP32 throughput.

which is why I hope the app update includes the SM_86 gencode.
____________

I think the existing app source code can just be recompiled with the new genarch parameters to get the app working like it already does with previous generations.

To get the app working and use the extra FP32 pipeline, I think the app will need to be rewritten to add the PTX ISA JIT compiler that is in the new 11.1 CUDA toolkit. That seems to be the key component to parallelize the code to get the extra FP32 register working on compute.

But just with the single FP32 register in play, we should see some pretty good enhancement just from the architecture changes.

was the F@h app compiled with SM_86? I couldn't find that info. as referenced in my previous comment from the nvidia documentation, that might be needed to get the full benefit from the double FP32 throughput.

which is why I hope the app update includes the SM_86 gencode.

I guess so. They must not have encoded a minimum or maximum level that can be used. Not looking for some specific bounds like the app here does and is currently finding the Ampere cards out of range.

At 320 Watts the PCIe cables and connectors will melt. I make my own PSU cables with a thicker gauge wire but those chintzy plastic PCIe connectors can't handle the heat. If you smell burnt electrical insulation or plastic that's the first place to check.

A standard 8 pin PCIE cable can provide 150W. All the cards have either two or three 8 pin connectors which provide 300 or 450 watts PLUS the 75 watts from the PCIE slot. The founders edition cards have that new 12 pin connector which is just a duplicate capacity of two 8 pin connectors. Only six wires have 12V on them, the rest are grounds.

The plastic connector housing has nothing to do with current capability. It is the wires and pins the provide that.

Pulling the blanket from the link in this eXaPower post...
I arrived to this other interesting article about the new Nvidia 12 pin power connector.
As Keith Myers remarks, it isn't a mere evolution of 6/8 pin current connectors, but a complete new specification.
This new specification includes connector's shape, constructive materials, wires and pins gauges, and electrical ratings.
On the paper, it might handle as much as 600 Watts...
To general public: Please, don't test it with a bent paperclip! ;-)

I arrived to this other interesting article about the new Nvidia 12 pin power connector.
As Keith Myers remarks, it isn't a mere evolution of 6/8 pin current connectors, but a complete new specification.
This new specification includes connector's shape, constructive materials, wires and pins gauges, and electrical ratings.
On the paper, it might handle as much as 600 Watts...

They carry a higher power rating because the base wiring spec is that of a higher gauge than the old cables.

The connector gets hot as a consequence of the small gauge wiring getting hot.

I’ve been crunching a long time with many high power GPUs and never had a cable or connector melt.

The 12-pin connector (or 2x 6-pin, or 2x 8-pin) is totally sufficient if you’re using a modern PSU that’s already built over spec with 16ga wire. if you’re using some cheap PSU with barely spec wiring, then yeah you might have issues. But most PSUs from a reputable brand like EVGA or Corsair or Seasonic, you’ll never have an issue unless you do something stupid like using cheap SATA or MOLEX to PCIe adapters


____________

Yes, the new 12 pin connector is using 16 ga. wire which reduces the circuit resistance and has higher ampacity. The main thing to avoid is heating either the wire or the pins up as that can cause a runaway thermal condition which just increases the contact resistance which increases the heating and it just keeps building until failure.

Just moving from cheap 20 ga wire in typical power supply cables to the new 16 ga wire gains you an increase of 2X capacity. From 11A for 20 ga. to 22 A for 16 ga.

I agree it would have been better to have silver, silver-cadmium or gold plating but contact mechanical grip also is a big factor. The more surface area in contact with the mating connector and the higher the gripping and insertion force there is, the lower the overall contact resistance and the lower amount of heating due to current transmission.

American Conductor Stranding - AWG Table

I've gone through hundreds of GPUs in the 20 years I've been doing this and I've had a fair number of PCIe cables burn up at both the PSU end and the GPU end. The biggest problem is due to the wimpy crimps. Sometimes you can use an extraction tool and remove the terminal pin from the plastic connector and the terminal pin is just flopping around on the wire. Heats up like Keith described and the plastic melts. I found examples that I hadn't thrown in the trash yet. I'll photograph them and post later.

Also all my PSUs are Corsair AX1200s with a few AX860s and TX750s. I got the AX1200s back when I thought it was a good idea to run 4 GPUs on a single motherboard. I now use only one or two and they must be on 16x 3.0 sockets. I believe PSUs are most efficient at 80% load so I'm under loading them now but I use what I have.

When I make cables this is the 16 gauge wire I use: https://mainframecustom.com/shop/cable-sleeving/lc-custom-16awg-cable-sleeving-wire-black-25ft/

This is my crimping tool: https://mainframecustom.com/shop/cable-sleeving/cable-sleeving-tools/mc-ratchet-crimper/

My terminal extraction tool: https://mainframecustom.com/shop/cable-sleeving/mainframe-customs-terminal-extractor/

I don't use sleeving as it seems it would just trap heat. Just an inch of heat shrink tube near each connector to bunch the wires together.

Anyone remember the Sapphire Radeon HD 5970??? It was notorious for frying cables. http://www.vgamuseum.info/media/k2/items/cache/8dd1b15b07ee959fff716f219ff43fb5_XL.jpg

Thanks for the links, I'll read those articles. I'm curious if one needs a new model PSU to use the 12-pin connector.

I got the AX1200s back when I thought it was a good idea to run 4 GPUs on a single motherboard. I now use only one or two and they must be on 16x 3.0 sockets. I believe PSUs are most efficient at 80% load so I'm under loading them now but I use what I have.

Still using Corsair AX1200 power supplies running 3 gpus on two hosts. It is a good supply.

Never ran AMD cards so far. The Nvidia cards I have owned have been moderate power consumers mainly. Never overheated a cable or connector yet.

Good for you making your own cables. You are using good tools and wire it seems.
I have all kinds of crimping tools myself. I build custom cables for telescope mounts.

I believe you that you have examples of burnt wires and connectors. I have seen many posted examples of burnt connector myself. I have always used good cables and supplies and keep the loads within reason.

I run two hosts with 4 gpus each on EVGA 1600T2 power supplies pulling over 1100W with no issues for years now.

Lots of quality modular power supply makers are providing either a free upgrade 12 pin Nvidia cable or for a modest charge.

Never had burnt 8 pin or 6 pin but always mechanical stuff die. Pumps or fans plus the GPU die. Like I said I had 4 Turing go cold out of 6. Overclocking the core shouldn't kill it all. Other generations didnt bed so easy.

I'd rather run a motherboard with 5 GPU or more. The mb lasted 5 years with 5 GPUs. Unexpectedly happens no matter what the best psus you run. I have 2 1200W psu on 1 motherboard. GPU maybe dont like win 8.1 The Turing that died should last offering a 3 year warranty. GPU Turing years lifespan length is lottery. Sometimes next generation better to wait for. (A few weeks past the first release.) Ampere release a mess. Worst in awhile. Good luck buying one today.

I have had only one gpu electrical death in probably 30 cards or so in ten years. But the pumps in the hybrid gpus die fairly regularly.

Nine Turing cards in use and running well so far.

Not too annoyed at the Ampere availability as they don't work here yet anyway.

Once there is an app that will run Ampere properly I might start getting a little antsy.

https://www.phoronix.com/scan.php?page=article&item=nvidia-rtx3080-compute&num=1

https://www.phoronix.com/scan.php?page=article&item=blender-290-rtx3080&num=1

The 3080 is faster at more power than 2080ti at various benchmarks.

https://www.phoronix.com/scan.php?page=article&item=nvidia-rtx3080-compute&num=1

https://www.phoronix.com/scan.php?page=article&item=blender-290-rtx3080&num=1

The 3080 is faster at more power than 2080ti at various benchmarks.

Yes, not very efficient at all. On Windows at least with the standard gpu control utilities you can shove the power usage down and cut the boost clocks and save some power.

Yes, not very efficient at all. On Windows at least with the standard gpu control utilities you can shove the power usage down and cut the boost clocks and save some power.

I do know that. I was being terse and the main advantage of Windows utilities is the ability to undervolt.

You can undervolt and save a lot of power and not have any appreciable drop in performance or increase in crunching times.

Yes, not very efficient at all. On Windows at least with the standard gpu control utilities you can shove the power usage down and cut the boost clocks and save some power.

You can do it under Linux as well. You know that.

Judging by the relevant benchmarks:

...snip...

For MD simulations there's no point to upgrade the RTX 2080Ti to the RTX 3080.
You get more performance in direct ratio of the power consumption.

i still believe that when such an app comes, the 3080 WILL be better "per watt" than the 2080ti.

From my reading of tech deep-dive at AnandTech on the Ampere arch, the extra *new* FP32 register is just another generic register like the original in Turing.

Not anything to do with rasterization or Tensor cores. The dual purpose INT32/FP32 register can do EITHER integer or floating point calcs. Not both at the same time.

And since we don't do INT32 calcs for GPUGrid I believe, we are never going to use the INT32 function of the register.

Rasterization on a gpu involves integer instructions. But we are not blitting pixels to a screen, so nothing to worry about. That statement is valid only for a card not driving a display. If you only have the single card doing both drive display and crunching, that register will be busy driving pixels.

However, if you are running other projects on the card, Einstein for example, I believe it's tasks do a considerable amount of INT32 calcs at the beginning of the task. So trying to run both projects at the same time should hamper the performance of GPUGrid work.

FP32 is FP32. Doesn’t matter what they are used for. They can be used for that operation.

Ampere has twice the number of FP32 cores than Turing. And if all you’re doing is FP32, all can be used at the same time. That’s where the 2x performance figures come from. If the workload is primarily FP32 operations, then you can expect to see major speed improvements. This has been widely reported.

Half of the FP32 cores can be used every cycle (the same number as Turing) . Half of them are “extra” added in a separate data pipeline. These are the ones that are split between FP32/INT32. The only way you won’t be able to use them, is if your workload is using a significant percentage of INT32 operations, and/or your application isn’t compiled in a way to take advantage of the additional pipeline.
____________

FP32 is FP32. Doesn’t matter what they are used for. They can be used for that operation.
Ampere has twice the number of FP32 cores than Turing. And if all you’re doing is FP32, all can be used at the same time.

That’s where the 2x performance figures come from. If the workload is primarily FP32 operations, then you can expect to see major speed improvements. This has been widely reported.

Half of the FP32 cores can be used every cycle (the same number as Turing).
Half of them are “extra” added in a separate data pipeline.

Sorry to say, but you are misinformed or have misinterpreted the specs. The 2x FP32 is the new data path.

FP32+INT32 performance is the same as Turing

FP32+FP32 is where there is increased performance with Ampere. Pure FP32 loads will see the most benefit.



https://www.tomshardware.com/amp/features/nvidia-ampere-architecture-deep-dive

You seem to be misinformed on Turing operation also. Turing absolutely can do FP32+INT32 simultaneously. That was the big feature of the Turing architecture when it was released.

https://developer.nvidia.com/blog/nvidia-turing-architecture-in-depth/

The Turing architecture features a new SM design that incorporates many of the features introduced in our Volta GV100 SM architecture. Two SMs are included per TPC, and each SM has a total of 64 FP32 Cores and 64 INT32 Cores. In comparison, the Pascal GP10x GPUs have one SM per TPC and 128 FP32 Cores per SM. The Turing SM supports concurrent execution of FP32 and INT32 operations

Waiting for the compute results for the cards. Would be the correct comparison of compute performance against the RTX 2080 Ti since the wattages are comparable.

we need a proper CUDA 11.1+ app from GPUGRID before we can compare anything
____________

Well I tried and failed to get my hand on Ampere cards once again.
I wanted to buy 2. No thanks am not going pay 1000$ or 1200usd to a Ebay scalper for 3070 or 3080.

If you haven't heard: Nvidia is saying no ample 3080 3090 supply until 2021.

The Rtx 3070 was sold out instantly this morning at microcenter store in Cambridge MA.
(Same place I purchased a 2080ti openbox Asus rog cod edition for under 800$.)

Amazon and newegg are sold out too. Better luck next time. I'd like to test 3080 and 3070 out against the 2080ti.

Unbelievable how much demand there is for Ampere.
I arrived 30 mins before opening. Crazy how people waited overnight.

Funny thing is among all the creators and gamers only a few have heard of BOINC platform.
Many knew about folding at home and of course coin mining. Sadly none heard about GPUGRID.

All this hoopla reminds me of old apple phone releases.

you have to go in-person (and usually at least a day in advance, waiting in line like black friday) to get the cards from Microcenter. you can't buy 30-series online.

one of the better methods for online shopping (in the US) is the queue system at EVGA's website. that's what I did this morning. they had a launch-day specific sku with a free t-shirt on their 3070 Black model for $499. they only had a limited number from what I can tell since you could only be added to the queue for about 5 minutes from about 6:05-6:10am PST.

I'm in line for both the launch-day 3070 Black and the normal 3070 Black. I expect to get my email to buy the launch day card either today or tomorrow. You can still get in the queue for the 3070 or 3080, or 3090, but just know that you'll have thousands ahead of you at this point so you might be waiting a long time before you are able to actually buy it.

don't even bother trying to buy from Best Buy or Newegg. they don't have any bot protections like EVGA does. most of the cards sold there this morning went to the bots.

but even if you have a Ampere card in-hand. you will be unable to test on GPUGRID until the admins/devs release a new application that supports the new cards. A few people have tried already and the tasks just error out immediately because the current app doesnt support them.
____________

well. my email came, order placed.

EVGA RTX 3070 Black on the way :)

now just need to wait for GPUGRID to release a new app!
____________

Congratz Ian. Now you get to be the first guinea pig of a new app.

Ian, have you tried FAH scoring it yet?
I guess you might be able to crunch for Greg Bowman until Toni's crew updates ACEMD. Folding@Home is more generous with points. I don't know if they are based on cobblestones or not. If so, Bowmanlab's app runs faster.

Too bad that F@H is no longer on the BOINC platform so that points would apply.

I know Dr Bowman is a Stanford graduate and they competed with Berkley on this stuff during development. I think they need to develop the F@H user interface further to match the power that BOINC gives experienced crunchers.

That said, I don't know of any other GPU based research dealing directly with a COVID-19 vaccine out there and in communicating with Greg I have found him to be quite approachable and conversational.

I will receive the card next Thursday 11/5.

I don’t participate in FaH. And from what I understand, their application is not CUDA, but rather OpenCL. At least FAHbench is. So it likely won’t see the most benefit from the new architecture. Having a CUDA 11.1 app is crucial for this.

What I did do in order to try to test CUDA performance is ive recompiled the SETI special CUDA application for CUDA 11.1. And I have an offline benchmarking tool and some old SETI workunits so I can check relative CUDA performance between a 2080ti and the 3070. This should give me a good baseline of relative performance to expect with Ampere when the GPUGRID app finally gets updated.

____________

Thanks for pointing out the openCL vs CUDA factor, Ian. I didn't think about FAHcore using a different function.

That makes me wonder if PassMark GPU direct compute scores might be a better reference than FAH scores for predicting performance running ACEMD.
(edit)After doing some checking I see PassMark's bench is not CUDA either. Is there a CUDA1.1 benchmark test anywhere?

I don’t participate in FaH. And from what I understand, their application is not CUDA, but rather OpenCL. At least FAHbench is.

As of today, your folding GPUs just got a big powerup! Thanks to NVIDIA engineers, our Folding@home GPU cores — based on the open source OpenMM toolkit — are now CUDA-enabled, allowing you to run GPU projects significantly faster. Typical GPUs will see 15-30% speedups on most Folding@home projects, drastically increasing both science throughput and points per day (PPD) these GPUs will generate.

... core22 0.0.13 should automatically enable CUDA support for Kepler and later NVIDIA GPU architectures ...

So it likely won’t see the most benefit from the new architecture. Having a CUDA 11.1 app is crucial for this.

What I did do in order to try to test CUDA performance is ive recompiled the SETI special CUDA application for CUDA 11.1. And I have an offline benchmarking tool and some old SETI workunits so I can check relative CUDA performance between a 2080ti and the 3070.

This should give me a good baseline of relative performance to expect with Ampere when the GPUGRID app finally gets updated.

Thank you for sharing your insight, Zoltan. I much appreciate your perspective and experience in DC.

After the recent rollout of CUDA support on F@H, that I shared with you in the very same thread initially, there now exist specific CUDA cores that helped to increase performance of NIVIDA cards tremendously on F@H. As Zoltan pointed out, there are still many unknown variables around the provided data, so take the performance charts provided on their website.

Risking to be off-topic in this thread... I like to diversify my contribution and occasionally run F@H for a couple WUs before switching back to GPUGrid. What I like about their infrastructure is that you can check out a project description for all currently running WUs in addition to a preference you can set directly in the software to specify what cause (disease) you want to focus your computational sources on. You can choose from various targets, such as high priority projects, Parkinsons, cancer, Covid-19, Alzheimer, Huntington. Those projects change from time to time depending on what research is currently being conducted. In the past there have been other research projects around Dengue fever, Chagas disease, Zika virus, Hepatitis C, Ebola virus, Malaria, antibiotic resistance and Diabetes. If there is currently no work for the chosen preference, it immediately switches to the highest priority projects. In that sense, I feel like F@H offers an informational advantage over GPUGrid, but certainly lacks the flexibility and options for cruncher that F@H lacks. My card (GTX750 Ti) finishes most tasks at F@H ahead of the first deadline and thus receives the early finish bonus points. It matches my RAC on GPUGrid if running exclusively 24/7 at around 100k credits. In the end I believe both projects have their own place and complement each other.

Anyways, I am also looking forward to seeing those benchmark results. Awesome that you managed to grab one in spite of the current sparsity of supply!

if they get the CUDA changes into the FAHbench application, I’ll try it out. But last time I looked at it, the benchmarking app still only used opencl. When comparing something like two different cards you need to eliminate as many variables as possible. Using the standard FAH app without the control the run the same exact work units over and ove the best I could do is run run each card for a few months to get average PPD numbers from each. I’m just not willing to put that level of effort into it, when I can likely get the same results from a quicker benchmark.

From what I’m reading, to get the benefit of the new data pipeline, you need CUDA 11.1, not just cuda 11.

While seti does different types of calculation, the comparison of architectures and models should be very comparable. When I moved from SETI to GPUGRID the hierarchy of GPUs remained the same. So the relative performance of 3070 to 2080ti observed with SETI should translate pretty closely with what we will probably observe on GPUGRID (provided they build an 11.1 app also).
____________

I got my EVGA 3070 black today and did some testing

PLEASE KEEP IN MIND: this is very preliminary testing with an application that does different calculation types than GPUGRID. this is merely an attempt to compare against another BOINC project using a CUDA app in a controlled benchmark kind of way.

testing platform:
Asrock X570M Pro4 mATX (x16 slots on position 1 and 4)
AMD Ryzen9 3900X @ 4.20GHz
16GB GSkill DDR4-3600 CL14
Phanteks mATX case with only 4 PCI expansion slots
SETI@home special application recompiled by me for CUDA 11.1 on Linux

Had a bit of trouble getting it to work in my Linux system at first. Thought it might be the drivers, 3070 "needs" the 455.38 driver, but previous tests and info from others have shown that slightly old drivers usually work just fine for new cards when the architecture is the the same, so 455.23.04 *should* work just fine, only reporting a generic card name like "Nvidia Graphics Device". So i went through the trouble to install the nvidia drivers from the run file. installed 455.38 with little issue, and it still was being very flaky, randomly dropping the GPU, extreme lag navigating the desktop, very low GPU utilization, or even failing to boot.

Narrowed it down to the PCIe gen3 ribbon risers and the card constantly trying to run at PCIe gen4 no matter what settings I used in the BIOS (I set all slots to gen3 but nvidia-settings still reported running at gen4). due to the motherboard and case layout, i was unable to install the card directly into the motherboard bottom slot due to lack of enough space and i don't have any gen4 capable risers. So i carefully removed and set aside the 2080ti which is custom watercooled and hooked in-line with the CPU to be able to plug the 3070 directly into the top slot.

Finally working as it should. no random issues.

Testing results with CUDA 11.1 special app:
about 75-85% speed vs. my 2080ti on my collection of WUs
1980-1995 MHz (same as 2080ti)
14000MHz mem clock on both 2080ti and 3070
98-99% GPU Utilization reported
but only ~190-200W power as reported by nvidia-smi unrestrained, 220W PL (2080ti used 225W, at the PL)

keep in mind that Ampere went back to 128 cuda cores per SM, like Pascal, but unlike Pascal the data paths are different. Also remember that the special app seems to heavily favor SM count, likely due to the way it is coded. it is my guess that either SETI calculations use more integer math, or both FP32 data paths are not being used.

3070 has 46 SMs, the 2080ti has 68 SMs
so with 67% the SMs (and CUDA cores), its giving ~80% the performance, not bad.

but power used comparison isn't so kind.
at 89% power used, its giving ~80% performance.
this might be able to be improved upon with power limiting, but I didn't test this right now

So I haven't lost hope yet. I'm hopeful that if GPUGRID does use mostly FP32, and they recode their app to take advantage of both pipelines in a CUDA 11.1 app, that we could see better performance.

I'm waiting for the new app to be ready.
____________

i attempted to build FAHbench with CUDA support (the information says it's possible), but I'm hitting a snag at configuring OpenMM.

I get errors when trying to configure OpenMM with CMake (as outlined in the FAHbench instructions)
I cant check the OpenMM docs since http://docs.openmm.org seems to be a dead link
I can't ask about it on the forums unless I register
I can't register unless they accept my request (says it'll take 72 hrs)
____________

I got my EVGA 3070 black today and did some testing
...
Testing results with CUDA 11.1 special app:
about 75-85% speed vs. my 2080ti on my collection of WUs

Kudos to you! From what I understand so far, I have to support Zoltan's opinion. I really must stress that I am very keen on efficiency as that is sth that everyone should factor in their hardware decisions.

Anyway, it still seems to offer a valid value proposition for me. Especially at this price point. And it is yet very early and future support of RTX 30xx cards could definitely offer some potential for optimisation.

I got my EVGA 3070 black today and did some testing
...
Testing results with CUDA 11.1 special app:
about 75-85% speed vs. my 2080ti on my collection of WUs

Thank you for all the effort you have put in this benchmark!
Regrettably your benchmarks confirmed my expectations.
Performance wise it's a bit better than I've expected (67.6%+10%~74.4% of the 2080 Ti).
Power consumption wise it seems as of yet that it's not worth to invest in upgrading from the RTX 2*** series for crunching.

From the issue raised on the OpenMM github repo, it seems they let their SSL certificate expire back in September.

And no one has done anything about it.

https://github.com/openmm/openmm-org/issues/38

https://github.com/openmm/openmm

This OpenMM forum?

https://simtk.org/plugins/phpBB/indexPhpbb.php?group_id=161&pluginname=phpBB

The compiler optimizations for Zen 3 haven't made it into any linux kernel yet either. GCC11 and CLANG12 are supposed to get znver3 targets in the upcoming 5.10 kernel next April for the 21.04 distro release.

https://www.phoronix.com/scan.php?page=news_item&px=AMD-Zen-3-Linux-Expectations

got the 3070 up and running on Einstein@home for some more testing.


As far as Einstein performance:

the 2080ti does the current batch GW tasks in about 4 minutes, using 225W (360 t/day)
the 3070 does the current batch GW tasks in about 5 minutes, using 170W (288 t/day)
the 2080ti does the FGRP tasks in about 6 minutes, using 225W (240 t/day)
the 3070 does the FGRP tasks in about 7:20 minutes, using 150W (196.4 t/day)

so for Einstein GW tasks, the 3070 is about 5% more efficient
and for the Einstein GR tasks, the 3070 is about 19% more efficient

now this isnt to say you should buy Ampere (or even nvidia cards) for Einstein, since some of the newer AMD cards perform much better there (faster and overall more efficient). But this will serve as an additional data point with a different processing type. you can see the efficiency improvements here, especially on the GR tasks.
____________

I wonder if you slowed down the 2080Ti (reducing GPU voltage accordingly as well) by 20% to match the speed of the 3070, would it be about the same effective? (or even better in the case of GW tasks)

the 2080ti was already power limited to 225W. its not possible (under Linux) to reduce the voltage. there are just no tools for it. reducing the power limit will have the indirect effect of reducing voltage, but I don't have control of the voltage directly. I could power limit further, but it will only slow the card further. you start to lose too much clock speed below 215-225W in my experience (across 6 different 2080tis). the 2080ti was also watercooled with temps never exceeding 40C so it had as much of an advantage as it could have had.

the 3070 was run at a 200W power limit, but it never even came close to that in these Einstein loads, with speed probably only limited by temp/clock boost bins. But that's really par for the course on mid-level Nvidia cards running Einstein tasks, the GR tasks are just light weight and dont pull a lot of power, and the GW tasks are more CPU bound than anything else so you dont get full GPU utilization. further efficiency gains could probably be made here on the 3070 with more power limiting and overclocking, but I didn't bother for this test.
____________

Any apps i can use yet on my 30s cards?

Any apps i can use yet on my 30s cards?

https://www.gpugrid.net/workunit.php?wuid=27026028

with no Ampere compatible app, and no mechanism to prevent work from being sent to incompatible systems, situations like this will only become more common as more and more users upgrade to these new cards.

3 out of the 6 systems that have handled this WU were using Ampere cards and failed because of it.

If we had an Ampere compatible CUDA 11.1 app, this would have been completed by the first system.
____________

I've "saved" a workunit earlier (my host was the 7th crunching it - the 1st successful one). It had been sent to two ampere cards before.

i have a couple like that that I similarly saved. even some _7s (8th)

this one, 50% of the users had RTX 30-series https://www.gpugrid.net/workunit.php?wuid=27025291
____________

Is there any update regarding nVidia Ampere Workunits??
My Ampere cards are getting bored -.-

my 3080Ti (limited to 300W mind you) did this ADRIA task in under 9.5hrs

https://www.gpugrid.net/result.php?resultid=32642251

anyone with a high power 3090 or 3080Ti run faster? my model 3080Ti will only go up to 366W, but I know some 3080Tis and 3090s can reach into the 400-500W range.
____________

I am not running 3090s or 3080ti cards but I do have some times/comparisons for high-end Turing and Ampere GPUs for Adria tasks.

Nvidia Quatro RTX6000- 8.4 hours
Nvidia RTX A6000- 6.7 hours

I am not running 3090s or 3080ti cards but I do have some times/comparisons for high-end Turing and Ampere GPUs for Adria tasks.

Nvidia Quatro RTX6000- 8.4 hours
Nvidia RTX A6000- 6.7 hours

(The real world GPUGrid performance of the Ampere architecture cards scales with the half of the advertised number of CUDA cores).

Just guessing here but since every new generation of Nvidia cards has basically doubled or at least increased the CUDA core count and since the GPUGrid app as well as a very few other project apps are really well coded for parallelization of computation, you can state that the crunch time scales with more cores.

You can tell how well optimized an application is by how much sustained utilization it produces and how close to the max TDP of the card the app runs.

The GPUGrid apps and the Minecraft apps are the only two apps that I know of that will run at 97-100 utilization through the entire computation at the full power capability of the card.

Kudos to the app developers of these projects. Job well done!

(The real world GPUGrid performance of the Ampere architecture cards scales with the half of the advertised number of CUDA cores).

Is that true for all NVidia GPUs or just Ampere? Just out of curiosity, why is it this way?

As long as you can keep an INT32 operation out of the warp scheduler, then Ampere series can do two FP32 operations on the same clock.

and since the real performance doesn't scale with the FP core count, that leads us to the conclusion that the GPUGRID app must be coded with a fair number of INT operations which cut into the FP cores available (half of the FP cores are actually shared FP/INT cores and can only do one type of operation at a time, while the other half are dedicated FP32). This explains the massive performance boost of Turing cards over Pascal (at the same FP count) for GPUGRID since Turing introduced concurrent FP/INT processing.

https://developer.nvidia.com/blog/nvidia-turing-architecture-in-depth/

the Turing SM adds a new independent integer datapath that can execute instructions concurrently with the floating-point math datapath. In previous generations, executing these instructions would have blocked floating-point instructions from issuing

... the GPUGRID app must be coded with a fair number of INT operations which cut into the FP cores available (half of the FP cores are actually shared FP/INT cores and can only do one type of operation at a time, while the other half are dedicated FP32).

Einstein scales much better with FP core count on Ampere ...

all depends on how the app is coded to use the hardware, and sometimes you can't make an app totally optimized for a certain GPU architecture depending on what kinds of computations you need to do or what coding methods you use.

... the GPUGRID app must be coded with a fair number of INT operations which cut into the FP cores available (half of the FP cores are actually shared FP/INT cores and can only do one type of operation at a time, while the other half are dedicated FP32).

Perhaps MD simulations don't rely on that many INT operations, so it's independent from the coder and from the cruncher's wishes (demands).

Einstein scales much better with FP core count on Ampere ...

The Einstein app is not a native CUDA application (it's openCL), it's not good at utilizing (previous) NVidia GPUs, making this comparison inconsequential regarding the GPUGrid app performance improvement on Ampere. It's the Ampere architecture that saved the day for the Einsten app, so if the Einstein app would be coded (in CUDA) the way it could run great on Turing also, you would see the same (low) performance improvement on Ampere.

all depends on how the app is coded to use the hardware, and sometimes you can't make an app totally optimized for a certain GPU architecture depending on what kinds of computations you need to do or what coding methods you use.

Well, how the app is coded depends on the problem (the research area) and the methodology of the given research, and the program language, which is chosen by the targeted range of hardware. The method is the reason for the impossibility of "the GPUGRID app must be coded with a fair number of INT operations" demand. (FP32 is needed to calculate trajectories.) The targeted (broader) range of hardware is the reason for Einstein is coded in openCL, resulting in lower NVidia GPU utilization on previous GPU generations.

👏👍