I've done some research in the past, and RTX 2060 to 2080 (including 2060 Super and 2070Super), can run 'fine' at 125W.
With a little overclock, and power capped to 125W, the 2060, 2060 Super, and 2070 ; both open ended and blower style GPUs run pretty much at stock speed values (but at 125W instead of 170 to 225W).

The 2070 Super and 2080 is best ran between 129 and 135W for an open bench, and depending on the cooling around 145-155W for inside a case.

The 2080Ti makes too much of a performance penalty at those wattages, depending on the project you'll have to set it anywhere from 180W to 225W on an open bench, and add about +15% of wattage when using inside a case.


For multi GPUs under a debian based Linux:
(like all the Ubuntu variants, Debian, or Mint), you have to run the latest supported Operating system which is 18.04 LTS.
Later versions will give desktop errors.
Then install the Nvidia .run drivers from their website (geforce.com/drivers or nvidia.com/download )
- After downloading the drivers, right click the .run time file, and enable it to execute (or go to terminal and do: "chmod +x nvidia_driver.run")
- log out of your GUI
- Go into terminal (CTRL+ALT+F1) (sometimes it's F2 or F8, depending on OS)
- To terminate GUI do:

if you're on ubuntu or mint (which is probably most people), it's a lot easier to use the PPA to install the nvidia drivers.

The 2070 Super and 2080 is best ran between 129 and 135W for an open bench, and depending on the cooling around 145-155W for inside a case.

I ran some tests on my RTX 2070s (that's plural, not s for Super) over the last several days.

I think I will settle on 150W PL, +125core/+400mem OC. this gives me a good balance of power draw and overall output, core clocks stay around 1900MHz, some more some less these are the low end EVGA black models. I'm sure I can increase efficiency going deeper into power limiting, but I don't want to give up too much more raw performance. I was able to match or even exceed overall output at 150W/+125core/+400mem than I was at 165W/+75core/+400mem, reducing system power draw by about 100 watts (7 GPUs) and reducing GPU temps by about 5C across the board. win-win.

I also found that, at least for GPUGRID, there's not much benefit overclocking the memory beyond the stock P0 clocks. (+400 on the RTX cards in Linux, +200 in Windows) and you can probably get away with even less. GPUGRID isn't too bottlenecked by the memory speed, and more dependent on GPU core clocks and PCIe bandwidth. GPU memory use and bus utilization is rather low on GPUGRID tasks. overclocking the memory lot will just eat into your power budget and reduce your core clocks, resulting in less output overall at a set power limit.
____________

I'm trying to understand this but the terminology you guys are using differs from what's in the nvidia-smi documentation: http://developer.download.nvidia.com/compute/DCGM/docs/nvidia-smi-367.38.pdf

E.g., using sudo nvidia-smi -q reports 4 clock speeds for a 1080 Ti:
Graphics: 1911 MHz
SM : 1911 MHz
Memory : 5005 MHz
Video : 1620 MHz
In one place the clocks are referred to as GPU & RAM and in another as Core & Mem. Aren't they all a form of RAM?
To set the clocks:
-ac, --applications-clocks=MEM_CLOCK,GRAPHICS_CLOCK
Specifies maximum <memory,graphics> clocks as a pair (e.g. 2000,800) that defines GPU's speed while running applications on a GPU.
My guess is that GPU or Core=GRAPHICS_CLOCK and RAM or Mem=MEM_CLOCK.

I was wondering about the p0 option and will explore that.
What about the persistence mode? Any reason to implement that?

sudo nvidia-smi -pl 150 -pm -p0

aurum@Rig-26:~$ sudo nvidia-smi -pm 1 ; sudo nvidia-smi -pl 150
Persistence mode Enabled for GPU 00000000:02:00.0.
All done.
Power limit for GPU 00000000:02:00.0 was set to 150.00 W from 150.00 W.
All done.

Changes power limit fine but I'm stuck in P2:
Performance State : P2
Power Limit : 150.00 W
Clocks
Graphics : 1645 MHz
SM : 1645 MHz
Memory : 5005 MHz
Video : 1392 MHz

I tried to set the Performance State using a startup command:
sh -c '/usr/bin/nvidia-settings --load-config-only -a GPULogoBrightness=0 -a GpuPowerMizerMode=1'
This turns off the baby-blinky lights but still stuck in P2.

aurum@Rig-26:~$ sudo nvidia-smi -pm 1 ; sudo nvidia-smi -pl 150 ; sudo nvidia-smi -ac 5405,2036
Persistence mode is already Enabled for GPU 00000000:02:00.0.
All done.
Power limit for GPU 00000000:02:00.0 was set to 150.00 W from 150.00 W.
All done.
Setting applications clocks is not supported for GPU 00000000:02:00.0.
Treating as warning and moving on.
All done.

How can I set the clocks or force P0???

sounds like you're running Linux? I can't see since your systems are hidden. i do all my overclocking through the Nvidia X Server settings application that is installed with the nvidia drivers on Linux, and when I've found stable settings, I drop it all into a script and I run it at startup.

but 'GPU' and 'core' are synonymous. they are not the same as memory clocks, which are 'ram' or 'mem'.

~1900 MHz is pretty normal for a 1080ti crunching with no tweaks. you can usually bump that close to 2000MHz with sufficient power. but if you powerlimit the card, it will by default pull the clocks down. you can apply an overclock to the GPU/core with the powerlimit in place and it will compensate and try to give more clocks within the power budget.

your memory clocks are running at the standard 'P2' state that you are stuck in with compute loads. you cannot get the card to run in P0 as it is a limitation in the driver itself. the only thing you can do is overclock the memory in the P2 state to mimic what you would get in the P0 state. your clocks now are 5000MHz, which is "10Gbps", to get the clocks up to the "11Gbps" (5500MHz) you need to overclock the memory.

in the command line you set clocks using an offset with nvidia-settings, not a straight number with nvidia-smi

for one of my RTX 2070s, the command looks like this:

So do I just set -pl 150 and let P2 pick the clocks???
In trying to understand this thread I couldn't tell if it was
-pl AND clocks
or
-pl OR clocks.

I've never overclocked my GPUs but reducing power consumption is very appealing. It's starting to get hot here and I cannot afford my power bill any more. I already pulled the second GPU from all my computers and have started selling them on fleaBay.

PL AND clocks. PL alone will have the affect of reducing power AND losing performance. the point of overclocking on top of the power limit is to gain back lost performance.

here's the startup script I use for my 10x 2070 system. you should be able to figure what's doing what. just modify it to however many GPUs you have in a system, change the offsets to whatever suits you, and change the [4]s to [3]s for Pascal cards. it sets power limits, clocks, and fan speeds all in one script. you need to have applied the coolbits tweak to your Xorg.conf file for these commands to work.

To set the clocks:
-ac, --applications-clocks=MEM_CLOCK,GRAPHICS_CLOCK

[edit]

In one place the clocks are referred to as GPU & RAM and in another as Core & Mem. Aren't they all a form of RAM?

To set the clocks:
-ac, --applications-clocks=MEM_CLOCK,GRAPHICS_CLOCK

This method of setting clocks only work for Maxwell or lower cards. Deprecated for Pascal and later.

For clock control in the later models you need to use the nvidia-settings application.

You use the nvidia-smi application for power level setting.

Did you read my previous post?

You can’t force a high end GTX 10-series card into P0. Only the professional level Quadro and Tesla line cards, and low end cards like 1050s and lower. There was a way to do it on 700 series cards and maybe 900 series cards in Windows, but not Linux if I recall.
____________

The reason that Maxwell and later are not directly overclockable is because Nvidia has never released signed firmware images for the cards.

They have for Kepler and earlier. I had the breakdown in card families wrong earlier. I used to use the -ac method of overclocking on my GTX 670's

This is the original "coolbits" Phoronix.com post.

https://www.phoronix.com/scan.php?page=news_item&px=MTY1OTM

And this snippet explains why. https://www.phoronix.com/scan.php?page=article&item=nouveau-summer-2018&num=1

Re-Clocking
The biggest issue that plagues the Nouveau driver with modern NVIDIA hardware and really hurts its potential adoption is the lack of re-clocking support.

For GeForce GTX 600/700 "Kepler" graphics cards there is manual re-clocking support that has been stable for a while now in the kernel and requires writing "0f" (or the other desired performance state) to /sys/kernel/debug/dri/0/pstate (or similar path depending upon GPU index) to switch from the boot clock frequencies to the highest performance state for the GPU core and memory. At this stage, Nouveau doesn't have any dynamic/automatic re-clocking for their GPUs to switch frequencies based upon GPU utilization. The benchmarks in this article on Kepler hardware as well as Maxwell 1 show the re-clocked performance to their highest (0f) performance state, but unfortunately there isn't re-clocking for Maxwell (GTX 900 series) and newer.

The Maxwell GeForce GTX 900 and Pascal GeForce GTX 1000 series still do not have proper re-clocking support in the Nouveau driver due to the shift to signed firmware images. While NVIDIA released the necessary signed firmware images to Nouveau developers to support hardware acceleration on Maxwell/Pascal, right now it's still stuck to the boot clock frequencies that are very low compared to their advertised clock speeds. NVIDIA would need to release the PMU firmware and likely other documentation for the open-source driver to get to the stage of re-clocking offered on Kepler hardware. But there's been no indication of that happening so unless the Nouveau developers discover some black magic for getting re-clocking working and being able to ramp up the fan speeds too as part of that process, the situation isn't looking good at this moment in time.

Did you read my previous post? You can’t force a high end GTX 10-series card into P0. Only the professional level Quadro and Tesla line cards, and low end cards like 1050s and lower. There was a way to do it on 700 series cards and maybe 900 series cards in Windows, but not Linux if I recall.

Did you read my previous post? You can’t force a high end GTX 10-series card into P0. Only the professional level Quadro and Tesla line cards, and low end cards like 1050s and lower. There was a way to do it on 700 series cards and maybe 900 series cards in Windows, but not Linux if I recall.

Yes but it still reported being in P0 for a 1080 Ti. I just can't get it to work.
<snip>

With nvidia-smi -q reporting:

Attached GPUs : 1 GPU 00000000:02:00.0 Product Name : GeForce GTX 1080 Ti Product Brand : GeForce Display Mode : Enabled Display Active : Enabled Persistence Mode : Disabled Fan Speed : 75 % Performance State : P2

I fixed it, forgot the sudo in my Startup Aps command:

I don't know how it got set to P0 this morning but it was.

That must be it since I immediately ran the nvidia-smi -q query.
It seems to be working good. As GG WUs finish I'll propagate it to my other computers. If I really get a 36% reduction in power consumption that will be amazing. I'm surprised this isn't common knowledge among BOINCers.

Yes, every time the card unloads a compute task, the driver lets the card go back to P0 power state. If you have applied a core and memory overclock to the lower P2 state to approximate the normal P0 power state under a compute load, the overclocks get applied briefly to the P0 state and can sometimes crash the card if the overclocks are a bit much.

Over at Seti, our Linux developer Petri came up with the keepP2 utility which you run in the background and keeps a small compute load on the card to prevent it from transitioning from the P2 state to the P0 state with overclocks applied.

keepP2.zip

That way it is always safe to run an overclock and never fear a lockup when the BOINC compute load disappears.

This made a big difference in the stability of Pascal cards which have an enormous compute penalty compared to Turing. There was no way I could get away with my +2000Mhz memory overclock when the compute load was removed and the card transitioned to P0 state. Instant crash.

Also before I forget, you can get an overclock applied to P2 state in Windows by using the Nvidia Profile Inspector and setting the "Force P2" setting in the Common settings.

https://www.guru3d.com/files-details/nvidia-profile-inspector-download.html

http://i.imgur.com/EhBH1e0.png%5D