you'd need to explain how you setup MPS. what environment variables did you use? the more information you can provide, the better.

I use all the environment variables you've posted.
This

CUDA_MPS_ACTIVE_THREAD_PERCENTAGE=70

is just an educated guess, or it is backed by some empirical evidence?

Its backed by empirical evidence by the TAAT and GPUUG team members who have been running the mps-server for over a year now. Most with multiple gpus in each host and most running from 2-6 concurrent task on each gpu.

You need to experiment with each gpu and task type according to how much VRAM each card has and the effective compute time running multiples. Most people running 2-3 tasks settle on 70% active thread percentage with 40% being popular for >4 task running.

you'd need to explain how you setup MPS. what environment variables did you use? the more information you can provide, the better.

I use all the environment variables you've posted.
This

CUDA_MPS_ACTIVE_THREAD_PERCENTAGE=70

is just an educated guess, or it is backed by some empirical evidence?

how many tasks are you running at one time? MPS is really only beneficial when you run 2 or more.

a 70% active thread percentage is good to run 2x tasks per GPU.

if you want to try more, you can drop it to 40% and run 3x tasks per GPU.

the runtime for each task will increase. but it should be by less than than the factor of increased tasks per GPU, so you will get more daily production as a whole. roughly 20-30% boost in overall tasks per day or ppd wouldnt be unexpected (vs just running 1x task without MPS). I saw good performance uplift like this last time ATM had a lot of work.

---

Right now I'm running 2 tasks simultaneously. The power consumption went up to 380~460W.
Later I'll give a try to run more at one time.
It's a bit cumbersome to evaluate the PPD gain with the present mixture of tasks. :)