Managing non-high-end (slow) hosts

Current extremely long ACEMD3 Gpugrid tasks represent a serious challenge for non-high-end hosts like mine ones.
My current fastest host, based on a Turing GTX 1660 Ti GPU, takes about 1 day and 4,5 hours to process one of these tasks (around 103.000 seconds).
My slowest one, based on a Pascal GTX 1050 Ti GPU, takes about 3 days and 9 hours doing the same kind of tasks (around 291.000 seconds).

Are these kind of slower hosts worth for processing such heavy tasks?
My personal opinion: Absolutely yes, as long as they be reliable systems.
It would be of no use if one host returned an invalid/failed result after several days retaining a task.
Let's take one example for sustaining my opinion:
Task e7s106_e5s196p1f1036-ADRIA_AdB_KIXCMYB_HIP-1-2-RND0214_7 was recently processed at my mentioned slowest host.
It took 290.627 seconds for it to return a valid result. That is: 3 days, 8 hours, 43 minutes and 47 seconds...
But taking a close look to Work Unit #27082868 it was hanging of, it had previously failed on 7 other hosts.
Maximum allowed number of failed tasks for current Work Units is 7, so that the task would have not be resent to any other host if mine had also failed, then resulting in a lost work unit for its intended scientific purposes...

With time, I've had to adapt BOINC Manager settings at my hosts, trying to squeeze the maximum performance for eviting to miss deadlines.
Here are my experiences:

My Computing preferences for a 4 cores CPU host look as follows:

Where:
-1) Use at most 50% of the CPUs, for not overcommitting the CPU for feeding GPU. This leaves two CPU full cores free for attending general system requirements.
-2) Use at most 100% of the CPU time, for preventing to throttle the CPU.
-3) Never suspending, for the task to be processed with the minimum pauses possible (preferably in one go).
-4) Setting tasks buffer to a minimum, for not wasting time in waiting to the current task to finish.
-5) Switch between tasks every 9999 minutes, for giving enough time for the current task be processed in one go, and only then switching to the next.

My Network preferences look as follows:

I set a certain Download/Upload rate limitation, for not saturating network bandwidth for my other hosts.
But I try to set it to high limits, because Download + Upload times count when task deadline is close to the end, or to the credit bonus limit...

And my Disk and memory preferences look as follows:

-1) Current Gpugrid environment takes a high disk space usage. I set all available space to be usable by BOINC, except for a certain security margin for the disk not resulting saturated.
-2) Regarding memory usage, I raise the default limits to empirically tested values, to maximums for the system not becoming unresponsive to its other general tasks.

And regarding system reliability:
- I empirically test every system and each GPU for safe overclock settings, if I apply any. I do prefer a robust reliable system than a slightly faster but sporadically failing one.
- I frequently check temperatures at every hosts, and try to maintain them at reasonable low levels. Lower temperatures result in a more reliable and faster system.
- I perform preventive hardware maintenances when a temperature raise is detected at some element, or when a host starts failing tasks without a known reason.
Regarding this last matter, I share my experiences at The hardware enthusiast's corner thread.

When you are commanding a fleet of assorted slow GPUs (in my case currently 8), it is difficult to hold in mind how long will it take for every of them to finish their tasks.
Here is a screenshot of the spreadsheet that I use for this purpose, at the time of writing this:



For me, it is useful for being aware to request new tasks when the ones in process are next to finish.
This helps to maintain my GPUs crunching most of the time.
An editable copy of this spreadsheet can be downloaded from here.

The slowest GPU's that I am using for GPUgrid are GTX 1070's. These average about 36hrs to process on Windows hosts.

They seem to work fine, just need to let them run. A stable system with a reliable connection is really the best you can do.

I have not tried anything slower like a GTX 1060 or some old Nvidia Grid K2 cards. They are only running Milkyway jobs which they do well with.

There comes a time when the technology outpaces the physical hardware we have available but I am a firm believer of using what we have as long as possible.

Remember, I'm the guy with the "Ragtag fugitive fleet" of old HP/HPE servers and workstations that I have saved from the scrap pile.

There comes a time when the technology outpaces the physical hardware we have available but I am a firm believer of using what we have as long as possible.

Remember, I'm the guy with the "Ragtag fugitive fleet" of old HP/HPE servers and workstations that I have saved from the scrap pile.

I've been running these tasks on a 1060 6gb, they take 161,000 - 163,000 seconds to complete. I will keep running tasks on this card until it can no longer meet the deadlines, I've been trying to hit the billion point milestone which would take another year or so if I could reliably get work units. Lets hope I can still run this card for that long!

Some considerations about Gpugrid tasks deadlines:

The criteria for receiving credits for a valid returned task is not strictly that it fits inside the 5 days deadline... Let's explain it.

The official deadline for a Gpugrid task is currently 5 days, counting from the moment of Gpugrid server starts sending it to the receiving host.
5 days are exactly 432000 seconds.
This time includes the task download time, the resting time until the task starts being processed, the pause times (if any), the total processing time, and the upload time of the result.
When this time is past, the server will generate a copy of the same task, and resend it to a new host.

Here comes the hint:
An overdue task will not receive any credits for a returned result when it goes outside its deadline AND a valid result is received from the other host first.

Continuing the process of a task past its deadline, is in some way a bet.
Depending on the receiving host, in practice, the deadline is extended by the time that this new host takes to receive, process, and report a valid result for the task.
If the new host is the fastest one for the current set of ADRIA tasks, it will take a mere 5 hours to process... That is the minimum deadline extension that you might expect for an overdue task.
My fastest host takes 1 day and 4,5 hours with these tasks. That is the minimum extension that you could expect from my fleet.
If the newly re-sent task gets a slower-than-medium host, you could expect an extension even longer.
- If you report a valid result for your overdue task even 1 second before the new host, both tasks will receive the base (no bonus) credits amount.
- If the new host reports a valid result even 1 second before yours, it will get the credits awarded, and your task will be labeled with "Completed, too late to validate", 0 credits.

One practical example:
I decided to test if I could get an old Maxwell GTX 750 Ti GPU to fit in deadline one of the current ADRIA extremely long tasks.
I awakened my host #325908, I applied an aggressive +230 MHz overclock to the GPU, and I received task #32657449
It took 473615 seconds to process this task. Too long for fitting the deadline!
This task was hanging from WU #27084724
My host returned a valid result 45150 seconds past the original deadline, but 17295 seconds before than the host that received the re-sent task.
Both tasks received 450000 credits... This time I won my bet! 🎉

If I had aborted, extended beyond 17295 more seconds, or failed my task, the other host would have received 675000, 50% bonused credits, for returning its result inside the first 24 hours.
It is an undesirable side effect, and I apologize for this.
My example task exceeded the 5 days deadline by 45150 seconds (12 hours, 32 minutes, 30 seconds in excess)
I'll give a last try to a new task by carefully readjusting the overclock for this 46 Watts low-power-consuming GPU and its harboring host.
If It is still failing the 5 days deadline, I'll retire this GPU definitely for processing the current Gpugrid ADRIA tasks.
Conversely, if I'm successful, I'll publish the measures taken. I strongly doubt it, since I have to trim more than 13 hours in processing time 🎬⏱️

Well said ServicEnginIC

At one time there was data on the Performance tab that would give users a rough idea of where the GPU's they had would fit in.

There are a great number of variables that affect GPU performance but they could at least tell if they were in the ballpark.

I would very much like to see the Performance tab reinstated so that these general comparisons were available.

While it's good for users to keep using GPU's as long as they are viable there comes a time when they should be retired or used for other projects.

I hate to see people burn time and energy with little or no result to show for it.

Maybe someone could come up with a general guide on what GPU's are useful for GPUgrid that would go back 2 or 3 generations.

If users had a breakpoint where they should seriously consider upgrading to something more recent then they will at least have something to work toward.

If users had a breakpoint where they should seriously consider upgrading to something more recent then they will at least have something to work toward.

I fully agree, jjch.

I reproduce my own words at Message #53367, in response to user Piri1974, where both slopes of your exposition were mentioned...

But anyway, I would not recommend buying the GT710 or GT730 anymore unless you need their very low consumption.

I find theese both models perfect for office computers.
Specially fanless models, that offer a silent and smooth working for office applications, joining their low power consumption.
But I agree that their performance is rather scarce to process at GPUGrid.
I've made a kind request for Performance tab to be rebuilt
At the end of this tab there was a graph named GPU performance ranking (based on long WU return time)
Currently this graph is blank.
When it worked, it showed a very useful GPUs classification according to their respective performances at processing GPUGrid tasks.
Just GT 1030 sometimes appeared at far right (less performance) in the graph, and other times appeared as a legend out of the graph.
GTX 750 Ti always appeared borderline at this graph, and GTX 750 did not.
I always considered it as a kind invitation for not to use "Out of graph" GPUs...

So, in many cases, in order to be up to date GPU-wise, it would mean to buy a new PC :-(

Overclocking to wacky (?) limits

Conversely, if I'm successful, I'll publish the measures taken. I strongly doubt it, since I have to trim more than 13 hours in processing time 🎬⏱️

sudo nvidia-xconfig --thermal-configuration-check --cool-bits=28 --enable-all-gpus

Yesterday, a new batch of tasks "_ADRIA_BanditGPCR_APJ_b0-" came out.
It seems that the project has attended the request for reducing the size of tasks.
This will allow for slower GPUs to process them into the 5 days deadline.

Based on estimations from this morning (about 7:00 UTC), all my currently working GPUs will return their tasks in time.
The GTX 1660 Ti is running for getting full bonus (+50% for result returned before 24 hours)
From GTX 1650 SUPER down to GTX 1050 Ti are running for getting mid bonus (+25% for result returned before 48 hours)
Even GTX 750 Ti will get with no problem its base credit for returning result into the 5 days deadline (before 120 hours)



This might also solve the problem of sporadic oversized result files not being able to upload. Good.

could it be that a GTX1650 is not able to crunch the current series of tasks?

Today, for the first time I tried GPUGRID on my host with a GTX1650 inside, and the task failed after some 4 hours:

https://www.gpugrid.net/result.php?resultid=32721627

excerpt from the stderr:

ACEMD failed:
Error invoking kernel: CUDA_ERROR_LAUNCH_TIMEOUT (702)
19:28:44 (6344): bin/acemd3.exe exited; CPU time 13408.734375

So, as sorry as I am, I will go back to crunch other projects on this one host :-(

could it be that a GTX1650 is not able to crunch the current series of tasks?

Today, for the first time I tried GPUGRID on my host with a GTX1650 inside, and the task failed after some 4 hours

could it be that a GTX1650 is not able to crunch the current series of tasks?

well, the GPU temperature was at 60/61°C - so not too high, I would guess.
GPU runs at stock frequency, but I could try to go below as suggested by Zoltan.

However, meanwhile my suspicion is that the old processor
Intel(R) Core(TM)2 Duo CPU E7400 @ 2.80GHz
may be the culprit.

I will give it one more try; if it fails again, I guess this host is not able to successfully run GPUGRID (however, it works fine with WCG GPU tasks, and with F&H)

However, meanwhile my suspicion is that the old processor
Intel(R) Core(TM)2 Duo CPU E7400 @ 2.80GHz
may be the culprit.

However, meanwhile my suspicion is that the old processor
Intel(R) Core(TM)2 Duo CPU E7400 @ 2.80GHz
may be the culprit.

Surely you're right.
For a two-core CPU, I'd recommend to set at BOINC Manager Computing preferences "Use at most 50 % of the CPUs".
This will cause that one CPU core to remain free for feeding the GPU.

Every of my five GTX 1650 graphics cards are currently taking less than 48 hours, in time for getting mid bonus (+25%).
I published a table of computing times at Message #57886

Every of my five GTX 1650 graphics cards are currently taking less than 48 hours, in time for getting mid bonus (+25%).
I published a table of computing times at Message #57886

But, as said, I'll give it another try once new tasks become available.

Firstly, I don't think your issue that you experienced is related to overclock or GPU temps at all, usually if temps or OC are the culprit you'll get a particle coordinate is nan error (but the nan error can also be a bad WU and not your fault, more on that later). Your error was a CUDA timeout. likely the driver crashed and the app couldn't hook back in. I'm on the fence if your CPU is the ultimate reason for this or not. certainly it's a very old platform to be running on Windows 10, so it's possible there are some issues. If your comfortable trying Linux, particularly a lightweight version with less system overhead, you might try that to see if you have a better experience with such an old system.

Your CPU being a Core2Duo, this architecture does not have a dedicated PCIe link to the CPU. it uses the older architecture where the PCIe and memory connect to the Northbridge chipset and the chipset is what has a single 10.6GB/s link to the CPU, the memory will take most of this bandwidth unfortunately, and since GPUGRID is pretty heavy on bus use, I can see some conflicts happening on this kind of architecture. but the CPU power itself shouldnt be an issue if you're trying to run 1 GPU and no CPU work.

Secondly, with regards to the work that is flowing this morning, a lot of them are bad WUs giving the nan error, so you can't jump to conclusions that whatever you changed wasn't effectual. I have errored like 80% of the WUs received this morning and my system is rock solid stable. If you check the WU record for your tasks this morning you will see that all of your wingmen errored out too, so it's not just you.
____________

Ian&Steve C., thanks for the thorough explanations.

I tried it one more time, and again the task failed after some 7.000 seconds.

Excerpt from stderr:

ACEMD failed:
Error invoking kernel: CUDA_ERROR_LAUNCH_TIMEOUT (702)

the complete report can be seen: https://www.gpugrid.net/result.php?resultid=32722583

So it seems clear that the current setting (hardware, software) is not working with GPUGRID :-(
Which is too bad, because before, with a GTX750ti inside, I had successfully crunched many hundreds of GPUGRID tasks.
Maybe the new GTX1650 does not fit well into the overall setting, or the GPUGRID tasks strain the systems more than ever before.

As mentioned earlier, everything works well with the GPU tasks from WCG and with Folding&Home.

The very first thing I would recommend you try is to totally wipe out your existing nvidia drivers with DDU: https://www.guru3d.com/files-details/display-driver-uninstaller-download.html

boot into safe mode, and run DDU to do a complete removal of the driver from all areas of your system, including the registry, and make sure to select the option to prevent Windows from installing the driver automatically (or unplug the network cable so it can't). Then re-install the latest stable (WHQL) nvidia driver for your platform. this will eliminate driver corruption (common on windows) as a potential cause of your problem.

But if that still doesn't help, then I think the high PCIe use of GPUGRID ACEMD3 tasks is what's killing you, combined with the old architecture of that platform which wasnt designed to handle this kind of thing, or maybe even instability/errata with the chipset itself (they do age/degrade just like any other silicon). folding and other projects do not exhibit that behavior with the hardware and wont stress the same subsystems that GPUGRID will. maybe the 750Ti wasnt fast enough to bring this problem to light and the 1650 is causing more stress? certainly possible, but without some trial and error testing it's impossible to give a conclusive answer.

Honestly I would recommend just replacing the whole platform with something more modern. just replace the CPU/Motherboard/Memory. You can get stuff only a few years old for dirt cheap, will outperform the old setup, and will pay for itself over time being more energy efficient and lower power use. something like even first gen AMD Ryzen or like a 14nm Intel platform with 4 cores, 8+GB DDR4 memory, and a low end motherboard will be very cheap, run circles around your current platform, be more compatible with modern system and software, and use the same or less power doing it.

just my .02
____________

Big slug of bad work went out with NaN errors.

Both my GTX 1060 on a Windows 10 host and GTX 1650 on a Windows 11 host have completed and validated their tasks.
Tullio
____________

However, meanwhile my suspicion is that the old processor
Intel(R) Core(TM)2 Duo CPU E7400 @ 2.80GHz
may be the culprit.

I was lucky again, the host received another workunit and it's running just fine for 90 minutes. (it needs another 12 hours to complete).
The Core2Duo is definitvely struggling to feed the GTX1080Ti (the GPU usage has frequent deep drops), but I don't think it will run into that "Error invoking kernel: CUDA_ERROR_LAUNCH_TIMEOUT (702)" error. We'll see. I've tried to maximize GPU usage by changing process affinities and the priority of the acemd3.exe, making not much difference.

However, meanwhile my suspicion is that the old processor
Intel(R) Core(TM)2 Duo CPU E7400 @ 2.80GHz
may be the culprit.

I've reanimated (that was quite an adventure on its own) one ancient DQ45CB motherboard with a Core2Duo E8500 CPU in it, and I've put a GTX 1080Ti in it to test with GPUGrid, but there's no work available at the moment. You can follow the unfolding of this adventure here.
EDIT: I've managed to receive one task...
EDIT2: It failed because I've forget to install the Visual C++ runtime :(

The task is finihed successfully in 12h 35m 23s.
On a Core i3-4xxx it takes about 12h 1m 44s, so it take 34m more for the Core2Duo, it was only 4.6% slower than an i3-4xxx.
I've noticed that the present acemd3 app does not use a full CPU core (thread) on Windows while it does on Linux. There's a discrepancy between the run time and the CPU time, also the CPU usage is lower on Windows.

I've noticed that the present acemd3 app does not use a full CPU core (thread) on Windows while it does on Linux. There's a discrepancy between the run time and the CPU time, also the CPU usage is lower on Windows.

I've noticed that the present acemd3 app does not use a full CPU core (thread) on Windows while it does on Linux. There's a discrepancy between the run time and the CPU time, also the CPU usage is lower on Windows.

hm, I acutally cannot confirm, see here:

e7s141_e3s56p0f226-ADRIA_BanditGPCR_APJ_b1-0-1-RND0691_0 27100764 588817 10 Dec 2021 | 12:29:46 UTC 11 Dec 2021 | 5:50:18 UTC Fertig und Bestätigt 31,250.27 31,228.75 420,000.00 New version of ACEMD v2.19 (cuda1121)

Your CPU being a Core2Duo, this architecture does not have a dedicated PCIe link to the CPU. it uses the older architecture where the PCIe and memory connect to the Northbridge chipset and the chipset is what has a single 10.6GB/s link to the CPU, the memory will take most of this bandwidth unfortunately, and since GPUGRID is pretty heavy on bus use, I can see some conflicts happening on this kind of architecture. but the CPU power itself shouldnt be an issue if you're trying to run 1 GPU and no CPU work.

it would be a better comparison to run the same GPU on both systems for a more apples to apples comparison.

something about your 1% PCIe use doesnt seem right. last year I had a 1650 and it used the normal PCIe bandwidth @ ~80% on a PCIe 3.0 x4 link. and about 20% on a PCIe 3.0 x16 link. and just spot checked my 2080Ti system, which showed ~20-25% use on a PCIe 3.0 x16 link, and ~40% on a PCIe 3.0 x8 link

other than PCIe generation (3.0), what are the link widths for each card? how do you have them populated? I'm assuming the two topmost slots? that should be x16 and x4 respectively.

also, what tasks were being processed when you did the test and took the screenshots? only the ACEMD3 tasks exhibit the high PCIe use. However, I've seen much lower on the Python beta tasks.

finally, keep in mind that the PCIe measurement percentage is measured GPU-side, this value comes from the nvidia driver. as a percentage of the GPU's total bandwidth. a bottleneck on the CPU side will not be reflected here. you could very well be 40% on the PCIe 2.0 GPU, but totally maxed on the CPU side as the limiting factor. This is likely the case.
____________

something about your 1% PCIe use doesnt seem right.

other than PCIe generation (3.0), what are the link widths for each card? how do you have them populated? I'm assuming the two topmost slots? that should be x16 and x4 respectively

One thing that I just noticed. All of your hosts are running the New Feature Branch 495 drivers. These are “kinda-sorta” beta and the recommended driver branch is still the 470 branch. So I wonder if this is just a reporting issue. Does the Nvidia-Settings application report the same PCIe value as Psensor?

Can you change one of these systems back to the 470 driver and re-check?
____________

One thing that I just noticed. All of your hosts are running the New Feature Branch 495 drivers

Good to know that psensor is at least still reading the correct value from the new driver.

Definitely interested to know if the reading goes back after the switch back to 470 when you have time.
____________

Definitely interested to know if the reading goes back after the switch back to 470 when you have time.

One thing that I just noticed. All of your hosts are running the New Feature Branch 495 drivers. These are “kinda-sorta” beta and the recommended driver branch is still the 470 branch. So I wonder if this is just a reporting issue. Does the Nvidia-Settings application report the same PCIe value as Psensor?

Definitely interested to know if the reading goes back after the switch back to 470 when you have time.

I am too.
But I've experienced that there is a sure Gpugrid task crash when Nvidia driver update is done and the task is restarted.
It crashes with the same error that when a task is restarted in a different device on a multi-GPU host.
Luckily, my Host #186626, finished its Gpugrid task overnight, so I've reverted it to Nvidia Driver Version 470.86
Curious that PCIe load reduction is only reflected in my newer systems. As shown, PCIe usage at older ones remains about the same than usual.
Now waiting to receive some (currently scarce) new task...

The later kernels are not necessarily considered beta. In fact the latest stable kernel is 5.15.7 as per the kernel.org site. https://www.kernel.org/

Some kernels are considered long-term or LTS and that is what distros ship that aren't rolling release distros.

Generally stated, later kernels are more secure and have more bug fixes and are faster than earlier kernels, not withstanding some speed regressions that pop up in the beta branches every once in a while.

Michael Larabel at Phoronix.com does the Linux community a great service by continually testing new kernels and does kernel bisecting regularly for the community to identify where the kernel regression is occurring and passes that info along to the developers and kernel maintainers.

We should all support his efforts by turning off adblockers on his website or contribute as a paying member.

I do.

I see that you picked up a task today :)

glad to see my hunch about the driver being the culprit was correct.

my second hunch is that it might not impact processing speed, and that the issue is solely with what the driver is reporting, not what's actually being used. I think your PCIe actual use with the 495 driver is normal for GPUGRID (~30%), but that the driver is incorrectly reporting it as 1%.

another test might be to put the older 470 driver on your old LGA775 system and see what the PCIe % reports then. maybe 100%? that might narrow it down to a scaling issue. or maybe the old system and older GPUs are using an older/legacy API that's fetching the correct data, but newer GPUs on a newer API are fetching the wrong data.

there's precedent for this kind of thing, where you have different API calls that fetch the same "thing" but return a different result. For example, in the stock BOINC code, everyone knows that Nvidia GPUs with >4GB memory will only report 4GB, no matter the true size, only returning the true size for values less than 4GB. this is because stock code uses a very old function that clamps the result to 32-bits (4095MB value). but a newer function exists that BOINC is not reading, that returns the correct result. both functions are trying to fetch the same thing, memory size, and in the cases of >4GB VRAM, both functions return a different result.
____________

The Nvidia 495.46 driver isn't considered a beta release now. Just a New Feature branch.

I would posit the main difference between the PCIe usage between the 470 and 495 branch drivers is that the 495 branch now has the GBM (Generic Buffer Manager) API built into it in support of the Wayland compositor.

Don't know exactly what the API is doing but surmise that since the name involves "buffer" that is might be decreasing the amount of traffic going over the PCIe bus.

I dont think the PCIe use at 1% is "real". I think it's just misreporting the real value. GBM doesnt seem to have anything to do with PCIe traffic. that would be up to the app. It seems the app is coded to store a bunch of stuff in system memory rather than GPU memory and it's constantly sending data back and forth. this is up to the app, not the driver.
____________

With the arrival of the last phase of Python GPU beta tasks for Linux systems, every of my hosts started to fail them.
Then I read an explanation from abouh, stating that minimum GPU dedicated RAM should be 6 GB or more.
I've temporary configured Gpugrid preferences for not to request beta apps at every of my 4 GB RAM GPUs.
I also deduced from Ian&Steve C. Message #58172 that minimun system RAM should be 16 GB.

Picking elements from several of my non-high-end hosts, I configured one of them that passed a first task, e1a18-ABOU_rnd_ppod_11-0-1-RND5755_4.
It was at my Test Host #540272
Then, I reproduced that configuration at my regular Host #569442 and a second task e1a16-ABOU_rnd_ppod_12-0-1-RND1123_2 also succeeded.
Therefore, I'm taking this configuration as of "Python GPU tasks minimum requirements" for me, with my currently available hardware.
- GPU: Asus DUAL-GTX1660TI-O6G (reworked), endowed with 6 GB dedicated RAM
- CPU: Intel CORE I3-9100F @3.60 GHZ, 4 cores / 4 threads
- System RAM: 16 GB DDR4 @2666 MHz
- Operating system: Ubuntu Linux 20.04.3 LTS
- Nvidia drivers: V495.44

I was fortunate to be present when catching the second mentioned task, and I was able to document several data since it started and got processing stability.
When this happened, my Host #569442 was previously processing a PrimeGrid GPU task.
When Python task started, an evident drop in GPU activity was observed.
The %progress of task was increasing from 0% to 10%, while free system RAM was decreasing from 44% to 1% and GPU RAM usage increased until almost reaching 100% of its 6 GB.
Then, %progress of task got fixed at 10% up to the end, after about 9 hours of true execution time.
As can be followed at this image, about 4 minutes of erratic CPU activity, followed by another 8 more minutes of intense CPU activity, gave way to an stable eye-catching periodic pattern:

This image can be better understood by reading abouh explanation at his Message #58194

During the task, the agent first interacts with the environments for a while, then uses the GPU to process the collected data and learn from it, then interacts again with the environments, and so on.

Lately, In Canary Islands (Spain), electricity supply costs have been getting higher and higher, month after month.
I have an ongoing "experiment":
I've swapped my currently highest performance graphics card, this reworked GTX 1660 Ti, to my daily use host, and I've switched off every of my other crunching machines.
On next month(s) I should be able to estimate the effects for this.
It should bring some savings in my electricity bill (and collaterally, in my BOINC RAC ;-) ...