Hello, I am using two GTX Titans and two Tesla K40 and my average is 474,000. It seems low for all four cards on the same machine, but I think that the Tesla driver is conflicting with the GTX Titans. I have already reported that to the Nvidia driver team.

I have seen the triple buffering in the Nvidia Control Panel and I was wondering if to enable it will help get higher numbers.

I am using EVGA Precision and I am running both Titans at 1162mhz GPU clock steady without any problems.

I am also doing only long runs to get about 120,000 credit per unit every 6 hours.

Thanks

Hello, I am using two GTX Titans and two Tesla K40

You can turn up the clocks on the K40 with the application clocks feature of nvidia-smi

I have seen the triple buffering in the Nvidia Control Panel and I was wondering if to enable it will help get higher numbers.

No, it won't.

Matt

Your system should eventually get a RAC of over 2M.

Your Tesla K40c doesn't perform very well. I'm getting similar performances from a GTX770, which should be at least 30% slower. There are a few clear reasons for this, and a few potentials.

Linux is 12.5% faster for a GTX770 than Windows Vista, W7 and newer (with the 2008R2 server being the exception; about 8% slower IIRC).
Not sure if you have ECC on or not, but I expect it's faster off.
Don't have a K40C so don't know if you can turn double precision off, as with the Titans. If you can I suggest you do that.

You are using a Beta driver that may cause your GPU's to operate at a non-Boost clock, rather than boosting. If this is happening it could cost you about 15% in performance.
Your 'AMD Phenom(tm) II X6 1090T' is a decent processor for a mid range system, but it is over 3 years old, those are high end cards, and you have 4 of them! The processor is about 80% slower than a high end Intel so your. They system is also limited to PCIE2.0. I don't know what motherboard you are using but it's likely the cards are running at PCIE2 x8 or less. My guess is that the last slot is PCIE 2.0 X4. So there is a slight performance hit from the CPU and from the PCIE bus as well as the OS. Of course all these hits are cumulative, and even a 1 or 2 % hit multiplies up.

Matt, unless it's overclocked I think the K40's device clock is being misreported here; it's supposed to be 745MHz, tasks say 875MHz. I expect it's not the Boost clock as other cards show the non boost clock.

---

FAQ's

HOW TO:
- Opt out of Beta Tests
- Ask for Help

Thanks for the reply.

The ECC is off, DP is off At least for the Titans in Control Panel

The motherboard is a Gigabyte GA-990FXA-UD7

Titans are in slots PCIEX16 and PCIEX16

K40 are in slots PCIEX8 and PCIEX8

Ubuntu 64 Linux might be better or windows 2008 R2 or 2012?

Any other recommendations??

Thanks

Ubuntu 64 would be the fastest. Have not tried 2013 server, yet.

---

FAQ's

HOW TO:
- Opt out of Beta Tests
- Ask for Help

Matt, unless it's overclocked I think the K40's device clock is being misreported here; it's supposed to be 745MHz, tasks say 875MHz. I expect it's not the Boost clock as other cards show the non boost clock.

The Teslas don't boost dynamically like the Geforces do. The K20 and K40 do have alternate clock settings, which are accessible through nvidia-smi (Linux), which I think he's set to get up to 875.

There's a bit more about that in:
http://www.nvidia.com/content/PDF/kepler/Tesla-K40-Active-Board-Spec-BD-06949-001_v03.pdf

You can't turn off DP on the Teslas.

Matt

Yes, use Linux. Try out the live image I posted about elsewhere if you want to try it out before installing.

Matt

Installed Linux last night and what a hassle to get the drivers installed. Thought that Linux had gotten better at that.

Went back with Windows 7, but I think I am going to try windows 2012 and see.

I am sure that I will need another motherboard with integrated graphics and PCI 3.0, 2- PCIE x16 slots to run the two Teslas K40 to get better performance. All four cards don't work together.