SOLVED: Purchase mistake ? [Nvidia Quadro M6000]

[StainlessS]

I've just (last night) ordered an nvidia Quadro M6000 24GB GDDR5 from ebay {~£200.00, used}:-
https://www.techpowerup.com/gpu-specs/quadro-m6000-24-gb.c2824
NEW on Amazon.co.uk (£830.00):-
https://www.amazon.co.uk/PNY-Interactive-Expression-Express-Graphics/dp/B01DPPPCM2?ufe=app_do%3Aamzn1.fos.d7e5a2de-8759-4da3-993c-d11b6e3d217f }

with a sort-of intention to use for LOCAL ONLY AI experimentation, however,
to use said card I would need to get a larger PSU (500W to 600W or more likely ~750W->850W)
for my HP Omen 880-112na {i7-8700, 32GB DDR4, Nvidia GTX 1070, 500W PSU).

Its only just occurred to me that the GPU is older (slightly, ie about 3 months) than my current GTX 1070 and
has older CUDA version and does not support some numeric types (int8,int16,fp16) that are supported
by my GTX 1070.

A thought occurred to me that new card might not be supported by eg DgIndex / DgIndexNV,
I do remember that there was some post by Donald Graft outlining minimum GPU family for use with those
utilities, but I seem not to have kept a link to it. [EDIT: Quadro M6000 is Maxwell 2.0]

Anybody know if the Quadro M6000 is ok, if not then I'll have to skip purchase of PSU and shove the GPU
in a cupboard, or something.


EDIT: At least it does support 4K which is another worry I had.
EDIT: According to the 1st link, Relative Performance box, my current GTX 1070 is 3% faster than Quadro M6000, so not losing much for general speed.
EDIT: Both my GTX 1070 and Quadro M6000 are PCIe 3.0, as is the HP machine.

[Z2697]

I think it's safe to use 500W PSU if you don't stress other components too much...?

[StainlessS]

Nope, dont think so, currently 3 hard drives (1x2TB, 2x3TB{seems very heavy and current consumptive, Hitachi-HUA723030ALA640}), + 1x1TB + 1x 2TB nvme.
I took one of my hard drives out of the machine some time ago because I thought it was struggling power wise (weird shut downs and such, might have been a 3rd Hitachi, as above).
[EDIT: mystery shutdowns no longer a problem since drive removal.]

The HP machine is my 'daily driver' despite my having a bunch of other machines (mostly 1 Litre or NUC type machines, and three 2.5 litre HP Z2 Mini Workstations; 1 Gen 3 i7-6700, and two Gen 5 i7-10700, with one of them a K
{and all three with quadro GPU's, about 4GB memory each},
my only other full size desktop is 11th Gen i7(maybe K, not sure) which cannot power the Quadro M6000, cant really even add another spinning hard drive without power concerns
[also M6000 would not fit in the case, currently just works with GTX 1650, I think, physically small and power light].

[Columbo]

It should be fine with DGDecNV.

Two caveats specific to that card:

HEVC: GM200 has no full hardware HEVC decode (only GM206 in the Maxwell family does). On a card that supports only hybrid decoding, you must enable D3D mode and DXVA mode, and the readme notes GTX 1050 and better is highly recommended for HEVC work. No VP9 or AV1 decode at all.

Drivers: NVIDIA has moved Maxwell to legacy status. Game Ready updates for Maxwell and Pascal ended with the 581.80 Windows driver in late November 2025, with only security updates through 2028, and the 590 branch does not support Maxwell. So the card must stay on the 58x branch.

[StainlessS]

It should be fine with DGDecNV.
Ah, good, (assume that includes DGDecode too for DVD).

HEVC: GM200 has no full hardware HEVC decode (only GM206 in the Maxwell family does).

from TechPowerUp page on GTX 1070,
GP104 GPU Notes
NVENC: 6th Gen
NVDEC: 3rd Gen
PureVideo HD: VP8
VDPAU: Feature Set H

from TechPowerUp page on Quadro M6000,
NVENC: 5th Gen
NVDEC: 1st Gen presume this one the problem.
PureVideo HD: VP6
VDPAU: Feature Set E

you must enable D3D mode and DXVA mode
No idea what that means, I guess I have to investigate settings in DGIndexNV.

I usually (for eg hevc input) Index in DGIndexNV, and auto generate an AVS script (from a template) which requires a few manual settings
for eg border croppings (after using RoboCrop to output coords to DebugView).
Set standard easy mod settings in the script for various tweaks.
For actual re-encode to hevc/avc, I usually copy across onto any of a number of 2.5 litre or 1 litre machines, some of which
have no discrete GPU, in which case the script that would call DgDecNV would fail and retry using Lsmash decode.

If I have problems, I guess that I could always index and setup the avs files on the GTX 1650 machine.

All hail Columbo (try to remain humble),
https://www.youtube.com/watch?v=Q8TUSsqWCTM

EDIT: I guess I'll order a PSU now, thanx for your replies.
MARKED as SOLVED.

[Columbo]

DGDecode does not use the GPU. ;) Also, DGDecNV works on DVDs too.

When the time comes hit me up again and I'll explain the D3D mode and DXVA mode stuff. Here or in DG forum is fine.

If you are just going to use this GPU for LLM stuff and the 1070/1650 for decoding, then the HEVC limitation is moot, as you observed.

Thanks for the youtube link!

[StainlessS]

Somebody on YouTube just pointed to an nvidia Tesla P40 as being an option for AI stuff, however it has no graphics output, ie does not drive a monitor.
https://www.techpowerup.com/gpu-specs/tesla-p40.c2878

To use the tesla P40, you would need to be able to drive a monitor [unless headless AI server] and also have the tesla card in the slot.
My HP Omen (and other HP's too), dont allow you to eg drive monitor via integrated intel port, whilst also using a separate discrete GPU monitor driver,
if you install external card, the HP disables use of the internal graphics card.
My omen only sports a single PCIe v3.0 16 lane device, so I could not use the P40 unless can use 'built-in' graphics at same time as discrete card is mounted.
Not sure what would happen if you try drive monitor via integrated card, and have discrete GPU WITHOUT any display ports on-board.
(Might need eg MSI or Gigabyte or similar non HP and non Dell machine to have simultaneous multi GPU at once).

For anybody that can use multiple GPU's, then perhaps the Tesla P40 is a 24GB VRAM solution for AI tinkering.
On ebay, I found several Tesla P40 24GB for about £215.

However, see also, DONT Buy these GPU's for Local AI! (learn from my mistake) :-
https://www.youtube.com/watch?v=ozqorev2_34
which I'm guessin would have been a good idea for me to read too before purchase of the M6000.
I'll watch that video later, gotta have a nap, I'm knackered.

EDIT: And a whole lot more clips of Tesla P40.
https://www.youtube.com/results?search_query=tesla%20p40

EDIT: WARNING, the Tesla P40 has NO COOLING fans at all. [sounds problematic]
(The Quadro M6000 does have one fan)

[Z2697]

Tesla P40 still sounds... too old, maybe it's just me being one that bought a decent card when the price was still barely reasonable.

I think even with less VRAM, a more modern (at least RTX30 era) 16GB card would be better (not talking about price of course...).
Intel is probably a no-no, but AMD with ROCm maybe worth a shot.

[StainlessS]

Just before RAM/SSD price rocketed, I bought a used GMKtec K9 Intel® Ultra 5 125H Mini PC
https://www.gmktec.com/products/intel-ultra-5-125h-mini-pc-nucbox-k9?_pos=1&_sid=2dc1b4ed1&_ss=r
with a little bit of a view to toying with AI, so I also bought 2x 48GB DDR5 sticks (used) for a total of 96GB RAM,
and also got 2x 2TB nvme sticks for a total of 4TB storage.

What was swirling through my mind was to use the Ultra 5 125H ARC integrated GPU with a bigger chunk of unified memory
(which turned out to be 54GB by default [EDIT: might be 57GB, not sure], but changeable in Intel driver settings prog).
So, I though that I was all set for some el-cheapo AI stumbling about.
I can use larger models now in LM-Studio and have also installed BIONIC agent [LM-Studio agent], but not tried that one yet.
I think the biggest model I tried was about 70GB in size (after increase unified memory to ~80GB), but was slow as hell.
EDIT: Actually, I think biggest model I've tried was more like ~50GB [EDIT: Actually 45.1GB] not 70GB, but I had increased unified mem to ~72GB beforehand (leaving about 24GB non-unified).

This is a question I posed the AI prior to posting this thread. [and prior to pushing the buy button on ebay for the
M6000 {in fact I think the AI answer was still pending/thinking when I pressed the buy button}]

27 billion parameter model [~16GB]
The nvidia quadro m6000 gpu does not support int4, int8, int16, fp16, or bf16 numeric formats.
How badly does this affect AI output from lm-Studio.


qwen/qwen3.8-27b
Thought for 103 minutes and 30 seconds.

Then generated a quite long answer at 0.93 tokens / second.


I will not force you to read the long reply (about 26 KB, excluding the also quite [EDIT: equally] long thinking output).

So, thats how I found myself looking for cheap GPU.

However, I came across an article somewhere that said that you need some kind of additional code stub
so as to use the intel ARC iGPU instead of the CPU. [EDIT: So, I was actually using CPU in above query, not iGPU]

Locally Running AI model with Intel GPU
https://www.reddit.com/r/LocalLLaMA/comments/1m2furm/locally_running_ai_model_with_intel_gpu/

There seem to be several alternative options, I have not looked at any of them yet,
with a bit of luck, maybe I dont even need the M6000 GPU.

EDIT: Maybe one day I might also read the LM-Studio manual, to try figure out what it is I am tryin' to do.

[StainlessS]

Well the Quadro M6000 turned up today, I removed 2 harddrives and 1 nvme and checked out that it works ok with 500W PSU.
As it happens, the nvme I removed was in a pcie adapter and that adapter slot was also full size 16 lane slot,
I thought that I only had a single 16 lane GPU slot in my HP omen.

So, I plugged in the GTX 1070 too, and both seem to be seen by firmware so I ordered an 850W PSU and its being delivered tomorrow.
I cant use that machine right now, I dropped a screw inside somewhere and have to locate it before putting it all together.

In the pub right now, feeling good, maybe we get this AI thingy up and working ok.



(WOW D9 is too busy more often than working)

EDIT: Quadro M6000 is PNY, so far as I've seen, should have 4x DisplayPort + 1xDVI, but mine has 3xDP, + 1xHDMI, + 1xDVI.
Have not seen any others with HDMI, so far. (Not even other PNY)

[Columbo]

Reading your post triggered my ocular migraine!

[StainlessS]

Thank you very much, its very kind of you to say so.
Yip, I've often been praised because my dazzling posts.
Right Bobby Dazzlers. :)

[StainlessS]

Well, here an update on how I've got on with the Nvidia Quadro M6000 24GB GPU thingy.
I have eventually found a driver that works for the device,
NVIDIA RTX / Quadro Desktop and Notebook Driver Release 545:- https://www.nvidia.com/download/driverresults.aspx/216365/en.us/
Version: R545 U2 (546.01)
Release Date: 2023.10.31
Operating System: Windows 10 64-bit, Windows 11
Language: English (US)
File Size: 523.89 MB
Works with W10 20H2 -> 22H2.

You would probably wanna see the release notes PDF from Additional Information tab.

I've had to use alone without my GTX 1070 in-situ, but I suppose that I can get used to that.

Using LM-Studio on my Gmktec K9, Ultra 5 215H [EDIT: 125H] CPU and with AI question being,

"Give me a concise recipe for highland shortbread."

Thought for 19.86 (1st time was 26.xx) seconds & produces,
We need to respond to user: "give me a concise recipe for highland shortbread."
Need produce final concise recipe. Need likely include ingredients and steps. Keep concise.

then,

Highland Shortbread

Ingredients
- 225g (2 cups) all-purpose flour
- 115g (½ cup) cold butter, cubed
- 115g (½ cup) sugar
- Pinch of salt
- Optional: 1 tsp vanilla or a pinch of cinnamon

Method
1. Mix flour, sugar, and salt. Rub in cold butter until crumbly.
2. Press into a lined 9-inch square pan; prick all over with a fork.
3. Bake at 300°F/150°C for 45–55 minutes until pale golden.
4. Cool 10 minutes, then cut into bars while warm. Cool fully before serving.

@ 1.12 tok/sec, time 20.6 seconds.

Using, Quadro M6000 with previous given driver,

Thought for 4.09 seconds
Need produce final concise recipe. Include ingredients steps maybe tips. Ensure safe? no issue. Keep concise.

Highland Shortbread

Ingredients

1 cup (230 g) unsalted butter, softened
½ cup (65 g) powdered sugar
2 cups (250 g) all-purpose flour

Method

Cream butter and powdered sugar until smooth.
Mix in flour until a soft dough forms.
Press into an ungreased 9-inch square pan; prick with a fork.
Bake at 300°F/150°C for 35–40 minutes, until pale golden.
Cool completely, then cut into bars or fingers.

@ 10.09 tok/sec, time 4.68 seconds.

The [EDIT: Ingredients] measurements on 2nd example are a bit weird.

I'm currently having problems, not detecting 2nd monitor for some reason, but was doing that when I was using the GTX 1070
and only had Quadro M6000 without driver and disabled.
Might be related to my using a KVM for 2nd monitor, also same problem in Linux.

The 2nd timing was on Quadro M6000 GPU, (i7-8700 CPU), whereas 1st timing on ultra 5 125H using CPU.

I shoulda done a timing on the i7-8700 CPU for comparison but dont as yet know how to when GPU installed.

EDIT: Model for both, LM-STUDIO COMMUNITY MODEL:- Qwen3.8-27B-Q4_K_M.gguf (16.5GB)

[Columbo]

Perfect! I'm half Scottish. ;)

[StainlessS]

I think actual ratio should be 3:2:1 (Probably 3=flour and 1=sugar<granulated not castor>)

Also, just chuck ingredients (butter=room temp) into a bowl and squeeze through fingers, separating into a loose wet sand like consistency.
When fully 'loose-wet-sand-like', then mould/compact into a single solid ball (like you are making a sand castle) , then roll out into a baking tin, about 1/2 inch thick.
Prick with fork all over the surface, about 1" apart. perhaps score into 3x1.5 rectangles.
Then bake.

Presented with a bowl of ingredients, a 7 year old should be able to master the technique and be amazed at what they have produced.

EDIT: I don't really eat biscuits, but, I make an exception for Highland Shortbread, truly a magnificent treat.

[Columbo]

Well shortbread is not really a biscuit in USA vernacular. We'd say cookie. I could give you my mailing adress if you need a beta tester. ;)

"I sometimes post sober."

Why?

[StainlessS]

In the patwah of the colonies it may well be cookie, but in English, it is biscuit. :)

If you got a grandkid, abouts 7, oder 8, then givem a task of Highland Shortbread, they'll be making it every time they visit.
Myself, I have not made any for about 5 years, not used the oven in about 2 years since getting a bread making machine, and
a little hot air frying oven thingy. [I live alone and mostly cook just for me]

[Columbo]

Lucky you. I don't have any grandkids so here's my solution. Slàinte Mhath!

https://rationalqm.us/misc/biscuit2.jpg

[StainlessS]

Dem der biscuits look good, but cant beat still slightly warm home cooked. [can bung packeted bickies in micro wave for 5-7 seconds for a (nearly) just baked taste].

"I sometimes post sober."

Why?

Yeah, you're right, does seem like a waste.

EDIT: Problem with 2nd monitor resolved, was a faulty DP to VGA adapter, KVM is for 4xPC -> 1x VGA monitor.
(setup as for 1xdual monitor, or 2nd monitor any of 3 other PC's [EDIT: Mouse/Keyboard switch with the 2nd monitor, ie 2nd monitor is Primary])
I got a 8xHDMI/8xDP KVM thingy, had for several months but scared of all dem der wires and such, gotta be a tangle.
https://www.amazon.co.uk/dp/B0FXGFFYYX
https://i.postimg.cc/LYNDb1R3/71a-Xs3-lvk-L-SL1500.jpg (https://postimg.cc/LYNDb1R3)

VGA KVM's are a lot cheaper than HDMI/DP. (HDMI/DP KVM's were a lot more expensive until relatively recently, like £500, somewhat cheaper now but still expensive)

Here is a 4xPC->1xVGA KVM on Amazon, I got 2 or 3 of these (still lots of wires to tangle, and probably need lots of DP or HDMI adapters).
https://www.amazon.co.uk/dp/B092Q2DVC2
https://i.postimg.cc/5QWzp9Qh/612li-TSn-c-L-SL1500.jpg (https://postimg.cc/5QWzp9Qh)

[Columbo]

Glad y'all got yer 2nd monitor working. So nuttin for me to do to support your efforts?

Here's a secret for you, don't tell anyone. I'm working on some changes to DGDecodeNV that will accelerate decoding by ~20%, tested on 8K 10-bit material. Gotta stay ahead of BestSource() dontcha know.

[StainlessS]

I hav'ta go read the Quadro M6000 release notes again, think I cannot use both GTX 1070(graphics) and M6000(CUDA) together, but not sure.
Not ever used BestSource, but am aware.

[Columbo]

look into "headless GPU".

[StainlessS]

Thanks, will do.

[Columbo]

Crud, I reached only 17%. Now to improve further I need DDR5 memory. But that would help the Big Satan too (BS = BestSource() get it?).