Compensating for incorrect capture by planes/endianness?

qyot27 · 28th August 2022, 02:35

I recently bought an EVGA XR1 Pro, and in the process of trying it out for some desktop captures, I found something very wrong:

This only happens through the capture box, and happens at a low enough level that it can't be blamed on capture software not understanding it - even mpv's V4L2 sink looked like that when I had it open /dev/video0. Hooking the Mac in question to a typical HDMI switch, the colors are fine (you can see that exact wallpaper in this blog post). It looks like it's misinterpreting the position of the RGB planes before the capture occurs, since the card only does YUY2 and NV12 natively.

The XR1 is not at fault, though. Captures from a Raspberry Pi 4B and a Nintendo Switch were fine, but those were HDMI end-to-end. The Quadro FX4500 in the Mac only has DVI-I and S-Video out, so I have both a small DVI-HDMI plug adapter as well as a long DVI-HDMI cable to use to hook to current equipment. Both of them resulted in that pink/green image through the box, but when plugged into a regular monitor, the colors are correct.

The best I can guess as to what's happening here is that:

Because this is a Power Mac G5, the graphics card is outputting in a big-endian RGB format.
Monitors are equipped to negotiate this, but the XR1 clearly isn't. The Pi and Switch are both ARM-based, and would be using a little-endian output format that the capture card or its internal framebuffer expects. I haven't tried with either a PS3 or Wii U, which might have similar issues due to being PowerPC-based like the G5.

The question here, though, is whether there's anything that I can do post-capture to fix the problem.

qyot27 · 3rd September 2022, 21:54

I still don't know exactly why there's an issue with DVI-D -> HDMI -> XR1 Pro, but I did discover one thing: the capture portion doesn't like the Mac, but if I hook another HDMI cable to the XR1's HDMI out port so that I can use passthrough to my regular monitor, the passthrough feed looks correct (and like I said, the DVI->HDMI cable is fine; if that's hooked directly to a monitor the colors are correct).

However, I remembered that I had a DVI-A to VGA adapter, and the ports on the graphics card are, in fact, DVI-I, so the analog signal is also there. I also happened to have a VGA->HDMI active converter from months ago when I wanted to more easily work with an iMac G5 with a possibly-dying graphics card.

The result of using the DVI->VGA adapter, hooked to the VGA->HDMI converter box, hooked to the XR1 Pro is correct. Thank you analog hole. So ultimately I won't have to do any special post-processing with AviSynth just to get it to look like it should have in the first place.

28th August 2022, 02:35	#1 \| Link
qyot27 ...? Join Date: Nov 2005 Location: Florida Posts: 1,420	Compensating for incorrect capture by planes/endianness? I recently bought an EVGA XR1 Pro, and in the process of trying it out for some desktop captures, I found something very wrong: This only happens through the capture box, and happens at a low enough level that it can't be blamed on capture software not understanding it - even mpv's V4L2 sink looked like that when I had it open /dev/video0. Hooking the Mac in question to a typical HDMI switch, the colors are fine (you can see that exact wallpaper in this blog post). It looks like it's misinterpreting the position of the RGB planes before the capture occurs, since the card only does YUY2 and NV12 natively. The XR1 is not at fault, though. Captures from a Raspberry Pi 4B and a Nintendo Switch were fine, but those were HDMI end-to-end. The Quadro FX4500 in the Mac only has DVI-I and S-Video out, so I have both a small DVI-HDMI plug adapter as well as a long DVI-HDMI cable to use to hook to current equipment. Both of them resulted in that pink/green image through the box, but when plugged into a regular monitor, the colors are correct. The best I can guess as to what's happening here is that: Because this is a Power Mac G5, the graphics card is outputting in a big-endian RGB format. Monitors are equipped to negotiate this, but the XR1 clearly isn't. The Pi and Switch are both ARM-based, and would be using a little-endian output format that the capture card or its internal framebuffer expects. I haven't tried with either a PS3 or Wii U, which might have similar issues due to being PowerPC-based like the G5. The question here, though, is whether there's anything that I can do post-capture to fix the problem. __________________ AviSynth+ native Linux/macOS/BSD support \| AviSynth+ utilities and plugins for non-Windows and non-x86(-64)

3rd September 2022, 21:54	#2 \| Link
qyot27 ...? Join Date: Nov 2005 Location: Florida Posts: 1,420	I still don't know exactly why there's an issue with DVI-D -> HDMI -> XR1 Pro, but I did discover one thing: the capture portion doesn't like the Mac, but if I hook another HDMI cable to the XR1's HDMI out port so that I can use passthrough to my regular monitor, the passthrough feed looks correct (and like I said, the DVI->HDMI cable is fine; if that's hooked directly to a monitor the colors are correct). However, I remembered that I had a DVI-A to VGA adapter, and the ports on the graphics card are, in fact, DVI-I, so the analog signal is also there. I also happened to have a VGA->HDMI active converter from months ago when I wanted to more easily work with an iMac G5 with a possibly-dying graphics card. The result of using the DVI->VGA adapter, hooked to the VGA->HDMI converter box, hooked to the XR1 Pro is correct. Thank you analog hole. So ultimately I won't have to do any special post-processing with AviSynth just to get it to look like it should have in the first place. __________________ AviSynth+ native Linux/macOS/BSD support \| AviSynth+ utilities and plugins for non-Windows and non-x86(-64)