Welcome to Doom9's Forum, THE in-place to be for everyone interested in DVD conversion.

Before you start posting please read the forum rules. By posting to this forum you agree to abide by the rules.

 

Go Back   Doom9's Forum > Capturing and Editing Video > NLE - Non Linear Editing

Reply
 
Thread Tools Search this Thread Display Modes
Old 25th March 2020, 00:49   #1  |  Link
Stereodude
Registered User
 
Join Date: Dec 2002
Location: Region 0
Posts: 1,436
Resolve 16.2 mangles chroma

Is there a way to get Resolve to not mangle the chroma of a video with non square pixels? (I'm guessing the non square pixels are the problem)

I have a NTSC 16:9 enhanced DVD (720x480), I converted it to a .mov containing 8-bit 420 H.264 (crf 1) with VD2 so I can mess with the grade in Resolve. I put it into Resolve into a 720x480 NTSC DV 16:9 timeline, do some color grading and then output it as uncompressed 10bit 422 YUV in a .mov and I get this crap.

Here's the input (scaled to 2x by madVR)


Here's the output (scaled to 2x by madVR)


I've tried other output formats, like Cineform and got the same thing. I tried a custom 720x480 timeline, but that doesn't seem to stick. It keeps reverting back to 720x480 NTSC DV 16:9.

Are there some settings I'm missing (I'm very far from a Resolve guru) or is Resolve just broken?

Last edited by Stereodude; 25th March 2020 at 00:57.
Stereodude is offline   Reply With Quote
Old 25th March 2020, 03:48   #2  |  Link
Stereodude
Registered User
 
Join Date: Dec 2002
Location: Region 0
Posts: 1,436
So I did a little more testing. I'm not sure if it is Resolve's 420 -> 422 conversion that's the problem or perhaps an issue with H.264, but if I convert the DVD to ProRes 422 instead of near lossless H.264 and feed that into Resolve the chroma in the output is fine.
Stereodude is offline   Reply With Quote
Old 25th March 2020, 04:36   #3  |  Link
poisondeathray
Registered User
 
Join Date: Sep 2007
Posts: 5,345
Quote:
Originally Posted by Stereodude View Post
So I did a little more testing. I'm not sure if it is Resolve's 420 -> 422 conversion that's the problem or perhaps an issue with H.264, but if I convert the DVD to ProRes 422 instead of near lossless H.264 and feed that into Resolve the chroma in the output is fine.
Quote:
I've tried other output formats, like Cineform and got the same thing
If that's the case why would cineform 422 be affected as well ?
poisondeathray is offline   Reply With Quote
Old 25th March 2020, 04:47   #4  |  Link
Stereodude
Registered User
 
Join Date: Dec 2002
Location: Region 0
Posts: 1,436
Quote:
Originally Posted by poisondeathray View Post
If that's the case why would cineform 422 be affected as well ?
Because the CineForm 422 was an output format I tried with the same near lossless 420 H.264 input. When I changed the input format to ProRes 422 or uncompressed v210 the problem goes away in the output.

I can't narrow down the issue on the input side any more because Resolve won't take uncompressed 8-bit or 10-bit 420 video in a .mov container as a source file. So I'm not sure if it's a H.264 420 specific issue or a 420 specific issue. Resolve didn't want to take a 4:2:2 H.264 input file either.

They desperately need to support VFW AVI codecs.

Last edited by Stereodude; 25th March 2020 at 04:50.
Stereodude is offline   Reply With Quote
Old 25th March 2020, 04:51   #5  |  Link
poisondeathray
Registered User
 
Join Date: Sep 2007
Posts: 5,345
Quote:
Originally Posted by Stereodude View Post
Because the CineForm 422 was an output format I tried with the same near lossless 420 H.264 input.
I misunderstood, I though you meant you tested cineform as the input and output (yes, even though it said output... )

Quote:
They desperately need to support VFW AVI codecs.
This will never happen
poisondeathray is offline   Reply With Quote
Old 25th March 2020, 05:37   #6  |  Link
Cary Knoop
Cary Knoop
 
Cary Knoop's Avatar
 
Join Date: Feb 2017
Location: Newark CA, USA
Posts: 397
Did you verify it is not an interlacing thing instead of a chroma thing?
Cary Knoop is offline   Reply With Quote
Old 25th March 2020, 05:46   #7  |  Link
Stereodude
Registered User
 
Join Date: Dec 2002
Location: Region 0
Posts: 1,436
Quote:
Originally Posted by Cary Knoop View Post
Did you verify it is not an interlacing thing instead of a chroma thing?
The source is progressive 30/1.001.
Stereodude is offline   Reply With Quote
Old 25th March 2020, 06:41   #8  |  Link
poisondeathray
Registered User
 
Join Date: Sep 2007
Posts: 5,345
It looks like point sampled (nearest neighbor) chroma . It's just duplicating samples up to 4:4:4 then RGB, then back down to 4:2:2 when exporting a 4:2:2 format

So if you export some 4:2:2 format, you can downsample back to 4:2:0 using chromaresample="point" and it will look close to what you started with

#exported 4:2:2 format like prores
ConvertBits(8)
ConvertToYV12(chromaresample="point")

Or you can use avsresize eg. for 10bit , but 4:2:0
z_ConvertFormat(pixel_type="YUV420P10", resample_filter="point")

(I tested this on some patterns in Resolve Studio 15, similar conditions 720x480 x264 4:2:0 import, 720x480 timeline settings, 422 export)



Last edited by poisondeathray; 25th March 2020 at 06:59.
poisondeathray is offline   Reply With Quote
Old 25th March 2020, 07:18   #9  |  Link
Stereodude
Registered User
 
Join Date: Dec 2002
Location: Region 0
Posts: 1,436
Quote:
Originally Posted by poisondeathray View Post
It looks like point sampled (nearest neighbor) chroma . It's just duplicating samples up to 4:4:4 then RGB, then back down to 4:2:2 when exporting a 4:2:2 format
I'll take really poor algorithm choices for $1000 Alex.

BTW, thanks for figuring this out!

Last edited by Stereodude; 25th March 2020 at 14:52.
Stereodude is offline   Reply With Quote
Old 25th March 2020, 17:52   #10  |  Link
poisondeathray
Registered User
 
Join Date: Sep 2007
Posts: 5,345
BTW that bottom graphic is labelled in correctly it should be avsresize point 10bit420, not 10bit422 - (ie. 422=>420 just discarding the samples that were duplicated)
poisondeathray is offline   Reply With Quote
Old 28th March 2020, 18:25   #11  |  Link
Stereodude
Registered User
 
Join Date: Dec 2002
Location: Region 0
Posts: 1,436
Quote:
Originally Posted by poisondeathray View Post
So if you export some 4:2:2 format, you can downsample back to 4:2:0 using chromaresample="point" and it will look close to what you started with

#exported 4:2:2 format like prores
ConvertBits(8)
ConvertToYV12(chromaresample="point")

Or you can use avsresize eg. for 10bit , but 4:2:0
z_ConvertFormat(pixel_type="YUV420P10", resample_filter="point")
So I decided to measure the PNSR of this. The numbers aren't stellar. I can't speak to whether the results are visually apparent or not. I didn't attempt to examine that aspect.

This is lossless:
Convert YV12 source to v210 in Avisynth -> uncompressed v210 .mov (VD2) -> Resolve -> uncompressed v210 .mov -> Avisynth convert back to YV12

This is not even close:
Compress the YV12 source to near lossless (qp 1 / keyint 1) H264 -> Resolve -> v210 .mov -> Avisynth convert back to YV12

The near lossless H264 compression is good for -~70-75dB

The Resolve -> YV12 in Avisynth step degrades it quite a bit further dipping all the way down to ~46dB for a section on my 5000 frame test clip.



My guess is that it has something to do with the chroma placement and the transformation matrix not aligning between the 4:2:0 -> 4:2:2 conversion in Resolve and 4:2:2 -> 4:2:0 back in Avisynth.

Edit: It seems like this is potentially a significant issue for anyone who uses 4:2:0 video from their cameras directly in Resolve and then exports from Resolve in a lossless or with a "light touch" codec to encode in something else. You can visually improve the chroma with a point resample of it back to 4:2:0, but you can't actually reverse the operation.


Also for a point of reference while I was at it I measured Cineform's performance (forcing a re-encode in Resolve) using the same flow vs. v210 uncompressed and got this:



Convert YV12 source to v210 in Avisynth -> CineForm FS3 v210 .mov (VD2) -> Resolve -> CineForm Best v210 .mov -> Avisynth convert back to YV12

Last edited by Stereodude; 28th March 2020 at 18:34.
Stereodude is offline   Reply With Quote
Old 28th March 2020, 21:27   #12  |  Link
poisondeathray
Registered User
 
Join Date: Sep 2007
Posts: 5,345
Quote:
Originally Posted by Stereodude View Post
My guess is that it has something to do with the chroma placement and the transformation matrix not aligning between the 4:2:0 -> 4:2:2 conversion in Resolve and 4:2:2 -> 4:2:0 back in Avisynth.
There is an intermediate RGB step in Resolve too. If there wasn't, and Resolve could work in YUV, you would expect Y to be the same since you're measuring Y psnr in those graphs, not Y, U, V or aggregate YUV PSNR. For programs that are YUV capable, there is no alteration of the Y plane . v210 gets special treatment - it actually gets special treatment in other NLE's too, even ones that work internally in RGB . It's a native format that gets passthrough . For some Windows based NLE's, you can use IYUV (8bit 4:2:0, with fourcc IYUV, or 8bit 4:2:2 with fourcc UYVY) and those gets passthrough , but Resolve is really devloped from a Mac environment , there is no way to get it to work with uncompressed 8bit 4:2:0

Was this internal avs function conversion, or z_convert_format?

Recall there is a chroma shifting issue in the internal avs resizers . It's not a true point resize. There are workarounds such as resize8, but z_convert_format works properly

But it would partially depend on how resolve is defining the chroma placement too.

Last edited by poisondeathray; 28th March 2020 at 21:52.
poisondeathray is offline   Reply With Quote
Old 28th March 2020, 21:57   #13  |  Link
poisondeathray
Registered User
 
Join Date: Sep 2007
Posts: 5,345
On a synthetic test clip (the one above, but there is animation, rotation, but it's just simple text motion)

Same thing 720x480 YV12 source => Resolve => Export v210 => Convert back to YV12

ConvertBits(8).ConvertToYV12(chromaresample="point")
[Parsed_psnr_4 @ 000000702506d280] PSNR y:71.482031 u:48.201779 v:48.396939 average:53.027946 min:52.964134 max:53.075222

Z_ConvertFormat(pixel_type="YV12", resample_filter="point")
[Parsed_psnr_4 @ 000000a5d07f7800] PSNR y:68.920585 u:61.097875 v:73.214861 average:66.519582 min:64.670853 max:67.972861

The U, V scores are much lower for the internal avs as expected with the known shifting

I'll try some real clips when I get a chance , but with synthetic test patterns it might be possible to figure out how it's interpeting the chroma and reverse it more closely

Last edited by poisondeathray; 28th March 2020 at 22:04.
poisondeathray is offline   Reply With Quote
Old 28th March 2020, 23:54   #14  |  Link
poisondeathray
Registered User
 
Join Date: Sep 2007
Posts: 5,345
UHD YV12 1152 frame , fairly high scene complexity with foliage, people, movement

Spline16Resize => 720x480p23.976 . Did not compensate for colormatrix or anything else. This is the new source "orig"

Encoded x264 --qp 1 --keyint 1 --tune psnr

Import qp1_intra version into Resolve, matched all import/export settings. Check marked retain sub black and super white on export (this clip has some since it was a native camera shot)

Calculated per frame metrics, but I didn't plot it out, I'm just posting the averages . FFmpeg psnr

self test check orig vs. orig
lossless

x264_qp1_intra vs. orig
PSNR y:68.087383 u:67.117369 v:67.088122 average:67.733770 min:67.343042 max:68.028963

ConvertBits(8).ConvertToYV12(chromaresample="point") vs. orig
PSNR y:67.782155 u:65.525216 v:66.421621 average:67.082155 min:65.246296 max:67.911099

ConvertToYUV420(chromaresample="point").ConvertBits(8) vs. orig
#In theory 10bit downsample before converting to 8 is better, but it doesn't matter when you're just dropping samples with "point", it should with other kernals
PSNR y:67.782155 u:65.525216 v:66.421621 average:67.082155 min:65.246296 max:67.911099

z_convertformat(pixel_type="YV12", resample_filter="point", resample_filter_uv="point") vs. orig
PSNR y:53.656829 u:66.398999 v:66.530673 average:55.305442 min:53.734979 max:57.186774


There is some loss , but PSNR Y,U,V > 65 is pretty good . Minus 1 or 2 db per channel vs. the qp1_keyint1 input into Resolve . You expect some loss because of the internal YUV=>RGB=>YUV conversion in resolve, in addition to, and separate from the subsampling/upsampling

Using long GOP, you expect that up/down pattern in those charts, especially with b-frames using default I/P/B ratios. Maybe try repeating with intra

Maybe a big problem with zimg Y plane 10bit => 8bit conversion. A 12-13db PSNR delta is massive. I have a feeling user error somewhere but I repeated results with vapoursynth version , same values. I'll recheck everything, but on that colored text clip it was lower than internal AVS too.

Last edited by poisondeathray; 29th March 2020 at 00:00.
poisondeathray is offline   Reply With Quote
Old 29th March 2020, 00:43   #15  |  Link
Stereodude
Registered User
 
Join Date: Dec 2002
Location: Region 0
Posts: 1,436
Quote:
Originally Posted by poisondeathray View Post
There is an intermediate RGB step in Resolve too. If there wasn't, and Resolve could work in YUV, you would expect Y to be the same since you're measuring Y psnr in those graphs, not Y, U, V or aggregate YUV PSNR. For programs that are YUV capable, there is no alteration of the Y plane . v210 gets special treatment - it actually gets special treatment in other NLE's too, even ones that work internally in RGB . It's a native format that gets passthrough . For some Windows based NLE's, you can use IYUV (8bit 4:2:0, with fourcc IYUV, or 8bit 4:2:2 with fourcc UYVY) and those gets passthrough , but Resolve is really devloped from a Mac environment , there is no way to get it to work with uncompressed 8bit 4:2:0
I guess I can't speak to what resolve is doing internally. I presumed it converts to RGB all the time, but maybe not. I unchecked the box for "Bypass re-encode when possible". Obviously I couldn't make any changes to the video or I'd lose my ability to do a PNSR. I guess I could have use two inline grade nodes that cancel each other out. I didn't try that.

I also didn't realize I was only looking at the Y plane. Oops. I'm brand new to using the MSU VQMT tool.

Quote:
Was this internal avs function conversion, or z_convert_format?

Recall there is a chroma shifting issue in the internal avs resizers . It's not a true point resize. There are workarounds such as resize8, but z_convert_format works properly

But it would partially depend on how resolve is defining the chroma placement too.
I tried both internal avs function and z_convert. I need to retest using YUV PNSR.

How are you measuring PNSR with ffmpeg? The free version of MSU VQMT can't do 10-bit which makes this a big pain.

Last edited by Stereodude; 29th March 2020 at 00:45.
Stereodude is offline   Reply With Quote
Old 29th March 2020, 01:02   #16  |  Link
Stereodude
Registered User
 
Join Date: Dec 2002
Location: Region 0
Posts: 1,436
Checking with YUV PNSR, z_lib (avsresize) is lossless 4:2:0 -> 4:2:2 -> 4:2:0 using point resize. The internal Avisynth functions are not lossless.

A quick test of a few other z_lib resize kernels show they are are not lossless.
Stereodude is offline   Reply With Quote
Old 29th March 2020, 01:38   #17  |  Link
poisondeathray
Registered User
 
Join Date: Sep 2007
Posts: 5,345
Quote:
Originally Posted by Stereodude View Post

How are you measuring PNSR with ffmpeg?
https://ffmpeg.org/ffmpeg-filters.html#psnr


Quote:
The free version of MSU VQMT can't do 10-bit which makes this a big pain.
Nor can it do >720p . But the GUI and plots are a nice touch and it's user friendly
poisondeathray is offline   Reply With Quote
Old 29th March 2020, 02:00   #18  |  Link
Stereodude
Registered User
 
Join Date: Dec 2002
Location: Region 0
Posts: 1,436
I used a MPEG-2 DVD source with non-square pixels since that's what I've been working on in Resolve. Maybe that has something to do with the numbers.

I compressed to H264 using --qp 1 --keyint 1 --tune psnr using the x264 engine in the latest VD2. (it's the upper set of lines on the graphs)

I redid my tests using YUV

I converted the v210 422 Quicktime back to 8-bit 420 with
z_ConvertFormat(pixel_type="YV12", resample_filter="point", colorspace_op="170m:601:170m:f=>170m:601:170m:f")


This gave different results
ConvertToYUV420(matrix="PC.601", chromaresample="point").ConvertBits(8, dither=0)

Last edited by Stereodude; 29th March 2020 at 02:11.
Stereodude is offline   Reply With Quote
Old 29th March 2020, 04:36   #19  |  Link
poisondeathray
Registered User
 
Join Date: Sep 2007
Posts: 5,345
Seems a bit low for some of those sections. Is something special about them visually?

I tried vqmt and got slightly different values (maybe slightly different psnr calcs) , but the trend was the same, average in the mid sixties. Nothing dropping that low, maybe 59.something was the lowest. Nothing to suggest a significant difference between ffmpeg psnr and msu vqmt's psnr

Mine was oversampled non square pixel , 720x480 . It was encoded with --sar 32:27 too but I doubt that would make a difference

Can you upload small clip test section that includes a peak and valley ?
poisondeathray is offline   Reply With Quote
Old 29th March 2020, 10:10   #20  |  Link
kolak
Registered User
 
Join Date: Nov 2004
Location: Poland
Posts: 2,843
Resolve handling of 420/422 was far from correct.
It was enough to export v210 5x back to v210 (which was always going through RGB) and you had red channel shifted few pixels! Then they improved it a bit, but it's still not very good. They will also now pass v210 (and not only) without any processing if you do just cut editing.

Last edited by kolak; 29th March 2020 at 10:36.
kolak is offline   Reply With Quote
Reply

Tags
chroma bug, resolve

Thread Tools Search this Thread
Search this Thread:

Advanced Search
Display Modes

Posting Rules
You may not post new threads
You may not post replies
You may not post attachments
You may not edit your posts

BB code is On
Smilies are On
[IMG] code is On
HTML code is Off

Forum Jump


All times are GMT +1. The time now is 09:35.


Powered by vBulletin® Version 3.8.11
Copyright ©2000 - 2024, vBulletin Solutions Inc.