Welcome to Doom9's Forum, THE in-place to be for everyone interested in DVD conversion.

Before you start posting please read the forum rules. By posting to this forum you agree to abide by the rules.

 

Go Back   Doom9's Forum > Capturing and Editing Video > Avisynth Usage

Reply
 
Thread Tools Search this Thread Display Modes
Old 1st November 2016, 19:09   #1  |  Link
SuperLumberjack
French Registered User
 
SuperLumberjack's Avatar
 
Join Date: Aug 2016
Location: France
Posts: 268
Loss of details in gradients when I convert YUV 4:2:2 to YUV 4:2:0

Hello

Today I tried to do a capture of the game "Rayman Origins" in 60 fps with Dxtory, just to see if it works (I haven't any SSD).

I got good results with the codec "Ut Video YUV422 BT.709 VCM" with a stable framerate. Strangely, it was less stable with "Ut Video YUV420 BT.709 VCM"

So, I need to use the YUV 4:2:2 format. But in output from Sony Vegas, I will use the codec "Ut Video YUV420 BT.709 VCM", because I want to encode my video with Handbrake.

And now is the problem !

I noticed when I converted YUV 4:2:2 to YUV 4:2:0, that there is a big degradation of the gradients. I know it's normal that there is a little loss, but here it's a bit too much

This is a screenshot from the capture with the codec "Ut Video YUV422 BT.709 VCM" :



With the "Ut Video YUV420 BT.709 VCM" :



And after I converted the video in YUV 4:2:2 to YUV 4:2:0 with "Ut Video YUV420 BT.709 VCM" :



Compared to the original YUV 4:2:0 capture, it's worse !

My only possibility should be to use the codec Lagarith in the Sony Vegas output, but one day I use Lagarith and the colors vere wrong with Handbrake. It's why I used the codec Ut Video, because it's flagged with the good color matrix.

How can I do to avoid this big loss of quality in the gradients do you think ?

Is it possible with AviSynth before I put the sequence in Sony Vegas ? Or have you another solution please ?


Thanks for you help !

(sorry for my bad English)
SuperLumberjack is offline   Reply With Quote
Old 1st November 2016, 19:34   #2  |  Link
Mounir
Registered User
 
Join Date: Nov 2006
Posts: 718
Try to disable smart resample in sony vegas (look on youtube-> disable smart resample, sony vegas)
Mounir is offline   Reply With Quote
Old 1st November 2016, 19:59   #3  |  Link
SuperLumberjack
French Registered User
 
SuperLumberjack's Avatar
 
Join Date: Aug 2016
Location: France
Posts: 268
Hi

I always disable the smart resample in Sony Vegas But here, I converted my video with VirtualDub. But it's the same with Vegas.
SuperLumberjack is offline   Reply With Quote
Old 1st November 2016, 21:53   #4  |  Link
SuperLumberjack
French Registered User
 
SuperLumberjack's Avatar
 
Join Date: Aug 2016
Location: France
Posts: 268
I tried with my video in YUV 4:2:2, to export it with the codec Lagarith in YV12.

It's a bit better :



But it's not as good as the original YUV 4:2:0.

The other problem is with Handbrake :



As you can see, reds looks orange. I think Handbrake think that the video use the Rec.601 color matrix, and then convert it to Rec.709.

It's maybe because there is no flag for the color matrix with Lagarith.

So it's not the solution

Any idea ?

Thanks !
SuperLumberjack is offline   Reply With Quote
Old 1st November 2016, 23:41   #5  |  Link
FranceBB
Broadcast Encoder
 
FranceBB's Avatar
 
Join Date: Nov 2013
Location: Germany
Posts: 678
@SuperLumberjack... why don't you save your video in 4:2:2 Ut Video, then convert it to 4:2:0 using avisynth Converttoyv12(interlaced=false) and then save it to lossless Ut Video? Besides, you could use flash3kyuu to deband your clip after Converttoyv12() and then save it to 4:2:0 Ut Video.
FranceBB is offline   Reply With Quote
Old 1st November 2016, 23:58   #6  |  Link
SuperLumberjack
French Registered User
 
SuperLumberjack's Avatar
 
Join Date: Aug 2016
Location: France
Posts: 268
Thanks for you answer

I tried with AviSynth to convert the YUY2 to YV12, but I found that it was a bit blurry. But I will try again with your suggestions tomorrow

Before, I tried with Fraps too instead of Dxtory, and I can capture a video in 60 fps very well.

The problem, as you see at 4 min 46 in this video, is that the videos in YUV have wrong colors compared to RGB

https://youtu.be/dn3umK0gir0?t=4m46s
SuperLumberjack is offline   Reply With Quote
Old 2nd November 2016, 00:24   #7  |  Link
FranceBB
Broadcast Encoder
 
FranceBB's Avatar
 
Join Date: Nov 2013
Location: Germany
Posts: 678
Hang on.
If RGB has the correct colours, then why don't you save it in RGB and then convert it with avisynth to yv12 and convert the color matrix to BT709 using colormatrix in order to avoid having the wrong colours once you have the YUV clip?
FranceBB is offline   Reply With Quote
Old 2nd November 2016, 00:34   #8  |  Link
SuperLumberjack
French Registered User
 
SuperLumberjack's Avatar
 
Join Date: Aug 2016
Location: France
Posts: 268
Since I noticed these wrong colors in YUV with Fraps, I use RGB.

The problem is that it's impossible to have smooth videos with this in 60 fps without have a SSD. So I'm limited with 30 fps with RGB.

Fraps in YUV has clearly the advantage to be very reactive. Even more that Dxtory I find.
SuperLumberjack is offline   Reply With Quote
Old 2nd November 2016, 01:38   #9  |  Link
raffriff42
Retried Guesser
 
raffriff42's Avatar
 
Join Date: Jun 2012
Posts: 1,377
For the color shift, try adding this in the handbrake x264 extra options textbox (I have not tested this)
colormatrix=bt709:colorprim=bt709:transfer=bt709

For the gradients, it looks like the captured source has minor banding issues (not unusual for a video game); color conversion in 8-bit arithmetic is making it more noticeable. Try adding a very small amount of dither (called add noise in Vegas)
raffriff42 is offline   Reply With Quote
Old 2nd November 2016, 02:01   #10  |  Link
SuperLumberjack
French Registered User
 
SuperLumberjack's Avatar
 
Join Date: Aug 2016
Location: France
Posts: 268
Thanks raffriff42

I will try. But why is there more banding when I convert YUV 4:2:2 to YUV 4:2:0, if I compare to the original YUV 4:2:0 ?

Shouldn't it look almost identical ?

Another question : do you understand why the colors are wrong in YUV with Fraps compared to RGB ? It's too bad, because the videos in YUV with Fraps are very smooth

Why anybody noticed this ? Although I contacted the Fraps team and talked to them about the problem, but they didn't really understand

And I show them with pictures like that :



That's a shame because Fraps is a good product. Without this default, it could be even better !

Last edited by SuperLumberjack; 2nd November 2016 at 02:07.
SuperLumberjack is offline   Reply With Quote
Old 2nd November 2016, 02:22   #11  |  Link
raffriff42
Retried Guesser
 
raffriff42's Avatar
 
Join Date: Jun 2012
Posts: 1,377
> But why is there more banding when I convert YUV 4:2:2 to YUV 4:2:0
Because the color channels are being resampled, and this always introduces errors, especially-noticeable with 8-bit video, and super-especially-noticeable on shallow gradients.

>why the colors are wrong in YUV with Fraps compared to RGB
They are not wrong if they are decoded properly and transcoded properly. It uses an unusual format: full range "Rec.709."

I wrote a program (FRAFS Test Pattern) to help screen-capture users like you and me spot color problems right away, at any point in the signal chain -- from screen to the end viewer.
Quote:
Do you worry if Fraps (or other computer video capture tool) is giving you accurate colors? Do you wish you had some kind of standard, recordable (DirectX 9) source with known colors? Well, that's what this is, so you're in luck.
raffriff42 is offline   Reply With Quote
Old 2nd November 2016, 02:37   #12  |  Link
SuperLumberjack
French Registered User
 
SuperLumberjack's Avatar
 
Join Date: Aug 2016
Location: France
Posts: 268
You're the boss raffriff42 !

I will see this later, because I need to sleep. It's 2h37 in France !
SuperLumberjack is offline   Reply With Quote
Old 2nd November 2016, 16:40   #13  |  Link
SuperLumberjack
French Registered User
 
SuperLumberjack's Avatar
 
Join Date: Aug 2016
Location: France
Posts: 268
Very good test raffriff42 !

I tried with Fraps in RGB, and no problem :



But colors are different in YUV as I said :



I noticed a little difference with the Ut Video codecs in YUV too, but it's more correct :



You can see this with the color picker in MS Paint.

So, what should I do ?
SuperLumberjack is offline   Reply With Quote
Old 2nd November 2016, 18:44   #14  |  Link
SuperLumberjack
French Registered User
 
SuperLumberjack's Avatar
 
Join Date: Aug 2016
Location: France
Posts: 268
I came back on the story of YUV 4:2:2 converted to YUV 4:2:0

I wanted to try to encode directly the YUV 4:2:2 in YUV 4:2:0 with Handbrake without any convertion before. I find the result is very good !

The MKV x264 from Handbrake in YUV 4:2:0 (YUV 4:2:2 input) :



The original YUV 4:2:2 :



Maybe the compression helps a bit too (but we lost a bit the depth I found)

So, it's the best solution and the simplest !

What do you think ? Is it possible to do better ? I tried with AviSynth, but the reds became more red

But with Fraps YUV, how can I correct the colors ?

Last edited by SuperLumberjack; 2nd November 2016 at 21:12.
SuperLumberjack is offline   Reply With Quote
Old 2nd November 2016, 21:58   #15  |  Link
raffriff42
Retried Guesser
 
raffriff42's Avatar
 
Join Date: Jun 2012
Posts: 1,377
Glad you like my test! But how are the colors bad? Yes, the RGB bars all measure 191 exactly, and some of the YUV bars measure 189, but I consider that small deviation to be acceptable.


If you mean the loss of fine color detail, that comes with the YV12 format.
raffriff42 is offline   Reply With Quote
Old 2nd November 2016, 22:10   #16  |  Link
SuperLumberjack
French Registered User
 
SuperLumberjack's Avatar
 
Join Date: Aug 2016
Location: France
Posts: 268
Yeah, good test !

No, the problem is not the loss of details. But yes, I can accept a small deviation like with the Ut Video. The problem is for the Fraps YUV. It's more visible I think, especially on this picture :



The third image was the Fraps RGB converted in YUV 4:2:0 with the codec Ut Video. The Ut Video is nearer to the RGB I find.
SuperLumberjack is offline   Reply With Quote
Reply

Thread Tools Search this Thread
Search this Thread:

Advanced Search
Display Modes

Posting Rules
You may not post new threads
You may not post replies
You may not post attachments
You may not edit your posts

BB code is On
Smilies are On
[IMG] code is On
HTML code is Off

Forum Jump


All times are GMT +1. The time now is 12:15.


Powered by vBulletin® Version 3.8.11
Copyright ©2000 - 2019, vBulletin Solutions Inc.