Welcome to Doom9's Forum, THE in-place to be for everyone interested in DVD conversion.

Before you start posting please read the forum rules. By posting to this forum you agree to abide by the rules.

 

Go Back   Doom9's Forum > Capturing and Editing Video > Avisynth Usage

Reply
 
Thread Tools Search this Thread Display Modes
Old 25th July 2014, 23:07   #1  |  Link
Trephin
Registered User
 
Trephin's Avatar
 
Join Date: Aug 2002
Location: Stockholm, Sweden
Posts: 17
Chroma flicker when deinterlacing DVD

I'm trying to deinterlace a 25 fps PAL DVD box set that I bought. It is an old childrens PAL TV show that I want to convert to AVC for my NAS and tablet, because a couple of episodes didn't play on my TV when ripped with MakeMKV.

What happens is that when I deinterlace with QTGMC (or Yadif or Bob.SelectEven), every other frame has a different hue. I'm not sure if it is like that for all scenes, but in some places it is obvious.

Maybe this is some kind of mastering error where the luma and chroma are interlaced differently?


Any help would be appreciated. I have been at it for days.


These are two adjacent frames that show the issue:




This is my script:

Code:
MPEG2Source("VTS_01_1_20.d2v")
Bob().SelectEven()
It looks fine when I let VLC deinterlace the stream, but I guess it does the deinterlacing after upsampling the color space.

One way I stumbled upon that appears to fix the colors was to upsample as progressive to YUY2 before deinterlacing, but that feels like the wrong solution (or maybe it is a good way to smooth the chroma?). Anyway, this is the script that appears to fix the chroma flicker:

Code:
MPEG2Source("VTS_01_1_20.d2v")
ConvertToYUY2()
Bob().SelectEven()
ConvertToYV12()
Here is a link to a 5 MiB VOB sample from DGSplit:
http://dropproxy.com/f/89D

And here is a link to a 3.8 MiB AVC MKV after encoding from the first script above:
http://dropproxy.com/f/89E

This is the log info from DGIndex:

Code:
Stream Type: MPEG2 Program
Profile: main@main
Frame Size: 720x576
Display Size: [not specified]
Aspect Ratio: 4:3 [2]
Frame Rate: 25.000000 fps
Video Type: PAL
Frame Type: Interlaced
Coding Type: B
Colorimetry: BT.470-2 B,G*
Frame Structure: Frame
Field Order: 
Coded Number: 275130
Playback Number: 3
Frame Repeats: 0
Field Repeats: 0
VOB ID: 1
Cell ID: 2
Bitrate: 
Bitrate (Avg): 
Bitrate (Max): 
Audio Stream: 80: AC3 2/0 224
Timestamp: 1:33:53
Elapsed: 0:04:28
Remain: FINISH
FPS: 
Info:
Thanks in advance,
// Trephin.
__________________
remember sammy jankis

Last edited by Trephin; 26th July 2014 at 23:04. Reason: Corrected spelling error in title
Trephin is offline   Reply With Quote
Old 26th July 2014, 01:04   #2  |  Link
Reel.Deel
Registered User
 
Join Date: Mar 2012
Location: Texas
Posts: 1,084
Flickering it's only present in the U channel, maybe some very aggressive temporal smoothing can fix this?
Reel.Deel is offline   Reply With Quote
Old 26th July 2014, 13:17   #3  |  Link
Trephin
Registered User
 
Trephin's Avatar
 
Join Date: Aug 2002
Location: Stockholm, Sweden
Posts: 17
Quote:
Originally Posted by Reel.Deel View Post
Flickering it's only present in the U channel, maybe some very aggressive temporal smoothing can fix this?
Yes, probably, but I'm thinking that the correct chroma information is restorable in some way. Like a shift of one field or frame or something like that, but I haven't managed to find a way yet. I'm not very experienced in deinterlacing and color sampling. Maybe the progressive ConvertToYUY2 is the best way to smooth the chroma temporally?

Sent from my SM-G900F using Tapatalk
__________________
remember sammy jankis
Trephin is offline   Reply With Quote
Old 26th July 2014, 13:54   #4  |  Link
StainlessS
HeartlessS Usurer
 
StainlessS's Avatar
 
Join Date: Dec 2009
Location: Over the rainbow
Posts: 5,653
There is an UpConv arg to Mpeg2Source that could bear some inspection.
Also AutoYUY2 is an alternative.

PS, this 'Chorma' you mention, would that be Chicken. or Beef Chorma ?
__________________
I sometimes post sober.
StainlessS@MediaFire ::: AND/OR ::: StainlessS@SendSpace

"Some infinities are bigger than other infinities", but how many of them are infinitely bigger ???

Last edited by StainlessS; 26th July 2014 at 16:05.
StainlessS is offline   Reply With Quote
Old 26th July 2014, 19:53   #5  |  Link
Reel.Deel
Registered User
 
Join Date: Mar 2012
Location: Texas
Posts: 1,084
Ok so I had some time to mess with this and somehow managed to put something together that actually looks pleasing. The following script is mainly a copypasta from here and here. Thanks to the original authors for sharing.

Code:
MPEG2Source("VTS_01_1_20.d2v")
source = last
oo = ConvertToYUY2().ConvertToYV12(interlaced=true).SeparateFields() # wrong but works in this case
SeparateFields()
o = last

# Luma destripe
FIR = Mt_Convolution(Horizontal=" 48 64 96 64 48 ", vertical ="1")
Diff = Mt_Makediff(Last, FIR)

THR=string("256")
MedianDiff = MT_Luts(Diff, Diff, mode="med", pixels = " 0 0 1 0 2 0 -1 0 -2 0 " ,  expr = " X Y - X Y - X Y - abs 1 + * X Y - abs 1 + "+THR+" 1 >= "+THR+" 0.5 ^ "+THR+" ? + / - 128 +"). \
fft3dfilter(bw=2, bh=16, ow=1, oh=8, bt=1, sigma=16, sigma2=0.75, sigma3=16, sigma4=16, plane=0)
ReconstructedMedian = mt_makediff(Diff, MedianDiff)
Mt_AddDiff(FIR, ReconstructedMedian)

Y = last

# U channel
rest=oo.UToY()
ux = rest.width()
uy = rest.height()

calm = rest.temporalsoften(1,255,255,32,2).merge(rest,0.25)#.repair(rest,1).removegrain(11)
calm = calm.temporalsoften(1,255,255,16,2).merge(calm,0.25)

LOP = calm.bicubicresize(ux/4,uy/4).bicubicresize(ux,uy,1,0) #  can be altered, but ~25% of original resolution seems reasonable
HIP = rest.bicubicresize(ux/4,uy/4).bicubicresize(ux,uy,1,0) #  can be altered, but ~25% of original resolution seems reasonable
HIP = mt_makediff(rest,HIP)
mix = LOP.mt_adddiff(HIP,U=2,V=2)

sup1=rest.msuper(levels=1, hpad=0, vpad=0) # only use hpad and vpad if there's absolutely no black borders
sup2=mix.removegrain(11,0).msuper(hpad=0, vpad=0) # only use hpad and vpad if there's absolutely no black borders

bv2=sup2.manalyse(isb=true, delta=2,blksize=16,overlap=8,DCT=5)
bv1=sup2.manalyse(isb=true, delta=1,blksize=16,overlap=8,DCT=5)
fv1=sup2.manalyse(isb=false,delta=1,blksize=16,overlap=8,DCT=5)
fv2=sup2.manalyse(isb=false,delta=2,blksize=16,overlap=8,DCT=5)

U = rest.mdegrain2(sup1,bv1,fv1,bv2,fv2,thSAD=640, plane=0)

# V channel
rest=oo.VToY()
resto=o.VtoY()
vx = rest.width()
vy = rest.height()

calm = rest.temporalsoften(1,255,255,32,2).merge(rest,0.25)#.repair(rest,1).removegrain(11)
calm = calm.temporalsoften(1,255,255,16,2).merge(calm,0.25)

LOP = calm.bicubicresize(vx/4,vy/4).bicubicresize(vx,vy,1,0) #  can be altered, but ~25% of original resolution seems reasonable
HIP = rest.bicubicresize(vx/4,vy/4).bicubicresize(vx,vy,1,0) #  can be altered, but ~25% of original resolution seems reasonable
HIP = mt_makediff(rest,HIP)
mix = LOP.mt_adddiff(HIP,U=2,V=2)

sup1=resto.msuper(levels=1, hpad=0, vpad=0) # only use hpad and vpad if there's absolutely no black borders
sup2=mix.removegrain(11,0).msuper(hpad=0, vpad=0) # only use hpad and vpad if there's absolutely no black borders

bv2=sup2.manalyse(isb=true, delta=2,blksize=16,overlap=8,DCT=5)
bv1=sup2.manalyse(isb=true, delta=1,blksize=16,overlap=8,DCT=5)
fv1=sup2.manalyse(isb=false,delta=1,blksize=16,overlap=8,DCT=5)
fv2=sup2.manalyse(isb=false,delta=2,blksize=16,overlap=8,DCT=5)

V = resto.mdegrain2(sup1,bv1,fv1,bv2,fv2,thSAD=640, plane=0)


YToUV(U,V,Y)
Weave()
QTGMC()
I'm sure this script can be fine tuned for better performance/results. Here's a video comparing before and after (source.QTGMC() vs script above).

Last edited by Reel.Deel; 3rd August 2014 at 13:05. Reason: typo
Reel.Deel is offline   Reply With Quote
Old 26th July 2014, 23:17   #6  |  Link
Trephin
Registered User
 
Trephin's Avatar
 
Join Date: Aug 2002
Location: Stockholm, Sweden
Posts: 17
Quote:
Originally Posted by StainlessS View Post
There is an UpConv arg to Mpeg2Source that could bear some inspection.
Also AutoYUY2 is an alternative.
Thanks, I will look in to them, but I would like to stay away from upsampling if I don't have to.

Quote:
Originally Posted by StainlessS View Post
PS, this 'Chorma' you mention, would that be Chicken. or Beef Chorma ?
Lamb Thank you for correcting me. I have updated the thread title.
__________________
remember sammy jankis
Trephin is offline   Reply With Quote
Old 26th July 2014, 23:45   #7  |  Link
Trephin
Registered User
 
Trephin's Avatar
 
Join Date: Aug 2002
Location: Stockholm, Sweden
Posts: 17
Quote:
Originally Posted by Reel.Deel View Post
Ok so I had some time to mess with this and somehow managed to put something together that actually looks pleasing. The following script is mainly a copypasta from here and here. Thanks to the original authors for sharing.

Code:
-snip-
I'm sure this script can be fine tuned for better performance/results. Here's a video comparing before and after (source.QTGMC() vs script above).
Wow! Thank you for taking the time to dig into my video and write this out. There is a lot going on in your script and a lot of it is new to me. I will try to understand and maybe fine tune it, if I can. I'm away from my computer now for a week, so I can't mess with the video now, but I can read up on the filters used in your script. Did you compare your results with a source.ConvertToYUY2().QTGMC().ConvertToYV12()?
__________________
remember sammy jankis
Trephin is offline   Reply With Quote
Old 27th July 2014, 09:01   #8  |  Link
Reel.Deel
Registered User
 
Join Date: Mar 2012
Location: Texas
Posts: 1,084
Quote:
Originally Posted by Trephin View Post
Did you compare your results with a source.ConvertToYUY2().QTGMC().ConvertToYV12()?
I don't have QTGMC setup to process YUY2 so I compared to this (similar result to your method):
Code:
MPEG2Source("VTS_01_1_20.d2v")
ConvertToYV16()
ConvertToYV12(interlaced=true)
QTGMC()
Even thought is technically wrong (resizing interlaced fields) it helps against the erratic flickering. The downside is that we're crippling the chroma temporal resolution. It's really not noticeable here but may or may not work well in other types of scenes. In my script above I used this trick on the U channel only and used it for the motion analysis in the V channel.
Reel.Deel is offline   Reply With Quote
Old 1st August 2014, 09:43   #9  |  Link
qwerty1983
Registered User
 
Join Date: Mar 2013
Posts: 44
I think i had a similar problem.

Look at this thread : http://forum.doom9.org/showthread.php?t=168473
qwerty1983 is offline   Reply With Quote
Old 3rd August 2014, 11:01   #10  |  Link
Trephin
Registered User
 
Trephin's Avatar
 
Join Date: Aug 2002
Location: Stockholm, Sweden
Posts: 17
Quote:
Originally Posted by qwerty1983 View Post
I think i had a similar problem.

Look at this thread : http://forum.doom9.org/showthread.php?t=168473
Yes, it looks similar. I found your thread before, but the difference is that my clip doesn't flicker until I deinterlace it, so I had hopes that the correct chroma information was there in some way.
__________________
remember sammy jankis
Trephin is offline   Reply With Quote
Old 3rd August 2014, 12:22   #11  |  Link
Trephin
Registered User
 
Trephin's Avatar
 
Join Date: Aug 2002
Location: Stockholm, Sweden
Posts: 17
Quote:
Originally Posted by Reel.Deel View Post
...
I have now analyzed your script and think I understand what it does. Good catch on the vertical striping! Thanks again Reel.Deel!
__________________
remember sammy jankis
Trephin is offline   Reply With Quote
Old 3rd August 2014, 13:26   #12  |  Link
Reel.Deel
Registered User
 
Join Date: Mar 2012
Location: Texas
Posts: 1,084
@Trephin

I fixed a small typo in the script (fix is in red).
Reel.Deel is offline   Reply With Quote
Old 3rd August 2014, 15:21   #13  |  Link
Trephin
Registered User
 
Trephin's Avatar
 
Join Date: Aug 2002
Location: Stockholm, Sweden
Posts: 17
Can anyone understand what is going on with my source video?

I have now ripped the second disc in the box set and DGIndex gives a message that a field order transition was detected. I found where the transition happens, and it seems that after that point, the chroma flickering has disappeared. That leads me to believe that the part before the transition is fixable by some kind of field operation.

DGIndex offers to correct the transition and creates a corrected .d2v file and a .d2v.bad file that is uncorrected.

For the part before the transition
MPEG2Source("VTS_01_1.d2v.bad").Bob().SelectEven() == MPEG2Source("VTS_01_1.d2v").Bob().SelectEven()

For the part after the transition
MPEG2Source("VTS_01_1.d2v.bad").Bob().SelectEven() == MPEG2Source("VTS_01_1.d2v").Bob().SelectEven().DeleteFrame(0)

DGIndex outputs the following log for disc 2 (there are some differences from the first disc):

Code:
Stream Type: MPEG2 Program
Profile: main@main
Frame Size: 720x576
Display Size: 720x576
Aspect Ratio: 4:3 [2]
Frame Rate: 25.000000 fps
Video Type: PAL
Frame Type: Interlaced
Coding Type: B
Colorimetry: BT.470-2 B,G
Frame Structure: Field
Field Order: 
Coded Number: 256334
Playback Number: 2
Frame Repeats: 0
Field Repeats: 0
VOB ID: 3
Cell ID: 21
Bitrate: 
Bitrate (Avg): 
Bitrate (Max): 
Audio Stream: 80: AC3 2/0 224
Timestamp: 1:07:17
Elapsed: 0:02:02
Remain: FINISH
FPS: 
Info:
Here is a link to a 6 MiB VOB sample where the field order transitions: http://dropproxy.com/f/8AF

I'm starting to think that the person that mastered this DVD is either incompetent or evil.
__________________
remember sammy jankis
Trephin is offline   Reply With Quote
Old 3rd August 2014, 15:49   #14  |  Link
videoh
Registered User
 
Join Date: Jul 2014
Posts: 417
Quote:
Originally Posted by Trephin View Post
I'm starting to think that the person that mastered this DVD is either incompetent or evil.
DVDs are intended to be played by players with MPEG2 decoders. These players are able to adjust the field order on the fly following the TFF/RFF syntax. It is only when you transcode to a format that does not contain a syntax element to signal field order that problems arise. So, the mastering is not incompetent or evil, unless you believe the publisher has an obligation to support ripping and transcoding. More often, publishers would like to discourage that.
videoh is offline   Reply With Quote
Old 3rd August 2014, 18:55   #15  |  Link
Trephin
Registered User
 
Trephin's Avatar
 
Join Date: Aug 2002
Location: Stockholm, Sweden
Posts: 17
Quote:
Originally Posted by videoh View Post
DVDs are intended to be played by players with MPEG2 decoders. These players are able to adjust the field order on the fly following the TFF/RFF syntax. It is only when you transcode to a format that does not contain a syntax element to signal field order that problems arise. So, the mastering is not incompetent or evil, unless you believe the publisher has an obligation to support ripping and transcoding. More often, publishers would like to discourage that.
I believe you are right. It's just frustrating :-)

The DVDs play perfectly fine in the physical player. I haven't encountered a DVD that was this hard to back up before though. I had to use another computer to copy them, because mine wouldn't read the discs. I think there was some kind of structural copy protection and the chapters are a mess. And then there are those chroma and field order transition annoyances.

I think maybe the source material used for the DVDs wasn't the easiest to work with, being an old TV show, and then it has been chopped up and reordered for the DVD, so that might have introduced that field order transition.
__________________
remember sammy jankis
Trephin is offline   Reply With Quote
Reply

Thread Tools Search this Thread
Search this Thread:

Advanced Search
Display Modes

Posting Rules
You may not post new threads
You may not post replies
You may not post attachments
You may not edit your posts

BB code is On
Smilies are On
[IMG] code is On
HTML code is Off

Forum Jump


All times are GMT +1. The time now is 07:57.


Powered by vBulletin® Version 3.8.11
Copyright ©2000 - 2018, vBulletin Solutions Inc.