Frames still blended after deinterlacing?

[VideoMilk78]

I'm working on a PAL VHS. When I deinterlace, frames from the previous and next scenes are still blended. Is there anything I can do about this, or is this the best it's going to get?

Here is an example of one such frame
https://i.ibb.co/My8m43St/deint.png (https://ibb.co/dsbRqzMV)

[StainlessS]

The guys are gonna wanna sample.

EDIT: At least 10 seconds with constant movement.
EDIT: Are you saying blended at scene changes ?,
if so, then provide a sample with a couple of scene changes.
EDIT: Also, PAL/VHS, what format container, And showing your script too might be a good idea.

[AVIL]

Hi,

You can deintrlace the frames in scene changes with nnedi3, You can left top filed, bottom field (result video with same framecount) or both. (framecount increases). It depends on your footage. Another trick is duplicate the bottom and top field in scene changes (the result is two consecutive frames with both fields equal). Then apply same deinterlace as the rest of the video. Framecount is increased. A sample as StainsessS can help.

[VideoMilk78]

The guys are gonna wanna sample.

https://92.gigafile.nu/0812-c0a7efe55419ddda182d39126d9db091f

EDIT: Are you saying blended at scene changes ?
Yes


EDIT: Also, PAL/VHS, what format container, And showing your script too might be a good idea.

MKV

video=FFVideoSource("C:\users\crossover\downloads\CDFRAW.MKV")
audio=FFAudioSource("C:\users\crossover\downloads\ CDFRAW.MKV.mkv")
clip=audiodub(video,audio)
deint=clip.qtgmc(preset="slow",fpsdivisor=2)
return deint
prefetch(10)

[Selur]

try

TFM()
Cdeblend(omode=2,xr=4,yr=4) # alternatively sRestores post processing modes

[VideoMilk78]

try

TFM()
Cdeblend(omode=2,xr=4,yr=4) # alternatively sRestores post processing modes


After QTGMC I presume?

[AVIL]

Hi

Telecined footage is too complicated to my skills. My best approach:


avisource("X:\Dat4TB_1\Temp\test.avi").assumetff().separatefields().selectodd().nnedi3(field=0, dh=true)

[VideoMilk78]

Hi

Telecined footage is too complicated to my skills. My best approach:


avisource("X:\Dat4TB_1\Temp\test.avi").assumetff().separatefields().selectodd().nnedi3(field=0, dh=true)

This is no different, maybe a little worse than qtgmc from what I can tell side by side.

[Selur]

After QTGMC I presume?

No. Instead.

[StainlessS]

VideoMilk78,
I think that tfm() matches fields [works better when DgIndex d2v source for DVD VOB], and Selur suspects that
blending happens because scene cut occurs mid frame, ie one field belongs to pre scene and other belongs to the post cut scene.
(so your result deinterlaced frame seems to be blend)

[think that what Selur is thinking]

[Selur]

Agree.
When bobbing content and half of the (non-scene change) content is trash, testing field matching on its own usually is a good idea.
After that only the blends on the scene changes were present and sRestores p-modes and cDeblend can replace those by duplicating a neighboring frame.
Did seem to work in a quick test before I went to bed, yesterday.
:)

[pbristow]

To my eye it looks not just telecine'd, but also standards converted. Which would make sense if the original production is a Japanese TV show (i.e. using NTSC-J), but it means that some frames may have been permanently blended during the standards conversion to PAL; Just recovering the converted PAL frames correctly can't fix that (and as far as I'm aware at this time, nothing can) and these side-effects from the conversion may confuse some deinterlacers to the point where they have no idea how to identify the correct PAL fields anyway.

Best of luck with this! =:o1

[VideoMilk78]

To my eye it looks not just telecine'd, but also standards converted. Which would make sense if the original production is a Japanese TV show (i.e. using NTSC-J), but it means that some frames may have been permanently blended during the standards conversion to PAL; Just recovering the converted PAL frames correctly can't fix that (and as far as I'm aware at this time, nothing can) and these side-effects from the conversion may confuse some deinterlacers to the point where they have no idea how to identify the correct PAL fields anyway.
This is a telecine of a film print, it should all be PAL.
Best of luck with this! =:o1
Thanks, I think I got it sorted out. It's never gonna be perfect, all you can do is improve.

[poisondeathray]

Thanks, I think I got it sorted out. It's never gonna be perfect, all you can do is improve.

What did you do to sort it out ? are you happy with what you have now ?

Because I have some other suggestions to improve it

[VideoMilk78]

What did you do to sort it out ? are you happy with what you have now ?

Because I have some other suggestions to improve it
Happy would be an overstatement but it's certainly improved, still open to suggestions though

[poisondeathray]

It's not only scene changes that are affected - other frames far from the scenechange are also blended. Also, there is more than 1 type of blend.

Was test.mkv the original source , or a re-encoded sample ? Do you have a proper 4:2:2 source ? Some of the chroma ghosting artifacts could be due to improper progressive resampling of the chroma , instead of interlace aware resampling. ie. when 4:2:2 interlace chroma is converted to 4:2:0 with progressive resampling while still organized in interleaved fields.

One type of blend is a warped/blurred field. 1 field of the pair is always worse. Selur mentioned this above. But it's not always top or always bottom . You can remove the fpsdivisor and go through the scenes to see this, or any double rate deinterlacer. A botched telecine, and probably multiple conversions

The problem with qtgmc(preset="slow",fpsdivisor=2) is it really means AssumeBFF().QTGMC().SelectOdd() . You are always choosing the bottom field . That happens to be "cleaner" for some scenes, but more blended on other scenes.

eg.

qtgmc(preset="slow",fpsdivisor=2)
https://i.postimg.cc/wvYqnYKV/528-bff.png

assumetff().qtgmc(preset="slow",fpsdivisor=2)
https://i.postimg.cc/rFZ8MrXz/528-tff.png

The top field is less blended, and every field (deinterlaced to frame) in that scene is less blended if you choose top. A clue is the line in the wall - so the background is less blended, but the FG actor still has some blends

You can partially deblend a frame by subtracting it's blended precursor frame by a certain %, then adjusting the brightness and saturation to match. It doesn't always work great because of 8bit clipping, lossy compression, etc... but it can sometimes improve some types of blends

This is an apng - it should animate in most browsers
https://i.postimg.cc/GhBy0YHh/compare-bff-tff-overlaysubtractdeblend-apng.png (https://postimages.org/)

I used something like this for the scene after , where b is current frame, a is the one before, from the selection of "good" set of frames. The reason Overlay_MTools was used , is the math is slightly different and seemed to work slightly better than internal Overlay(mode="subtract") in this case


Overlay_MTools(b,a, mode="subtract", opacity=0.4).smoothlevels(5,1.5,255,0,255)


I used this for after the scenechange in the sample below . It's just a rough levels compensation for the subtract, you probably have to adjust it a bit


LWLibavVideoSource("test.mkv")
bwdif(field=3)
bw=last
Overlay_MTools(bw.trim(1050,-1),bw.trim(1048,-1), mode="subtract", opacity=0.4).smoothlevels(5,1.5,255,0,255) + \
Overlay_MTools(bw.trim(1052,-1),bw.trim(1050,-1), mode="subtract", opacity=0.4).smoothlevels(5,1.5,255,0,255) + \
Overlay_MTools(bw.trim(1054,-1),bw.trim(1052,-1), mode="subtract", opacity=0.35).smoothlevels(5,1.4,255,0,255) + \
Overlay_MTools(bw.trim(1056,-1),bw.trim(1054,-1), mode="subtract", opacity=0.3).smoothlevels(6,1.4,255,0,255) + \
Overlay_MTools(bw.trim(1058,-1),bw.trim(1056,-1), mode="subtract", opacity=0.3).smoothlevels(6,1.4,255,0,255)
assumefps(25)
mergechroma(awarpsharp2(depth=48)) #try to improve the chroma bleeding / misalignment - you might not need it if starting with proper 4:2:2 source


A double rate deinterlace + srestore(25) is supposed to be "smarter" and choose the less blended 25 frames out of 50 fps, instead of "blindly" selecting the odd ones, but it's not going to be as accurate as manually checking. It makes the wrong decisions in this sample, but maybe you can play with the settings and get it more accurate

You can interpolate frames using "video prediction." Instead of interpreting the frame(s) between 2 "good" reference points, you can take 2 adjacent "good" frames as reference to predict the next, or the previous frame. The key is "good" because if you have blended or garbage frames used as reference, the predicted frame will be blended or garbage as well. See this thread for more info .
https://forum.doom9.org/showthread.php?p=1985227


https://www.mediafire.com/file/fxx2yuest9y6mtk/compare_deblend.mp4/file
Here are a few frames before/after that scene change, slowed down, comparing qtgmc(preset="slow",fpsdivisor=2) to the correct field selection with partial deblending. The scenechange frame before was interpolated with DMVFN. You can also try the avs functions PredictFromPrevious/Next based on mvtools2, but they generally produce signifcantly worse results, or you can try other deblending methods. If you step through frame by frame, the predicted frame is temporally coherent - ie. shadows move, car tail lights move etc...

This is an apng of the frame in the 1st post, it should animate in most browsers
https://i.postimg.cc/yxmy0ZcH/sc-DMVFN-apng.png (https://postimages.org/)


Also, I'd be careful using QTMGC with deblending operations - because the temporal processing can create new blending, or make existing blending worse, or create other problems like warping of lines/objects. Sometimes it might be ok, but always double check

[AVIL]

Hi;

Another "poor man" approach:

avisource("X:\Dat4TB_1\Temp\test.avi").converttoyv16().nnedi3(field=1, dh=false).vhsclean().converttoyuy2()

You can grab VHSCLEAN script from this thread:

https://forum.doom9.org/showthread.php?t=185261

[VideoMilk78]

It's not only scene changes that are affected - other frames far from the scenechange are also blended. Also, there is more than 1 type of blend.

Was test.mkv the original source , or a re-encoded sample ? Do you have a proper 4:2:2 source ? Some of the chroma ghosting artifacts could be due to improper progressive resampling of the chroma , instead of interlace aware resampling. ie. when 4:2:2 interlace chroma is converted to 4:2:0 with progressive resampling while still organized in interleaved fields.

One type of blend is a warped/blurred field. 1 field of the pair is always worse. Selur mentioned this above. But it's not always top or always bottom . You can remove the fpsdivisor and go through the scenes to see this, or any double rate deinterlacer. A botched telecine, and probably multiple conversions

The problem with qtgmc(preset="slow",fpsdivisor=2) is it really means AssumeBFF().QTGMC().SelectOdd() . You are always choosing the bottom field . That happens to be "cleaner" for some scenes, but more blended on other scenes.

eg.

qtgmc(preset="slow",fpsdivisor=2)
https://i.postimg.cc/wvYqnYKV/528-bff.png

assumetff().qtgmc(preset="slow",fpsdivisor=2)
https://i.postimg.cc/rFZ8MrXz/528-tff.png

The top field is less blended, and every field (deinterlaced to frame) in that scene is less blended if you choose top. A clue is the line in the wall - so the background is less blended, but the FG actor still has some blends

You can partially deblend a frame by subtracting it's blended precursor frame by a certain %, then adjusting the brightness and saturation to match. It doesn't always work great because of 8bit clipping, lossy compression, etc... but it can sometimes improve some types of blends

This is an apng - it should animate in most browsers
https://i.postimg.cc/GhBy0YHh/compare-bff-tff-overlaysubtractdeblend-apng.png (https://postimages.org/)

I used something like this for the scene after , where b is current frame, a is the one before, from the selection of "good" set of frames. The reason Overlay_MTools was used , is the math is slightly different and seemed to work slightly better than internal Overlay(mode="subtract") in this case


Overlay_MTools(b,a, mode="subtract", opacity=0.4).smoothlevels(5,1.5,255,0,255)


I used this for after the scenechange in the sample below . It's just a rough levels compensation for the subtract, you probably have to adjust it a bit


LWLibavVideoSource("test.mkv")
bwdif(field=3)
bw=last
Overlay_MTools(bw.trim(1050,-1),bw.trim(1048,-1), mode="subtract", opacity=0.4).smoothlevels(5,1.5,255,0,255) + \
Overlay_MTools(bw.trim(1052,-1),bw.trim(1050,-1), mode="subtract", opacity=0.4).smoothlevels(5,1.5,255,0,255) + \
Overlay_MTools(bw.trim(1054,-1),bw.trim(1052,-1), mode="subtract", opacity=0.35).smoothlevels(5,1.4,255,0,255) + \
Overlay_MTools(bw.trim(1056,-1),bw.trim(1054,-1), mode="subtract", opacity=0.3).smoothlevels(6,1.4,255,0,255) + \
Overlay_MTools(bw.trim(1058,-1),bw.trim(1056,-1), mode="subtract", opacity=0.3).smoothlevels(6,1.4,255,0,255)
assumefps(25)
mergechroma(awarpsharp2(depth=48)) #try to improve the chroma bleeding / misalignment - you might not need it if starting with proper 4:2:2 source


A double rate deinterlace + srestore(25) is supposed to be "smarter" and choose the less blended 25 frames out of 50 fps, instead of "blindly" selecting the odd ones, but it's not going to be as accurate as manually checking. It makes the wrong decisions in this sample, but maybe you can play with the settings and get it more accurate

You can interpolate frames using "video prediction." Instead of interpreting the frame(s) between 2 "good" reference points, you can take 2 adjacent "good" frames as reference to predict the next, or the previous frame. The key is "good" because if you have blended or garbage frames used as reference, the predicted frame will be blended or garbage as well. See this thread for more info .
https://forum.doom9.org/showthread.php?p=1985227


https://www.mediafire.com/file/fxx2yuest9y6mtk/compare_deblend.mp4/file
Here are a few frames before/after that scene change, slowed down, comparing qtgmc(preset="slow",fpsdivisor=2) to the correct field selection with partial deblending. The scenechange frame before was interpolated with DMVFN. You can also try the avs functions PredictFromPrevious/Next based on mvtools2, but they generally produce signifcantly worse results, or you can try other deblending methods. If you step through frame by frame, the predicted frame is temporally coherent - ie. shadows move, car tail lights move etc...

This is an apng of the frame in the 1st post, it should animate in most browsers
https://i.postimg.cc/yxmy0ZcH/sc-DMVFN-apng.png (https://postimages.org/)


Also, I'd be careful using QTMGC with deblending operations - because the temporal processing can create new blending, or make existing blending worse, or create other problems like warping of lines/objects. Sometimes it might be ok, but always double check

Thank you so much, this has been very useful!

[StainlessS]

Here a bit of an experiment to detect blends at scene change. [EDIT: ONLY at scene change]
DetBlendFrameSC.avs - EXPERIMENTAL (but works) :- https://forum.doom9.org/showthread.php?t=183790

You could swap blended frame for one of the frames before/after it.

EDIT:

DetBlendFrameSC(clip c,String "FramesFile"="BlendFrameSC.txt",Float "dfactor"=0.90,Float "dMin"=40.0,
\ int "x"=16,int "y"=16, int "w"=-x, int "h"=-y,Bool "ClipClopIx"=False, Bool "Show"=false,
\ Bool "Force_ZOK"=False,Bool "Force_AltOK"=False)

Purpose:- To write a frames file of frame numbers that are single frame blend at scene change. (blend of last frame of scene and first frame of following scene).
These Target frames are very difficult to detect as scene changes.
Can fix blended frame using eg FrameSel or ClipClop, replace with frame before or after the blended frame.
To Replace blended frame with eg ClipClop clip 1, use a clip src.DuplicateFrame(0) [replace with previous frame], OR src.DeleteFrame(0) [replace with following frame].

We use a modified result from RT_LumaCorrelation for distance measure between frames.
See Pearson’s Distance here: http://en.wikipedia.org/wiki/Pearson_product-moment_correlation_coefficient
where Pearson's Distance is equivalent to:

PD(x,y) = 1.0 - Correlation(x,y) # PD Range 0.0 -> 2.0

However we mod to

PD(x,y) = (1.0 - Correlation(x,y)) * 255.0 # Range 0.0 -> 510.0



n = current_frame
Prv = (1 - RT_LumaCorrelation(Last,Last,n=n-1,n2=n ,x=x,y=y,w=w,h=h)) * 255.0 # Distance, prev <-> current
Nxt = (1 - RT_LumaCorrelation(Last,Last,n=n ,n2=n+1,x=x,y=y,w=w,h=h)) * 255.0 # Distance, current <-> Next
Sum = Prv+Nxt
Z = Sum * dFactor + dMin

Alt = (1 - RT_LumaCorrelation(Last,Last,n=n-1,n2=n+1,x=x,y=y,w=w,h=h)) * 255.0 # Distance, Prev <-> Next, either side of current
pAl = (1 - RT_LumaCorrelation(Last,Last,n=n-2,n2=n ,x=x,y=y,w=w,h=h)) * 255.0 # Distance, either side of previous
nAl = (1 - RT_LumaCorrelation(Last,Last,n=n ,n2=n+2,x=x,y=y,w=w,h=h)) * 255.0 # Distance, either side of next

ZOK = (Z < Alt)

AltHICOR = 0.5
AltLOCOR = 0.65
AltHI = (1.0 - AltHICOR) * 255 # Distance 127.5.00 equivalent to correlation 0.50
AltLO = (1.0 - AltLoCOR) * 255 # Distance 89.25 equivalent to correlation 0.65

AltOK = (Alt > AltHi && (pAl < AltLo || nAl < AltLo)) # ALT must look like scenechange and at least one of prv/nxt ALTs like NOT Scene change
# AltOK helps avoid false +ve's, and nearly detects single frame blends OK all on its own.

Blend = (ZOK||Force_ZOK) && (AltOK || Force_AltOK)


Maybe of use, maybe not.

[VideoMilk78]

I used this for after the scenechange in the sample below . It's just a rough levels compensation for the subtract, you probably have to adjust it a bit


LWLibavVideoSource("test.mkv")
bwdif(field=3)
bw=last
Overlay_MTools(bw.trim(1050,-1),bw.trim(1048,-1), mode="subtract", opacity=0.4).smoothlevels(5,1.5,255,0,255) + \
Overlay_MTools(bw.trim(1052,-1),bw.trim(1050,-1), mode="subtract", opacity=0.4).smoothlevels(5,1.5,255,0,255) + \
Overlay_MTools(bw.trim(1054,-1),bw.trim(1052,-1), mode="subtract", opacity=0.35).smoothlevels(5,1.4,255,0,255) + \
Overlay_MTools(bw.trim(1056,-1),bw.trim(1054,-1), mode="subtract", opacity=0.3).smoothlevels(6,1.4,255,0,255) + \
Overlay_MTools(bw.trim(1058,-1),bw.trim(1056,-1), mode="subtract", opacity=0.3).smoothlevels(6,1.4,255,0,255)
assumefps(25)
mergechroma(awarpsharp2(depth=48)) #try to improve the chroma bleeding / misalignment - you might not need it if starting with proper 4:2:2 source


How might this be applied to the full film?

[Arx1meD]

VideoMilk78, try it:

v01 = last.SelectEvery(2, 0) # Select 0,2,4,6... frames
v02 = v01.Trim(2,0) # shift by 2 frames
Overlay(v01, v02, mode="subtract", opacity=0.35).SmoothLevels(5, 1.5, 255, 0, 255)

Or:

v00 = last.SelectEvery(2, 0) # Select 0,2,4,6... frames
v01 = v00.DuplicateFrame(0,0) # add 2 frames to the beginning
v02 = v00
Overlay(v01, v02, mode="subtract", opacity=0.35).SmoothLevels(5, 1.5, 255, 0, 255)

I don't know which way would be correct.