Welcome to Doom9's Forum, THE in-place to be for everyone interested in DVD conversion.

Before you start posting please read the forum rules. By posting to this forum you agree to abide by the rules.

 

Go Back   Doom9's Forum > Capturing and Editing Video > VapourSynth

Reply
 
Thread Tools Search this Thread Display Modes
Old 29th December 2016, 19:31   #1  |  Link
feisty2
I'm Siri
 
feisty2's Avatar
 
Join Date: Oct 2012
Location: Los Angeles, California
Posts: 2,133
New filter: Fix Telecined Fades

binary(windows x64, requires msvcr 2017):https://github.com/IFeelBloated/Fix-...eleases/tag/r5
git repo:https://github.com/IFeelBloated/Fix-...ter/Source.cpp

the filter gives a mathematically perfect solution to such(fades were done AFTER telecine which made a picture perfect IVTC pretty much impossible) problem, and it's now time to kiss "vinverse" goodbye cuz "vinverse" is old and low quality.
unlike vinverse which works as a dumb blurring + contra-sharpening combo and very harmful to artifacts-free frames, this filter works by matching the brightness of top and bottom fields with statistical methods, and also harmless to healthy frames.

Code:
core.ftf.FixFades(clip, mode=0, threshold=0.002, color=[0.0, 0.0, 0.0], opt=True)
clip: clip to be processed

mode: could be 0(default), 1, or 2
0: adjust the brightness of both fields to match the average brightness of 2 fields.
1: darken the brighter field to match the brightness of the darker field
2: brighten the darker field to match the brightness of the brighter field

threshold: threshold for the average difference per pixel, on a scale of 0.0 - 1.0, but could go beyond 1.0, the frame will remain untouched if the average difference between 2 fields goes below this value

color: base color of the fade, default is [0.0, 0.0, 0.0](black)

opt: call the fastest possible functions if opt=True, else call the C++ functions.

INPUT CLIP MUST BE 32BITS FLOATING POINT FORMAT!!!
apply this filter AFTER field matching!!!
__________________
If I got new ideas, will post here: https://github.com/IFeelBloated

Last edited by feisty2; 28th January 2017 at 17:15.
feisty2 is offline   Reply With Quote
Old 29th December 2016, 20:03   #2  |  Link
feisty2
I'm Siri
 
feisty2's Avatar
 
Join Date: Oct 2012
Location: Los Angeles, California
Posts: 2,133
comparison against vinverse
input
Code:
import vapoursynth as vs
core = vs.get_core()
clp = core.lsmas.LWLibavSource("rule6")
clp = core.vivtc.VFM(clp,0)
clp.set_output()

vinverse
Code:
import vapoursynth as vs
core = vs.get_core()
clp = core.lsmas.LWLibavSource("rule6")
clp = core.vivtc.VFM(clp,0)

clp = core.vinverse.Vinverse(clp)

clp.set_output()

FTF
Code:
import vapoursynth as vs
core = vs.get_core()
clp = core.lsmas.LWLibavSource("rule6")
clp = core.vivtc.VFM(clp,0)
clp = core.fmtc.bitdepth(clp,bits=32,fulls=False,fulld=True)

clp = core.ftf.FixFades(clp)

clp.set_output()
__________________
If I got new ideas, will post here: https://github.com/IFeelBloated
feisty2 is offline   Reply With Quote
Old 29th December 2016, 20:16   #3  |  Link
feisty2
I'm Siri
 
feisty2's Avatar
 
Join Date: Oct 2012
Location: Los Angeles, California
Posts: 2,133
and vinverse failed miserably at the top-right part of the image, there's clear aliasing around edges, and ftf produced a picture perfect reconstruction.
__________________
If I got new ideas, will post here: https://github.com/IFeelBloated
feisty2 is offline   Reply With Quote
Old 29th December 2016, 21:22   #4  |  Link
Mystery Keeper
Beyond Kawaii
 
Mystery Keeper's Avatar
 
Join Date: Feb 2008
Location: Russia
Posts: 711
Does it do combing detection? Can you do better combing detection than TDM?
__________________
...desu!
Mystery Keeper is offline   Reply With Quote
Old 30th December 2016, 04:36   #5  |  Link
feisty2
I'm Siri
 
feisty2's Avatar
 
Join Date: Oct 2012
Location: Los Angeles, California
Posts: 2,133
Quote:
Originally Posted by Mystery Keeper View Post
Does it do combing detection? Can you do better combing detection than TDM?
No combing detection, it pretty much won't affect the normal frames so no masking required. Technically every pixel will be processed but only the problem frames will be affected.
__________________
If I got new ideas, will post here: https://github.com/IFeelBloated
feisty2 is offline   Reply With Quote
Old 30th December 2016, 06:35   #6  |  Link
ShogoXT
Registered User
 
Join Date: Dec 2011
Posts: 93
Does this work in general on IVTC scene transitions? Since using Vapoursynth (through Staxrip) ive been having such a problem with duplicate frames and scthresh with VFM and Vdecimate (no blending???), that ive been forced to stay with mode=0, but scene changes look awful. Ive also been having trouble with dealing with rainbows and dotcrawl in motion, but I guess thats another problem as all the solutions are temporal only.

EDIT: Also scrolling scenes stink...

Last edited by ShogoXT; 30th December 2016 at 06:38.
ShogoXT is offline   Reply With Quote
Old 30th December 2016, 09:42   #7  |  Link
feisty2
I'm Siri
 
feisty2's Avatar
 
Join Date: Oct 2012
Location: Los Angeles, California
Posts: 2,133
Quote:
Originally Posted by ShogoXT View Post
Does this work in general on IVTC scene transitions? Since using Vapoursynth (through Staxrip) ive been having such a problem with duplicate frames and scthresh with VFM and Vdecimate (no blending???), that ive been forced to stay with mode=0, but scene changes look awful. Ive also been having trouble with dealing with rainbows and dotcrawl in motion, but I guess thats another problem as all the solutions are temporal only.

EDIT: Also scrolling scenes stink...
not sure what you meant by "IVTC scene transitions", it works on any "simple" telecined fades(fade from/to pure color, eg. black), and it won't work on telecined cross-fades(2 or more scenes blending into each other), shit like that is simply beyond repair.
__________________
If I got new ideas, will post here: https://github.com/IFeelBloated
feisty2 is offline   Reply With Quote
Old 30th December 2016, 15:33   #8  |  Link
WolframRhodium
Registered User
 
Join Date: Jan 2016
Posts: 110
Great algorithm. The result is fantastic.

Question:
Will the results be even better if the algorithm acting on local rather than global?

Last edited by WolframRhodium; 30th December 2016 at 15:37.
WolframRhodium is offline   Reply With Quote
Old 30th December 2016, 18:48   #9  |  Link
feisty2
I'm Siri
 
feisty2's Avatar
 
Join Date: Oct 2012
Location: Los Angeles, California
Posts: 2,133
r2:
fixed a stupid memory leak
new parameter "threshold"
new parameter "color"
__________________
If I got new ideas, will post here: https://github.com/IFeelBloated
feisty2 is offline   Reply With Quote
Old 30th December 2016, 18:49   #10  |  Link
feisty2
I'm Siri
 
feisty2's Avatar
 
Join Date: Oct 2012
Location: Los Angeles, California
Posts: 2,133
Quote:
Originally Posted by WolframRhodium View Post
Question:
Will the results be even better if the algorithm acting on local rather than global?
done, use the "threshold" parameter
__________________
If I got new ideas, will post here: https://github.com/IFeelBloated

Last edited by feisty2; 30th December 2016 at 18:59.
feisty2 is offline   Reply With Quote
Old 30th December 2016, 19:06   #11  |  Link
feisty2
I'm Siri
 
feisty2's Avatar
 
Join Date: Oct 2012
Location: Los Angeles, California
Posts: 2,133
another demonstration for something other than fading into blackness
input
Code:
import vapoursynth as vs
core = vs.get_core()
clp = core.lsmas.LWLibavSource("rule6")
clp = core.vivtc.VFM(clp,0)
clp = core.fmtc.bitdepth(clp,bits=32,fulls=False,fulld=True)
clp = core.std.Expr(clp, ["1.0 x -", "x"]) #invert Y so it would be literally fading into whiteness
clp.set_output()

FTF
Code:
import vapoursynth as vs
core = vs.get_core()
clp = core.lsmas.LWLibavSource("rule6")
clp = core.vivtc.VFM(clp,0)
clp = core.fmtc.bitdepth(clp,bits=32,fulls=False,fulld=True)
clp = core.std.Expr(clp, ["1.0 x -", "x"]) #invert Y so it would be literally fading into whiteness

clp = core.ftf.FixFades(clp, color=[1.0, 0.0, 0.0])

clp.set_output()
__________________
If I got new ideas, will post here: https://github.com/IFeelBloated
feisty2 is offline   Reply With Quote
Old 30th December 2016, 20:22   #12  |  Link
jackoneill
unsigned int
 
jackoneill's Avatar
 
Join Date: Oct 2012
Location: 🇪🇺
Posts: 723
Quote:
Originally Posted by feisty2 View Post
and it won't work on telecined cross-fades(2 or more scenes blending into each other), shit like that is simply beyond repair.
No, no, fades like that can still be saved. By which I mean that you can obtain a 24 fps fade with smooth motion even if field matching is impossible in the fade. The solution involves QTGMC, so the detail preservation isn't awesome, but it's still good, considering the situation.

Steps:
1. Invoke source filter to obtain the untouched 30 fps clip.
2. Extract the fade using Trim.
3. Pass to QTGMC to obtain a 60 fps clip.
4. Stare at the output of QTGMC really carefully. It's been a few months since I did this, so I forget exactly what you're looking for.
5. Extract 4 frames out of a cycle of 10 using SelectEvery.
Tada!
__________________
Buy me a "coffee" and/or hire me to write code!
jackoneill is offline   Reply With Quote
Old 30th December 2016, 21:44   #13  |  Link
ShogoXT
Registered User
 
Join Date: Dec 2011
Posts: 93
Quote:
Originally Posted by jackoneill View Post
No, no, fades like that can still be saved. By which I mean that you can obtain a 24 fps fade with smooth motion even if field matching is impossible in the fade. The solution involves QTGMC, so the detail preservation isn't awesome, but it's still good, considering the situation.

Steps:
1. Invoke source filter to obtain the untouched 30 fps clip.
2. Extract the fade using Trim.
3. Pass to QTGMC to obtain a 60 fps clip.
4. Stare at the output of QTGMC really carefully. It's been a few months since I did this, so I forget exactly what you're looking for.
5. Extract 4 frames out of a cycle of 10 using SelectEvery.
Tada!
I was under the impression that if the source is 2:3 telecined that it HAD to be ivtc and not QTGMC. QTGMC is very nice but I thought it was for interlace 121212 only. Vinverse has helped me a little, does the color conversation and fixfade cost much speed?
ShogoXT is offline   Reply With Quote
Old 30th December 2016, 22:08   #14  |  Link
fAy01
Registered User
 
Join Date: Jun 2010
Posts: 74
Quote:
Originally Posted by jackoneill View Post
No, no, fades like that can still be saved. By which I mean that you can obtain a 24 fps fade with smooth motion even if field matching is impossible in the fade. The solution involves QTGMC, so the detail preservation isn't awesome, but it's still good, considering the situation.

Steps:
1. Invoke source filter to obtain the untouched 30 fps clip.
2. Extract the fade using Trim.
3. Pass to QTGMC to obtain a 60 fps clip.
4. Stare at the output of QTGMC really carefully. It's been a few months since I did this, so I forget exactly what you're looking for.
5. Extract 4 frames out of a cycle of 10 using SelectEvery.
Tada!
Is there a possibility to write a script/plugin that deinterlaces and matches correctly at the same time without using QTGMC?
fAy01 is offline   Reply With Quote
Old 30th December 2016, 22:47   #15  |  Link
Mystery Keeper
Beyond Kawaii
 
Mystery Keeper's Avatar
 
Join Date: Feb 2008
Location: Russia
Posts: 711
Quote:
Originally Posted by ShogoXT View Post
I was under the impression that if the source is 2:3 telecined that it HAD to be ivtc and not QTGMC. QTGMC is very nice but I thought it was for interlace 121212 only. Vinverse has helped me a little, does the color conversation and fixfade cost much speed?
After IVTC you can still find a lot of combed frames of various origins.
__________________
...desu!
Mystery Keeper is offline   Reply With Quote
Old 31st December 2016, 00:43   #16  |  Link
WolframRhodium
Registered User
 
Join Date: Jan 2016
Posts: 110
Quote:
Originally Posted by feisty2 View Post
done, use the "threshold" parameter
I mean maybe the calculation of TopFieldSum/BottomFieldSum can be done in a small neighborhood, for example, 16x16 block?

There seems to be some residual fades after filtering, and I think the reason is the calculation is done in global, currently.
WolframRhodium is offline   Reply With Quote
Old 31st December 2016, 07:16   #17  |  Link
feisty2
I'm Siri
 
feisty2's Avatar
 
Join Date: Oct 2012
Location: Los Angeles, California
Posts: 2,133
Quote:
Originally Posted by WolframRhodium View Post
I mean maybe the calculation of TopFieldSum/BottomFieldSum can be done in a small neighborhood, for example, 16x16 block?

There seems to be some residual fades after filtering, and I think the reason is the calculation is done in global, currently.
the statistical info(TopFieldSum/BottomFieldSum) should be no-local according to the weak law of large numbers.
if you got residual combing still even with threshold=0, it's then most likely your video suffers from something other than telecined fades and this filter won't help
__________________
If I got new ideas, will post here: https://github.com/IFeelBloated
feisty2 is offline   Reply With Quote
Old 31st December 2016, 07:23   #18  |  Link
feisty2
I'm Siri
 
feisty2's Avatar
 
Join Date: Oct 2012
Location: Los Angeles, California
Posts: 2,133
Quote:
Originally Posted by jackoneill View Post
No, no, fades like that can still be saved. By which I mean that you can obtain a 24 fps fade with smooth motion even if field matching is impossible in the fade. The solution involves QTGMC, so the detail preservation isn't awesome, but it's still good, considering the situation.

Steps:
1. Invoke source filter to obtain the untouched 30 fps clip.
2. Extract the fade using Trim.
3. Pass to QTGMC to obtain a 60 fps clip.
4. Stare at the output of QTGMC really carefully. It's been a few months since I did this, so I forget exactly what you're looking for.
5. Extract 4 frames out of a cycle of 10 using SelectEvery.
Tada!
my definition of "repair" is, to perfectly restore the frame without deinterlacing.
__________________
If I got new ideas, will post here: https://github.com/IFeelBloated
feisty2 is offline   Reply With Quote
Old 31st December 2016, 08:14   #19  |  Link
WolframRhodium
Registered User
 
Join Date: Jan 2016
Posts: 110
Quote:
Originally Posted by feisty2 View Post
the statistical info(TopFieldSum/BottomFieldSum) should be no-local according to the weak law of large numbers.
if you got residual combing still even with threshold=0, it's then most likely your video suffers from something other than telecined fades and this filter won't help
Thank you for your reply. I misunderstood the function of this filter before and now I understand it better.
WolframRhodium is offline   Reply With Quote
Old 15th January 2017, 10:01   #20  |  Link
feisty2
I'm Siri
 
feisty2's Avatar
 
Join Date: Oct 2012
Location: Los Angeles, California
Posts: 2,133
r3:
AVX and FMA3 optimizations, will probably crash if your CPU is a predecessor of the Haswell microarchitecture (just use r2 in that case)
__________________
If I got new ideas, will post here: https://github.com/IFeelBloated
feisty2 is offline   Reply With Quote
Reply

Thread Tools Search this Thread
Search this Thread:

Advanced Search
Display Modes

Posting Rules
You may not post new threads
You may not post replies
You may not post attachments
You may not edit your posts

BB code is On
Smilies are On
[IMG] code is On
HTML code is Off

Forum Jump


All times are GMT +1. The time now is 23:30.


Powered by vBulletin® Version 3.8.11
Copyright ©2000 - 2019, vBulletin Solutions Inc.