ABurns
3rd February 2022, 05:45
I'm working with some VHS captures that have a couple of different problems at scene changes. The last frame before the scene change has a discoloration at the bottom that I think might be from the physical film print where it was glued, taped, or something similar. It doesn't happen on every change, but it's on a vast majority. See example below.
https://thumbs4.imagebam.com/db/95/a2/ME7GGZR_t.jpg (https://www.imagebam.com/view/ME7GGZR) https://thumbs4.imagebam.com/cb/f7/e8/ME7GGZY_t.jpg (https://www.imagebam.com/view/ME7GGZY)
Then after the scene change, the chroma gets smeared (for lack of a better word) for the first 1-2 frames of the new scene. Sometimes there is a big difference in tint or saturation that clears up, and sometimes it's just smeared and nasty looking. Example below.
https://thumbs4.imagebam.com/44/91/c2/ME7GH8M_t.jpg (https://www.imagebam.com/view/ME7GH8M) https://thumbs4.imagebam.com/43/ea/f7/ME7GH8T_t.jpg (https://www.imagebam.com/view/ME7GH8T) https://thumbs4.imagebam.com/35/f4/35/ME7GH90_t.jpg (https://www.imagebam.com/view/ME7GH90)
As far as I can tell, it's just the chroma that's affected. I really don't want to do frame replacement because that will create a noticeable stutter on action scenes, and I haven't been able to find anything that can interpolate new frames without having both a previous and next keyframe, which obviously doesn't work at a scene change. Then I started looking for a way to just copy the chroma from the adjacent frame because it should be close enough I can get away with it, something like MergeChroma(source frame, target frame) but that doesn't appear to exist either.
So I'm at a loss what to do with these frames. The luma is perfectly good and they should be salvageable, but I'm out of ideas. Any suggestions?
zorr
4th February 2022, 23:23
That sounds feasible if I can get it to work. I'm afraid MVTools is a bit above my pay grade, as I've never really understood its toolbox. What might that function look like?
Here's a simple script to demonstrate the concept. It's taking the chroma from the next frame (use fwd1 in MCompensate instead of bak1 to get chroma from the previous frame). Final output has the original frame, mixed frame (luma from current and chroma from next frame) and the MCompensated frame.
This would still need the logic to only apply the correction after a scene change.
Note that the functions are used with default values so they are not in any way optimal, you can get much better MCompensated frames with the right settings. If you provide a short clip with typical motion (also those action scenes) I could try to find optimal settings with Zopti.
FFVideoSource("clip.avi")
src=last
pre=last
superfilt = MSuper(pre)
superR = MSuper(src)
bak1 = MAnalyse(superfilt, isb=true, delta=1)
fwd1 = MAnalyse(superfilt, isb=false, delta=1)
MCompensate(superR, bak1, scbehavior=false, thSCD1=10000)
mix = MergeChroma(src, last)
return Interleave(src, mix, last)
zorr
12th February 2022, 00:25
All right, got the results. I did a pretty comprehensive Zopti search with about 134k results (and a total of 58k results in three earlier script variations) using dynamic iteration count.
The result landscape looks like this. Here all results:
https://i.postimg.cc/B6PwR44T/mcompensate4-top100.png
Zoomed to top 70% of results:
https://i.postimg.cc/9fnnxr5X/mcompensate4-top70.png
And finally the top 30%
https://i.postimg.cc/0NQ4d74H/mcompensate4-top30.png
Below the script with the best GMSD score which outputs the original frame and a fixed frame where chroma is taken from the MCompensated next frame. Note that this works for frames just after a scene change, for frames before scene change you need to take the chroma from the previous frame (just comment the line with # from next frame and uncomment the line with # from prev frame). Thanks to Dogway for the script template, I used his MDegrain script as the base.
FFVideoSource("sample.mov")
#FFVideoSource("sample2.mov")
#pre = FFVideoSource("mcompensate_prev-pre.avi")
frames = FrameCount()
#return last
src=last
thSADR = 50 # optimize thSADR = _n_ | 0..200 ; filter:x 10 % 0 == | thSADR
BlkSize = 16
BlkSizeR = 8
overlap = 4
overlapR = 4
pel = 1
sharp = 2 #NO optimize sharp = _n_ | 2 | sharp
scaleCSAD = 2
trymany = false
truemotion = false #NO optimize truemotion = _n_ | true,false | truemotion
truemotionR = false #NO optimize truemotionR = _n_ | false,true | truemotionR
temporal = false # optimize temporal = _n_ | true,false | temporal
# Settings that depend on source noise
# lsad the noisier the lower
# sigma the noisier the higher
# badSAD
# TRUEMOTION SETTINGS
lambda = 270 # optimize lambda = _n_ | 0..300 ; filter:x 10 % 0 == | lambda
# lambdaR normally optimizes between 1.6 and 2.0 times lambda
lambdaR = 0 # optimize lambdaR = _n_ | 0..200 ; filter:x 20 % 0 == | lambdaR
# pnew: Default is 0 for truemotion = false and 50 for truemotion = true.
pnew = 4 # optimize pnew = _n_ | 0..80 ; filter:x 2 % 0 == | pnew
pnewR = 0 # optimize pnewR = _n_ | 0..50 ; filter:x 2 % 0 == | pnewR
# lambda is not used when pzero is 0 (zero vector)
# there's a relationship between pzero and searchRangeR (and searchRangeR with searchRangeFinest)
pzero = 176 # optimize pzero = _n_ | 0..200 ; filter:x 2 % 0 == | pzero
lsad = 21 # optimize lsad = _n_ | 1..1000 ; filter:x 1 - 10 % 0 == | lsad
# plevel: Default is 0 for truemotion = false and 50 for truemotion = true
plevel = 59 # optimize plevel = _n_ | 1..200 ; filter:x 2 % 0 != | plevel
lvl = 1 # typically plevel is set same as level
# lambda is not used for global predictor
pglobal = 5 # optimize pglobal = _n_ | 0..40 | pglobal
badrange = 144 # optimize badrange = _n_ | 0..150 ; filter:x 2 % 0 == | badrange
badSAD = 0 # optimize badSAD = _n_ | 0..500 ; filter:x 50 % 0 == | badSAD
dct = 0 #NO optimize dct = _n_ | 0 | dct
dctre = 0 # optimize dctre = _n_ | 0,6,7,9 | dctre
rfilter = 3
searchAlgo = 1 #NO optimize searchAlgo = _n_ | 1 | searchAlgo
searchAlgoR = 5 # optimize searchAlgoR = _n_ | 0,1,2,4,5 | searchAlgoR
searchRange = 4 # optimize searchRange = _n_ | 1..20 | searchRange
searchRangeR = 60 # optimize searchRangeR = _n_ | 2..60 | searchRangeR
searchRangeFinest = 10# optimize searchRangeFinest = _n_ | 9..30 | searchRangeFinest
sglobal = true # optimize sglobal = _n_ | true,false | sglobal
C=ConvertBits(16,fulls=false)
pre=C.ex_FluxSmoothST(2,2,255,0,false,UV=3).ex_Luma_Rebuild(s0=3,tv_range=true).ConvertBits(8,dither=-1,fulls=true)
#return pre
Recalculate=true
superfilt = MSuper(pre, hpad=16, vpad=16, sharp=sharp, rfilter=rfilter, pel=pel, mt=false)
superR = MSuper(C, hpad=16, vpad=16, levels=lvl, sharp=sharp, rfilter=rfilter, pel=pel, mt=false, chroma=true)
superRe = MSuper(pre, hpad=16, vpad=16, levels=lvl, sharp=sharp, rfilter=rfilter, pel=pel, mt=false)
bak1 = MAnalyse(superfilt, isb=true, delta=1, blksize=BlkSize, overlap = overlap, search=searchAlgo, searchparam=searchRange, pelsearch=searchRangeFinest, dct=dct, mt=false, scaleCSAD=scaleCSAD, pnew=pnew, pzero=pzero, truemotion=truemotion, badSAD=badSAD, badrange=badrange, temporal=temporal,lsad=lsad, lambda=lambda, pglobal=pglobal, plevel=plevel, trymany=trymany, global=sglobal)
fwd1 = MAnalyse(superfilt, isb=false, delta=1, blksize=BlkSize, overlap = overlap, search=searchAlgo, searchparam=searchRange, pelsearch=searchRangeFinest, dct=dct, mt=false, scaleCSAD=scaleCSAD, pnew=pnew, pzero=pzero, truemotion=truemotion, badSAD=badSAD, badrange=badrange, temporal=temporal,lsad=lsad, lambda=lambda, pglobal=pglobal, plevel=plevel, trymany=trymany, global=sglobal)
bak1 = Recalculate ? MRecalculate(superRe, bak1, blksize=BlkSizeR, overlap = overlapR, search=searchAlgoR, searchparam=searchRangeR, dct=dctre, mt=false, scaleCSAD=scaleCSAD, pnew=pnewR, thSAD=thSADR, truemotion=truemotionR, lambda=lambdaR) : bak
fwd1 = Recalculate ? MRecalculate(superRe, fwd1, blksize=BlkSizeR, overlap = overlapR, search=searchAlgoR, searchparam=searchRangeR, dct=dctre, mt=false, scaleCSAD=scaleCSAD, pnew=pnewR , thSAD=thSADR, truemotion=truemotionR, lambda=lambdaR) : fwd
C.MCompensate(superR, bak1, mt=false, thSCD1=10000) # from next frame
#C.MCompensate(superR, fwd1, mt=false, thSCD1=10000) # from prev frame
#mix = MergeChroma(C, last)
#return Interleave(C, mix, last).Histogram("levels")
output = last.ConvertBits(10,fulls=true)
mix = MergeChroma(src, output)
return Interleave(src, mix).Histogram("levels")
return Interleave(src, mix, output).Histogram("levels")
#return last
src = src.Trim(0,176) + src.Trim(180, 281) + src.Trim(286, frames-2)
Trim(0,176) + Trim(180, 281) + Trim(286, frames-2)
resultFile = "perFrameResults.txt" # output out1="GMSD: MIN(float)" out2="time: MIN(time) ms" file="perFrameResults.txt"
# write the number of frames to tell the optimizer how many result lines to expect
WriteFileStart(resultFile, "FrameCount()")
GMSD(src.ConvertBits(32, fulls=false, fulld=true), ConvertBits(32, fulls=false, fulld=true), show=false)
global GMSD = 0.0
FrameEvaluate(last, """
GMSD = propGetFloat("_PlaneGMSD")
global GMSD = (GMSD == 0.0 ? 1.0 : GMSD)
""",local=false)
# measure runtime, plugin writes the value to global avstimer variable
global avstimer = 0.0
AvsTimer(frames=1, type=0, total=false, name="Optimizer")
# per frame logging (GMSD, time)
global delimiter = "; "
WriteFile(resultFile, "current_frame", "delimiter", "GMSD", "delimiter", "avstimer")
#Prefetch(4)
#return Interleave(src.ConvertBits(32, fulls=false, fulld=true), last, C.ConvertBits(32, fulls=false, fulld=true))
return last
More in the next post, this one can't fit more...
vBulletin® v3.8.11, Copyright ©2000-2026, vBulletin Solutions Inc.