Welcome to Doom9's Forum, THE in-place to be for everyone interested in DVD conversion.

Before you start posting please read the forum rules. By posting to this forum you agree to abide by the rules.

 

Go Back   Doom9's Forum > Capturing and Editing Video > Avisynth Development

Reply
 
Thread Tools Search this Thread Display Modes
Old 30th June 2022, 10:15   #801  |  Link
DTL
Registered User
 
Join Date: Jul 2018
Posts: 1,041
"What does bad blocks mean in that context?"

( blocks[i].GetSAD() > nTh1 )
bad (thSCD1) blocks.

I think it is blocks with SAD > thSCD1 .
DTL is offline   Reply With Quote
Old 8th July 2022, 12:53   #802  |  Link
LeXXuz
21 years and counting...
 
LeXXuz's Avatar
 
Join Date: Oct 2002
Location: Germany
Posts: 716
I just checked most of my scripts and they are running with mvtools 2.7.44

I have no idea why I haven't checked this more thoroughly.

I already had pinterf's 2.7.45 in my library, no idea why I didn't use it.

DTL you wrote 2.7.45+ so I guess there is already never version to use?
LeXXuz is offline   Reply With Quote
Old 8th July 2022, 15:25   #803  |  Link
DTL
Registered User
 
Join Date: Jul 2018
Posts: 1,041
Still no next official release like 2.7.46 from pinterf. With 2.7.45+ I mean post-2.7.45 builds I typically put at https://github.com/DTL2020/mvtools/releases .

I hope someday may be in this or next year it will be more or less merged to pinterf branch and after supplementing and testing finally we will see 2.7.46 or may be even 2.8 release. The number of strategic new features and processing modes continue to increase after first fork in 2021. I think to finish adding most of new features after interpolated overlap I plan to start at this week. But after reading mvtools docs one more time at this year I found MFlow with per-sample processing precision mode and got new idea to test it with MDegrainN frame-operation pipeline (with easy to use multi-mode). So it may be next more or less slow development feature to the MDegrainN. Also all new features added and approved need to supplement with SIMD processing functions for better speed for most or all supported colour formats and at least some commonly used block sizes and bitdepths. Currently only some formats have at least AVX2 execution in addition to very slow C-reference like only YV12 and only 8x8 block size of 8bit for UseSubShift mode of MVPlane (fully working only in MDegrainN now and planned for MAnalyse and other filters to save RAM and/or better speed at some use cases).

Typically that builds are full featured of this and https://forum.doom9.org/showthread.php?t=183517 thread described features but most of latest builds are with DX12-ME support and will not load where no DX12 installed (so no Win7 and old). So it is required separate build config to build all-win compatible executable without using DX12-ME mode of MAnalyse but using all new features of MDegrainN for example. The overlap single pass MDegrainN for colour formats are also ready to use after 06.07.22 commits but I still not make release build - you can only build yourself from latest git sources image. I plan to make new release after adding the also very strategic interpolated overlap feature to finally get solution against blockiness at some scenes with no-overlap supported hardware accelerated MAnalyse now.

Also the shader need to be fixed after final runtime subshift of block processing implementation based on old mvtools Wiener kernel - it is delayed because 12.05.22 release have not best but about good working SAD compute shader and fast rebuild of shader still not show better results and need more debug.

Some new features like single pass colour MDegrainN or MVLPF interfiltering are not depend on colour format and block size so expected to work already with all old formats. Single pass colour proc also do not require new options so compatible with old scripts directly (expected a bit better speed with same quality).

mvtools will soon be 1/5 of century aged - in January of 2024. So may we will have version 2.8.0 prepared to that celebration. Or may be final release of 2.7.46 .

Last edited by DTL; 24th July 2022 at 08:03.
DTL is offline   Reply With Quote
Old 7th March 2023, 20:59   #804  |  Link
DTL
Registered User
 
Join Date: Jul 2018
Posts: 1,041
Version for testing of (partial) fix for issue with chroma subsampled formats (both MAnalyse and MDegrainX/N) - https://github.com/DTL2020/mvtools/r.../r.2.7.45-fix1 . The description in https://github.com/pinterf/mvtools/issues/59

One of my test with x264 encoding shows a bit lower output MPEG bitrate and a bit better details at 200% size comparison.
Code:
denoise script:
for YV12 source
tr=8
super=MSuper(last,chroma=true, mt=false, pel=2)
multi_vec=MAnalyse (super, multi=true, delta=tr, search=3, searchparam=2, overlap=4, chroma=true, mt=false, truemotion=false, pnew=0, pzero=0, levels=0)
MDegrainN(last,super, multi_vec, tr, thSAD=300, thSAD2=290, mt=false, thSCD1=500)

orig 2.7.45= 
x264.exe --profile high --crf 18 --tff --ref 4 -b 10 --me umh --subme 9 --merange 16 --deblock -3:-3 --b-adapt 2 --psy-rd 1.0:0.7 --transfer "bt70
9" --colorprim "bt709" --colormatrix "bt709" -o out.264 src_t.avs
avs [info]: 960x540i 0:0 @ 25/1 fps (cfr)
x264 [warning]: interlace + weightp is not implemented
x264 [info]: using cpu capabilities: MMX2 SSE2Fast SSSE3 SSE4.1 Cache64
x264 [info]: profile High, level 3.1, 4:2:0, 8-bit
x264 [info]: frame I:5     Avg QP:14.23  size:102544
x264 [info]: frame P:203   Avg QP:17.24  size: 31718
x264 [info]: frame B:781   Avg QP:20.50  size:  6645
x264 [info]: consecutive B-frames:  1.7%  3.2%  9.1% 11.3%  8.6% 49.1%  4.2%  2.4%  2.7%  3.0%  4.4%
x264 [info]: mb I  I16..4: 14.7% 66.3% 19.0%
x264 [info]: mb P  I16..4:  1.8%  4.7%  1.0%  P16..4: 42.6% 23.4% 19.0%  0.0%  0.0%    skip: 7.4%
x264 [info]: mb B  I16..4:  0.1%  0.1%  0.0%  B16..8: 31.0%  7.6%  2.1%  direct: 7.9%  skip:51.3%  L0:35.5% L1:32.5% BI:32.0%
x264 [info]: field mbs: intra: 36.6% inter:41.6% skip:31.1%
x264 [info]: 8x8 transform intra:61.4% inter:72.9%
x264 [info]: coded y,uvDC,uvAC intra: 70.9% 82.6% 65.8% inter: 29.9% 25.5% 7.7%
x264 [info]: i16 v,h,dc,p: 29% 19% 15% 36%
x264 [info]: i8 v,h,dc,ddl,ddr,vr,hd,vl,hu: 14% 12% 14%  7%  9% 10% 11%  9% 13%
x264 [info]: i4 v,h,dc,ddl,ddr,vr,hd,vl,hu: 13% 33%  7%  5%  8%  8%  9%  7% 11%
x264 [info]: i8c dc,h,v,p: 46% 27% 19%  8%
x264 [info]: ref P L0: 49.9% 28.4% 14.0%  4.8%  1.4%  1.4%  0.0%  0.0%
x264 [info]: ref B L0: 70.4% 23.7%  3.4%  1.9%  0.2%  0.2%
x264 [info]: ref B L1: 78.0% 20.4%  0.9%  0.8%
x264 [info]: kb/s:2455.20

encoded 989 frames, 1.42 fps, 2455.20 kb/s

2.7.45 fix1=
x264.exe --profile high --crf 18 --tff --ref 4 -b 10 --me umh --subme 9 --merange 16 --deblock -3:-3 --b-adapt 2 --psy-rd 1.0:0.7 --transfer "bt70
9" --colorprim "bt709" --colormatrix "bt709" -o out.264 src_t.avs
avs [info]: 960x540i 0:0 @ 25/1 fps (cfr)
x264 [warning]: interlace + weightp is not implemented
x264 [info]: using cpu capabilities: MMX2 SSE2Fast SSSE3 SSE4.1 Cache64
x264 [info]: profile High, level 3.1, 4:2:0, 8-bit
x264 [info]: frame I:5     Avg QP:14.14  size:102566
x264 [info]: frame P:202   Avg QP:17.24  size: 31792
x264 [info]: frame B:782   Avg QP:20.51  size:  6558
x264 [info]: consecutive B-frames:  1.4%  3.4%  9.7%  8.5% 10.6% 51.6%  4.2%  3.2%  1.8%  1.0%  4.4%
x264 [info]: mb I  I16..4: 14.8% 65.9% 19.3%
x264 [info]: mb P  I16..4:  1.8%  4.6%  1.1%  P16..4: 42.6% 23.5% 19.1%  0.0%  0.0%    skip: 7.3%
x264 [info]: mb B  I16..4:  0.1%  0.1%  0.0%  B16..8: 30.9%  7.5%  2.1%  direct: 7.7%  skip:51.6%  L0:35.5% L1:32.3% BI:32.1%
x264 [info]: field mbs: intra: 36.0% inter:41.4% skip:31.1%
x264 [info]: 8x8 transform intra:61.4% inter:72.9%
x264 [info]: coded y,uvDC,uvAC intra: 71.4% 82.9% 66.0% inter: 29.7% 25.1% 7.5%
x264 [info]: i16 v,h,dc,p: 30% 20% 15% 35%
x264 [info]: i8 v,h,dc,ddl,ddr,vr,hd,vl,hu: 14% 12% 14%  7% 10% 10% 11%  9% 13%
x264 [info]: i4 v,h,dc,ddl,ddr,vr,hd,vl,hu: 13% 32%  7%  5%  8%  8%  9%  7% 11%
x264 [info]: i8c dc,h,v,p: 48% 26% 18%  7%
x264 [info]: ref P L0: 49.6% 28.6% 14.2%  4.7%  1.5%  1.4%  0.0%  0.0%
x264 [info]: ref B L0: 70.6% 23.8%  3.3%  1.9%  0.2%  0.2%
x264 [info]: ref B L1: 78.2% 20.3%  0.8%  0.7%
x264 [info]: kb/s:2439.41

encoded 989 frames, 1.42 fps, 2439.41 kb/s
Currently for best results it is recommended to use 2.7.45 and 2.7.45-fix1 in 4:4:4 mode only.

Last edited by DTL; 8th March 2023 at 00:00.
DTL is offline   Reply With Quote
Old 8th March 2023, 11:32   #805  |  Link
DTL
Registered User
 
Join Date: Jul 2018
Posts: 1,041
Updated fix02 - https://github.com/DTL2020/mvtools/r...r.2.7.45-fix02
Found some error in MAnalyse for 4:2:2 and added chroma shift fix to MCompensate to use in QTGMC and other MC-denoise scripts based on MCompensate.
DTL is offline   Reply With Quote
Old 8th March 2023, 23:08   #806  |  Link
zorr
Registered User
 
Join Date: Mar 2018
Posts: 447
Thanks DTL, your development of new MVTools features and fixes is impressive. Interesting that you found a bug with trymany=true as I've found sometimes trymany=false has the better quality even when it clearly shouldn't.

I haven't tested your version so far but might have some time to do some Zopti investigations. I'm mostly interested in the potential quality improvements of MAnalyse as I have a current use case for it and MCompensate.

Can you provide a list of the new parameters and perhaps a short explanation? That would help constructing appropriate Zopti scripts and make sure I don't miss anything relevant.
zorr is offline   Reply With Quote
Old 8th March 2023, 23:13   #807  |  Link
kedautinh12
Registered User
 
Join Date: Jan 2018
Posts: 2,153
I still don't use 2.7.46 ver of DTL cause many change in parameters will affect different result in many scripts base mvtools dependencies
kedautinh12 is offline   Reply With Quote
Old 8th March 2023, 23:46   #808  |  Link
DTL
Registered User
 
Join Date: Jul 2018
Posts: 1,041
"Interesting that you found a bug with trymany=true"

In my builds of post-2.7.45 in may be my own missing memory init when I add performance optimizations to MAnalyse (skipping already checked predictors positions). Somewhere around end of 2021. I not use trymany=true in that years so not found that bug. In versions 2.7.45 and before all predictors always checked so no need to fill the results vector for each predictor with some 'verybig cost' value because it will be overwritten. After optimization of skipping refining of already checked predictor - some memory were not updated and may keep better cost and result in not correct MV output.

" I'm mostly interested in the potential quality improvements of MAnalyse as I have a current use case for it and MCompensate."

Better check that approach - https://forum.doom9.org/showthread.p...40#post1984140 multi-generation MVs refining with intermediate denoising by MDegrainN. It gives most of MVs quality boost even at 2nd generation. And it also have main (and secondary) options to zopti-optimizing.

Main params of multi-generation MVs refining are thSAD of first and next MDegrainN. Secondary - may be MAnalyse for 1st and next generations. Currently I think we have too big step in quality after 1st generation and very few in 3 and next. May be better to have more slow quality gain at each generations and it may help to keep more details. So may be the thSAD for 1st MDegrainN may be reduced and the total thSAD(n) vector is subject to check for best quality.
Possible examples of thSAD(generation number) vectors:
1. Current large improvement after 1st generation (400, 200, 200, 200, 200) (result of gen2 is about good so may be used for better performance with stopping iterations after gen2)
2. Possible better quality (250, 240, 230, 220, 200) (result of gen2 lower in quality but gen5 may be better - may be used for better quality with lower performance)

"Can you provide a list of the new parameters and perhaps a short explanation?"

There were update of documentation: https://github.com/DTL2020/mvtools/b.../mvtools2.html . Though still not all new params are listed - mostly about MPB-processing in MDegrainN (still in the very development and some may be removed in future). Also some features support only very few number of block size and bitdepth (other either slow in performance or even crash with error). Mostly checked is my typical used block size 8x8 and 8bit. I will try to add most of new params to March of 2023 to that documentation file soon.

Also there is significant difference already between 'very old 2.7.45' version and current post-2.7.45 builds in features list.

" That would help constructing appropriate Zopti scripts"

That may be complex task even for zopti-robot. Total number of params of MAnalyse+MDegrainN may be >50 now and quality-adjusting on per-scene basis may be >20.
If look into Interface.cpp file https://github.com/DTL2020/mvtools/b.../Interface.cpp
MAnalyse args[40]
c[blksize]i[blksizeV]i[levels]i[search]i[searchparam]i[pelsearch]i[isb]b[lambda]i[chroma]b[delta]i[truemotion]b[lsad]i[plevel]i[global]b[pnew]i[pzero]i[pglobal]i[overlap]i[overlapV]i
[outfile]s[dct]i[divide]i[sadx264]i[badSAD]i[badrange]i[isse]b[meander]b[temporal]b[trymany]b[multi]b[mt]b[scaleCSAD]i[optsearchoption]i[optpredictortype]i[scaleCSADfine]f
[accnum]i[UseSubShift]i[SuperCurrent]c[SearchDirMode]i[DMFlags]i

MDegrainN args[43]
ccci[thSAD]i[thSADC]i[plane]i[limit]f[limitC]f[thSCD1]i[thSCD2]i[isse]b[planar]b[lsb]b[thsad2]i[thsadc2]i[mt]b[out16]b[wpow]i[adjSADzeromv]f[adjSADcohmv]f[thCohMV]i
[MVLPFCutoff]f[MVLPFSlope]f[MVLPFGauss]f[thMVLPFCorr]i[adjSADLPFedmv]f[UseSubShift]i[IntOvlp]i[mvmultirs]c[thFWBWmvpos]i
[MPBthSub]i[MPBthAdd]i[MPBNumIt]i[MPB_SPCsub]f[MPB_SPCadd]f[MPB_PartBlend]b[MPBthIVS]i[showIVSmask]b[mvmultivs]c[MPB_DMFlags]i[MPBchroma]i[MPBtgtTR]i[MPB_MVlth]i

+MSuper
c[hpad]i[vpad]i[pel]i[levels]i[chroma]b[sharp]i[rfilter]i[pelclip]c[isse]b[planar]b[mt]b[pelrefine]b

Also MRecalculate may be useful in some scenarios - args[25].
cc[thsad]i[smooth]i[blksize]i[blksizeV]i[search]i[searchparam]i[lambda]i[chroma]b[truemotion]b[pnew]i[overlap]i[overlapV]i[outfile]s[dct]i[divide]i[sadx264]i[isse]b[meander]b[tr]i[mt]b[scaleCSAD]i[optsearchoption]i[optpredictortype]i[DMFlags]i
total is over 100. Thats huge. I understand why people like AI Neural Nework 'auto-magic' tools like RIFE with close to no one adjusting param.

Because MDegrainN is the only and main blending engine in latest 'multi-generation' MVs refining approach - all params of MDegrainN used for the intermediate super clips processing may affect the quality of final output MAnalyse MVs in a sequence.

" I have a current use case for it and MCompensate."

If your consumer filter is MCompensate with 2frames mvclip - you can change last MAnalyse in a multi-generation chain to 2-frames mode (multi=false and set isb and delta params).

Here is drawing how is macro-function of MultiGenerations MG-MAnalyse is designed (for 3 generations of MVs refining):

Standard 2.7.45 mvtools MDegrainN output of single pass MAnalyse is 'g1'. Also MAnalyse of 2.7.45 is single input only for single super-clip. With post-2.7.45 builds 2-input MAnalyse used (Ref and Current super-clips inputs).
Each last output of MDegrainN may be used as degrained output or each last output of MAnalyse may be used as mv-clip for mvtools consumer filters.
MV3 out with SuperRef and SrcCurrent may be used with next MDegrainN for g3-degrained output and so on.

I think of creation of schematic drawing of internal MDegrainN design with internal processing and where different params are going. May be sometime later and in some CAD software (it will be large enough drawing).

The 'elementary building block' of in theory infinite generations MVs refinement is 'MDegrain+Super+MAnalyse'. It have inputs of mvclip, super_ref and src. Its output is refined mvclip in current generation.
So it can be designed in function for single step:
Code:
Function RefineMV(clip mvclip, clip super_ref, clip src)
{
g_next=MDegrainN(src, super_ref, mvclip, tr, thSAD=my_thSAD, thSAD2=my_thSAD2, mt=false, wpow=4, thSCD1=my_thSCD, \
adjSADzeromv=0.5, adjSADcohmv=0.5, thCohMV=16, MVLPFGauss=0.9, thMVLPFCorr=50, adjSADLPFedmv=0.9, IntOvlp=3)

super_g_next=MSuper(g_next,chroma=true, mt=false, pel=my_pel)

return=MAnalyse(super_g_next, SuperCurrent=super_ref, multi=true, delta=tr, search=3, searchparam=2, trymany=my_trymany, \
overlap=0, chroma=true, mt=false, optSearchOption=1, truemotion=false, pnew=my_pnew, pzero=my_pzero, pglobal=my_pglobal, \
global=my_global, levels=4, DMFlags=my_DMFlags, optPredictorType=my_oPT)
}

And usage:
my_src=last
s=Super(my_src)
mv1=MAnalyse(s)
mv2=RefineMV(mv1, s, my_src)
mv3=RefineMV(mv2, s, my_src)
mv4=RefineMV(mv3, s, my_src)
and so on
Also one more param to Zopti check may be tr for each generation. May be SuperCurrent may be named simply super2 because it may be no difference if it current or refs frames to search. Still not tested this idea.

Last edited by DTL; 9th March 2023 at 10:39.
DTL is offline   Reply With Quote
Old 10th March 2023, 01:17   #809  |  Link
zorr
Registered User
 
Join Date: Mar 2018
Posts: 447
Quote:
Originally Posted by DTL View Post
" I'm mostly interested in the potential quality improvements of MAnalyse as I have a current use case for it and MCompensate."

Better check that approach - https://forum.doom9.org/showthread.p...40#post1984140 multi-generation MVs refining with intermediate denoising by MDegrainN. It gives most of MVs quality boost even at 2nd generation. And it also have main (and secondary) options to zopti-optimizing.
Will do, thanks. Do you think this way is better than the more traditional prefiltering with another denoiser? Have you done comparisons?

Quote:
Originally Posted by DTL View Post
Also some features support only very few number of block size and bitdepth (other either slow in performance or even crash with error). Mostly checked is my typical used block size 8x8 and 8bit.
My preliminary tests indicate that block size 32 will work much better than 8, at least in the first phase before MRecalculate. Am I out of luck if I want to use that size? Are the limitations also in the CPU implementation or just the HW accelerated version? Also I recall someone mentioned the overlap of MAnalyse is not supported in all cases?

Quote:
Originally Posted by DTL View Post
" That would help constructing appropriate Zopti scripts"

That may be complex task even for zopti-robot. Total number of params of MAnalyse+MDegrainN may be >50 now and quality-adjusting on per-scene basis may be >20.
...
total is over 100. Thats huge. I understand why people like AI Neural Nework 'auto-magic' tools like RIFE with close to no one adjusting param.
Yes, the number of parameters is inhumane. Zopti was created pretty much because I was frustrated with MVTools and wanted an easier way to tune it. So far it has fared well - even with around 50 parameters. It's just a matter of running with a large enough population and iterations. Sometimes I restrict the number of parameters by only optimizing a subset of them and then freezing some parameters and unlocking new ones. Some paramers don't benefit much from trying all possible values, for example you can test LSAD in increments of 100 or even bigger.

Quote:
Originally Posted by DTL View Post
" I have a current use case for it and MCompensate."

If your consumer filter is MCompensate with 2frames mvclip - you can change last MAnalyse in a multi-generation chain to 2-frames mode (multi=false and set isb and delta params).
My use case is a bit special - I have two different videos and I want to match the other one to the first one by moulding it as closely as possible to resemble the first one but still maintaining its own characteristics like colours. The second video has pretty much no noise so I think I would only need to use this multi-generation MAnalyse on the first video and get a clean MSuper clip from it before I interleave it with the second video.

Quote:
Originally Posted by DTL View Post
I think of creation of schematic drawing of internal MDegrainN design with internal processing and where different params are going. May be sometime later and in some CAD software (it will be large enough drawing).
That would certainly be interesting.
zorr is offline   Reply With Quote
Old 10th March 2023, 05:00   #810  |  Link
DTL
Registered User
 
Join Date: Jul 2018
Posts: 1,041
"Do you think this way is better than the more traditional prefiltering with another denoiser? Have you done comparisons?"

The denoiser for each generation may be not only MDegrain if apply previous generation MVs to the next generation denoiser. It is new way of MAnalyse with saving from accumulation of search errors. Still very few comparisons - only with user provided result in https://forum.doom9.org/showthread.p...42#post1984142 . I typically not use prefiltering because not found it useful. So do not have working scripts for compare.

"My preliminary tests indicate that block size 32 will work much better than 8, at least in the first phase before MRecalculate. Am I out of luck if I want to use that size?"

You may try. At least it is not have SIMD optimized dismetric (SSIM, VIF, covariance) and runtime subshifting functions because too large from SSE2 and AVX2 register file so only slow C-reference. But you can use standard SAD and standard MSuper pelrefining. I not test 32 block size.

"or just the HW accelerated version? "

DX12 search also not support 32 blocksize directly. Yon can only simulate it with feeding /4 divided frame size and setting blocksize of 8 and use MScaleVect after MAnalyse to scale resulted MVs to full frame size. Also may try MRecalculate or onCPU MAnalyse for refine.

"overlap of MAnalyse is not supported in all cases?"

Yes - overlap also not supported with DX12 search modes. There currenly interpolated overlap modes in MDegrain or you can simulate overlap (like diagonal 2x) with scripting - make 1/2 blocksize diagonally shifted copy of clip and use second MAnalyse to create second MVs clip and BlockOverlap pluging to combinr 2 MDegrain clips back.

"for example you can test LSAD in increments of 100 or even bigger."

Yes - some params are float point and need either some not very fine quantization to test not infinite number of points or approaching algorithm to iteratively go to best value like dividing quantization step 2 times around currently found best value.

Full working script example with refineMVs functions now in https://forum.doom9.org/showthread.p...76#post1984176

Last edited by DTL; 10th March 2023 at 05:05.
DTL is offline   Reply With Quote
Old 10th March 2023, 13:03   #811  |  Link
TDS
Formally known as .......
 
TDS's Avatar
 
Join Date: Sep 2021
Location: Down Under.
Posts: 965
Quote:
Originally Posted by kedautinh12 View Post
I still don't use 2.7.46 ver of DTL cause many change in parameters will affect different result in many scripts base mvtools dependencies
Yes, I gave it a try today, and it doesn't work for me
__________________
Long term RipBot264 user.

RipBot264 modded builds..
TDS is offline   Reply With Quote
Old 10th March 2023, 16:03   #812  |  Link
DTL
Registered User
 
Join Date: Jul 2018
Posts: 1,041
Unfortunately it may be not compatible with many old scripts (because of still non-fixed bugs with different block sizes and may be something else). So it is no good to 'install to common plugins folder'. Only load from current working directory for current script.

It was expected pinterf will someday port most important and useful features to more official and final 2.7.46 update. Still waiting. At the begining of 2024 it looks we will have great all planet celebration of 20 years to mvtools so may be some more official release after 2.7.45 expected before January 2024. May be even great version change to 2.8.

Current list of ideas to try still not empty: There is an idea to add more predictors to MAnalyse - to fill big gap between near predictors and global motion predictor. Add 'area' predictors of the bestMV of closest blocks from all previous search levels and median of them. So in the 'all' predictors search mode (optPredictorType=0) area predictors check may be added without refine (and with trymany=true - with all refine) and in reduced predictors mode like (optPredictorType=1) only median predictor (with trymany=true - with refine around this predictor).

Addition: New strategic idea for 202x and later: For trymany=true the AVX2/AVX512 for exhaustive (search=3) may process >1 predictors at single SIMD pass and make significant benefit to current much slower trymany=true mode of MAnalyse. It is close to optSearchOption=3 and 4 idea. But for single current source block and different predictors.

Last edited by DTL; 11th March 2023 at 22:27.
DTL is offline   Reply With Quote
Old 13th March 2023, 08:48   #813  |  Link
joka
Registered User
 
Join Date: May 2010
Posts: 28
I want to re-write some scrips using "multi" Variants. Is there no "multi" MSCDetection? Any alternative / Work arround?
joka is offline   Reply With Quote
Old 14th March 2023, 14:43   #814  |  Link
DTL
Registered User
 
Join Date: Jul 2018
Posts: 1,041
Yes - it looks MSCD can only accept single search pair mvclip. It is really only call to FakePlaneOfBlocks::IsSceneChange() via some steps. multi is a special mode for MAnalyse and I still do not have an ideas if it possible somehow easy to convert multi-mv-clip to pair-mv-clip. Though may be some playing with frame-rate/frame number may help.

In multi mode the MAnalyse select the pair to search and output using special conversion of requested frame number in the required pair:
Code:
  const int		ndiv = (_multi_flag) ? _delta_max * 2 : 1;
  const int		nsrc = n / ndiv;
  const int		srd_index = n % ndiv;
If multi enabled - ndiv=_delta_max*2, if disabled (standard pair) - it is 1.

So to request required pair of frame mvs from MAnalyse in multi-mode it required to calculate correct frame number to request and it will output the correct mv frame. But in AVS scripting it may be not very easy (I still not have ideas how to do it - may be play with FrameRate and so on).

Also: MDegrainN still not dead. Completely new processing mode is in development. It is finally motion to the object-oriented moving pictures reconstruction that was expected from AI/neural networks but still no working tools released.

Algorithm (for each block):
1. For the current input pool of blocks in tr-scope - find the most equally (the same) looking block and save it to local memory (with some surrounding area to have ability to perform motion compensation in limited MVs length). Standard thSAD threshold used to separate mostly likely equal blocks from significantly changed. Possible solution - simply save total frame where the 'best' block is found. But it will require very many RAM for storage unknown number of frames (max is total number of blocks in frame to have ability to compensate any MV length, but it looks very rarely required).

2. Analyse current tr-pool of blocks to found best in current pool (+-tr from current frame).

3. Compare best found block in 2. with stored in 1. for visual difference.

3.1. If visual difference (using available dis-metrics - SAD/SSIM/VIF or partial combinations) is below threshold - replace current frame block with stored block (make 100% curing operation).
3.2. If visual difference is above threshold (scenechange, object's view too much change) - replace stored block with current (or best looking in the pool) and output current (or best looked) block.

So for enough denoised clip with first MDegrainN pass it will convert slow motion and slow changed blocks in completely non-changed (with still ability to move around more or less far) and will significantly help to downstream MPEG encoder to mark these blocks as 100% motion compensated and 'skipped'. See skip %-stats in x264 output for example.

So it is threshold-based non-linear processing for complete cut-out of the residual temporal noise over a several frames in a sequence (up to total cut-scene duration).

Something like this already used in modern MPEG encoders (like 265 and may be others) but current implementation is to be used with all possible MPEG encoders like a pre-processing (from MPEG-1 and later). So it is designed to make widely used xvid/x.264 MPEG-4ASP/AVC releases better.

Last edited by DTL; 14th March 2023 at 15:30.
DTL is offline   Reply With Quote
Old 14th March 2023, 16:41   #815  |  Link
anton_foy
Registered User
 
Join Date: Dec 2005
Location: Sweden
Posts: 702
Quote:
Also: MDegrainN still not dead. Completely new processing mode is in development. It is finally motion to the object-oriented moving pictures reconstruction that was expected from AI/neural networks but still no working tools released.
Have you watched Vcmohan's neuralnet and other plugins? He told me that they could help finding temporal differences too. PM him maybe?

Edit: I think now it is called "neural" in "manyPlus" package.

Last edited by anton_foy; 14th March 2023 at 16:46.
anton_foy is offline   Reply With Quote
Old 14th March 2023, 19:27   #816  |  Link
DTL
Registered User
 
Join Date: Jul 2018
Posts: 1,041
" they could help finding temporal differences "

The task for AI robot denoiser is really very very simple: Using input noised frames sequence _understand_ the scene content (objects, textures, transformations, lighting), create best possible clean textures for all scene objects and simply create scene from scratch using textures and lighting and objects transformations without adding inter-frame noise.

So the main task of denoising is not looking for input noise - it is about looking for clean objects textures (real clean object's views) and simply creating output frames using restored clean textures.

The described above algorithm is some step to 'ideal AI denosier'. It have all elements of this processing: Method to create as clean and as true as possible texture of a block and to create new output frame sequence using only restored textures (if possible without significant distortions).

The main issue of 'linear' MDegrainN processing (it is sort of FIR filter now) - it can not reach non-changing output frame sequence even with large enough and slow in processing tr-value and big thSAD value because at each output frame the total tr-pool of frames advances +1 frame in the input frame sequence so it lost 1 last frame and add 1 next frame and even ideal average blending process will typically create slightly different block result even for ideally motion compensated blocks.
So to create some temporal-thresholding (non-linearity at small enough temporal differences) it is require addition of internal memory for the previous frame output block and some estimatie and resolve algorithm about keep output old block or select new. It looks like IIR filter with possibility to infinite output single and completely same block if some conditions met.

Addition:

First idea about noise measurement: The typical PSNR-type is not work well with very low noise levels still critical to MPEG encoders with high and very high compression ratios (about 1000:1).

So suggested temporal noise measurement in noise-bitrate. In noise bits per second.
Calculation:
Nnps x Ns x FPS
where:
Nnps - number of noise bits per sample
Ns - number of samples in frame (width x height)
FPS - number of frames per second.

So for the 1noise bit per each sample (very close to quantization noise) giving PSNR high above 40 dB the nosie-bitrate for FullHD is still about 50 Mbit/s and MPEG encoder can not go significantly lower in output bitrate without significant distortions. It is also remarkable this noise-bitrate is not depend on bitdepth (so equal in relative amplitude 10bit noise bitrate to 8bit is about 4 times larger). Single frame sequence like Trim(1,1).Loop() have a temporal noise bitrate of true zero.

And 1noise bit per 8bit image already look as 'visually noiseless'. But for MPEG encoder it is still very loaded with bad noise. So much higher denoise ratios required to finally put noise-bitrate good below target MPEG output bitrate (with good enough quality CRF-encoding). So much denoise ratios looks like impossible to reach with 'linear' denoise modes of MDegrainN.

Now made an encoding test: For static camera scene from documentary title with sitting and talking person taking about 10..20% of total frame samples in 1920x1080i 25fps, x264 crf=18 encoding:

After 'visually noiseless' MDegrainN denoising -
2427 Kbit/s,
mb P skip: 40.9%
mb B skip: 76.2%

After creating 1bit BW mask in MS Paint masking talking person with white and all other (about 80% frame) in black and simple scripting:
Code:
mask_clip=ImageReader("mask.bmp")
bckgr=Trim(1,1)
bckgr=Loop(bckgr,1000)

Overlay(bckgr, last, mask=mask_clip)
1383 Kbit/s
mb P skip: 85.2%
mp B skip: 93.2%


So with completely static background the number of skipped blocks in MPEG encoder is significantly higher and output bitrate is about 75% lower.

Attempt to apply internal noise-reduction of x264 --nr=600:
1647 K
mb P skip: 37.1%
mb B skip: 77.5%

So even high enough noise-reduction setting for x264 encode can not reach such high skip ratio and so low output bitrate as possible with total block-duplication for non-changed blocks of static parts of the scene.

", the number of parameters is inhumane. "

It is still not very inhumane - it is only some progress from simple denoise application to mv-search engine to more advanced tool. There possible to design control console like that for colour grading pro-work to make realtime adjustments easier (for example some Davinchi products have dedicated hardware control consoles). It is like

Last edited by DTL; 16th March 2023 at 10:04.
DTL is offline   Reply With Quote
Old 8th April 2023, 07:17   #817  |  Link
flossy_cake
Registered User
 
Join Date: Aug 2016
Posts: 605
Is it possible to have MVTools output a single floating point value somewhere that defines the overall amount of motion in the scene? I want to use this value in a ScriptClip.

Currently I am using YDiff and/or weighted YUVDiff (like what RT_FrameDifference uses) but I'm finding at times it's not very accurate because of what's in the frame - high APL panning tends to generate higher values, low APL lower values, plus some randomness of the image contents of the frame, eg. if new bright or dark or differently shaped objects are on screen then that could result in a much higher/lower value despite the motion speed being the same.

Thanks
flossy_cake is offline   Reply With Quote
Old 8th April 2023, 12:02   #818  |  Link
DTL
Registered User
 
Join Date: Jul 2018
Posts: 1,041
There is some hacky way - convert mvclip to 'frame-clip' (MStoreVect()) and pick dx,dy MVs values for analysis using samples-picking in scripting (also SAD or other dismetric is available). Or some filter redesign required to calculate required value and put to some script-readable form.

It is also way to detect dupe frames - if MStoreVect (cropped-out first line with header data) return 0% zero black frame - it mean all dx=dy=SAD=0 (frames equal).

Last edited by DTL; 8th April 2023 at 12:07.
DTL is offline   Reply With Quote
Old 23rd April 2023, 20:35   #819  |  Link
flossy_cake
Registered User
 
Join Date: Aug 2016
Posts: 605
Quote:
Originally Posted by DTL View Post
There is some hacky way - convert mvclip to 'frame-clip' (MStoreVect()) and pick dx,dy MVs values for analysis using samples-picking in scripting (also SAD or other dismetric is available). Or some filter redesign required to calculate required value and put to some script-readable form.

It is also way to detect dupe frames - if MStoreVect (cropped-out first line with header data) return 0% zero black frame - it mean all dx=dy=SAD=0 (frames equal).
Thanks, but I couldn't figure it out. Seems to require 2 passes?

For now I'm using CFrameDiff which is ok - more consistent than YUVDiff in that it still detects motion even when APL is the same. Because of this it's not as good for detecting scenechanges - YUVDiff seems better for that imo. But for motion CFrameDiff is probably the best cheap, fast solution. Some useful thresholds seem to be 0-10=very low motion, 10-25=low motion, 25-45=normal motion, >45=high motion. It also has a fast denoising option in case noise somehow interferes with the metrics.

Last edited by flossy_cake; 23rd April 2023 at 20:49.
flossy_cake is offline   Reply With Quote
Old 24th April 2023, 05:50   #820  |  Link
DTL
Registered User
 
Join Date: Jul 2018
Posts: 1,041
" require 2 passes?"

No. MAnalyse return MVs data for current stream processed. So you can request any 2 frames from current clip MV analysis at any current frame in clip processing (typically current frame and some other frame, but it may be possible to request any 2 frames from clip - not only current + other).

If you need scenechange detection - it may be not good idea with mvtools. It sort of not search MV data over scenechange at all (it try but result is very bad). The scenechange detection in mvtools is poor enough. It simply look on 'quality of MVs' data as best dismetric over found blocks and if it is very bad over most of frame - it looks like attempt to search for motion over scenechange point in time axis. So may be no great difference in mean MV with static frames of single scene or many random very bad MVs over scenechange. Also mvtools have already implemented very simple filter like MSCDetect that simply output bool flag from internal scenechange detector used by may be all other internal mvtools filters based on SAD/dismetric analysis as described (using 'global' thSCD1 and thSCD2 values).

Also if you like to implement different algorithm for scenechange detection in scripting - you can also grab SAD/dismetric for each block form pseudo-frame clip output by MStoreVect and try to make your own math analysis.

Also for scenechange detection you can try to use any full-ref image quality analysis tools with 2-frames input (like PSNR/SAD/SSIM/VIF/many_other). If the sim/dissim metric over checked pair of frames is very bad - so it looks like frames from different scenes.

If you try to detect scenechange from MV data it may be same task as currently attempt to implement in MPB detector of 'stable MV data' over time axis. But it is much more complex in compare with simple mean value. If MV data is very bad it mean MVs have and random lengths and random directions (angles). So the algorithm for detection if MVs data is very bad and unnatural for typical good intrascene search need to separate many co-directed MVs in areas from mostly random directed MVs in very noisy and low detailed areas or scenechanges. It sort of measuring noise-level in dx/dy data over total set of MVs from single frame pair search. May be even making Fourier domain transform and analyse the shape of Fourier spectrum - for random white nosie it sort of equal-energy over all valid frequencies and for natural motion it may be mostly low-frequencies energy located. But also total length of MVs must be analysed (for single scene and simple dark frame with noise the MVs typically random too). So advanced scenechange detector may be very complex and math and computing loaded and slow.

I also not understand what you need to measure in motion. The mean MV is sort of global MV vector and it close to zero length for different cases (like random noised static scene and also static clean scene). May be you need mean of MVs length in the current pairs of frames analysed ? It may show some measure of motion in a frame, but for static scene with noise it will also correlate with simple noise level if noise is comparable or more of the real scene objects contrast. So for better motion measure the MVs data may be need to be processed with sort of area-based low-pass filtering to found noise-cleaned areas mean MVs at first.

Last edited by DTL; 24th April 2023 at 06:18.
DTL is offline   Reply With Quote
Reply

Thread Tools Search this Thread
Search this Thread:

Advanced Search
Display Modes

Posting Rules
You may not post new threads
You may not post replies
You may not post attachments
You may not edit your posts

BB code is On
Smilies are On
[IMG] code is On
HTML code is Off

Forum Jump


All times are GMT +1. The time now is 01:11.


Powered by vBulletin® Version 3.8.11
Copyright ©2000 - 2024, vBulletin Solutions Inc.