Welcome to Doom9's Forum, THE in-place to be for everyone interested in DVD conversion. Before you start posting please read the forum rules. By posting to this forum you agree to abide by the rules. |
8th July 2022, 12:53 | #802 | Link |
21 years and counting...
Join Date: Oct 2002
Location: Germany
Posts: 716
|
I just checked most of my scripts and they are running with mvtools 2.7.44
I have no idea why I haven't checked this more thoroughly. I already had pinterf's 2.7.45 in my library, no idea why I didn't use it. DTL you wrote 2.7.45+ so I guess there is already never version to use? |
8th July 2022, 15:25 | #803 | Link |
Registered User
Join Date: Jul 2018
Posts: 1,041
|
Still no next official release like 2.7.46 from pinterf. With 2.7.45+ I mean post-2.7.45 builds I typically put at https://github.com/DTL2020/mvtools/releases .
I hope someday may be in this or next year it will be more or less merged to pinterf branch and after supplementing and testing finally we will see 2.7.46 or may be even 2.8 release. The number of strategic new features and processing modes continue to increase after first fork in 2021. I think to finish adding most of new features after interpolated overlap I plan to start at this week. But after reading mvtools docs one more time at this year I found MFlow with per-sample processing precision mode and got new idea to test it with MDegrainN frame-operation pipeline (with easy to use multi-mode). So it may be next more or less slow development feature to the MDegrainN. Also all new features added and approved need to supplement with SIMD processing functions for better speed for most or all supported colour formats and at least some commonly used block sizes and bitdepths. Currently only some formats have at least AVX2 execution in addition to very slow C-reference like only YV12 and only 8x8 block size of 8bit for UseSubShift mode of MVPlane (fully working only in MDegrainN now and planned for MAnalyse and other filters to save RAM and/or better speed at some use cases). Typically that builds are full featured of this and https://forum.doom9.org/showthread.php?t=183517 thread described features but most of latest builds are with DX12-ME support and will not load where no DX12 installed (so no Win7 and old). So it is required separate build config to build all-win compatible executable without using DX12-ME mode of MAnalyse but using all new features of MDegrainN for example. The overlap single pass MDegrainN for colour formats are also ready to use after 06.07.22 commits but I still not make release build - you can only build yourself from latest git sources image. I plan to make new release after adding the also very strategic interpolated overlap feature to finally get solution against blockiness at some scenes with no-overlap supported hardware accelerated MAnalyse now. Also the shader need to be fixed after final runtime subshift of block processing implementation based on old mvtools Wiener kernel - it is delayed because 12.05.22 release have not best but about good working SAD compute shader and fast rebuild of shader still not show better results and need more debug. Some new features like single pass colour MDegrainN or MVLPF interfiltering are not depend on colour format and block size so expected to work already with all old formats. Single pass colour proc also do not require new options so compatible with old scripts directly (expected a bit better speed with same quality). mvtools will soon be 1/5 of century aged - in January of 2024. So may we will have version 2.8.0 prepared to that celebration. Or may be final release of 2.7.46 . Last edited by DTL; 24th July 2022 at 08:03. |
7th March 2023, 20:59 | #804 | Link |
Registered User
Join Date: Jul 2018
Posts: 1,041
|
Version for testing of (partial) fix for issue with chroma subsampled formats (both MAnalyse and MDegrainX/N) - https://github.com/DTL2020/mvtools/r.../r.2.7.45-fix1 . The description in https://github.com/pinterf/mvtools/issues/59
One of my test with x264 encoding shows a bit lower output MPEG bitrate and a bit better details at 200% size comparison. Code:
denoise script: for YV12 source tr=8 super=MSuper(last,chroma=true, mt=false, pel=2) multi_vec=MAnalyse (super, multi=true, delta=tr, search=3, searchparam=2, overlap=4, chroma=true, mt=false, truemotion=false, pnew=0, pzero=0, levels=0) MDegrainN(last,super, multi_vec, tr, thSAD=300, thSAD2=290, mt=false, thSCD1=500) orig 2.7.45= x264.exe --profile high --crf 18 --tff --ref 4 -b 10 --me umh --subme 9 --merange 16 --deblock -3:-3 --b-adapt 2 --psy-rd 1.0:0.7 --transfer "bt70 9" --colorprim "bt709" --colormatrix "bt709" -o out.264 src_t.avs avs [info]: 960x540i 0:0 @ 25/1 fps (cfr) x264 [warning]: interlace + weightp is not implemented x264 [info]: using cpu capabilities: MMX2 SSE2Fast SSSE3 SSE4.1 Cache64 x264 [info]: profile High, level 3.1, 4:2:0, 8-bit x264 [info]: frame I:5 Avg QP:14.23 size:102544 x264 [info]: frame P:203 Avg QP:17.24 size: 31718 x264 [info]: frame B:781 Avg QP:20.50 size: 6645 x264 [info]: consecutive B-frames: 1.7% 3.2% 9.1% 11.3% 8.6% 49.1% 4.2% 2.4% 2.7% 3.0% 4.4% x264 [info]: mb I I16..4: 14.7% 66.3% 19.0% x264 [info]: mb P I16..4: 1.8% 4.7% 1.0% P16..4: 42.6% 23.4% 19.0% 0.0% 0.0% skip: 7.4% x264 [info]: mb B I16..4: 0.1% 0.1% 0.0% B16..8: 31.0% 7.6% 2.1% direct: 7.9% skip:51.3% L0:35.5% L1:32.5% BI:32.0% x264 [info]: field mbs: intra: 36.6% inter:41.6% skip:31.1% x264 [info]: 8x8 transform intra:61.4% inter:72.9% x264 [info]: coded y,uvDC,uvAC intra: 70.9% 82.6% 65.8% inter: 29.9% 25.5% 7.7% x264 [info]: i16 v,h,dc,p: 29% 19% 15% 36% x264 [info]: i8 v,h,dc,ddl,ddr,vr,hd,vl,hu: 14% 12% 14% 7% 9% 10% 11% 9% 13% x264 [info]: i4 v,h,dc,ddl,ddr,vr,hd,vl,hu: 13% 33% 7% 5% 8% 8% 9% 7% 11% x264 [info]: i8c dc,h,v,p: 46% 27% 19% 8% x264 [info]: ref P L0: 49.9% 28.4% 14.0% 4.8% 1.4% 1.4% 0.0% 0.0% x264 [info]: ref B L0: 70.4% 23.7% 3.4% 1.9% 0.2% 0.2% x264 [info]: ref B L1: 78.0% 20.4% 0.9% 0.8% x264 [info]: kb/s:2455.20 encoded 989 frames, 1.42 fps, 2455.20 kb/s 2.7.45 fix1= x264.exe --profile high --crf 18 --tff --ref 4 -b 10 --me umh --subme 9 --merange 16 --deblock -3:-3 --b-adapt 2 --psy-rd 1.0:0.7 --transfer "bt70 9" --colorprim "bt709" --colormatrix "bt709" -o out.264 src_t.avs avs [info]: 960x540i 0:0 @ 25/1 fps (cfr) x264 [warning]: interlace + weightp is not implemented x264 [info]: using cpu capabilities: MMX2 SSE2Fast SSSE3 SSE4.1 Cache64 x264 [info]: profile High, level 3.1, 4:2:0, 8-bit x264 [info]: frame I:5 Avg QP:14.14 size:102566 x264 [info]: frame P:202 Avg QP:17.24 size: 31792 x264 [info]: frame B:782 Avg QP:20.51 size: 6558 x264 [info]: consecutive B-frames: 1.4% 3.4% 9.7% 8.5% 10.6% 51.6% 4.2% 3.2% 1.8% 1.0% 4.4% x264 [info]: mb I I16..4: 14.8% 65.9% 19.3% x264 [info]: mb P I16..4: 1.8% 4.6% 1.1% P16..4: 42.6% 23.5% 19.1% 0.0% 0.0% skip: 7.3% x264 [info]: mb B I16..4: 0.1% 0.1% 0.0% B16..8: 30.9% 7.5% 2.1% direct: 7.7% skip:51.6% L0:35.5% L1:32.3% BI:32.1% x264 [info]: field mbs: intra: 36.0% inter:41.4% skip:31.1% x264 [info]: 8x8 transform intra:61.4% inter:72.9% x264 [info]: coded y,uvDC,uvAC intra: 71.4% 82.9% 66.0% inter: 29.7% 25.1% 7.5% x264 [info]: i16 v,h,dc,p: 30% 20% 15% 35% x264 [info]: i8 v,h,dc,ddl,ddr,vr,hd,vl,hu: 14% 12% 14% 7% 10% 10% 11% 9% 13% x264 [info]: i4 v,h,dc,ddl,ddr,vr,hd,vl,hu: 13% 32% 7% 5% 8% 8% 9% 7% 11% x264 [info]: i8c dc,h,v,p: 48% 26% 18% 7% x264 [info]: ref P L0: 49.6% 28.6% 14.2% 4.7% 1.5% 1.4% 0.0% 0.0% x264 [info]: ref B L0: 70.6% 23.8% 3.3% 1.9% 0.2% 0.2% x264 [info]: ref B L1: 78.2% 20.3% 0.8% 0.7% x264 [info]: kb/s:2439.41 encoded 989 frames, 1.42 fps, 2439.41 kb/s Last edited by DTL; 8th March 2023 at 00:00. |
8th March 2023, 11:32 | #805 | Link |
Registered User
Join Date: Jul 2018
Posts: 1,041
|
Updated fix02 - https://github.com/DTL2020/mvtools/r...r.2.7.45-fix02
Found some error in MAnalyse for 4:2:2 and added chroma shift fix to MCompensate to use in QTGMC and other MC-denoise scripts based on MCompensate. |
8th March 2023, 23:08 | #806 | Link |
Registered User
Join Date: Mar 2018
Posts: 447
|
Thanks DTL, your development of new MVTools features and fixes is impressive. Interesting that you found a bug with trymany=true as I've found sometimes trymany=false has the better quality even when it clearly shouldn't.
I haven't tested your version so far but might have some time to do some Zopti investigations. I'm mostly interested in the potential quality improvements of MAnalyse as I have a current use case for it and MCompensate. Can you provide a list of the new parameters and perhaps a short explanation? That would help constructing appropriate Zopti scripts and make sure I don't miss anything relevant. |
8th March 2023, 23:46 | #808 | Link |
Registered User
Join Date: Jul 2018
Posts: 1,041
|
"Interesting that you found a bug with trymany=true"
In my builds of post-2.7.45 in may be my own missing memory init when I add performance optimizations to MAnalyse (skipping already checked predictors positions). Somewhere around end of 2021. I not use trymany=true in that years so not found that bug. In versions 2.7.45 and before all predictors always checked so no need to fill the results vector for each predictor with some 'verybig cost' value because it will be overwritten. After optimization of skipping refining of already checked predictor - some memory were not updated and may keep better cost and result in not correct MV output. " I'm mostly interested in the potential quality improvements of MAnalyse as I have a current use case for it and MCompensate." Better check that approach - https://forum.doom9.org/showthread.p...40#post1984140 multi-generation MVs refining with intermediate denoising by MDegrainN. It gives most of MVs quality boost even at 2nd generation. And it also have main (and secondary) options to zopti-optimizing. Main params of multi-generation MVs refining are thSAD of first and next MDegrainN. Secondary - may be MAnalyse for 1st and next generations. Currently I think we have too big step in quality after 1st generation and very few in 3 and next. May be better to have more slow quality gain at each generations and it may help to keep more details. So may be the thSAD for 1st MDegrainN may be reduced and the total thSAD(n) vector is subject to check for best quality. Possible examples of thSAD(generation number) vectors: 1. Current large improvement after 1st generation (400, 200, 200, 200, 200) (result of gen2 is about good so may be used for better performance with stopping iterations after gen2) 2. Possible better quality (250, 240, 230, 220, 200) (result of gen2 lower in quality but gen5 may be better - may be used for better quality with lower performance) "Can you provide a list of the new parameters and perhaps a short explanation?" There were update of documentation: https://github.com/DTL2020/mvtools/b.../mvtools2.html . Though still not all new params are listed - mostly about MPB-processing in MDegrainN (still in the very development and some may be removed in future). Also some features support only very few number of block size and bitdepth (other either slow in performance or even crash with error). Mostly checked is my typical used block size 8x8 and 8bit. I will try to add most of new params to March of 2023 to that documentation file soon. Also there is significant difference already between 'very old 2.7.45' version and current post-2.7.45 builds in features list. " That would help constructing appropriate Zopti scripts" That may be complex task even for zopti-robot. Total number of params of MAnalyse+MDegrainN may be >50 now and quality-adjusting on per-scene basis may be >20. If look into Interface.cpp file https://github.com/DTL2020/mvtools/b.../Interface.cpp MAnalyse args[40] c[blksize]i[blksizeV]i[levels]i[search]i[searchparam]i[pelsearch]i[isb]b[lambda]i[chroma]b[delta]i[truemotion]b[lsad]i[plevel]i[global]b[pnew]i[pzero]i[pglobal]i[overlap]i[overlapV]i [outfile]s[dct]i[divide]i[sadx264]i[badSAD]i[badrange]i[isse]b[meander]b[temporal]b[trymany]b[multi]b[mt]b[scaleCSAD]i[optsearchoption]i[optpredictortype]i[scaleCSADfine]f [accnum]i[UseSubShift]i[SuperCurrent]c[SearchDirMode]i[DMFlags]i MDegrainN args[43] ccci[thSAD]i[thSADC]i[plane]i[limit]f[limitC]f[thSCD1]i[thSCD2]i[isse]b[planar]b[lsb]b[thsad2]i[thsadc2]i[mt]b[out16]b[wpow]i[adjSADzeromv]f[adjSADcohmv]f[thCohMV]i [MVLPFCutoff]f[MVLPFSlope]f[MVLPFGauss]f[thMVLPFCorr]i[adjSADLPFedmv]f[UseSubShift]i[IntOvlp]i[mvmultirs]c[thFWBWmvpos]i [MPBthSub]i[MPBthAdd]i[MPBNumIt]i[MPB_SPCsub]f[MPB_SPCadd]f[MPB_PartBlend]b[MPBthIVS]i[showIVSmask]b[mvmultivs]c[MPB_DMFlags]i[MPBchroma]i[MPBtgtTR]i[MPB_MVlth]i +MSuper c[hpad]i[vpad]i[pel]i[levels]i[chroma]b[sharp]i[rfilter]i[pelclip]c[isse]b[planar]b[mt]b[pelrefine]b Also MRecalculate may be useful in some scenarios - args[25]. cc[thsad]i[smooth]i[blksize]i[blksizeV]i[search]i[searchparam]i[lambda]i[chroma]b[truemotion]b[pnew]i[overlap]i[overlapV]i[outfile]s[dct]i[divide]i[sadx264]i[isse]b[meander]b[tr]i[mt]b[scaleCSAD]i[optsearchoption]i[optpredictortype]i[DMFlags]i total is over 100. Thats huge. I understand why people like AI Neural Nework 'auto-magic' tools like RIFE with close to no one adjusting param. Because MDegrainN is the only and main blending engine in latest 'multi-generation' MVs refining approach - all params of MDegrainN used for the intermediate super clips processing may affect the quality of final output MAnalyse MVs in a sequence. " I have a current use case for it and MCompensate." If your consumer filter is MCompensate with 2frames mvclip - you can change last MAnalyse in a multi-generation chain to 2-frames mode (multi=false and set isb and delta params). Here is drawing how is macro-function of MultiGenerations MG-MAnalyse is designed (for 3 generations of MVs refining): Standard 2.7.45 mvtools MDegrainN output of single pass MAnalyse is 'g1'. Also MAnalyse of 2.7.45 is single input only for single super-clip. With post-2.7.45 builds 2-input MAnalyse used (Ref and Current super-clips inputs). Each last output of MDegrainN may be used as degrained output or each last output of MAnalyse may be used as mv-clip for mvtools consumer filters. MV3 out with SuperRef and SrcCurrent may be used with next MDegrainN for g3-degrained output and so on. I think of creation of schematic drawing of internal MDegrainN design with internal processing and where different params are going. May be sometime later and in some CAD software (it will be large enough drawing). The 'elementary building block' of in theory infinite generations MVs refinement is 'MDegrain+Super+MAnalyse'. It have inputs of mvclip, super_ref and src. Its output is refined mvclip in current generation. So it can be designed in function for single step: Code:
Function RefineMV(clip mvclip, clip super_ref, clip src) { g_next=MDegrainN(src, super_ref, mvclip, tr, thSAD=my_thSAD, thSAD2=my_thSAD2, mt=false, wpow=4, thSCD1=my_thSCD, \ adjSADzeromv=0.5, adjSADcohmv=0.5, thCohMV=16, MVLPFGauss=0.9, thMVLPFCorr=50, adjSADLPFedmv=0.9, IntOvlp=3) super_g_next=MSuper(g_next,chroma=true, mt=false, pel=my_pel) return=MAnalyse(super_g_next, SuperCurrent=super_ref, multi=true, delta=tr, search=3, searchparam=2, trymany=my_trymany, \ overlap=0, chroma=true, mt=false, optSearchOption=1, truemotion=false, pnew=my_pnew, pzero=my_pzero, pglobal=my_pglobal, \ global=my_global, levels=4, DMFlags=my_DMFlags, optPredictorType=my_oPT) } And usage: my_src=last s=Super(my_src) mv1=MAnalyse(s) mv2=RefineMV(mv1, s, my_src) mv3=RefineMV(mv2, s, my_src) mv4=RefineMV(mv3, s, my_src) and so on Last edited by DTL; 9th March 2023 at 10:39. |
10th March 2023, 01:17 | #809 | Link | ||||
Registered User
Join Date: Mar 2018
Posts: 447
|
Quote:
Quote:
Quote:
Quote:
That would certainly be interesting. |
||||
10th March 2023, 05:00 | #810 | Link |
Registered User
Join Date: Jul 2018
Posts: 1,041
|
"Do you think this way is better than the more traditional prefiltering with another denoiser? Have you done comparisons?"
The denoiser for each generation may be not only MDegrain if apply previous generation MVs to the next generation denoiser. It is new way of MAnalyse with saving from accumulation of search errors. Still very few comparisons - only with user provided result in https://forum.doom9.org/showthread.p...42#post1984142 . I typically not use prefiltering because not found it useful. So do not have working scripts for compare. "My preliminary tests indicate that block size 32 will work much better than 8, at least in the first phase before MRecalculate. Am I out of luck if I want to use that size?" You may try. At least it is not have SIMD optimized dismetric (SSIM, VIF, covariance) and runtime subshifting functions because too large from SSE2 and AVX2 register file so only slow C-reference. But you can use standard SAD and standard MSuper pelrefining. I not test 32 block size. "or just the HW accelerated version? " DX12 search also not support 32 blocksize directly. Yon can only simulate it with feeding /4 divided frame size and setting blocksize of 8 and use MScaleVect after MAnalyse to scale resulted MVs to full frame size. Also may try MRecalculate or onCPU MAnalyse for refine. "overlap of MAnalyse is not supported in all cases?" Yes - overlap also not supported with DX12 search modes. There currenly interpolated overlap modes in MDegrain or you can simulate overlap (like diagonal 2x) with scripting - make 1/2 blocksize diagonally shifted copy of clip and use second MAnalyse to create second MVs clip and BlockOverlap pluging to combinr 2 MDegrain clips back. "for example you can test LSAD in increments of 100 or even bigger." Yes - some params are float point and need either some not very fine quantization to test not infinite number of points or approaching algorithm to iteratively go to best value like dividing quantization step 2 times around currently found best value. Full working script example with refineMVs functions now in https://forum.doom9.org/showthread.p...76#post1984176 Last edited by DTL; 10th March 2023 at 05:05. |
10th March 2023, 16:03 | #812 | Link |
Registered User
Join Date: Jul 2018
Posts: 1,041
|
Unfortunately it may be not compatible with many old scripts (because of still non-fixed bugs with different block sizes and may be something else). So it is no good to 'install to common plugins folder'. Only load from current working directory for current script.
It was expected pinterf will someday port most important and useful features to more official and final 2.7.46 update. Still waiting. At the begining of 2024 it looks we will have great all planet celebration of 20 years to mvtools so may be some more official release after 2.7.45 expected before January 2024. May be even great version change to 2.8. Current list of ideas to try still not empty: There is an idea to add more predictors to MAnalyse - to fill big gap between near predictors and global motion predictor. Add 'area' predictors of the bestMV of closest blocks from all previous search levels and median of them. So in the 'all' predictors search mode (optPredictorType=0) area predictors check may be added without refine (and with trymany=true - with all refine) and in reduced predictors mode like (optPredictorType=1) only median predictor (with trymany=true - with refine around this predictor). Addition: New strategic idea for 202x and later: For trymany=true the AVX2/AVX512 for exhaustive (search=3) may process >1 predictors at single SIMD pass and make significant benefit to current much slower trymany=true mode of MAnalyse. It is close to optSearchOption=3 and 4 idea. But for single current source block and different predictors. Last edited by DTL; 11th March 2023 at 22:27. |
14th March 2023, 14:43 | #814 | Link |
Registered User
Join Date: Jul 2018
Posts: 1,041
|
Yes - it looks MSCD can only accept single search pair mvclip. It is really only call to FakePlaneOfBlocks::IsSceneChange() via some steps. multi is a special mode for MAnalyse and I still do not have an ideas if it possible somehow easy to convert multi-mv-clip to pair-mv-clip. Though may be some playing with frame-rate/frame number may help.
In multi mode the MAnalyse select the pair to search and output using special conversion of requested frame number in the required pair: Code:
const int ndiv = (_multi_flag) ? _delta_max * 2 : 1; const int nsrc = n / ndiv; const int srd_index = n % ndiv; So to request required pair of frame mvs from MAnalyse in multi-mode it required to calculate correct frame number to request and it will output the correct mv frame. But in AVS scripting it may be not very easy (I still not have ideas how to do it - may be play with FrameRate and so on). Also: MDegrainN still not dead. Completely new processing mode is in development. It is finally motion to the object-oriented moving pictures reconstruction that was expected from AI/neural networks but still no working tools released. Algorithm (for each block): 1. For the current input pool of blocks in tr-scope - find the most equally (the same) looking block and save it to local memory (with some surrounding area to have ability to perform motion compensation in limited MVs length). Standard thSAD threshold used to separate mostly likely equal blocks from significantly changed. Possible solution - simply save total frame where the 'best' block is found. But it will require very many RAM for storage unknown number of frames (max is total number of blocks in frame to have ability to compensate any MV length, but it looks very rarely required). 2. Analyse current tr-pool of blocks to found best in current pool (+-tr from current frame). 3. Compare best found block in 2. with stored in 1. for visual difference. 3.1. If visual difference (using available dis-metrics - SAD/SSIM/VIF or partial combinations) is below threshold - replace current frame block with stored block (make 100% curing operation). 3.2. If visual difference is above threshold (scenechange, object's view too much change) - replace stored block with current (or best looking in the pool) and output current (or best looked) block. So for enough denoised clip with first MDegrainN pass it will convert slow motion and slow changed blocks in completely non-changed (with still ability to move around more or less far) and will significantly help to downstream MPEG encoder to mark these blocks as 100% motion compensated and 'skipped'. See skip %-stats in x264 output for example. So it is threshold-based non-linear processing for complete cut-out of the residual temporal noise over a several frames in a sequence (up to total cut-scene duration). Something like this already used in modern MPEG encoders (like 265 and may be others) but current implementation is to be used with all possible MPEG encoders like a pre-processing (from MPEG-1 and later). So it is designed to make widely used xvid/x.264 MPEG-4ASP/AVC releases better. Last edited by DTL; 14th March 2023 at 15:30. |
14th March 2023, 16:41 | #815 | Link | |
Registered User
Join Date: Dec 2005
Location: Sweden
Posts: 702
|
Quote:
Edit: I think now it is called "neural" in "manyPlus" package. Last edited by anton_foy; 14th March 2023 at 16:46. |
|
14th March 2023, 19:27 | #816 | Link |
Registered User
Join Date: Jul 2018
Posts: 1,041
|
" they could help finding temporal differences "
The task for AI robot denoiser is really very very simple: Using input noised frames sequence _understand_ the scene content (objects, textures, transformations, lighting), create best possible clean textures for all scene objects and simply create scene from scratch using textures and lighting and objects transformations without adding inter-frame noise. So the main task of denoising is not looking for input noise - it is about looking for clean objects textures (real clean object's views) and simply creating output frames using restored clean textures. The described above algorithm is some step to 'ideal AI denosier'. It have all elements of this processing: Method to create as clean and as true as possible texture of a block and to create new output frame sequence using only restored textures (if possible without significant distortions). The main issue of 'linear' MDegrainN processing (it is sort of FIR filter now) - it can not reach non-changing output frame sequence even with large enough and slow in processing tr-value and big thSAD value because at each output frame the total tr-pool of frames advances +1 frame in the input frame sequence so it lost 1 last frame and add 1 next frame and even ideal average blending process will typically create slightly different block result even for ideally motion compensated blocks. So to create some temporal-thresholding (non-linearity at small enough temporal differences) it is require addition of internal memory for the previous frame output block and some estimatie and resolve algorithm about keep output old block or select new. It looks like IIR filter with possibility to infinite output single and completely same block if some conditions met. Addition: First idea about noise measurement: The typical PSNR-type is not work well with very low noise levels still critical to MPEG encoders with high and very high compression ratios (about 1000:1). So suggested temporal noise measurement in noise-bitrate. In noise bits per second. Calculation: Nnps x Ns x FPS where: Nnps - number of noise bits per sample Ns - number of samples in frame (width x height) FPS - number of frames per second. So for the 1noise bit per each sample (very close to quantization noise) giving PSNR high above 40 dB the nosie-bitrate for FullHD is still about 50 Mbit/s and MPEG encoder can not go significantly lower in output bitrate without significant distortions. It is also remarkable this noise-bitrate is not depend on bitdepth (so equal in relative amplitude 10bit noise bitrate to 8bit is about 4 times larger). Single frame sequence like Trim(1,1).Loop() have a temporal noise bitrate of true zero. And 1noise bit per 8bit image already look as 'visually noiseless'. But for MPEG encoder it is still very loaded with bad noise. So much higher denoise ratios required to finally put noise-bitrate good below target MPEG output bitrate (with good enough quality CRF-encoding). So much denoise ratios looks like impossible to reach with 'linear' denoise modes of MDegrainN. Now made an encoding test: For static camera scene from documentary title with sitting and talking person taking about 10..20% of total frame samples in 1920x1080i 25fps, x264 crf=18 encoding: After 'visually noiseless' MDegrainN denoising - 2427 Kbit/s, mb P skip: 40.9% mb B skip: 76.2% After creating 1bit BW mask in MS Paint masking talking person with white and all other (about 80% frame) in black and simple scripting: Code:
mask_clip=ImageReader("mask.bmp") bckgr=Trim(1,1) bckgr=Loop(bckgr,1000) Overlay(bckgr, last, mask=mask_clip) mb P skip: 85.2% mp B skip: 93.2% So with completely static background the number of skipped blocks in MPEG encoder is significantly higher and output bitrate is about 75% lower. Attempt to apply internal noise-reduction of x264 --nr=600: 1647 K mb P skip: 37.1% mb B skip: 77.5% So even high enough noise-reduction setting for x264 encode can not reach such high skip ratio and so low output bitrate as possible with total block-duplication for non-changed blocks of static parts of the scene. ", the number of parameters is inhumane. " It is still not very inhumane - it is only some progress from simple denoise application to mv-search engine to more advanced tool. There possible to design control console like that for colour grading pro-work to make realtime adjustments easier (for example some Davinchi products have dedicated hardware control consoles). It is like Last edited by DTL; 16th March 2023 at 10:04. |
8th April 2023, 07:17 | #817 | Link |
Registered User
Join Date: Aug 2016
Posts: 605
|
Is it possible to have MVTools output a single floating point value somewhere that defines the overall amount of motion in the scene? I want to use this value in a ScriptClip.
Currently I am using YDiff and/or weighted YUVDiff (like what RT_FrameDifference uses) but I'm finding at times it's not very accurate because of what's in the frame - high APL panning tends to generate higher values, low APL lower values, plus some randomness of the image contents of the frame, eg. if new bright or dark or differently shaped objects are on screen then that could result in a much higher/lower value despite the motion speed being the same. Thanks |
8th April 2023, 12:02 | #818 | Link |
Registered User
Join Date: Jul 2018
Posts: 1,041
|
There is some hacky way - convert mvclip to 'frame-clip' (MStoreVect()) and pick dx,dy MVs values for analysis using samples-picking in scripting (also SAD or other dismetric is available). Or some filter redesign required to calculate required value and put to some script-readable form.
It is also way to detect dupe frames - if MStoreVect (cropped-out first line with header data) return 0% zero black frame - it mean all dx=dy=SAD=0 (frames equal). Last edited by DTL; 8th April 2023 at 12:07. |
23rd April 2023, 20:35 | #819 | Link | |
Registered User
Join Date: Aug 2016
Posts: 605
|
Quote:
For now I'm using CFrameDiff which is ok - more consistent than YUVDiff in that it still detects motion even when APL is the same. Because of this it's not as good for detecting scenechanges - YUVDiff seems better for that imo. But for motion CFrameDiff is probably the best cheap, fast solution. Some useful thresholds seem to be 0-10=very low motion, 10-25=low motion, 25-45=normal motion, >45=high motion. It also has a fast denoising option in case noise somehow interferes with the metrics. Last edited by flossy_cake; 23rd April 2023 at 20:49. |
|
24th April 2023, 05:50 | #820 | Link |
Registered User
Join Date: Jul 2018
Posts: 1,041
|
" require 2 passes?"
No. MAnalyse return MVs data for current stream processed. So you can request any 2 frames from current clip MV analysis at any current frame in clip processing (typically current frame and some other frame, but it may be possible to request any 2 frames from clip - not only current + other). If you need scenechange detection - it may be not good idea with mvtools. It sort of not search MV data over scenechange at all (it try but result is very bad). The scenechange detection in mvtools is poor enough. It simply look on 'quality of MVs' data as best dismetric over found blocks and if it is very bad over most of frame - it looks like attempt to search for motion over scenechange point in time axis. So may be no great difference in mean MV with static frames of single scene or many random very bad MVs over scenechange. Also mvtools have already implemented very simple filter like MSCDetect that simply output bool flag from internal scenechange detector used by may be all other internal mvtools filters based on SAD/dismetric analysis as described (using 'global' thSCD1 and thSCD2 values). Also if you like to implement different algorithm for scenechange detection in scripting - you can also grab SAD/dismetric for each block form pseudo-frame clip output by MStoreVect and try to make your own math analysis. Also for scenechange detection you can try to use any full-ref image quality analysis tools with 2-frames input (like PSNR/SAD/SSIM/VIF/many_other). If the sim/dissim metric over checked pair of frames is very bad - so it looks like frames from different scenes. If you try to detect scenechange from MV data it may be same task as currently attempt to implement in MPB detector of 'stable MV data' over time axis. But it is much more complex in compare with simple mean value. If MV data is very bad it mean MVs have and random lengths and random directions (angles). So the algorithm for detection if MVs data is very bad and unnatural for typical good intrascene search need to separate many co-directed MVs in areas from mostly random directed MVs in very noisy and low detailed areas or scenechanges. It sort of measuring noise-level in dx/dy data over total set of MVs from single frame pair search. May be even making Fourier domain transform and analyse the shape of Fourier spectrum - for random white nosie it sort of equal-energy over all valid frequencies and for natural motion it may be mostly low-frequencies energy located. But also total length of MVs must be analysed (for single scene and simple dark frame with noise the MVs typically random too). So advanced scenechange detector may be very complex and math and computing loaded and slow. I also not understand what you need to measure in motion. The mean MV is sort of global MV vector and it close to zero length for different cases (like random noised static scene and also static clean scene). May be you need mean of MVs length in the current pairs of frames analysed ? It may show some measure of motion in a frame, but for static scene with noise it will also correlate with simple noise level if noise is comparable or more of the real scene objects contrast. So for better motion measure the MVs data may be need to be processed with sort of area-based low-pass filtering to found noise-cleaned areas mean MVs at first. Last edited by DTL; 24th April 2023 at 06:18. |
Thread Tools | Search this Thread |
Display Modes | |
|
|