MVTools

[um3k]

Thanks, again.

[um3k]

Ok, I've run in to a little problem. The filter has a tendency to undercorrect fast motion--and I suspect the cause is MFlow's limitation of any vector over 127 being thrown out. Whether it happens depends on the resolution of the video and what "pel" value one is using. I need to find a workaround. If I use pel=1, it fixes it on lower resolutions, but also causes jaggies. I tried using MRecalculate, but as I expected, it did not help. I thought about trying MFlowInter, but it only seems able to generate an in-between frame, and I want to simply warp the current frame toward its neighbors. Any suggestions? I'd really like to make this work, one way or another.

EDIT: I tried MFlowInter, not only did it add undesirable frame blending, but it also did not fix the problem.

[hydra3333]

This question more properly resides here under the MVtools thread, I think.

Originally Posted by hydra3333

Just wondering, some people seem to use the DePanEstimate function last updated January 27 2008, whilst the more recent version of MVtools2 last updated 8.11.2009 suggests using the newer MSuper/MAnalyse/MDePan.

Would it be fair to guess that the newer MSuper/MAnalyse/MDePan functions would incorporate more recent updates and motion detection approaches than the older DePanEstimate functions and maybe some other things like multithreading ?

[XhmikosR]

I just got a crash using AviSynth 2.5.8 SEt's MT mod and VirtualDub points to mvtools2.dll.

SetMTMode(5,4)
DGDecode_mpeg2source("MyCapture.d2v", info=3)
ColorMatrix(hints=true, interlaced=true, threads=4)
SetMTMode(2)
TempGaussMC_beta2(EdiMode="EEDI2")
SelectEven()
crop(12,0,-12,-4)
Spline36Resize(640,480)

VirtualDub crash report -- build 32809 (release)
--------------------------------------

Disassembly:
02f64660: 660f6fda        movdqa xmm3, xmm2
02f64664: 660f61da        punpcklwd xmm3, xmm2
02f64668: 660f70d300      pshufd xmm2, xmm3, 00h
02f6466d: 660f6eda        movd   xmm3, edx
02f64671: 8b5520          mov    edx, [ebp+20h]
02f64674: 660f6fe3        movdqa xmm4, xmm3
02f64678: 660f61e3        punpcklwd xmm4, xmm3
02f6467c: 53              push   ebx
02f6467d: 660f70dc00      pshufd xmm3, xmm4, 00h
02f64682: 660f6ee0        movd   xmm4, eax
02f64686: 56              push   esi
02f64687: 8b7510          mov    esi, [ebp+10h]
02f6468a: 660f6fec        movdqa xmm5, xmm4
02f6468e: 57              push   edi
02f6468f: 8b7d08          mov    edi, [ebp+08h]
02f64692: 660f61ec        punpcklwd xmm5, xmm4
02f64696: 660fefc0        pxor   xmm0, xmm0
02f6469a: 660f70e500      pshufd xmm4, xmm5, 00h
02f6469f: c744240c100000  mov    dword ptr [esp+0ch], 00000010
          00              
02f646a7: 2bd1            sub    edx, ecx
02f646a9: 2bf1            sub    esi, ecx
02f646ab: 8bc1            mov    eax, ecx
02f646ad: 2bf9            sub    edi, ecx
02f646af: bb02000000      mov    ebx, 00000002
02f646b4: eb0a            jmp    02f646c0
02f646b6: 8da42400000000  lea    esp, [esp+00]
02f646bd: 8d4900          lea    ecx, [ecx+00h]
02f646c0: f30f7e2c02      movq   xmm5, [edx+eax]
02f646c5: f30f7e30        movq   xmm6, [eax]
02f646c9: 660f60e8        punpcklbw xmm5, xmm0
02f646cd: 660fd5ec        pmullw xmm5, xmm4
02f646d1: 660ffde9        paddw  xmm5, xmm1
02f646d5: 660f60f0        punpcklbw xmm6, xmm0
02f646d9: 660fd5f3        pmullw xmm6, xmm3
02f646dd: 660ffdf5        paddw  xmm6, xmm5
02f646e1: f30f7e2c06      movq   xmm5, [esi+eax]      <-- FAULT
02f646e6: 660f60e8        punpcklbw xmm5, xmm0
02f646ea: 660fd5ea        pmullw xmm5, xmm2
02f646ee: 660ffdee        paddw  xmm5, xmm6
02f646f2: 660f71d508      psrlw  xmm5, 08h
02f646f7: 660f67e8        packuswb xmm5, xmm0
02f646fb: 660fd62c07      movq   [edi+eax], xmm5
02f64700: 83c008          add    eax, 08h
02f64703: 83eb01          sub    ebx, 01h
02f64706: 75b8            jnz    02f646c0
02f64708: 8b7d08          mov    edi, [ebp+08h]
02f6470b: 8b7510          mov    esi, [ebp+10h]
02f6470e: 8b5520          mov    edx, [ebp+20h]
02f64711: 037d0c          add    edi, [ebp+0ch]
02f64714: 037514          add    esi, [ebp+14h]
02f64717: 035524          add    edx, [ebp+24h]
02f6471a: 034d1c          add    ecx, [ebp+1ch]
02f6471d: 836c240c01      sub    dword ptr [esp+0ch], 01h
02f64722: 897d08          mov    [ebp+08h], edi
02f64725: 897510          mov    [ebp+10h], esi
02f64728: 895520          mov    [ebp+20h], edx
02f6472b: 0f8576ffffff    jnz    02f646a7
02f64731: 5f              pop    edi
02f64732: 5e              pop    esi
02f64733: 5b              pop    ebx
02f64734: 8be5            mov    esp, ebp
02f64736: 5d              pop    ebp
02f64737: c3              ret    
02f64738: cc              int    3
02f64739: cc              int    3
02f6473a: cc              int    3
02f6473b: cc              int    3
02f6473c: cc              int    3
02f6473d: cc              int    3
02f6473e: cc              int    3
02f6473f: cc              int    3
02f64740: 55              push   ebp
02f64741: 8bec            mov    ebp, esp
02f64743: 83e4f0          and    esp, 0f0h
02f64746: 51              push   ecx
02f64747: 0fbf4d28        movsx  ecx, word ptr [ebp+28h]
02f6474b: 0fbf552c        movsx  edx, word ptr [ebp+2ch]
02f6474f: b880000000      mov    eax, 00000080
02f64754: 660f6ec8        movd   xmm1, eax
02f64758: 0fbf4530        movsx  eax, word ptr [ebp+30h]
02f6475c: 660f6fd1        movdqa xmm2, xmm1

Built on Aegis on Sat Feb 20 22:12:08 2010 using compiler version 1400

Windows 6.1 (Windows Vista x86 build 7600) []

EAX = 1be01728
EBX = 00000002
ECX = 1be01728
EDX = ff3a0000
EBP = 0addf1c8
ESI = e41fe8d8
EDI = ec207af0
ESP = 0addf1b0
EIP = 02f646e1
EFLAGS = 00010287
FPUCW = 027f
FPUTW = ffff

Crash reason: Access Violation

Crash context:
An out-of-bounds memory access (access violation) occurred in module 'mvtools2'...

...reading address 00000000.

Pointer dumps:

EAX   1be01728: 393b3b3b 39393938 8d7b5938 9190918f 93919091 95949592 91919190 91919091
ECX   1be01728: 393b3b3b 39393938 8d7b5938 9190918f 93919091 95949592 91919190 91919091
ESP   0addf1b0: 07f5e8e8 07fb4628 07f5e838 00000010 00000000 00000248 00000000 02f62cbd
      0addf1d0: 08009218 00000010 00000000 000002e0 1be01728 000002e0 1b1a1728 000002e0
      0addf1f0: 00000058 00000054 00000054 00000004 0288f968 00000001 00000000 000000fe
      0addf210: 000002e0 010102e0 00000000 00000000 00000000 1be01728 000002e0 00000001
EBP   0addf1c8: 00000000 02f62cbd 08009218 00000010 00000000 000002e0 1be01728 000002e0
      0addf1e8: 1b1a1728 000002e0 00000058 00000054 00000054 00000004 0288f968 00000001
      0addf208: 00000000 000000fe 000002e0 010102e0 00000000 00000000 00000000 1be01728
      0addf228: 000002e0 00000001 1b1a1728 00000054 00000001 00000000 000002e0 00000170

Thread call stack:
02f646e1: mvtools2!_AvisynthPluginInit2@4 [02f50000+6740+dfa1]
10009559: AviSynth!avs_release_value [10000000+75d0+1f89]
1000adb7: AviSynth!avs_release_value [10000000+75d0+37e7]
1005417a: AviSynth!DllGetClassObject [10000000+da50+4672a]
02ec9f71: mt_masktools!_AvisynthPluginInit2@4 [02e90000+2cd60+d211]
10009559: AviSynth!avs_release_value [10000000+75d0+1f89]
1000adb7: AviSynth!avs_release_value [10000000+75d0+37e7]
1005417a: AviSynth!DllGetClassObject [10000000+da50+4672a]
10009559: AviSynth!avs_release_value [10000000+75d0+1f89]
1000adb7: AviSynth!avs_release_value [10000000+75d0+37e7]
02ec3e38: mt_masktools!_AvisynthPluginInit2@4 [02e90000+2cd60+70d8]
02ec199d: mt_masktools!_AvisynthPluginInit2@4 [02e90000+2cd60+4c3d]
02ec1a3c: mt_masktools!_AvisynthPluginInit2@4 [02e90000+2cd60+4cdc]
10009559: AviSynth!avs_release_value [10000000+75d0+1f89]
1000adb7: AviSynth!avs_release_value [10000000+75d0+37e7]
02ec9f71: mt_masktools!_AvisynthPluginInit2@4 [02e90000+2cd60+d211]
02ec78c8: mt_masktools!_AvisynthPluginInit2@4 [02e90000+2cd60+ab68]
02ec31f1: mt_masktools!_AvisynthPluginInit2@4 [02e90000+2cd60+6491]
02ebbfc1: mt_masktools!0002bfc1
02ed1bea: mt_masktools!_AvisynthPluginInit2@4 [02e90000+2cd60+14e8a]
10009559: AviSynth!avs_release_value [10000000+75d0+1f89]
1000adb7: AviSynth!avs_release_value [10000000+75d0+37e7]
02ec6f88: mt_masktools!_AvisynthPluginInit2@4 [02e90000+2cd60+a228]
02ec9e10: mt_masktools!_AvisynthPluginInit2@4 [02e90000+2cd60+d0b0]
02ec199d: mt_masktools!_AvisynthPluginInit2@4 [02e90000+2cd60+4c3d]
02ec1a3c: mt_masktools!_AvisynthPluginInit2@4 [02e90000+2cd60+4cdc]
10009559: AviSynth!avs_release_value [10000000+75d0+1f89]
1000adb7: AviSynth!avs_release_value [10000000+75d0+37e7]
02ec7d68: mt_masktools!_AvisynthPluginInit2@4 [02e90000+2cd60+b008]
02ebc377: mt_masktools!0002c377
02ec3f70: mt_masktools!_AvisynthPluginInit2@4 [02e90000+2cd60+7210]
10009559: AviSynth!avs_release_value [10000000+75d0+1f89]
1000adb7: AviSynth!avs_release_value [10000000+75d0+37e7]
02ec1bc1: mt_masktools!_AvisynthPluginInit2@4 [02e90000+2cd60+4e61]
02f61b93: mvtools2!_AvisynthPluginInit2@4 [02f50000+6740+b453]
02ec9e10: mt_masktools!_AvisynthPluginInit2@4 [02e90000+2cd60+d0b0]
02ec199d: mt_masktools!_AvisynthPluginInit2@4 [02e90000+2cd60+4c3d]
02ec1a3c: mt_masktools!_AvisynthPluginInit2@4 [02e90000+2cd60+4cdc]
10009559: AviSynth!avs_release_value [10000000+75d0+1f89]
1000adb7: AviSynth!avs_release_value [10000000+75d0+37e7]
02ec78c8: mt_masktools!_AvisynthPluginInit2@4 [02e90000+2cd60+ab68]
10009559: AviSynth!avs_release_value [10000000+75d0+1f89]
1000adb7: AviSynth!avs_release_value [10000000+75d0+37e7]
7c3416b3: MSVCR71!__crtLCMapStringA [7c340000+13ae+305]
100372e6: AviSynth!DllGetClassObject [10000000+da50+29896]
10009559: AviSynth!avs_release_value [10000000+75d0+1f89]
1000adb7: AviSynth!avs_release_value [10000000+75d0+37e7]
1006492e: AviSynth!DllGetClassObject [10000000+da50+56ede]
771e2149: ntdll!RtlAllocateHeap [77190000+5209d+ac]
10009559: AviSynth!avs_release_value [10000000+75d0+1f89]
1000adb7: AviSynth!avs_release_value [10000000+75d0+37e7]
1003bd40: AviSynth!DllGetClassObject [10000000+da50+2e2f0]
10009559: AviSynth!avs_release_value [10000000+75d0+1f89]
1000adb7: AviSynth!avs_release_value [10000000+75d0+37e7]
7c3416b8: MSVCR71!__crtLCMapStringA [7c340000+13ae+30a]
1006492e: AviSynth!DllGetClassObject [10000000+da50+56ede]
7c3416b8: MSVCR71!__crtLCMapStringA [7c340000+13ae+30a]
10009559: AviSynth!avs_release_value [10000000+75d0+1f89]
1000adb7: AviSynth!avs_release_value [10000000+75d0+37e7]
1000464d: AviSynth!0000464d
1005533b: AviSynth!DllGetClassObject [10000000+da50+478eb]
10005337: AviSynth!00005337
10005407: AviSynth!00005407
1005627b: AviSynth!DllGetClassObject [10000000+da50+4882b]
771d5e6c: ntdll!NtWaitForSingleObject [77190000+45e60+c]
754a179c: KERNELBASE!WaitForSingleObjectEx [754a0000+1730+6c]
754a17c4: KERNELBASE!WaitForSingleObjectEx [754a0000+1730+94]
10009559: AviSynth!avs_release_value [10000000+75d0+1f89]
1000adb7: AviSynth!avs_release_value [10000000+75d0+37e7]
1000e2cd: AviSynth!DllGetClassObject [10000000+da50+87d]
76b71194: kernel32!BaseThreadInitThunk [76b20000+51182+12]
771eb3f5: ntdll!RtlInitializeExceptionChain [77190000+5b392+63]
771eb3c8: ntdll!RtlInitializeExceptionChain [77190000+5b392+36]

-- End of report

Fizick, is there anything I can do to help you pinpoint this problem?

[TSchniede]

I suppose this is quite obvious, but unfortunately the best and fastest way to fix any bug is to replicate it. This involves finding the simplest test case that triggers it.

1) which filters are needed / is it dependent on a specific input video (or source filter) / does it depend on specific options in the filters? (this helps pinpoint which filter / which part of a filter is responsible)
2) how fast does it occur? (first frame / <1000 frames / >50000)
3) is it always on the same frame (+-10 frames)?
4) Is there a relationship with other system components - like maxed out CPU load or network traffic?
5) And since you mentioned MT - is there a difference on the amount of threads used?
I suppose it never occurs single-threaded?

[Terka]

Fizick, do you have some plans about mvtools - to add some new features/improvements?
There is no update on your pages for 6months.

[kypec]

Hasn't been mvtools replaced with mvtools2? I thought so but I might be very wrong. I'm just a noob regarding any external plugins and scripts...

[Fizick]

i do not like the limitation of vector length to 64 (for pel=2).

[Terka]

yes, was replaced. i ment mvtools as 'filter name' regardless of a version. so the question was in fact about mvtools version2.

[pbristow]

I have a suggestion for you guys (Terka, Fizick, et al.) to think about...

We have the function MMask, which can convert MVTools2's vector data into a "normal" clip, where, using kind=5, colours (in U,V) represent motion vectors (in X,Y). Could we have a function that does the inverse of this, i.e. turns the "kind=5" colourmap back into a vector clip (such as produced by MAnalyse), suitable for driving MCompenstae, Mflow, etc.?

This would then open the door to manipulating vector data using various avisynth functions, and applying the results to the source clip. For example, to obtain a map of the *acceleration* over three frames, you could do:

MotionMap = MMask(Source, Vectors, kind=5)
AccelMap = Subtract(MotionMap, MotionMap.DeleteFrame(0))

... and then use that data, via the new function (called, perhaps, "MUnmask"?) to create a parabolic, rather than linear, interpolation of the motion between frames n-1 and n+1. Take things one step futher, and you've got cubic interpolation, and so on.

You could also apply a combination of masks and blurs to a motion map before using it in compensation, to create an effect similar to varying the mvtools "lamda" factor (vector coherence) across the frame, depending on detected content. I.e., you could enforce higher vector coherence (strong "blur" of U & V data) where regular textures/patterns are detected, but reduce it where an edge is detected between two separately moving objects. Lots of possibilities open up for more precise, or more creative, motion manipulation! =:o}

Interested?

(Hmmm... Should I putting this in development?)

[Gavino]

Originally Posted by pbristow

For example, to obtain a map of the *acceleration* over three frames, you could do:
MotionMap = MMask(Source, Vectors, kind=5)
AccelMap = Subtract(MotionMap, MotionMap.DeleteFrame(0))

Don't you have to subtract the value at two different positions in order to compensate for the movement between frames?

[Boulder]

Edit: never mind, I got it..MDepan must be returned in order to create a log file.

[Terka]

MVtools are esencial avisynth plugin. So every development of them is big plus. I think the vectors modifying could be useful.
There aslo exists some 'requests' about what new feature(s) to have, Didee posted some here in forum. Maybee he could have something to say.
Also there are some papers over internet about phase correlation etc.

[Boulder]

A bug report: if I use range=1 in MDepan, the debug text (with info=true) is only displayed in the first frame when loading the script. It disappears when seeking to the next frame.

[tormento]

One question: is MDegrain3 more precise and stronger or only stronger in removing noise?

I'd like to use as a standard avs part a very light grain removal part to increase compressibility without remove too much details. I am thinking about a thSAD=200 or so with the usual motion compensated denoise script of the documentation.

Do you think that the thSAD number is too high or low? I read in the documentation that a too low number could cause "staggered denoising" but what is a too low number for that parameter? As I want remove grain only, do you think I should use a different thSADC for chroma?

And, returning to my first question, as far as PRECISION and DETAILS are concerned, is it better to use MDegrain3 or MDegrain2?

[Didée]

In that sense, there is no "precision" in MDegrainX. The "precision" is in MAnalyse.

Before we had MVTools, the problem was that the dumb PC could not distinguish between noise and detail.

Now that we have MVTools, the problem is that the dumb PC still can't distinguish between noise and detail.

What has changed is that, due to motion compensation, the likelyhood to catch more noise than detail has become bigger. But the basic problem hasn't changed. Good old TemporalSoften uses differences between single pixels to make its decisions. Shiny new MVTools uses differences between single pixels, summed up over 8x8 blocks, to make its decisions.

The method as a whole has become more robust, but the mechanism of making decisions is still the same.
("Hey! Now there is weighting, too!" -- [bored:] "ttempsmooth!")

[*.mp4 guy]

Originally Posted by Didée

What has changed is that, due to motion compensation, the likelyhood to catch more noise than detail has become bigger. But the basic problem hasn't changed. Good old TemporalSoften uses differences between single pixels to make its decisions. Shiny new MVTools uses differences between single pixels, summed up over 8x8 blocks, to make its decisions.

With dct=5, you get usably fast satd, instead of very fast sad, which can give a nice boost to perceived (and probably actual) quality of denoising. Anyway, "Doing It Right" with "intelligent" metrics would be ridiculously slow, were talking orders of magnitude worse then tnlmeans.

Also, "its just differences of blocks of pixels instead of singly pixels" rather misses the point. VQ is just quantizing blocks of pixels instead of single pixels, but the coding gain is ridiculous. And its ridiculous because working on blocks of pixels is not comparable to working on single pixels. When dealing with blocks, even if your metric is incredibly simple, you are matching features whenever you do a block search, you can't do block matching without getting some feature detection. This combined with the higher signal correlation between frames as opposed to pixels within a frame are why mvtools is so good. The high correlation allows for fast metrics and lazy pixel grouping to work very well, thereby enabling a "fast" implementation. You can actually apply the same methodology to pixels within a frame (which is basically what tnlmeans does) but because correlation is so much lower, you have to have much more granular pixel grouping to get usable results, and to get truly good result you would probably need a much better metric then sad or sse as well.

[tormento]

Guys, your knowledge is fascinating but is it better, precision related, Degrain2 or Degrain3?

Didée, you told precision is in MAnalyse. Do you think that using Degrain3, as far as precision is concerned, is similar do Degrain2 if the same MAnalyse process is used? If positive, does Degrain3 removes only more noise?

I thought that thSAD was the only parameter that decided the amount of noise (let me say that) to be removed. You overwhelmed me.

[Didée]

mp4guy - hey, of course I have exaggerated a bit, wasn't that obvious? I just wanted to make the point that the algorithms still aren't "seeing" anything. And, even with (inherently) better feature correlation, turing's machine still will not tell apart whether any given difference is due to a change in a detail's reprentation from one frame to the next, or because of noise, or because of something else. It's just a difference.
In particular, any "weak" detail that is "hidden" below the noise (signal amplitude of detail is same - or even lower - than the noise level) by chance might survive, or by chance will get filtered away. Dice game.

When dealing with blocks, even if your metric is incredibly simple, you are matching features whenever you do a block search, you can't do block matching without getting some feature detection.

Sure. The best part is when there is a more or less feature-less something (sky, wall, whatever), and the motion engine still is detecting "features" and compensates them. Everybody loves floating walls.


tormento: MDegrain3 removes more noise. MDegrain2 keeps more detail. That's basically it.


If you guys insist, I'll dig out my Babylon-5 samples once more. SAD fights a losing battle there. And SATD won't do better.

[*.mp4 guy]

Originally Posted by Didée

mp4guy - hey, of course I have exaggerated a bit, wasn't that obvious?

Yes, but I wasn't sure where the exaggeration ended.

Originally Posted by Didée

I just wanted to make the point that the algorithms still aren't "seeing" anything. And, even with (inherently) better feature correlation, turing's machine still will not tell apart whether any given difference is due to a change in a detail's reprentation from one frame to the next, or because of noise, or because of something else. It's just a difference.
In particular, any "weak" detail that is "hidden" below the noise (signal amplitude of detail is same - or even lower - than the noise level) by chance might survive, or by chance will get filtered away. Dice game.

The more correlation an algorithm can leverage, the more selective it is in removing noise. This is why compression related features almost always work well for denoising (linear transforms, motion compensation, etc.). The more correlation a given algorithm can exploit, the further down the noise floor below which details are indistinguishable from noise is pushed (on average, there will always be certain cases that break a given algorithm).

Originally Posted by Didée

tormento: MDegrain3 removes more noise. MDegrain2 keeps more detail. That's basically it.

yep

Originally Posted by Didée

If you guys insist, I'll dig out my Babylon-5 samples once more. SAD fights a losing battle there. And SATD won't do better.

SATD essentially always does better, but I'm sure the results would still be quite bad, so no need to dig it up. That source is more difficult then it should be, If I had to take a guess I would say that there is a good chance that it is due to compression artifacts. More specifically I have seen cases where not-so-great mpeg2 compression will cause detail to become "intermittent" (similar to jumping b-frames in xvid) when this happens mvtools can almost always be relied on to remove whatever detail is left, or leave all of the noise alone, because the feature that mvtools uses to separate noise from detail has been ruined by the compression. The original R1 dvd of the matrix was a classic example of this problem.

All of that said I think we are essentially in agreement and that I am just quibbling over terminology (that I still don't agree with) but that is irrelevant to the actual point.Also, no offense was meant, there was just a certain ways your previous comment could have been read that bothered me.