MVTools-pfmod

[tormento]

Now we need a revised masktools

[real.finder]

Originally Posted by pinterf

New release:

• MvTools2 2.7.5.22 (20161119)
  General support of 10-16 bit formats with Avisynth Plus (r2294 or newer recommended)
  with new MDegrain4 and MDegrain5 filters.
• Depan: 2.13.1 (20161119)
• DepanEstimate: 2.10 (20161119)
  General support of 10-16 bit formats with Avisynth Plus
  Compiled for 64 bit for the first time
  Supporting YV16 and YV24 for 8 bit and their 10+ bits equivalents
  Requires 2.6 interface

Download:
MvTools 2.7.5.22 + Depan + DepanEstimate for 32 and 64 bits

For general info see readme or the first page of the thread.
Have fun.

Todo:

• greyscale support (not tested)
• Port remaining inline assembly to SIMD intrinsics
• satd 16 bit to SIMD
• fix reported bugs

Energetic as always

has the Depan fix this bug (turns the video green)?

[Reel.Deel]

Originally Posted by pinterf

New release:

• MvTools2 2.7.5.22 (20161119)
  General support of 10-16 bit formats with Avisynth Plus (r2294 or newer recommended)
  with new MDegrain4 and MDegrain5 filters.
• Depan: 2.13.1 (20161119)
• DepanEstimate: 2.10 (20161119)
  General support of 10-16 bit formats with Avisynth Plus
  Compiled for 64 bit for the first time
  Supporting YV16 and YV24 for 8 bit and their 10+ bits equivalents
  Requires 2.6 interface

Download:
MvTools 2.7.5.22 + Depan + DepanEstimate for 32 and 64 bits

For general info see readme or the first page of the thread.
Have fun.

Todo:

• greyscale support (not tested)
• Port remaining inline assembly to SIMD intrinsics
• satd 16 bit to SIMD
• fix reported bugs

Awesome work as always, thanks pinterf!

Originally Posted by real.finder

has the Depan fix this bug (turns the video green)?

I think Fizick fixed it in the latest release, see here: http://forum.doom9.org/showthread.ph...63#post1763563

Originally Posted by tormento

Now we need a revised masktools

A while back tp7 started working on a 16-bit MaskTools, unfortunately it was not finished. See here: https://github.com/tp7/masktools/commits/16bit

Maybe someone will come along and finish it. Another route is porting VS' Expr and co.

[feisty2]

Someday, maybe there will be a complete version of mvtools that could merge all mvtools variations into one
One binary serves as both a vaporsynth plugin and an avisynth plugin, supporting bitdepths from 8 to 32 and arbitrary temporal radius..

[real.finder]

Originally Posted by Reel.Deel

I think Fizick fixed it in the latest release, see here: http://forum.doom9.org/showthread.ph...63#post1763563

I missed this update...

[real.finder]

MvTools2 2.7.5.22d is slower than mvtools2 2.7.0.22d

~1.7 fps vs ~1.5 fps with same complex script

is this because it built without ICC?

[Groucho2004]

Originally Posted by real.finder

MvTools2 2.7.5.22d is slower than mvtools2 2.7.0.22d

~1.7 fps vs ~1.5 fps with same complex script

is this because it built without ICC?

No. It's because ill-tempered fairies have invaded your computer and are eating CPU cycles.

If you post the script that causes this behaviour and possibly mention what CPU you have you'll probably get a better answer.

[pinterf]

Originally Posted by real.finder

MvTools2 2.7.5.22d is slower than mvtools2 2.7.0.22d

~1.7 fps vs ~1.5 fps with same complex script

is this because it built without ICC?

Yes.
Since then I found some bottlenecking places when using __forceinline helped poor vs2015.

Testing on a typical MDegrain3 script:
2.7.5.22: 4.13 fps (VS2015)
2.7.0.22d: 4.69 fps
2.7.futu.re: 4.62 fps (VS2015)

Promising.

[Groucho2004]

Originally Posted by pinterf

Testing on a typical MDegrain3 script:
2.7.5.22: 4.13 fps (VS2015)
2.7.0.22d: 4.69 fps

What CPU do you use for these tests? Also, what switches for ICC?

[pinterf]

i7 ivy bridge.
Icc build was for sse2 with optional sse4.1 code paths

[real.finder]

Originally Posted by pinterf

Yes.
Since then I found some bottlenecking places when using __forceinline helped poor vs2015.

Testing on a typical MDegrain3 script:
2.7.5.22: 4.13 fps (VS2015)
2.7.0.22d: 4.69 fps
2.7.futu.re: 4.62 fps (VS2015)

Promising.

why stop using ICL in last ver.? for amd users?

[Groucho2004]

Originally Posted by pinterf

Icc build was for sse2 with optional sse4.1 code paths

Well, that's just one switch (QaxNNN). From my experience with Intel compilers, there are at least 5-7 other switches that can affect performance. Just a little selection:

/O3    optimize for maximum speed and enable more aggressive optimizations

/Qipo[n]  enable multi-file IP optimization between files

/Qunroll-aggressive

/Qopt-ra-region-strategy[:<keyword>]
          select the method that the register allocator uses to partition each
          routine into regions
            routine - one region per routine
            block   - one region per block
            trace   - one region per trace
            loop    - one region per loop
            default - compiler selects best option

/Qprof  profiling

[pinterf]

Will check it later now i'm from mobile.

[Groucho2004]

Originally Posted by pinterf

Icc build was for sse2 with optional sse4.1 code paths

One more thing about the automatic CPU dispatcher (enabled with QaX...) in the Intel compiler - This could actually have an impact on AMD CPUs since I suspect that even with the latest incarnation of the compiler it may chose sub-optimal optimizations for those.
If I build for specific instruction sets, I always "hard-code" them by using "Qx..." instead of "QaX...". This way all CPUs use the same code path. That of course means building binaries for each instruction set.
I also recommend testing if the SSE4.x or even AVX options really make a difference. More often than not, they don't. As usual, it all depends on the code.

[jpsdr]

Originally Posted by Groucho2004

If I build for specific instruction sets, I always "hard-code" them by using "Qx..." instead of "QaX...". This way all CPUs use the same code path. That of course means building binaries for each instruction set.

Oh... You too.

Didn't know some of the options :

/Qunroll-aggressive
/Qopt-ra-region-strategy[:<keyword>]

I'll have to check themn even if... "aggressive". Just seeing the word make me a little suspicious.

[Groucho2004]

Originally Posted by jpsdr

Oh... You too.

Didn't know some of the options :

/Qunroll-aggressive
/Qopt-ra-region-strategy[:<keyword>]

I'll have to check themn even if... "aggressive". Just seeing the word make me a little suspicious.

Don't worry.

I suppose my way of building binaries differs a lot from what everyone else does. I don't use the IDE, I create makefiles for my projects and build from the command line with batch files. The makefiles have easily accessible compiler and linker options so I can quickly change them, rebuild and test.

As for all compiler options - run "icl -help" and pipe into a text file. And there are of course the Intel docs that come with the compiler which have lots of stuff about optimization (which almost nobody reads, I guess).

[pinterf]

Originally Posted by Groucho2004

One more thing about the automatic CPU dispatcher (enabled with QaX...) in the Intel compiler - This could actually have an impact on AMD CPUs since I suspect that even with the latest incarnation of the compiler it may chose sub-optimal optimizations for those.
If I build for specific instruction sets, I always "hard-code" them by using "Qx..." instead of "QaX...". This way all CPUs use the same code path. That of course means building binaries for each instruction set.
I also recommend testing if the SSE4.x or even AVX options really make a difference. More often than not, they don't. As usual, it all depends on the code.

Not to mention the aligment hints, hints for typical loop size, etc.

Finally I couldn't find my old ICC settings. Sure, loop unrolling was at default, so it was not fine-tuned, but I've seen unrolled loops in the asm output (sometimes I check the asm code that compilers generate). Global optimization was on, and the maximum optimization, too.

When I have successfully gained back a lot of speed, I used VS2015's performance profiler that showed me the parts where the code spends most of the time. Then I forced these functions to be inline.

There were other optimizations I have made, so perhaps the ICC version could also be faster, from the current codebase.

Interesting article on the optimizer changes came with VS2015 update 3:
https://blogs.msdn.microsoft.com/vcb...ode-optimizer/

[Groucho2004]

There are also oddball cases where the compiler options for max. speed have the opposite effect. I have a couple of programs where turning off automatic inlining or even using "O1" instead of "O2/O3" results in faster binaries. Always test, if possible on Intel and AMD CPUs.

[jpsdr]

Originally Posted by Groucho2004

And there are of course the Intel docs that come with the compiler which have lots of stuff about optimization (which almost nobody reads, I guess).

Yes, i've read it... a long time ago.

[pinterf]

New version. 2.7.6.22
Depan and DepanEstimate are unchanged.

https://github.com/pinterf/mvtools/r...s/tag/2.7.6.22

Change log
2.7.6.22 (20161204) - fixes and speedup

• fix: sumLumaChange underflow (used for dct=2,6,9) (regression during 16 bit support)
• fix: MeanLumaChange scale for 10-16 bits (used for dct=2,6,9)
• fix: Mask fix: 8 bit mask resizer bug in SIMD intrinsics - Thx real.finder
  (regression on inline asm -> SIMD transition)
• Fix: dctmode=1,2: pixel distance was not corrected for 16 bit pixel sizes
• speed: Let's help VS2015 with some __forceinline directives to recognize the truth.
• speed: Misc optimizations throughout the code (bit shifts instead of div or mul)
• speed: FFTW DCT: C code replaced with SIMD SSE2/SSE4 (FloatToBytes, BytesToFloat)
• speed: 16 bit SAD: a few optimizations in SSE2, AVX-coded SSE2 and AVX2 codepath
• VS2015 compiler: /MT -> /MD (from static to dynamic dlls - now it really needs VS2015 redistributables)