[Neo] FFT3D / DFTTest

[MeteorRain]

This thread serves Neo-FFT3D and Neo-FFTTest.

Neo filters family is not to be confused with AviSynth-Neo. Neo filters family is not affiliated in any way with AviSynth-Neo.

[Neo] FFT3D → GitHub (https://github.com/HomeOfAviSynthPlusEvolution/neo_FFT3D)

FFT3DFilter is a 3D Frequency Domain filter - strong denoiser and moderate sharpener. It was originally written by Alexander G. Balakhnin aka Fizick, and later modified by martin53 for AviSynth 2.6 and later modified by Ferenc Pintér aka pinterf for further improvement, high bit depth, and more. Kudos to them for creating and improving this fantastic tool.

Difference compared to pinterf/FFT3DFilter


Dual interface, supporting AviSynth+ and VapourSynth
Removed YUY2
Removed multiplane options, added y, u, v options for plane control
Removed existing 3DNow and SSE code
Fixed an incorrect coefficient in Apply2D
Added SSE, AVX and AVX512 routine
Added multi-threading, both internal and AVS-MT prefetch


Result bit-identicalness


Neo-FFT3D C - FFT3DFilter C: identical except for a few situations
Except: bt=2 and sharpen and dehalo where the incorrect coefficient is fixed
Except: bt=4, 5 where the floating point numbers are added in different way ( (fp2r + fpr) + (fcr + fnr) in Neo-FFT3D, (fp2r + fpr + fcr + fnr) in FFT3DFilter ) resulting rounding error
Except: bt=0 and pfactor > 0 causing crash, need your investigation and fixes
===== ===== ===== =====
Neo-FFT3D SSE - C: identical
Neo-FFT3D AVX - C: identical
Neo-FFT3D AVX512 - C: theoretically identical


[Neo] DFTTest → GitHub (https://github.com/HomeOfAviSynthPlusEvolution/neo_DFTTest)

DFTTest is a 2D/3D Frequency Domain denoiser. It was originally written by tritical, and later modified by Ferenc Pintér aka pinterf for further improvement, high bit depth, and more. VapourSynth-DFTTest was ported to VapourSynth interface by HolyWu with further clean ups and efficient SSE2 and AVX2 SIMD routines. Kudos to them for creating and improving this fantastic tool.

Difference compared to pinterf/dfttest


Dual interface, supporting AviSynth+ and VapourSynth
Parameter names and opt levels follow VapourSynth-DFTTest
Removed Y, U, V options, added y, u, v options for plane control
Removed hacked 16 bit I/O
Replaced internal multi-threading with C++ Parallel STL, removed Windows specific APIs
Improved precision of floating point reciprocal using Newton Raphson method
sigma/sigma2/sbsize/sosize/tbsize etc. has different default value


Difference compared to HomeOfVapourSynthEvolution/VapourSynth-DFTTest

Dual interface, supporting AviSynth+ and VapourSynth
Added dither option from AVS counterpart
Added internal multi-threading
Replaced division by Newton Raphson method reciprocal in SSE2
Replaced reciprocal approximation by Newton Raphson method in AVX2


Result bit-identicalness


Neo-DFTTest C - (AVS)DFTTest C: identical except when nlocation is provided, due to order of floating point multiplication
Neo-DFTTest: data = data * (scale * (wscale2 / wscale) * alpha)
(AVS)DFTTest: data = data * scale * (wscale2 / wscale) * alpha
Neo-DFTTest C - (VS)DFTTest C: identical
===== ===== ===== =====
Neo-DFTTest SSE - C: identical except when dither > 2, due to rounding errors in dithering
Neo-DFTTest AVX - C: identical except when dither > 2, due to rounding errors in dithering

[feisty2]

I took a peek at your dual interface, I think the argument fetching part, particularly the part dealing with array parameters could be improved.
I don't think it's a good idea to assume array parameters are std::vector<T>, and to erase the vector while fetching the arguments.
people might not use vector to store array parameters, it could also be std::array or a pointer to some pre-allocated memory or something else for various reasons.
it's also not a good practice to erase the parameter container, the container might hold more default values than the number of values specified by the user. you lose all default values when you erase the container and the user loses the flexibility to specify a partial array, leaving the rest of default values untouched.

I think it's best if the argument object itself behaves like a container (it does not necessarily to be an actual container) and let the filter developer decide how the arguments should be handled.

// this is bad
args.read("sigma_array", sigma_array);

// this is better, and allows the user to specify a partial array
for (auto x : range{ args["sigma_array"].size() })
sigma_array[x] = args["sigma_array"][x];

// or if you don't need the flexibility to specify a partial array
for (auto sigma : args["sigma_array"])
sigma_array.push_back(sigma);


the dual interface also doesn't yet seem to handle frame properties for vaporsynth, I have completed this functionality and a Rec601ToRGB example is also available. if you plan to add support for frame properties, the syntax should be the same as, or at least very similar to the aforementioned "arguments" thing for the sake of consistency.

[MeteorRain]

Thanks for taking your time to look at the code. You are absolute right about parameter processing. Part of those code was copied from my previous projects years ago, and there's lots of room for improvement.

At this moment I'm trying to make scaffolding as quick as possible (this wrapper was made after your wrapper, so you know it's still very young). And my hope is if anyone has interests they can send merge requests to improve it. Of course, at this moment I'd aim at C++17 and not 20 or 23. Let's wait until C++20 or 23 becomes mainstream before moving forward.

Your point about array makes a lot sense. I'll need some time to think about it. For now vectors are adequate to my needs, but I'll try to make it better. As you suggested, begin() and end() would be a good start.

[MeteorRain]

Please test dfttest beta1 (https://down.7086.in/AviSynthPlus%20Filters/neo-dfttest-beta1~20200415.zip).

Example: neo_dfttest(ftype=0, sigma=8.0, sigma2=12.0, pmin=0.0, pmax=500.0, sbsize=4, smode=1, sosize=2, tbsize=1, tmode=0, tosize=0, opt=0, zmean=True, y=3, u=3, v=3, nlocation="0,0,20,40", slocation="0.0:3.0, 0.3 14.0 1.0,10.0", ssystem=1, threads=6)

Syntax is similar to original AVS dfttest except:

Y/U/V=true/false is now y/u/v=3/2/1 where 3=process, 2=copy, 1=garbage.

nstring is now alpha and nlocation.
For example, (nstring="0,0,20,40 a=5.2") is now (nlocation="0,0,20,40", alpha=5.2).

sstring is now ssystem and sstring.
For example, (sstring="$ 0.0:3.0 1.0:10.0") is now (sstring="0.0,3.0,1.0,10.0", ssystem=1).

sstring, ssx, ssy, sst supports any delimiter of "," or " " or ":".
For example, (ssx="0.0:3.0 1.0:10.0") equals (ssx="0.0,3.0,1.0,10.0").

opt is now 0=auto, 1=C, 2=SSE2, 3=AVX2.

No hacked 16bit IO. Only native HBD is supported.

threads=X only applies to SSE2 and AVX2 routine, C code always runs at single thread.

sigma and sigma2 defaults to 8.0. Manually set to 16 to match original result.

C/SSE2/AVX2 now produces bit identical results except for random dithered 8bit, which gets some rounding errors.

On VapourSynth, nlocation, slocation, ssx, ssy, sst are still arrays.

Let me know any issues.

[StainlessS]

which gets some rounding errors.
Maybe in future use term "rounding differences", allay [lessen] user worries about errors.

[MeteorRain]

dither > 2 uses Mersenne Twister random number generator to dither data, so outputs would never be the same every time. Whether there are rounding "errors" or "differences" no longer matters, I think.

[StainlessS]

Its just that some use a translator to read docs, and sometimes translators can change meaning a little and may result in much exaggerated significance.

[ChaosKing]

Please test dfttest beta1 (https://down.7086.in/AviSynthPlus%20Filters/neo-dfttest-beta1~20200415.zip).



Tested a bit in VS:
- 8-16 + 32 bit depth works fine
- clip.neo_dfttest.DFTTest(ftype=0, sigma=8.0, sigma2=12.0, pmin=0.0, pmax=500.0, sbsize=4, smode=1, sosize=2, tbsize=1, tmode=0, tosize=0, opt=0, zmean=True, nlocation="0,0,20,40", slocation="0.0:3.0, 0.3 14.0 1.0,10.0", ssystem=1, threads=6)
produces
Core freed but 5 filter instance(s) still exist
Core freed but 5 filter instance(s) still exist
Core freed but 537600 bytes still allocated in framebuffers
Core freed but 537600 bytes still allocated in framebuffers
EDIT:
The parameters above triggersValueError: invalid literal for int() with base 10: '0,0,20,40'

- No warnings with a simple neo_dfttest.DFTTest(sigma=8, sigma2=8)
- speed:
-- vapoursynth dfttest 7.9 fps
-- vapoursynth neo dfttest 5.5 fps

[MeteorRain]

Yea I haven't tested thoroughly on VS.

On VS nlocation and slocation and ss? are still arrays. I should have said those changes only apply to AVS. On VS array inputs are the same as vapoursynth-dfttest.

I'll look on memory leaks later.

Speed difference is probably because I've set it as serialized filter (so single threaded from VS point of view) for debugging purpose. Will change to parallel on release.

[ChaosKing]

So this would be correct?
neo_dfttest.DFTTest(nlocation=[0,0,20,40])
It crashes immediately :D

[MeteorRain]

beta2 (https://down.7086.in/AviSynthPlus%20Filters/neo-dfttest-beta2~20200415.zip)
Fixes fmParallel
Fixes nlocation crashes

When nlocation does not crash, there seems no memory leaks found.

[ChaosKing]

beta2 (https://down.7086.in/AviSynthPlus%20Filters/neo-dfttest-beta2~20200415.zip)
Fixes fmParallel
Fixes nlocation crashes

When nlocation does not crash, there seems no memory leaks found.
Works good now. The only thing I found was a msgCore freed but 1 filter instance(s) still exist
Core freed but 1 filter instance(s) still exist
if you specify an incomplete parameter like this DFTTest(nlocation=[8]). Idk if this counts as an error.

[MeteorRain]

Supplying incorrect parameters may trigger an error with some temporary data not being freed correctly. I'll check where it leaks, but I don't think it's a big issue.

[real.finder]

I did some tests

ColorBars(width=640, height=480, pixel_type="yv12")
admfilter(custom_filter="neo_fft3d(Sigma=(adSigma+1.0)/f)")
Prefetch(4)

with https://forum.doom9.org/showthread.php?p=1904672#post1904672

I get
https://i.postimg.cc/23Wgtx5H/Untitled.png (https://postimg.cc/23Wgtx5H)

and with https://forum.doom9.org/showthread.php?p=1906016#post1906016

it's just freeze! and sometimes

https://i.postimg.cc/146wN8Wn/Untitled.png (https://postimg.cc/146wN8Wn)

and sometimes same as the 1st one

[MeteorRain]

I can't get your script work. I assume I should use "Advanced Denoising and anime bob v1.87"?

I get -- I don't know what 'MotionRampadc' means.

[real.finder]

I can't get your script work. I assume I should use "Advanced Denoising and anime bob v1.87"?

I get -- I don't know what 'MotionRampadc' means.

you only need https://github.com/realfinder/AVS-Stuff/raw/master/avs%202.5%20and%20up/AdvancedDenoising.avsi and https://github.com/realfinder/AVS-Stuff/raw/master/avs%202.5%20and%20up/Zs_RF_Shared.avsi

and from that error you get seems you didn't use any runtime MT fixed avs+ (both test avs+ builds I post)

edit: also you maybe missed this GRunT update (http://www.mediafire.com/file/77uromd0j0z56ap/GRunT-1.0.2tst.7z/file) pinterf made years ago

[MeteorRain]

Okey grunt it is. Now I can see it crashed. Let me check.

That's easy to find. Filter initialization was executed concurrently, crashed the fftw library. I don't know if initialization should be thread safe or not. I'll add a mutex then.

[real.finder]

Okey grunt it is. Now I can see it crashed. Let me check.

That's easy to find. Filter initialization was executed concurrently, crashed the fftw library. I don't know if initialization should be thread safe or not. I'll add a mutex then.

DFTTest was also crashed, but pinterf fix it, didn't try neo DFTTest yet, but last DFTTest by pinterf work

edit: it fixed since this one https://forum.doom9.org/showthread.php?p=1904777#post1904777

[MeteorRain]

Yes my solution is similar to that. Usually initialization is single threaded, so you don't have to think about thread safety. In this case multiple instances were initialized so there needs to be a global mutex there. pinterf used static global, and I used a static local, the result would be similar.

beta3 (https://down.7086.in/AviSynthPlus%20Filters/neo-dfttest-beta3~20200415.zip) thread-safe initialization.

[real.finder]

beta3 (https://down.7086.in/AviSynthPlus%20Filters/neo-dfttest-beta3~20200415.zip) thread-safe initialization.

it's seems work

btw, how you end-up with this case? https://forum.doom9.org/showthread.php?p=1905043#post1905043

and aside from that, do you plan to did the same with all/most/some https://github.com/HomeOfVapourSynthEvolution ? (Dual interface with neo renames)

[MeteorRain]

I believe the change of bit depth should not change a parameter, at least in most cases.
Both vapoursynth-dfttest and dfttest scale parameters to the bit depth so it wasn't a problem.

I haven't decided what to do next, but I may start with a few. mvtools-sf is also a good candidate. Depending on my time availability.

[real.finder]

I believe the change of bit depth should not change a parameter, at least in most cases.
Both vapoursynth-dfttest and dfttest scale parameters to the bit depth so it wasn't a problem.


so, parameter for those now float right? (if they not already), since float more flexible, For example if the parameter 0-255 with float we can set it like 112.5 and so, but with int we have only 256 options and that not suitable for more than 8bit

I haven't decided what to do next, but I may start with a few. mvtools-sf is also a good candidate. Depending on my time availability.

since mvtools already has HBD (still there are no float motion analyzing) and it's complex maybe it's good to start with basic things that has no HBD yet in avs+ like VagueDenoiser, EEDI3 and Dither_resize16/fmtconv and then back to mvtools in the end

[MeteorRain]

Which parameters are you talking about? DFTTest has all parameters in float I believe (except for those modes, sizes and frame numbers).

And I've uploaded the release (r7).

[real.finder]

Which parameters are you talking about? DFTTest has all parameters in float I believe (except for those modes, sizes and frame numbers).

if they are, then no need to do anything :) I am just saying if any (Whether here or not)

[tormento]

And I've uploaded the release (r7).
I saw your readme.md is pointing to official FFTW download page.

Unfortunately they built only 3.3.5 version of dll, while there are many updated ones such as 3.3.7 or 3.3.8. The latest I can find is here (https://forum.doom9.org/showthread.php?p=1907464#post1907464) and a fresh compile from you or pinterf with VS and clang (and perhaps Intel) to test would be nice.

[real.finder]

is [Neo] FFT3D / DFTTest stable yet? cuz I want to make scripts use them instead of non-neo one

also as I said in #22 more ports from VS will be nice since avs+ still need those updates for HBD, so I will make another wave of scripts updates :)

[MeteorRain]

Whether it is stable depends on your bug report. If you're happy about it then I think it's stable enough.

What are the things that you are looking to replace in your script from VS counterparts? Those 3 you mentioned?

[real.finder]

Whether it is stable depends on your bug report. If you're happy about it then I think it's stable enough.

What are the things that you are looking to replace in your script from VS counterparts? Those 3 you mentioned?

yes and along with SangNomMod those the most important for now, but there are others with kinda less important like TTempSmooth, EEDI2, TCanny (there are TCannymod by chikuzen but I think with no HBD), Yadifmod (there are Yadifmod2 by chikuzen with not completed HBD), IT, DeblockPP7

and after that maybe these VS only like Bwdif, W3FDIF, VMAF, BM3D

[StvG]

ffvideosource()
neo_ff3d() #r3 crash with no message, r1v4 is ok (x64)

[Myrsloik]

I believe the change of bit depth should not change a parameter, at least in most cases.
Both vapoursynth-dfttest and dfttest scale parameters to the bit depth so it wasn't a problem.

I haven't decided what to do next, but I may start with a few. mvtools-sf is also a good candidate. Depending on my time availability.

This is one of the cases where you can argue that it should change. We're dealing with transforms so there's a direct connection between input range and transformed range. Just my thoughts. I suppose scaling is easier but then a 0-255 range makes no sense. Either have a proper 0-1 float or make the int range a percentage or whatever.

Don't propagate 0-255 scaled ranges, they make me very unhappy since they make no sense AND lack fine enough steps to for higher bitdepths.

[MeteorRain]

ffvideosource()
neo_ff3d() #r3 crash with no message, r1v4 is ok (x64)

Can't reproduce here.

Source file resolution? (Tested 1280x720)
Source bit depth? (Tested 8-bit)
FFMS version? (Mine stamped at 2019-8-11)
Anything that I might have missed?

[StvG]

Can't reproduce here.

Source file resolution? (Tested 1280x720)
Source bit depth? (Tested 8-bit)
FFMS version? (Mine stamped at 2019-8-11)
Anything that I might have missed?
Tested with 1280x720 / 1920x1080 8-bit video, blankclip(pixel_type="yv12").
Added ConvertBits(16) before neo_fft3d() doesn't change the situation.
Tested with this FFMS2 (https://forum.doom9.org/showthread.php?p=1906345#post1906345), latest LSMASH (https://github.com/HolyWu/L-SMASH-Works).
AVSMeter - https://images2.imgbox.com/e6/ba/oz2BaBV8_o.png
AVX-512 CPU.

[MeteorRain]

Oh avx 512. Yea I don't have any testing environment. Can you try debugging it for me?

(Or I can try to use a CPU emulator or rent a server to debug it. Or I can simply remove it since it's not such a big deal to support it now.)

[MeteorRain]

Should have fixed it. The issue is more stupid than I thought.

Please try v4.

Also opt parameter is introduced so you can disable AVX512 manually.

[StvG]

Thanks. It works now.
The output of opt=1 and opt=2 is identical.
The output of opt=3 and opt=4 is identical.
The output of opt=1/2 is a bit different than opt=3/4.

[MeteorRain]

Do you have a combination of resolution-depth-parameters that I can test?

By the way I did find some issues when running 2 instances together. I'll check that part later.

[StvG]

ColorBars(pixel_type="yv12") #whatever pixel_type
a=neo_fft3d(opt=3)
neo_fft3d(opt=2)
Compare(a,"yuv")
Live action video would have bigger difference.

[MeteorRain]

Please try r5 as I fixed the race condition when multiple instances are running together.

[StvG]

The difference is still there.

[kedautinh12]

The difference is still there.

can you comment more detail?

[pinterf]

Usually I'm using this comparison at the end of my scripts. It may show if difference has geometric pattern.

...
StackVertical(old, new, Diff(old, new))
return last

Function Diff(clip src1, clip src2)
{
return Subtract(src1.ConvertBits(8),src2.ConvertBits(8)).Levels(120, 1, 255-120, 0, 255, coring=false)
}

[MeteorRain]

Thanks for reporting, there's an issue with degrid on. The issue was introduced in r3. I should change for all 3 SIMD files but ended up only changing 1 place, leaving 2 files with incorrect parameters.

I'm uploading r6.

[kedautinh12]

is [Neo] FFT3D / DFTTest stable yet? cuz I want to make scripts use them instead of non-neo one

also as I said in #22 more ports from VS will be nice since avs+ still need those updates for HBD, so I will make another wave of scripts updates :)

I think it's stable now, can you add your scripts??

[real.finder]

I think it's stable now, can you add your scripts??

I am already plan for this next update, but I was waiting for more plugins to did MCTD update first since it's not work with HBD at all (many plugins need updates)

[almosely]

I just found, that there's this NEO-version of FFT3DFilter - very nice! :-) Although a test is waiting ... Question: Is FFTW still needed for this NEO-version? I have FFTW 3.3.8 (AVX) (Wolfberry's compile from 29-01-2019) already installed and in use since then. I just realized that AvsPmod GPo has gotten very slow, since I have updated to the newest AviSynth+ Version 3.5.1 and also updated my Visual C++ Redists 2015-2019 to the latest. Now I am checking everything, I use in combination with AviSynth+ and update all of it - maybe, the slowdown is gone after that.

Ahm, I can vaguely remember, that there was a bug in pinterf's FFT3DFilter (maybe it has been there since the beginning, as there was this luma-bug too): I compared the outcome of RgTools's RemoveGrain vs. FFT3DFilter (with chroma filtering), both on and off, with Histogram("color"/"color2") and there were differences when FFT3DFilter was in the filter chain, although turned off. With RemoveGrain, there was no difference to the unfiltered source. But I noticed that a year ago and thought, that I just use RemoveGrain for chroma filtering and not to bother pinterf a second time ;-) Maybe someone else noticed that too and maybe that misbehaviour is resolved in the NEO-version?

[MeteorRain]

almosely:

I'm having difficulties reading your words. Let's split it into parts and you can clarify if I misunderstood.

> Is FFTW still needed for this NEO-version?

Yes. In fact, FFTW is required for all plugins that needs regular FFT transform. FFTW would be the fastest and easiest way to do it.

Of course if it's a GPU plugin than it'll need something else.

> when FFT3DFilter was in the filter chain, although turned off

What do you mean turned off? Does that mean you apply fft3dfilter on a clip but did not use the result? In which part did you notice differences?

[almosely]

Ok, then I leave FFTW where it is ;-) Thank you.

I made a new test regarding chroma filtering, although I realized that I had a thinking error (RemoveGrain cannot be turned off, of course). But I can put FFT3DFilter or neo_fft3d into the script and turn the sigma off (setting it to 0.1 = filter is off). When I do that with luma filtering only, then there is absolutely no difference between the original material and the one with either one of the two fft3d filters (in Histogram=luma), as it should be. But when I do that with UV-filtering turned on, then I can see a difference, although there should not be any difference. I made 3 screenshots.

This happens not, when I leave the chroma settings at 2 (y=3,u=2,v=2 neo) or 0 (plane, fft3dfilter). But it happens, when I only filter the chroma (y=2,u=3,v=3 neo / plane=3 fft3dfilter) or all of it (all=3 neo / plane=4 fft3dfilter).

And another question: What is the difference between 1 and 2 (y,u,v)?

And I also installed the newest version of AvsPmod GPo. The first thing, that came up, was an error-message, that Shibatch.dll is installed twice (but it's only within the plugins+ and plugins64+ folders once, each). That one is "btw". But the next one is regarding your neo_fft3d filter. AvsPmod GPo also spit out an error message, that "neo-fft3d" is containing irregular letters. I muted that message for the next times, sadly. But - IIRC - there was a third message, telling me, that the "-" is the villain and it should be a "_" or anything else to be correct. So I renamed your .dll to "neo_fft3d.dll".

My first benchmarks between neo_fft3d and FFT3DFilter resulted in a speed gain of about 9% (only luma filtering) and about 17% with all planes filtered (compression tests of 720p material with other filters turned on, as I usually do). Nice! :-)

And another thing I am wondering about. With opt=0 I got exactly the same results as with opt=2. But with opt=3 the x264 results were a bit different all over the place (bitrate, I-,P-,B-frame sizes and counts etc.). I have an Intel Core i7-3770 (with AVX). Shouldn't the results be all the same?

opt=0

[2020-05-03][02:20:13] y4m [info]: 1280x720p 1:1 @ 30/1 fps (cfr)
[2020-05-03][02:20:13] x264 [info]: using SAR=1/1
[2020-05-03][02:20:13] x264 [info]: using cpu capabilities: MMX2 SSE2Fast SSSE3 SSE4.2 AVX
[2020-05-03][02:20:13] x264 [info]: profile High, level 4.0, 4:2:0, 8-bit
[2020-05-03][02:22:13] x264 [info]: frame I:23 Avg QP:15.27 size: 60969
[2020-05-03][02:22:13] x264 [info]: frame P:372 Avg QP:19.54 size: 21633
[2020-05-03][02:22:13] x264 [info]: frame B:1125 Avg QP:23.53 size: 7449
[2020-05-03][02:22:13] x264 [info]: consecutive B-frames: 2.2% 2.5% 11.1% 48.9% 21.1% 14.2%
[2020-05-03][02:22:13] x264 [info]: mb I I16..4: 14.4% 69.4% 16.2%
[2020-05-03][02:22:13] x264 [info]: mb P I16..4: 4.9% 20.2% 2.0% P16..4: 33.5% 12.5% 7.4% 0.2% 0.0% skip:19.2%
[2020-05-03][02:22:13] x264 [info]: mb B I16..4: 0.6% 3.1% 0.2% B16..8: 29.6% 8.9% 2.3% direct: 3.8% skip:51.5% L0:45.4% L1:46.8% BI: 7.9%
[2020-05-03][02:22:13] x264 [info]: 8x8 transform intra:74.7% inter:72.8%
[2020-05-03][02:22:13] x264 [info]: direct mvs spatial:99.6% temporal:0.4%
[2020-05-03][02:22:13] x264 [info]: coded y,uvDC,uvAC intra: 66.7% 75.0% 44.9% inter: 11.0% 12.6% 1.9%
[2020-05-03][02:22:13] x264 [info]: i16 v,h,dc,p: 18% 12% 4% 67%
[2020-05-03][02:22:13] x264 [info]: i8 v,h,dc,ddl,ddr,vr,hd,vl,hu: 21% 15% 7% 7% 8% 11% 9% 11% 11%
[2020-05-03][02:22:13] x264 [info]: i4 v,h,dc,ddl,ddr,vr,hd,vl,hu: 18% 13% 3% 8% 14% 14% 13% 8% 8%
[2020-05-03][02:22:13] x264 [info]: i8c dc,h,v,p: 37% 21% 22% 20%
[2020-05-03][02:22:13] x264 [info]: Weighted P-Frames: Y:7.3% UV:5.1%
[2020-05-03][02:22:13] x264 [info]: ref P L0: 59.7% 6.0% 18.4% 7.7% 7.9% 0.2%
[2020-05-03][02:22:13] x264 [info]: ref B L0: 87.6% 8.4% 2.8% 1.2%
[2020-05-03][02:22:13] x264 [info]: ref B L1: 96.6% 3.4%
[2020-05-03][02:22:13] x264 [info]: kb/s:2815.29
[2020-05-03][02:22:13] encoded 1520 frames, 12.13 fps, 2815.42 kb/s

opt=2

[2020-05-03][02:22:20] y4m [info]: 1280x720p 1:1 @ 30/1 fps (cfr)
[2020-05-03][02:22:20] x264 [info]: using SAR=1/1
[2020-05-03][02:22:20] x264 [info]: using cpu capabilities: MMX2 SSE2Fast SSSE3 SSE4.2 AVX
[2020-05-03][02:22:20] x264 [info]: profile High, level 4.0, 4:2:0, 8-bit
[2020-05-03][02:24:21] x264 [info]: frame I:23 Avg QP:15.27 size: 60969
[2020-05-03][02:24:21] x264 [info]: frame P:372 Avg QP:19.54 size: 21633
[2020-05-03][02:24:21] x264 [info]: frame B:1125 Avg QP:23.53 size: 7449
[2020-05-03][02:24:21] x264 [info]: consecutive B-frames: 2.2% 2.5% 11.1% 48.9% 21.1% 14.2%
[2020-05-03][02:24:21] x264 [info]: mb I I16..4: 14.4% 69.4% 16.2%
[2020-05-03][02:24:21] x264 [info]: mb P I16..4: 4.9% 20.2% 2.0% P16..4: 33.5% 12.5% 7.4% 0.2% 0.0% skip:19.2%
[2020-05-03][02:24:21] x264 [info]: mb B I16..4: 0.6% 3.1% 0.2% B16..8: 29.6% 8.9% 2.3% direct: 3.8% skip:51.5% L0:45.4% L1:46.8% BI: 7.9%
[2020-05-03][02:24:21] x264 [info]: 8x8 transform intra:74.7% inter:72.8%
[2020-05-03][02:24:21] x264 [info]: direct mvs spatial:99.6% temporal:0.4%
[2020-05-03][02:24:21] x264 [info]: coded y,uvDC,uvAC intra: 66.7% 75.0% 44.9% inter: 11.0% 12.6% 1.9%
[2020-05-03][02:24:21] x264 [info]: i16 v,h,dc,p: 18% 12% 4% 67%
[2020-05-03][02:24:21] x264 [info]: i8 v,h,dc,ddl,ddr,vr,hd,vl,hu: 21% 15% 7% 7% 8% 11% 9% 11% 11%
[2020-05-03][02:24:21] x264 [info]: i4 v,h,dc,ddl,ddr,vr,hd,vl,hu: 18% 13% 3% 8% 14% 14% 13% 8% 8%
[2020-05-03][02:24:21] x264 [info]: i8c dc,h,v,p: 37% 21% 22% 20%
[2020-05-03][02:24:21] x264 [info]: Weighted P-Frames: Y:7.3% UV:5.1%
[2020-05-03][02:24:21] x264 [info]: ref P L0: 59.7% 6.0% 18.4% 7.7% 7.9% 0.2%
[2020-05-03][02:24:21] x264 [info]: ref B L0: 87.6% 8.4% 2.8% 1.2%
[2020-05-03][02:24:21] x264 [info]: ref B L1: 96.6% 3.4%
[2020-05-03][02:24:21] x264 [info]: kb/s:2815.29
[2020-05-03][02:24:21] encoded 1520 frames, 12.03 fps, 2815.42 kb/s

opt=3

[2020-05-03][02:24:27] y4m [info]: 1280x720p 1:1 @ 30/1 fps (cfr)
[2020-05-03][02:24:27] x264 [info]: using SAR=1/1
[2020-05-03][02:24:27] x264 [info]: using cpu capabilities: MMX2 SSE2Fast SSSE3 SSE4.2 AVX
[2020-05-03][02:24:27] x264 [info]: profile High, level 4.0, 4:2:0, 8-bit
[2020-05-03][02:26:28] x264 [info]: frame I:23 Avg QP:15.26 size: 60963
[2020-05-03][02:26:28] x264 [info]: frame P:372 Avg QP:19.56 size: 21631
[2020-05-03][02:26:28] x264 [info]: frame B:1125 Avg QP:23.52 size: 7453
[2020-05-03][02:26:28] x264 [info]: consecutive B-frames: 2.2% 2.5% 11.1% 48.9% 21.1% 14.2%
[2020-05-03][02:26:28] x264 [info]: mb I I16..4: 14.5% 69.3% 16.2%
[2020-05-03][02:26:28] x264 [info]: mb P I16..4: 5.0% 20.2% 2.0% P16..4: 33.6% 12.5% 7.4% 0.2% 0.0% skip:19.2%
[2020-05-03][02:26:28] x264 [info]: mb B I16..4: 0.6% 3.2% 0.2% B16..8: 29.6% 8.8% 2.3% direct: 3.8% skip:51.4% L0:45.5% L1:46.7% BI: 7.9%
[2020-05-03][02:26:28] x264 [info]: 8x8 transform intra:74.7% inter:72.7%
[2020-05-03][02:26:28] x264 [info]: direct mvs spatial:99.6% temporal:0.4%
[2020-05-03][02:26:28] x264 [info]: coded y,uvDC,uvAC intra: 66.7% 75.0% 44.8% inter: 11.0% 12.6% 1.9%
[2020-05-03][02:26:28] x264 [info]: i16 v,h,dc,p: 18% 12% 4% 67%
[2020-05-03][02:26:28] x264 [info]: i8 v,h,dc,ddl,ddr,vr,hd,vl,hu: 21% 15% 7% 7% 8% 11% 9% 11% 11%
[2020-05-03][02:26:28] x264 [info]: i4 v,h,dc,ddl,ddr,vr,hd,vl,hu: 18% 13% 3% 8% 14% 14% 13% 8% 8%
[2020-05-03][02:26:28] x264 [info]: i8c dc,h,v,p: 37% 21% 22% 20%
[2020-05-03][02:26:28] x264 [info]: Weighted P-Frames: Y:7.3% UV:5.1%
[2020-05-03][02:26:28] x264 [info]: ref P L0: 59.6% 6.0% 18.5% 7.7% 8.0% 0.2%
[2020-05-03][02:26:28] x264 [info]: ref B L0: 87.6% 8.4% 2.8% 1.2%
[2020-05-03][02:26:28] x264 [info]: ref B L1: 96.7% 3.3%
[2020-05-03][02:26:28] x264 [info]: kb/s:2815.81
[2020-05-03][02:26:28] encoded 1520 frames, 12.10 fps, 2815.93 kb/s

[MeteorRain]

> turn the sigma off (setting it to 0.1 = filter is off)

Bug confirmed. I'll investigate what happened.

> What is the difference between 1 and 2 (y,u,v)?

2 = copy, 1 = garbage data.

> But with opt=3 the x264 results were a bit different

Make sure you have r6 which should have fixed the issue.

[almosely]

Nice again! :-) It would be fine to use only one filter for luma and croma, I think (even so RemoveGrain is fast as hell).

I still don't get it. What means "garbage data" or "Do not touch, leaving garbage data". But, all I have to really know is: When I only want to filter luma, I use y=3, u=2, v=2 or shall I use y=3, u=1, v=1? I guess 3-2-2 is the way to go?

Hm, I do have r6 installed :-(

[real.finder]

Nice again! :-) It would be fine to use only one filter for luma and croma, I think (even so RemoveGrain is fast as hell).

I still don't get it. What means "garbage data" or "Do not touch, leaving garbage data". But, all I have to really know is: When I only want to filter luma, I use y=3, u=2, v=2 or shall I use y=3, u=1, v=1? I guess 3-2-2 is the way to go?

Hm, I do have r6 installed :-(

"garbage data" or don't care (see mode=-1 in RemoveGrain or y,u,v =1 in masktools) is the fastest option useful for scripts maker/editer