[Neo] FFT3D / DFTTest

[almosely]

A test with RemoveGrain -1,u,v translated the effects to my brain :-) -1 removes the channel completely from the further script progress. So 2 is the way to go for me, when I only want to filter luma (3-2-2).

But it seems since I have updated to the newest AviSynth+ and MSVC++ 15-19, something is not working right anymore. AvsPMod GPu is slow (the old was and the new version is too), even when I switch the values of RemoveGrain, when nothing else is within the script. And sometimes RemoveGrain mode -1 removed the luma channel, and sometimes not - like a bitchy programm. Weird. And I never noticed a red cross in the upper right corner of AvsPMod GPu before the update to the new AviSynth+ and MSVC++ - now I see it almost every time I change a filter value. I know, thats the wrong thread, but I can't help myself to mention that. Sadly I deleted the old avisinth.dlls and devil.dlls from version 3.4.0, which I had installed before.

- edit -

just downloaded the portable version of AviSynth+ 3.4.0 and will test with that.

[MeteorRain]

almosely:

Do not attach images like this. You can use a public image hosting service such as imgur. Upload your picture to imgur and paste the image link here.

If you have r6 and still have issues with opt=3 and opt=2 please give me a minimal sample script that you can reproduce the issue.

garbage data means it can return anything and you should not use the data from that plane. For (y=3, u=2, v=1), y plane is filtered, u plane is copied from source, v plane contains garbage and you should not consider v plane as a valid image.
Whether use 2 or 1 depends on your actual script.

[almosely]

Ok, now I got the meaning of "-1" - that explains the "bitchy" ;-)

I will use imgur.com the next times.

I made another test, with almost the same script (only the BlackmanResize has different values) and another clip. Now I got 3 different results ...

opt=0

[2020-05-03][05:26:36] y4m [info]: 1280x536p 1:1 @ 24000/1001 fps (cfr)
[2020-05-03][05:26:36] x264 [info]: using SAR=1/1
[2020-05-03][05:26:36] x264 [info]: using cpu capabilities: MMX2 SSE2Fast SSSE3 SSE4.2 AVX
[2020-05-03][05:26:36] x264 [info]: profile High, level 4.0, 4:2:0, 8-bit
[2020-05-03][05:28:30] x264 [info]: frame I:65 Avg QP:15.61 size:101225
[2020-05-03][05:28:30] x264 [info]: frame P:548 Avg QP:20.40 size: 26703
[2020-05-03][05:28:30] x264 [info]: frame B:1387 Avg QP:23.77 size: 8405
[2020-05-03][05:28:30] x264 [info]: consecutive B-frames: 5.5% 5.6% 14.1% 59.8% 7.2% 7.8%
[2020-05-03][05:28:30] x264 [info]: mb I I16..4: 4.0% 70.9% 25.1%
[2020-05-03][05:28:30] x264 [info]: mb P I16..4: 1.7% 17.6% 3.4% P16..4: 36.0% 16.7% 14.0% 1.0% 0.3% skip: 9.3%
[2020-05-03][05:28:30] x264 [info]: mb B I16..4: 0.3% 3.6% 0.7% B16..8: 36.3% 9.1% 2.5% direct: 3.9% skip:43.6% L0:38.6% L1:43.1% BI:18.3%
[2020-05-03][05:28:30] x264 [info]: 8x8 transform intra:76.4% inter:62.4%
[2020-05-03][05:28:30] x264 [info]: direct mvs spatial:99.6% temporal:0.4%
[2020-05-03][05:28:30] x264 [info]: coded y,uvDC,uvAC intra: 85.4% 86.9% 59.8% inter: 16.9% 17.8% 3.0%
[2020-05-03][05:28:30] x264 [info]: i16 v,h,dc,p: 22% 7% 6% 64%
[2020-05-03][05:28:30] x264 [info]: i8 v,h,dc,ddl,ddr,vr,hd,vl,hu: 25% 9% 7% 7% 9% 12% 9% 12% 11%
[2020-05-03][05:28:30] x264 [info]: i4 v,h,dc,ddl,ddr,vr,hd,vl,hu: 24% 5% 3% 8% 13% 16% 9% 12% 8%
[2020-05-03][05:28:30] x264 [info]: i8c dc,h,v,p: 36% 18% 27% 18%
[2020-05-03][05:28:30] x264 [info]: Weighted P-Frames: Y:9.1% UV:5.1%
[2020-05-03][05:28:30] x264 [info]: ref P L0: 62.4% 14.0% 15.2% 4.9% 3.2% 0.3% 0.0%
[2020-05-03][05:28:30] x264 [info]: ref B L0: 88.5% 8.3% 2.6% 0.6%
[2020-05-03][05:28:30] x264 [info]: ref B L1: 96.5% 3.5%
[2020-05-03][05:28:30] x264 [info]: kb/s:3152.49
[2020-05-03][05:28:30] encoded 2000 frames, 17.02 fps, 3152.57 kb/s

opt=2

[2020-05-03][05:28:35] y4m [info]: 1280x536p 1:1 @ 24000/1001 fps (cfr)
[2020-05-03][05:28:35] x264 [info]: using SAR=1/1
[2020-05-03][05:28:35] x264 [info]: using cpu capabilities: MMX2 SSE2Fast SSSE3 SSE4.2 AVX
[2020-05-03][05:28:35] x264 [info]: profile High, level 4.0, 4:2:0, 8-bit
[2020-05-03][05:30:29] x264 [info]: frame I:65 Avg QP:15.61 size:101210
[2020-05-03][05:30:29] x264 [info]: frame P:548 Avg QP:20.40 size: 26700
[2020-05-03][05:30:29] x264 [info]: frame B:1387 Avg QP:23.81 size: 8408
[2020-05-03][05:30:29] x264 [info]: consecutive B-frames: 5.5% 5.6% 14.2% 59.4% 7.5% 7.8%
[2020-05-03][05:30:29] x264 [info]: mb I I16..4: 4.0% 70.9% 25.1%
[2020-05-03][05:30:29] x264 [info]: mb P I16..4: 1.7% 17.6% 3.4% P16..4: 36.0% 16.7% 14.0% 1.0% 0.3% skip: 9.3%
[2020-05-03][05:30:29] x264 [info]: mb B I16..4: 0.2% 3.6% 0.7% B16..8: 36.4% 9.2% 2.5% direct: 3.9% skip:43.6% L0:38.6% L1:43.1% BI:18.3%
[2020-05-03][05:30:29] x264 [info]: 8x8 transform intra:76.4% inter:62.3%
[2020-05-03][05:30:29] x264 [info]: direct mvs spatial:99.6% temporal:0.4%
[2020-05-03][05:30:29] x264 [info]: coded y,uvDC,uvAC intra: 85.4% 86.9% 59.8% inter: 16.9% 17.8% 3.0%
[2020-05-03][05:30:29] x264 [info]: i16 v,h,dc,p: 22% 7% 7% 64%
[2020-05-03][05:30:29] x264 [info]: i8 v,h,dc,ddl,ddr,vr,hd,vl,hu: 25% 9% 7% 6% 9% 12% 9% 12% 11%
[2020-05-03][05:30:29] x264 [info]: i4 v,h,dc,ddl,ddr,vr,hd,vl,hu: 24% 5% 4% 8% 13% 16% 9% 12% 8%
[2020-05-03][05:30:29] x264 [info]: i8c dc,h,v,p: 37% 18% 27% 18%
[2020-05-03][05:30:29] x264 [info]: Weighted P-Frames: Y:9.1% UV:5.1%
[2020-05-03][05:30:29] x264 [info]: ref P L0: 62.4% 14.0% 15.3% 4.9% 3.2% 0.3% 0.0%
[2020-05-03][05:30:29] x264 [info]: ref B L0: 88.5% 8.2% 2.6% 0.6%
[2020-05-03][05:30:29] x264 [info]: ref B L1: 96.5% 3.5%
[2020-05-03][05:30:29] x264 [info]: kb/s:3152.61
[2020-05-03][05:30:29] encoded 2000 frames, 17.00 fps, 3152.69 kb/s

opt=3

[2020-05-03][05:30:35] y4m [info]: 1280x536p 1:1 @ 24000/1001 fps (cfr)
[2020-05-03][05:30:35] x264 [info]: using SAR=1/1
[2020-05-03][05:30:35] x264 [info]: using cpu capabilities: MMX2 SSE2Fast SSSE3 SSE4.2 AVX
[2020-05-03][05:30:35] x264 [info]: profile High, level 4.0, 4:2:0, 8-bit
[2020-05-03][05:32:28] x264 [info]: frame I:65 Avg QP:15.62 size:101210
[2020-05-03][05:32:28] x264 [info]: frame P:548 Avg QP:20.39 size: 26706
[2020-05-03][05:32:28] x264 [info]: frame B:1387 Avg QP:23.78 size: 8408
[2020-05-03][05:32:28] x264 [info]: consecutive B-frames: 5.5% 5.6% 14.1% 59.8% 7.2% 7.8%
[2020-05-03][05:32:28] x264 [info]: mb I I16..4: 4.1% 71.0% 25.0%
[2020-05-03][05:32:28] x264 [info]: mb P I16..4: 1.7% 17.6% 3.4% P16..4: 36.0% 16.7% 14.0% 1.0% 0.3% skip: 9.3%
[2020-05-03][05:32:28] x264 [info]: mb B I16..4: 0.2% 3.6% 0.7% B16..8: 36.3% 9.1% 2.5% direct: 4.0% skip:43.6% L0:38.6% L1:43.2% BI:18.3%
[2020-05-03][05:32:28] x264 [info]: 8x8 transform intra:76.4% inter:62.3%
[2020-05-03][05:32:28] x264 [info]: direct mvs spatial:99.6% temporal:0.4%
[2020-05-03][05:32:28] x264 [info]: coded y,uvDC,uvAC intra: 85.4% 86.8% 59.7% inter: 16.9% 17.9% 3.0%
[2020-05-03][05:32:28] x264 [info]: i16 v,h,dc,p: 22% 7% 7% 64%
[2020-05-03][05:32:28] x264 [info]: i8 v,h,dc,ddl,ddr,vr,hd,vl,hu: 25% 9% 7% 7% 9% 12% 9% 12% 11%
[2020-05-03][05:32:28] x264 [info]: i4 v,h,dc,ddl,ddr,vr,hd,vl,hu: 24% 5% 3% 8% 13% 16% 9% 12% 8%
[2020-05-03][05:32:28] x264 [info]: i8c dc,h,v,p: 36% 18% 27% 18%
[2020-05-03][05:32:28] x264 [info]: Weighted P-Frames: Y:9.1% UV:5.1%
[2020-05-03][05:32:28] x264 [info]: ref P L0: 62.4% 14.0% 15.2% 4.9% 3.2% 0.3% 0.0%
[2020-05-03][05:32:28] x264 [info]: ref B L0: 88.5% 8.3% 2.6% 0.6%
[2020-05-03][05:32:28] x264 [info]: ref B L1: 96.6% 3.4%
[2020-05-03][05:32:28] x264 [info]: kb/s:3152.91
[2020-05-03][05:32:28] encoded 2000 frames, 17.06 fps, 3152.98 kb/s

The script

DGSource("test v2.dgi", crop_l=0, crop_r=0, crop_t=0, crop_b=0, deinterlace=0, use_top_field=true, use_pf=true)

CompTest(5, 40)

RequestLinear(rlim=50, clim=50)

BlackmanResize(1280, 536, taps=9)

neo_fft3d(sigma=1.0, beta=1.0, bw=48, bh=48, sharpen=0.040, scutoff=0.01, y=3, u=2, v=2, bt=5, opt=0)

GradFun3(thr=0.35)

avstp_set_threads(1)

Prefetch(3)

return last

The CompTest-Script (.avsi)

function CompTest(clip c, float percent, float frames)
{
frange = floor((frames/1.)/(percent/100.))
return SelectRangeEvery(c, frange, int(frames))
}

- edit -

Ha! Could it be the mtmodes.avsi? I did not make any new entry for neo_fft3d ... Should be like ...

SetFilterMTMode("neo_fft3d", MT_MULTI_INSTANCE)

.. this could explain it, couldn't it?

[MeteorRain]

Don't do set filter mt mode.

Can you please give me a sample script like the one below?

some_source
some_preprocessing

v1 = neo_fft3d(some parameters, opt=1)
v2 = neo_fft3d(some parameters, opt=2)
compare(v1, v2)


Make sure you read non-identical result. Try to give me a minimal script so I can narrow down issue easily.

EDIT:

To narrow down further, please tell me whether you can reproduce it under non-MT mode. Is the problem specific to MT mode only?

[almosely]

Is this different (enough)?

https://i.imgur.com/5q09cfQ.png

DGSource("test v3.dgi", crop_l=0, crop_r=0, crop_t=0, crop_b=0, deinterlace=0, use_top_field=true, use_pf=true)

RequestLinear(rlim=50, clim=50)

BlackmanResize(640, 360, taps=16)

v1 = neo_fft3d(sigma=1.0, beta=1.0, bw=48, bh=48, sharpen=0.040, scutoff=0.01, y=3, u=2, v=2, bt=5, opt=1)
v2 = neo_fft3d(sigma=1.0, beta=1.0, bw=48, bh=48, sharpen=0.040, scutoff=0.01, y=3, u=2, v=2, bt=5, opt=2)
compare(v1, v2)

avstp_set_threads(1)

Prefetch(3)

return last

- edit -

There's no difference between ST and MT mode with this (I mean: There are still these differences like in the picture above).

# avstp_set_threads(1)

# Prefetch(3)

[MeteorRain]

Thanks.

3 issues were fixed.

1) When input is Y8, the engine was incorrectly skipped.

2) Chroma processing had incorrect negative value float to int rounding.
For positive values, the rounding was correct.
For example, x = 2.1, int(x + 0.5) = 2.
However for a negative value, x = -2.1, int(x + 0.5) -> int(-1.6) -> -1.
Since chroma is processed as signed integer, negative values were incorrectly rounded.

3) bt=5 had a rounding error between C and SIMD code.
Basically in SIMD I was doing result=(a+b)+(c+d)+e while in C code it's result=a+b+c+d+e, resulting a minor rounding error.
Now the C code is the same as SIMD code, and is different to fft3dfilter.

Please try r7.

[almosely]

Yeehaa! Thank you very much :-) I can't see any difference with the last script anymore.

... but there's still a minor difference regarding chroma:

https://i.imgur.com/OwlRvJd.png
https://i.imgur.com/kilLcML.png

[MeteorRain]

Try sigma=0.001.

[almosely]

What a magical mystery! ;-))) ... tried, worked ... set the value-range-change back to 0.1 after that test. Thank you again!

[StvG]

v1 = neo_fft3d(sigma=1.0, beta=1.0, bw=48, bh=48, sharpen=0.040, scutoff=0.01, y=3, u=2, v=2, bt=5, opt=4)
v2 = neo_fft3d(sigma=1.0, beta=1.0, bw=48, bh=48, sharpen=0.040, scutoff=0.01, y=3, u=2, v=2, bt=5, opt=2)
compare(v1, v2)
There is difference with r7.

[MeteorRain]

StvG: I don't have AVX512 and every time I test with AVX512 I have to rent a server. Can you confirm if the difference only appears on opt=4? What bit depth?

[almosely]

I made a test with prefetch(3) and (5) and got different results. Now I am wondering, if that's a normal behaviour (and has always been like that for all my scripts, but I did'nt realize) or if that's because of neo_fft3d.

script(s) for prefetch-test

DGSource("test.dgi", crop_l=0, crop_r=0, crop_t=0, crop_b=0, deinterlace=0, use_top_field=true, use_pf=true)

RequestLinear(rlim=100, clim=100)

neo_fft3d(sigma=1.0, beta=1.0, bw=48, bh=48, sharpen=0.040, scutoff=0.01, y=3, u=2, v=2, bt=5, opt=3)

avstp_set_threads(1)

Prefetch(3)

# Prefetch(5)

return last

x264 result for prefetch(3)

[2020-05-03][18:59:26] y4m [info]: 1280x720p 1:1 @ 30000/1001 fps (cfr)
[2020-05-03][18:59:26] x264 [info]: using SAR=1/1
[2020-05-03][18:59:26] x264 [info]: using cpu capabilities: MMX2 SSE2Fast SSSE3 SSE4.2 AVX
[2020-05-03][18:59:26] x264 [info]: profile High, level 4.0, 4:2:0, 8-bit
[2020-05-03][19:06:29] x264 [info]: frame I:33 Avg QP:14.82 size:164557
[2020-05-03][19:06:29] x264 [info]: frame P:1785 Avg QP:18.28 size: 28082
[2020-05-03][19:06:29] x264 [info]: frame B:7078 Avg QP:26.95 size: 2907
[2020-05-03][19:06:29] x264 [info]: consecutive B-frames: 1.5% 1.5% 5.1% 6.4% 18.9% 66.6%
[2020-05-03][19:06:29] x264 [info]: mb I I16..4: 3.6% 55.1% 41.3%
[2020-05-03][19:06:29] x264 [info]: mb P I16..4: 0.5% 2.9% 2.3% P16..4: 44.1% 16.1% 16.4% 0.9% 0.4% skip:16.4%
[2020-05-03][19:06:29] x264 [info]: mb B I16..4: 0.0% 0.1% 0.2% B16..8: 34.9% 2.9% 0.7% direct: 0.7% skip:60.5% L0:46.9% L1:44.0% BI: 9.1%
[2020-05-03][19:06:29] x264 [info]: 8x8 transform intra:49.6% inter:31.3%
[2020-05-03][19:06:29] x264 [info]: direct mvs spatial:99.9% temporal:0.1%
[2020-05-03][19:06:29] x264 [info]: coded y,uvDC,uvAC intra: 86.8% 59.0% 33.7% inter: 7.2% 2.9% 0.3%
[2020-05-03][19:06:29] x264 [info]: i16 v,h,dc,p: 7% 53% 12% 28%
[2020-05-03][19:06:29] x264 [info]: i8 v,h,dc,ddl,ddr,vr,hd,vl,hu: 8% 33% 15% 5% 5% 4% 8% 6% 16%
[2020-05-03][19:06:29] x264 [info]: i4 v,h,dc,ddl,ddr,vr,hd,vl,hu: 10% 28% 8% 6% 9% 7% 11% 7% 14%
[2020-05-03][19:06:29] x264 [info]: i8c dc,h,v,p: 40% 38% 14% 9%
[2020-05-03][19:06:29] x264 [info]: Weighted P-Frames: Y:12.2% UV:0.8%
[2020-05-03][19:06:29] x264 [info]: ref P L0: 62.4% 13.9% 18.1% 3.4% 1.9% 0.2% 0.0%
[2020-05-03][19:06:29] x264 [info]: ref B L0: 86.7% 10.4% 2.5% 0.4%
[2020-05-03][19:06:29] x264 [info]: ref B L1: 98.0% 2.0%
[2020-05-03][19:06:29] x264 [info]: kb/s:2051.98
[2020-05-03][19:06:29] encoded 8896 frames, 20.88 fps, 2052.00 kb/s

x264 result for prefetch(5)

[2020-05-03][19:06:33] y4m [info]: 1280x720p 1:1 @ 30000/1001 fps (cfr)
[2020-05-03][19:06:33] x264 [info]: using SAR=1/1
[2020-05-03][19:06:33] x264 [info]: using cpu capabilities: MMX2 SSE2Fast SSSE3 SSE4.2 AVX
[2020-05-03][19:06:33] x264 [info]: profile High, level 4.0, 4:2:0, 8-bit
[2020-05-03][19:14:10] x264 [info]: frame I:33 Avg QP:14.82 size:164580
[2020-05-03][19:14:10] x264 [info]: frame P:1785 Avg QP:18.28 size: 28080
[2020-05-03][19:14:10] x264 [info]: frame B:7078 Avg QP:26.96 size: 2907
[2020-05-03][19:14:10] x264 [info]: consecutive B-frames: 1.5% 1.5% 5.1% 6.4% 18.9% 66.6%
[2020-05-03][19:14:10] x264 [info]: mb I I16..4: 3.4% 55.2% 41.4%
[2020-05-03][19:14:10] x264 [info]: mb P I16..4: 0.5% 2.9% 2.3% P16..4: 44.1% 16.2% 16.3% 0.9% 0.4% skip:16.4%
[2020-05-03][19:14:10] x264 [info]: mb B I16..4: 0.0% 0.1% 0.2% B16..8: 34.9% 2.9% 0.7% direct: 0.7% skip:60.5% L0:46.9% L1:44.0% BI: 9.1%
[2020-05-03][19:14:10] x264 [info]: 8x8 transform intra:49.8% inter:31.3%
[2020-05-03][19:14:10] x264 [info]: direct mvs spatial:99.9% temporal:0.1%
[2020-05-03][19:14:10] x264 [info]: coded y,uvDC,uvAC intra: 86.8% 59.0% 33.8% inter: 7.2% 2.9% 0.3%
[2020-05-03][19:14:10] x264 [info]: i16 v,h,dc,p: 7% 53% 11% 29%
[2020-05-03][19:14:10] x264 [info]: i8 v,h,dc,ddl,ddr,vr,hd,vl,hu: 7% 33% 15% 5% 5% 4% 8% 6% 16%
[2020-05-03][19:14:10] x264 [info]: i4 v,h,dc,ddl,ddr,vr,hd,vl,hu: 10% 29% 8% 6% 9% 7% 11% 7% 14%
[2020-05-03][19:14:10] x264 [info]: i8c dc,h,v,p: 40% 38% 14% 9%
[2020-05-03][19:14:10] x264 [info]: Weighted P-Frames: Y:12.2% UV:0.8%
[2020-05-03][19:14:10] x264 [info]: ref P L0: 62.4% 13.9% 18.1% 3.4% 1.9% 0.2% 0.0%
[2020-05-03][19:14:10] x264 [info]: ref B L0: 86.7% 10.4% 2.5% 0.4%
[2020-05-03][19:14:10] x264 [info]: ref B L1: 98.0% 2.0%
[2020-05-03][19:14:10] x264 [info]: kb/s:2051.85
[2020-05-03][19:14:10] encoded 8896 frames, 19.35 fps, 2051.87 kb/s

compare-script after that

v1 = DGSource("test pf 3.dgi", crop_l=0, crop_r=0, crop_t=0, crop_b=0, deinterlace=0, use_top_field=true, use_pf=true)

v2 = DGSource("test pf 5.dgi", crop_l=0, crop_r=0, crop_t=0, crop_b=0, deinterlace=0, use_top_field=true, use_pf=true)

Compare(v1, v2)

return last

https://i.imgur.com/PGp9fQz.jpg

- edit -

Maybe my encoding-settings are relevant too:

[2020-05-03][18:59:23] Creating encoder process:
[2020-05-03][18:59:23] "...\Simple x264 Launcher\toolset\x64\x264_x64.exe" --output-depth 8 --crf 18.0 --preset medium --tune film --trellis 2 --direct auto --me umh --partitions all --vbv-maxrate 24000 --vbv-bufsize 30000 --b-adapt 2 --bframes 5 --merange 24 --ref 5 --keyint 300 --subme 10 --aq-mode 1 --sar 1:1 --rc-lookahead 40 --output "...\test pf 3.mkv" --frames 8896 --demuxer y4m --stdin y4m -

- edit -

And another question: Should I use opt=0 or opt=3 with my CPU. Is neo_fft3d switching between different settings/routines while one job is running (opt=0) or is it using the same code for the whole job, once the decision is set at the beginning?

https://i.imgur.com/wQUVssc.png

[MeteorRain]

> using the same code for the whole job

This.

opt is only for debugging purpose. Normal users should never need to set this.

> got different results

Spent a few hours on comparing the results.

First thing first, it's not multi-threading related. It's multi-run related, which means it was caused by multiple runs of the script, not because multiple runs have different thread count.

Secondly, I can't figure out the fundamental reason. It seems to me like floating point computing error.

For example,
out[lfp.w][0] = incur[lfp.w][0];
out[lfp.w][1] = incur[lfp.w][1];
This code gives identical result for each run. However,
out[lfp.w][0] = incur[lfp.w][0] * 0.98f;
out[lfp.w][1] = incur[lfp.w][1] * 0.98f;
This code gives different result for each run.

I have no clue what this is. I tried both /fp:precise and /fp:strict if that matters.

[Groucho2004]

avstp_set_threads(1)Do you even know what that does? Are you aware that there's nothing in your script that uses avstp?

[almosely]

@MeteorRain

Okay, I have dropped the parameter "opt" from my call for neo_fft3d.

Do you know that fftw itself is internally multithreaded too? Maybe this is causing it?

@Groucho

Are you 100% sure about that?

avstp is made by the author of dither tools. It handles the internal single-/multi-threading-functionality of the dither tools. Since I regularly use GradFun3, avstp is important to my scripts. If I only use GradFun3, then I turn off multithreading of my script (#prefetch) and don't use "avstp_set_threads" at all (#), so that GradFun3 can work multithreaded internal. If I use some filters which work faster multithreaded in my scripts, then I turn on "my" multithreading (prefetch) and turn off the internal mt of the dither tools (avstp_set_threads(1)), so that they do not interfere each other.

"avstp_set_threads" has to be written at the very end of a script. Therefore I put it straight before the two very last commands of my scripts (prefetch and return) and handle it manually, by myself, to be absolutely in control of "my" mt and the avstp mt - and not within the mtmodes.avsi (I deleted the function within there).

Nobody knows which routine is using avstp.dll for sure, right? Whether dither.avsi nor mt_xxpand_multi.avsi is including "avstp_set_threads" or "avstp". So this setting must be made from the dither.dll - or could; so any other .dll can do this too and make use of avstp.dll. I can check the thread-count with AVSMeter, but do I know 100% if a .dll is working mt or st at everytime? Maybe sometimes the thread-count changes during a job - I simply don't know, how routines are written internally; I can't look inside.

Therefore, setting "avstp_set_threads(1)" is a safety measure everytime I turn on "my" mt with prefetch().

[Groucho2004]

I was referring to the script you posted in post #62. There's no plugin in that script that uses avstp. A far as I'm aware, only Dither.dll and mvtools2.dll use avstp.
In some cases, such as QTGMC, avstp can be beneficial in combination with Avisynth+'s internal MT, you have to experiment yourself.
avstp initializes a thread pool in its constructor (IIRC) which could make things slower even if there's no plugin using avstp. The statement avstp_set_threads(1) does not disable avstp. I did some experiments with it some time ago and its sheer presence in the plugin directory influences the number of threads and the rendering speed, albeit slightly.

Just some food for thought.

[almosely]

"could make things slower even if there's no plugin using avstp. The statement avstp_set_threads(1) does not disable avstp. I did some experiments with it some time ago and its sheer presence in the plugin directory influences the number of threads and the rendering speed"

Oha! Thank you for that advice :-) Then I will figure out a new routine of doing my scripting, probably with "LoadPlugin("xyz.dll")" calls, I don't know.

- edit -

@MeteorRain

I am sorry, I am not a programmer (anymore); I used to be one, but that's 22 years ago. Since then I reorganized my brain to function in other ways, less rational (at least different from that time), more emotional, less mathematic, more optical etc. (studied design, now I am going to make some music again, after about 26 years ago, etc.). So I can't help you with C or something related, sadly.

[Groucho2004]

"could make things slower even if there's no plugin using avstp. The statement avstp_set_threads(1) does not disable avstp. I did some experiments with it some time ago and its sheer presence in the plugin directory influences the number of threads and the rendering speed"

Oha! Thank you for that advice :-) Then I will figure out a new routine of doing my scripting, probably with "LoadPlugin("xyz.dll")" calls, I don't know.Well, mainly my point is to not just dump avstp into your plugin directory, but rather figure out if you really need it.

[almosely]

So, meanwhile I benched AviSynth+ 3.4.0 vs 3.5.1. The difference was negligibly if not accidental. Perhaps my memory is fooling me or it has something to do with the latest MSVC++ 2015-2019 Update - or my recent RAM upgrade from 2x4 GB to 4x4 GB. I don't know and I don't care anymore. AVS+ 3.5.1 will stay installed.

And I made a little test regarding avstp.dll. At first, I put avstp.dll out of the avs-plugins64+ folder into another folder. From there I can sideload it as necessary.

I used the following scripts ...

ST

DGSource("test 1080p.dgi", crop_l=0, crop_r=0, crop_t=0, crop_b=0, deinterlace=0, use_top_field=true, use_pf=true)

Trim(0, 1000)

RequestLinear(rlim=100, clim=100)

GradFun3(thr=0.35)

return last

ST mit avstp.dll

LoadPlugin("...\AviSynth+ sideload\plugins64+\avstp.dll")

DGSource("test 1080p.dgi", crop_l=0, crop_r=0, crop_t=0, crop_b=0, deinterlace=0, use_top_field=true, use_pf=true)

Trim(0, 1000)

RequestLinear(rlim=100, clim=100)

GradFun3(thr=0.35)

return last

MT (pf 4)

DGSource("test 1080p.dgi", crop_l=0, crop_r=0, crop_t=0, crop_b=0, deinterlace=0, use_top_field=true, use_pf=true)

Trim(0, 1000)

RequestLinear(rlim=100, clim=100)

GradFun3(thr=0.35)

Prefetch(4)

return last

MT (pf 4) mit avstp.dll

LoadPlugin("...\AviSynth+ sideload\plugins64+\avstp.dll")

DGSource("test 1080p.dgi", crop_l=0, crop_r=0, crop_t=0, crop_b=0, deinterlace=0, use_top_field=true, use_pf=true)

Trim(0, 1000)

RequestLinear(rlim=100, clim=100)

GradFun3(thr=0.35)

Prefetch(4)

return last

MT (pf 4) mit avstp.dll mit avstp_set_threads(1)

LoadPlugin("...\AviSynth+ sideload\plugins64+\avstp.dll")

DGSource("test 1080p.dgi", crop_l=0, crop_r=0, crop_t=0, crop_b=0, deinterlace=0, use_top_field=true, use_pf=true)

Trim(0, 1000)

RequestLinear(rlim=100, clim=100)

GradFun3(thr=0.35)

avstp_set_threads(1)

Prefetch(4)

return last

https://i.imgur.com/P6HTlne.png

In that specific case I don't need avstp.dll neither single- nor multi-threaded (it only hurts).

So, thanks for the hint again, Groucho ;-)

- edit -

... sorry for the german language (some words) sneaking in ;-))

[MeteorRain]

Seems like bw=48 was the problem. Can you try some different numbers like 32/40/56/64/96/128?

I still haven't figured out what went wrong, but this might be the right approach to the problem.

[almosely]

Hm, I just realized, that the original author set bw=32 as default, after he has set it to 48 at some time. Oddly I remembered that bw=48 would still be the default. I will try some other sizes.

Maybe it has something to do with his statement?


Features and limitations

Filter works only in YV12 or YUY2 color format.
Only specified single color plane may be processed (Y, U or V). Since v1.8.4 it is possible to process several (all) color planes.
Since v1.3 it works with both progressive and interlaced clips.
Tested with Avisynth v2.55, v2.56.
Filter uses fast external FFTW library version 3 (http://www.fftw.org)
as Windows binary DLL (compiled with gcc under MinGW by Alessio Massaro), which support for threads and have AMD K7 (3dNow!) support in addition to SSE/SSE2.
It may be downloaded from ftp://ftp.fftw.org/pub/fftw/fftw3win32mingw.zip
You MUST put FFTW3.DLL file from this package (not fft3dfilter.dll) to some directory in path (for example, C:\WINNT\SYSTEM32).
Filter will NOT work without it!
FFTW works most efficiently for arrays whose size (bw, bh) can be factored into small primes 2, 3, 5, and 7 (and one 11, 13), and otherwise it uses a slower general-purpose routine.
First versions were really slooow! But recent versions are simple slow :-).
Algorithm is improved and assembler 3DNow! and SSE instruction are used for some modes
Algorithm is optimized by speed for forward sequential frames access only.
The sharpening method is experimental, however is quite good since v1.1 (and v1.7).
The noise pattern method is experimental too.
degrid option is not implemented for Kalman filter.
For noisy video it is useful to use median prefilter (for example DeGrainMedian) before FFT3Dfilter.
Multithread mode ncpu>1 is almost not tested (I do not have core duo), try to use ncpu=2 if you have.


- edit -

Now I know why I remember bw=48 as default, grml ...


Function parameters:

first parameter - input clip

sigma - given noise value for all (or highest) frequencies (float>0, default=2.0)
beta - noise margin (float>=1.0, default=1.0 for no noise left):
control noise amount left in the restoration for Wiener filter, so max filter noise attenuation = (beta-1)/beta.
plane - processed color plane: 0 - luma(Y), 1 - chroma U, 2 - chroma V,
3 - chroma planes U and V, 4 - both luma and chroma (default = 0)
bw - block width (integer, default = 48 since v.1.2)
bh - block height (integer, default = 48 since v.1.2)


... but very little at the very bottom of the documentation:


Version changes:
Version 1.9.2 - 11 september 2006 - changed default bw=bh=32; added mode bt=5

[pinterf]

StvG: I don't have AVX512 and every time I test with AVX512 I have to rent a server. Can you confirm if the difference only appears on opt=4? What bit depth?
I was using Intel Emulator (https://software.intel.com/en-us/articles/intel-software-development-emulator) with the following jolly-joker batch file
"%~dp0/sde" -- cmd.exe
@rem "%~dp0/sde" -- cmd.exe
@rem "c:\sde\sde" -mrm -- cmd.exe
@rem "c:\sde\sde" -wsm -- cmd.exe
@rem "c:\sde\sde" -skx -- c:\virtualdub\VirtualDub64.exe
"c:\sde\sde" -skx -- cmd.exe
@rem "c:\sde\sde" -wsm -- s1.bat
@rem -mrm Set chip-check and CPUID for Merom
@rem -pnr Set chip-check and CPUID for Penryn
@rem -nhm Set chip-check and CPUID for Nehalem
@rem -hsw Set chip-check and CPUID for Haswell
@rem -wsm Set chip-check and CPUID for Westmere, no AVX
@rem -skx Set chip-check and CPUID for SkyLakeX. AVX512F and AVX512BW

Of course, not for speed tests :) but for visual tests this tool was good enough and I could even debug from Visual Studio by setting project properties|debugging to sde

[MeteorRain]

SDE causes BSOD on my computer 100% time so I gave up after 2 hard CPU reset in 5 minutes.

[pinterf]

Then I was lucky (i7-7700 - Win10)

[almosely]

Can you try some different numbers like 32/40/56/64/96/128?

Test-Script (avstp.dll not loaded anymore)

DGSource("test.dgi", crop_l=0, crop_r=0, crop_t=0, crop_b=0, deinterlace=0, use_top_field=true, use_pf=true)

RequestLinear(rlim=50, clim=50)

neo_fft3d(sigma=1.0, beta=1.0, bw=32, bh=32, sharpen=0.040, scutoff=0.01, y=3, u=2, v=2, bt=5)

Prefetch(3)

return last

32 + 3

[2020-05-04][08:09:02] y4m [info]: 1280x720p 1:1 @ 30000/1001 fps (cfr)
[2020-05-04][08:09:02] x264 [info]: using SAR=1/1
[2020-05-04][08:09:02] x264 [info]: using cpu capabilities: MMX2 SSE2Fast SSSE3 SSE4.2 AVX
[2020-05-04][08:09:02] x264 [info]: profile High, level 4.0, 4:2:0, 8-bit
[2020-05-04][08:16:36] x264 [info]: frame I:33 Avg QP:14.81 size:164393
[2020-05-04][08:16:36] x264 [info]: frame P:1781 Avg QP:18.28 size: 28087
[2020-05-04][08:16:36] x264 [info]: frame B:7082 Avg QP:26.96 size: 2921
[2020-05-04][08:16:36] x264 [info]: consecutive B-frames: 1.5% 1.4% 5.0% 6.5% 18.4% 67.2%
[2020-05-04][08:16:36] x264 [info]: mb I I16..4: 3.6% 55.0% 41.4%
[2020-05-04][08:16:36] x264 [info]: mb P I16..4: 0.5% 3.0% 2.4% P16..4: 44.0% 16.2% 16.3% 0.9% 0.4% skip:16.4%
[2020-05-04][08:16:36] x264 [info]: mb B I16..4: 0.0% 0.1% 0.2% B16..8: 35.1% 2.9% 0.7% direct: 0.7% skip:60.3% L0:47.0% L1:44.0% BI: 9.0%
[2020-05-04][08:16:36] x264 [info]: 8x8 transform intra:49.6% inter:31.2%
[2020-05-04][08:16:36] x264 [info]: direct mvs spatial:99.9% temporal:0.1%
[2020-05-04][08:16:36] x264 [info]: coded y,uvDC,uvAC intra: 86.4% 59.0% 33.8% inter: 7.1% 2.9% 0.3%
[2020-05-04][08:16:36] x264 [info]: i16 v,h,dc,p: 7% 52% 12% 29%
[2020-05-04][08:16:36] x264 [info]: i8 v,h,dc,ddl,ddr,vr,hd,vl,hu: 8% 33% 16% 5% 5% 4% 8% 6% 15%
[2020-05-04][08:16:36] x264 [info]: i4 v,h,dc,ddl,ddr,vr,hd,vl,hu: 10% 29% 8% 6% 9% 7% 11% 7% 14%
[2020-05-04][08:16:36] x264 [info]: i8c dc,h,v,p: 40% 38% 14% 9%
[2020-05-04][08:16:36] x264 [info]: Weighted P-Frames: Y:12.0% UV:0.7%
[2020-05-04][08:16:36] x264 [info]: ref P L0: 62.3% 13.9% 18.2% 3.4% 1.9% 0.2% 0.0%
[2020-05-04][08:16:36] x264 [info]: ref B L0: 86.6% 10.5% 2.5% 0.4%
[2020-05-04][08:16:36] x264 [info]: ref B L1: 98.0% 2.0%
[2020-05-04][08:16:36] x264 [info]: kb/s:2052.01
[2020-05-04][08:16:36] encoded 8896 frames, 19.47 fps, 2052.03 kb/s

32 + 5

[2020-05-04][08:16:41] y4m [info]: 1280x720p 1:1 @ 30000/1001 fps (cfr)
[2020-05-04][08:16:41] x264 [info]: using SAR=1/1
[2020-05-04][08:16:41] x264 [info]: using cpu capabilities: MMX2 SSE2Fast SSSE3 SSE4.2 AVX
[2020-05-04][08:16:41] x264 [info]: profile High, level 4.0, 4:2:0, 8-bit
[2020-05-04][08:24:11] x264 [info]: frame I:33 Avg QP:14.81 size:164393
[2020-05-04][08:24:11] x264 [info]: frame P:1781 Avg QP:18.28 size: 28087
[2020-05-04][08:24:11] x264 [info]: frame B:7082 Avg QP:26.96 size: 2921
[2020-05-04][08:24:11] x264 [info]: consecutive B-frames: 1.5% 1.4% 5.0% 6.5% 18.4% 67.2%
[2020-05-04][08:24:11] x264 [info]: mb I I16..4: 3.6% 55.0% 41.4%
[2020-05-04][08:24:11] x264 [info]: mb P I16..4: 0.5% 3.0% 2.4% P16..4: 44.0% 16.2% 16.3% 0.9% 0.4% skip:16.4%
[2020-05-04][08:24:11] x264 [info]: mb B I16..4: 0.0% 0.1% 0.2% B16..8: 35.1% 2.9% 0.7% direct: 0.7% skip:60.3% L0:47.0% L1:44.0% BI: 9.0%
[2020-05-04][08:24:11] x264 [info]: 8x8 transform intra:49.6% inter:31.2%
[2020-05-04][08:24:11] x264 [info]: direct mvs spatial:99.9% temporal:0.1%
[2020-05-04][08:24:11] x264 [info]: coded y,uvDC,uvAC intra: 86.4% 59.0% 33.8% inter: 7.1% 2.9% 0.3%
[2020-05-04][08:24:11] x264 [info]: i16 v,h,dc,p: 7% 52% 12% 29%
[2020-05-04][08:24:11] x264 [info]: i8 v,h,dc,ddl,ddr,vr,hd,vl,hu: 8% 33% 16% 5% 5% 4% 8% 6% 15%
[2020-05-04][08:24:11] x264 [info]: i4 v,h,dc,ddl,ddr,vr,hd,vl,hu: 10% 29% 8% 6% 9% 7% 11% 7% 14%
[2020-05-04][08:24:11] x264 [info]: i8c dc,h,v,p: 40% 38% 14% 9%
[2020-05-04][08:24:11] x264 [info]: Weighted P-Frames: Y:12.0% UV:0.7%
[2020-05-04][08:24:11] x264 [info]: ref P L0: 62.3% 13.9% 18.2% 3.4% 1.9% 0.2% 0.0%
[2020-05-04][08:24:11] x264 [info]: ref B L0: 86.6% 10.5% 2.5% 0.4%
[2020-05-04][08:24:11] x264 [info]: ref B L1: 98.0% 2.0%
[2020-05-04][08:24:11] x264 [info]: kb/s:2052.01
[2020-05-04][08:24:11] encoded 8896 frames, 19.63 fps, 2052.03 kb/s

40 + 3

[2020-05-04][08:24:14] y4m [info]: 1280x720p 1:1 @ 30000/1001 fps (cfr)
[2020-05-04][08:24:14] x264 [info]: using SAR=1/1
[2020-05-04][08:24:14] x264 [info]: using cpu capabilities: MMX2 SSE2Fast SSSE3 SSE4.2 AVX
[2020-05-04][08:24:14] x264 [info]: profile High, level 4.0, 4:2:0, 8-bit
[2020-05-04][08:31:26] x264 [info]: frame I:33 Avg QP:14.82 size:164351
[2020-05-04][08:31:26] x264 [info]: frame P:1781 Avg QP:18.27 size: 28114
[2020-05-04][08:31:26] x264 [info]: frame B:7082 Avg QP:26.98 size: 2912
[2020-05-04][08:31:26] x264 [info]: consecutive B-frames: 1.5% 1.5% 5.0% 6.4% 18.9% 66.7%
[2020-05-04][08:31:26] x264 [info]: mb I I16..4: 3.8% 55.4% 40.8%
[2020-05-04][08:31:26] x264 [info]: mb P I16..4: 0.5% 3.0% 2.3% P16..4: 44.1% 16.1% 16.4% 0.9% 0.4% skip:16.3%
[2020-05-04][08:31:26] x264 [info]: mb B I16..4: 0.0% 0.1% 0.2% B16..8: 35.0% 2.9% 0.7% direct: 0.7% skip:60.4% L0:46.9% L1:44.1% BI: 9.0%
[2020-05-04][08:31:26] x264 [info]: 8x8 transform intra:49.8% inter:31.3%
[2020-05-04][08:31:26] x264 [info]: direct mvs spatial:99.9% temporal:0.1%
[2020-05-04][08:31:26] x264 [info]: coded y,uvDC,uvAC intra: 86.6% 59.1% 33.9% inter: 7.2% 2.9% 0.3%
[2020-05-04][08:31:26] x264 [info]: i16 v,h,dc,p: 7% 53% 12% 28%
[2020-05-04][08:31:26] x264 [info]: i8 v,h,dc,ddl,ddr,vr,hd,vl,hu: 8% 33% 15% 5% 5% 4% 8% 6% 16%
[2020-05-04][08:31:26] x264 [info]: i4 v,h,dc,ddl,ddr,vr,hd,vl,hu: 10% 29% 8% 6% 9% 7% 11% 7% 14%
[2020-05-04][08:31:26] x264 [info]: i8c dc,h,v,p: 40% 38% 14% 9%
[2020-05-04][08:31:26] x264 [info]: Weighted P-Frames: Y:12.2% UV:0.7%
[2020-05-04][08:31:26] x264 [info]: ref P L0: 62.2% 14.0% 18.2% 3.4% 2.0% 0.2% 0.0%
[2020-05-04][08:31:26] x264 [info]: ref B L0: 86.6% 10.6% 2.5% 0.4%
[2020-05-04][08:31:26] x264 [info]: ref B L1: 98.0% 2.0%
[2020-05-04][08:31:26] x264 [info]: kb/s:2051.38
[2020-05-04][08:31:26] encoded 8896 frames, 20.52 fps, 2051.41 kb/s

40 + 5

[2020-05-04][08:31:29] y4m [info]: 1280x720p 1:1 @ 30000/1001 fps (cfr)
[2020-05-04][08:31:29] x264 [info]: using SAR=1/1
[2020-05-04][08:31:29] x264 [info]: using cpu capabilities: MMX2 SSE2Fast SSSE3 SSE4.2 AVX
[2020-05-04][08:31:29] x264 [info]: profile High, level 4.0, 4:2:0, 8-bit
[2020-05-04][08:38:40] x264 [info]: frame I:33 Avg QP:14.82 size:164351
[2020-05-04][08:38:40] x264 [info]: frame P:1781 Avg QP:18.27 size: 28114
[2020-05-04][08:38:40] x264 [info]: frame B:7082 Avg QP:26.98 size: 2912
[2020-05-04][08:38:40] x264 [info]: consecutive B-frames: 1.5% 1.5% 5.0% 6.4% 18.9% 66.7%
[2020-05-04][08:38:40] x264 [info]: mb I I16..4: 3.8% 55.4% 40.8%
[2020-05-04][08:38:40] x264 [info]: mb P I16..4: 0.5% 3.0% 2.3% P16..4: 44.1% 16.1% 16.4% 0.9% 0.4% skip:16.3%
[2020-05-04][08:38:40] x264 [info]: mb B I16..4: 0.0% 0.1% 0.2% B16..8: 35.0% 2.9% 0.7% direct: 0.7% skip:60.4% L0:46.9% L1:44.1% BI: 9.0%
[2020-05-04][08:38:40] x264 [info]: 8x8 transform intra:49.8% inter:31.3%
[2020-05-04][08:38:40] x264 [info]: direct mvs spatial:99.9% temporal:0.1%
[2020-05-04][08:38:40] x264 [info]: coded y,uvDC,uvAC intra: 86.6% 59.1% 33.9% inter: 7.2% 2.9% 0.3%
[2020-05-04][08:38:40] x264 [info]: i16 v,h,dc,p: 7% 53% 12% 28%
[2020-05-04][08:38:40] x264 [info]: i8 v,h,dc,ddl,ddr,vr,hd,vl,hu: 8% 33% 15% 5% 5% 4% 8% 6% 16%
[2020-05-04][08:38:40] x264 [info]: i4 v,h,dc,ddl,ddr,vr,hd,vl,hu: 10% 29% 8% 6% 9% 7% 11% 7% 14%
[2020-05-04][08:38:40] x264 [info]: i8c dc,h,v,p: 40% 38% 14% 9%
[2020-05-04][08:38:40] x264 [info]: Weighted P-Frames: Y:12.2% UV:0.7%
[2020-05-04][08:38:40] x264 [info]: ref P L0: 62.2% 14.0% 18.2% 3.4% 2.0% 0.2% 0.0%
[2020-05-04][08:38:40] x264 [info]: ref B L0: 86.6% 10.6% 2.5% 0.4%
[2020-05-04][08:38:40] x264 [info]: ref B L1: 98.0% 2.0%
[2020-05-04][08:38:40] x264 [info]: kb/s:2051.38
[2020-05-04][08:38:40] encoded 8896 frames, 20.51 fps, 2051.41 kb/s

56 + 3

[2020-05-04][08:38:44] y4m [info]: 1280x720p 1:1 @ 30000/1001 fps (cfr)
[2020-05-04][08:38:44] x264 [info]: using SAR=1/1
[2020-05-04][08:38:44] x264 [info]: using cpu capabilities: MMX2 SSE2Fast SSSE3 SSE4.2 AVX
[2020-05-04][08:38:44] x264 [info]: profile High, level 4.0, 4:2:0, 8-bit
[2020-05-04][08:45:29] x264 [info]: frame I:33 Avg QP:14.81 size:165176
[2020-05-04][08:45:29] x264 [info]: frame P:1780 Avg QP:18.27 size: 28208
[2020-05-04][08:45:29] x264 [info]: frame B:7083 Avg QP:26.94 size: 2909
[2020-05-04][08:45:29] x264 [info]: consecutive B-frames: 1.5% 1.6% 4.9% 6.2% 18.6% 67.2%
[2020-05-04][08:45:29] x264 [info]: mb I I16..4: 3.9% 55.1% 41.1%
[2020-05-04][08:45:29] x264 [info]: mb P I16..4: 0.5% 2.9% 2.3% P16..4: 44.2% 16.1% 16.4% 0.9% 0.4% skip:16.3%
[2020-05-04][08:45:29] x264 [info]: mb B I16..4: 0.0% 0.1% 0.2% B16..8: 34.9% 2.8% 0.7% direct: 0.7% skip:60.5% L0:46.9% L1:44.0% BI: 9.1%
[2020-05-04][08:45:29] x264 [info]: 8x8 transform intra:49.7% inter:31.4%
[2020-05-04][08:45:29] x264 [info]: direct mvs spatial:99.9% temporal:0.1%
[2020-05-04][08:45:29] x264 [info]: coded y,uvDC,uvAC intra: 86.9% 59.0% 33.7% inter: 7.2% 2.9% 0.3%
[2020-05-04][08:45:29] x264 [info]: i16 v,h,dc,p: 7% 52% 12% 29%
[2020-05-04][08:45:29] x264 [info]: i8 v,h,dc,ddl,ddr,vr,hd,vl,hu: 8% 33% 15% 5% 5% 4% 8% 6% 16%
[2020-05-04][08:45:29] x264 [info]: i4 v,h,dc,ddl,ddr,vr,hd,vl,hu: 9% 28% 7% 6% 9% 7% 11% 7% 14%
[2020-05-04][08:45:29] x264 [info]: i8c dc,h,v,p: 40% 38% 14% 9%
[2020-05-04][08:45:29] x264 [info]: Weighted P-Frames: Y:12.1% UV:0.7%
[2020-05-04][08:45:29] x264 [info]: ref P L0: 62.4% 14.0% 18.1% 3.4% 1.9% 0.2% 0.0%
[2020-05-04][08:45:29] x264 [info]: ref B L0: 86.6% 10.5% 2.5% 0.3%
[2020-05-04][08:45:29] x264 [info]: ref B L1: 98.0% 2.0%
[2020-05-04][08:45:29] x264 [info]: kb/s:2055.54
[2020-05-04][08:45:29] encoded 8896 frames, 21.85 fps, 2055.56 kb/s

56 + 5

[2020-05-04][08:45:32] y4m [info]: 1280x720p 1:1 @ 30000/1001 fps (cfr)
[2020-05-04][08:45:32] x264 [info]: using SAR=1/1
[2020-05-04][08:45:32] x264 [info]: using cpu capabilities: MMX2 SSE2Fast SSSE3 SSE4.2 AVX
[2020-05-04][08:45:32] x264 [info]: profile High, level 4.0, 4:2:0, 8-bit
[2020-05-04][08:52:17] x264 [info]: frame I:33 Avg QP:14.81 size:165176
[2020-05-04][08:52:17] x264 [info]: frame P:1780 Avg QP:18.27 size: 28208
[2020-05-04][08:52:17] x264 [info]: frame B:7083 Avg QP:26.93 size: 2909
[2020-05-04][08:52:17] x264 [info]: consecutive B-frames: 1.5% 1.6% 4.9% 6.2% 18.6% 67.2%
[2020-05-04][08:52:17] x264 [info]: mb I I16..4: 3.9% 55.1% 41.1%
[2020-05-04][08:52:17] x264 [info]: mb P I16..4: 0.5% 2.9% 2.3% P16..4: 44.2% 16.1% 16.4% 0.9% 0.4% skip:16.3%
[2020-05-04][08:52:17] x264 [info]: mb B I16..4: 0.0% 0.1% 0.2% B16..8: 34.9% 2.8% 0.7% direct: 0.7% skip:60.5% L0:46.9% L1:44.0% BI: 9.1%
[2020-05-04][08:52:17] x264 [info]: 8x8 transform intra:49.7% inter:31.4%
[2020-05-04][08:52:17] x264 [info]: direct mvs spatial:99.9% temporal:0.1%
[2020-05-04][08:52:17] x264 [info]: coded y,uvDC,uvAC intra: 86.9% 59.0% 33.7% inter: 7.2% 2.9% 0.3%
[2020-05-04][08:52:17] x264 [info]: i16 v,h,dc,p: 7% 52% 12% 29%
[2020-05-04][08:52:17] x264 [info]: i8 v,h,dc,ddl,ddr,vr,hd,vl,hu: 8% 33% 15% 5% 5% 4% 8% 6% 16%
[2020-05-04][08:52:17] x264 [info]: i4 v,h,dc,ddl,ddr,vr,hd,vl,hu: 9% 28% 7% 6% 9% 7% 11% 7% 14%
[2020-05-04][08:52:17] x264 [info]: i8c dc,h,v,p: 40% 38% 14% 9%
[2020-05-04][08:52:17] x264 [info]: Weighted P-Frames: Y:12.1% UV:0.7%
[2020-05-04][08:52:17] x264 [info]: ref P L0: 62.4% 14.0% 18.1% 3.4% 1.9% 0.2% 0.0%
[2020-05-04][08:52:17] x264 [info]: ref B L0: 86.6% 10.5% 2.5% 0.3%
[2020-05-04][08:52:17] x264 [info]: ref B L1: 98.0% 2.0%
[2020-05-04][08:52:17] x264 [info]: kb/s:2055.56
[2020-05-04][08:52:17] encoded 8896 frames, 21.83 fps, 2055.58 kb/s

64 + 3

[2020-05-04][08:52:21] y4m [info]: 1280x720p 1:1 @ 30000/1001 fps (cfr)
[2020-05-04][08:52:21] x264 [info]: using SAR=1/1
[2020-05-04][08:52:21] x264 [info]: using cpu capabilities: MMX2 SSE2Fast SSSE3 SSE4.2 AVX
[2020-05-04][08:52:21] x264 [info]: profile High, level 4.0, 4:2:0, 8-bit
[2020-05-04][08:59:29] x264 [info]: frame I:33 Avg QP:14.81 size:165277
[2020-05-04][08:59:29] x264 [info]: frame P:1778 Avg QP:18.28 size: 28214
[2020-05-04][08:59:29] x264 [info]: frame B:7085 Avg QP:26.92 size: 2904
[2020-05-04][08:59:29] x264 [info]: consecutive B-frames: 1.4% 1.6% 5.0% 6.4% 18.2% 67.4%
[2020-05-04][08:59:29] x264 [info]: mb I I16..4: 3.8% 55.2% 41.0%
[2020-05-04][08:59:29] x264 [info]: mb P I16..4: 0.5% 2.9% 2.3% P16..4: 44.2% 16.1% 16.3% 0.9% 0.4% skip:16.3%
[2020-05-04][08:59:29] x264 [info]: mb B I16..4: 0.0% 0.1% 0.2% B16..8: 34.9% 2.8% 0.7% direct: 0.7% skip:60.6% L0:46.8% L1:44.2% BI: 9.0%
[2020-05-04][08:59:29] x264 [info]: 8x8 transform intra:49.8% inter:31.5%
[2020-05-04][08:59:29] x264 [info]: direct mvs spatial:99.9% temporal:0.1%
[2020-05-04][08:59:29] x264 [info]: coded y,uvDC,uvAC intra: 87.0% 59.1% 33.8% inter: 7.2% 2.9% 0.3%
[2020-05-04][08:59:29] x264 [info]: i16 v,h,dc,p: 7% 52% 12% 29%
[2020-05-04][08:59:29] x264 [info]: i8 v,h,dc,ddl,ddr,vr,hd,vl,hu: 7% 33% 15% 5% 5% 4% 8% 6% 16%
[2020-05-04][08:59:29] x264 [info]: i4 v,h,dc,ddl,ddr,vr,hd,vl,hu: 9% 28% 7% 6% 9% 7% 11% 7% 14%
[2020-05-04][08:59:29] x264 [info]: i8c dc,h,v,p: 40% 38% 14% 9%
[2020-05-04][08:59:29] x264 [info]: Weighted P-Frames: Y:12.1% UV:0.7%
[2020-05-04][08:59:29] x264 [info]: ref P L0: 62.4% 14.0% 18.1% 3.4% 1.9% 0.2% 0.0%
[2020-05-04][08:59:29] x264 [info]: ref B L0: 86.7% 10.4% 2.5% 0.4%
[2020-05-04][08:59:29] x264 [info]: ref B L1: 98.0% 2.0%
[2020-05-04][08:59:29] x264 [info]: kb/s:2053.48
[2020-05-04][08:59:29] encoded 8896 frames, 20.66 fps, 2053.50 kb/s

64 + 5

[2020-05-04][08:59:33] y4m [info]: 1280x720p 1:1 @ 30000/1001 fps (cfr)
[2020-05-04][08:59:33] x264 [info]: using SAR=1/1
[2020-05-04][08:59:33] x264 [info]: using cpu capabilities: MMX2 SSE2Fast SSSE3 SSE4.2 AVX
[2020-05-04][08:59:33] x264 [info]: profile High, level 4.0, 4:2:0, 8-bit
[2020-05-04][09:06:42] x264 [info]: frame I:33 Avg QP:14.81 size:165277
[2020-05-04][09:06:42] x264 [info]: frame P:1778 Avg QP:18.28 size: 28214
[2020-05-04][09:06:42] x264 [info]: frame B:7085 Avg QP:26.92 size: 2904
[2020-05-04][09:06:42] x264 [info]: consecutive B-frames: 1.4% 1.6% 5.0% 6.4% 18.2% 67.4%
[2020-05-04][09:06:42] x264 [info]: mb I I16..4: 3.8% 55.2% 41.0%
[2020-05-04][09:06:42] x264 [info]: mb P I16..4: 0.5% 2.9% 2.3% P16..4: 44.2% 16.1% 16.3% 0.9% 0.4% skip:16.3%
[2020-05-04][09:06:42] x264 [info]: mb B I16..4: 0.0% 0.1% 0.2% B16..8: 34.9% 2.8% 0.7% direct: 0.7% skip:60.6% L0:46.8% L1:44.2% BI: 9.0%
[2020-05-04][09:06:42] x264 [info]: 8x8 transform intra:49.8% inter:31.5%
[2020-05-04][09:06:42] x264 [info]: direct mvs spatial:99.9% temporal:0.1%
[2020-05-04][09:06:42] x264 [info]: coded y,uvDC,uvAC intra: 87.0% 59.1% 33.8% inter: 7.2% 2.9% 0.3%
[2020-05-04][09:06:42] x264 [info]: i16 v,h,dc,p: 7% 52% 12% 29%
[2020-05-04][09:06:42] x264 [info]: i8 v,h,dc,ddl,ddr,vr,hd,vl,hu: 7% 33% 15% 5% 5% 4% 8% 6% 16%
[2020-05-04][09:06:42] x264 [info]: i4 v,h,dc,ddl,ddr,vr,hd,vl,hu: 9% 28% 7% 6% 9% 7% 11% 7% 14%
[2020-05-04][09:06:42] x264 [info]: i8c dc,h,v,p: 40% 38% 14% 9%
[2020-05-04][09:06:42] x264 [info]: Weighted P-Frames: Y:12.1% UV:0.7%
[2020-05-04][09:06:42] x264 [info]: ref P L0: 62.4% 14.1% 18.1% 3.4% 1.9% 0.2% 0.0%
[2020-05-04][09:06:42] x264 [info]: ref B L0: 86.7% 10.4% 2.5% 0.4%
[2020-05-04][09:06:42] x264 [info]: ref B L1: 98.0% 2.0%
[2020-05-04][09:06:42] x264 [info]: kb/s:2053.46
[2020-05-04][09:06:42] encoded 8896 frames, 20.63 fps, 2053.48 kb/s

[almosely]

I just made two further tests with bw=48 and surprisingly they are identical. Now I do some tests with avstp.dll loaded and opt=3, as I did in first place.

[MeteorRain]

Don't just test two tests. Take more tests. Trim the video short.
In my tests they have certain results, and you may happen to hit the identical one in that 2 attempts.

[almosely]

Okay, bw=48 with opt=3, with loaded or unloaded or loaded and "deactivated" avstp.dll were all different from each other.

Then I ran the test again for 32, 40, 48, 56, 64, without avstp.dll and without opt=3, for each bw-size 3 times per prefetch (3 or 5). This time trimmed (0-1000 frames).

Only bw=32 was consistent. Alle three pf 3 were the same, all three pf 5 and all six of them together. Additionally only bw=56 with pf 5 was consistent. Everything else had different results.

This time I ran 3 jobs at a time, parallel. All of the last tests were one job at a time, after another.

- edit -

If your results are consistent, could it be that my system is the error? Maybe my undervolted cpu or my newly added second ram-kit? My CPU is undervolted since a year. I never had issues. I tested it that time with Intel Processor Diagnostic Tool and some x264 encodings. From that point where IPDT and/or x264 gave error messages I went up with the offset for 0.030 Volt. Since then I never had any issue with the offset of -0.190 V. And with my RAM: I did runs with Memtest86+ 5.01, for my old 2x 4GB Kit and one for the new constellation of 4x 4GB (two 2-RAM-Kits of the same type from the same vendor). Both tests passed without an error. HWiNFO displays the same adjustment results for each configuration (8/16 GB); the only different thing is the RAM command rate (changed from 1T to 2T). I really hope, that my system is not the reason for that x264 results.

- edit -

I am running the same tests with fft3dfilter right now. Seems that it ends the same way as with neo_fft3d. Till now, all six bw=32 tests gave identical results. All six bw=40 had different results from each other. The rest is running.

- edit -

So, almost the same with fft3dfilter. All six bw=32 tests gave identical results. All other bws gave different results to each other run. And there is a little difference between neo_fft3d and fft3dfilter (both bw=32). Please do not compare the fps, they are not representative in this case.

neo_fft3d

[2020-05-05][00:18:50] y4m [info]: 1280x720p 1:1 @ 30000/1001 fps (cfr)
[2020-05-05][00:18:50] x264 [info]: using SAR=1/1
[2020-05-05][00:18:50] x264 [info]: using cpu capabilities: MMX2 SSE2Fast SSSE3 SSE4.2 AVX
[2020-05-05][00:18:50] x264 [info]: profile High, level 4.0, 4:2:0, 8-bit
[2020-05-05][00:21:08] x264 [info]: frame I:5 Avg QP:16.06 size:156722
[2020-05-05][00:21:08] x264 [info]: frame P:207 Avg QP:18.58 size: 28493
[2020-05-05][00:21:08] x264 [info]: frame B:789 Avg QP:26.66 size: 3141
[2020-05-05][00:21:08] x264 [info]: consecutive B-frames: 1.6% 1.6% 8.4% 8.0% 17.5% 62.9%
[2020-05-05][00:21:08] x264 [info]: mb I I16..4: 5.2% 48.4% 46.5%
[2020-05-05][00:21:08] x264 [info]: mb P I16..4: 0.5% 2.8% 2.3% P16..4: 44.0% 15.2% 16.4% 1.1% 0.6% skip:17.1%
[2020-05-05][00:21:08] x264 [info]: mb B I16..4: 0.0% 0.1% 0.2% B16..8: 36.4% 3.1% 0.7% direct: 1.0% skip:58.5% L0:44.7% L1:44.3% BI:11.1%
[2020-05-05][00:21:08] x264 [info]: 8x8 transform intra:47.7% inter:34.3%
[2020-05-05][00:21:08] x264 [info]: direct mvs spatial:99.2% temporal:0.8%
[2020-05-05][00:21:08] x264 [info]: coded y,uvDC,uvAC intra: 84.9% 57.5% 31.5% inter: 8.2% 3.2% 0.3%
[2020-05-05][00:21:08] x264 [info]: i16 v,h,dc,p: 7% 51% 12% 30%
[2020-05-05][00:21:08] x264 [info]: i8 v,h,dc,ddl,ddr,vr,hd,vl,hu: 7% 33% 15% 5% 5% 4% 8% 6% 16%
[2020-05-05][00:21:08] x264 [info]: i4 v,h,dc,ddl,ddr,vr,hd,vl,hu: 10% 29% 8% 6% 8% 7% 11% 7% 14%
[2020-05-05][00:21:08] x264 [info]: i8c dc,h,v,p: 41% 37% 13% 8%
[2020-05-05][00:21:08] x264 [info]: Weighted P-Frames: Y:25.1% UV:1.0%
[2020-05-05][00:21:08] x264 [info]: ref P L0: 66.9% 14.3% 14.3% 2.7% 1.6% 0.2% 0.0%
[2020-05-05][00:21:08] x264 [info]: ref B L0: 88.4% 9.2% 2.0% 0.3%
[2020-05-05][00:21:08] x264 [info]: ref B L1: 98.2% 1.8%
[2020-05-05][00:21:08] x264 [info]: kb/s:2194.06
[2020-05-05][00:21:08] encoded 1001 frames, 7.06 fps, 2194.25 kb/s

fft3dfilter

[2020-05-05][02:18:35] y4m [info]: 1280x720p 1:1 @ 30000/1001 fps (cfr)
[2020-05-05][02:18:35] x264 [info]: using SAR=1/1
[2020-05-05][02:18:35] x264 [info]: using cpu capabilities: MMX2 SSE2Fast SSSE3 SSE4.2 AVX
[2020-05-05][02:18:35] x264 [info]: profile High, level 4.0, 4:2:0, 8-bit
[2020-05-05][02:20:59] x264 [info]: frame I:5 Avg QP:16.05 size:156878
[2020-05-05][02:20:59] x264 [info]: frame P:207 Avg QP:18.59 size: 28492
[2020-05-05][02:20:59] x264 [info]: frame B:789 Avg QP:26.72 size: 3146
[2020-05-05][02:20:59] x264 [info]: consecutive B-frames: 1.6% 1.6% 8.4% 8.0% 17.5% 62.9%
[2020-05-05][02:20:59] x264 [info]: mb I I16..4: 5.3% 48.2% 46.4%
[2020-05-05][02:20:59] x264 [info]: mb P I16..4: 0.5% 2.9% 2.3% P16..4: 44.2% 15.1% 16.3% 1.1% 0.5% skip:17.1%
[2020-05-05][02:20:59] x264 [info]: mb B I16..4: 0.1% 0.1% 0.2% B16..8: 36.4% 3.1% 0.7% direct: 1.0% skip:58.5% L0:44.8% L1:44.2% BI:11.0%
[2020-05-05][02:20:59] x264 [info]: 8x8 transform intra:47.9% inter:34.6%
[2020-05-05][02:20:59] x264 [info]: direct mvs spatial:99.2% temporal:0.8%
[2020-05-05][02:20:59] x264 [info]: coded y,uvDC,uvAC intra: 84.9% 57.5% 31.6% inter: 8.2% 3.2% 0.3%
[2020-05-05][02:20:59] x264 [info]: i16 v,h,dc,p: 7% 51% 12% 30%
[2020-05-05][02:20:59] x264 [info]: i8 v,h,dc,ddl,ddr,vr,hd,vl,hu: 7% 33% 15% 5% 5% 4% 8% 6% 16%
[2020-05-05][02:20:59] x264 [info]: i4 v,h,dc,ddl,ddr,vr,hd,vl,hu: 9% 29% 8% 6% 8% 7% 11% 7% 14%
[2020-05-05][02:20:59] x264 [info]: i8c dc,h,v,p: 41% 37% 13% 8%
[2020-05-05][02:20:59] x264 [info]: Weighted P-Frames: Y:25.1% UV:1.0%
[2020-05-05][02:20:59] x264 [info]: ref P L0: 66.8% 14.3% 14.4% 2.7% 1.6% 0.2% 0.0%
[2020-05-05][02:20:59] x264 [info]: ref B L0: 88.4% 9.3% 2.0% 0.3%
[2020-05-05][02:20:59] x264 [info]: ref B L1: 98.3% 1.7%
[2020-05-05][02:20:59] x264 [info]: kb/s:2195.05
[2020-05-05][02:20:59] encoded 1001 frames, 6.72 fps, 2195.24 kb/s

[MeteorRain]

A few assumptions / points here:

1) For 1 frame long clip at bw=48 I get 3 possible results after ~10 runs.
x264 [info]: kb/s:6861.36
x264 [info]: kb/s:6863.66
x264 [info]: kb/s:6864.24

So I assume there are certain results possible, and are deterministic. I get one of them randomly. Maybe it's memory bound -- different memory position / alignment gives different result?

2) prefetch should not affect results. If it's deterministic, it is no matter prefetch or single threaded.

3) Both filter didn't use avstp, so you shouldn't need to take it into consideration.

4) I don't think your hardware is at fault.

5) Core code (algorithm) remains the same between fft3dfilter and neo_fft3d, hence they behaved the same on bw=32/40/48.

[ChaosKing]

A few assumptions / points here:

1) For 1 frame long clip at bw=48 I get 3 possible results after ~10 runs.
x264 [info]: kb/s:6861.36
x264 [info]: kb/s:6863.66
x264 [info]: kb/s:6864.24



Have you also tested with only 1 thread in x264?(idk if multiple threads have any impact on 1 frame, but better be sure)

[pinterf]

Isn' that kb/s the bitrate after compression instead of the processing speed? And if they are different for multiple runs that means that the output is also different?

[MeteorRain]

Have you also tested with only 1 thread in x264?(idk if multiple threads have any impact on 1 frame, but better be sure)

Yes, that was the first thing to test. Also with bw=32 it's always the same result regardless of threads in x264.

Isn' that kb/s the bitrate after compression instead of the processing speed? And if they are different for multiple runs that means that the output is also different?

Yes it's compression result. And yes, the output seems to be different.

I've checked the result, they have very little different at LSB.

TL;DR:
fftwf_execute_dft_r2c() -> deterministic
Apply() -> nondeterministic

Inside Apply(), for example bt=2, inside Apply3D2(), there is code like below
out[w][0] = something * 0.5f;
This multiplication caused nondeterministic. If it's addition instead of multiplication (or *1.0f which is no-op) the result is still deterministic.

I have no clue how this happened. I'm guessing there's some padding issue (because bw=32 always gives deterministic result), but it can also be problems inside fftw (fftw claimed nondeterministic results in certain circumstances).

[almosely]

In my tests, the results were "almost" consistent, as long as I ran them one-by-one, after each other and while not working at my pc at best. Since (almost) every test had (way more) different results (except bw=32) when running the jobs parallel, I think, it might be correlated with multi-/hyper-threading or something like that. Maybe one task is not giving back the results to the corresponding task, e.g. FFTW is called from task 1 but giving back the result to task 2. But I do not know if something like that is possible at all. But I do know that some plugins do behave wrong (mostly, but sometimes not), if not called sequential in a mt-script. Maybe with a block size greater than 32 there is built up a chain with splitted parts to compute, but put back together in the wrong order. And with a shitload of tasks running parallel these parts are (more often) not sequential anymore, as they would be (by accident/by speed) when running the jobs one-by-one.

[MeteorRain]

The computing error is very little, so I don't think there's any major issue in the process.
Differences between each run is less than 0.01% (i.e. 1 or 2 bit at LSB in fftw_complex).

As I'm reading articles I noticed that floating points may not get reproducible results for their operations. I wonder if that is part of the reason for this issue.

[almosely]

Hm, okay. Seems that this is no issue to worry about. Then I will use neo_fft3d from now on (and without restrictions). Thank you again! :-)

[pinterf]

So it's fftw? And the old fft3d have those issues as well?

[almosely]

I am running the same tests with fft3dfilter right now. Seems that it ends the same way as with neo_fft3d. Till now, all six bw=32 tests gave identical results. All six bw=40 had different results from each other. The rest is running.

- edit -

So, almost the same with fft3dfilter. All six bw=32 tests gave identical results. All other bws gave different results to each other run.


At least in my last comparison test, it seems to be the same issue with both.

[tormento]

RTFM:

Question 3.7. FFTW gives results different from my old FFT.
People follow many different conventions for the DFT, and you should be sure to know the ones that we use (described in the FFTW manual). In particular, you should be aware that the FFTW_FORWARD/FFTW_BACKWARD directions correspond to signs of -1/+1 in the exponent of the DFT definition. (Numerical Recipes uses the opposite convention.)
You should also know that we compute an unnormalized transform. In contrast, Matlab is an example of program that computes a normalized transform. See Q3.10 `Why does your inverse transform return a scaled result?'. Finally, note that floating-point arithmetic is not exact, so different FFT algorithms will give slightly different results (on the order of the numerical accuracy; typically a fractional difference of 1e-15 or so in double precision).

Question 3.8. FFTW gives different results between runs
If you use FFTW_MEASURE or FFTW_PATIENT mode, then the algorithm FFTW employs is not deterministic: it depends on runtime performance measurements. This will cause the results to vary slightly from run to run. However, the differences should be slight, on the order of the floating-point precision, and therefore should have no practical impact on most applications.
If you use saved plans (wisdom) or FFTW_ESTIMATE mode, however, then the algorithm is deterministic and the results should be identical between runs.

[almosely]

Ahhh, there shall be light :-) So, everything is fine then. Thank you!

[MeteorRain]

tormento: Just so you know that I already read that part during my research.
Regarding to 3.8 it runs under estimate mode, so it shouldn't be nondeterministic.
Regarding to 3.7 it's possible that rounding errors in float caused the issue. It's just that I can't say "for sure" that it "is" the cause of the issue. I can only say it's "likely" the cause.

[tormento]

tormento: Just so you know that I already read that part during my research.
Aaaaaaand which version is almosely running? :p

[MeteorRain]

You know I was talking about your "Question 3.7 / Question 3.8" right?

[tormento]

You know I was talking about your "Question 3.7 / Question 3.8" right?
Definitely.

[MeteorRain]

I don't know but I used the version from fftw website.

[tormento]

I don't know but I used the version from fftw website.
If you did not compile on your own, the latest official windows binary you can get from the site is 3.3.5. Try and use the latest versions have been around since some weeks on this forum.

[almosely]

I have FFTW 3.3.8 (AVX) (Wolfberry's compile from 29-01-2019) already installed and in use since then.

He compiled this version on request by me (here at this forum). I could'nt find any v3.3.8 AVX x64 that time. I don't know in which mode that compile runs. I thought, that would depend on the kind of call, not the compile. I am surprised.

FFTW 3.3.8 built with ICC 19.0 now in my signature.

Have fun :)

Should I use a more recent MSVC build? Clang is for Mac, GCC etc. for what? Nowadays there are so many builds, I don't know which one to use.

[StvG]

StvG: I don't have AVX512 and every time I test with AVX512 I have to rent a server. Can you confirm if the difference only appears on opt=4? What bit depth?
Sorry for the late reply.
Used script:LoadPlugin(".\NeoFFT3D_r7\clang-x64\neo-fft3d.dll")
ColorBars(1920,1080, pixel_type="rgbp16") #tested rgbp, rgbps, yv12, yuv420p16, yuv420ps, yuv444ps (same behavior always)
n=3
a=neo_fft3d(sigma=1.0, beta=1.0, bw=48, bh=48, sharpen=0.040, scutoff=0.01, y=3, u=2, v=2, bt=5, opt=n)
b=neo_fft3d(sigma=1.0, beta=1.0, bw=48, bh=48, sharpen=0.040, scutoff=0.01, y=3, u=2, v=2, bt=5, opt=1)
Compare(a,b)
Subtitle("neo_fft3d(sigma=1.0, beta=1.0, bw=48, bh=48, sharpen=0.040, scutoff=0.01, y=3, u=2, v=2, bt=5, opt="+string(n)+")"+"\n"+"neo_fft3d(sigma=1.0, beta=1.0, bw=48, bh=48, sharpen=0.040, scutoff=0.01, y=3, u=2, v=2, bt=5, opt=1)", lsp=1, y=1040)

https://thumbs2.imgbox.com/ae/0a/SBZEn8T5_t.png (http://imgbox.com/SBZEn8T5) https://thumbs2.imgbox.com/79/a6/WpKGRVGh_t.png (http://imgbox.com/WpKGRVGh) https://thumbs2.imgbox.com/0a/fc/48odxsGP_t.png (http://imgbox.com/48odxsGP)

[tormento]

Should I use a more recent MSVC build? Clang is for Mac, GCC etc. for what? Nowadays there are so many builds, I don't know which one to use.
Enjoy (https://forum.doom9.org/showthread.php?p=1908016#post1908016).

ICL, usually, runs best on Intel.

[almosely]

Thanks! :-)

I tested all FFTW (I guess it's 3.3.8?) versions (from your link above) against each other, including my old 3.3.8 ICC 19.0, and the fastest and least cpu consuming version was GCC 9.3. I have all MS Visual C++ versions from 2005-2019 installed and updated (for this test) my Intel C++ Redist from 15.0.xx (2015) to the latest 19.0.8 Update 8 (2019) version. Sadly, the newest Intel C++ version (2020 19.1) is not downloadable (Intel-Link is broken). The slowest version was MSVC 2019, then Clang 10.0.

Detailed test results are in the corresponding thread, here (https://forum.doom9.org/showthread.php?p=1911115#post1911115).

I rerun my bw-tests also with some of this versions (ICL 19.1, MSVC 2019, GCC 9.3) and the results were the same as with my old FFTW-version: bw=32 was identical, every time; every other block-size got different results from each other run.

[tormento]

Sadly, the newest Intel C++ version (2020 19.1) is not downloadable (Intel-Link is broken).
Yep, Intel site is pure shit from that point of view.

Anyway, I uploaded there:

Intel C++ redist 2020.1·216 (https://www.mediafire.com/file/2f576c2qokgen4j/Intel_C++_redist_2020.1%B7216.msi/file)

Intel C++ redist 2020.1·216_x64 (https://www.mediafire.com/file/efea2ncb0r27vew/Intel_C++_redist_2020.1%B7216_x64.msi/file)

Tell me if you need Fortran ones too. :p