[Neo] f3kdb

[MeteorRain]

Neo filters family is not to be confused with AviSynth-Neo. Neo filters family is not affiliated in any way with AviSynth-Neo.

[Neo] F3KDB → GitHub (https://github.com/HomeOfAviSynthPlusEvolution/neo_f3kdb)

F3kdb is a deband filter. It was originally written for AviUtl (author uncovered) and later ported to AviSynth (Original F3KDB) (https://forum.doom9.org//showthread.php?t=161411) by SAPikachu many years ago.

r5

* New dual synth wrapper platform update.

r2

* Add sample_mode 3 and 4.

r1

* Added AviSynth+ native high bit depth, based on 2.0 branch.
* Supporting YUV4xxP8/P10/P12/P14/P16 and I420.
* Dropped support for legacy AviSynth versions.

[real.finder]

:thanks:

any plan for float support?

[Natty]

thaaaaanks

[real.finder]

I think there are bugs

ColorBars(width=640, height=480, pixel_type="yv12")
F3KDB()

will give "Unsupported output depth (-1)" in both avs+ and old avs 2.6

with
F3KDB(output_depth=8)

it work, in both avs+ and old avs 2.6

[MeteorRain]

I see, should have initialized output-depth to be the same as input.

https://down.7086.in/f3kdb-2.1rc2.zip

Regarding floating support, I have no interests on that. Supporting float involves extensive coding that will come with minimum benefits.

For high precision processing, 16 bit should be good enough for most of the cases.

[StvG]

There is random "access violation writing 0x0000000000000000". Just used:ffvideosource(10-bit.video)
f3kdb()

[MeteorRain]

StvG: Thanks for the report.

Can you please do ffvideosource(10-bit.video).info and check what colorspace/width/height etc is?

Also are you on latest AviSynth+(2772)? Or an older version?

[StvG]

avs+ 2772 x64 used. Info (https://images2.imgbox.com/e1/c6/9xx4ukdN_o.png); error (https://images2.imgbox.com/72/80/FpvPA0kO_o.png) (trying to run three times in a row).

Edit: The behavior is the same with 8-bit video too.

[MeteorRain]

Reviewing the code I actually found an out of bounds array access issue. Will fix it and let you try again.

https://down.7086.in/f3kdb-2.1rc3.zip

Please let me know how it works for you.

[StvG]

Same behavior. (https://images2.imgbox.com/18/8a/oPtBlQbq_o.png)
Btw no crashes when f3kdb_*_sanitize* stuff (https://github.com/msg7086/flash3kyuu_deband/commit/0ad779c3c736647d97c4146195bc2c1e3e11a6eb) isn't removed but sometimes the speed is lower (~30%).

[ChaosKing]

only 64bit version seems to be affected, crashes also in 8 bit. 32bit version seems to run fine and is even faster in 16bit - 180fps vs 430fps :O

ffms2(dvd-source)
ConvertBits(16)
f3kdb()

Edit:
avsmeter confirmed the 8bit - 16bit huge fps gap. 16bit is more optimized?

[MeteorRain]

Oh 64 bit version, missed that. Let me check that part and see if I can find something.

[Natty]

only 64bit version seems to be affected, crashes also in 8 bit. 32bit version seems to run fine and is even faster in 16bit - 180fps vs 430fps :O

ffms2(dvd-source)
ConvertBits(16)
f3kdb()

Edit:
avsmeter confirmed the 8bit - 16bit huge fps gap. 16bit is more optimized?

convertbits(16)
f3kdb()

gives half green screen

example of 32 bit pls? and you mean 32 is faster than 16?

[ChaosKing]

32bit = x86 :-)

[MeteorRain]

but sometimes the speed is lower (~30%).
Seems to be an uninitialized multi-threading flag that caused random results (randomly mt / st).

[MeteorRain]

32bit version seems to run fine and is even faster in 16bit - 180fps vs 430fps :O

ffms2(dvd-source)
ConvertBits(16)
f3kdb()

Edit:
avsmeter confirmed the 8bit - 16bit huge fps gap. 16bit is more optimized?

Try (dither_algo = 1). 16bit doesn't need dithering and thus is supposedly faster.

[Natty]

Try (dither_algo = 1). 16bit doesn't need dithering and thus is supposedly faster.

16bit only supports dither_algo=3 according to docs.
https://f3kdb.readthedocs.io/en/stable/usage.html#cmdoption-arg-dither_algo
is this correct? or is there some other docs for your ported version?
and can you enable dither_algo=2 for 16bit? :thanks:

[MeteorRain]

https://down.7086.in/f3kdb-2.1rc4.zip

Fixed an undefined behavior in C++.
Fixed mt flag not properly set.

[MeteorRain]

16bit only supports dither_algo=3 according to docs.
https://f3kdb.readthedocs.io/en/stable/usage.html#cmdoption-arg-dither_algo
is this correct? or is there some other docs for your ported version?
and can you enable dither_algo=2 for 16bit? :thanks:

Internal processing was under 16 bit, so for 16 bit output no dithering is needed. This is what I understood by reading the code, please correct me if I was wrong.

"This parameter is ignored if output_depth = 16." is what the doc says. The "ignore" doesn't mean it uses the default, this means it's not needed.

[ChaosKing]

https://down.7086.in/f3kdb-2.1rc4.zip

Fixed an undefined behavior in C++.
Fixed mt flag not properly set.

x64 seems to work now. It is about 22% faster than the x86 version.

[Natty]

Internal processing was under 16 bit, so for 16 bit output no dithering is needed. This is what I understood by reading the code, please correct me if I was wrong.

"This parameter is ignored if output_depth = 16." is what the doc says. The "ignore" doesn't mean it uses the default, this means it's not needed.

https://f3kdb.readthedocs.io/en/stable/usage.html#cmdoption-arg-output_depth

till now i thought it will apply floyd dithering as it is default mode of f3kdb in 16bit output. i am confused now. :confused:

that means specifying dither_algo is waste then as f3kdb will ignore it

[MeteorRain]

Natty:
Dither is only used on downgrading bit depth, i.e. from internal precision (16 bit) down to output bit depth (14 bit for example).
If you want 16 bit output, it just gives out the original result, without touching it at all.

[Natty]

:thanks: understood

[StvG]

x64 seems to work now...

I have no crashes too.

Thanks for this version.

[FranceBB]

Original post (/showthread.php?t=161411)

Added AviSynth+ native high bit depth, based on 2.0 branch.
Supporting YUV4xxP8/P10/P12/P14/P16 and I420.
Dropped support for legacy AviSynth versions.


Hi,
I tried the new version of f3kdb and it works, however I noticed that as you introduced support for regular, planar high bit depth, you removed support for 16bit stacked and 16bit interleaved. I don't actually think this is the right thing to do as it will definitely break things in many other scripts that make use of f3kdb as a filter (dependency) and that were working in either 16bit stacked or 16bit interleaved.
f3kdb was working with basically two modes: input_mode=1 and input_mode=2. Mode 1 means 16bit stacked, while Mode 2 means 16bit interleaved.
I think that you are absolutely right to add support for regular, planar high bit depth, however you should not remove those two modes as they are really important for people relying on the old Avisynth 2.6.1 or for filters that use f3kdb as a sub-filter.


This is an example of what I noticed when I was trying to encode a documentary provided in Apple ProRes 10bit planar that still had banding in low light scenes anyway:

16bit interleaved (https://i.imgur.com/EMmGxwB.png) - 16bit stacked (https://i.imgur.com/AU9iA2P.png) - 10bit planar (https://i.imgur.com/TQwGyKc.jpg)

In this case there's no reason to use 16bit stacked or interleaved, but as I said, removing them will break support for many filters that rely on f3kdb as dependency and will also make life harder to those using Avisynth 2.6.1.

#10bit planar input, 8bit planar output works

video=FFVideoSource("\\VBOXSVR\Share_Windows_Linux\Production\RAW\IMN_190319_v2.mov")
audio=FFAudioSource("\\VBOXSVR\Share_Windows_Linux\Production\RAW\IMN_190319_v2.mov")
Audiook=MergeChannels(audio, audio, audio, audio)
AudioDub(video, audiook)

f3kdb(range=15, Y=70, Cb=40, Cr=40, grainY=0, grainC=0, sample_mode=2, blur_first=true, dynamic_grain=false, opt=3, mt=true, keep_tv_range=true, output_depth=8)


### ### ###
#10bit planar input -> 16bit planar -> 8bit planar output works

video=FFVideoSource("\\VBOXSVR\Share_Windows_Linux\Production\RAW\IMN_190319_v2.mov")
audio=FFAudioSource("\\VBOXSVR\Share_Windows_Linux\Production\RAW\IMN_190319_v2.mov")
Audiook=MergeChannels(audio, audio, audio, audio)
AudioDub(video, audiook)

ConvertBits(16)

f3kdb(range=15, Y=70, Cb=40, Cr=40, grainY=0, grainC=0, sample_mode=2, blur_first=true, dynamic_grain=false, opt=3, mt=true, keep_tv_range=true, output_depth=8)


### ### ###
#10bit planar input -> 16bit planar -> 16bit planar output works

video=FFVideoSource("\\VBOXSVR\Share_Windows_Linux\Production\RAW\IMN_190319_v2.mov")
audio=FFAudioSource("\\VBOXSVR\Share_Windows_Linux\Production\RAW\IMN_190319_v2.mov")
Audiook=MergeChannels(audio, audio, audio, audio)
AudioDub(video, audiook)

ConvertBits(16)

f3kdb(range=15, Y=70, Cb=40, Cr=40, grainY=0, grainC=0, sample_mode=2, blur_first=true, dynamic_grain=false, opt=3, mt=true, keep_tv_range=true, output_depth=8)


### ### ###
#10bit planar input -> 16bit planar ->16bit stacked -> 16bit stacked does not work
#f3kdb does not have a named argument "input mode"

video=FFVideoSource("\\VBOXSVR\Share_Windows_Linux\Production\RAW\IMN_190319_v2.mov")
audio=FFAudioSource("\\VBOXSVR\Share_Windows_Linux\Production\RAW\IMN_190319_v2.mov")
Audiook=MergeChannels(audio, audio, audio, audio)
AudioDub(video, audiook)

ConvertBits(16)
ConvertToStacked()

f3kdb(range=15, Y=70, Cb=40, Cr=40, grainY=0, grainC=0, sample_mode=2, blur_first=true, dynamic_grain=false, opt=3, mt=true, keep_tv_range=true, input_mode=1, input_depth=16, output_mode=1, output_depth=16)


### ### ###
#10bit planar input -> 16bit planar ->16bit interleaved -> 16bit interleaved does not work
#f3kdb does not have a named argument "input mode"

video=FFVideoSource("\\VBOXSVR\Share_Windows_Linux\Production\RAW\IMN_190319_v2.mov")
audio=FFAudioSource("\\VBOXSVR\Share_Windows_Linux\Production\RAW\IMN_190319_v2.mov")
Audiook=MergeChannels(audio, audio, audio, audio)
AudioDub(video, audiook)

ConvertBits(16)
ConvertToDoubleWidth()

f3kdb(range=15, Y=70, Cb=40, Cr=40, grainY=0, grainC=0, sample_mode=2, blur_first=true, dynamic_grain=false, opt=3, mt=true, keep_tv_range=true, input_mode=2, input_depth=16, output_mode=2, output_depth=16)

[Natty]

input depth, input mode and output mode dont work. only output depth works. I like it. just hope it works correctly internally. :p

[MeteorRain]

however you should not remove those two modes as they are really important for people relying on the old Avisynth 2.6.1 or for filters that use f3kdb as a sub-filter

The only thing that was done in 2.1 was to add high bit depth (and remove legacy stacked and interleave).

If you want to use legacy I/O, simply use the old version (2.0, 1.x, etc).

[FranceBB]

The only thing that was done in 2.1 was to add high bit depth (and remove legacy stacked and interleave).

If you want to use legacy I/O, simply use the old version (2.0, 1.x, etc).

I know, and having normal high bit depth is great, but I think you should put 16bit stacked and 16bit interleave back because, as I said, there are third party plugins that are calling f3kdb with input and output modes that are now deprecated.

Tl;dr I'm in favor of adding regular planar high bit depth, but I don't think that stacked and interleave support should be removed for compatibility reasons.

[MeteorRain]

I see your point. However having both format could cause confusion.

Suppose you are f3kdb'ing 8 bit input and asking for 16 bit output. Pre 2.1 output mode defaults to 1 = stacked, and now native YUV420P16. If we return stacked by default, it's confusing for normal usage. If we return native by default, it still breaks compatibility.

My recommendation is that you keep the old version auto loaded, and manually load the native version when needed, until all your existing scripts are converted.

Upgrading to native HBD is a pain process. All moving parts need to be replaced by the HBD version. In my case, that means DGSource, L-Smash-works, Delogo, f3kdb and the final step avs4x26x or whatever piper.

PR welcome, but I'm not going to spend time on reverting the code / adding smart or dumb condition checks to improve compatibility. Sorry.

[real.finder]

is it possible to disable masking in F3KDB?

[MeteorRain]

Can you be more precise? I'm not very familiar with the core part of f3kdb.

[real.finder]

Can you be more precise? I'm not very familiar with the core part of f3kdb.

with

ImageSource("uonzj6V.png (http://i.imgur.com/uonzj6V.png)")
PointResize(720,480)
ConvertToYV12()
mt_merge(last,FlipHorizontal(),BlankClip(last,color = $FFFFFF).FadeIO(500))
f3kdb()
luma_histogram() (https://forum.doom9.org/showthread.php?t=161411)


you can see masking artifacts in the dividing line

edit: to make sure it's masking problem use f3kdb(grainy=0)

[real.finder]

There is no internal masking in f3kdb. The artifacts you see are resulted from how the reference pixels are selected by the algorithm. By using sample_mode=2 which selects pixels diagonally around the current pixel, the pixels above the dividing line and the pixels below the dividing line differ too much, hence the algorithm will return the original pixels instead of the blurred ones. By making the algorithm select pixels horizontally (let's call it sample_mode=0), the artifacts will go away.

https://i.postimg.cc/ctq0L8BB/Original.png (https://postimg.cc/ctq0L8BB) https://i.postimg.cc/9zTcNJc8/sample-mode-2.png (https://postimg.cc/9zTcNJc8) https://i.postimg.cc/ctQZg4Dy/sample-mode-0.png (https://postimg.cc/ctQZg4Dy)


aside from sample_mode=0 was removed in pre2, with sample_mode=0, grainy=0 the filter didn't do anything and still there are artifacts

edit: I think you had a modified version, it's better to chose 3 instead of 0

edit: adding mode 4 for vertical will be nice, and then the output of both can mixed later

[real.finder]

Sorry for the confusion. The sample_mode=0 was added by myself in my compiled f3kdb. The functionality is not the same as the old removed one. I only meant to demonstrate the result of selecting the reference pixels horizontally and I just arbitrarily assigned it to sample_mode=0 for distinguishing.

I already get it, no need to be sorry :) see my edit

[real.finder]

That's exactly what sample_mode=1 does.

ok, what about 3 for horizontal, and 4 for both but separated then mixed (like by simple merge (http://avisynth.nl/index.php/Merge) or something)

[DJATOM]

Guess you will have the same result with (if I understood your request right)
f3kdb(sample_mode=3)
TurnLeft()
f3kdb(sample_mode=3)
TurnRight()

[real.finder]

Guess you will have the same result with (if I understood your request right)
f3kdb(sample_mode=3)
TurnLeft()
f3kdb(sample_mode=3)
TurnRight()


for now there are no sample_mode=3, with sample_mode=1 it will work I think, but with more blur and slowness

[ChaosKing]

@MeteorRain The download link on the first page is not working.

[MeteorRain]

Fixed the link. But I'm planning to clean up the building system a little bit and then release a GA version.

[Stereodude]

GA version?

Is there any change we can talk you into integrating madshi's suggested improvements for it? https://forum.doom9.org/showthread.php?p=1652256#post1652256

[MeteorRain]

Stereodude, I'm not an algorithm guy and have no idea about the actual implementation. If there's an easy-to-integrate patch available I'd love to merge it in. Otherwise It'll be very hard for me to implement it.

I mainly took over the work to create an AVS+HBD compatible plugin and that's all I'm capable of doing.

[Natty]

:goodpost:

[MeteorRain]

FranceBB,

After a while of thinking, I believe it's possible to expose a legacy function where it assumes all input and output mode as interleaved (which is effectively the same as HBD), so in the case of referencing f3kdb from foreign scripts, simply changing the function name should do the work, if they were working under interleaved mode. For stacked mode, a conversion is mandatory.

# Proposal1
interleaved_output = interleaved_input.f3kdb_i(input_depth=10, output_depth=12)
stacked_output = stacked_input.ConvertFromStacked().ConvertToDoubleWidth().f3kdb_i(input_depth=10, output_depth=12).ConvertFromDoubleWidth().ConvertToStacked()

The other option is to compile and release a legacy version with function name changed. You keep both DLLs in the system until all legacy scripts move on.

# Proposal2
output = input.f3kdb_legacy(input_depth=10, input_mode=1, output_depth=12, output_mode=2)

Which do you all prefer? I can also do both.

Understanding the difficulties of moving from hacks to HBD, I'd like to reduce even just a bit of pain for you.

[real.finder]

MeteorRain, but you can do it like this
input_mode=0 mean no hacks (8 bit or HBD)
output_mode=3 mean HBD always

you can also add some check with input_mode > 0 and HBD ThrowError("f3kdb: you can't use input_mode > 0 with HBD")

edit: you can add things I add here https://forum.doom9.org/showthread.php?p=1881052#post1881052 (All or some of them) and of course without change the default settings (mean if you add sample_mode 3 and 4 the default will stay on 2 not like I set it to 4)

[MeteorRain]

Would it be better if I leave the mode conversion to a wrapper script and call it a day? I'm feeling that we should move on to HBD as fast as possible and not leaving that much legacy code in the code base.

By the way I didn't realize you made f3kdb_3. Do you think it's still necessary to add legacy modes and builds while your script can already deal with it?

Bottom line: I don't want to put modes back, but I'd like to have some sort of mechanism so that people would have no pain living with both version and use whatever version they want, until all plugins are overhauled.

[real.finder]

Would it be better if I leave the mode conversion to a wrapper script and call it a day? I'm feeling that we should move on to HBD as fast as possible and not leaving that much legacy code in the code base.

By the way I didn't realize you made f3kdb_3. Do you think it's still necessary to add legacy modes and builds while your script can already deal with it?

Bottom line: I don't want to put modes back, but I'd like to have some sort of mechanism so that people would have no pain living with both version and use whatever version they want, until all plugins are overhauled.

well, pinterf didn't remove the lsb from mvtools, even he didn't remove it from masktools (which no one use or even know)

there are some people still use even avs25 but that not important, avs+ plugins work with avs26

anyway I am with keeping lsb and other modes even if I will not use them, I am with you about move on to HBD as fast as possible and also with keep lsb and interleaved format

[ChaosKing]

well, pinterf didn't remove the lsb from mvtools, even he didn't remove it from masktools (which no one use or even know)

there are some people still use even avs25 but that not important, avs+ plugins work with avs26

anyway I am with keeping lsb and other modes even if I will not use them, I am with you about move on to HBD as fast as possible and also with keep lsb and interleaved format

There is also XP compatibility, but it doesn't mean it should be supported :devil:
Sometimes it's good to drop old stuff -> less bugs and work.

[real.finder]

There is also XP compatibility, but it doesn't mean it should be supported :devil:
Sometimes it's good to drop old stuff -> less bugs and work.

support old os is different topic

avs26 still has reasons to use it, lsb and interleaved not important only for avs26 but as FranceBB said before there are many scripts has them

and indeed drop old stuff -> less bugs and work but also it's enjoyable to code to support them :D but anyway I am not with add them to plugins that don't has them but I am with only keep them on plugins that already has them

[FranceBB]

FranceBB,

After a while of thinking, I believe it's possible to expose a legacy function where it assumes all input and output mode as interleaved (which is effectively the same as HBD), so in the case of referencing f3kdb from foreign scripts, simply changing the function name should do the work, if they were working under interleaved mode. For stacked mode, a conversion is mandatory.

# Proposal1
interleaved_output = interleaved_input.f3kdb_i(input_depth=10, output_depth=12)
stacked_output = stacked_input.ConvertFromStacked().ConvertToDoubleWidth().f3kdb_i(input_depth=10, output_depth=12).ConvertFromDoubleWidth().ConvertToStacked()

The other option is to compile and release a legacy version with function name changed. You keep both DLLs in the system until all legacy scripts move on.

# Proposal2
output = input.f3kdb_legacy(input_depth=10, input_mode=1, output_depth=12, output_mode=2)

Which do you all prefer? I can also do both.

Understanding the difficulties of moving from hacks to HBD, I'd like to reduce even just a bit of pain for you.


The first proposal you offered would be better.

As to me, myself, I was thinking about something like this:

(please note that this is written in pseudo-code and won't work in Avisynth):


function f3kdb_mod(clip c, int range, int Y, int Cb, int Cr, int grainY, int grainC,
\ int sample_mode, float seed, bool blur_first, bool dynamic_grain, int opt,
\ bool mt, int dither_algo, bool keep_tv_range, int input_mode,
\ int input_depth, int output_depth, int random_algo_ref,
\ int random_algo_grain, float random_param_ref,
\ float random_param_grain, string preset)

{

#input mode 0 = Planar (8bit planar, 10bit planar, 12bit planar, 14bit planar, 16bit planar)
#input mode 1 = Stacked
#input mode 2 = Interleaved

#output mode 0 = Planar (8bit planar, 10bit planar, 12bit planar, 14bit planar, 16bit planar)
#output mode 1 = Stacked
#output mode 2 = Interleaved

#Input Planar Output Planar
planar_i_planar_o = (input_mode=0 && input_depth >= 8 && output_mode=0) ? true : false

f3kdb = (planar_i_planar_o=true) ? f3kdb(c, range=range, Y=Y, Cb=Cb, Cr=Cr, grainY=grainY,
\ grainC=grainC, sample_mode=sample_mode, seed=seed, blur_first=blur_first
\ dynamic_grain=dynamic_grain, opt=opt, mt=mt, dither_algo=dither_algo,
\ keep_tv_range=keep_tv_range, output_depth=output_depth,
\ random_algo_ref=random_algo_ref, random_algo_grain=random_algo_grain,
\ random_param_ref=random_param_ref, random_param_grain=random_param_grain,
\ preset=preset)


#Input Planar Output Stacked
planar_i_stacked_o = (input_mode=0 && input_depth >= 8 && output_mode=1 && output_depth=16) ? true : false

f3kdb = (planar_i_stacked_o=true) ? f3kdb(c, range=range, Y=Y, Cb=Cb, Cr=Cr, grainY=grainY,
\ grainC=grainC, sample_mode=sample_mode, seed=seed, blur_first=blur_first
\ dynamic_grain=dynamic_grain, opt=opt, mt=mt, dither_algo=dither_algo,
\ keep_tv_range=keep_tv_range, output_depth=output_depth,
\ random_algo_ref=random_algo_ref, random_algo_grain=random_algo_grain,
\ random_param_ref=random_param_ref, random_param_grain=random_param_grain,
\ preset=preset).ConvertToStacked()

#Input Planar Output Interleaved
planar_i_interleaved_o = (input_mode=0 && input_depth >= 8 && output_mode=2 && output_depth=16) ? true : false

f3kdb = (planar_i_stacked_o=true) ? f3kdb(c, range=range, Y=Y, Cb=Cb, Cr=Cr, grainY=grainY,
\ grainC=grainC, sample_mode=sample_mode, seed=seed, blur_first=blur_first
\ dynamic_grain=dynamic_grain, opt=opt, mt=mt, dither_algo=dither_algo,
\ keep_tv_range=keep_tv_range, output_depth=output_depth,
\ random_algo_ref=random_algo_ref, random_algo_grain=random_algo_grain,
\ random_param_ref=random_param_ref, random_param_grain=random_param_grain,
\ preset=preset).ConvertToDoubleWidth()


#Input Stacked output Stacked
stacked_i_stacked_o = (input_mode=1 && input_depth=16 && output_mode=1 && output_depth=16) ? true : false

f3kdb = (stacked_i_stacked_o=true) ? ConvertFromStacked().f3kdb(c, range=range, Y=Y, Cb=Cb, Cr=Cr, grainY=grainY,
\ grainC=grainC, sample_mode=sample_mode, seed=seed, blur_first=blur_first
\ dynamic_grain=dynamic_grain, opt=opt, mt=mt, dither_algo=dither_algo,
\ keep_tv_range=keep_tv_range, output_depth=output_depth,
\ random_algo_ref=random_algo_ref, random_algo_grain=random_algo_grain,
\ random_param_ref=random_param_ref, random_param_grain=random_param_grain,
\ preset=preset).ConvertToStacked()


#Input Stacked output Interleaved
stacked_i_interleaved_o = (input_mode=1 && input_depth=16 && output_mode=2 && output_depth=16) ? true : false

f3kdb = (stacked_i_stacked_o=true) ? ConvertFromStacked().f3kdb(c, range=range, Y=Y, Cb=Cb, Cr=Cr, grainY=grainY,
\ grainC=grainC, sample_mode=sample_mode, seed=seed, blur_first=blur_first
\ dynamic_grain=dynamic_grain, opt=opt, mt=mt, dither_algo=dither_algo,
\ keep_tv_range=keep_tv_range, output_depth=output_depth,
\ random_algo_ref=random_algo_ref, random_algo_grain=random_algo_grain,
\ random_param_ref=random_param_ref, random_param_grain=random_param_grain,
\ preset=preset).ConvertToDoubleWidth()


#Input Interleaved output Interleaved
interleaved_i_interleaved_o = (input_mode=2 && input_depth=16 && output_mode=2 && output_depth=16) ? true : false

f3kdb = (stacked_i_stacked_o=true) ? ConvertFromDoubleWidth().f3kdb(c, range=range, Y=Y, Cb=Cb, Cr=Cr, grainY=grainY,
\ grainC=grainC, sample_mode=sample_mode, seed=seed, blur_first=blur_first
\ dynamic_grain=dynamic_grain, opt=opt, mt=mt, dither_algo=dither_algo,
\ keep_tv_range=keep_tv_range, output_depth=output_depth,
\ random_algo_ref=random_algo_ref, random_algo_grain=random_algo_grain,
\ random_param_ref=random_param_ref, random_param_grain=random_param_grain,
\ preset=preset).ConvertToDoubleWidth()


#Input Interleaved output Stacked
interleaved_i_interleaved_o = (input_mode=2 && input_depth=16 && output_mode=1 && output_depth=16) ? true : false

f3kdb = (stacked_i_stacked_o=true) ? ConvertFromDoubleWidth().f3kdb(c, range=range, Y=Y, Cb=Cb, Cr=Cr, grainY=grainY,
\ grainC=grainC, sample_mode=sample_mode, seed=seed, blur_first=blur_first
\ dynamic_grain=dynamic_grain, opt=opt, mt=mt, dither_algo=dither_algo,
\ keep_tv_range=keep_tv_range, output_depth=output_depth,
\ random_algo_ref=random_algo_ref, random_algo_grain=random_algo_grain,
\ random_param_ref=random_param_ref, random_param_grain=random_param_grain,
\ preset=preset).ConvertToStacked()


#Error handling
Assert (output_mode=1 && output_depth<=16) chr(10) + "Stacked output is supported with 16bit only!" + chr(10))
Assert (output_mode=2 && output_depth<=16) chr(10) + "Interleaved output is supported with 16bit only!" + chr(10))


return f3kdb
}

[MeteorRain]

https://down.7086.in/AviSynthPlus%20Filters/f3kdb-2.0-NBD-r1.zip

1. I think it might be better to use 2.0-NBD instead of 2.1 to avoid confusion. Original 2.0 will be re-compiled as 2.0-Legacy with function name f3kdb_legacy. Let me know what you think.

2. Added f3kdb_i with input_depth (default: 8) and always accepts and produces 8 bit (double width) video.

input_depth > 8 → double width input
output_depth > 8 → double width output

3. Since SAP is not actively maintaining this project, I'm going to host the project in my organization, and not merging back to his repo.

4. I'm trying to move the compiling system to cmake, and the binaries above is the production of cmake. Let me know what you think.

[real.finder]

I think you should name the 2.1 as f3kdb2() and flash3kyuu_deband2() if you don't do it like https://forum.doom9.org/showthread.php?p=1882340#post1882340

edit: and f3kdb_dither2() if it still there

and then will be no need for f3kdb_i and f3kdb_legacy

[MeteorRain]

Technically, it's not "2". It's what it *should* be -- doing non hack operations.

[real.finder]

Technically, it's not "2". It's what it *should* be -- doing non hack operations.

Technically yes it's not "2" but that what SAP did (give it the "2" and remove many things from f3kdb and add VS support but even so he don't recommended use it in avs since it was pre/beta) so it's (the part of not "2") half true in this case

so going with "2" in last of functions names is the best option, for both users and scripts editor (don't think there are others than me and I will not update all scripts that has f3kdb with lsb hacks just to make them work with 2.1 for end users! and most of those end users cant even edit simple script to replace with f3kdb_legacy and have 2 of f3kdb dll)

aside form that, I will ask for last time, if you don't like/can't do take the changes from f3kdb_3 then at least add sample_mode=3 (same as sample_mode=1 but horizontal) if not then seems I will not add support for 2.1 in f3kdb_3 since there are no benefit from that

[MeteorRain]

@real.finder
What you said makes sense. I'll take that into consideration.

Regarding to f3kdb_3 and other pending patches, I'll start looking at those after this is completed. I'd like to finish this version before starting another.

[MeteorRain]

I talked to SAP and he suggested having a different name, and suggested the word neo.

So, neo_f3kdb it is. All pending changes will be released as neo_f3kdb r1.

Also would like to know how sample_mode = 3 and 4 works. Thanks!

[real.finder]

I talked to SAP and he suggested having a different name, and suggested the word neo.

So, neo_f3kdb it is. All pending changes will be released as neo_f3kdb r1.

Also would like to know how sample_mode = 3 and 4 works. Thanks!

sample_mode = 3 same as sample_mode = 1 but in horizontal (sample_mode = 1 is vertical)

and sample_mode = 4 is mix of outputs from sample_mode = 1 and 3 https://forum.doom9.org/showthread.php?p=1880098#post1880098

[MeteorRain]

So mode 1 is
O
x
O

mode 2 is
O O
x
O O

mode 3 will be
O x O

and mode 4 will be
O
O x O
O

Correct or not?

[real.finder]

So mode 1 is
O
x
O

mode 2 is
O O
x
O O

mode 3 will be
O x O

and mode 4 will be
O
O x O
O

Correct or not?

mode 1 and mode 3 are correct, don't know how 2 work (I think it's comparing the up, left, down and right pixel with the center pixel)

but the 4 is not, it do up and down comparing, and another comparing for left and right and mix (blend/Average/merge) the result of both

if it can't be added that ok, if there mode 3 in the plugin then mix both output will be faster than mode 1 alone (I did use turn function with mode 1 in f3kdb_3 to simulation mode 3, so it will be slower than mode 3 in the plugin)

[MeteorRain]

https://down.7086.in/AviSynthPlus%20Filters/debug/debug-sample-mode-4-pure-c.7z

test build pure C code for sample mode 3 / 4.

neo_f3kdb(sample_mode=3, opt=0, ...)

neo_f3kdb(sample_mode=4, opt=0, ...)

Let me know if it's what you want.

[Natty]

will it be ever possible to introduce new dithering methods in f3kdb?
not asking for new feature to be added, just asking if its possible in future. example - sierra-2-4a
:thanks:

[real.finder]

https://down.7086.in/AviSynthPlus%20Filters/debug/debug-sample-mode-4-pure-c.7z

test build pure C code for sample mode 3 / 4.

neo_f3kdb(sample_mode=3, opt=0, ...)

neo_f3kdb(sample_mode=4, opt=0, ...)

Let me know if it's what you want.

mode 3 seems ok, but I can't get mode 4 work it show error about in must be between 1 and 3 only

[MeteorRain]

Whoops, please re-download it.

[real.finder]

Whoops, please re-download it.

yes, both seems do it as they should :goodpost:

test code

ImageSource("uonzj6V.png (http://i.imgur.com/uonzj6V.png)")
PointResize(720,480)
ConvertToYV12()
mt_merge(last,FlipHorizontal(),BlankClip(last,color = $FFFFFF).FadeIO(500))
Crop(26, 2, -26, -2)
last.PointResize(Height(),Width()).StackHorizontal(last.fturnright)
#~neo_f3kdb(sample_mode=3, opt=0)
neo_f3kdb(sample_mode=4, opt=0)
luma_histogram() (https://forum.doom9.org/showthread.php?t=161411)

[MeteorRain]

@Natty,

Dithering code is in dither_high.h, and is fairly easy to add new algorithms if someone decides to add that support.

[MeteorRain]

https://down.7086.in/AviSynthPlus%20Filters/debug/debug-sample-mode-34-sse.7z

SSE code for 3 & 4. Please do extensive testing for both pure C and SSE if you can, because I didn't do.

[real.finder]

https://down.7086.in/AviSynthPlus%20Filters/debug/debug-sample-mode-34-sse.7z

SSE code for 3 & 4. Please do extensive testing for both pure C and SSE if you can, because I didn't do.

you mean sse2? with opt=1 both seems ok with this https://forum.doom9.org/showthread.php?p=1884034#post1884034

[MeteorRain]

SSE2 and 3 version will be removed soon to clean up legacy polyfill code.
SSE4.1 is old enough to support bulldozer (2011) and Penryn (Core 2 series, 2007).
And I plan to add AVX2 soon.

Please test with opt=0 (C) and opt=3 (SSE4.1) at the moment.

[Myrsloik]

SSE2 and 3 version will be removed soon to clean up legacy polyfill code.
SSE4.1 is old enough to support bulldozer (2011) and Penryn (Core 2 series, 2007).
And I plan to add AVX2 soon.

Please test with opt=0 (C) and opt=3 (SSE4.1) at the moment.

Did you test so the SSE4.1 code is actually faster than SSE2? A lot of the time developers didn't bother to actually benchmark it...

[MeteorRain]

Blendv only exists in SSE 4.1 and has to be emulated. No way that's faster than native.

[Myrsloik]

Blendv only exists in SSE 4.1 and has to be emulated. No way that's faster than native.

You'll be surprised how small the difference can be sometimes. I also never claimed it to be faster.

[MeteorRain]

Brain farted just wake up please forgive me. I meant definitely slower than native.

Blendv has to be emulated by and and andnot, some mullo also has to be emulated. It might be true that they won't drag down speed much, I also think that see2 and 3 support are no longer necessary.

[ChaosKing]

Thx for supporting both VS and AVS.
I made a quick bench for r1 on a ryzen 2600 1080p source
clip=clip.neo_f3kdb.Deband(preset="veryhigh")
msvc-x64: 145fps
icc-x64: 155fps
(Had to download libmmd.dll for intel build on some random dll site)

[MeteorRain]

Thanks for letting me know about libmmd. I'll try to statically link it and re-upload the release.

[Myrsloik]

Thanks for letting me know about libmmd. I'll try to statically link it and re-upload the release.

If you want to do speed comparisons try clang-cl as well.

[ChaosKing]

Thanks for letting me know about libmmd. I'll try to statically link it and re-upload the release.

neo-f3kdb_r1-fix.zip still needs libmmd.dll (only tested x64)

[DJATOM]

@MeteorRain
Just add libmmds.lib into linking libs and it should work. At least worked for me in the past.

[MeteorRain]

Thanks for the help. I tried to install clang-cl and then ICC toolset blew up. Now having to completely remove and reinstall the freaking ICC. Will let you know after finishing all those craps.


Please re-download fix https://down.7086.in/AviSynthPlus%20Filters/neo-f3kdb_r1-fix.zip

[ChaosKing]

Thanks for the help. I tried to install clang-cl and then ICC toolset blew up. Now having to completely remove and reinstall the freaking ICC. Will let you know after finishing all those craps.


Please re-download fix https://down.7086.in/AviSynthPlus%20Filters/neo-f3kdb_r1-fix.zip
The intel build works now without libmmd.dll
Clang seems to be as fast as the intel build or even some fps faster. :thanks:

[MeteorRain]

Am I reading this wrong? Clang is almost twice speed than VC and Intel builds?
On 1080P bluray with opt=1/2, ICC=45fps and Clang=73fps.
Making me to suspect I didn't properly optimize the other 2 builds.

Or because I'm on AMD?

And 32-bit clang is even faster than 64-bit clang sometimes.

[StvG]

neo_f3kdb(opt=1) - clang x86, clang x64 and icc x64 have almost the same speed (~1% difference). icc x86 have ~4% lower speed.
Intel here.

[real.finder]

sample_mode=4 is the Default, right?

there are other new things that can be added, like chroma bool parameter, and adaptive grain parameter to make grain more or only in dark Part/Scenes

and it's off-topic but what about this? https://forum.doom9.org/showthread.php?p=1883458#post1883458

[DJATOM]

Time elapsed: 0:32.145 - 155.54746162610678084093 FPS # classic f3kdb (x64)
Time elapsed: 0:25.578 - 195.47774818344581149177 FPS # neo f3kdb (x64) (clang build)
Time elapsed: 0:33.242 - 150.41332423316677591174 FPS # neo f3kdb (x64) (icc build)
Time elapsed: 0:31.664 - 157.90663253049507375181 FPS # neo f3kdb (x64) (msvc build)
Measured on 3900X with 2 threads and set affinity for vsedit to 2 cores (1 thread assigned per active core). Anyway I can't utilize all cores with f3kdb (using DGSource), so I believe such comparison shows real difference in optimization.

[StvG]

neo_f3kdb(opt=1) - clang x86, clang x64 and icc x64 have almost the same speed (~1% difference). icc x86 have ~4% lower speed.
Intel here.

Tested with avs+ (avsmeter).
~10% CPU usage.

[MeteorRain]

sample_mode=4 is the Default, right?

there are other new things that can be added, like chroma bool parameter, and adaptive grain parameter to make grain more or only in dark Part/Scenes

and it's off-topic but what about this? https://forum.doom9.org/showthread.php?p=1883458#post1883458
Not yet. This is test build.

Minideen you probably have to ask original porter. I just port the exact code back.

[Natty]

please include a full documentation text file in final build. :thanks:

[MeteorRain]

I need some help on AVX2 version. After mirroring code to AVX2 from SSE4, performance dropped a lot (30-40% slower). I'd really like to know how that could happen.

[Myrsloik]

I need some help on AVX2 version. After mirroring code to AVX2 from SSE4, performance dropped a lot (30-40% slower). I'd really like to know how that could happen.

Did you put the AVX2 code somewhere so we can take a look for ourselves? And is this drop with only one or all the compilers?

[MeteorRain]

Myrsloik, it's on the same repo, avx2 branch. The code somehow can only be compiled by clang-cl (due to the weird template layer that I added) and probably g++.

Running code at opt=3 and opt=4 makes speed difference and I'm wondering what I did wrong.

BTW: Also would like to know if there's a better way to do this SSE/AVX abstraction layer.

[Myrsloik]

Myrsloik, it's on the same repo, avx2 branch. The code somehow can only be compiled by clang-cl (due to the weird template layer that I added) and probably g++.

Running code at opt=3 and opt=4 makes speed difference and I'm wondering what I did wrong.

BTW: Also would like to know if there's a better way to do this SSE/AVX abstraction layer.

Start by looking at the generated code. Since you say you added additional abstraction it could be that functions aren't inlined anymore and you need to liberally sprinkle __forceinline everywhere. For example here (https://github.com/HomeOfAviSynthPlusEvolution/neo_f3kdb/blob/368611ca6dc977033db1ca91fff28b7e479aee70/src/simd/T_AVX2.hpp#L55).
Then there's also the statements of the form static constexpr auto count_zero = _mm_setzero_si128; where you may end up with a function pointer and instead of inlining you end up having a separate function call made every time you call that function.

Better way to do it? Simply write two versions and keep them both clean and readable is one option. Of course the best way would be for you to learn to read the disassembled code so you can spot this kind of thing yourself.

[MeteorRain]

Exact same abstraction layer pointing to SSE is working as fast as it was, making me believing the abstraction layer is doing good.

I did read the disassembled code, and all AVX2 intrinsics were properly inline compiled (I read the debug version so it's filled with useless ymm assignments, but CPU instructions are there, not function calls.)

[Myrsloik]

Exact same abstraction layer pointing to SSE is working as fast as it was, making me believing the abstraction layer is doing good.

I did read the disassembled code, and all AVX2 intrinsics were properly inline compiled (I read the debug version so it's filled with useless ymm assignments, but CPU instructions are there, not function calls.)

You have to compare the optimized versions.

[MeteorRain]

I stopped at some heavily optimized code and got lost.

If I manually inline read_reference_pixels() into _process_plane_simd_impl() (basically just copy pasting code to the caller), the speed of SSE4 will slow down to be worse than AVX2, while AVX2 timing is not affected.
If I leave the code in a __forceinline function, SSE4 version is way faster.
Some magic happened within that function and with tmp_1/2/3/4 arrays. I tried to add volatile on those arrays, that also "corrected" the performance.

I also tried VTune Amplifier, but didn't get meaningful answers as well.

EDIT:

This is both fun and boring. A few hours spent and I finally track down to the place that matters.

Clang completely rewrite the (scalar fetch + _mm_load) sequence into unrolled _mm_insert_epi16, while under avx2 it simply went through the pain loop and a full _mm256_load.

EDIT2:

I rewrote the hotspot using the same style as Clang did. (https://github.com/HomeOfAviSynthPlusEvolution/neo_f3kdb/commit/f59924c92620b9f528045ca93a1016c80898b764) Still not as fast as SSE due to missing intrinsics in AVX2. If you'd like to spend ten or maybe twenty hours on this, please give it a try. Otherwise I'll abandon the AVX2 branch very soon.

[MeteorRain]

I'm preparing for r2 release. Please let me know which should be the default for sample_mode.

[real.finder]

I'm preparing for r2 release. Please let me know which should be the default for sample_mode.

I don't mind if it 2 or 4, so let's see what the other members will says

[Reel.Deel]

I'm preparing for r2 release. Please let me know which should be the default for sample_mode.

Probably the original default.

[MeteorRain]

I keep the default. Released r2.

[Natty]

hi, whats the difference between these two

neo-f3kdb_r2.zip

neo-f3kdb_r2_extra.zip
also in r1, the output mode and depth were same as inputs. will it be the same now?

[MeteorRain]

Compilers. Defaults to clang builds. Extra package comes with msvc and icc builds.

I didn't change output mode and depth code in r2, so should be the same.

[Natty]

using clang x64, had to put neo_f3kdb(...) instead of f3kdb(...)

works absolutely fine :thanks:

[Natty]

ConvertBits(16).neo_f3kdb(output_depth=10)

this is making encode look 1 point darker.

example (https://slowpics.org/comparison/e56cafd8-5676-44c8-a352-52b4999b1fe6)

if i use SmoothTweak to increase brightness by 1 point, it becomes fine (mostly).

plain neo_f3kdb() is also fine.

using 3.2+6-f46aa2b (https://msystem.waw.pl/x265/) x265 10bit with color parameters set to BT.709

sample clip (https://www9.zippyshare.com/v/jj04ufhz/file.html)

edit 1 : i think issue might be in x265 as this never happened before

[quietvoid]

ConvertBits(16).neo_f3kdb(output_depth=10)

this is making encode look 1 point darker.

example (https://slowpics.org/comparison/e56cafd8-5676-44c8-a352-52b4999b1fe6)

if i use SmoothTweak to increase brightness by 1 point, it becomes fine (mostly).

plain neo_f3kdb() is also fine.

using 3.2+6-f46aa2b (https://msystem.waw.pl/x265/) x265 10bit with color parameters set to BT.709

sample clip (https://www9.zippyshare.com/v/jj04ufhz/file.html)

edit 1 : i think issue might be in x265 as this never happened before
The bit depth conversion is causing a slight hue shift. It's something in your workflow.
There is no difference between x265 3.2+6 and 3.1+8 for me.

[Natty]

The bit depth conversion is causing a slight hue shift. It's something in your workflow.

this shift never happened before.

i have only updated ffms2, neo f3kdb and x265.

I'll test again. :thanks:

edit 1: tested with various videos. it happens only with some sources. couldn't understand why.

[real.finder]

chroma bool parameter seems forgotten

[real.finder]

chroma bool parameter seems forgotten

and I think there are bug
ColorBars(width=640, height=480, pixel_type="yv12")
converttoy8
ConvertBits(16)
neo_f3kdb

https://i.postimg.cc/xTymQcQH/Untitled.png (https://postimages.org/)

[MeteorRain]

I have uploaded r3 which is marked as pre-release yet. r3 is based on new dual synth platform and should not change functionality.

Y8 greyscale format issue is fixed.

chroma bool parameter -- what does it do? If you want to skip chroma processing you can set cb = cr = grainc = 0 and it will only copy data and change bit depth if required.

Let me know how it works, and I'll release it if all is well.

[real.finder]

now I got this https://i.imgur.com/mVH3Iva.png

with any bit depth

[ChaosKing]

vsedit does not even open the preview window. It just crashes with r3-pre.

[tormento]

now I got this
The color of love! :D

[l00t]

r3 prerelease is causing VSEditor to crash immediately upon function call. Furthermore y, cb, cr, grainy and grainc is renamed to Y, Cb, Cr, grainY, grainC. It would be good to document if you change variable names, as VapourSynth is case sensitive.

[MeteorRain]

Parameter name change was unintended. I'll check and change name according to last version.

I personally tested on AVS and VS with msvc-x64 and YUV420Pxx on VD2 and I didn't see any crashes.

Can you guys please provide (1) Program used and whether 32/64 bit (2) What colorspace did you pass in (3) If possible, a minimum reproducible BlankClip script.

It's very strange to me that I tested locally without problem and no one else can get it working..... Let's see what I missed.
https://www.hanselman.com/blog/content/binary/WindowsLiveWriter/IntroducingRockScroll_C29C/works-on-my-machine-starburst_3.png IRL.

[ChaosKing]

clip = core.std.BlankClip(format=vs.YUV420P8, width=1920, height=1080, length=500)
clip = clip.neo_f3kdb.Deband()

clang and msvc are crashing. tested only 64bit in vsedit.

[ChaosKing]

vspipe showed me this now (same script above used)
Cannot output clips with varying dimensions

[MeteorRain]

lol that was a stupid mistake, didn't set output-depth to input-depth if none given.

Let me finish it up and give you r4.

EDIT: Please try r4.

[real.finder]

with r4

https://i.imgur.com/SfuP3bA.png

ColorBars(width=640, height=480, pixel_type="yv12")
converttoy8
ConvertBits(16)
neo_f3kdb

[MeteorRain]

Yea sorry there's a regression. Please re-download.

[real.finder]

Yea sorry there's a regression. Please re-download.

it seems load fine this time :thanks:

[kedautinh12]

it seems load fine this time :thanks:

when you replace f3kdb to neo_f3kdb in f3kdb_s.avsi??

[real.finder]

when you replace f3kdb to neo_f3kdb in f3kdb_s.avsi??

it's already done https://github.com/realfinder/AVS-Stuff/commit/c63dd6b6e991c0ae8a351d473d1e1f23d63d305c#diff-5a47f060779727dcf6c5d1fe85ae4267

[MeteorRain]

FYI: mt and opt are no longer available in new versions.
mt is automatically enabled for planes (so 3 planes = 3 threads).
opt doesn't have too much point for consumers as it always detects supported routine.

[kedautinh12]

it's already done https://github.com/realfinder/AVS-Stuff/commit/c63dd6b6e991c0ae8a351d473d1e1f23d63d305c#diff-5a47f060779727dcf6c5d1fe85ae4267

Thanks

[real.finder]

FYI: mt and opt are no longer available in new versions.
mt is automatically enabled for planes (so 3 planes = 3 threads).
opt doesn't have too much point for consumers as it always detects supported routine.

I think opt is useful when debugging

[kedautinh12]

it's already done https://github.com/realfinder/AVS-Stuff/commit/c63dd6b6e991c0ae8a351d473d1e1f23d63d305c#diff-5a47f060779727dcf6c5d1fe85ae4267

Must load 2 plugins f3kdb.dll and neo-f3kdb? Cause i only load neo-f3kdb.dll replaced f3kdb.dll met error: "don't have function F3KDB"

[MeteorRain]

OK I'll add mt and opt back. I'll leave mt unused, and opt be a non-public parameter.

EDIT: check r5

[tormento]

OK I'll add mt and opt back. I'll leave mt unused, and opt be a non-public parameter.
Is ok to let a MT plugin run its own threads within an already MT i.e. Prefecth() script?

[real.finder]

Must load 2 plugins f3kdb.dll and neo-f3kdb? Cause i only load neo-f3kdb.dll replaced f3kdb.dll met error: "don't have function F3KDB"

did you use the r2 or newer? cuz if it as MeteorRain said, there are some changes (https://forum.doom9.org/showthread.php?p=1907413#post1907413) and I made the f3kdb_3 use the old f3kdb if there are error with neo_f3kdb

edit: maybe r5 should work as before

[real.finder]

OK I'll add mt and opt back. I'll leave mt unused, and opt be a non-public parameter.

EDIT: check r5

as tormento said, I think neo_f3kdb should be see if the Prefecth is used or not and if it used then it set mt=false like jpsdr resize plugin did (IIRC)

[kedautinh12]

did you use the r2 or newer? cuz if it as MeteorRain said, there are some changes (https://forum.doom9.org/showthread.php?p=1907413#post1907413) and I made the f3kdb_3 use the old f3kdb if there are error with neo_f3kdb

edit: maybe r5 should work as before

r5 still error, it's work normally if back to r2

[MeteorRain]

as tormento said, I think neo_f3kdb should be see if the Prefecth is used or not and if it used then it set mt=false like jpsdr resize plugin did (IIRC)

Any idea how to detect prefetch() and get thread count?

Is ok to let a MT plugin run its own threads within an already MT i.e. Prefecth() script?

Yes, it's totally fine.

For example, f3kdb itself initiates 3 threads, then you prefetch(4), that'll make it around 12 threads, but in reality it could spin up less due to thread pool size, so possibly 6 threads with 12 tasks waiting to be processed.

We are still waiting for C++23 where there will be executors and we can than do fine control on thread pool size. For now, it's managed by Windows system thread pool or Intel TBB library, which should be smart enough I'd say.


===

EDIT:
I should also point out that internal multi-threading is almost free to use while MT may come with memory usage penalty.
For example, filters usually allocate internal buffers for all 3 planes, so running 3 threads writing to and reading from their own buffer would not create any race condition. However in MT mode, multiple calls to the same function would cause race condition, and usually we'd have to create extra buffers for each individual thread. That's why in most case internal multi-threading runs lighter than MT.

[MeteorRain]

kedautinh12: Please try r5v2.

[real.finder]

Any idea how to detect prefetch() and get thread count?


I don't know since I am not Knowing c++, better see what pinterf said but in worst cases making mt=false by default and if someone set it to true then the filter should report itself as MT_SERIALIZED like rgtools
RgTools 0.94
Autoregister filter MT modes as NICE_FILTER for Avisynth+
(except for Clense: when reduceflicker is true, MULTI_INSTANCE MT mode is reported)


edit: this may help https://forum.doom9.org/showthread.php?p=1667553#post1667553

[MeteorRain]

real.finder, to be honest I don't like the idea to turn off mt by default and wait for (the less efficient) MT mode.

Like I said, internal mt is almost free to use. And I have been very conservative on number of total threads.
I'm hoping that we can reach to a point where you'll have no needs to go more than prefetch(4) when the heaviest parts are already running parallel.

Eventually we will have mainstream CPU with 32 cores or more. If you do prefetch(32) you could waste dozens of gigabytes of memory in some filters.

Let's see what pinterf has, but I haven't found anything useful yet.

[tormento]

real.finder, to be honest I don't like the idea to turn off mt by default and wait for (the less efficient) MT mode.
Benchmarks are the way to go.

Just run with prefetch with internal MT disabled and with internal MT with prefetch disabled and let's see results.

[MeteorRain]

Tormento, that's not how real world application works. AviSynth is a frame server, you take it as a pre-processor and you'll feed the result to an encoder. Most of the time the encoder should be taking most of your CPU resources, and the filters, in most cases, will only have probably a single core to spare.

The temporal median that I just finished a few days ago, can do 11400fps on a 1280x720 YV12 clip with MT (VapourSynth) but can only do 7500fps single threaded (internal mt). However in former case it took 80% of my CPU while the latter case it only took 20%. Now, those numbers may not be accurate, but it should tell you something. As a frame server it's not the faster the better, it should find a balance between high efficiency AND speed.

[tormento]

Tormento, that's not how real world application works.
Absolutely right.

That is why with "benchmaks" I meant x264/x265 encoding, not syntetic tests such as AVSMeter.

I just posted a message in AviSynth+ main thread where AVSMeter results and real encoding results are different.

[kedautinh12]

kedautinh12: Please try r5v2.

it's work, thanks

[FranceBB]

Just a suggestion: in f3kdb.hpp from line 111 to line 121


#define INVALID_PARAM_IF(cond) \
do { if (cond) { throw("Invalid parameter condition: " #cond); } } while (0)

INVALID_PARAM_IF(in_vi.Format.IsFamilyYUV != true);
INVALID_PARAM_IF(in_vi.Width < 16);
INVALID_PARAM_IF(in_vi.Height < 16);
INVALID_PARAM_IF(in_vi.Format.SSW < 0 || in_vi.Format.SSW > 4);
INVALID_PARAM_IF(in_vi.Format.SSH < 0 || in_vi.Format.SSH > 4);
INVALID_PARAM_IF(in_vi.Frames <= 0);
INVALID_PARAM_IF(in_vi.Format.BitsPerSample < 8 || in_vi.Format.BitsPerSample > INTERNAL_BIT_DEPTH);
INVALID_PARAM_IF(in_vi.Format.IsInteger != true);


Since you're checking for invalid parameters anyway, wouldn't it be better to throw a string describing the error so that non skilled Avisynth users can better understand what they did wrong instead of trowing just #cond?
I mean, I'm totally fine with that, but even if it's self-explanatory, perhaps someone else might not get it at a first glance, so I would do something like:

in_vi.Format.IsFamilyYUV != true

"Your input is not YUV; only YUV inputs are supported"

in_vi.Width < 16

"Input Width is smaller than 16, which is the minimum supported"

in_vi.Height < 16

"Input Hight is smaller than 16, which is the minimum supported"

in_vi.Frames <= 0

"No input frames received. You need at least one frame"

in_vi.Format.BitsPerSample < 8 || in_vi.Format.BitsPerSample > INTERNAL_BIT_DEPTH

"Your input is either smaller than 8, which is the minimum supported bit depth, or larger than 16, which is the maximum internal bit depth precision of this filter"

etc

I didn't really want to bother you, it's just a little hint for the future.
By the way, thank you for the keeping f3kdb alive and working on XP as well. :)

[jpsdr]

Any idea how to detect prefetch() and get thread count?

From the few tests i've made, it seems it's not possible to have the prefetch value used in the script neither in the AvisynthPluginInit3 or the Create_xxx function and not even in the constructor of the filter.
Only in the GetFrame method you can have access to this information.
Unfortunately, in my case, it's too late, i need the information in the Create_xxx function... :(
I'm unable to remember how i had access to the information...

Edit
I've been able to retrieve the information. You have to call :
IScriptEnvironment2::GetProperty(AEP_FILTERCHAIN_THREADS)
But as i said, the value returned is not correct even at the constructor stage of the filter.

[StainlessS]

In Avisynth, constructors are called in script [EDIT: FLOW] order, and so if Prefetch is last in script, so its constructor has not yet been called when your filters constructor is called.

[MeteorRain]

Thanks, that's useful. Do you think it makes sense to turn off internal MT when prefetch is large? (Like 6+)

[real.finder]

Do you think it makes sense to turn off internal MT when prefetch is large? (Like 6+)

I do, maybe it better if prefetch more than half of cpu cores then turn off internal MT

[StainlessS]

Have revised prev post to

In Avisynth, constructors are called in script [EDIT: FLOW] order, and so if Prefetch is last in script, so its constructor has not yet been called when your filters constructor is called.

So Program Flow order would include and eg looping or calls to script functions etc.
Also, eg Variables are assigned in program flow order, and so are then availalbe for use when eg calling filter contructor. Calling a filter is just calling its constructor.

The whole shabang, start to end program flow just builds the filter graph.

During some runtime environment (eg Scriptclip::GetFrame called via eg some other GetFrame), you enter another short lived period where a temp filter graph is created and each constructor within the runtime script is called,
finally delivering a single (current_frame) frame to the filter GetFrame that requested it of Scriptclip.

Sort of.

[MeteorRain]

I do, maybe it better if prefetch more than half of cpu cores then turn off internal MT

So I thoroughly thought about it, and I think I'm not gonna disable internal MT at this moment.

f3kdb has internal state cache, which means the more you enable external MT the more you'll be wasting memory space, by a LOT.

Running 10 threads external MT with iMT disabled will have the same speed as running 4 threads with iMT enabled but you are looking at doubling the memory usage. (That means it may go from 1GB to 2GB, per instance, and you gain nothing at all.)

So for f3kdb, instead of disabling iMT, I'd ask user to reduce MT threads to prevent wasting.

[real.finder]

So I thoroughly thought about it, and I think I'm not gonna disable internal MT at this moment.

f3kdb has internal state cache, which means the more you enable external MT the more you'll be wasting memory space, by a LOT.

Running 10 threads external MT with iMT disabled will have the same speed as running 4 threads with iMT enabled but you are looking at doubling the memory usage. (That means it may go from 1GB to 2GB, per instance, and you gain nothing at all.)

So for f3kdb, instead of disabling iMT, I'd ask user to reduce MT threads to prevent wasting.

isn't f3kdb NICE_FILTER? if it not, what about make report itself as MT_SERIALIZED when iMT is on?

[MeteorRain]

Never mind. Obviously f3kdb and fft3d got mixed in my mind. f3kdb is fine.

[Stereodude]

Sorry if this is a stupid question or has been discussed before. I didn't see any discussion on it. Where did the new sampling modes come from, or what is the theory behind them? Is there particular scenarios where one should be better than others, or is it just a try each and see what you like better?

[real.finder]

Sorry if this is a stupid question or has been discussed before. I didn't see any discussion on it. Where did the new sampling modes come from, or what is the theory behind them? Is there particular scenarios where one should be better than others, or is it just a try each and see what you like better?

see from here https://forum.doom9.org/showthread.php?p=1879897#post1879897

[Stereodude]

see from here https://forum.doom9.org/showthread.php?p=1879897#post1879897
That makes it seem like mode 4 is to perform better on synthetic test patterns, not real content (most of the time).

[real.finder]

That makes it seem like mode 4 is to perform better on synthetic test patterns, not real content (most of the time).

mode 4 is balanced and less fierce against the details

and I think in anime/cartoon you can also see mode 4 perform better

anyway, in the end it depends on what you see better, mode 2 still default

[Boulder]

Out of interest, has anyone else made any tests with UHD sources and the filter? I was astonished to notice that removing the line core.neo_f3kdb.Deband(clp, preset="medium", output_depth=16, sample_mode=4) from my test script produced a file that was ~35% bigger than the one with it.. the exact same settings in x265. The test material (Blade Runner 2049) is very clean and you can easily see the effect of debanding if you use DGHDRtoSDR.

[MeteorRain]

Because of the grain you added?

[FranceBB]

Because of the grain you added?

That must be it.

@Boulder... By default, f3kdb adds grain to compensate debanding which of course takes more bits to be encoded properly with crf. You can try to turn the grain off in f3kdb and play with the other parameters. I generally use:


neo_f3kdb(range=15, Y=45, Cb=30, Cr=30, grainY=0, grainC=0, sample_mode=2, blur_first=true, dynamic_grain=false, mt=true, keep_tv_range=true)


so grainY=0, grainC=0 and dynamic_grain=false will make sure that you don't add any grain to luma and chroma, thus making compressibility better.

[ChaosKing]

He wrote the file was bigger without neo_f3kdb, so with neo_f3kdb it is smaller.

[MeteorRain]

Oops, then I don't know.

[Boulder]

Yes, folks, it was the other way around which is why I was confused. Debanding made the result a whole lot smaller than the one in which the debanding line was commented out. Really weird.. and with SDR sources it definitely works like expected with an increase in filesize.

So I made another test, source with borders cropped off and downscaled to 2560x1064 with resamplehq, kernel bicubic and b=-0.56, c=0.25. 2673 frames in the test clip. AQ-mode 1, aq-strength 1.0, CRF 13 and pretty much preset --slower + HDR options in x265. In this case where I chose a scene with much more light (where K goes to visit Gaff), the difference is not so big. The earlier one was from a dark scene closer to the end of the movie where Deckard tries to shoot K.

No debanding: 6384 kbps
core.neo_f3kdb.Deband(clp, preset="medium", output_depth=16, sample_mode=4): 6069 kbps
core.neo_f3kdb.Deband(clp, preset="medium/nograin", output_depth=16, sample_mode=4): 5740 kbps

This is the amplified difference between the clip before and after debanding, so it's clearly adding a layer of grain over the whole frame (even with nograin in the preset):
https://i.ibb.co/RyJxwNG/bladerunner-vpy-105107.png (https://ibb.co/bJGS9Fj)

[MeteorRain]

I see, that's a bug in parsing preset string. The code didn't recognize slashes so "medium/nograin" is considered an invalid preset. For the time being can you test it without using slash preset? You can try preset="medium" with grainy/c = 0.

[Boulder]

I see, that's a bug in parsing preset string. The code didn't recognize slashes so "medium/nograin" is considered an invalid preset. For the time being can you test it without using slash preset? You can try preset="medium" with grainy/c = 0.

Thanks, that definitely fixed the grain thing and there's mostly the banded areas visible in the diff in frames with a lot of light.
I re-encoded the sample and it came out at 5800 kbps.

[Boulder]

Maybe the encoder doesn't like the banding, very hard to say. I definitely need to test this on the TV, as my PC display is a normal one. But I can tell that the non-graded frames look better in Vapoursynth Editor when debanded so maybe it's also true on the TV after encoding.

[ChaosKing]

I would say banding = hard edges = more bitrate is needed to preserve them. But 35% still is impressive, and it looks overall better I guess?
And there is also something like "moving banding", this would also disappear.

[mastrboy]

I got a warning in AVSPmod I've never seen before when loading this plugin:
neo-f3kdb.dll
Above plugin names contain undesirable symbols.
Rename them to only use alphanumeric or underscores,
or make sure to use them in short name style only.

Should I assume it is safe to ignore this warning?

[FranceBB]

I got a warning in AVSPmod I've never seen before when loading this plugin:
neo-f3kdb.dll
Above plugin names contain undesirable symbols.
Rename them to only use alphanumeric or underscores,
or make sure to use them in short name style only.

Should I assume it is safe to ignore this warning?

That's something that always pops up in AVSPmod when using dash like "plugin-name.dll", however I never had any troubles other than a simple warning when I open AVSPmod.
Besides, no other software seem to care and it's been like that for years, so I don't know, but I think it's kinda safe to ignore the warning...
My best guess is that this comes from the fact that AVSPmod *thinks* that the plugin name corresponds to the function name and therefore issues a warning.
In other words, although the DLL is called "neo-f3kdb" the function that you call inside Avisynth is "neo_f3kdb" which is a perfectly valid function, however if it was with the "-" instead of "_" it could have caused problems inside Avisynth, that's why AVSPmod complains.
(If you want it to disappear, though, you can easily rename it to neo_f3kdb.dll yourself).

[mastrboy]

(If you want it to disappear, though, you can easily rename it to neo_f3kdb.dll yourself).

Thanks for the quick reply and especially that rename tip.

[MeteorRain]

core.neo_f3kdb.Deband(clp, preset="medium/nograin", output_depth=16, sample_mode=4)

This should be working in r6.

[Dorohedoro]

Sorry guys, I'm a total noob on this things and I need help.

How can I use this on potplayer or ffdshow raw? I can't get this to work. For example, I'm using this in AviSynth (ffdshow):

LoadPlugin("C:\.....\neo-f3kdb.dll")
neo_f3kdb(clip, y=64, cb=64, cr=64, grainy=0, grainc=0)

and I'm getting an error saying "I don't know what clip means" or something like that.

Can anyone help me please? Thank you so much.

[Boulder]

You don't need the "clip" parameter there at all if your script is like that.

[Dorohedoro]

Cam you give me an example of a working script please?

Because this doesn't work, I'm also getting an error on line 2

LoadPlugin("C:\.....\neo-f3kdb.dll")
neo_f3kdb(y=64, cb=64, cr=64, grainy=0, grainc=0)

Thanks!

[Boulder]

What error do you get with that one?

[Dorohedoro]

Unknown parameter or something like that (I'm at work right now and I don't remember, sorry)

[Dorohedoro]

Got it working on ffdshow but it's too demanding for my pc.

Thanks.

[tormento]

I saw r7 on GitHub but no release yet. Why?

[GMJCZP]

I am migrating to this version but I am getting a 1280x480 green image, when it should be 640x480, I am using the r6 msvc x86 on w7 32 bit c2d e5800. My old script is this:

f3kdb(0,0,0,0,0,0, keep_tv_range=true, dither_algo=2, input_mode=2, input_depth=12, output_mode=0, output_depth=8) # for avs+

Now I am using like this but with the aforementioned problem:

neo_f3kdb(0,0,0,0,0,0, keep_tv_range=true, dither_algo=2, output_depth=8) # for avs+

Any idea please?

[GMJCZP]

I am not an expert in bit conversions, I have used f3kdb with the script that I put previously but with neo_f3kdb I don't know how to do it, I have a 12-bit video that converted to 8 bits.

Edit: Can't seem to do bit conversions with neo_ f3kdb

[mp3dom]

It works right on my end, tried with colorbars with 12bpc (4:2:0, 4:2:2 and 4:4:4) and all produce correct result (converted to 8bpc).
How are you loading your source? Unlike old f3kdb, neo_f3kdb accepts native hbd with no hacks (stacked or double width)

[GMJCZP]

LwLibavVideoSource("Video.mp4")

Thanks

Edit: Now I use Clang x86 version

[GMJCZP]

I don't want to back to f3kdb only for this problem, i need help

[Arx1meD]

In my videos, I used your neo_f3kdb settings and there was no problem.
As I understand it, you are loading a video with a color depth of 12 bits in AviSynth.
Try it:ConvertBits (12) # or comment this line
neo_f3kdb(0,0,0,0,0,0, keep_tv_range=true, dither_algo=2)
ConvertBits (8)

[GMJCZP]

In my videos, I used your neo_f3kdb settings and there was no problem.
As I understand it, you are loading a video with a color depth of 12 bits in AviSynth.
Try it:ConvertBits (12) # or comment this line
neo_f3kdb(0,0,0,0,0,0, keep_tv_range=true, dither_algo=2)
ConvertBits (8)

Thanks but the problem continúes, a green image of 1280x480

[mp3dom]

Does your processor supports SSE4.1?

[GMJCZP]

Does your processor supports SSE4.1?

It seems not, e5800 Dual core 45 nm.

Edit: fron wiki:

Requirements
AviSynth+ (x86/x64)
Supported color formats: Y8, YV12, YV16, YV24, [all planar Y/YUV formats (8/10/12/14/16bit) are supported]

[mp3dom]

Uhm, do you still see errors with this code?


colorbars(pixel_type="yuv422p12")
neo_f3kdb(output_depth=8)

[GMJCZP]

Uhm, do you still see errors with this code?


colorbars(pixel_type="yuv422p12")
neo_f3kdb(output_depth=8)


I'm getting the 640x480 colorbars at 29,970fps, but it's telling me virtualdub2 in File Information is on the internal dib decoder (yv16). I also use VirtualDubMod and it tells me that Couldn't locate decompressor for format 'yv16' (unknown).

Edit: if I delete neo_f3kdb line obtain a P216 colorbars clip.

[mp3dom]

It's perfectly fine that you're getting yv16 output colorbars, since neo_f3kdb is converting the bit depth from 12 to 8, and planar 8bit for yuv422 is yv16. If you're not getting any trouble and you see the colorbars right, then it shouldn't be a neo_f3kdb problem the green video you're getting, or it doesn't fully depends on neo_f3kdb.
You said you're opening the file with LWLibav... how you see the file with just plain LWLibav (no neo_f3kdb) and what's its colorspace inside VDub2?

[GMJCZP]

It's perfectly fine that you're getting yv16 output colorbars, since neo_f3kdb is converting the bit depth from 12 to 8, and planar 8bit for yuv422 is yv16. If you're not getting any trouble and you see the colorbars right, then it shouldn't be a neo_f3kdb problem the green video you're getting, or it doesn't fully depends on neo_f3kdb.
You said you're opening the file with LWLibav... how you see the file with just plain LWLibav (no neo_f3kdb) and what's its colorspace inside VDub2?

Just using LwLibavVideoSource I get a 1280x480 green image and it says Vdub2 in Decompressor that it is Internal DIB decoder (Yv12).

[mp3dom]

That's the culprit... maybe you can try to update LSMASH (pick one of the latest compile).
12 bpc file on mp4 it sounds a bit weird... what's also telling you MediaInfo?

[GMJCZP]

I have the last version of Lsmash, the 20210423.

Mediainfo data:

Format : HEVC
Format/Info : High Efficiency Video Coding
Format profile : Format Range@L3@Main
Codec ID : hvc1
Codec ID/Info : High Efficiency Video Coding
Duration : 21 min 2 s
Bit rate : 1 384 kb/s
Maximum bit rate : 3 199 kb/s
Width : 640 pixels
Height : 480 pixels
Display aspect ratio : 4:3
Frame rate mode : Constant
Frame rate : 25,000 FPS
Color space : YUV
Chroma subsampling : 4:2:0
Bit depth : 12 bits
Bits/(Pixel*Frame) : 0.180
Stream size : 208 MiB (93%)
Writing library : x265 2.0+4-43ca544799c2:[Windows][MSVC 1900][64 bit] 12bit
Encoding settings : wpp / ctu=64 / min-cu-size=8 / max-tu-size=32 / tu-intra-depth=1 / tu-inter-depth=1 / me=1 / subme=2 / merange=57 / no-rect / no-amp / max-merge=2 / temporal-mvp / no-early-skip / rskip / rdpenalty=0 / no-tskip / no-tskip-fast / strong-intra-smoothing / no-lossless / no-cu-lossless / no-constrained-intra / no-fast-intra / open-gop / no-temporal-layers / interlace=0 / keyint=250 / min-keyint=25 / scenecut=40 / rc-lookahead=20 / lookahead-slices=0 / bframes=4 / bframe-bias=0 / b-adapt=2 / ref=3 / limit-refs=3 / no-limit-modes / weightp / no-weightb / aq-mode=1 / qg-size=32 / aq-strength=1.00 / cbqpoffs=0 / crqpoffs=0 / rd=3 / psy-rd=2.00 / rdoq-level=0 / psy-rdoq=0.00 / no-rd-refine / signhide / deblock=0:0 / sao / no-sao-non-deblock / b-pyramid / cutree / no-intra-refresh / rc=crf / crf=21.0 / qcomp=0.60 / qpmin=0 / qpmax=51 / qpstep=4 / ipratio=1.40 / pbratio=1.30
Encoded date : UTC 2016-07-18 23:03:09
Tagged date : UTC 2016-07-18 23:03:14
Codec configuration box : hvcC

[mp3dom]

Have you tried also LSMASHVideoSource? Same results? Otherwise you can also try to force the output to 16bit:
LWLibavVideoSource(source,format="yuv420p16")

[GMJCZP]

Have you tried also LSMASHVideoSource? Same results? Otherwise you can also try to force the output to 16bit:
LWLibavVideoSource(source,format="yuv420p16")

Both with LsmahVideoSource and with the script that you propose, I get the same greenish image of 1280 * 480. If I add format = "yuv420p8" I do get the correct image but I would be giving up editing in Hbd and neo_f3kdb.

[mp3dom]

I think the last chance would be to share a trimmed part of your file so everyone can try to solve this.

[GMJCZP]

Sample:

Here (http://www.filedropper.com/sample_10)

[mp3dom]

I still find no issue whatsoever.
Correctly opened with both LW and LSMASH, and with VDub2 with ffmpeg decoder as well. Unless it's a recent bug of LW/LSMASH (you can try to use an older version just in case) I think you have something wrong somewhere on your system.

[GMJCZP]

I still find no issue whatsoever.
Correctly opened with both LW and LSMASH, and with VDub2 with ffmpeg decoder as well. Unless it's a recent bug of LW/LSMASH (you can try to use an older version just in case) I think you have something wrong somewhere on your system.

But with neo_f3kdb?
What is your script?

Edit: with Dss2mod and Vdub2 directly i can Open the File without problem.

Edit2: If I can open the file with dss2mod then the following question arises, why then use neo_f3kdb or f3kdb? of course this doesn't solve the problem with l-smash.

[mp3dom]

neo_f3kdb is not the problem here, because you told me that even the plain LWLibav (opening plugin) give you the problem. You can try an older version of Lsmashsource and see if something change (btw I'm using a 07/2020 version of the plugin).
If you open the file with dss2mod I guess you're opening it at 8bit with (good) or without (bad) the dither applied, so you lose hbd processing. F3kdb main use is for debanding, but can be used for dithering.
You have the latest Lsmashsource and latest neo_f3kdb... do you also have latest avs+ 3.7?

[GMJCZP]

neo_f3kdb is not the problem here, because you told me that even the plain LWLibav (opening plugin) give you the problem. You can try an older version of Lsmashsource and see if something change (btw I'm using a 07/2020 version of the plugin).
If you open the file with dss2mod I guess you're opening it at 8bit with (good) or without (bad) the dither applied, so you lose hbd processing. F3kdb main use is for debanding, but can be used for dithering.
You have the latest Lsmashsource and latest neo_f3kdb... do you also have latest avs+ 3.7?

You were right, the problem was with Lsamsh, I downloaded version 20200111 and everything now works perfectly, including neo_f3kdb, thanks.
In short, at some point in the development of Lsmash a bug was introduced.

PD: recently update to last version of Avs+

[StainlessS]

LWLibavVideoSource(".\Sample_yuv420p12le.mp4") # LSMASHSource_x86_20200322(HolyWu).dll
info

https://i.postimg.cc/3dc42rgK/g.jpg (https://postimg.cc/3dc42rgK)
click to see Info().

EDIT: OK, you got it sorted.

[GMJCZP]

The bug was introduced this year, I tested the 20200728 Lsmash version and works.

[GMJCZP]

LWLibavVideoSource(".\Sample_yuv420p12le.mp4") # LSMASHSource_x86_20200322(HolyWu).dll
info

https://i.postimg.cc/3dc42rgK/g.jpg (https://postimg.cc/3dc42rgK)
click to see Info().

EDIT: OK, you got it sorted.

TinMan, how did you go about getting the information from the video in yellow letters?

[StainlessS]

already posted, ie

LWLibavVideoSource(".\Sample_yuv420p12le.mp4") # LSMASHSource_x86_20200322(HolyWu).dll
info <<<<<<======= Here !!!
It would be a big help at times, if LSmash (and others) added version resouce to their dll's,
so we knew what ver$ worked, or had problems.

[EDIT: downloading all versions and comparing file size/MD5 is a real pain]

[GMJCZP]

already posted, ie

LWLibavVideoSource(".\Sample_yuv420p12le.mp4") # LSMASHSource_x86_20200322(HolyWu).dll
info <<<<<<======= Here !!!
It would be a big help at times, if LSmash (and others) added version resouce to their dll's,
so we knew what ver$ worked, or had problems.

[EDIT: downloading all versions and comparing file size/MD5 is a real pain]

Thanks, with a celular maybe difficult see some things.

The problem was introduced from this year, at least the latest version

[StvG]

No issue with latest lsmash version (20210423):

https://images2.imgbox.com/c6/d1/PFOttvu9_o.png

[GMJCZP]

No issue with latest lsmash version (20210423):

https://images2.imgbox.com/c6/d1/PFOttvu9_o.png

Thanks StvG, your comment motivated me to take action on the matter...

I was finally able to solve the problem, as follows:

- I downloaded the latest version of Lsmash again

- Then I noticed, by chance, that some videos were played wrong by mpc-hc, I updated it as well as lav filters (when I uninstalled it I had problems reinstalling it but it was solved)

- Finally I tried the video and it worked correctly.

It still seems a mystery to me that the latest version did not work at first, it is possible that there was a conflict with the programs I mentioned before.

Thanks everyone for your help.

[StainlessS]

Some time ago, there was some kind of problem concerning a VDub source plugin, and either ffmpeg or FfmpegSource, dll's.
It was something to do with path search order to find a dll, one softwares change in search order interfered with the other.
No idea if relevant.

[kedautinh12]

New commits, can anyone compile it??
https://github.com/HomeOfAviSynthPlusEvolution/neo_f3kdb

[l00t]

New commits, can anyone compile it??
https://github.com/HomeOfAviSynthPlusEvolution/neo_f3kdb

You can get it from Asd-g's plugin pack: https://gitlab.com/uvz/AviSynthPlus-Plugins-Scripts

[kedautinh12]

Thanks

[GMJCZP]

Thanks

But is there the r7 version?

[kedautinh12]

No, it's new compile 20 hours ago. It's still r6 but added new commits by asd-g

[StvG]

It's r7.

https://images2.imgbox.com/9c/1d/hfGdnnnT_o.png

[kedautinh12]

Oh, my mistake, sr

[kedautinh12]

New ver neo-f3kdb
https://gitlab.com/uvz/AviSynthPlus-Plugins-Scripts

[GMJCZP]

New ver neo-f3kdb
https://gitlab.com/uvz/AviSynthPlus-Plugins-Scripts

Thanks. Although it says that it is also an r7 version, the size is larger than the previous r7 version (282 kb against 224 kb).

[real.finder]

Thanks. Although it says that it is also an r7 version, the size is larger than the previous r7 version (282 kb against 224 kb).

if you use avs then you don't need to update anyway, the update is for vs only (there was bug in vs case)

[almosely]

Hideho,

me, again ;-)

Recently I installed Release 7 and used it with mt=true for a few encodings without problems.

But my latest encoding crashed at around 30% with mt=true and at around 60% with mt=false. Now I re-installed Release 6 and the encoding finished without problems.

My hardware:

- Intel Core i7-3770

- ASUS GTX660 TI

- 16 GB DDR3 SDRAM

My software:

- Windows 10 Pro 20H2 64-Bit

- Simple x264/x265 Launcher 64-Bit (2021-05-29)

- AviSynth+ 3.7.0 64-Bit

- L-SMASH-Works (2021-04-23) 64-Bit

- TIVTC 1.0.26 (22-02-2021) 64-Bit

- neo_f3kdb r6 (30-05-2020) 64-Bit (working)

- neo_f3kdb r7 (05-07-2021) 64-Bit (not working)

My AVS-script (used that script for about one year without problems):

SetCacheMode(1)

LWLibavVideoSource("Movie.mkv", format="YUV420P8", prefer_hw=1)

RequestLinear(rlim=200, clim=200, elim=100)

Crop(0, 138, -0, -138)

Spline64Resize(1280, 536)

Prefetch(4,8)

sharpen(0.128)

Prefetch(4,8)

neo_f3kdb(range=15, y=48, cb=48, cr=48, grainy=0, grainc=0, dither_algo=2)

Prefetch(4,8)

return last

My x265-parameters within Simple Launcher for nearly all 720p-encodings for about a year:

Encoder: x265 (HEVC)
Architecture: 64-Bit (x64)
Variant: 8-Bit
Rate Control: CRF
Quantizer/CRF: 17,0
Preset: medium
Tune: <None>
Profile: <Unrestricted>

Custom Encoder Parameters:

--no-open-gop --selective-sao 1 --lookahead-slices 1 --ref 5 --bframes 4 --weightb --rc-lookahead 40 --me 3 --subme 4 --max-merge 4 --hevc-aq --rd 4 --rdoq-level 2 --psy-rd 4.5 --psy-rdoq 8.0 --ctu 32 --merange 26 --keyint 240 --sar 1:1 --pools 8

I tested and refined these encoding (and AVS-script) settings a few weaks and tried nearly every x264 and x265 parameter (and resize and sharpening method for 1080p to 720p), checked with my eyes and FFMetrics, tuned for quality, speed and compatibility. These settings replaced my old x264 crf 17 settings as followed:

Quantizer/CRF: 17,0
Preset: slower

--rc-lookahead 40 --threads 13 --me umh --subme 10 --merange 24 --deblock -1:-1 --psy-rd 1.0:0.15 --aq-strength 1.0 --bframes 5 --ref 5 --keyint 240 --aq-mode 3 --sar 1:1

and:

Quantizer/CRF: 18,5
Preset: slower

--rc-lookahead 80 --threads 18 --me umh --subme 10 --merange 32 --deblock -1:-1 --psy-rd 1.0:0.15 --aq-strength 1.0 --bframes 5 --ref 9 --keyint 240 --aq-mode 3 --sar 1:1 --non-deterministic

(both with BlackmanResize within AVS, without sharpening).

So, for me, r7 is not working.

[Boulder]

Remove all those extra Prefetch lines and leave just one there as the last item of your script. You also do not need "return last" at all.

[almosely]

I tested and refined these encoding (and AVS-script) settings a few weaks and tried nearly every x264 and x265 parameter (and resize and sharpening method for 1080p to 720p), checked with my eyes and FFMetrics, tuned for quality, speed and compatibility.

I used to set only one "Prefetch(x)" at the end of my scripts before AviSynth got the update to handle multithreading differently ("Prefetch(x,y)").

Setting "return last" seems to be a question of faith, hm?

[kedautinh12]

Updated r8:
https://github.com/HomeOfAviSynthPlusEvolution/neo_f3kdb/releases

[PoeBear]

Updated r8:
https://github.com/HomeOfAviSynthPlusEvolution/neo_f3kdb/releases

What are the use cases of the new parameters: Y2, Cb2, Cr2?

[ChaosKing]

Source code says: `Y`, `Cb`, `Cr` are internally left shifted by `2` while `Y2`, `Cb2`, `Cr2` - by `5`.

[Julek]

What are the use cases of the new parameters: Y2, Cb2, Cr2?

https://github.com/HomeOfAviSynthPlusEvolution/neo_f3kdb/discussions/16#discussioncomment-3592269

[PoeBear]

https://github.com/HomeOfAviSynthPlusEvolution/neo_f3kdb/discussions/16#discussioncomment-3592269

What are the use cases though? Ie, what types of sources would we want to use the new parameters vs the old parameters? Were there issues/limitations with the old parameters, is it an 8 vs 16-bit thing, should we always use the new ones going forward, etc?

[Greenhorn]

What are the use cases though? Ie, what types of sources would we want to use the new parameters vs the old parameters? Were there issues/limitations with the old parameters, is it an 8 vs 16-bit thing, should we always use the new ones going forward, etc?

Stronger debanding, I guess. y/cb/cr are limited (and normalized, so they're ~the same regardless of bitdepth) to the range of 0-511, and this appears to allow circumventing that limitation without rewriting whatever code causes it-- "y=0,y2=64" produces seemingly the exact same output as "y=511,y2=-1", and debanding strength continues to get stronger as you increase y2 past 64.

I'm not sure how practical this actually is, but I guess this is meant to make f3kdb more competitive with other inherently stronger filters for really really messed up videos.

[kedautinh12]

Stronger debanding, I guess. y/cb/cr are limited (and normalized, so they're ~the same regardless of bitdepth) to the range of 0-511, and this appears to allow circumventing that limitation without rewriting whatever code causes it-- "y=0,y2=64" produces seemingly the exact same output as "y=511,y2=-1", and debanding strength continues to get stronger as you increase y2 past 64.

I'm not sure how practical this actually is, but I guess this is meant to make f3kdb more competitive with other inherently stronger filters for really really messed up videos.

Y2, Cb2, Cr2 (> r7)

When specified respectively Y, Cb, Cr do not have effect

[kedautinh12]

neo_f3kdb r9
https://github.com/HomeOfAviSynthPlusEvolution/neo_f3kdb/releases

[kedautinh12]

neo_f3kdb r10
https://github.com/HomeOfAviSynthPlusEvolution/neo_f3kdb/releases

Added Arm Neon support. (@yuygfgg)
Added sample_mode=6. (#13)
Added sample_mode=7. (#13)
Changed cache size to use only the required size.
sample_mode=5: made the output of scalar and SSE code identical.
Fixed CPU instructions detection when MSVC is used.
Added AVX2 and AVX512 code for sample_mode=5.
Changed range limit

[FranceBB]

Well, it seems to be working on Windows 10 Professional x64 and especially on my AVX512 capable Intel Xeon running Windows Server 2019 Standard x64. So far so good. :)

[tormento]

Well, it seems to be working on Windows 10 Professional x64 and especially on my AVX512 capable Intel Xeon running Windows Server 2019 Standard x64. So far so good. :)
If you can, please update wiki with the new sample modes. ;)

[FranceBB]

I will, but in the meantime, for those wondering, the build is not compatible with Windows XP / Windows Server 2003.

https://i.imgur.com/wOkqRGB.png

Even after getting rid of all the AVX512 and AVX2 references, remapping the SRW to the non SRW equivalents and the EX to the non EX equivalent, compiling with v141_xp and setting /Zc:threadSafeInit- it still won't work as it's using the new threading mechanism introduced in Windows Vista / Windows Server 2008 onward.

https://i.imgur.com/izmWK9Y.png

Looks like it's the end of the road. Those who want to use f3kdb on XP / Server 2003 can still do it, but as far as neo f3kdb goes I suggest sticking to version 5 which is the last still supported stable release that is still fully compatible without modifications / workarounds.