Welcome to Doom9's Forum, THE in-place to be for everyone interested in DVD conversion.

Before you start posting please read the forum rules. By posting to this forum you agree to abide by the rules.

 

Go Back   Doom9's Forum > Capturing and Editing Video > VapourSynth

Reply
 
Thread Tools Search this Thread Display Modes
Old 27th May 2011, 07:46   #1  |  Link
SAPikachu
Registered User
 
SAPikachu's Avatar
 
Join Date: Aug 2007
Posts: 218
flash3kyuu_deband 1.5.1 / 2.0pre2 with native VapourSynth support [2012-12-03]

Hi all,

I've ported a deband filter for AviUtl (http://www.geocities.jp/flash3kyuu/auf/banding17.zip) to AviSynth. Its algorithm is simple but it is quite effective for some anime sources.
Code:
2.0pre2 with native VapourSynth support: http://nmm.me/tr

Some notes:

* This is still an experimental version, it has not been fully 
  tested, so please use it with caution.
* sample_mode=0 and dither_algo=0 are removed in this 
  version since nobody is using them.
* Document has not been updated yet, please refer to test.vpy 
  for sample usage. All parameters are in lower case.
* Because input_mode/input_depth/output_mode can be 
  implied from clip properties, they are not settable in VS 
  interface. output_depth is still available though.

2.0pre2:
* Fixes compatibility with XP
* Exports public API, can be used as a standalone library
Change log:
1.5.1 (2012-04-07)
* Supports setting StdDev (sigma) for the Gaussian random number generator

1.5.0 (2012-03-12)
* (There isn't any new feature in this version, only some parameters are modified to reduce user confusion)
* ditherY/ditherC are renamed to grainY/grainC
* dynamic_dither_noise is renamed to dynamic_grain
* precision_mode is renamed to dither_algo, mode 4 and 5 are removed
* random_algo_dither is renamed to random_algo_grain
* enable_fast_skip_plane is removed, this optimization will be enabled mplicitly whenever possible (Filter result won't be changed by this optimization)

1.4.2 (2011-11-10)
* Fixed crash on some non-mod16 videos

1.4.1 (2011-11-05)
* Fixed broken YUY2 support (still slow)
* Improved default value handling of bitdepth-related parameters
* precision_mode 4 / 5 are now deprecated and may be removed in future versions, you can use output_mode 1 / 2 to achieve the same result

1.4.0 (2011-10-30)
* 9 ~ 16 bit-depth input/output
** Related parameters: input_mode/input_depth/output_mode/output_depth
* New random number generator, reference position and dither noise can be generated in uniform or gaussian distribution
** Related parameters: random_algo_ref / random_algo_dither
* diff_seed is replaced with dynamic_dither_noise, when enabled, noise pattern will be different for each frame
* Another new parameter: enable_fast_skip_plane
* Short filter alias: f3kdb
* Now the ICC-compiled DLL should be runnable on pre-SSE2 systems (untested)
* Several bug fixes

1.3.0 (2011-09-07)
* Added x64 version
* Added a downsample filter: f3kdb_dither
* Internal precision is increased to 16bit
* New parameter: keep_tv_range, please see readme.txt for details
* Default sample_mode is changed to 2 as it is better in most cases
* Fixed: Floyd-Steinberg dithering may produce incorrect result for full-range videos
* Fixed: Broken YUY2 debanding
* Minor optimizations

1.2.0 (2011-08-01)
* Added support for YUY2 (not optimized yet)
* Added support for all planar YUV format in AviSynth 2.6
* The filter is now compatible with both AviSynth 2.5 and 2.6
* 16bit output (precision_mode = 4/5)
* Note: The internal processing precision is still 14bit, this will be
improved in future versions

1.1.0 (2011-06-18)
* Fixed a bug that high threshold values would produce incorrect result in high precision modes.
* Threshold values was scaled based on other parameter in previous versions, it is unscaled now to increase flexibility. Using same value has weaker effect in new version. Effect of default parameter set is also changed.
* SSE optimization for high precision mode.
* Rejects some invalid parameter combination instead of silently skips them

1.0.2
* High precision mode
(currently non-optimized, SSE routine will be added later)
* Frame edges are properly processed now
* Fix crash in some cases (unaligned frames are handled correctly)
* Other bug fixes

Binary: x86 / x64 / MSVC
(The MSVC build is slower, use that only if the x86 build has problems)

Documentation: https://raw.github.com/SAPikachu/fla...yuu_deband.txt

Source code: https://github.com/SAPikachu/flash3k...and/tree/1.5.1

Samples: #1 #2 #3 #4 (1st in post #4) #5 (2nd in post #4) All

(Note: I used different image source for samples starting from 1.0.2, the original sources are not very banded indeed.)

Script for generating the samples (changed again on 6-19):
Code:
ImageReader("wallorig.png", end=0)
ConvertToYV12
pic=last

global i=0
    
LoadPlugin("flash3kyuu_deband.dll")
StackVertical(last.croppic.subtitle("original"), last.croppic.luma_histogram.subtitle("original"))


addtest(last,pic,"""flash3kyuu_deband(sample_mode=2,dither_algo=3)""") #1
addtest(last,pic,"""flash3kyuu_deband(sample_mode=2,dither_algo=3,y=64,cb=64,cr=64,grainy=32,grainc=32)""") #2
addtest(last,pic,"""flash3kyuu_deband(sample_mode=2,dither_algo=3,y=64,cb=64,cr=64,grainy=16,grainc=16)""") #3
addtest(last,pic,"""flash3kyuu_deband(sample_mode=2,dither_algo=3,y=64,cb=64,cr=64,grainy=0,grainc=0)""") #4
addtest(last,pic,"""flash3kyuu_deband(sample_mode=2,dither_algo=2,y=64,cb=64,cr=64,grainy=32,grainc=32)""") #5
addtest(last,pic,"""flash3kyuu_deband(sample_mode=2,dither_algo=1,y=64,cb=64,cr=64,grainy=32,grainc=32)""") #6
addtest(last,pic,"""gradfun3(smode=1,mask=0)""") #7
addtest(last,pic,"""gradfun3(smode=2,mask=0)""") #8



ConvertToRGB
ImageWriter("out",type="png")

function luma_histogram(clip c)
{
    return c.histogram("luma").ColorYUV(levels="PC->TV")
}

function croppic(clip c)
{
    return c.crop(100,100,400,400)
}

function addtest(clip c, clip pic, string expr)
{
    pic
    eval(expr)
    croppic
    
    
    global i=i+1
    expr="#" + string(i) + " " + expr
    
    StackVertical(last.subtitle(expr), luma_histogram.subtitle(expr))
    return StackHorizontal(c,last)
}
Comments are appreciated.

Last edited by SAPikachu; 3rd December 2012 at 14:22.
SAPikachu is offline   Reply With Quote
Old 27th May 2011, 09:05   #2  |  Link
CruNcher
Registered User
 
CruNcher's Avatar
 
Join Date: Apr 2002
Location: Germany
Posts: 4,949
Comparison with http://forum.doom9.org/showthread.ph...59#post1386559 seems missing
__________________
all my compares are riddles so please try to decipher them yourselves :)

It is about Time

Join the Revolution NOW before it is to Late !

http://forum.doom9.org/showthread.php?t=168004
CruNcher is offline   Reply With Quote
Old 27th May 2011, 09:12   #3  |  Link
mastrboy
Registered User
 
Join Date: Sep 2008
Posts: 287
could you provide some screenshots comparing this to GradFun2DBmod and using histogram(mode="luma") ?
mastrboy is offline   Reply With Quote
Old 27th May 2011, 10:22   #4  |  Link
cretindesalpes
͡҉҉ ̵̡̢̛̗̘̙̜̝̞̟̠͇̊̋̌̍̎̏̿̿
 
cretindesalpes's Avatar
 
Join Date: Feb 2009
Location: No support in PM
Posts: 610
Three comparisons done with flash3kyuu_deband v1.0.0, without High precision mode:


orig | gf2m | f3kds1 | f3kds2 | gf3sm0 | gf3sm1 | gf3sm2 (normal)
orig | gf2m | f3kds1 | f3kds2 | gf3sm0 | gf3sm1 | gf3sm2 (histogram)


orig | gf2m | f3kds1 | f3kds2 | gf3sm0 | gf3sm1 | gf3sm2 (normal)
orig | gf2m | f3kds1 | f3kds2 | gf3sm0 | gf3sm1 | gf3sm2 (histogram)


orig | gf2m | f3kds1 | f3kds2 | gf3sm0 | gf3sm1 | gf3sm2 (normal)
orig | gf2m | f3kds1 | f3kds2 | gf3sm0 | gf3sm1 | gf3sm2 (histogram)

orig = original
gf2m = Gradfun2dbMod (str=0)
f3kds1 = flash3kyuu_deband (sample_mode=1)
f3kds1 = flash3kyuu_deband (sample_mode=2)
gf3sm0 = GradFun3 (smode=0)
gf3sm1 = GradFun3 (smode=1)
gf3sm2 = GradFun3 (smode=2)

Conversion to RGB done with yv12torgb24hq ()

Edit 1: Histogram(mode="luma") looks broken and outputs PC-range lumas whatever the input range. I fixed the screenshots
Edit 2: One more comparison
Edit 3: imageshack.us resized silently most of the pictures so the comparison is now irrelevant.
__________________
dither 1.27.2 for AviSynth | avstp 1.0.3 for AviSynth development | fmtconv r19 for Vapoursynth | trimx264opt segmented encoding

Last edited by cretindesalpes; 24th August 2013 at 13:32. Reason: imageshack.us not trustable
cretindesalpes is offline   Reply With Quote
Old 27th May 2011, 10:40   #5  |  Link
SAPikachu
Registered User
 
SAPikachu's Avatar
 
Join Date: Aug 2007
Posts: 218
Added some more samples. Also thanks cretindesalpes for testing.
SAPikachu is offline   Reply With Quote
Old 27th May 2011, 18:17   #6  |  Link
mandarinka
Registered User
 
mandarinka's Avatar
 
Join Date: Jan 2007
Posts: 669
I see zero banding in the image in post #1 and only very very little banding in post #4 (not worht filtering imho). Monitor is Samsung SyncMaster 226bw (not exactly expensive TN, 4 years old), without any special calibration...

Anyone with good/calibrated display can comment? It's tricky if one can without knowing run debanding just because of deficiences of particular lcds...
mandarinka is offline   Reply With Quote
Old 28th May 2011, 02:34   #7  |  Link
SAPikachu
Registered User
 
SAPikachu's Avatar
 
Join Date: Aug 2007
Posts: 218
The original image have banding in lower sky. I can't find an image with more serious banding at this time... The third image in #4 should be more visible.
SAPikachu is offline   Reply With Quote
Old 28th May 2011, 02:44   #8  |  Link
Hiritsuki
Novice of AVS
 
Join Date: Oct 2009
Posts: 156
May I ask which is this filter used? GPU or CPU?
__________________
My PC
Hiritsuki is offline   Reply With Quote
Old 28th May 2011, 02:47   #9  |  Link
SAPikachu
Registered User
 
SAPikachu's Avatar
 
Join Date: Aug 2007
Posts: 218
It runs on CPU. It should be pretty fast with current CPU though.
SAPikachu is offline   Reply With Quote
Old 28th May 2011, 03:42   #10  |  Link
Hiritsuki
Novice of AVS
 
Join Date: Oct 2009
Posts: 156
Thank you. It's seems amazing.
I'll try it.
__________________
My PC
Hiritsuki is offline   Reply With Quote
Old 28th May 2011, 03:43   #11  |  Link
Hiritsuki
Novice of AVS
 
Join Date: Oct 2009
Posts: 156
BTW, this filter support with MT?
__________________
My PC
Hiritsuki is offline   Reply With Quote
Old 28th May 2011, 03:51   #12  |  Link
SAPikachu
Registered User
 
SAPikachu's Avatar
 
Join Date: Aug 2007
Posts: 218
I didn't test that, but it should support MT mode 2 with appropriate overlap.
SAPikachu is offline   Reply With Quote
Old 28th May 2011, 10:00   #13  |  Link
Chikuzen
typo lover
 
Chikuzen's Avatar
 
Join Date: May 2009
Posts: 597
Hi.
@SAPikachu
some Japanese on 2ch complain that "AviUtl's original can adjust params by 32 stages(0 to 31), but this can be adjusted only by three stages(value 15 of original corresponds to value 1 of this) except range".

would you mind improving it?
__________________
my repositories

Last edited by Chikuzen; 28th May 2011 at 16:12.
Chikuzen is offline   Reply With Quote
Old 28th May 2011, 10:08   #14  |  Link
SAPikachu
Registered User
 
SAPikachu's Avatar
 
Join Date: Aug 2007
Posts: 218
Yes, I plan to add 12-bit (or even 16-bit) processing when I have time. Parameter stages will be the same as the original plugin then.
SAPikachu is offline   Reply With Quote
Old 28th May 2011, 10:16   #15  |  Link
Chikuzen
typo lover
 
Chikuzen's Avatar
 
Join Date: May 2009
Posts: 597
Quote:
Originally Posted by SAPikachu View Post
Yes, I plan to add 12-bit (or even 16-bit) processing when I have time. Parameter stages will be the same as the original plugin then.
i see, thx
__________________
my repositories
Chikuzen is offline   Reply With Quote
Old 28th May 2011, 16:00   #16  |  Link
Hiritsuki
Novice of AVS
 
Join Date: Oct 2009
Posts: 156
My friend say it's cannot use MT with over 4threads, I have 980X 6c12t to run and it's slow.
__________________
My PC
Hiritsuki is offline   Reply With Quote
Old 29th May 2011, 01:58   #17  |  Link
SAPikachu
Registered User
 
SAPikachu's Avatar
 
Join Date: Aug 2007
Posts: 218
Quote:
Originally Posted by Hiritsuki View Post
My friend say it's cannot use MT with over 4threads, I have 980X 6c12t to run and it's slow.
Is there any error message? Could you try setting mt=false when using AVS MT? For some reasons I can't test AVS MT at the moment...
SAPikachu is offline   Reply With Quote
Old 29th May 2011, 15:26   #18  |  Link
Hiritsuki
Novice of AVS
 
Join Date: Oct 2009
Posts: 156
No error message.
I means this filter only use 4threads for multithreads to run.
__________________
My PC
Hiritsuki is offline   Reply With Quote
Old 30th May 2011, 02:29   #19  |  Link
SAPikachu
Registered User
 
SAPikachu's Avatar
 
Join Date: Aug 2007
Posts: 218
Actually the filter only uses 2 threads internally. It is more complicated to use more threads, I won't change it until 12-bit processing is completed. You may want to try MT-related filter to speed up.
SAPikachu is offline   Reply With Quote
Old 7th June 2011, 05:16   #20  |  Link
SAPikachu
Registered User
 
SAPikachu's Avatar
 
Join Date: Aug 2007
Posts: 218
Released 1.0.2.
SAPikachu is offline   Reply With Quote
Reply

Tags
avisynth, deband

Thread Tools Search this Thread
Search this Thread:

Advanced Search
Display Modes

Posting Rules
You may not post new threads
You may not post replies
You may not post attachments
You may not edit your posts

BB code is On
Smilies are On
[IMG] code is On
HTML code is Off

Forum Jump


All times are GMT +1. The time now is 07:40.


Powered by vBulletin® Version 3.8.11
Copyright ©2000 - 2018, vBulletin Solutions Inc.