amDCT "adaptive multifunction DCT", deblock, smooth, sharpen, range expansion filter.

[jconklin]

amDCT() is an adaptive video filter providing deblocking, sharpening, local range expansion, smoothing, and bright noise removal in a single filter that can be used on video of any level of quality. These four operations work synergistically to drastically reduce block artifacts while maintaining detail and increasing local contrast.

A key advantage of amDCT--and one way in which it is a major advance over previous approaches--is that it determines how much smoothing, local range expansion and sharpening should be done at three different levels: the frame as a whole, block-by-block, and pixel-by-pixel.

Adaptivity was built into the core algorithm in order to provide a filter that could automatically adjust its strength to varying conditions in badly-encoded material so that good frames or good parts of bad frames would not be oversmoothed and bad frames or bad parts of good frames would be smoothed and deblocked more.


amDCT is built around a processing core modifed from my SmoothD2 filter that runs each block of a frame thru a DCT quant dequant inverse DCT process. Each frame is run through this process up to 64 times. Each time through the loop the frame is shifted up down left or right, and the resultant frame is then shifted back so that it is aligned with the original image. The values of each frame generated by this loop are added to each other and then averaged to produce a new frame.


The basic problem I found when using SmoothD2 which led me to start work on what became amDCT is that some frames needed more smoothing than others did.

So I decided I would find a way for the program to measure the amount of smoothing a frame needed and adjust the strength of smoothing accordingly.
After all, how hard could that be?
The gods laugh; the wife cries.
Well it turns out doing a first swipe at it is fairly easy. Just take an existing deblocker and measure the amount of change it makes in the frame and use that as a measure for the amount of smoothing you need.
The first rush of success. I've been through this before and I know I am hooked.
The problem is that you then start seeing that there are cases that the simple method of determining the global amount of smoothing doesn't work all of the time.
For example a blocky but almost black frame with a small area of light with a lot of detail. The deblocker tells you the frame is very blocky so you should strongly smooth the frame.
Unfortunately if you smooth it at a high level the area of detail gets smoothed out.

When you start getting a number of special cases like this you figure that a measure of the internal detail of each block might be useful.
It is useful. I am not an expert in this, so think of a hamster running in a wheel as I try all kinds of different measures and try to figure out which ones work best.

During that exercise I realize that I might have a measure of the amount of smoothing that I want for each block of the frame. So I figure out how to rebuild the DCT processing loop so that the amount of smoothing can be controlled on a block by block basis.
And it works. It took awhile to get it to work.

Hamster running really fast, wheel falls out of bearings, wheel and hamster fly out of cage and down the hall.
Acidentally discover that if you screw up the quant dequant code you can make the DCT processing loop do local range expansion instead of smoothing.

A NEW TOY!!! Spend a bunch of time messing around with the quant dequant code and figure out how to make it sharpen the image.
Two new problems. Range expansion intoduces halos on very strong black white borders. Sharppening introduces artifacts.
Sharpening artifacts can be reduced by presmoothing with a hand crafted matrix using shift=2.

Notice the most visible noise is in the brightest parts of the image and that the most oversmoothed parts are in dark areas.
Add controls that allow the user to specify at what brightness and at what strength dark areas should start being protected from being oversmoothed.
Add controls that allow the user to specify at what brightness and at what strength bright areas should start being more smoothed.
To add the extra smoothing at bright areas required building a very smoothed frame, shift=2, quant=6, qtype=2, which it turns out also makes an excellent smoothed frame for mvtools2() to use.
The very smoothed frame is also used to build a boundary mask to reduce bright halos that would be introduced by range expansion.

Hamster finds note taped on monitor
todo list not getting done
todo list will be glued to monitor.

Note understood.
amDCT version 1.0 is out the door.
Let me know what works, what doesn't, and what can be improved. My own improved list will easily take a year.
I think I just heard laughter.
Download code and manual https://sites.google.com/site/amdctfilter/
or just the manual https://docs.google.com/viewer?a=v&pid=sites&srcid=ZGVmYXVsdGRvbWFpbnxhbWRjdGZpbHRlcnxneDoyYmRkM2JiYWViNjEyMmIz

[ChaosKing]

Nice filter!

I get about 7.5fps on a 480p source i5 3570K@4.4Ghz with amDCT(adapt=28, quant=9, expand=16, sharpWAmt=8, quality=4) 3fps more with quality=3.

I had to set expand to 30 to see a diffenrence at first (it's an anime source)

Can you also provide a 64Bit build?

[Boulder]

Thanks a huge bunch! I've missed a good deblocker since I switched to VapourSynth, thus, I second ChaosKing's request for a 64-bit build :) SmoothD2 was the item in my toolbox for all DVD sources, and many Blu-rays would also benefit.

[hydra3333]

+1
thank you.

I've missed a good deblocker since I switched to VapourSynth So have I.
DG has an NV based gpu accelerated deblock filter on on his to-do list ...

[jconklin]

Thanks for trying amDCT so quickly.
3fps on 480p sounds about right. I have just started trying to get some gpu code "arrayfire.com" to work. There are 3 parts.
1. Get it to compile and link with my code.
2. replace the frame analysis code which should make it a bit faster.
3. Rebuild the whole multi threaded dct quant-dequant idct back end to take advantage of the gpu.

I have no idea how long #3 will take. I have never done gpu code.

Getting a 64 bit version out sounds easy. ... I think I just heard laughter.

What else do I need to do to build a 64-bit filter besides
Download and install 64bit avisynth. Do I need to uninstall the current avisynth?
Compile amDCT with visual studio 2015 with Configuration manager active solution platform set to CPU_x64
It can't be that easy.

I have a couple of priority items first and then
I will give it a try and see what happens.

[real.finder]

Thanks for trying amDCT so quickly.
3fps on 480p sounds about right. I have just started trying to get some gpu code "arrayfire.com" to work. There are 3 parts.
1. Get it to compile and link with my code.
2. replace the frame analysis code which should make it a bit faster.
3. Rebuild the whole multi threaded dct quant-dequant idct back end to take advantage of the gpu.

I have no idea how long #3 will take. I have never done gpu code.

Getting a 64 bit version out sounds easy. ... I think I just heard laughter.

What else do I need to do to build a 64-bit filter besides
Download and install 64bit avisynth. Do I need to uninstall the current avisynth?
Compile amDCT with visual studio 2015 with Configuration manager active solution platform set to CPU_x64
It can't be that easy.

I have a couple of priority items first and then
I will give it a try and see what happens.

you don't need avs64 nor avs+ 64 for that, you just need avisynth.h

if you still use avs25 I put the avisynth.h that work with it in Attachments

but it's better to use the one from avs+ and support new avs+ colour format

and you don't need install avs 64 for test too, you can use mp_pipeline to load avisynth .dll inside it block


LoadPlugin("E:\mpp\MP_Pipeline.dll") #32 one
MP_Pipeline("""
### platform: win64
### dll: E:\avs64\AviSynth.dll
LoadPlugin("E:\amDCT.dll") #64one
ColorBars.converttoyv12
your filter here
### ###
""")


run that in any normal 32 avs editor/player

[Boulder]

For the 64-bit VapourSynth to be able to use Avisynth plugins, you need to compile with Avisynth 2.6 headers. I think that's recommended anyway because Avisynth has been officially at v2.6 for some time now.

I requested a port last year in this thread: https://forum.doom9.org/showthread.php?t=173866 . As you can see, Myrsloik was not very optimistic :D

[Groucho2004]

Getting a 64 bit version out sounds easy. ... I think I just heard laughter.

What else do I need to do to build a 64-bit filter besides
Download and install 64bit avisynth. Do I need to uninstall the current avisynth?
Compile amDCT with visual studio 2015 with Configuration manager active solution platform set to CPU_x64
It can't be that easy.
It's not easy. I had a quick glance at your code, there's a lot of inline asm which would have to be converted to SIMD intrinsics to maintain reasonable performance. Compared to that, fixing other (potential) x64 porting issues is simple.
Installing AVS+ x64 alongside AVS(+) x86 is also no problem.

[jconklin]

Thanks Groucho2004 for looking at my code. It never crossed my mind that the Microsoft linker could not link a 32 bit dll to a 64 bit program.

I will end up building a 64 bit version of amDCT so that I can use one of the open source image processing libraries which tend to be 64 bit. This should also improve the speed of the DCT loop. To get a real speedup the DCT loop will need to run on the GPU, something I have no experience with.

The image processing libraries will allow me to quickly test some ideas I have for improving the block level adaptivity.

[TheFluff]

To clarify: inline asm doesn't compile in x64 mode. At all. You must either port all of it to intrinsics (do this) or switch to a separate assembler like YASM (don't do this). Good luck.

[Myrsloik]

To clarify: inline asm doesn't compile in x64 mode. At all. You must either port all of it to intrinsics (do this) or switch to a separate assembler like YASM (don't do this). Good luck.

You forgot option #3. GO FULL DOOM9 STYLE!

Intel compiler is best compiler (assuming you manage to replace all register names)

[jpsdr]

It's not necessary to switch to another assembler, you can still put all your inline asm in external asm file, and still continue to use the assembler provided by Visual Studio. This will avoid the need of other who want to compile your code to have to install third party things, you'll just only need Visual Studio, nothing more.

[jconklin]

It looks like it should be fairly easy to take all of the inline asm and put it in a separate .asm file and have nasm process it. This is how the XVID dct idct routines are done. They %include "nasm.inc" which contains the following lines.

%ifdef ARCH_IS_X86_64
BITS 64
DEFAULT REL
%define SECTION_ALIGN 32

...

%else
%define SECTION_ALIGN 16
BITS 32

I hope this means that the .asm routines are aready setup to build and link to both 32 and 64 bit code depending on the visual studio settings.

I have several projects that I have to get done before I move the inline assembler to its own file.

In the meantime I uploaded a VERY basic architecture doc "amDCT Software Architecture Picture.pdf" at https://sites.google.com/site/amdctfilter/

[CkJ]

Original
https://i.imgbox.com/LZncX7CY.png

amdct(quant=16, matrix=8)
https://i.imgbox.com/FCxVSNxL.png

Is this filter require mod8? Any chance it support for mod4 too? Thank you.

[jconklin]

CKJ Both the height and width of the input video needs to be evenly divisible by 8. Anytime I have delt with non mod8 clips I did a AddBorders(val, val, 0, 0).amDCT().Crop(val, val, 0, 0)

I really like the images. amDCT() will sometimes enhance the detail if expand is used. In this case it might bring out some of the details of the patterns in the water. If I think expand will help I usually start with expand=15.

[CkJ]

Look like the expand parameter darkens black areas. I don't think it can restores the details lost. I played with some amdct parameters. This is what i can deal with a heavy blocky image by using amdct.

mt_merge(last, last.amdct(shift=0, quant=4, qtype=2), last.histogram("luma").mt_deflate.greyscale.mt_invert)
https://i.imgbox.com/VFvV8q5c.png

[jconklin]

CKJ using shift=0 does almost no deblocking and in some cases will make blocking worse. Try shift=3. It will be slower but the deblocking should be better.

[CkJ]

I have found out that amdct work better with supersampling. My final result with the max value of shift:
filtered = spline36resize(last.width*4, last.height*4).amdct(adapt=31, shift=5, quant=8, matrix=8, qtype=2).spline36resize(last.width, last.height)
mt_merge(last, filtered, last.histogram("luma").mt_deflate.blur(.4).mt_inpand.greyscale.mt_invert)

https://i.imgbox.com/7Y1qeki2.png

[jconklin]

CKJ I took a copy of your image marked original, and ran it thru

amDCT(quality=4, adapt=31, quant=9, matrix=8, shift=5, qtype=1, expand=16, sharpWPos=6, sharpWAmt=24, sharpTPos=7, sharpTAmt=31, darkStart=64, darkAmt=10, brightStart=205, brightAmt=30)
https://sites.google.com/site/amdctfilter/home/pictures/More%20Detail40001.png?attredirects=0

The top left is the original. Top right is output from amDCT
The bottom left is the makediff of the original and the output of amDCT()
The bottom right is the makediff output expanded so the diff is more obvious.

Note that doing a lot of sharpening might cause flickering of the highlights between frames. One of the things that I would like to do is to pass a number of motion compensated frames to amDCT and use them for higher quality processing including flicker reduction. At one point I built a prototype out of smoothD2. It showed some promise but its speed was several seconds per frame. It is on my upgrade list after getting the main DCT processing loop to run on a gpu.

What the arguments are doing to the image.
Note that the image I am starting with will have gone thru a compression to .png and a decompression back to a frame for AviSynth. This process will have lost some detail in the image and might have introduced some more and different blocking than the image you are working with. Also the image you are seeing has gone thru another encoding decoding process.

My process in setting the values for amDCT for your image.
adapt=31
Note: Improvements needed frame size HD vs 480. interlaced converted to progressive often has smoothing done along the top and bottom edges of the blocks in a vertical column. This is not accounted for in the adaptation calculation. Among other things that the adaptation calculation is used for is to determine the strength of the block edge to block edge deblocking. If you really want to see the problems at the pixel and block edge level then in VirtualDub display your output setting zoom to 200%, 1to1 pixel, and point resize, and calibrate you monitors gray scale. Thats what I use. I don't recommend it. You will never be satisfied with any of your filters!!!

quant=9 Make as large as necessary to remove noise and blockiness.

and now I don't have any detail left.

darkStart=64, darkAmt=10 Find dark details that have been smoothed out. Use colors.exe available at http://www.den4b.com/ to find the values of the dark details that have disappeared. Set darkStart larger then that value and darkAmt large enough to bring back some of the blacks.

expand=16 get most of the details back. Possibly go back and increase quant.

brightStart=205, brightAmt=30 Notice that expand has made some of the white highlights too bright. Use colors.exe program to see what the original values of the over brightened highlights are. Set brightStart lower than this value and the values above brightStart will be compressed toward brightStart by an amount controlled by brightAmt. This also reduces the blocking artifacts that show up as short thin bright lines.

sharpWPos=6, sharpWAmt=24, sharpTPos=7, sharpTAmt=31 start dialing in fine detail. Go back to quant and check the values.

[CkJ]

Never heard about colors.exe program. Where do I can download it?

[StainlessS]

Never heard about colors.exe program. Where do I can download it?

Bottom of first post.

EDIT: Ignore me, just downloaded zip from first post, not in there, no idea where to get it, sorry :(

Did a search on-site for "Colors.exe", just got the hits from this thread, nothing else.

Also seached
"colors.exe" site:forum.doom9.org
same again.

[jconklin]

The program colors is available at http://www.den4b.com/

[Gser]

Any chance of getting an x64 build?

[jconklin]

I am working on it. I can cleanly build a simple test filter in win32 release and debug mode as well as x64 release and debug mode.
The win32 versions load and run nicely. The x64 versions won't load.
AviSynth+ 0.1(r2455, MT, i386). Got another step while writing this. AviSynth+ thinks the filter is not a valid Win32 application.
Getting the build system to work well is probably my least favorite thing about programming.

[Chyrka]

There is a small misprint in amDCTc.avs
sharpWAmt = default( sharpTAmt, 0 )
should be
sharpTAmt = default( sharpTAmt, 0 )

[Mounir]

can you make it "interlaced safe" ? that would be awesome

[jconklin]

Chyrka Thank you for finding the bug in amDCTc.avs. I have uploaded the corrected version.

Mouniir I am not sure what you mean by "interlaced safe".

Gser I want to get back to the x64 build. Unfortunately life intervened so haven't been able to spend time on it. I have a list of things I want to try at 64 bits.

[Gser]

Chyrka Thank you for finding the bug in amDCTc.avs. I have uploaded the corrected version.

Mouniir I am not sure what you mean by "interlaced safe".

Gser I want to get back to the x64 build. Unfortunately life intervened so haven't been able to spend time on it. I have a list of things I want to try at 64 bits.

Well I'm guessing he means adding this kind of code straight into the script that can be triggered by an option
i=LAST
SeparateFields().PointResize(i.width,i.height)
amDCT().AssumeFrameBased()
GetParity(i) ? AssumeTFF() : AssumeBFF()
SeparateFields().SelectEvery(4,0,3).Weave()

Yeah there are some other updated things to consider like using avs+ headers and high bit depth support. But I suppose most high bit depth sources are from either codecs that have in-loop deblocking or from source material that is encoded in high enough bit rates to avoid compression issues.

[0ther]

When trying to use amDCT, I always get a pop up: "amDCT: out of memory".

I have already tried playing around with SetMemoryMax().

Have no problems with other plug-ins so far.

Any idea what I can do?

[StainlessS]

Just thought that I would point out that avisynth header VERSION 5 (in amDCT directory as avisynth.h.2.6.a5.new), is obsolete and would likely cause
major problems if it is used in your source, should change to avisynth VERSION 6 header (used since avs 2.60 standard Alpha 4, I think).
Prior to VERSION 6 header, used AvisynthPluginInit2 [also for v2.58] rather than AvisynthPluginInit3 of avs 2.60 FINAL and avs+.
VERSION 6 header of AVS+ is also the better choice and compatible with avs 2.60 standard VERSION 6 header.

Requires code something like this to support v2.58 and v2.6

#ifdef AVISYNTH_PLUGIN_25
extern "C" __declspec(dllexport) const char* __stdcall AvisynthPluginInit2(IScriptEnvironment* env) {
#else
// New 2.6 requirement!!!
// Declare and initialise server pointers static storage.

const AVS_Linkage *AVS_linkage = 0;

// New 2.6 requirement!!!

// DLL entry point called from LoadPlugin() to setup a user plugin.
extern "C" __declspec(dllexport) const char* __stdcall
AvisynthPluginInit3(IScriptEnvironment* env, const AVS_Linkage* const vectors) {

// New 2.6 requirment!!!
// Save the server pointers.
AVS_linkage = vectors;
#endif

env->AddFunction("ApparentFPS",
"c[DupeThresh]f[FrameRate]f[Samples]i[ChromaWeight]f[Prefix]s[Show]b[Verbose]b[Debug]b[Mode]i[Matrix]iii[oLapY]i",
Create_ApparentFPS, 0);

return "`ApparentFPS' ApparentFPS plugin";
// A freeform name of the plugin.
}


and for header inclusion something like this


#include <windows.h>
#include <stdio.h>
#include <stdlib.h>
#include <time.h>
#include <math.h>

#ifdef AVISYNTH_PLUGIN_25
#include "avisynth25.h"
#else
#include "avisynth.h"
#endif



EDIT: Here, IanB post introducing the removal of Baked Code, and new VERSION 6 header requirements [just prior to v2.60 Alpha 4].
https://forum.doom9.org/showthread.php?p=1580772#post1580772

Dear All,

I have had a chance to review my first efforts, resolve a few issues and add functionality for host applications to establish the AVS_Linkage.

The good news.

I have tagged the main instance of the AVS_Linkage structure static and const and I have created a Dll exported const pointer to it, AVS_linkage. This provides a static means for applications that use Avisynth.dll functions to establish the AVS_Linkage. Also I have extended IScriptEnvironment:: with GetAVSLinkage() as a dynamic method to establish the AVS_Linkage. To use the static linkage method #define AVS_LINKAGE_DLLIMPORT immediately before your #include "avisynth.h". To use the dynamic method declare AVS_linkage as in a plugin and after creating env with the CreateScriptEnvironment() do AVS_linkage = env->GetAVSLinkage().

...

This of course breaks binary compatibility so any early API adopters will need to recompile to match this version. Hopefully no other nasties appear and break future binary compatibility.

Thoughts and comments please.
//------------------------
// USER_PLUGIN.cpp side
//------------------------
...
/* New 2.6 requirement!!! */
// Declare and initialise server pointers static storage.
[B]const AVS_Linkage *AVS_linkage = 0;

/* New 2.6 requirement!!! */
// DLL entry point called from LoadPlugin() to setup a user plugin.
extern "C" __declspec(dllexport) const char* __stdcall
AvisynthPluginInit3(IScriptEnvironment* env, [B]const AVS_Linkage* const vectors) {

/* New 2.6 requirment!!! */
// Save the server pointers.
AVS_linkage = vectors;

// Add the name of our function
env->AddFunction("Plugin", "c", Create_Plugin, 0);

// Return plugin text identifier.
return "Plugin";
}

[Seedmanc]

What is a "local range expansion"? Is it like local contrast?
I remember back then I was subtracting binomialBlur from a chain of Blur(1.58)'s in order to sharpen the middle range of frequencies while avoiding both the high-range noise and low-range edges.

[jconklin]

Other I will try recompiling my code with StainlessS and IanB suggestions and see if that works.

[jconklin]

What is a "local range expansion"? Is it like local contrast?
I remember back then I was subtracting binomialBlur from a chain of Blur(1.58)'s in order to sharpen the middle range of frequencies while avoiding both the high-range noise and low-range edges.
Seedmanc local range expansion is a form of local contrast enhancement. It is done in the main quant dequant loop. See "How amDCT Works" page 2 of amDCT.pdf.

It works at the block level, 8x8 pixels, using an internal matrix designed to increase the amount of brightness change, of lines in any orientation that are 1 to 8 pixels wide and 1 to 8 pixels long, with thinner lines getting more change.

The amount of brightness change will be reduced for blocks that have a high blockiness value.

[0ther]

When trying to use amDCT, I always get a pop up: "amDCT: out of memory".

I have already tried playing around with SetMemoryMax().

Have no problems with other plug-ins so far.

Any idea what I can do?

Can anybody help me with this?

[WaxCyl]

Can anybody help me with this?
I think the problem may be caused by the video pixel dimensions not being a whole multiple of 16
Say, for instance, your resolution is 1280 x 900
Lanczos4Resize(1280, 960)
amDCT()
Lanczos4Resize(1280, 900)

Or else you could add black bars on both sides.

I Hope this helps.

[jconklin]

I have posted a new version of amDCT at https://sites.google.com/site/amdctfilter/
The new file is amDCT_v1.1_Testing.zip
The major difference is it uses the AviSynth Plus header files.
I have tested it on the following avs releases using setavs.bat to switch
between different versions and AVSMeter to confirm the AviSynth version.


setavs . . . . . . . . . . . . AVSMeter 2.8.7 (x86) - Copyright (c) 2012-2018, Groucho2004
NAME. . . . . . . . . . . . . . . . . NAME
AVS260 . . . . . . . . . . . . . . AviSynth 2.60, build:Feb 20 2015 [03:16:45] (2.6.0.5)
AVS261_Alpha . . . . . . . . . AviSynth 2.60, build:Mar 31 2015 [16:38:54] (2.6.0.6)
VS260, AVS260MT . . . . . . AviSynth 2.61, build:May 17 2016 [16:06:18] VC2008Exp (2.6.1.0)
AVSPLUS_x86 . . . . . . . . . AviSynth+ 0.1 (r2728, MT, i386) (0.1.0.0)
AVSPLUS_x64 . . . . . . . . . AviSynth+ 0.1 (r2728, MT, x86_64) (0.1.0.0)

The new version of amDCT in amDCT_v1.1_Testing.zip is compiled as a 32 bit dll.
The next step is to get a simple 64 bit dll to build and run.

[ChaosKing]

Nice filter!

I get about 7.5fps on a 480p source i5 3570K@4.4Ghz with amDCT(adapt=28, quant=9, expand=16, sharpWAmt=8, quality=4) 3fps more with quality=3.




Update with v1.1, tested on a Ryzen 1700 @ 3.65ghz
Now I get only 5.4fps with a 480p source (maybe 10% cpu). With Prefetch(8) it goes up to 37fps @50% cpu. Prefetch(16) almost 55fps 96% cpu.
quality=3 and Prefetch=16 -> 87fps.

So realtime on a DVD is easily achievable.

[Treaties Of Warp]

Fell in love with this filter when I tested it with short clips.

Now, it is a nightmare because most of the time I can't get it to work with anything longer than a few minutes without crashing.

Nothing like having a 6 hour job crash at 90%.

The most stable performance with short clips seems to be with ncpu=1 (the default) and the version 1.1 "testing" version posted in January, with 2 x264 encode threads and 4 Avisynth MT threads. Even with this is still crashes occassionally.

This is with amDCT as the only Avisynth filter.

REALLY needs a 64 bit version for Vapoursynth! :(

LoadCPlugin("C:\PROGRA~1\Hybrid\32bit\AVISYN~1\ffms2.dll")
LoadPlugin("C:\PROGRA~1\Hybrid\32bit\AVISYN~1\amDCT.dll")
SetFilterMTMode("DEFAULT_MT_MODE", MT_MULTI_INSTANCE)
# loading source: E:\USER\Test\Test.mp4
# input color sampling YV12
# input luminance scale tv
FFVideoSource("E:\USER\Test\Test.mp4")
# current resolution: 1920x1080
# deblocking
# deblocking using amDCT
# padding to mod16 for speed
# Current resilution: 1936x1088
AddBorders(8,4,8,4)
amDCT(adapt=31,shift=5,quality=4,quant=9,matrix=8,expand=16,sharpWpos=6,sharpTpos=7,sharpWAmt=24,sharpTAmt=31,brightStart=205,brightAmt=30)
# removing mod16 padding
Crop(8,4,-8,-4)
PreFetch(4)
return last

[Treaties Of Warp]

I have so far gotten the most stable performance of amDCT by using Hybrid, with LWLibavVideoSource, aus2yuv.exe, 2 x264 threads, and 4 Avisynth threads.

Despite that, this filter sometimes produces HORRIBLE anomalies (https://i.imgur.com/HxxTs1S.mp4). I've included below a 15 second clip that produces these anomalies with the author's settings from above (https://forum.doom9.org/showpost.php?p=1804132&postcount=19):

LoadPlugin("C:\PROGRA~1\Hybrid\32bit\AVISYN~1\LSMASHSource.dll")
LoadPlugin("C:\PROGRA~1\Hybrid\32bit\AVISYN~1\amDCT.dll")
SetFilterMTMode("DEFAULT_MT_MODE", MT_MULTI_INSTANCE)
# loading source: E:\USER\temp_dvd\Video\Test\cut_orig.mp4
# input color sampling YV12
# input luminance scale tv
LWLibavVideoSource("E:\USER\temp_dvd\Video\Test\cut_orig.mp4",cache=false,stacked=true,format="YUV420P8",repeat=true)
# current resolution: 1920x1080
# deblocking
# deblocking using amDCT
# padding to mod16 for speed
# Current resilution: 1936x1088
AddBorders(8,4,8,4)
amDCT(adapt=31,shift=5,quality=4,quant=9,matrix=8,expand=16,sharpWpos=6,sharpTpos=7,sharpWAmt=24,sharpTAmt=31,brightStart=205,brightAmt=30)
# removing mod16 padding
Crop(8,4,-8,-4)
PreFetch(4)
return last

https://i.imgur.com/HxxTs1S.mp4

I tried both amDCT_v1.0 and amDCT_v1.1_Testing... same behavior in both of them.

I don't trust this filter.

Here's the original (unprocessed) clip: https://bit.ly/2ZXH4Gp

[Selur]

produces HORRIBLE anomalies.
as a side note: those artifacts disappear when removing 'brightStart=205'

[jconklin]

Treaties Of Warp. I agree the "Horrible anomalies" really are are horrible.
Selur. Thanks for the side note about 'brightStart=205' making the artifacts disappear.

The problem is that I assumed that the video to be processed would be in the range of 16-235.
Given that I have thought about using amDCT() to clean and modify masks, the 16-235 range assumption is really embarrassing.
A quick workaround until I fix the code is to do the following

temp = orig.levels(0,1,255,16,235)
outTemp = temp.amDCT(...)
out = outTemp.levels(16,1,235,0,255)

This workaround is not as "accurate" as the fix will be.


Treaties Of Warp. I will start testing long runs on a 1920x1080 source and with luck reproduce your error.
It sounds like a memory leak.

[jconklin]

Bug Fix update.

Spent 2 weeks fixing a bad case of software rot.
The build system "Visual Studio" would not build.
I proved to myself that it was Visual Studio by trying several different versions of my shipped code which contained the build system for that version. None of them still worked.
Almost certainly some update to Visual Studio was to blame.

Rebuilt the build system.

Spent 1 week to find the cause and Fix the Horrible anomalies.
It turns out that in 1 place the data returned from the DctLoop routines was stored into an 8 bit value.

If you are just doing smoothing this works fine since the brightest and darkest pixels move toward the mean.
On the other hand range expansion and sharpening can move pixel values toward the extreme.
It never showed up in my testing since I had only tested sources with a range of 16 to 235.
It turns out that provided just enough headroom so the pixel values did not overflow.
Treaties Of Warp. Thank You for reporting the HORRIBLE anomalies and for providing a clip that would produce the problem. I would not have discovered the bug just using my testing.

Selur Thanks for noting that the artifacts disappear when removing 'brightStart=205'. It helped me get to the answer quicker.

Fixed the Horrible anomalies. Download the new version amDCT_v1.2Testing.zip from http://sites.google.com/site/amdctfilter

Now I begin on the memory problems. amDCT() is a memory hog as you can see from the following numbers.
A 1080p source frame is 1920x1080 pixels. Internally the frame has an 8 byte border surrounding the source frame.
Internally a 1080p frame uses 2,121,855 bytes or around 2 megabytes.
Depending on the number of cpus, specified by the ncpu argument, there are up to 4, 16 bit per pixel frames that are used to store the intermediate results of the DCT processing done on the frame.

There are currently 14 frame buffers allocated for things like the smooth buffer, the range expand buffer, the destination buffer, buffers for finding boundaries etc.
By the time you add it all up, amDCT mallocs and frees around 30 million bytes of data for each frame processed. Some of those buffers are used to try out ideas for improving amDCT.


I will try using loop() to feed the same clip to amDCT() forever.
If I can't get loop() to work I will build a very small front end that continually calls amDCT with the same frame.
This will give me a testbed to find any memory leaks.

When version 2.0 is done any frame buffers not used will be removed from the code.

[ChaosKing]

My 16gb ram is laughing at 28mb.

And thank you for the update :-)

[Lirk]

Is there x64 version?

[jconklin]

Other, Treaties Of Warp, and anyone else having memory problems or crashes using amDCT
I have posted a new version of amDCT at https://sites.google.com/site/amdctfilter/
The new file is amDCT_v1.3_Testing.zip

The problem sounds like a memory leak. Lets stress test it.
The problem sounds like a memory leak. Lets stress test it.

I ran a 1080P .avi video in a loop where each frame was processed 700 times.
Version1.2_Testing ran for 7 days with no errors. The max memory usage was stable around 1,120 MB. Reported by Task Manager for VirtualDub.

While this was running I dug into my memory management code, in Memory.c, carefully matching every malloc with its free.
It is NOT the first time I have done this exercise and almost certainly won't be the last.

What I discovered was a buffer was being freed in one section of code and then also being freed in the calling routine.
I don't know why this did not always crash.

I fixed the code and am currently running another 7 day test.
Then a series of long tests at different resolutions.

There is a new version amDCT_v1.3_Testing.zip that fixes freeing the same memory twice.

Please try it with a script that broke the v1.2 version of amDCT with version v1.3

If the new version doesn't fix the crashes please let me know.

[Selur]

I like the filter, any plans for a Vapoursynth version?

[jlpsvk]

is there 64bit version? :)

[eluvator]

Hello.
Does anyone have this filter?
The download link is no longer working "WebspaceNotFound".
Thanks in advance.

[WaxCyl]

Try here :
https://drive.google.com/file/d/1ymDBWcxKQPqp_RYJVznSTuniVMKsEnzp/view?usp=sharing
I think you only need the DLL (from memory)

[eluvator]

WaxCyl, thank you it works.