iip: Integrated Image Processor, e.g. DVD -> 720p HDTV

[Didée]

Hello folks,

this is an early version of iip: integrated image processor. (I don't like the name too much, but didn't come up with a better one.)

Its main purpose is upconversion from DVD resolutions to (pseudo-) HDTV resolutions. It should work, however, also for just processing at native input resolution, or even for traditional downsizing. The default parameters are chosen for upconversion, and probably should be adopted for smaller resolutions.
This script aims at natural sources only! For animated/cartoon content, you are probably better of with mf's mftoon & sharpresize.

I wrote this script for the following reasons:
- after denoising a clip, much detail is attenuated also, or even gone away completely.
- DVB broadcasting in Germany is often very poor, and this script is just what I needed for these sources.
- plain fun of fiddling around

So, what does it do exactly?

- it will test your patience! ;)

1. Basic denoising of the clip
2. Sharpen the picture up, excluding already hard edges to avoid oversharpening. This step can be done in a small supersampled stage, to get (a little) finer granularity.
3. Big Supersampling, applying Xsharpen for better edge definition.
4. Now search for edges again. Areas without edges get the original, denoised, not-sharpened picture applied. This should give the best relative compressability for any level of detail enhancement.


The denoising is currently hard-coded to PixieDust. If you want to try any other denoiser, you must change the function itself for now - no hard job, its only one line in the function.
However, in my opinion the dust filters still are offering the best ratio of "compressability/preserved detail", and that's why I've build the whole function around PixieDust.

The basic usage looks like this:
LoadPlugin("path\to\undot.dll") #
LoadPlugin("path\to\MaskTools_v1.4.9.dll") # These plugins
LoadPlugin("path\to\WarpSharp.dll") #
LoadPlugin("path\to\LoadPluginEx.dll") # are required!
LoadPlugin("path\to\dustv5.dll") #

Removed. See update on this thread's page 6 (http://forum.doom9.org/showthread.php?s=&postid=517100#post517100)!

#

As you see, there are quite some parameters to play around with. And you MUST play around with them.
Everything depends in the quality of the input, the final resolution, your taste regarding image quality, and the bitrate you are willing to spend ;)


Parameter description

dest_x, dest_y ,[int]
Obviously, these are specifying the final output resolution.

duststr, dustweight, antiflicker [int],[float 0.0~1.0],
duststr is the denoising level of PixieDust. default is 2, try 1~8 depending on how noisy the source is.
dustweight: instead of using the "dusted" clip, you can mix it back with the raw input image by a percentage of [i]dustweight.
antiflicker: I had other plans for it. For now, it simply calms the picture further down by a weak temporalsoften(). See if you like it better on or off.

ss1_x, ss1_y [float],[float]
These are the supersampling factors for the following sharpening stage. Default is 1.4, try 1.0~2.0. With 1.0 the supersampling is discarded.

detailcontr, contrast_radius, PreSharp ,[int],[float]
These are the values for detail enhancement.
[i]detailcontr is the strength of unsharp masking. Default is 112, try 40~80~160~255~512, this depends on the source quality and your taste.
contrast_radius is the radius for unsharp masking. Default is 2, try 3 and 4 also. This correlated with the supersampling factors above.
PreSharp is simply the value for a sharpen()-command after unsharp masking. Default is 0.8, try 0.0~1.0.
NOTE: to completely disable detail enhancement, set detailcontr=0 and PreSharp=0.0

ss2_x, ss2_y [float],[float]
These are the supersampling factors for the following Xsharpening stage. Default is 3.5 . For downsizing, go not under 4.0. For upsizing, 3.0 might be sufficient.

Xstren, Xlimit ,[int]
These are the strength and limit factors for Xsharpen(). Default is (255,255). If you don't have the patience for big enough supersampling, e.g. for 2* supersampling only, try something like (128,23).

[b]do_subpel
This envokes a sharpen(1.0) on the flat areas, and a blur(1.0) on the edges of the supersampled image, prior to XSharpen().

[b]flatweight [int]
At the very end, the denoised-but-not-sharpened original clip is copied back into the flat areas of the full-processed clip. Through [i]flatweight you can control the strength of this copying process. "0" will return the calmest picture with best compressability. Try values 0~32~64~128.


So, I suggest you play around with all the knobs, and see if you like it.

AviSynth virtuosos might want to play with the levels() commands for the masks. In this area there is probably room for improvement - you are invited.

perhaps upcoming features:
- integrated deringing
- perhaps a better flicker reduction

Have fun


- Didée

[Didée]

The script.

Copy&paste in you editor, save & import as "iip.avs".

EDIT:
Script removed. See updated version of iiP on page 6 (http://forum.doom9.org/showthread.php?s=&postid=517100#post517100)!

[Wilbert]

btw, there is still a bug in generalconvolution. (There is no division by the total number of weights ...) I hope Richard has time one day to correct this :)

[alky]

congratulations: you win!

http://web00020.ipax.tk/video

it preserves very much detail and even adds some compared to just resizing. the sharpening is also not too strong - this should look really good viewed on a normal television.

drawback: i cannot use it as it is since i have to crop black borders from the source before doing anything (pvr250 always adds 8-10 black pixel left/right, even if it gets a normal 4:3 signal) and i almost never get a mod 16 by this. maybe you could integrate cropping in the function?

[alky]

haha... was i speaking of practical usage?!
a divx5 1st-pass would need ~34 hours for one csi episode...

[Soulhunter]

A quick test... Hmmm... Yes, a nice script !!!

But I will take a closer look next weekend !!!


Bye

[Didée]

@ all

Keep in mind this is only the proof-of-concept version of iiP. The next version is almost ready, and delivers even better results. But sorry to kill your illusions - it will NOT deliver those results faster than now ;)

The script has its name for a reason. This is not an image filter. It is an image processor (!).


Wilbert:

Yes, I'm aware of that missing feature. However, for the given purpose, it just doesn't matter - in contrary.
Of course I would *love* to use anything other than the (slow) GeneralConvolution for edge detection. But, with all the bunch of filters that are now delivering edge masks, not one of these is versatile. Either they deliver binary masks only (I need graduaded masks), or they are not pixel-centered (2*2 pseudo kernels of MaskTools), or they cannot consider all 8 surrounding pixels (vector-matrix implementation of Manao within MaskTools), or they are ... simply not implemented (Kurosu mentions the LaPlacian kernel in MaskTool's Readme, but that feature is not in the DLL!).
Perhaps it's time to post on the development forum ...


alky:

Thank you very much for testing, much appreciated. I didn't test it on analog captures at all - bought a DVB-S card some time ago ;)
For the little problems you had - see further down.


Soulhunter:

Good timing. New version will definetly be online at weekend, perhaps even earlier. Perhaps.


Some little tips:

MOD16 issue
This is not too handy, I know. Next version will remove this limit.
For now, if you need other horizontal cropping, you should crop *after* iiP.
Example:
720ish frame, crop 4 at left, 10 at right, destination=640:
iiP(dest_x=654, ...).crop(4,top,-10,-bottom)
Yes, it's cumbersome, and it's not exact, ... it's a workaround for now. There are more evil things in the world.

Speed considerations
Shortly: good processing takes good time :P
You should try&see if, after all, you need Xsharpening + big supersampling. This alone takes huge time, and is most effective only if you are upsizing or keeping DVD resolution. For downsizing, you could probably live without it.
So, if you set ss2_x=ss2_y=1.0 and Xstren=Xlimit=0, things will go quite a bit faster. Still slow, but more bearable.

In case of a 2-pass scenario, it is not a bad idea to discard the ss2 supersampling + Xsharpening in the 1st pass, and do it only in the 2nd pass.

As of now, I am converting a 40min episode with (the upcoming) version, with the abovementioned fast 1st pass.
On my (only) Athlon1800, the first pass took 9.5 hours, second will crunch for about 16-19 hours.
Encoded framesize is 704*528 (anamorphic), upconverted from a 16:9 non-anamorphic source.
Destination ratio of bits/pixel*frame is >> 0.102 << (hello RealMedia ;)), and the test encodings looked ... well, not bad :D


For the next version, deringing will be integrated (and there are some scenes that definetly need this feature!), and the sometimes-appearing flickering of detail is mostly nailed down.

Stay tuned

- Didée

[Richard Berg]

Originally posted by Wilbert
btw, there is still a bug in generalconvolution. (There is no division by the total number of weights ...) I hope Richard has time one day to correct this :)
Done in the latest CVS. (Thought about doing multiplication too, but I think Overlay would be faster.)

Here's the new internal documentation:
/**
* GeneralConvolution(PClip clip, int divisor=1, int bias=0, string matrix)
* clip = input video
* divisor = divides the output of the convolution (calculated before adding bias)
* bias = additive bias to adjust the total output intensity
* matrix = the kernel (3x3 or 5x5). any kind of whitespace is ok, see example
*
* clip.GeneralConvolution("1 2 3
* 4 5 6
* 7 8 9")
**/

[SoonUDie]

I worked on something similar for a while, but I've given up for the time being. There are some filters I want to code before I give it another serious go, but I'm not a coder and I don't have a lot of free time right now :/

One thing I'd love but could never code myself would be a superresoultion filter - they regenerate a high resolution image from temporal information. There are a million papers available on the technique, but I haven't seen any effort to make one.

[Richard Berg]

Actually, that little example at the bottom is very old, and therefore wrong :rolleyes: Now that there are more arguments you have to say 'matrix="..."' or prepend it with '1, 0, "..."'.

[mf]

Originally posted by Didée
[BAs of now, I am converting a 40min episode with (the upcoming) version, with the abovementioned fast 1st pass.
On my (only) Athlon1800, the first pass took 9.5 hours, second will crunch for about 16-19 hours.[/B]
Thought of VBLE yet? Or don't you have the HD space? :)

[Didée]

[off-topic]
Originally posted by mf
Thought of VBLE yet? Or don't you have the HD space? :)
Of course that's one of the common procedures. I am used to do things that way too, yes.

My excuses:

1. My HD's are currently cluttered with WAY too much unfinished projects and not-yet-touched DVB captures.

2. I'm short with HD space generally: it's only ~ 500 GB

3. Since this script is fairly new for me, too, I decided to do a conventional 1st-pass and analyze it thourougly afterwards, to see how the actual compressability will turn out, and to be able to perhaps adjust the parameters slightly for the 2nd pass.

I'm going to put 13 episodes of 40~42 minutes each, at 704*528, to one DVD+R. Hence I don't want to make (too) much mistakes with such a crawler script.

Rule#1: First know your tools exactly, *then* start doing the tricks.

BTW, I was almost sure I'll get a kick in the a** from you, mf, for the crap I am scripting here :) :)

[/off-topic]

- Didée

[mf]

Originally posted by Didée
BTW, I was almost sure I'll get a kick in the a** from you, mf, for the crap I am scripting here :) :)
Yeah, I didn't look too closely yet, but I was tickled for a moment to script a quick hack that *does* allow the denoiser to be chosen ;).

[Mug Funky]

hmm. i don't use generalconvolution for edge masking, but are there any advantages do just "subtract(last,last.blur(1))" and some leveling or overlaying?

i'm pretty new to edgemasking, but this method works for me in most instances (except for the odd white-dot effect when colours stray outside the CCIR range)

[mf]

Originally posted by Mug Funky
i'm pretty new to edgemasking, but this method works for me in most instances (except for the odd white-dot effect when colours stray outside the CCIR range)
Fix for that is using YV12Subtract with tol=0 twice and layering them.
Example (semi-avs cause I'm lazy):
dark = yv12subtract(tol=0)
light = invert.yv12subtract(tol=0).invert
layer/overlay/whatever(dark, light, op=lighten)

That worked for me. The idea is that when using tol=0 only the dark values come out for some wicked reason. So if you invert twice you get the light values and then you layer those over the dark ones.

[Didée]

@ mf:...I was tickled for a moment to script a quick hack that *does* allow the denoiser to be chosen. Now, this is a really BRILLIANT idea:

I'll do the technical art, and you take the punishment of making my stuff more user friendly!

Do we have a deal? :D


originally posted by Mug Funky
i don't use generalconvolution for edge masking, but are there any advantages do just "subtract(last,last.blur(1))" and some leveling or overlaying?
A good idea, Mug Funky - which I had already 2 years ago ... [edit: in November 2002]

Subtract() will leave you with an image centered around Y=127, spreading the differences towards 0 and 255 both. This is not suited for masking (most times).
Almost 2 years ago, I proposed more than once that subtract() badly needed another mode where it returns [no difference]=[0] & [full difference]=[255]. But I was talking to the wind.
Of course it is possible to adjust the 127-centered result of subtract() to what is needed; but these calculations (you need both overlaying and levelling) also need some CPU time, so that doing it from start by GeneralConvolution turns out to be about just the same effort.
Another possibility is to use "adjust.dll" for making subtract()'s output suitable ... works pretty good also, but requires YV12<->YUY2 conversion.
After all, with Convolution I have better control over what's exactly happening. (Keep dreaming of "UltraFastLaPlacianKernelConvolution.DLL" ...)


Regarding iip itself:

New version is ready. Deringing is in. MOD16 is no more required. Sharpening works a little more tasty now (for me). Speed has not improved. Custom denoising is waiting for mf ;)

When I'll return home, I'll have to do a little more fine-tuning on the deringing part - deringing of natural sources with neither destroying nor "forgetting" something is probably a little more complicated than on animee ...

Should go online this night.

One more question:

Much overlaying is going on in this script, and ...

1. Overlay() is (yet) less optimized than MaskTool's functions, speak: it's noticeably slower.

2. MaskedMerge() of MaskTools seems to inheritely have small rounding errors: it may introduce slight color shifts.

What do you prefer ... ?

- Didée

[Didée]

Oops, didn't see that mf was replying while I was still typing ...

mf:

So one can get it to work that way, okay.

But then, how much faster is [2*subtract]+[invert]+[overlaying]+[SomeThingsMore] compared to GeneralConvolution()??

Plus, the latter still is more exact than this UnsharpMasking-like approach -- but: it is well suited to e.g. create simple deringing masks, agreed.

Should we create a new thread, about edge detection ... :)

- Didée

[mf]

Actually, I hate putting in the end-user stuff. I just like to release a filter and see the reactions, so I try to put in as much user-configurable stuff as possible. In that sense you could say I'm a fame junkie ;).

And you've given me an idea for an edge mask. It has to do with an old filter in MaskTools that's been a friend of mine for some time... :D Can you guess it?

[Didée]

I put (little) money on "MaskOverlay()" ? - I didn't try that one myself, not one single time :o

But I do remember the good old times, say a year ago, when we simply posted pure linear scripts, and simply stated "try this!".
Today, we shall produce full-featured functions, that catch every possible user error, are configurable from head to toe, do sing and dance, glow in the dark and defragment your HD in the background.
Oh my Lord. ( partly :) , partly :rolleyes: )

- Didée

[mf]

It has to do with Inflate :).

[Didée]

Oh c'mon, mf ... there are a myriad of things you can do by means of edgemasks, and you ask such a question. I'm not THAT good at reading minds (yet).

After having a big struggle in the logo war I'm fighting with "SAT1", and sleeping much longer than I expected, the new version of iiP is ready now. Phew!
I just have to prepare the instructions, so that you all know what do do with it ;)

- Didée

[scharfis_brain]

After having a big struggle in the logo war I'm fighting with "SAT1"

Please tell me more....

[mf]

Originally posted by Didée
Oh c'mon, mf ... there are a myriad of things you can do by means of edgemasks, and you ask such a question. I'm not THAT good at reading minds (yet).
Okay, it has to do with Subtract and Dilate! :)

[Didée]

Okay, here is the new version of iiP.

The changes are:

- input clip is no more required to be MOD16.
- sharpening stage now uses up to three iterations
- masking for detail processing internally slightly adjusted
- subpel operation simplified
- reduced the possible amplifying of pre-existing detail flickering
- added adjustable deringing option
- added debug/visualization option for setting up deringing
- all plane copying operations are now done by "Overlay()", "MaskedMerge()" was giving slight color shifts
- reduced overhead: processing steps that are set to "no-operation" through parameters 0|1|0.0|1.0 respectively, are effectively discarded


iiP uses a good amount of RAM. If you get unexpected AviSynth/Vdub crashes, try setting "SetMemorMax(64)" (at least, better (96) or even (128) ) at the start of your script.


The usage & default parameters now look like this:
(output resolution defaults to input resolution, of course)

iip( dest_x= 704, dest_y= 416,
\ ss1_x = 1.414, ss1_y = 1.414,
\ duststr = 2, dustweight = 1.00, antiflicker= true,
\ detailcontr1=104, detailcontr2 = 192, contr_radius = 2, PixSharp=0.4,
\ ss2_x = 3.5, ss2_y = 3.5,
\ Xstren = 255, Xlimit = 255,
\ subpelstren=1.58, flatweight = 0,
\ dering = 80, dering_weight= 1.0, dering_floor = 80, dering_bias=0.4,
\ debug= " "
\ )


Updated parameter description:

dest_x, dest_y ,[int]
Obviously, these are specifying the final output resolution.

duststr, dustweight, antiflicker [int],[float],
[i]duststr is the denoising level of PixieDust. Default is 2, try 1~8 depending on how noisy the source is.
dustweight: instead of using the pure "dusted" clip, you can mix it back with the raw input image by a percentage of dustweight.
antiflicker: I had other plans for it. For now, it simply calms the picture further down by a weak temporalsoften(). See if you like it better on or off.

ss1_x, ss1_y [float],[float]
These are the supersampling factors for the following sharpening stage. Default is 1.4, try 1.0~2.0. With 1.0 the supersampling is discarded.

detailcontr1, detailcontr1, contr_radius, PixSharp ,[int],[int],[float]
These are the values for detail enhancement.
[i]detailcontr1 & detailcontr2 are the strengths of two chained unsharp maskings. Default is 108/192, try 40~80~160~255~512. First value should be smaller than the second. (Unless it should be greater.)
contr_radius is the radius for the first unsharp masking. Default is 2, try 3 and 4 also. The second unsharp masking is hardcoded to a radius of 1.
PixSharp is simply the value for a sharpen()-command after unsharp masking. Default is 0.5, try 0.0~1.0.
Setting any of the strength values will discard the respective sharpening step.

ss2_x, ss2_y [float],[float]
These are the supersampling factors for the following Xsharpening stage. Default is 3.5 . For downsizing, it is NOT recommended to drop below 4.0. For upsizing, 3.0 might be sufficient.
Setting both of these to 1.0 will discard the second supersampling step. Remember to also disable subpel, then!!

Xstren, Xlimit ,[int]
These are the strength and limit factors for Xsharpen(). Default is (255,255). If you don't have the patience for big enough supersampling, e.g. for 2* supersampling only, try something like (128,23).
Setting both of these to 0 will discard this step.

[b]subpelstren [float]
This now envokes a "blur(subpelstren)" on the supersampled image, prior to XSharpen(). Default is 1.58, range is from -1.0 (full subpel sharpening) to 1.58 (full subpel smoothing).
The effect is minor, look very close and see for yourself if you want to use it or not. I mostly use it to avoid 'nervous edges' through XSharpen() - it helps stabilizing the picture, especially when the 2nd supersampling factor is not set to the preferred value ;-).
Setting subpelstren=0.0 will discard this step.

flatweight [int]
At the very end, the denoised-but-not-sharpened original clip is copied back into the flat areas of the full-processed clip. Through flatweight you can control the strength of this copying process. "0" will return the calmest picture with best compressability. Try values 0~32~64~128.

dering [int]
The basic strength to use for deringing. This is effectively the parameter for an "unfilter()" command, so the useful range is 5 to 100. Negative values are possible, but not useful at all ;-). Default is 80.
(Clarification: compared to unfilter(), the parameter's sign is reversed.)
Setting this to 0 will discard the deringing step.

dering_bias [float]
This value adjusts how strong the edges in a frame must be to get deringing applied to. Use with [i]debug="dering" to see how affected areas will change. Default is 0.4, reasonable range is 0.2~2.0 (or even more?).

dering_floor
This is a cutoff value, up from here deringing will start to kick in. Set this up also with [i]debug="dering". Default is 80.

dering_weight [float]
Similar to flatweight, this effectively is a percentage of how much of the calculated deringed will be applied to the frame. Default is 1.0.

debug [string]
This has (for now) only one useful value: "dering", which will overlay the deringing mask onto the clip in fine green.
Use this to find appropriate values for dering_bias and dering_floor for your clip.


Notes on usage:

- You should not rely on the default values, since I don't know what material you are going to process.

- all sharpening values must be seen in correlation to the first supersampling stage. The bigger the supersampling, the finer but weaker the sharpening will come out. Example: imagine you were using a supersampling(1) factor of 1.5 and found your favorite sharpening values. Later, you decide to reduce or disable the first supersampling, then you will definetly end up with strong oversharpening if you dont reduce the formerly found values!

- to completely disable detail enhancement, set detailcontr1=0, detailcontr2=0 and PreSharp=0.0. This will make iiP work more similar to the "SharpResize" function by mf, however without seperated color treatment.

- Attention: if you don't use the 2nd supersampling + Xsharpening, remember to set subpelstren=0.0 !!! Else, you will either destroy or over-smooth the clip!


Some more Tips:

- You want to use another base denoiser than PixieDust? Okay, set dustweight=0.0, and use your own denoiser before calling iiP. This is much simplier than bloating the script, still serves the very same effect ;).

- for clean sources (DVD mostly, DVB seldom, analog captures never), you might want to reduce dustweight to 0.75 or 0.5 to retain maximum detail.

- If the source has sufficient "depth sharpness" (can I say this for "Tiefenschärfe" ?), you might set detailcontr1=0, and possibly reduce detailcontr2 in favour to raise PixSharp.
In the opposite case, on muddy sources, you might want to rely even more on detailcontr1, and possibly raise contr_radiusfrom 2 to 3.
BTW, if you want to easily see where sharpening is kicking in, you can set one or both of the detailcontrX-values to something like 500, 1000 or even 2000. Not really useful for encoding, but at least you'll see something ...

- regarding flatweight: I would keep this at 0 as long as PixieDust leaves you with enough recoverable detail - what it will mostly do.
However, if you are using a video compressor that tends to throw blocks on very flat image areas, you might want to raise flatweight to something like 16 through 40. What do I know, see for yourself, please.

- to get the best relative compressability, set dustweight=1.0 and flatweight=0.

- Although it is generally not recommended to use subpel operation without the 2nd supersampling (because then, it is no more 'subpel'), exactly that was giving me surprisingly good results when I tried it on a certain, rather poor source. The result looked smooth, but not blurry. Experimenting is always a good idea.


Hope everything is clear as mud now. :)

- Didée

[edits]
#1 - Only one missing "/" in a tag can make things look pretty strange ...
#2 - typos, some small additions, usage example wasn't really at default values
#3 - usage example still wasn't at defaults. Now it is.

[Didée]

The new script:

edit 26.02., 0h20m: v0.3b: added workaround for Overlay()'s YV12 UV-bug (color shifting)

edit 28.02., 17h30m: v0.3c: More Speed! (less slow ;) ) Removed critical Overlay()'s, replaced with MaskTools functions
# iip : Integrated Image Processor, v0.3c
#
# (0.3c performs most plane operations (again) through MaskTools (v1.4.9) functions for more speed.
# Some are still done by Overlay(), since modes "lighten" and "exclusion" are only available there.)
#
# Currently includes:
# - Denoising
# - detail enhancement |< while avoiding edge oversharpening, ringing,
# - improved edge definition |< halos, and detail flickering
# - deringing: removes noise around edges,
# moderates pre-existing Edge-Enhancement (dark & bright halos around edges)
#
# It's often able to make the clip, at the same time, sharper & more compressable than the source itself.
# By tweaking the values, you can give more priority to either compressability or sharpness.
#
# ---------------
# | It is slow. |
# ---------------

function iip( clip clp,
\ int "dest_x", int "dest_y",
\ int "duststr", float "dustweight",
\ float "ss1_x", float "ss1_y",
\ int "detailcontr1", int "detailcontr2",
\ int "contr_radius", float "PixSharp",
\ float "ss2_x", float "ss2_y",
\ int "Xstren", int "Xlimit",
\ float "subpelstren", bool "antiflicker",
\ int "flatweight",
\ int "dering", float "dering_weight",
\ int "dering_floor", float "dering_bias",
\ string "debug"
\ )
{
ox = clp.width
oy = clp.height
xcorr=(int(ox/16.0+.9999)*16)-ox
dest_x = default(dest_x, clp.width)
dest_y = default(dest_y, clp.height)
ss1_x = default(ss1_x, 1.414)
ss1_y = default(ss1_x, 1.414)
ss2_x = default(ss2_x, 3.5)
ss2_y = default(ss2_x, 3.5)
duststr = default(duststr, 2)
dustweight = default(dustweight, 1.0)
antiflicker= default(antiflicker, true)
Xlimit = default(Xlimit, 255) # SS<3 ? 23 : 255
Xstren = default(Xstren, 255) # SS<3 ? 216 : 255
detailcontr1= default(detailcontr1, 108)
detailcontr2= default(detailcontr2, 192)
contr_radius= default(contr_radius, 2)
PixSharp = default(PixSharp, 0.5)
subpelstren = default(subpelstren, 1.58)
flatweight = default(flatweight, 0)
dering = default(dering, 80)
dering_bias = default(dering_bias, 0.4)
dering_floor= default(dering_floor, 80)
dering_weight=default(dering_weight, 1.0)
debug = default(debug, "false")

xx_ss1 = int((ox*ss1_x/8)+.5)*8
yy_ss1 = int((oy*ss1_x/8)+.5)*8
xx_ss2 = int(ox*ss2_x/16+.5)*16
yy_ss2 = int(oy*ss2_x/16+.5)*16

antiflicker == true ? clp.temporalsoften(3,2,3,23,2) : clp

dusted=last.addborders(0,0,xcorr,0).converttoyuy2().PixieDust(limit=duststr).converttoyv12().crop(0,0,-xcorr,-0)#.BlindPP(quant=8,cpu=4,moderate_h=45,moderate_v=55)
dusted = (dustweight<1.00) ? Overlay(last, dusted, opacity=dustweight) : dusted

edgebase=dusted.greyscale().bicubicresize(int(ox/1.33/4)*4,int(oy/1.33/4)*4,1.0,0.0).converttorgb()
edgeA = edgebase.\
GeneralConvolution(0,"
-3 -4 -3
-4 28 -4
-3 -4 -3 ").ConvertToYV12()
edgeB = edgebase.levels(0,1.0,255,255,0,false).\
GeneralConvolution(0,"
-3 -4 -3
-4 28 -4
-3 -4 -3 ").ConvertToYV12()

edge1 = Overlay(edgeA,edgeB,mode="lighten").inflate().inflate().inflate().blur(1.58).levels(24,1.0,208,0,255,false)
\ .inflate().inflate().inflate().inflate().greyscale().bicubicResize(xx_ss1,yy_ss1,1.0,0)

edge1b=Overlay(edgeA.levels(dering_floor,1.0,255,0,255,false).expand().inflate(),
\ edgeB.levels(dering_floor,1.0,255,0,255,false).expand().inflate(),
\ mode="Exclusion").levels(31,dering_bias,192,0,int(255*dering_weight),false).deflate().deflate().inflate().inflate().greyscale()

(ss1_x > 1.0 || ss1_y > 1.0) ? dusted.Lanczosresize(xx_ss1,yy_ss1)
\ : dusted

temp = last

detailcontr1 != 0 ? unsharpmask(detailcontr1,contr_radius,0) : NOP
detailcontr2 != 0 ? unsharpmask(detailcontr2, 1,0) : NOP
PixSharp != 0.0 ? sharpen( PixSharp ) : NOP

MaskedMerge(last,temp,edge1,Y=3,U=1,V=1).undot()

dering != 0 ? MaskedMerge(last,last.unfilter(-dering,-dering).lumafilter(1),edge1b.bicubicresize(xx_ss1,yy_ss1),Y=3,U=1,V=1)
\ : NOP

antiflicker == true ? temporalsoften(1,2,3,23,2) : NOP

xx_ss2 != ox && yy_ss2 != oy ? LanczosResize(xx_ss2,yy_ss2) : NOP

subpelstren != 0.0 ? blur(subpelstren) : NOP

Xstren != 0 && Xlimit !=0 ? xsharpen(Xstren,Xlimit) : NOP

LanczosResize(dest_x,dest_y).undot()

edgebase2=last.greyscale().bicubicresize(int(dest_x/1.2/8)*8,int(dest_y/1.2/8)*8,1.0,0.0).converttorgb()
edgeA2 = edgebase.\
GeneralConvolution(0,"
-3 -4 -3
-4 28 -4
-3 -4 -3 ").ConvertToYV12()
edgeB2 = edgebase.levels(0,1.0,255,255,0,false).\
GeneralConvolution(0,"
-3 -4 -3
-4 28 -4
-3 -4 -3 ").ConvertToYV12()
edge2 = overlay(edgeA2,edgeB2,mode="lighten")
\ .inflate().inflate()
\ .levels(24,3.0,192,flatweight,255,false).inflate()
\ .bicubicResize(dest_x,dest_y,1.0,0)
\ .inflate().inflate().blur(1.58).inflate().greyscale()
\ .temporalsoften(1,255,255,23,2)

MaskedMerge(dusted.lanczosresize(dest_x,dest_y),last,edge2,Y=3,U=1,V=1)

debug == "dering" ? overlay(last,blankclip(last,color=$008000),mask=edge1b.bicubicresize(dest_x,dest_y),mode="blend",opacity=dering_weight)
\ : NOP

return(last)
}

#

[Soulhunter]

Thanks for this toy, now I will play with it... ;)

Bye

[Soulhunter]

I get an error with this script... :(

mpeg2source("F:\DVD Backups\Hulk\Hulk.d2v")

LoadPlugin("C:\Programme\AviSynth 2.5\plugins\undot.dll")
LoadPlugin("C:\Programme\AviSynth 2.5\plugins\MaskTools_v1.4.9.dll")
LoadPlugin("C:\Programme\AviSynth 2.5\plugins\WarpSharp.dll")
LoadPlugin("C:\Programme\AviSynth 2.5\plugins\LoadPluginEx.dll")
LoadPlugin("C:\Programme\AviSynth 2.5\plugins\dustv5.dll")

Import("C:\Programme\AviSynth 2.5\plugins\iip.avs")

iip( dest_x= 1024, dest_y= 576)
Bye

[Didée]

Doctor, I don't know what I'm suffering from, but please give me a curing medicine."

What error do you get, soulhunter? Your report is a little thin ...
Your very script (but different source, of course) works fine here.

If it's some kind of AviSynth read error / access violation, then try
SetMemoryMax( X ), with X > 128 or 160 for testing.

I'm doing much chained steps in there, including mask/overlay operations on big framesizes, holding several of those big frames parallel in memory and mix them together later, - - that kind of stuff. May eat quite some memory ...

[... testing ...]

Yup, that's it:
SetMemoryMax(32): destination 1024*576 -- crashes

SetMemoryMax(64): destination 1024*576 -- OK
destination 1280*720 -- OK
destination 1920*1088 - crashes

SetMemoryMax(96): destination 1920*1088 - OK

Seems we have the culprit. I'll edit the usage description.
(My own script templates all start with SetMemoryMax(192), so I didn't notic that. Sorry.)

- Didée

[Soulhunter]

THX, I will see if this medicine helps me... :D

[EDIT1: Ok, now it work's !!!]

[EDIT2: 2h for 4500 frames...]

Bye

[Soulhunter]

So, its done...

Sample encode with iip's default settings @ 1024x576 !!!


The Matrix (lobby shootout)

More than 2h encoding time on my 2600XP... :eek:


Result:

- Razor-sharp picture... Nice !!!

- "Near" like no floating wall effect... :)

- Problems (flickering) with sharp edges near detailed arears... :confused:


Overall nice script !!!

But considering the speed factor... :rolleyes:


Bye

[Didée]

Soulhunter:

Fine that you liked at least the result.
Could you perhaps quickly compare it also in regards to compressability, or filesize @ fixed quant, against the unfiltered source, and perhaps one or two common denoisers? Just to get a *rough* idea about that aspect.

Considering the speed: well, heheh ...

Sophisticated evaluation and treatment of images-in-motion takes much time, simple as that. Look at how much slower XviD is with qpel+vhq4, compared to no-qpel + vhq0! There are worlds between.

Moreover, one of the basic ideas behind this, when starting out, was the supersampled Xsharpen (which I then tried to supply with more details to be sharpened ... iip was born). Now, would you please try out the latest version of mf's SharpResize of the SharpTools? This one does no denoising, nor does it any explicit detail enhancement. But you will find that, regarding the speed, it plays in about the same league ...

Now:

>> Razor-sharp picture... Nice !!!

Answer: :)

>> "Near" like no floating wall effect...

Answer: :D

>> Problems (flickering) with sharp edges near detailed arears...

Answer: Yeah, alas, still.
I have a couple of ideas how to make those problematic areas more steady. (But notice: mostly, this flickering is not really introduced by iip - it's very likely already there in the source, but gets amplified by the script).

I am working on that problem. The question is, which method will work out the best, and which will produce the least amount of additional CPU cycles ... lets see.

BTW, if anyone feels like hosting some short samples of a 704*528 encoded clip @ ~1000 kbps (0.105 bits/pix*frm !) - I'd have some available ;) (but no webspace).

- Didée

[mf]

Originally posted by Didée
Oh c'mon, mf ... there are a myriad of things you can do by means of edgemasks, and you ask such a question. I'm not THAT good at reading minds (yet).
- Didée
The correct answer was:
Corona Edge Mask (http://forum.doom9.org/showthread.php?s=&threadid=71540)
:D You and Mug Funky gave me the idea of using Inflate and Deflate with Subtract to create an edge mask.

[Soulhunter]

Originally posted by Didée
Fine that you liked at least the result.
Could you perhaps quickly compare it also in regards to compressability, or filesize @ fixed quant, against the unfiltered source, and perhaps one or two common denoisers? Just to get a *rough* idea about that aspect.
Yeah, its a bit vague... :rolleyes:

Just wait a bit !!!


Bye

[Soulhunter]

Ok, here is something less vague... ;)


Source: Matrix (PAL) / Lobby shootout chapter / First 1000 frames

Settings: XviD RC2 / Fixed quantizer 2 / MPEG matrix / VHQ1 / 1024x576


Only resized
Size = 30,7 MB / Encoding time was about 01:45 min. (2600XP)

Fluxsmooth(3,5)
Size = 24,7 MB / Encoding time was about 02:05 min. (2600XP)

Peachsmoother & Fluxsmooth(3,5)
Size = 19,3 MB / Encoding time was about 02:25 min. (2600XP)

Convolution3d("moviehq") & Fluxsmooth(3,5)
Size = 23,4 MB / Encoding time was about 02:35 min. (2600XP)

SharpResize & Fluxsmooth(3,5)
Size = 26,4 MB / Encoding time was about 15:20 min. (2600XP)

Peachsmoother
Size = 21,4 MB / Encoding time was about 01:55 min. (2600XP)

Convolution3d("moviehq")
Size = 28,1 MB / Encoding time was about 02:10 min. (2600XP)

SharpResize
Size = 32,2 MB / Encoding time was about 15:05 min. (2600XP)

IIP processing
Size = 32,7 MB / Encoding time was about 25:40 min. (2600XP)


Note1: For all encodes except the SharpResize & IIP one, Lanczos resize was used !!!

Note2: All smoothers where used after the resizing !!!

Note3: I dont take any responsibility for the results !!!

Note4: Want screenshots ??? Then tell me till next weekend !!!


Bye

[koszopal]

hi
ipp is very good but ive got problem with "red ghost" (see 3 rows and ipp column frames )here (http://koszopal.fm.interia.pl/movie/comp.htm)
koszopal

[Didée]

Soulhunter:

What? iip at defaults was bigger than the plain lanczos'ed source?

Wait a minute ...

... oh my! Did you simply call "iip(xxx,yyy)" with empty parameters, or did you use the complete "usage script" above posted ?

I just noticed a inconvience: default for dustweight should be 1.0, but in the usage script it is 0.75 ... doesn't do any harm, but won't give the best compressability. At defaults, in my tests so far iip was always smaller, or at least equal in size compared to not-filtering.

Usage script updated, sorry :o


koszopal:

Please, could you describe your "red ghost" problem a little better:
What I see on your web page is:
- source pictures that already look a little red-ish beforehand
- SharpResized pictures that look more greenish than the source
- iip'ed pictures that look similar to the source, color-wise

BTW, was that the new script version, or the older one? The old was giving red-ish pictures caused by MaskedMerge(), which is no more used now.

- Didée

[koszopal]

ive tryd both i mean
1st) iip( dest_x= 640, dest_y= 352)
and
2nd) iip( dest_x= 640, dest_y= 352,
\ ss1_x = 1.414, ss1_y = 1.414,
\ duststr = 2, dustweight = 1.00, antiflicker= true,
\ detailcontr1=104, detailcontr2 = 192, contr_radius = 2, PixSharp=0.4,
\ ss2_x = 3.5, ss2_y = 3.5,
\ Xstren = 255, Xlimit = 255,
\ subpelstren=1.58, flatweight = 0,
\ dering = 80, dering_weight= 1.0, dering_floor = 80, dering_bias=0.4,
\ debug= " "
\ )
in both case in frame 32525 ive got "red ghost" :(
ab version
im using new script (version 0.3) just few posts up
and im agree with you that iip is color-wise :)
koszopal
PS hehe and ab speed yes mostly 1fps or below (2200xp) :) (3h for const quant2 xvid rc2 7914 frames sample) :D

[koszopal]

ive just uploaded a part vob containg this frame (rename .gif to .vob)
get this (http://fire.prohosting.com/koszopal/emma/emmapart.gif)
koszopal

[Didée]

Will investigate, thanks.

- Didée

[Didée]

ALERT!

The script indeed produces output with screwed color alignment!.

The bug is confirmed and tracked down. A solution will be posted shortly.

Stay tuned - the bug lies elsewhere, and might affect many other scripts, too!

If anyone has currently any serious work running through iiP, (s)he might abort the job right NOW.

(Sorry for the flood of bolds)

- Didée

[Didée]

Okay, the miracle's solution.

After inspecting each and every single step in the script closely, I started to tear out my hair because I couldn't nail down anything.

Finally, I found it:

Overlay() shifts the color planes if you process YV12 clips with it!

Now, this is sort of a problem for iip ... As a temporal workaround, I changed the script to do all related internal processing in YUY2 colorspace. This doesn't exacly help the speed, but it's the only solution I can offer quickly.

My initial usage of MaskedMerge() from MaskTools is no solution, since its little inaccurate averaging will produce color changes as well: mostly reddish, sometimes greenish ...

I'll see how to isolate the color planes from the process in a satisfying way, but I don't want to hack that in head-over-heels. So you have to live with the intermediate YUY2 version, for the time being.

As a bonus, I changed the deringing part to work even a little finer than before :)

I'll edit the above posted script in a minute.
Please Copy & Paste the script again.


- Didée,

dropping a tear,

while watching about 120 hours worth of encoding going into the trashbin.

[koszopal]

@didee
i can confirm with 03b isnt any problems with "red ghosts" , great work :D,also i've updated my SharpResize/IIP/Hypp comp page (http://koszopal.fm.interia.pl/movie/comp.htm)
(added comp version 0.3 and 0.3b and hypp (0.6)
koszopal

[Ark]

Damn, i've finally solved the LoadPluginEx/Dustv5 loading problem, and suddently a new problem comes up:

When i use iip, Avisynth throws in this error:

Avisynth open failure
Script error: there is no function named "unsharpmask"

I've all the plugins needed and there's no problem in loading them, but this seems to be a plugin missed or something else to my eyes...

[Didée]

You probably don't have the "right" WarpSharp package, Ark. There is more than one package floating around with then name WarpSharp in it.

Grab the one from this site (http://www.geocities.co.jp/SiliconValley-PaloAlto/2382/), and ....

Bah, I'm talking nonsens! If you are sucessfully using LoadPluginEx, then you have the right package.
Are you really loading the "WarpSharp.dll" plugin at your script's start? You need "WarpSharp" to get "UnsharpMask" ...

- Didée

[Ark]

@Didèe

Ok, now works fine. It was broken because i wanted to keep the other warpsharp (don't ask me why...) I had, so I've done a subdirectory with the new package, but I forgot to add to the loading path in the .avs this subdir...

Now I want to thank you for this awesome script, I'm testing it with Matrix and if i could use it at more than 1 fps...

I've also a little question: is this script called an "image processor" because it do the same things as a dedicated processor, but in software?

Thanks and keep up improving!

[Didée]

Well, I have plans in store to speed the script up by a big amount: 1.5 fps should be in reach, perhaps even 2.0 fps on fast machines ...

Seriously, the script will be about 20% faster once it will be (again) using MaskTool's functions, as far as appliable.
Moreover, I have another idea that *could* accelerate the whole thing quite a bit, but I cannot say much about that yet. Have to try the idea first, and it's not that simple to implement. Could perhaps work, perhaps it will not.

See this also in conjunction with you question about the "image processor" terminology:

I have, honestly, no idea at all what dedicated hardware processors are actually doing. :)
It is just like this: the script does (sort of) an analysis where the interesting picture areas are, and performs suited operations to the different kinds of image areas - with some little twists, somewhere inbetween.

Hence I thought "image processor" wouldn't be such a bad name - and, most of all, this name is also a good excuse for the nerve-breaking speed, or better the lack thereof.
Moreover, the script uses *several* techniques to process the source, and nobody forces you to use *all* of them. As described above, you can disable almost every single procedure by setting the approbriate parameters to either 0(.0) resp. 1(.0).
In this respect, iiP could perhaps be seen as a "Mini Processing Suite" - although a little tricky to set up, agreed.
If time allows, there will be implemented even more possibilities to process the picture, like e.g. ***censored***.

All together, it is simply so, that a lot of "expensive" operations have to be done, in order to get a result like that. You simply cannot put your car in your trouser's pocket ...

- Didée

[Ark]

Thanks for your reply, and i asked that because this script seemed to me to pratically do the same job that do a processor in a dvd-player.

In many high-level dvd-players you can adjust many parameters regarding image quality, like sharpness, or noise reduction, and even in some upscale the resolution to HDTV standard and others (such as 576p, 720p, 1080i/p). There're even dedicated machines that do only this type of operations, taking any video input and pocessing it to a different output (composite<->s-video<->component) while processing it.

These pieces of hardware are usually very expensive, so it's incredible to me that i can have the same function for no cost at all!

Yes, I've to wait a month (for now) to do a backup, but who cares?? The quality is astonishing! :)

Now I think that only time and Didée's work will raise the usability of this script (maybe with a P7 9000mhz..:D ), so I've just to wait quietly.

[Didée]

Or, better said, less slow ;)

Guys, the workaround version was really slow!

Most of the plane copying functions now are done again by MaskTools, which speeds the whole thing up by quite a little bit, especially when compared to the YUY2/Overlay() version 0.3b ...
The colors are safe.

Functionality and usage stays the same, so the script is still here (http://forum.doom9.org/showthread.php?s=&postid=447991#post447991) on page 2.

Some rough numbers:

( CPU 1533Mhz / clip ~40 min, 704*432 -> 704*528 anamorph )

Overlay()/YV12: ~19:00 hours (buggy)

Overlay()/YUY2: ~25:20 hours

MaskedMerge/YV12: ~14:30 hours


Still slow, of course. But nonetheless a noticeable improvement.

- Didée

[Didée]

Bug report:

iiP is not yet compatible with AviSynth/03032004!

For the moment, stay with AviSynth/30012004, if you're going to use iiP. I'm adapting the script - but it's a littly tricky...

The good news:
After finishing the changes, iip will gain another 10% of "speed" ;)


Moreover, a friend was so nice to make some webspace available. So, here we have some

Samples of iiP (http://kotisivu.mtv3.fi/ii00i/index.htm)

for those who might be interested.

- Didée

[Manao]

BTW, considering speed issue, if you're working only in YV12, and if only the luma plane is interesting, the following will be faster than a levels(a,gamma,b,c,d) :YV12LUT(Yexpr = "x a - b a - / 1 gamma / ^ d c - * c +")simply because it doesn't process the chroma planes. It is cheap optimizations, yet, it is 3/2 times faster.