iip: Integrated Image Processor, e.g. DVD -> 720p HDTV

[Didée]

One correction of my own false comment, before you start out:

To disable Pixie, use dustweight = 0.0 ! When setting "duststr=0.0", it is also (effectively) disabled ... but PixieDust would first do its full work, changing nothing, and then get copied over ... a pretty slow No-Op. By zeroing dustweight, Pixie will actually be bypassed - that's what you want to do.

But remember:

The more *you* like the preview you're getting -

- the more your *encoder* will HATE it


Me has taken some screenshots, wait a little.


- Didée

[Didée]

A case example.

Yesterday I did my first full encoding with iiP v.0.5a. You see, I myself am still learning to use it

Target was Enterprise episode 1x09 - "Die Saat". (am too lazy now to browse for the original title. perhaps "The Seed" - but probably not)

Here are some screenshots comparing the naked RAW DVB stream to iiP's output. Note that this is an upscale from 4:3 letterboxed to 16:9 anamorphic:


source-1:


iiP-1:



source-2:


iiP-2:



source-3:


iiP-3:



source-4:


iiP-4:



source-8:


iiP-8:


source-9:


iiP-9:



The used script looked like this:

v1=mpeg2source("ENTERPRISE_0327_1.d2v", idct=7, cpu=4,moderate_h=45,moderate_v=60,showQ=false)

v1.trim(12,60029) 
crop(8,72,-8,-72)

MyDelogo()

## version05a
iip( dest_x = 708,      dest_y = 532, 
 \   duststr = 2,       dustweight = 1.0, 
 \   ss1_x = 1.414,     ss1_y = 1.414, 
 \   detailcontr1= 127, detailcontr2=255+40,  contr_radius=2, PixSharp=0.38, 
 \   ss2_x=3.25,        ss2_y=3.25, 
 \   Xstren=255,        Xlimit=255,  
 \   subpelstren = 1.0, antiflicker1=false, antiflicker2=true,
 \   flatweight = 0,
 \   protect_floor = 0, protect_bias = 12,
 \   dering = -80,      dering_weight = 1.0,
 \   dering_floor = 10, dering_bias = 12,
 \   EQ = 2,            detail_floor = 15,
 \   warp_Y = true,     warp_UV=true, exborder = true,
 \   debug="-showall", cropx=36, cropy=56
 \   )

crop(2,2,-2,-2)

colorYUV(cont_u=18,cont_v=12)

limiter()

return last

Notes:

Be sure to view those screenshots at an sufficient brightness/gamma level, in particular the dark ones. You know, all those parts that are invisible, "lost in the dark", in normal desktop viewing, spring to the eye very obviously when played through overlay in a dark room ...

As you can see from the output, I was careful with the sharpening & enhancing. One can very easily overdo things with iiP - but of course I don't want to do over-processing. My goal with this source is to bring it back to what it should have been in first place (and what it *was*, originally. But we dumb TV consumers are getting fed with such big crap sometimes, it's just ridiculous).
Moreover, my aim is to put each season onto 2 DVDs - that makes 13 episodes, each about ~42 min, per DVD. Given the resolution, you can do the math yourself -- Below the summing line, I end up with an average bitrate of 1050 kbps, taken the overall for all episodes. This really isn't that impressing high for a framesize of 704*528
I'm ending up with an average P-quantizer of 4.0~4.2 using SixOfNine-HVS in XviD. "Translated" to the standard matrix, that would be a quant of ~2.6.

The source is somewhat noisy - not too bad, but definetly too much to simply pipe the source unfiltered into XviD. This results in annoying "floating wall" artefacts.

For my taste, the source is also too blurry. You see I've used relatively high sharpening values - "detailcontr2" is already in overdrive range. For cleaner sources, one could simply go higher with "detailcontr1", but on these particular sources, bitrate climbs way too much. 127 is my limit in this specific case.

protect_floor is down to zero - always a good choice to not enhance any remaining noise more than necessary.
protect_bias at 12 was figured by try'n'error. 16 gave not enough sharpening, at 10 some edge artefacting started to appear.

DeRinging was almost standard:floor-bias at 10-12 gave a good compromise at catching source artefacts even at weaker levels, while still masking everything strong enough.
I was teared between 8-10 and 12-12 for different scenes, so I just took the golden middle
Now that I review it, I could have put it a little higher values for dering, perhaps -100 would have been better.
Moreover, now I have the impression I should feather-out the whole deringing mask a little bit more.

The ss2_x|y values were set a little lower. The source's details are not as dense as in a good DVD movie, so I could afford this.

detail_floor was lowered to 15 after inspection of several scenes with weak detail. However, this is more important for compression than for the visuals.

All warp options were used, for the formerly mentioned reasons. Same goes for border exclusion.

EQ mode was "2". The source comes along with an (un)fair amount of small, but quite annoying artefacts (best seen in motion, there's no so much to tell from the screenshots), which here makes EQ=1 to look not very nice. And EQ=3 would be far too strong for a source with such insufficient sharpness.

Lastly, I applied a weak color correction. Somehow the series seems to suffer from NTSC-television-color-flatness, or whatever damage they might have done. It just looks too "empty", occasionally.

Very lastly: No, the the removal of that wonderful logo was NOT done by iiP


Okay, that was the little walk-through, or better said, run-through.

Also, now you have an idea what crap I am producing all day long


- Didée

[Inc]

Very lastly: No, the the removal of that wonderful logo was NOT done by iiP

What procedure did u use for removing the logo in such a nice quality? Yo want to keep it secret?

Greets
Inc.

[Didée]

Secret? What secret?

1. The name is "Didée".

2. The filter is "DeLogo".

3. The search button is at the top of the page.

4. The remainder is hard work.


- Didée

[Soulhunter]

Originally posted by Didée
One correction of my own false comment, before you start out:

To disable Pixie, use dustweight = 0.0 ! When setting "duststr=0.0", it is also (effectively) disabled ... but PixieDust would first do its full work, changing nothing, and then get copied over ... a pretty slow No-Op. By zeroing dustweight, Pixie will actually be bypassed - that's what you want to do.

Ok, I set them both to zero...


THX n' Bye

[Soulhunter]

Ok, here is my "replace dust with deen" example...

iip( dest_x=     1024,   dest_y=        576,
 \   ss1_x =    1.414,  ss1_y =      1.414,
 \   duststr =      2,  dustweight =  1,  antiflicker1= true, antiflicker2= true,
 \   detailcontr1=127,  detailcontr2 = 255,  contr_radius =  3, PixSharp=0.25,
 \   ss2_x =      3.5,  ss2_y =        3.5,
 \   Xstren =     255,  Xlimit =       255,
 \   subpelstren= 1.0,  flatweight =  64,
 \   protect_floor= 4,  protect_bias =  16,
 \   dering =    -60,  dering_weight= 1.0, dering_floor = 16, dering_bias=8,
 \   detail_floor= 1,  EQ = 2,
 \   warp_Y = false,     warp_UV = false,
 \   debug= "showall | compareH/V | protect | dering",  
 \   cropx=32, cropy=16
 \   )

edeen(3,3,3,1,1,true)

iip( dest_x=     1024,   dest_y=        576,
 \   ss1_x =    1.414,  ss1_y =      1.414,
 \   duststr =      0,  dustweight =  0,  antiflicker1= true, antiflicker2= true,
 \   detailcontr1=127,  detailcontr2 = 255,  contr_radius =  3, PixSharp=0.25,
 \   ss2_x =      3.5,  ss2_y =        3.5,
 \   Xstren =     255,  Xlimit =       255,
 \   subpelstren= 1.0,  flatweight =  64,
 \   protect_floor= 4,  protect_bias =  16,
 \   dering =    -60,  dering_weight= 1.0, dering_floor = 16, dering_bias=8,
 \   detail_floor= 1,  EQ = 2,
 \   warp_Y = false,     warp_UV = false,
 \   debug= "showall | compareH/V | protect | dering",  
 \   cropx=32, cropy=16
 \   )

[malkion]

Originally posted by Soulhunter
Ok, here is my "replace dust with deen" example...

It seems like pixiedust looks a tad sharper on still samples.

How does it look when in motion?

[DarkNite]

@Didée

When I first looked at that source (thinking DVB content would actually be much sharper) I thought it might've been one of those hazed over love scenes.

Those results looks amazingly better than that soft source.

@Soulhunter

Any particular reason for using eDeen the spatial monster, instead of the kinder, faster, more tweakable, and much less prone to access violations, Deen?

Looking at the upper left corner wall section, and the lateral obliques of Mr. "Russel Crowe beats my a%$!" there's some strange oil painting effect going on. Looks like more experimenting may be called for.

[Soulhunter]

Originally posted by DarkNite
Any particular reason for using eDeen the spatial monster, instead of the kinder, faster, more tweakable, and much less prone to access violations, Deen?

Not really, was only a quick "try n' see" attempt...

Originally posted by DarkNite
Looking at the upper left corner wall section, and the lateral obliques of Mr. "Russel Crowe beats my a%$!" there's some strange oil painting effect going on. Looks like more experimenting may be called for.

Yeah, I will do further experiments !!!


Bye

[Soulhunter]

Now a "replace Dust with MipSmooth" example...

IIP as before:




MipSmooth MovieHQ:




Bye

[tedkunich]

@Didée

I've been trying out your iip processor for my LD caps and have run into a bit of a snag that I'm hoping you might be able to suggest some guidance with:

with the following parameters:

iip( dest_x=     720,   dest_y=        360,
 \   ss1_x =   1.414,  ss1_y =      1.414,
 \   duststr =      2,  dustweight =  1,  antiflicker1= false, antiflicker2= false,
 \   detailcontr1=127,  detailcontr2 = 255,  contr_radius =  2, PixSharp=0.4,
 \   ss2_x =      3.5,  ss2_y =        3.5,
 \   Xstren =     255,  Xlimit =       255,
 \   subpelstren= 1.0,  flatweight =  0,
 \   protect_floor= 4,  protect_bias =  16,
 \   dering =    -60,  dering_weight= 1.0, dering_floor = 16, dering_bias=8,
 \   detail_floor= 1,  EQ = 2,
 \   warp_Y = false,     warp_UV = false,
 \   debug= "",  
 \   cropx=32, cropy=16
 \   )

I get some really ugly artifacts/grain/noise in the returned clip. I have experimented with the detailcontr1 settings without much success in returning a clean clip. Setting detailcontr2 along with all of the other requisite settings to zero, reduces the introduced noise somewhat. Setting the Dust strength higher (8) does nothing but lose detail - noise/artifacts in the returned clip are unaffected. Setting ss1_x and ss1_y to something like 5 will eliminate the artifacts, but then your script does not really sharpen up any edges. Any thoughts as to what to try next?

T


PS I'll try to get a sample of what I am talking about and get it hosted somewhere.....

[Didée]

tedkunich:

Yes, please make a short sample available. Without, everything is guesswork.

However, you definetly should raise detail_floor from 1 to something >>8. Realize that default is 20...
Check with "showall" that the "detail" debug frame appears with not too much noise in it (a _moderate_ noise level in this frame is okay). Raise detail_floor until it starts to appear cleaner.

Also, those two antiflicker switches could help to improve.

Lastly, are you sure about your dering settings? Default is -80/8/16. Your settings of -60/16/8 will push "deringing" on big amounts of frame area - and therefore also in areas of weak detail (real or artificial), that might tend to be spatially instable from the very start. That could lead to additional "artefacts", namely those transitions between "deringed" and not "deringed" parts, in areas where it was not intended to be used. (The actual [new] deringer is mostly designed to clean around hard edges.)

Perhaps your input is simply not clean enough and would require additional pre-processing. As mentioned time ago, iiP is not so much a "repairer". Its focus lies on enhancing. If source's artefact aren't fully catched by Dust & deringing, then something else must be done beforehand.

A sample would help.

[tedkunich]

@Didée

Here are two small samples... In clip 1, my settings distort the face, giving it almost a cartoonish look. Clip2, there is lots of artifacts in the sky background....

http://www.geocities.com/tedk_1/clips.htm

My POS ISP has no hosting, so I had to go thorugh a free Geocities account... right now, I get a message saying to check back later.... dunno what that is all about.... give it an hour or so and the clips should be available.


I did ramp up the detail_foor setting to 60 and was able to get much better results, although I get a lot of noise around the sharp transitions in detail...

Would it be to difficult to put together a usage guide of your script for us noobs? kind of a first things first approach with maybe a bit of a narrative as to settign up the parameters. You have descriptions of the parameters scattered about in this thread as the script has evolved, but I for one, still have not been able to determine what are the effects of each of the paramters. As an example, I was able to figure out the "DERING" debug view, but is there an prefered order in adjusting the parameter so that they are not skewed way off? I had no clue as to what the "DETAIL" view was telling me, prior to your reply - from what I gather now, the white mask areas are where there are details that will not be processed????


Thanks for all your efforts!!!


T

[Didée]

I love GeoCities

Now it tells me the site has exceeded its limit. Some minutes before, the links appeared, but could not be accessed ("... is temporarily unavailable").

>> Would it be to difficult to put together a usage guide

Yes. I'm currently a little out of shape, and have neither the energy nor the mood for such an effort.

Everything lies somewhere in the thread ... be adventureous, and read carefully

[JanBing]

Wow, I am a bittle overwhelmed by this thread... any chances for some collected info? Simply as: Which plugins do I need, and what's the script *g*

There are so many version (and so many info in general) floating around in this thread, that I - as a newbie to AviSynth - am definintely knocked out.

So... to cut a long story short... (and this is where you finish this sentence )

[Didée]

tedkunich:

Finally geoCities delivered.

Well ... difficult source to get HD quality from (joke). The Han Solo sequence can't get good-looking at all: too much noise, too much detail exceeded by noise floor. And faces start looking ugly much earlier than anything else...

Perhaps more motion compensated pre-processing via MVTools + (e.g.) DeGrainMedian could prepare the source better.

For iiP alone, perhaps try the following:

- manually replace PixieDust with GoldDust
- settings suggestion:

crop(0,0,-0,-4)
converttoyv12()
iip( dest_x=       720,  dest_y=        288,
 \   ss1_x =     1.414,  ss1_y =      1.414,
 \   duststr =       3,  dustweight =   1.0,  antiflicker1= true,  antiflicker2= true,
 \   detailcontr1= 127,  detailcontr2 = 255,  contr_radius =    3,  PixSharp=0.3,
 \   ss2_x =       3.5,  ss2_y =        4.0,
 \   Xstren =      255,  Xlimit =       255,
 \   subpelstren=  1.0,  flatweight =     0,
 \   protect_floor = 0,  protect_bias =  8,
 \   dering =      -60,  dering_weight= 1.0, dering_floor = 8,  dering_bias=12,
 \   detail_floor = 25,  EQ = 2,
 \   warp_Y = true,      warp_UV = false,    exborder = false,
 \   debug= "-compareV", 
 \   cropx=40, cropy=16
 \   )

I didn't play too much with it, but (together with GoldDust) the 2nd scene looks rather OK, and Harry looks not too bad when in motion. Not sure if warpY should be used or not, though.

BTW, Harry looked pretty awful when I fed the source without any filtering to XviD

[tedkunich]

@Didée

Thanks, FYI I spent a bit of time trying to find some of the worst scenes in the capture for those clips... the rest is not that bad... especially considering that it is a LD capture with a $50 card...


I'm processing the clip now with some other settings I came up with.... It has about 4 more hours to go, so I'll let it finish, then I'll try your recomendations.

Any tips on the MVTools? I have never used it before. How would that work with DegrainMedian?


Thanks


Ted

[audioslave]

@Didée
Great work!
This just gets better and better. Any chance of getting a zip file with all filters + the script? Or, maybe there already is one that I missed...
Thanks again!

[Jo_Gors]

May be I'm posting a boring question, but I tried iip several times with no success.
Whatever warsharp version I use, seems doesn't works.

With the version dideè linked in his post, I allways get a "hieroglyplical like" error message (japanese?) and with the other one I've found in the web, I get "..there's no function named unsharpmask".


The loadpluginex is present both in the script and the plugins' folder.
I'm running the latest 2.55 avisynth version (august,31)

Tried on 2 different PC with the same results.

Any suggestions?



greetings

Jo

[Didée]

msvcr70.dll/msvcr71.dll
msvcp70.dll/msvcp71.dll
--> %windir%/system32

Plus for safety, put WarpSharp.dll *NOT* in the pluginfolder, but load it manually. Autoloading may cause problems, too.