DVD "The road" and "dancing noise" problem.

[maximo]

First of all, sorry for my english. :o

This is a short clip (45 sec.) from my DVD9 PAL Version "The road":

http://www.mediafire.com/?3vb3u6o77kx61h9

There is a terrible "dancing noise" problem.

For a good and clean x264 encoding, can you help me with an avs script that solve the problem? :confused:

Thank you.
Maximo.

[Didée]

# FluxSmooth.dll
# RemoveGrain.dll
# mt_masktools-25.dll
# mvtools2.dll
# nnedi3.dll

mpeg2source("D:\__DL\The road\VIDEO_TS\VTS_01_1.d2v")
crop(0,72,-0,-72,true)
oo=last

nnedi3(field=-2,nsize=0,nns=3)
merge(selecteven(),selectodd())
D1=mt_makediff(oo,last)
D2=mt_makediff(last,last.removegrain(11,-1))
last.mt_adddiff(D2.repair(D1,13,-1).mt_lutxy(D2,"x 128 - y 128 - * 0 < 128 x 128 - abs y 128 - abs < x y ? ?"),U=2,V=2)
o=last

x1 = o.fluxsmootht(3)

x2 = o.removegrain(11,-1)
x22 = o.mt_makediff(mt_makediff(x2,x2.removegrain(20,-1)))
enhD = mt_lutxy(x22,x22.removegrain(4),"128 x y - abs 2 / 1 1.6 / ^ 2.51 * x y - x y - abs 0.001 + / * +",U=2,V=2)
enh = o.mt_adddiff(enhD,U=2,V=2)

BLK = 16 # 8
ME1 = 5
ME2 = 2 # 8
_DCT = 5

sup1 = enh.MSuper(hpad=16, vpad=16, pel=2, levels=0,sharp=1)
sup2 = x1.removegrain(11).MSuper(hpad=16, vpad=16, pel=2, sharp=0)
bv2 = MAnalyse(sup2,delta=2,truemotion=false,global=true,blksize=BLK,overlap=BLK/2,search=ME1,searchparam=ME2,isb=true, dct=_DCT)
bv1 = MAnalyse(sup2,delta=1,truemotion=false,global=true,blksize=BLK,overlap=BLK/2,search=ME1,searchparam=ME2,isb=true, dct=_DCT)
fv1 = MAnalyse(sup2,delta=1,truemotion=false,global=true,blksize=BLK,overlap=BLK/2,search=ME1,searchparam=ME2,isb=false,dct=_DCT)
fv2 = MAnalyse(sup2,delta=2,truemotion=false,global=true,blksize=BLK,overlap=BLK/2,search=ME1,searchparam=ME2,isb=false,dct=_DCT)

o.mdegrain2(sup1,bv1,fv1,bv2,fv2,thsad=321,thSCD1=256,thSCD2=96)

# -- visualisations --
# stackvertical(o,last)
# interleave(o,last)
# levels(15,1.0,144,0,255,false)

return(last)

If you think the detail enhancement is too much, reduce the "2.51" in the "enhD=..." line.

[Didée]

BTW, the picture quality is ... a bit strange. There is a *tremendous* amount of detail/resolution in horizontal direction, while the resolution in vertical direction is *particularly poor* (looks almost like height has been halved, then point-resized again to original height.)

Performing some antialiasing might be be indicated ...

Also, is all of the movie so dimmed and low-contrast, or is it just the scenes of this sample that happen to be so?
(You see, I don't know that movie.) (Yet.);)

[talen9]

Didée, I didn't see the clip maximo uploaded, but the movie (and the cinematograhy) is very bleak and subdued... worst seeing anyway :)

[Taurus]

# Plugins needed: Happy hunting for the right versions for your OS and Avisynth version!
# fftw3.dll # copy to windows\system32 if 32bit
# DGDecode.dll
# RemoveGrain.dll
# Repair.dll
# mt_masktools-25.dll
# mvtools2.dll
# Hqdn3d.dll
# FFT3DFilter.dll

mpeg2source("VTS_01_1.d2v")
crop(0,72,-0,-72,true)

# Autolevels() # to make sh... visible, not necessary for encode
temporaldegrain() # heavy gun, flatens the most out of it....
LSFMod(Defaults="Slow", strength=200) # only use on progressive content, or Didée will come after you...

# You have to tweak the filters yourself!

As Didée wrote: The picture quality is bad.
Is this on commercial DVD ? :confused::devil::D

[maximo]

Hi Didée, thanks for your help... and for the script. :)

As soon as I return to home from the job, I will test your script... and I will post the output.

BTW, the picture quality is ... a bit strange. There is a *tremendous* amount of detail/resolution in horizontal direction, while the resolution in vertical direction is *particularly poor* (looks almost like height has been halved, then point-resized again to original height.)

You're too technical for my knowledge :)... the VOB is untouched, I've only cut the sample... anyway, i tell you that this is a "dark" movie... all of the movie is like the sample... but it is a GREAT movie.

For antialiasing, which filter you suggest?

[Didée]

In any case, I've edited the above script to include antialiasing.
(That's probably daa(), and probably the 150th time that I've written that code in plain, instead of using a function.):scared:

Looks better now, as far as pixel rendering is concerned.

[maximo]

Hi Taurus, thanks for the reply... tonight I will test your script.

As Didée wrote: The picture quality is bad. Is this on commercial DVD ? :confused::devil::D

YES... this is a Italian DVD9 of the movie. :angry::angry::angry:

[Didée]

Thinking about "autolevels" ... did we already have sth along the lines of

a = last
b = a.autolevels

diff = mt_makediff(a,b)

a.overlay(diff, mode="softlight") # or maybe hardlight

? ;)

[maximo]

In any case, I've edited the above script to include antialiasing.
(That's probably daa(), and probably the 150th time that I've written that code in plain, instead of using a function.):scared:

Looks better now, as far as pixel rendering is concerned.

OK... later I will post results of my test. :)

[maximo]

Hi, I use LoRd_MuldeR's Simple x264 Launcher for my encoding.

These are the parameters that I have used for the test:
Mode: 2-Pass
Target Bitrate: 2500 (kbit/s)
Preset: Medium
Tuning: None
Profile: High
Params: --level 4.1 --sar 64:45 --deblock -1:-1 --bframes 5 --b-adapt 2 --ref 5 --rc-lookahead 60 --merange 32 --me umh --subme 10 --direct spatial --trellis 2 --psy-rd 1.0:0.15 --no-dct-decimate --no-fast-pskip --threads auto --thread-input

Didèe script output:
http://www.mediafire.com/?1ccvabf0fpofq9b

Taurus script (@default) output:
http://www.mediafire.com/?x228l6808ek2vv1

What do you think?

For me:

1) Taurus output is more clean (dancing noise is less then Didée output), but there is a more evident banding??? artifact;
2) Didée output is more detailed then Taurus output.

This is obvious, right? More aggressive is denoiser, more clean but less detailed will be the output, right?

Is possible to obtain a better result whit this kind of source? If yes, how?

Confronting the screenshoot, I opt for more detailed Didée output... but in movement, maybe, I would opt for more clean Taurus output.

Which you would choose?

Also, on PC the banding artifact can be diminished with FFDShow Deband filter... but on TV???

Is possible to remove banding directly in encoding phase?

Can you post me an avs script for this?

Thanks all, :)
Maximo.

[Bloax]

I suppose you mean all the haloing?
If so, then this script of Didée's (http://forum.doom9.org/showpost.php?p=1532192&postcount=61) should do the trick. (DeHalo_Alpha)

http://img809.imageshack.us/img809/7400/stuff00.th.png (http://img809.imageshack.us/img809/7400/stuff00.png)

[aegisofrime]

I suppose you mean all the haloing?
If so, then this script of Didée's (http://forum.doom9.org/showpost.php?p=1532192&postcount=61) should do the trick. (DeHalo_Alpha)

http://img809.imageshack.us/img809/7400/stuff00.th.png (http://img809.imageshack.us/img809/7400/stuff00.png)

Holy crap, is that screenshot from the DVD?

[Didée]

Hmmh, I'm not sure if those halos should be touched. Look close, and note that it's only bright gloom - whereas the dark neighborhood has not been darkened, i.e. it's not a general sharpening or unsharp-masking operation, but really a gloom effect. Also, considering the film plot, these two seem to be about the last true manchilds in a world of human animals, or something. (I've only read the plot on IMDB.)

For these reasons, and unless a better source (BluRay) would show up a different picture, I would readily believe that these "halos" in fact are an intended artistic expression. "Positive aura", you know.

[Didée]

@maximo: Are you seriously asking for opinions, or are you trying to pull our legs? :D

Looking at just those two videos, I see one time a flat detail-less soup with oversharpened borders, and one time something that looks remotely like a "cinema movie".

Methinks that TemporalDegrain is far, far too strong for this source. At it's default settings anyway. Several parameters could be weakened, but even then I think that TemporalDegrain is overkill in this case. It has been made for heavy grain. But what we have here is far away from being "heavy". In contrary, it is moderately mild film grain, just that it has been malformed by mpeg-2 compression. No need to shoot a butterfly on a wheel, as far as denoising is concerned.

* * * * *

I've altered the script a little: for one a noise-dampener that should better prevent unwanted noise from being sharpened, and a mini-tweak to reduce sharpening aggressive-ness by one notch. Both together should result in a more clean result overall, and make the detail-enhancement a bit less pronounced.

Incidentally, after many years, this is my first "public" usage of mt_mappedblur (which once upon a time was created upon my behalf...) :o :)

# FluxSmooth.dll
# RemoveGrain.dll
# mt_masktools-25.dll
# mvtools2.dll

function sbr(clip o) {
rg11=o.removegrain(11,-1)
rg11D=mt_makediff(o,rg11)
rg11DD=mt_makediff(rg11D,rg11D.removegrain(11,-1)).mt_lutxy(rg11D,"x 128 - y 128 - * 0 < 128 x 128 - abs y 128 - abs < x y ? ?")
return(o.mt_makediff(rg11DD,U=2,V=2))
}

function MinMapBlur(clip c) {
map=mt_lutxy(c.mt_expand(),c.mt_inpand(),"x y - 1 3 / * 3 +")
a = c.removegrain(4,-1)
b = c.mt_mappedblur(map,"1 2 1 2 8 2 1 2 1","dump")
aD=mt_makediff(c,a)
bD=mt_makediff(c,b)
return(c.mt_makediff(mt_lutxy(aD,bD,"x 128 - y 128 - * 0 < 128 x 128 - abs y 128 - abs < x y ? ?"),U=2,V=2))
}


mpeg2source("D:\__DL\The road\VIDEO_TS\VTS_01_1.d2v")
crop(0,72,-0,-72,true)
oo=last

nnedi3(field=-2,nsize=0,nns=3)
merge(selecteven(),selectodd())
D1=mt_makediff(oo,last)
D2=mt_makediff(last,last.removegrain(11,-1))
last.mt_adddiff(D2.repair(D1,13,-1).mt_lutxy(D2,"x 128 - y 128 - * 0 < 128 x 128 - abs y 128 - abs < x y ? ?"),U=2,V=2)
o=last

x1 = o.fluxsmootht(3)

x2 = o.removegrain(11,-1)
x22 = o.mt_makediff(mt_makediff(x2,x2.removegrain(20,-1))) .MinMapBlur()
x222 = x22.removegrain(4,-1)
x222 = x222.sbr().merge(x222,0.51)
enhD = mt_lutxy(x22,x222,"128 x y - abs 2 / 1 1.6 / ^ 2.51 * x y - x y - abs 0.1 + / * +",U=2,V=2)
enh = o.mt_adddiff(enhD,U=2,V=2)


BLK = 16 # 8
ME1 = 5
ME2 = 2 # 8
_DCT = 5

sup1 = enh.MSuper(hpad=16, vpad=16, pel=2, levels=0,sharp=1)
sup2 = x1.removegrain(11).MSuper(hpad=16, vpad=16, pel=2, sharp=0)
bv2 = MAnalyse(sup2,delta=2,truemotion=false,global=true,blksize=BLK,overlap=BLK/2,search=ME1,searchparam=ME2,isb=true, dct=_DCT)
bv1 = MAnalyse(sup2,delta=1,truemotion=false,global=true,blksize=BLK,overlap=BLK/2,search=ME1,searchparam=ME2,isb=true, dct=_DCT)
fv1 = MAnalyse(sup2,delta=1,truemotion=false,global=true,blksize=BLK,overlap=BLK/2,search=ME1,searchparam=ME2,isb=false,dct=_DCT)
fv2 = MAnalyse(sup2,delta=2,truemotion=false,global=true,blksize=BLK,overlap=BLK/2,search=ME1,searchparam=ME2,isb=false,dct=_DCT)

o.mdegrain2(sup1,bv1,fv1,bv2,fv2,thsad=321,thSCD1=256,thSCD2=96)

# -- visualisations --
# stackhorizontal(oo.crop(40,0,-40,-0),last.crop(40,0,-40,-0))
# interleave(oo,last)
# levels(15,1.0,144,0,255,false)

return(last)

edit#1: "return(last)" right after mpeg2source was not intended. :-)
edit#2: sbr() usage reduced to half stregth
edit#3: typo! :-D

[Taurus]

Didée's modified script is retaining much more details (look at the boys wollen hat) and the aliasing is much less.
But it's slower...
You can modify temporaldegrain, just look into the avsi :D
And yes, Didée was involved in this wonderful script too.
Take your poison.

offtopic: what have the poor actors done to deserve this :confused::devil:

@Didée: Good grief, you're a workokolic.
will try your attempt in a minute.
And yes, everything you wrote above is true!

[maximo]

I answer to all... but for me is very difficult to write in a correct English... and to understand your answers correctly... anyway...

I suppose you mean all the haloing?
http://img809.imageshack.us/img809/7400/stuff00.th.png (http://img809.imageshack.us/img809/7400/stuff00.png)

Yes Bloax... but these artifacts are haloing or banding???:confused: because FFDShow Deband filter seems to improve the situation :confused:

Holy crap, is that screenshot from the DVD?

No, this is the output... but the source, for me, is worse... if you want, in first post of the thread, I've linked the source.

@maximo: Are you seriously asking for opinions, or are you trying to pull our legs? :D

I'm serious... my avisynth knowledge is very low... with good DVD source, I don't have problems to make an h264 backup of my DVD collection... but with this kind of source I don't know how to proceed.

You can see my other help request at this old thread:
http://forum.doom9.org/showthread.php?t=161469

Same "terrible" source.

If I use an aggressive denoiser, I lose detail and haloing/banding artifacts appear... If I use a moderate denoiser, the output is not very clean... I repeat... I don't know how to proceed.

Here because in mine two thread (this one and the old one above linked) I've posted the source... I would want to see the "BEST" (I know the forum rule :)) output that an "avisynth expert" can obtain and compare it to mine.

That's all. :)

Tonight I will test your new script.

Thanks all,
Maximo.

[Dogway]

maximo, to prevent banding when heavy denoising use the Dither tools. If you don't know how to set up the code for high bit depth denoising then use my smdegrain mod and use lsb=true.
It can probably denoise too much specially when using a high temporal radius, but that is because source is very low in contrast, so that motion vectors cannot detect the details. The trick is to feed a nice clear and contrasty image as a prefilter.

[maximo]

use my smdegrain mod and use lsb=true.

I've downloaded SMDegrain1.5d.avsi but with this script:

**********************************************
DGDecode_mpeg2source("H:\Ripping\Video\The road\Dogway\The road.d2v")

Trim(111948,113075)

crop( 0, 72, 0, -72)

Import("C:\Program Files (x86)\Ripping\AviSynth\plugins\SMDegrain1.5d.avsi")

SMDegrain(lsb=true)
**********************************************

appear this error:

**********************************************
Analyzing source file:
Avisynth error:
Script error: MDegrain3 does not have a named argument "lsb"
(C:\Program Files (x86)\Ripping\AviSynth\plugins\SMDegrain1.5d.avsi, line 502)
(H:\Ripping\Video\The road\Dogway\The road.avs, line 9)

Fatal Error: Failed to analyze the source video!

Fatal Error: Failed to create the process!
**********************************************

Can you post me an example script?

The trick is to feed a nice clear and contrasty image as a prefilter.

How??? Can you post me an example script?

Thank you Dogway :)
Maximo.

[Taurus]

@Didée:
Just tried your second attempt.
It leaves the mood of this video intact.
Not death calm like my approach.
Thanks for sharing.
--------------------------------------------
My eyes are still bleeding from this source.....
--------------------------------------------