remove halo

[franketto]

I have a dvd video with some halo effect especially visible in certain frames.
I have deinterlaced it, applied Srestore to get it back to the original 25fps, denoised and sharpened. But the halo was there also at the beginning, even if a bit less.
Original frame:
http://i.imgur.com/ncXNhRO.png

Final frame:
http://i.imgur.com/sfjbdBW.png

I tried
abcxyz(rad=4.0, ss=1)
and
Masked_DHA( rx=5 , ry=5 )
They remove it a bit but they also create a sort of "toon" effect especially visible in the faces.

Which filter should I use to remove these white halos?

[feisty2]

what did u do exactly?
I see ringings in "final frame" are mighty enhanced from "original frame"

[franketto]

I did:


source = DGDecode_mpeg2source("Z:\3NipotiDVD\01-01\VTS_01_1.d2v")
bobby = source.QTGMC( Preset="Super Fast")
calmy = bobby.Clense(reduceflicker=false).merge(bobby,0.5).crop(8,0,-8,0)
V2 = bobby.Srestore(dclip=calmy, frate=25, omode=6, mode=-4)

# denoise 1:
super = V2.MSuper(pel=2)
bv1 = MAnalyse(super, isb = true, delta = 1, overlap=4)
fv1 = MAnalyse(super, isb = false, delta = 1, overlap=4)
bv2 = MAnalyse(super, isb = true, delta = 2, overlap=4)
fv2 = MAnalyse(super, isb = false, delta = 2, overlap=4)
bv3 = MAnalyse(super, isb = true, delta = 3, overlap=4)
fv3 = MAnalyse(super, isb = false, delta = 3, overlap=4)
V2noise = V2.MDegrain3(super,bv1,fv1,bv2,fv2,bv3,fv3,thSAD=400)
V2 = V2noise.FFT3DFilter(bt=-1, sharpen=2) #only sharpen

#OR denoise 2:
V2a = V2.DeGrainMedian(limitY=2 , mode=1)
V2 = V2a.fft3dfilter(sigma=1.5, bt=5, bw=32, bh=32, ow=16, oh=16, sharpen=1.5, ncpu=4)

[Guest]

Is that the complete script? I don't see an appropriate return statement or assignment to last.

[franketto]

Well, it's not complete, but I thought the last line was not important as for the problem, anyway this is:


return V2.crop(8,0,-6,0).addBorders(6,0,8,0)

Also it lacks all the plugin load at first. For the rest, yes it's all.

[johnmeyer]

Since this is a TV show that was shot on film, deinterlacing is probably the wrong thing to do to it, and that may be contributing to whatever problem you are seeing. If it is PAL, then you probably don't need to do anything to it. If it is an NTSC DVD, then you might need to do inverse telecine. I don't think QTGMC is the right tool for the job. If you can post a small sample, I can tell you for sure.

[bxyhxyh]

You can try this script (http://forum.doom9.org/showthread.php?p=1679763#post1679763).
It can be buggy.

# Your script
...
dehalo_blah(rx=1.6,ry=1.6,darkstr=0.45)

It is best i can do for this screenshot.
if it removes too much try lower rx and ry.

[franketto]

Thank you johnmeyer, you're right in supposing IVTC and so on... But I should have clarified better from the beginning.
This film was shot @24f, but they used a PAL master to make the NTSC DVD... Furthermore they did it the wrong way, so it has some field blended, very bad.
It definitely needs Srestore, as IVTC doesn't work well with it. I also need it @25f.
As I need bob for Srestore I used QTGMC, particularly because it uses NNEDI3 that gave a lot more details.
See here (http://forum.doom9.org/showthread.php?t=170583).

[Guest]

Looks like the sharpener is worsening the haloes. Don't sharpen it. Or try a more forgiving sharpener, such as LimitedSharpenFaster().

[franketto]

@neuron2, I tried LSFmod but it's similar.
Also I tried only to denoise, but halo is still there, of course a little less:
http://i.imgur.com/qYDISbi.png


Thank you, bxyhxyh!
http://i.imgur.com/Esp9I0c.png
It seems far better!

Also I tried your dehalo with fft sharpen or with LSFmod:
http://i.imgur.com/CS3Mdhj.png

With LSFmod the dehalo looks bit better, but I prefer fft sharpen because of the other details, also in all the other scenes where there's no halo.

Of course if you have another sharpen suggestion for this kind of material, I will try it, thank you.

[foxyshadis]

You're pretty much beyond the limit of restoring original detail and into synthesizing new details, now. To me, the telltale signs of nnedi are obvious. What is the final effect you're looking for? SeeSaw and FineSharp will give you other different effects, as well, and of course you can combine sharpeners at lower strength to get the best of both.

[franketto]

Maybe you're right.
But what I would like is to have a video a bit less blurred of what it was the original. Especially when seeing it on large FullHD display that blur themself the material for the scaling. That's the reason I want to sharpen it.

For the NNEDI3 I tried tdeint, yadif and also QTGMC without nnedi3 but I saw straight diagonal lines sometimes get to "flicker" with jagged borders and it's not good to see: with nnedi3 that's solved.
ie (without denoise-sharpen-dehalo):
http://i.imgur.com/Whkk3W9.png
That is especially visible while playing, not with stills.


I will try SeeSaw and fineSharp.


BTW:
The halo is there also in the PAL broadcasted stuff, so it was there also in the PAL master they used to make the NTSC DVD:
http://i.imgur.com/0BE0L9A.png

[franketto]

So, I launched x264.exe from a bat file in cmd with an .avs input of a dvd.
At 62% it has blocked.
x264.exe --preset=medium --crf 16 --ref 3 --bframes 2 --keyint 100 --subme 3 --sar 8:9 --output "out.mkv" F:\****.avs
i7 920 4core (8 HT) Win7 64bit

Is it possible to restore the process or should I restart from the beginning?

What could have caused the block?
I used:


#loadplugins...

setmtmode(2,8)

V2 = AviSource("****.avi") #huff file

SetMtMode(2)

super = V2.MSuper(pel=2)
bv1 = MAnalyse(super, isb = true, delta = 1, overlap=4)
fv1 = MAnalyse(super, isb = false, delta = 1, overlap=4)
bv2 = MAnalyse(super, isb = true, delta = 2, overlap=4)
fv2 = MAnalyse(super, isb = false, delta = 2, overlap=4)
bv3 = MAnalyse(super, isb = true, delta = 3, overlap=4)
fv3 = MAnalyse(super, isb = false, delta = 3, overlap=4)
V2 = V2.MDegrain3(super,bv1,fv1,bv2,fv2,bv3,fv3,thSAD=400).f3kdb()

V2 = V2.LSFMod(defaults="slow")
V2 = V2.dehalo_blah(rx=1.6,ry=1.6,darkstr=0.45)

return V2.crop(8,0,-6,0).addBorders(6,0,8,0)

[bxyhxyh]

At 62% it has blocked.
What could have caused the block?

Do you mean it has crashed? Or stucked?
dehalo_blah() uses lsfmod for halo detection. Calling lsfmod twice can be caused memory problem. But I don't think so.
Or you can try different modes for source filter.
SetMTMode(5,8) for instance.


Is it possible to restore the process

If your encode crashed and your incomplete file still exists, yes.
To do that, you have to know where it is stopped.
1.
Load your incomplete video with any source filter in AvsPmod or VirtualDub.
Then write down the last frame number.

2.
Since x264 has GOP structure, last gop of your encode is not completed when it has crashed. So you have to cut out that part using MKVMerge.
Choose Split Mode: split after frame/field numbers
At first try number-30 (if it has crashed at 20000th frame, start from 19970).
Start Muxing.
If your cut video's duration is still same as your failed encode, try number-60 (19940) etc. Keep doing it until you cut out last incomplete gop.

3.
Load your cut video with frame accurate source filter in AvsPmod or VirtualDub
See the last frame and its number

4.
Open your original avisynth script. Compare the last frame of cut video with original script's equivalent frame to make sure frame number.
Add Trim(framenumber+1,0) at the end of your original script.
Then encode it using exactly same settings as previous encode.

5.
Append your cut video and new encodded video using MKVMerge.

My English is not so good, but i think you would understand its point.

[bxyhxyh]

I've updated dehalo_blah()
Now you don't have to call lsfmod before dehalo_blah(), because it internally sharpens the video before removes halo.
So your
V2 = V2.LSFMod(defaults="slow")
V2 = V2.dehalo_blah(rx=1.6,ry=1.6,darkstr=0.45)

will become
V2 = V2.dehalo_blah(rx=1.6,ry=1.6,darkstr=0.45,sharpness=100,sharptype="slow")

And your script will become slightly faster and will use lower memory.

[franketto]

Thank you bxyhxyh, I will use your suggestions.