need some help with project...pics included

[neo211]

i'm working on a GSD special titled Our Music (i think). the only raw i was able to obtain is of very poor quality. i would say VHS. i'm new to the AVISynth scene so i was wondering if i can get some help from some of the veterans. i've got two screencaps in PNG format so u guys/gals can take a look. I dont know much about what filters do what or how to even identify specific artifacts/noise. so any help regarding this and how to best clean this up is greatly appreaciated. this is going to be a parody =). oh and i'm looking to encode with Xvid (1.1 Beta Koepi build). TIA for any help you may provide.

i dont really know how to post the images o_o

[Scarfac3]

http://www.free-webhosts.com/free-image-hosting.php
there

[Mug Funky]

pardon my ignorance, but what's a GSD?

when i see the pics, i'll be able to make a more useful post :)

[neo211]

here are the images (hope they work)

http://files.bighosting.net/at3607.png
http://files.bighosting.net/nm3628.png

and GSD is Gundam Seed Destiny

[Chainmax]

First of all you'll want to get rid of the haloing (check out the picture on the second screenshot). I recommend you use BlindDeHalo3 (http://forum.doom9.org/showthread.php?s=&threadid=74003&perpage=20&highlight=blinddehalo3&pagenumber=4) for that. The thread itself is quite self-explanatory. I advice you to try the defaults first to see if they work. Then you might want to use some light smoothing, like Deen("a2d",1,3,5) or Convolution3D(preset="movieHQ"). Or, if you feel gutsy, try Didée's IIP (http://forum.doom9.org/showthread.php?s=&threadid=70916&perpage=20&highlight=IIP&pagenumber=6).

[neo211]

k i'll give those a try. is the Halo effect all the little lines around the people? just trying to find out how to recognize it in the future. thanx for the help

[neo211]

wow i'm completely confused as to how to use BlindDeHalo. i'm supposed to use it with MaskTools which i got. but i cant figure out how.the few times i've used AVISynth i've used plugins that are DLLs and not AVS scripts (BlindDeHalo). so i'm a little confused as to how to use the BlindDeHalo script within my own script let alone how i go about actually using BlindDeHalo with MaskTools. the thread explains what the parameters do but not really how i go about using it. thanx for any help. i'm gonna keep looking around and see if i ca find an FAQ or something.

[neo211]

ok i've tried figuring out how exactly to use BlindDeHalo but to no avail. right now i have a script that looks like this:

Loadplugin("C:\Program Files\AviSynth 2.5\plugins\convolution3d.dll")
Loadplugin("C:\Program Files\AviSynth 2.5\plugins\MaskTools.dll")
Import("C:\Program Files\AviSynth 2.5\plugins\BlindDeHalo3.avs")
AviSource("e:\1gb_our_music_raw.avi")
ConvertToYUY2()
Convolution3d (preset="movieLQ")

VDubMod opens it with now problem. i'm not sure how to implement BlindDeHalo here. I've gotten errors constantly with whatever i have tried. i have not changed anything in the original BlindDeHalo AVS to ensure that it can at least load it. also i'm not sure if i'm supposed to be able to see the difference in the VdubMod output window but if i am then Convolution3d is not doing anything. its probably me though. :(

[Chainmax]

I will give you a detailed description on how to use avs functions (like BlindDeHalo3) tomorrow (gotta go sleep now :)). Haloing is also known as ghosting and you notice it when there's a transparent outline around people and objects. BTW, why converting to YUY2? Isn't the source already YUY2 since it's a capture? If not, just go to www.avisynth.org/warpenterprises and download Convolution3D YV12.


P.S: just one thing, if the plugins are in avisynth's default plugins folder (like it is in your case) you don't have to load them as they are loaded automatically. The problem you have with BlindDeHalo3 is that it has an avs extension, and it should have an avsi extension if you want to autoload it. In any case, I would recommend you to put all of your filters and avs functions in a different folder than avisynth's default one and load them manually like you tried to do.

[neo211]

thank you so much. i'll be sure to move the plugins and functions.

[neo211]

ok well i was playing around with BlindDeHalo3 and Convolution3D and Deen. i was actually able to get some results. it seems to have less noise but theres also loss of detail. More than i thought there would be. it seems BlindDeHalo3 is a bit to strong. This is my current script:


#PLUGINS:
LoadPlugin("C:\Documents and Settings\Neo\My Documents\Software - Editing\Tools - Video\AVISynth\plugins\deen.dll")
LoadPlugin("C:\Documents and Settings\Neo\My Documents\Software - Editing\Tools - Video\AVISynth\plugins\Convolution3DYV12.dll")
LoadPlugin("C:\Documents and Settings\Neo\My Documents\Software - Editing\Tools - Video\AVISynth\plugins\MaskTools.dll")
Import("C:\Documents and Settings\Neo\My Documents\Software - Editing\Tools - Video\AVISynth\plugins\BlindDeHalo3.avs")

#SOURCE:
AviSource("F:\1gb_our_music_raw.avi", false)

#Deen:
Deen("a2d",1,3,5)

#BlindDeHalo3
BlindDeHalo3()


the top image is just with Deen. i dont see any difference from the original that had no filtering. The bottom one is with both Deen and BlindDeHalo3. As u can see the colors seem darker and the image is more "washed-out". Although it did clean up alot of the noise. i figure i just have to tweak it a bit. Any help in that regard is welcome as i am confused by how to use BlindDeHalo3. Here are the images:



http://files.bighosting.net/wg4723.png

http://files.bighosting.net/sa4730.png

[Chainmax]

I'm terribly sorry about not having given you the description but I had some things to take care of. I'll send you a PM tomorrow, I promise :).


P.S: BDH3 is way too strong with default settings. Try something like BlindDeHalo3(rx=2.0,ry=2.0,strength=100,lodamp=4.0,PPmode=2).

[Maximillius]

Also try resharpening it with LimitedSharpen after doing the BlindDeHalo to restore some of the detail.

[neo211]

no worries man i know u have a life outside this forum lol. i'll give the new settings a try and post whatever i get from them. i await your message tomorrow =).

Maximillius:
could u maybe give a rough estimate as to what parameters i should use for LimitedSharpen()? thanx

[Maximillius]

LimitedSharpen(ss_x=2.0,ss_y=2.0,Smode=3,strength=100) works pretty good for me.

[neo211]

thank u

[Chainmax]

Smode=3 and Strength=100 are defaults for LS, you only need to put LimitedSharpen(ss_x=2.0,ss_y=2.0).

[neo211]

this is my current script:

#PLUGINS:
LoadPlugin("C:\Documents and Settings\Neo\My Documents\Software - Editing\Tools - Video\AVISynth\plugins\deen.dll")
LoadPlugin("C:\Documents and Settings\Neo\My Documents\Software - Editing\Tools - Video\AVISynth\plugins\MaskTools.dll")
LoadPlugin("C:\Documents and Settings\Neo\My Documents\Software - Editing\Tools - Video\AVISynth\plugins\WarpSharp.dll")
Import("C:\Documents and Settings\Neo\My Documents\Software - Editing\Tools - Video\AVISynth\plugins\BlindDeHalo3.avs")
Import("C:\Documents and Settings\Neo\My Documents\Software - Editing\Tools - Video\AVISynth\plugins\LimitedSharpen.avs")

#SOURCE:
AviSource("F:\1gb_our_music_raw.avi", false)

#Deen:
Deen("a2d",1,3,5)

#BlindDeHalo3
BlindDeHalo3(rx=2.0,ry=2.0,strength=100,lodamp=4.0,PPmode=2)

#LimitedSharpen()
LimitedSharpen(ss_x=2.0,ss_y=2.0,Smode=3,strength=100)

it did clean it up a bit and it looks much nicer now but i'm still having trouble getting rid of some noise mainly the noise around the peoples heads. you can notice it very well in the Pride pic above. even with the current filters that noise still sticks around a bit. i can leave it the way it is now and be ok with it but i would really like to get rid of that noise. i spent last night tweaking the settings on the three filters to see what i get and the script above gave me the best results between noise removal and detail. Deen does an ok job but i cant tell if LimitedSharpen is doing anything. I figure its not a strong sharpener so as to preserve more detail. If u wanna see a pic of the current encode let me know and i'll post it. thanx for the great help guys.

[Chainmax]

I don't see the noise you're speaking of, but I'd bet it's slight mosquito noise (the kind I described in the PM as being akin to the granularity on overcompressed jpegs). Try RemoveGrain (http://forum.doom9.org/showthread.php?s=&threadid=79751&highlight=removegrain) (mode=1) or (mode=2). As for LimitedSharpen, you can always put higher strength values if you want more sharpening.

[neo211]

heres a pic with the noise i'm talking about circled. i originally thought this is what is called the "halo effect" but now i'm not sure.

http://files.bighosting.net/qy6674.png

[scharfis_brain]

this is caused by transmission echoes.

[Chainmax]

I have no idea how to remove that kind of artifacting, sorry :o. BTW, the halo effect is most evident around the rguy's right arm.

[Leak]

Originally posted by Chainmax
I have no idea how to remove that kind of artifacting, sorry :o. BTW, the halo effect is most evident around the rguy's right arm.

Maybe try my AWarpSharpDering (http://leak.no-ip.org/AviSynth/AWarpSharpDering/) script; I've only tested it on anime, but it might also work here... :)

(Documentation is at the beginning of the script file...)

np: Digitalis - Rentrer (Snarecrisp Mix) (Silent Season Dub)

[neo211]

thanx i'll give it a try =)

[Kagura]

http://i136.exs.cx/img136/502/output3re.th.png (http://img136.exs.cx/my.php?loc=img136&image=output3re.png)

That is the best I can do with it with just a picture. If there is a small lossless of the section, I could do more, as I am currently limited to just spatial filters. The type of noise seems to be temporal. I think the characters are swaying from side to side, which causes the haloing and edge noise. For your video, I would probably insert a fluxsmootht(5) and/or temporalsoften(2,3,3,6,2) somewhere. and see if that works better than purely spatial filtering.

Let me know if I killed too much detail. Too much anime encoding for me means that I've lost my edge for filting real life clips =(.

Oh, limitedsharpen wouldn't be good for this, imho.

My script is:

yv12lutxy(last, last.unfilter(-30,-30), Yexpr = "y x < y x ?")
maskedmerge(last, last.deen("a3d",1,3,5), BlankClip(input, color=$4C4C4C))
lanczosresize(1280,960)
_2dcleanyuy2()
_2dcleanyuy2()
lanczosresize(640,480)
hqdering()
unfilter(-5,-5)
mergechroma(awarpsharp(10,1))

[neo211]

that looks amazing! it is lacking some detail, most noticeably in the two front people (the clothing). le me know which setting i have to tweak to get some of the detail back and i'll be the happiest person on Earth. btw would u happen to know how to reduce/remove the noise that is normally found around subtitles? is it the same method (temporal/spatial filtering)? if u need a pic of what i'm talking about just let me know =). thanx again for the amazing job and postng your script as well.

[Chainmax]

Just wanted to chime in by saying that aWarpSharpDering is awesome. Out of all the dehaloers I tried it's the one that keeps the most detail.

[Kagura]

That's the price of using unfilter to dehalo. For real life clips, there are few hard edges, and those edges aren't the ones to contain halos, so there is no good way to mask unfilter so that it only gets halos, in contrast to anime. I used this method because it looks better to my eyes and requires less tweaking than bdh.

There is no real good way to "regain" the details on the coat, which look fake (halos) to my eyes anyways.

Noise around subtitles, if you're adding them after filtering, are caused by overcompression. Noise around subtitles present in the source should be "destroyed" by the filter chain I posted.

[neo211]

ok great. r there any particular filters i need to use that script? i already have HQDering but i'm getting an error:"no function named YV12layer". i also have the latest MaskTools as well as unfilter and awarpsharp. r the other params part of AviSynth?

i forgot to ask this on my last post but why use LanczosResize? i'm trying to understand all the steps in the script but am having trouble.sorry for being such a noob.

[Kagura]

hqdering requires the not so latest version of masktools. YV12layer is, iirc, part of 1.4.16. You can include both versions to be able to use all masktools functions.

I used lanczos because it's good at not changing the image when upsizing for supersample filtering.

[neo211]

ok so let me see if i understand this correctly, by increasing the resolution the filters can do a better job? i then go back to the original size and all should be fine?

[Kagura]

Supersampling is one of two techniques (the other is masking) that I believe are essential to good filtering. When it is needed to use a filter at strong settings but still not have it damage so much detail, supersampling is perfect.

[neo211]

cool i'm gonna see how this works out on my new project =)

[Chainmax]

neo211, I encourage you to try aWarpSharpDering. The function can be downloaded from the link Leak posted in this page. It also has a link to the needed AWarpSharp filter.
In any case, you can also make HQDering work with the latest MaskTools if you susbtitute the YV12Layer line with this (as suggested by Didée in the HQDering thread):

Overlay(amplifiedmask.Invert(), thickmask, mode="multiply").greyscale


P.S: by the way, Leak, can I safely use the version in that page for regular aWarpSharp(depth=16,cm=1) usage? What are the exact differences in this new version?

[neo211]

i tried grabbing it when u originally told me but the link was dead. i think i was PG2 so that might have been the problem. i was always forget bout that. i'll try it again and see if i can get it. i used this script since i couldnt use HQDering or the aWarpSharpDering.

Unfilter(-30,-30)
awarpsharp(10,1)
temporalsoften(2,3,3,6,2)
Convolution3D(preset="animeLQ")
FastLineDarken(strength=100, thinning=0)

i gave me some pretty good results. got rid of most if not all of the noise around the lines and sharpened up a bit. i still have ghosting of the subtitles but its not that bad. i'm gonna give your suggestions a try tomorrow morning and see what i get.

[Leak]

Originally posted by Chainmax
P.S: by the way, Leak, can I safely use the version in that page for regular aWarpSharp(depth=16,cm=1) usage? What are the exact differences in this new version?

Well, it's an improved version of aWarpSharp SoulHunter brought about in this thread (http://forum.doom9.org/showthread.php?threadid=87514), I just put it somewhere more safe than GMX and added to the docs what I could make out from looking at the filter... :)

I've got to say it's the aWarpSharp version I've used ever since, I've never had a problem with it. YMMV, of course.

@neo211: what's the reason you can't use AWarpSharpDering? :confused:

np: Apollo 440 - Hustler Groove (Dude Descending A Staircase (Disc 1))

[neo211]

i tried the aWarpSharpDering link again today with PG2 off (Peer Guardian 2) and nothing. i think the link is broken.

[Leak]

Originally posted by neo211
i tried the aWarpSharpDering link again today with PG2 off (Peer Guardian 2) and nothing. i think the link is broken.

Might be because the server it's on runs Apache on port 8888, as the link I gave ultimately redirects to http://140.78.145.103:8888/~kp/AviSynth/AWarpSharpDering/ ...

Sorry, there's not much I can do about it... :(

[neo211]

i finally got it =). apparently even though i killed PG2 the process was still running in the background. Got both files that were there. thanx