Welcome to Doom9's Forum, THE in-place to be for everyone interested in DVD conversion.

Before you start posting please read the forum rules. By posting to this forum you agree to abide by the rules.

 

Go Back   Doom9's Forum > Capturing and Editing Video > Avisynth Usage

Reply
 
Thread Tools Search this Thread Display Modes
Old 11th March 2008, 03:13   #1  |  Link
EroKaos
Registered User
 
Join Date: Feb 2005
Posts: 27
Blocky artifacts.

What kind of mistake/error am I making to get the artifacts (if you can call it that) that appear on the hair?

Sample: http://www.mediafire.com/?ipb1lnhnzbd

It's not the first time I see it, had it before plenty of time, which is the reason I have never completed anything I have tried to finish. But since I have plenty of time now I decided to finish unfinished stuff now.

Thanks
EroKaos is offline   Reply With Quote
Old 11th March 2008, 03:19   #2  |  Link
Ranguvar
Registered User
 
Ranguvar's Avatar
 
Join Date: Feb 2007
Location: ::1
Posts: 1,236
Please post your script.

Also, can you provide PNG screens of just it filtered? The Xvid compression may cause mosquito noise or blocks.
Ranguvar is offline   Reply With Quote
Old 11th March 2008, 03:32   #3  |  Link
EroKaos
Registered User
 
Join Date: Feb 2005
Posts: 27
I tried two different ways to do it, this one is megui:

Code:
# Set DAR in encoder to 2253 : 1250. The following line is for automatic signalling
global MeGUI_darx = 2253
global MeGUI_dary = 1250
LoadPlugin("G:\E Drive\Apps\Encoding Tools\YATTA 6.118\Plugins\UnFilter.dll")
LoadPlugin("G:\E Drive\Apps\Encoding Tools\YATTA 6.118\Plugins\SangNom.dll")
import("G:\E Drive\Apps\Encoding Tools\YATTA 6.118\Plugins\aaa.avs")
DGDecode_mpeg2source("G:\Trinity Blood\megui\VTS_02_1.d2v",cpu=4,info=3)
ColorMatrix(hints=true,interlaced=true)
tfm(order=-1).tdecimate(hybrid=1)
crop( 4, 0, -4, 0)

Lanczos4Resize(720,480) # Lanczos4 (Sharp)
mergechroma(blur(1.3)) # Little Noise
aaa()
Images from huffy file:
http://img364.imageshack.us/my.php?i...0140487jo6.png
http://img364.imageshack.us/my.php?i...0787143ia4.png
http://img355.imageshack.us/my.php?i...4158179js7.png

Images from avs:
http://img355.imageshack.us/my.php?i...0409032zw6.png
http://img364.imageshack.us/my.php?i...0983771fh4.png
http://img364.imageshack.us/my.php?i...5052430mc5.png

(They are not the same frames.)
EroKaos is offline   Reply With Quote
Old 11th March 2008, 04:18   #4  |  Link
TheRyuu
warpsharpened
 
Join Date: Feb 2007
Posts: 787
It appears as though it's xvid causing the macroblocking and since the source is probably a dvd, I bet it doesn't have that same problem.

I'd recommend to just up the bitrate or something if it's giving you that much trouble. If you're going for a set size, cap the quants at 1 to 5 for all of them, don't use AQ, don't use trellis.
TheRyuu is offline   Reply With Quote
Old 11th March 2008, 04:41   #5  |  Link
EroKaos
Registered User
 
Join Date: Feb 2005
Posts: 27
Quote:
Originally Posted by wizboy11 View Post
It appears as though it's xvid causing the macroblocking and since the source is probably a dvd, I bet it doesn't have that same problem.

I'd recommend to just up the bitrate or something if it's giving you that much trouble. If you're going for a set size, cap the quants at 1 to 5 for all of them, don't use AQ, don't use trellis.
The thing is that it also appears in the huffy files and when I preview it in megui/vdub.

Edit: Found out the reason for the blocking stuff. It's aaa() causing it. So I can just remove it now, but what can I use to replace aaa()? I need to run some sort of anti-alaising.

Last edited by EroKaos; 11th March 2008 at 05:29.
EroKaos is offline   Reply With Quote
Old 11th March 2008, 05:45   #6  |  Link
themostestultimategenius
Kwon BoA
 
themostestultimategenius's Avatar
 
Join Date: Feb 2007
Location: Singapore
Posts: 83
Add gradfun2db() after aaa()
__________________
Jeremy Clarkson - Getting fat is just evolution
themostestultimategenius is offline   Reply With Quote
Old 11th March 2008, 11:15   #7  |  Link
Ranguvar
Registered User
 
Ranguvar's Avatar
 
Join Date: Feb 2007
Location: ::1
Posts: 1,236
Also, try swapping out mergechroma... UnDot is good for light mosquito noise, LRemoveDust_YV12(17,1) is good for anime (change the 1 to 2 for heavier denoising), DeHalo_alpha may remove some around edges, and FFT3DFilter/FFT3DGPU usually do a good job...
Ranguvar is offline   Reply With Quote
Old 11th March 2008, 20:57   #8  |  Link
MadRat
Registered User
 
MadRat's Avatar
 
Join Date: Jan 2008
Posts: 110
You've got a little blocking in Able Nightroad's clothing but the girl's hair isn't blocking, that's banding. Normally you see that kind of banding as a denoising artifact but I'm not sure if mergechroma(blur(1.3)) is strong enough to do it. It might be that's just a weak spot to begin with and even a little denoising created the banding.

I agree with themostestultimategenius and Ranguvar and I'd also suggest a "better" anti-aliasing. You'll see a big slow down with these filters but you'll get better quality. Try something like this replace mergechroma and aaa with...

Code:
fft3dfilter(sigma=1.0,bt=4,plane=4,bw=32,bh=32,ow=16,oh=16) # denoise
gradfun2db() # deband 
LanczosResize(width*2,height*2).TurnLeft().SangNom().TurnRight().SangNom().LanczosResize(width,height) #anti-alias
To make the denoising on FFT3DFilter stronger or weaker adjust the sigma, it's a floating point number so you can change it from 0.1 to 10.0. You could also add some sharpening to make your picture clearer. So for example, fft3dfilter(sigma=2.0,bt=4,plane=4,sharpen=0.5,bw=32,bh=32,ow=16,oh=16) would increase the denoising and add a sharpen of 0.5 out of 0.1 to 1.0.

You can also put a floating point number between the () in gradfun2db to make it stronger or weaker, I think the default is 1.2.

Hopefully if I told you the wrong thing or over simplified too badly someone will come along, shake their finger and correct me.
MadRat is offline   Reply With Quote
Old 11th March 2008, 23:42   #9  |  Link
Ranguvar
Registered User
 
Ranguvar's Avatar
 
Join Date: Feb 2007
Location: ::1
Posts: 1,236
Just did some testing, LRemoveDust_YV12 is actually beating out the slower FFT3DFilter on most of my anime

Make sure you try them both.

And MadRat, how is your little function better for anti-aliasing than AAA? (Not challenging you, just wondering)
Ranguvar is offline   Reply With Quote
Old 11th March 2008, 23:49   #10  |  Link
TheRyuu
warpsharpened
 
Join Date: Feb 2007
Posts: 787
Quote:
Originally Posted by EroKaos View Post
The thing is that it also appears in the huffy files and when I preview it in megui/vdub.

Edit: Found out the reason for the blocking stuff. It's aaa() causing it. So I can just remove it now, but what can I use to replace aaa()? I need to run some sort of anti-alaising.
Most anime shouldn't even need anti-aliasing, especially if it's a dvd source.

Also, if you're going to use gradfun, make sure to try out gradfunkmirror because it does the same thing just doesn't destroy edges. But one again, if the source is dvd, you shouldn't need that either. Keep it simple. Gradfun would likely cause destruction of detail in dark areas (will "blend" it together sometimes, depends on the raw).

There is a better function then AAA for anti-aliasing. It's slower but it is higher quality. That being said, it still should not be needed for a dvd source.

Better AA function (IMO):
Code:
function antialiasing( clip orig, int "th_luma", int "th_croma", string "type", int "order", int "aath", bool "HQedge") 
{

# "th_luma" and "th_croma" affect directly the edge detection: higher values = more edges filtered 
# set "order = 0" for Top Field First; order = 1 --> Bottom Field First
# aath = anti-aliasing strenght (default should be fine)


th_luma  = Default(th_luma, 20)
th_croma = Default(th_croma, 20)
type     = Default(type, "sobel")
order    = Default(order, 1)
aath     = Default(aath, 48)
HQedge   = Default(HQedge, false)
ox = orig.width
oy = orig.height
dx = orig.width * 2
dy = orig.height * 2

orig.convertToYV12()
a=last
b=lanczos4Resize(dx, dy).TurnLeft().SangNom(order,aath).TurnRight().SangNom(order,aath)

# native chroma edges:
#c=b.lanczosresize(ox,oy).EdgeMask(0, th_luma, 0, th_croma, type,Y=3,U=3,V=3)

# use luma edges for chroma:
c=b.lanczosresize(ox,oy).EdgeMask(0, th_luma, 0, th_croma, type,Y=3,U=1,V=1).FitY2UV()

d= logic( b.DEdgeMask(0,255,0,255,"5 10 5 0 0 0 -5 -10 -5", divisor=4,Y=3,U=1,V=1)
 \       ,b.DEdgeMask(0,255,0,255,"5 0 -5 10 0 -10 5 0 -5", divisor=4,Y=3,U=1,V=1)
 \       ,"max").greyscale.levels(0,0.8,128,0,255,false).FitY2UV().lanczosresize(ox,oy)
b=b.lanczosresize(ox,oy)

HQedge ? MaskedMerge(a,b,d,Y=3,U=3,V=3) : MaskedMerge(a,b,c,Y=3,U=3,V=3)
}
GradFunkMirror:
Code:
Function GradFunkMirror( clip c, float "strength" )
{
#counters edge effects for gradfun2db()
#gradfun2db() requires width mod8 and height mod2
	strength	=	default( strength, 1.2 )
	w			=	c.width()
	h			=	c.height()
	vflip		=	c.FlipVertical()
	hflip		=	c.FlipHorizontal()
	
	stackhorizontal( hflip.crop( w-16, 0, 16, h ).addborders( 0, 16, 0, 16 ),
	\	stackvertical( vflip.crop( 0, h-16, w, 16 ), c, vflip.crop(0, 0, w, 16 ) ),
	\	hflip.crop( 0, 0, 16, h ).addborders( 0, 16, 0, 16 ) )
	gradfun2db( strength )
	crop( 16, 16, -16, -16 )

	Return last
}
TheRyuu is offline   Reply With Quote
Old 12th March 2008, 00:14   #11  |  Link
EroKaos
Registered User
 
Join Date: Feb 2005
Posts: 27
This is the main reason I use some antialaising:

http://img530.imageshack.us/my.php?i...6980105nj2.png

It's not just this frame, there are a bunch of other frames that have tiny jagged borders all over the video (nothing serious but annoying to see over and over.) If it weren't for this I would'nt use antialaising.
EroKaos is offline   Reply With Quote
Old 12th March 2008, 00:34   #12  |  Link
TheRyuu
warpsharpened
 
Join Date: Feb 2007
Posts: 787
Quote:
Originally Posted by EroKaos View Post
This is the main reason I use some antialaising:

http://img530.imageshack.us/my.php?i...6980105nj2.png

It's not just this frame, there are a bunch of other frames that have tiny jagged borders all over the video (nothing serious but annoying to see over and over.) If it weren't for this I would'nt use antialaising.
Bad field detection maybe with telecine (or tfm) so it's setting it to post process.
TheRyuu is offline   Reply With Quote
Old 12th March 2008, 00:41   #13  |  Link
EroKaos
Registered User
 
Join Date: Feb 2005
Posts: 27
So as long as I get the field detection correct I shouldn't get that kind of jagged border? I'm using the megui automated avisynth script creator to try it (never used it before), like the *press Analyze, wait 2-3 mins* and poof, script is done, wheter it's completely perfect or flawed, I don't know. Just using it while I remember stuff.
EroKaos is offline   Reply With Quote
Old 12th March 2008, 10:09   #14  |  Link
MadRat
Registered User
 
MadRat's Avatar
 
Join Date: Jan 2008
Posts: 110
Quote:
Originally Posted by Ranguvar View Post
And MadRat, how is your little function better for anti-aliasing than AAA? (Not challenging you, just wondering)
I spent 4 or 5 hours one morning testing every anti-aliasing function I could find and that one subjectively looked the best and was reasonably fast. There might have been anti-aliasing functions and plug-ins that I missed.

I'm really interested to try the functions wizboy11 mentioned and see how they compare to what I've been using.
MadRat is offline   Reply With Quote
Old 12th March 2008, 13:00   #15  |  Link
Ranguvar
Registered User
 
Ranguvar's Avatar
 
Join Date: Feb 2007
Location: ::1
Posts: 1,236
At least in one of my tests so far, MadRat's anti-aliasing had the best appearance (may not be best in quality, but it looked like it) over antialiasing() and AAA(). And thanks, wizboy, for intdocuing me to gradfunkmirror, works well, especially when using (foxyshadis' I believe) little AddGrain() mod.
Ranguvar is offline   Reply With Quote
Reply

Thread Tools Search this Thread
Search this Thread:

Advanced Search
Display Modes

Posting Rules
You may not post new threads
You may not post replies
You may not post attachments
You may not edit your posts

BB code is On
Smilies are On
[IMG] code is On
HTML code is Off

Forum Jump


All times are GMT +1. The time now is 20:34.


Powered by vBulletin® Version 3.8.11
Copyright ©2000 - 2024, vBulletin Solutions Inc.