Welcome to Doom9's Forum, THE in-place to be for everyone interested in DVD conversion.

Before you start posting please read the forum rules. By posting to this forum you agree to abide by the rules.

 

Go Back   Doom9's Forum > Capturing and Editing Video > Avisynth Usage

Reply
 
Thread Tools Search this Thread Display Modes
Old 3rd November 2003, 23:20   #1  |  Link
NightMare
Registered User
 
NightMare's Avatar
 
Join Date: Jul 2003
Posts: 73
GuavaComb for rainbows or...

i'm encoding escaflowne right now and i noteced some rainbows in the source and i'm thinkin of using GuavaComb to get rid of them should i use it or is there somthin better

thanx
NightMare is offline   Reply With Quote
Old 4th November 2003, 00:04   #2  |  Link
qwerpoi
Registered User
 
qwerpoi's Avatar
 
Join Date: Oct 2002
Posts: 298
For an up-to-date discussion on removing rainbows, check out this thread here:

http://forum.doom9.org/showthread.php?threadid=62873

Somewhere on the second page of the thread you will find a script sh0dan came up with to remove rainbows, reprinted here:


Code:
org = last
org_u = utoy(org)
org_v = vtoy(org)

msharpen(threshold = 10, mask=true)
reduceby2()
greyscale()
uv = blur(1.5).levels(0,2.0,255,0,255).blur(1.5)

filtered_u = org_u.blur(1.5).blur(1.5).blur(1.5).temporalsoften(2,255,0,3,2)
filtered_v = org_v.blur(1.5).blur(1.5).blur(1.5).temporalsoften(2,255,0,3,2)

u_final = MaskedMerge(org_u, filtered_u, uv)
v_final = MaskedMerge(org_v, filtered_v, uv)

ytouv(u_final, v_final, org)
I found it to be very effective in removing rainbows, read the thread to see how it works. You could also try the suggestions on the first page of the thread if you want to experiment, there's a lot there to play around with.
qwerpoi is offline   Reply With Quote
Old 4th November 2003, 00:30   #3  |  Link
NightMare
Registered User
 
NightMare's Avatar
 
Join Date: Jul 2003
Posts: 73
GuavaComb failed misribly even on the highest setings it couldent remove the rainbow using cnr2 right now its much better

thanx for the script but i'm lookin for a dll filter but thanx anyway it really helps
NightMare is offline   Reply With Quote
Old 4th November 2003, 01:09   #4  |  Link
qwerpoi
Registered User
 
qwerpoi's Avatar
 
Join Date: Oct 2002
Posts: 298
Glad you found something that you like. As far as using dlls go, you could also try using MipSmooth (see my post on the first page of the thread I mentioned for the settings). Also, scripts are just as convinient as dll filters if you make them into a function. For example, you can convert sh0dan's script into a function as follows:

EDIT: See sh0dan's post below for a working script.

Simply save the above code into a new text file and rename it to "function.avs". Now you can load it as if it were a dll filter using the Import() command, ie,
Code:
Import("C:\function.avs")
Avisource("C:\something.avi")
DeRainbow()
If you want to use the function like an autoloading filter, name the text file to "function.avsi" instead and put it in the same directory as your autoloading plugins- now you don't need to use the Import() command. Keep in mind that sh0dan's script requires you to have msharpen as well as an updated version of Masktools in order to work.

Functions are one of the more fun things you can do with avisynth, although one might argue it requires a bit of programming background to do some of the more complicated things. If you don't know how to program, I think that learning AviSynth functions is a simple and great way to get started, since it does not have a lot of the difficulties involved in learning a language like c++ from scratch. Check out the shared functions on the AviSynth homepage to see what some other people are doing, and try some functions out for yourself in your free time, it will really help you get the most out of AviSynth.

Last edited by qwerpoi; 13th November 2003 at 06:58.
qwerpoi is offline   Reply With Quote
Old 4th November 2003, 09:58   #5  |  Link
NightMare
Registered User
 
NightMare's Avatar
 
Join Date: Jul 2003
Posts: 73
Quote:
Originally posted by qwerpoi
Glad you found something that you like. As far as using dlls go, you could also try using MipSmooth (see my post on the first page of the thread I mentioned for the settings). Also, scripts are just as convinient as dll filters if you make them into a function. For example, you can convert sh0dan's script into a function as follows:
Code:
function DeRainbow(clip org)
{
org_u = utoy(org)
org_v = vtoy(org)

msharpen(threshold = 10, mask=true)
reduceby2()
greyscale()
uv = blur(1.5).levels(0,2.0,255,0,255).blur(1.5)

filtered_u = org_u.blur(1.5).blur(1.5).blur(1.5).temporalsoften(2,255,0,3,2)
filtered_v = org_v.blur(1.5).blur(1.5).blur(1.5).temporalsoften(2,255,0,3,2)

u_final = MaskedMerge(org_u, filtered_u, uv)
v_final = MaskedMerge(org_v, filtered_v, uv)

return ytouv(u_final, v_final, org)
}
Simply save the above code into a new text file and rename it to "function.avs". Now you can load it as if it were a dll filter using the Import() command, ie,
Code:
Import("C:\function.avs")
Avisource("C:\something.avi")
DeRainbow()
If you want to use the function like an autoloading filter, name the text file to "function.avsi" instead and put it in the same directory as your autoloading plugins- now you don't need to use the Import() command. Keep in mind that sh0dan's script requires you to have msharpen as well as an updated version of Masktools in order to work.

Functions are one of the more fun things you can do with avisynth, although one might argue it requires a bit of programming background to do some of the more complicated things. If you don't know how to program, I think that learning AviSynth functions is a simple and great way to get started, since it does not have a lot of the difficulties involved in learning a language like c++ from scratch. Check out the shared functions on the AviSynth homepage to see what some other people are doing, and try some functions out for yourself in your free time, it will really help you get the most out of AviSynth.
allready did that named it antirainbow

i really dont know much about advanced scriptin i wish i could learn it an maybe some day i will write somthin like this

thanx alot u helped me alot
NightMare is offline   Reply With Quote
Old 4th November 2003, 10:28   #6  |  Link
Wilbert
Moderator
 
Join Date: Nov 2001
Location: Netherlands
Posts: 6,354
Quote:
GuavaComb failed misribly even on the highest setings it couldent remove the rainbow using cnr2 right now its much better
If that's the case you are not really experiencing rainbows (in the sense of GuavaComb). Have a look at the analog capture guide for the difference between the two. Maybe you can post a screenshot?
Wilbert is offline   Reply With Quote
Old 4th November 2003, 11:01   #7  |  Link
NightMare
Registered User
 
NightMare's Avatar
 
Join Date: Jul 2003
Posts: 73
Quote:
Originally posted by Wilbert
If that's the case you are not really experiencing rainbows (in the sense of GuavaComb). Have a look at the analog capture guide for the difference between the two. Maybe you can post a screenshot?
i used this
GuavaComb(Mode="NTSC",Recall=85,MaxVariation=100,Activation=70)
NightMare is offline   Reply With Quote
Old 4th November 2003, 11:16   #8  |  Link
Wilbert
Moderator
 
Join Date: Nov 2001
Location: Netherlands
Posts: 6,354
Could you upload a screen shot of the original clip somewhere (or the clip itself)?
Wilbert is offline   Reply With Quote
Old 4th November 2003, 11:23   #9  |  Link
NightMare
Registered User
 
NightMare's Avatar
 
Join Date: Jul 2003
Posts: 73
Quote:
Originally posted by Wilbert
Could you upload a screen shot of the original clip somewhere (or the clip itself)?
http://www.geocities.com/nightmare30002003/01.jpg
NightMare is offline   Reply With Quote
Old 4th November 2003, 12:21   #10  |  Link
mf
·
 
mf's Avatar
 
Join Date: Jan 2002
Posts: 1,729
Quote:
Originally posted by NightMare
http://www.geocities.com/nightmare30002003/01.jpg
Those certainly look like rainbows. Are you sure you're using guavacomb BEFORE any IVTC or deinterlacing? If the source is already deinterlaced, use progressive rainbow cleaners like mfRainbow.

(No, I wasn't referring to my own creation to advertise, but because it was the one I could find easily and of which I know it works .)
mf is offline   Reply With Quote
Old 4th November 2003, 12:34   #11  |  Link
Wilbert
Moderator
 
Join Date: Nov 2001
Location: Netherlands
Posts: 6,354
Btw, GuavaComb only removes Rainbows in static scenes. Although I don't know how your clip goes, I guess the person moves?
Wilbert is offline   Reply With Quote
Old 4th November 2003, 15:00   #12  |  Link
MasterYoshidino
Yoshi
 
MasterYoshidino's Avatar
 
Join Date: Jan 2003
Posts: 108
you can also try this setting for rainbows in static scenes (works fine for me, not strong enough for temporal errors from minor patterns, but works fairly well with static rainbow, line noise, and that hideous dot crawl)

GuavaComb(mode="NTSC",recall=85,maxvariation=35,activation=30)
watch out for patterns...

and did you give "smoothuv" a try? that works spatially. does not work as well as smoothiq but its way faster.

I looked at your "problem.avi" and my guavacomb setting should work pretty well to get rid of that line noise. tweak maxvariation if too much line noise comes through.
__________________
member of www.clan-ee.com

Last edited by MasterYoshidino; 4th November 2003 at 15:05.
MasterYoshidino is offline   Reply With Quote
Old 4th November 2003, 15:27   #13  |  Link
NightMare
Registered User
 
NightMare's Avatar
 
Join Date: Jul 2003
Posts: 73
Quote:
Originally posted by mf
Those certainly look like rainbows. Are you sure you're using guavacomb BEFORE any IVTC or deinterlacing? If the source is already deinterlaced, use progressive rainbow cleaners like mfRainbow.

(No, I wasn't referring to my own creation to advertise, but because it was the one I could find easily and of which I know it works .)
i used GC befor ivtc
and i used the script at the start of the forum it removed it completly
NightMare is offline   Reply With Quote
Old 4th November 2003, 15:30   #14  |  Link
NightMare
Registered User
 
NightMare's Avatar
 
Join Date: Jul 2003
Posts: 73
Quote:
Originally posted by MasterYoshidino
and did you give "smoothuv" a try? that works spatially. does not work as well as smoothiq but its way faster.

I looked at your "problem.avi" and my guavacomb setting should work pretty well to get rid of that line noise. tweak maxvariation if too much line noise comes through.
smoothuv where can i find it i will give it a try
and if u can help me with my "problem" i will be very greatful
NightMare is offline   Reply With Quote
Old 4th November 2003, 17:25   #15  |  Link
MasterYoshidino
Yoshi
 
MasterYoshidino's Avatar
 
Join Date: Jan 2003
Posts: 108
i remember there was a thread one time... where is it now...
a/w here is the dll for avisynth2.5.

http://members.cox.net/hentai/SmoothUV.rar

syntax is (radius,strength,interlaced boolean)
example SmoothUV(5,16,false)
accepts YV12 data only.

hits subtitle and credit rainbows pretty easily, some rainbows it only hits at too high a strength and ends up cross coloring / color bleeding. if that happens, stuck with slow as can be smoothiq.
__________________
member of www.clan-ee.com
MasterYoshidino is offline   Reply With Quote
Old 9th November 2003, 21:02   #16  |  Link
High Speed Dubb
Registered User
 
Join Date: Jan 2002
Posts: 283
About Shodan’s chroma smoothing script — It will work substantially better if it only smooths vertically within a field. What you really want for this purpose with NTSC video is a 1:2:1 FIR filter. (In other words, weight 1 for the line above, 2 for the current line, and 1 for the line below.) For PAL you would want (this gets icky) 1:0:2:0:1.

For Guava stuff, I wouldn’t raise MaxVariation all the way to 100 — That’ll show a lot of artifacts. For a more permissive setting, I’d reduce Recall and possibly Activation, with something like
GuavaComb(Mode="NTSC", Recall=70, Maxvariation=50, Activation=40)

But the best MaxVariation does depend on your input card. And like Wilbert mentioned, regardless of its settings, Guava will only work on completely static parts of a picture.
__________________
9:) Lindsey Dubb
High Speed Dubb is offline   Reply With Quote
Old 11th November 2003, 07:50   #17  |  Link
jarthel
plebian
 
Join Date: Nov 2001
Posts: 775
I followed what was said below. I have the following lines in my .avs file

-----
LoadPlugin("D:\downloads\windows\utilities\divx\avisynth\plugins25\MPEG2Dec3.dll")
LoadPlugin("D:\downloads\windows\utilities\divx\avisynth\plugins25\undot.dll")
LoadPlugin("D:\downloads\windows\utilities\divx\avisynth\plugins25\Decomb.dll")
LoadPlugin("D:\downloads\windows\utilities\divx\avisynth\plugins25\STMedianFilter.dll")
LoadPlugin("D:\downloads\windows\utilities\divx\avisynth\plugins25\awarpsharp.dll")
LoadPlugin("D:\downloads\windows\utilities\divx\avisynth\plugins25\masktools.dll")
LoadPlugin("D:\downloads\windows\utilities\divx\avisynth\plugins25\msharpen.dll")
Import("D:\downloads\windows\utilities\divx\avisynth\plugins25\rainbow.avsi")

mpeg2source("c:\dvdrips\escaflowne18\escaflowne18.d2v",cpu=6,iPP=true)
crop(12,0,696,480)
DeRainbow()
Telecide(Post=true,blend=true)
Decimate(cycle=5)
awarpsharp(20,2,0.5,2)
lanczosresize(640, 480)
-----------------

Every time I load the avs file, it says msharpen has invalid arguments. I am using the latest one from Donald Graft's website.

thanks for the reply.

Quote:
Originally posted by qwerpoi
Glad you found something that you like. As far as using dlls go, you could also try using MipSmooth (see my post on the first page of the thread I mentioned for the settings). Also, scripts are just as convinient as dll filters if you make them into a function. For example, you can convert sh0dan's script into a function as follows:
Code:
function DeRainbow(clip org)
{
org_u = utoy(org)
org_v = vtoy(org)

msharpen(threshold = 10, mask=true)
reduceby2()
greyscale()
uv = blur(1.5).levels(0,2.0,255,0,255).blur(1.5)

filtered_u = org_u.blur(1.5).blur(1.5).blur(1.5).temporalsoften(2,255,0,3,2)
filtered_v = org_v.blur(1.5).blur(1.5).blur(1.5).temporalsoften(2,255,0,3,2)

u_final = MaskedMerge(org_u, filtered_u, uv)
v_final = MaskedMerge(org_v, filtered_v, uv)

return ytouv(u_final, v_final, org)
}
Simply save the above code into a new text file and rename it to "function.avs". Now you can load it as if it were a dll filter using the Import() command, ie,
Code:
Import("C:\function.avs")
Avisource("C:\something.avi")
DeRainbow()
If you want to use the function like an autoloading filter, name the text file to "function.avsi" instead and put it in the same directory as your autoloading plugins- now you don't need to use the Import() command. Keep in mind that sh0dan's script requires you to have msharpen as well as an updated version of Masktools in order to work.

Functions are one of the more fun things you can do with avisynth, although one might argue it requires a bit of programming background to do some of the more complicated things. If you don't know how to program, I think that learning AviSynth functions is a simple and great way to get started, since it does not have a lot of the difficulties involved in learning a language like c++ from scratch. Check out the shared functions on the AviSynth homepage to see what some other people are doing, and try some functions out for yourself in your free time, it will really help you get the most out of AviSynth.

Last edited by jarthel; 11th November 2003 at 07:54.
jarthel is offline   Reply With Quote
Old 12th November 2003, 09:40   #18  |  Link
sh0dan
Retired AviSynth Dev ;)
 
sh0dan's Avatar
 
Join Date: Nov 2001
Location: Dark Side of the Moon
Posts: 3,480
The function above did not work. I fixed it and added a YUY2 version (much slower though), but with no chroma downsampling. I tweaked it a bit for best results on my current material. I also changed the blurs to MipSmooth.

Code:
function DeRainbow(clip org, int "thresh")
{ 
  assert(org.isYV12(),"DeRainbow() requires YV12 input!")
  thresh = default(thresh, 10)

  org_u = utoy(org)
  org_v = vtoy(org)

  msharpen(org, threshold = thresh, mask=true)
  reduceby2()
  greyscale()
  uv = blur(1.5).levels(0,2.0,255,0,255, coring=false).blur(1.5).blur(1.5).levels(50,2.0,255,0,255, coring=false)

  filtered_u = org_u.mipsmooth(spatial=255, temporal=255, scenechange=3, show=false, method="strong", scalefactor=0.5)  
  filtered_v = org_v.mipsmooth(spatial=255, temporal=255, scenechange=3, show=false, method="strong", scalefactor=0.5)

  u_final = MaskedMerge(org_u, filtered_u, uv)
  v_final = MaskedMerge(org_v, filtered_v, uv)

  return ytouv(u_final, v_final, org)
}


function DeRainbowYUY2(clip org, int "thresh")
{
  assert(org.isyuy2(),"DeRainbowYUY2() requires YUY2 input!")
  thresh = default(thresh, 10)

  org_yv12 = org.converttoyv12()
  org_u = utoy(org).converttoyv12()
  org_v = vtoy(org).converttoyv12()

  msharpen(org_yv12, threshold = thresh, mask=true)
  bilinearresize(last.width/2, last.height)
  greyscale()
  uv = blur(1.5).levels(0,2.0,255,0,255, coring=false).blur(1.5).blur(1.5).levels(50,2.0,255,0,255, coring=false)

  filtered_u = org_u.mipsmooth(spatial=255, temporal=255, scenechange=3, show=false, method="superstrong", scalefactor=0.5)
  filtered_v = org_v.mipsmooth(spatial=255, temporal=255, scenechange=3, show=false, method="superstrong", scalefactor=0.5)

  u_final = MaskedMerge(org_u, filtered_u, uv).converttoyuy2()
  v_final = MaskedMerge(org_v, filtered_v, uv).converttoyuy2()

  return ytouv(u_final, v_final, org)
}
Usage:

YV12: DeRainbow(int thresh)
YUY2: DeRainbowYUY2(int thresh)

Threshold is optional, and defaults to 10 (which is fairly strong).

Required plugins:
MipSmooth.
Masktools.
mSharpen.
__________________
Regards, sh0dan // VoxPod

Last edited by sh0dan; 12th November 2003 at 20:56.
sh0dan is offline   Reply With Quote
Old 12th November 2003, 16:56   #19  |  Link
NightMare
Registered User
 
NightMare's Avatar
 
Join Date: Jul 2003
Posts: 73
thanx for the new script will try it soon
NightMare is offline   Reply With Quote
Old 12th November 2003, 18:50   #20  |  Link
Suikun
The void that is my self
 
Join Date: Mar 2002
Location: Germany
Posts: 47
mf hates me for sometimes using this, but you could try
Code:
BlindPP(cpu2="ooxxox", quant=31)
which will do some heavy chroma filtering and usually kills about all chroma noise including rainbows, but might lead to "glowing edges".
BlindPP is part of MarcFD's MPEG2Dec3.
Oh, and don't try this with non-anime stuff...
__________________
Kono yo ni aku ga aru to sureba
sore wa hito no kokoro da.
Suikun is offline   Reply With Quote
Reply

Thread Tools Search this Thread
Search this Thread:

Advanced Search
Display Modes

Posting Rules
You may not post new threads
You may not post replies
You may not post attachments
You may not edit your posts

BB code is On
Smilies are On
[IMG] code is On
HTML code is Off

Forum Jump


All times are GMT +1. The time now is 00:46.


Powered by vBulletin® Version 3.8.11
Copyright ©2000 - 2022, vBulletin Solutions Inc.