Welcome to Doom9's Forum, THE in-place to be for everyone interested in DVD conversion.

Before you start posting please read the forum rules. By posting to this forum you agree to abide by the rules.

 

Go Back   Doom9's Forum > Capturing and Editing Video > Avisynth Usage

Reply
 
Thread Tools Search this Thread Display Modes
Old 18th September 2002, 23:12   #21  |  Link
High Speed Dubb
Registered User
 
Join Date: Jan 2002
Posts: 283
Could anyone post a sample of PAL video with rainbows or dot crawl?

Pretty please?
__________________
9:) Lindsey Dubb
High Speed Dubb is offline   Reply With Quote
Old 19th September 2002, 11:34   #22  |  Link
Kurosu
Registered User
 
Join Date: Sep 2002
Location: France
Posts: 432
@Defiler
I was already using your clip, which is usefull. However, the main problem is now that high motion provokes ghosting artifacts, due to the heavy temporal smoothing (averaging in fact)necessary to kill rainbow effects. I still haven't tested the 60fps version, but I don't think that will be much different from the previous clips.

@Taranli Maren
Thanks, I'm dl'ing it. I think it will be a good test for the ghosting artifacts.

@High Speed Dubb
Never Twice the Same Color
However I have rarely see PAL material with this problem, unless it was a NTSC master awfully transfered to PAL (some DVD editors, more particularly regarding anime, let the DVD player hardware do their job...)
Anyway, I haven't such material.
Btw, if you have links or docs (OutPinged_ had some, but in Russian) dealing with the problem of aliasing between Y "carrier" and U/V ones in analog signal, it could be of great help to design a better filter. The one is a bit dumb

(hope that time my post will be recorded)
Kurosu is offline   Reply With Quote
Old 19th September 2002, 13:23   #23  |  Link
Kurosu
Registered User
 
Join Date: Sep 2002
Location: France
Posts: 432
@Defiler
No particular enhancement using the 60fps clip.

@Taranli Maren
Your clip has no rainbow artifacts. However it was useful, because one can see that the filter tends to filter areas which don't show such rainbows.

@All
Using those 2 videos (from Taranli Maren and Defiler), setting the filter to... filter perfectly Defiler's clip provokes ghosting artifacts in Taranli Maren's clip. But settings avoiding ghosting artifacts in his clip does a worse job on Defiler's clip. So this filter is far from perfect.

The solutions I may use when processing the video, macroblock-wise:
- only filter pixels with high luma (Y), because chroma variations there are more sensitive. This will prevents areas with edges, but sufficiently dark to not show rainbows, to stay unfiltered.
- looking at the U/V planes, maybe I should count the number of pixels where variations are higher than a chroma threshold. Because it's more their number than the intensity that reveals the rainbow effects. Moreover, it may be possible to only use that criterium (not the edge one) on those kind of videos.

Considering other videos (not macroblock-wise), I still wonder how to do an easy job on them...

I'm opened to any idea
Kurosu is offline   Reply With Quote
Old 19th September 2002, 13:34   #24  |  Link
kyousuke
Anime Team
 
Join Date: Jul 2002
Posts: 86
wow

i tested it, it's very impressive with a good setting.

just one thing, be careful about the color degradation with the 4th int parameter.
i tested the four parameters, the 0,1 and 2 gave good result on my video (band of brothers) but the 3 changed the colors too much if i remember well. for the 4th, i forgot ^^;

i'm sure that your next version will be better, more and more.

congratulation
see ya!
kyousuke is offline   Reply With Quote
Old 19th September 2002, 15:40   #25  |  Link
Kurosu
Registered User
 
Join Date: Sep 2002
Location: France
Posts: 432
I'm maybe posting too much and should test more before posting, but I come with a total new setup. More explainations in the previous posts, and of course the avisynth.txt, cleaned this time.
Beware, it's macroblock based, so myabe it has now a too strong effect for your needs.

Basically, it still doesn't test motion, but should be more accurate. However, it will be very picky with the 2 first values (see text file for setting it up). So, visually, a maybe less effective result, even with more testing, but a lot less possible degradations (or so I hope).

I'm still searching a good metric to test if the macroblock has rainbow, but the one used seems ok...

Good luck with testing
Kurosu is offline   Reply With Quote
Old 20th September 2002, 04:29   #26  |  Link
Guest
Guest
 
Join Date: Jan 2002
Posts: 21,901
Bump due to delayed approval of attachment.
Guest is offline   Reply With Quote
Old 21st September 2002, 02:05   #27  |  Link
High Speed Dubb
Registered User
 
Join Date: Jan 2002
Posts: 283
@Kurosu,

Take a look at the online chapter from The Engineer's Guide to Decoding & Encoding
http://www.snellwilcox.com/reference/pdfs/edecod.pdf

[um... except that it seems to be down right now -- would you like me to email it to you?]

That’s a good description of each of the color encoding formats, as well as ways to separate them from the luma signal.

There’s an especially interesting looking algorithm for PAL color separation (called a “diagonal comb filter”) near the end of the document — But I don’t trust myself to program it right without some actual test material to work on.

So based on theory, PAL should have color crosstalk problems, too. But apparently they’re less visible. One interesting thing I’ve noticed from the boards is that people in the US tend to talk about problems with rainbows, while people in Europe tend to notice problems with chroma noise. I suspect this is due to the color encoding differences.

On a related topic — Here’s one way to avoid artifacts with this. Instead of using just information for the past few frames, keep a field-sized array which tracks how much each pixel has shown evidence for crosstalk. Slowly decay the old values for evidence, and add in the new ones. When the cumulative amount exceeds some threshold, you can decide to smooth the chroma (and luma, for that matter). This isn’t an easy approach, and it will miss any crosstalk that is only in a couple of frames, but it works well for any crosstalk in stationary areas.
__________________
9:) Lindsey Dubb
High Speed Dubb is offline   Reply With Quote
Old 1st October 2002, 02:58   #28  |  Link
Kurosu
Registered User
 
Join Date: Sep 2002
Location: France
Posts: 432
@High Speed Dubb

Thanks for the very interesting link, and sorry for the late reply. As your idea even more underlines, I fear that there isn't a good method: I'm considering 16x16 blocks (because videos I had were only affected on entities of that size), you're dealing with PAL signal properties. I'm not familiar with those considerations, but I'll look more closely: maybe a good idea will pop up

Concerning your idea:
- to detect possible crosstalk, I count how many pixels are likely to be crosstalks. Looking at the blocks in UV planes clearly show high activity. I simply count the number of pixels which derivative (calculed by Robert's filters) is higher than a certain threshold, then consider a threshold number of pixels for which the block is considered as to be temporally smoothed (the more sample the better, if no ghosting artifacts )
- your idea would likely give a more precise result by only affecting some pixels, not the whole 16x16 block. However, the only material I have is from MPEG source. Maybe people from the capture and TV encoding forums could provide us with more general "rainbow artifact" (not shadow ^^. As a result, I dunno. All the matter involved is "when can we consider that a pixel is a crosstalk". My MPEG source show a block basis:
Original image
Y Plane
U Plane
- I'm not convinced by the the spatial smooth. I put some sort of spatial smoothers (blur, then filling with average on parts of the 16x16 block) but their result are awful. Just imagine a logo or something of the like with a lot of chroma activity (high frequencies I mean), detected as crosstalk. Smooth it. Yuk...

My idea is to expect that the crosstalk is due to a random signal added to the UV planes. Then, by temporal smoothing it, you will reduce it. The next idea is to use local information, like Y values, to do a sort of deconvolution to remove the aliasing due to Y signal. Fact is that I dunno if people would be interested in a very good filter but slow as hell. Anyway the doc you pointed out will certainly be really helpfull. I tried to find such a doc. Moreover, i've bookmarked the whole site for "theory basis"

Btw, I had no answer like effective/uneffective, bugged, hard to setup and the like on my lattest build... I think I'll do for my own sake some additionnal developments, but is there anyone that would like improvements? Or should I crosspost to the capture forum ?
Kurosu is offline   Reply With Quote
Old 5th October 2002, 13:28   #29  |  Link
ronnylov
Registered User
 
Join Date: Feb 2002
Location: Borås, Sweden
Posts: 492
I tried the macro block based antiblink on a captured PAL movie. It works very well except in the scene changes. I get blocks with different colours that not look very nice the 2 frames before and 2 frames after each scene change.

I attach an image of how it looks on a frame before a scene change.
__________________
Ronny
ronnylov is offline   Reply With Quote
Old 5th October 2002, 14:44   #30  |  Link
Kurosu
Registered User
 
Join Date: Sep 2002
Location: France
Posts: 432
Good! Feedback, I love feedback

I prepared something to verify scene changes (very basic at start) and was waiting for such a report. Drawback: even slower (unless you feel speed is ok )

I'll give news whenever the results are ok with my sample clips.
Kurosu is offline   Reply With Quote
Old 5th October 2002, 15:59   #31  |  Link
ronnylov
Registered User
 
Join Date: Feb 2002
Location: Borås, Sweden
Posts: 492
Well, I found out that it worked better if I used noise filtering after antiblink instead of before. I also changed to Antiblink(3,32,2,4) for dirty sources. It seems that my analogue TV capture needed this setting. Now the blocks is not as visible but there is still some kind of smearing at scene changes. I attach an image right after the scene change above. The color of the blue wall and the faces are left on the first frame after the scene change and after 2-3 frames the picture is normal again.

My capture resolution was 528x576 and my destination is 704x576 DVD (I did not have enough hard disc space to capture full resolution this time).

I used this script:
--------------------------------------------
SetMemoryMax(40)
#
LoadPlugin("E:\Download\Videoverktyg\Avisynth Plugins\Convolution3D\Convolution3D.dll")
LoadPlugin("E:\Download\Videoverktyg\Avisynth Plugins\Antiblink\AntiBlink.dll")
#
source=SegmentedAviSource("G:\AVI_IO Capture\capture.avi").trim(0,201588)
#
# Macroblockoptizing of film (plus top letterboxing)
Film=Crop(source,17,131,492,312)
Film=LanczosResize(Film,672,320)
Film=Antiblink(Film,3,32,2,4)
Film=Convolution3D(Film,0,6,16,6,16,2.7,0)
Film=AddBorders(Film,16,128,16,0)
#
#
# Macroblockoptimizing of subtitles (plus bottom letterbox)
Text=Crop(source,54,445,420,80)
Text=LanczosResize(Text,560,80)
Text=Antiblink(Text,3,32,2,4)
Text=Temporalsoften(Text,3,32,128)
Text=AddBorders(Text,80,0,64,48)
#
# Display the subtitle below the film
Return(StackVertical(Film,Text))
---------------------------------------------
__________________
Ronny
ronnylov is offline   Reply With Quote
Old 5th October 2002, 16:15   #32  |  Link
Kurosu
Registered User
 
Join Date: Sep 2002
Location: France
Posts: 432
Quote:
Well, I found out that it worked better if I used noise filtering after antiblink instead of before.
You're totally right. the video should be unfiltered/untouched when fed to AntiBlink. In order to speed the whole think and have better results, I don't test edges anymore: I just look at the activity of the 16x16 block (as described in the .txt in the archive) in the U plane. Using a spatial (and certainly a temporal filter) such at SpatialSmoothMMX, Convolution3D and such will reduce the effectiveness of the detection, unless you set them for only affecting luma (Y).

For the ghosting artifact, that's quite simple: I don't test scene change when temporal smoothing. when using Antiblink(3,32,2,4), you check if half of the pixels (32) from the 16x16 blocks have an activity higher than 3, then you ask to use the 2 previous pixels (in time axis) and the 2 next to do the temporal smoothing (last parameter has been disabled in this release for it is bugged). Therefore, a scene change occuring in one of those frames will affect the pixels used for tempsmooth.

I'll be using luminance for scene change detection: using chroma is ineffective because of the rainbow effect.

btw, I set up the filter for not filtering the x first and x last frames of the video (x being your tempsmooth strength).
Kurosu is offline   Reply With Quote
Old 6th October 2002, 16:27   #33  |  Link
Defiler
Asker of Questions
 
Join Date: Oct 2001
Location: Florida
Posts: 433
It seems to me that these rainbow edges have a couple of distinct characteristics:
1. They occur in areas of high luminance. If you've got a black rainbow, you can't see it.
2. They have high frequencies in chroma-space.
3. They change drastically from frame to frame.

What would happen if you did a Fourier transform on all the blocks in chroma-space to find all the blocks with lots of color frequency data, masked out all the areas that fell below a certain luminance threshold, and then compared that to adjacent frames?
Sounds a bit like the "correlation surface" that the really advanced motion estimators use.
__________________
"The real trick to optimizing color space conversions is of course to not do them." --trbarry, April 2002
Defiler is offline   Reply With Quote
Old 6th October 2002, 19:26   #34  |  Link
Kurosu
Registered User
 
Join Date: Sep 2002
Location: France
Posts: 432
Hi Defiler,

1. More visible indeed. In the current filter, I skip pixel which luminance is beneath 31
2. True: that's why I try to find by detecting how much pixels are likely to be from a rainbow artifact. I call that activity, and it's even more visible in U/V planes
3. I agree, therefore the "AntiBlink"

Your ideas are great. If you remember earlier antiblink.txt coming within the archive, you know I was thinking of dct or whatever mean to access frequency components. Three ways then:
- using normal pixels value and track the pixels with too much variations in chroma planes (for instance, by looking at previous and next frames). That is High Speed Dubb. it could however be highly sensitive to motion (scene change detection)
- do a DCT/FFT on the 16x16 block and see if frequency values are more or less regularly distributed (something close to physic white noise - maybe testing high frequencies would be sufficient as you said), then filter most away. This would do spatial smoothing. However, the error is most disturbing because it "blinks". A temporal smoother seems better there
- another way still not explored, maybe lying in the document High Speed Dubb linked. Maybe a more computational exepnsive processing is needed, but it would yield quite better result, as we would know more precisely what to remove.

The FFT/DCT solution is a good one, could benefit from using existing optimized code (XviD for instance), but I don't consider myself as a seasoned programmer: the code isn't optimized at all, and won't certainly be.

Unless forgiving souls would point me some online usefull resources and bear with my annoying questions

Notice that this filter is more geared towards particular sources. I've now seen such result due to typical aliasing (grid lines) on pal video (therefore, High Speed Dubb, tell me if you want some - I don't use them but I'll capture some). But the files I want to work with for now (unless someone really want me to do test on a particular file) are of the previous type.

For now I'm working on ghosting artifact and rewrite quite a bit the code. But more bugfix to do, then :/
Kurosu is offline   Reply With Quote
Old 6th October 2002, 22:48   #35  |  Link
High Speed Dubb
Registered User
 
Join Date: Jan 2002
Posts: 283
Kurosu,

Yeah, it does look like color crosstalk in compressed material is different than crosstalk in broadcast video. With NTSC, regions of crosstalk can be pretty much any shape (though they tend to be tall and thin, following vertical lines in the picture). The changes caused by compression probably mean that any spatial comb filters will fail. (And I haven’t had any luck getting one working for broadcast material, either.)

Yes, the method I described is very susceptible to motion, but that’s easy to fix — just check that the pixel value at field t=-4 is sufficiently similar to the pixel value at field t=0. (That’s for NTSC — I think it should be t=-8 and t=0 for PAL.) Combine that with reasonable values for the “is it shimmering” threshold and motion artifacts are avoided. (But it does cause problems with strobe effects — That’s probably unavoidable since the algorithm is pretty much designed to detect strobing.)

I agree — moving to frequency space is worth a try. It could even be pretty cheap to do for broadcast material, since you’re just looking for a single specific frequency (the color carrier frequency). But from the pictures, I’m not sure whether compressed material will still have the same signal.

And yes, it would be great if you’d capture some PAL color artifacts. I’m pretty sure there are still some problems with what I’ve written for PAL.
__________________
9:) Lindsey Dubb
High Speed Dubb is offline   Reply With Quote
Old 9th October 2002, 04:06   #36  |  Link
Kurosu
Registered User
 
Join Date: Sep 2002
Location: France
Posts: 432
@All
I'd like some (can be short, like 25 frames, of any sources) videos sequences with both "rainbow" effect and motion (like ronnylov's) to check how the new code does its job. I urge everyone interested in the matter to provide High Speed Dubb appropriate sequences, as his work is directed in another direction than me. Both works concern everyone having trouble with "rainbow" artifacts.

@High Speed Dubb
I'd like to go slowly from the simple observations-ideas I had to more general ones. Indeed I've worked on material that were particuliarly fitting what I attempt to filter. But I don't know if it should be pixel-wise or block-wise. Both "activity" basis and High Speed Dubb's ideas are good ideas of what's happening. I guess that his idea is more general and more suited to artifact coming from video captures.
About the fix: I guess you're speaking in terms of luminance, calculating something like SAD for motion estimation.

Regarding the frequency analysis, this is going to be my next step. When analysing the causes of the rainbow artifacts (aliasing of Y signal and U or V), it seems some better results are possible there. And it would be a finer measure for my "activity thresholding", the one I'm using right now.

(I'm really doing long posts )
Kurosu is offline   Reply With Quote
Old 9th October 2002, 10:14   #37  |  Link
ronnylov
Registered User
 
Join Date: Feb 2002
Location: Borås, Sweden
Posts: 492
Hi Kurosu!

I would love to send you a short clip of the TV-captured movie but unfortunately I have already deleted my original capture because I needed the disc space for other capture. However I found that the final result after compression (MPEG2 on DVD-R) was very clean from the rainbows! And the scene change artifacts was hardly noticable in full speed playback. It compressed very well also.

I have a channel that produces rainbows at capture so I can try to capture something representative. How (where) should I upload the sample clips?

I capture with AIW Radeon and AVI_IO. In AVI_IO I can only tune the channels to fixed channel numbers but in ATI MMC capture software I can set the TV channel frequency more exact and thus get ride of some of the rainbow artifacts. I can try I short capture with both softwares on the same channel as a comparison. I don't capture longer clips with ATI MMC because of the file size limitation on FAT32.
__________________
Ronny
ronnylov is offline   Reply With Quote
Old 9th October 2002, 15:36   #38  |  Link
Kurosu
Registered User
 
Join Date: Sep 2002
Location: France
Posts: 432
@ronnylov
I'm very pleased you already find my filter usable in spite of the error you reported! Sometimes, for captures, I use _YUY2Clean() that gives pretty good results at removing little noise, but produce some effects of the kind. Btw, was your video subbed?

For uploading the video, I'm still wondering: I'll give you in private the details, and why.

In my case, I use an ATI Radeon Vivo DDR (7200 now, I think), and I let the taperecoder do the tuner work
I use VirtualDub as it gives me good performance, and I can tweak the program. I indeed don't capture anymore in MMC because of that limitation ( I really like the possibility in VDub to give different "spill drives").
Kurosu is offline   Reply With Quote
Old 9th October 2002, 17:17   #39  |  Link
ronnylov
Registered User
 
Join Date: Feb 2002
Location: Borås, Sweden
Posts: 492
Hi Kurosu!

Please post a link to the _YUY2Clean() filter.
I could not find it by a quick search in this forum
or searching internet.

I think "the error" I reported was that I used "wrong" parameters and that I did not filter after antiblink(). This filter needs another filter afterwards to remove the blocky edges. I can see the ghosting sometimes at scene changes but it is less annoying than the chroma noise I had on saturated colours and sharp edges before.

The rmPal filter in Virtualdub helps but it is not as efficient as your antiblink filter. RmPal does not remove the raimbows.
__________________
Ronny
ronnylov is offline   Reply With Quote
Old 9th October 2002, 17:46   #40  |  Link
Kurosu
Registered User
 
Join Date: Sep 2002
Location: France
Posts: 432
@ronnylov

for the thread (didn't test if it was still up). Additionally, it requires an Athlon or PIII comp (int SSE+MMX)
http://forum.doom9.org/showthread.php?s=&threadid=31092
I had also mistaken its correct name

The error I was talking of is the ghosting, neither a wrong setup of the AntiBlink nor the order of filters. I have no appropriate video sequences to test it, and additionnaly, my pm to you was lost... I'll resend.

Just to keep you warm and help you wait, the filter syntax uses a difference between blocks to choose not to use a block in the filtering (depending on a threshold you'll have to use), and another threshold for the level of luminance above which to filter (dark pixels won't be filtered).
Kurosu is offline   Reply With Quote
Reply

Thread Tools Search this Thread
Search this Thread:

Advanced Search
Display Modes

Posting Rules
You may not post new threads
You may not post replies
You may not post attachments
You may not edit your posts

BB code is On
Smilies are On
[IMG] code is On
HTML code is Off

Forum Jump


All times are GMT +1. The time now is 00:25.


Powered by vBulletin® Version 3.8.11
Copyright ©2000 - 2024, vBulletin Solutions Inc.