Need pixel locking and fixed warpsharp

[Tofusensei]

Hi,

I am a fairly experienced anime fansub encoder, and recently having moved to Japan I have begun capturing the raws directly for the shows that I am subbing. I have been a longtime lurker in these forums, but recently I have noticed a few things that might really make some encoders' lives much easier, especially with the hurdles I am facing doing analog captures from broadcast w/ a not-so-clean signal. And now that my 5-day Brady Bill is up, I can post ;)

Anyway, the first thing I was hoping I could get addressed is the temporalcleaner port for avisynth 2.5.

The pixel locking feature from the virtualdub version of the filter was removed, and I understand the reasons behind doing this. But, although it does not help aid compression, it is by far the best tool I have seen for removing noise from analog anime captures. The ability to lock pixels across frames has a drastic effect on removing that random analog noise that a lousy cable signal has. I am politely requesting (begging ^_^) that the pixel locking feature be reimplemented into the avisynth 2.5 version for this very reason. It is the most effective method I have seen for removing that noise. Also, if anyone has any suggestions for alternative methods for doing this, I'd be glad to hear it.

The next filter I would like to see fixed is the XSharpen port for avs 2.5 that is included with warpsharp.dll. I know someone else had mentioned this, but at high levels it messes up and starts acting very oddly. I can post screenshots if you'd like. A very effective technique that can be used to clean up anime sources is to do a very high xsharpen inside of a supersampling, so it'd be nice to see this problem with the filter addressed.

Keep up the good work, guys :)

-Tofu

[sh0dan]

I made this PixelLock (http://forum.doom9.org/showthread.php?s=&threadid=34406&highlight=pixellock) filter a while ago. IMO it doesn't provide that good results, often resulting in smearing. (it's a filter for v2.0)

You should rather look at Temporalsoften - it provides much better smoothing - it can be used with small thresholds. Try:

TemporalSoften(4,3,3,mode=2, scenechange=5)
or
TemporalSoften(4,3,3,mode=1, scenechange=5)

Instead of simply locking pixels, the "almost-the-same" pixels are blended across 9 frames (in this example). With such conservative settings as above,
Finally combine it with Donald's Dup filter for anime. Be sure to use Dup before temporalsoften. Try the Dup 2.20 (http://forum.doom9.org/showthread.php?s=&threadid=44500&highlight=Dup) with improved blending!

Regarding Xsharpen - it is a straight port from ffdshow, and is delivered "as is". Anyone is very welcome to rewrite or improve the current algorithm.

I just found out about fansubs here a short a while ago - great concept!

[digitize]

Regarding xsharpen, I have had the same problems with it at high levels. It gives jagged lines and in turn bad output ~_~. @sh0dan i pm'ed you a couple of weeks ago with a picture showing the results. And I also pm'ed neuron2 about the matter, since he is the original author of the vdub filter. Unfortunatly he's out of town atm, and cannot help too much. For other's, here is a screen shot of the results xsharpen gives (supersample'd video (1920x1440), and settings of (255,255)).

http://mf.onthanet.nl/supersampled_xsharpen/xsharpencrap.png

[Tofusensei]

Okay, I will try out the TemporalSoften filter. Of course I have been using Dup, any anime encoder worth jack better be using it ;)

I'll report back here with my results w/ the temporal soften.

Oh, and a comment about Dup usage.

If I use it before I apply other filters, what happens is, there is so much random analog noise that I'd have to set the threshold to a very high level to get any effective usage. This renders it useless because what happens is, you get scenes in anime, where EVERYTHING is standing exactly still, except for a very small segment of pixels (the person's mouth flapping). On a clean anime raw source, if you use the default dup threshold setting of 3, this is even too strong. (On a side note, this is why the pixel locking feature of temporalcleaner is so darn effective on messy analog anime captures. Because of the fact that almost everything is intended to stay exactly the same you want to exactly lock the pixels across frames. If you look at some HQ raw anime encodes coming out of Japan these days, it's quite apparent they do a lot of pixel locking/Frame duplication.)

For example, the other day I was ripping some Groove Adventure Rave Region-2 DVDs I rented. With the default Dup level it would dup frames that shouldn't be duped, making some scenes look very choppy and awkward. I ended up having to lower the threshold on Dup to below 1 even, for it to not do this!

So what I am saying is, if I have to set the threshold really high to compensate for analog noise, it renders it useless, and if I have to set the threshold low to keep from the mouth-flapping issue happening... Basically it makes the filter useless.

So what I have been doing is placing the Dup filter at a reasonable threshold at the end of my filter chain, as opposed to the start, and this seems to make it quite useful.

Never underestimate the power of analog noise ^_^ I showed clips of it to digitize and JediNight and they agree that it's fugly.

Sorry for typing so much :P

[Tofusensei]

Hi, I've done a lot more testings with your suggestions, and temporalsoften is causing some really odd effects with the video, making it jump around like crazy and white flashes. I wonder if this is a bug, or if it has something to do with my filter chain? Would running two temporal filters back-to-back be a bad idea?

For example:

TemporalCleaner(7,11)
TemporalSoften(4,3,3,mode=2,scenechange=5)

That results in the encode I was speaking of. Is this possibly some glitch in the code?

Also, back to my original post, I've been doing intensive testing for prettying these television anime captures, and nothing comes close to the effectiveness of doing really high-level pixel locking with temporal cleaner. :/ I can provide example clips if you'd like to see.

Also, I was unable to try out that pixellock filter you wrote because I am really restricted to avs 2.5 at this point. I'm already encoding at under 2fps, and in the world of anime raw encoding, you're constantly working against the clock. ^^;

edit: interesting to note that the weird behavior of temporalsoften only occurs when mode=2 is selected and not mode=1. I am using a Pentium IV-Mobile 1.8Mhz processor, and as far as I can tell it should be ISSE capable.

[mf]

Originally posted by Tofusensei
I am a fairly experienced anime fansub encoder, and recently having moved to Japan I have begun capturing the raws directly for the shows that I am subbing.
Wow! A raw capturer that knows what he's doing! :D
On behalf of all people that hate jerky motion, please do only Telecide() and not Decimate()! It seems ivTC is being done by the raw capturers, so the encoders can't do anything about it, so I ask of you, please introduce HQ 29.97fps raws! I've tested it with DVD sources and it's much better with pans and computer animation. Another thing, distribute them in XviD constant quant 1 :D. And buy a HDTV set when you get the money :p.

[Tofusensei]

Haha xvid quantizer 1, eh? :D

I like your point about not IVTCing, though, it's true. Also, the new trend you see coming on a lot of japanese raws is to use 120fps (with a lot of delta frames.) I'm still not sure why they do that... :\

But yeah, I'm still new to this whole capturing thing, so working out kinks but I'm getting to a pretty good level of quality right now :)

Maybe I should open up my encoding struggles to the public, it'd be for the benefit of the community, really...

If anyone wants to try and see what filter chains you can come up with to pretty my raw captures, be my guest. I'll gladly upload chunks of my DV capture (unadulterated) to an ftp or something if you wanna try. It's quite a challenge, really. Think of it as a learning exercise :D

[mf]

Originally posted by Tofusensei
Haha xvid quantizer 1, eh? :D

Yeah! Or 2 :p. You can also do constant quality 100% and set I-frame quantizer to 1, P-frame quantizer to 2, and B-frame quantizer to 200%, and then set 4 max b-frames.

I like your point about not IVTCing, though, it's true. Also, the new trend you see coming on a lot of japanese raws is to use 120fps (with a lot of delta frames.) I'm still not sure why they do that... :\

Well, whatever the reason, if you decimate it right you can rid of quite some artifacts (if there are any) by merging all the duplicate frames together and forming one average.

But yeah, I'm still new to this whole capturing thing, so working out kinks but I'm getting to a pretty good level of quality right now :)

Good, good. Send me a raw some time, so I can test filter chains on it :).

Maybe I should open up my encoding struggles to the public, it'd be for the benefit of the community, really...

If anyone wants to try and see what filter chains you can come up with to pretty my raw captures, be my guest. I'll gladly upload chunks of my DV capture (unadulterated) to an ftp or something if you wanna try. It's quite a challenge, really. Think of it as a learning exercise :D
Challenge accepted :D.

[Tofusensei]

OK, check your personal message thingy ;)