What the hell is wrong here (Grave of the Fireflies DVD)?

[Chainmax]

I'm trying to make an SVCD out of my disc in order to be able to watch it with burned spanish subs with some friends. When trying to decide wether to IVTC or Deinterlace it, I noticed a very weird pattern when stepping through VDubMOD with only the mpeg2source line. Let's say that X = interlaced frame and Y = Progressive frame. The pattern I get is:

XYYYXYYYYYYYYYYX

Also, I think I saw faint combing in some of the progressive frames. There also appears to be some blending going on. Does anyone have any idea as to what am I facing here?

[zettai]

I havent gotten the new print yet, but it sounds like teh joys of Central Park Media encoding to me :)

Get the japanese Region 2 release :P

[scharfis_brain]

please make a sample available.

use ChopperXP or something else to cut the VOBs

[Chainmax]

Sure, I just have to find some webspace to put it in. Do you prefer 5sec or 10sec?


[edit]
One more thing: while stepping through in VDubMOD, the whole frame seems to "tremble".
BTW, the disc has two angles. I ripped the one SmartRipper chose by default.

[scharfis_brain]

5 secs should be okay

[Chainmax]

OK, I made two clips that show different issues. Clip 1 is 2.75MBytes and Clip 2 is 5.45MBytes.

[link deleted].

[link deleted].

[scharfis_brain]

hehe weird stuff:

just look at the objects patterns:
the flames has its own pattern
and the man has its own pattern
both patterns had been put together

[Chainmax]

I'm not sure if it's that the man and the flames have different patterns or if the movie itself changes patterns. There could be many more different patterns throughout the movie for all I know.
What do you suggest I do?

[resonator]

I don't know for sure, but judging by my quick examination of Clip2 I wouldn't worry too much about the changing patterns, I think it's just the animators adding some additional motion at certain parts for a smoother pace than regular animation at 12fps, or in other words, they went for full 24fps while saving some time and money on the parts where 12fps animation was enough.

I ran this script:


LoadPlugin("MPEG2Dec3dg.dll")
LoadPlugin("decomb510.dll")
Mpeg2Source("test.d2v",idct=0)
Telecide(order=1)
blur(-1.0)
blur(1.0)
Decimate(5)
ConvertToYUY2()


The blur is in there to catch stray combs without loosing too much detail. And I think the result is kinda ok. I guess you can do even better with tweaked Telecide settings and without the usage of the blurs, but I think it's sufficient for an SVCD you can watch with your friends.

[Chainmax]

So, I'm dealing with VFR here? I'm goint to have to YATTA an 88 minute movie :scared:?

[resonator]

Actually no, it isn't VFR because this was a theatrical feature, so there must be a way to use 24fps all the way trough the movie. It's just that Anime don't uses full 24fps all the time. Actually, most of the time it's animated at 12fps and they just repeat every frame. Sometimes, when the scenes have very little motion even 8fps is sufficient, then they repeat every frame twice. A VFR encoding only makes sense when there are different framerates like 24 and 30, but not when the framerates are 24, 24/2 and 24/3.

[Chainmax]

Can anyone run the clips through Telecide() and then post what the original/dupe frame pattern is (if it even exists)? I'm having a hard time detecting it.

[scharfis_brain]

telecide(order=1,post=0).decimate(5)
leaves a nearly clean 24fps-clip.

better use

smartdecimate(24,60)

[Chainmax]

Ok, I guess I'll use the following script then:

mpeg2source("C:\MPEG-4\Grave of the Fireflies\GotF.d2v")
Telecide(order=1,guide=1,post=1,hints=true)
KernelDeint(order=1,sharp=true)
SmartDecimate(24,60)

(I'd rather have some postprocessing just in case) What about the 12fps parts resonator seems to have found? Should I leave them alone? Also, in what cases is smartdecimate better than Decimate(cycle=5,mode=2,quality=3)?

[scharfis_brain]

You didn't read the smartdecimate-doc!

just use

mpeg2source("blabla.d2v"
smartdecimate(24,60)


maybe
mpeg2source("blabla.d2v")
smartdecimate(24,60,bob=bobmatcher(last))
gives better results, if the upper one returns combed images or partially bobbed sequences

do not deinterlace before applying smartdecimate

[Chainmax]

Sorry, that was just me being overenthusiastic as always :rolleyes: :o. I'll try that. Do you want me to post a short clip with the final SVCD stream once I'm done?

[scharfis_brain]

If this sample isn't larger than 2 or 3 Megs....

[Chainmax]

mpeg2source("blabla.d2v").smartdecimate(24,60) gives me some combing issues. In the clips I posted, some combing artifacts appear on the flames and every now and then the wooden beams on the houses display some jaggyness. Therefore, I'm going to include bobmatcher in the mix.
One more thing: I made an AVSI out of the script you posted in the "bobbing deinterlacer" thread and plan to autoload everything (except avisynth_c and smartdecimate). Is that ok or will it interfere with the requirement of mpeg2dec.dll having to be called before mpeg2dec3(dg).dll?

[Chainmax]

Never mind. The following script:

loadplugin("C:\MPEG-4\SmartDecimate Plugin\unfilter.dll")
loadplugin("C:\MPEG-4\SmartDecimate Plugin\kerneldeint140.dll")
loadplugin("C:\MPEG-4\SmartDecimate Plugin\sangnom.dll")
loadplugin("C:\MPEG-4\SmartDecimate Plugin\mpeg2dec.dll")
loadplugin("C:\MPEG-4\SmartDecimate Plugin\mpeg2dec3.dll")
loadplugin("C:\MPEG-4\SmartDecimate Plugin\avisynth_c.dll")
loadCplugin("C:\MPEG-4\SmartDecimate Plugin\smartdecimate.dll")
mpeg2source("C:\MPEG-4\Grave of the Fireflies\GotF.d2v")
AssumeTFF()
SmartDecimate(24,60,bob=bobmatcher(last))

gives me an access violation error. Any idea as to why? The d2v was made with DVD2AVI 1.76, if that helps.

[resonator]

Remove this line from your script:

loadplugin("C:\MPEG-4\SmartDecimate Plugin\mpeg2dec.dll")

and see if it works. I'm not sure, but I think you shouldn't load two different mpeg2dec plugins at once.

[scharfis_brain]

bobmatcher needs mpeg2dec.dll .


but try to play with setmemorymax(x)
where x is size in MB

and try to shift avisynth_c & smartdecimate.dll to the top

[Chainmax]

No dice. I put those DLLs at the top and used setmemorymax(112) as I have only 128Mbytes of ram (I had 256, long story), but I get the same message. I'm going to use plain Telecide and postprocess with KernelDeint. Thanks anyway :(.

[Chainmax]

When postprocessing with KD there is still some combing left in the flames. Also, on a scene before the clips, a plane is flying by, the propeller blades show some significant interlacing. Will this change if I use Telecide's own post processing or should I just be happy that it's not worse?

[DarkNite]

I get the best results when designating the combed frames in an ovr (framenumber +), using post=0 in telecide so that the only hints passed are the correct ones you've given in the ovr, and TMCPost as the postprocessor with the vertical filter set to false (assuming you don't want that softening effect).

BTW: That would be the UNDECIMATED framenumber.

You may get results that are just as good with KernelDeint as long as the usage is the same. See which one you like better.

[Chainmax]

But wouldn't that imply examining every frame? I really don't want to check 150000+ frames for something not that noticeable...I am going to try plain Telecide postprocessing with back=1 and Dup at default settings, maybe that will clear things up a bit more.

[DarkNite]

Hmm... back=1 might help you out a lot considering that 2 out of 3 of your combed frames coming through are p matches (from what I've seen in the samples given and the settings you posted). IIRC the bthresh is triggered by the v metric and so the the bthresh should be lower than your vthresh (probably considerably lower) if you want it to actually invoke a p match instead of postprocessing.

[Chainmax]

Well, I was checking out the plane scene and a bit of the fire scene with the following script:

mpeg2source("C:\MPEG-4\Grave of the Fireflies\GotF.d2v")
Telecide(order=1,guide=1,post=2,vthresh=25,blend=false,chroma=true,back=1,bthresh=25)
Decimate(cycle=5,mode=2,quality=3)
Dup()
TextSub("C:\MPEG-4\Grave of the Fireflies\GraveOTF.srt")

The interlacing in the propeller blades was completely gone, and the artifacting in the fire scene was slightly reduced. As per your recommendation, I'm going to lower bthresh to 15. I'm also going to add MipSmooth(preset = "movieHQ) at the end of the script, hopefully that might further hide any remaining artifacts.

[Chainmax]

I tried the new script and got some very weird errors. Some of them (like the blue screen) appear at different frames every time I load the avs in VDubMod, and when I stepped back and forth between a certain blue frame and the next and following ones, the frame alternated between blue frame, extremely interlaced and very blurry. I also get some avisynth violation errors that weren't there with the previous script. I don't know WTF is wrong with my PC. Can anyone try these two scripts

Telecide(order=1,guide=1,post=2,vthresh=25,blend=false,chroma=true,back=1,bthresh=25)
Decimate(cycle=5,mode=2,quality=3)
Dup()

and

Telecide(order=1,guide=1,post=2,vthresh=25,blend=false,chroma=true,back=1,bthresh=15)
Decimate(cycle=5,mode=2,quality=3)
Dup()
MipSmooth(preset = "movieHQ")

on the clips and tell me which one do you think looks better?


P.S: here are some screenshots from the artifacts
Blue Screen (http://members.arstechnica.com/subscriptors/x/master_burrfoot/GotFBlue.png).
Blurring (http://members.arstechnica.com/subscriptors/x/master_burrfoot/GotFBlur.PNG).
Messed up (http://members.arstechnica.com/subscriptors/x/master_burrfoot/GotFFU.PNG).

[zettai]

What the hell is decoding YV12 data on your system?!

That's not an avisynth problem, that's a decoder/overlay issue.

I'd also not recommend dup as it's terrible in anime scenes with very subtle elements like rain etc (too many false positives).

[Chainmax]

I'm not sure if it's a decoder issue, the first of the two scripts I posted didn't give me any problems. How can I detect what is decoding YV12? Anyway, could you please try the two scripts (with and without dup) on the two clips and tell me which one you think looks better?

[DarkNite]

Originally posted by Chainmax
How can I detect what is decoding YV12?

Easy Route: Open an .avs in VirtualDubMod go to File\File Information and look in the Decompressor field.

[Chainmax]

Even weirder stuff: playing back the avs in Media Player 6.4, it shows as a 540x480 stream, but VDubMOD reports the correct frame size (720x480) :confused: Can anyone confirm me if the VOBs have an 1.85:1 aspect ratio?

[Chainmax]

I'm going to make a 1CD XviD version for myself since I want to try the latest XviD version. Since I'm using MipSmooth just to hide combing artifacts, should I use it before or after the resizing?

[DarkNite]

The general rule for anything doing spatial smoothing is to do it before your final resize (yes, sometimes it happens more than once in a filterchain).

Heh... That reminds me of an old freind who used to upsize all his anime encodes by 150% before applying any smoothers. That was back in the day when detail\edge detection in the few smoothers available were haphazard at best, and "Mail it to me!" was still associated with envelopes and postage stamps.

For some reason I feel like I should give a shout out to Donald Graft and Klaus Post now. :D

[Chainmax]

Yeah, I know that, but I figure that if I resize first then the combing artifacts will get smaller, making it easier for a post-resize smoothing to eliminate them. BTW DarkNite, could you please try the two scripts (without dup) on the clips I posted and tell me which one you think looks better?

[Chainmax]

Maybe the picture got f***** up because bthresh is too low?
Anyway, when I opened the avs in Media Player 6.4, it only showed "video renderer" as a filter.

[DarkNite]

1) I repeat: Open an .avs in VirtualDubMod go to File\File Information and look in the Decompressor field.

For example, my YV12 decompressor is HelixYV12. You'll more commonly see DivX or XviD.

2) Yes, that movie has a 1.85:1 aspect ratio.

3) it looks better with MipSmooth(preset="moviehq") than without, and, other than the Dup default threshold being 2x higher than I like, your settings worked fine on those test clips.

[Chainmax]

Yeah, the decompressor was XviD. I misread "VDubMod" for "WMP" :rolleyes::o Now I am using HelixYV12 (thx for the rec, BTW).
I got too enthusiastic and hasty, so I asked lots of needless questions. Thanks a lot for your patience, DarkNite :).

[DarkNite]

No problem. :)