Ripping the "Simpsons 1st Season" DVDs, need help tweaking my script.

[Chainmax]

As you might know, those DVDs are very badly mastered, with tons of noise, moving blocks, etc. Here's my script so far:

mpeg2source("C:\MPEG-4\Simpsons\Episodio1\Ep1.d2v")
ComplementParity()
Bob(0,0.5)
VagueDenoiser(threshold=1.2,method=1,nsteps=6,chroma=true)
SeparateFields()
SelectEvery(4,1,2)
Weave()
Telecide(order=1,guide=1,post=1,hints=true)
KernelDeint(order=1,sharp=true)
Decimate(cycle=5,mode=2,quality=3)
Undot()
MipSmooth(preset="animeHQ",method="superStrong",downsizer="bilinear",upsizer="lanczos",scalefactor=0.65)
SSXSharpen()

The reason lines 2-7 are like that is because I read somewhere in here that such a method is better than plain SeparateFields().Filter.Weave(). Is that true?
Also, when stepping through VDubMod I noticed that some frames still exhibit some combing and even if I set KernelDeint's threshold to 0 that combing isn't acknowledged (note: I don't zoom on the picture when stepping through the video). Should I switch to another deinterlacer?

Any recommendation on alternatives to the filter I'm using would be appreciated too.this (http://members.arstechnica.com/subscriptors/x/master_burrfoot/Simpimg.png) is a screenshot from the unfiltered source.

[DarkNite]

Ever tried YATTA? ;)

If you're too lazy for that try back=1 and tweak the bthresh, that source has a lot of p matches in the worst cases of IVTC failure using Decomb (that I noticed.)

Hope that helps you at least a little.

[Wilbert]

The reason lines 2-7 are like that is because I read somewhere in here that such a method is better than plain SeparateFields().Filter.Weave(). Is that true?
That's only true in case of temporal / spatio-temporal smoothing. VagueDenoiser is a spatial smoother, so it doesn't matter in this case.

[Chainmax]

Darknite: I'd rather not use YATTA since I'm going to rip all 13 episodes and that would take just too much time. So, your advice is to ditch KD and use full decomb+postprocessing then? I might give that a shot, thx for the advice :).

Wilbert: ok, but then again using separatefields() would mean that only a 720x240 picture would be processed at a time, so I won't be able to set the nteps setting too high. Does this method slow down the encoding too much when compared to separatefields().filter.weave()?

[Chainmax]

I get the following message when playing the AVS through WMP 6.4:

Avisynth: caught an access violation at 0x00b7f3a0, attempting to read from 0x02587300

Any ideas as to why this happens? I've used both avisynth v2.53 and v2.54 and am running on an 850Mhz Athlon T-Bird w/160Mb PC100 ram and WinXP Pro.

[Chainmax]

I recall reading somewhere on the net that in the DivX Isonews forum (or something like that) it was mentioned that the Simpsons and Futurama DVDs were telecined in a very wierd way and that you had to use a cycle 25 decimate command before any IVTCing.
I'd say that's not true, since stepping through the unfiltered source I saw a continous "3 progressive frames, 2 interlaced ones" pattern; then stepping through the source with only telecide showed that each fifth frame was a dupe of the fourth.

One thing I find strange is that telecide reported many "out of pattern"-"in pattern" switches in the beginning of the episode I tested (the christmas special) only to stabilize to "in pattern" after a couple of minutes later.

BTW, the settings I'm going to use when encoding (if they are important at all when recommending alternatives) are the following:

QPel, VHQ4, chroma motion, cartoon mode, b-frames @ 1/1.5/0.75, MPEG matrix, Trellis quant, adaptive quantization. The average video bitrate for each episode will be of around 1200kbps.

[BoNz1]

Hi, I have encoded most of these episodes too and here is what I can recommend. First of all there is no substitute for YATTA, use it :). However, one of the most ugly atifacts in these dvds are jumpy lines and I mean really jumpy lines. I really know of only one way to fix it and it isn't perfect. Basically, I just abuse Dup but using really high thresholds and a low number of max copies. This seems to get rid of some of the jumpiness without making it jerky. I am encoding a episode from season 3 right now, here is my avisynth script:
setworkingdir("c:\video\avisynth plugins\")
loadplugin("mpeg2dec3dg.dll")
loadplugin("decomb510.dll")
loadplugin("dup.dll")
loadplugin("aWarpSharp.dll")
loadplugin("deen.dll")
loadplugin("bicublinresize.dll")
mpeg2source("C:\SIMPSONSS3D4\VIDEO_TS\The Otto Show.d2v")
Telecide(order=1,guide=0,post=0,ovr="C:\SIMPSONSS3D4\VIDEO_TS\The Otto Show.txt").Decimate()
crop(8,0,700,476)
deen("a2d",2,10,12)
Dup(threshold=15,chroma=false,maxcopies=5)
BicublinResize(640,480,1/3,1/3)
aWarpSharp(depth=10,cm=1)
trim(0,31194)

This is what I have found works the best for me on these episodes. Granted it isn't perfect but it is the best I have found. It is a little slow too. As for XviD settings, I know this is the AviSynth forum but I think your encodes could be better if you used something like this:

1st pass:
Quantization Tyoe: H.263
Adaptive Quantization: Off
Qpel: Off
B-frames: 2,1.5,1
Motion Search Precision: 4 High
VHQ: 0
Chroma Motion: Off
Cartoon Mode: On
Trellis Quantization: Off

2nd pass:
Quantization Tyoe: H.263
Adaptive Quantization: Off
Qpel: Off
B-frames: 2,1.5,1
Motion Search Precision: 6
VHQ: 4
Chroma Motion: On
Cartoon Mode: On
Turbo: On
Trellis Quantization: On

I realize some of my settings might be a little controversial but I have found they work nice. You can even get a nice little speedup in the first pass if you don't turn on a lot of things and it doesn't hurt quality at all IMO. Also, I would not use any adaptive quantization on cartoon IMO it hurts quality. Also, I see no benefit to using Qpel it doesn't help quality at all (seems to actually harm it on this kind of material) and just slows down encoding. Mpeg quantization doesn't help either unless you have some sort of matrix specifically tuned for this kind of material, in which case I would like to see it :). Also, using b-frames 2,1.5,1 is better, contrary to popular opinion you can use quite a few b-frames on cartoons without it looking bad and even at relatively high quants.

[Chainmax]

BoNz1: thanks for the advice, I guess I'm going to have to use YATTA. I don't know anything at all about it, but I guess I can always read its documentation :). Your XviD settings are pretty interesting, I'll give them some though. Just one word of advice: I don't think it's a good idea to change the Motion Search Precision value in between passes. ABout your avisynth settings, I have some doubts:

1) Shouldn't you crop before IVTC?
2) Is dup necessary after decimate()? Don't they do the same thing?

[BoNz1]

Ok, about the motion search precision. I can understand if you think that might harm some things. If you don't think it is a good idea you can just use 6 of course, the speedup isn't really that huge from using 4 instead of 6. But I will say that I have tested side by side an encode with these fast settings and without and I could see no visible problems. About your other questions, I believe you are supposed to IVTC before doing anything including cropping, I may be wrong but this is the way I have done it always. About dup(), basically what dup does is it has frame differencing algorithms which enable to detect whether there is much or any difference in between two frames. If the difference is below a certain threshold a copy of the previous frame will replace that frame. Decimate() will remove duplicates but obviously there are still going to be frames that are duplicates or are very close to being duplicates and might as well be the same. Put it this way, if you IVTC from 29.976fps to 23.976fps there will still be duplicates because the video was not made at 23.976fps probably it was done at a much lower framerate. So to summarize decimate() removes duplicates and dup() will make more where it doesn't matter, making the material much easier to encode. You will notice you bitrate drop significantly when using dup especially if you use cartoon mode :).
EDIT: Just one more thing, when you are using dup(), you might want to tweak your settings from the defaults and not the ones I have posted since mine are quite agressive.

[DarkNite]

1) Shouldn't you crop before IVTC?

I have 2 comments on this matter. First of all, N. Secondly, O.

If done properly, you "could" crop before IVTC, but nobody is going to say you "should" crop before IVTC.

[Mug Funky]

@ BoNz1:

have you tried the "fast 1st pass" method of the latest Xvid? basically it does the same kind of thing, but will give you a TREMENDOUS speed boost without sacrificing apparent quality (i haven't tested this). i think all it does is throw the 2nd pass size off by ~ 10kb which we can all live with.

personally i use 1 pass CQ for everything unless i'm really serious about fitting onto specific volumes.

@ all:

if you think the R1 simsons and futurama are bad... the R4 PAL versions are the same thing run through a converter box.

so there's incessant blending to deal with. i can't even be bothered backing up my futurama DVDs. too much pain. i'll just wait for dual layer DVD burners.

[Chainmax]

Mug Funky: I asked several times what "fast first pass" is all about and got no answer. Could you tell me what it is and how to activate it?
My Simpsons DVDs are region 4 NTSC discs. I guess I got lucky :).

[BoNz1]

I think fast first pass is set by a flag in the plugin_2pass1.c but I don't think it can be turned on in the vfw. I'm not sure if it is on by default now. To be honest I haven't been staying very current with my builds now that Koepi is releasing builds again. I check though tomorrow if I have time. I think FAST1PASS is reasonably similar to the settings I posted though anyway.
EDIT: I just asked sysKin on IRC and he said FAST1PASS is on by default (now not in the latest beta I don't think but if you check out the cvs and build it or get one of gamr's builds once he fixes his build system you can use fast1pass) <- actually this stuff in brackets I think is wrong, AFAIK the latest beta 1st pass is faster so I think it is enabled in beta3.

[Leak]

Originally posted by BoNz1
EDIT: I just asked sysKin on IRC and he said FAST1PASS is on by default

As I asked sysKin how to disable fast first pass - you can disable it by using a zone to encode the first pass at fixed quantizer 2; that gets rid of all the optimizations. I prefer having a first pass file with optimal quality that I can just use if the file comes out undersized.

np: Fluke - Hang Tough (Puppy)

[Mug Funky]

@leak:

i thought xvid has been tweaked such that if the first pass comes out undersized it'll increase the quants to 1 for some frames. not sure.

i don't use 2pass all that much as most of my final output is either DV or MPEG-2. i mainly use xvid for super-low-bitrate encodes.

[Leak]

Originally posted by Mug Funky
i don't use 2pass all that much as most of my final output is either DV or MPEG-2. i mainly use xvid for super-low-bitrate encodes.

Errr... no. IIRC, one of the developers said you could set the min quantizer to 1 in the 2nd pass if you really want to have bigger frames. But nothing will do that automatically.

np: A Grape Dope - Dr. Moonorgun Please (The No Music Of Aiff's)

[Chainmax]

Here's my (almost) final script:

mpeg2source("C:\MPEG-4\Simp\Episodio1\Ep1.d2v")
Telecide(order=0,guide=1,post=2,vthresh=30,blend=false,chroma=true,back=0,ovr="C:\MPEG-4\Simp\Episodio1\EspNav.txt")
Decimate(cycle=5,mode=2,quality=3)
crop(10,4,704,472,align=true)
Undot()
MipSmooth(preset="animeHQ",method="superStrong",downsizer="bilinear",upsizer="lanczos",scalefactor=0.65)
Dup()
LanczosResize(640,480)
SSXSharpen()

I have one last question. There are two big issues I have with these discs. From time to time, the colors get darker (brightness/contrast issue? No idea). Also, large flat colored areas often show massive blocking, especially stuff like the skies and curtains. I expect mipsmooth to correct the latter but I'm not 100% sure those blocks are macroblocks.
Anyway, I was thinking maybe lumafilter could help. What do you guys think? If I use it, which part of the script should it be put in?

[Chainmax]

bUmP.

[BoNz1]

Is the blocking in the source or in the final encode? If it is the later you might want to post your XviD settings I might be able to help you fine tune them. That said I don't think lumafilter is going to help you at all here. I don't know how mipsmooth works since I have never used it, usually I use deen. The only thing that will really help if you have bad blocking in your source is to postprocess it with mpeg2dec filtering it probably will not help but just post process the parts where you feel are blocking badly. Lumafilter IMO will not help in this respect usually it makes movies a lot more compressible and makes those flat areas even flatter. I did not notice any brightness or contrast change when I was encoding these DVDs but it was quite some time since I encoded season 1.
EDIT: the blocks could also be because the choice of the pattern is not good I have seen this from time to time but I know you have gone through it with Yatta so probably this isn't the case.

[Chainmax]

The blocking is in the source, and the blocks are pretty much present throughout the whole movie. It's so bad it can even be noticed when being watched through a standard TV.