choppiness with TIVTC, mouth combs with Decomb

[xxxyzzzz]

Hi,

First off, I would like to thank neuron2 and tritical for their work in creating the two excellent filters Decomb and TIVTC.

I have a situation with which (I am sure) many are familiar with. I have the DVDs for Avatar: The Last Airbender (fun show!) and I am encoding them. It is hybrid material, and I am trying to get it so the end result is not jerky and that there are no 'mouth combs'. Decomb seems to apply IVTC very well to reduce jerkiness, but leaves some combing. TIVTC does the opposite. TIVTC seems to be the smoothest on full frame motion... it just seems to be a little choppier with characters moving and other action. I can't put my finger on it though, it just seems that way.

Has anyone else had this problem with TIVTC?

The 'smoothest' I have gotten with TIVTC is the following:

tfm(mode=0,pp=7)

I know it doesn't have tdecimate. I have tried almost all the combinations of tfm and tdecimate, with all the mode,pp,hybrid,etc. of each. And this seems to works the best for me with this material. The best with Decomb is the following:

Telecide(guide=1,post=2,back=1).Decimate(mode=1)

I have even tried wierd combos like tfm.decimate and telecide.tdecimate (I wanted to be thorough). I used FDecimate as well. I admit that I haven't done a lot with thresholds, only a dozen or so tests, and couldn't tweak them to help me.

Maybe I have just been staring at the screen so long that everything looks choppy now. Any Ideas?


Thanks,
Z

[neuron2]

Post a link to an unprocessed source sample containing a problematic scene that we may use to duplicate your issue.

[DeathAngelBR]

The way I fixed the mouth combing thing (when it happens) was using blockx=8,blocky=8 in tfm(). Sometimes blockx/blocky need to be as low as 4. On my sources, that is.

[xxxyzzzz]

Here are clips from the first episode. I hope a demuxed video portion from DGMPGDec is ok. If not, let me know what specifically to provide you with.

The first one here is one instance where tfm seems to be choppier than telecide with every setting I tested.

http://www.wikiupload.com/download_page.php?id=24998

The second one here has combing with all three of the kids speaking. This occurs with Decomb, but not with TIVTC.

http://www.wikiupload.com/download_page.php?id=24999

Thank you for taking the time to diagnose my issue. Once again, it may be that I am just too picky.

Z

[Dreassica]

Oddly, a plain force film works perfectly on both samples, here yet doesn't seem to work properly with tivtc.

[tritical]

First, you don't need a field matcher. You can open either of those clips using only mpeg2source() and step through them and there is no interlacing. Second, the first clip has 30p sections where there is movement in every frame. So you definitely shouldn't be converting to 24 fps (it will make things choppy). Based on only those two clips, the only recommendation to make is to encode it without field matching or decimation.

[xxxyzzzz]

@Tritical - Thank you for your reply. I looked at the clips again and you are correct, I shouldn't try and decimate the content.

I have uploaded a third clip from the same episode to show some visible interlacing. Here it is:

http://www.wikiupload.com/download_page.php?id=25080

Also, when I run it through DGMPGDec; any clip I pull, it says it is NTSC interlaced; including the ones I uploaded earlier.

More tests with Decomb gave me some better results; using Telecide(post=4,back=1) took away most of the combing in the episode, but with a loss in fluidity.

Perhaps this is just too messy.

Any other suggestions?

[cools]

hmm... I think I know how to fix this... Check out the clip attached at the bottom, if you think its good, let me know. I think this has done the trick...

[Adub]

@cools,

It helps if you a) tell us what you did, like post your scipt, and b) upload to a external upload site.

[neuron2]

We're not going to approve large video clip attachments. Use an upload site.

[neuron2]

The clip has irregular field pulldown. When you field match it you lose smoothness because the motion is no longer distributed across fields but across frames. This is the price you have to pay if you want to make this progressive. It's a low-motion anime, so it shouldn't be a big deal.

[TheRyuu]

MPEG2Source("C:\Documents and Settings\Dragon\My Documents\Encoding\doom9help\clip3-interlaced.d2v", info=3)

assumetff()
tfm(d2v="C:\Documents and Settings\Dragon\My Documents\Encoding\doom9help\clip3-interlaced.d2v").tdecimate()
colormatrix(hints=true)

tfm+tdecimate is probably your best bet at recovering the progressive frames although as neuron2 said, you'll loose the smoothness because of the irregular field pulldown.

[xxxyzzzz]

@cools - I am interested in knowing what script you used.

@neuron2 - Thank you for your comment; I am gradually learning more and more about these many useful tools/filters.

@wizboy11 - Thanks to you as well. Is there a reason you used ColorMatrix, or is it generally just a good idea to use it?

It seems that tfm(mode=0,pp=7) by itself is the smoothest with full frame motion. Is there some reason that you chose to decimate it, wizboy11? Will decimating the content (which reduces the frame rate) make the whole of the cartoon seem more fluid compared to just leaving the frame-matched content?

Also, Is there a way to encode the fields one at a time with mpeg4 (xvid/x264) and then have the decoder match them up like with mpeg2? (I may be incorrect in the way this works.)

I apologize for the many questions.

Thanks,
Z

[Adub]

decimating will not make the anime seem for fluid.

Also, using colormatrix is just a good idea, although it does depend on the source and the desired output, however, it is better to use it more often than not.

[foxyshadis]

Something I would do for an irregular source like this: Double-rate field matching.

even=TFM(field=0,pp=7)
odd=TFM(field=1,pp=7)
Interleave(even,odd)

This probably won't work as-is. (It definitely won't work for BFF, swap field orders for that.) But with some massaging you could probably keep it from stuttering, get extra smoothness, and either get better decimation or use TTempSmooth and possibly Dup to keep frames identical that should be. (Just to save space, it won't really affect it visually.) If that doesn't work, build something out of a field-matcher and a bobber - prefer the field-matcher unless it's way out of line with the deinterlaced footage, that should totally prevent double-rate stutter.

Then just increase the B-frames in the encoder to compensate for all the extra frames.

That's because I value smoothness over simplicity in all cases, and I'll make a script as complicated as I have to. If you're happy with what you have now, ignore this, madness lies not far past.

[xxxyzzzz]

I believe I found the solution:

http://avisynth.org/mediawiki/VFR

And to think it was right under my nose. I used two scripts, one for each pass to turn it into a VFR mkv file. Almost exactly like it says on the wiki:

Pass 1:
tfm(mode=0,pp=7,mi=40,output="tfm.txt").tdecimate(mode=4,output="stats.txt")

Pass2:
tfm(movie,mode=0,pp=7).tdecimate(mode=5,hybrid=2,dupThresh=1.0,input="stats.txt",tfmin="tfm.txt",mkvout="timecodes.txt")

Then I muxed the video and audio with the timecodes and everthing was just how it needed to be. All the interlacing was taken care of with the field matching; all chopiness was removed with proper decimation in the FILM areas.

As is with a lot of the forum threads here, I was able to come to the best answer for myself with a lot of research - but only with your help to guide me. Thank you.

One more thing, though; do any of you use VFR mkv's for hybrid material? If not, why not?

Are there any other disadvantages to them that are not mentioned in the wiki?

Thanks again,
Z

[foxyshadis]

Well, I wrote a good chunk of that article, so I can vouch for its utility. :p Only other possible disadvantage is hypothetical, future standalone hardware players might hiccup on them... or might not.

I thought you had video where the pulldown was applied in different amounts in different places, or with a strange ratio. Obviously not. It's good that you found this to work well.

[martino]

Only other possible disadvantage is hypothetical, future standalone hardware players might hiccup on them... or might not.
Personally I can't see a problem with this. If VFR MP4s can be played on an XBOX360, then I see no reason why a VFR MKV would not play on the "to come" hardware player. Anyway, just an assumption...

[xxxyzzzz]

@foxyshadis - I am not sure what you mean about the pulldown.

The pulldown for this show is 3:2; but, it is in different places (some of the show is progressive in places.) I guess I don't understand how this would be bad when doing VFR.

Also, I can now vouch for the usefulness of vfr mkv. Thank you for sharing your knowledge so folks like myself can learn.

Z

[delacroixp]

I believe I found the solution:

Variable FrameRate video (http://avisynth.org/mediawiki/VFR).

Awesome... a natural extension to variable bitrate...
It certainly compliments my idea of a unique resolution and quantizer for each movie.


Well, I wrote a good chunk of that article, so I can vouch for its utility. :p
Only other possible disadvantage is hypothetical, future standalone hardware players might hiccup on them... or might not.

I just love MKV... so it's nice to see a further validation for the container...
Hardware support for MKV will be the litmus test indicating MKV's ultimate value and/or success as a multi-media container.


:):devil::D
Pascal


*** btw ***
Are there any Variable Resolution codecs out there ?
Not only could the codec choose an optimal resolution for the entire movie (greyscale, 2D/3D animation, photo-realistic) but it could also optimise the resolution for each scene-change... and collectively upscale on playback.
I suppose the problems of dealing with resolution are dwarfed by the issue of allocating 32-bit-colour addressing to each pixel... not to mention special effects !

[hkazemi]

Awesome... a natural extension to variable bitrate...
It certainly compliments my idea of a unique resolution and quantizer for each movie.

I just love MKV... so it's nice to see a further validation for the container...
Hardware support for MKV will be the litmus test indicating MKV's ultimate value and/or success as a multi-media container.

:):devil::D
Pascal

*** btw ***
Are there any Variable Resolution codecs out there ?
Not only could the codec choose an optimal resolution for the entire movie (greyscale, 2D/3D animation, photo-realistic) but it could also optimise the resolution for each scene-change... and collectively upscale on playback.
I suppose the problems of dealing with resolution are dwarfed by the issue of allocating 32-bit-colour addressing to each pixel... not to mention special effects !

I haven't seen any, nor did a quick Google search point any out, but I did see the following papers from 1987 and 1995:

Variable Resolution Codecs:
http://ieeexplore.ieee.org/Xplore/login.jsp?url=/iel6/49/25833/01146635.pdf
A Variable Resolution Video Codec for Low Bit-Rate Applications
Chiariglione, L.; Fontolan, S.; Guglielmo, M.; Tommasi, F.
Selected Areas in Communications, IEEE Journal on
Volume 5, Issue 7, Aug 1987 Page(s): 1184 - 1189
Digital Object Identifier
Summary: Demand for interpersonal video and audio communication at bit rates available on basic and subprimary ISDN accesses is growing and it is expected that service implementation will require large numbers of low-cost high-quality equipment. This paper describes the concept and simulation results of a coding scheme of the hybrid (intraframe transform and temporal DPCM) type whose main feature is the possibility of regaining high resolution for slowly changing pictures. Methods for controlling the spatio/temporal resolutions and simulation results are given. A byproduct of the algorithm allows the reduction of the transmission buffer to negligible values.

http://csdl2.computer.org/persagen/DLAbsToc.jsp?resourcePath=/dl/proceedings/&toc=comp/proceedings/vlsid/1995/6905/00/6905toc.xml&DOI=10.1109/ICVD.1995.512111
http://doi.ieeecomputersociety.org/10.1109/ICVD.1995.512111
8th International Conference on VLSI Design p. 214
VLSI implementation of variable resolution image compression
H. Sahabi, Dept. of Comput. Sci., Alberta Univ., Edmonton, Alta., Canada
A. Basu, Dept. of Comput. Sci., Alberta Univ., Edmonton, Alta., Canada
M. Fiala, Dept. of Comput. Sci., Alberta Univ., Edmonton, Alta., Canada

Citation: H. Sahabi, A. Basu, M. Fiala, "VLSI implementation of variable resolution image compression," vlsid, p. 214, 8th International Conference on VLSI Design, 1995

Variable Resolution (VR) compression of images is a new compression technique which has been developed based on the characteristics of the human eye. VR compression reduces the bandwith requirements in telepresence and teleconferencing systems. In order to perform this type of compression in real-time, a special purpose hardware is required. We describe the design of a VR Encoding/Decoding subsystem, which as designed to be interfaced, to SBus (Sun SPARC machines system Bus). The main component of this hardware unit is a VR Coder/Decoder (CODEC) VLSI chip which supports real time VR compression and decompression of up to 1024/spl times/1024 pixels at video rates. Some experimental results regarding VR compression and the quality of output images in a prototype teleconferencing system are provided.

[foxyshadis]

Multiscale wavelet codecs (like snow and dirac) work that way inherently; the more bits you provide to a frame, the higher its resolution, in general terms. (I-frames especially; this gets a little weird with delta frames, where some areas can be very sharp and others very fuzzy in a low-bit frame, depending on where there's good compensation.) They also automatically code low-detail areas better than small DCTs (no blocking), as a tradeoff for slightly less efficient high-detail (more ringing).

Better wavelets (directional, curvelets, ridgelets, etc) have much better ringing performance but are rarely seen outside academia.