TFF and RFF-like support in avi/mpeg4

[nuked]

OK, this is probably a crazy idea from a guy who doesn't know enough. The biggest headache in mpeg4 backups has been and still is how to deal with mixed interlaced/telecined and badly telecined material. As far as I can tell the mpeg4/avi combination does not explitly support fields (of course there are always odd and even lines that CAN be interpreted as fields) and doesn't have support.. or at least not implemented for field ordering and repeating flags.. TFF and RFF found in mpeg2.

so the only way to encode bad hybrids is to lose smoothness on the interlaced parts or record a bunch of repeated fields in teh telelecined parts that add files size needlessly. Sure codecs SHOULD be able to recompress the duplicates at nearly no cost, but that functionaly as far as I can tell doesn't really work in divx 5 and xvid is not at a user friendly state yet and is still hard to get it to behave well.

SO why not just do the simple thing and keep the original mpeg2 frames and copy the tff and rff flags into the avi for interpretation on playback... I know I know but avi/mpeg4 (I don't know which) doesn't support this or even fields for that matter. well.. I say humbug... If you're telling me you can't encode a few bits of extra information in a format desinged to take megabits per second of data then I say you're not very creative. No one would have created virtual private networks with that attitude. Why not encode the info directly into the video stream if needed, like in an extra macroblock somewhere and have a decode filter read it back, store the info and remove the extra block. I don't know enough but something like this might even be doable inbetween the compression and decompression codecs to make it codec independant, so long as you remove whatever data you put in and so long as it fits in the avi format. Of course you'd probably need a 2 staged decoder for that aproach, like one to strip the data pass the rest to the divx decoder and another one above that to do what's needed with the fields using the data stripped from the first filter, but now I'm just making stuff up... I'm sure folks who know more can be creative though.

It seems like all these headaches for hybrid sources are just silly and unecesary. Why not just leave these sources the way they were. Only thing to worry about is audio syncing.

[nuked]

by the way... I'm thinking a nice bonus of this would be significant speed up of anime encodes. I think there already exist filters that ecan look for anime frames that are equal to within some noise level and replace them with identical ones which will compress better, but the the codec still has to re-anilyze the identical frames just to figure out it doesn't need any bits after the filter already figured that out. With this type of support an avisynth filter could replace the repeated fames with a single UNREPEATED frame and adjust the repeat tags acordingly. An expansion of the repeat tag formalism would be needed for this to allow mulitple repeats and repeats of top and bottom at the same time, but this would be a trivial addition and might reduce encode times non-trivialy for anime.

[nuked]

just figured out the easy way to get around the lack of enthusiasm. I will just construct a text file with the flags. Making filters should be easy. Next time I have for programming anything is the holidays. Maybe in January I'll have something worked out.. we'll see.

[ThePanda]

This sounds like a very good idea to me. Having to IVTC or deinterlace makes encoding anime impossible to do perfectly, but maybe your idea could make it play back just like the dvd (although my dvd player even has some trouble playing anime correctly).

[nuked]

yeah... some animes are really messed up. With my scheme some filtering could be aplied to figure out the right filed patterns tough if you don't trust the ones coming off the dvd, and they aren't always what they seem because you can have material that was edited and speed changed several times and has all kinds of goofy telecining and still gets laid down with a no repeat pattern or a variety of misleading patterns. This trick wont solve everything. There are still animes with blended fields... yuck, and with screwed enough patterns that you'll need filters to fix the patterns, but the blended fields will confuse the filters. It's not a fix all, but it fixes alot, and the more creative you are with the filters, the more it can fix.

I have a few ideas on implementation of the field tags and filters.. even ideas of how to make it compatible with new options to use variable frame rates that may be emerging. If anyone is intersted I'm glad to discuss. I have fewer ideas on how to push these bits through the current works though.. and honeslty I'm deluding myself if I say I'm likely program anything myself soon. I have to much to do in my real work. I am starting to think using a separate file would probably cause serious audio problems with fixed frame rate containers, but I don't really know enough to be sure. I've looked at mpeg4 standards some now and it's redicously complex and I'm certain a bit or two can be pushed through it somewhere without even anything as hackish as putting in the dct stream. For instance, I saw that there is a word for DCT scan order, and acording to an old post by -h it is determined for every block yet there's one left for the whole frame that is not being used for anything. There are at least 3 scan orders so that's 2 bits I KNOW are available, and probably something cleaner could be done. But good luck getting anyone to implement this is into their mpeg4 codecs... even though it's really quite simple.

I have some alternative go around ideas, like adding stupid color blocks outside the image that should be pretty compressible and detectable on playback and woudln't require changes to mpeg4. Or just filters that replace a frame constructed of 1 repeated field and one prepeated field with an exact duplicate of the previous field, then a filter on play back that replaces any duplicates(now not exact possibly) with a top from before and bottom from after. This has the advantage that it's completely compatible,and doesn't require screwing with audio syncing. The repeated frames added will compress to near zero in a codec. Without the right playback filter this will just be jerky but will still play. It WILL require a good chunk of cpu for the right filter(maybe too much) and in fact will add a half frame timing error in places where there were naturually duplicates followed by a change... but a half frame error at 30fps is acceptable as we know from telecining... a whole frame error at 24fps is not... Not ideal, but has some neat compatibility advantages, like anyone who can hack up Mr. Graft's Dup filter can make it happen without breaking into mpeg4 stuff.

I think this kind of thing can happen, but people with the skills have to be convinced it's a nice idea and worth their time (for which us laymen are all forever greatful).

[nuked]

I've found out more about the codec status. SOme kind of interlace minimal ability to encode interlaced fields is needed for this to work. It doesn't have to do a perfect job of supressing field redunancy and since there's only one motion vector but fields may move diferently, they will not. I ahve read that it is possible to implement field separated motion vectors. Xvid has a a sorta field optimized DCT but not field separated motion vectors. Xvid also has broken B-frames with the field DCT. So... this is good enough altohough I'd like working B-frames. I don't think interlaced motion estimation will come anytime soon... if it would I'd drop the whole tff rff idea, as that would come close enough to providing the same savings or more(the combination would be best) with just an encoding time cost. The flags are I'm sure much easier to program though than the field M.E.

DivX5 has an interlaced option.. I tried it in quality mode.. quant=2, no visual improvement and slightly larger file... no good. NOt to mention I don't know how to use a bob-filter on the divx5 playback codec, and the ffdshow codec seems to be adding artifacts to my interlaced divx5's... some color wierd, persitent color striping even in stationary areas. I haven't tested if their option helps at lower bit rates.

So currently to me the only option for 60 field/s playback is XVID with no B-frames. you lose as much as 20% files size on redundant fields and another 20% on B-frames... all total a big hit. My flags would fix the redundant fields and hopefully B-frames will be fixed someday.

P.S. alot of the dvd's that look bad even from the dvd will look good on a tv or using a 60fps bob... which is what I suggesting anyway in the end. This separates out the fields in time so the wrong ones don't get merged.