Questions re. interlace support

[sprit]

I noticed -h said:
"Interlaced frame support is mostly working and will be committed in day or so."

I'm recording hockey games from TV with Virtual Dub. The source is interlaced, so I deinterlace in VD when compressing from M-JPEG to XviD.

So, will interlace frame support make the deinterlace unnecessary or is it "just" useful when watching the XviD on an interlaced device (ie TV-out)?

/sprit

[Neo Neko]

If you plan on using a TV for output it can handle the interlacing just fine. The thing with Xvid and most other codecs is that when you encode interlaced material it makes the over all file size much larger than if they were deinterlaced. There for for best compression and size deinterlace is needed. The new code should make improvements on compressing interlaced material without deinterlacing. I will still be deinterlacing myself for a while though. 80% of the time I watch my encodes on the PC an not on the TV.

[-h]

MPEG4 interlacing doesn't remove the interlacing artifacts, but preserves them by separating fields when suitable. It'll allow you to capture interlaced content in either 1) better quality or 2) smaller files.

It's a tad slower, and I've decided to hold back committing until B-frames are in (hint hint) and I've fully debugged field-based motion estimation.

-h

[Nic]

(hint, hint)? Tell me more :)

-Nic

[avih]

great to hear it's working well.

question+idea:
does interlace "support" means it compresses interlaced clips better (i.e. internally in 2 fields) only?

or.. does it also "support" playback somehow. i.e. would it be possible to play interlaced clip with double framerate than the capture (=actual field rate) with some de-interlacing method (i.e. bob) with current interlace support?.

this could be an option in the dshow project (another q: can a dshow filter modify (double) the frame rate? or is it just possible from core/vfw and the dshow filter will have to control core/vfw??)

playing back @50fps would be fantastic, and shouldn't take too much effort to implement nor should it take too much cpu. i might work on it if no one else would :)

since (double framerate)*(half v-res) will cause massive vert jitter, deinterlasing method that doubles fields v-size and applies 1 pixel shift would have to be applied. in that case we're talking doubling pixel throughput ->doubling filter load. but if we don't use filters (the need for postprocessing filters decreases dramatically with double framerate) cpu load should not suffer much.

this would enable to capture once (interlaced), and play it right away as we would expect interlaced clip to play (deinterlaced @original field rate). exciting :).

cheers
avi

[-h]

does interlace "support" means it compresses interlaced clips better (i.e. internally in 2 fields) only?

It switches between a field-based and frame-based DCT on a per-macroblock basis (the field-dct decision is what slows it down - 448 subtractions and 112 additions per macroblock in C), and adds field-based motion estimation as another per-macroblock option (2 16x8 motion vectors per block). The frames themselves aren't separated into fields, it just applies several tricks to work around interlacing artifacts.

You can specify a top-field-first flag in the bitstream, but this doesn't actually affect any steps in encoding/decoding, but is rather information that the playing app can use.

this could be an option in the dshow project (another q: can a dshow filter modify (double) the frame rate? or is it just possible from core/vfw and the dshow filter will have to control core/vfw??)

Perhaps Nic could field this (the AVI filter would be specifying the framerate, right?), but I believe individual filters in the chain could modify the frame rate. In that case, it'd be a simple buffering trick to get 50 half-height fields out per second (just specify stride*2 with a line offset between fields, no extra memcpy()).

Oh and and about the hint-hint... it turns out there won't be anything stopping me committing tonight after all ;)

-h

[avih]

:D
i'll check the cvs 1st thing in the morning
! :)

[Nic]

Oh and and about the hint-hint... it turns out there won't be anything stopping me committing tonight after all

Yay!
(I want to get privy to all this insider info :D )

(Back on topic :) )
Unfortunatly DShow filters for decoding are weird like that....Decoder filters are known as Transform filters (the main function that must be implemented from the interface is a function called Transform). This function just has an input buffer & and an output buffer. Thats all, so controlling the actual flow/framerate can't really be done from inside the actual decoder filter (as its done from the AVI Splitter :( )

Cheers,
-Nic

[trbarry]

-h

Are you just adding support for storing interlaced fields, or are you actually adding a motion comp deinterlace to Xvid?

I've been reading up a bit and it appears that, like many video filters, motion comp deinterlace applies a clever algorithm to do one thing followed by some judicious blending to hide any new artifacts created. ;)

If that is really the case then a good adaptive blending deinterlace process might fit well as post motion comp step.

- Tom

[avih]

hehe, what a hype :)

@-h: wouldn't a constant (during encoding) flag make things faster? no decisions to make, just run full steam ahead. afterall, we should know if the clip is interlaced or not...

i understand that your method should produce better results in terms of file size, but if we're talking capture, where cpu load matters, the trade-of between smaller file and *not dropping frames* can be acceptable for many users.

[avih]

Originally posted by trbarry
-h

Are you just adding support for storing interlaced fields, or are you actually adding a motion comp deinterlace to Xvid?

I've been reading up a bit and it appears that, like many video filters, motion comp deinterlace applies a clever algorithm to do one thing followed by some judicious blending to hide any new artifacts created. ;)

If that is really the case then a good adaptive blending deinterlace process might fit well as post motion comp step.

- Tom

i think that in the case of deinterlaced playback, all the codec has to do is output @actual field rate with half vert resolution (unless the dshow filter can't modify frame size, in which case, the codec should produce (double framerate)*FRAMEsize.

since all the motion vectors are available during playback, it could be an excellent starting point to play with motion compensated deinterlacing. THIS would be a breakthrough in (RT) software deinterlacers (we got rid of the need for perliminary MV calculations during playback). afaik, only hardware deinterlacers use MC methods these days.

we're expecting exciting times :)

[-h]

@-h: wouldn't a constant (during encoding) flag make things faster? no decisions to make, just run full steam ahead. afterall, we should know if the clip is interlaced or not...

Well once I get field-ME working (won't be in the first cvs commit, pmvfast mods will take a couple days), you'd only want to perform a 16x16 match followed by a 16x8 match - there's no real reason to perform the inter4v step given the speed hit. You could probably get away with discarding the 16x16 search altogether, for a super-fast capture mode. Anyway, the first implementation will use the same pmvfast options as normal non-interlaced encoding.

The field/frame-dct choice should be at least 16 times faster in mmx (and faster in xmm), so I'll do that tonight and commit - forcing every macroblock to use the field-based DCT would probably bloat files by quite a bit.

-h

[avih]

>"forcing every macroblock to use the field-based DCT would probably bloat files by quite a bit."

more than "standard" encoding?? we still have the same number of blocks. how bad can it get?

:) oh well, i'll just try it out with a private custom build...

[-h]

more than "standard" encoding?? we still have the same number of blocks. how bad can it get?

Hm I'll run a test and see.. but I would think the additional vertical frequency energy created by "de-interlacing" blocks which didn't exhibit any interlacing artifacts, will blow out the coefficients. Might get compensated for by vertical prediction if we do every block... should be interesting.

All the field-DCT actually does is, instead of sending lines 0 through 7 and 8 through 15 to fdct(), even lines 0 through 14 and odd lines 1 through 15 are sent instead. If the fields aren't exhibiting interlacing artifacts, we'll actually be creating vertical complexity instead of removing it.

Oh and I put the wrong smiley on my hint-hint - there won't be anything else complicating my commit :(

Heh well not tonight anyway :)

Yes teasing is fun... and no one knows if I'm telling the truth!

-h

[avih]

>" All the field-DCT actually does is, instead of sending lines 0 through 7 and 8 through 15 to fdct(), even lines 0 through 14 and odd lines 1 through 15 are sent instead. If the fields aren't exhibiting interlacing artifacts, we'll actually be creating vertical complexity instead of removing it. "

well, now i really hope i understand the process correctly. afaik, the fdct is still operating on 8x8 blocks, in which case, the high vertical energy will fall exactly between blocks, and won't increase the vertical freq inside the block. so the block coeff souldn't get higher...


ok, i thought again, and i'm wrong. i understand now that the higher v-freq is between every 2 lines, since they're not adjucent originally. each adjucent fdct 2 lines were originally separated by 1 line. sorry for the rush question.


>"Yes teasing is fun... and no one knows if I'm telling the truth! "

it doesn't matter as long as new ideas keep flowing :) ideas will get implemented tonight or some other time, bu u or by someone else. comming up with the ideas is the tough (and fun) part ;) although there IS great reward in seeing them implemented...

good luck -h

night ;)

[wing1]

Take your time -h. Anticipation is a good feeling to have :p

[Nic]

I kinda guessed not to get too excited....your such a tease lol :D

-Nic