pixel accuracy

[Zarxrax]

While I love mpeg-4 compression and xvid, i'm rather disappointed that even at maximum bitrates and quality settings, no mpeg-4 codecs currently availble are able to accurately (visually lossless) reproduce the video. While under most cercumstances, I cant tell the difference between the source and the mpeg-4 encode, however I recently found a situation in which the differences are VERY visible. I used virtualdub and the subtitle filter to overlay some subtitles onto the video. It looked very nice in the virtualdub output pane, but once i saved it to an avi, I saw extreme differences! I tested divx3, divx4, xvid, and huffyuv (just to confirm that the video was accually getting sent to the compressor correctly). Both divx and xvid produced the same error. I've got the images here to show the problem, and i've blown them up 400% so everyone can see it clearly. Most of the subtitles look fine, but the "@" symbol seems to have extreme difficulty compressing. If the xvid creators could look into this problem, and hopefully fix it, i would... well i wouldn't do anything, i'd just be very happy :)

http://www4.ncsu.edu/~baclontz/subs-uncompressed.png
http://www4.ncsu.edu/~baclontz/subs-xvid.png

[saVe]

the problem you're adressing is that all our beloved video codecs are made for natural content with natural lines and colors.
text isn't natural and therefore it doesn't look very nice when compressed. that's also the reason why credits become so big when you encode a movie using real vbr.

my view on things isn't very technical, correct me if i'm wrong, please!

[MaTTeR]

Isn't most everybody using VobSub to overlay the subs anyway? Forgive me if I'm missing the point:)

[Koepi]

MPEG4 wasn't designed to code text in that way, MPEG2 neither. Guess why subtitles are in separate files.
That "test" just clarifies this.

Use an appropiate bitrate and it will look good.

I really dislike it when unrealistic demands are made - btw., demands?

I demand some money for the developers in return for that :P

SCNR

[RadicalEd]

If your problem lies soley in the red @, I can explain that.

For whatever reason every modern high-compression codec horribly pixellates pure colors. By 'pure' I mean 255 bits of red, 255 bits of green, or 255 bits of blue. It is to my experience that it can also happen with colors close to pure as well. I believe this issue is related to the quantization of the uniform color areas and high frequencies, but I'm in no way an expert on that (yet...)

I have however posted an example (http://www.angelfire.com/anime3/airanime/comparered.html) of this strangeness. Thats not even the worst example I've ever seen, it occurs in DVDs as well and has been very obvious on all of the HD broadcasts I've seen thus far.

Anywho I wish I had a solid explanation for why this occurs but I can tell you that the problem isn't specific to XviD, its a generic problem with all real compression and we have to live with it for the time being.

Subs themselves can have a lot of ringing, but I'm not even going to start on that :rolleyes:

[-h]

This is because MPEG-4 (in its current incarnation) is only storing information in the YUV 4:2:0 colorspace - thus finely detailed colour areas are, well, savaged. Compare the detail in the black&white text (using the fully-stored Y channel) to the red symbol (decimated U values).

You could try using MPEG-2 in the 4:2:2 colorspace to fix this, but even that might not be enough (4:4:4 ?).

Blah, got sick of my face.

-h

[trbarry]

Blah, got sick of my face.

If I'm the only one left with my real photo then I guess I'll have to round up some suitable anime character icon.

But nothing immediately springs to mind. :(

Anyway, MPEG4 studio profile 4:4:4 would be nifty for HDTV. But I guess it wouldn't accomplish anything if my sources were still YV12.

- Tom

[-h]

But nothing immediately springs to mind. :(

I've been looking for clips from Astroboy, haven't found many so I might have to capture the scant few episodes I have on tape.

By the way, have you got the time to check the sse2 stuff I committed on a decent P4 box? I'm uploading some more tonight (the rest of quantization, maybe some interpolation if I go mad), but it doesn't seem to give all that great a benefit on this crappy box.

-h

[Zarxrax]

Hmmm, very interesting. So does MPEG-4 Advanced Profile (or whatever the next step up is called) support this 4:2:2 and 4:4:4 encoding?

[-h]

Hmmm, very interesting. So does MPEG-4 Advanced Profile (or whatever the next step up is called) support this 4:2:2 and 4:4:4 encoding?

Yeah, but I don't think you'll see it in XviD for a *long* time - it's quite a substantial modification, and I'm not sure what else would have to change in order to support it.

Also, I've never seen anywhere in the spec that even talks about it. I.e., the spec isn't there yet for that feature.

-h

[trbarry]

-h

Sorry. I haven't done anything at all with Xvid or any of my usual projects for the last few days. I got in manic mode and a thread over at the Xvid forum made me determined to write a good motion compensating video deinterlace filter, first for Avisynth. I probably won't do anything else until that's done so I can finish it before I lose momentum.

On 4:4:4 and 4:2:2 ... I've got a book called Video Compression Demystified. It talks about something called a MPEG4 'studio profile' that supports the higher color depths, for professional use. But I'm not sure it ever got approved yet.

- Tom

[TheBashar]

Originally posted by trbarry

I got in manic mode and a thread over at the Xvid forum made me determined to write a good motion compensating video deinterlace filter, first for Avisynth.

Tom,

Are you doing something along the lines of what Alexis detailed in his thesis? Just curious. It was an intersting read for me, but I have no skill or extrodinary interest in that area.

[avih]

tom :) good luck with that. pls make me a beta tester :)

and don't forget vdub and dscaler too! :)

avi

[Koepi]

Hm, can that avisynth filter contain some motion compensated denoise as well? Temporal smoother with knowledge of the occuring motion will be a huge step for better TV Rips/encodings (but will be hellish slow ;) but I know you are a magician in optimizing ASM/MMX/XMM/... ).

tom, please, pretty please! :)

Best regards,
Koepi

[trbarry]

Are you doing something along the lines of what Alexis detailed in his thesis?

TheBashar -

The couple times I tried his web site I couldn't get there so I haven't read it. For some obstinate reason I'm trying to do an optimized MMX hierarchical approach, mostly because he said it wouldn't work in your thread there. ;)

But nothing very elaborate. I am not at all an expert in ME so it may not pan out at all. But I think it is going to.

avih -

I don't write Vdub/RGB filters but yes, the eventual target is DScaler also, if I can make it fast enough. But it is easier to develope for Avisynth first.

Koepi -

Don't know about the magi issue but it's way too early to start thinking about bells & whistles. It would just confuse me.

Everybody -

Sorry for the obvious OT thread hijack. Back to coding.

- Tom

[TheBashar]

Originally posted by trbarry

The couple times I tried his web site I couldn't get there so I haven't read it.


@trbarry

If you are interested you can get two of his papers from:
http://home.hiwaay.net/~stephena/4310-101.pdf
http://home.hiwaay.net/~stephena/indexthesis.pdf

The first paper is almost identical to Ch. 9 of the second paper (his thesis). It was interesting reading for me, but I wasn't interested in doing ME deinterlacing. You may aprpeciate it more.

Cheers!

[trbarry]

TheBasher -

Thsnks. It turns out I had aready read the first paper. I think someone posted another link to it here recently.

- Tom

[avih]

tom, how about starting with xvid playback? kinda like postprocessing. u already have all the MV u need ( u can download the source to my MV visualization), just use the existing MVs to optimize your MV based deinterlacer.

once the deinterlacer itself is satisfying, start writing your own ME code (or just take xvid code or wait till it improves even more)

then you'll also integrate deinterlacing into xvid playback, which would be a most welcome feature (i wanted to do it with bob, but u have much more experience with deinterlacing from what i understand)

the only problem i thought of is how to double the playback rate to 50/60fps. still didn't look at the sources enough to tackle this issue.

cheers
avi

[trbarry]

avih -

The Xvid decoder is hard to develop for first, at least the way I like to do things. I usually get partially working code running by setting breakpoints in the debugger environment and stepping through my code one instruction at a time watching the registers and other values. Maybe I'm doing something wrong but I've never been able to get away with setting VS6 break points in the dshow filter. It crashes my machine.

So I would probably do an Xvid show addition later, after I had it working for Avisynth, and probably DScaler.

And I don't think we would want to double the frame rate during deinterlace, just remove the weave artifacts. Probably almost nobody could play any decent resolution clip at 60 fps. But if they could, maybe the decoder could just lie about the frame rate from the beginning. I'm still not all that knowledgeable about dshow programming.

BTW, you are right that I probably should have taken advantage of the sophisticated ME code already in Xvid. But I had an idea I wanted to try out.

- Tom

[avih]

tom,
the whole MV array is there for each frame, u can just use it to manipulate the image. again, have a look at my code if u can't find this array.
there are macro-blocks vectors (for 16x16) blocks and block vectors (for interV4 macroblocks mode). just follow the last function (decoder_decode) in decoder.c file.

indeed, it's hard debugging the xvid core (there's no need for the dshow filter though), but how about adding little code to xvid decoder such that it writes a file of pure yuv sequence while decoding, ALONG with the array of MV for each frame (in a different file). this way you have a clean yuv sequence file and an accompaning file of the MV for each of the frames. this way u can test and debug your code without writing the ME code at 1st.

as you said, u can utilize xvid's me code without even looking at it.

avi

ps.
if you do decide to modify the image in decoder_decode (the current frame image is dec->cur) make sure u modify a copy of it, because dec->cur is beeing used further by the codec, amd modifying it will cause MAJOUR artifacts :)

ps#2
i didn't look at the new interlaced mode -h introduced, but it should have MV for each field in the frame, on areas that interlacing artifacts are detected. i'm not sure how do these MV stored in that array, but it shouldn't be hard to understand probably.

[-h]

i didn't look at the new interlaced mode -h introduced, but it should have MV for each field in the frame, on areas that interlacing artifacts are detected. i'm not sure how do these MV stored in that array, but it shouldn't be hard to understand probably.

It should, but doesn't ;)

Field-based motion estimation isn't terribly complex, it's just a tremendous pain to do inside XviD's structure of 6 linearly-stored blocks (the int16_t data[6*64] style arrays).

When it is done, and I really want to do it before I leave, it'll just store two 16x8 field vectors in the mvs[] array, one for the top field and one for the bottom field, plus a couple flags that tell which field the vectors actually reference (since the top field vector can reference the bottom field of the previous frame).

-h

[b0b0b0b]

I have an idea for a hack to accomplish 4:2:2. Just double your x & y resolution for your movie and halve it on playback and voila!

I am going to try this tonight to see how bad it looks. At any rate the 4:2:0 really bugs me because everything looks so soft.

[b0b0b0b]

Oh yeah I just remembered that my machine is unable to play high resolution clips because it is so slow.

[trbarry]

I am going to try this tonight to see how bad it looks. At any rate the 4:2:0 really bugs me because everything looks so soft.

If you are going to be compressing to ratios typical of what we do here then I'm not sure 4:2:0 is a bad compromise. But what bugs me is the idea that maybe the color info gets a bit softer each time it is converted to YUY2 and back to YV12 again. It's maybe not just a performance issue.

- Tom