Cvs 2002-04-17

[-h]

Just added support for lumi masking and hinted ME (didn't get a chance to try modulated quantization, though I assume it's still broken :().

I would recommend that people try enabling lumi masking on *both* passes, just to see what kind of effect it has. I think it'll help quality, Koepi thinks it'll hurt ;)

I still have to look at:

- Modulated quantization + hinted ME problems
- Custom quant matrix problems
- Enabling/disabling controls (heh I'm bad at this)
- Weird credits behaviour (crashes, bad frames, etc.)
- .. anything else??

-h

[Teegedeck]

how about
- drinking a cup of tea and watching telly?

Thanks for all your work.

[Selur]

and
- ordering something to eat and relax :D

Cu Selur

[-h]

- drinking a cup of tea and watching telly?

Try, lying in bed eating cashews to bad british sitcoms ;)

Why do you think I'm so long between updates?

-h

[-h]

I just realised how much my face is going to show up in this forum now.

-h

[Koepi]

XviD-17042002-1:
- EPSZ and EPSZ^2 activated.
- Fresh CVS checkout
- Fixes for hinted MVs and luma masking


As -h pointed out, I recommand using luma masking only on second pass (though we need to put in some fixes to curve treatment for that ;) ).

Best regards,
Koepi

[Teegedeck]

BTW, I forgot to report a bug that I found on the weekend; don't know if it still is there: VDub reproducably crashed with a compression error upon a constant-quantizer encoding after a couple of hundred frames when I left the mv-hint file option ticked. Apparantly, XviD tried to read from the mv-hint file -- I think it should only try to do this on a 2nd pass.

[Hanty]

Nice pic -h, now the feds know who to look for. ;)

Seriously though, about luma masking.
If it works like it should in theory then Luma masking would produce higher quality, no? The question is more like wether it actually performs like in theory and giving a different result in practise, no?

This begs the question, does Luma masking currently exactly do what it claims to do, nothing added nothing taken?

[Teegedeck]

Excuse me if I try to answer.

Originally posted by Hanty

If it works like it should in theory then Luma masking would produce higher quality, no?

No, not really. It should reduce quality where it is less visible. The question is whether it really is less visible. Little enough to accept that loss for the gain in _overall_ movie quality that the resultant bitrate savings give.

[Doom9]

if we're at the subject of enabling and disabling controls why not label everything that's I frame related consitently? last time I checked there still were i-frames and keyframes used interchangeably..

[LotionBoy]

lumi masking is a very subjective adjustment, the value of which really depends on the person who is encoding the movie. In theory, it should make dark areas look slightly worse and light areas look slightly better, and I assume it does this fine (though I've never used the Xvid lumi maksing). Whether or not this looks 'better' is something that depends on both the source and the viewer.

LotionBoy

[Hanty]

Yes yes, I see. The quality gained from using luma masking would be the quality used bu the "saved" luma masking bits. THAT however is an impossibility to measure.

Only way to reliably measure it would be to make two identically set up encodes with luma masking on in one and off in the other. To make a consensus on it would be to produce dozens of encodes in this manner.

Thus it is interesting to hear how there is a difference in opinion by the xvid codec builder on Luma Masking.

[mrbungle]

where to get your binaries -h ?

[Neo Neko]

I believe -h and keopi pool their work into the collective binaries found on keopi's page. Not to mention some of Nic's work is there as well.

[Nic]

'Bout time I did an update...my DVD2AVI project has been quite frutiful so far so ill start spending time on XviD again soon.

Nice pic -h....What bad british sitcoms? Arent they all bad....I haven't seen an episode of "Neighbours" seen ive been working....seriously missing Dionne... :)

-Nic

[Belgabor]

@-h: Just a small question, is the block bug supposed to be gone with the new cvs? Just asking because it seems to be gone in the clip i sent you, but its still in another clip I have.

Regards
Belgabor

[-h]

@-h: Just a small question, is the block bug supposed to be gone with the new cvs? Just asking because it seems to be gone in the clip i sent you, but its still in another clip I have.

Really? Nothing much has been changed in that regard (I believe Isibaar is having a play with your clip), so I'm not sure how it disappeared in the one you sent :)

Could you post a screenshot of the other clip's artifact?

-h

[omol]

Originally posted by -h
I would recommend that people try enabling lumi masking on *both* passes, just to see what kind of effect it has. I think it'll help quality, Koepi thinks it'll hurt ;)

It helps a lot! This is on Sting's Brand New Day Live. Lots of slow cross-fades and dark scenes.

regards,
omol

[Belgabor]

Originally posted by -h

Could you post a screenshot of the other clip's artifact?


Sure, here they come =)

Two small notes:
- These Screencaps were taken from a testclip, the effect is much worse in the whole anime ep (= her entire leg is full of blocks/blotches)
- The effect turns up when encoding with constant quant, too (2 in this case). If you want the clip, its much shorter than the other one (4.5 Mb, perhaps i can cut it down even more). If you want it, just say so ;)

Regards
Belgabor

This is the original

[Belgabor]

And this is the ecoded (buggy) frame

[-h]

So weird.

Just a couple things I'd like you to try :) :

- Decode buggy clip with DivX5 and ffdshow, to make sure it is an XviD encoding problem
- Try H.263 and MPEG quantization (can't remember if you've already tried this?)
- Try encoding the same sequence on a different PC, to see if it's assembly at fault

If that doesn't single it out.. we might need that clip :)

-h

[avih]

Belgabor, why do harass developers over such a small gray square. just use photoshop to correct it! ;) lol

seriously now, this is a known issue for long time, i also wonder if someone is looking at it. actually i think it's one of those bugs that might be hard to find (because it's rare), that has been there too long, so no one remember when it was introduced :), and the most visible visual bug xvid has.

i do also hope someone will fix that, but each of the developers is working on features atm, (unofficial bug fixing time was about a month ago). i believe it'll eventually be solved, but it's so rare, that for me, in 99% of encoding, i just don't see it.

maybe if u have the time, can u do some tests and find exactly which parameters/video-type/resolution/etc cause it to appear?? i'm sure the dev team will be appreciate it a lot.

cheers
avi

[Aktan]

The small grey block is cause by the deinterlacer! not due to the codec in any way. Try the fram in full res without ANY deinterlacer or and filters, and you will see what I mean.

[Milkman Dan]

Well, Sasami is still open to host a new clip if the problem cannot be worked out on his end.

[Belgabor]

Originally posted by -h
- Decode buggy clip with DivX5 and ffdshow, to make sure it is an XviD encoding problem

I took the caps with vdubs 'save to bmp' feature, so the decode should be coming form the core, right? Well, I'll try with divx5 when i go back to my flat at university tomorrow. ffdshow shows it just the same.

Originally posted by -h
- Try H.263 and MPEG quantization (can't remember if you've already tried this?)

I did for the clip I sent you, and I'm pretty sure I did for this one, too. its in both.

Originally posted by -h
- Try encoding the same sequence on a different PC, to see if it's assembly at fault

That will be a bit tricky and probably take about a week. The only other pc (with a windoze OS on it) I could easily encode on is a 1.2 GHz Athlon, which shouldnt be really diffrent form my own 1.4 GHz Athlon. But I'll see what I can do :)

Two more things I found out yesterday:
- the bug (at least in this clip) only occurs on quants 2 and (really really evilly) quant 1. Its gone with Q3 and up (only checked H.263 on that).
- another, completely unrelated bug: vdub goes down in flames when you select 'no output. check only speed' and click on 'advanced options', though thats pretty much minor and cosmetic ;)

Regards
Belgabor

[Aktan]

As I said before, I think its really caused by the deinterlacer, not the codec. The reason why quat 2 and 1 can see it is becuase it retains the most detail in which other quants blurr that box away.

[Belgabor]

Originally posted by Aktan
As I said before, I think its really caused by the deinterlacer, not the codec. The reason why quat 2 and 1 can see it is becuase it retains the most detail in which other quants blurr that box away.

What I posted was the original image and the encoded one. no interlacing/deinterlacing involoved between these steps, only XviD encoding. you might be right that a detail not encoded by q3+ causes the error, but its definately not there before encoding. (and the q3+ encodes dont look like theres a box being blurred away)

Belgabor

[-h]

Hm if ffdshow displays it, it's definitely a bad encoding job on XviD's behalf (looks like a DC scaler issue?). So weird.

If it only happens at quant=1 or 2, it sounds like an overflow issue. However that should all be catered for..

Well, if you can try it on the other PC it'd be greatly appreciated. I have a 1.2 Ghz K7 as well, so if it appears on yours it should do the same on mine.

Also it's strange to hear that the Null (test speed) mode crashes on yours - all it does is say the frame used 0 bytes and return an ICM success code - that it could crash seems to say there's something else at play..

-h

[kastro68]

Where can I get a hold of that?

It is so that I can ask for -h's autograph in case I see him on a train in Australia.

[Psyche]

- another, completely unrelated bug: vdub goes down in flames when you select 'no output. check only speed' and click on 'advanced options', though thats pretty much minor and cosmetic
Wow, that was funny.
Yeah, it's right, so this one goes...

@-h:
In config.c, function adv_mode, parameter mode can take a value you did not have into account, when you have Null - test speed in drop down list mode is 6 which falls outside modes[] and the for (i=0 ; i<lengths[mode] ; ++i) is making an access violation. Just making an const int null_testspeed_disable[] array (whatever you think in this one) and adding it to modes[] and lengths[] makes it work correctly.

See ya!

[Aktan]

If its not the deinterlacer, then I think its a MPEG 4 issue, becuase I see the same "boxes" on DivX 3,4, & 5.

[-h]

@-h:
In config.c, function adv_mode, parameter mode can take a value you did not have into account, when you have Null - test speed in drop down list mode is 6 which falls outside modes[] and the for (i=0 ; i<lengths[mode] ; ++i) is making an access violation. Just making an const int null_testspeed_disable[] array (whatever you think in this one) and adding it to modes[] and lengths[] makes it work correctly.

Yeah, I read the post wrong - didn't pick up the "clicking Advanced Options" thing, since there have been problems in the past with that mode.

There is a lot to fix in vfw..

I hate GUI coding with such a passion..

Oh and I stuck my pic back on, because it's the right thing to do.

-h

[Belgabor]

Ok, like I said, it took some time. ;)

I just checked on a Intel machine (dunno excatly which, prolly a PIII). Same story, same picture, same bug.

Regards
Belgabor

[-h]

I just checked on a Intel machine (dunno excatly which, prolly a PIII). Same story, same picture, same bug.

Awesome, is there any way you can post the clip somewhere? This would make Isibaar a happy man :)

-h

[Belgabor]

Sure I could, would require some short time interaction though.

Is he (or you for that matter) somewhere/somewhen to find on irc?

Regards
Belgabor

P.S: clip is 4.5 Mb and encoded with glzw.

[-h]

Is he (or you for that matter) somewhere/somewhen to find on irc?

I can be on irc if you want, might help to organise a time though.

-h

[Aktan]

Um, i siad this same porblem happens on divx 5 divx 4 and divx 3.........

[-h]

Um, i siad this same porblem happens on divx 5 divx 4 and divx 3.........

It's most likely a problem with how the quantization routine actually goes about quantizing. There's nothing in MPEG4 that makes it impossible to properly encode certain sequences.

-h

[Belgabor]

Hi -h

Heh, should have thooght of this before. Cut off some more frames to make it small enogh for attaching ;)

Regards
Belgabor

P.S.: Reencoded to HuffYuv

[OUTPinged_]

if there would been a build with PSNR values for each frame in debug, it would be far easyer to detect that problem *hint* hint*.


PS> noticed "inverted macroblock-like" artifact one time too, but thought it was buggy PC's problem (didnt happen on mine pc yet)

[-h]

if there would been a build with PSNR values for each frame in debug, it would be far easyer to detect that problem *hint* hint*.

Actually, the internal PSNR for such cases shows that no error occurs (i.e. the internal buffer is bug-free). It seems to be a problem with writing bad values to the bitstream, or corruption during a stage between internal decoding and bitstream functions.

PS> noticed "inverted macroblock-like" artifact one time too, but thought it was buggy PC's problem (didnt happen on mine pc yet)

It's a strange one, that's for sure.

I might hold off on enabling Belgabor's attachment pending a fix for the other thread, as it will most likely fix both instances of this bug.

-h