Misplaced edges/blocks when using b-frames (1.0RC2) !

[Leguman]

Hello,

I noticed that when using b-frames, sometimes there are misplaced blocks in the picture, this is especially noticeable around edges.
This problem is also present with Koepi's 24062003 build but less noticeable than in recent XVID's releases.

To better describe the problem I encouter, here are 2 short vids with default x-vid 1.0RC2 settings except VHQ4, Trellis & chroma optimiser enabled (no qp/aq/gmc).
1st pass is about 1400 KB in size, so 1st/2nd pass ratio is close to 2 in this example.

http://home.tele2.fr/pouet/with_b-vop_(2-150-100).avi
http://home.tele2.fr/pouet/without_b-vop.avi
http://home.tele2.fr/pouet/frame155_with_b-vop.jpg
http://home.tele2.fr/pouet/frame155_without_b-vop.jpg
http://home.tele2.fr/pouet/divx.avi

As you can see, when using b-vop, the overall quality is hopefully better, but these misplaced blocs artefacs are *very* annoying (see frames 155,162 or 367 for example) because there are very 'unnatural' : no blocky but really wrong images parts!
Same results with ffdshow and xvid decoder.
The divx output is bad at this bitrate but I included it just because it do not show any misplaced bloc artefacts contrary to the x-vid version.

Setting only 1 b-vop instead 2 and reducing b-vop quantizer dramatically reduce these artefacts, but also increase 1st pass size, so it is not a good solution especially with 1CD rips. Of course the higher is the bitrate, the less are these artefacts. It seems that enabling QPEL helps also.

Note also that these artefacts are quite random, whenever I change any setting, the bad frames positions are not the same (but such frames are still present).

I don't know if it's a bug or if these artefacts are inherents to xvid algorithms but I would be pleased if this problem can be solved (at least reduced) before 1.0 becomes final.

Any workaround or advice in the meantime ?

Thanks.

[kilg0r3]

Sorry I can't help with this problem just two hints to enhance your doom9 experience :):

1. not all links seem to be working ...

2. Please document your encoding chain.

+ hardware platform
+ operating System
+ applications used for encoding
+ b-frame settings

3. In the meantime you could try to encode with chroma optimizer unchecked.

Else, I only can confirm that the avi with b-frames has a problem also on my system.

I haven't followed the discussions enough lately but as far as I can survey the situation you have earned yourself the :helpful:-Award.

[Leguman]

1. Hehe, sorry for the broken links kilg0r3, there are fixed now :)

2. Well, I'm under XP SP1 with an AMD XP 2000+. I used DVD Dcrypter 3.1.7.0 / DVD2AVIdg1.0.0RC2 & Vitualdubmod 1.5.10.1 (fast recompress, no vdub filter).
Here are the AVS parameters
mpeg2source("R:\15ANS_DVD1\VIDEO_TS\1.d2v")
crop(12,4,696,568)
KernelDeInt(order=1, threshold=0, sharp=true)
BicubicResize(512,384,0,0.5)
undot()
FluxSmooth(7,7)
limiter()
I didn't mentionned it, but the video is interlaced PAL.

3. I've just encoded with chroma optimizer unchecked, but I don't see obvious improvements. Some frames are better (#155 is ok), others are worst. Globaly I would say it is better, but far from 'perfect'.

[Selur]

yup, this are misplaced macroblocks, you system isn't overclocked in any way, is it?

[Leguman]

Originally posted by Selur
yup, this are misplaced macroblocks, you system isn't overclocked in any way, is it?

Well, my system is slightly overclocked but it is stable ;)
I get bit identical results at nominal speed (same crc)

[sh0dan]

What about VHQ = 0?

Also, try disabling all assembler extensions.

It seems like a bad ME choice or bad compensation. Either way it does look like some sort of bug somewhere.

[sysKin]

Originally posted by sh0dan
What about VHQ = 0?

Also, try disabling all assembler extensions.

It seems like a bad ME choice or bad compensation. Either way it does look like some sort of bug somewhere. Yes. I've seen that before. No idea what this is.

VHQ will not help because it's in a b-frame, where VHQ will have no effect (or will have a random effect, like this chroma optimizer off).

Can anyone find a short source which I can use to reproduce bugs like this? It would help me a lot. Actually this clip here will also help me, just wait a second until I'll discover more :)

Radek

[crusty]

well, looked at it, but could only find it after looking at the jpeg's.
It has the feeling of some random noise if you look at it for the first time.

I can confirm this with RC2 on my system.

[BeNooL]

I noticed the same kind of artefacts on anime encodes.
It started in XviD 1.0 beta 2 and still is present up to RC2.
All soft and configuration is indentical to all encodes, only XviD codec was changed.

'things' start appearing around some edges where they shouldn't be...
A few frame later, when object moves, they are in the right place.

It's really strange and occurs randomly it seems...

Here's a pic to illustrate the effect:
http://benool.free.fr/bframebug.png

the black spot below is not noise, it's not present in the original .avs. From what I checked it only happens on bframes.

[sysKin]

From what I see in this clip, all artifacts are from not-coded macroblocks. Since not-coded ones are easy (don't even have a motion vector, don't have DCT information), I have to conclude that the actual decision whether to use them is wrong.

It would be very good if I had any sample which I can use to recreate the artifacts. I would be sure if it's the case and I would make some thresholds higher in order to fix this.

Radek

[Leguman]

I made an encoding with no VHQ and no performances optimizations upon sh0dan suggestion, the bug still occurs.

I uploaded a lossless huffyuv encoded sample here, so anyone will be able to make its own tests: http://pouetos.free.fr

This is the avs output (already deinterlaced & resized), so please do not add any filter while encoding.

As there are many parameters in xvid, I suggest you first try with 100% default settings (load defaults, especially default b-vop settings 2-1.50-1.00) because I'm sure there are artefacts with these settings.
Then make a 2 pass encoding with 750 KB target size, not more (remember the more the bitrate is high, the less the misplaced blocks are visible)

Hope this helps!

[sysKin]

OKay I've got some bad news. These are ugly compression artifacts that can't be hidden in 1.0 tree. I will commit a small change that helps with some of them, at the cost of all other frames, but not much.

Just a small explaination: when encoder tries to discover the most correct motion vector or mode, it calculates sum of all errors (meaning all pixels). If 250 pixles have no error but 6 pixels are completely wrong, total sum is not big and encoder thinks it's a good match. It's not of course.

I already know what needs to be done, but 1.0 tree is frozen and I can't. 1.1 tree will open as soon as 1.0.0 final is out, and I'll fix it as soon as I can.

I talked to nanoflower on irc and he said that not using too many filters helps. He's probably right, especially with temporal smoothers. I'd give it a try.

At worst, you can use zones to reduce number of b-frames in these areas.

Radek

[Selur]

hmm,.. the thing with the feature freeze is understandable, but should it stop anyone from fixing bugs? I mean if it's not that much of a fix, that could introduce a lot of other bugs, a fix should be done. (but that's only my opinion, and I understand it, if the feature freeze overrules bugfixes,..)

[sysKin]

Originally posted by Selur
hmm,.. the thing with the feature freeze is understandable, but should it stop anyone from fixing bugs? I mean if it's not that much of a fix, that could introduce a lot of other bugs, a fix should be done. (but that's only my opinion, and I understand it, if the feature freeze overrules bugfixes,..) The thing is it's not a bug. It's a compression artifact, just like blocks and ringing.

And no, fixing it (aka. doing something to avoid it) will require some major redesign. Definitely not something to be done in 1.0.

[Selur]

okay, thx for clearing this up :)

Cu Selur

[Andrey]

>>OKay I've got some bad news. These are ugly compression artifacts
O !
Remember my investigation of anamorphic 1CD encode ? (It of course, use many B frames)
I wrote about strange "noise" around sharp edges...
Seems that anamorphic encode looked better not because it is anamorphic, :) but because of resize, that change/smoother such an artifacts a bit.
Anyway, that anamorphic thing needs reinvestigation after this bug will be fixed.

[bond]

hm, is this a big problem?

i mean you guys, who encoded already 2hours or so and saw the artifacts how often do they appear approximately? 2 times in the movie or every 5 minutes or whatever...?

[sysKin]

Originally posted by bond
hm, is this a big problem?

i mean you guys, who encoded already 2hours or so and saw the artifacts how often do they appear approximately? 2 times in the movie or every 5 minutes or whatever...? I'm pretty sure it only happens with ultra-clean, possibly overfiltered source.

[Leguman]

Originally posted by bond
hm, is this a big problem?

i mean you guys, who encoded already 2hours or so and saw the artifacts how often do they appear approximately? 2 times in the movie or every 5 minutes or whatever...?

Well, certainly it's not the biggest problem ever in xvid ;)
With 1CD rips, movies that don't exhibit this problem at all are rare, some have numerous artefacts. It seems to be unpredictable. Moreover when these artefacts show up, they are not always visible if you don't look the picture carefully.
However, when they are visible, they win the "ugliest artefact" award,= ;) It's not just like a little ringing or blocky part of the image.

With each new release, I guess the xvid devs try to make the codec better (correct bugs & improve image quality) so I far I'm concerned, trying to get rid of this encoding problem is not just a good idea, it's simply normal behavior :)

I'm pleased to see sysKin takes into account this problem, and I hope to see a visual improvement with these kind of artefacts in the future 1.1 release, even if I have to wait a few month.

Edit : to precisely answer your question, in most case these artefacts become visible more about once an hour than once a minute. The sample I choose is the worst I found up to now, showing more than 1 artefact per second ;)

[Leguman]

Originally posted by sysKin
I'm pretty sure it only happens with ultra-clean, possibly overfiltered source.

First, thanks for your replies :)
I tried with no filter except kernedeint & bicubicresize, but I saw no improvements.
Regarding the original video, you're right it's a rather clean one (at least this sample).
Morevover the background colors are very close, I guess it don't help the codec to make the best choice.
I can upload the original sample without any filtering/resizing if you're interested.

[Sharktooth]

Originally posted by Leguman
I can upload the original sample without any filtering/resizing if you're interested.
Yes, please:)

[MfA]

Pretty much the same effect omni pointed out a while ago BTW. Just a better example.

[sOy]

@Leguman

I have found, with build 24062003 sometimes 2 max b-frame would produce same kind of artifacts. Could you change the b-frame setting to 1-150-100 and test again? Thanks.

hmmm, if I am wrong please correct me.:o

[bond]

damn, a whole bunch of encodes was waiting for 1.0 and now this :(

[Aktan]

Damn, I should have posted this bug earlier. I have a clip which has it all this time. I've been testing varies settings on RC1 and RC2, and the best way to get rid of it is to set b-frame max to 1. Or u can set b-frame sensitivty to -33 in which those area use 1 b-frame at a time anyway :D . I can post a huffyuv clip is u want sysKin.

[Koepi]

Well - the fix won't make it into 1.0 (it's for sure mostly affecting anime) - if you are sensible to these small glitches (they should be very seldom IMO), wait for 1.1 alpha to show up :)

But please, test 1.0RC3 so we can make xvid 1.0 final and start with 1.1 (it's in your interest too, isn't it? ;) )

Regards
Koepi

[crusty]

But please, test 1.0RC3 so we can make xvid 1.0 final and start with 1.1 (it's in your interest too, isn't it? )
LOL!

Interesting...1.0 isn't even out and already the list for 1.1 materializes...

So IIRC, fixes with ETA at v1.1:
- IDCT decoder switch
- This 'Total-sum-error' issue creating artifacts especially in anime

Then we have the request for the ability of setting a zone as a 'default'

I would further like to add a request for a feature which I think would come in handy:
-a 'dump settings to textfile' button in the debug tab so people can post their full settings here by just copy&paste from that file. Don't know if it's much work but it would certainly come in handy for bug reports. :)

But no rush, keep it cool.