Newest codecs: new artifacts?

temporance · 10th December 2003, 09:22

I've noticed a change in the artifacts I'm seeing from my MPEG-4 codecs (DivX, xvid). A year or two ago, a too-low bitrate would have mostly been evidenced by blocking and ringing. With DivX 5.1.1 and xvid 1.0b, blocking and ringing are no longer the most objectionable artifacts: what seems to be more common now is a loss of details and motion instability. Postprocessing is not a factor here: I usually judge my encodes without PP.

Loss of details: softening or blurring of minor details / texture. I'm not talking of something as bad as RV9 or WMV9, but things seem to be going that way. Basically caused by high Q. But why does the high Q not seem to produce as much blocking/ringing as older codecs?

Motion instability: things that are supposed to be still move. Basically caused by a noisy motion vector, when Q is too high for the erroneous motion to be corrected by residual encoding.

Blocking is still occasionally an issue in large single-tone areas with a slight gradient. But detail loss and instability are the new enemies.

Anyone else noticed this change? What could be causing it? Too much reliance on PSNR by devlopers? Better ME revealing new weaknesses in the MPEG-4 spec?

unixfs · 10th December 2003, 18:12

I noticed this same issue; Libavcodec shows the same artefacts.
It's funny that detecting stillness is so hard for encoders.
Maybe both codecs are over-sensitive to motion?

sysKin · 11th December 2003, 03:23

Actually, addressing "floating walls" is one of my plans for post-1.0 release.
I'm not sure how to do that yet - I have to detect them first (I have some ideas how) and I have to prevent them (intra-coding every couple of frames? keeping the motion still? Both will cause blocking you know). Or decreasing quant? Will not be very effective...

Anyway, if you have some ideas how to do something about that, you're welcome to share them.

Radek

temporance · 11th December 2003, 09:08

No, I don't have any good ideas besides trying to think of the cause: why do we get floating walls with the newest codecs? Is it a ME weakness?

unixfs · 11th December 2003, 12:56

I have a nice idea: don't do ME at all and reuse the motion information
of the original video

(a kind of ReJig between different flavours of MPEG)

Jokes apart, is there any kind of compatibility/reusability between
the motion infos of MPEG 1/2/4 ?
How practical would such an approach be?

temporance · 11th December 2003, 13:57

One problem with MV reuse is that you don't know that the original MV's aren't a load of old cr@p. If the original MVs are far from optimum then you won't be able to compress very well with them.

Q. How do you know if the original MV's are good?
A. You do a ME to find out!

Teegedeck · 11th December 2003, 13:58

Reusing motion vectors won't work. Even a different quantizer of the adjacent frames changes motion vectors. (Thus there's no reusing MVs in 2nd pass anymore.) And it would completely ruin the efficiency of VHQ.

edit: whoa, temporance beat me by a second on this one!

sysKin · 11th December 2003, 14:03

Quote:

Originally posted by unixfs
Jokes apart, is there any kind of compatibility/reusability between
the motion infos of MPEG 1/2/4 ?

You would have to repeat the same frame pattern as in mpeg-2, which isn't really useful as b-frames are much less useful mpeg-4.

Quote:

How practical would such an approach be?

Not at all probably - different prediction, different modes... do you rememeber how xvid used to reuse motion info from first pass ("hinted ME")? It was very bad, and this would be much worse.

Radek

temporance · 11th December 2003, 14:04

Quote:

Originally posted by Teegedeck
Reusing motion vectors won't work. Even a different quantizer of the adjacent frames changes motion vectors. (Thus there's no reusing MVs in 2nd pass anymore.) And it would completely ruin the efficiency of VHQ.

Agreed, but this is not a killer. We should be able to take some information from existing MVs even if we can't use them as-is. In principle you could use this information to accelerate ME in xvid (e.g. using original MVs as seed for a quick region search).

But even using original MVs as a seed for a sawn-off ME algo means that we must trust that those existing MVs are good. Garbage-in == garbage-out.

Bear in mind that regular xvid ME is so fast now that to beat it you really don't have time to do many SADs. So you really do have to trust the input.

Didée · 11th December 2003, 15:50

I don't have some very clever suggestions, alas. In contrary: the more I think about it, the harder the task becomes - amongst other things, because the floating wall problem appears in different flavours:

1.
If there is e.g. a really static & flat texture, the problem is visible, but it's not unbearable bad. (Adding just a little noise on playback already hides a lot of this 'static floating'.)
This could be improved by something similar to 'cartoon mode', where 'no-motion' is always tested also.

2.
But I've come across a scenario where the result most times is really bad, bad, bad: rooms that are lightend by flaring fire. When the light & shadows are flaring on the walls and ceilings, XviD produces very annoying "shifting back'n'forth". (LOTR FOTR has a lot of these). Even quant 2 is not enough to hide these motion artefacts!
I kinda understand how this effect is produced - but I've no idea at all how to avoid it ...

3.
"Tearing motion":
Imagine you have a pretty dark scene. Something in the foreground is in focus, the background is almost flat colored (black sky, e.g.).
Now, if the thing in the foreground starts moving, the background starts moving also (!), as if it was "teared" by the foreground object.
This should be resolveable by a more sophisticated creation of motion vectors (don't create and trust predictors too blindly). However, that's faaar beyond my horizon.

BTW, while we're at artefacts:
The floating is more pronounced with ffdshow than with xvid.ax.
Moreover, we still have qpel smearing with ffdshow (AND with DivX5 for decoding, also, AND with Nero decoder, also. Didn't test 3ivx yet).
I really start asking myself: Are all the other decoders buggy, and XviD is the ONLY one doing things right? ... or could it perhaps be the other way round?

- Didée

Koepi · 11th December 2003, 16:10

Did you set ffdshow to use xvid idct (walkem idct)? Not using that will produce qpel smearing for sure.

Regards
Koepi

temporance · 12th December 2003, 09:52

Quote:

Originally posted by Didée

1.
If there is e.g. a really static & flat texture, the problem is visible, but it's not unbearable bad. (Adding just a little noise on playback already hides a lot of this 'static floating'.)
This could be improved by something similar to 'cartoon mode', where 'no-motion' is always tested also.

2.
But I've come across a scenario where the result most times is really bad, bad, bad: rooms that are lightend by flaring fire. When the light & shadows are flaring on the walls and ceilings, XviD produces very annoying "shifting back'n'forth". (LOTR FOTR has a lot of these). Even quant 2 is not enough to hide these motion artefacts!
I kinda understand how this effect is produced - but I've no idea at all how to avoid it ...

3.
"Tearing motion":
Imagine you have a pretty dark scene. Something in the foreground is in focus, the background is almost flat colored (black sky, e.g.).
Now, if the thing in the foreground starts moving, the background starts moving also (!), as if it was "teared" by the foreground object.

Didée: Great summary - this rings well with my experience. Presumably these artefacts do not have a great effect on PSNR but because the eye is very sensitive to unnatural motion, they are perceptibly annoying. Agreed, the fix is probably through motion estimation, but my only idea is to add a bias which favors MV (0, 0).

Btw, do you have any webspace to host examples of these bugs? (and maybe uncompressed source too!?) I'm sure if all the xvid developers could see and analyze example of the problem that it could be fixed.

Didée · 12th December 2003, 10:37

Quote:

Originally posted by Koepi
Did you set ffdshow to use xvid idct (walkem idct)? Not using that will produce qpel smearing for sure.

Yes, of course I did.
It's not only smearing. There is also image degration over time: if there is a keyframe inserted in the middle of a long, still scene, there is a noticeable "keyframe refresh" effect with ffdshow (& DivX & NeVideo.ax also), which does not show up with xvid.ax.
Therefore, I strongly assume that IDCT diferences are still an issue. But I cannot tell on which side the faulty things happen. My trust in XviD is great, but the others are the majority ...

Oh, I must admit one thing: I'm still using ffdshow 23-05-2003, because the later ones seem too unstable for me. Perhaps there were related changes in libavcodec since then? However, with ffdshow 23-05 those issues are 100% reproducable.

@ temporance:
No, not one single byte of webspace, sorry. Anyone provide me with a temporal directory to upload, and I'll happily do.

- Didée

Koepi · 12th December 2003, 11:37

There were some changes in ffdshow since then. I'm currently using 28102003-build without any problems - i even leave idct at "autoselect" and it works like charme.
But maybe my samples just don't provoke such behaviour? (keyframe "pumping" is a ntural phenomenon which i also get with xvid decoder, no special effects with ffdshow for my eyes.)

Regards
Koepi

sysKin · 12th December 2003, 15:15

As for ffdshow and idct, I can swear that the change to "xvid" is not always respected. As far as I can tell it sometimes works, sometimes doesn't - looks like a gui/config bug.

I can't confirm that without debugger :/

Didée: great summary. Sometimes a wall is static but moves (because of prediction or because of luminance changes), but sometimes a wall actually moves. Current ME has no way of distinguishing between the two...

My idea of detection: if total high-frequecy energy in a block is lower than total high-frequency error, the block is "bad" (ie block is flat in original, but appears "noisy" after motion compensation, with total noise bigger than total texture).

IMHO detection is easier than doing something about such blocks.

Radek

mfluder · 12th December 2003, 16:10

Yeah, I can confirm what Didée wrote above. It's not exactly smearing but more like image quality degrades until next keyframe. When keyframe arives you can clearly see how image refreshes. Now, interesting thing is that when simple IDCT is used for encoding (older Koepi's builds) and simple IDCT is set in ffdshow there is no image quality degradation and it looks _exactly_ the same as if XviD was used for decoding (the build that was encoded with, of course).

Which leads me to conclusion that libavcodec doesn't have a problem with XviD's qpel. I think that something is wrong in ffdshow and it's implementation of XviD's IDCT or it simply doesn't select the correct IDCT. I'm saying this because from my tests there is no difference if 'normal' or 'XviD' IDCT is used. You get the same level of image degradation but far less than with 'simple' IDCT.

Also, Koepi mentioned that he leaves IDCT at autoselect. When one do this IDCT for XviD is 'normal'. Why is that? Isn't it better and more logical that IDCT is set to 'XviD'?

All this has been tested with the new unofficial ffdshow builds compiled by athos.

Edit: Oh, I haven't seen that sysKin's post but it looks that I'm right about ffdshow not selecting the correct IDCT, which is a good news because it should be easy to fix.

mfluder

yaz · 12th December 2003, 16:20

@didée
i've made some tests so as to get the taste of the new release (1.0b2). my first intention was ... horrible. after a more through investigation i found some strange(?) thing.

1st of all, it's not horrible at all :-) in general, i found the output quite pleasant. great work, devels ! thx

i've tested 5 different sources so as to see the probs listed above & i found that the compression was always higher (1st pass size is smaller than with devapi3, checked with statsreader) but the quantizer distribution of the 2nd pass was narrower(no such tailing towards higher quants).
by checking 'the last flight of osiris' i found the duell scene (lottsa detailes, fine textures) much(!) better but the ship scenes (dark, shady, less detailed) somewhat worst than with devapi3. so the 'detail loss' effect really exists but it has a special taste :-)

'static floating' : still(!) exists, i had this prob with devapi3 too. former 'solution' (unfilter(+,+), blockbuster(noise), mpeg quant) did not work here. adding noise as postproc made it almost invisible.

summing up so far : it seems as if the new codec would remove more from where it's less but less from where it's more (???)

other probs as u descibed. in add, there is this 'green blocks on orange surfaces' problem annoying me the most. i can do only chroma-noising against it on the postproc level.

further, i played a bit with the decoders.
ffdshow/libav(idct=xvid) : the effects are the most pronounced.
xvid : the effects are visible but much less.
by swithing off both (removing xvid.ax), mpc drops a filter chain like this :
wmr9 (renderless)
color space converter
avi decoder(xvid)
the last 2 greyed out. with this i only found the effects just because i knew they were there. so i'm sure, a relevant part of the problems listed is decoder related.

i hope it tells sg to the devels, cus i'm a bit confused with it at the moment.

the bests
y

[EDIT] i didn't play with xvid settings, i used the same as before, exept for trellis. switching it on&off did not solved the problems.

Didée · 16th December 2003, 09:41

Quote:

Originally posted by Didée
I must admit one thing: I'm still using ffdshow 23-05-2003, because the later ones seem too unstable for me.

Originally posted by Koepi
There were some changes in ffdshow since then. I'm currently using 28102003-build without any problems - i even leave idct at "autoselect" and it works like charme.

Okay, so I went on testing the behaviour of ffdshow 28-10-2003.

My experiences:

23-05-2003:
- qpel-smearing
- I-frame-pumping
- exept those two: no other issues

28-10-2003:
- qpel-smearing: just the same
- I-frame-pumping: just the same
- much slower: unsuitable for "HDTV" streams on an XP 1800+ (100% CPU & some stuttering, where 23-05 is doing bored fingertipping at 60% CPU)
- 'use overlay mixer' b0rked ("3-states-checkbox"): no more auto AR correction with anamorphic matroska
- used as postprocessor only (decoding done by xvid.ax), ffdshow interprets XviD's output as IYUV, forcing vertical flipping, and me un/checking approbriate checkboxes all the time ...

Ceterum censo: For me, there are no benefits, but only drawbacks with ffdshow newer than May 2003.

Back to 23-05, and waiting for a stable build.

- Didée

yaz · 16th December 2003, 10:22

@didée
ffdshow : i've just tried the last 4 versions from athos (200308xx-200311yy). 08 & 11 worked fine while 10 was somehow 'unstable'. the quality of 09 was the worst.
all produced some kinda smearing but to different extent.
i-frame pumping: is it a decoder problem ?
idct: other than xvid produced quality loss.

the bests
y

[EDIT]in the meantime i installed the very last athlon compile from gamrdev & koepi's b3 decoder. it improved the output definitely. i should start my tests over :-)

10th December 2003, 09:22	#1 \| Link
temporance Registered User Join Date: Mar 2002 Posts: 486	Newest codecs: new artifacts? I've noticed a change in the artifacts I'm seeing from my MPEG-4 codecs (DivX, xvid). A year or two ago, a too-low bitrate would have mostly been evidenced by blocking and ringing. With DivX 5.1.1 and xvid 1.0b, blocking and ringing are no longer the most objectionable artifacts: what seems to be more common now is a loss of details and motion instability. Postprocessing is not a factor here: I usually judge my encodes without PP. Loss of details: softening or blurring of minor details / texture. I'm not talking of something as bad as RV9 or WMV9, but things seem to be going that way. Basically caused by high Q. But why does the high Q not seem to produce as much blocking/ringing as older codecs? Motion instability: things that are supposed to be still move. Basically caused by a noisy motion vector, when Q is too high for the erroneous motion to be corrected by residual encoding. Blocking is still occasionally an issue in large single-tone areas with a slight gradient. But detail loss and instability are the new enemies. Anyone else noticed this change? What could be causing it? Too much reliance on PSNR by devlopers? Better ME revealing new weaknesses in the MPEG-4 spec?

11th December 2003, 03:23	#3 \| Link
sysKin Registered User Join Date: Jun 2002 Location: Adelaide, Australia Posts: 1,167	Actually, addressing "floating walls" is one of my plans for post-1.0 release. I'm not sure how to do that yet - I have to detect them first (I have some ideas how) and I have to prevent them (intra-coding every couple of frames? keeping the motion still? Both will cause blocking you know). Or decreasing quant? Will not be very effective... Anyway, if you have some ideas how to do something about that, you're welcome to share them. Radek __________________ Visit #xvid or #x264 at irc.freenode.net

11th December 2003, 13:58	#7 \| Link
Teegedeck Moderator, Ex(viD)-Mascot Join Date: Oct 2001 Posts: 2,564	Reusing motion vectors won't work. Even a different quantizer of the adjacent frames changes motion vectors. (Thus there's no reusing MVs in 2nd pass anymore.) And it would completely ruin the efficiency of VHQ. edit: whoa, temporance beat me by a second on this one! __________________ It's a man's life in Doom9's 52nd MPEG division. "The cat sat on the mat." ATM I'm thoroughly enjoying the Banshee - a fantastic music player/ripper for Linux. Give it a whirl!

11th December 2003, 15:50	#10 \| Link
Didée Registered User Join Date: Apr 2002 Location: Germany Posts: 5,390	I don't have some very clever suggestions, alas. In contrary: the more I think about it, the harder the task becomes - amongst other things, because the floating wall problem appears in different flavours: 1. If there is e.g. a really static & flat texture, the problem is visible, but it's not unbearable bad. (Adding just a little noise on playback already hides a lot of this 'static floating'.) This could be improved by something similar to 'cartoon mode', where 'no-motion' is always tested also. 2. But I've come across a scenario where the result most times is really bad, bad, bad: rooms that are lightend by flaring fire. When the light & shadows are flaring on the walls and ceilings, XviD produces very annoying "shifting back'n'forth". (LOTR FOTR has a lot of these). Even quant 2 is not enough to hide these motion artefacts! I kinda understand how this effect is produced - but I've no idea at all how to avoid it ... 3. "Tearing motion": Imagine you have a pretty dark scene. Something in the foreground is in focus, the background is almost flat colored (black sky, e.g.). Now, if the thing in the foreground starts moving, the background starts moving also (!), as if it was "teared" by the foreground object. This should be resolveable by a more sophisticated creation of motion vectors (don't create and trust predictors too blindly). However, that's faaar beyond my horizon. BTW, while we're at artefacts: The floating is more pronounced with ffdshow than with xvid.ax. Moreover, we still have qpel smearing with ffdshow (AND with DivX5 for decoding, also, AND with Nero decoder, also. Didn't test 3ivx yet). I really start asking myself: Are all the other decoders buggy, and XviD is the ONLY one doing things right? ... or could it perhaps be the other way round? - Didée __________________ - We´re at the beginning of the end of mankind´s childhood - My little flickr gallery. (Yes indeed, I do have hobbies other than digital video!)

11th December 2003, 16:10	#11 \| Link
Koepi Moderator Join Date: Oct 2001 Location: Germany Posts: 4,455	Did you set ffdshow to use xvid idct (walkem idct)? Not using that will produce qpel smearing for sure. Regards Koepi __________________ Koepi's new media development site

10th December 2003, 18:12	#2 \| Link
unixfs Registered User Join Date: May 2002 Posts: 308	I noticed this same issue; Libavcodec shows the same artefacts. It's funny that detecting stillness is so hard for encoders. Maybe both codecs are over-sensitive to motion?

11th December 2003, 09:08	#4 \| Link
temporance Registered User Join Date: Mar 2002 Posts: 486	No, I don't have any good ideas besides trying to think of the cause: why do we get floating walls with the newest codecs? Is it a ME weakness?

11th December 2003, 12:56	#5 \| Link
unixfs Registered User Join Date: May 2002 Posts: 308	I have a nice idea: don't do ME at all and reuse the motion information of the original video (a kind of ReJig between different flavours of MPEG) Jokes apart, is there any kind of compatibility/reusability between the motion infos of MPEG 1/2/4 ? How practical would such an approach be?

11th December 2003, 13:57	#6 \| Link
temporance Registered User Join Date: Mar 2002 Posts: 486	One problem with MV reuse is that you don't know that the original MV's aren't a load of old cr@p. If the original MVs are far from optimum then you won't be able to compress very well with them. Q. How do you know if the original MV's are good? A. You do a ME to find out!

12th December 2003, 11:37	#14 \| Link
Koepi Moderator Join Date: Oct 2001 Location: Germany Posts: 4,455	There were some changes in ffdshow since then. I'm currently using 28102003-build without any problems - i even leave idct at "autoselect" and it works like charme. But maybe my samples just don't provoke such behaviour? (keyframe "pumping" is a ntural phenomenon which i also get with xvid decoder, no special effects with ffdshow for my eyes.) Regards Koepi __________________ Koepi's new media development site

12th December 2003, 15:15	#15 \| Link
sysKin Registered User Join Date: Jun 2002 Location: Adelaide, Australia Posts: 1,167	As for ffdshow and idct, I can swear that the change to "xvid" is not always respected. As far as I can tell it sometimes works, sometimes doesn't - looks like a gui/config bug. I can't confirm that without debugger :/ Didée: great summary. Sometimes a wall is static but moves (because of prediction or because of luminance changes), but sometimes a wall actually moves. Current ME has no way of distinguishing between the two... My idea of detection: if total high-frequecy energy in a block is lower than total high-frequency error, the block is "bad" (ie block is flat in original, but appears "noisy" after motion compensation, with total noise bigger than total texture). IMHO detection is easier than doing something about such blocks. Radek __________________ Visit #xvid or #x264 at irc.freenode.net

12th December 2003, 16:10	#16 \| Link
mfluder Registered User Join Date: Aug 2002 Posts: 75	Yeah, I can confirm what Didée wrote above. It's not exactly smearing but more like image quality degrades until next keyframe. When keyframe arives you can clearly see how image refreshes. Now, interesting thing is that when simple IDCT is used for encoding (older Koepi's builds) and simple IDCT is set in ffdshow there is no image quality degradation and it looks _exactly_ the same as if XviD was used for decoding (the build that was encoded with, of course). Which leads me to conclusion that libavcodec doesn't have a problem with XviD's qpel. I think that something is wrong in ffdshow and it's implementation of XviD's IDCT or it simply doesn't select the correct IDCT. I'm saying this because from my tests there is no difference if 'normal' or 'XviD' IDCT is used. You get the same level of image degradation but far less than with 'simple' IDCT. Also, Koepi mentioned that he leaves IDCT at autoselect. When one do this IDCT for XviD is 'normal'. Why is that? Isn't it better and more logical that IDCT is set to 'XviD'? All this has been tested with the new unofficial ffdshow builds compiled by athos. Edit: Oh, I haven't seen that sysKin's post but it looks that I'm right about ffdshow not selecting the correct IDCT, which is a good news because it should be easy to fix. mfluder Last edited by mfluder; 12th December 2003 at 16:14.

12th December 2003, 16:20	#17 \| Link
yaz n00b ever Join Date: May 2002 Posts: 627	@didée i've made some tests so as to get the taste of the new release (1.0b2). my first intention was ... horrible. after a more through investigation i found some strange(?) thing. 1st of all, it's not horrible at all :-) in general, i found the output quite pleasant. great work, devels ! thx i've tested 5 different sources so as to see the probs listed above & i found that the compression was always higher (1st pass size is smaller than with devapi3, checked with statsreader) but the quantizer distribution of the 2nd pass was narrower(no such tailing towards higher quants). by checking 'the last flight of osiris' i found the duell scene (lottsa detailes, fine textures) much(!) better but the ship scenes (dark, shady, less detailed) somewhat worst than with devapi3. so the 'detail loss' effect really exists but it has a special taste :-) 'static floating' : still(!) exists, i had this prob with devapi3 too. former 'solution' (unfilter(+,+), blockbuster(noise), mpeg quant) did not work here. adding noise as postproc made it almost invisible. summing up so far : it seems as if the new codec would remove more from where it's less but less from where it's more (???) other probs as u descibed. in add, there is this 'green blocks on orange surfaces' problem annoying me the most. i can do only chroma-noising against it on the postproc level. further, i played a bit with the decoders. ffdshow/libav(idct=xvid) : the effects are the most pronounced. xvid : the effects are visible but much less. by swithing off both (removing xvid.ax), mpc drops a filter chain like this : wmr9 (renderless) color space converter avi decoder(xvid) the last 2 greyed out. with this i only found the effects just because i knew they were there. so i'm sure, a relevant part of the problems listed is decoder related. i hope it tells sg to the devels, cus i'm a bit confused with it at the moment. the bests y [EDIT] i didn't play with xvid settings, i used the same as before, exept for trellis. switching it on&off did not solved the problems.

16th December 2003, 10:22	#19 \| Link
yaz n00b ever Join Date: May 2002 Posts: 627	@didée ffdshow : i've just tried the last 4 versions from athos (200308xx-200311yy). 08 & 11 worked fine while 10 was somehow 'unstable'. the quality of 09 was the worst. all produced some kinda smearing but to different extent. i-frame pumping: is it a decoder problem ? idct: other than xvid produced quality loss. the bests y [EDIT]in the meantime i installed the very last athlon compile from gamrdev & koepi's b3 decoder. it improved the output definitely. i should start my tests over :-)