XviD with BLACK Solid Background looks Terrible !!

bobby8798 · 9th February 2003, 04:19

Tried encoding 1-pass XviD with colorful moving objects in the foreground against a PITCH BLACK Background

used 3000 kbits (increasing bit rate has zero impact)
changing i-frame quantizer has zero impact (even tried 1-2)
using latest koepi unstable build XviD-02022003-1
tried h263 and mpeg
1 pass CBR, or 100% quality, or quantizer=3
lumi-masking OFF; on is just as bad
CBR smoother from 0-100, no impact

compared this to DivX3, and DivX is far superior than XviD !!!

the XviD encode has a lot of noise in the background which looks really horrible. looks like color from the foreground is leaking everywhere. Extremely blocky color leakage !!!
The divx encode looks very clean. It has only about 10% of the XviD noise.

If the background is lighter gray, the situation REVERSES and the XviD is far superior than the Divx. It seems like XviD has a very serious limitation against SOLID DARK colors ???

anyone know how to fix this?

Y7 · 9th February 2003, 05:38

Use a filter to adjust the levels so that you have more black. Does 2-pass help?

Zarxrax · 9th February 2003, 07:41

Maybe you could post a clip to show what you are talking about?
I don't think xvid has any trouble with black, as I just created a test video myself to see if I could cause some trouble in the codec (pure black background with colored text zooming all over the place) and for all quantizers below 8, it looked very nice.
Probably you just need to filter your source some, maybe adjust the levels like Y7 said. Spatial and temporal cleaners might help as well.

Teegedeck · 9th February 2003, 12:20

Yup; post a screenshot, please.

Koepi · 9th February 2003, 13:45

Or even better: search the avisynth forum for "luma offset".

Use marc's mpeg2dec3.dll and add this to your avisynth scripts:

LumaFilter()
UnFilter(-5,-5) # - is smoothing, + is sharpening

Regards
Koepi

9th February 2003, 13:49

maybe you should use the refdivx's lumimasking code (koepi's build ^^)

great for dark scene ^^

edit: oops just seen: " just as bad" --->

iago · 9th February 2003, 14:39

Listen to Koepi. That issue was discussed extensively in the past and a working solution was finally proposed/achieved in the "Lumi Masking" thread in this very XviD forum.

regards,
iago

Use LumaFilter(-2) / LumaFilter(-3) / LumaFilter(-3,0.8) etc. depending on the source, together with UnFilter.

Lothar · 9th February 2003, 16:27

Thanks for the info, it's the same problem I posted about some time ago

Now I'm gonna test this solution, thanks again

bobby8798 · 9th February 2003, 20:30

LumaFilter cleans the noisy blocks !!!

It tends to brush out and use one standard color to remove the blocks and noise.

LumaFilter works nicely to remove the noise in a dark background. I will try a setting of (-1) instead of the default of (-2) because I don't want some of the fine details to have its color transformed to black. LumaFilter also tends to add more of those tiny white DOTs.

I didn't try the UnFilter, because it was not part of the avisynth v.250 package.

Thanks to everyone for being so helpful

angelyote · 9th February 2003, 23:24

Here's the thread everyone's referencing.
It's well worth reading and is a who's who of doom9 personalities.

educate me

Dave

McQuaid · 10th February 2003, 05:45

I also was recently dealing with that issue. I tried block buster and its seems to have solved the problem quite nicely. I did not try the luma/unfilter trick yet though. The reason I thought block buster would be better suited for this is because it is selective on where it adds noise instead of unfilter doing this everywhere.

Or am I not understanding the effect of the combination of luma/unfilter?

But even though a workaround has been found for this. It's strange the problem is there much more in mpeg4 than mpeg1. I had a tv clip that had this problem and the mpeg1 encoding barely showed the blocks whereas xvid had this problem no matter what I set the bit rate at.

MaTTeR · 10th February 2003, 05:52

Quote:

Originally posted by McQuaid
Or am I not understanding the effect of the combination of luma/unfilter?

Well actually none of us have yet to figure out why it works so well. lol

Quote:

But even though a workaround has been found for this. It's strange the problem is there much more in mpeg4 than mpeg1.

I've been scratching my head about the same thing. All the MPEG-1 & 2 encoding I've done never really show the blocks during playback but it's a constant battle with all the MPEG-4 codecs

lamer_de · 10th February 2003, 19:27

Quote:

Here's the thread everyone's referencing.
It's well worth reading and is a who's who of doom9 personalities.

But it is aswell 14 pages of tech talk. I don't have either the time nor the concentration to read up such a huge text just to find the solution on page 10 of 14. I have no problem with reading 10 pages of "me too" or "Great work", but such threads where every post could contain the information and which deal with very technical settings are just over my head. This is aswell true for some other popular threads, which reach up to 30 pages or more.
Sure, the information is there, but a more "user-friendly approach" (you could call it aswell "cheap solution for the lazy bums") wouldn't hurt in my eyes (like in this thread, it saved me reading the 14 pages). Maybe if a solution is found on such topics, it could be added to the first post or made a link to the post which contains the solution. Or made a sticky for some time?

Only my 2 cents,
lamer_de

bobby8798 · 10th February 2003, 20:59

It would be nice if threads like that were deleted, and the conclusions / solutions drawn be added to an "XviD Tips & Tricks"
Sticky Post

Koepi · 10th February 2003, 22:36

Delete all that knowledge? Are you serious?

Sometime later we might need such infos.

Not believing what I read,
Koepi

Belgabor · 11th February 2003, 00:09

But this part

Quote:

Originally posted by bobby8798
solutions drawn be added to an "XviD Tips & Tricks"
Sticky Post

is actually a good idea (imho).

McQuaid · 11th February 2003, 00:10

Well, I've been trying SansGrip's Blockbuster a little more and have had amazing results with this setting:

Blockbuster(method="dither",block_size=3,luma_offset=-2,luma_threshold=30,detail_min=1, detail_max=30)

Although, I saw someone else post the block_size=3 and gave it a try, but I think it should divide evenly into the frame, no?

On the encode I just tested of a TV cap of a cartoon, I had to look out for dark blocks watching through my TV, while normally they are so glaring it's unacceptable.

Dither seems to be the key, as it stop the blocks from having a 'life of their own' (i.e. moving around or flickering where the image is relatively static).

SansGrip mentions in the readme that dither is experimental so maybe this can still be improved.

The place where I see blocks the most now is in somewhat bright solid colour areas. Instead of increasing the variables, I wonder if blockbuster can be modified to tackle a frame at both ends of the luma scale?

Anyways try it out and report back your findings.

Get his plugin here.

Teegedeck · 11th February 2003, 07:34

Give the 'sharpen'-method (instead of 'dither') a try; very nice, too.

drebel · 11th February 2003, 13:10

Removing existing mpeg2 macroblocs and not reproducing mpeg4 marcoblocks are two completely different things...

Blockbuster performs the first one, lumafilter+unfilter make sure about the second
I'd go with Teegedeck on this one : "sharpen" seems to be the key for me
regards,
george

PS: u could try the luma_offset=-2 option of blockbuster with unfilter(+x,+x) as a combination of both

McQuaid · 11th February 2003, 21:51

Ok, I'm trying out sharpen right now to see the results.

Regarding luma filter/unfilter.

Almost every post I've seen is from someone using the luma function that's part of mpeg2dec3.

I'm doing tv captures so obivously, I am not using mpeg2dec3. How do I offset luma without it, if I'm not using blockbuster.

I saw one post using lumafilter(), but I can't find anything on this. Is this part of aviysynth 2.5? I'm still using 2.07.

9th February 2003, 04:19	#1 \| Link
bobby8798 Registered User Join Date: Aug 2002 Posts: 67	XviD with BLACK Solid Background looks Terrible !! Tried encoding 1-pass XviD with colorful moving objects in the foreground against a PITCH BLACK Background used 3000 kbits (increasing bit rate has zero impact) changing i-frame quantizer has zero impact (even tried 1-2) using latest koepi unstable build XviD-02022003-1 tried h263 and mpeg 1 pass CBR, or 100% quality, or quantizer=3 lumi-masking OFF; on is just as bad CBR smoother from 0-100, no impact compared this to DivX3, and DivX is far superior than XviD !!! the XviD encode has a lot of noise in the background which looks really horrible. looks like color from the foreground is leaking everywhere. Extremely blocky color leakage !!! The divx encode looks very clean. It has only about 10% of the XviD noise. If the background is lighter gray, the situation REVERSES and the XviD is far superior than the Divx. It seems like XviD has a very serious limitation against SOLID DARK colors ??? anyone know how to fix this?

9th February 2003, 12:20	#4 \| Link
Teegedeck Moderator, Ex(viD)-Mascot Join Date: Oct 2001 Posts: 2,564	Yup; post a screenshot, please. __________________ It's a man's life in Doom9's 52nd MPEG division. "The cat sat on the mat." ATM I'm thoroughly enjoying the Banshee - a fantastic music player/ripper for Linux. Give it a whirl!

9th February 2003, 13:45	#5 \| Link
Koepi Moderator Join Date: Oct 2001 Location: Germany Posts: 4,454	Or even better: search the avisynth forum for "luma offset". Use marc's mpeg2dec3.dll and add this to your avisynth scripts: LumaFilter() UnFilter(-5,-5) # - is smoothing, + is sharpening Regards Koepi __________________ Koepi's new media development site

9th February 2003, 13:49	#6 \| Link
Sigmatador Guest Posts: n/a	maybe you should use the refdivx's lumimasking code (koepi's build ^^) great for dark scene ^^ edit: oops just seen: " just as bad" ---> Last edited by Sigmatador; 9th February 2003 at 13:53.

9th February 2003, 16:27	#8 \| Link
Lothar Registered User Join Date: Apr 2002 Location: Italy Posts: 24	Thanks for the info, it's the same problem I posted about some time ago Now I'm gonna test this solution, thanks again __________________ Never loose that feeling

9th February 2003, 05:38	#2 \| Link
Y7 Registered User Join Date: Jan 2003 Posts: 5	Use a filter to adjust the levels so that you have more black. Does 2-pass help?

9th February 2003, 07:41	#3 \| Link
Zarxrax Registered User Join Date: Dec 2001 Posts: 1,219	Maybe you could post a clip to show what you are talking about? I don't think xvid has any trouble with black, as I just created a test video myself to see if I could cause some trouble in the codec (pure black background with colored text zooming all over the place) and for all quantizers below 8, it looked very nice. Probably you just need to filter your source some, maybe adjust the levels like Y7 said. Spatial and temporal cleaners might help as well.

9th February 2003, 14:39	#7 \| Link
iago retired Join Date: Jun 2002 Location: hollywood Posts: 1,013	Listen to Koepi. That issue was discussed extensively in the past and a working solution was finally proposed/achieved in the "Lumi Masking" thread in this very XviD forum. regards, iago Use LumaFilter(-2) / LumaFilter(-3) / LumaFilter(-3,0.8) etc. depending on the source, together with UnFilter.

9th February 2003, 20:30	#9 \| Link
bobby8798 Registered User Join Date: Aug 2002 Posts: 67	LumaFilter cleans the noisy blocks !!! It tends to brush out and use one standard color to remove the blocks and noise. LumaFilter works nicely to remove the noise in a dark background. I will try a setting of (-1) instead of the default of (-2) because I don't want some of the fine details to have its color transformed to black. LumaFilter also tends to add more of those tiny white DOTs. I didn't try the UnFilter, because it was not part of the avisynth v.250 package. Thanks to everyone for being so helpful Last edited by bobby8798; 9th February 2003 at 21:14.

9th February 2003, 23:24	#10 \| Link
angelyote The frumious Bandersnatch Join Date: Sep 2002 Posts: 78	Here's the thread everyone's referencing. It's well worth reading and is a who's who of doom9 personalities. educate me Dave

10th February 2003, 05:45	#11 \| Link
McQuaid Registered User Join Date: Feb 2002 Posts: 93	I also was recently dealing with that issue. I tried block buster and its seems to have solved the problem quite nicely. I did not try the luma/unfilter trick yet though. The reason I thought block buster would be better suited for this is because it is selective on where it adds noise instead of unfilter doing this everywhere. Or am I not understanding the effect of the combination of luma/unfilter? But even though a workaround has been found for this. It's strange the problem is there much more in mpeg4 than mpeg1. I had a tv clip that had this problem and the mpeg1 encoding barely showed the blocks whereas xvid had this problem no matter what I set the bit rate at.

10th February 2003, 20:59	#14 \| Link
bobby8798 Registered User Join Date: Aug 2002 Posts: 67	It would be nice if threads like that were deleted, and the conclusions / solutions drawn be added to an "XviD Tips & Tricks" Sticky Post

10th February 2003, 22:36	#15 \| Link
Koepi Moderator Join Date: Oct 2001 Location: Germany Posts: 4,454	Delete all that knowledge? Are you serious? Sometime later we might need such infos. Not believing what I read, Koepi __________________ Koepi's new media development site

11th February 2003, 00:10	#17 \| Link
McQuaid Registered User Join Date: Feb 2002 Posts: 93	Well, I've been trying SansGrip's Blockbuster a little more and have had amazing results with this setting: Blockbuster(method="dither",block_size=3,luma_offset=-2,luma_threshold=30,detail_min=1, detail_max=30) Although, I saw someone else post the block_size=3 and gave it a try, but I think it should divide evenly into the frame, no? On the encode I just tested of a TV cap of a cartoon, I had to look out for dark blocks watching through my TV, while normally they are so glaring it's unacceptable. Dither seems to be the key, as it stop the blocks from having a 'life of their own' (i.e. moving around or flickering where the image is relatively static). SansGrip mentions in the readme that dither is experimental so maybe this can still be improved. The place where I see blocks the most now is in somewhat bright solid colour areas. Instead of increasing the variables, I wonder if blockbuster can be modified to tackle a frame at both ends of the luma scale? Anyways try it out and report back your findings. Get his plugin here.

11th February 2003, 07:34	#18 \| Link
Teegedeck Moderator, Ex(viD)-Mascot Join Date: Oct 2001 Posts: 2,564	Give the 'sharpen'-method (instead of 'dither') a try; very nice, too. __________________ It's a man's life in Doom9's 52nd MPEG division. "The cat sat on the mat." ATM I'm thoroughly enjoying the Banshee - a fantastic music player/ripper for Linux. Give it a whirl!

11th February 2003, 13:10	#19 \| Link
drebel Registered User Join Date: Nov 2002 Location: Athens,Hellas Posts: 152	Removing existing mpeg2 macroblocs and not reproducing mpeg4 marcoblocks are two completely different things... Blockbuster performs the first one, lumafilter+unfilter make sure about the second I'd go with Teegedeck on this one : "sharpen" seems to be the key for me regards, george PS: u could try the luma_offset=-2 option of blockbuster with unfilter(+x,+x) as a combination of both Last edited by drebel; 11th February 2003 at 13:16.

11th February 2003, 21:51	#20 \| Link
McQuaid Registered User Join Date: Feb 2002 Posts: 93	Ok, I'm trying out sharpen right now to see the results. Regarding luma filter/unfilter. Almost every post I've seen is from someone using the luma function that's part of mpeg2dec3. I'm doing tv captures so obivously, I am not using mpeg2dec3. How do I offset luma without it, if I'm not using blockbuster. I saw one post using lumafilter(), but I can't find anything on this. Is this part of aviysynth 2.5? I'm still using 2.07.