Severe color banding after resize

[brett]

A lot of times I'll notice some greenish color banding in darker scenes, but I found a clip that gives me truly horrible results in DivX 4.11. What I don't understand is why the banding only appears after I scale the image down.

http://home.columbus.rr.com/luna/drip_cap_mj19.jpg

drip_cap_mj19.jpg

This is the original capture. I captured straight to PicVideo MJPEG quality 19 using VDub 1.4.7. I used YUY2, although I later read this gives washed-out colors in MJPEG. The colors look good here, though.

http://home.columbus.rr.com/luna/drip_deint_q100.jpg

drip_deint_q100.jpg

The same clip encoded with DivX 4.11 100% quality. Results are similar using 2-pass encoding at various bitrates. The only filter used was the Smart Deinterlacer. The clip still looks great.

http://home.columbus.rr.com/luna/drip_deint_resize_q100.jpg

drip_deint_resize_q100.jpg

Same as above except resized. Filter order is (1) smart deinterlacer, (2) null transform to crop a few lines, (3) precise bicubic resize, A=-0.60. Banding is the same with and without cropping, and I have tried all bilinear and bicubic resize filters. Why does this banding show up?

http://home.columbus.rr.com/luna/drip_deint_resize_mj19.jpg

drip_deint_resize_mj19.jpg

This clip was produced the exact same way, except I re-encoded to MJPEG instead of DivX. The banding is gone.

http://home.columbus.rr.com/luna/drip_deint_resize_mj_then_q100.jpg

drip_deint_resize_mj_then_q100.jpg

Just to be sure, I tried re-encoding the resized MJPEG clip as DivX. Banding is once again introduced. If the banding is the fault of the DivX codec, why didn't it show up before I resized the clip?

[b0b0b0b]

I think that's called posterization.

[lar1r]

Quick question - what video mode should be used when using MJPEG?
I use YUY2 now.

[b0b0b0b]

I think Steady discovered a MJPEG bug somewhere related to converting between RGB and YUV... maybe in virtualdub maybe in avisynth.

Has to do with images coming out too dark or something.

[brett]

No, that's not posterization. Posterization is when you reduce a picture to a very low number of colors without dithering, which will cause banding in gradients, but it's a much more drastic effect which really doesn't have anything to do with video editing.

[brett]

Ideally, you should use YUY2 when capturing video in Huffy or MJPEG, but some people say there is a bug in MJPEG that causes colors to look washed-out (although I never noticed) so you should use 24-bit RGB for MJPEG.

So, nobody has a clue why I'm getting these weird results after I resize and encode to DivX?

[b0b0b0b]

what are your resize-to dimensions?

[b0b0b0b]

Also I think posterization is a real problem in video display. I think you're thinking more of extreme posterization applied as a photoshop effect.

[brett]

I thought the resolution couldn't have anything to do with it, but I tried some testing and found some interesting stuff.

During the capture, I cropped to 640x440, since there's some distortion in the edges and it looks weird having so many extra lines below the network logo.

In all my initial testing, I was cropping to 640x427 (1.50:1), then resizing to 480x320 (1.50:1).

For higher quality, I'll often take my 640x440 capture (1.46:1) then resize to 512x352 (1.46:1). I tried that with this clip, and everything looked fine, no banding.

I redid it by taking the 640x440 capture (1.46:1), then resizing to 480x330 (1.46:1), which isn't divisible by 8, so I re-cropped to 480x320. It still looked horrible. No matter what order I put the filters, a final resolution of 480x320 looks horrible if encoded in DivX.

I tried it again by resizing to 384x256, and everything looked fine. Hmm.

Just to make sure it wasn't something about the resizing filter when going to 480x320, I tried simply cropping the video to 480x320. It looked horrible again. It didn't matter what height I cropped to, as long as it was 480 wide. 480x416 looked horrible. 480x288 looked horrible.

As soon as I tried a different horizontal resolution, though, it looked fine again, even just cropping it down to 448x288.

Has anybody else encountered a bug with encoding DivX with a horizontal resolution of 480? I've tried it with 4.11 and 4.12.

[b0b0b0b]

You should submit a bug report.

I've never tried encoding at 480 horiz resolution, but I believe what you're seeing.

[Ookami]

> I think Steady discovered a MJPEG bug somewhere related to converting between RGB and YUV...

A bug in what? VirtualDub? Or in a MJPEG codec? If second, what codec (Picvideo, Leadtools...)?

[b0b0b0b]

my bad.... blight noticed it and steady wrote a filter to adjust for it:

http://forum.doom9.org/showthread.php?s=&threadid=9997&highlight=mjpeg