Horrible Video Noise

[Thracks]

Hello everybody!

I guess this is the right section, but I'm not too sure. I've been playing around with encoding AVI to DVD lately, as I've gotten the AVI down myself a long time ago, and I thought maybe I would start working on encoding in the other direction.

I've been using a copy of Star Wars Episode V that I ripped for myself a while back. The basic stats for it are:

1406kbps video (XViD)
192kbps stereo AC3
640x288 (2.22:1)

If you gents need any other stats about the video (Such as how I encoded it), I'll be glad to post it.

Granted, I know converting from an original source to XViD back to DVD isn't the best idea in the world as it's lossy the entire way, but I don't think the errors introduced by this project are very typical.

Here's a shot of the original XViD frame. Looks pretty crisp, no glaring video artifacts...

XViD Source Frame (PNG) (http://thrax.warp1.net/StarWars(XViD).png)

Here it is again with the contrast turned way up to show the lack of artifacts in the frame. I also turned the contrast up because this problem only appears on a TV (Brighter black), or when you turn the contrast up on the video card's overlay to simulate a TV. Notice the black starfield is still very clean, looks great even.

High Contrast XViD Source Frame (PNG) (http://thrax.warp1.net/StarWars_High_Contrast(XViD).png)

Now here is the same frame captured from a normal contrast, and then again in a higher contrast.. This time the capture comes from the encoded MPEG:

MPEG Source Frame (PNG) (http://thrax.warp1.net/StarWars(MPEG).png)

High Contrast MPEG Source Frame (PNG) (http://thrax.warp1.net/StarWars_High_Contrast(MPEG).png)

Notice the hideous compression artifacting that appears all over the black? On a TV, it looks like bright green blobs swirling and jiggling around the screen. ANY time there's a scene with a high black content, this problem rears its head.. Throughout the opening credits crawl, in space scenes, so on and so forth. On bright images, the picture's just fine.

Now here's the lowdown of my encode. I'm using CCE SP and AviSynth v2.55.. I'm using this AVS:

Import("LimitedSharpen.avs")
LoadPlugin("H:\Programs\VCD\AviSynth 2.5\plugins\warpsharp.dll")
LoadPlugin("H:\Programs\VCD\AviSynth 2.5\plugins\MaskTools.dll")
LoadPlugin("H:\Programs\VCD\AviSynth 2.5\plugins\VagueDenoiser.dll")
AVISource("test.avi")
VagueDenoiser(threshold=0.8,method=1,nsteps=6,chromaT=0.8)
Limitedsharpen(ss_x = 1.5, ss_y = 2,dest_x = 720, dest_y = 320,strength = 300)
AddBorders(0, 80, 0, 80, $000000)

Just using Didee's limited sharpening and the appropriate plugins, and running the suggested denoising pass before the plugins do their thing. This script is loaded into CCE SP, and is processed with the following settings:

Multi-pass ES VBR mpeg2: 0-9532kbps, avg 4358
Picture complexity set to max difficult.
Top field first.
Open GOP.
Zigzag block scanning order.
Default quantization matrix.
Progressive flag.
Luminance 16-235.

Even when the scene is PURE BLACK, those ugly green/blue blocks are everywhere. I thought maybe that CCE might be at fault, so I ran the script through TMPGEnc too, and the result was exactly the same.

I've tried three different versions of AVISynth (2.55, 2.56 and the newest alpha). I've also tried forcing colourspaces (Input_Pixel=RGB), Converttoyuy2(). I've tried forcing the CIRC601 (I think that's what it is) luminance with the PC->TV commands, I've tried multiple versions of the XViD codec.. So on and so forth.

Even with the lossy conversion process, I can't image the result looking that bad. Anyone have any ideas?

//EDIT: I should note that even when the script was as basic as:

AVISource("test.avi")
Lanczos4REsize(720,320)
AddBorders(0, 80, 0, 80, $000000)

The problem was still there.

[Pookie]

Have you tried just feeding the AVI directly into your encoder to rule out AVISynth ?

[Thracks]

Aye, I forgot to mention that. It still occurs.

[Mug Funky]

Picture complexity set to max difficult.

is this the slider that gives priority to either complex or flat scenes?

a source without much noise (like an xvid) will be mostly flat areas as far as CCE is concerned, so changing this setting accordingly might help (i always just leave it in the middle...).

[Thracks]

Still gives me the hideous encode. :(

[Mug Funky]

hmm. how did you capture the first xvid bit? it looks like it's been clamped compared to the mpeg screencap.

maybe there's a problem with colour ranges going on? what colourspace are you feeding CCE with? (i assume yuy2 or yv12, but avisource can sometimes give other formats depending on system config. it can be forced with "avisource("blah.avi",pixel_type="YV12").

it's very possible the wiggling blobs are in the xvid encode, but the noise is just being clamped off on playback. if you've got a TV output on your viddy card, try play the xvid straight to your TV and see what happens.

try this script:

Import("LimitedSharpen.avs")
LoadPlugin("H:\Programs\VCD\AviSynth 2.5\plugins\warpsharp.dll")
LoadPlugin("H:\Programs\VCD\AviSynth 2.5\plugins\MaskTools.dll")
LoadPlugin("H:\Programs\VCD\AviSynth 2.5\plugins\VagueDenoiser.dll")
AVISource("test.avi", pixel_type="yv12")
VagueDenoiser(threshold=0.8,method=1,nsteps=6,chromaT=0.8)
Limitedsharpen(ss_x = 1.5, ss_y = 2,dest_x = 720, dest_y = 320,strength = 300)
AddBorders(0, 80, 0, 80, $000000)
converttoyuy2()

not sure if vaguedenoiser is needed, either. you could enable deblocking in xvid's playback settings and get a similar result but faster. any noise still present in a low-bitrate MPEG-4 encode can easily be handled by a high bitrate mpeg-2 encode. you might also consider adding noise with BlockBuster, as it's designed to tackle this kind of thing (blocks in dark scenes), but will increase bitrate requirements a little.

[scharfis_brain]

The comparision is invalid, because both contrast enhanced images were not enchaned by the same amount.

in the first image the black levels are cutted away, while they are clearly showed in the 2nd one.

so redo the contrast enhancement with equal settings!

[Thracks]

Hey chaps, thanks for the help thus far.

For all four screenshots, I captured the D3D overlay from the videocard with hypersnapDX. On the MPEG, I merely cut off the black letterboxing to save some space.. Curiously enough, the artifacting doesn't appear in the letterboxing, but does in the video. Maybe that's not curious at all, not sure.

Unfortunately, I have no way of getting a direct video out to the TV.. Wish I did.

Basically, upping the video gamma on the D3D overlay on the XViD shows no evidence of the artifacting. The minute I encode and pop up that gamma in the overlay to the same levels, the artifacts are very clear.. The artifacts are EXTREMELY clear on my TV, thanks to the higher natural brightness of the TV display. As for colorspaces..

I've tried forcing YUY2 since CCE doesn't deal with YV12 (Damn...), I've also tried telling AVISynth that the source is RGB, and then forcing YUY2 and so on. I'll try your script and see if I get lucky.. Thanks Mug Funky

----

Scharfi..

The comparison is valid, as I only increased the gamma on the mpeg to clearly show the artifacting. If I had run the same gamma test on both pictures, the artifacting on the MPEG would not have shown, even though it's clearly there when on a TV or stared at closely on the PC monitor.

I understand apples-to-apples comparison, but I still think this one is fair.

//EDIT:

It appears that CCE handles YV12 imput, but can't process in YV12 space. The script processes okay, and the encode starts. I'll let you guys know what happened when it's through.

//EDIT #2:

Alright, tried your script mug, and the green blocks still crawl all over the damn scene. When I load the script and process it real time with Media Player Classic, looks great.. Even when I adjust the gamma levels to reveal the artifacting. The blocks pop up in encoding, and it doesn't matter if I use TMPEG or CCE.

Is it possible that the XVID codec is rendering out the noise? I don't have any deblocking or decoding processing enabled.