The Ultimate Flickering Luma Block Thread

[Kagura]

Before anyone says to use the search button (this subject has been covered, although scattered among different threads), let me start by saying that the problem has not been solved yet. Anyways, it's better to start from a clean slate anyways as this is probably one of the top three encoding problems yet unsolved nowadays and newer filters may have found a solution.

Oh, this thread doesn't belong in the xvid forums because although xvid does enhance and exacerbate the problem, the blocks are inherent in the source.

The problem: Blocks in dark scenes

More specifically: A while ago, I compared a few of my encodes between my CRT and my LCD monitor. There was a world of difference between them. Not only was the CRT richer and more vibrant in colour, the LCD also had horrendous blocking on dark scenes.

Going through pixel by pixel, I found that both light areas and dark areas (this is anime, btw) of what should be solid colours had minute fluctuations of value. The luma (haven't tested chroma yet) had value differences of just 1 in those areas. While the bright scenes, because of the contrast differences, showed up as "solid" to the eye, the dark scenes, which had the same difference in luma values, exhibited horrendous blocking. Mind you, this isn't your typical blocking. Rather, they can appear as either flickers or solid static blocks staying through motion. They are also rather large. For example, in a patch of dark black (a character's hair), the left half might have a value of 23 while the right half might have 24. Thus, no filters will touch the differences between the two because a) the area is too large for small radii or b) it is detected as a natural "gradient" and not touched.

Experimentation: I experimented with the "established" ways to solve these problems. I tried the lumafilter/unfilter "solution," which only split the dark scenes into smaller "swimming sea" of colour that was off by 1 pixel (still too large to filter). Blockbuster does the same thing with blur and dither and has an even worse effect with noise. Because of that, I don't think this problem can be classified as purely "blocks" but rather... "flickering/static sea of minute luma differences"?

Anyone?

[FredThompson]

LCD is junk compared to CRT, regardless of what advertising hype claims.

Additionally, your eyes are more keen to detect these types of changes in the images when there are right angles and lossy compression is very visible in near-blacks. You'll see the same thing in any area of highly similar colors. It's just most noticeable in near-blacks.

There is no easy solution other than increased bandwidth or a lossless source.

Consider it the equivalent of dust on vinyl records or something like that. It's the inherent junk of digital compression.

[Leak]

Originally posted by FredThompson
LCD is junk compared to CRT, regardless of what advertising hype claims.

Huh? LCDs/TFTs are junk because you see more artifacts that have been encoded into the video? Now that's a strange way of reasoning if I've ever seen one... :confused:

Care to explain this a bit further?

[FredThompson]

YOU stitched the statements together, I didn't.

LCD doesn't have the subtle range of CRT yet.

Human vision doesn't work on a grid system.

Nature has very few right angles.

The nature of block-based lossy compression yields a form of artificial motion.

Human vision is most centric on motion.

All those combine to make us more sensitive to unnatural blobs of uniform color where they wouldn't exist in nature.

--

I tried all kinds of smoothing techniques and noise reduction in near-blacks for months. There is no real solution for this. It's a lack of information in the source. LCD just makes it more visible.

Well, OK, the entire image could have a false texture applied and the edges of the blobs would become less visible but that's about as much a solution as smearing petroleum jelly all over the screen.

[JasonFly]

I 've also tested many filters to decrease this blocking problem but I haven't got any good result. The only thing I found to workaround that is to encode at higher resolution(1920, 1280). But that means that you're forced to switch to DVD because that require a lot more space. This is the drawback, but after that, no more blocks(at least they are far less visible), the quality is amazing.

[esby]

I do agree that LCD will make the rendering worse.
That is true for some trinitron based monitor too,(at least some I have seen).
Now I don't know the solution, the best I can think is not watching material with this type of 'faulty' monitor.

esby

[Zarxrax]

One thing I have done on some recent encodes is to adjust the levels so the blacks arent as black. Pretty light settings like Levels(14,1,255,0,255)

This is acceptable to me because I really can't tell the levels have been adjusted unless I compare it directly to the source, and it alleviated the blocking a LOT in my case.

[Mug Funky]

That is true for some trinitron based monitor too,(at least some I have seen)

really? the ones i use make everything look nice. i have to get it on a computer monitor (LCD or otherwise) just to see how bad the source really is... and what also sucks is the "blue only" mode on the new trinitrons is actually blue instead of a B&W image of the blue channel... this makes it useless for spotting flaws... grrr sony. i could punch them.

and another thing... why are the $80,000 sony digibeta decks not multiformat? they're physically identical, but you can't play NTSC with the PAL decks. double grrr sony.

back on topic...

blocking in dark areas happens because the codec is assuming a linear response, but CRTs, LCDs and humans are all nonlinear, favouring the dark tones. assumption is the mother of all truck ups i suppose. FWIW i've not seen dark blocking on QuEnc, HC, or latest Xvids. but i haven't been doing many encodes lately, and my eyes are crap.

[Kagura]

Well, the blocks are not just present in dark scenes. They are in areas of what should be solid colour but instead have pixels that are off in rgb space by +/- 2 for each channel and off by +/- 20 in YV12 colourspace. The only reason that it kinda looks like solid colour on CRTs is that the luma and chroma average out to produce a close enough colour to the original. It also helps that CRTs don't render contrast as much in those places.

If I'm messing up my technical monitor related stuff, someone correct me. I do not claim to have detailed knowledge of how the different display types work. LCDs just have a tendency of amplifying the "shit" on encodes, or so I've found. Everything is more obvious, the edge noise, the blocks, the luma stuff that I'm discussing in this thread, halos, oversharpening, weak lines, etc.

Well, after much experimentation and discussion with other encoders, I've found what seems to be an acceptable solution that uses a combination of the following:

Zarxrax's levels for correction on dark scenes. Reduce the first value to 5 as 8 is slightly too destructive for my eyes.

blockbuster (method = "noise")
blockbuster (method = "blur")
dnr2(5)

The first two will kill much of the blocks and the dnr will stop it from swimming as much.

If all comes to nothing, isolate the scene and create a b&w mask of the "solid colour." Then, use yv12lut (or one can redraw the scene) to clamp everything in a certain range of luma and chroma to a specified value. Then, mask that back into the original clip so that the the rest of the scene isn't harmed. This is a lot of work though.

[communist]

Originally posted by Mug Funky
and what also sucks is the "blue only" mode on the new trinitrons is actually blue instead of a B&W image of the blue channel...
Thats probably not new - just about every professional (PAL/NTSC) studio monitor I came across shows a blue/black picture in blue-mode only.. though I never really needed it (PAL user). If you need b/w just tone down the saturation ;)