Reducing Blocks in Dark Areas?

[futurex]

Hello,

i am encoding cinderella man @ 1436kbps, 704x288. looks great :)

but there is absolutely horrible deblocking in dark areas, which really ruin the experience. they are very noticeable on a 1280x1024 monitor. using R417, latest x264-megui. the settings are sharktooth's HQ-slowest profile, tweaked a little to my taste (decreased deblocking to -3, -2 to get rid of the airbrushed look)

any ideas?

example (i resized the image to make it easier to see. but it gets much worse in other scenes and at 1280x1024):

http://img149.imageshack.us/img149/1509/snapshot200602122222167zf.jpg

[foxyshadis]

Blockbuster was designed to mitigate this. AQ may or may not help, ut blockbuster definitely will if you can live with a bit of graininess in those areas.

[DarkZell666]

Ugliness in dark areas is a known "problem" with x264 (no idea about AVC in general).

Actually the problem extends to "low-contrast" areas (which dark areas also are ;))

Is it really blocking or is it more like smearing/excessive blurring ?

If it's smearing : use lower values for deblocking (e.g. -6:-6) or disable it all together
If it really is blocking : try using the --no-fast-pskip switch.

This option has been introduced because x264 has always used a fast (and less accurate) pskip detection (leading to blocks since nearly identical blocks were condidered as identical) ... and because this issue had already been reported ;)

[foxyshadis]

If you turn the monitor brightness up you can pretty clearly see the mess it makes in the lower right corner, xvid does the exact same thing. It's like a really blocky form of dithering. It's not bad macroblocks, which pskip causes; it has to do with the DCT and the quant matrices. The Blockbuster manual (http://mf.creations.nl/avs/filters/Blockbuster.html) has more information and sadly no pictures.

[AlexB17]

Maybe its an artifacts of noise reduction coded, or bad source?

[shon3i]

I using blockbuster("dither") long time but some tell me that LumaYV12(-2) and Unfilter(-5,-5) have good efect

[dude051]

Skip to the bottom if you dont want to read my long-winded stuff :p

This is a common problem in the entire MPEG4 spec, mainly its DCT routine. It's caused because the encoders have a natural taste to place less bits in areas with less contrast ('dark areas') because the human eye usually can't see this. If you have a higher brightness and less deep blacks in your monitor.. you will see them. To understand how to get rid of the things though, you have to understand what causes them. A few are:
1. Original blocks in source.
2. Lack of detail or range of detail in that area.

So this makes two things that usually help (not fix, it's near impossilbe to make it not block at all, you can simple reduce it). That is, deblock the orignal source with something like DeBlock(), or SmoothD(). There are many threads on this topic, it's an on going fight, just search for it. Secondly for the latter, you need to add detail, a.k.a. grain, to the affected area. Like said above, the trick LumaYV12(-2) sometimes works... but this isn't a sure go thing. Unfilter(-5,-5) is the equivelant of UnDot(), and works good for denoising a source... we don't want this though. In this situation you want detail. Blockbuster is a great idea, but decreases the compressability of your entire source by huge amounts.Try using blockbuster and deblocking set to 0,0. If that doesn't work, try using a new matrix. Matricies change the quantisization method and therefore will change where bits are placed. This is the best way, in my opinion, to add bits to the dark areas preventing blocking.

History aside... your best friend is detail. Denoise very little, try to not remove too much grain and make areas flat. This goes in the area of 'range of detail'. Above all else, try and use RDO and multi-hex for the best picture. Hope this helps....

[Oline 61]

What are some good quantization matrices for x264 in dark scenes? I use eqm_avc_hr.cfg most of the time.

[dude051]

There isn't any all purpose block busting custom matricies for AVC yet. The only one I can really recommend is the one you mentioned, EQM_AVC_HR 1 by Sharktooth. If your feeling daring, give the one by Soulhunter, Soulhunters AVC v2, a try. Each are still being worked on and the latter is still searching for an area it works on in my book... still havent found a use for it heh. Just give em all a whirl, the default, JVT, EQM_AVC_HR and Soulhunters AVS V2.

I mention this method because it could work, and if it did its one of the best ways. Maybe you can make your own custom matrix futurex :). Just stray from 'low bitrate' matricies. You want something mid bitrate and that preserves detail.

[Sharktooth]

EQM AVC HR should help with the issue, infact it has a good quantization for low lumas, but it requires some more bits (HR = high bitrate).

[Dayvon]

Nice. This might be what I'm looking for (see here (http://forum.doom9.org/showthread.php?p=783879#post783879)). Sharktooth, you're still helping people even while at the hospital. You're the best!! Get well soon!

[sjchmura]

Has anyone noticed this is the ONE area where Nero recode is superior? If you turn on "mild" Psycho Enhancments these "blocks" seem to not be noticible. Interestingly, in my encodes of BattleStar HD stuff from NBC, the HIGH psycho enhancment actually made it worse!!!!

This was 720p@ 1GB/Hr

[foxyshadis]

"Psycho enhancements" are AQ (Adaptive Quantization). And yes, it's well-known that Nero's AQ is better than x264's. It's pretty complex logic, and presumably slower, to make a really good one.

[Oline 61]

AQ is a patch for x264, correct? It's not in the official source is it?

[sjchmura]

I was not criticizing x264 - I thought this was not even implemented yet. How do we "play" with these settings.

The "AQ" in nero is REALLY SLOW but to be honest for those dark scenes (and dark space battles) ends up coming out better at the same bit rate and it is obvious on my 65" 1080p DLP with good contrast etc.

Still I think the average picutre quality and "sharpness" for lack of a better word is better wtih x264. The dual core support and Megui (once you get it working) is really really easy

HQ_Slower :0 - definitely needs AMD X2 cores :)

[sjchmura]

One question - why does "high" AQ in Nero look WORSE in the dark scenes yet take so much more time? Has ATEME ever commented on this or am I just crazy??

[foxyshadis]

mild/medium/high probably correspond to the quant variance and thresholds, so that more (mostly low-complexity) areas in high mode are compressed much more strongly than in mild mode (which is already stronger than no-psycho). A higher quant means less details and more blurriness from inloop. Dark scenes often register with a lower complexity, so strong AQ like Xvid's and Nero's strongest tend to strip them of detail. x264's is sort of a patch to give bits to smooth areas compressed too hard; maybe nero's light works that way, I don't remember the details now.

I understand the more recent recode betas have even better psycho, but I was never in the testing trials so I can't say.

[Oline 61]

I want to apply this patch to my x264 builds. I build my own from SVN using GCC. Where can I get the patch, and how do I apply it?

[foxyshadis]

http://files.x264.nl/Sharktooth/?dir=./x264_patches

[Oline 61]

Now off to find a proxy server, because I still can't access files.x264.nl with a direct connection.

Update: 9th proxy I tried worked (don't you love free proxy lists where only 10% of the IP addresses are actually running a proxy server :)). Patched rev. 437, all hunks passed, encoding now with --aq-sensitivity 0.5.

Well it encoded fine, although I can't tell any difference except slower speed :).

[sjchmura]

So for those of us that have never compiled anything but turbo pascal on an 80286 and assemlby on a C=64, anyone want to share a compiled late build with me :)

[foxyshadis]

ChronoCross made one just last night.
http://chronocrossdev.com/apps/x264

[sjchmura]

Thanks!!!

Any suggestions where to start (in terms of strength)? I just have to figure out how to add this to my MeGUI - I think I can just "show command line" and add it there.

Thanks again!!!!!!! I can't wait to test this. I will post a few screen shots from "Kobol's Last Gleeming hope" (1080i->720p 1GB/HR rate. The "explosion" scenese show it best - stark, brightareas of contrast with teh subtle background of "near black" when looking at "AQ" Nero and no "AQ" x264

[foxyshadis]

From the patch:

+ " --aq-strength <float> Amount to adjust QP per MB [0.0]\n"
+ " 0.0: no AQ\n"
+ " 1.1: strong AQ\n"
+ " --aq-sensitivity <float> \"Flatness\" threshold to trigger AQ [15.0]\n"
+ " 5: applies to almost all blocks\n"
+ " 22: only flat blocks\n"

So that should give you some pointers for tweaking. A scene with very different motion profiles in parts of the frames sounds exactly like where it'd be useful. =D

[bkman]

Chrono's build seems to quit straight out when trying to encode with aq...

[Oline 61]

I can make a Linux build if anyone wants. If there is any demand post it here. Don't know how to or want to learn about Windows builds.

[asdfsauce]

Chrono's build seems to quit straight out when trying to encode with aq...

Same here.

[dude051]

I'm one hour into the first pass with --aq-strength 0.8 --aq-sensitivity 15

[ChronoCross]

Same here.

I'm trying to figure out why. I tried applying the patch both manually and automatically. Either way it does the same thing. I'll let you know.

[ChronoCross]

it's not the patches or x264. it's a problem with the latest gpac cvs. I'll see if a new CVS checkout fixes it.

[foxyshadis]

For all the grumbles over VFW it really seems as if gpac may be the single biggest source of x264 bugs. :p They could probably use some better change management and regression testing for their commits. But is there a really good reason to always use the current cvs?

[ChronoCross]

Fixed copy is now on my FTP. I tested it fully with 3 passes. It has all the patches in the folder installed.

[desta]

I'm gonna start by apologising for my apparent stupidity...

Ok, now that's out the way, how do I apply the AQ patch that's mentioned in this thread? I downloaded the x264_438_cli and the x264_aq.5-398.diff, but don't know what to do next with them.

I'm using the 0.2.3.2090 build of MeGUI.

I'll go stand in the corner and wait...

[foxyshadis]

It's a patch to the source code; unless you have a compiler it's not going to be much use to you. I linked to CC's site, where you can download a compiled version with the patch included.

[desta]

Ah right... well it was from that link that I got the x264_438_cli.rar - I just extracted the new exe and replaced the original on my pc... so I'll presume I'm now using the patched x264.exe.

I've also just noticed that the AQ setting is accessed by command line, so again apologies for stupidity.


edit: the reason I was drawn to this thread, was that I'm encoding from a clean dvd source, but still getting a blocky output. The only setting that seems to get rid of it for me is using CQ at it's highest... but obviously that's giving me huge output file sizes.

[dude051]

Yes, currently you have to run it from command line. I recommend a batch file since currently no GUI supports that new patch. As for you problem... try just using a custom matrix first. This is supported by MeGUI (which I think is better suited for you, no offence), and will give you results. The new AQ patch is rather experimental at this stage. Again, I recommend Sharktooth's EQM AVC HR (if your running a rather high bitrate, say 1500kbps? More if you can.)

[bkman]

Actually, you can just add --aq-strength x into the Custom Commandline Options field under Zones in MEgui.

[desta]

Well, I've just tried a mix of adding the command line in as bkman suggested, and using the custom matrice as dude051 said to do...... the results were far better. Many, many thanks. :)

On the subject of blockiness, I see many people saying to turn off the in-loop deblocking or reduce it completely to avoid blocks occuring. Obviously I'm on expert, but I would've thought (though probably wrongly) that raising the deblocking to '+1, 0' would actually reduce blocks. Again, I'm not trying to make out I know what's the case here, but from some experience of encoding to mpeg2, I understand that you have to add a deblock filter to remove blocks and/or noise to prevent them.

I was reading this...

----

http://www.mplayerhq.hu/DOCS/HTML/en/menc-feat-x264.html

"Many people seem to think it is a good idea to lower the deblocking filter strength by large amounts (say, -3). This is however almost never a good idea, and in most cases, people who are doing this do not understand very well how deblocking works by default."

----


... I'm not trying to seem smart by posting that. Just thought it might make an interesting read for people, and of course would be good to get opinions.

[nm]

On the subject of blockiness, I see many people saying to turn off the in-loop deblocking or reduce it completely to avoid blocks occuring.
Where do you see people saying that? In-loop deblocking is reduced to preserve "surface texture" along with a high enough bitrate, not to avoid blocking.

[desta]

Ok.. I've obviously misunderstood what's been said then.

[Oline 61]

The only time you turn off deblocking is when decoding is too slow, or you're using --qp 0

[desta]

The only time you turn off deblocking is when decoding is too slow, or you're using --qp 0
I'm not overly clued up on the commandlines... is that one referring to the minimum quantizer?

Also, when I add the --aq-strength/sensitivity commands in the 'zones' tab, they add to end of the commands... is that ok?

Cheers.

[foxyshadis]

--qp 0 is lossless encoding.

And yeah, commands can be in any order.

[Manao]

The only time you turn off deblocking is when decoding is too slow, or you're using --qp 0Actually, deblocking disables itself when the quantizer is under 16 - strength.

[dude051]

Actually, you can just add --aq-strength x into the Custom Commandline Options field under Zones in MEgui.
Wow, very nice, I didn't know about this option in MeGUI... I was wrong then :scared: . Thanx for the tip btw.

As for the in-loop filter... all it does is smooth more for + values and not smooth more for - values. This doesn't sharpen the source, just prevents the filter from over smoothing. This should also go that you should raise this only if you have aparent blocking in the source, this is not to prevent blocks from occuring. A thread to read up on this would probably be..... http://forum.doom9.org/showthread.php?t=105858