Ghosting artifacts in x264 encoding

[Sasovics]

Hi Folks,

I am experiencing weird ghosting artifact in my encodings, especially when a light scene suddenly turns completely dark. These ghosting artifacts (ghosting as these are silhouettes from previous bright scenes) stay there as long as the current scene remains black.

See screenshoots:

http://img409.imageshack.us/img409/2513/ghostingky9.th.jpg (http://img409.imageshack.us/my.php?image=ghostingky9.jpg)

Zoomed

http://img409.imageshack.us/img409/6251/ghostingzoomnm1.th.jpg (http://img409.imageshack.us/my.php?image=ghostingzoomnm1.jpg)

I suspect this might be because of wrongly calculated settings in my x264 commandline.. but might be something else..

Please comment...

My Graph:

Haali Splitter -> WMVideo Decoder DMO

My AVS:

DirectShowSource("Untitled.grf", fps=23.976, framecount=165876, audio=false)
Crop(0,140,0,-136)
Spline36Resize(1280,720)
Trim(0,2000)


My x264 Cmdline:

x264 --pass 1 --bitrate 5708 --stats ".stats" --bframes 3 --b-pyramid --direct auto --deblock -2:-1 --subme 1 --analyse none --me dia --threads auto --thread-input --cqmfile "prestige.cfg" --deadzone-inter 8 --deadzone-intra 4 --progress --no-psnr --no-ssim -o NUL Video.avs

x264 --pass 2 --bitrate 5708 --stats ".stats" --ref 5 --mixed-refs --no-fast-pskip --bframes 3 --b-pyramid --b-rdo --bime --weightb --direct auto --deblock -2:-1 --subme 7 --analyse p8x8,b8x8,i4x4,i8x8 --8x8dct --trellis 0 --aq-strength 0.5 --aq-sensitivity 12 --deadzone-inter 8 --deadzone-intra 4 --me umh --threads auto --thread-input --cqmfile "prestige.cfg" --progress --no-psnr --no-ssim --output VIDEO.MKV Video.avs

PS: I tried to raise bframes to 16 as suggested by Sagekilla but same result :(

[Dark Shikari]

Can we see the screenshots of the source for those same frames?

[Sasovics]

The original source frame does not exhibit this anomaly, I can tell you this for sure now...

Anyway, I will post the screenshot of the source frame later today..


EDIT: The source frame:

http://img263.imageshack.us/img263/2774/ghostoriginalbb3.th.jpg (http://img263.imageshack.us/my.php?image=ghostoriginalbb3.jpg)

[Ranguvar]

Not sure this is the best solution, but if you're using a screen that ghosts easily (I have experience with the PSP's LCD) the best way is to increase brightness with Tweak() in AviSynth. You can adjust contrast and saturation to compensate for the increased brightness and make it look better.

EDIT: Yeah, never mind. You have actual ghosting in your encode.

[Guest]

Why can't I see any ghosting? I have an LCD monitor.

[Sharktooth]

cant see anything there at normal brightness...
raising it to ridiculous levels i can see some blocks... but that's not ghosting... that's blocking
http://www.webalice.it/f.corriga/temp/ghostingzoomnm1.png
i think you can fix it by using a VAQ enabled x264 build.
however, yes, x264 is behaving incorrectly when creating those blocks, maybe it's due to a rounding error.
let's hear what the specialists have to say.
i think you should also provide a small clip of the encoding and the source so they can debug...

[Dark Shikari]

Yes, a clip of the .h264 file would be extremely useful for debugging your problem.

[Sasovics]

Folks, Here (http://rapidshare.com/files/93416079/VIDEO.MKV.html) is my encoded video chunk. The blocking I'm describing can be found at around 1:29 timestamp.

To notice this anomaly, turn off the post-processing deblocking in your decoder preferences and boost-up the brightness on your LCD (if needed)

EDIT: @Sharktooth: Where can I get VAQ enabled x264 build ? For the chunk above I have used 709C build downloaded from www.x264.nl. How do I know if this build is VAQ enabled or no ? Please advise!

[LoRd_MuldeR]

o notice this anomaly, turn off the post-processing deblocking in your decoder preferences
Sasovics, you are saying you have disabled the Inloop Deblocking in your H.264 decoder? :eek:

The "Inloop" Deblocking is an integral part of H.264, it's not optional like MPEG-2/MPEG-4 postprocessing filters.
All H.264 encoders (e.g. x264) will rely on the decoder (e.g. libavcodec) to do a proper H.264 deblocking!

So if you disable the Inloop Deblocking in your decoder, you are supposed to see significant blocking (or "ghosting", as you call it) in the output!


EDIT: @Sharktooth: Where can I get VAQ enabled x264 build ?
From http://x264.tk/ for example, or via MeGUI auto-update...

[Sasovics]

This is confusing me a little bit. I am using CoreAVC 1.5.0.1 Video decoder and that one has an option to "Disable deblocking" in it's decoder preferences.

So if this is the case, is it my decoder what is causing this blocking ???

[LoRd_MuldeR]

This is confusing me a little bit. I am using CoreAVC 1.5.0.1 Video decoder and that one has an option to "Disable deblocking" in it's decoder preferences.

So if this is the case, is it my decoder what is causing this blocking ???
Well, every "proper" H.264 decoder must do the "Inloop Deblocking" as defined by the H.264 standards and as expected by the encoder.
If your decoder skips the deblocking (for performance boost), then "corrupted" output is the consequence! You will see (blocking) artifacts...

The CoreAVC people really should rename that option to "Disable deblocking (NOT RECOMMENDED)" ;)


Some more info, taken from Wikipedia:
H.264 deblocking filter

In contrast with older MPEG-1/2/4 standards, the H.264 deblocking filter is not an optional additional feature in the decoder. It is a feature on both the decoding path and on the encoding path, so that the in-loop effects of the filter are taken into account in reference macroblocks used for prediction. When a stream is encoded, the filter strength can be selected, or the filter can be switched off entirely. Otherwise, the filter strength is determined by coding modes of adjacent blocks, quantization step size, and the steepness of the luminance gradient between blocks.

[Snowknight26]

Tried a differen't decoder in your graph?

[LoRd_MuldeR]

Tried a differen't decoder in your graph?
As long as he forces the decoder to produce "corrupted" output by disabling the Inloop Deblocking, he won't see correct output - no matter which decoder. He should stick with CoreAVC and just re-enable the deblocking! Further investigation is only needed, if the so-called "ghosting" remains with proper deblocking...

[Sasovics]

I see no blocking when Decoder's deblocking is ON. I think your answer is fair enough to me ... seems there is no problem with my encoding then.

[Sasovics]

One more question I have, getting answer to that I can consider this topic closed ;)

Can one of you experts comment my x264 cmdline settings above if it is a good choice for a very grainy source, such as 300 (1080p, Blu-Ray source)

[Ranguvar]

300 is a BEAST to encode. Search around, especially in recent AVC forum and AviSynth forum posts. Tons of info on making it look good. But you're in for a nightmare.

[Dark Shikari]

You have five choices for 300:

1. Degrain it with a relatively fast but detail-destroying degraining script. Resize to 720p or 480p, 'cause there's no HD left to keep.

2. Degrain it with a monstrously slow script like TemporalDegrain.

3. Store the grain with a high bitrate (10-12 megabits for 1080p) and Prestige matrix + low deadzones.

4. Don't degrain, and use a low bitrate, destroying the quality.

5. Don't bother re-encoding.

[Snowknight26]

3. Store the grain with a high bitrate (10-12 megabits for 1080p) and Prestige matrix + low deadzones.

10930 kbps with above settings:
x264 [info]: slice I:2067 Avg QP:20.23 size:188622
x264 [info]: slice P:96348 Avg QP:22.96 size: 78468
x264 [info]: slice B:69237 Avg QP:26.22 size: 23159

Needs more bitrate!

[Didée]

Some name it "anti-piracy grain" ... ;)

@ Dark Shikari: No GrainOptimizer in that list?

[Dark Shikari]

Some name it "anti-piracy grain" ... ;)

@ Dark Shikari: No GrainOptimizer in that list?You could try it, I guess. Its still not a magic bullet.

[Prettz]

Hi Folks,

I am experiencing weird ghosting artifact in my encodings, especially when a light scene suddenly turns completely dark. These ghosting artifacts (ghosting as these are silhouettes from previous bright scenes) stay there as long as the current scene remains black.

See screenshoots:

http://img409.imageshack.us/img409/2513/ghostingky9.th.jpg (http://img409.imageshack.us/my.php?image=ghostingky9.jpg)

Zoomed

http://img409.imageshack.us/img409/6251/ghostingzoomnm1.th.jpg (http://img409.imageshack.us/my.php?image=ghostingzoomnm1.jpg)

I suspect this might be because of wrongly calculated settings in my x264 commandline.. but might be something else..

Please comment...

My Graph:

Haali Splitter -> WMVideo Decoder DMO

My AVS:

DirectShowSource("Untitled.grf", fps=23.976, framecount=165876, audio=false)
Crop(0,140,0,-136)
Spline36Resize(1280,720)
Trim(0,2000)


My x264 Cmdline:

x264 --pass 1 --bitrate 5708 --stats ".stats" --bframes 3 --b-pyramid --direct auto --deblock -2:-1 --subme 1 --analyse none --me dia --threads auto --thread-input --cqmfile "prestige.cfg" --deadzone-inter 8 --deadzone-intra 4 --progress --no-psnr --no-ssim -o NUL Video.avs

x264 --pass 2 --bitrate 5708 --stats ".stats" --ref 5 --mixed-refs --no-fast-pskip --bframes 3 --b-pyramid --b-rdo --bime --weightb --direct auto --deblock -2:-1 --subme 7 --analyse p8x8,b8x8,i4x4,i8x8 --8x8dct --trellis 0 --aq-strength 0.5 --aq-sensitivity 12 --deadzone-inter 8 --deadzone-intra 4 --me umh --threads auto --thread-input --cqmfile "prestige.cfg" --progress --no-psnr --no-ssim --output VIDEO.MKV Video.avs

PS: I tried to raise bframes to 16 as suggested by Sagekilla but same result :(
I'm chiming in just to say that I have an awesome CRT monitor (at 85Hz) and I can't see a damn thing in either screenshot. Sorry, man.

[Atak_Snajpera]

First of all screenshot should be saved in png instead of jpg

[Sasovics]

I'm chiming in just to say that I have an awesome CRT monitor (at 85Hz) and I can't see a damn thing in either screenshot. Sorry, man.

I have a very high quality high-brightness dual-lamp powered LCD panel so for me the blocks were obvious even without boosting the monitor brightness on...

First of all screenshot should be saved in png instead of jpg

Thanks for notifying this.. I will keep that in mind in the future ;)

[Sasovics]

You have five choices for 300:

1. Degrain it with a relatively fast but detail-destroying degraining script. Resize to 720p or 480p, 'cause there's no HD left to keep.

2. Degrain it with a monstrously slow script like TemporalDegrain.

3. Store the grain with a high bitrate (10-12 megabits for 1080p) and Prestige matrix + low deadzones.

4. Don't degrain, and use a low bitrate, destroying the quality.

5. Don't bother re-encoding.

Shikari, what values do you suggest for "low deadzones" for a grainy source ?

[Dark Shikari]

Shikari, what values do you suggest for "low deadzones" for a grainy source ?Maybe 5 for both deadzones?

[kristiXS]

The Prestige matrix is found where?

[Sasovics]

Use the forum's Search (http://forum.doom9.org/search.php) feature

[kristiXS]

Use the forum's Search (http://forum.doom9.org/search.php) feature

I would not have posted had i not already attempted that. Searching titles for a post regarding that gets you:
Sorry - no matches. Please try some different terms.


I am afraid, Sasovics, that your post was not helpful. Try again with a more specificic recommendation please.

[jeffy]

I am not sure, maybe it is this one?
http://forum.doom9.org/showthread.php?p=989214#post989214

[nm]

You know, it is also possible to search for words within posts and use intuition to pick promising threads from the results. Or just google it ("prestige cqm" gave me the right thread as the first hit) ;)

But, I already did the work for you: http://forum.doom9.org/showthread.php?t=124755

Ah, and so did jeffy :)

[kristiXS]

I am not sure, maybe it is this one?
http://forum.doom9.org/showthread.php?p=989214#post989214

Thank you. I did a title search all forums with no result, and searched within posts but in the wrong sections. I could have googled Prestige but did not know to use the term cqm etc along with it, so that didnt get me anywhere (other than finding the movie, The Prestige on Amazon.com lol.) Finding something is all in knowing where to look, and more importantly, descriptive terms for what you seek. I am afraid that while i know enough to get by, i would never have guessed to search for cqm. Matrix maybe, but not an abbreviation for Custom Quantization Matrix. Experience goes a long way, and I appreciate help from those in the know.

[Sharktooth]

do not use prestige on non heavily grainy sources...

[Sasovics]

Yes, I agree with Sharktooth.

Prestige is good for grainy sources (most of the real-life sources are pretty grainy),
for non-grainy sources I use eqm_avc_hr matrix instead ;)

[Ranguvar]

Yeah. And if you do use Prestige, check for funky artifacting. Search around, there was a thread with a comparison between *.mp4 guy's high quality matrix, Sharktooth's AVC HR, and Prestige. Although I would do your own comparison.

[Sasovics]

I've read that thread thoroughly and from my experience Prestige is still the best choice for very grainy sources... ;)

[CruNcher]

Hi i also encountered this problem but it's not only a X264 problem also other H.264 encoders have problems with this.
Their is somekind of fog/shadow in the source above the actual film layer (it seems to appear very often in CGI scenes when light sources are rendered it can get very visible) this fog is very pixelated (very fine) and so only gets percepted very low in the original (even wrong calibrated it doesn't hurt much).
After the quantization it seems the structure of it gets decimated (the higher the quantization the more problematic it gets, visually) and so it becomes a block like appearance (the inloop deblocker can't do anything against it) with ffdshow it gets really visible (Fullscreen) (with CoreAVC it's much less problematic) (i think because of the different luminance representation, seems these artifacts are very well hidden in the luminance).

Here is an example:

Original Percepted Frame: (Source)
http://s4.directupload.net/images/080320/jfk9q7cm.png

Luma Noise/Grain: (before quantization)
http://s7.directupload.net/images/080320/wmohnfja.png

Luma Noise/Grain: (after high quantization) H.264
http://s7.directupload.net/images/080320/z5d4qizn.png

My guess is you have to calibrate everything correctly to avoid this decimation (encoding artifact) becoming visible in your viewing setup (Monitor,Renderer,Software Brightness and Gama settings)

Tough imho it's better to distribute the bits differently (if you don't want to post-pro kill it) and try to avoid such extreme low quantization in such scenes (even if the viewer is not correctly calibrated it isn't percepted that bad then,and the decimation in motion in the brighter scenes isn't visible @ all)

X264 - CVS (CruNchers Visual System ©) Main Profile Level 3 2mbit Full SD Dark_Shikari AQ
http://s5.directupload.net/images/080320/6ugs6igy.png