x264 artifacting question

[SamNeal2001]

The commandline: --preset veryslow --tune film --level 3 --crf 16

The result: http://www.sendspace.com/file/sy7rqz

Forehead on the closeup of the redhead girl looks terrible. I always seam to have that problem and can never get rid of it.

Suggestions?

[Dark Shikari]

I can't see anything wrong; can you point out specifically what the problem is?

[SamNeal2001]

On the top right side of her head between her eyebrow and the edge of frame there is a bunch of dancing blocks. Most noticeable between 32-39 seconds. Everything, and everyone else looks flawless.

[Dark Shikari]

I've replayed the section a few dozen times but I can't see anything:

http://i.imgur.com/wWX8A.png

Can you point out where the problem is, specifically? Are you sure it's not a problem with your player?

[jethro]

i see it too especially in darker part of forehead on the right side

[SamNeal2001]

What he said.

Now I know you can't judge an encode by frame grabbing, but the artifacting sticks around for pretty much the entire duration of every closeup of the red headed girl.

http://i.imgur.com/a1APs.png

Just trying to figure how how a section can look so bad, considering the whole scene, when everything else, in all of the other shots looks near flawless.

[Guest]

Why do those two have different overall luma levels? What is your script?

[Blue_MiSfit]

That's very visible. I'm not sure how to describe it, and it's definitely only visible in motion.

More bits? You're encoding full resolution SD at 1mbps. Well... okay it's 2.40 AR, but still. This seems like an unusually low bitrate for CRF 16, so maybe do 2 pass at 2mbps, or maybe try CRF 14 or so?

@Dark_Shikari: This is quite visible, you don't see it?? The rest of the frame is flawless throughout the whole video, but her forehead grabs my attention each time.

One other wild guess.... try disabling mbtree?
Not a good idea. Disregard!

[Atak_Snajpera]

This movie has weird colors. It looks like some kind of pirated "screening" version downloaded from torrent site.
Regarding artefacts. I see nothing wrong. (latest ffdshow + MPCHC) Frankly I wouldn't expect perfect image quality when you upscale during playback 720x354 (~1Mbps) to 1920x1080 (22''+). It is normal that all blocks/artefacts will be more visible (enlarged)

[SamNeal2001]

This is from the original store bought physical DVD (yes they still do exist). And I'd consider the transfer pretty damn good. It is encoded at 8000mb/sec.

And you don't even need to scale that up to HD to notice the artifacting. It is very obvious.

I posted this question only because it is said that if you use crf 16 you can get perfect quality. But I have to go down to crf 12 to get rid of that artifacting. But the resulting bitrate becomes around 4500mb/sec.

I don't know everything about encoding, but considering the whole scene, I don't understand how everything can look perfect (even when scaled up to HD resolution) and a portion, or a section of the image can look like garbage in comparison.

What is so difficult about that part of the image that the encoder suffers to much?

[SamNeal2001]

Here's the source.
http://www.sendspace.com/file/a1sjdo

[Atak_Snajpera]

what player do use ? again i see nothing what could be described as garbage on forehead

[SamNeal2001]

Media Player Home Classic on my PC. Dune as my hardware player.

[Dark Shikari]

@Dark_Shikari: This is quite visible, you don't see it?? The rest of the frame is flawless throughout the whole video, but her forehead grabs my attention each time.Do you mean that extremely faint vertical artifact? Other than that I don't see much...

Are you watching the video upscaled to fullscreen? I'm only watching it at its original resolution; it's always the case that when watching a video upscaled, it needs to be higher quality (and thus lower CRF) to be completely artifact-free.

One other wild guess.... try disabling mbtree?I don't see any reason why "disabling mbtree" would be useful here -- if it has any effect, it's purely luck, and is just as likely to hurt quality other places. Making random suggestions like this is how misleading rumors get started.

[cyberbeing]

There seems to be large patches of blurry 'grain' artifacts mixed in with her forehead, which remain semi-static when she turns her head. I assume that's the same distracting problem Blue_MiSfit saw.

The following seems decent @1.5Mbps:
--preset veryslow --tune grain --level 3 --crf 16 --aq-strength 1.0 --aq-mode 2

[Dark Shikari]

That's caused by a combination of psy and the motion search not quite getting the "true" motion vectors when she turns her head.

There isn't any easy solution, unfortunately. It might help to not use --tune grain, but I can't give any guarantees.

[cyberbeing]

I already did some test encodes with his source link, and at least to my eyes, --tune grain with --aq-mode 2 seemed to give better results than any single override or addition to --tune film settings at the same bitrate.

[Blue_MiSfit]

@ Dark_Shikari: Indeed, just a wild guess :)

Retracted.

I was watching this full screen on a 1920x1200 24" LCD, so yeah that would explain why it seems much worse to me. When watching windowed its' scarcely noticeable.

[Blue_MiSfit]

There seems to be large patches of blurry 'grain' artifacts mixed in with her forehead, which remain semi-static when she turns her head. I assume that's the same distracting problem Blue_MiSfit saw.

The following seems decent @1.5Mbps:
--preset veryslow --tune grain --level 3 --crf 16 --aq-strength 1.0 --aq-mode 2

1.5mbps is a 50% increase in bitrate compared to the original sample provided. This alone could definitely account for the difference!

[cyberbeing]

1.5mbps is a 50% increase in bitrate compared to the original sample provided. This alone could definitely account for the difference!
It didn't though. I tested from 1Mbps to 4Mbps with both --tune film and --tune grain, along with a mixture of setting overrides (tweaking deblock, psy, qcomp, aq, or even enabling tesa with --tune film didn't seem to help at all), before reaching that conclusion. As SamNeal2001 mentioned above, --tune film didn't start looking decent on the forehead until bitrate exceeded 3Mbps.

As for which of the --tune grain specific settings made the biggest difference, who knows. SamNeal2001's 1min 13sec source sample is available for anybody who has a desire to figure that out.

[jsquare]

It didn't though. I tested from 1Mbps to 4Mbps with both --tune film and --tune grain, along with a mixture of setting overrides (tweaking deblock, psy, qcomp, aq, or even enabling tesa with --tune film didn't seem to help at all), before reaching that conclusion. As SamNeal2001 mentioned above, --tune film didn't start looking decent on the forehead until bitrate exceeded 3Mbps.

As for which of the --tune grain specific settings made the biggest difference, who knows. SamNeal2001's 1min 13sec source sample is available for anybody who has a desire to figure that out.
Cyberbeing, try this:
--crf 18 --no-fast-pskip --no-mbtree --b-adapt 2 --b-pyramid strict --no-weightb --direct auto --trellis 0 --no-psy --weightp 0 --aq-mode 2 --no-dct-decimate
I know I'll be flamed by this comments but MBTree is a detail killer at low bit rates, I stopped using it a long time ago. Trellis increases the bit rate with no visual gain (at least to my eyes), and PSY-RD is another option that I recently got rid off on my encodes.

[Dark Shikari]

I don't really think that CRF 18 counts as "low bit rates", by at least an order of magnitude. I wouldn't call anything below CRF ~26-30 "low bit rate".

A lot of the settings in your command don't make much sense, like --b-pyramid strict (which cannot possibly improve anything).

[jsquare]

I don't really think that CRF 18 counts as "low bit rates", by at least an order of magnitude. I wouldn't call anything below CRF ~26-30 "low bit rate".

A lot of the settings in your command don't make much sense, like --b-pyramid strict (which cannot possibly improve anything).
I normally use CRF 20-21, the 18 value was used as a reference, anything above 22 is worthless. As far as --b-pyramid strict it was there since I took it off my own Blu-ray compatible profile minus some other stuff.

[Dark Shikari]

A CRF above 22 might be useless if you're looking for a transparent encode. But nothing near that is ever going to be "low bit rate".

Plenty of people use x264 at much higher CRFs when not looking for a transparent encode. Facebook for example used CRF 28 at one point.

In my experience, MB-tree helps more at lower bitrates and less at the highest bitrates (e.g. CRF 18). But I don't think this particular case of artifacts is particularly near MB-tree's corner cases, so I doubt it's related.

[jsquare]

A CRF above 22 might be useless if you're looking for a transparent encode. But nothing near that is ever going to be "low bit rate".

Plenty of people use x264 at much higher CRFs when not looking for a transparent encode. Facebook for example used CRF 28 at one point.

In my experience, MB-tree helps more at lower bitrates and less at the highest bitrates (e.g. CRF 18). But I don't think this particular case of artifacts is particularly near MB-tree's corner cases, so I doubt it's related.
I agreed, a higher CRF could be better for some other cases or applications. I've gone as high as 24-26 in my encodes for some portable devices that requires a baseline 1.3 and the job of x264 still excellent as far as quality and size. But I always reserve MBTree for some serious compression at lower CRF.
Anyways Dark, your feed back is always welcome.

[AnonCrow]

I wonder if encoding the source with --qp (and 1.00 ip/pbratios) would be useful in this case,
if for no other purpose than to see how low a quantizer the problematic area would need
to look good enough for the OP; then consider using zones.

[7ekno]

The source also looked pretty good encoded with:

--preset veryslow --tune film --level 3 --crf 16 --aq-strength 1.15

7ek

[benwaggoner]

A lot of the settings in your command don't make much sense, like --b-pyramid strict (which cannot possibly improve anything).
It can improve trick play in many hardware devices; drop non-ref B-frames for 2x playbac, drop all B-frames for 4x playback.

[SassBot]

It can improve trick play in many hardware devices; drop non-ref B-frames for 2x playbac, drop all B-frames for 4x playback.

Which wasn't really what he was referring to. He was saying that it would offer no improvements when it comes to the blocking issue.

[*.mp4 guy]

This is actually a very common artifact introduced by x264. It is caused by psy interacting with motion search that is having trouble with texture that it cannot distinguish from noise during motion. There would be artifacts without psy, but they would be less energetic (probably just excessive detail loss). Psy forces the codec to try to put more energy into these areas, often without increasing bitrate significantly, which can make the artifacts worse. Fiddling with aq can help by increasing the bitrate in such places, better motion search also helps, sometimes greatly, but is unreliable in general. Lowering psy rd can be helpful, but may also be counter-productive.

In this case, the 'sliding' makes me think that the use of badly predicted non-coded mv's contributes to the problem as well.

Frankly I thought people were just ignoring this problem, because it's essentially a universal issue for x264. It is also rarely mentioned in the case of other block transform codecs, for example, the artifact is also present in the source mpeg2, albeit to a much reduced extent concomitant to the higher bitrate.

These issues tend to be greatly exacerbate by each re-encoding step, as each codec in succession is faced with the same difficulty, but is increasingly misled be the previous mistakes of its predecessors. That's always how it goes when there are systemic artifacts in a codec, or codec family.

[benwaggoner]

Frankly I thought people were just ignoring this problem, because it's essentially a universal issue for x264. It is also rarely mentioned in the case of other block transform codecs, for example, the artifact is also present in the source mpeg2, albeit to a much reduced extent concomitant to the higher bitrate.
We had this problem with VC-1 as well. We built in a simple edge-preserving noise reduction filter which could be turned on for sources that showed this problem.

An improved dynamic motion vector cost mode to address this was the last quality improvement added to Microsoft's VC-1 implementations.

These issues tend to be greatly exacerbate by each re-encoding step, as each codec in succession is faced with the same difficulty, but is increasingly misled be the previous mistakes of its predecessors. That's always how it goes when there are systemic artifacts in a codec, or codec family.
Reencoding from visually lossy source is always asking for trouble :).

[SamNeal2001]

This is actually a very common artifact introduced by x264. It is caused by psy interacting with motion search that is having trouble with texture that it cannot distinguish from noise during motion. There would be artifacts without psy, but they would be less energetic (probably just excessive detail loss). Psy forces the codec to try to put more energy into these areas, often without increasing bitrate significantly, which can make the artifacts worse. Fiddling with aq can help by increasing the bitrate in such places, better motion search also helps, sometimes greatly, but is unreliable in general. Lowering psy rd can be helpful, but may also be counter-productive.

In this case, the 'sliding' makes me think that the use of badly predicted non-coded mv's contributes to the problem as well.

Frankly I thought people were just ignoring this problem, because it's essentially a universal issue for x264. It is also rarely mentioned in the case of other block transform codecs, for example, the artifact is also present in the source mpeg2, albeit to a much reduced extent concomitant to the higher bitrate.

These issues tend to be greatly exacerbate by each re-encoding step, as each codec in succession is faced with the same difficulty, but is increasingly misled be the previous mistakes of its predecessors. That's always how it goes when there are systemic artifacts in a codec, or codec family.

Thanks for the explaination *.mp4 guy :)

And I'm not trying to poke holes in x264 when I say this, but yes, I've notice this same problem all over the place. Everything looks perfect, then a simple little section of the image will cause x264 to block up like crazy. And they are never high motion or complex scenes.

Is it possible x264 can address this issue the way VC-1 does/did. Or is it the very nature of H.264 itself that x264 has no control over.

We had this problem with VC-1 as well. We built in a simple edge-preserving noise reduction filter which could be turned on for sources that showed this problem.

An improved dynamic motion vector cost mode to address this was the last quality improvement added to Microsoft's VC-1 implementations.
.

Does expression encode give me access to all of VC-1's features? Or is it a stripped down version? If so, which piece of software (preferably free) will give me complete access to VC-1's features?

[Dark Shikari]

"Blocking up" doesn't seem to describe this artifact at all; there's no blocking, it's more like a slight smearing of detail that doesn't match the real motion vector. I certainly can't see any "blocking" anywhere in the video.

mp4guy is largely correct as to the cause. "Better" motion vector costs might help, but I don't think they'd be a silver bullet, especially since there's no real way to know what exactly is "better" universally.

[deadrats]

@SamNeal2001

hey, i too have seen some sources where under some circumstances x264 seems to fall apart at certain parts of a scene. after more test encodes than i can count and using practically every combination of settings that exist (and considering how many parameters are adjustable that's a load of combinations) i have finally settled on using the following approach that seems to eliminate all the problems i was having:

1) don't use crf or anything related to it; it's silly to place your trust in any piece of software to decide quality for you, use a bit rate based approach and lot's of it. i'm talking 4 mb/s for SD, 8 mb/s for 720p and 12 mb/s for 1080p.

2) i use these settings: i start off with the ultra fast preset and then add the following: no mb tree, no deblocking (i hate deblocking with a passion, it kills details and results in crappy encodes), 3 reference frames, 3 b-frames, SATD exhaustive (that's what xmedia recode calls it, tmpg calls it hadamard), auto direct mode prediction, i only use intra 8x8 and 4x4 everything else i disable, no psy, chroma me enabled, fast p-skip, dct decimation, mixed references, cabac, aq mode to auto, weighted p smart analysis, adaptive b optimal, weighted b implicit (this is what tmpg calls it). i've also found that motion search range can improve quality, at times quite a bit, depending on source and i have experimented with ranges up to 128, but this brings pc's down to their knees, i've seen noticeable quality up to 64 but in all honesty the difference between 64 and 32 wasn't that great.

anyway, that's me, i'm sure that there will probably be some people that read this and say ?

[*.mp4 guy]

It's always possible to address a compression induced problem by using less compression, or using less advanced compression. However this avoidance cannot be conflated with a solution and in this case, I don't expect anyone will be able to stomach those recommendations. Minimal bitrate bloat is desired.

[mandarinka]

1) don't use crf or anything related to it; it's silly to place your trust in any piece of software to decide quality for you, use a bit rate based approach and lot's of it. i'm talking 4 mb/s for SD, 8 mb/s for 720p and 12 mb/s for 1080p.

That's silly. I have no problem with using higher bitrates rather than lower (bitrate being the only thing that never hurts, after all...), but you are doing it wrong, pardon my trollspeak.

If you want to spend more bits, just use a lower crf number. It's definitely going to be more effective than random shots you describe.

And BTW - you say you don't want to believe a software to do the decission. However that software is analysing the video and as such it actually has a clue how compressible/bitrate hungry teh material is.

You on the other hand have no clue about that, and yet you trust your judgement that 720p shall need 8 mbps and etc... (while I mean this constructivelly, :mad: ...)

[Audionut]

1) don't use crf or anything related to it; it's silly to place your trust in any piece of software to decide quality for you, use a bit rate based approach and lot's of it. i'm talking 4 mb/s for SD, 8 mb/s for 720p and 12 mb/s for 1080p.

CRF and 2 pass use the exact same bit distribution algorithm (http://forum.doom9.org/showthread.php?p=1579680#post1579680).

So everything you said up there is silly.


2) i use these settings:

Virtually disable everything so that you can increase --me-range as far as you can tolerate for the speed loss on your computer.

[corneliu68]

Yes it's a sad moment when you are happy with the great results you get from x264 and suddenly such a problem comes up....
A cumbersome method I tried, to get rid of this problem, was to add a not-animated Kodak grain filter with an 0.1 intensity in After Effects.
For the eye there is nothing but a 1 frame "black" png with the static noise is almost 1 Mb.
The problem is that AE is very slow adding the grain about 150 000 times... and to get a blu-ray source into AE is another long story...

[rahzel]

@OP, Tune Film sets psy-trellis to 0.15 and aq-strength to 1.0. I would try raising psy-rd and aq-strength a bit (maybe aq 1.10 and psy-rd 1.05) and lower psy-trellis to ~0.05. In my experience, psy-trellis increases the bitrate pretty significantly with CRF encodes as you raise it, so you could probably lower CRF a bit to say, 15.8 or 15.9 if you want to target a similar size.