DXVA playback leads to flickering, how to prevent it?

[kenpachi]

My encodes have suffered from flickering in dark areas recently. I discovered what happened to be the problem. It is not the script nor x264 parameters to be the reason. Raw script preview in mplayerc (with default AVI decompresser) is correct. Decoding the compressed material with coreAVC 3.0.1, LAV or libavcodec is correct as well. Decoding with newest mplayerc-hc DXVA internal (nVidia newest drivers, win7) leads to macroblocked flickering in e.g. dark blue areas as in the provided sample - the woman's shirt and some part of the background flickers in an awful macroblock style. What I ask is what to disable in x264 to prevent from DXVA playback flickering mentioned and why there is any problem at all with DXVA presets. I use meGUI x264-DXVA-SD-HQ with following modifications:

automated 2pass, bitrate 2000, high profile, level 4.1, target playback DXVA, ref number 5, Psy-RD Strength 1.08

Below there is a link to the sample and the used script as well.

http://www.sendspace.com/file/p93w7g

MPEG2Source(".\sample.d2v")
Crop(16, 0, -12, -6)

DeSpot(...)

super = MSuper(pel=2, sharp=1)
backward_vec2 = MAnalyse(super, isb = true, delta = 2, overlap=4)
backward_vec1 = MAnalyse(super, isb = true, delta = 1, overlap=4)
forward_vec1 = MAnalyse(super, isb = false, delta = 1, overlap=4)
forward_vec2 = MAnalyse(super, isb = false, delta = 2, overlap=4)
MDegrain2(super, backward_vec1,forward_vec1,\
backward_vec2,forward_vec2,thSAD=400,thSCD1=200,thSCD2=80)

last.LimitedSharpenFaster(...)

GradFunkMirror(strength=1.5)

Lanczos4Resize(672, 560)
GrainFactory3(g1str=1, g2str=2, g3str=8)

[benwaggoner]

Level 4.1 seems like serious overkill for your bitrate and frame size. If you're doing 25p or less, you could go down to Level 3.0.

All of that should work in theory. One thing you might try is turning off weighted P-frames. That could cause some issues with a problematic decoder. However, with the latest drivers you shouldn't have one.

[kenpachi]

I've tried to reproduce the problem but meanwhile my friend had downgraded nVidia drivers and it's impossible now. I don't see the mentioned difference between CPU and DXVA decoding anymore. I have downloaded new x264 presets and now I'm encoding with DXVA-SD-HQ (2-pass automated, bitrate-2400(672x560), High@3.1, DXVA playback, slow preset(5ref), deblock-1-1, aq1.0, psy1.12). Quality acceptable. One question though.

@benwaggoner: Why is it said to encode with L3.1 for SD materials and L4.1 for HD ones? Does 4.1 needs more bitrate to prevent blocking? What is the main advantage of 4.1 over 3.1 and why it shouldn't be used to encode SD?

PS. How do you Guys predict needed bitrate for your encodes? I usually encode noisy blocked paper-DVD materials that need heavy degraining, deblocking, sometimes quality-deinterlacing and new grain. I would've always thought if movie is low-motion I could lower the bitrate but my experience shows me everything from low-quality dvds needs extra bitrate not to enlarge blocking artifacts. I usually do some sample encodes with different bitrate to observe changes, how do You do that?

PS2. Movies from Paper-DVDs often come with colored noise/notNoise - compression or NTSCtoPAL/deinterlacing artifacts. How do You deal with it? Neither deblock nor MDegrain2 can do the trick because this is rather still. Anything You use and wish to recommend?

thank You for the response.

[benwaggoner]

Originally Posted by kenpachi

I've tried to reproduce the problem but meanwhile my friend had downgraded nVidia drivers and it's impossible now. I don't see the mentioned difference between CPU and DXVA decoding anymore.

Ah, driver issue! As it so often is...

@benwaggoner: Why is it said to encode with L3.1 for SD materials and L4.1 for HD ones? Does 4.1 needs more bitrate to prevent blocking? What is the main advantage of 4.1 over 3.1 and why it shouldn't be used to encode SD?

People encoding for archival use or PC playback, instead of for device-compatible content delivered to customers? For real-world content, you can get aweseome quality within the Level requirements for the lowest level that provides the frame size and rate of your content.

4.1 allows for >25 Mbps bitrates versus 4.0. But with good modern encoders and flexible decoders, you just don't need bitrates that high that often. For Blu-ray, maybe with some edge cases, but Level 4.1 also forces 4 slices, 1 second GOPs, strictly heirarchical pyramids, and other things that degrade compression efficiency a little. But with x264, I can make a Level 4.0 compliant file that looks as good as anything I can do at Level 4.1.

I generally use:
SD (up to 720x576p25 or 720x480p30): Level 3.0
720p: Level 3.1
1080p: Level 4.0

PS. How do you Guys predict needed bitrate for your encodes?

Use CRF if you care about quality instead of bitrate. Generally my task is to provide highest quality at specific bitrates, so the bitrates are the constant and how to spend them is the craft.

I usually encode noisy blocked paper-DVD materials that need heavy degraining, deblocking, sometimes quality-deinterlacing and new grain. I would've always thought if movie is low-motion I could lower the bitrate but my experience shows me everything from low-quality dvds needs extra bitrate not to enlarge blocking artifacts.

Motion hides noise and detail well, particularly for film sources with that nice 1/48th of a second shutter speed and lots of motion blur. A static shot with grain or noise can make compression artifacts much more obvious.

I usually do some sample encodes with different bitrate to observe changes, how do You do that?

Trick question ?

PS2. Movies from Paper-DVDs often come with colored noise/notNoise - compression or NTSCtoPAL/deinterlacing artifacts. How do You deal with it? Neither deblock nor MDegrain2 can do the trick because this is rather still. Anything You use and wish to recommend?

There's only so much you can do with bad source. For my own work, I then request earlier-generation copies of the sources from the vendor so I can do preprocessing right. It's way harder to fix someone else's preprocessing than it is to do it over right.

But if that's not possible, well, that's what After Effects is for. That can be a rathole, though: sufficiently advanced preprocessing is indistriguishable from rotoscoping (apologies to Arthur C. Clarke).

[Blue_MiSfit]

Level 4.1 also forces 4 slices, 1 second GOPs, strictly heirarchical pyramids, and other things that degrade compression efficiency a little

Wait a second... are you sure this applies to Level 4.1 in general?? I thought it applied to BluRay specifically.

[sneaker_ger]

Blu-Ray only.

[benwaggoner]

Originally Posted by Blue_MiSfit

Wait a second... are you sure this applies to Level 4.1 in general?? I thought it applied to BluRay specifically.

Yes, those are just for Blu-ray.

Level 4.1 might be better than Level 4.0 for Blu-ray encoding with very challenging sources, even if it wouldn't a material difference with a more flexible, fully Level 4.0 compliant decoder.