My x264 encodes look smeared

[thymej]

Sorry if I posted this in the wrong forum, the other option was MPEG4 GUIs

Am new to x264 and I just start experimenting with x264 via MeGUI as the front end. I’m testing with OTA HD recordings that I convert to MP4 files. I leave the resolution the same as it was broadcasted OTA. I only de-interlace and or decimate to bring the frame rate to 23.976fps.

Here is what I noticed. The MP4 file when comparing side by side with the original mpeg2 TS files, the MP4 from x264 looks smeared; as if a smother filter was used on it. I lose lots of detail. Like with black hair you can no longer make out the hair streaks in the MP4 x264 file. In the original you can see all the hair streaks.

In my testing I found if I change the MeGUI profile on the ‘RC and ME’ rate control section, I change the setting ‘Temp. Blur of est. Frame complexity’ to 1 (the default is 20) Changing this setting adds the x265 option –cplxblur 1.

With only the –cplxblur 1 option the problem I had is 99% gone. The over smoothing is gone and I can see the detail in the black hair and dark areas again.

My question is about this setting –cplxblur, the comment in MeGUI say leave the default if you don’t know what you are doing. Will changing this setting make the MP4 file less compatible or does this setting only change the image (or does this setting change how its decoded during playback)?

What am I doing wrong, or what other setting needs to be changed to stop x264 from overly smoothing the image?

ThymeJ

[Dark Shikari]

cplxblur should change basically nothing, especially if AQ is enabled.

"Temporal blur of frame complexity" has nothing to do with smearing or blurring whatsoever, its just a ratecontrol parameter that controls subtly how ratecontrol distributes bits.

Are you sure you didn't change anything else, like updating x264 or changing the bitrate?

[J_Darnley]

That rate control did you use and what setting did you use it at? I.e. If you used a target bitrate, what bitrate did you choose? You may just not be giving the video enough bits to compress file detail. What in-loop filter settings did you use? Post the command line out of the MeGUI config. window. Can you also tell us what version of x264 you are using? You can find this out from the update window.

[thymej]

My input file is:

-------
MyVideoD2V="HDTest.d2v"
SetMemoryMax(1024)
MPEG2Source(MyVideoD2V, idct=3)
TFM(PP=0,d2v=MyVideoD2V,slow=2)
TDecimate()
--------------

I use the MeGUI profile PD-PS3-Xbox360, I will have to run two encodes to get the command line since I use auto2pass with a bit rate of 4000k

The only setting I changed in the profile is the 'Temporal blur of frame complexity'. I change it to 1 from 20.

So --cplxblur is not a Temporal smother that smooths out micro blocks? So if my question shows my newness to this stuff.

[Dark Shikari]

So --cplxblur is not a Temporal smother that smooths out micro blocks? So if my question shows my newness to this stuff.No, x264 has no filters that run on the input video stream except for --nr, which doesn't even work in multithreaded mode (and is off by default).

[thymej]

Temp blur set to 1
pass1
--pass 1 --bitrate 4000 --stats "HDTest.stats" --level 4.1 --bframes 3 --direct auto --subme 1 --partitions none --ipratio 1.1 --pbratio 1.1 --vbv-bufsize 9000 --vbv-maxrate 20000 --qcomp 0.5 --cplxblur 1 --qblur 0.3 --me dia --merange 12 --threads auto --thread-input --sar 1:1 --progress --no-psnr --no-ssim --output NUL "HDTest.avs"

pass2
--pass 2 --bitrate 4000 --stats "HDTest.stats" --level 4.1 --ref 3 --mixed-refs --bframes 3 --b-rdo --bime --weightb --direct auto --subme 6 --trellis 1 --partitions p8x8,b8x8,i4x4,i8x8 --8x8dct --ipratio 1.1 --pbratio 1.1 --vbv-bufsize 9000 --vbv-maxrate 20000 --qcomp 0.5 --cplxblur 1 --qblur 0.3 --me umh --merange 12 --threads auto --thread-input --sar 1:1 --progress --no-psnr --no-ssim --output "HDTest.mp4" "HDTest.avs"

Temp blur set to 20 (default)
pass1
--pass 1 --bitrate 4000 --stats "HDTest.stats" --level 4.1 --bframes 3 --direct auto --subme 1 --partitions none --ipratio 1.1 --pbratio 1.1 --vbv-bufsize 9000 --vbv-maxrate 20000 --qcomp 0.5 --qblur 0.3 --me dia --merange 12 --threads auto --thread-input --sar 1:1 --progress --no-psnr --no-ssim --output NUL "HDTest.avs"

pass2
--pass 2 --bitrate 4000 --stats "HDTest.stats" --level 4.1 --ref 3 --mixed-refs --bframes 3 --b-rdo --bime --weightb --direct auto --subme 6 --trellis 1 --partitions p8x8,b8x8,i4x4,i8x8 --8x8dct --ipratio 1.1 --pbratio 1.1 --vbv-bufsize 9000 --vbv-maxrate 20000 --qcomp 0.5 --qblur 0.3 --me umh --merange 12 --threads auto --thread-input --sar 1:1 --progress --no-psnr --no-ssim --output "HDTest.mp4" "HDTest.avs"

[thymej]

I dont have the HDTest clip here at my work so I will re-run the clip at home (that had the black haired person in it that I lost the detail in).

[sysKin]

What do you use for playback? If ffdshow (esp. older versions) make sure you disable postprocessing / deblocking.

[thymej]

That was it, ffdshow post processing, I had blur/nr turned on. All looks good now. Thanks for your help.