Problem encoding with pc scale.

[Selur]

I encode my input file Color Range Test.MOV (https://drive.google.com/drive/folders/0B_WxUS1XGCPASUZibG5XZkRfeTg?usp=sharing) using:
ffmpeg -y -loglevel fatal -threads 8 -i "C:\Users\Selur\Desktop\Color Range Test.MOV" -map 0:0 -an -sn -vsync 0 -vf zscale=rangein=full:range=full -pix_fmt yuv420p -f yuv4mpegpipe - | x264 --crf 18.00 --profile high --level 4.1 --sync-lookahead 27 --vbv-maxrate 62500 --vbv-bufsize 78125 --sar 1:1 --non-deterministic --range pc --colorprim bt709 --transfer bt709 --colormatrix bt470bg --demuxer y4m --fps 30000/1001 --output-depth 8 --output "H:\Temp\19_35_41_6110_01.264" -
then I compare it using:
# Imports
import vapoursynth as vs
core = vs.get_core()
# Loading Plugins
core.std.LoadPlugin(path="G:/Hybrid/64bit/vsfilters/SourceFilter/LSmashSource/vslsmashsource.dll")


# Loading H:\Output\Color Range Test.MOV using LibavSMASHSource
clip = core.lsmas.LibavSMASHSource(source="C:/Users/Selur/Desktop/Color Range Test.MOV")
# making sure input color matrix is set as 470bg
clip = core.resize.Point(clip, matrix_in_s="470bg")
# making sure frame rate is set to 30000/1001
clip = core.std.AssumeFPS(clip, fpsnum=30000, fpsden=1001)
# Making sure input color range is set to PC (full) range.
clip = core.std.SetFrameProp(clip=clip, prop="_ColorRange", intval=0)
clip = core.text.Text(clip=clip, text="Original")
clip = core.hist.Levels(clip=clip)

#### x264
clip10 = core.lsmas.LWLibavSource(source="H:/Output/x264.mkv", cache=0)
# making sure input color matrix is set as 470bg
clip10 = core.resize.Point(clip10, matrix_in_s="470bg")
# making sure frame rate is set to 30000/1001
clip10 = core.std.AssumeFPS(clip10, fpsnum=30000, fpsden=1001)
# Making sure input color range is set to PC (full) range.
clip10 = core.std.SetFrameProp(clip=clip10, prop="_ColorRange", intval=0)
clip10 = core.text.Text(clip=clip10, text="x264")
clip10 = core.hist.Levels(clip=clip10)

clip = core.std.StackVertical([clip, clip10])
# Output
clip.set_output()

and the luma channel is totally decimated.
https://s9.postimg.cc/i5q19l9e3/compare_x264.png (https://postimg.cc/image/i5q19l9e3/)

Using x265, DivX265, Kvazaar, NVEnc (H.264), NVEnc (H.265) and vp9 this does not happen.
Usin 'rawvideo' (and '-demuxer raw' in x264) instead of using 'yvu4mpegpipe' doesn't make a difference.

-> Is something broken on my end? Can someone reproduce this? Is this a bug or am I making a mistake somewhere and simply don't see it?

I'm using:
x264 0.155.2901 7d0ff22
(libswscale 5.2.100)
(libavformat 58.13.100)
(ffmpegsource 2.30.0.0)
built on Apr 20 2018, gcc: 7.3.0
x264 configuration: --chroma-format=all
libx264 configuration: --chroma-format=all
x264 license: GPL version 2 or later
libswscale/libavformat/ffmpegsource license: GPL version 2 or later
and
ffmpeg version N-91072-gcae004cabb Copyright (c) 2000-2018 the FFmpeg developers
built with gcc 7.3.0 (Rev2, Built by MSYS2 project)
configuration: --disable-debug --enable-gmp --enable-libmp3lame --enable-libopus --enable-libvorbis --enable-libvpx --enable-libx264 --enable-libx265 --enable-fontconfig --enable-gnutls --enable-libass --enable-libbluray --enable-libfreetype --enable-libmfx --enable-libmysofa --enable-libopencore-amrnb --enable-libopencore-amrwb --enable-libopenjpeg --enable-libsnappy --enable-libsoxr --enable-libspeex --enable-libtheora --enable-libtwolame --enable-libvidstab --enable-libvo-amrwbenc --enable-libwavpack --enable-libwebp --enable-libxml2 --enable-libzimg --enable-libshine --enable-gpl --enable-avisynth --extra-cflags=-DLIBTWOLAME_STATIC --extra-libs=-lstdc++ --extra-cflags=-DLIBXML_STATIC --enable-version3
libavutil 56. 18.102 / 56. 18.102
libavcodec 58. 19.101 / 58. 19.101
libavformat 58. 13.102 / 58. 13.102
libavdevice 58. 4.100 / 58. 4.100
libavfilter 7. 22.100 / 7. 22.100
libswscale 5. 2.100 / 5. 2.100
libswresample 3. 2.100 / 3. 2.100
libpostproc 55. 2.100 / 55. 2.100
which should be up-to-date afaik.

Cu Selur

Ps.: This also happens to me when encoding to 10bit x264.

[poisondeathray]

The 8bit to 8bit looks ok to me; you get some minor expected loss at crf8 (you used crf18; I used 8bit avs input this time), but there is no line artifact in the waveform (check the classic histogram). The 8bit (scaled to 10bit), 10bit encoding result is unexpected - it's as if some low midtone values congregate at 1 value. You can interleave the results in vapoursynth or check in avspmod tabs if using avisynth , it's very bizzarre result . I would expect a more random distribution instead of a line

[Selur]

The 8bit to 8bit looks ok to me;
What about the totally flat luma? (see histogram in screenshot)

there is no line artifact in the waveform
Yes, 8bit doesn't have the line artifacts. :)

[poisondeathray]

What about the totally flat luma? (see histogram in screenshot)



It depends on what you mean by "flat"

You're actually looking at 2 different views with histogram("levels") vs histogram("classic") mode. A higher "peak" in histogram("levels") mode just means more values of that number.

The values are still 0 to 255 (ie you still have min of zero and max of 255) , there is just a redistribution in the midrange values and the errors aggregate around a low midtone value, creating that line .

It's a bizarre unexpected pattern, but if you scrub through all the frames, they are all affected



Do you have another section, another scene from that clip ?

[Selur]

Do you have another section, another scene from that clip ?
Sadly no. Got the clip from a user of Hybrid, which stumbled over it complaining that Hybrid did something wrong.

The values are still 0 to 255 (ie you still have min of zero and max of 255) , there is just a redistribution in the midrange values and the errors aggregate around a low midtone value, creating that line .
I totally overlooked that, just compared the curves. :)

From the looking in classic view, x264 seems to do some additional scaling.
https://s9.postimg.cc/hj0bbvbxn/compare_x264_classic.png (https://postimg.cc/image/hj0bbvbxn/)
Not sure how to express this properly, seems like the 0-255 range is scaled like during a 16:235->0:255 conversion and then the 'overflow' is removed.

Cu Selur

[poisondeathray]

From the looking in classic view, x264 seems to do some additional scaling.

Not sure how to express this properly, seems like the 0-255 range is scaled like during a 16:235->0:255 conversion and then the 'overflow' is removed.



Are you referring to the 8bit to 8bit scenario

or

(8bit to 10bit) to 10bit encoding scenario ?


It doesn't look like that for me in my encode ; but your screenshot shows an issue , like it was scaled twice . How did you do that encode ?

[poisondeathray]

I think you have a command issue ; for x264 you need to specify --input-range pc --range pc , or maybe your pipe isn't working (strict -1 for full range? I used rawvideo pipe in the other example) . Eitherway it looks like you're scaling instead of passing through values. You can see the banding in the histogram

As expected, the 8bit to 8bit case is very similar
https://s31.postimg.cc/cephtwlsr/source.png
https://s31.postimg.cc/rao11j7i3/8bit_x264_crf8.png

[Selur]

added two additional sources (go google drive folder), will test and answer question tonight after work :)