Help with removing artefacts from encode

[Oblak]

Hi

First of all, I am complete noob. So sorry for any "stupid" questions where you all may know the answers.

tl;dr I am trying to do 4k HDR HEVC encode from BR and as result I get these artefacts in some dark scenes https://i.imgur.com/XylLpHe.jpg and here is the same area in original video https://i.imgur.com/YQUWWG8.jpg
I am using ffworks I tried to increase bitrate and some changes.

The only improvement I have seen was when I changed Coder type from Arithmentic (CABAC) which is as default to NONE. And Scaler to Accurate rounding.

But at the end I am still getting artefacts.

Here are the parameters I am getting (2 pass encode)

wpp / no-pmode / no-pme / no-psnr / no-ssim / log-level=2 / input-csp=1 / input-res=3840x2160 / interlace=0 / total-frames=0 / level-idc=51 / high-tier=1 / uhd-bd=0 / ref=3 / no-allow-non-conformance / repeat-headers / annexb / no-aud / no-hrd / info / hash=0 / no-temporal-layers / open-gop / min-keyint=24 / keyint=250 / gop-lookahead=0 / bframes=4 / b-adapt=2 / b-pyramid / bframe-bias=0 / rc-lookahead=20 / lookahead-slices=8 / scenecut=40 / radl=0 / no-splice / no-intra-refresh / ctu=64 / min-cu-size=8 / no-rect / no-amp / max-tu-size=32 / tu-inter-depth=1 / tu-intra-depth=1 / limit-tu=0 / rdoq-level=0 / dynamic-rd=0.00 / no-ssim-rd / signhide / no-tskip / nr-intra=0 / nr-inter=0 / no-constrained-intra / strong-intra-smoothing / max-merge=3 / limit-refs=1 / no-limit-modes / me=1 / subme=2 / merange=57 / temporal-mvp / no-hme / weightp / no-weightb / no-analyze-src-pics / deblock=0:0 / sao / no-sao-non-deblock / rd=3 / early-skip / rskip / no-fast-intra / no-tskip-fast / no-cu-lossless / b-intra / no-splitrd-skip / rdpenalty=0 / psy-rd=2.00 / psy-rdoq=0.00 / no-rd-refine / no-lossless / cbqpoffs=0 / crqpoffs=0 / rc=cbr / bitrate=20000 / qcomp=0.60 / qpstep=4 / stats-write=0 / stats-read=0 / vbv-maxrate=20000 / vbv-bufsize=5000 / vbv-init=0.8 / ipratio=1.40 / pbratio=1.30 / aq-mode=3 / aq-strength=1.00 / cutree / zone-count=0 / no-strict-cbr / qg-size=32 / no-rc-grain / qpmax=69 / qpmin=0 / no-const-vbv / sar=1 / overscan=0 / videoformat=5 / range=0 / colorprim=9 / transfer=16 / colormatrix=9 / chromaloc=0 / display-window=0 / master-display=G(13250,34500)B(7500,3000)R(34000,16000)WP(15635,16450)L(10000000,1)cll=0,0 / min-luma=0 / max-luma=1023 / log2-max-poc-lsb=8 / vui-timing-info / vui-hrd-info / slices=1 / no-opt-qp-pps / no-opt-ref-list-length-pps / no-multi-pass-opt-rps / scenecut-bias=0.05 / no-opt-cu-delta-qp / no-aq-motion / hdr / hdr-opt / no-dhdr10-opt / no-idr-recovery-sei / analysis-reuse-level=5 / scale-factor=0 / refine-intra=0 / refine-inter=0 / refine-mv=0 / refine-ctu-distortion=0 / no-limit-sao / ctu-info=0 / no-lowpass-dct / refine-analysis-type=0 / copy-pic=1 / max-ausize-factor=1.0 / no-dynamic-refine / no-single-sei / no-hevc-aq / no-svt / no-field / qp-adaptation-range=1.00

Here is mediainfo from original video
Video
ID : 1
Format : HEVC
Format/Info : High Efficiency Video Coding
Format profile : Main 10@L5.1@High
HDR format : SMPTE ST 2086, HDR10 compatible
Codec ID : V_MPEGH/ISO/HEVC
Duration : 1h 30mn
Bit rate : 63.2 Mbps
Width : 3 840 pixels
Height : 2 160 pixels
Display aspect ratio : 16:9
Frame rate mode : Constant
Frame rate : 24.000 fps
Color space : YUV
Chroma subsampling : 4:2:0 (Type 2)
Bit depth : 10 bits
Bits/(Pixel*Frame) : 0.317
Stream size : 39.8 GiB (100%)
Title : MPEG-H HEVC Video / 63315 kbps / 2160p / 24 fps / 16:9 / Main 10 @ Level 5.1 @ High / 4:2:0 / 10 bits / HDR10 / BT.2020
Language : English
Default : Yes
Forced : No
Color range : Limited
Color primaries : BT.2020
Transfer characteristics : PQ
Matrix coefficients : BT.2020 non-constant
Mastering display color primaries : BT.2020
Mastering display luminance : min: 0.0001 cd/m2, max: 1000 cd/m2


Thanks in advance for any advice

[sneaker_ger]

Why so little buffer? Why maxrate = avg. bitrate?
rc=cbr / bitrate=20000 / vbv-maxrate=20000 / vbv-bufsize=5000

Can you post your complete command-line (and any scripts or whatever it is you use in your workflow)? It is a bit difficult to see problems in such a big block of settings.

[Asmodian]

The only improvement I have seen was when I changed Coder type from Arithmentic (CABAC) which is as default to NONE.

This is very odd, why is there a none option? Arithmetic coding is like zip compression, it should not affect quality, just size and decoding requirements. What is this controlling? I am not familiar with ffworks but it sounds like it has some non-standard options or something? :confused:

[Natty]

stats-write=0 / stats-read=0 / rc=cbr

this is not 2 pass like you said.

[RanmaCanada]

Why so little buffer? Why maxrate = avg. bitrate?


Can you post your complete command-line (and any scripts or whatever it is you use in your workflow)? It is a bit difficult to see problems in such a big block of settings.

Agree. We need your command line as the parameters don't give us enough information.

[Oblak]

thank you for replies

@Asmodian - that should be -coder parameter in ffmpeg no idea why it is turned on

@sneaker_ger reason is that I locked bitrate at 20k.

@Natty - posted export was done from sample. I did 2 pass on full video, artefacts were still there and looked the same.

regardig command line, as mentioned in first post I am using ffworks, there is no command line

[sneaker_ger]

@sneaker_ger reason is that I locked bitrate at 20k.
Ok, but why?
Usually we don't set vbv so low unless you have a good reason to do otherwise (which it seems you don't have). Set buffer size and maximum rate to at least 50000. (Average bitrate can stay at 20000.)

[Oblak]

Ok, but why?
Usually we don't set vbv so low unless you have a good reason to do otherwise (which it seems you don't have). Set buffer size and maximum rate to at least 50000. (Average bitrate can stay at 20000.)

sorry for late reply I was away

I tied, got little bit better result, but it still needs improvement.

Again I am no expert on this stuff, but I believe it would be one of those stupid settings which need change, not only bitrate...

anyway here is result https://i.imgur.com/PT75Zvn.jpg

/ wpp / no-pmode / no-pme / no-psnr / no-ssim / log-level=2 / input-csp=1 / input-res=3840x2160 / interlace=0 / total-frames=0 / level-idc=51 / high-tier=1 / uhd-bd=0 / ref=3 / no-allow-non-conformance / repeat-headers / annexb / no-aud / no-hrd / info / hash=0 / no-temporal-layers / open-gop / min-keyint=24 / keyint=48 / gop-lookahead=0 / bframes=4 / b-adapt=2 / b-pyramid / bframe-bias=0 / rc-lookahead=20 / lookahead-slices=8 / scenecut=40 / radl=0 / no-splice / no-intra-refresh / ctu=64 / min-cu-size=8 / no-rect / no-amp / max-tu-size=32 / tu-inter-depth=1 / tu-intra-depth=1 / limit-tu=0 / rdoq-level=0 / dynamic-rd=0.00 / no-ssim-rd / signhide / no-tskip / nr-intra=0 / nr-inter=0 / no-constrained-intra / strong-intra-smoothing / max-merge=3 / limit-refs=1 / no-limit-modes / me=1 / subme=2 / merange=57 / temporal-mvp / no-hme / weightp / no-weightb / no-analyze-src-pics / deblock=0:0 / sao / no-sao-non-deblock / rd=3 / early-skip / rskip / no-fast-intra / no-tskip-fast / no-cu-lossless / b-intra / no-splitrd-skip / rdpenalty=0 / psy-rd=2.00 / psy-rdoq=0.00 / no-rd-refine / no-lossless / cbqpoffs=0 / crqpoffs=0 / rc=abr / bitrate=20000 / qcomp=0.60 / qpstep=4 / stats-write=0 / stats-read=0 / vbv-maxrate=50000 / vbv-bufsize=50000 / vbv-init=0.8 / ipratio=1.40 / pbratio=1.30 / aq-mode=3 / aq-strength=1.00 / cutree / zone-count=0 / no-strict-cbr / qg-size=32 / no-rc-grain / qpmax=69 / qpmin=0 / no-const-vbv / sar=1 / overscan=0 / videoformat=5 / range=0 / colorprim=9 / transfer=16 / colormatrix=9 / chromaloc=0 / display-window=0 / master-display=G(13250,34500)B(7500,3000)R(34000,16000)WP(15635,16450)L(10000000,1)cll=0,0 / min-luma=0 / max-luma=1023 / log2-max-poc-lsb=8 / vui-timing-info / vui-hrd-info / slices=1 / no-opt-qp-pps / no-opt-ref-list-length-pps / no-multi-pass-opt-rps / scenecut-bias=0.05 / no-opt-cu-delta-qp / no-aq-motion / hdr / hdr-opt / no-dhdr10-opt / no-idr-recovery-sei / analysis-reuse-level=5 / scale-factor=0 / refine-intra=0 / refine-inter=0 / refine-mv=0 / refine-ctu-distortion=0 / no-limit-sao / ctu-info=0 / no-lowpass-dct / refine-analysis-type=0 / copy-pic=1 / max-ausize-factor=1.0 / no-dynamic-refine / no-single-sei / no-hevc-aq / no-svt / no-field / qp-adaptation-range=1.00

and I forgot to turn on the variable bitrate setting, so here it is https://i.imgur.com/f6X7wTe.jpg

[Oblak]

ok I noticed new thing, I tried to change YUV420p10le to YUV420p12le and this is result https://i.imgur.com/96mjJt9.jpg. It is not "perfect", but it is for sure huge improvement and result is good enough for me.

So is there any way how to achieve this with YUV420p10le?
If I leave it on YUV420p12le can this cause some trouble with playback in different devices/players?

Thanks

[Natty]

it depends on your whole avs/vs script and output depth of encoder

[sneaker_ger]

Such a big difference between 10 and 12 bit is unexpectet. The problem is few people here know or use ffworks. It's hard to tell what is does internally. Can you cut e.g. 30 seconds from the problematic part using mkvtoolnix (or ffworks in video copy mode if that is possible)? Then others can test if they have the same problem with x265 or if it is a problem specific to ffworks.

[HolyWu]

regardig command line, as mentioned in first post I am using ffworks, there is no command line

Do you see there is a "Show FFmpeg Terminal Output" icon like the one in https://youtu.be/9m-y2ImYvYg?t=100? If so, post whatever it showed.

[Oblak]

@HolyWu good find!

I stopped it after few frames. This is my original config with limited framerate.. just to give idea what is going on when doing encode

Library/Application\ Support/FFmpegTools/ffmpeg -probesize 50M -analyzeduration 100M -i videoname.mkv -pass 1 -passlogfile iffmpeg_5468652E466F672E313938302E32313630702E5548442E426C755261792E52454D55582E4844522E484556432E4454532D48442E4D412E352E312D455053694C4F4E2E6D6B76 -fastfirstpass 1 -an -map 0:0 -c:v libx265 -b:v 20000k -minrate 20000k -maxrate 20000k -bufsize 5000k -r 24000/1001 -s 3840x2160 -aspect 16:9 -pix_fmt yuv420p10le -x265-params profile=main10:level=51:me=hex:scenecut=40:bframes=4:min-keyint=24:keyint=48:hdr=1:hdr-opt=1:master-display="G(13250,34500)B(7500,3000)R(34000,16000)WP(15635,16450)L(40000000,1)":max-cll="1242,436" -color_primaries bt2020 -color_trc smpte2084 -colorspace bt2020_ncl -qmin 3 -qmax 51 -threads 4 -sn -f rawvideo -y /dev/null

ffmpeg version N-94454-gad97be9fa3-tessus https://evermeet.cx/ffmpeg/ Copyright (c) 2000-2019 the FFmpeg developers
built with Apple LLVM version 10.0.1 (clang-1001.0.46.4)
configuration: --cc=/usr/bin/clang --prefix=/opt/ffmpeg --extra-version=tessus --enable-avisynth --enable-fontconfig --enable-gpl --enable-libaom --enable-libass --enable-libbluray --enable-libdav1d --enable-libfreetype --enable-libgsm --enable-libmodplug --enable-libmp3lame --enable-libmysofa --enable-libopencore-amrnb --enable-libopencore-amrwb --enable-libopenh264 --enable-libopenjpeg --enable-libopus --enable-librubberband --enable-libshine --enable-libsnappy --enable-libsoxr --enable-libspeex --enable-libtheora --enable-libtwolame --enable-libvidstab --enable-libvmaf --enable-libvo-amrwbenc --enable-libvorbis --enable-libvpx --enable-libwavpack --enable-libwebp --enable-libx264 --enable-libx265 --enable-libxavs --enable-libxvid --enable-libzimg --enable-libzmq --enable-libzvbi --enable-version3 --pkg-config-flags=--static --disable-ffplay
libavutil 56. 33.100 / 56. 33.100
libavcodec 58. 55.100 / 58. 55.100
libavformat 58. 30.100 / 58. 30.100
libavdevice 58. 9.100 / 58. 9.100
libavfilter 7. 58.100 / 7. 58.100
libswscale 5. 6.100 / 5. 6.100
libswresample 3. 6.100 / 3. 6.100
libpostproc 55. 6.100 / 55. 6.100
Input #0, matroska,webm, from 'videoname.mkv':
Metadata:
title : The Fog (1980)
encoder : libebml v1.3.9 + libmatroska v1.5.2
creation_time : 2019-09-08T13:08:02.000000Z
Duration: 01:30:06.54, start: 0.000000, bitrate: 63188 kb/s
Stream #0:0(eng): Video: hevc (Main 10), yuv420p10le(tv, bt2020nc/bt2020/smpte2084), 3840x2160 [SAR 1:1 DAR 16:9], 24 fps, 24 tbr, 1k tbn, 24 tbc (default)
Metadata:
title : MPEG-H HEVC Video / 63315 kbps / 2160p / 24 fps / 16:9 / Main 10 @ Level 5.1 @ High / 4:2:0 / 10 bits / HDR10 / BT.2020
BPS-eng : 63185874
DURATION-eng : 01:30:06.542000000
NUMBER_OF_FRAMES-eng: 129757
NUMBER_OF_BYTES-eng: 42702135287
_STATISTICS_WRITING_APP-eng: mkvmerge v37.0.0 ('Leave It') 64-bit
_STATISTICS_WRITING_DATE_UTC-eng: 2019-09-08 13:08:02
_STATISTICS_TAGS-eng: BPS DURATION NUMBER_OF_FRAMES NUMBER_OF_BYTES
Codec AVOption fastfirstpass (Use fast settings when encoding first pass) specified for output file #0 (/dev/null) has not been used for any stream. The most likely reason is either wrong type (e.g. a video option with no video streams) or that it is a private option of some encoder which was not actually used for any stream.
Stream mapping:
Stream #0:0 -> #0:0 (hevc (native) -> hevc (libx265))
Press [q] to stop, [?] for help
[libx265 @ 0x7fbc1a806000] Unknown option: profile.
x265 [info]: HEVC encoder version 3.1+10-459d3822c608
x265 [info]: build info [Mac OS X][clang 10.0.1][64 bit] 10bit
x265 [info]: using cpu capabilities: MMX2 SSE2Fast LZCNT SSSE3 SSE4.2 AVX FMA3 BMI2 AVX2
x265 [info]: Main 10 profile, Level-5.1 (Main tier)
x265 [info]: Thread pool created using 4 threads
x265 [info]: Slices : 1
x265 [info]: frame threads / pool features : 4 / wpp(34 rows)
x265 [info]: Coding QT: max CU size, min CU size : 64 / 8
x265 [info]: Residual QT: max TU size, max depth : 32 / 1 inter / 1 intra
x265 [info]: ME / range / subpel / merge : hex / 57 / 2 / 3
x265 [info]: Keyframe min / max / scenecut / bias: 24 / 48 / 40 / 5.00
x265 [info]: Lookahead / bframes / badapt : 20 / 4 / 2
x265 [info]: b-pyramid / weightp / weightb : 1 / 1 / 0
x265 [info]: References / ref-limit cu / depth : 3 / off / on
x265 [info]: AQ: mode / str / qg-size / cu-tree : 2 / 1.0 / 32 / 1
x265 [info]: Rate Control / qCompress : ABR-20000 kbps / 0.60
x265 [info]: VBV/HRD buffer / max-rate / init : 5000 / 20000 / 0.750
x265 [info]: tools: rd=3 psy-rd=2.00 early-skip rskip signhide tmvp b-intra
x265 [info]: tools: strong-intra-smoothing lslices=8 deblock sao
Output #0, rawvideo, to '/dev/null':
Metadata:
title : Videoname
encoder : Lavf58.30.100
Stream #0:0(eng): Video: hevc (libx265), yuv420p10le(bt2020nc/bt2020/smpte2084), 3840x2160 [SAR 1:1 DAR 16:9], q=3-51, 20000 kb/s, 23.98 fps, 23.98 tbn, 23.98 tbc (default)
Metadata:
title : MPEG-H HEVC Video / 63315 kbps / 2160p / 24 fps / 16:9 / Main 10 @ Level 5.1 @ High / 4:2:0 / 10 bits / HDR10 / BT.2020
BPS-eng : 63185874
DURATION-eng : 01:30:06.542000000
NUMBER_OF_FRAMES-eng: 129757
NUMBER_OF_BYTES-eng: 42702135287
_STATISTICS_WRITING_APP-eng: mkvmerge v37.0.0 ('Leave It') 64-bit
_STATISTICS_WRITING_DATE_UTC-eng: 2019-09-08 13:08:02
_STATISTICS_TAGS-eng: BPS DURATION NUMBER_OF_FRAMES NUMBER_OF_BYTES
encoder : Lavc58.55.100 libx265
Side data:
cpb: bitrate max/min/avg: 20000000/0/20000000 buffer size: 5000000 vbv_delay: -1
frame= 10 fps=0.0 q=0.0 size= 0kB time=00:00:00.00 bitrate=N/A speed= 0x
frame= 19 fps= 18 q=0.0 size= 0kB time=00:00:00.00 bitrate=N/A speed= 0x
frame= 26 fps= 16 q=0.0 size= 0kB time=00:00:00.00 bitrate=N/A speed= 0x
frame= 28 fps=9.0 q=0.0 size= 0kB time=00:00:00.00 bitrate=N/A speed= 0x
frame= 32 fps=6.3 q=-0.0 size= 4kB time=00:00:00.00 bitrate=N/A speed= 0x
frame= 34 fps=6.1 q=-0.0 size= 5kB time=00:00:00.08 bitrate= 492.3kbits/s speed=0.015x
frame= 37 fps=5.9 q=-0.0 size= 6kB time=00:00:00.20 bitrate= 228.3kbits/s speed=0.0335x



@sneaker_ger what software do you recommend? I thought ffmpeg is the thing most people use.. again I am noob in this
here is the sample. not sure which hosting is best so I hope mega is ok for you https://mega.nz/#!hJdgGYIT!4QQzFYkir7tXNUxwRC9sy80OyrLVLeyY7DQ3n8rIr9k 307MB

[RanmaCanada]

This is piracy related, rule 6 violation.

[Oblak]

This is piracy related, rule 6 violation.

I am not able to simulate this issue on other video so I had to post 30s from the original

but yes, you can be a proactive good boy, report it to mods and get it removed you will maybe even get a cookie for being such a good boy.

but don't help finding solution to this specific technical issue which can maybe help lots of other people in future.

[RanmaCanada]

I am not able to simulate this issue on other video so I had to post 30s from the original

but yes, you can be a proactive good boy, report it to mods and get it removed you will maybe even get a cookie for being such a good boy.

but don't help finding solution to this specific technical issue which can maybe help lots of other people in future.
No, your problem is you downloaded an pirate scene remux of a movie and you want to shrink it, but you don't know how, so you are asking the community how to pirate materials. Violation of rule 6.

You've edited your post to take out the original filenames, and then you turned hostile on me when I pointed out that you actively broke the rules. Rules that keep this board safe.

[Blue_MiSfit]

Closing