x265 HEVC Encoder [Archive] - Page 120

brumsky

10th March 2018, 06:47

The first part of your question asks what has better detail retention. the higher --subme value, the higher chance of fine detail retention. --tu-inter-depth does not effect detail retention as I understand it.

--tu-inter-depth is for compression or size. It is how much compression to shoot for. higher value, try to compress more. --limit-tu value effects how long to spend on compression target of --tu-inter-depth.

Hopefully I have read the command line options correctly. Try these set of options to see if they help you, it is what I use for anime "--qcomp 0.8 --rc-lookahead 30 --rc-grain --me 2 --subme 4 --no-strong-intra-smoothing --no-sao"

Thanks for the info! It does help and makes sense! I'll give some of those settings a try as well. I read a different thread where BenWaggner mentioned using --tskip and --tskip-fast with good results with anime. I've read the description a few times and can't quiet figure out what exactly it is skipping... any tips on those?

Asmodian

10th March 2018, 07:45

The residual, the difference between the predicted image and the original image, is coded after a DCT-like transformation, except with tskip this transformation can be skipped. The encoder compares the size of the coded block after the transformation to its size without a transformation and picks whichever is smaller. As a speed boost tskip fast does not compare the size after skipping the transformation at all the possible block size combinations, only a good subset of them.

LigH

10th March 2018, 22:56

x265 2.7+12-98e48e8dd6ab (https://www.mediafire.com/file/5oibs47vn3jpbm1/x265_2.7%2B12-98e48e8dd6ab.7z)

--refine-intra <0..4> Enable intra refinement for encode that uses analysis-load.
...
- 4 : Re-evaluate all intra blocks, does not reuse data from save encode.

--[no-]dynamic-refine Dynamically changes refine-inter level for each CU. Default disabled

--[no-]idr-recovery-sei Emit recovery point infor SEI at each IDR frame

Barough

18th March 2018, 15:27

x265 v2.7+15-b9f5b5d7bf95 (http://www.mediafire.com/file/kkupnoml2ka8b7p/) (GCC 7.3.0, 32 & 64-bit 8/10/12bit Multilib Windows Binaries)

https://bitbucket.org/multicoreware/x265/commits/branch/default

enctac

26th March 2018, 16:20

Problem about x265 2.7+17 and L-SMASH r1459/r1450

L-SMASH : https://github.com/l-smash/l-smash
r1459 binary : http://www.mediafire.com/file/b7w5m1bfh7b5wk6/L-SMASH_rev1459_20180310.zip

x265 2.7+14-d7c26df32fae:[Windows][MSVC 1913][64 bit] 8bit+10bit+12bit
x265 2.7+17-2e370d98c806:[Windows][MSVC 1913][64 bit] 8bit+10bit+12bit

sample.avs
#---
ColorBarsHD(1280,720).Trim(0,500).ConvertToYV12()
ShowFrameNumber(scroll=true,size=128)
#---

ffmpeg.exe -i sample.avs -f yuv4mpegpipe - | x265.exe --y4m - --pass 1 --bitrate 1000 -o sample.265
ffmpeg.exe -i sample.avs -f yuv4mpegpipe - | x265.exe --y4m - --pass 2 --bitrate 1000 -o sample.265

muxer.exe -i "sample.265"?fps=30000/1001 -o sample.mp4

Result

x265 2.7+14 -> No problem.

x265 2.7+17 -> muxer.exe freeze or crash.

Which is wrong ? x265 ? or L-SMASH ?

LigH

26th March 2018, 20:12

Let's check the last 3 commits (https://bitbucket.org/multicoreware/x265/commits/all) ... which may be the most probable: 5be53f0 (https://bitbucket.org/multicoreware/x265/commits/5be53f09eee6ab28bd9dbb2c0ac447f50eb29655) (Clean up SEI::write function)? Sounds like it changed the output in important values.

Sp00kyFox

27th March 2018, 19:49

edit: seems the artifacts were caused by aq-motion. qg-size 8 just counteracts that but the default qg-size 32 without aq-motion actually looks better.
https://mega.nz/#F!DMYT2ICb!4aufAuilmDJAn-Bj-Gzcpg
__________

original post:
about the qg-size discussion. a friend of mine recently asked me about some artifacts he noticed with his x265 encodings. turned out the default qg-size was the culprit.
--qg-size <64|32|16|8>
Enable adaptive quantization for sub-CTUs. This parameter specifies the minimum CU size at which QP can be adjusted, ie. Quantization Group size. Allowed range of values are 64, 32, 16, 8 provided this falls within the inclusive range [maxCUSize, minCUSize].

from what I understand a lower qg-size allows the adaptive quantization to better adjust itself to small details. so why even use a higher value than the lowest one possible? at least for HD resolutions or lower it seems essential to simply set it to 8 to avoid artifacts that can appear otherwise. without being familiar with the parameters of the encoder people would be tempted to just increase their crf value instead.

here check out this scene. source is included if you wanna experiment yourself with it.
link removed

this is the general commandline I used. the different clips only differ at the chosen qg-size value, see the file names for that.
--preset slow --crf 23 --profile main10 --no-sao --aq-motion

after the first scene cut focus on the two guys walking back to their truck. notice the artifact corona that surrounds them. qg-size 8 pretty much fixes it. this also slightly increased the bitrate. so I'm not saying that using --qg-size 8 improves quality at the same file size. but when using crf it might be useful to do that to avoid artifacts you otherwise would need to generally lower crf for.

I suppose the default value is set to 32 instead because the codec is designed for 4k content in mind where it's probably useful to avoid a subblock adaptation at the lowest level due to details being larger.

what's your opinion on this?

Motenai Yoda

28th March 2018, 00:01

coz all was 32x based at early development stages

brumsky

28th March 2018, 00:29

Is it possible to use a UHD HDR source to create an HD SDR encode? I ask because I'm having issues with the encode being desaturated due to the HDR source and BT2020.

How can I convert the color palette and convert to SDR?

sneaker_ger

28th March 2018, 00:33

x265 library/cli doesn't offer HDR->SDR conversion by itself. You can do it using other software, e.g. ffmpeg or VapourSynth and then feed that to x265 (or use the x265 integrated in ffmpeg).

https://forum.doom9.org/showthread.php?t=175125
https://forum.doom9.org/showthread.php?p=1800675
https://github.com/ifb/vapoursynth-tonemap/

enctac

28th March 2018, 08:15

Problem about x265 2.7+17 and L-SMASH r1459/r1450
...

Result

x265 2.7+14 -> No problem.

x265 2.7+17 -> muxer.exe freeze or crash.

Which is wrong ? x265 ? or L-SMASH ?

Let's check the last 3 commits (https://bitbucket.org/multicoreware/x265/commits/all) ... which may be the most probable: 5be53f0 (https://bitbucket.org/multicoreware/x265/commits/5be53f09eee6ab28bd9dbb2c0ac447f50eb29655) (Clean up SEI::write function)? Sounds like it changed the output in important values.

x265 2.7+17 outputs broken SEI. (maybe SEIuserDataUnregistered, extra 16bytes)
Not only 2pass encoding.
Need to fix this.

Left side: 2pass encoding
Right side: 1pass encoding

LigH

28th March 2018, 08:25

I already mentioned that in the x265 developer mailing list. Maybe you would like to register there as well and attach your report.

enctac

28th March 2018, 09:56

OK, I sent a mail.

Boulder

28th March 2018, 10:24

qg-size 8 pretty much fixes it. this also slightly increased the bitrate.

It seems to increase the bitrate quite a lot. I just tested on a 4000-frame 720p encode:

qg-size 64 : 3147,41 kbps
qg-size 32 : 3648,01 kbps
qg-size 16 : 3954,23 kbps

Didn't test qg-size 8 yet.

Average QP is about the same, a little bit less than 21.

Ashok Kumar Mishra

28th March 2018, 10:31

Sp00kyFox

28th March 2018, 12:05

It seems to increase the bitrate quite a lot. I just tested on a 4000-frame 720p encode
how does it visually compare? (inb4: well ok, more bitrate is probably gonna look better) I would think at 720p or even SD a smaller qg-size is even more important since the situation demonstrated in the clip I uploaded is even more likely to appear. from what I've tested edges and details when adjacent to flat areas become more refined with lower qg-size.

Boulder

28th March 2018, 13:47

Didn't do any visual comparisons with those ones yet. I'll now run a 2-pass encode with all four values, using the lowest bitrate (3147 kbps) for all of them as the average bitrate and see how they look. Previously when I just tested changing qg-size and kept all the rest the same, qg-size 32 looked best compared to the original frame (least distortion) on a frame-by-frame comparison.

microchip8

28th March 2018, 17:50

my experience with qg-size on movies (full hd) is that the lower it is, the more fine detail it preserves but also increases ringing around smaller objects. the higher it is, the more smoother the picture and also the less ringing you get. i keep mine at 32 most of the time

Sp00kyFox

28th March 2018, 18:40

wanted to let you know that my first conclusion was overhasty. seems the artifacts were caused by aq-motion. theoretically this options seems to be a nice idea but the approach can cause problem with this kind of scene where you have some small moving details that get crippled by this option. qg-size 8 counteracts that but the result is still worse than the default qg-size 32 without aq-motion. I'm doing some more comparisons regarding qg-size wihout aq-motion this time.

edit: well after further tests I can say qg-size 32 is a good default value. SSIM-wise it seems to be the best choice for general content. in the scene I provided earlier qg-size 64 is little bit too smooth while 32 gives a good balance. 16 and 8 both produce noticeable artifacts.

Barough

29th March 2018, 06:04

x265 v2.7+19-1fafca24a399 (http://www.mediafire.com/file/pl4imr5zqppo4jc/) (GCC 7.3.0, 32 & 64-bit 8/10/12bit Multilib Windows Binaries)

https://bitbucket.org/multicoreware/x265/commits/branch/default

Boulder

29th March 2018, 09:15

edit: well after further tests I can say qg-size 32 is a good default value. SSIM-wise it seems to be the best choice for general content. in the scene I provided earlier qg-size 64 is little bit too smooth while 32 gives a good balance. 16 and 8 both produce noticeable artifacts.

Yes, I agree that the default 32 is a good choice. Fortunately so, as the 25% bitrate increase from the smaller values was a bit steep :)

enctac

29th March 2018, 10:27

Don't use 2.7+15 - 2.7+19.
2.7+14 maybe safe.
We should wait until developers fixes SEI bug.

Result

x265 2.7+14 -> No problem.

x265 2.7+17 -> muxer.exe freeze or crash.

Which is wrong ? x265 ? or L-SMASH ?

x265 2.7+17 outputs broken SEI. (maybe SEIuserDataUnregistered, extra 16bytes)
Not only 2pass encoding.
Need to fix this.

Thank you for reporting this bug. We are fixing it.

pradeeprama

30th March 2018, 06:42

For the curious, of course we will be at NAB in Vegas from April 9th - 12th! We will be one of the demos at the MulticoreWare booth in SU-14708. If you want to discuss all things media, swing by the booth! Suggested topics include, but are certainly not limited to the soon-to-publish AVX512 acceleration, content adaptive optimizations for ABR encoding with x265, upcoming overhaul of AQ and associated visual improvements. or a request to take a selfie with the creators of the world’s most popular HEVC encoder.

See you in Vegas!

PS: Make sure to mention this post on doom9 if you stop by to stand a chance to win some open-source memorabilia!

Midzuki

30th March 2018, 17:33

x265.exe 2.7+20-3440a56acc78

fix bug in SEI::write clean up

https://forum.videohelp.com/threads/357754-%5BHEVC%5D-x265-EXE-mingw-builds?p=2516009#post2516009

Asmodian

1st April 2018, 16:36

Midzuki, I get a report "Trojan:Script/Cloxer.A!cl" for that link and Windows 10 automatically deletes it. This is the first time I have had a warning from x265.exe, is it a false alarm?

LigH

1st April 2018, 16:48

Your antivirus (which?) probably caught a JavaScript related to advertizing in a forum site. It seems not to be related to the attached x265 build.

If you use NoScript and an ad blocker (like uBlock Origin) in your web browser, you will probably not get any advertizing network scripts confusing you before you even see the Video Help forum thread.

Midzuki

1st April 2018, 19:04

Midzuki, I get a report "Trojan:Script/Cloxer.A!cl" for that link and Windows 10 automatically deletes it. This is the first time I have had a warning from x265.exe, is it a false alarm?

LigH answered well. I stopped using anti-virus software ages ago because 1) they became bloatware and 2) too many false positives.

Regarding browsers's add-ons: I use NoScript against most sites, but not against the ones I visit regularly (such as Doom9 and Videohelp). My HUGE hosts file is sufficient to block most /all ads on most sites, except on the anti-social networks (Twitter and Fakebook specifically).

TL; DR — Get rid of Windows 10, problem solved ;)

P.S.: for what it's worth...... https://www.virustotal.com/#/file/1da62a91f04465432f4805b5bb7b5cb715b798554449e77a65133b891be1e336/detection

LigH

2nd April 2018, 12:39

x265 2.7+20-3440a56acc78 (https://www.mediafire.com/file/goscyr6whgpllca/x265_2.7%2B20-3440a56acc78.7z)

fix bug in SEI::write clean up

Barough

2nd April 2018, 12:47

x265 v2.7+22-946f82dbf4e8 (http://www.mediafire.com/file/p97a266kp5wazj7/) (GCC 7.3.0, 32 & 64-bit 8/10/12bit Multilib Windows Binaries)

https://bitbucket.org/multicoreware/x265/commits/branch/default

LigH

2nd April 2018, 19:05

x265 2.7+22-946f82dbf4e8 (https://www.mediafire.com/file/b909czo6z3og1hq/x265_2.7%2B22-946f82dbf4e8.7z)

dynamic-refine tunings

Can anyone report advantages of this feature?

K.i.N.G

2nd April 2018, 22:51

I've been wondering what's the difference between your binaries, Barough's and LigH's?

Also, i read in the x265 online manual (x265.readthedocs.io (http://x265.readthedocs.io/en/default/cli.html#cmdoption-aq-mode)) that AQ-Mode 3 is good for low bitrate 10bit encodes... But what is considered low bitrates for Full-HD and 4K?

I backup my bluray movie collection @ Full-HD between 4000 and 8000kbs (depends mostly on the amount of grain, some movies can go higher but those are the exception. Animations tend to go lower)...
Is that considered low bitrates?

LigH

2nd April 2018, 22:59

I've been wondering what's the difference between your binaries, Barough's and LigH's?

No clue how exactly Barough builds them ... but probably negligible.

I use the media-autobuild_suite as MSYS2 base environment, but still run separate build scripts in MinGW32 and MinGW64. No additional optimizations, just semi-automatic preparation for building the archive to be uploaded.

Barough

2nd April 2018, 23:03

No clue how exactly Barough builds them ... but probably negligible.

I use the media-autobuild_suite as MSYS2 base environment, but still run separate build scripts in MinGW32 and MinGW64. No additional optimizations, just semi-automatic preparation for building the archive to be uploaded.Default MABS compiles here

Sent from my SM-G965F via Tapatalk

K.i.N.G

2nd April 2018, 23:22

Ok, thank you both for the answers!

Although I know absolutetly nothing about compiling so that doesn't tell me much... :D

LigH

2nd April 2018, 23:26

Means: We use the same tools to build them, and very similar ways to do that. You may not notice any differences (e.g. in speed) when comparing both while encoding the same material with the same parameters.

K.i.N.G

2nd April 2018, 23:40

Allright, thank you :)

RieGo

3rd April 2018, 10:48

x265 2.7+22-946f82dbf4e8 (https://www.mediafire.com/file/b909czo6z3og1hq/x265_2.7%2B22-946f82dbf4e8.7z)

dynamic-refine tunings

Can anyone report advantages of this feature?

was wondering the same.
i did some test encodes with default parameters and "--dynamic-refine --refine-intra 4" @1000kbit/s Two Pass
i couldn't really tell a difference...
how are you supposed to see a difference when changing one parameter? that's almost impossible in my eyes - without picking single frames, which doesn't make much sense anyways.
decoding at ultra low bitrate?

so in theory this setting should be an improvement, right?

LigH

3rd April 2018, 14:59

Well, I did not expect obvious differences for random material and convenient bitrates; someone with more insight might know academic cases where an improvement can be expected. My guess would be: questionable cost-benefit ratio.

But surprise me who can!

Sp00kyFox

3rd April 2018, 17:05

had some thoughts about the aq-motion option I wanted to share. on the previous pages I referred to a scene where this option creates serious and very noticeable artifacts. here is the source clip and encodings to compare. focus on the two guys walking back to the truck after the first scene cut:
https://mega.nz/#F!DMYT2ICb!4aufAuilmDJAn-Bj-Gzcpg

commandline that was used with --aq-motion / --no-aq-motion resp.:

--preset slow --crf 23 --profile main10 --no-sao

this is the description of the feature from the x265 documentation

--aq-motion, --no-aq-motion
Adjust the AQ offsets based on the relative motion of each block with respect to the motion of the frame. The more the relative motion of the block, the more quantization is used. Default disabled. Experimental Feature

this is apparently also applied when the frame doesn't move at all. so even when the scene is still, moving parts will use more quantization. I'd assume the original intention of this option is to only use more quantization in moving scenes where the block in question doesn't follow the global motion direction.

with a faster moving scene more quantization should be less noticeable on blocks with alternate direction. so to make this option more viable I'd suggest to scale the option with the speed of the global motion. with this still scenes shouldn't be affected at all and slow scenes less which avoids ugly results like demonstrated with the sample.

foxyshadis

4th April 2018, 04:55

this is apparently also applied when the frame doesn't move at all. so even when the scene is still, moving parts will use more quantization. I'd assume the original intention of this option is to only use more quantization in moving scenes where the block in question doesn't follow the global motion direction.

with a faster moving scene more quantization should be less noticeable on blocks with alternate direction. so to make this option more viable I'd suggest to scale the option with the speed of the global motion. with this still scenes shouldn't be affected at all and slow scenes less which avoids ugly results like demonstrated with the sample.

By the time the encoder would have even a rough idea of the global motion of the frame, more than half the blocks would already be quantized and done. x265 doesn't do any global motion estimation in the lookahead phase, it just decides if a scenecut is warranted, and everything else happens in a huge combined phase that outputs blocks as it gobbles them up.

Maybe a check for the inverse of emergency VBV could be enabled: Once the frame is over, go back and say "hey, that frame was pretty easy after all and didn't need to be so overcompressed." Or since global motion is known by the end, look back and see the oops before flushing the frame. Performance would take a pretty big hit every time it happened, especially if it affects future frames that are already mid-encode, but you need quality modes too. --aq-motion has promise in increasing overall quality, managing it just seems difficult.

Sp00kyFox

4th April 2018, 14:21

thanks foxyshadis for your explanations, I'm not that familiar with the inner workings of the encoder. well, there goes my idea. maybe there are some other approaches to improve aq-motion. I think it definitely has potential and behaves well in many cases.

LigH

4th April 2018, 15:01

x265 v2.7+25 introduces the CLI option single-sei (to write all SEI messages in one single NAL, which may cause compatibility issues with reference decoders); but its documentation is missing in the full help, so I'll wait for the next patch.

benwaggoner

4th April 2018, 18:27

By the time the encoder would have even a rough idea of the global motion of the frame, more than half the blocks would already be quantized and done. x265 doesn't do any global motion estimation in the lookahead phase, it just decides if a scenecut is warranted, and everything else happens in a huge combined phase that outputs blocks as it gobbles them up.

Maybe a check for the inverse of emergency VBV could be enabled: Once the frame is over, go back and say "hey, that frame was pretty easy after all and didn't need to be so overcompressed." Or since global motion is known by the end, look back and see the oops before flushing the frame. Performance would take a pretty big hit every time it happened, especially if it affects future frames that are already mid-encode, but you need quality modes too. --aq-motion has promise in increasing overall quality, managing it just seems difficult.

Good analysis. That said, doing a coarse estimate of global motion in a downrezzed frame during lookahead is a perfectly reasonable strategy. Also the 1st pass global motion is known in later passes, so the above could work with 2-pass or even refine analysis. Global motion should be pretty consistent across frame sizes, so data from a lower Rez can be reused it doing an adaptation set.

LigH

6th April 2018, 15:23

https://cosgan.de/images/smilie/konfus/g080.gif

307 patches with AVX-512 (and other improved assembly) code uploaded to the developer mailing list. That will take a little while to review.

pingfr

6th April 2018, 19:34

https://cosgan.de/images/smilie/konfus/g080.gif

307 patches with AVX-512 (and other improved assembly) code uploaded to the developer mailing list. That will take a little while to review.

Sounds like a good candidate for x265-3.0 to me.

Kavitha

9th April 2018, 10:11

x265 2.7+22-946f82dbf4e8 (https://www.mediafire.com/file/b909czo6z3og1hq/x265_2.7%2B22-946f82dbf4e8.7z)

dynamic-refine tunings

Can anyone report advantages of this feature?

x265 has static levels of refinement(--refine inter <level>/refine intra <level>) which can be used with --analysis-reuse-level 10.
Efficiency in terms of quality increases as the levels of refinement increases. This quality increase results from additional computation thereby increasing the overall encoding time.
For a better quality-speed trade-off, dynamic refinement was introduced where the encoder dynamically switches between different inter refine levels.
This basically exploits the fact that not all CUs are required to be encoded with same level for better performance/quality.
Considering the complexity of video content and the analysis information from first pass, the encoder can intelligently decide the optimal level of refinement for each CU.
Intra frames are usually encoded with best quality as they are used as references by the consecutive frames. Hence error introduced in intra frames due to reusing analysis data can propagate to frames that use these intra frames as reference.
To minimize the chances of error propagation, refine-intra 4 (level with best quality) restricts reusing analysis data for intra frames and forces the encoder to perform full intra analysis in the second pass.
This is why x265 documentation suggests to use dynamic refinement along with refine-intra 4 and this setting is expected to give improved quality than other refine intra levels for some videos.

LigH

10th April 2018, 07:56

:thanks: @Kavitha; so I believe quality enthusiasts who are willing to spend a sensible amount of time will probably use this as default...

RieGo

10th April 2018, 11:19

yeah, i think i will use this as default, combined with slow preset.
afaik there are some settings that haven't really been adopted to presets. are there any plans to revise presets?
btw. can't wait for that avx 512 - ready to overheat my cpu :)

LigH

10th April 2018, 13:17

afaik there are some settings that haven't really been adopted to presets. are there any plans to revise presets?

I'm no developer, but ... probably; presets have been revised in x264 several times as well. And in addition, there are also tunings to be revised and added. But first: Satisfy needs of paying customers. Second: e.g. complete spec coverage as much as sensible; etc. etc. The developers will surely face no boredom. :rolleyes: