Log in

View Full Version : x265 HEVC Encoder


Pages : 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 58 59 60 61 62 63 64 65 66 67 68 69 70 71 72 73 74 75 76 77 78 79 80 81 82 83 84 85 86 87 88 89 90 91 92 93 94 95 96 97 98 99 100 101 102 103 104 105 106 107 108 109 110 111 112 113 114 115 116 117 118 119 120 121 122 123 124 125 126 127 128 129 130 [131] 132 133 134 135 136 137 138 139 140 141 142 143 144 145 146 147 148 149 150 151 152 153 154 155 156 157 158 159 160 161 162 163 164 165 166 167 168 169 170 171 172 173 174 175 176 177 178 179 180 181 182 183 184 185 186 187 188 189 190 191 192 193 194 195 196 197

K.i.N.G
4th December 2018, 15:49
I have to glue several shots, which all have different resolution, together.
Ideally I'd like to keep the original resolution.
Is it possible to have variable resolution? (If so, is it UHD compliant/will it play nicely on modern UHD devices?)

Selur
4th December 2018, 15:53
I have to glue several shots, which all have different resolution, together.
...
Is it possible to have variable resolution?
iirc mkv added support for this a while back, but that has nothing to do with x265
(best ask Mosu to be sure)

If so, is it UHD compliant/will it play nicely on modern UHD devices?
No.

K.i.N.G
5th December 2018, 15:56
Ah, too bad... Thanks for the quick & clear reply!

mandarinka
9th December 2018, 16:31
iirc mkv added support for this a while back
I tried searching the changelog for resolution, variable and so on, but could not find anything. Do you recall which version was it roughly, or where can I find information about that? Support for variable resolution, or better said, variable display aspect ratio as well is something I tried to solve for years. The problem was that matroska stored such data in a single header entry valid for the whole file; back when designing it, nobody imagined files where there would be need to switch these parameters.

AFAIK the best possible solution was to use separate files and link them using the next/previous segment linking fucntionality.

Selur
9th December 2018, 16:43
@mandarinka: Like I wrote 'best ask Mosu to be sure' :)
I thought there was a way to combine file with multiple resolutions and the problem was to find a player supporting it, but Mosu most definitely can shine some light on it whether I just remember this wrong of if there is a way to create such files.

benwaggoner
12th December 2018, 19:58
@mandarinka: Like I wrote 'best ask Mosu to be sure' :)
I thought there was a way to combine file with multiple resolutions and the problem was to find a player supporting it, but Mosu most definitely can shine some light on it whether I just remember this wrong of if there is a way to create such files.
The only software player I know of that can handle arbitrary resolution changes while maintaining constant displayed image size and aspect ratio is Windows with a MFT from Expression Encoder installed.

That at least works with variable-sized VC-1; I've never tried it with H.264. If a player is attending to SAR, doing a full-screen playback should Just Work. But it seems players rarely account for the SAR changing each GOP even if they can handle different resolutions.

This does work in most players on living room devices, though, since it's common for SAR to change (e.g. 720x480 to 1280x720 both at 16:9) with stream switching.

benwaggoner
12th December 2018, 20:14
After a bit of a draught, we have five new checkins for x265! Three for Dolby Vision Profile 5 support, one to make muxing of chunked encoding easier, and one to allow cutree to be used in analysis reuse, finally.

https://bitbucket.org/multicoreware/x265/commits/all

LigH
13th December 2018, 16:00
x265 2.9+14-3023bd8b05c0 (https://www.mediafire.com/file/gqaq9939bjwgdhw/x265_2.9%2B14-3023bd8b05c0.7z) (MSYS2, MinGW32 + GCC 7.4.0 / MinGW64 + GCC 8.2.1)

support for Dolby Vision profile 5 and RPU multiplexing; Cutree offset for analysis reuse

imhh11
14th December 2018, 00:05
Thanks for the update.

is DV profile 5 single layer ?
if yes, then does it mean we can now convert our 2 layer DV uhd disc to a single layer file playable via usb into our TV ?

utack
14th December 2018, 01:19
Netflix dropped a new article.
H264/H265/VP9 comparison using HVMAF with reference encoders and production encoders.
x265 is doing quote ok but can't always beat HM, EVE-VP9 does similarly well
Relative Bitrate savings for low and high quality using three test sets:
https://i.imgur.com/Cf7fQ9V.png


https://medium.com/netflix-techblog/performance-comparison-of-video-coding-standards-an-adaptive-streaming-perspective-d45d0183ca95

Barough
14th December 2018, 15:01
x265 v2.9+15-81373aab81df (http://www.mediafire.com/file/mfww6xxjiaodna9/) (32 & 64-bit 8/10/12bit Multilib Windows Binaries) (32bit : GCC 7.4.0 / 64bit : GCC 8.2.1)

https://bitbucket.org/multicoreware/x265/commits/branch/default

LigH
14th December 2018, 15:08
^ just a fixed compiler warning

kuchikirukia
16th December 2018, 04:15
I tried searching the changelog for resolution, variable and so on, but could not find anything. Do you recall which version was it roughly, or where can I find information about that? Support for variable resolution, or better said, variable display aspect ratio as well is something I tried to solve for years. The problem was that matroska stored such data in a single header entry valid for the whole file; back when designing it, nobody imagined files where there would be need to switch these parameters.

AFAIK the best possible solution was to use separate files and link them using the next/previous segment linking fucntionality.

Yup, linked mkvs and ordered chapters (the latter has more widespread support) can handle frame rate changes as well as resolution.

mpv used to use the bitstream DAR instead of the container until I pointed out that I could troll mpv users by encoding a bitstream with rapidly shifting DARs that would work perfectly fine on every other player.
There's really little real-world need to change DAR midstream given the video resolution has to be constant, and container DAR is better than bitstream since that allows it to be trivially fixed if wrong.

jlpsvk
16th December 2018, 20:24
Thanks for the update.

is DV profile 5 single layer ?
if yes, then does it mean we can now convert our 2 layer DV uhd disc to a single layer file playable via usb into our TV ?

i would like to know that too. :) and if yes, how. :)

seandarcy
16th December 2018, 21:34
Do I use -tune grain, or just decrease crf for the same bitrate ?

I've ripped some old film-based blurays which have film grain that I'd like to retain. I understand that -tune grain will retain the grain, at a 50% increase in file size, or equivalently bitrate. So, as I understand it, if a given crf has 2000kbs, adding -tune grain will result in 3000kbs. But what if I decreased the crf until I got 3000kbs without -tune grain ? Will I retain some/all of the original grain ?

Assume I have a specific bitrate budget, say 3000kbs.

Am I better off with -tune grain crf X (say 29), or no grain and crf X - 2 or 3 (say crf 26 ) ? Again I've picked the crf's so that the output has the same bitrate.

x265, 2.9.
My command line :
ffmpeg -i in.mkv -c:a copy -c:v libx265 -preset medium [-tune grain] crf X -pix_fmt yuv420p10le -x265-params colorprim=bt709:colormatrix=bt709:transfer=bt709 out.mkv

I'm sure that some place in this thread this question has been asked and answered, but none that I've found discuss grain vs crf for a fixed bitrate.

asarian
17th December 2018, 09:13
So, does x265 need to be compiled manually, or does someone distribute the binaries too?

LigH
17th December 2018, 09:21
You did not read this thread before asking that?!

asarian
17th December 2018, 09:24
You did not read this thread before asking that?!

I read the x265 documentation listed in the start post, but they don't seem to offer binaries (and if binaries exist here, they're, subsequently, usually listed in the start-post too). But I'll keep reading. :)

LigH
17th December 2018, 09:29
Try bottom-up; just one or two pages ago, you will find about 3-4 people offering latest builds. Including mine.

asarian
17th December 2018, 09:33
Try bottom-up; just one or two pages ago, you will find about 3-4 people offering latest builds. Including mine.

Thx. :) I took yours.

MeteorRain
17th December 2018, 19:42
So do I, although I rarely post in this thread.

seandarcy
18th December 2018, 16:16
I've been ripping a bunch of 480p DVDs with x264. A lot of them are early technicolor.

I've been wondering whether 10 bit x265 would better preserve the color. (I don't have anything that would play 10 bit x264). But if I do that, what's the color range, still TV (limited) or full ? And if TV, what is the limited color range for 10 bit ? 16-235 goes to xxx - yyy ?

And the colorprim stays smpte170m, correct ?

Anybody tried this ?

sean

SeeMoreDigital
18th December 2018, 16:28
Given the low cost of HDD's these days, why not back-up the original (8-bit) MPEG-2 stream from the DVD?!

No encoding required... Easy.


Cheers

LigH
18th December 2018, 16:43
The 8 or 10 bit of the AVC or HEVC encoder "engine" is not related to the bit depth of the image, but to the resolution of encoder internal quantization parameters after a mathematical transformation. If you never learned about the Fourier transformation of discrete samples to a frequency spectrum, you may not understand much what that means... and despite a limited range, the Discrete Cosine Transform as well as the HEVC Integer Transform are still closely related to Fourier sequences (which are originally infinite).

MPEG-2 video may technically have 8..11 bit internal DCT precision, but in DVD Video media most commonly 9 bit, rarely 10 bit in high quality productions. Recoding will lose quality already by the requantization of the material. I can only agree to SeeModeDigital: The best way to preserve the quality in your DVD material is not to convert it anymore. Just remultiplex the main movie PGC to an MKV (makeMKV) for easy handling on a PC or smart TV.

seandarcy
18th December 2018, 18:07
OK, I'll remux to mkv. But what about the soft telecine ? This is a 24fps film soft telecined to 29.97. Can I undo the soft telecine without reencoding ?

sean

videoh
18th December 2018, 18:52
You can undo the soft telecine with DGPulldown.

seandarcy
18th December 2018, 20:08
Unless I'm misreading this, DGPulldown telecines a stream. It doesn't undo the telecine.

videoh
18th December 2018, 20:18
From the user manual:

"Note that if the custom rate conversion is selected, and if the source rate is specified as equal to the destination rate, then all pulldown is removed and the stream is flagged as having a rate equal to the specified destination rate."

FranceBB
19th December 2018, 04:32
I've been wondering whether 10 bit x265 would better preserve the color. (I don't have anything that would play 10 bit x264).

x265 uses high bit depth internally and 10bit files are compatible with the majority of players; in other words, even if you feed it with an 8bit stream and you choose Main10, it will use high bit depth internally and encode it to 10bit.


But if I do that, what's the color range, still TV (limited) or full ?


You can encode it in both Full Range or Limited Range, depending on your target.
Please note that the standard is Limited Tv Range.
Besides, if the content was originally in Tv Range, you shouldn't change it.


And if TV, what is the limited color range for 10 bit ? 16-235 goes to xxx - yyy ?


The old 8bit Tv Range was 0.0-0.7V, in other words, 16-235.
For 10bit, Tv Range is 64-940.
x265 has a flag to specify "limited range" which is --range limited (just like x264), and it also has two additional parameters --min-luma 64 --max-luma 940 to apply clipping if you have to be 100% sure that values don't exceed Tv Range (useful for TV Stations).


Can I undo the soft telecine without reencoding ?


If it's really soft telecine (just a flag specified while muxing the original MPEG-2 file), yes, you can, otherwise if it's really telecined with repeated fields, you gotta do reverse telecine and re-encode.

MPEG-2 video may technically have 8..11 bit internal DCT precision, but in DVD Video media most commonly 9 bit, rarely 10 bit in high quality productions.

I used to make official DVDs time ago and you are correct, but I've seen some atrocities done in the past, including a playback 1:1 from a Sony VTR playing a BetaCAM "master", or playback using an Omneon SD playback port connected via SDI, or "recently" an Omneon FULL HD playback port routed via SDI to an hardware downscaler which performed FastBilinear and outputted the result to the Master DVD copy, when all this could have been done better and faster via software... but hey, stone-age procedures and stubborn co-workers...
Luckily, these days are over and nowadays everything it's handled via software.

BLKMGK
22nd December 2018, 19:35
Some advice on chunked encoding for a newb please!

I've been working out how to use the chunk commands to encode pieces of a file with an eye towards using some form of cluster to speed single file encoding. One thing that I'm observing is slow encode speed but it may be because of the VM I'm using despite 12 cores being made available. When chunk is used and the specified frames are deep into the file the encoder is able to skip to those frames yes, it need not start at the beginning and search for them? I'm not currently specifying key frames and am allowing chunk to find those on its own, I also realize it will encode some frames before and after the keyframes it chooses to start on but I'd like to be sure that in the end I've saved time! I'm encoding a test now with a csv file specified for statistics and it's got nothing in it after 5 mins, I was hoping to see some information as it went along in hopes of determining progress :(

In some tests I've done things like specify color space, key intervals, and other settings, am I better off allowing the encoder to do most of that itself? My intuition says yes lol.

I have attempted to use a cropping filter in ffmpeg feeding X265, would I be better off doing this in X265? I've had trouble figuring that out so far so an example would rock - especially if it uses the same syntax as ffmpeg. Would cropping in ffmpeg prevent a video from seeking to the chunk frame I desire or be slower than using X265? I'm just trying to ensure I'm doing this as efficiently as possible, honestly must ffmpeg be used at all if I'm not adding filters? I've not been able to use X265 standalone to accomplish what I want yet.

I've also noticed that when I encode a chunk like this that VLC won't open the file with a MKV extension but if I use .265 (which I've associated with VLC) it opens, it's as if assumptions are made from the extension that prove invalid. When I've concatenated a number of chunks together and muxed in audio the result has been fine though.

Here are two examples of commandline I'm working with right now:

ffmpeg -hide_banner -i INPUT.mkv -f yuv4mpegpipe - | x265 - --no-open-gop --chunk-start 166675 --chunk-end 169500 --colorprim bt709 --transfer bt709 --colormatrix bt709 --crf=20 --fps 24000/1001 --min-keyint 24 --keyint 240 --sar 1:1 --preset slow --ctu 16 --y4m --pools + -o chunky-nocrop.265

ffmpeg -hide_banner -i INPUT.mkv -filter:v "crop=1920:800:0:140" -f yuv4mpegpipe - | x265 - --no-open-gop --chunk-start 166675 --chunk-end 169500 --crf=20 --fps 24000/1001 --preset slow --ctu 16 --y4m -o chunky1.mkv

Any advice would be welcomed, I'm doing this in Linux FWIW. :thanks:

FranceBB
22nd December 2018, 20:59
Encoding in a Virtual Machine is gonna be slower than encoding files on your physical hardware; it doesn't matter if you allow many cores and you have. Since you are doing this on Linux in a VM, unless you need a frameserver (like Avisynth) to do any sort of post-processing, drop the Windows Virtual Machine and do it on Linux via ffmpeg and x265 (they can both be compiled using GCC, CMake and NASM).
As to the player, MPV works perfectly well on Linux, you just need to install the additional codecs and you are good to go; no need to use VLC.

For the records, I use Windows at work, but I have Fedora at home. ;)

BLKMGK
22nd December 2018, 21:33
Encoding in a Virtual Machine is gonna be slower than encoding files on your physical hardware; it doesn't matter if you allow many cores and you have. Since you are doing this on Linux in a VM, unless you need a frameserver (like Avisynth) to do any sort of post-processing, drop the Windows Virtual Machine and do it on Linux via ffmpeg and x265 (they can both be compiled using GCC, CMake and NASM).
As to the player, MPV works perfectly well on Linux, you just need to install the additional codecs and you are good to go; no need to use VLC.

For the records, I use Windows at work, but I have Fedora at home. ;)

Right now it's a Linux VM on ESX for testing, I'm able to run the same command in Windows however as it's crossplatform (my intent). What seems odd to me is the destination file remains empty for an extended length of time but in the end has data. I've shortened up the gap between the chunks to 2875 frames to speed things and am dropping encoding quality to try and speed further but it still takes awhile.

My concern is that I'm somehow forcing it to read through and do processing on the entire file before it reaches it's target range and does real work - that would make breaking jobs up pointless!

End goal: break a large HD encoding task into 60 or more "jobs" and throw those jobs to encoding containers on various machines around the house that have idle cycles. Combine the resulting sections of video, mux in audio, have a complete video. Kubernetes or something like that to manage the containers. Have to have a proper working "job" before I can start the clustering part though hence my testing.

I've done this already, not using chunk, by indexing a video, making jobs that started and ended on keyframes, and then combining etc. but "chunk" seems made for this and allows me to skip the indexing, mathing of start/end points, and sounds like it uses the encoder better.

Hoping that makes more sense!

P.S. Yes, I intend to share whatever I come up with. Every other project I've found that does anything similar hasn't been updated in ages other than RipBot which works great on Windows but I've got more Linux resources and desire cross-platform as a result. :cool:

asarian
22nd December 2018, 21:57
Encoding in a Virtual Machine is gonna be slower than encoding files on your physical hardware; it doesn't matter if you allow many cores and you have. Since you are doing this on Linux in a VM, unless you need a frameserver (like Avisynth) to do any sort of post-processing, drop the Windows Virtual Machine and do it on Linux via ffmpeg and x265 (they can both be compiled using GCC, CMake and NASM).
As to the player, MPV works perfectly well on Linux, you just need to install the additional codecs and you are good to go; no need to use VLC.

For the records, I use Windows at work, but I have Fedora at home. ;)


I have done many (x264) encodings in a VM (ESXi). Surprisingly, it actually isn't all that much slower, as, while the CPU's are virtualized, of course, they're not like software-emulated or something, and, for the most part, are accessed almost directly. The speed drop was about 2-3% vs. the real deal. Mostly VM I/O was actually a bottleneck, for me. :)

The beauty of VM encoding, is that you can suspend the entire VM, mid-encoding! And it will continue just fine where you left of, when you start it up again. I stopped doing it, though, as I love using GPU-assisted encoding, and figured, if I have to buy a second video-card, just to make it passthru to ESXi, then I'm better off just using my main card directly, outside Vmware.

BLKMGK
22nd December 2018, 22:04
Agree, I don't see ESX slowing things too much. I am presently shying from GPU encoding mostly because I'm under the impression it's not as high a quality and only one Windows machine I've got could do it. Learning abnout clustering is also an incentive I'll admit.

That said - I have two jobs running right now in CMD Windows in WIN10. One seems to be saying it's run through 94K frame, the other 115K - they've been running over an hour each. The chunk start and end were only 2800 frames apart! One is moving at 23FPS, the other 56FPS, neither datafile has any damn data in it yet.

Am I doing something wrong? Surely chunk isn't meant to read from SOF and should jump to where it's been told to begin encoding right? I really don't want to use --seek and --frames to do this :(

Edit: Rereading the little I can find on chunk. Surely they don't mean ALL frames preceeding do they?! I had read this as a function to help multiple machines work on a single file but it's surely not that if ALL preceeding frames are encoded and discarded as you get into a file :(

option:`--chunk-start and --chunk-end` Frames preceding first frame of chunk in display order will be encoded, however, they will be discarded in the bitstream. Frames following last frame of the chunk in display order will be used in taking lookahead decisions, but, they will not be encoded. This feature can be enabled only in closed GOP structures. Default disabled.

BLKMGK
22nd December 2018, 22:30
Just answered part of my own question "chunk" reads and encodes from start of file all the way to it's given starting point to create it's "chunk". It then reads to the given end point and completes. I've seen this mentioned as something that could be used for clustering jobs but not with this behavior it isn't! Is this intended? Why does it not advance to it's given start frame, look backwards to a previous keyframe, and start encoding there? It could discard the frames prior to it's given start but have given the encoder frames to prime it. If this is intended behavior I don't understand the usefulness.

Edit: Just tried an end point for the chunk that wasn't right against EOF. When it hits its endpoint it keeps encoding but discards the work.

asarian
22nd December 2018, 22:41
Agree, I don't see ESX slowing things too much. I am presently shying from GPU encoding mostly because I'm under the impression it's not as high a quality and only one Windows machine I've got could do it.

Coding HEVC directly with (hardware-accelerated) NVENC, for instance, is an order of magnitude faster (and sometimes several orders of magnitude even) than what we can achieve with x265 itself, but you have no control over the process, and the quality is definitely lower than that of the latter.

I was more talking about OpenCL in x264; and especially used for the KNLMeansCL denoiser (QTGMC), where it speeds things up significantly. Even though I recently heard even OpenCL yields a theoretical lower quality.

That said - I have two jobs running right now in CMD Windows in WIN10. One seems to be saying it's run through 94K frame, the other 115K - they've been running over an hour each. The chunk start and end were only 2800 frames apart! One is moving at 23FPS, the other 56FPS, neither datafile has any damn data in it yet.

As for the latter, typically you have to refresh your explorer window (a few times) to see how much is actually output already, or it will stay at 0 bytes. :)

As for chunks, I never used them before, so I can't really say something useful about that.

Ma
22nd December 2018, 23:37
If you want to encode without VBV buffer, you could use --seek and --frames options, for example:
for %i in (0 1000 2000) do ffmpeg -i ../lighthouse_lossless.mp4 -v warning -f yuv4mpegpipe - | x265 --y4m - --crf 20 --seek %i --frames 1000 c%i.hevc

which expands to:
ffmpeg -i ../lighthouse_lossless.mp4 -v warning -f yuv4mpegpipe - | x265 --y4m - --crf 20 --seek 0 --frames 1000 c0.hevc

ffmpeg -i ../lighthouse_lossless.mp4 -v warning -f yuv4mpegpipe - | x265 --y4m - --crf 20 --seek 1000 --frames 1000 c1000.hevc

ffmpeg -i ../lighthouse_lossless.mp4 -v warning -f yuv4mpegpipe - | x265 --y4m - --crf 20 --seek 2000 --frames 1000 c2000.hevc

Finally you can put these 3 parts together:
mkvmerge -o c-whole.mkv c0.hevc + c1000.hevc + c2000.hevc
-----------------
example for Windows

BLKMGK
23rd December 2018, 00:10
Coding HEVC directly with (hardware-accelerated) NVENC, for instance, is an order of magnitude faster (and sometimes several orders of magnitude even) than what we can achieve with x265 itself, but you have no control over the process, and the quality is definitely lower than that of the latter.

I was more talking about OpenCL in x264; and especially used for the KNLMeansCL denoiser (QTGMC), where it speeds things up significantly. Even though I recently heard even OpenCL yields a theoretical lower quality.



As for the latter, typically you have to refresh your explorer window (a few times) to see how much is actually output already, or it will stay at 0 bytes. :)

As for chunks, I never used them before, so I can't really say something useful about that.

Ah, makes sense to use them for programmatic speedups and for filters I agree. I may try using my GPU a few times just to see if the results are acceptable to me but so far haven't been able to get it working the few times I've tried :(

This definitely wasn't a refresh issue in explorer, Chunk really doesn't appear to save any data until it hits its start frame and continues encoding after it hits its end frame. Ugh!

If you want to encode without VBV buffer, you could use --seek and --frames options, for example:
for %i in (0 1000 2000) do ffmpeg -i ../lighthouse_lossless.mp4 -v warning -f yuv4mpegpipe - | x265 --y4m - --crf 20 --seek %i --frames 1000 c%i.hevc

which expands to:
ffmpeg -i ../lighthouse_lossless.mp4 -v warning -f yuv4mpegpipe - | x265 --y4m - --crf 20 --seek 0 --frames 1000 c0.hevc

ffmpeg -i ../lighthouse_lossless.mp4 -v warning -f yuv4mpegpipe - | x265 --y4m - --crf 20 --seek 1000 --frames 1000 c1000.hevc

ffmpeg -i ../lighthouse_lossless.mp4 -v warning -f yuv4mpegpipe - | x265 --y4m - --crf 20 --seek 2000 --frames 1000 c2000.hevc

Finally you can put these 3 parts together:
mkvmerge -o c-whole.mkv c0.hevc + c1000.hevc + c2000.hevc
-----------------
example for Windows

I've done this except I started and ended on keyframes with the understanding that not doing so could have poor joins. The issue with that is it requires first pulling an index and then skipping through it to build the jobs. The first test attempted I built the latter part by hand (PITA) and after encoding I couldn't detect the seams at all. Seemed a good path forward!

I then noticed chunk in the release notes. Chunk, from its description and some discussion I'd seen here, seemed tailor made for clutering but my tests sure don't seem to prove it out. I'll try your example and see if I can detect the seams, perhaps I've been too paranoid?

Thank you for the example of how to build the jobs!

Edit: Tested this method ignoring keyframes and thought I detected a hitch in the video. Sure enough VLC shows two dropped frames where the hitch occurs and I see it elsewhere too. I'll keep testing and see if the keyframes are the issue. I'll pull an index and create some jobs. I need an automated way to build jobs from the index but not at every break lol

Ma
23rd December 2018, 01:29
I tried to add --chunk-start/--chunk-end to the example and it is like this:
Encoding in parts of 1000 frames (+100 before, +100 after):

ffmpeg -i ../lighthouse_lossless.mp4 -v warning -f yuv4mpegpipe - | x265 --y4m - --crf 20 --no-open-gop --seek 0 --frames 1100 --chunk-start 1 --chunk-end 1000 c0.hevc

ffmpeg -i ../lighthouse_lossless.mp4 -v warning -f yuv4mpegpipe - | x265 --y4m - --crf 20 --no-open-gop --seek 900 --frames 1200 --chunk-start 101 --chunk-end 1100 c1.hevc

ffmpeg -i ../lighthouse_lossless.mp4 -v warning -f yuv4mpegpipe - | x265 --y4m - --crf 20 --no-open-gop --seek 1900 --frames 1200 --chunk-start 101 --chunk-end 1100 c2.hevc

mkvmerge -o chunks.mkv c0.hevc + c1.hevc + c2.hevc

BLKMGK
23rd December 2018, 05:00
I tried to add --chunk-start/--chunk-end to the example and it is like this:
Encoding in parts of 1000 frames (+100 before, +100 after):

ffmpeg -i ../lighthouse_lossless.mp4 -v warning -f yuv4mpegpipe - | x265 --y4m - --crf 20 --no-open-gop --seek 0 --frames 1100 --chunk-start 1 --chunk-end 1000 c0.hevc

ffmpeg -i ../lighthouse_lossless.mp4 -v warning -f yuv4mpegpipe - | x265 --y4m - --crf 20 --no-open-gop --seek 900 --frames 1200 --chunk-start 101 --chunk-end 1100 c1.hevc

ffmpeg -i ../lighthouse_lossless.mp4 -v warning -f yuv4mpegpipe - | x265 --y4m - --crf 20 --no-open-gop --seek 1900 --frames 1200 --chunk-start 101 --chunk-end 1100 c2.hevc

mkvmerge -o chunks.mkv c0.hevc + c1.hevc + c2.hevc

That looks like a very good idea to get the encoder to take into account frames before and after the specified range in order for the encoder to be more efficient! It's not breaking on keyframes, am I wrong about their importance?

I tried your example on my sample file, I'm not spotting glitches or dropped frames in the statistics and I managed to extend it out some.

ffmpeg -i INPUT.MKV -v warning -f yuv4mpegpipe - | x265 --y4m - --crf 20 --no-open-gop --seek 3100 --frames 1200 --chunk-start 101 --chunk-end 1100 --csv stats.csv c3.hevc

If I'm wrong about the keyframes that will make life much easier. I'll try doing an entire film once I've incorporated this into my bash script, mux in sound, and check the synch.

Thank you! I really appreciate you providing examples too as it makes understanding what you're doing much simpler. :thanks:

nghiabeo20
23rd December 2018, 05:39
What can SAO do? Does it improve quality/file size?
I'm targeting good quality with reasonable bitrate (5-6000 kbps for 1080p10, BD remux source) with CRF 21. Thanks!

excellentswordfight
23rd December 2018, 13:42
What can SAO do? Does it improve quality/file size?
I'm targeting good quality with reasonable bitrate (5-6000 kbps for 1080p10, BD remux source) with CRF 21. Thanks!
Yes, SAO increases efficiency/compression. But it usually does it with the drawback of an slight reduction in fine detail. So its pretty great for low bitrate encodings and animation, but not so much if you wanna keep most of the detail.

Its best if you test it for yourself, and see what settings gives you the best results, but if you target that bitrate range I whould say that you might wanna turn it off.

And btw, I think crf21 will end up at a much lower bitrate for most content if you dont us tune grain or something similar. I get arround those numbers with crf18.

nghiabeo20
23rd December 2018, 16:46
Yes, SAO increases efficiency/compression. But it usually does it with the drawback of an slight reduction in fine detail. So its pretty great for low bitrate encodings and animation, but not so much if you wanna keep most of the detail.

Its best if you test it for yourself, and see what settings gives you the best results, but if you target that bitrate range I whould say that you might wanna turn it off.

And btw, I think crf21 will end up at a much lower bitrate for most content if you dont us tune grain or something similar. I get arround those numbers with crf18.

Maybe I'm doing something wrong. All I edit from the veryslow preset is these commands

bframes=6:subme=3:rd=5:qcomp=0.75:no-sao:deblock=-2:-2

however CRF 22 10bit without any tuning gives me a whopping 8500kbps for a 30s test clip from my The Hundred-Foot Journey bluray!

Boulder
23rd December 2018, 16:59
The bitrate depends a huge deal on the content. Grainy or noisy stuff requires much more than some clean CGI content.

excellentswordfight
24th December 2018, 00:24
Maybe I'm doing something wrong. All I edit from the veryslow preset is these commands

bframes=6:subme=3:rd=5:qcomp=0.75:no-sao:deblock=-2:-2

however CRF 22 10bit without any tuning gives me a whopping 8500kbps for a 30s test clip from my The Hundred-Foot Journey bluray!
As boulder said, thats very source dependant. And a 30s sample might not be representative either. But with that said i think your qcomp setting plays a role here (maybe deblock as well).

singhkays
24th December 2018, 04:17
Can anybody give any advice if there is any benefit by taking the following approaches? and is either one of the splitting methods preferred?


Split a video file on keyframes
Split a video file based on the shots (i.e. different camera angles)


other than the obvious ability to parallelize the encoding task

Selur
24th December 2018, 08:09
other than the obvious ability to parallelize the encoding task
Aside from further parallelizing the encoding more there is no reason to split a source unless you want to.
When having the split a source for parallelizing the encoding I would prefer to split 'a video file based on the shots' since the source simply might have key frames placed in a fixed interval not caring about coding efficiency. That said splitting on key frames is easier.
But splitting on key frames in respect to scenes and using the segmented encoding feature of x265 is probably the best easy to implement way. Ideally you wound split at the points where the encoder would want to set key frame for coding efficiency.

singhkays
24th December 2018, 23:28
But splitting on key frames in respect to scenes and using the segmented encoding feature of x265 is probably the best easy to implement way. Ideally you wound split at the points where the encoder would want to set key frame for coding efficiency.

Thank for the advice. I have specific question on the below

splitting on key frames in respect to scenes

How would this work? Should I split on keyframes within the scenes?

segmented encoding feature of x265

I haven't heard about this. How does this feature work?

Ideally you wound split at the points where the encoder would want to set key frame for coding efficiency

How do I figure out where the encoder wants to set key frames? I thought that was a user controlled option i.e. keyint

filler56789
27th December 2018, 13:32
To whom this may interest...
latest revision(s) cannot be compiled under MinGW-w64 && MSYS(2).
Some recent changes prevent the libraries from being linked to the command-line .EXE.

Ma
27th December 2018, 14:32
Some recent changes prevent the libraries from being linked to the command-line .EXE.

Multilib builds are broken (and builds with ENABLE_HDR10_PLUS too).
Simplest solution:
diff -r 537bba0b7fdc source/encoder/encoder.cpp
--- a/source/encoder/encoder.cpp Thu Dec 27 11:56:52 2018 +0530
+++ b/source/encoder/encoder.cpp Thu Dec 27 14:25:51 2018 +0100
@@ -69,7 +69,7 @@
int doviProfileId;
}DolbyVisionProfileSpec;

-DolbyVisionProfileSpec dovi[] =
+static DolbyVisionProfileSpec dovi[] =
{
{ 1, 1, 1, 1, 1, 5, 1, 2, 2, 2, 50 },
{ 1, 1, 1, 1, 1, 5, 0, 16, 9, 9, 81 },
diff -r 537bba0b7fdc source/x265.cpp
--- a/source/x265.cpp Thu Dec 27 11:56:52 2018 +0530
+++ b/source/x265.cpp Thu Dec 27 14:25:51 2018 +0100
@@ -690,7 +690,7 @@
args[argCount++] = token;
token = strtok(NULL, " ");
}
- args[argCount] = '\0';
+ args[argCount] = NULL;
CLIOptions cliopt;
if (cliopt.parseZoneParam(argCount, args,param, i))
{