xvc - a next-generation video codec at xvc.io

[MoSal]

@foxyshadis

1- You might want to check what FUD means.

2- I would trust the actual lawyers, with actual relevant experience and expertise, who are hired by multiple corporations (not just Google), to okay everything that gets added to AV1, over an unpractical scheme, augmented with inconcrete promises of legal protection, and tired attempts at spreading FUD against the competition.

[iwod]

I am starting to understand and guess what xvc might be.

It tries to be an all JVET H.266 encoder, and once all the body and decision are made regarding to tools and features used in spec, xvc with its feature selection will be the first JVET h.266 encoder on the market.

[augman000]

ffmpeg -y -loglevel fatal -threads 8 -i "10bit-dnxhd.mov" -frames:v 100 -an -sn -vsync 0 -pix_fmt yuv422p -f yuv4mpegpipe - | xvcenc_dev.exe -input-file - -verbose 1 -input-chroma-format 2 -input-bitdepth 10 -speed-mode 2 -qp 25 -explicit-encoder-settings "aqp_strength 16" -output-file d:\output.xvc

Is there any way to modify this to also add the decoding (xvcdec) after encoding? If so, how?

[LigH]

Maybe if you use "-output-file -" and pipe xvcdec_dev ... but I would not recommend it: if it works at all, it would not leave any file on your disk.

[augman000]

Maybe if you use "-output-file -" and pipe xvcdec_dev ... but I would not recommend it: if it works at all, it would not leave any file on your disk.

I did try that and for one reason or another it did not work properly.

But I think you're right. As long as xvc takes to encode a video, not having the actual file to revert back to is a pretty bad idea.

By the way: I am really liking xvc. It is beating AV1, x265, and VP9 in my tests with PSNR and SSIM scores. I'd like to measure VMAF as well but I have been unsuccessful in all my attempts to compile a working win64/win32 binary.

Thanks.

[Selur]

Main show stoppers for me with xvc are atm.
a. no decoding through libav/ffmpeg
b. terribly slow encoding

[augman000]

Main show stoppers for me with xvc are atm.
a. no decoding through libav/ffmpeg
b. terribly slow encoding

I do agree regarding the lack of ffmepg integration. But, for speed, with lower resolutions at leaast (haven't tested higher yet), it isn't so bad for the quality it produces.

Here is a quick test on my i7-3770k, Win 7 x64 PC of a 10 second, 428x240 clip using Timer 3.01 from Igor Pavlov (author of 7-Zip) for accurate timings:
PSNR SSIM TIME CODEC / OPTIONS
================================================================
44.845179 0.983561 252.955s libaom-av1 -b:v 100K -cpu-used 4
46.537460 0.988025 334.294s xvcenc -qp 25 -speed-mode 2
45.024133 0.983995 444.758s libaom-av1 -b:v 100K -cpu-used 3

And here is the same clip at 480p (854x480):
PSNR SSIM TIME CODEC / OPTIONS
================================================================
47.507486 0.990071 1179.726 libaom-av1 -b:v 300K -cpu-used 4
47.905909 0.991473 1237.431 xvcenc -qp 25 -speed-mode 2
47.662076 0.990272 1939.154 libaom-av1 -b:v 300K -cpu-used 3

Obviously this is just one 10s clip at a very low resolution. But for my me, these results are pretty good.

[jonatans]

We have now added support for multi-threaded encoding in the dev branch of xvc. I have created new binaries available here:

https://drive.google.com/file/d/1jFSHB4TkZZiUgG8gqimda9X0EPf0A2pE/view?usp=sharing

Threading is disabled by default. In order to turn it on, set the '-threads' parameter to the number of threads desired. The implementation is a variant of picture-based threading with the main benefit of not having any impact on picture quality (it gives identical bitstreams as when running without threads). It has the downside of increasing memory usage linearly with number of threads. Also note that it takes some time for utilization to reach its peak performance. As an example, when using 8 threads you typically need to code at least 128 pictures for this to happened.

Please let me know if you have any feedback or any suggestions for improvements.

[LigH]

Just tried to build the sources in the MSYS2/MinGW environment of MABS, but since I also have MSVC 2015 installed, CMake detects it first and prefers to create MSVC solutions, so I had to explicitly request
cmake -G "MSYS Makefiles" ..
Building the googletest sources fails with GCC 7.3.0 (error: 'AutoHandle' does not name a type; did you mean 'LongToHandle'?); but decoder and encoder have been built already.

MediaFire: xvc 2018-03-15 3e1db91 (https://www.mediafire.com/file/me5etoudwvl04lw/xvc_2018-03-15_3e1db91.7z) (MSYS2/MinGW 32+64, GCC 7.3.0)

The help output of xvcenc does not yet mention "-threads" as available parameter.

In addition: "Error: Unknown argument: -threads" ... apparently my MinGW builds do not know about threading. Is that not enabled in the github repo?
_

Aah, I should have taken the "dev" branch, not "master". Once again...

MediaFire: xvc-dev 2018-05-18 d2355cd (https://www.mediafire.com/file/rgudhswfap81vvh/xvc-dev_2018-05-18_d2355cd.7z) (MSYS2/MinGW 32+64, GCC 7.3.0)

[jonatans]

Aah, I should have taken the "dev" branch, not "master". Once again...

Yes, the '-threads' option is currently only available in the dev branch. That is where all new tools and features are added in preparation for the release of the next version of xvc.

[pie]

Building the googletest sources fails

iirc with mingw you need to pass -Dgtest_disable_pthreads=ON or something to that effect.

[LigH]

@pie: That was the old "master" version.
_

@jonatans:

This dev version crashes reading from a Y4M:

Assertion failed!

Program: E:\Programme\xvc\Win64\xvcenc.exe
File: H:/development/media-autobuild_suite-master/build/xvc/app/xvc_enc_app/y4m_reader.cc, Line 124

Expression: pic_fmt.input_bitdepth != 0

Source is the usual "foreman" clip from Derf's collection on xiph.org, only patched to PAL frame rate:

YUV4MPEG2 W352 H288 F25:1 Ip A128:117
FRAME

Maybe a "C420" should be added to the Y4M header so that your Y4M reader knows exactly which YUV variant it is?

[jonatans]

Yes, our y4m parser assumed that the C parameter would be present in the y4m header.

I have just changed so that "C420" is assumed if there is no C parameter present.

I also added support for "Cmono".

[LigH]

A good reason to build anew: xvc-dev 2018-05-18 bbf1325 (https://www.mediafire.com/file/gzai00ew4jjr0bm/xvc-dev_2018-05-18_bbf1325.7z)

[benwaggoner]

By the way: I am really liking xvc. It is beating AV1, x265, and VP9 in my tests with PSNR and SSIM scores. I'd like to measure VMAF as well but I have been unsuccessful in all my attempts to compile a working win64/win32 binary.
SSIM and especially PSNR are not very good metrics, particularly with advanced new codecs. Or anything that is psychovisually tuned.

VMAF is probably the least-bad objective metric we have, but still has some pretty big limitations and blind spots. It wasn't tested with anything >1080p or below something like 300 Kbps. SDR 8-bit only. A quite weak temporal comparison module. And it was trained on just x264, and so doesn't know what to do with new types of artifacts AV1 and HEVC can have.

Plus there's the whole question of how you aggragate individual frame scores into a clip score. Just the mean of the metrics can't discriminate between content that is consistently mediocre versus oscillating between terrible and pristine.

With a new codec, actually looking at it is really the only thing that can give a better than a rough ballpark. Certainly a difference of less than 5 VMAF, 4 PSDR dB, and 3 SSIM dB should be verified visually.

[maxcoolzero]

Wow. Looks cool

[jonatans]

SSIM and especially PSNR are not very good metrics, particularly with advanced new codecs. Or anything that is psychovisually tuned.

VMAF is probably the least-bad objective metric we have, but still has some pretty big limitations and blind spots. It wasn't tested with anything >1080p or below something like 300 Kbps. SDR 8-bit only. A quite weak temporal comparison module. And it was trained on just x264, and so doesn't know what to do with new types of artifacts AV1 and HEVC can have.

Plus there's the whole question of how you aggragate individual frame scores into a clip score. Just the mean of the metrics can't discriminate between content that is consistently mediocre versus oscillating between terrible and pristine.

With a new codec, actually looking at it is really the only thing that can give a better than a rough ballpark. Certainly a difference of less than 5 VMAF, 4 PSDR dB, and 3 SSIM dB should be verified visually.

Thanks Ben, those are all very valid points.

I agree that PSNR is not a very good metric for determining visual quality. It is good for determining how close the compressed pictures are to the original pictures (in a mathematical sense - after all that is what PSNR measures, on a sample by sample basis). So if two different implementations are tuned to minimize MSE ("tune PSNR") then PSNR gives a good indication of how good the implementations are at doing just that. And even though this generally doesn't correlate well with visual quality it is good in the sense that it does not give any "false positives" (i.e. if one compressed picture A is tuned towards PSNR you cannot easily create another compressed picture B with higher PSNR by making it look worse than A).

We have put some results for the NETVC test conditions and test sequences at awcy.divideon.com, comparing xvc to AV1, both codecs with PSNR tuning (https://awcy.divideon.com/?job=1pass-av1-debargha-default-0202%402018-02-16T14%3A34%3A19.005Z&job=1pass-xvc-180226-sp0-chroma-1%402018-02-26T10%3A34%3A38.388Z) including results for PSNR (16.5% savings) MS-SSIM (23.3% savings) and VMAF (18.5% savings). It can be seen that the gains are quite sequence dependent but the overall trend is very clear and quite consistent among the metrics.

There is also a comparison between xvc and HM (https://awcy.divideon.com/?job=1pass-HM-16.17-ra10%402017-11-17T12%3A29%3A11.462Z&job=1pass-xvc-180226-sp0-chroma-1%402018-02-26T10%3A34%3A38.388Z) which shows slightly larger bitrate savings.

But, as you correctly point out, what matters in the end is the visual quality which can only be determined by actually looking at compressed video (encoded under fair conditions when it comes to complexity etc.)

Please share your impressions if you have had a chance to look at any xvc encoded sequences or made any visual comparisons.

[LigH]

xvc-dev 2018-05-24 9722ca7 (https://www.mediafire.com/file/ob3xwv4lkef2kld/xvc-dev_2018-05-24_9722ca7.7z/file) (dev branch commits (https://github.com/divideon/xvc/commits/dev))

[jonatans]

New version of xvc just released!

The source code is available in the master branch at https://github.com/divideon/xvc.

More info about the release in the official press release: http://www.releasewire.com/press-releases/divideon-releases-xvc-20-an-open-source-video-codec-with-a-royalty-free-baseline-1005793.htm

A few highlights:
- Better compression performance
- A royalty-free baseline profile
- Multi-threaded encoding
- Dual-licensing scheme: LGPL and commercial license

Enjoy! / Jonatan

[LigH]

Fresh build of xvc v2.0 2018-07-04 (commit 283009f) in my archive (https://www.mediafire.com/?4dcbb2dwc5tby). No obvious code additions since last dev build in May.

[colinhunt]

^ Thanks!

[colinhunt]

Hmm, I can't make xvc 2.0 do multi-threading. Tried with "-threads 10" and "-threads -1" but it insists on running on a single core. Cmdline:

xvcenc.exe -input-file moto.yuv -input-width 1920 -input-height 1080 -input-bitdepth 8 -framerate 24 -verbose 1 -threads -1 -speed-mode 2 -qp 30 -explicit-encoder-setti
ngs "aqp_strength 16" -output-file output.xvc

[jonatans]

Thanks for making it available LigH. And thanks for testing colinhunt!

Did you try to run it long enough for the multi-threading to kick in?

Our multi-threading implementation uses picture level parallelism and due to the prediction structure it is only after the first two pictures have been completely encoded that multiple pictures are encoded in parallel. (Before that everything is coded in a single thread)

I just tried both my own compiled xvcenc and the one provided by LigH and both are able to do multi-threading.

[colinhunt]

Well, shoot. My impatience got the best of me yet again! I'll re-run the cmdline, and this time I'll wait for longer than 30 seconds. Thanks Jonatan!

update: it has now processed 19 pictures, based on the list it outputs on screen, but I still don't see multi-threading. Average CPU usage hovers at 5-6% and it looks like only a single core/thread is running at 100%.

...okay, I noticed now there is *some* life in the graphs for 7-8 threads but those peak at 5% before quickly dying back down to 0%. Not quite what I expected tbh.

update #2: I changed "-threads -1" to "threads 20" and now re-running the cmdline. A moment ago xvc's CPU usage hit 47%, while 10-ish threads were running at 100%. Encoder spat a number of lines of text on screen, followed by CPU usage dropping to 15% ... It jumped to 25% while I was typing that previous sentence ... And now to 45%. Here comes another set of text on screen, CPU use goes down to 10% and the cycle starts again.

[colinhunt]

Encoding finished in 94 minutes. Some months ago the same source file with same encoding parameters (except multi-threading, of course) took 183 minutes to encode. So almost 50% faster now with multi-threading.

[jonatans]

Hmm, that does not seem to be right. You should get substantially more than 2x speed-up (unless you are actually running it on a 2 core CPU ;) ).

Perhaps you can try using this binary instead:

https://drive.google.com/open?id=1sP7KQdj6C8VPe6wdnP5Mdixkvo5TqsSI

The reason for why you don't see higher than 50% CPU usage could potentially be related to memory-usage. With 20 threads and 1080p resolution, xvc might try to allocate more memory than what is available on the system. Are you running on a 20 core system? Perhaps you can try using something in the order of 8 or 10 threads instead to get more stable performance. Or try with lower resolution video just to see if you can get all cores fully utilized.

As you have correctly noted there is currently no way of letting xvcenc figure out the optimal number of threads to use (as is done in xvcdec). This is also related to the memory-usage. With additional testing on a wide variety of combinations of CPU-cores and memory-availability, it should be possible to figure out a good scheme for doing this automatically, but we haven't done that yet.

[iwod]

The first version of xvc was released in September 2017 and was able to clearly outperform all existing video codecs including HEVC. The second version of xvc delivers even better performance and makes it possible to reduce the bitrate compared to the recently released AV1 codec by between 10% and 20% for the same visual quality.

I really rather like the PR. The humble, tone down figures. Pretty sure the first version already had similar to AV1 performance or better. At least according to other members clips and testing.

[colinhunt]

Perhaps you can try using this binary instead
I'll give it a shot.

The reason for why you don't see higher than 50% CPU usage could potentially be related to memory-usage. With 20 threads and 1080p resolution, xvc might try to allocate more memory than what is available on the system.
It was hogging a bit over 5GB, on a 32GB system.

Are you running on a 20 core system?
No, the CPU's a 10-core / 20-thread Xeon.

Perhaps you can try using something in the order of 8 or 10 threads instead to get more stable performance.
I'll try with 10 threads next time. Thanks Jonatan.

[Leeloo Minaï]

I tried binary from jonatans and multi-threading seems to work well, at least for me.

Is it possible to show encoding progress in the next release ?

[colinhunt]

xvcenc.exe -input-file moto.yuv -input-width 1920 -input-height 1080 -input-bitdepth 8 -framerate 24 -verbose 1 -threads 20 -speed-mode 2 -qp 30 -explicit-encoder-settings "aqp_strength 16" -output-file output.xvc

v1 binary: 183 minutes, no multi-threading.
v2 binary (LigH's): 94 minutes. 20 threads, CPU usage peaked at 47%.
v2 binary (Jonatan's): 51,3 minutes. 20 threads, RAM usage 5.9GB, CPU usage peaked at 56%.

v2 binary (Jonatan's): 51 minutes. 10 threads, CPU usage peaked at 55%.

v2 binary (Jonatan's): 51 minutes. 40 threads, RAM usage 8.5GB, CPU usage peaked at 59%.
v2 binary (Jonatan's): 51 minutes. 8 threads, RAM usage 2.1GB.

[LigH]

No clue why mine is slower. Plain GCC 7.3.0 build. What's different in jonatans', new dev revision or higher CPU optimization while compiling C sources?

[MoSal]

- Dual-licensing scheme: LGPL and commercial license

Is the first L in LGPL intentional?

[Zebulon84]

LGPL = Lesser General Public License.
Lesser because it allows the work to be linked to work with other type of license. Here it allows it to be used as a library (.dll) in any software. If it was GPL, it could only be used within GPL software.

[MoSal]

LGPL = Lesser General Public License.
Lesser because it allows the work to be linked to work with other type of license. Here it allows it to be used as a library (.dll) in any software. If it was GPL, it could only be used within GPL software.

I know. That's why I'm asking. Dual licensing is usually GPL/commercial. Or GPL/LGPL.

LGPL/commercial still makes sense if it's about the commercial licensee making source modifications and not contributing back. But, if this is about other limitations in the LGPL, then one would be better served with another license like the MPLv2.

[jonatans]

Is the first L in LGPL intentional?

Yes.

No clue why mine is slower. Plain GCC 7.3.0 build. What's different in jonatans', new dev revision or higher CPU optimization while compiling C sources?

I don't know really. Mine is built from xvc version 2, master branch, commit 283009f using the Visual Studio compiler.

I also get much slower encoding times with your build. Will try to look further into what causes the difference.

xvcenc.exe -input-file moto.yuv -input-width 1920 -input-height 1080 -input-bitdepth 8 -framerate 24 -verbose 1 -threads 20 -speed-mode 2 -qp 30 -explicit-encoder-settings "aqp_strength 16" -output-file output.xvc

v1 binary: 183 minutes, no multi-threading.
v2 binary (LigH's): 94 minutes. 20 threads, CPU usage peaked at 47%.
v2 binary (Jonatan's): 51,3 minutes. 20 threads, RAM usage 5.9GB, CPU usage peaked at 56%.

v2 binary (Jonatan's): 51 minutes. 10 threads, CPU usage peaked at 55%.

v2 binary (Jonatan's): 51 minutes. 40 threads, RAM usage 8.5GB, CPU usage peaked at 59%.
v2 binary (Jonatan's): 51 minutes. 8 threads, RAM usage 2.1GB.

Thanks for the extensive testing colinhunt. The reason why you do not see higher speed-up might be related to that the sequence you are encoding is roughly equally long as the segment length you are using. If you would be using a sequence that was 4 times as long or if you would set -max-keypic-distance to something like 160 you might get a higher speed-up factor (but that would of course also affect the compressed result).

[colinhunt]

The reason why you do not see higher speed-up might be related to that the sequence you are encoding is roughly equally long as the segment length you are using.
The source file, which I've shared earlier in this thread, is 468 frames long. What's the default segment length? That's what I'm (probably?) using.

[jonatans]

The source file, which I've shared earlier in this thread, is 468 frames long. What's the default segment length? That's what I'm (probably?) using.

Default segment length is 640. To get high level of parallelism with a 468 frames long sequence you might try for example -max-keypic-distance 160 or even -max-keypic-distance 128.

[jonatans]

Some more speedups and printout of PSNR has been added to the master branch of https://github.com/divideon/xvc (use -tune 1 if you want to perform analysis based on PSNR and -tune 0 for best visual quality).

It would be interesting to hear more reports on the usage of threading in xvc in different configurations and on different platforms to see if there are settings that can be adjusted for improved performance.

Other comments and suggestions are of course also welcome.

[LigH]

New upload: xvc v2.0 2018-08-30 #1cbe133 (https://www.mediafire.com/file/n89izdm9z96zu9f/xvc_v2.0_2018-08-30_1cbe133.7z/file) (MSYS2, MinGW32 + GCC 7.3.0 / MinGW64 + GCC 8.2.0)

[Tommy Carrot]

I made only 1 test encode, so i dont know if it's a general behaviour or not, but the resulting video is identical to the one made with the older build from may, but it took nearly twice the time to finish. After the first keyframe is finished, it waits several minutes seemingly doing nothing (using 25% cpu on a 4 core haswell i5) until it starts to encode the inter-frames. Maybe i remember incorrectly, but i dont think the older build did that long wait. After that it seems to work ok, using 96-98% cpu at times. I set the number of threads to 4.

[LigH]

Does the 32 bit (GCC 7.3.0) build delay the same way as the 64 bit (GCC 8.2.0) build?

[Tommy Carrot]

Yes, the 32 bit build is doing the same. I had a 10 minute "idle time" between the first keyframe and the first inter-frame encoded. The whole encoding is taking 23 minutes. The old build did it in 11 minutes with identical output.

[jonatans]

I have uploaded 64-bit binaries built with Visual Studio. Available here: https://drive.google.com/open?id=1K0KLaaHRvH9i-SZHodEB_vDrj5XrajkA

[Tommy Carrot]

Your visual studio build doesn't do that long delay, it finished the same encode in 10 minutes.

edit: just to clarify, it still waits 2-3 minutes before it finishes the first inter-frame, it just seems to be generally faster.

[LigH]

Then we may need someone with experience in GNU profiling tools to discover the reason...

[benwaggoner]

Your visual studio build doesn't do that long delay, it finished the same encode in 10 minutes.

edit: just to clarify, it still waits 2-3 minutes before it finishes the first inter-frame, it just seems to be generally faster.
I'm guessing there is some kind of long lookahead taking place, so a bunch of frames are getting encoded and refined in RAM before the first inter frame in decode order gets written. This happens in all codecs to some degree. For example, a GOP like

IbBbPbBbP

Actually needs to get written to the file as:
IPBbbPBbb

The second frame can't be written until the fifth frame is finalized.

[LigH]

New upload: (MSYS2, MinGW32 + GCC 7.3.0 / MinGW64 + GCC 8.2.0)

xvc v2.0 2018-09-23 #d3e8b46 (https://www.mediafire.com/file/xjvj0e1s368av38/xvc_v2.0_2018-09-23_d3e8b46.7z/file)

New commit d3e8b46 in master: Allow parsing one picture nal after segment header

[Gravitator]

xvc v2.0 2018-09-23 #d3e8b46
C:\>xvcenc

Usage:
-input-file <string> -output-file <string> [Optional parameters]

Optional parameters:
-rec-file <string>
-input-width <16..65535>
-input-height <16..65535>
-input-chroma-format <0..3>
0: Monochrome
1: 4:2:0 (default)
2: 4:2:2
3: 4:4:4
-input-bitdepth <8..14>
-internal-bitdepth <8..14>
-framerate <0.0054..90000>
-skip-pictures <int>
-temporal-subsample <int>
-max-pictures <int>
-sub-gop-length <1..64> (default: 16)
-max-keypic-distance <int> (default: 640)
-closed-gop <int> (default: 0)
-num-ref-pics <0..5> (default: 2)
-checksum-mode <0..1>
0: Reduced checksum verification (default)
1: Maximum checksum robustness
-deblock <0..1> (default: 1)
-beta-offset <-32..31>
-tc-offset <-32..31>
-qp <-64..63> (default: 32)
-multi-passes <0..2>
0: Single-pass (default)
1: Single pass with start picture determination
2: Multi-pass
-speed-mode <0..2>
0: Placebo
1: Slow (default)
2: Fast
-tune <0..1>
0: Visual quality (default)
1: PSNR
-profile <0..1>
0: Full (default)
1: Baseline
-threads <int>
-1: auto-detect
0: disabled (default)
1+: number of threads
-verbose <0..1>
There is no "-explicit-encoder-settings "aqp_strength 16"" ?

[Gravitator]

Can you add the ability to pause xvc coding in ffmpeg?

[jonatans]

Can you add the ability to pause xvc coding in ffmpeg?

Sounds interesting but it's not something we have looked in to. Is this a capability that is available for any other codecs in ffmpeg? Can you provide an example?