xvc - a next-generation video codec at xvc.io

[jonatans]

We have quite recently launched the new next-generation video codec xvc at xvc.io (https://xvc.io/). The complete source code for encoding and decoding is available at GitHub with the development happening in the dev branch: https://github.com/divideon/xvc/tree/dev.

The xvc codec is free to use for everyone for personal use, evaluations and research. There is also a commercial license available.

The ambition is for xvc to be the world's most efficient video codec while still keeping decoding complexity at a reasonable level in order to support efficient software decoding on various platforms.

In an article about xvc (http://www.streamingmedia.com/Articles/Editorial/Featured-Articles/Divideon-Creates-xvc-an-HEVC-Codec-With-Reasonable-Pricing-122180.aspx), Streaming Media comments on the performance of xvc as "Pretty impressive performance" with xvc being the quality leader compared to HEVC, H.264, VP9 and AV1 for the two "real-world files".

Please try it out and share what you think of it. All sorts of feedback is highly appreciated. The encoder has not yet been optimized for speed and is currently very slow (unless your point of reference is AV1, in which case the xvc encoder is actually very fast :)).

[Selur]

Divideon has developed several proof of concept integrations for xvc playback using ExoPlayer, VLC and ffmpeg. Feel free to contact support@xvc.io if you have any questions or need help integrating with your platforms.
Will those integrations be part of the normal releases of VLC, ffmpeg any time soon?
Encoding

The following is an example command line for encoding a raw YUV sequence using an intra picture interval of 256 pictures:
Also would be nice if the command line encoder would accept y4m content via pipe to avoid humongous yuv files.

Also: any recompiled Windows 64bit binaries available atm?

Cu Selur

[jonatans]

Will those integrations be part of the normal releases of VLC, ffmpeg any time soon?

We have not initiated integration with the normal releases of VLC and FFmpeg. We plan to make the wrapper code available together with the next official release of xvc and we are open to make such integration at any point in time.

Also would be nice if the command line encoder would accept y4m content via pipe to avoid humongous yuv files.

Good point! This functionality has just been checked in to the dev branch.

Also: any recompiled Windows 64bit binaries available atm?

Not yet. We'll probably start publishing binaries for different platforms at the next official release of xvc (or possibly even sooner). For the time being we recommend users to compile xvc themselves. Let me know if you have any questions related to building xvc.

[nevcairiel]

Will those integrations be part of the normal releases of VLC, ffmpeg any time soon?

The reference software is non-free, so it'll face some challenges being distributed with open-source software like VLC or FFmpeg. If the codec is to become popular, a free and open-source decoder would be needed, since those make the basis for a large share of video players and tools.

[iwod]

While I think that is a nice way of solving the patent problem, but what happen if a patents was responsible for let say 10% of the compression gain, and at a later stage had to be pulled.

Companies who decide on codec had those cost saving in mind. And all of a sudden it is no longer there?

Sometimes I wonder how much does it actually cost to buy all the patents in the current HEVC Pool.

[jonatans]

While I think that is a nice way of solving the patent problem, but what happen if a patents was responsible for let say 10% of the compression gain, and at a later stage had to be pulled.

Companies who decide on codec had those cost saving in mind. And all of a sudden it is no longer there?

True. And that is why we try to make it very clear that there is a risk that there will be third party patent assertions, just as with any other codec. And users of the codec will have to account for that risk and be prepared to react to it. The main difference with xvc and other codecs is that xvc has a framework for dealing with third party patent assertions from organizations that are not interested in taking part of the licensing program.

Whenever a third party patent infringments is asserted, the first option is to invite the patent holder to join the licensing program. If this is not successful, the validity of the infringement assertion will be assesed (to determine if the assertion should be challenged in court). And only if none of these two options is successful will the technology be removed.

In practice, we do not expect this situation to occur very often (if it occurs at all) and we are confident that we will be able to quickly circumvent such technology and offer an alternative solution with similar performance.

In fact, most of the compression tools provides quite small compression gain in isolation (below 1%) so the impact of turning off just a couple of them would be very minor.

[nevcairiel]

But doesn't that then potentially mean that previously encoded files become un-decodable as some features are disabled?

[jonatans]

But doesn't that then potentially mean that previously encoded files become un-decodable as some features are disabled?

Correct. If a specific technology needs to be removed from xvc in the next xvc version, bitstreams that made use of that technology can no longer be decoded by the latest xvc decoder.

This is somewhat similar to how for example different generations of MPEG codecs or H.26x codecs or VPx codecs, not necessarily offer support for decoding bitstreams of their previous generation. With the difference that with xvc, it might potentially occur more frequently.

If technology removal occurs, service providers with large assets of xvc bitstreams have the options of:
1) transcode their bitstreams to the new xvc version (which might be a lightweight process depending on the technology that is being replaced and which might even offer better compression if the new xvc version also includes new compression tools)
2) come to an agreement with the patent holder outside of the xvc licensing program in order to be able to continue to use the technology.
3) ditch the xvc version of their legacy assets and fall back to their h.264 version for those assets (which they'll probably anyway have, in order to support some legacy platforms) and just use the new xvc version for new assets.

[jonatans]

We have just added an online comparison of xvc and h.264 for low bit rate streaming scenarios: https://www.divideon.com/products-and-services/mobile-video-streaming-with-xvc/

Please have a look and share your thoughts!

[iwod]

We have just added an online comparison of xvc and h.264 for low bit rate streaming scenarios: https://www.divideon.com/products-and-services/mobile-video-streaming-with-xvc/

Please have a look and share your thoughts!

At this moment i really wish someone could explain to me how to always be optimistic, because they say I am a pessimist.

If technology removal occurs, service providers with large assets of xvc bitstreams have the options of:
1) transcode their bitstreams to the new xvc version (which might be a lightweight process depending on the technology that is being replaced and which might even offer better compression if the new xvc version also includes new compression tools)
2) come to an agreement with the patent holder outside of the xvc licensing program in order to be able to continue to use the technology.
3) ditch the xvc version of their legacy assets and fall back to their h.264 version for those assets (which they'll probably anyway have, in order to support some legacy platforms) and just use the new xvc version for new assets.

1. Well there goes Youtube, Hulu, Netflix, or basically every other big streaming site or providers. So after patents problem 1 they re-encode their files, then when patents problem number 2 happens they repeat that again?

2. What if you cant come to agreement if they purposely charge you an outrageous amount?

3. Ditching everything they have done?

I dont think this is a technology problem, but an go to market problem.

[ChaosKing]

Quality looks good but it would be nice to know which h.264 codec was used + settings.

[birdie]

We have just added an online comparison of xvc and h.264 for low bit rate streaming scenarios: https://www.divideon.com/products-and-services/mobile-video-streaming-with-xvc/

Please have a look and share your thoughts!

Without x265, VP9 and AV1 results (which is close to completion) this comparison looks quite barren.

[jonatans]

Quality looks good but it would be nice to know which h.264 codec was used + settings.

Thanks!
For the h.264 encoding, x264 was used with preset placebo.

I have uploaded the uncompressed y4m file here (114 MB):
https://drive.google.com/open?id=1dXxwA7P_81EjOyn_0GFA6Lvm3OX8rnKQ

Encoding with x264:
ffmpeg -i tos.y4m -c:v libx264 -preset placebo -x264-params keyint=1080 -crf 38 tos_x264_120kbps.mp4
Encoding with xvc:
xvcenc.exe -input-file tos.y4m -speed-mode 0 -max-keypic-distance 1080 -qp 31-internal-bitdepth 8 -explicit-encoder-settings "aqp_strength 16" -output-file tos_xvc_120kbps.xvc -verbose 1

FFmpeg version: 20170305-035e932
xvc commit 79466050f8818e352ab0fbc6abe6bb4ed0b455c5

File sizes:
tos_x264_120kbps.mp4 - 358 KB
tos_xvc_120kbps.xvc - 346 KB

[jonatans]

Without x265, VP9 and AV1 results (which is close to completion) this comparison looks quite barren.

I agree that it would be interesting with visual comparisons with other codecs. At this point though, h.264 is the only codec supported for playback in all browsers. And since h.264 is the codec that is most widely used today I would argue that it is still a relevant comparison, in order to get an understanding of what level of quality difference to expect when switching to xvc.

I created the following HEVC encoding using x265:
https://drive.google.com/open?id=1wYBwYipissnaOnS9ydDQ5cfPPxU3XGmz

from the following commands:
x265-64bit-8bit-2018-01-22.exe --input tos.y4m --keyint 1080 --preset placebo --crf 34 tos_x265_124kbps.265
ffmpeg -i tos_x265_124kbps.265 -codec copy tos_x265_124kbps.mp4
File size: 375 KB

Perhaps someone from the AOM group would like to create an AV1-encoding for comparison? ;)
The uncompressed y4m file is available here (114 MB):
https://drive.google.com/open?id=1dXxwA7P_81EjOyn_0GFA6Lvm3OX8rnKQ

[Selur]

any news on pipe support for those y4m files?

[Asmodian]

Pipe support for y4m would be great. It would be a huge usability improvement, especially since I already use y4m piping for encoding with x264 and x265. :)

[jonatans]

any news on pipe support for those y4m files?

Thanks for the feedback.

Pipe support for y4m files was added to the dev branch in December. The dev branch is where all development happens and I would suggest that you use that one if you are trying out xvc.

However, I have now also applied that commit to the master branch.

I've also added information about valid ranges for the command line parameters.

Please let me know if something doesn't seem correct or if you have any other feedback.

[Selur]

Please let me know if something doesn't seem correct or if you have any other feedback.
Some (optional) progress indication would be nice even when not using '-verbose 1'. First thought nothing was happening until I realized that xvcenc actually was using <7% of the available cpu using:
ffmpeg -y -loglevel fatal -threads 8 -i "H:/sequence/ED-360-png/%05d.png" -frames:v 15691 -an -sn -vsync 0 -pix_fmt yuv420p -f yuv4mpegpipe - | xvcenc.exe -input-file - -verbose 1 -output-file h:\test.xvc
The process indication should at least output the current frame number (missing that with 'verbose 1' too ;)), since at least for longer sequences it is kind of hard to at least have a rough

Also a question what does qp stand for? I assumed that it would stand for quantizer parameter, but a range from -64 to 63 seems rather strange to me since I would normally assume that a quantizer of 0 would mean lossless. So what would happen with negative quantizers?
With the default qp (32) and verbose output enabled I see that the qp is fluctuating

So:
Good news: Encoding seems to be working.
(kind of expected) Bad news: no multi threading at all.


Cu Selur

Ps.: took me 35269 s to encode 15691 pictures, so more than 2 seconds per frame with 6-7% cpu usage. :)

[jonatans]

Thank you Selur for the comments.

The only level of progress indication right now is with "-verbose 1" and it is done on SubGop level (and not frame level) since the print-out is coupled with the delivery of encoded data from the encoder lib to the command line application.

You are completely correct in that there is no multi-threading support on the encoder yet. This is of course a desirable feature that we would like to see support for soon (together with other speed improvements) but so far we are still more focused on producing really high quality with no specific encoding time limitations.

You are right in that "-qp" stands for "quantization parameter". In general terms, a higher qp gives coarser quantization steps (lower quality) and a lower qp gives finer quantization steps (higher quality). However, internally xvc will use the "-qp" as input for calculating which base qp to use for different pictures, depending on the SubGop length and the current picture's position in the SubGop. Furthermore, within a picture, different blocks will use different qp depending on the local energy.

The range of "-qp" is -64 to 63. There is no mathematical lossless mode in xvc and depending on the internal bitdepth it might actually make sense to use qp below 0, although in general I would say that "-qp" should probably be between 15 and 45 in order to produce useful result.

Did you get a chance to compare your encoded result to some other codec? Any comments on the quality?

[Selur]

Did you get a chance to compare your encoded result to some other codec? Any comments on the quality?
Not so far plan to do some encoding over the week, wanted to wait for a reply regarding the qp to get a rough idea what it's about. ;)

[Selur]

Argh,... I give up all the following calls:
ffmpeg -y -loglevel fatal -threads 8 -i "H:/sequence/ED-360-png/%05d.png" -frames:v 15691 -an -sn -vsync 0 -pix_fmt yuv420p -f yuv4mpegpipe - | xvcenc.exe -input-file - -verbose 1 -qp 18 -explicit-encoder-settings "aqp_strength 16" -output-file h:\elephantsDream_360.xvc
ffmpeg -y -loglevel fatal -threads 8 -i "H:/sequence/ED-1080-png/%05d.png" -frames:v 15691 -an -sn -vsync 0 -pix_fmt yuv420p -f yuv4mpegpipe - | xvcenc.exe -input-file - -verbose 1 -qp 18 -explicit-encoder-settings "aqp_strength 16" -output-file h:\elephantsDream_1080.xvc
ffmpeg -y -loglevel fatal -threads 8 -i "H:/sequence/tearsofsteel-4k.y4m" -an -sn -vsync 0 -pix_fmt yuv420p -f yuv4mpegpipe - | xvcenc.exe -input-file - -verbose 1 -qp 18 -explicit-encoder-settings "aqp_strength 16" -output-file h:\tearsofsteel-4k.xvc
ended with xvcenc crashing after running 2days or so,.. (got another call running, but I won't test this again until an new release,..)
Using another encoder like x264 or x265 instead of xvcenc didn't crash.

Cu Selur

Ps.: I also reencoded a 8bit console lagaringth capture using:
ffmpeg -y -loglevel fatal -threads 8 -i "F:\TestClips&Co\files\Console 8bit\ng2.avi" -an -sn -vsync 0 -pix_fmt yuv420p -f yuv4mpegpipe - | xvcenc.exe -input-file - -verbose 1 -qp 18 -explicit-encoder-settings "aqp_strength 16" -output-file h:\ng2.xvc

ffmpeg -y -loglevel fatal -threads 8 -i "F:\TestClips&Co\files\Console 8bit\ng2.avi" -an -sn -vsync 0 -pix_fmt yuv420p -f yuv4mpegpipe - | xvcenc.exe -input-file - -verbose 1 -qp 24 -explicit-encoder-settings "aqp_strength 16" -output-file h:\ng2_24.xvc

ffmpeg -y -loglevel fatal -threads 8 -i "F:\TestClips&Co\files\Console 8bit\ng2.avi" -an -sn -vsync 0 -pix_fmt yuv420p -f yuv4mpegpipe - | xvcenc.exe -input-file - -verbose 1 -qp 30 -explicit-encoder-settings "aqp_strength 16" -output-file h:\ng2_30.xvc

ffmpeg -y -loglevel fatal -threads 8 -i "F:\TestClips&Co\files\Console 8bit\ng2.avi" -an -sn -vsync 0 -pix_fmt yuv420p -f yuv4mpegpipe - | xvcenc.exe -input-file - -verbose 1 -qp 34 -explicit-encoder-settings "aqp_strength 16" -output-file h:\ng2_34.xvc

ffmpeg -y -loglevel fatal -threads 8 -i "F:\TestClips&Co\files\Console 8bit\ng2.avi" -an -sn -vsync 0 -pix_fmt yuv420p -f yuv4mpegpipe - | xvcenc.exe -input-file - -verbose 1 -qp 40 -explicit-encoder-settings "aqp_strength 16" -output-file h:\ng2_40.xvc

ffmpeg -y -loglevel fatal -threads 8 -i "F:\TestClips&Co\files\Console 8bit\ng2.avi" -an -sn -vsync 0 -pix_fmt yuv420p -f yuv4mpegpipe - | xvcenc.exe -input-file - -verbose 1 -qp 50 -explicit-encoder-settings "aqp_strength 16" -output-file h:\ng2_50.xvc

ffmpeg -y -loglevel fatal -threads 8 -i "F:\TestClips&Co\files\Console 8bit\ng2.avi" -an -sn -vsync 0 -pix_fmt yuv420p -f yuv4mpegpipe - | xvcenc.exe -input-file - -verbose 1 -qp 60 -explicit-encoder-settings "aqp_strength 16" -output-file h:\ng2_60.xvc
uploaded the files (source&reencodes) to: https://drive.google.com/drive/folders/1fnyiP8j0JZDE7aO6uRyU9Ne0WwWRvfkp?usp=sharing

[jonatans]

Thanks Selur for sharing.

I tried to replicate your ED-360 encoding using both master (730987c) and dev (452170c) and both of my encodings completed successfully.

Did you use the same compiled encoder for all your runs? (I.e. version 1 of xvc - from the master branch)

Did you change anything compared to the other run that you reported in your earlier post (which from my understanding, was completed successfully) other than using -qp 18 -explicit-encoder-settings "aqp_strength 16"?

Please share any information you have about the crash (potential error message or printout, verbose output etc.)

Did you compile for 32-bit or 64-bit architecture?

[Selur]

Did you use the same compiled encoder for all your runs?
yes.

Did you change anything compared to the other run that you reported in your earlier post (which from my understanding, was completed successfully) other than using -qp 18 -explicit-encoder-settings "aqp_strength 16"?
Nope. I simply used the calls I posted.

Please share any information you have about the crash (potential error message or printout, verbose output etc.)
There was no additonal, uncommon printout or error message, just the usual Windows popup that complained about xvcenc crashing.

Did you compile for 32-bit or 64-bit architecture?
64bit. Uploaded the two binaries (xvcenc, xvcdec) to https://drive.google.com/drive/folders/0B_WxUS1XGCPASUZibG5XZkRfeTg?usp=sharing .

Cu Selur

[Tommy Carrot]

I ran a few tests with Selur's build (thanks btw). I did not test a wide variety of bitrate ranges and just tried a few test videos, but this encoder consistently beats av1 in all cases. Very promising, good job jonatans!

[jonatans]

I ran a few tests with Selur's build (thanks btw). I did not test a wide variety of bitrate ranges and just tried a few test videos, but this encoder consistently beats av1 in all cases. Very promising, good job jonatans!

Thank you Tommy!

[jonatans]

New results for xvc published at xvc.io (https://xvc.io/concept/performance/). The xvc codec performs clearly better than HEVC (HM) and AV1, with bitrate savings of 26% and 20%, respectively, for the 360p test sequences in the NETVC test set.

[colinhunt]

Not sure if I'm doing something wrong here. It took my 12-core Xeon roughly 53 seconds per frame to encode a 10-frame test, using a 1080p60 source. I used Selur's (hi!) build of xvcenc and one of his encoding scripts from post #21.

update: There's apparently some sort of a set-up cost at the beginning of the process. A 300-frame long encode just finished in 3179 seconds, i.e. an average of 11 seconds per frame. CPU load held steady at 5.5% during the encode.

[colinhunt]

I uploaded two source files and two encoding tests here: https://drive.google.com/open?id=1Hv9FcBRjEixEuKNNi3UnL-VbQybO6Zfd

The 1080p59.94 source footage was shot by me. Feel free to use the provided source files for your encoding testing (altho the footage is not for re-sale).

The 15-second source clips are encoded in 1) 8-bit 4:2:0 AVC at 42 Mbps (.mp4, 76MB) and 2) 8-bit 4:2:2 DNxHR SQ at 291 Mbps (.mov, 525MB).

[jonatans]

Thanks colinhunt! Nice to see that it's working ok.

Quality seems fine on your encodings, given that it's a compression ratio of around 50:1 compared to the mp4 file and around 350:1 compared to the mov file.

It's true that the encoding is slow, and there is no multi-threading.

However, a nice property of the .xvc files is that you can concatenate any two of them to create a new valid .xvc file.

So if you want to encode a longer sequence and use more than a single core, you can split up the work and then merge the output files at the end. It will however result in closed-GOP intra pictures so it will have a negative visual impact (especially at low rates and at short clips).

I created a simple python script for doing this. It expects a y4m input file but it should be possible to extend it to support any input format by adding a call to ffmpeg and using a seek operation to fetch the right part.
https://drive.google.com/open?id=1hRWN8n9124hM5t_S57ijjl72SpSGMYYV

I have also uploaded new binaries built from the latest commit of the dev branch. The default speed mode is now even slower than in version 1 (but also produces better quality) but there is also a speed-mode 2 which should give faster encoding times than xvc version 1.0 (and hopefully also better visual quality). It should be noted that these are experimental binaries for evaluation purposes only. It has not yet been decided what the final set of coding tools in xvc 2.0 will be.
https://drive.google.com/open?id=1kWFt0ehuhScfkvn7qPuQTZP-Rqqdy9Jx

[colinhunt]

Quality seems fine on your encodings, given that it's a compression ratio of around 50:1 compared to the mp4 file and around 350:1 compared to the mov file.
One of the encodes was actually created from an uncompressed 8-bit 4:2:2 .AVI source, not the .mov. The .avi is almost 4 gigabytes which is wayyy too much for my slow internet connection, hence it's not in the shared folder.

Thanks for the python script and new binaries!

[colinhunt]

Got an error from the new encoding binary (I named it xvcenc_dev.exe). First here's the script I ran:

ffmpeg -y -loglevel fatal -threads 8 -i "10bit-dnxhd.mov" -frames:v 100 -an -sn -vsync 0 -pix_fmt yuv422p -f yuv4mpegpipe - | xvcenc_dev.exe -input-file - -verbose 1 -input-chroma-format 2 -input-bitdepth 10 -speed-mode 2 -qp 25 -explicit-encoder-settings "aqp_strength 16" -output-file d:\output.xvc

Note: source is 10bit 4:2:2 DNxHD 1080p59.94. ffplay has no problem playing it, and ffmpeg transcodes it just fine.

Error msg: Assertion failed: 0, file C:\Users\jonat\proj\xvc\app\xvc_enc_app\y4m_reader.cc, line 93

Currently encoding the same source using Selur's build, same parameters except without "-speed-mode 2". Hasn't crashed so far.

update - uploaded the DNxHD source to shared folder: "fruit--dnxhd-10bit-422-440mbps--1080p5994.mov" (1.16 GB)

[jonatans]

Thanks for pointing this out!

The reason for the assert is that the y4m read function did not have support for 422. We have just checked in a fix to that. (There was no such support in version 1 either, but that binary might have been compiled without the asserts)

Here is a new binary: https://drive.google.com/open?id=1iJsjuMZtTsnUiv6jNsyQmWu6EwPwOQ10

Now it should be able to understand both 422 and 444 directly from the y4m file so there is no need to indicate "-input-chroma-format 2 -input-bitdepth 10". I hope it works fine now!

[colinhunt]

Thanks!

Hmm.. I'm seeing this upon running the new binary, regardless of whether parameters include "-input-chroma-format 2 -input-bitdepth 10" or not (tried both ways):

Input: -
Output: d:\output.xvc
Size: 1920x1080
Bitdepth: 8
Framerate: 59.9401
QP: 25

Selur's build reported a bitdepth of 10 with same source file.

[jonatans]

Hi again,

I think the problem relates to the settings for the ffmpeg pipe. In order to support 10 bit you need to add "-strict -1" and change from "-pix_fmt yuv422p" to "-pix_fmt yuv422p10le". Or remove the "-pix_fmt" parameter altogether if you want ffmpeg to pipe the same format as the original file.

That seems to be working fine for me.

[colinhunt]

^ Thank you!

I let that previous "Bitdepth: 8" encode finish. When I played it, decoder reported both output and internal bitstream had a bitdepth of 10. Well, no matter, I removed "-pix_fmt" and added "-strict -1" , and encoder is now reporting 10bit.

[colinhunt]

Added a new source file to the share. It's a 10-bit 4:2:2 1080p24.00 DNxHD .mov of motocross bikes rushing off from the starting line. Also added a qp32 xvc encode of that source with a bitrate of approx. 1 Mbps.

Source files in share:

Fruits
https://saitti.kuvat.fi/kuvat/encode_src/fruit-dnxhd-10bit-422.png/_smaller.jpg

Moss
https://saitti.kuvat.fi/kuvat/encode_src/moss-dnxhrsq-8bit-422.png/_smaller.jpg

Moto
https://saitti.kuvat.fi/kuvat/encode_src/moto-dnxhd-10bit-422.png/_smaller.jpg

[foxyshadis]

So is this an automatic 2-pass? The length of time before real encoding starts seems to indicate that it's doing some serious analysis beforehand. Is there a plan for developing a streaming encoder, or is the format generally designed only for archival?

[jonatans]

Also added a qp32 xvc encode of that source with a bitrate of approx. 1 Mbps.

Thanks for sharing additional encodings. Have you compared it to any other codec at the same bitrate?

So is this an automatic 2-pass? The length of time before real encoding starts seems to indicate that it's doing some serious analysis beforehand. Is there a plan for developing a streaming encoder, or is the format generally designed only for archival?

The xvc codec is targeting streaming rather than archival, but so far only offline (non-realtime) encoding. There is no speed mode in xvc (yet) that can do realtime encoding. In the dev version of xvc there is experimental support for automatic multi-pass encoding (-multi-passes 2) and there is also a single pass mode that includes a quick determination of which start picture to use (-multi-passes 1) but the default is normal single pass encoding (-multi-passes 0) and in that mode there is no analysis of the video prior to encoding.

The encoding is quite slow so it will take some time before the first picture has been encoded (and the statistics of that picture is output if -verbose 1 is used), so maybe that is the reason why it appears to take long time before encoding is started.

[colinhunt]

Thanks for sharing additional encodings. Have you compared it to any other codec at the same bitrate?

Ran a comparison just now. Encoded the "moto" clip at 1mbps, using x265 Slow 1-pass preset, to match xvc encode's bitrate. Below is a graph depicting the 3SSIM metric for x265 and xvc encodes compared to the DNxHD original. Closer to 1.000 is better.

https://saitti.kuvat.fi/kuvat/encode_src/moto--dnxhd-10bit-422-1080p2400fps-468frames_xvc--moto-1mbps-speed2-10bit422-1080p5994_x265--moto-1mbps-slow1pass-10bit422-1080p5994-ffmpeg_3ssim.png?img=medium

x265 (green) catches up with xvc (red) briefly at frame 250, also somewhere around frames 332-335, and from frame 385 onwards it fares slightly better than xvc on average. It's perhaps worth noting that I used speed-mode 2 for the xvc encode.

(Argh. Noticed now that I had misnamed the files; instead of 1080p5994 both should have read 1080p24.)

[jonatans]

Ran a comparison just now.

Thanks colinhunt for running the comparison and sharing the results!

[nevcairiel]

The high peaks (or low valleys, if you want) look disturbing. I can only assume those are I frames, a smoother difference between the frame types would feel more appropriate.

[jonatans]

The high peaks (or low valleys, if you want) look disturbing. I can only assume those are I frames, a smoother difference between the frame types would feel more appropriate.

The peaks are not related to I-pictures. They represent the temporal layer 0 pictures. The default coding structure in xvc is a hierarchical B-picture tree with 15 B-pictures.

Giving higher quality to pictures of lower temporal layers actually improves the overall quality in general. If the coding settings are modified to give more equal distribution you will get worse quality in most pictures. The quality of the pictures in low temporal layers needs to be reduced and since the pictures in higher temporal layers use those pictures as reference pictures, there will be a negative impact on the prediction.

As long as the lowest quality pictures are of good quality and there are no visual problems I don't think there is anything inappropriate with a bit of peaks and valleys in objective quality scores. Colinhunt's graph indicates that for most of the sequence the lowest quality xvc pictures are actually better than the highest quality x265 pictures.

I would be very happy to hear comments and feedback regarding the visual quality.

[colinhunt]

It might be interesting to hear/learn why x265 catches up and overtakes xvc from frame 385 onwards in my sample clip encodes.

[colinhunt]

I ran a few encodes today, with the aim of comparing xvc and av1. Now, this is far from a definitive comparison, seeing as I often have trouble finding my own butt with both hands, and generally just mess about with encoders like a drunkard stumbling around in the dark.

Aaanyway, here's how I set up the encodes.

source: 1080p24 8bit 4:2:0 469 frames
xvc: speed mode 2, qp30, explicit setting "aqp_strength 16", single pass
av1: cpu_used 2, tile-columns 6, usage vbr, threads 14, bitrate-target 1370kbps, single pass

Encoder versions

xvc: binary from Jonatans' post #32 in this thread
av1: LigH's binary 0.1.0-8449-g6471e8bd7

Encoding times, reported bitrates and filesizes

xvc: 10984 seconds / 1375 kbps (by encoder) / 3 360 139 bytes (.xvc)
av1: 9120 seconds / 1404 kbps (by Mediainfo) / 3 580 224 bytes (.webm)

It should be noted that xvc ran on a single thread at a cpu load of 4%, while av1 ran on 14 threads at an average cpu load of 20% (bouncing between 4-60%).

Metrics
Red is xvc, green is av1. Higher, i.e. closer to 1.000, is better.

3SSIM
https://saitti.kuvat.fi/kuvat/VQMetrics/3SSIM-xvc-av1.png/_medium.jpg

MSSIM
https://saitti.kuvat.fi/kuvat/VQMetrics/MSSIM-xvc-av1.png/_medium.jpg

[LigH]

:eek: A new toy!

:( Slower than AV1...

:rolleyes: MABS won't include it, fearing MPEG license related problems; but gave manual building instructions (https://github.com/jb-alvarado/media-autobuild_suite/issues/734#issuecomment-362054956) which seem to be successful.

[colinhunt]

:( Slower than AV1...
20 or so percent slower, yeah, but in my test xvc was running on single thread while av1 was running on 14 threads. It'll be interesting to see xvc performance once/if they implement multi-threading.

:rolleyes: MABS won't include it, fearing MPEG license related problems; but gave manual building instructions (https://github.com/jb-alvarado/media-autobuild_suite/issues/734#issuecomment-362054956) which seem to be successful.
Harsh words there by the MABS dev...

[jonatans]

Thanks colinhunt for reporting more performance numbers!

Just a quick comment on the patent situation in xvc. Divideon (the developers of the xvc codec) is an MPEG member and we are in continuous dialog with the patent holders to try our best to get all the necessary agreements in place. But even though that may take some time, I would like to highlight that xvc is already in a better position than other codecs when it comes to licensing. Unless you are using a codec that is more than 25 years old, you will be at risk of infringing patents (and that would also be the case for software implementations like x264/x265 and libaom), and you will be at risk of receiving requests for paying a license for using those patents. The difference with xvc and the other codecs is that we promise to help you out (e.g. by designing around the patent in question) if the patent holder makes an unreasonable claim.

[MoSal]

I would like to highlight that xvc is already in a better position than other codecs when it comes to licensing. Unless you are using a codec that is more than 25 years old, you will be at risk of infringing patents (and that would also be the case for software implementations like x264/x265 and libaom), and you will be at risk of receiving requests for paying a license for using those patents.


I'm not sure why you are not getting it. FUDing upfront is going to be as fruitful as FUDing down the line, that is, not fruitful at all.

Good luck with your post-xvc endeavors.

[LigH]

I will focus on the technical aspects and follow the development loosely until I see the chance to actually use it practically. Best wishes from my side.

[foxyshadis]

I'm not sure why you are not getting it. FUDing upfront is going to be as fruitful as FUDing down the line, that is, not fruitful at all.

Good luck with your post-xvc endeavors.

That's not FUD. FUD is Google boldly carrying On2's torch with a promise that nothing they make has ever been touched by a patent, and basically being forced to indemnify their adopters when the inevitable patent pool formed once the source code came out. (Which, to their credit, they actually did for all VP8 users up to that point. Pre-Google adopters could take it up with On2.)

Saying that "things happen" and the next version will excise any patent-protected process is a realistic warning to their users to hope for the best but expect rocky roads and re-encodings.