xvc - a next-generation video codec at xvc.io

[LigH]

No clue why mine is slower. Plain GCC 7.3.0 build. What's different in jonatans', new dev revision or higher CPU optimization while compiling C sources?

[MoSal]

- Dual-licensing scheme: LGPL and commercial license

Is the first L in LGPL intentional?

[Zebulon84]

LGPL = Lesser General Public License.
Lesser because it allows the work to be linked to work with other type of license. Here it allows it to be used as a library (.dll) in any software. If it was GPL, it could only be used within GPL software.

[MoSal]

Originally Posted by Zebulon84

LGPL = Lesser General Public License.
Lesser because it allows the work to be linked to work with other type of license. Here it allows it to be used as a library (.dll) in any software. If it was GPL, it could only be used within GPL software.

I know. That's why I'm asking. Dual licensing is usually GPL/commercial. Or GPL/LGPL.

LGPL/commercial still makes sense if it's about the commercial licensee making source modifications and not contributing back. But, if this is about other limitations in the LGPL, then one would be better served with another license like the MPLv2.

[jonatans]

Originally Posted by MoSal

Is the first L in LGPL intentional?

Yes.

Originally Posted by LigH

No clue why mine is slower. Plain GCC 7.3.0 build. What's different in jonatans', new dev revision or higher CPU optimization while compiling C sources?

I don't know really. Mine is built from xvc version 2, master branch, commit 283009f using the Visual Studio compiler.

I also get much slower encoding times with your build. Will try to look further into what causes the difference.

Originally Posted by colinhunt

xvcenc.exe -input-file moto.yuv -input-width 1920 -input-height 1080 -input-bitdepth 8 -framerate 24 -verbose 1 -threads 20 -speed-mode 2 -qp 30 -explicit-encoder-settings "aqp_strength 16" -output-file output.xvc

v1 binary: 183 minutes, no multi-threading.
v2 binary (LigH's): 94 minutes. 20 threads, CPU usage peaked at 47%.
v2 binary (Jonatan's): 51,3 minutes. 20 threads, RAM usage 5.9GB, CPU usage peaked at 56%.

v2 binary (Jonatan's): 51 minutes. 10 threads, CPU usage peaked at 55%.

v2 binary (Jonatan's): 51 minutes. 40 threads, RAM usage 8.5GB, CPU usage peaked at 59%.
v2 binary (Jonatan's): 51 minutes. 8 threads, RAM usage 2.1GB.

Thanks for the extensive testing colinhunt. The reason why you do not see higher speed-up might be related to that the sequence you are encoding is roughly equally long as the segment length you are using. If you would be using a sequence that was 4 times as long or if you would set -max-keypic-distance to something like 160 you might get a higher speed-up factor (but that would of course also affect the compressed result).

[colinhunt]

Originally Posted by jonatans

The reason why you do not see higher speed-up might be related to that the sequence you are encoding is roughly equally long as the segment length you are using.

The source file, which I've shared earlier in this thread, is 468 frames long. What's the default segment length? That's what I'm (probably?) using.

[jonatans]

Originally Posted by colinhunt

The source file, which I've shared earlier in this thread, is 468 frames long. What's the default segment length? That's what I'm (probably?) using.

Default segment length is 640. To get high level of parallelism with a 468 frames long sequence you might try for example -max-keypic-distance 160 or even -max-keypic-distance 128.

[jonatans]

Some more speedups and printout of PSNR has been added to the master branch of https://github.com/divideon/xvc (use -tune 1 if you want to perform analysis based on PSNR and -tune 0 for best visual quality).

It would be interesting to hear more reports on the usage of threading in xvc in different configurations and on different platforms to see if there are settings that can be adjusted for improved performance.

Other comments and suggestions are of course also welcome.

[LigH]

New upload: xvc v2.0 2018-08-30 #1cbe133 (MSYS2, MinGW32 + GCC 7.3.0 / MinGW64 + GCC 8.2.0)

[Tommy Carrot]

I made only 1 test encode, so i dont know if it's a general behaviour or not, but the resulting video is identical to the one made with the older build from may, but it took nearly twice the time to finish. After the first keyframe is finished, it waits several minutes seemingly doing nothing (using 25% cpu on a 4 core haswell i5) until it starts to encode the inter-frames. Maybe i remember incorrectly, but i dont think the older build did that long wait. After that it seems to work ok, using 96-98% cpu at times. I set the number of threads to 4.

[LigH]

Does the 32 bit (GCC 7.3.0) build delay the same way as the 64 bit (GCC 8.2.0) build?

[Tommy Carrot]

Yes, the 32 bit build is doing the same. I had a 10 minute "idle time" between the first keyframe and the first inter-frame encoded. The whole encoding is taking 23 minutes. The old build did it in 11 minutes with identical output.

[jonatans]

I have uploaded 64-bit binaries built with Visual Studio. Available here: https://drive.google.com/open?id=1K0...EB_vDrj5XrajkA

[Tommy Carrot]

Your visual studio build doesn't do that long delay, it finished the same encode in 10 minutes.

edit: just to clarify, it still waits 2-3 minutes before it finishes the first inter-frame, it just seems to be generally faster.

[LigH]

Then we may need someone with experience in GNU profiling tools to discover the reason...

[benwaggoner]

Originally Posted by Tommy Carrot

Your visual studio build doesn't do that long delay, it finished the same encode in 10 minutes.

edit: just to clarify, it still waits 2-3 minutes before it finishes the first inter-frame, it just seems to be generally faster.

I'm guessing there is some kind of long lookahead taking place, so a bunch of frames are getting encoded and refined in RAM before the first inter frame in decode order gets written. This happens in all codecs to some degree. For example, a GOP like

IbBbPbBbP

Actually needs to get written to the file as:
IPBbbPBbb

The second frame can't be written until the fifth frame is finalized.

[LigH]

New upload: (MSYS2, MinGW32 + GCC 7.3.0 / MinGW64 + GCC 8.2.0)

xvc v2.0 2018-09-23 #d3e8b46

New commit d3e8b46 in master: Allow parsing one picture nal after segment header

[Gravitator]

xvc v2.0 2018-09-23 #d3e8b46

C:\>xvcenc

Usage:
-input-file <string> -output-file <string> [Optional parameters]

Optional parameters:
-rec-file <string>
-input-width <16..65535>
-input-height <16..65535>
-input-chroma-format <0..3>
0: Monochrome
1: 4:2:0 (default)
2: 4:2:2
3: 4:4:4
-input-bitdepth <8..14>
-internal-bitdepth <8..14>
-framerate <0.0054..90000>
-skip-pictures <int>
-temporal-subsample <int>
-max-pictures <int>
-sub-gop-length <1..64> (default: 16)
-max-keypic-distance <int> (default: 640)
-closed-gop <int> (default: 0)
-num-ref-pics <0..5> (default: 2)
-checksum-mode <0..1>
0: Reduced checksum verification (default)
1: Maximum checksum robustness
-deblock <0..1> (default: 1)
-beta-offset <-32..31>
-tc-offset <-32..31>
-qp <-64..63> (default: 32)
-multi-passes <0..2>
0: Single-pass (default)
1: Single pass with start picture determination
2: Multi-pass
-speed-mode <0..2>
0: Placebo
1: Slow (default)
2: Fast
-tune <0..1>
0: Visual quality (default)
1: PSNR
-profile <0..1>
0: Full (default)
1: Baseline
-threads <int>
-1: auto-detect
0: disabled (default)
1+: number of threads
-verbose <0..1>

There is no "-explicit-encoder-settings "aqp_strength 16"" ?

[Gravitator]

Can you add the ability to pause xvc coding in ffmpeg?

[jonatans]

Originally Posted by Gravitator

Can you add the ability to pause xvc coding in ffmpeg?

Sounds interesting but it's not something we have looked in to. Is this a capability that is available for any other codecs in ffmpeg? Can you provide an example?