Welcome to Doom9's Forum, THE in-place to be for everyone interested in DVD conversion.

Before you start posting please read the forum rules. By posting to this forum you agree to abide by the rules.

 

Go Back   Doom9's Forum > Video Encoding > New and alternative video codecs

Reply
 
Thread Tools Search this Thread Display Modes
Old 9th February 2018, 20:41   #21  |  Link
Selur
Registered User
 
Selur's Avatar
 
Join Date: Oct 2001
Location: Germany
Posts: 5,623
Argh,... I give up all the following calls:
Code:
ffmpeg -y -loglevel fatal -threads 8  -i "H:/sequence/ED-360-png/%05d.png" -frames:v 15691 -an -sn  -vsync 0 -pix_fmt yuv420p  -f yuv4mpegpipe - | xvcenc.exe -input-file - -verbose 1 -qp 18 -explicit-encoder-settings "aqp_strength 16" -output-file h:\elephantsDream_360.xvc
Code:
ffmpeg -y -loglevel fatal -threads 8  -i "H:/sequence/ED-1080-png/%05d.png" -frames:v 15691 -an -sn  -vsync 0 -pix_fmt yuv420p  -f yuv4mpegpipe - | xvcenc.exe -input-file - -verbose 1 -qp 18 -explicit-encoder-settings "aqp_strength 16" -output-file h:\elephantsDream_1080.xvc
Code:
ffmpeg -y -loglevel fatal -threads 8  -i "H:/sequence/tearsofsteel-4k.y4m" -an -sn  -vsync 0 -pix_fmt yuv420p  -f yuv4mpegpipe - | xvcenc.exe -input-file - -verbose 1 -qp 18 -explicit-encoder-settings "aqp_strength 16" -output-file h:\tearsofsteel-4k.xvc
ended with xvcenc crashing after running 2days or so,.. (got another call running, but I won't test this again until an new release,..)
Using another encoder like x264 or x265 instead of xvcenc didn't crash.

Cu Selur

Ps.: I also reencoded a 8bit console lagaringth capture using:
Code:
ffmpeg -y -loglevel fatal -threads 8  -i "F:\TestClips&Co\files\Console 8bit\ng2.avi" -an -sn  -vsync 0 -pix_fmt yuv420p  -f yuv4mpegpipe - | xvcenc.exe -input-file - -verbose 1 -qp 18 -explicit-encoder-settings "aqp_strength 16" -output-file h:\ng2.xvc
Code:
ffmpeg -y -loglevel fatal -threads 8  -i "F:\TestClips&Co\files\Console 8bit\ng2.avi" -an -sn  -vsync 0 -pix_fmt yuv420p  -f yuv4mpegpipe - | xvcenc.exe -input-file - -verbose 1 -qp 24 -explicit-encoder-settings "aqp_strength 16" -output-file h:\ng2_24.xvc
Code:
ffmpeg -y -loglevel fatal -threads 8  -i "F:\TestClips&Co\files\Console 8bit\ng2.avi" -an -sn  -vsync 0 -pix_fmt yuv420p  -f yuv4mpegpipe - | xvcenc.exe -input-file - -verbose 1 -qp 30 -explicit-encoder-settings "aqp_strength 16" -output-file h:\ng2_30.xvc
Code:
ffmpeg -y -loglevel fatal -threads 8  -i "F:\TestClips&Co\files\Console 8bit\ng2.avi" -an -sn  -vsync 0 -pix_fmt yuv420p  -f yuv4mpegpipe - | xvcenc.exe -input-file - -verbose 1 -qp 34 -explicit-encoder-settings "aqp_strength 16" -output-file h:\ng2_34.xvc
Code:
ffmpeg -y -loglevel fatal -threads 8  -i "F:\TestClips&Co\files\Console 8bit\ng2.avi" -an -sn  -vsync 0 -pix_fmt yuv420p  -f yuv4mpegpipe - | xvcenc.exe -input-file - -verbose 1 -qp 40 -explicit-encoder-settings "aqp_strength 16" -output-file h:\ng2_40.xvc
Code:
ffmpeg -y -loglevel fatal -threads 8  -i "F:\TestClips&Co\files\Console 8bit\ng2.avi" -an -sn  -vsync 0 -pix_fmt yuv420p  -f yuv4mpegpipe - | xvcenc.exe -input-file - -verbose 1 -qp 50 -explicit-encoder-settings "aqp_strength 16" -output-file h:\ng2_50.xvc
Code:
ffmpeg -y -loglevel fatal -threads 8  -i "F:\TestClips&Co\files\Console 8bit\ng2.avi" -an -sn  -vsync 0 -pix_fmt yuv420p  -f yuv4mpegpipe - | xvcenc.exe -input-file - -verbose 1 -qp 60 -explicit-encoder-settings "aqp_strength 16" -output-file h:\ng2_60.xvc
uploaded the files (source&reencodes) to: https://drive.google.com/drive/folde...kp?usp=sharing
__________________
Hybrid here in the forum, homepage

Last edited by Selur; 9th February 2018 at 23:34.
Selur is offline   Reply With Quote
Old 12th February 2018, 15:53   #22  |  Link
jonatans
Registered User
 
Join Date: Oct 2017
Posts: 30
Thanks Selur for sharing.

I tried to replicate your ED-360 encoding using both master (730987c) and dev (452170c) and both of my encodings completed successfully.

Did you use the same compiled encoder for all your runs? (I.e. version 1 of xvc - from the master branch)

Did you change anything compared to the other run that you reported in your earlier post (which from my understanding, was completed successfully) other than using -qp 18 -explicit-encoder-settings "aqp_strength 16"?

Please share any information you have about the crash (potential error message or printout, verbose output etc.)

Did you compile for 32-bit or 64-bit architecture?
jonatans is offline   Reply With Quote
Old 13th February 2018, 04:44   #23  |  Link
Selur
Registered User
 
Selur's Avatar
 
Join Date: Oct 2001
Location: Germany
Posts: 5,623
Quote:
Did you use the same compiled encoder for all your runs?
yes.

Quote:
Did you change anything compared to the other run that you reported in your earlier post (which from my understanding, was completed successfully) other than using -qp 18 -explicit-encoder-settings "aqp_strength 16"?
Nope. I simply used the calls I posted.

Quote:
Please share any information you have about the crash (potential error message or printout, verbose output etc.)
There was no additonal, uncommon printout or error message, just the usual Windows popup that complained about xvcenc crashing.

Quote:
Did you compile for 32-bit or 64-bit architecture?
64bit. Uploaded the two binaries (xvcenc, xvcdec) to https://drive.google.com/drive/folde...Tg?usp=sharing .

Cu Selur
__________________
Hybrid here in the forum, homepage
Selur is offline   Reply With Quote
Old 14th February 2018, 12:23   #24  |  Link
Tommy Carrot
Registered User
 
Tommy Carrot's Avatar
 
Join Date: Mar 2002
Posts: 839
I ran a few tests with Selur's build (thanks btw). I did not test a wide variety of bitrate ranges and just tried a few test videos, but this encoder consistently beats av1 in all cases. Very promising, good job jonatans!
Tommy Carrot is offline   Reply With Quote
Old 14th February 2018, 15:55   #25  |  Link
jonatans
Registered User
 
Join Date: Oct 2017
Posts: 30
Quote:
Originally Posted by Tommy Carrot View Post
I ran a few tests with Selur's build (thanks btw). I did not test a wide variety of bitrate ranges and just tried a few test videos, but this encoder consistently beats av1 in all cases. Very promising, good job jonatans!
Thank you Tommy!
jonatans is offline   Reply With Quote
Old 7th March 2018, 15:56   #26  |  Link
jonatans
Registered User
 
Join Date: Oct 2017
Posts: 30
New results for xvc published at xvc.io. The xvc codec performs clearly better than HEVC (HM) and AV1, with bitrate savings of 26% and 20%, respectively, for the 360p test sequences in the NETVC test set.
jonatans is offline   Reply With Quote
Old 8th March 2018, 20:14   #27  |  Link
colinhunt
Registered User
 
Join Date: Dec 2002
Posts: 998
Not sure if I'm doing something wrong here. It took my 12-core Xeon roughly 53 seconds per frame to encode a 10-frame test, using a 1080p60 source. I used Selur's (hi!) build of xvcenc and one of his encoding scripts from post #21.

update: There's apparently some sort of a set-up cost at the beginning of the process. A 300-frame long encode just finished in 3179 seconds, i.e. an average of 11 seconds per frame. CPU load held steady at 5.5% during the encode.

Last edited by colinhunt; 8th March 2018 at 21:08.
colinhunt is offline   Reply With Quote
Old 9th March 2018, 01:46   #28  |  Link
colinhunt
Registered User
 
Join Date: Dec 2002
Posts: 998
I uploaded two source files and two encoding tests here: https://drive.google.com/open?id=1Hv...UnL-VbQybO6Zfd

The 1080p59.94 source footage was shot by me. Feel free to use the provided source files for your encoding testing (altho the footage is not for re-sale).

The 15-second source clips are encoded in 1) 8-bit 4:2:0 AVC at 42 Mbps (.mp4, 76MB) and 2) 8-bit 4:2:2 DNxHR SQ at 291 Mbps (.mov, 525MB).
colinhunt is offline   Reply With Quote
Old 9th March 2018, 13:34   #29  |  Link
jonatans
Registered User
 
Join Date: Oct 2017
Posts: 30
Thanks colinhunt! Nice to see that it's working ok.

Quality seems fine on your encodings, given that it's a compression ratio of around 50:1 compared to the mp4 file and around 350:1 compared to the mov file.

It's true that the encoding is slow, and there is no multi-threading.

However, a nice property of the .xvc files is that you can concatenate any two of them to create a new valid .xvc file.

So if you want to encode a longer sequence and use more than a single core, you can split up the work and then merge the output files at the end. It will however result in closed-GOP intra pictures so it will have a negative visual impact (especially at low rates and at short clips).

I created a simple python script for doing this. It expects a y4m input file but it should be possible to extend it to support any input format by adding a call to ffmpeg and using a seek operation to fetch the right part.
https://drive.google.com/open?id=1hR...ijjl72SpSGMYYV

I have also uploaded new binaries built from the latest commit of the dev branch. The default speed mode is now even slower than in version 1 (but also produces better quality) but there is also a speed-mode 2 which should give faster encoding times than xvc version 1.0 (and hopefully also better visual quality). It should be noted that these are experimental binaries for evaluation purposes only. It has not yet been decided what the final set of coding tools in xvc 2.0 will be.
https://drive.google.com/open?id=1kW...uQTZP-Rqqdy9Jx
jonatans is offline   Reply With Quote
Old 9th March 2018, 13:52   #30  |  Link
colinhunt
Registered User
 
Join Date: Dec 2002
Posts: 998
Quote:
Originally Posted by jonatans View Post
Quality seems fine on your encodings, given that it's a compression ratio of around 50:1 compared to the mp4 file and around 350:1 compared to the mov file.
One of the encodes was actually created from an uncompressed 8-bit 4:2:2 .AVI source, not the .mov. The .avi is almost 4 gigabytes which is wayyy too much for my slow internet connection, hence it's not in the shared folder.

Thanks for the python script and new binaries!
colinhunt is offline   Reply With Quote
Old 9th March 2018, 16:10   #31  |  Link
colinhunt
Registered User
 
Join Date: Dec 2002
Posts: 998
Got an error from the new encoding binary (I named it xvcenc_dev.exe). First here's the script I ran:

ffmpeg -y -loglevel fatal -threads 8 -i "10bit-dnxhd.mov" -frames:v 100 -an -sn -vsync 0 -pix_fmt yuv422p -f yuv4mpegpipe - | xvcenc_dev.exe -input-file - -verbose 1 -input-chroma-format 2 -input-bitdepth 10 -speed-mode 2 -qp 25 -explicit-encoder-settings "aqp_strength 16" -output-file d:\output.xvc

Note: source is 10bit 4:2:2 DNxHD 1080p59.94. ffplay has no problem playing it, and ffmpeg transcodes it just fine.

Error msg: Assertion failed: 0, file C:\Users\jonat\proj\xvc\app\xvc_enc_app\y4m_reader.cc, line 93

Currently encoding the same source using Selur's build, same parameters except without "-speed-mode 2". Hasn't crashed so far.

update - uploaded the DNxHD source to shared folder: "fruit--dnxhd-10bit-422-440mbps--1080p5994.mov" (1.16 GB)

Last edited by colinhunt; 9th March 2018 at 16:45.
colinhunt is offline   Reply With Quote
Old 9th March 2018, 17:37   #32  |  Link
jonatans
Registered User
 
Join Date: Oct 2017
Posts: 30
Thanks for pointing this out!

The reason for the assert is that the y4m read function did not have support for 422. We have just checked in a fix to that. (There was no such support in version 1 either, but that binary might have been compiled without the asserts)

Here is a new binary: https://drive.google.com/open?id=1iJ...yQmWu6EwPwOQ10

Now it should be able to understand both 422 and 444 directly from the y4m file so there is no need to indicate "-input-chroma-format 2 -input-bitdepth 10". I hope it works fine now!
jonatans is offline   Reply With Quote
Old 9th March 2018, 17:54   #33  |  Link
colinhunt
Registered User
 
Join Date: Dec 2002
Posts: 998
Thanks!

Hmm.. I'm seeing this upon running the new binary, regardless of whether parameters include "-input-chroma-format 2 -input-bitdepth 10" or not (tried both ways):

Input: -
Output: d:\output.xvc
Size: 1920x1080
Bitdepth: 8
Framerate: 59.9401
QP: 25

Selur's build reported a bitdepth of 10 with same source file.
colinhunt is offline   Reply With Quote
Old 9th March 2018, 18:41   #34  |  Link
jonatans
Registered User
 
Join Date: Oct 2017
Posts: 30
Hi again,

I think the problem relates to the settings for the ffmpeg pipe. In order to support 10 bit you need to add "-strict -1" and change from "-pix_fmt yuv422p" to "-pix_fmt yuv422p10le". Or remove the "-pix_fmt" parameter altogether if you want ffmpeg to pipe the same format as the original file.

That seems to be working fine for me.
jonatans is offline   Reply With Quote
Old 9th March 2018, 18:48   #35  |  Link
colinhunt
Registered User
 
Join Date: Dec 2002
Posts: 998
^ Thank you!

I let that previous "Bitdepth: 8" encode finish. When I played it, decoder reported both output and internal bitstream had a bitdepth of 10. Well, no matter, I removed "-pix_fmt" and added "-strict -1" , and encoder is now reporting 10bit.

Last edited by colinhunt; 9th March 2018 at 18:52.
colinhunt is offline   Reply With Quote
Old 11th March 2018, 12:08   #36  |  Link
colinhunt
Registered User
 
Join Date: Dec 2002
Posts: 998
Added a new source file to the share. It's a 10-bit 4:2:2 1080p24.00 DNxHD .mov of motocross bikes rushing off from the starting line. Also added a qp32 xvc encode of that source with a bitrate of approx. 1 Mbps.

Source files in share:

Fruits


Moss


Moto

Last edited by colinhunt; 11th March 2018 at 12:25.
colinhunt is offline   Reply With Quote
Old 13th March 2018, 09:46   #37  |  Link
foxyshadis
ангел смерти
 
foxyshadis's Avatar
 
Join Date: Nov 2004
Location: Lost
Posts: 9,374
So is this an automatic 2-pass? The length of time before real encoding starts seems to indicate that it's doing some serious analysis beforehand. Is there a plan for developing a streaming encoder, or is the format generally designed only for archival?
__________________
There are four boxes to be used in defense of liberty: soap, ballot, jury, and ammo. Please use in that order. ~ Ed Howdershelt
foxyshadis is offline   Reply With Quote
Old 13th March 2018, 13:26   #38  |  Link
jonatans
Registered User
 
Join Date: Oct 2017
Posts: 30
Quote:
Originally Posted by colinhunt View Post
Also added a qp32 xvc encode of that source with a bitrate of approx. 1 Mbps.
Thanks for sharing additional encodings. Have you compared it to any other codec at the same bitrate?

Quote:
Originally Posted by foxyshadis View Post
So is this an automatic 2-pass? The length of time before real encoding starts seems to indicate that it's doing some serious analysis beforehand. Is there a plan for developing a streaming encoder, or is the format generally designed only for archival?
The xvc codec is targeting streaming rather than archival, but so far only offline (non-realtime) encoding. There is no speed mode in xvc (yet) that can do realtime encoding. In the dev version of xvc there is experimental support for automatic multi-pass encoding (-multi-passes 2) and there is also a single pass mode that includes a quick determination of which start picture to use (-multi-passes 1) but the default is normal single pass encoding (-multi-passes 0) and in that mode there is no analysis of the video prior to encoding.

The encoding is quite slow so it will take some time before the first picture has been encoded (and the statistics of that picture is output if -verbose 1 is used), so maybe that is the reason why it appears to take long time before encoding is started.
jonatans is offline   Reply With Quote
Old 13th March 2018, 21:42   #39  |  Link
colinhunt
Registered User
 
Join Date: Dec 2002
Posts: 998
Quote:
Originally Posted by jonatans View Post
Thanks for sharing additional encodings. Have you compared it to any other codec at the same bitrate?
Ran a comparison just now. Encoded the "moto" clip at 1mbps, using x265 Slow 1-pass preset, to match xvc encode's bitrate. Below is a graph depicting the 3SSIM metric for x265 and xvc encodes compared to the DNxHD original. Closer to 1.000 is better.



x265 (green) catches up with xvc (red) briefly at frame 250, also somewhere around frames 332-335, and from frame 385 onwards it fares slightly better than xvc on average. It's perhaps worth noting that I used speed-mode 2 for the xvc encode.

(Argh. Noticed now that I had misnamed the files; instead of 1080p5994 both should have read 1080p24.)

Last edited by colinhunt; 14th March 2018 at 15:03.
colinhunt is offline   Reply With Quote
Old 14th March 2018, 11:10   #40  |  Link
jonatans
Registered User
 
Join Date: Oct 2017
Posts: 30
Quote:
Originally Posted by colinhunt View Post
Ran a comparison just now.
Thanks colinhunt for running the comparison and sharing the results!
jonatans is offline   Reply With Quote
Reply

Tags
codec, compression, video codec, video encoding, xvc

Thread Tools Search this Thread
Search this Thread:

Advanced Search
Display Modes

Posting Rules
You may not post new threads
You may not post replies
You may not post attachments
You may not edit your posts

BB code is On
Smilies are On
[IMG] code is On
HTML code is Off

Forum Jump


All times are GMT +1. The time now is 16:23.


Powered by vBulletin® Version 3.8.11
Copyright ©2000 - 2018, vBulletin Solutions Inc.