Welcome to Doom9's Forum, THE in-place to be for everyone interested in DVD conversion.

Before you start posting please read the forum rules. By posting to this forum you agree to abide by the rules.

 

Go Back   Doom9's Forum > Video Encoding > New and alternative video codecs

Reply
 
Thread Tools Search this Thread Display Modes
Old 15th April 2022, 22:04   #41  |  Link
FranceBB
Broadcast Encoder
 
FranceBB's Avatar
 
Join Date: Nov 2013
Location: Royal Borough of Kensington & Chelsea, UK
Posts: 2,883
I'm really surprised to see ffv1 outperforming x265 lossless, even with long GOP. About huffyuv, that's what I generally use as intermediate format through the ffvhuff implementation and it's fast enough for my tastes given that the files I produce are really temporary and used in between before getting to the compressed version, but I always thought that x265 would have outperformed ffv1, huffyuv and lagarith by a margin and I didn't expect ffv1 to actually be better than x265. Thanks for your comparison. Next time I would suggest adding UTVideo to the test just to see how it performs in terms of compression and speed.
FranceBB is offline   Reply With Quote
Old 16th April 2022, 11:53   #42  |  Link
PCU
Registered User
 
Join Date: Oct 2017
Posts: 327
Quote:
Originally Posted by FranceBB View Post
I'm really surprised to see ffv1 outperforming x265 lossless, even with long GOP. About huffyuv, that's what I generally use as intermediate format through the ffvhuff implementation and it's fast enough for my tastes given that the files I produce are really temporary and used in between before getting to the compressed version, but I always thought that x265 would have outperformed ffv1, huffyuv and lagarith by a margin and I didn't expect ffv1 to actually be better than x265. Thanks for your comparison. Next time I would suggest adding UTVideo to the test just to see how it performs in terms of compression and speed.
Michael Niedermayer:

you mean HDR10+ or something else ? about HDR10+ i did send a mail to the guy who added this for libvpx in ffmpeg with matroska and the same code should work with ffv1 if it was just not intentionally vpx specific i think

if i hear nothing back from that guy (plausible) and i dont forget (i tend to forget these things) then ill move it over so it can be used with any codec

that should unlock HDR10+ for ffv1 but no way to be 100% sure if thats enough before implementing it

i didnt look into dolby vision yet so i cannot comment but i suspect its not hugely different

Last edited by PCU; 16th April 2022 at 13:22.
PCU is offline   Reply With Quote
Old 16th April 2022, 15:29   #43  |  Link
kolak
Registered User
 
Join Date: Nov 2004
Location: Poland
Posts: 2,843
Quote:
Originally Posted by FranceBB View Post
I'm really surprised to see ffv1 outperforming x265 lossless, even with long GOP. About huffyuv, that's what I generally use as intermediate format through the ffvhuff implementation and it's fast enough for my tastes given that the files I produce are really temporary and used in between before getting to the compressed version, but I always thought that x265 would have outperformed ffv1, huffyuv and lagarith by a margin and I didn't expect ffv1 to actually be better than x265. Thanks for your comparison. Next time I would suggest adding UTVideo to the test just to see how it performs in terms of compression and speed.
ffv1 was written from the ground as lossless codec. h264/5 are not about lossless mode at all. Lossless compression is very different than lossy and I doubt h264/5 lossless modes are polished.
kolak is offline   Reply With Quote
Old 19th April 2022, 01:20   #44  |  Link
Balling
Registered User
 
Join Date: Feb 2020
Posts: 539
Quote:
Originally Posted by kolak View Post
ffv1 was written from the ground as lossless codec. h264/5 are not about lossless mode at all. Lossless compression is very different than lossy and I doubt h264/5 lossless modes are polished.
It is not very different it uses YCgCo-R presentation internally (without tagging stuff in matrix in VUI), so is rather effecient, 9 bit YCgCo is enough for RGB 8 bit. Complexity of ffv1 is not comparable with H.26X series. Both hevc_nvenc and h264_nvenc support mathematically lossless!

x265 encoder library lossless mode is not very optimised, so it is known, but that is just because it does not yet support SCC, just CABAC. Read this, as you can see reference encoder lossy and lossless are so much better with scc https://www.cambridge.org/core/servi...48770315000116

Not yet even ffmpeg native hevc decoder support it yet, but there are patches from Intel https://patchwork.ffmpeg.org/project...t/?series=2878

While hevc with Ice Lake of Intel is supporting SCC already.

JPEG 2000 and AV1 also have lossless modes, last one can outperform (because it has SCC-like already).

Grain generation for HEVC can increase it even further (H.274)

Last edited by Balling; 19th April 2022 at 01:42.
Balling is offline   Reply With Quote
Old 19th April 2022, 13:03   #45  |  Link
Balling
Registered User
 
Join Date: Feb 2020
Posts: 539
Quote:
Originally Posted by benwaggoner View Post
Do AV1 encoders generally enable the lossless option? x265 via command line is the only HEVC encoder that does off the top of my head.

AV1 encode is generally quite slow and decode is as well. And in many/most cases a lossless AV1 would be going through software decode to due level requirements, so even slower to use in practice.

Off the top of my head I can't think of any reason AV1 would have a material file size advantage over HEVC for the lossless case.
AV1 had a bug in lossless encoding in ffmpeg for a long time but it was fixed. Now crf 0 works losslessly, I mean I participated in fixing this bug. All encoders, VP9, AVC, HEVC, AV1 supports lossless.
Balling is offline   Reply With Quote
Old 19th April 2022, 13:09   #46  |  Link
nevcairiel
Registered Developer
 
Join Date: Mar 2010
Location: Hamburg/Germany
Posts: 10,344
Quote:
Originally Posted by Balling View Post
I mean I participated in fixing this bug..
You are banned on the ffmpeg development mailing list. You participated in nothing constructive.
__________________
LAV Filters - open source ffmpeg based media splitter and decoders
nevcairiel is offline   Reply With Quote
Old 22nd April 2022, 12:43   #47  |  Link
tormento
Acid fr0g
 
tormento's Avatar
 
Join Date: May 2002
Location: Italy
Posts: 2,542
Quote:
Originally Posted by FranceBB View Post
I'm really surprised to see ffv1 outperforming x265 lossless
The huge difference is that you can have x265 lossless by a gpu card, with almost no CPU resorces used.
__________________
@turment on Telegram
tormento is offline   Reply With Quote
Old 22nd April 2022, 13:44   #48  |  Link
FranceBB
Broadcast Encoder
 
FranceBB's Avatar
 
Join Date: Nov 2013
Location: Royal Borough of Kensington & Chelsea, UK
Posts: 2,883
Quote:
Originally Posted by tormento View Post
The huge difference is that you can have x265 lossless by a gpu card, with almost no CPU resorces used.
Yeah but it's not supported by the Blackmagic Decklink capture cards, so... nope for my use case.
FranceBB is offline   Reply With Quote
Old 22nd April 2022, 20:27   #49  |  Link
excellentswordfight
Lost my old account :(
 
Join Date: Jul 2017
Posts: 322
Quote:
Originally Posted by tormento View Post
The huge difference is that you can have x265 lossless by a gpu card, with almost no CPU resorces used.
Since when did x265 support hw-encoding? Or are actually meaning HEVC in general, e.g. using nvenc?

Quote:
Originally Posted by FranceBB View Post
Yeah but it's not supported by the Blackmagic Decklink capture cards, so... nope for my use case.
Cant you pipe the input to nvenc? FFMPEG can be built with decklink support.

edit.
Just tried the lossless mode in nvenc for hevc, actually surprisingly good, outperformed x265 preset fast (x265 really is bad for lossless), still both beaten by x264 (at 1080p) at preset fast (which isnt that much slower then nvenc on a modern CPU). x264 lossless mode is actually rather good, but a bit limited in that it doesnt support 10bit (afaik).

Last edited by excellentswordfight; 22nd April 2022 at 20:48.
excellentswordfight is offline   Reply With Quote
Old 23rd April 2022, 09:51   #50  |  Link
PCU
Registered User
 
Join Date: Oct 2017
Posts: 327
Lossless mode with HEVC? how did you do that!?
PCU is offline   Reply With Quote
Old 23rd April 2022, 09:53   #51  |  Link
PCU
Registered User
 
Join Date: Oct 2017
Posts: 327
Quote:
Originally Posted by nevcairiel View Post
You are banned on the ffmpeg development mailing list. You participated in nothing constructive.
How to unsub from the FFmpeg mailing list? I still receive emails from them.
PCU is offline   Reply With Quote
Old 23rd April 2022, 11:33   #52  |  Link
rwill
Registered User
 
Join Date: Dec 2013
Posts: 343
Quote:
Originally Posted by Balling View Post
x265 encoder library lossless mode is not very optimised, so it is known, but that is just because it does not yet support SCC, just CABAC. Read this, as you can see reference encoder lossy and lossless are so much better with scc https://www.cambridge.org/core/servi...48770315000116
If you had actually understood the paper you are referencing you would have seen that the SCC extensions only work well with screen content like flat areas and text. The paper you referenced even shows clearly that there no gain for the video content like computer animation or camera captured content (normal video content this thread is about?).

SCC tools are completely useless except for screen content. It is very strong in video conferencing though. What you did is like saying PNG performs higher than JPEG.

CABAC is the entropy coder by the way, not a tool like SCC, you compared an Apple with an Orange.

Are you just typing words in google and use some GPT3 model to generate these forum posts ?
rwill is offline   Reply With Quote
Old 23rd April 2022, 11:35   #53  |  Link
rwill
Registered User
 
Join Date: Dec 2013
Posts: 343
Quote:
Originally Posted by PCU View Post
How to unsub from the FFmpeg mailing list? I still receive emails from them.
Google "ffmpeg mailing list unsubscribe". Its the first hit (the FFMPEG FAQ).
rwill is offline   Reply With Quote
Old 23rd April 2022, 15:42   #54  |  Link
poisondeathray
Registered User
 
Join Date: Sep 2007
Posts: 5,346
Quote:
Originally Posted by excellentswordfight View Post
x264 lossless mode is actually rather good, but a bit limited in that it doesnt support 10bit (afaik).
It does

1920x1080 10bit422 src (709,SDR) test, default settings incl. gop size (lossless outputs verified with psnr)

x264 445Mb/s
aom-av1 447Mb/s (bloody slow. Basically useless)
x265 454Mb/s
ffv1 462Mb/s
poisondeathray is offline   Reply With Quote
Old 23rd April 2022, 18:46   #55  |  Link
excellentswordfight
Lost my old account :(
 
Join Date: Jul 2017
Posts: 322
Quote:
Originally Posted by poisondeathray View Post
It does

1920x1080 10bit422 src (709,SDR) test, default settings incl. gop size (lossless outputs verified with psnr)

x264 445Mb/s
aom-av1 447Mb/s (bloody slow. Basically useless)
x265 454Mb/s
ffv1 462Mb/s
Strange, when I feed 10bit to x264 it looks like this:

x264 [info]: profile High 4:4:4 Predictive, level 5.1, 4:2:0, 8-bit

and when i set high10 I get:

x264 [error]: high10 profile doesn't support lossless

edit.
Hmm when I set --input-depth 10 --output-depth 10 the output is 10bit, but then its not using the High 4:4:4 Predictive profile, its using high10 which --fullhelp say doesnt support lossless (the output is suspiciously low at 500Mbps for 2160p movie content so I dont think its lossless), adding --profile high444 doesnt help. So what switches should I use?

edit2.
Using:

x264 0.164.3079 d9a19f0
(libswscale 6.1.101)
(libavformat 59.9.102)
(ffmpegsource 3.0.1.0)
(lsmash 2.16.1)
built on Dec 9 2021, gcc: 11.2.0
x264 configuration: --chroma-format=all
libx264 configuration: --chroma-format=all

Last edited by excellentswordfight; 23rd April 2022 at 19:10.
excellentswordfight is offline   Reply With Quote
Old 23rd April 2022, 20:25   #56  |  Link
poisondeathray
Registered User
 
Join Date: Sep 2007
Posts: 5,346
Quote:
Originally Posted by excellentswordfight View Post
Strange, when I feed 10bit to x264 it looks like this:

x264 [info]: profile High 4:4:4 Predictive, level 5.1, 4:2:0, 8-bit

and when i set high10 I get:

x264 [error]: high10 profile doesn't support lossless

edit.
Hmm when I set --input-depth 10 --output-depth 10 the output is 10bit, but then its not using the High 4:4:4 Predictive profile, its using high10 which --fullhelp say doesnt support lossless (the output is suspiciously low at 500Mbps for 2160p movie content so I dont think its lossless), adding --profile high444 doesnt help. So what switches should I use?

edit2.
Using:

x264 0.164.3079 d9a19f0
(libswscale 6.1.101)
(libavformat 59.9.102)
(ffmpegsource 3.0.1.0)
(lsmash 2.16.1)
built on Dec 9 2021, gcc: 11.2.0
x264 configuration: --chroma-format=all
libx264 configuration: --chroma-format=all


Not sure, I used an old r3000 x264 tmod binary by jpdsr
https://github.com/jpsdr/x264/releases

I didn't set profile for any of them, they were all "auto". This is what mediainfo said for 3 methods below

Quote:
Format profile : High 4:4:4 Predictive@L4
Frame rate : 23.976 (24000/1001) FPS
Color space : YUV
Chroma subsampling : 4:2:2
Bit depth : 10 bits

For direct input, I used demuxer lavf , src was prores file . If your binary does not have high bit depth support you won't get that "resize [warning]: converting from yuv422p10le to yuv422p16le"
Quote:
.
.
.
lavf [info]:
Format : mov,mp4,m4a,3gp,3g2,mj2
Codec : prores ( ProRes (iCodec Pro) )
PixFmt : yuv422p10le
Framerate : 2997/125
Timebase : 1/23976
Duration : 0:00:26
lavf [info]: 1920x1080p 0:1 @ 24000/1001 fps (cfr)
lavf [info]: color matrix: GBR
mp4 [info]: audio muxing feature is disabled.
resize [warning]: converting from yuv422p10le to yuv422p16le
x264 [info]: using cpu capabilities: MMX2 SSE2Fast SSSE3 SSE4.2 AVX FMA3 BMI2 AVX2
x264 [info]: profile High 4:4:4 Predictive, level 4.0, 4:2:2, 10-bit
.
.
.
1) demuxer lavf , direct input
Code:
"x264_x64.exe" --demuxer lavf --input-csp i422 --input-depth 10 --input-res 1920x1080 --fps 24000/1001 --output-csp i422 --output-depth 10 --qp 0 -o "x264_demuxerlavf.mp4"  "input.mov"
2) Or, for x264cli , you can use rawvideo pipe , e.g. ffmpeg rawvideo pipe
Code:
ffmpeg -i "input.mov" -pix_fmt yuv422p10le -f rawvideo - | "x264_x64.exe" --demuxer raw --input-csp i422 --input-depth 10 --input-res 1920x1080 --fps 24000/1001 --output-csp i422 --output-depth 10 --qp 0 -o x264_rawpipe.mp4 -
3) Or you can use ffmpeg libx264 directly, it will automatically set the pixel format to 10bit422 if input was 10bit422 .

Code:
ffmpeg -i input.mov -an -c:v libx264 -qp 0 libx264.mp4
All 3 methods were verified lossless by psnr
poisondeathray is offline   Reply With Quote
Old 24th April 2022, 10:41   #57  |  Link
PCU
Registered User
 
Join Date: Oct 2017
Posts: 327
Quote:
Originally Posted by rwill View Post
Google "ffmpeg mailing list unsubscribe". Its the first hit (the FFMPEG FAQ).
- Results:
Ignoring non-text/plain MIME parts
***@***.com is not a member of the ffmpeg-user mailing list

I still receive emails from the list!
PCU is offline   Reply With Quote
Old 24th April 2022, 13:59   #58  |  Link
MasterNobody
Registered User
 
Join Date: Jul 2007
Posts: 552
Quote:
Originally Posted by excellentswordfight View Post
Strange, when I feed 10bit to x264 it looks like this:

x264 [info]: profile High 4:4:4 Predictive, level 5.1, 4:2:0, 8-bit

and when i set high10 I get:

x264 [error]: high10 profile doesn't support lossless

edit.
Hmm when I set --input-depth 10 --output-depth 10 the output is 10bit, but then its not using the High 4:4:4 Predictive profile, its using high10 which --fullhelp say doesnt support lossless (the output is suspiciously low at 500Mbps for 2160p movie content so I dont think its lossless), adding --profile high444 doesnt help. So what switches should I use?

edit2.
Using:

x264 0.164.3079 d9a19f0
(libswscale 6.1.101)
(libavformat 59.9.102)
(ffmpegsource 3.0.1.0)
(lsmash 2.16.1)
built on Dec 9 2021, gcc: 11.2.0
x264 configuration: --chroma-format=all
libx264 configuration: --chroma-format=all
1) H.264 lossless mode is supported only in high444 irrespectively of bit depth (8-bit or 10-bit). So you shouldn't specify profile (keep auto) or specify high444.
2) To enable lossless in x264 you should use `--qp 0` (not `--crf 0` especially for 10-bit).
3) To enable 10-bit encoding you should specify `--output-depth 10` (because default is 8 irrespectively of source).

Last edited by MasterNobody; 24th April 2022 at 14:04.
MasterNobody is offline   Reply With Quote
Old 24th April 2022, 15:06   #59  |  Link
excellentswordfight
Lost my old account :(
 
Join Date: Jul 2017
Posts: 322
Quote:
Originally Posted by MasterNobody View Post
1) H.264 lossless mode is supported only in high444 irrespectively of bit depth (8-bit or 10-bit). So you shouldn't specify profile (keep auto) or specify high444.
2) To enable lossless in x264 you should use `--qp 0` (not `--crf 0` especially for 10-bit).
3) To enable 10-bit encoding you should specify `--output-depth 10` (because default is 8 irrespectively of source).
Thank you! Using qp 0 and not crf did the trick! I can also see now that fullhelp states that qp 0 is the switch for lossless mode. But its a bit strange though as it seems like the encoder recognize crf 0 as lossless mode in some combinations.

Last edited by excellentswordfight; 24th April 2022 at 15:16.
excellentswordfight is offline   Reply With Quote
Old 24th April 2022, 21:37   #60  |  Link
rwill
Registered User
 
Join Date: Dec 2013
Posts: 343
Quote:
Originally Posted by PCU View Post
- Results:
Ignoring non-text/plain MIME parts
***@***.com is not a member of the ffmpeg-user mailing list

I still receive emails from the list!
Maybe you are subscribed to the devel list...

Have you checked the links to the respective different ffmpeg mailing lists here:

https://ffmpeg.org/contact.html#MailingLists ?

Otherwise I guess you have to pester some ffmpeg admin.
rwill is offline   Reply With Quote
Reply

Thread Tools Search this Thread
Search this Thread:

Advanced Search
Display Modes

Posting Rules
You may not post new threads
You may not post replies
You may not post attachments
You may not edit your posts

BB code is On
Smilies are On
[IMG] code is On
HTML code is Off

Forum Jump


All times are GMT +1. The time now is 08:32.


Powered by vBulletin® Version 3.8.11
Copyright ©2000 - 2024, vBulletin Solutions Inc.