Welcome to Doom9's Forum, THE in-place to be for everyone interested in DVD conversion.

Before you start posting please read the forum rules. By posting to this forum you agree to abide by the rules.

 

Go Back   Doom9's Forum > Video Encoding > New and alternative video codecs

Reply
 
Thread Tools Search this Thread Display Modes
Old 17th July 2023, 14:52   #41  |  Link
ksec
Registered User
 
Join Date: Mar 2020
Posts: 118
And ECM 9.1 is out. https://vcgit.hhi.fraunhofer.de/ecm/ECM/-/tree/ECM-9.1

According to the latest test from JVET Geneva meeting, ECM 9.1 finally achieved *overall* 30% BD-Rate vs VTM 11 in Random Access.
__________________
Previously iwod
ksec is offline   Reply With Quote
Old 18th July 2023, 17:38   #42  |  Link
benwaggoner
Moderator
 
Join Date: Jan 2006
Location: Portland, OR
Posts: 4,783
Quote:
Originally Posted by ksec View Post
And ECM 9.1 is out. https://vcgit.hhi.fraunhofer.de/ecm/ECM/-/tree/ECM-9.1

According to the latest test from JVET Geneva meeting, ECM 9.1 finally achieved *overall* 30% BD-Rate vs VTM 11 in Random Access.
30% is impressive! Hopefully the irreducible decode complexity increase isn't >>2x to get those gains.
__________________
Ben Waggoner
Principal Video Specialist, Amazon Prime Video

My Compression Book
benwaggoner is offline   Reply With Quote
Old 18th July 2023, 17:59   #43  |  Link
nevcairiel
Registered Developer
 
Join Date: Mar 2010
Location: Hamburg/Germany
Posts: 10,349
30% sounds about average, its a similar ballpark as VVC claimed over HEVC at the slower speeds. Much less and adopting a new codec is barely worth it.
__________________
LAV Filters - open source ffmpeg based media splitter and decoders
nevcairiel is offline   Reply With Quote
Old 22nd October 2023, 08:19   #44  |  Link
ksec
Registered User
 
Join Date: Mar 2020
Posts: 118
And ECM 10 is out in the usual place

According to the latest test from JVET meeting, ECM 10 achieved overall 33% BD-Rate vs VTM 11 in Random Access. With lots of improvement going in Low-Delay.

The most impressive thing is we get ~44% of BD-Rate in Text and Graphics in Motion Category. ( The overall result exclude this Category )

Quote:
Originally Posted by benwaggoner View Post
30% is impressive! Hopefully the irreducible decode complexity increase isn't >>2x to get those gains.
Compared to VTM we are looking at 8x increase in Encoding Complexity ( Fair but far from perfect ) and 8x increase in decoding. ( oouh )

I am not entirely sure how they intend to achieve 60% BD-Rate ( Yes 60%, not 50% as they are usually aiming at ). We might have hit the end of the S curve in terms of how we could further compress video.
__________________
Previously iwod

Last edited by ksec; 22nd October 2023 at 08:23.
ksec is offline   Reply With Quote
Old 23rd October 2023, 21:23   #45  |  Link
benwaggoner
Moderator
 
Join Date: Jan 2006
Location: Portland, OR
Posts: 4,783
Quote:
Originally Posted by ksec View Post
And ECM 10 is out in the usual place

According to the latest test from JVET meeting, ECM 10 achieved overall 33% BD-Rate vs VTM 11 in Random Access. With lots of improvement going in Low-Delay.
Yeah, we got the same update at the SMPTE Media Technology Summit last week. Quite impressive progress for just three years since standardization

Quote:
The most impressive thing is we get ~44% of BD-Rate in Text and Graphics in Motion Category. ( The overall result exclude this Category )
Oh, that is good. Not having screen content extensions be mandatory in Main in HEVC is one of my biggest frustrations from it. Those would make game streaming a lot better.

Quote:
Compared to VTM we are looking at 8x increase in Encoding Complexity ( Fair but far from perfect ) and 8x increase in decoding. ( oouh )
Yeah, 8x decoder complexity is going to be a hard sell; we've never seen close to that big a jump in successive codecs. Although AV2's ML extensions might mean an atypical jump on the AOM side as well. Certainly the $ and mm^2 for a current-get video decoder are WAY down compared to MPEG-2. Even if we had an 8x jump circa 2026. Moore's Law keeps on giving.

Quote:
I am not entirely sure how they intend to achieve 60% BD-Rate ( Yes 60%, not 50% as they are usually aiming at ). We might have hit the end of the S curve in terms of how we could further compress video.
At least not without some serious ML on the encoder side to make some more advanced transforms feasible to encode. For example, a spherical warp.

Getting a good film grain synthesis feature built-in is probably the biggest lower-hanging fruit left, as so much of the energy of the most difficult to encode content is grain or other temporally and spatially random noise.

Comcast and others are pushing for AOM's AVFG1 proposal, which would allow any codec to trigger the AV1 FGS post filter via a SEI message. AV1 FGS isn't great, but it certainly can be made better with better parameterization (SVT-AV1 sells it way short). The biggest gap is that the grain synthesis is defined relative to display resolution, not source content resolution, so you get cases where grain particle sizes vary on different displays. Since grain is a physical property, a given piece of synthetic grain should take up a specified percentage of the content, so a low resolution screen should show proportionally smaller grain.
__________________
Ben Waggoner
Principal Video Specialist, Amazon Prime Video

My Compression Book
benwaggoner is offline   Reply With Quote
Old 10th February 2024, 15:54   #46  |  Link
ksec
Registered User
 
Join Date: Mar 2020
Posts: 118
ECM 11 is out, tiny improvement made on top of EMC 10, very small encoding and decoding efficiency improvement. Looking at possibly merging work on Neural Network to further push down BD-Rate.
__________________
Previously iwod
ksec is offline   Reply With Quote
Old 10th February 2024, 20:10   #47  |  Link
Tommy Carrot
Registered User
 
Tommy Carrot's Avatar
 
Join Date: Mar 2002
Posts: 863
Anyone care to share a Windows build of this encoder? I'm very curious how it performs.
Tommy Carrot is offline   Reply With Quote
Old 11th February 2024, 23:30   #48  |  Link
Jamaika
Registered User
 
Join Date: Jul 2015
Posts: 709
https://www.sendspace.com/file/ymwwdd
Jamaika is offline   Reply With Quote
Old 12th February 2024, 16:22   #49  |  Link
Tommy Carrot
Registered User
 
Tommy Carrot's Avatar
 
Join Date: Mar 2002
Posts: 863
Thank you very much!

In random access mode unfortunately it crashes at the 2nd frame every time, but i could get it work in allintra mode, so i can at least examine how does it perform for still image compression.
Tommy Carrot is offline   Reply With Quote
Old 12th February 2024, 18:44   #50  |  Link
Jamaika
Registered User
 
Join Date: Jul 2015
Posts: 709
I didn't check codecs. I created encoder only in AVX. I had no time.
Jamaika is offline   Reply With Quote
Old 13th February 2024, 10:15   #51  |  Link
ksec
Registered User
 
Join Date: Mar 2020
Posts: 118
Quote:
Originally Posted by Tommy Carrot View Post
Thank you very much!

In random access mode unfortunately it crashes at the 2nd frame every time, but i could get it work in allintra mode, so i can at least examine how does it perform for still image compression.
Last time I checked on it ( ECM4 I think in late 2022 or early 2023? ) It was ridiculously good. To the point I think we might as well forget VVC and move forward with FVC / H.267.
__________________
Previously iwod
ksec is offline   Reply With Quote
Old 13th February 2024, 11:17   #52  |  Link
hajj_3
Registered User
 
Join Date: Mar 2004
Posts: 1,127
ecm paper: https://arxiv.org/pdf/2401.02145.pdf
hajj_3 is offline   Reply With Quote
Old 13th February 2024, 12:38   #53  |  Link
Tommy Carrot
Registered User
 
Tommy Carrot's Avatar
 
Join Date: Mar 2002
Posts: 863
Quote:
Originally Posted by Jamaika View Post
I didn't check codecs. I created encoder only in AVX. I had no time.
No worries mate, i'm grateful for your build anyway.
Quote:
Originally Posted by ksec View Post
Last time I checked on it ( ECM4 I think in late 2022 or early 2023? ) It was ridiculously good. To the point I think we might as well forget VVC and move forward with FVC / H.267.
Well, i could not test random access mode yet, but allintra is definitely very promising, still image quality is very noticably better than in VVC, and compared to HEIF, it's a completely different class.
Tommy Carrot is offline   Reply With Quote
Old 13th February 2024, 20:41   #54  |  Link
kurkosdr
Registered User
 
Join Date: Aug 2009
Posts: 315
Quote:
Originally Posted by ksec View Post
Last time I checked on it ( ECM4 I think in late 2022 or early 2023? ) It was ridiculously good. To the point I think we might as well forget VVC and move forward with FVC / H.267.
Personally, I still can't believe technology was able to improve significantly beyond HEVC. I had a university course that covered the basics up to AVC in 2014, I have read about the neat tricks HEVC employs to go beyond AVC, but VVC and beyond are a mystery to me. HEVC was already supposed at the limits of entropy and the limits of post-processing.

Also, are the gains for ECM consistent across the resolution range? I mean, AVC can already do 720x576p25 at 2mbps with decent quality, so does this mean that with ECM we'll be able to get 2*0.6*0.6*0.7= 0.5Mbps SD video with decent quality?

Last edited by kurkosdr; 13th February 2024 at 20:51.
kurkosdr is offline   Reply With Quote
Old 14th February 2024, 20:04   #55  |  Link
benwaggoner
Moderator
 
Join Date: Jan 2006
Location: Portland, OR
Posts: 4,783
Quote:
Originally Posted by ksec View Post
Last time I checked on it ( ECM4 I think in late 2022 or early 2023? ) It was ridiculously good. To the point I think we might as well forget VVC and move forward with FVC / H.267.
VVC is a completed standard starting to go into silicon. H.267 may be the best codec to use in 2030, but I'd rather not have to stick to HEVC until then.
__________________
Ben Waggoner
Principal Video Specialist, Amazon Prime Video

My Compression Book
benwaggoner is offline   Reply With Quote
Old 14th February 2024, 20:14   #56  |  Link
benwaggoner
Moderator
 
Join Date: Jan 2006
Location: Portland, OR
Posts: 4,783
Quote:
Originally Posted by kurkosdr View Post
Personally, I still can't believe technology was able to improve significantly beyond HEVC. I had a university course that covered the basics up to AVC in 2014, I have read about the neat tricks HEVC employs to go beyond AVC, but VVC and beyond are a mystery to me. HEVC was already supposed at the limits of entropy and the limits of post-processing.
Whomever said that had a limited imagination! There are tons of encoding techniques that haven't been implemented into a codec yet. Complex 3D warping. More elaborate forms of texture synthesis than just film grain.

A speculative codec from 25 years now could be a ML kernel that turns into an optimized entropy decoder for other ML kernels that synthesize a movie (including actors, acting, lighting, sets, dialog), with residuals as other ML kernels that correct the first approximation. A future codec could be a superset of H.267, Unreal Engine, ChatGPT, and three other things we've never considered. A video codec is just a stream of bits that turn into a sequence of images.

Quote:
Also, are the gains for ECM consistent across the resolution range? I mean, AVC can already do 720x576p25 at 2mbps with decent quality, so does this mean that with ECM we'll be able to get 2*0.6*0.6*0.7= 0.5Mbps SD video with decent quality?
Older codecs weren't as well optimized for higher resolutions, so you'll typically see bigger gains there. AVC High Profile, which added 8x8 blocks was to make HD encoding more efficient. AVC Main was originally tuned much more for SD. HEVC can do 32x32 texture units, which is a whole lot better. VVC has some advantages over HEVC for 8K. Reference encoders are SLOW, so there has been a natural tendency to focus on lower resolutions as experiments can be run much faster. 4K is 24x the pixels of 720x480!

We're likely past the point of diminishing returns for improving higher resolutions more than lower ones. For moving images, it's really hard to resolve more than 4K anyway, and pretty much impossible for 24p with 1/48th sec motion blur.
__________________
Ben Waggoner
Principal Video Specialist, Amazon Prime Video

My Compression Book
benwaggoner is offline   Reply With Quote
Old 15th February 2024, 17:24   #57  |  Link
kurkosdr
Registered User
 
Join Date: Aug 2009
Posts: 315
Quote:
Originally Posted by benwaggoner View Post
Whomever said that had a limited imagination! There are tons of encoding techniques that haven't been implemented into a codec yet. Complex 3D warping. More elaborate forms of texture synthesis than just film grain.

A speculative codec from 25 years now could be a ML kernel that turns into an optimized entropy decoder for other ML kernels that synthesize a movie (including actors, acting, lighting, sets, dialog), with residuals as other ML kernels that correct the first approximation. A future codec could be a superset of H.267, Unreal Engine, ChatGPT, and three other things we've never considered. A video codec is just a stream of bits that turn into a sequence of images.
Personally, I'd never use an ML encoder, since it can create things that aren't there. I've made my peace with the concept of lossy compression by considering it a form of clever downscaling in the frequency domain (the weights of the higher frequencies are recorded with less accuracy, that's what the whole "integer-divide the DCT'ed block by the quantization table" does). But ML could create things that were never there. For example, how such a video would be admissible as evidence, and how can we claim any kind of creative intent is maintained? And let's be real, in the real world no residual will be recorded in the name of bitrate reduction, much like both AVC and HEVC are typically pushed to their limits by content providers/broadcasters. But imagine instead of loss of detail and blockiness you get rogue details.

Anyway, back on topic, any good info on what ECM does to achieve its compression gains over VVC (and VVC over HEVC)?

Last edited by kurkosdr; 15th February 2024 at 17:32.
kurkosdr is offline   Reply With Quote
Old 15th February 2024, 20:16   #58  |  Link
benwaggoner
Moderator
 
Join Date: Jan 2006
Location: Portland, OR
Posts: 4,783
Quote:
Originally Posted by kurkosdr View Post
Personally, I'd never use an ML encoder, since it can create things that aren't there. I've made my peace with the concept of lossy compression by considering it a form of clever downscaling in the frequency domain (the weights of the higher frequencies are recorded with less accuracy, that's what the whole "integer-divide the DCT'ed block by the quantization table" does).
Good description of pretty much all important codecs from JPEG on. It's really down to a perceptually optimized multidimensional scaling at heart.

Quote:
But ML could create things that were never there. For example, how such a video would be admissible as evidence, and how can we claim any kind of creative intent is maintained? And let's be real, in the real world no residual will be recorded in the name of bitrate reduction, much like both AVC and HEVC are typically pushed to their limits by content providers/broadcasters. But imagine instead of loss of detail and blockiness you get rogue details.
Yeah, "hallucinations" become an increasingly big risk with ML. And even with non-ML techniques to some degree, as compression ratios get more and more complex. Film Grain Synthesis can create a different grain texture. Complex error concealment, deranging, or deblocking can interpolate detail that wasn't there originally instead of just losing detail.

In general principle, the more advanced compression gets, the more plausible the results of a bitstream error will be, as everything but entropy gets squeezed out. Bits spent that can indicate a later bit is wrong is a failure of arithmetic encoding.

One way around this would be deterministic ML - if we have models that always respond the same way to the same input, we can ensure that pixels we see in QA are the same pixels that will be seen anywhere. We can think about this like the transition from MPEG-2 using floating point to iDCT in modern codecs. We don't need future codecs to be compatible with general purpose ML engines for decode; we can continue to specify particular behavior in decoders.
__________________
Ben Waggoner
Principal Video Specialist, Amazon Prime Video

My Compression Book
benwaggoner is offline   Reply With Quote
Old 18th February 2024, 06:48   #59  |  Link
ksec
Registered User
 
Join Date: Mar 2020
Posts: 118
Quote:
Originally Posted by benwaggoner View Post
VVC is a completed standard starting to go into silicon. H.267 may be the best codec to use in 2030, but I'd rather not have to stick to HEVC until then.
VVC 1.0 was standardised in June 2020 at the time it was VTM 9. But in reality it was 2.0 in 2022 April with all compliance and standard test ready. Which at the time it was VTM 16.

I think FVC / ECM 11 right now is much closer to VTM 9 / VVC 1.0 stage. And in terms of Next Gen Codec, this is by far the earliest reach by historic standard. Part of me think that is due to COVID and researcher got insane productivity while being stuck at home.

I still remember I was impressed by VVC during VTM development, well turns out we squeeze another ~30% BD-Rate out of it within 2-3 years time.

But on the other hand it will likely be the first codec that requires hardware decoder to work.
__________________
Previously iwod
ksec is offline   Reply With Quote
Old 18th February 2024, 16:42   #60  |  Link
Jamaika
Registered User
 
Join Date: Jul 2015
Posts: 709
Test VVC encoder

vvencapp: VVenC, the Fraunhofer H.266/VVC Encoder, version 1.6.1-c5da1b5 [Windows][GCC 14.0.0][64 bit][SIMD=SSE42]

Encoder always converts videos to 10bit. Currently frame rate isn't read in libavcodec.

VVEncoderApp_avx.exe -i "113.yuv" -o "output_vvenc_10bit.vvc" -v 5 -t 4 -s 1280x720 --fps 3000/1001 -c yuv420 -ip 256 --internal-bitdepth 10 --bitrate 3Mbps -f 200 --passes 2 --preset medium --level auto --tier main

mp4box.exe -new -add output_vvenc_10bit.vvc output_vvenc_10bit.mp4
Track Importing VVC - Width 1280 Height 720 FPS 3000/1001
VVC Import results: 200 samples (377 NALUs) - Slices: 2 I 0 P 198 B - 0 SEI - 1 IDR - 0 CRA
VVC Stream uses forward prediction - stream CTS offset: 31 frames
Code:
[vvc @ 0000019cf30e4ef0] Intra Block Copy is not implemented. Update your FFmpeg version to the newest one from Git. If the problem still occurs, it means that your file has a feature which has not been implemented.
[vvc @ 0000019cf30e4ef0] frame     3, P(  0,   0) failed with -1163346256
[vvc @ 0000019cf30e4ef0] Intra Block Copy is not implemented. Update your FFmpeg version to the newest one from Git. If the problem still occurs, it means that your file has a feature which has not been implemented.
[vvc @ 0000019cf30e4ef0] frame     4, P(  5,   0) failed with -1163346256
[vvc @ 0000019cf30e4ef0] Intra Block Copy is not implemented. Update your FFmpeg version to the newest one from Git. If the problem still occurs, it means that your file has a feature which has not been implemented.
[vvc @ 0000019cf30e4ef0] Intra Block Copy is not implemented. Update your FFmpeg version to the newest one from Git. If the problem still occurs, it means that your file has a feature which has not been implemented.
[vvc @ 0000019cf30e4ef0] frame     6, P(  7,   0) failed with -1163346256
[vvc @ 0000019cf30e4ef0] Intra Block Copy is not implemented. Update your FFmpeg version to the newest one from Git. If the problem still occurs, it means that your file has a feature which has not been implemented.
[vvc @ 0000019cf30e4ef0] frame     8, P(  0,   1) failed with -1163346256
[vvc @ 0000019cf30e4ef0] frame     5, P(  2,   0) failed with -1163346256
VVCSoftware: VTM Encoder Version 23.1-b4dd0e7 [Windows][GCC 14.0.0][64 bit] [SIMD=AVX]

ffvvc doesn't like VTM. Video doesn't play well.

VTMEncoderApp_avx.exe --SummaryVerboseness -c "encoder_randomaccess_vtm.cfg" --InputFile=114.yuv --BitstreamFile=output_vtm_10bit.vvc --SourceWidth=1280 --SourceHeight=720 --FrameRate=29.970 --InputBitDepth=10 --InternalBitDepth=10 --OutputBitDepth=10 --MSBExtendedBitDepth=10 --InputChromaFormat=420 --ChromaFormatIDC=420 --ConformanceWindowMode=1 --FramesToBeEncoded=200 --MatrixCoefficients=1 --InputColorPrimaries=-1 --LMCSSignalType=0 --Level=4 --BDPCM=1 --Tier=main --HashME=1 --IBC=1 --MaxCUWidth=16 --MaxCUHeight=16 --CTUSize=32 --QP=32 --RateControl=1 --TargetBitrate=3000000 --MaxBTLumaISlice=32 --MaxBTChromaISlice=32 --MaxBTNonISlice=32 --MaxTTLumaISlice=32 --MaxTTChromaISlice=32 --MaxTTNonISlice=32 --ColorTransform=0 --VideoFullRange=0 --InputSampleRange=0 --AspectRatioInfoPresent=1 --ChromaLocInfoPresent=1 --OverscanInfoPresent=1 --Log2MaxTbSize=5 --VirtualBoundariesPresentInSPSFlag=1 --EnableDecodingCapabilityInformation=1 --DecodingRefreshType=1 --HrdParametersPresent=0

mp4box.exe -new -add output_vtm_10bit.vvc output_vvenc_10bit.mp4
Track Importing VVC - Width 1280 Height 720 FPS 25000/1000
OpenGOP detected - adjusting file brand
VVC Import results: 48 samples (95 NALUs) - Slices: 3 I 0 P 45 B - 0 SEI - 1 IDR - 2 CRA
VVC Stream uses forward prediction - stream CTS offset: 31 frames

VVCSoftware: ECM Encoder Version 11.0-ce78934 (VTM-10.0-45dfe06) [Windows][GCC 14.0.0][64 bit] [SIMD=AVX]

ECM isn't probably VVC codec. Non-standard.

ECMEncoderApp_avx.exe --SummaryVerboseness -c "encoder_randomaccess_ecm.cfg" --InputFile=114.yuv --BitstreamFile=output_ecm_10bit.vvc --SourceWidth=1280 --SourceHeight=720 --FrameRate=29.970 --InputBitDepth=10 --InternalBitDepth=10 --OutputBitDepth=10 --MSBExtendedBitDepth=10 --InputChromaFormat=420 --ChromaFormatIDC=420 --ConformanceWindowMode=1 --FramesToBeEncoded=200 --MatrixCoefficients=1 --InputColorPrimaries=-1 --LMCSSignalType=0 --Level=4 --BDPCM=1 --Tier=main --HashME=1 --IBC=1 --MaxCUWidth=16 --MaxCUHeight=16 --CTUSize=32 --QP=32 --RateControl=1 --TargetBitrate=3000000 --MaxBTLumaISlice=32 --MaxBTChromaISlice=32 --MaxBTNonISlice=32 --MaxTTLumaISlice=32 --MaxTTChromaISlice=32 --MaxTTNonISlice=32 --ColorTransform=0 --VideoFullRange=0 --InputSampleRange=0 --AspectRatioInfoPresent=1 --ChromaLocInfoPresent=1 --OverscanInfoPresent=1 --Log2MaxTbSize=5 --VirtualBoundariesPresentInSPSFlag=1 --EnableDecodingCapabilityInformation=1 --DecodingRefreshType=1 --HrdParametersPresent=0

mp4box.exe -new -add output_ecm_10bit.vvc output_ecm_10bit.mp4
[VVC] wrong num tile columns 207 in PPS
[VVC] Error parsing NAL unit type 16
[VVC] Error parsing Picture Param Set

Code:
[vvc @ 000001692f8cc2b0] sps_delta_qp_in_val_minus1[i][j] out of range: 610, but must be in [0,255].
[vvc @ 000001692f8cc2b0] Failed to read unit 1 (type 15).
[vvc @ 000001692f8cc2b0] Failed to parse picture unit.
[vvc @ 000001692f8c1760] Could not find codec parameters for stream 0 (Video: vvc, none): unspecified size
Consider increasing the value for the 'analyzeduration' (0) and 'probesize' (5000000) options
https://www.sendspace.com/file/ar9enm

Last edited by Jamaika; 18th February 2024 at 17:36.
Jamaika is offline   Reply With Quote
Reply

Tags
iso, itu, successor, vvc

Thread Tools Search this Thread
Search this Thread:

Advanced Search
Display Modes

Posting Rules
You may not post new threads
You may not post replies
You may not post attachments
You may not edit your posts

BB code is On
Smilies are On
[IMG] code is On
HTML code is Off

Forum Jump


All times are GMT +1. The time now is 03:29.


Powered by vBulletin® Version 3.8.11
Copyright ©2000 - 2024, vBulletin Solutions Inc.