Is XVID still used?

[rwill]

Originally Posted by kurkosdr

1. Don't care.
2. As previously said, the problem with MPEG-2 is that, in its most common form (DVD-Video), it forces you to use 720x756/480 resolution if you want widescreen. This means you are looking at a 4GB file at minimum for an ordinary movie.

Isnt most Divx/Xvid content on the internet derived from DVD ? I remember it was in the year ~2000. That just came about with people starting to re-encoding DVDs with the hacked Divx 3.11 Alpha codec in resolutions like 640x360. People could have used some Mpeg-2 encoder too but there was no one available for free I guess. MP3 started off the same, with some hacked Fraunhofer encoder and Napster. Ah, piracy... "Sure it looks bad, but at least it's small". Mpeg-2 video could have done this as well, SVCD being one popular standard.

[kurkosdr]

Originally Posted by SeeMoreDigital

Actually you are wrong.

In the case of MPEG-2 DVD or DTV the 720x756/480 or 'D1' pixel frame size is always encoded along with either 4:3 or 16:9 DAR (ie: aspect ratio signalling). It is never distributed at 1:1...

Yes, 720x756/480 is always anamorphic (either for 16:9 or for 4:3), I never questioned that.

The real issue is that, according to the DVD-Video specs, only the 720x756/480 resolution can have the 16:9 flag, everything else (for example 352x288/240) can only be flagged as 4:3. Which is the real problem. If your content is 16:9, you have to use the 720x756/480 resolution (if you use the DVD-Video format), which means you have to use higher bitrates, which means you have to use at least 4GB for a half-decent picture.

So, where was I wrong? As I said in my previous post, DVD-Video forces you to use the 720x756/480 resolution if you want widescreen (16:9).

Originally Posted by rwill

"Sure it looks bad, but at least it's small". Mpeg-2 video could have done this as well, SVCD being one popular standard.

No, it couldn't. And it's not for lack of trying by various people in the past.

First of all, SVCD doesn't do 16:9 anamorphic. Not reliably at least, aka with the players capable of signaling it properly (for 16:9 TVs) or putting black bars (for 4:3 TVs). So, you have to letterbox during authoring and manually crop/zoom in the TV's widescreen settings when viewing on a widescreen TV. DVD-Video only does widescreen at 720x756/480, which requires high bitrates (for the standards of the day, at least). And that's before we take into account the fact MPEG2 degrades much worse than MPEG4 ASP.

Basically, it all comes down to this: Have you tried squeezing a 2-hour movie on an SVCD? Or even on 2 SVCDs? It's unwatchable, even on a CRT television. You have to go to 4GB (DVD) at minimum, which was considered a huge filesize back then. Meanwhile, a 2-hour movie on 2CDs with MPEG4 ASP at 640x360 offered acceptable quality (for the standards of the day, at least), and even 1CD was considered watchable.

Now, why is MPEG4 ASP used today? The answer is it's pretty small (700MB or 1400MB) and plays on pre-H.264 players. Nobody will download a 4GB (or 4.38GB) MPEG2 file for their car DVD player or for their non-HD TV-DVD combo in the kitchen. At least that's what I get by looking at the availability online. And for their big screen TVs, they will download the H.264 version

[j7n]

None of the existing MPEG-2 encoders that I'm aware of would encode to long GOPs, and would enforce a strict pattern of P/B frames that sometimes didn't even respect a scene change, leading to low quality. For playback on a computer you could always encode to square pixels, same as XviD, no?

[rwill]

Originally Posted by kurkosdr

...

Damn.

First the necro and then the borderline insane way to present random fact snippets as if they support your argument... and being dense.

Are you Ballings Twin ?

[FranceBB]

Originally Posted by kurkosdr

So, where was I wrong?

Here:

Originally Posted by kurkosdr

DVD-Video forces you to use the 720x756/480 resolution

720x576 is PAL anamorphic
720x480 is NTSC anamorphic
both can be flagged either 4:3 or 16:9 and it's gonna be the player that will re-scale them on the fly.

Originally Posted by kurkosdr

Now, why is MPEG4 ASP used today?

'cause people don't wanna throw away their 2002-era players and that's just sad...

Originally Posted by j7n

None of the existing MPEG-2 encoders that I'm aware of would encode to long GOPs, and would enforce a strict pattern of P/B frames that sometimes didn't even respect a scene change, leading to low quality.

The state of open source MPEG-2 encoders is just sad...

The MPEG-2 Libavcodec encoder allows you to set an arbitrary GOP, however by default it assigns waaaaaaaay too many bits to the Intra, thus bit-starving P and B and the result is just... very poor.

x262 doesn't support all interlaced chroma sampling modes as it was a work-in-progress encoder and unfortunately it stayed that way 'cause it has been abandoned eons ago, so... nope (and I know 'cause I begged the creator to keep going back then).

[kurkosdr]

Originally Posted by FranceBB

Here:
720x576 is PAL anamorphic
720x480 is NTSC anamorphic
both can be flagged either 4:3 or 16:9 and it's gonna be the player that will re-scale them on the fly.

I mean 576, sorry. Typo that got copied pasted all over. My point stays the same. DVD-Video forces you to use full D1 resolution if you want anamorphic 16:9, which, together with the fact MPEG-2 degrades much worse than MPEG4 ASP, means you are looking at a 4GB file minimum, which is more than what people are willing to download for SD.

Originally Posted by FranceBB

'cause people don't wanna throw away their 2002-era players and that's just sad...

Yeah, but nothing you or me can do about it, right? Also, things like car DVD players can't be easily upgraded.

[FranceBB]

Originally Posted by kurkosdr

things like car DVD players can't be easily upgraded.

As a broke person trying to save money, wishing to get on the property ladder and become a home owner one day, I drive a poor-people car, so a 2009 Nissan Micra where everything is analog and I had to buy a bluetooth to jack adapter to even use my phone... eheheheheh

Anyway, my father's car, a Mitsubishi, does indeed have a DVD player which reads MPEG-2 with either MP2 or AC3 audio, but my parents have never ever used it 'cause due to silly Italian laws they're not allowed to use it while driving as it might distract the driver (even if he's not looking at it at all). The Italian firmware made it so that if you're driving, it won't play and it will tell you that the playback will resume once the car stops, thus making it completely useless, 'cause let's face it, who in the world would sit in his car, in a car park, to watch a movie on a small screen? xD

Originally Posted by kurkosdr

DVD-Video forces you to use full D1 resolution if you want anamorphic 16:9

Uhm... it's actually the other way round: DVD Forces you to use full resolution if you want anamorphic 4:3.

DVD 720x480 flagged as 4:3 will be rescaled to 640x480
DVD 720x480 flagged as 16:9 will be rescaled to 848x480

with those being MPEG-2.
Now, if you re-encode in xvid, you can toss away the ugly anamorphic thing and re-encode to a proper 1.77 (so 16:9) or 1.33 (so 4:3) and your argument stands, but for 4:3 'cause 640x480 4:3 1.33 is smaller than 720x480 anamorphic flagged 4:3, so you save space, but it doesn't apply for 16:9 'cause 848x480 16:9 1.77 is larger than 720x480 anamorphic :P

(I don't wanna be pedantic, I got what you're trying to say, but you know this forum is read by thousands of people every day, so I like to clarify things for people who will read it in the future ).

Originally Posted by kurkosdr

MPEG-2 degrades much worse than MPEG4 ASP

True, in general, if you compare apples with apples.
In the case of DVDs, though, you have MPEG-2 25i or 30i which are generally deinterlaced before being re-encoded to xvid. Even though xvid is better than MPEG-2, we should note that encoding an interlaced source takes less bitrate than encoding a progressive source, if the same codec is used. In other words, if I were to use MPEG-2 to encode the same source, but one time progressive and one time interlaced, the latter would take less bitrate.
Still, given that xvid is better than MPEG-2, it will still be able to yield an advantage even if it's progressive.

Originally Posted by kurkosdr

you are looking at a 4GB file minimum, which is more than what people are willing to download for SD.

"De Gustibus"
(I think it's widely used across the world, but if you don't know it's an ancient latin sentence used during the Roman Empire and you can find the meaning on Wikipedia)

[kurkosdr]

Nobody encodes Divx avi in 848x480 and most pre-H.264 devices out there won't even play it (because it's beyond the maximum resolution allowed by the "Divx Home Theater" profile, which is what those devices implement). Most widescreen Divx AVI files are 640x360, which is substantially less pixels than the 720x480 minimum resolution that DVD-Video mandates if you want to have anamorphic widescreen. Coupled with the fact MPEG4 ASP degrades less bad than MPEG2, it allows for substantial bitrate savings (for example, an entire movie on 1400MB or 700MB), without the sea of artifacts you would encounter if you targeted such filesizes on DVD-Video.

BTW, don't get me wrong, I wish more stuff was made available as 4GB, 4.38GB, or even 7.96GB DVD-video for people with old devices, but that's not what I am seeing around. Most stuff out here is either Divx avi (700MB or 1400MB) or H.264 (MKV/MP4).

[rwill]

Here are some Tears of Steel 720x300 encodes:

https://drive.google.com/drive/folde...UW?usp=sharing

Mpeg2 and Mpeg4 look pretty equal to me.

[FranceBB]

Originally Posted by kurkosdr

Nobody encodes Divx avi in 848x480 and most pre-H.264 devices out there won't even play it (because it's beyond the maximum resolution allowed by the "Divx Home Theater" profile, which is what those devices implement).

Right! I see! I forgot!
I took a look and yeah, I saw old ancient SD 4:3 encodes as 640x480 and my oooooooooooold SD 16:9 xvid encodes as 704x396 so that they were gonna stay within the profile constraint. To be fair, I should have remembered it given that I was one of the people who encoded in 640x480 and 704x396 in 2006-2007 for the Italian """branch""" of ADC-Elites (which then turned into OPF-Italia), but I forgot... it's been ages ago...

[kurkosdr]

Originally Posted by rwill

Here are some Tears of Steel 720x300 encodes:

https://drive.google.com/drive/folde...UW?usp=sharing

Mpeg2 and Mpeg4 look pretty equal to me.

No mention of encoder and encoder settings used for each? All I can see is that the MPEG4 ASP one was done with the XviD encoder but nothing else.

Also, the MPEG4 ASP looks better to me, and there is no mention of PSNR and SSIM either (can't be bothered to track down an uncompressed copy of the clip to do it myself, sorry).

PS: It would seem weird to me that MPEG would go through all the effort of defining MPEG4 ASP and breaking compatibility with MPEG2 without at least some kind of significant improvement in the coding tools offered.

[rwill]

Originally Posted by kurkosdr

No mention of encoder and encoder settings used for each? All I can see is that the MPEG4 ASP one was done with the XviD encoder but nothing else.

Also, the MPEG4 ASP looks better to me, and there is no mention of PSNR and SSIM either (can't be bothered to track down an uncompressed copy of the clip to do it myself, sorry).

PS: It would seem weird to me that MPEG would go through all the effort of defining MPEG4 ASP and breaking compatibility with MPEG2 without at least some kind of significant improvement in the coding tools offered.

In case you wonder why it looks crap, it does not look that bad on a CRT. Going through my archive, things looked even worse back then in ~2002.

Encoder and configuration used, of course I used the best Mpeg2 and Mpeg4 encoder I had available with the best settings I could come up with.

That says it all I think.

...
...
...
...



Ok maybe not...

I used xvid_encraw which reports

xvidcore build version: xvid-1.3.7
Bitstream version: 1.3.7

And y262 in the 08/15/2022 git version.

Configuration was:

./xvid_encraw.exe -i tos_720x300_8b.yuv -type 0 -csp i420 -w 720 -h 300 -framerate 24.0 -bitrate 900 -pass1 -full1pass -max_key_interval 300 -quality 6 -vhqmode 4 -bvhq -masking 2
./xvid_encraw.exe -i tos_720x300_8b.yuv -type 0 -csp i420 -w 720 -h 300 -framerate 24.0 -bitrate 900 -pass2            -max_key_interval 300 -quality 6 -vhqmode 4 -bvhq -masking 2 -o mpeg4.m4v

./y262.exe -in tos_720x300_8b.yuv -size 720 300 -threads 1 2 -profile main -level high -chromaf 420 -rcmode 1                -mpout stats.p1 -bitrate 900 -vbvrate 2000 -vbv 600 -quant 3 -quality 100 -frcode 2 -arinfo 1 -nump 18 -numb 2 -flatmat -videoformat 709
./y262.exe -in tos_720x300_8b.yuv -size 720 300 -threads 1 2 -profile main -level high -chromaf 420 -rcmode 2 -mpin stats.p1 -mpout stats.p2 -bitrate 900 -vbvrate 2000 -vbv 600 -quant 3 -quality 100 -frcode 2 -arinfo 1 -nump 18 -numb 2 -flatmat -videoformat 709
./y262.exe -in tos_720x300_8b.yuv -size 720 300 -threads 1 2 -profile main -level high -chromaf 420 -rcmode 2 -mpin stats.p2 -mpout stats.p3 -bitrate 900 -vbvrate 2000 -vbv 600 -quant 3 -quality 100 -frcode 2 -arinfo 1 -nump 18 -numb 2 -flatmat -videoformat 709 -out mpeg2.m2v

The tos_720x300_8b.yuv was derived from the "Ben Waggoner HEVC encoding challenge" ToS source in the HEVC subforum.
Its scaling was done with some ffmpeg version with such a filter: "-filter:v scale=720x300".

*edit*
Its also telling that you make broad claims about the quality of 'Mpeg2' and 'Mpeg4' without mentioning encoder and configuration as well but complain about them missing when others disagree with your claims.

Oh and the Mpeg2 video looks better to me.

[Katie Boundary]

Originally Posted by kurkosdr

Basically, it all comes down to this: Have you tried squeezing a 2-hour movie on an SVCD? Or even on 2 SVCDs? It's unwatchable, even on a CRT television. You have to go to 4GB (DVD) at minimum, which was considered a huge filesize back then. Meanwhile, a 2-hour movie on 2CDs with MPEG4 ASP at 640x360 offered acceptable quality (for the standards of the day, at least), and even 1CD was considered watchable.

"Watchable"? 700 MB per 2-hour movie was the standard. Only as movies approached 2.5 hours did you start to see the 2-CD encodes become more common.

[outhud]

Originally Posted by rwill

Isnt most Divx/Xvid content on the internet derived from DVD ? I remember it was in the year ~2000. That just came about with people starting to re-encoding DVDs with the hacked Divx 3.11 Alpha codec in resolutions like 640x360. People could have used some Mpeg-2 encoder too but there was no one available for free I guess. MP3 started off the same, with some hacked Fraunhofer encoder and Napster. Ah, piracy... "Sure it looks bad, but at least it's small". Mpeg-2 video could have done this as well, SVCD being one popular standard.

I remember a short lived one called 3ivX also for a while.

[benwaggoner]

Originally Posted by outhud

I remember a short lived one called 3ivX also for a while.

IIRC, that was another MPEG-4 part 2 implementation.

[benwaggoner]

Originally Posted by kurkosdr

PS: It would seem weird to me that MPEG would go through all the effort of defining MPEG4 ASP and breaking compatibility with MPEG2 without at least some kind of significant improvement in the coding tools offered.

MPEG-4 part 2 was a disappointment. While it was better than MPEG-2, particularly for lower bitrates and resolutions, it wasn't so much better for 480i/576i that broadcasters were motivated to replace the millions of set top boxes to get a ~20% bitrate reduction. Plus there were IP ambiguities that made big bets by big companies seem unwise for the first few years.

Dark Shikari had a great insightful post about the design defects in p2 ASP somewhere I can't find. A couple issues I recall is that the requirement that adaptive quantization go up and down by even numbers relative to a previously determined frame QP was a challenge and annoying to optimize. And Global Motion Compensation could really only save something like 1 bit per frame. Hopefully someone else can dredge it up.

The broadcast market back then is what sold encoders, and therefore drove encoder development. MPEG imagined a big competitive ecosystem of vendors trying to squeeze out an extra 1% here and there to win contracts. That's what helped make MPEG-2 as good as it was, and later provided the many years of continuous improvement of H.264 and HEVC.

MPEG-4 Pt 2 never got that kind of love, both due to, and a cause of, its relatively small paying market.

Also, the streaming market back then was dominated by the proprietary QuickTime, Windows Media, and RealVideo. By the time they had had robust Pt 2 support, better codecs were available.

Lastly, as it was pretty obvious that Pt2 was in trouble, H.264 (MPEG-4 Pt 10) development was accelerated and arrived a lot sooner than the typical 10 year gap between major MPEG codecs, had obviously superior compression efficiency, and had a simple licensing regime. That's what broadcasters targeted, and what the encoder vendors all pivoted to competing on. The commercial market for H.264 encoders was quickly 100x bigger than the Pt 2 market ever was, which funded a lot of encoder developers. And then x264 happened with a remarkably brilliant set of developers and a big global audience competing on encoding fast and beautifully and providing a much broader base of feedback and fine tuning than any encoder before it ever got. It's weird to think that traditional codec development back then didn't even look at animation content!

[StainlessS]

Dark Shikari had a great insightful post about the design defects in p2 ASP somewhere

On D9 ?

Perhaps amongst this lot
Google

"Dark Shikari" NEAR ("ASP" AND ("p2" OR "part 2"))  site:forum.doom9.org

https://www.google.co.uk/search?q=%2...client=gws-wiz


amend if you remember better search stuff

EDIT: There is this post by bond (but not on defects), MPEG-4 ASP Information, What is MPEG-4?
https://forum.doom9.org/showthread.php?t=73022

[outhud]

Originally Posted by benwaggoner

MPEG-4 part 2 was a disappointment. While it was better than MPEG-2, particularly for lower bitrates and resolutions, it wasn't so much better for 480i/576i that broadcasters were motivated to replace the millions of set top boxes to get a ~20% bitrate reduction. Plus there were IP ambiguities that made big bets by big companies seem unwise for the first few years.

Dark Shikari had a great insightful post about the design defects in p2 ASP somewhere I can't find. A couple issues I recall is that the requirement that adaptive quantization go up and down by even numbers relative to a previously determined frame QP was a challenge and annoying to optimize. And Global Motion Compensation could really only save something like 1 bit per frame. Hopefully someone else can dredge it up.

The broadcast market back then is what sold encoders, and therefore drove encoder development. MPEG imagined a big competitive ecosystem of vendors trying to squeeze out an extra 1% here and there to win contracts. That's what helped make MPEG-2 as good as it was, and later provided the many years of continuous improvement of H.264 and HEVC.

MPEG-4 Pt 2 never got that kind of love, both due to, and a cause of, its relatively small paying market.

Also, the streaming market back then was dominated by the proprietary QuickTime, Windows Media, and RealVideo. By the time they had had robust Pt 2 support, better codecs were available.

Lastly, as it was pretty obvious that Pt2 was in trouble, H.264 (MPEG-4 Pt 10) development was accelerated and arrived a lot sooner than the typical 10 year gap between major MPEG codecs, had obviously superior compression efficiency, and had a simple licensing regime. That's what broadcasters targeted, and what the encoder vendors all pivoted to competing on. The commercial market for H.264 encoders was quickly 100x bigger than the Pt 2 market ever was, which funded a lot of encoder developers. And then x264 happened with a remarkably brilliant set of developers and a big global audience competing on encoding fast and beautifully and providing a much broader base of feedback and fine tuning than any encoder before it ever got. It's weird to think that traditional codec development back then didn't even look at animation content!

This is a great post on the history of it, thanks!

[FranceBB]

Great piece of history indeed by our Ben.
If there's someone I'd point my finger against (as an "antagonist") in history that would be Sony and their MPEG-2 pursuit 'till the bitter end.
If it wasn't for Sony, MPEG-2 would have been dead in the water, but after the bitter disappointment of MPEG-4 ASP (i.e XVID) in 2001, most broadcasters didn't gamble with MPEG-4 Part 10 (i.e H.264) in 2004 for SD contents (720x480 29,970i - 720x576 25i) and decided to stick with MPEG-2. Back then, formats like IMX50 based on MPEG-2 All Intra 50 Mbit/s 4:2:2 were popular for SD files.
In 2006, as little as 2 years after H.264 was introduced, the world moved to HD (and then FULL HD) and Sony made the XDCAM set of standard which is essentially MPEG-2 for both HD and FULL HD.
Guess what happened? Many companies wanted "stability" and adopted it instead of H.264, but that was a big, big mistake. As result, most broadcasters are still using MPEG-2 for FULL HD contents to this very day and are hog-tied to this ancient, no longer supported, unoptimized codec with banding problems and what not.
If Sony didn't do this, the world would have moved to H.264 for HD and FULL HD and MPEG-2 would have been dead by now.

[excellentswordfight]

Originally Posted by FranceBB

Great piece of history indeed by our Ben.
If there's someone I'd point my finger against (as an "antagonist") in history that would be Sony and their MPEG-2 pursuit 'till the bitter end.
If it wasn't for Sony, MPEG-2 would have been dead in the water, but after the bitter disappointment of MPEG-4 ASP (i.e XVID) in 2001, most broadcasters didn't gamble with MPEG-4 Part 10 (i.e H.264) in 2004 for SD contents (720x480 29,970i - 720x576 25i) and decided to stick with MPEG-2. Back then, formats like IMX50 based on MPEG-2 All Intra 50 Mbit/s 4:2:2 were popular for SD files.
In 2006, as little as 2 years after H.264 was introduced, the world moved to HD (and then FULL HD) and Sony made the XDCAM set of standard which is essentially MPEG-2 for both HD and FULL HD.
Guess what happened? Many companies wanted "stability" and adopted it instead of H.264, but that was a big, big mistake. As result, most broadcasters are still using MPEG-2 for FULL HD contents to this very day and are hog-tied to this ancient, no longer supported, unoptimized codec with banding problems and what not.
If Sony didn't do this, the world would have moved to H.264 for HD and FULL HD and MPEG-2 would have been dead by now.

In XDCAM 50 defense, when it comes to well adopted standards with a broad compliance there is still no HD standard that that does 1080i at the same or lower bitrate that is viable as house format! AVC-I and XAVC-I is 100Mbps, and when re-compressed for contribution at limited bitrates it makes pretty much no difference for the end consumer. The GOP versions of AVC/HEVC is not universally supported in the same way as xdcam and the intra flavors of AVC.

I think its crazy that the "smallest" viable house format for UHD XAVC-I Class300 is 500Mbps (50fps mode for pal regions), thats a 10x increase from xdcam 50! Tbh thats absurd given that mpeg-2 is over two decades old, and we still cant "save" space in the broadcasting world. This makes transitions to things like 1080p and 2160p very expensive to migrate to, so most just stick to 1080i/XDCAM. I tried to explain this to management when calculating on on storage cost for UHD migration, cause they couldnt understand why UHD would take up more than 4x than our old XDCAM format.

It also doesnt help that that hw-decode and encode of 10bit 4:2:2 formats is abysmal. Like it would make so much sense for a AVC/HEVC mezz standard to also have good hw support in gpu-solutions, without the need for expensive purpose built acc-cards. If there were full out support in qsync/nvenc/VCE and implemented in a broad amount of applactions as NLEs etc for fast encode and decode I think it would accelerate adoption alot. It would also lower requirement for realtime ingest a lot.