possible improvements beyond XviD 1.1 final [thread split]

[DeathTheSheep]

not everyone uses XviD for DVD rips. I use XviD for encoding my webcam movies for instance. And I only have an 18 GB hard drive, with a slow cpu. Not everyone has the latest stuff.

You have a point there. This is why many people can't simply switch to x264 for low-br encodes or for DVD-player compatibility.

In fact, many people's devices can't even handle Qpel/GMC. I've noticed DivX's Insane Mode disables Qpel/GMC while bolstering quality. Perhaps a similar "insane mode" can be implemented for XviD--perhaps not in the sense that I described earlier, but instead as quality increases targeted to these people who don't happen to have the latest stuff (like myself, for instance ;)). Perhaps to increase DivX compatibility, it might be a good idea for these minor quality increases to target such an audience.

[Elias]

You have a point there. This is why many people can't simply switch to x264 for low-br encodes or for DVD-player compatibility.

In fact, many people's devices can't even handle Qpel/GMC. I've noticed DivX's Insane Mode disables Qpel/GMC while bolstering quality. Perhaps a similar "insane mode" can be implemented for XviD--perhaps not in the sense that I described earlier, but instead as quality increases targeted to these people who don't happen to have the latest stuff (like myself, for instance ;)). Perhaps to increase DivX compatibility, it might be a good idea for these minor quality increases to target such an audience.Yes. I have friends who still have 300 mhz computers, good luck trying to get them play AVC on that.

[Teegedeck]

You have a point there. This is why many people can't simply switch to x264 for low-br encodes or for DVD-player compatibility.

In fact, many people's devices can't even handle Qpel/GMC. I've noticed DivX's Insane Mode disables Qpel/GMC while bolstering quality. Perhaps a similar "insane mode" can be implemented for XviD--perhaps not in the sense that I described earlier, but instead as quality increases targeted to these people who don't happen to have the latest stuff (like myself, for instance ;)). Perhaps to increase DivX compatibility, it might be a good idea for these minor quality increases to target such an audience.
The profiles are meant for that.

[IgorC]

Perhaps a similar "insane mode" can be implemented for XviD

Who said you that Xvid hasn't an insane mode? ;)
And let's say RDO (rate distortion optimization). And not "insane" "deadbrain" "craziest" and other labels to indicate quality level to newbies.
Full quality first pass at desired quantizer (in zones) with maxed settings. That would be an insane mode for Xvid. Why do you compare it to divX insane mode?Slow doesn't mean always better quality. Xvid is optimized for better visual quality and also for speed.
You can do 10 passes and the quality will be same as for 3 passes.
I think FFdshow VFW has some RDO + multipass for Xvid but I didn't play much with it.

[sysKin]

Who said you that Xvid hasn't an insane mode? ;)
Insane is easy - add trellis to VHQ :)

[DeathTheSheep]

The profiles are meant for that.

Yes, the profiles force compatibility with certain (DivX-compatible) profiles, I know ;). I was only saying that the targets of proposed quality increases should be aimed at these (compatible) profiles because that's where people without the latest stuff need the quality :).

Sorry 'bout being vague and all, just thought I'd clarify so people won't think I'm quite that unenlightened (still). :D

[Sharktooth]

Insane is easy - add trellis to VHQ :)
!!!!!

[BoNz1]

Hi, I think a good idea to look for improvement after 1.1 is the treatment of dark scenes. It's well know that XviD performs quite poorly on these. I've encoded a lot of movies and from what I can see the problem is threefold:

1) Ratecontrol. It isn't uncommon for these scenes to only get 300-400kbps. Unfortunately, this is not because it overquantizes these scenes. It usually does use quant 2 for P frames and 4 for B frames. Even if it does, the bitrate still goes ridiculously low. What I have been doing lately is making zones and setting the quant to 1 or the weight to 2 and allowing the ratecontrol to use quant 1. This seems to help considerably. Also, quant 1 is still pretty cheap for these kind of scenes, the bitrate is still fairly low. So allowing the ratecontrol to preferentially use quant 1 on scenes with very low energy I think is a good idea. We might spend bits on scene that already look good but it shouldn't be that wasteful.

2) Frametype decision. Biggest problem I have noticed so far is that I frames are often only used in these scenes when they have to be ie the max I-frame interval and often what I would think should be a clear scenechange is missed. When I make a zone and create more I-frames in the scene this boosts up the quality of the surrounding frames. It shouldn't be any surprise. Again this is a pretty cheap way to increase the quality of these scenes. I haven't really looked at P vs B frame decision in these scenes but I imagine that this too could be improved.

3) Skip decision. Is always a problem with flat blocks. It has improved greatly in 1.1 but it's still very problematic in these scenes. Blocks in the background that move a tiny bit due to all the noise in the source become very jumpy in the final encode. I guess it's hard to distinguish noise from real motion. But we gotta find some way to keep them stable.

[DeathTheSheep]

More possible improvements:
Trellis for I-Frames, Psychovisual Enhancements, more RD- Decisions, better ME algos (for example umhex)

[Emp3r0r]

Hi, I think a good idea to look for improvement after 1.1 is the treatment of dark scenes. Would be nice


2) Frametype decision.

I think a new setting for for I frame insertion could help improve quality that would allow for a threshold. If your max I frame interval is 240 and the next scene change is at 250 frames from the previous I frame, it doesn't really make much sense to put a keyframe at 240.

[Manao]

I think a new setting for for I frame insertion could help improve quality that would allow for a threshold. If your max I frame interval is 240 and the next scene change is at 250 frames from the previous I frame, it doesn't really make much sense to put a keyframe at 240.Huh ? Why not raise the frame interval to 250 then ? If you don't care about frame interval, then why not even raise it to 1000+ ? Finally, there's no way for the codec to know that a scene change will occur in the next 10 frames ( except in two passes, but right now, since frame type is decided on first pass, that won't work )

[Emp3r0r]

If you don't care about frame interval, then why not even raise it to 1000+ I do currently disable min i frame interval and I only do 2 pass encoding and this seems like a possible way to improve the 2nd pass.

[Teegedeck]

I-frame intervalls high as that make searching a real test of patience. Now if sysKin actually meant that bit about researching whether 2-pass ME (resulting in one additional first-pass) could help quality - and was successful in doing so - we'd stand a good chance that this would pretty much solve the problem of ideal I-frame placement, too...

Come to think of it, if someone would actually work on that, very nice things would be theoretically within reach; like XviD assigning different CQMs per scene, depending on picture complexity, HVS-data or whatever. Ah, there I go, daydreaming again...

[SenorKaffee]

I´m no expert in this, but would it make sense to bundle more matrices and attach them to a certain set of presets? Or do non-standard matrices break compatibility with standalone players?

[Teegedeck]

Yes, it would certainly make sense, and yes, CQMs can break compatibility with some old, rotten standalones but I wouldn't mind. :D

[celtic_druid]

If you care about standalone compatibility then you would use the Home Theatre profiles which means h.263 quants.

[stephanV]

Come to think of it, if someone would actually work on that, very nice things would be theoretically within reach; like XviD assigning different CQMs per scene, depending on picture complexity, HVS-data or whatever. Ah, there I go, daydreaming again...
Do you want XviD to stay MPEG4 compliant or not?

[Elias]

Do you want XviD to stay MPEG4 compliant or not?It's still not 100% MPEG-4 compliant anyway. If I were in charge, I'd remove Packed bitstream and other incompliant features.

[stephanV]

It's still not 100% MPEG-4 compliant anyway. If I were in charge, I'd remove Packed bitstream and other incompliant features.
?

You are not forced to use packed bit stream... and what other features?

[Didée]

Another one of "possible improvements" ... to quote myself: (from here (http://forum.doom9.org/showthread.php?p=742936#post742936))

In particular, one thing I'd like to see is something like "Penalty for Bframe sensitivity in dark scenes" : The usual setup of 2 (or even 3) max. consecutive Bframes at times is counterproductive during dark scenes, regarding perceived quality ... sure many Bframes during dark scenes "compress like hell", but they just don't look good. And usually the max. Bframe usage is reached much earlier in the dark than it is in the light ...
That's at least my experience: in bright scenes, it seems a good thing to have more B-frames - sometimes even pushed with a little more sensitivity.
But during dark or generally low-contrast scenes, to my eyes its much better when at max. 1 consecutive Bframe is used ... but when a max. of 2 or 3 is set up in the configuration, then XviD tends to use just that many in those scenes it thinks they were "easy".

I think this also should be somewhat simple to implement, opposed to Lambda plugins and HVS modelling. What do you think?

[Elias]

?

You are not forced to use packed bit stream... and what other features?The fact that it's enabled by default, means that most XviD users will use it, and thereby produce incompliant MPEG-4 video. Then we have avi (MPEG-4 in avi is not compliant), the hybrid quantizer of H.263 and MPEG. There are some other non compliant features that I don't know of I'm sure. Anyway, XviD is very close to 100% MPEG-4 compliance, but it's still not there all the way.

[Teegedeck]

Do you want XviD to stay MPEG4 compliant or not? Yeah, I guess that there was something in the specs about the matrix needing to be embedded in the VOL-header, and we only have one such per stream AFAIK. But hey, just a daydream ;-), we don't even have anyone who would volunteer to write such a feature.

Edit: Hm? Wasn't 'hybrid' quantization removed a long time ago?

[Manao]

It was called custom quant, and it has been removed before 1.0.

Elias : you're actually complaining about vfw not being mp4 compliant ( because packed bitstream is default only in the GUI ). It's not XviD's fault if you use vfw. Encraw is compliant, so use it ( and encraw doesn't use packedbitstream nor avi

[Elias]

It was called custom quant, and it has been removed before 1.0.

Elias : you're actually complaining about vfw not being mp4 compliant ( because packed bitstream is default only in the GUI ). It's not XviD's fault if you use vfw. Encraw is compliant, so use it ( and encraw doesn't use packedbitstream nor aviThat's good. Why not remove Packed bitstream too? I don't see the point with it.

[stephanV]

The fact that it's enabled by default, means that most XviD users will use it, and thereby produce incompliant MPEG-4 video.
You quite forget that it is a useful for a lot of people too. But the ability to give extra features to ensure people having compatible streams with a large scale of hardware devices doesn't make XviD incompliant, certainly not if the encoder works perfectly fine without that feature.

Then we have avi (MPEG-4 in avi is not compliant),
Compliant with what exactly? And in what way does XviD enforce you to use AVI? You don't have to use the VFW interface, although it makes a lot of things much easier.

the hybrid quantizer of H.263 and MPEG.
Wasn't that removed a while ago? I certainly can't find it anymore...

There are some other non compliant features that I don't know of I'm sure.
:eek:

Elias, please don't let your search for compliance reach to such an extend that everything becomes impractical to use.

[stephanV]

Yeah, I guess that there was something in the specs about the matrix needing to be embedded in the VOL-header, and we only have one such per stream AFAIK. But hey, just a daydream ;-), we don't even have anyone who would volunteer to write such a feature.

In AVI it should be possible though ;)

[Elias]

You quite forget that it is a useful for a lot of people too. But the ability to give extra features to ensure people having compatible streams with a large scale of hardware devices doesn't make XviD incompliant, certainly not if the encoder works perfectly fine without that feature.So you're saying that without Packed bitstream, XviD will not work on DivX certified products? Still though, I don't think it should be enabled by default, because most people who encode to XviD do it in VirtualDub, and most of them don't know that packed bitstream is incompliant. It's quite easy to remove packed bitstream from XviD video streams with various tools, but imho there ought to be a big fat warning next to it if it's going to be enabled by default.Elias, please don't let your search for compliance reach to such an extend that everything becomes impractical to use.In my opinion, it's important with full compliance. I guess that's where we're different.

[stephanV]

So you're saying that without Packed bitstream, XviD will not work on DivX certified products?
It might work, but there is no guarantee. Since DivX Profiles are introduced in XviD VFW (which is good for users) a while ago, it is quite an important feature to keep in. It also doesn't hurt quality and can be losslessly removed.

Still though, I don't think it should be enabled by default, because most people who encode to XviD do it in VirtualDub, and most of them don't know that packed bitstream is incompliant. t's quite easy to remove packed bitstream from XviD video streams with various tools, but imho there ought to be a big fat warning next to it if it's going to be enabled by default
Most of them that don't know, probably also don't care. But I have no problems with an informative explanation of when and when not to use packed bit stream in a tooltip or something.

In my opinion, it's important full compliance. I guess that's where we're different.
That is certainly true. :)

[SeeMoreDigital]

LOL..... Lets hear it for DivX certification!

As far as I'm aware DivX5.0.1 introduced packed bit-stream with "1B-VOP", when it was launched 09 Apr 2002. DivX certification was launched 19 Sep 2002 (five months later) and only allowed for "1B-VOP".

I seem to remember when DivX first introduced multiple B-VOP encoding for themselves (to one of their DivX5.1.x's versions), it did not include packed bit-stream, broke their compliace and did not work very well in DivX certified devices anyway!

[Manao]

Still though, I don't think it should be enabled by default, because most people who encode to XviD do it in VirtualDub, and most of them don't know that packed bitstream is incompliantAccording to you, encoding with vdub isn't compliant, so why then care about packedbitstream, which concerns only vdub and avi, and which is a better way to put mpeg4 into avi because it's actually more compliant with decoders that supports mpeg4 into avi.

[celtic_druid]

Default for VfW used to be packed bitstream disabled. It was enabled by default to stop all the newbie I get this decoder lag error type posts. Decoding/editing via VfW is much easier with packed bitstream enabled so it makes sense to have it the default setting for VfW encoding.

As said it is just the VfW front end anyway. mencoder also has packed bitstream disabled by default. You can enable it manually or it is enabled automatically with DXN profiles.

Complaining about mp4 specs whilst using avi as a container makes little sense anyway.

[Sharktooth]

As CD said, PB is necessary for editing in VFW (virtualdub, premiere etc). The xvid VFW interface has to have it enabled by default.
for xvid DS encoder it may be optional and xvid CLI (encraw) may not have the PB option at all...

[Elias]

According to you, encoding with vdub isn't compliant, so why then care about packedbitstream, which concerns only vdub and aviEncoding with VDub is compliant if you know what you're doing, i.e. put the XviD avi stream into the mp4 container once VDub is done with the encoding :)which is a better way to put mpeg4 into avi because it's actually more compliant with decoders that supports mpeg4 into avi.No it's not. It's more compatible at the moment. Huge difference. MPEG-4 wasn't standardised for avi.

[Teegedeck]

This is getting increasingly off-topic. If there's call for further discussion about standard-compliance I'm gonna split the latest posts off this thread.

[bond]

According to you, encoding with vdub isn't compliant, so why then care about packedbitstream, which concerns only vdub and avi, and which is a better way to put mpeg4 into avi because it's actually more compliant with decoders that supports mpeg4 into avi.a plain mpeg-4 decoder cant decode packed bitstream without having special support for packed bitstream

therefore packed bitstream can be seen as breaking the mpeg-4 part2 standard

[SeeMoreDigital]

a plain mpeg-4 decoder cant decode packed bitstream without having special support for packed bitstream

therefore packed bitstream can be seen as breaking the mpeg-4 part2 standardI did not know that :eek: