XviD-1.1.0-Beta1

[Koepi]

Ahoy,

the announcement on xvid.org is still missing but will be there anytime soon - XviD 1.1.0 Beta1 is available.

XviD-1.1.0-Beta1-16012005:
- {core}: Rate-Distortion mode decision for bvops.
- {core}: Two new postprocessing functions, brightness and deringing.
- {core}: VBV support in 2pass mode
- {core}: Overal encoder speedup, especially for b-frames and all vhq modes.
- {core}: Overal speedup of the decoder, merging decoding steps together
in order to save significant memory bandwidth, enabled mmx
qpel support that was misteriously disabled in 1.0.x. All optimizations
could lead to a 25% speedup, and something closer to 12% in average.
- {core}: Fixes to the CBR/ABR rate controller, it should work much better with
bvops now.
- {core}: Fix to the 2pass2 code that could cause doubling the overflow.
- {dshow}: Support for brightness control.
- {vfw}: Many small improvements.

Please head over to www.koepi.org and test the new binary and report errors! :)

Cheers
Koepi

[CruNcher]

Jep finaly :)

[Enigmax]

Santa Claus could bring you an official beta release

:rolleyes:
Koepi = Santa Claus
:) :) :) :) :)

Greetings

[PlazzTT]

Great! Thanks Koepi!

Is the date on that change log above correct? November?

[Koepi]

Oops, of course not. I didn't properly change the numbers in the releasenotes, sorry. Uploading fixed build, should be available now.

Cheers
Koepi

[CruNcher]

Koepi i think you should also credit the changes that aren't Windows Build specific.


+ Brand new PowerPC port by Christoph Naegeli
(MacOSX users may be interested by the Quicktime component,
have a look at http://people.ee.ethz.ch/~naegelic/index.php)

+ Brand new amd64 Linux 64bit port by Andre Werthmann

+ Out of the box yasm support (Required by the amd64 port, see:
http://www.tortall.net/projects/yasm/)

+ More ELF friendly assembly code for the ia32/amd64 ports
(functions are declared as functions, and their size is also written to the resulting object files)

[Koepi]

Well, that's in the sources, ok, but it's not affecting the windows build at all, that's why I left that stuff away. Who's on linux or has a PowerPC or x86-64 will read the source code release notes anyway... :)

Regards
Koepi

[Sharktooth]

This time it is a straight-from-CVS build.
Is there a hope to see the (in)famous strict/loose curve scaling option again in future builds?

[Koepi]

If I find the time I'll try to merge foxer's code again, yes. This can take some time though.
(So do you like strict scaling better than loose scaling?)

Cheers
Koepi

[IgorC]

Celtic D compilations are faster on intel CPU (sse2)than Koepi´s .
will be there a new compilation from Celtic?

P.S. great work!

[Sharktooth]

@Koepi: yep, i'd like to choose loose or strict coz i think they are both usefull, depending on the source material.

@IgorC: yes C_D constantly compiles xvid builds. So it's a matter of days (or maybe hours?) :)

[celtic_druid]

I'll put up some new builds later (been busy today), however the biggest change between this and my last build is the bitstream version, everything else is basically identical other than some cosmetic, header, etc. changes.

[Plub]

Xvid doens't show up in my the list in virtualdub after i installed it.
I can change the propetries via the startmenu, which works, but doens't change anything -_-
//edit: i deinstalled and reinstalled it several times, now it works

[Dams]

Nice work as usual Koepi !

Is this build can be called "stable" as I use only VBV for my home standalone player (Keyplug 4810) , because I don't use any "pro" feature like BF or other things like this ?

Is VBV before this build , worked only on first pass ?? (- {core}: VBV support in 2pass mode )

I use actualy Nic build 13/12/04 compiled 8.1 Intel

Ps : I've got an Duron 1.6@2.2 Ghz

[Leak]

Originally posted by Koepi
XviD 1.1.0 Beta1 is available.
Ahem... w00t! :D

np: Zorn - Nachtbus (The City's Collapsing (But Not Tonight))

[jk888]

qpel wasn't working before?!? Would this be the reason why Xvid did poorly on Doom9's codec shootout? Well not exactly "poorly" but I was expecting better results after the 1.1 releases.

[Nazgul]

Originally posted by jk888
qpel wasn't working before?!? Would this be the reason why Xvid did poorly on Doom9's codec shootout? Well not exactly "poorly" but I was expecting better results after the 1.1 releases.

It doesn't sound like Qpel itself was disabled, just some mmx optimizations for Qpel that would've helped the speed. Unless I'm reading the changelog wrong.

[superdump]

It's official now, the release and full changelog has been posted at XviD.org (http://www.xvid.org/). Hopefully the new improvements prove worthwhile for your use. :) Enjoy the new version people!

[Prettz]

Are these changes compared to the 1.1.-127 build, or just compared to the 1.0.3 builds?

Also, the 1.1.-127 builds produced awesome encodes for me, so this version should be the same.

[CQ]

celtic druid what's the diference between the icl7.0 and icl8.1 versions? Its the software that compiles it or what :rolleyes: ?

[Shinobu]

kooooool, i'm going to test it, but not multi thread support to bad for those how use dual cpu like me....

keep up your good job, thanks

++

[mikeX]

Originally posted by CQ
celtic druid what's the diference between the icl7.0 and icl8.1 versions? Its the software that compiles it or what :rolleyes: ?

ICL refers to the Intel C compiler that he is using to compile XviD. Some time ago, it was reported that icl8.1 favored Intel cpus more than AMD ones, disabling certain optimisations that normally were also available for AMD. That's why icl7.0 is still widely used (I think that's what Koepi is using as well).

[ChronoCross]

Originally posted by jk888
qpel wasn't working before?!? Would this be the reason why Xvid did poorly on Doom9's codec shootout? Well not exactly "poorly" but I was expecting better results after the 1.1 releases.

you should probably re-read the quality report off dooom9.org.

Xvid 1st in speed
Xvid 3rd in quality. It got beat by the two h.264 AVC codecs in developement by nero digital. other than that it's the best codec out there and in my opinion there was hardly any difference between xvid and the H.264 codecs.

[CQ]

Originally posted by mikeX
ICL refers to the Intel C compiler that he is using to compile XviD. Some time ago, it was reported that icl8.1 favored Intel cpus more than AMD ones, disabling certain optimisations that normally were also available for AMD. That's why icl7.0 is still widely used (I think that's what Koepi is using as well).

Thanks! It's so clear for me now! :)

[Ezhihua]

:devil: :devil:
thanks

I just have a problem.

is it for windows2K/XP/2003?
however when I launch the tools in win2K,such as cal.exe,they crash!

Are they packed by upx?it seems that the problem caused by compressing.

excuse for my poor english....:sly:

[IgorC]

in fact i tested Celtic and Koepi compilations. maybe Celtic compilation approx. 0.317 % is faster (very liitle difference and there is possibility of error) so the speed is iqual for all compilations.

[Blue_MiSfit]

Originally posted by ChronoCross
you should probably re-read the quality report off dooom9.org.

Xvid 1st in speed
Xvid 3rd in quality. It got beat by the two h.264 AVC codecs in developement by nero digital. other than that it's the best codec out there and in my opinion there was hardly any difference between xvid and the H.264 codecs.

I find that for AVS/Virtual Dub encoding ( my favorite method ), there is nothing better than XviD. ND AVC is obviously an option, VP6 doesn't come close IMHO. DivX is always a step or 10 behind, and the experimental codecs x264 and snow are amazing yet still not "complete" enough for me to use all the time.

XviD is so customizable, dependible, understandable, fast, and bug free that I really find no reason to constantly use anything else. :)

A little OT I realize, but I was inspired

more on topic, the additions to XviD in 1.1 are awesome. VHQ for b-frames really helps boost compressibility especially in cartoon content for me. Good to see we have a friendly kopei release of the 1.1 tree to play with, not that there's much difference between it and Celtic-Druid's nightlies IMHO :)

~misfit

[peteag]

Is XviD full featured (now): 4MV, splines, etc. in the decoder and the encoder???

newbie-question, I know.

[LiFe]

Originally posted by ChronoCross
you should probably re-read the quality report off dooom9.org.

Xvid 1st in speed
Xvid 3rd in quality. It got beat by the two h.264 AVC codecs in developement by nero digital. other than that it's the best codec out there and in my opinion there was hardly any difference between xvid and the H.264 codecs.

Nero has two codecs, one ASP and one AVC. ASP gave xvid a sore run for it's money (from what I read) and AVC out did it a little. Keep in mind there are still a few AVC features they need to add, and then overall tweaking to be done.

[Leak]

Originally posted by LiFe
Nero has two codecs, one ASP and one AVC. ASP gave xvid a sore run for it's money (from what I read) and AVC out did it a little. Keep in mind there are still a few AVC features they need to add, and then overall tweaking to be done.

Nope - what Doom9 tested were the AVC Main Profile and AVC Hi Profile codecs; Nero's MPEG4 ASP codec wasn't even looked at.

Well, AVC beating MPEG4 ASP - who'da thunk? :D

[b0b0b0b]

congrats on the beta release, dling to test now

[peteag]

Think fully exhausted ASP- and AVC-Codecs are not comparable, because these are two extremely different technologies. At the moment, nero's AVC-implementation isn't that better that our new XviD-release and it can't hold the promises of what the AVC-standard has expected to be. There's a lot of time to invest.

I like XviD very well because it's a technologie which is also not finished in implementation, so there's still a place for ASP beside AVC. Look some years ago. The "holy DivX4 - The incredible MPEG4-Codec". Look now. XviD compared to DivX4 ... :p !!!

So, the time and the hard work of the developers of XviD and some other people, who made these wondeful free avaiable releases, has to be honored. But how? My Way -> USE THIS CODEC WITHOUT SPOTTING ABOUT IT'S QUALITY COMPARED TO MUCH NEWER TECHNOLOGIES !!!

Without these great guys you wouldn't have anything to encode your full-lenght-movies to a single CD in that quality.

Great job guys!

[Yong]

Keep the good work, XviD developers.:)

[jk888]

You're right you can't even begin to compare AVC and Xvid, because AVC freaking destroys Xvid for quality. On a 10 point scale I give Nero AVC 10 and Xvid 5. Yeah it's that much better.

I had a 2 & 1/2 hour movie with lots of dark + grey + foggy scenes and changing special effects (Japanese movie Casshern). I wanted to encode it into 2 CDs with Xvid, and the quality looked like crap, especially on the grey or foggy scenes.

Then 1 month later I read Doom9's codec shoot-out and then tried using AVC. I used the same movie with the exact same avs script, and WOW!!! The difference was unbelievable! I'm talking sharp! All the dark and foggy scenes were crystal clear!

I don't know how good AVC will become in the future, but right now it's a hands down winner already.

It's great for a new release of Xvid and thumbs up for everyone's hard work, but this time, after 1.5 years of using Xvid I decided I don't need to download the new version. Thx Xvid for giving me great encodes for the past year, please don't be offended now that I'm moving on to the future: AVC.

[OMINUS]

Originally posted by jk888
You're right you can't even begin to compare AVC and Xvid, because AVC freaking destroys Xvid for quality. On a 10 point scale I give Nero AVC 10 and Xvid 5. Yeah it's that much better.

I had a 2 & 1/2 hour movie with lots of dark + grey + foggy scenes and changing special effects (Japanese movie Casshern). I wanted to encode it into 2 CDs with Xvid, and the quality looked like crap, especially on the grey or foggy scenes.

Then 1 month later I read Doom9's codec shoot-out and then tried using AVC. I used the same movie with the exact same avs script, and WOW!!! The difference was unbelievable! I'm talking sharp! All the dark and foggy scenes were crystal clear!

I don't know how good AVC will become in the future, but right now it's a hands down winner already.

It's great for a new release of Xvid and thumbs up for everyone's hard work, but this time, after 1.5 years of using Xvid I decided I don't need to download the new version. Thx Xvid for giving me great encodes for the past year, please don't be offended now that I'm moving on to the future: AVC.


good for you 888 that you can now make superb movies with avc
although i think you are a little excessive about the quality gap between xvid and avc

anyway you made your point,now pls let the rest of us enjoy this new release of xvid-if we are making mistake because we still use it well i will find this out by mysfelf not from you

cheers lad

[suspiciousBob]

I'm getting slightly undersized files with XviD-1.1.0-Beta1

Source is r2 dvd of bbc's porridge, pal, 25fps, interlaced

LoadPlugin("C:\PROGRA~1\GORDIA~1\DGdecode.dll")
LoadPlugin("C:\PROGRA~1\GORDIA~1\decomb.dll")
LoadPlugin("C:\PROGRA~1\GORDIA~1\UnDot.dll")
mpeg2source("D:\Encoding Storage\Porridge 1x01\P 1x01.d2v")
crop(16,2,692,572)
FieldDeinterlace(blend=false)
BicubicResize(640,496,0,0.5)
Undot()

2bframes, vhq for bframes too, packed bitstream, ultra high/wide search, chroma motion, turbo, 250 to kf

aiming for 194MB, getting 192MB

the same kind of 2-3 MB undersize is apparent on all 6 episodes encoded.

[Koepi]

Can you please post:

- no. of frames for the encode
- entered "desired kbytes" (or bitrate)
- resulting kbytes(!) (MBs are calculated by the OS and therefore not really reliable. A size difference of i.e. 600kb can be the difference in shown 192/194MB due to rounding taking place)

Cheers
Koepi

[suspiciousBob]

43172 frames (28m46s)
average bitrate: 943kbps, aiming for filesize of 198715 KB
final avi size: 200,433,664 bytes (195736 KB)

[Sergei_Esenin]

Originally posted by jk888
You're right you can't even begin to compare AVC and Xvid, because AVC freaking destroys Xvid for quality. On a 10 point scale I give Nero AVC 10 and Xvid 5. Yeah it's that much better.

Not if you want to use high bitrates and know how to tweak the settings for high bitrates, in which case Nero AVC smoothes out the detail too much compared to XviD with a custom matrix, VHQ at max, etc. Until Nero gives more control over the encoding matrix etc. it won't match XviD at high bitrates or fixed low quantizers.

[obieobieobie]

Very nice. Going to play around with it.

Btw, "misteriously" must be a typo. The correct word is "mysteriously". :)

[gotaserena]

ASP gave xvid a sore run for it's money

Am I the only one who finds the phrase above funny (regardless of its accuracy)? :D

[len0x]

Are there any plans for doing VBV for single pass mode?

[Koepi]

Originally posted by len0x
Are there any plans for doing VBV for single pass mode?

AFAIK noone's working on this (and I lack the time to look into it).
Sorry for the bad news. Maybe someone feels motivated to do so though? ;)

Cheers
Koepi

[Sharktooth]

Uhm... i think it's quite difficult to implement VBV for a 1pass encode... the results will be suboptimal and the average bitrate could be completely screwed up.

[len0x]

But for fixed quantizer encoding we don't really care about average bitrate...

[Sharktooth]

Well, yes coz the encode must respect the "max bitrate" for that profile...
So if a fixed quantizer encode exceeds a xxxx bitrate the VBV should lower it to respect the profile...
IMHO 1pass with VBV is useless.

[len0x]

There are quite a few ppl doing quality based encoding, so standalone support would be beneficial for them.

As for me - I like comp test result (i.e. quant=2 encoding) to be as close as possible to the two pass encoding, while VBV for two passes can change result of comp test quite a lot in some cases...

[Sharktooth]

Uhm... with 1 pass VBV i think it will be "out of target" anyways.
That's maybe why DivX does the first pass at constant bitrate instead of constant Q....

[_rEuTeL_]

I'm experiencing a small bug with the brightness slider

after I've decoded something, the slider seems to have moved slightly, resulting in a brighter (too bright) playback
nothing serious, but annoying nonetheless :p

[suspiciousBob]

Originally posted by _rEuTeL_
I'm experiencing a small bug with the brightness slider

after I've decoded something, the slider seems to have moved slightly, resulting in a brighter (too bright) playback
nothing serious, but annoying nonetheless :p I also see this.

[Sharktooth]

Yes it was there even in previous builds...
Once you restore the decoder defaults it won't "move" anymore...

[_rEuTeL_]

yes, but if you enable deblocking again, it happens again and again

[Sharktooth]

...i know...

[Marcel]

Originally posted by Sharktooth
Well, yes coz the encode must respect the "max bitrate" for that profile...
So if a fixed quantizer encode exceeds a xxxx bitrate the VBV should lower it to respect the profile...
IMHO 1pass with VBV is useless.

I like to encode short funny clips for my mobile (Nokia 3650), 208*1xx pixels, 12.5 or 15 FPS. File size doesn't matter, so I go for Q5 or Q6. In some scenes this throws the bitrate above 200 kbit/s, where the SmartMovie Player starts to cry for more performance (but I didn't find any Socket 775 inside the phone yet, any help on that? ;)). That's a case where 1pass with VBV is useful - or did I get the whole VBV idea wrong?

[Sharktooth]

I'm saing i dont know if a CORRECT 1 pass VBV is possible, but i dont think it is.
VBV is not only something that gets enabled or disabled...

[Tri]

Selecting XviD in VirtualDub and clicking the "about" button shows the GPL License with a "Dismiss" button. Is that intended? :confused:

[Marcel]

Originally posted by Sharktooth
I'm saing i dont know if a CORRECT 1 pass VBV is possible, but i dont think it is.
VBV is not only something that gets enabled or disabled...
Because with VBV you have to answer the same question as with a target size / target bitrate for the whole video: How many bits for frame x? ?
So I better go for a 2pass encode with fixed quantizer, if I want to have a sensible use of VBV.

[Luminaria]

What is VBV exactly? I noticed that on the tab containing the @ profile recomendations but am not really sure what purpose it serves.

[JarrettH]

Nice profiles too:D

[ObiKenobi]

Originally posted by Luminaria
What is VBV exactly? I noticed that on the tab containing the @ profile recomendations but am not really sure what purpose it serves.

It's used so that the video stream's bitrate doesn't overwhelm the buffer of the decoder. It's really only useful when you are playing the encodes on a hardware player or a PDA.

[peteag]

Does XviD contain all features of the ASP-standard now or is it far away from this (4mv, slices etc.)? Same the decoder?

[LordIntruder]

Originally posted by ObiKenobi
It's used so that the video stream's bitrate doesn't overwhelm the buffer of the decoder. It's really only useful when you are playing the encodes on a hardware player or a PDA.

I was going to ask the same question. Is that really only for a question of hardware? Someone said above that he gets a better compression with this option on.

By enabling this option, what am I supposed to get?

I will of course test this option but it is just to know what basically this new one is intended for.

Thanks :)

[ObiKenobi]

Originally posted by LordIntruder
I was going to ask the same question. Is that really only for a question of hardware? Someone said above that he gets a better compression with this option on.

By enabling this option, what am I supposed to get?

I will of course test this option but it is just to know what basically this new one is intended for.

Thanks :)

That's what the option is used for, it has nothing to do with compression level. It's so the decoders buffer doesn't get overwhelmed by spikes in the bitrate causing stuttering playback. I don't ever use the option so I have no clue what it does to compressibility, but I remember one time I had forgotten to turn it off when I had done a new install of XviD and it made the quality go down a noticeable amount.

[BoNz1]

BTW for all the people wanting to know about 1 pass VBV and whether or not it is worthwhile or not, I urgue you to think about the world beyond your DVD rips :P. You definitely need strict VBV if you want to do any kind of streaming. I guess you would need some sort of look ahead though too so it wouldn't be a completely blind one pass. Anyway, is it just me or would it not be cool to see streaming XviD? I'm sick of all these proprietary streaming garbage. Oh, well it will probably happen one day when someone needs it.

[madness]

Hi all, when I encode videos in xvid I tend to have closed gov enabled, but on this beta I found the closed gov option is gone. Is it been disable for this beta or it got replaced Rate-Distortion mode decision for bvops?

Thanks.

[ObiKenobi]

Originally posted by madness
Hi all, when I encode videos in xvid I tend to have closed gov enabled, but on this beta I found the closed gov option is gone. Is it been disable for this beta or it got replaced Rate-Distortion mode decision for bvops?

Thanks.

I'd be willing to bet its now just a default option.

[AsTimeGoesBy]

I have installed Xvid 1.1 beta yester evening.... :)

So if i have understand correctly the posts here in this thread, enabling this VHQ feature also for b-frames may improve quality, right!?
Edit: i found something about this in a post by Sharktooth (http://forum.doom9.org/showthread.php?&threadid=88525)

Another thing, is there any possibility to overtake my zone settings from Xvid v1.0 to v1.1?
Till now i only had to re-activiate the encoding jobs in VirtualDub's job list but that doesn't work any longer. Also copying and pasting registry strings doesn't work any longer... :(
I have a problematical vide to encode and i'm wondering how Xvid 1.1 will solve it with just the same zone settings.

[peteag]

which format does xvid use: 4:1:1, 4:2:0 o.s. ?

[Koepi]

YV12 is defined as planar Y'CbCr 4:2:0.

The registry settings between 1.0.x and 1.1.x are incompatible. you can take the zones alone from the 1.0-registry export and import those to the settings in 1.1.x.

[AsTimeGoesBy]

Originally posted by Koepi
...
The registry settings between 1.0.x and 1.1.x are incompatible. you can take the zones alone from the 1.0-registry export and import those to the settings in 1.1.x. Thanks, i suppose i only must separate values with "zone" in the name!?
Ok, i will try it (again)... ;)

_____________
Edit:
If you are doing this, not only take values beginning with the string "zone". There is also a value called "num_zones" you should save, otherwise it's very probable that the number of zones displayed in the Xvid configuration window is not correct.

[loni_blues]

Hi,

I am getting some colour artifacts in B&W scenes (remnants of colour in some parts of the picture after a sudden change in the movie from colour to B&W). Sorry if this isn't new, but I hadn't noticed it before.

Regards,
loni_blues

[Koepi]

Originally posted by loni_blues
Hi,

I am getting some colour artifacts in B&W scenes (remnants of colour in some parts of the picture after a sudden change in the movie from colour to B&W). Sorry if this isn't new, but I hadn't noticed it before.

Regards,
loni_blues

This happens if a scene change isn't detected as such (thus the I-frame is missing so the picture gets "all b/w").

Let's see if sysKin has an idea how to solve this. Most probably you have a source where the difference between the coloured scene and the b/w scene is minimal, so there's no clear cut/scene change when switching from colour to b/w?

Cheers Koepi

[RadicalEd]

Well, the obvious answer would be to add a special case to scenechange detection involving the zeroing of the chroma planes. For now, you could just make the b/w segment a zone.

[Koepi]

...and force the keyframe that way. Good idea. (Not even a need for using greyscale encoding - there shouldn't be any colour artefacts in the original, right? ;) )

Cheers
Koepi

[Dams]

Is this build can be called "stable" as I use only VBV for my home standalone player (Keyplug 4810) , because I don't use any "pro" feature like BF or other things like this ?

Is VBV before this build , worked only on first pass ?? (- {core}: VBV support in 2pass mode )

I use actualy Nic build 13/12/04 compiled 8.1 Intel

Ps : I've got an Duron 1.6@2.2 Ghz

[SirCanealot]

Is anyone having any problems with rate control?
I have a file I want to encode to about 925kbs, but I'm having to set it to around 880kbs to hit my target filesize.

I have VHQ4(+B-Frames) on, Trellis Quant on, quants set to 2-31, chroma motion on, MSP on 6, cartoon mode on, and everything else is just about defaults.

If there isn't a probolem, I guess there might be some sort of problem with my first-pass data - I think I'll run another one...

[CruNcher]

Don't use cartoon mode it's bugged

[SirCanealot]

Originally posted by CruNcher
Don't use cartoon mode it's bugged

Just in this version? :/

[Sharktooth]

I feel confident it will be fixed before the final release.

[OMINUS]

something i am confused about
when i want to encode a full b/w movie do i have to enable both
<begin with keyframe> and <greyscale> or just <greyscale>?

[lark]

just grayscale.
but enabling begin with keyframe doesn't hurt, since the 1st frame will be key anyway...

regards
t :)

[Marcel]

Originally posted by Marcel
I like to encode short funny clips for my mobile (Nokia 3650), 208*1xx pixels, 12.5 or 15 FPS. File size doesn't matter, so I go for Q5 or Q6. In some scenes this throws the bitrate above 200 kbit/s, where the SmartMovie Player starts to cry for more performance (but I didn't find any Socket 775 inside the phone yet, any help on that? ;)). That's a case where 1pass with VBV is useful - or did I get the whole VBV idea wrong?

Resorted my thoughts.
VBV adjustments can only be done by selecting a profile, right? This leaves me with 654720 bits of buffer size and a bitrate of 128 or 384 kbit/s. First too low, second too high.
Maybe I should stick with 1pass, target bitrate of 200 kbit/s or something like that (should depend on the number of pixels per frame) and minimum quantizer = 5 (Q4 gives no visual improvement over Q5 on that tiny 4096 colors screen). All these VBV values are a matter of intense testing, as the video player is not based on hardware, but on software.

[Prettz]

Originally posted by peteag
Does XviD contain all features of the ASP-standard now or is it far away from this (4mv, slices etc.)? Same the decoder?
No one more knowlegeable than me has bothered to answer you so I'll take a crack at it.

I believe that Xvid does support all of the defined ASP features. It's had 4mv for years, has b-frames and s-frames with 3 warp-point GMC (if that's what you meant by "slices"), Q-pel, custom quant matrices, adaptive quantization, reduced resolution, and allows you to limit the encoder to a certain profile and level. I think that's everything.

[bond]

regarding mpeg-4 asp, xvid doesnt support error resilience, thats it i think ;)

[bREAkDoWN]

Originally posted by madness
Hi all, when I encode videos in xvid I tend to have closed gov enabled, but on this beta I found the closed gov option is gone. Is it been disable for this beta or it got replaced Rate-Distortion mode decision for bvops?

Thanks.

Good question, but i haven't still found the answer. Can someone of the xvid gurus ;) answer?

Bye!

[squid_80]

Originally posted by bREAkDoWN
Good question, but i haven't still found the answer. Can someone of the xvid gurus ;) answer?

Bye!
I'm no guru, but looking at vfw/src/codec.c it would seem that yes, closed gov is now always on if bframes are enabled.

[RadicalEd]

Originally posted by Prettz
No one more knowlegeable than me has bothered to answer you so I'll take a crack at it.

I believe that Xvid does support all of the defined ASP features. It's had 4mv for years, has b-frames and s-frames with 3 warp-point GMC (if that's what you meant by "slices"), Q-pel, custom quant matrices, adaptive quantization, reduced resolution, and allows you to limit the encoder to a certain profile and level. I think that's everything.

Reduced res isn't a feature of ASP, and incidentally isn't supported by XviD anymore.

XviD lacks some of the other rate control mechanisms besides VBV (namely VCV and VMV), but they're largely inconsequential.

[riggits]

Originally posted by SirCanealot
Is anyone having any problems with rate control?
I have a file I want to encode to about 925kbs, but I'm having to set it to around 880kbs to hit my target filesize.

I have VHQ4(+B-Frames) on, Trellis Quant on, quants set to 2-31, chroma motion on, MSP on 6, cartoon mode on, and everything else is just about defaults.

If there isn't a probolem, I guess there might be some sort of problem with my first-pass data - I think I'll run another one...

I've gotten undersized encodes from changing the min. quantizers to 2. Back to 1 and I get perfect size every time. Cartoon mode also gimps things a bit.

[Jackzeripper]

Hello,
Many many thanks for the VBV support on XviD1.1 beta. It’s a great feature that seems to work perfectly (2 tests OK : bitrate beyond 3000k in AS@L4).
The new DNX profile with 4000k max bitrate is quite good for standalones.
Thanks for your great work !
A french fan

[TripleA]

Hello.

I'm cooking a one DL DVD collection of the Lord of the Rings trilogy Special Extended Edition.

I'm getting corrupted frames with XviD-1.1.0-Beta1-16012005 (Koepi's build, if it makes a difference. And January 16th is my birthday, btw, so thanks for the gift!;)). I noticed this corruption in the second and third parts, but not the first. Though they were all encoded in sequence in one session using essentially the same settings (credits treatment differed for Return of the King).

I checked an earlier encode made using XviD-1.1.-127-13102004 and the same frames are OK. But I haven't watched an entire encode made using the earlier build yet, so I cannot be sure the issue is not present elsewhere. I'm planning to watch a re-encode of Return of the King ASAP and I'll report then, but it may take some time since I have finals and stuff.

It looks like it could be bad RAM. But I'm quite sure my RAM is OK since it was extensively tested using MemTest86+ not too long ago (a 3 day run, if I remember correctly, less than 6 months ago). And my system is reasonably stable: it got through two passes of the entire LotR trilogy without a restart, for example. In fact, I don't remember the last time it crashed. So I'm pretty sure the hardware is OK.

Both XviD-1.1.-127-13102004 and XviD-1.1.0-Beta1-16012005 encodes were made using the same settings (manually re-entered after uninstall/reinstall of XviD, if you're wondering), so I'm pretty sure the bits of the software related to me are OK as well.

But I keep an open mind.

All the data related is still on my HDD. And I'm willing to help anyone who wants to investigate this in any way I can.

Here are links to a sample frame (frame #69472 of region 2 DVD set of the Two Towers SEE):

Source: http://img186.exs.cx/img186/211/frame69472source6xm.png
XviD-1.1.-127-13102004: http://img197.exs.cx/img197/5933/frame69472xvid11127131020042qh.png
XviD-1.1.0-Beta1-16012005: http://img169.exs.cx/img169/1161/frame69472xvid110beta116012005.png

Edit:

Forgot to mention XviD settings, originally. Silly me. Here:

Profile @ Level == unrestricted
Quantization type == H.264
Adaptive Quantization == 1
Interlaced Encoding == 0
Quarter Pixel == 1
Global Motion Compensation == 1
Reduced Resolution == 0
B-VOPs == 1
Max. cons. BVOPs == 5
Quantizer ratio == 1.50
Quantizer offset == 1.00
Packed bitstream == 0
Closed GOV == 1

Pixel Aspect Ratio == Square

Motion search precision == 6
VHQ == 4
Use VHQ for BF == 1
Chroma motion == 1
Turbo == 1
Frame drop == 0
Max. IF Int. == 132
Cartoon Mode == 0

All Quantization min == 2
All Quantization max == 31
Trellis == 1

Credits at const. quant == 20
Credits at Weight == 0.25 for Return of the King

Athlon XP default optimizations in effect.

YV12 AVISynth script is the source. But I don't see how that could be related considering the source frame is OK, as decoded by VDubMod (v1.5.10.1 build 2439, in case that matters).

Can't think of any other info that might be relevant. Let me know if anyone needs anything. I really want to know what's causing this. If it's my PC that's broken, I want it fixed. And if it's me, well, I want that fixed ASAP too: can't b0rk in finals season! ;)

[Mug Funky]

I have a file I want to encode to about 925kbs, but I'm having to set it to around 880kbs to hit my target filesize.

i get this a little. i've done a bit of testing (and i was kinda hoping someone else would notice this so i knew i wasn't mad).

i _suspect_ that VHQ b-frames are the culprits, but i only got this hunch while reading this thread here, and i'm embarrassed to say that in all my test encodes i didn't once disable VHQ b-frames :)

one thing i've noticed is that the cleaner the source, the more accurate the rate-control is. this is very strange, but thinking out loud it does indeed seem like the b-frames might be causing this somehow (b-frames are quanted higher, therefore more noise will be removed from them and not considered during ME? but when there's no noise to start with, the deviation is much smaller? am i even close?)

if you've got any real noisy video, play with that and see what happens. i'll do that too i guess :)

BTW, thanks y'all for this xvid milestone. without you guys, we'd all be using Divx (TM) or, god forbid, WMV.

[edit]

okay, looks like VHQ bframes doesn't hurt ratecontrol. it's something else :(

anyone can test this by adding noise to a source, or possibly just using a noisy source and sharpening it. shoot for smaller frame-sizes (my machine is slow, so i'm doing tests at 320x240, which may not be representative of xvid's performance on larger frames).

[edit 2]

tried it on full pal d1, interlaced, with the same result.

i also tried a few other versions (including 1.0.x), and they all seem to be thrown off by the amount of noise in an encode. i guess it's fine if your movie is long enough, as i've never encountered really undersized encodes for full-length stuff. anything over about 5 minutes long is pretty solid for 2-pass. i might have to set a long, noisy encode going to check this out (bladerunner springs to mind)

[edit 3]

tried the first few chapters of the good, the bad and the ugly, r4 PAL, using varying lengths to see if the undersizing improved with the length of the encode. it looks like it does somewhat.

all encodes start at frame 4329, and target bitrate is always 1000kbps . scene is when it fades up from the opening titles to some guy's face... i confess i haven't actually seen this film, but i figured "it's old, so it'll be grainy", and i couldn't find blade runner. as it turns out, the clip isn't all that grainy, but the picture shakes a tad and there's spots on the film in parts, as well as a fair bit of mosquito noise.

1000 frames, 821kbps
2000 frames, 837kbps
4000 frames, 907kbps
8000 frames, 929kbps

i'd call the 929kbps encode an acceptable deviation from target bitrate, but that's 5 mins 20, which is a bit long for a ratecontrol to stabilize. if you're encoding music clips, or doing a movie by chapters (not that anyone would do it like that...), this could be a problem.

settings: 2-pass, target 1000kbps, h.263 matrix, VHQ1+bframes, turbo mode, fast 1st pass, halfpel, no GMC, no AQ, no chroma motion (no difference with these on though). i haven't tested trellis yet.

[Jan Marijniszoon]

I might have seen the same bug as the person above...

I was encoding the Matrix (re-mastered and PAL version) just for fun.

I encoded the dojo-fight, the lobby, the trinity chase, neo stopping bullets and the fight with smith in the subway.

Everything went fine, except there is an error in the subway sequence. I used the newest XviD to decode. I also tried libavcodec both in Windows and Linux (with mplayer), but it gave the same error.

I used this script for the scenes:

LoadPlugin("DGDecode.dll")
LoadPlugin("warpsharp.dll")
MPEG2Source("matrix.d2v")
Crop(0,80,-0,-80)
sxsinput = last
targetwidth = 1024
targetheight = 416

sxsinput.Lanczos4Resize(targetwidth*4, targetheight*4)
XSharpen(255, 255)
Lanczos4Resize(targetwidth, targetheight)

And I used this settings for XviD:

SINGLE PASS with QUANT 3
Profile UNRESTRICTED
MPEG-CUSTOM --> Didees SixOfNine Max=20.qmatrix
Adaptive Quantifisation
Quarter Pixel
GMC
B-VOPS: 2
1.50
1.00
Packed bitstream = OFF
Closed GOV = ON
Aspect Ratio DEFAULT

Chroma motion = ON
Motion search = 6
VHQ = 4
VHQ for b-frames = ON
Frame drop ratio = 0
Maximum I-interval = 250
Cartoon Mode = OFF

All Quant restrictions are MIN 2 and MAX 20
Trellis = ON

Chromo optimizer = ON

And finally, here you can find the screenshots:

http://members.lycos.nl/LudoSanders/video-images/

I hope this didn't sound too lame or something. But I think XviD is so great, that I should help in case this might be an important bug.

[b0b0b0b]

I think they need a clip from your xvid encode to debug

[Didée]

I did a full movie encode (only 1 up to now :o) with Koepi's build, and experienced no errors.

Jan Marijniszoon, you should cross-check if the same errors occur on the same frames when you encode the same script again with the same settings.
Your 4*SSXSharpen-script is somewhat memory consuming, and I see no SetMemoryMax() in your script. It could well be possible that e.g. Windows did some strange things while swapping memory in&out ... especially if did anything else with the PC, during encoding.

edit: Swapped end by and, Xoepi by Koepi, ...

[Jan Marijniszoon]

I left the PC entirely alone during this encode.

Could you explain me more about that max memory setting?
Isn't it very strange that Windows (I use Win2k with the latest updates) should cause an error just like that?

I say this, because I also encoded alot of scenes from Reloaded and Revolutions with exactly the same script. I only used XviD 1.0 with them and there was never any error to be spotted.

[Koepi]

Originally posted by Didée
edit: Swapped end by and, Xoepi by Koepi, ...

I assure you that I'm not Xsexual or anything like that :D Thanks for changing that! ;)


Jan Marijniszoon:
Can you test the same script and sequence with another codec and check for an error in that place? Also, as Dideé noted, something fishy can happen if your system isn't absolutely well tuned and swapping is going on. So a second test with the same script and xvid-1.1-beta1 to recheck that it wasn't something random would be nice.

[Some background to this recommendation: computer components get affected by aging, too. Just because your system worked well until now doesn't mean that i.e. the memory is still 100% in order. Imagine you overclocked your system for playing a game or so, and now your system is back to normal settings. Due to the higher temperature your chips were working at, the aging of the components did accelerate by a noticable amount (rule of thumb: 10°C hotter means only half the longevity of a microchip if operated always at that temprature.)
If you did use a higher voltage for your chips it gets even worse, current CPUs/memory chips use structures that small that the electron flux can lead to material weakening. And so on.]


You used a custom quant matrix. Is the error reproducable with MPEG or h263 quantisation?

Thank you for testing that,

cheers
Koepi

EDIT: corrected "migration" to "flux". That's quite a difference %)

[Didée]

I didn't say "that IS the cause", I said "that *could* ... ".

Isn't it very strange that Windows (I use Win2k with the latest updates) should cause an error just like that?General rule: never trust Windows.

I say this, because I also encoded alot of scenes from Reloaded and Revolutions with exactly the same script. I only used XviD 1.0 with them and there was never any error to be spotted. If you have a scene that produced erroneous frames, please repeat that encoding with the very same script, codec settings, etc. If the errors are 100% reproduceable, then the problem can be tracked down. If the errors are not reproduceable but random,
then the cause is random.


BTW, if you like 4*SSXSharpen, you might like this (http://forum.doom9.org/showthread.php?s=&threadid=84196) even more - same quality if not better, only that it's a little faster ;)


edit: should learn to refresh before submitting posts ...

[yaz]

Originally posted by Didée
If you have a scene that produced erroneous frames, please repeat that encoding with the very same script, codec settings, etc. If the errors are 100% reproduceable, then the problem can be tracked down. If the errors are not reproduceable but random,
then the cause is random. sounds good but what if different ppl produce the same error but each only once? :-)) ok ...
i had the same problem as triplea and jan had. it had happened at a high motion scene and looked exactly the same as on jan's pictures. at a moment the high motion part of the scene fall apart to moving blocks. as the quantizers were raised extremely on that part.
my 1st guess is(was) vbv as only that relases produce such problems which officially has working vbv. so, now i'm on testing different profiles up to 'unrestricted'.
my next guess is the rate-controll part which was cleared a week ago or so. but, amof, i don't know how to test that part.

the bests
y

[Didée]

Got your point, yaz - and I think you got mine.

I know there were quite a few bug reports about latest build from koepi, most or all about corrupted or blocky frames. But if one hundred people just state "look here: bad frames! That's a bug!" and nothing else, it doesn't help all too much in tracking the bug ...

But my quote (should) stay true: if bad frames are produced indeed by the codec, then the errors should be 100% reproducable when repeating the *very same* encoding setup - there's not much room for "chance" on the encoding side :)
So, if repetition of the very same encoding shows no errors, or errors in a different place, then probability is high it's something else, but not the codec.

BTW, didn't CruNcher report "Cartoon Mode" were a little br0ken?

[yaz]

@didée
yes, u're right, an error must be reproducible so as to track down. no doubt. i'm back on testing (and reproducing:-)
the bests
y

[SeeMoreDigital]

Yes, it's that old chestnut again :eek:

As with previous versions of XviD, I can't generate B-VOP encodes and play them using my Xcard with this version either!

However, I have stumbled across the following observation... which may (or may not) be useful at establishing a remedy.

I often use StarWars 2 Ch41 (PAL 6min 17sec) with an movie AR of 2.35:1 as an source.

Anyway I've found that the XviD B-VOP problem (ie: dropped frames, tearing at the bottom of frame, stuttering etc) is far worse when the encodes are generated at 1:1 (ie 720x480/576 pixels). But seem to work "almost" perfectly when cropped/resized to smaller frames, say 640x272!

I've tried various B-VOP settings, I've also used as many as 5No B-frames and the results are the same. At 640x272, the Xcard can play fine but at 720x576, they are b0rked.

Is this any help to anyone who cares?

[Jan Marijniszoon]

Guys...

I re-encoded with the exact same settings.
I only changed this in the script:

SetMemoryMax(128)

I already threw away DGDECODE 1.0 and used the newer 1.10 version.
Would this matter?

Also, I enabled debugging in XviD.

The produced AVI-file is now 100% flawless.

Maybe it was the DGDECODE or was it that swap-file problem? Whatever, it seems it was not the brilliant XviD codec!

If you guys need me to do more tests, I would be happy to co-operate.

Let me finsish by thanking you guys for the info and especially Didée for that tip on sharpening!

[yaz]

@all (interested :-)
i checked the last 4 encodes i'd made w/1.1b1 and i found shitloads in each :-((( quite disappointing as the source is available only for the very last one. (ye know, the nature of tv caps) however, i made some observations.

by analyzing the bad streams i found that round the critical parts there were always some kinda frame 'misplacements'. there were many(!) i-frames missing a/o replaced w/p/b/s(!)frames. when i reencoded the stream w/the same .pass file the errors were xactly there but ... when i made a new 2pass encode everything went fine (???). so, the errors were generated during the 1st pass or it is just wrong signing during the second pass. the latter is supported by the observation that it seems as if the i-frames were there (same sizes at the same places) but signed falsely (???).
my bad is that in the meantime i tried also some previous 1.1b releases, so this kinda 'reproduction' is a bit ... messy. maybe it's just an installation misery (???) i haven't got closer :-(

dunno whether this story tells anything to anyone but i spent hours w/it and i thought it'd worth to share

the bests
y

[TripleA]

Originally posted by Didée
If you have a scene that produced erroneous frames, please repeat that encoding with the very same script, codec settings, etc. If the errors are 100% reproduceable, then the problem can be tracked down. If the errors are not reproduceable but random,
then the cause is random.


I basically agree.

But look at the problem from my perspective:

I have a 40+ hour encode session consisting of about a million video frames the output of which is showing about 6 to 10 broken frames (I'm not counting frames dependant on broken frames as broken themselves) at seemingly random places in the second two thirds of the session (The Fellowship did not have any problems that I noticed. And while I wasn't watching every frame hawk-like, I did later notice the broken frames several times in the later two parts despite not having changed my watching mode).

I have no idea what might be causing this problem. What if it's a signed integer that should be unsigned but only overflows after one-hundred-thousand frames? What if the integer is in VDubMod and starts b0rking only after 10 or so hours of work, for whatever reason? What if it's the rate-control code that's the culprit? Surly I cannot test that without re-encoding the entire clip. And what if, for whatever reason, it's random frames that get b0rked? Is it reasonable to spend 11 hours watching video to, perhaps, catch those few frames (or not) by pure luck? Isn't there something that might automate this?

The problem with me trying to troubleshoot this is that I have no idea how the insides of any component work. I don't even know what's possible and what's effectively stating here-be-dragons.

So what I was hoping for when I posted here was for someone who knows how the insides of XviD work to take a look at the frames and say something like "nope, can't be XviD", "if it's XviD, it would be such and such. Do this to check if it is" or "not enough information. Do this and get back to us with the results and we'll proceed from there"... before I start Easter-egging in the dark. I can do that on my own, thank you very much. I certainly didn't post here for that piece of advice. Unless you're telling that's the best that can be done, under the circumstances...

[Teegedeck]

Sorry, I'm sure everyone would have been happy to try and help you but it obviously won't work without you putting some work into it yourself. Beta-testers working together with developers in the bug-hunt etc. Granted, it is a bit much to encode it all again.

[Jan Marijniszoon]

Sorry guys...bad news...

I didn't feel right about the last encode and I saw I forgot something in the script. I used LanczosResize instead of Lanczos4Resize.

So I did it again...I also downloaded the older version of DGDECODE and made a new d2v-project. So it was now exactly the same as the firt encode, with the addition SetMemoryMax(128) to the script to prevent memory problems.

And the error was reproduced at the exact same frames.
So I think this is really a bug.

EDIT: trying it with H263 now...I'll report later.

[Koepi]

If you change something in your avisynth script and the error occurs _then_, what does this mean? An xvid error? For sure not. It's either your lanczos4-resize (unlikely) or a "broken" dgmpegdec.dll (very probable).

Cheers
Koepi

[yaz]

@koepi
i must say (w/my greatest respect) u're wrong. i didn't use dgmpegdec (i worked w/xvid precoded tv caps), i didn't use lanczos4 (cus i don't like that), i didn't tamper w/my script (as it's fairly optimized for that caps) but i have had the same errors as others had. in add, i've found it in four different streams. it is very likely a codec issue. we tried different sources, different scripting, different codec settings but the error is the same. what i've found so far should tell sg to someone knowing the xvid internals. i'm sure that there's no avs script being able to hijack xvid's vhq so rude as i outlined above.
pls, don't be off-hand. maybe u're right, and it's just a 'very-rare' case, but for me it seems to be rather a 'may ever happen'. this or that, imho, it'd worth to chase this beast down.
the bests
y

[Jan Marijniszoon]

LOL yaz, I think that Koepi was addressing me.

Koepi...

I did the exact same encode again with H263 and then the error was gone. So is the fault in the custom mpeg matrix? How come, I have always used it with succes.

[TripleA]

Originally posted by Teegedeck


Sorry, I'm sure everyone would have been happy to try and help you but it obviously won't work without you putting some work into it yourself. Beta-testers working together with developers in the bug-hunt etc. Granted, it is a bit much to encode it all again.



Painful words. But nevermind: you're right; I wasn't doing all I could. Not burning enough neurons on the problem, etc. I could plead other-things-to-do, but everyone has those so that excuse doesn't work.

Anyway, I now have a 5014 frame section (frames 68041 to 73054 inclusive, of region 1 DVD (IVTCed using DeComb) Two Towers SEE) that exhibits this problem when encoded with XviD-1.1.0-Beta1-16012005 and doesn't when encoded with XviD-1.1.-127-13102004. I reran the two passes of the XviD-1.1.0-Beta1-16012005 run and got binary-identical output (except for headers) and .pass files to the first run, so it's pretty conclusive, it seems to me.

I tried to make the interval smaller (smaller file sizes if test clips are to be passed around etc.), but couldn't... Wait a minute! I have an idea!!... Oh, I love myself I'm so clever!: I re-encoded the XviD-1.1.-127-13102004 output using XviD-1.1.0-Beta1-16012005 and it b0rks, too! So now I have a 15.4MiB test clip ready to go... ;)

Your move.

@Koepi: you probably know more about these things than I. But to me it seems that at this point in the related softwares' life, if the original AVISynth script when viewed in VDub does not look broken then anything contained in it can be ruled out as the cause of the problem. It might be that XviD b0rks when fed frames that have specific characteristics caused by one part or another of the AVISynth script, true, but that would still be an XviD bug, in my book.

Just dismissing a problem reported by 3 different people out-of-hand to me seems bad scientific practice. Perhaps the developers should also be prepared to do some work in the "bug-hunt"...

[Edit: Ahh... minor irrelevant correction: region 1 DVD, not region 2. The three parts aren't from the same region so I got confused, sorry.]

[Sharktooth]

@Jan Marijniszoon: What's the custom MPEG matrix you used?
Was trellis quant enabled or disabled?

[Jan Marijniszoon]

Originally posted by Sharktooth
@Jan Marijniszoon: What's the custom MPEG matrix you used?
Was trellis quant enabled or disabled?

I used Didees SixOfNine Max=20
With all the quant ranges from min 2 to max 20
And yes, I had Trellis enabled.

EDIT: I think the quant ranges do not ever matter, becaused I used a fixed quant of 3

[Sharktooth]

Can you repeat the same encode with the same custom quant matrix BUT with trellis disabled?

[ChronoCross]

Originally posted by TripleA
@Koepi: you probably know more about these things than I. But to me it seems that at this point in the related softwares' life, if the original AVISynth script when viewed in VDub does not look broken then anything contained in it can be ruled out as the cause of the problem. It might be that XviD b0rks when fed frames that have specific characteristics caused by one part or another of the AVISynth script, true, but that would still be an XviD bug, in my book.


I'd actually like to comment on this seeing as I have ran into certain issues with this particular thing before. Just because something looks perfect in the vdub preview window does NOT mean that it will come out like that when it goes through xvid. There are external factors like real time decisions that filters make based on previous frames and whatnot especially with deinterlacing that can change the entire look of the encode. So the is a very good possibility that it is a error in your filter chain.

[Koepi]

Well, the error looks like misplaced MBs. This is very weird, because if you have problems concerning the Custom Quant Matrix (or i.e. Trellis), it should give errors like wrong blocks/visible blocks (like too bright/inverted/distorted/something like that blocks). It shouldn't affect motion estimation/compensation.

One thing which would be possible is maybe a broken bitstream due to a certain bit sequence indicating "end of MB" so that texture data gets interpreted as motion vectors. Just a very wild guess, but there is no other relation that comes to my mind.

@yaz & TripleA: I was indeed replying to Jan as he wrote this happened with an old version of dgmepgdec. Updates in dgmpegdec are usually further bugfixes so I assumed the bug was there. You're of course right, if there is no (visible) artefact in the avs output this isn't (necessarily) true.

Cheers
Koepi

[TripleA]

Well, I took extra care in these last tests I ran to make sure all options are identical in all test runs using the two different XviD builds (the same options I mentioned earlier, btw: no custom matrices or anything fancy like that). I even took screenshots of the dialogue boxes to compare and be absolutely sure (if you have my memory, you quickly learn to totally distrust it). The only different thing is Closed GOV, and that only because the option has been removed in Beta 1. All other software, filters, scripts, etc. are exactly identical. I didn't even update DGMPGDec. To the extent of my control, the only difference is the XviD build used. This coupled with the binary-identical output of two runs using Beta 1 to me strongly indicate the cause is a bug inside XviD Beta 1. Does anyone have any other explanation? What further tests would you suggest? I'm open to suggestions.

P.S. I am right now watching the Return of the King re-encoded with XviD-1.1.-127-13102004. I'm trying to pay extra attention to the video quality, but, despite my best efforts, the movie is distracting me, at times :). We'll see what happens.

[yaz]

Originally posted by Koepi
Well, the error looks like misplaced MBs.yep. now i'm sure, it's sg like that. in the meantime i examined my fawlty streams further on and i found sg strange in one of them. some of these ugly blocks are shown at quite different places than they should be. so, the block looks perfect but shown not at the right place. (sg like p.i.p. on most modern tvs)

@jan
yes, i know, koepi answered to u. i only wanted to point that the only common in our misery is the codec (and the misplaced blocks). anything else is different. that's why i'm conculeded there that it must be sg related to the codec.

has any of u examined the stream round the blocky part? can anyone confirmed that the frames are 'mis-typed'? (see my comments above) it's quite easy w/ffdshow.

@triplea
pls, try to reencode the blocky part from a pass file produced by 1.1b1 but using an other codec version not producing this error. this way i found that (some of) the errors are generated in the 1st pass.

the bests
y

[TripleA]

Finished RotK.

Good movie. Good trilogy in general, in fact. I'm very impressed. Everyone who hasn't seen this should and everyone who doesn't have it should buy it. Amazon has the entire trilogy for something like US$80.

But I would have been impressed infinitely more had I not read the book, before: the book is *much* better. Read it.

And the movie diverges quite a bit from the book, at times, if I remember correctly. But it has been a while since I read it. And my memory is... well... read comment in my previous post.

Oh! And the encode is OK, so far as I can tell.

I'm not sure what yaz suggested (using the .pass created by one build of XviD with another) is legal. I suspect that such practice would create its own set of unpredictable problems, actually. But then we're already having problems, so I went ahead and tried my little test clip (5000 frames is just *soooo* much better than 1000000 ;)) in the four possible combinations using the two XviD builds concerned in two passes. And guess what? It seems the problem shows only when *both* passes were done using XviD-1.1.0-Beta1-16012005!!

I hope that narrows things down a bit.

Though I would suggest it would be better if someone were to take a look at this, find out what's going wrong and fix it.

[TripleA]

Not sure this is significant or helpful, but the broken frame in my short test clip isn't exactly the same as the one in the full film:

Full: 69471
Test: 69468

Both on the same scale, naturally: that of the full film.

[Edit: And when re-encoding the test clip encoded by XviD-1.1.-127-13102004 by XviD-1.1.0-Beta1-16012005 (read it a couple of times. It can be understood) it breaks at yet another position: 69474!]

[Jan Marijniszoon]

Originally posted by Sharktooth
Can you repeat the same encode with the same custom quant matrix BUT with trellis disabled?

Sure I can.

It's indeed the trellis. Without trellis, the error is gone.
So can you geniuses fix this? :-)

[TripleA]

Here, too, things are OK without Trellis.

[dragongodz]

trellis bugs with a custom matrix is not a new bug but an already known and already discussed one.
http://forum.doom9.org/showthread.php?threadid=84999

now xvid 1.03 has this change listed
- Fixed trellis optimization overflow for quant <= 2.

and also in 1 of Koepi's earlier 1.1 builds he fixed it, to quote from that thread
Silent update on my site: reduced the range from 11 to 10 as well. (in the 1.1.-127 test build)

now i just had a look at the source from Koepi's page used for the latest build and it has the TL_SHIFT at 11 again. so in theory all Koepi should have to do is reduce it to 10 again to mayby fix your problems. atleast worth testing.

[celtic_druid]

The fix from 1.0.x was merged into 1.1.x.
The fix by the way doesn't involve TL_SHIFT
http://forum.doom9.org/showthread.php?s=&threadid=84999&perpage=20&pagenumber=2

[dragongodz]

The fix by the way doesn't involve TL_SHIFT
the original fix that was used for the earlier Koepi build did. since that doesnt seem to have the problem ,according to the earlier posts, that is why i mentioned it may be worth trying.

and yes i see the
if(sum && (pMB->quant > 2) && (frame->vop_flags & XVID_VOP_TRELLISQUANT)) {
is in the latest source aswell. shouldnt that be >1 or >=2 aswell as Koepi said in the other thread since the problem was only at Q1 ?

[Koepi]

Can you please check which decoder gets used when you see the problems?

Isibaar suggested that it's a problem with the DivX decoder which has trouble decoding "long" motion vectors (or much motion)... so make sure that either ffdshow or XviD decodes the stream! This would be much of help,

thank you.

Cheers
Koepi

[TripleA]

DivX is not installed on my system. Hasn't been in years.

When I noticed this first, here, the clip I was watching was being decoded by ffdshow-20041223. I later investigated things and took captures using VDub. If I am not mistaken, that means it was being decoded by XviD VfW, since I have nothing else that would decode MPEG4 installed: only ever needed XviD. Is there a way to be certain of who decodes what, in VDub?

[Koepi]

In VDub, go to the "File"-menu and select "file information". This shows you the codec used for decoding. (ffdshow's name is misleading in the meantime, it has a quite mighty vfw interface in the meantime :) ).

Regards
Koepi

[yaz]

nah, divx decoder is big no-no for me. my system was installed a week ago, there's only xvid and ffdshow (and nero w/nbr) installed. i used to test my encodes w/xvid only but when i watch them i use ffdshow pp in a funky way. i switch rawyv12 support on in ffdshow which chains ffdshow as a 'postprocessor'. it's worked fine so far but now sg is fishy w/the new builds as i must switch rawyuy2 to make ffdshow kick in (???)
insisted by this i checked the decoder, and yes, if i leave (or set) on default it seems as if xvid would output yuy2. is it ok or sg's malfunctioning ?
further on, i don't have any deringing (stated in the installer and in the announcement) and the brightness slider seems to live its own life. it accepts any changes i set but sometimes it flips back to default (haven't found what triggers this).
is it the decoder devels wanted to release ?

as regards block-shits, i don't think it's the trellis+cqm problem as i did use the mpeg matrix (never reported to have any problem w/trellis). in add, i use trellis since it was introduced (actually, a big fun, i am) and it's never produced such problems so far.

anyway, i'm about to give it all up. i spent yesterday evening (and half of the night) w/tracking but i found ... nothing. the only way i can reproduce the errors is using the old .pass file originally generated. w/it i always got the 'random blocks' and the 'frame-misery', independently of what ver of 1.1encoder i use, and the output is always shitty, but no other way to pop this in. (???) i've made a new 2pass encode from the only source still i have and i haven't found any prob even if watching the problematic parts frame-by-frame ... baah ... it's too much for my simple mind.

the bests
y

[TripleA]

I thought that only gave the FourCC! But now that you mention it and that I pay closer attention, it does say "XviD MPEG-4 Codec" next to "Decompressor:" under the FourCC field.

So I guess we're sure about that, now.

I'm not sure what you mean by "ffdshow's name is misleading in the meantime", but I'm sure it's the filter that plays back stuff in DShow on my system. And the corruption showed, there.

[celtic_druid]

I would think he meant the fact that it can decode and encode via VFW where as the name suggests that it is dshow only.

[Koepi]

yaz: too bad the error isn't reproducable. Maybe something very suspicous like earth rays falling in with a 30° angle produced the error that one time.
</joking> Thank you for your efforts anyways.

TripleA: do i understand you correctly, the error shows both with xvid decoder and with ffdshow? EDIT: reread the last posts, yes, I understood it correctly.

Since there are only these two reports for this issue I think there occured some corruption the the stats file or something in the computer system might have caused that. It's not somethig systematically reproducable, and since XviD doesn't do any random decisions this is a likely assumption.

:(

Please try to find a small sample input which produces this error reliable and send or upload it. Else it's impossible to track this bugger down.

Thanks,
Koepi

[TripleA]

Koepi: please reread my last few posts more carefully. I know they're a bit longwinded and contorted at times, but I think the English is understandable, still.

I have a 15.4MiB clip that will reproduce this issue every time it's encoded. The output of all runs is binary identical both for .pass file and video output, except for headers of the last. So it's all quite deterministic and non-random, here.

I have no way of hosting the file. But give me some place to upload it to and I will.

[Koepi]

Sorry, my bad. I'm always in a hurry when I read/answer from work, so please forgive me :)

I will setup a temporary account on my space for you this evening and PM you the necessary data. Thanks in advance!

Cheers
Koepi

[Jan Marijniszoon]

Originally posted by Koepi

Since there are only these two reports for this issue I think there occured some corruption the the stats file or something in the computer system might have caused that. It's not somethig systematically reproducable, and since XviD doesn't do any random decisions this is a likely assumption.


Are you also talking about my report?
With me it is reproducable. Even if I trim the clip, it's always at the same frames. But it's related to a custom matrix combined with the trellis option.

Please try to find a small sample input which produces this error reliable and send or upload it. Else it's impossible to track this bugger down.

I can do this if you want. Tell me if you are interested.

[*.mp4 guy]

I was just using the ffdshow Xvid encoder and I noticed that Motion Search precisoin 6 is not the highest search option available. As I am a quality fanatic and not fond of the ffdshow gui or rate control, would it be possible torelease an Xvid 1.1 Beta build with an exhaustive search option?

[Koepi]

ffdshow does use it's own front-end for the xvid options, so there's nothing more than MSP6 in XviD. ffdshow just calls it another way and has no difference between i.e. search precision 9 and search precision 6 (don't know exactly how the sclaing 1-6 -> 1-9 is done in ffdhsow. But even in XviD MSP 2+3 are the same internal options %).

Regards
Koepi

[yaz]

it seems as if this misery would be left unsolved, so i made some further investigations and i found some common in my shitty encodes.

all scenes having this heavy blocking are :
- (very) high motion scenes, in a special way. in most such scenes sg (or sy) moves quickly away but the camera follows it (so as to keep it in the focus), so the real movement is in/on the background which sweeps away very quickly. blocks are tipical where the background just reveals from behind the moving object(subject), so round edges a/o contours.
- quite uniform as regards luma and chroma too. (just think of the most scenes of matrix1)
- quite blurry by nature (resulting from the high motion in the fore and in the background)
just for sure, i don't stay that scenes having these characteristics are all shitty, but only that scenes having this problem looks like that.

up to this, it seems not too complicated (for me:-)) but the codec has also some issue here

- round these scenes i always found some missing i-frames. it seems as if the codec replaced them w/interframes. the most funny is when it happenes w/s-frames resulting in a quite funky blocking pattern (blocks spreads all over holding clear fragments from other parts of the same or from some previous frames)

and there must be sg coming from the process (system?) itself, cus i found exactly what triplea reported
...the broken frame in my short test clip isn't exactly the same as the one in the full film ... And when re-encoding ... it breaks at yet another positionthat's why i didn't find the same error in my reencoded clip. it's shifted away to the next similar scene :-(
it seems as if there were sg overflowing/wearing out/giving up only after a certain amount of torture, making the codec to drop some bad macro-blocks so as to heal that wounds. after that everything goes fine ... for awhile.

dunno does it help or not, but i'm sure it's not a rare, occasional effect hitting just some 'lucky guys', but all of those making longer encodes would have it sooner or later. (maybe just not noticed yet)

the bests
y

[AsTimeGoesBy]

I also (still) have found hard block structures in some encodes. I just have used default settings as mentioned here (http://forum.doom9.org/showthread.php?threadid=89026) so no custom matrix was used.
Indeed it happens on fast motion scenes, the faster the more, but also the darker the the more visible blocks. The blocks i have found weren't misplaced, the image had has any mosaic touch rather.

That sounds now like the 5th wheel on a car.... Wouln't possible to implement any optional 'anti-block control loop' in Xvid's code in order to detect 'too strongly visible' blocks and to re-encode such a frame with a lower quantizier to get better quality!?
That would help at least... altough i must say the blocky appearences are rare, and compared to Xvid 1.03 they are even less probable.

[Jan Marijniszoon]

Hey Koepi, why did you not reply to me?

I think it is really a bug.

I also encoded the piece with XviD 1.03 WITH the custom matrix and the trellis option enabled and there is nothing wrong there.

So it is a malfunction in the new beta.

[Teegedeck]

Everyone, you did hit 'load defaults' after installing that build, right?

[TripleA]

I generally write things with the assumption that someone will read them, btw.

Originally posted by TripleA
Both XviD-1.1.-127-13102004 and XviD-1.1.0-Beta1-16012005 encodes were made using the same settings (manually re-entered after uninstall/reinstall of XviD, if you're wondering), so I'm pretty sure the bits of the software related to me are OK as well.

Also, I need the space and I will have to delete these files (about 80GBs in total related to this project) soon, if no one is interested in them.

24 hours to go, let's say.

[TripleA]

Originally posted by TripleA
Also, I need the space and I will have to delete these files (about 80GBs in total related to this project) soon, if no one is interested in them.

24 hours to go, let's say.

I truly can be such an idiot, at times.

Sorry for that.

The small test clip will survive for quite a while since it's, well, small. But if anyone would like to take a look at any of the MPEG2 source, they'd better come forward quickly.

Well, not that critical, either, actually: I can always rip the DVD again. It'll just be such a bother and I'm quite lazy, so please be nice and don't force me to do much work...

[APF_Gandalf]

for those of you getting strange blocking in motion, try disabling the GMC.
I got blocking with the 1.1Beta1 builds from Koepi and Celtic druid.I also tested an older 1.1 build from Celtic druid (end of december), and it has the same blocking problem.
disabling gmc fixed the encoding bug for me.

I hope it can help.

[Mr_Schizo]

Originally posted by APF_Gandalf
for those of you getting strange blocking in motion, try disabling the GMC.
I got blocking with the 1.1Beta1 builds from Koepi and Celtic druid.I also tested an older 1.1 build from Celtic druid (end of december), and it has the same blocking problem.
disabling gmc fixed the encoding bug for me.

I hope it can help.
I noticed that too and uploaded a little package which contains the source file, used avs and the b0rked clips i made.

Just follow the link to get the package if you're interested.
http://s2.yousendit.com/d.aspx?id=1IRU00JPFMEFR26USGL9OUJOF1
dunno how long its online

[APF_Gandalf]

I just tried the same settings with XviD-1.1.-127-06112004.exe from Koepi (thanks to Celtic druid for correcting my mistake) and gmc is safe in this one, no blocking at all.

[celtic_druid]

XviD-1.1.-127-06112004.exe would be from Koepi and if I recall correctly it isn't a pure cvs compile.

[yaz]

i've found what apf_gandalf had. the problem exist back to 041220 build from c_d. now i'm about to test earlier builds.
however, i wouldn't suspect gmc as this part was last touched a year ago or so (iirc). my main suspect is b-vhq. if i read the changelog correctly, this part was heavily reworked in the past months. it's supported by my findings about the 'missing i-frames'. as the "two vhqs" were conflicting. (btw, how many vhqs do we have ? or is it a stupid question ?)
the next (much weaker) suspect is the trellis-hack introduced because of the conflict w/some cqm.
the bests
y

[Ark]

I don't think that 2 vhqs can be in conflict, vhq is only a search (more search and even more search :D )process no? It is intended to do better ME that without it afaik...:confused:

However currently only mode 1 vhq is allowed for b-frames.

[Heini011]

Hi,

i have a 3 hour movie encoded with koepi's xvid 1.1 beta 1 and xvid 1.03. i used q-pixel, adapt quant, b-vop (3,1.5,0.7), eqm v3-hr matrix, vhq 4, b-vhq, q 2-31, trellis. (i don't use gmc at all).

the second pass is 5% under first pass (q2) size.

i watched the encodes carefully and compared many frames against each other. there is a small quality improvement in xvid 1.1 in relation to details and sharpness. no problems with misplaced blocks at all! :-)

only the decoding was not always smothly on motion with the 1.1 beta 1 build. no problems with ffdshow.

many thanks @xvid-team for your great work!

greetings, Heini011.

my system: athlon xp @ 2 Ghz, nforce 2, 512 mb ddr-sdram, win me

[APF_Gandalf]

Originally posted by yaz
i've found what apf_gandalf had. the problem exist back to 041220 build from c_d. now i'm about to test earlier builds.
however, i wouldn't suspect gmc as this part was last touched a year ago or so (iirc). my main suspect is b-vhq. if i read the changelog correctly, this part was heavily reworked in the past months. it's supported by my findings about the 'missing i-frames'. as the "two vhqs" were conflicting. (btw, how many vhqs do we have ? or is it a stupid question ?)
the next (much weaker) suspect is the trellis-hack introduced because of the conflict w/some cqm.
the bests
y

I will test the same encode with GMC and without Treillis and then without vhq for B-frames.
As I said, I already tried with GMC off, treillis and vhq for B-frames on without having the bug. so... :confused:
maybe treillis+GMC, or "something"+GMC produces this result.

[ChronoCross]

I just got done with a test of Samurai X: The Movie 1 hr 30 mins. I have to say I am impressed with the quality. However I did find a problem. there are approx 5 points in the movie at far apart times where there is a massive what looks like a frame freeze that creates super blocks until it hits the next keyframe. I'm testing it again with a more basic script to see if it was a filter from avisynth but I doubt it. If the problem reoccurs I'll provide a video clip segment and if someone will point me to a good vob cutter a vob segment of the scenes that become corrupt when encoded in xvid.

Edit: I fully Reproduced the error. Here's a short Clip.
http://www.chronocrossdev.com/encoding/kenshin_error.avi

[yaz]

@chronocross
what settings did u use as regards especially b-vhq/gmc(/trellis)? at the moment those are the main suspects for such 'super-blocking' (good phrase, but i hope we shouldn't learn it :-)
the bests
y

[ChronoCross]

Originally posted by yaz
@chronocross
what settings did u use as regards especially b-vhq/gmc(/trellis)? at the moment those are the main suspects for such 'super-blocking' (good phrase, but i hope we shouldn't learn it :-)
the bests
y

All of the Above

GMC
qpel
Trellis
B-VHQ

[yaz]

@chronocross
pls, try without b-vhq (a/o gmc). some reported that may help.
the bests
y

[dragongodz]

really you should test it disabling(and re-enabling) 1 at a time for all of them. if you can reproduce it by just doing a small section instead of the whole movie it should not take too long.

so try it without GMC. and see if its still there. turn GMC back on turn TRELLIS off and try and see if its the same. again turn TRELLIS back on and QPEL off etc. that should show which is causing the problem.

[Luminaria]

I've also encountered the same problem ChronoCross is having, although the super-blocking as it may be wasnt as bad as his. first encode was done with b-frame vhq, trellis, and gmc enabled (produced super blocks), second was done w\o gmc, but with trellis and b-frame vhq and had no super blocks. I dont have that encode with me anymore, so I cant give an example, but I thought this might help track the bug down.

[ChronoCross]

okay I did some tests.

Trellis,GMC, qpel, BVHQ - Blockiness Error
Test File 1 (http://www.chronocrossdev.com/encoding/test_GMC_BVHQ_QPEL_Trellis.avi)
Trellis,GMC,Qpel - Blockiness Error
Test File 2 (http://www.chronocrossdev.com/encoding/test_GMC_QPEL_Trellis.avi)
Trellis,GMC - Blockiness Error
Test File 3 (http://www.chronocrossdev.com/encoding/test_GMC_Trellis.avi)
Trellis - No Errors
Test File 4 (http://www.chronocrossdev.com/encoding/test_Trellis.avi)
Default Settings - No Errors
Test File 5 (http://www.chronocrossdev.com/encoding/test_defaults.avi)
Trellis,Qpel, BVHQ - No Errors
Test File 6 (http://www.chronocrossdev.com/encoding/test_BVHQ_QPEL_Trellis.avi)
GMC - Blockiness Errors
Test File 7 (http://www.chronocrossdev.com/encoding/test_GMC.avi)

Well it seems that GMC is the culprit. when by itself or with anything else it's always blocky. but when I used the other modes it was fine. Perhaps the xvid develpers can use my tests to figure out what the problem is. Hope this helps.

[TripleA]

Oh dear.

This, IMHO, is not going anywhere good.

See, in *my* test clip disabling Trellis did the trick (i.e. the problem "went away"). Yes, GMC was still enabled, in case anyone is wondering.

We can all encode stuff till the cows come home, but unless some devs are willing to take a test clip that's known (alleged) to cause the problem(s) and take a look at what happens inside the CODEC, this problem is not going anywhere.

Just IMHO.

In the mean time, there's XviD-1.1.-127-06112004.

[Edit: shortening the clip solved the problem for me, too. In the sense that there were no more broken frames when encoding the shorter clip. Should be a good solution for an FAQ somewhere: "only encode clips under 4000 frames in length." :)]

[ChronoCross]

yeah cept my problem was less than 300 frames and reproduced EVERY time I tried it. so shortening the clip does absolutely nothing. lol this is interesting indeed. I wish I knew more about coding so I could help the devs out on this.

[ChronoReverse]

I've encountered a similar problem with GMC frames too with the 1.1.0 beta. I was test encoding Read or Die and a short section had a strange blocking error. Interestingly enough, when I switched to h263 from a custom matrix (hvs-good) it went away. It's probably because I forgot the workaround for custom matrices, but I thought I might as well chime in

[yaz]

so as to make things more fuzzy; switching off b-vhq solved the problems for me. another quick solution was encoding only the problematic parts. that way i always got perfect clips. (sew a knob onto this :confused: ) so ... so, i'm quite sure it's a fairly complex problem (for us having no look into the core)

@chronocross
would u, pls, check your clips around the blocky parts as regards frame types. i find (again and again) that when super-blocks kick in there are always some 'frame-misery' too, i.e. i-frames are missing from where they were in the flawless encodes.
i would do it by myself, but 'up/downloding multimedia content is forbidden' for me. if u uploaded the files zipped (or whatever it hides their 'trivial nature') i could put my hands on (and i would :)

the bests
y

[ChronoCross]

Here you go yaz....all testfiles put into the same rar. let me know if it works for you.

Test Files 1-4 rar (http://www.chronocrossdev.com/encoding/trivial_nature.rar)

[yaz]

@chronocross
many thx ! all files worx. i'm about to analyze them. what i noticed at the 1st glance is that the critical part contains only(!) s-frames. nothing else just series of s-frames. quite strange.
scene changes detected perfectly independently of the settings (might have been wrong about it?)
now i'm going deeper (underground (long live, godzilla!):-))
the bests
y

[Josip Tosic]

Can we have a standalone XviD video decoder once the final V1.1 is released? What with the new post-processing routines, it's looking better and better and since most people simply want to watch video, not encode it...

Now if only someone would add support for DivX V3.11a... :)

[AsTimeGoesBy]

blocks
Belong the posts here blocks may appear in particular when GMC is enabled (i never use it myself...) but i even have seen (with Xvid's own decoder) blocky frames without GMC specially at low bitrates with 2 pass encodings. That's why I think Xvid has a certain block-weakness on (darker) fast motion secnces, not really 'scene wasting' of course - and also v1.1beta is slightly better in that situations than v1.0.3.

trellis & CQM
If i have followed all posts correctly... It is still no good idea to use a custum quantization matrix with trellis enabled?
I hope disabling trellis won't eleminate the benefit of a custom matrix...

[yaz]

@astimegoesby
the blocking issue here ('super-blocking') is sg different than u listed. it's not 'black-blocking' as it occurs in the normal-to-high luma region too. it's not the 'trellis-does-not-like-this-cqm-blocks' as it occurs even when the standard matrices are used.
and i think it's not gmc directly, however it triggers that (see below)

@chronocross (and all interested)
i've checked your files. what to say, it's amazing. u did a lot of things i'd never make but it helped me a lot. clip is resized without cropping so we have decent black bars above and under the video stripe, it is a full q1/2 encode (never seen before :-), aso, aso ... ok, what i found.

stream encoded by 'default settings' shows quite normal frame pattern (ipbbpbbpbb...) but the critical region is full p w/q1. this is a typical anime 'hi-mo' scene where the motion is just 'imitated' by waving slanting lines in the background and by jerking the figure forth and back, up and down in the forground.
when gmc switched on all(!) these p-frames are replaced w/s-frames. (my clips showed sg similar but there were b-frames too)
it's clearly seen that the reference(s) is(are) lost sw in the middle of the scene as there the frame falls apart into super-blocks showing nice but pretty annoying block-patterns. it happens typically when the background and forground moves in opposite direction.
in add, there are clear evidences for block(mv?)-misalignment as some blocks get out of the video stripe showing up on the black bars. within the frames it's clearly seen that blocks are placed to wrong locations, left side appears on the right, aso aso.

qpel and trellis(?) varies the pattern making it a bit finer but super-blocks are there. b-vhq does not seem to be effective (maybe, cus there's no b-frame anywhere)

i also found some decoder dependency too. when encoded w/ffdshow(libav) the block misalignment is more serious than w/xvid, some blocks are not simply misaligned but are deformed to longer blobs and sometimes to long bars reaching the frame borders.

so, the problem can be related to s-frames, really, but i think it's because they are placed there wrong. what i know about s-frames (not too much) tells they are for global(uniform) motions such as scans, zooms, rotations. but this scene isn't any of that. if no b-frame is fitting there why would s-frames do? i can't see any reason to a 'change all these p-frames to s-frames' decision.

would any of the devels comment on these ?

tx
y

[ChronoCross]

@yaz
good analysis. hopefully that will help the dev's because that is a little wierd. The clip is cropped however the original DVD is NTSC non-anamorphic and if I try to change it to anamorphic, it can be done however at different parts of the movie the Credits get cut off as well as some karaoke subtitles that ADV(the creator company) added. so I found it best to leave it in 4:3 pulldown in order to preserve the original DVD form of the movie.

[Heini011]

hi yaz,

what is a s-frame ?? (xvid uses i,p and b...)

greetings

[yaz]

Originally posted by Heini011
what is a s-frame ??
as u see above, i don't know to much about the technique of video encoding, so i don't know how an s-frame is coded but it's the frame-type generated by gmc (global motoion compensation) one of the devels (maybe, syskin) dropped a short explanation about it. use search to find it, or see the faq (but don't expect any tecky-talk:-)).Originally posted by Heini011
... xvid uses i,p and b ...yep ... and s and n frames too.

the bests
y

ps would we expect any answer to the problem, anyway ? i switched off gmc (w/bleeding heart) and i hope the bests.

[Teegedeck]

Now I encountered that bug, too. I decided to re-do the second pass on a movie at a higher bitrate and thus get close to saturation. The blocking occured in a scene that is OK in the lower-bitrate one. Alas, I could not reproduce the bug in a constant-quantizer (quant=2) encode of that same scene. Grrr...

The sequence in question is a rotation, typical GMC stuff, and the sequence was (damaged frames bold, quantizers in brackets):

s(3) - beginning of damaged sequence - b(4) s(2) b(4) s(3) b(4) s(2) b(4) s(3) - end of damaged sequence - i(2)

As usual, I had every XviD gadget activated that is available. SixOfNine. On the input side, I cropped to mod 8, not mod 16. No resizing.

Edit: Notable about the sequence: the rotating background seems alright, but the figure standing at the center of the rotation gets dissolved into blocks.

edited: my English. It still is bad though... ;)

[yaz]

Originally posted by Teegedeck
Now I encountered that bug, too. welcome to the heartbreak hotel! ... what i really don't get how other ppl don't have it.

anyway ... brushing up my mind popped up sg. quite awhile back, we had sg similar w/gmc. that time we had plain black blocks instead of this color mess but the whole resembles pretty much. that prob was solved by syskin within a day (khmm ... i don't want to urge anyone, really :-) is this one so much more complicated ?

i'm checking my encodes made round christmas and i haven't found any super-blocking. maybe, that helps too.

the bests
y

[foxyshadis]

Hmm, I couldn't find this in a search, but I found a minor but annoying bug. When a stats file gets put into a non-existant folder, it'll go through the first pass merrily and uselessly, then die on the second, unable to find anything. I hit it when gordian knot messed with m xvid settings and never reverted them; at most it cost me a half hour of encoding time, but it seems like it'd be an easy fix to die on the first pass instead.

This is the 1.1 beta1 compile.

[ChronoCross]

Originally posted by foxyshadis
Hmm, I couldn't find this in a search, but I found a minor but annoying bug. When a stats file gets put into a non-existant folder, it'll go through the first pass merrily and uselessly, then die on the second, unable to find anything. I hit it when gordian knot messed with m xvid settings and never reverted them; at most it cost me a half hour of encoding time, but it seems like it'd be an easy fix to die on the first pass instead.

This is the 1.1 beta1 compile.

the problem is your using gknot....gknot is not built for the latest beta...the registry keys are different. if your gonna use gknot then use the latest stable 1.0.3. if your gonna use the beta then your gonna have to do manual encodes in vdub/vdubmod.

[AsTimeGoesBy]

I got recently a error message that the maximum number of zones is reached, and if i would like more zones i should recompile the codec...

Would it be possible maybe to (massively) increase that maximum value allow more than 64 zones?
Of would this have any bad conseqences i can't see!?


Generally on low motion movies or videos with long corsssfading the Xvid (but DivX too:)) always uses the maximum keyframe interval and these keyframes mostly are at a quite stupid place, often the frame before looks just the sames. Thus I like zones to force keyframes - this is more efficent in many cases and often gives a better seek point for playpack/forwarding too.

I can imagine that 'more zones' not are a frequently posted wish but if that depends only on a simple values in the code... please set it higher. :)

[ChronoCross]

I think if your using more than 64 zones you should just give up.....nothing needs that many separate zones.

[celtic_druid]

Looks to be as simple as changing "#define MAX_ZONES 64"
Untested build with 512 zones: http://celticdruid.no-ip.com/test/xvidvfw1.1.0beta1512z.7z
If it works ok and you need more, give us a new MAX_ZONES value and I will do a new compile.

[Didée]

Originally posted by ChronoCross
I think if your using more than 64 zones you should just give up.....nothing needs that many separate zones.
Well, I have this 4.38GB encoding of LOTR-FOTR SEE @ 960*528 @ 24fps here, which I encoded chapter by chapter, with all quantizer distribution done manually, through zones. There were quite some chapters i had to break in 2 parts, only because of the zones limit ... :)

Although I'm not going to make such an effort again anytime soon:
Thanks, celtic_druid!

[TripleA]

Well, now: "Never say never". There's always LotR - TTT SEE and LotR - TRotK SEE. Can't have the collection incomplete, you know... Though, perhaps, it would not be trivial to get source material.

I salute your dedication.

BTW, it's been my experience (with the DVDs) that FotR is much more compressible than the others.

[ChronoCross]

Originally posted by Didée
Well, I have this 4.38GB encoding of LOTR-FOTR SEE @ 960*528 @ 24fps here, which I encoded chapter by chapter, with all quantizer distribution done manually, through zones. There were quite some chapters i had to break in 2 parts, only because of the zones limit ... :)

Although I'm not going to make such an effort again anytime soon:
Thanks, celtic_druid!

Next thing you know you'll be coding the movies in binary by yourself Didée lol. I actually think that zones are particularly only needed in real special sections of movies, the internal RD algorithms handle most any situation flawlessly. but if your gonna do a 3 hour movie manually be my guest....all I can say is wow lol.

[ninegoodthings]

Originally posted by ChronoCross
the problem is your using gknot....gknot is not built for the latest beta...the registry keys are different. if your gonna use gknot then use the latest stable 1.0.3. if your gonna use the beta then your gonna have to do manual encodes in vdub/vdubmod.

Actually, it works fine for me... I have the newest 0.34.4beta though.

On a related note, I am also having the blocking problems using trellis, gmc, and the V2 Sharp matrix for AutoGK. Thanks to all of you on the front lines (that know what they're doing) for testing around with it! :)

[Sharktooth]

This version seems to have some problems with trellis and/or GMC.
Anyways it should not be a matrix problem. I succesfully use it with 1.03 and with previous 1.1 builds.

[Didée]

/* OT
Originally posted by ChronoCross
Next thing you know you'll be coding the movies in binary by yourself Didée lol. I actually think that zones are particularly only needed in real special sections of movies, the internal RD algorithms handle most any situation flawlessly. but if your gonna do a 3 hour movie manually be my guest....all I can say is wow lol. You have to know that that encoding was done a while back, with an _alpha_ version of XviD 1.0. Rate control was somewhat delicate (br0ken) at that time ;)

OT */

[yaz]

Originally posted by Sharktooth
This version seems to have some problems with trellis and/or GMC. i'm sure it's not trellis, or at least not in a straight way. i've made several tests and, accordingly, i would recommend not to use gmc (until the next note :confused: ) trellis & b-vhq seem to be ok.
Originally posted by Sharktooth
Anyways it should not be a matrix problem.sure, it's not. i didn't use other than the mpeg matrix in my tests, and i got heavy super-blocking.
the bests
y

[AsTimeGoesBy]

Originally posted by celtic_druid
Looks to be as simple as changing "#define MAX_ZONES 64"
Untested build with 512 zones: http://celticdruid.no-ip.com/test/xvidvfw1.1.0beta1512z.7z
If it works ok and you need more, give us a new MAX_ZONES value and I will do a new compile. Wow, a 512-zone-compilation, great! :)

And if it's already mentioned here... I like to use that compilation to stick all the landscape scenes together of the LotR movies, but since i use long crossfading the codec never sets a keyframe so i like to help a bit by defining up a zone. - Thanks again!

[yaz]

khm ... would we expect anything about this super-blocking issue or must we live w/it from now on ?
thx
y

[CruNcher]

@ChronoCross

did you used Cartoon Mode ?
if yes could you do the same test encode without b-vops ?
if you didn't ignore this

[ChronoCross]

no Cartoon mode. Gave up on it awhile back cause it didn't give any noticeable result.

[*.mp4 guy]

Extremely annoying flickering that is less noticible, but still present the higher the compression is. I don't know how to describe it really so here is a link to a small segment. Also some matrices are worse then others, but all of them display the flickering block things to some extent.

http://putfile.com/media.php?n=example

Edit: Thanks to didee the file now has the correct extension:D

I used one of the latest 1.1 betas to encode this. There was no flickering in the source and I used these settings different from defaults.

Edit: ahh it apears I did not give all the relevent info. It is koepi's First Xvid1.1 beta compile I would be more specific but the installer was sent to the recycle bin long ago. I use ffdshow (once again im sorry but i dont know the version) with no post processing to decode the file.

-adaptive Q
-qpel
-gmc
-no packed bitstream
-chroma optimizer
-highest vhq for everything
-msp6
-trellis

[Didée]

Originally posted by *.mp4 guy

http: //img237.exs.cx/img237/2308/example5tr.jpg

It's really an avi file, just right click it and choose "save link as" and rename it to .avi extension
Please! You should not abuse free image hosting like that!

You can host clips without problem e.g. at www.putfile.com.

[yaz]

just found a new cvs snapshot on celtic's site. would it be the 'super-block-free' release ? :confused:

anyaway, might we expect a decent athlon-xp build too ? ;)

thx
y

[celtic_druid]

core:
- sad_mmx register dependency speed optimisation
- assume that fcode also limits average MV in mcsel==1 blocks. fixes a visual bug caused by different prediction from such MV
vfw:
- set xvid_init_t.debug on decompress_begin()

Build was ICL7.0 so should be fine on an athlon-xp.

[yaz]

@celtic_druid
thx for the info ! but wht's the answer ? :confused: would someone having a 'super-block-prone' clip at hand test this release, pls ?

the bests
y

[TripleA]

The XviD-1.1.-127-13102004 encoded test clip I still have (read previous posts for details, if you want) doesn't exhibit the issue any more.

What that exactly means, I have no idea. But if someone took a look at things and "fixed it", then it seems likely that it is, indeed, fixed.

But I would still recommend some caution, if what you're encoding is anything important.

Thank you very much.

[yaz]

maybe, chronocross would check that gmc-killer clip. ;)

the bests
y

[ChronoCross]

checked the clip and it doesn't seem to be happening anymore. which is a good thing. YAY xvid

[yaz]

so gmc seems to be back again. (amof, i've been missing it very much) chronocross, pls, would u make your new encodes also available, d'ye know, packed only that short part w/super-blocks.
thx
y

[ChronoCross]

I'll do so as soon as I have time. Perhaps late tonight or sunday. I'm headed out of town for the weekend tomorrow morning(saturday)

[ChronoCross]

Here it is: Xvid latest Head build encoding test.

BVHQ, Trellis, GMC, qpel
Successful Clip (http://www.chronocrossdev.com/encoding/test(GMC,qpel,Trellis,BVHQ).avi)

YAY Xvid.

[Chainmax]

Hopefully Koepi will make a new build out of these fixes :).

[ninegoodthings]

Can someone explain to me the difference between the CVS head and any release Koepi would make?

[ChronoCross]

the CVS head is basically all changes up to the point of compilation. or the latest version.

Keopi will usually only release offcial beta's or test versions. I've never seen him make daily builds. He'll probably wait to make a new build till Beta2

[Koepi]

I was doing "daily builds" some years ago, but then I got too deep involved into a) development of XviD and b) work, so there's less time i can spend on this. Also, I'm not capable of giving as much support as I gave back then - so there are my reasons for NOT giving out daily binaries.

Beta 2 shouldn't be too far away now.

Cheers
Koepi

[ninegoodthings]

Great to hear, thanks for all your work Koepi.