XviD-05042003-1

[Koepi]

XviD-05042003-1:
- Fresh CVS checkout.
- sysKin has made refdivx' lumimasking P4-compatible
- the same applies to VHQ
- some speedups by sysKin, he replaced ABS-macros for better compiler optimizations
- inspired by Cruncher: removed fdct_sse2, he suggested it produces worse quality
- still no negative thresholds for bframes
- statsreader updated to 2.0a (merging stats support)

Thanks Cruncher for all the useful hints.

Best regards
Koepi

[JagPanzer]

Koepi, thanx for a new build. : )
I will do some tests now.

[Didée]

>> statsreader updated to 2.0a (merging stats support)

Mmmmmmhhhh...
that's really nice. "Bundesverdienstkreuz" for Koepi!

[Lefungus]

Thanks Koepi and all others xvid developers. I'm a P4 user and xvid has been buggy with these cpus for a long time. Well i'm now able to test all its features again :)

[jarthel]

I assume this build has the same changes as umaniacs. right?

The problem with Umaniacs changelog is that it's too technical. Unless you are developer, you wouldn't have a clue.

Thanks

Jayel

[Koepi]

Sorryx, but what's not to understand about XviD-05042003-1:
- Fresh CVS checkout. ? Of course it has the same changes - and more.

Sorry if it's too technical for you. But unfortunately the changelog is from developers for developers.

Regards
Koepi

[Sharro]

A long thread with just thanks is of no use to anybody, still... not thanking all the work you guys do is rude.

Thanks a million Koepi, Syskin and Cruncher!!

Sharro

[Didée]

@ Koepi

Just an idea:
In addition to the new "merge" support of statsreader, a possibility to "save range" of a statsfile could come handy.
E.g. you made a 2-pass encoding, and then find that several scenes apparently could've needed more bitrate ... then you could build a "cusomized" statsfile with not too much effort.

(Yes, in my mind there is floating something similar to Nandub's stats editor, or even divx's EKG application. Please don't flame me ... I simply like to do some things manually, on occasion)

Didée

[Koepi]

Didee,

that's a good idea indeed. If I find some time (damn work all the tme :( ) I'll implement it.

All:

Thanks for the thanks :)

Regards
Koepi

[gotaserena]

It would be a dream to have a CCE-like interface to make judgments over bitrate distribution. A bit far-fetched, but a dream nevertheless.

Good build, koepi, hasn't crashed with me so far. I hope that sse2 comes back soon.

[kilg0r3]

high as it seems xvid does not insert i frames at the correct position. i just skipped through a movie looking for a good cut point. the iframes i could find always were positioned in a range of 1 frame before to 6 frames after the actual scene change. I checked this in vdub and with ffdshow. furthermore, when loading the statsfile into vdubmod, i could see that the iframes reported internally by vdubmod were not at the same position as those reported by the stats file.
http://www.mynetcologne.de/~nc-allgeife8/img/capture.jpg
however, the iframes in the stats file also were not located at the correct positions. at one occasion e.g. the iframe in the stream was the frame before the scene change, while the stats file reported an i frame one frame after the SC.

btw, is it save to use packed bitstream?

The stream i used for this testing was the first-pass of an encode which i have done with the following settings. this might explain some of the problems but not all. i will do the second pass today.

XviD BUILD (05.04.2003 18:17:50)
Mode: 2 Pass - 1st pass
Discard 1st Pass: OFF
Motion Search Precision: 6 - Ultra
Quantization Type: MPEG
FourCC Used: XVID
Max I-frame Interval: 400, Min I-frame Interval: 10
Lumimasking: ON
Interlacing: OFF
Greyscale: OFF
Max B-frames: 3
B-frames Quantizer Ratio: 126%
Packed Bitstream: OFF
DX50 B-VOP Compatibility: ON
Print debug info on each frame: OFF
Min I-frame Quantizer: 2, Max I-frame Quantizer: 31
Min P-frame Quantizer: 2, Max P-frame Quantizer: 31
Start Credits 0-589, End Credits: 135131-140772
Encode credits in greyscale: OFF
Credits I-frame Quantizer: 20, Credits P-frame Quantizer: 20

[sungey]

hmm my encode without bframe/gmc/qpel with this build has the I-frame inserted correctly.... (at the scene change)...

[kilg0r3]

in a second attempt, a partial encode of 1500 frames with _enabled_ option 'packed bitstream', the iframe seems to set at the right position now. yet, the divergence between stats file and video stream still persists.

therefore, again, is it safe to use packed bitstream?

[Koepi]

Erm. There wasn't mentioned that 2nd pass dynamic IPB-decision is fixed yet. Thus you can insert whatever you want into the statsfile, the frame_type_ won't be respected.
This is not the case when disabling bframes (max bframes: -1) as there the good old working code still is used there.

The difference between the stats and vdub-reported frame-positions is due to padframes at the beginning of the stream, internally xvid handles that correctly.

Koepi

[kilg0r3]

Originally posted by Koepi
Erm. There wasn't mentioned that 2nd pass dynamic IPB-decision is fixed yet. okay, this one of those bugs that had eluded me:)

The difference between the stats and vdub-reported frame-positions is due to padframes at the beginning of the stream, internally xvid handles that correctly.

Shouldn't the diffrence be constant then? Well heck, it's working:)

btw i managed to cut the stream properly. i don't know why, but it was even harder to find the right position this time. this cutting issue really is a pain in the ... But that's not really an xvid issue, is it?

cheers and sorry for the fuzz.

[drebel]

@ kilg0r3
I know i'm getting OT now,but i couldnt avoid noticing some "strange" values in your settings

- In Xvid sticky there's a suggestion of Unfilter(3,3).I thought the filter goes in steps of 5(0,5,10 etc.(Is the nearest value used?)
- B-frames Quantizer Ratio: 126% .I thought it goes in steps of 50,but i might be mistaken .Pls correct me if i'm wrong :)

regards,
george

[kilg0r3]

(1.)
@drebel
i am just guessing, but i don't think there is something like a 50% step.

(2.)

Just to make a bit more noise ;). Cuold somebody explain me what a message 'BrokenB-Frame, TPP=0 TBP=2' means. I get it when seeking frame-by-frame in a stream encoded with the above mentioned settings.http://www.mynetcologne.de/~nc-allgeife8/img/Capture2.jpg

[Selur]

Little bug report:

VHQ>1 + qpel causes flowting walls
(VHQ)>1 on it's own doesn't)

I know this has den reported earlier, but then VHQ>1 itself caused this, but now it only apears when vhq&qpel are enabled.
So could it be that qpel is not acurate enough, and we need something like 'simple qpel' ;)?

Cu Selur

ps.: @kilg0r: got that Broken-B-Frame-problem, some time back, with an earlier version of VirtualDubMod, but it doesn't apear with the current version.

[kilg0r3]

@Selur
I am using the latest version of vdm, strange.

@All

I have compiled a little collection of clips i encoded with different combinations of bframes vhq and qpel. Have a look (http://www.mynetcologne.de/~nc-allgeife8/downloads/material.rar) and judge yourself. The encodes are done with constant quant 2. When using bframes the settings were 3,BFR 100,BFO 100. These are very mild settings not stressing the codec very much.

The file is about 2mb. I recommend looking at in vdm at 2x zoom.

[Selur]

tried your clips and for me it gives the same error and I'm able to reproduce this with clips I encoded myself,.. (same error if you try it with VirtualDub and an avi file)

Strange, now I'm confused. I'm sure I had this problem before and it disapeared, but maybe I just didn't scroll so much in clip I used B-frames and VHQ,...

Cu Selur

[kilg0r3]

Perhaps it is specific to the resent xvid builds or(!) to not using DX50 compatibility mode. When i seek through a movie with a player, the playback starts with a broken frame (incoherent blocks). I cant describe it more exactly because it is visible only for a fraction of a second (presumable 1/25 sec :D).

Anyway for me it is not a problem, just being curious.

[Prettz]

What is the correct term for when, during fades (particularly fade-ins), bright areas in the video appear to expand/contract or "bulge", rather than fading in or out normally?
Xvid is the only codec that shows this problem, and even then I only see it when decoding with ffdshow; when I check "Use Xvid" in ffdshow (I'm using Koepi's latest Xvid build) the problem disappears.

[PowerMacG4]

YAY. B-Frames successfully stay disabled in this build.

[gruntle8]

Prettz: I may be wrong, but I believe that's a lumi masking artifact.

[Sharro]

Originally posted by Prettz
when I check "Use Xvid" in ffdshow (I'm using Koepi's latest Xvid build) the problem disappears.

Errr.. Wouldn't you say ...that this is a decoding problem not related to Xvid?

Try ffdshow forum...OR... Decode with xvid.dll

I think...

All the best,

Sharro

PS: I'm quite sure I read something here (or somewhere else) about ffdshow latest alpha not being able to properly decode some features of most recent Xvid builds. But I'm nobody....

[Prettz]

Originally posted by gruntle8
Prettz: I may be wrong, but I believe that's a lumi masking artifact.
I didn't use lumimasking in any of the xvid encodes where I saw the problem.


Originally posted by Sharro
Errr.. Wouldn't you say ...that this is a decoding problem not related to Xvid?

I mentioned it here because this problem only appears with xvid files, and it appears frequently in them (decoding with ffdshow, at least).

[sungey]

Prettz .. got a clip ?

[Koepi]

Turn off the automatic workarounds in ffdshow and try again decoding xvid with it. You may find that ffdshow is a bit outdated in the meantime.

Regards
Koepi

[kastro68]

I've compared several encodes of Hero using these two builds, I spent lots of time comparing the several scenes over and over.

My conclusion is that I think the 17-02-2003 build gives slightly better results than the 5-04-2003. It is still very hard to tell the difference between the two unless you watch the same scenes over and over and over.

Using 3 b-frames, vhq1, qpel, chroma me, everything else on default.

for 5-04-2003 I used a b-frame threshold of 50.

The 17-02-2003 build gave a first pass size of 1156 and the 5-04-2003 gave a 1st pass size of 1683.

Just and observation that I thought I should share.

[iago]

@Koepi

After a long long time, heya again my friend! Recently, time has really become my enemy! ;)

Now I'm searching through the threads, gathering information, and doing some tests with your new build, feeling a total newbie! ;)

ciao,
iago

[Didée]

Ahoy Iago!!!

Glad to see you back here - I was already worrying what might have happened.
As for encoding, you may relax: we're still using mpeg-4 techniques ... ;)

Didée

[Koepi]

Merhaba iago!

I'm happy to see you back on the xvid-track! ;) I hope there didn't happen too much in the meantime, I can't remember huge steps we made at least. Except for bframes are of amazing quality now, no flickering anymore, we use SimpleIDCT now, sysKin added VHQ modes,...

Looking forward to your impressions :)

Best regards dostum!
Koepi

[Sharro]

Iago,

Good to see that email kind of "awake" you!!!!!

Where have you been? Damn Life who doesn't leave us proper time for things we like eh?

All the best,


Sharro

PS: @koepi I asked Iago to show up on irc!!! Hope he has time for it.

[JasonFly]

I didn't use qpel very often but i did some tests in the lasts weeks and I saw some artifacts which seems to be related to qpel.

I have tested with différent config(bframe+qpel+vhq,Bframe+Qpel without vhq, qpel alone....) and the only clips which had some artifacts were those were I used qpel.

The artifacts I m talking about are the those:
Some parts of the image seem to "slide" and that's not very beautiful.This appears more otfen in movies with noise but also in others.
This also happens when camera make some "reverse" movement(it moves up and then it moves down for example)

This phenomenon is more visible in high quant(even in q=6, the result is very ugly).

I haven't read all xvid threads (or I have read them but don't remember such a comment on qpel) so I not sure if this problem is already known or if this only happens with me.

[cca]

I also have noticed the sliding problem JasonFly mentions. Maybe there is still bugs with Qpel.

[Sigmatador]

Originally posted by JasonFly
Some parts of the image seem to "slide" and that's not very beautiful.This appears more otfen in movies with noise but also in others.
This also happens when camera make some "reverse" movement(it moves up and then it moves down for example)

This phenomenon is more visible in high quant(even in q=6, the result is very ugly).

i notice this problem too

[BoNz1]

What are you guys using to decode your movie? I was using an older version of ffdshow which had this problem but I switched to Nic's directshow and now it is gone. If you are reporting a bug you should really give more info on how you are decoding since it is quite possible this is the problem.

[Sigmatador]

i know how to decode a xvid ^^:
ffdshow20030103 with "xvid" and "use xvid" enabled (and xvid for idct)

but right i didn't try last nic's dshow, i'm going to ^^

but i notice this problem too with a divx502 movie (with divx502 dshow decoder, but worth than xvid ^^) so i think i'm not a qpel fan (well, most of my encode are anime so it's really useless ^^)
on a movie with h.263 matrice it's not really noticiable, it's a perfectionnist critic ^^

[drebel]

i know how to decode a xvid ^^:
ffdshow20030103 with "xvid" and "use xvid" enabled (and xvid for idct)

but right i didn't try last nic's dshow, i'm going to ^^

I'm sure you do.But to eliminate qpel smearing problem u must have the latest alpha ffdshow 18.04.04 by athos and use simple IDCT...
...or Nic's decoder ;)

regards,
george

[cca]

Well, I AM using the latest alpha ffdshow 18.04.04 by athos with simple IDCT selected! The sliding problem is there...

[JasonFly]

The problem also exists with nic latest dec build.I usually decode with Nic's decoder.I havn't tested with ffdshow.

Does ffdshow really is better than Nic's decoder or is it just better for qpel decoding that there is a difference?

EDIT
I have just tested with the latest ffdshow build and the result seems to be good(no more "slidding" effects in the two movies I had noticed this problem.

So far, I didn't found any interest in ffdshow over Nic's decoder but qpel decoding is definitely in favour of ffdshow.
I haven't tested with older builds of ffdshow but Sigmatador said that this didn't work well with them too.

[Sigmatador]

Originally posted by drebel
I'm sure you do.But to eliminate qpel smearing problem u must have the latest alpha ffdshow 18.04.04 by athos and use simple IDCT...
...or Nic's decoder ;)

regards,
george

i'm going to test with the 21.04.04 one because 18.04.04 doesn't work properly on my computer (2k adv server).

[wotef]

unfortunately, i agree - still get qpel smearing with the newest ffdshow - and btw, doesn't "use xvid" just give the decoding job back to the codec? i.e. you guys are getting different results simply cause you have different xvid binaries installed, i.e. using different xvid.ax files

for now, i'm going to use the old trick of installing newer binaries, but overwriting my xvid.ax with nic's 16-feb version; it has the best qpel decoding

[Menion2k]

Koepi this build crashes between the first and the second pass with Virtualdubmod v2, as one older build (I don't remeber exactly the build, but there is a thread about this). Tested with Virtualdubmod v2 (latest binaries), video imported via avisynth 2.5.1 (latest build), Athlon XP 2700+, WinXP SP1 Italian, 512Mb RAM. The different xvid settings don't seem to influence the crash. Bye!

[Koepi]

Menion,

please check the location of your stats-file: is the space for it sufficient? Does the path exist?

You're the first one to report this behaviour, so it's unlikely to be a xvid bug.

Best regards
Koepi

[Menion2k]

stats file is in D:\, the space if more than sufficient and it was created succesfull, in fact the first pass was correctly completed. Now I restarted the second pass, and the encoding is proceding correctly, later I'll post the result. Bye!

Update: The second pass was succeffully created!

[InfoCynic]

When I was home for Easter, I saw this behavior at my friend's house (Athlon XP 2200+, WinXP SP1, AviSynth 2.5, VDubMod latest). VDubMod would throw an error & crash after each first pass. It was after the pass, because the stats file was complete and the second pass completed fine after that. I'll see if I can get him to post here with more details. He would have been using XviD-22032003-1.exe.

[PerCIVaL]

are you using the joblist?

With virtualdubmod and said koepi build, vdbmod crashes after the first pass (first job) finishes and the second pass (second job) is about to start with me.

Same with divx 5.04 btw.

[InfoCynic]

That was using GKnot 0.28 beta 3, IIRC. (Which is to say, no, b/c GKnot does not use VDM's job list.)

[Koepi]

And I always thought GKnot setups VDub via the joblist because that's what it's there for.

Please take a look at the vdub forum and see if it is somewhere mentioned (some internal variables changed versus original vdub, thus it might be that gknot messes with the wrong ones).

And: do your encoding properly, without one-click-and-go-tools. As you can see, they aren't adopted to the new builds of the used tools and can't unleash the full features available, or mess up otherwise.

Still no xvid bug. It's gknot or vdubmod which are failing.

Koepi