My XVid binaries

[Nic]

Hi, I know I shouldn't really post the binaries but:

http://xvidbinary.20megsfree.com/

I set the codec to use the XVID decoder rather than the DIVX DShow decoder & Ive fixed the bug in the CVS that causes loading/saving the stats problematic (as an open dialog box would open rather than a save)

Hope this helps testing & evaluation of the codec. (sorry for all those cr*p banners from the poor hosting)

Cheers,
-Nic

ps
If Doom9 or anyone is really against the posting of links to the binaries I will take them down (as well as the actual files).

[aleksander]

It doesn't work. I cannot download.
I get the following message:

Forbidden
Remote Host: [213.25.33.98]

You do not have permission to access http://xvidbinary.20megsfree.com/XVID.ZIP
Data files must be stored on the same site they are linked from.

Thank you for using default.partner


Did you take it down already????
I want to download them since this open/close stats file bug is making me crazy....
take care
aleksander

[Nic]

Im on it, they should be there, it seems like a cr*p host but I cant get access to my normal dumping ground, hold tight. Ill get back in just a sec.

-Nic

[Nic]

Try here:

http://xvid.stormpages.com/

This one is picky too, but it should be better (remember to left click no right click to download)

Cheers,
-Nic

[Nic]

Would people prefer I reset it back to using the DivX Decoder? Rather than the new XVID one?

-Nic

[Teegedeck]

...why don't you just include TWO .inf-files; one that install the DShow and one that doesn't?

[Teegedeck]

...and don't forget to include an uninstall.bat, too, for those that change their minds... ;)

[Nic]

I didn't realise the xvid decoder also accepted DivX fourcc as well as having such a high merit.

However, I think that the xvid filter only accepts the XVID mediatype. So it wont play divx files, I could change this but things could get confusing because then it would be used for both xvid & divx.

So I'd rather have one or the other, then again people can always set the fourcc of the AVI files with AviC.

plus, the inf file that comes with xvid doesnt install the filter, so I dont want to change too many things.

-Nic

[Doom9]

isibar explicitly told me not to use the xvid ds filter for my comparisions.. it's not ready yet.. hence I would suggest you default to the divx fourcc

[Nic]

Thank you for the info, ill change it back to using the DivX fourcc.

Cheers,
-Nic

ps
Now changed back! :)

[Koepi]

I can only agree - the xvid DS filter (for playback) is unusable as it hasn't any controls for brightness/contrast/saturation/... - it's not using overlay as my NVIDIA drivers don't allow me to adjust those values as well on xvid playback... so all the playbacks via that filter are way too dark for me :)

I hope this helps.

Btw., thanks for the binary, helps me not to get into the trouble of compiling it myself ;)

Regards,
Koepi

[Teegedeck]

@Nic: thanks for compiling the sources! Perhaps you should include a message on your site that download will only work with IE. (Yes, there are people actually using OPERA! ;))

[Koepi]

@Tee:

Make it "Download NOT working with opera"!
I use mozilla (most recent nightly builds) and it worked like charme. If you want to, i can test lynx, links and wget as well, but I don't think it's necessary!

Btw., is this browser-thingy not a little Off-Topic? ;)

Regards,
Koepi

[Teegedeck]

@Koepi: BTW, nice to see you here! When I look at the topic 'My XviD binaries', I think that my posting wasn't too off-topic. But I'll try to adhere to forum rules a bit more, now.

Hope you stick around.

[Nic]

Latest version compiled:

http://xvidbinary.20megsfree.com/

or

http://www.stormpages.com/xvid/

Cheers,
-Nic

[MaTTeR]

@Nic

Thanks so much for the binaries. I just installed your latest 01-28 build but the about box in the config window is showing 01-27. Anyone else see this?

[Nic]

When I state the date, that is the date on which I got the source from the CVS and compiled it. Which was this morning (28/01) at about 9:00am GMT :)

Ill update it frequently... :)

Cheers,
-Nic

[Koepi]

Hi again,

I tried using the 1 pass CBR mode - but it doesn't start encoding. The best result until now I got was a division by zero error (the build from yesterday didn't even react to anything, made vdub crash... i had to use the task manager to kill the process...).

Any ideas where this comes from?

And btw., if the XviD developers are around here as well: I posted a new topic on the powercoding.de general devel forum about some (de)interlacing involving something fancy like fake-re-encoding ;-) You may want to look at it
SCNR *hide* :)


Regards,
Koepi

[Teegedeck]

Hello Koepi,

sorry, never ever tried CBR... -h could perhaps help, but won't be online for some hours. I guess you'd better try it over at Isibaar's place...

[EDIT:] ...that's what he told me anyway :)

Till then

Tee

[-h]

I tried using the 1 pass CBR mode - but it doesn't start encoding. The best result until now I got was a division by zero error (the build from yesterday didn't even react to anything, made vdub crash... i had to use the task manager to kill the process...).

Something has.. gone wrong. Looks like a core issue too..

And btw., if the XviD developers are around here as well: I posted a new topic on the powercoding.de general devel forum about some (de)interlacing involving something fancy like fake-re-encoding ;-) You may want to look at it
SCNR *hide* :)

Couldn't find the web page?

-h

[Koepi]

Sorry, my fault, I didn't think before writing. Mixed up something I read in some threads here before with the thing I was actually doing - i meant videocoding.de of course.

Regards,
Koepi

[-h]

Oops.. dead CBR seems to be my fault ;)

I won't have CVS access for 10 hours or so however.

-h

[-h]

@Koepi - can you try CBR encoding with lumi masking disabled? There seems to be an issue with the new lumi masking code, and CBR mode.

Edit: scratch that, I have to change something in the CVS to allow CBR encodes. Lumi masking issue is still there though.

-h

[Koepi]

You're right, reproducable here. Without luma masquerading it works! :-)

Thanks for the hint :)

Regards,
Koepi

[Nic]

Latest build 30/01/02 is now at:

Main:
www.freewebz.com/xvid

Mirror:
xvid.stormpages.com

Cheers,
-Nic

ps
(http://xvidbinary.20megsfree.com/ -> Not updating any more!, will be closed)

[Nic]

The reason the about box reads: 27/01/02 is because the code uses __TIMESTAMP__ inside config.c, which writes the time for when config.c was last changed, not the last compilation. Ill recommend to videocoding.de that they change this to __TIME__ or similar.

-Nic

[MaTTeR]

Originally posted by Nic
....Ill recommend to videocoding.de that they change this to __TIME__ or similar.

-Nic

Thx Nic. This would be most helpful. I sometimes lose track of which build I installed last;)

[wing1]

-> Koepi

how did you get cbr to work. I've tried both with/without lumina masking option, and both crashed immediately with vdub1.48 on win2K o/s using AthalonXP 1900+. xvid produces great quality if you use 1-pass quality encoding but the bitrate climbs over 300Kbps, while the quantizer 1-pass method drops the bitrate way under 90Kbps and it produces way too much block noise.

-> nic
thanks for sharing the binaries :)
btw..the new 1/30/02 binaries does play back divx4.12 movies without 4.12codec installed.

[Nic]

Cheers for the heads up, ill look into both those points & get the next CVS version soon (which will hopefully get these things fixed (or maybe ill get chance to address them this weekend)

-Nic

[Koepi]

Wing1, did you try to disable mmx optimization as well on the advanced tab? IIRC that was another option next to disabling luma masking making 1 pass CBR work again...

I hope this helps,

Regards,
Koepi

[-h]

CBR encoding without lumi masking should be working with the current CVS, with or without MMX/MMX_EXT optimizations. If it doesn't... then yuck.

I'm not sure if Isibaar is onto it yet, so I can play this weekend.

-h

[Nic]

It works without Lumi on mine (Celeron 1000)

-Nic

[Nic]

Latest build 01/02/02 is now at:

Main:
www.freewebz.com/xvid

Mirror:
xvid.stormpages.com

This version may well work with MPEG2AVI.

Cheers,
-Nic

[-h]

Before the latest CVS update, MPEG2AVI was crashing for me when attempting a 2-pass encode in YUV colorspaces. It didn't crash for any other codecs, like DivX4, but the output avi was corrupt.

After the last change, MPEG2AVI no longer crashes when XviD attempts 2-pass mode. But the avis were still all corrupt :)

I assume that, if it worked for people with DivX4, it'll now work with XviD too.

I have no idea what I did to corrupt so many avis (raw frames, huffyuv, divx 3.11, divx 4, indeo, mpeg2avi corrupted them all on my computer).

-h

[Nic]

Just in case anyone thought I was slacking :) The reason I haven't put the latest build up is because I have heard that it is having quite big problems with Pink & Green blocks.

As soon as this is resolved Ill put up the latest version,
Cheers,
-Nic

[-h]

Blah I've been too busy to do much of anything lately. Weird block issue seems to be the new 3DNow! interpolation code (just disable 3DNow! optimizations in the CPU config - the only change is that you'll go back to using MMX interpolation). Haven't been able to test to make sure though, not sure if CBR is still broken either.

-h

[Nic]

Busy? Tell me about it :)

Latest build 06/02/02 is now at:

Main:
www.freewebz.com/xvid

Mirror:
xvid.stormpages.com

Changes:
This version has a bug in the 3DNow! interpolation code, so if your using AMD turn 3dNow
optimisation off.

If using the MPEG quantizer then change fourcc to XVID so that the XVID decoder is used,
This is because of a bug in the DivX decoding code.

Cheers,
-Nic

[rui]

Originally posted by Nic

Latest build 06/02/02

Changes:
This version has a bug in the 3DNow! interpolation code, so if your using AMD turn 3dNow
optimisation off.

If using the MPEG quantizer then change fourcc to XVID so that the XVID decoder is used,
This is because of a bug in the DivX decoding code.


I have an AMD XP. In comparison with the 2/02/02 build, will i get any speed gain, since i am turning 3dnow optimization off and going back to mmx, like -h said above?
If no, i think i will wait for the next one ;)

[Nic]

Probably your best bet, hopefully this will be resolved soon :)

-Nic

[easyfab]

With my athlon I encode with 3dnow! on .
In DivX decoder, there are the problems you said but when i've changed to XVID decoder (fourcc to XVID ) there aren't any problems ?
Is the Xvid encoder the problem or the Divx decoder Which not support the new version of the codec ?

[Nic]

The problem with the 3DNow code has now been solved. Ill compile a new version & put it up tommorrow morning (then ill post here)

As with the decoders, the DivX decoder has problems decoding files that were encoded using the MPEG quantizer (rather than the h.263 one). The problem with decoding the MPEG quantizer is a problem with DivX compatability (not xvid).

-Nic

[rui]

Originally posted by Nic
The problem with the 3DNow code has now been solved. Ill compile a new version & put it up tommorrow morning (then ill post here)
-Nic

These are really GREAT news :)
I will be looking forward for tomorrow's build :cool:

[Nic]

Latest build 07/02/02 is now at:

Main:
www.freewebz.com/xvid

Mirror:
xvid.stormpages.com

Changes:
Fixes the AMD 3DNow! Interpolation bugs, i.e. Not so many Pink Blocks

Cheers,
-Nic

[-h]

Sounds like the 3DNow! issues have only been 'partly' resolved.

-h

[rui]

These are not so great news :)

[-h]

New bugs are always good news! It shows you that the developers haven't given up yet.

.. and that they still hate testing :)

-h

[ChristianHJW]

:lol: ...

[MoonWalker]

How do u change the fourcc? With the prog that Doom9 has??

Thanks,
MoonWalker

[Nic]

Its mentioned in the readme.txt

the program is called AviC (you may have to goto the main software page)

it does the job very nicely.... :)

-Nic

[wing1]

down load the new build and did noticed a few pinkish/purplish area during play back( using DivX4.12 decoder ). These area only show up on a few frames after scene changes especially in the high contrasted area. I used H263 quantizer with motion estimation set at 5, while encoding with 1-pass CBR mode @ 1600kpbs. Did noticed a slightly sharper video with this version. I also had 3D-Now and 3D-Now2 options turned on for CPU optimization setting. I will turn it off and re-encode to see if the pinkish/purplish will go away.

[Nic]

Thanks for the info :) Once you've re-encoded without 3DNow! perhaps you could post your results of how it went here & at www.videocoding.de :)

Cheers,
-Nic

[wing1]

Nic,

The result got worst. Now the purple is more prevelent then the pink. The discoloration issue can be seen in the character's faces more pronounce. My processor is an Athalon 1900+ XP. MMX, SSE integer, SSE , and SSE2 are the only thing that's on. I've turned off both 3Dnow and 3Dnow2 off. Again Divx4.12 filters and Xvid were used (remove Divx4.12 codec from system to use Xvid only).

[rui]

Originally posted by wing1
Nic,

The result got worst. Now the purple is more prevelent then the pink. The discoloration issue can be seen in the character's faces more pronounce. My processor is an Athalon 1900+ XP. MMX, SSE integer, SSE , and SSE2 are the only thing that's on . I've turned off both 3Dnow and 3Dnow2 off. Again Divx4.12 filters and Xvid were used (remove Divx4.12 codec from system to use Xvid only). [/i]

You are choosing manually the cpu optimizations, correct?
Don't choose SSE2, if you have an AMD Athlon XP, because it doesn't support those.

[Nic]

Hopefully all problems will be solved very soon, also post very detailed accounts at www.videocoding.de to help the developers.

Also when you post a problem could people post what quantizer they are using (i.e. MPEG or H.263). Thank you very much.

Unfortunatly if a fix does come out, I wont be able to update the site until first thing monday morning (im a bit ill at present :-(

Cheers,
-Nic

[NoLogo]

First, excuse my poor english, not my mother tongue :(

There seems to be a problem with XviD: with some videos, XviD allows too many keyframes, which look much worse than P-frame, and I didn't find how i can configure this in VDub.
My question is: can this be solved, and if yes, will it be in the future ?

Second: I heard that DivX 4.5 is going to use P-Frame and B-Frame (KPPBBKPP), a soltion which is lighter than KPPPPKPP), will it be used with XviD too ?

PS: Great job :)

[rui]

Originally posted by NoLogo
First, excuse my poor english, not my mother tongue :(


I wish i could type in english the way you typed that :)
I have to use MS Word's dictionary to correct some errors ;)

[-h]

There seems to be a problem with XviD: with some videos, XviD allows too many keyframes, which look much worse than P-frame, and I didn't find how i can configure this in VDub.

What kind of source are you encoding from? XviD's keyframe decision kicks in when more than half of the image changes. I've noticed many keyframes that it misses with this criteria..

Second: I heard that DivX 4.5 is going to use P-Frame and B-Frame (KPPBBKPP), a soltion which is lighter than KPPPPKPP), will it be used with XviD too ?

B-frame decoding is working, but the encoding is much more complex than I-P only. Disable B-frames in TMPGEnc to see the kind of speed hit it will bring.

-h

[NoLogo]

-h:

I encode from an .avs (given by GKnot, but first from .vob), but it doesn't occur everytime. Maybe the video is too complex and forces VDub to use so many KF...:confused:
Maybe i'll post a pic of my frame graph, so that you can see what i mean.
But i don't now exactly where it comes from, not a true specialist at all :(
Wouldn't it be possible to have, with XviD, the same correction for KF than in NanDub, which allow to correct the problem by the "min kf interval" command ?

rui:

I use a dictionnary too ! ;)

[-h]

But, is it anime? High-action video clips?

The sad fact is, I-frames are more compact than P-frames in certain situations.

There are going to be further enhancements to motion estimation (for fades, etc.) so this may not be an issue forever. I guess a min-kf-interval could be implemented as a temporary measure.. but usually the codec knows best. If half the blocks in an image are being coded as intra, odds are the frame is better off as intra.

-h

[rui]

Is there any difference between Nic's and uManiacs XviD builds??
Are they compiled using the same software, and the same optimisations? I mean could be that one of them uses a compiler that makes better use of Intel processors than AMD ones?

P.S. Last night i went to the local cinema see a movie called Black Hawk Down . The movie has 2H25 duration, but about 1H30 is all war, with fast combat images, both during day and night period. It will be a real hard test to XviD or Divx when the DVD comes out ;)

[Nic]

Have you noticed a difference between our builds? (when they are done on the same day of course)

Both me & uManiac use VC6 + DX8. I dont if he has the VC SP5 + latest ProPack installed, but I do.

Ill try compiling in VC7 next week, see if that helps/hurts speed & stability.

Cheers,
-Nic

[Koepi]

EDIT:

Question:
Hm. What do I need next to the DirectX8.1 C++ SDK to compile XviD?

Answer:
Nothing. Add following dir to "Extra"-"Options" - "(Include) Directories"

E:\Programme\Microsoft Visual Studio\Common\DXSDK\samples\Multimedia\DirectShow\BaseClasses
Or wherever you installed the DX 8.1 SDK.

I hope this helps :)

Regards,
Koepi

[NoLogo]

-h:

The movie was an action one, called "Warriors" (it's about UNO soldiers in Bosnia), with some fast scenes but also slow ones.

Other fact: I tried to recompress with the binary from the 7/2, et I got superb pink uniforms, what didn't occurs with the former binaries (not with the 30/7). I have a PIII, so I used detected optimization (MMX, SSE, integer SSE). Everything's nice but pink...

[rui]

I can confirm NoLogo words.
I just finished an encoding using the 7/02 Nic' build, the movie is Independence Day. I used H.263 quantization, motion precision search 5 and luma in the second pass. I used the internal second pass.
I got pink colours right in the beginning of the movie, when the alien space ship is passing the moon.
I am returning to the 2/2 build (i believe that this build and the 30/1 one are the same, just 30/1 was compile by uManiacs and the 2/2 was compiled by Nic), because with this one i made 3 movies with no problems.
Hope you guys can fix this pink problem.

Nic: No, i didn't notice any differences between your build and uManiac's build. Just wanted to know if could be possible that you two choose different kinds of optimisations.
The question first came to my mind after seeing this site :
http://www.geocities.com/avihpit/xvid/cpu-opt.html
This site was posted by avih in this thread : http://forum.doom9.org/showthread.php?s=&threadid=15798

[Nic]

Thanks for the links rui! Interesting stuff....

@Koepi:
Didn't quite know what you were on about :) If you were trying to help people compile the xvid dshow filter then great. :) (obviously I already knew how to do that)

Ill put up a new build on Monday, hopefully things will be better, if not ill also put up a link to the 2/2 build on the site for now.

Cheers,
-Nic

[Koepi]

Nobody warned me that I'm dealing with C here instead of C++... well, just a last thing to figure out (how to read out the value of a radio-button correctly) and I've finished what I wanted to do first.

Until now I've done only very easy things:

In 2pass mode I cleaned up the debug output. It now shows something like "string:value string:value..." so that it's possible to mark intra-frames with debugview :)
Another thing I'm still fiddling with is a radio-button with which you can choose whether the FourCC of the encoded movie should be DivX or XviD... nothing where we don't ave tools for, but I thought it's a nice gimmick :)

Maybe I'll put up binaries later, it's a snapshot from tonight (9.2.2002, ~3:30h CET) (just if there is interest...).

Regards,
Koepi

[cofferscuffs]

An interest is here....

[Koepi]

@Nic:

Sorry, that wasn't meant to offend you. It's just a general thing I found out now ;) And I think there might be some other people trying to compile XviD themselfes and maybe they get stuck in the same corner.

(before editing it was just a question, I wanted to indicate that I found the solution).

Best regards,
Koepi

[NoLogo]

I got my pink colors with:
MPEG quantization, motion precision search 5, no luma in second pass.
But I use DivX4 to decode XviD, maybe there is a dedicated decoder for XviD ?

[Koepi]

Ok, I've put up my first own binaries, fetch them at

http://www.roeder.goe.net/~koepi/

The changelog is there as well. The modified sources are included as well.

Have fun!

Best regards,
Koepi

[cofferscuffs]

Yummie. Thanks 10x Koepi!!!

[NoLogo]

Koepi, which sources did you use ? I mean date.

EDIT: the URL doesn't seem to work :(

[cofferscuffs]

The URL does work.

09.02.2002: I did compile my own version of the XviD decoder/encoder today. The sources are from 09.02.2002, ~3:30h CET (CVS).

[NoLogo]

Well, big mystery...does work now...:confused:
Maybe my firewall is guilty...
Thx cofferscuffs.

[Koepi]

sorry for the trouble with that link. We have trouble in our MAN / VLAN here regarding a central DNS, even our mailers are affected. Should be more stable in some hours (hopefully)...

Regards,
Koepi

[-h]

@Koepi:

Haven't tried your changes, but the registry issue is known (we're just too lazy to separate every config setting into registry keys). In config.h I've been updating the value of XVID_REG_NAME to get around this (beautiful).

Just wondering, why'd the additions to xvid.h have to be global? ;)

I'd just commit it all to CVS, no one really worries about non-core stuff that gets merged. I wish there was a global changelog though..

-h

[Koepi]

@-h:

Oopsi, did I make changes global? ;)

To tell you the trueth, these were points that gave me a real itch to scratch and so I just started coding blindly. I'll recheck if I really did something messy, but since I'm not really a C freak I have my problems with it, and I'm glad that I made it work at least - maybe I get closer to C and can revisit my changes and clean them up. Using the class manager is so much easier ;)

What gives me a real headache is my plan with adding overlay support to dshow - in C++ it isn't too easy, but now in C?

Well, I want to learn that and I'll find a way ;)

Thanks for looking at my changes!!! :)

Best regards,
Koepi

[-h]

The stuff in xvid.h couild be in codec.c, just under the "int bytes" declaration, that's all :)

How would one go about adding overlay support? I was hoping that the dshow framework did that for us, and all we managed was decompression. Why can't you use C++?

-h

[Koepi]

Mixing C with C++ is a BIG no no! It results in unmaintainable sources and I'm not even sure if it would compile correctly.

Just because my hack was "global" (compiler complained about unreferenced variables so I moved both declarations to the header, problem solved...) doesn't mean that my style is bad in all ways ;)

Regards,
Koepi

[-h]

Mixing C with C++ is a BIG no no! It results in unmaintainable sources and I'm not even sure if it would compile correctly.

Oh I thought you were talking about only being able to use C to add overlay support to the dshow filter. I got a bit lost seeing as it's already CPP :)

I'm merging some vfw stuff tonight, so I can add yours in with it if you want. I'm moving a lot of the tab content around (finally making a 2-pass tab) so it might be cleaner that way.

-h

[Koepi]

@-h:
I didn't look into dshow sources yet, I just compiled those. Since vfw is in C I assumed dshow is in C as well.
If it's a C++/MFC project then I'm more than glad because I can help there way better :)

Regards,
Koepi

P.S.: you may of course incorporate my changes (as long as you don't sell them as yours ;) ) - didn't you want to upload those into CVS?

[-h]

If it's a C++/MFC project then I'm more than glad because I can help there way better :)

Yep, don't think it's actually possible to write dshow code in pure C. The dshow code is very 'clean' at the moment.

I merged a whole bunch of vfw stuff a few minutes ago, credits, fourcc, tab moves etc. All kinds of stuff people are going to complain about.

But.. I'm going to bed, so I'll miss all the bug reports for a while :)

-h

PS. forgot to do the extra 2-pass output, might add it tomorrow.

[Koepi]

Thanks mate, I'll re-checkout the sources tomorrow then... extra debug-output? I simply commented out the original output (were just 2 lines ;) ) and added some, it's meant as a substitution - else you'd get 2 lines of output for every frame ;)

G'night, sleep well!

Regards,
Koepi

[Ripe73]

HI!
I try to change the FourCC to Xvid DSfiler with Ogg like this.
Open video in FourCC
FourCC Description Code:q
FourCC Used Codec:XVID
Apply
but the movie cant be played after that,someone know how to change to XviD DS filter with Ogg?

I hope
;)

EDIT:I must be genius:D i changed the FourCC before i mux
I think i will beat Einstein(maybe only with my penislenght)

[Nic]

Latest build 11/02/02 is now at:

Main:
www.freewebz.com/xvid

Mirror:
xvid.stormpages.com

Changes:
The pink & green bugs should be alot better,
Now has a Koepi's check button to select the FourCC used.

Cheers,
-Nic

[rui]

Hey, we now have three codec supliers: Nic, uManiac and Koepi. :)

We the users must urry, i was just starting to test Koepi,'s build, and now we have a fresh new one from Nic. I can't keep up the pace :D :D

[Nic]

I think mine & Koepi's will be somewhat similar :)

-Nic

[Koepi]

Ahoy :)

No offense nic, but please test your binary - first pass always uses quant 16 (or whatever quantizer you put in for the credits). While adding the credits stuff there went something wrong. (Damn, if i weren't that keen on a little debug output I would never've figured that out!)

I fixed that bug.

My binary+modified sources is available at http://www.roeder.goe.net/~koepi/

Changelog is inside the zip and on that page.

I forgot to add that the values get screwed due to the registry stuff... So just enter the default values (or whatever you like) and then close the configuration interface. After reopening it the values should be in range again.

I hope this helps,

Best regards,
Koepi

[Franko30]

Just as I posted a few minutes ago

http://forum.doom9.org/showthread.php?s=&postid=82094#post82094


Frank


P.S.: Thanks Koepi for fixing it that fast!

[rui]

Well, some time ago i placed here a question about the diferent XviD builds we have. My question was about the possibility of having diferent compilation optimizations.
I've read an article over Ace's Hardware about this:
http://www.aceshardware.com/read.jsp?id=40000189
I read in the XviD org forums that you guys are having dificulties introducing SSE2 optimizations, but this article says that by using Intel's c++ compiler, SSE2 code is generated automatically: http://www.aceshardware.com/Spades/read.php?article_id=40000192
http://www.aceshardware.com/Spades/read.php?article_id=40000193

Humm. I forgot that this applies to floating point, but this encoding thing uses integer most, correct?

[Franko30]

to Koepi:

Hi again,

I just installed and tested your fixed binaries with my before mentioned 1 minute test clip, using H.263/Motion Search 5/2pass/2ndpass int./desired filesize 6651.

But there's still a bug when using "credits start 0". 1st pass finishes OK but the second pass produces the error message "Credits larger than desired file size" directly after starting.

Frank

[Koepi]

Uh, really?

I'm still stuck in my first pass (>6h to go from now...), so I've just tested first pass yet.
I'll try to find the wrong part. Btw., I didn't fix it that fast, I just checked out the changes that -h applied to the CVS and ran into the same trouble as you :)

But maybe now I'll be fixing that fast ;)

Regards,
Koepi

[Koepi]

Let's see if this is fast...

You're absolutely right. -h introduced a range-check for the credits (if they are within the movie...). But as the codec can't know the size of the movie this is simply impossible.
I removed that check, hopefully it works now.
Updated binaries are in the same place again ;)

Thanks for your report Frank,

best regards,
Koepi

[Franko30]

Deleted my message that I do it a second time to be sure, as you found the error faster than I could do a second try...

Frank

[Koepi]

Can you please try to confirm that it works now?
And don't use credits out of range of the movie, there is _no_ check at all now ;)

Regards,
Koepi

[Nic]

It isn't really my job to test the binaries is it? If someone puts silly mistakes into the CVS, im supposed to find them am I?

Well fine then,

Site Closed.

-Nic

[-h]

You're absolutely right. -h introduced a range-check for the credits (if they are within the movie...). But as the codec can't know the size of the movie this is simply impossible.

I only merged enough to fix encoding with the CVS just then - the sources on my side seem to be a mess (yuck).

Why can't the range check work? codec_2pass_init() just adds the total of all the credits frames in the .stats file - if that value is larger than the desired file size, there's no way the movie will reach the proper size since the credits aren't being rescaled (they're the same size in the first and second passes). I know it isn't terribly likely, but at least it's the start of some (much needed ;)) error checking in the frontend.

I still can't believe I committed the wrong code.. looks like it's missing some other stuff I had done too. Heh oh well.

-h

[Koepi]

Can you please addin my changes then as well? I guess it's unneccessary to add that every VFW change in CVS again ;)

Do you like the idea of a dropdownlist? It makes the GUI more uniform...

Just my 2 € cent...

And nic, nope, you're not supposed to check if it works, as I wrote, no offense in that!

i appreciate your work. My codebase can be totally flawed as well and then it's necessary to have your binaries directly from CVS.
So please, reopen your site!

Regards,
Koepi

[Nic]

Nope.

Tell Doom9 to link to your binaries from his guide.

-Nic

[Franko30]

to Koepi:


I did the test encodings of my one minute clip again:

Settings: H.263/Motion Search 5/2 pass int./Credit Quant 16/Desired filesize 6651/luma masking off

Credits start 0: resulting filesize is 1382, awfully blocky

Credits start 1000 (out of 1497): resulting filesize 1344, awfully blocky

I uninstalled the XVID codec again, reinstalled your fix1 binaries, double checked, same, blocky results. :(


So far, Nic's binaries work best (when not choosing "0" as credits start).

Frank

[-h]

@Koepi: I'll add them tomorrow most likely, I'm just going through a diff of CVS and my bad tree to see what else sneaked in (nothing bad so far). You can seriously add them if you want, no one will mind. Just look at the crud that I've been committing! :)

@Nic: Apologies if you got any angry emails about the build being broken. It isn't the distributor's job to test new functionality before making builds available, it sets a bad precedent. One good thing that came of this is how fast it was resolved by user feedback - thanks to binary sites like yours getting the (however stuffed it may be ;)) codec out there.

-h

[Koepi]

Damnit. Well, then let's wait for -h to bring up his changes to CVS :)

Btw., you can use debugview (included with earlier nandub releases, downloadable at sysinternals.com [or wininternals.com? dunno]) to see the quantizer used during first pass... much easier to check then if everything works or not...

I don't know what i broke now that quant 16 seems to be used again :-(

-h, what's the status on your side? Can you please post when you updated the CVS?

@Nic: do it the way I do... set a marker "Do not use this build from <DATE>" ... should be enough. Don't let childish behaviour rule your mind. A little bit of constructive criticism is a MUST.

-h, what about the following way to scale the credits:

You have the first pass, credits get quant. <16>. In second pass, jump to the credits start position, sum up all frame sizes up to the end, subtract that from <desired size>, get the scale factor and then encode up to config->credits_frame and after that just apply quant. <16>?

I didn't look into the way you do that now, I just tried to avoid the reported error message by commenting out the check ;) Since it didn't work, we can play around now, if you like ;)

Do you have ICQ or something? Maybe we can coordinate our work then to avoid double or unneccessary work...

Regards,
Koepi

[-h]

CVS is patched as of about 20 minutes ago, most every internal 2-pass iteration I could think of I tested and found it working.

What you described for the credits is pretty much exactly what it does now :)

I'm on AIM as hst1nk (most days) and icq as hstink , though I don't know how many others share the name on icq. Can't remember my number, it's been months since I used it.

-h

[Koepi]

Hmpf, can't search users with trillian and i don't like to register on AIM since I am afraid of the plenty security leaks (well ICQ has them as well, but not as bad...)

25073073 - just contact me ;)

Regards,
Koepi

P.S.: I looked into the new codec.c and it looks WAY better now, that's a better attempt to do that correctly. I'll rebuild a binary ASAP, have to readd my debug output stuff ;)

[Franko30]

@ koepi:

downloaded debugview and the results are:

Quant 2 during first pass, 1st frame 2nd pass Quant 31, then Quant 16.

Frank

[Koepi]

Thanks Frank,

that brings light into that matter - so I broke something on second pass, first pass works correctly :)

Thanks again,
Koepi

[-h]

The current codec.c isn't how I'd like to implement it, but I just wanted to fix this issue asap. There's still a lot that needs cleaning up.

An ICQ search for your number isn't turning anything up (just freezing on my end). Try finding me on 85734223.

-h

[Koepi]

Amazing, found exactly that number via the icq webpage ;)
I already contacted you... :)

Regards,
Koepi

P.S.: unzipping fresh checkout now...

[-h]

I never got anything via ICQ - people have complained that I never reply to their ICQ messages, seems they just disappear somewhere along the line :confused:

Ah well, I've got to get up in 5 hours anyway - if you find anything scary.. merge the fix! :)

-h

[MaTTeR]

Sigh...Nics site is indeed closed:mad:

Whats up with this? I'll be traveling all this week on business but when I get back I'll get a compiler installed and put up a site myself;) I will welcome your feedback...this is what it's all about right?

[Koepi]

hehehe Matter ;)

-h, do you have anything fancy like zone alarm installed or are behind a restrictive router/firewall in your network? This is indeed strange...

Well, I just put up new binaries, you may check if they work (I can't sionce I'm still stcuk in a first pass with the old binaries... :-/ )

Please use debugview to see if everything works the way it should...
I added something more on the debug output/first pass, but it may be trash and maybe doesn't work at all...

Please, send reports :)

Regards,
Koepi

[Nic]

:)

-Nic

[Koepi]

Hidden update to XviD-11022002-3 ;)

I modified 1st pas sdebug output to actually show the total filesize as well (I need that for checking compression ratio...).

The last build (-2) should work as well.
Modified sources are as usual included in my zip-archive.

Can anyone test this binary on 2nd pass as well?
I had to stop my encoding session right in the middle (4 hours for nothing :( ) just to verify that at least 1st pass works.

Regards,
Koepi

[Franko30]

@ Koepi:

OK - it seems like you've done it (Fix2), the results stick to what you enter in the codec dialogue.

One minute file:

Credits Start 0: Quant 2 through whole 1st pass, then varying Quantizers in 2nd pass.

Credits Start 1100: 1st pass - Quant 2 till frame 1100, then Quant 16. 2nd pass - varying Quantizers till frame 1100, then Quant 16.


And the "pink" problem seems to be gone, too, with these builds - or should I take off my sunglasses? :cool:


Thanks, Koepi!


Frank

P.S.: I can't get rid of the feeling that I wanted to do s.th. else today besides testing binaries :D

[Koepi]

Thanks a million for testing Frank :)

I'm glad that it works now again :)

@Nic: the CVS is in usable state again, compiling without modification should work like charme again.

Regards,
Koepi

[Nic]

Well done Koepi :)

It will be great when all the pink/green blocks have disappeared....

(remember to tell Doom9 to download from your site from now on.....I dont want him (or me :) to get complaints :)

Take Care,
-Nic

[Franko30]

Originally posted by Koepi
Thanks a million for testing Frank :)
Regards,
Koepi


It was my pleasure!

BTW: I tested Fix 3 and it seems to work, too.


Hey, what about adding a "start credits" feature next? A lot of movies start with Credits and not much moving at all :D :D


Frank

[MaTTeR]

Which compiler are you guys using? Dibrom seems to recommend the Intel compiler for his LAME builds.

[Nic]

If you look through videocoding.de you'll see a similar thread about compilers, it doesn't look like the ICL is the way to go. I think everyone is using VC6. I've got VC7 which I was going to try out today & put up a binary.....ahh..I doubt it would have made much difference anyway,

Cheers,
-Nic

[MaTTeR]

Thanks Nic:)

If you decide to try VC7 please post your results. I guess I'll eventually get into the wonderful world of compiling:D

[Koepi]

It shouldn't be too hard to implement "start credits". I could do so - but first I want my movie to finish sometime close - it's on my HD since saturday, I'm in the 5th start now...

Regards,
Koepi

[Franko30]

Originally posted by Koepi
I'm in the 5th start now...

Regards,
Koepi


Take your time!

And don't crash your computer during your 5th try ;)

Frank

[Koepi]

Hi,

I added experimental start credits support to my binaries.
They might not work at all.
The GUI can be broken.

I just hacked in what i think is needed, but since my compi is occupied I can't test the new binary.

Please keep me posted with results.

Regards,
Koepi

[Nic]

Ive given up ;) :

11/02/02 part 2 (latest CVS)

www.freewebz.com/xvid

or

xvid.stormpages.com

With -h's fixes & Koepi's additions.

Cheers,
-Nic

[Koepi]

Heh, you added my additions? Nice :) They aren't that useless then I think - which makes me happy .)

Btw., did you already use M$ .Net VC?
Is the new dll faster?

Best regards,
Koepi

[Nic]

No im in the middle of a big project for a client & I really don't want any delays so I havent put it on my comp yet (Just in case)...But I think ill take the plunge tommorrow.

(I didn't add anything to the build, its straight from the CVS)

Cheers,
-Nic

[Franko30]

Originally posted by Koepi
Hi,

I added experimental start credits support to my binaries.
They might not work at all.
The GUI can be broken.
Regards,
Koepi


Hi again,

after doing some actual encoding (instead of testing) and watching the encoded Farscape episode with a feeling of "Wow!", I now had the time to test the "start credits" option.

And you're right, the GUI is "broken": :(


When checking the "Use start credits" box, start frame and end frame stay greyed out and cannot be used. Doesn't matter if "2 pass 1st pass" or "second pass int." or anything else is selected.

Frank

[Koepi]

Ok, thanks, I'll look at into it. Strange thing is, all the code is in there. Maybe I have to add a message handler ;)

I'm trying to involve a new option for two pass now, separate Intra frame quantizer settings - should give much better quality on low bitrate condition, but can break file size predictability... I'll test around with that for myself first.

Regards,
Koepi

[Koepi]

Ok, small bugfix without version numbering change:
I disabled the greyed out fields. They just will get respected when "Use start credits" is checked (hopefully....).

Frank, can you please redownload the binaries and check again?

Thanks,
Koepi

[Franko30]

Originally posted by Koepi

Frank, can you please redownload the binaries and check again?


Sorry, but unfortunately the "end frame" box in the "start credits" option is still greyed out.

Frank

[Koepi]

Damnit.

Well, in 5 minutes my 8 hours first pass is finsihed and then I can start testing here again (well, at least install other binaries).

I'll put up new binaries when i figured out why the box is still greyed out. But please don't use that binary for regular purposes since I (tried to) hardcode a Intra frame DRF range of 2x-3x on 2 pass modes(in the second part).... dunno if that works correctly. But for testing start credits it should be suitable.

I'll write another post when I updated the page.

Regards,
Koepi

[Franko30]

Well, I have to get up early tomorrow, so it's good night for me - if you succeed with the start credits and the other stuff, I can test anything you want later tomorrow morning.:D



Regards

Frank

[Koepi]

One more _damnit_!

I couldn't resolve doom9.org for ~2 1/2 hours now...

The new binary is up, the fields are working now. I added an extra that I'm testing ATM:
locked quantizers for keyframes.
It's hardcoded to 2x-3x until now, but if the result looks good I'll include it to the configuration GUI.

Still I need someone to test the start-credits option.

Sleep well guys,

regards,
Koepi

[vinetu]

Hi Koepi,
I've just tested the new build - Start and End credits works correctly.
The only mistake I found is uncorrect file sizes:
expected file size - 55 Mb
1pass(2passes) - 62 Mb
2pass(2passes) - 22 Mb
( avi lenght - 2500 frames ,25 fps)

BTW the "end credits" Quantizer is active right now on the first pass
instead of only on second pass.

[Ripe73]

HI!

If i have done a statsfile with a previous Xvidbuild can i use the same statsfile for the latest build or will it not work correct?

[-h]

.stats will always be the same between builds, so you can safely use the old one you made.

@Koepi: I'm going to start another overhaul of the vfw :) Wish I could get you on icq or aim. suxen_drol came up with a pretty good way to organise the frontend, so I'm going to move some tabs around (again), add some popup dialogs and see if I can merge your changes (or just recode them as per my own messy style). Credits will be supported by all modes, not just 2-pass. Also, the registry will store separate keys, not a struct blob (should make external automation easier?)

Going to start coding now, should be done in an hour or so ....

-h

[Teegedeck]

It depends. If your .stats aren't too old, you might get away with it. If they're several months old, you'll certainly have problems as meanwhile the internal inter-/intra-decision (keyframes) has changed.

[Ripe73]

Originally posted by Teegedeck
It depends. If your .stats aren't too old, you might get away with it. If they're several months old, you'll certainly have problems as meanwhile the internal inter-/intra-decision (keyframes) has changed.

I forgot to say they are from the 02-02 build.
Thanks

[Franko30]

Originally posted by Koepi
One more _damnit_!

Still I need someone to test the start-credits option.

Koepi

Hi Koepi,

I just had the time to test the start credits option:

The checkboxes work, but debugview shows that in first pass, quantizer 2 is used till the start of the end credits, then quant 16 as specified. In 2nd pass the specified credits quantizer gets used for start and end credits. Thus, the desired filesize 6651 (for a 1 minute file) isn't reached - resulting filesize is 6040. I guess is because of the Quantizer 2 during start credits in first pass, isn't it?

Frank

@ vinetu:

from what I learned so far, the credits Quantizer (16) is supposed to work in 1st AND 2nd pass. But I might be wrong, let's wait for a moderator's reply to that.

[Koepi]

Uh, have to look for that stupid mistake where i forgot to use that fixed quantizer on first pass.
Since the algorithm is VERY easy (just subtract the fixed quantizer regions [the space they need] from the desired size and use the residue to scale down the rest) it of course produces false results then :-/

sorry for that stupid mistake.
I have to checkout -h's changes and try to add my ideas then as well.
@-h:

can you make up some space for fixed keyframe quantizer as well? In fact those are the same as min and max quantizer, but special ones for keyframes... I only want to use them on 2 pass mode/2nd pass, so it shouldn't be a general tab holding this ;)

Thanks,
best regards,
Koepi

[-h]

can you make up some space for fixed keyframe quantizer as well? In fact those are the same as min and max quantizer, but special ones for keyframes... I only want to use them on 2 pass mode/2nd pass, so it shouldn't be a general tab holding this ;)

A whole lot of stuff is changing - credits are calculated with a rate % (same as gknot). There's going to be a dialog box for the internal 2-pass mode, so we can cram as much stuff on there as we want :)

Code is going well so far..

Finally got your icq messages, did you get the one I sent back?

-h

[Koepi]

Nope, i didn't get those - you should upgrade to an icq client that supports the new v7/v8 protocol. ICQ programmed their servers to drop randomly messages from clients with older protocols.
I recommand trillian - i think the URL was www.trillian.cc ...

Thanks,
Koepi

[-h]

How awesome of icq...
downloading now.

-h

[Koepi]

Well, in the meantime...

The experimental intraframe quantizer lock gave very nice results. I still wonder if they were nicer if we disallow quantizer jumps >1 or >2 after a keyframe... for example, if I had:

INTRA qu.3 (upped!)
Inter qu.11
Inter qu. 7
Inter qu. 5
Inter qu. 4
...
Inter qu. 3
Inter qu. 4
Inter qu. 4
Inter qu. 4
Inter qu. 3
(This happened quite a few times, but mostly it worked out correctly. Even these scenes still look awesome for:
640x352 TV Source rip 1h52m on 1 700MB CDR, using MPEG quantizer... there was still some minor noise left in the images which result in a vivd picture - but it's way better than the first version I tried...) ;)
Maybe the quality is even better if we have

INTRA qu. 3 (upped!)
Inter qu. 5
Inter qu. 7
Inter qu. 6 (or 7)
Inter qu. 5
Inter qu. 4
Inter qu. 4
Inter qu. 4
Inter qu. 3
...


?

We had to check in something global like "last_quantizer" and correct the decision we make:


if (frame->quant < last_quantizer-2)
{
frame->quant = last_quantizer - 2;
}
if (frame->quant > last_quantizer+2)
{
frame->quant = last_quantizer + 2;
}
last_quantizer = frame->quant;


This has only to apply for interframes of course.

What do you think -h?

Koepi

[Koepi]

Well well, since there were yet no changes commited to the CVS i cleaned up my version of XviD.
Changelog is inside the package and on my site, so you know where to fetch it ;)

- Added intraframe quantizer lock, hardcoded to 2x-3x until now.
- Added a routine to prevent quantizer jumping, largest jump amount is +-2 now.
- start credits should finally work. forgot to add them to the first pass as well ;)

As a note, this release is HIGHLY experimental due to the hardcoded quantizer lock, use with care. I got very good results with these intraframe quantizer locks, but have to test quality with the quantizer jumping prevention a little more. A first, fast test showed a once-more-improved image.

Have fun!

Best regards,
Koepi

[saVe]

i just wanted to thank you guys for the great work you've been doing! you should know that it is really appreciated! in the last two days or so i've become an xvid fan i think, the first thing i do after standing up is to check what's new around here... ;)

again, thanks a lot!

[-h]

@Koepi: everything's finished, except that credits aren't working properly (blah) and the edit box for bitrate, quality etc. doesn't automatically update the slider. It's too late to keep working, so it'll have to wait til tomorrow.

I think people might start getting used to such fast-paced development, that's a scary thought ;)

-h

[Koepi]

@-h:
hm, so should I wait until you finsihed the sliders as well? I know how to do such stuff in C++, but I'm still very bad in C, especially when it comes to the windows message handling.

And, what's wrong with the credits? They worked the way they were before, I even updated "my" binary/sources to work as start credits as well - it's the same code and got approval yetserday that it works (well, except for my modifications of course ;) ).

If you're going to sleep now, then good night & dream well!

Regards,
Koepi

[-h]

I'm about to turn in now.. looks like I've found an __int64 bug in VC6 :rolleyes:

The slider is easy to update, I'm just having trouble catching keystrokes in the text box. The credits work differently to before, no more fixed quantizers (a few more calculations).

I'd have debugged it by now, but VC is doing things like this:

if (codec->config.credits_start && frame <= codec->config.credits_start_frame)
{
start += nns.bytes;
}

This snippet actually adds "nns.bytes" to "codec->config.credits_start", instead of adding it to "start". I can't figure out why either.

Ah well, should be wrapped up tomorrow.

-h

[Ripe73]

HI!

I start a second pass with debugview and there was some frames with
Quant:1
Is that possible?I thought the lowest was 2.:confused:

EDIT
I used XviD-12022002-1

and there is a folder"modified_sources" and its only for your coder i hope:)

So Xvid can handle DRF:1-->better quality

[Koepi]

There is a Min+Max quantizer edit box on the config GUI. Guess why there is a "1" by default in the min. quant.? ;)
So it is possible to have that quantizer (it doesn't work with DivX3 since the decoder can't decode DRF 1).

To get further into the materia, you should post which binary you use. Since i added modifications and -h as well, and some things aren't in CVS so these changes don't apply to e.g. Nic's binaries, it's easier to track that down :)

Regards,
Koepi

[Franko30]

Originally posted by Koepi

- Added intraframe quantizer lock, hardcoded to 2x-3x until now.
- Added a routine to prevent quantizer jumping, largest jump amount is +-2 now.
- start credits should finally work. forgot to add them to the first pass as well ;)

Have fun!

Best regards,
Koepi


Hi, I'm back (had a little work to do today).

I tested your Feb. 11th Fix 3 binaries against the above mentioned.

Today I used a 15 min. test clip/H.263/2 pass/2nd pass int./desired file size 87632 for 19506 frames

First test with Fix 3 binaries without end credits enabled (start frame 0) to see what the quantizers would do. Resulting File size 88098. Mostly Quantizer 3 and 4 were used (remember: I use input from an analogue source, although the tv signal gets broadcasted digitally to the d-box decoder, system see below, no filtering enabled except smart de-interlace, normal bicubic resize was used). Quant 2 was used 16 times.

Second test was with the above mentioned experimental binaries, start credits from 0 to 1500 and the end credits were the last 500 frames. That is about 10% credits (more than in a usual movie) so the actual happenings in between should have enough bitrate to produce the "Wow!" effect. Resulting bitrate was 88100 and mostly Quantizer 3 and 4 were used - but MORE Quant 3 than 4 (compared both Debugview logfiles). Quant 2 was used 218 times.

Now to what I saw:

I didn't notice any visible difference, everything looks just great!
Only when examining real closely (nose to the screen) with the "fixed" Quantizers there wasn't as much noise around "sharp edges", shapes like people against sky etc.

I guess that from now on I can't contribute anymore to the subject of "quality comparison", as I use analogue input and no DVD-Rips...

But this also means:
Start and end credits are usable, even the experimental build can be used for normal encoding, which I will do tonight ;)

A great Codec and GREAT WORK :D :D :D done by Koepi and the others (I don't know how close all of you work together).


Thanks for another highly usable XVID "manifestation". Keep up the good work!

Frank

[wing1]

It's getting better and better everyday :) As I've said earlier, this codec has a very bright future. I too am using analog capture source as input for encoding.

[Koepi]

Thanks Frank for soing such a great testing job :)
I'm glad that the locked intraframe quantizers and the jumping-quantizer-prevention work as expected :)
(My actual encoding is a much harder job, tested with MPEG quantizer and no jumping prevention, looks still nice, but let's see what happens when it's finsihed with the new build... :) )

Btw., -h will add another way to scale down credits - it's a percentage of the desired bitrate - so if you enter 50% and are encoding with 800kbit/s, it will use 400kbit on credits... so I'll add my additions to his new version as soon as he publishes his changes to CVS (if he didn't do it yet ;) ).

Best regards,
Koepi

[-h]

The "credits rate" functionality has entirely replaced the fixed quantizer option, which I think is a bit unfortunate since having a higher credits quantizer in the first pass made encoding the credits significantly faster.

Once it's settled down, it shouldn't be hard for me (or you ;)) to re-add the fixed-quantizer mode.

-h

[Koepi]

NONONO! I don't want fixed quantizers back! Scaling with just another variable isn't bad and carries the thought of "constant quality" (which 2 pass true VBR should be...).

I meant the other changes:

- intraframe quantizer lock
- quantizer aliasing (jumping prevention)
- and finally, the debug output :)

Would be nice if you could add them, then we wouldn't have to spread X different versions of XviD, which is a BAD thing IMHO.

I would really appreciate it!

Thanks in advance ;)

Warm regards,
Koepi

[-h]

I-frame quant settings are done, quant smoothing (last_quant +- 2) is done, debug I'll get around to ;) but I'm still getting this ridiculous VC bug, even on my work comp with SP5.

Either way, it'll be done tonight. I might just get fed up and commit the whole lot anyway :)

-h

[gldblade]

Wow, that's what I call fast development.

Just wondering, I heard somewhere that Koepi's latest build does not have the pink bug. Can anyone confirm? I've done a little testing and I don't see any pink, but I just want to make sure.

[Koepi]

I don't see any pink either.
This has nothing to do with that I'm color blind...

sorry, I'm not :)

But well, sometimes there is pink: when it belongs there. Since I encode a western movie ATM there is some pink ;)

The bug is solved in the core-lib in CVS since 11.02. IIRC.

Regards,
Koepi

-h:
Thanks mate, saves me a lot of work :) As I'm writing and watching the debug output of the second pass I'm _really_ impressed what those modifications are able to do. For me it's never SBC/DivX/Nandub again (I hope). :)

[Ripe73]

HI Coders!!

I encoded a movie with XviD-12022002-1(Koepi)
177501 Frames(118min)
XviD Settings
2 pass Int.H:263
M.Search 5
Q.Type 5
No L.Masking(both pass)
Endcredits:169883 Q:12
Final size:1396MB(GKnot calculated) saved 4mb for subbs!

Well i did the movie twice i did not know that the endcredits start should be on the first pass too so the movie was undersized:D
I have not seen the whole movie just a couple of scenes and they look amazing:) so clear
I will report later if i see something strange

Thanks

[-h]

!@#$!@#$%!@#$!@#

All that's holding back a cvs commit is this VC6 __int64 bug - any better way to work with the values we're going to see in .stats files? I can commit now, but if the 1st pass is bigger than 4 GB there'll be trouble.

-h

[Koepi]

Hmmmm... since XviD is alpha there shouldn't be a problem with that limitation. It's for advanced users in it's current state as I understand it, and those wouldn't dare to compress anything that exceeds 4GB on 1st pass to 1 or 2 cd.... ;)

Just my oppinion.
If it's still the error, that the wrong variable gets used... It works for me if you just declare an int. Isn't that int64 anyways by default?

Maybe I'm still too sleepy to get the point there, but this is just an idea.

Regards,
Koepi

P.S.: thanks Ripe73 for the feedback. Nice to know that it works out well for you as well :)

[-h]

@Koepi - I'm trying to message you on icq but trillian is spitting out "Error while sending IM: Not supported by client."

I think a second pair of eyes would be very useful right about now.

-h

[Nic]

@Koepi the standard int/long is a 32bit integer....

@h:
What is the bug with the int64 (ive never come across it!) If you could tell me how I could re-create it quick, I can do some tests for you.....Have you got service pack 5 on?

Cheers,
-Nic


ps
looking forward to the CVS commit :)

[-h]

@Nic: I think I'm going insane.

Anyway, I'll commit what I've done only use long's instead of __int64's. The issue I have at the moment is:

if (codec_is_in_credits(codec, frame))
{
credits += nns.bytes;
}

Now in that bit of code, if "credits" is a long or int, it adds fine - if it's an __int64, it stays 0 no matter how much I add to it. Driving me insane.

Also, 2-pass mode is currently undersized, but that can be fixed later. Best to just commit this stuff and let the bugs be fixed by people who haven't got brain-freeze :)

-h

[Nic]

LoL -h

Ill do a little test with _int64
im assuming is credits a int64 and nns.bytes is a int...right?
(im in the UK so ive only just got up so my brain should work for about the next twenty mins :)

Could you also do a little explanation on how to do an external 2pass with the new code (or how to do an internal one?) Looks very confusing now (also both stats boxes load/open stats...is that right??? :)

Cheers,
-Nic

[Nic]

__int64 credits=0;
int num = 5;
long a = 12413423;
credits += a;
credits += a;
credits += num;
printf("\n\n64: %ld",credits);

Just tried the above code, didn't get any problems......
Do you have service pack 5 on?

Good luck, hope you dont pull you hair out :)

-Nic

[-h]

I just committed my "changes".

I encountered the __int64 issue when "unsigned long total" and "unsigned long credits" in codec.c are __int64 instead.

I have no idea why internal 2-pass encodes are undersized now.. I assume I'm missing something obvious.

Internal encode instructions:
- perform 1st pass
- set mode to "2-pass - 2nd pass Int.", click on the "Internal 2-pass options" button and enter desired file size

External:
- perform 1st pass
- set mode to "2-pass - 2nd pass Ext.". Load 1st pass stats file in GKnot, do stuff to it, save as a new .stats file. Load the new stats file in XviD's second .stats text box.

-h

[Nic]

I hope the changes go in straight away cos im just downloading them :)

Im going to look into the codec.c __int64 problem as well. Oh, & maybe ill compile this with VC7.....Better go install it :)

Cheers,
-Nic

[Nic]

Latest Version 13/02/02

main:
www.freewebz.com/xvid

mirror:
xvid.stormpages.com

Cheers,
-Nic

Changes:
Compiled with VC7
-h's & Koepi's changes (Start Credits bitrate, etc)
(maybe slight bug with undersized 2pass files)

ps
Could people please test this version for me :) Compiling with VC7 has produced a smaller .DLL file & might be quicker or cause problems.
(The DShow is the older version, couldn't get it to compile under VC7 yet)

[saVe]

just an idea: couldn't it be possible that the undersizing in 2pass happens due to quantizer aliasing? when the codec can't choose quantizers that differ more than 2 from the previous one like it would without quantizer aliasing it's clear that filesize is less predictable.

can you confirm this?

[Koepi]

Nope, that's not the reason. Jumping prevention + intraframelock are known to possibly produce oversized files when too strict settings are used.
This is a matter of the new credits scaling algorithm IMHO. I'll look deeper at it when -h figured out how to solve the __int64 problem. Until that I'll stay with my build as it works as I expect it to ;) (No offense in here again, but I like my work and want to use it a little ;) )

Regards,
Koepi

[-h]

The credits undersizing is just mind-boggling.

During testing, I get:
1st pass size = 160000
desired size = 50000
curve = 3.2 (1st / desired)

Voila! Same as it was before, and I didn't change any of the 2-pass frame-size generation.

After a VC re-install, __int64 seems happy again, so I merged it (along with a minor interface enable/disable fix). Now I've just got to figure out what's going wrong with the curve calculation.

-h

[Koepi]

My encoding should be finished in ~2 hours, I'll checkout the CVS then again and try to give you a helping hand.
Mathematics often gave me a heaache because I just saw what I wanted to do and not what the thing I wrote down actually does - sometimes it's a blockade in the mind, so I think I can try to help! :)

Regards,
Koepi

[Nic]

Doing a quick test, it appears that a VC6 compile is slightly quicker than a VC7 one!

Then again ASM cant be speeded up, so I guess any difference is always going to be minimal.

-Nic

[-h]

The math is right, and it's generating the right curves, but "something" is breaking down when it comes to actually generating quants.

Maybe I'll just add "curve_movie *= 1.02" at the end ;)

-h

[rui]

Well, i just made a quick test using the latest Nic's build. The source was an AVI file, already encoded with divx4 (i am at work, and this is the only thing i have). I used quantization H.263, 5 search precision, no credits start or end, 2 pass using the Xvid 2 pass, not Gnot.
The source file has 8000 Kb in size, i didn't convert any audio, and pointed a 2000 KB end file size for the codec, but it turned out that the final AVI had ~ 15.000 Kb.
I don't know if this was a very small test and isn't valid, but the end file got much bigger than the source file.

[-h]

@rui - I certainly haven't seen that!

Could you list all the (20 odd ;)) parameters you used for encoding? sounds like credits or quantizer settings have messed it up.

-h

[Koepi]

rui,

the intraframe quantizer lock can screw up here, you should disable it for such small files.
(Intraframe quantizer lock can produce oversized files in case the bigger intraframes can't be compensated by interframes. As addition, the quantizer jumping prevention takes away more possibilities to compensate....)

Hopefully this solves the trouble,

Regards,
Koepi

[rui]

Well, lets see: my settings were: for the first pass i used motion search precision 5, quantization H.263, max keyframe interval 300, quantization tunning max 31 and min 1, fourCC Divx.
i didn't touch the debug tab, so everything was as default. I didn't choose smooth quantizer fluctuation.
For the 2 pass, i didn't use any credits options, both start credits and end credits weren't selected. I didn't choose Luminance masking either.
Desired filesize was 2000 Kb.
The source file was the AVI that Tom's Hardware Guide made about that running a quake3 loop and then removing the fan of the cpu.
Its an AVI divx4, i believe. It has a total size of 10.230 Kb (not the 8000 i said, but). The final AVI has a total of 15.174 Kb
Any more info please tell me, i will be glad to help.

[rui]

Well, i just made the exact same test, with the exact same settings, EXCEPT, that this time i choosed in the debug tab the "smooth quantizer fluctuation +-between 2 frames".
This time, the encoding time was MUCH faster (i am using a P2 350), i even saw in my monitor ~20 fps (??). And the final filesize was 2104Kb. :)

What did the "smooth quantizer fluctuation +-between 2 frames" made to change both the speed and final size?? Is this the "the intraframe quantizer lock" that Koepi talked about?

Thanks

[lar1r]

I tend to record TV shows and fit about 4 hrs on one CD. But because each hour is recorded seperately I don't know what the expected file size should be. Some files are less complex than others.

Will there be an option to simply state the bitrate for 2 pass mode?
(or have a choice b/w filesize or bitrate)

Thx

[wing1]

Is it my computer? Just download's the latest build13/2/2002. Removed Divx4.12 codec from system, installed xvid codec and registered filter,xvid.ax. Encoded a matrix trailer clip using (1-pass CBR, 1800kpbs, 5 motion, H263 quantizer, quantize min=1 & max=7, xvid 4CC, default for the rest). Play back using WMP and it acts as if there is no codec existed in current system, and it attempts to download it from web page. It went to the motion of searching for codec for about 30seconds, and then load video. Why is this happening? It does not act like this before. NT4.0(service pak6, 733 PIII, 256Mb, 8Mb ATI video card).

[-h]

@lar1r:
Setting a bitrate for 2-pass mode is the same as setting a desired file size - setting a desired size just saves you the calculation.

If you want the complexity spread across the 4 files, so they all have the same quality (thus different file sizes), you should do all 4 1st passes, find the file size that the .stats file captured (GKnot can do this?), then set the desired size for each file to be a scaled down copy of the 1st pass output so that it fits onto 1 CD. Or there might be a better way I've overlooked.

@wing1:
The XviD .ax filter you downloaded is not configured to play DivX content - you'll have to change the FourCC code of your avi to XVID instead of DIVX (the new codec frontend gives this option in a drop down box).

Edit: hmm just read that you set the fourcc, I'm not sure what's going on in that case.

-h

[wing1]

I also noticed during full screen playback, the filter is using pal mode framesize. The clip frame size is 512x384 @ 24fps. The codec that was used on the original clip is divx3.11. I used vdub to re-encode the clip to xvid for testing purposes.

[Franko30]

@ Koepi:

Yesterday evening, after reading, that you clened up the XVID dialogue etc. I downloaded and installed your Feb 13 binaries.

Trying to encode the evry same testfile as during the last few days, I encountered nothing but crasehs when starting to encoded, no matter what settings I used for XVID (set fourcc DivX or XVID, Quantizers etc.). Mostly VirtualDub crashed in a way that I couldn't even get a crashinfo - the first time an application made my whole XP system freeeze...

Then at 3 a.m. I gave up. :confused:

Just now I had the idea to download Nic's newest binaries (that should be the same now concerning XVID dialogue and features) and voilà - it works! :D


Only half an hour ago I could finally get a crashinfo.txt from VirtualDub. I'll try to add it to this post as attachment (don't want to fill this forum up including it into the post).

BTW:

Why is your xvid.dll 212 kb in size and Nic's only 180 kb?

Frank

edit: Well, it seems that file attachments still aren't possible.

[rui]

I placed some posts above a message saying that I did a test with Nic’s latest XviD build, and got an oversized file (bigger than the source file). That was true, but for that test I used a small avi file already compressed to divx4. So I wasn’t sure about the results being valid.
Now I have made another test, this time using a chapter ripped from a Pal DVD (The Haunting), and I must say this:
XviD is the best!! The file size was exactly the one that Gnot calculations gave me.
I want to apologize to Nic, –h and Koepi, the ones that are the face for XviD in these forums, for having some (but little) doubts about their XviD builds.
I made the same test again, this time using divx4, and the resulting avi file, besides being undersized, was inferior in quality to the XviD avi.
Ok, my world is in peace again. Keep the excelent work.
P.S. I am still using fourCC Divx, because of the postprocessing.

[-h]

Trying to encode the evry same testfile as during the last few days, I encountered nothing but crasehs when starting to encoded, no matter what settings I used for XVID (set fourcc DivX or XVID, Quantizers etc.). Mostly VirtualDub crashed in a way that I couldn't even get a crashinfo - the first time an application made my whole XP system freeeze...

Heh we should be proud then! :)

Only half an hour ago I could finally get a crashinfo.txt from VirtualDub. I'll try to add it to this post as attachment (don't want to fill this forum up including it into the post).

Attachments are probably still stuffed (I didn't see attachments pending approval listed in the mod menu), feel free to post the crash info here.

Why is your xvid.dll 212 kb in size and Nic's only 180 kb?

Nic's is compiled with VC7, Koepi's with VC6. Different compiler cores, I would say.

-h

[Franko30]

Originally posted by -h
feel free to post the crash info here.

-h

OK, here it comes!

Frank


Crashinfo VirtualDub 1.4.8:

VirtualDub crash report -- build 13719
--------------------------------------

Disassembly:
77d1ca00: jbe 77d1ca12
77d1ca02: push ebx
77d1ca03: mov bl,[ecx]
77d1ca05: mov dl,[eax]
77d1ca07: mov [eax],bl
77d1ca09: inc eax
77d1ca0a: mov [ecx],dl
77d1ca0c: dec ecx
77d1ca0d: cmp ecx,eax
77d1ca0f: ja 77d1ca03
77d1ca11: pop ebx
77d1ca12: ret 0008
77d1ca15: sub ecx,00000069
77d1ca18: jz 77d1c85c
77d1ca1e: sub ecx,00000007
77d1ca21: jz 77d47faf
77d1ca27: sub ecx,00000003
77d1ca2a: jz 77d1caf5
77d1ca30: dec ecx
77d1ca31: dec ecx
77d1ca32: jz 77d1c863
77d1ca38: sub ecx,00000003
77d1ca3b: jnz 77d1c97d
77d1ca41: cmp byte ptr [ebp+11],00
77d1ca45: push 10
77d1ca47: pop ebx
77d1ca48: jz 77d1c867
77d1ca4e: cmp dword ptr[ebp-32],00000000
77d1ca52: setz al
77d1ca55: dec al
77d1ca57: and al,e0
77d1ca59: add al,78
77d1ca5b: mov [ebp+11],al
77d1ca5e: jmp 77d1c867
77d1ca63: cmp [ebp-20],eax
77d1ca66: jnz 77d4ed06
77d1ca6c: mov ecx,[ebp-12]
77d1ca6f: mov ecx,[ecx-4]
77d1ca72: mov [ebp-44],ecx
77d1ca75: mov [ebp-40],eax
77d1ca78: jmp 77d1c89d
77d1ca7d: add ecx,esi
77d1ca7f: jmp 77d1c9d5
77d1ca84: mov dl,[eax] <-- FAULT
77d1ca86: inc eax
77d1ca87: test dl,dl
77d1ca89: jnz 77d1ca84
77d1ca8b: sub eax,esi
77d1ca8d: xor esi,esi
77d1ca8f: xor edx,edx
77d1ca91: cmp [ebp-16],edx
77d1ca94: jge 77d4a796
77d1ca9a: sub [ebp-8],eax
77d1ca9d: cmp esi,edx
77d1ca9f: jnz 77d3ff80
77d1caa5: cmp dword ptr[ebp-28],00000000
77d1caa9: jnz 77d4eca1
77d1caaf: cmp dword ptr[ebp-8],00000000
77d1cab3: jg 77d4eccc
77d1cab9: test eax,eax
77d1cabb: jz 77d1cad3
77d1cabd: dec eax
77d1cabe: cmp dword ptr[ebp-4],00000000
77d1cac2: jz 77d1c939
77d1cac8: mov dl,[ecx]
77d1caca: mov [edi],dl
77d1cacc: inc edi
77d1cacd: inc ecx
77d1cace: dec dword ptr[ebp-4]
77d1cad1: jmp 77d1cab9
77d1cad3: dec eax
77d1cad4: mov [ebp-24],eax
77d1cad7: test esi,esi
77d1cad9: jz 77d1c929
77d1cadf: push dword ptr[ebp-36]
77d1cae2: push 00
77d1cae4: push dword ptr[077d6d1a4h]
77d1caea: call dword ptr[077d112e4h]
77d1caf0: jmp 77d1c929
77d1caf5: test esi,esi
77d1caf7: jnz 77d3ff4b
77d1cafd: add dword ptr[ebp-12],00000000

Windows 5.1 (Win2000 build 2600) []

EAX = 00000002
EBX = 0000000a
ECX = 00000002
EDX = 00000000
EBP = 0477f7d8
DS:ESI = 0023:00000003
ES:EDI = 0023:0477f8bd
SS:ESP = 0023:0477f7a0
CS:EIP = 001b:77d1ca84
FS = 0038
GS = 0000
EFLAGS = 00010202

MM0 = 0000000000000000
MM1 = 0000000000000000
MM2 = 0080008000800080
MM3 = 0000000000000000
MM4 = ddc1666666666800
MM5 = aa25c89a02752000
MM6 = ddc1666666666800
MM7 = f800000000000000
Crash reason: Access Violation

Thread 00000428 (Main thread)
T:\projects\VirtualDub_old\main\Init.cpp(115)
T:\projects\VirtualDub_old\main\Init.cpp(134)
T:\projects\VirtualDub_old\main\Init.cpp(152)
T:\projects\VirtualDub_old\main\Init.cpp(214)
T:\projects\VirtualDub_old\main\Main.cpp(181)
T:\projects\VirtualDub_old\main\Main.cpp(204)
T:\projects\VirtualDub_old\main\FilterSystem.cpp(427)
Thread 000004dc (FastWriteStream)
Thread 000004d8 (Processing)
T:\projects\VirtualDub_old\main\Dub.cpp(2809)
T:\projects\VirtualDub_old\main\Dub.cpp(2813)
T:\projects\VirtualDub_old\main\VideoSequenceCompressor.cpp(336)
Thread 000004e8 (I/O processing)


77d1ca84: USER32!wsprintfA(0, 0) [77d10000+c96a+11a]
77d1c97c: USER32!wsprintfA(77f888, f8a04e) [77d10000+c96a+12]
00f61e50: xvid!00001e50(477f888, f8a030)
00f6566b: xvid!xvid_encore(df2258, df2264) [00f60000+4930+d3b]
00f64973: xvid!xvid_encore(3a9de, f8ffffff) [00f60000+4930+43]
00f6152b: xvid!0000152b(df3968, 477f92c)
77f416f5: ntdll!RtlFreeHeap(12, e409e8) [77f40000+1597+15e]
77f416f5: ntdll!RtlFreeHeap(77f42e0f, 77f4180b) [77f40000+1597+15e]
77f42e0f: ntdll!RtlTimeToTimeFields(77f4180b, 0) [77f40000+2a1d+3f2]
77f4180b: ntdll!RtlAllocateHeap(0, 88) [77f40000+16f8+113]
77f416f5: ntdll!RtlFreeHeap(77f417cd, 0) [77f40000+1597+15e]
77f417cd: ntdll!RtlAllocateHeap(0, 2) [77f40000+16f8+d5]
00f6433c: xvid!DriverProc(df3968, 477fe04) [00f60000+4140+1fc]
77f416f5: ntdll!RtlFreeHeap(77f417cd, 77f4180b) [77f40000+1597+15e]
77f417cd: ntdll!RtlAllocateHeap(77f4180b, 143450) [77f40000+16f8+d5]
77f4180b: ntdll!RtlAllocateHeap(143450, 7ffd8000) [77f40000+16f8+113]
1a41c670: urlmon!CompareSecurityIds(14, 1729a8) [1a400000+1bfc3+6ad]
1a4015ca: urlmon!000015ca(0, 1a41c60a)
1a41c60a: urlmon!CompareSecurityIds(477fb40, 1) [1a400000+1bfc3+647]
1a41c445: urlmon!CompareSecurityIds(1729a8, 0) [1a400000+1bfc3+482]
76261694: CRYPT32!00001694(1, 0)
77f41858: ntdll!RtlAllocateHeap(400000, 1) [77f40000+16f8+160]
77f41898: ntdll!RtlImageDirectoryEntryToData(400000, 1) [77f40000+185b+3d]
77f55c91: ntdll!RtlConvertExclusiveToShared(400000, 1) [77f40000+15c40+51]
77f55cc3: ntdll!RtlConvertExclusiveToShared(477fc64, 1) [77f40000+15c40+83]
77f6e3aa: ntdll!LdrInitializeThunk(0, 18) [77f40000+2e386+24]
77f55e66: ntdll!RtlConvertExclusiveToShared(477fc50, 77f48b49) [77f40000+15c40+226]
77f48b49: ntdll!CsrNewThread(7ffd8000, 7ffdf000) [77f40000+8a8e+bb]
77f48a8d: ntdll!RtlUnicodeStringToInteger(77f6f3ef, 77f48a57) [77f40000+8862+22b]
77f6f3ef: ntdll!NtTestAlert(77f48a57, 477fd30) [77f40000+2f3e3+c]
77f48a57: ntdll!RtlUnicodeStringToInteger(477fd30, 0) [77f40000+8862+1f5]
77f48a57: ntdll!RtlUnicodeStringToInteger(77f6e5bf, 77f4107b) [77f40000+8862+1f5]
77f6e5bf: ntdll!NtContinue(77f4107b, 477fd30) [77f40000+2e5b3+c]
77f4107b: ntdll!KiUserApcDispatcher(477fd30, 1) [77f40000+106c+f]
73b4175d: MSVFW32!ICSendMessage(df3968, 1) [73b40000+1734+29]
73b4469c: MSVFW32!ICCompress(73b59460, 4008) [73b40000+463b+61]
0477fe98: 0477fe98(0, ffffff)
0046d99c: VideoSequenceCompressor::packFrame(73b59460, 1)
0477fe98: 0477fe98(0, ffffff)
00467167: Dubber::WriteVideoFrame(3ad0000, 477fec3)
0477fe98: 0477fe98(826376b1, 477ff70)
77e578b0: kernel32!WaitForSingleObjectEx(77e59d6a, c) [77e40000+17800+b0]
77e59d6a: kernel32!WaitForSingleObject(c, 3e8) [77e40000+19d5b+f]
0040b740: AVIPipe::getReadBuffer(40b75a, adafb0)
0040b75a: AVIPipe::getReadBuffer(adafb0, 0)
00467b23: Dubber::ProcessingThread(467bcf, 4880000)
00467bcf: Dubber::ProcessingThread(4880000, 3ad0000)
00467a96: Dubber::ProcessingThreadKickstart(47f7c8, ad9f90)
0047f7c8: _threadstart@4(ad9f90, 0)
77e602ed: kernel32!OpenConsoleW(3df04e8, 0) [77e40000+20235+b8]

-- End of report

[-h]

Darn that's not from any of the .asm files - it looks like a compiler issue. I couldn't trace that back to the code itself unless I had the asm listings generated when that dll was compiled :(

-h

[Koepi]

I'll rebuild my dll today, checking the code. It doesn't crash here, that's amazing... Maybe I included an older build instead that one I'm currently working with....
EDIT:
Nope, it's the right dll, doesn't crash here... If I use XviD and start saving an avi more than once, I've to reopen the XviD compressor dialog because else I would get an "video encoder error: -100".
Maybe it's some of the variables I use, maybe this is a memory leak,... I've to check that. Expect an update in a few minutes.

I'm sorry that you tested for nothing :-(

Regards,
Koepi

[-h]

Dude Koepi.. what on earth is trillian doing!

Can you describe what you mean by saving an avi more than once? Sounds like something I did.

-h

[-h]

Trillian died again. I just committed my changes, nothing major but should fix that crashing bug & roll in some more debug output.

-h

[Nic]

Ill put a new build up tomorrow (taken today off to be with my girl) :)

-Nic

[Koepi]

Well well, so then I put up a new build ;)

I fixed the issue with the first pass crashing (dumb error from my side, tried to print out an __int64 as int...).
Further -h solved the problem where aborting an encoding process and starting over would result in a "can't start video compressor: -100" error.
Else it's just the most recent CVS which most of my changes incorporated already, but readded debug output on both passes to see if everything works alright.

Best Regards,
Koepi

[ChristianHJW]

@koepi, @nic, @-h :

nice battle you have here ... who is the leader in nr. of new builds per day right now :D ? If any of you could find interest in writing a MCF parser instead you made me a happy man .... Tronic stated there would be happening something on monday .... and he wanted me to organize that a MCF parser is being done .... but i honestly dont have a clue who to ask now :( .... Nic would be my 1st bet, with respect to his unrivalled background in DirectShow, but he fell in love with XviD it seems .... ;)

[-h]

Heh well I haven't used, let alone compiled the dshow filter yet, and haven't even looked at dshow as an API. So I might not be your best bet :)

Then again, I'm running low on ideas for XviD that I'm experienced enough to implement. Might be time for a change.

-h

[ChristianHJW]

You're gladly invited to help us out -h .... i remember when Matthias Lenk coded the excellent Dedynamic filter, he said it wasnt big deal at all. The main core of his Dedynamic.exe was coded in assembler ( for speed ).... all he did was to download the M$ DirectShow SDK and played with one of the example filters, i guess it was called 'gargle' or so .... 2 days after that i had the 1st alpha on my HDD .... it worked fine from the very 1st day ... Blacksun implemented it in PowerDivX and the rest is history .... you see, its very easy :D ...

[Nic]

I did offer to give up the "battle"....But I think Koepi's winning, but ill keep mine up for now :) (Ive got it so it does the CVS download & compile from a batch file :)

(Hmmmm...The new project manager at work has very good DShow experience....maybe I can rope him in as well :) :)

-Nic

[Nic]

Latest build up :)

www.freewebz.com/xvid

-Nic

[rui]

Nic, the dll file is the same size than Koepi's dll.
This means that you are back to VC 6?
Any problems with VC 7?

[Koepi]

Erm... I'm not in a battle here. I'm trying to do some extra coding on XviD to improve it - so it has nothing to do with the usual binaries. And I may not do this for long.

*grrrrr*

All my releases have some "extra features" that might break something, result in unpredictable behaviour, is unusable for very low bitrates etc etc.

I won't upload my actual binary for that reason, just to show that I'm not willing to be in a combat.

Just a hint:

The test with modulating the quant_type on a per-frame base (dependent on the quantizer) works like charme and results in a sharper overall movie. But it triggers the DivX4-DS-Filter playback bug giving some green areas (very seldom, but they are there).

Best regards,
Koepi

[saVe]

is there no way you could make up your mind? i'd love to test your "special" binaries..... please?

[Nic]

@Koepi:
Oh, theres no battle, your doing far more than me...Keep up the good work, I only started to do it, because uManiac was updating too slowly....& because its very easy.

@Rui:
I did go back to VC6 your right (very sharp :) I did a very brief test on a SVCD of Lord of the Rings....I was getting about 26fps with VC6 & about 25 with VC7, I think???? Im not sure...There definitely isn't any huge speed increase....However there are some settings I could play around with in VC7 to make it that little bit faster.
Seeing that most of the important bits of the codec are ASM the compiler cant speed that up. So for now VC6, unless anyone noticed improvements when using VC7....In which case ill go back to 7.

(I should try compiling DVD2AVI & Lame now with VC7 :)

-Nic

[-h]

Erm... I'm not in a battle here. I'm trying to do some extra coding on XviD to improve it - so it has nothing to do with the usual binaries. And I may not do this for long.

Ominous words? BTW the extra debug output that you added is in the CVS, just uncomment the junk after DEBUG2P(), DEBUG1ST() and DEBUG2ND() in codec.h, and put it on the same line as the respective #define. I remember reading that you thought it was only enabled in CVS Debugg builds?

The test with modulating the quant_type on a per-frame base (dependent on the quantizer) works like charme and results in a sharper overall movie. But it triggers the DivX4-DS-Filter playback bug giving some green areas (very seldom, but they are there).

Yeah they're dragging their feet when it comes to fixing the MPEG quant issue. Just wondering, what threshold did you use to switch from MPEG to H.263, and back again?

-h

[rui]

Originally posted by -h


Yeah they're dragging their feet when it comes to fixing the MPEG quant issue
-h

Arghh!!

Those divx4 guys... :rolleyes:

Just wondering.. It's still far the time when XviD will have its own post processing filter? Then we wouldn't need them anymore.

[-h]

Just wondering.. It's still far the time when XviD will have its own post processing filter? Then we wouldn't need them anymore.

Ehh....

If I have some spare time this weekend, I'll look at duct-taping ffmpeg's post-processor onto XviD. I've never used it before, nor do I know if it's any good whatsoever, but the source is there to play with.

-h

[Koepi]

Originally posted by -h

Ominous words? BTW the extra debug output that you added is in the CVS, just uncomment the junk after DEBUG2P(), DEBUG1ST() and DEBUG2ND() in codec.h, and put it on the same line as the respective #define. I remember reading that you thought it was only enabled in CVS Debugg builds?

Uh, stupid me! I just tried commenting that out without moving it on the same line... well, I'm still no pro when it comes to coding ;)

Yeah they're dragging their feet when it comes to fixing the MPEG quant issue. Just wondering, what threshold did you use to switch from MPEG to H.263, and back again?

I hardcoded using MPEG4 quant_type for quantizers 1-3, .h263 for 4-31.



Best Regards,
Koepi

[rui]

Koepi, i too would like to try your builds. The alternating quantizer seems a good ideia, at least to me. (i am no coder). Then we hould have the best from both worlds.
Your site clearly (is this correct?) states that the build is experimental, so anyone that tries it should accept any little bugs.
Your work is very apreciated, be sure of this.

Hope -h can get that post processing working ;)

[Koepi]

I made my stand, to show that I'm not willing to "fight" or anything I'll post no release today.
There's something left to do which improves the release a (very) little, something you could describe as "dictating the core to use I or P frame at our will".
With that modification there won't be any I-frames out of the lock range.

I may post that resulting binary tomorrow or so - but nothing today!

Best regards,
Koepi

[Nic]

:)

I think people know theres no fight or anything Koepi....Why would there be? :)

Im sure people will look forward to it tomorrow :)

Take Care,
-Nic

[ChristianHJW]

Originally posted by Nic Hmmmm...The new project manager at work has very good DShow experience....maybe I can rope him in as well :) :)Nic
.... yeah, good idea, drag him here :D

[ChristianHJW]

Originally posted by Koepi Erm... I'm not in a battle here. I'm trying to do some extra coding on XviD to improve it - so it has nothing to do with the usual binaries. And I may not do this for long.
Sorry mate, i didnt want to step on anybodies feet, definitely not my intention. I was basically posting my comment because i thought it could help to win Nic for the MCF parser project :D .... but he decided to help my anyway and couldnt get through to me because my inbox here is going crazy ( ?? ) ..... never mind, keep up the excellent work on XviD my friend !!

[Ripe73]

HI Again!!!

Encoded a movie with XVIDCVS 150202
132678 frames
2pass H:263
L.mask second pass
M.Search 5
I-Frame Min:1 Max:4
Smooth Quantizer
The movie was 3MB oversized:D

Did a new second pass with I-frame Min:1 Max:6
the rest samme settings
perfect size:)

The Debugview did not work with this build for me. :mad:

Thanks

[Koepi]

As I wrote earlier, I-frame quantizer lock can produce oversized files when given to strong tresholds.

So, nothing to be surprised of :P

Regards,
Koepi

P.S.: the debug output is disabled by default in CVS.

[-h]

As I wrote earlier, I-frame quantizer lock can produce oversized files when given to strong tresholds.

I haven't actually encoded anything over 2 minutes in length with XviD - are long movies uniformly oversized by 2 or 3 MB? I have always wondered about the 2-pass code not taking into account AVI's overhead per frame - over a long movie, this has the chance to add up to a few MB oversizing, and it'd be easy to just add the AVI overhead (~24 bytes per frame?) to the 1st pass frame size before it's written to the .stats to fix this.

Then again, if no uniform oversizing is occurring... I'm just confused :)

And yes debug output is disabled by default in the CVS. My next commit will enable the 2-pass-specific debug output (you don't want to see everything outputted by XviD..)

-h

[Koepi]

Hm. I tried a single movie 3 times, targeted ~610mb, got always the _same_ size, ~613mb.
We might try to add 24 bytes per frame, the movie had 122000 pictures, resulting in...
2.79MB overhead.
would explain the matter perfectly!
But I would simply use that on second pass, not for the stats file, but in overflow += frame->length + 24 ;)

Best regards,
Koepi

[Foo]

Originally posted by -h
I haven't actually encoded anything over 2 minutes in length with XviD - are long movies uniformly oversized by 2 or 3 MB?



Yes :). I've done several full length features, both one disc and two. Each one is just *slightly* oversized. I actually came here tonight to ask about this...nice timing.


I have always wondered about the 2-pass code not taking into account AVI's overhead per frame - over a long movie, this has the chance to add up to a few MB oversizing, and it'd be easy to just add the AVI overhead (~24 bytes per frame?) to the 1st pass frame size before it's written to the .stats to fix this.


Please do! Thanks for all the hard work everyone (-h, Nic, Koepi, etc). This would just be one more nice little thing about Xvid. Right now I target at 698 to be safe.


foo

[Koepi]

@-h:

Morning,

I just did a few modifications to codec.c, may you have a look at this?

in "int codec_get_frame_quant"

I extended "frame->intra = -1" to


if ((codec->config.mode == DLG_MODE_2PASS_2_INT) || (codec->config.mode == DLG_MODE_2PASS_2_EXT))
{
frame->intra = (nns1->quant & NNSTATS_KEYFRAME);
}
else
{
frame->intra = -1;
};


and (just a temporal workaround since my stats file is already done)


case DLG_MODE_2PASS_2_INT :
case DLG_MODE_2PASS_2_EXT :
// Modified by Koepi
codec->overflow += bytes2 - frame->length -24;


added a -24 (AVI frame overhead) in "int codec_2pass_update".

What do you think about these modifications, are they correct?

Regards,
Koepi

[Koepi]

Well, there were no objections so I uploaded new binaries.
Find them at http://roeder.goe.net/~koepi/ .

Some of the changes:

- Quant type modulation implemented, you have to select that on second pass in the config GUI (quant type: modulated).
- Added the AVI header size to overflow on 2nd pass internal. File size should now be precise to some kb, if the config settings aren't too strict.
- Frames that were smallest size possible (around 101 bytes) don't get scaled anymore (well, 101 bytes hardcoded now, have to figure out how to find the treshold).
...
and some things I forgot.

Modifications to the sources are included in the archive as usual.

Best regards,
Koepi

[Demone]

I encoded Aliens with Koepy's build (not the last one) and I cant tell the quality in % but I think its something near 60-70% at 576*xxx
in 3cds with dolby. I must say that I cant notice pink green and other shit, but there is something strange...when there is a scene change u actually can notice the keyframe, I mean there is a jump in bitrate that u can see in picture quality and after (quite fast) the quality deteriorate, mantaining a constant (but not good as the keyframe) quality. It seems to me that or the decoder is not working as it should or someone must put hand on the i-p-b frame relation.

I got and AthlonXP 1900 + K7V266-e geforceMX +latest
I did a two pass with the second internal, giving 1718000 as final size using mpeg-quant 300maxkf and no lumi in either 1st or second pass.

[Koepi]

It is important to know WHICH build you used.
This phenomenon you describes sounds like iframe quant lock without quantizer aliasing.

Regards.
Koepi

[Demone]

Its the 10022002 but was umaniacs...sorry koepy going to try ur latest.

[Koepi]

I made the attached graph below to explain what quantizer aliasing does (and what it can't do...).

See those intraframes (red). Those got caught by the iframe quantizer lock, as you can easy tell from the frame behind, which is smaller than the usual ones (right after that).

With the older builds, there was no quant aliasing and the frame right after such an intraframe got _very_ small resulting in a ugly frame/image.

now you can see, the frame still gets smaller, but the negative overflow is distributed over some frames (the other yellow arrows show this. There is a bigger frame missing where "usually" there would have been some).

This looks way better when watching the movie.

But I wrote already earlier, that this iframe quant lock and quantizer aliasing an't do wonders. It won't look nice if you set the values too strict as the p-frames get smaller (a higher quantizer is used) resulting in an overall nice, but strange look of the images.

I hope this helps explaining that phenomenon a little.

Best regards,
Koepi

[Franko30]

Originally posted by Koepi
Well, there were no objections so I uploaded new binaries.

- Quant type modulation implemented, you have to select that on second pass in the config GUI (quant type: modulated).
Best regards,
Koepi


Hi Koepi,

I just installed the new binaries and while encoding my first test file, I was wondering why Quantizer modulation should only be used on 2nd pass. I guess there's an obvious reason I just don't get the hang of right now...

Frank

[Koepi]

You need the statistical data on first pass on a constant DRF, that's why. The same reason applies to luma masking :)

The scaling on second pass would go wrong if you use that on first pass, resulting in a very ugly video...

Regards,
Koepi

[Franko30]

@ koepi:

Thanks!

I knew it was s.th. like that, just couldn't put it to words.

Another question:

Some people say that the MPEG quantization brings the effect where, on non-moving areas of one colour, the surfaces seem to be moving.

Did you notice that, too? Not that it matters much anyway, with my analogue sources...

Frank

[rui]

Originally posted by Koepi
The test with modulating the quant_type on a per-frame base (dependent on the quantizer) works like charme and results in a sharper overall movie. But it triggers the DivX4-DS-Filter playback bug giving some green areas (very seldom, but they are there).

Koepi, couldn't this be avoid by selecting XviD fourCC, in the codec tab? Or is the quantizer modulation for itself that triggers divx4 post processing even if we do that?
And regarding the question Franko30 made about the quantizer modulation being choose in the 2 pass:
in the first pass we can choose either H.263 or MPEG?

Thanks

[Koepi]

hm. in the first pass you can choose whatever you want (but don't choose modulated as that would result in MPEG quant type ;) ).
I use .h263 there, because most of my frames will use that quant_type later on. But you can try as well using MPEG, maybe this gives better results, mybe worse, maybe the same :) (if I understood it correctly, MPEG quant_type frames are bigger at higher quantizers, so the frames get scaled to big on second pass... it finally shoudl depend on the resulting average quantizer then. If you have more 2x and 3x frames, you should use MPEG on first pass, but if you have more frames >3x you should use .h263)

And nope, those modifications don't need the DivX4 DS filter, it's just that, if you use FourCC DIVX, that bug can get triggered. Everything should be fine with the XVID FourCC.

Regards,
Koepi

[serbersan]

I just encoded a movie with the 15 fb build of Nic and appears some macrobloks like freeze frames in divx3 several times along the whole movie but now I don't have antishit.

The effect isn't the same as freeze frames but very similar. The colours of them are strange red, pink, green, blue, pure colours.

In fact in one scene I've noticed the bottom of the screen with those colours merged during 3 or4 seconds.

I encoded the movie using the default values and using luma masking in 2º pass.

I've noticed too, black horizontal bars when the screen is absolutely in black.

Without this the overall quality is very good.

[MaTTeR]

Originally posted by serbersan
I just encoded a movie with the 15 fb build of Nic and appears some macrobloks ...

The effect isn't the same as freeze frames but very similar. The colours of them are strange red, pink, green, blue, pure colours.

In fact in one scene I've noticed the bottom of the screen with those colours merged during 3 or4 seconds.

I encoded the movie using the default values and using luma masking in 2º pass.


I can verify this as well. I used the same settings and got the same results. Try the latest build from Koepi now:)

[-h]

Well, there were no objections so I uploaded new binaries.
Find them at http://roeder.goe.net/~koepi/ .

Oops, haven't been online for a while.

- Frames that were smallest size possible (around 101 bytes) don't get scaled anymore (well, 101 bytes hardcoded now, have to figure out how to find the treshold).

How often does this happen? The smallest possible frame size (with current builds) is (Vopheader + 1 bit per macroblock + markers). Or it should be, anyway..

In terms of frame size for H.263 and MPEG quantization, the relationship seems to be fairly linear. If I get around to coding a replacement for the frame.quant decision, I'll try implementing it and refining the 2-pass mode further when quant modulation is enabled.

Tomorrow should be a big day.

-h

[Koepi]

Hehe, fine then :)

My GF is away for ~48hours from now, so I can spend some more time on the computer without getting into trouble, so it would be nice if we could use that time :)

Regards,
Koepi

[Koepi]

As I'm currently in the last minutes of encoding my movie, I see one thing we must change as well:

we should lift the iframe quant lock if credits are active... debugview output is REALLY ugly at the moment ;)

Regards,
Koepi

[Koepi]

Hi again,

now that the movie is finsihed, I wanted to post my results:

The quality is unbelievable. It is impossible to achieve with DivX3/Nandub IMHO (with these size limits etc).
My desired size was 569.6875MB, it came out 569.7109MB (so just a thing caused by iframe quantizer lock in the last few frames). So it's definatly been the 24 bytes avi-header-overhead per frame which gave a little size unpredictability!

Btw., the movie is The World Is Not Enough, 2 hours 1 minute long. I'll mux 2 soundtracks (vorbis, qual 0.01) with it into 1 ogg and hopefully it hits 700MB... :)

So far my results,
Regards,
Koepi

[Synth]

What res what the encode at?

[Koepi]

640x272.

As you would expect, sometimes large unicolour areas are a little blocky, but hey, I now have a bilingal movie of 2 hours in really NICE quality on 1 cd.

:)

[ChristianHJW]

@koepi

What was the average bitrate of your Vorbis audio streams at q 0.01 ?

[Demone]

I tried ur latest build and I'm comparing it with 3.11, for backgrounds xvid is better, but it seems that the 3.11 has a finer detail, seems that xvid is more brutal as image quality, I mean grainy, but I'll do more tests.
Can u put up some seconds of your encoded movie ?

[Ripe73]

HI!
Use XviD-16022002-1 and when i watch the Debugview i see alot of double keyframes and sometimes more like 5-6 keyframes(in a row) .Whats wrong?

EDIT
[3524] [XviD-2nd] Quant:2 .h263 INTRA Stats1:31079 ScaledTo:23324 Actual:31079 Overflow:-21920
[3524] [XviD-2nd] Quant:3 .h263 Inter Stats1:31055 ScaledTo:23306 Actual:21472 Overflow:-20110
[3524] [XviD-2nd] Quant:2 .h263 INTRA Stats1:29100 ScaledTo:21838 Actual:29100 Overflow:-27396
[3524] [XviD-2nd] Quant:2 .h263 INTRA Stats1:30710 ScaledTo:23047 Actual:30710 Overflow:-35083
[3524] [XviD-2nd] Quant:2 .h263 INTRA Stats1:30710 ScaledTo:23047 Actual:30710 Overflow:-35083
[3524] [XviD-2nd] Quant:5 .h263 Inter Stats1:29614 ScaledTo:22224 Actual:12368 Overflow:-32887

EDIT AGAIN :D
When i jump between keyframes in V-dub i cant see any (K) on these framenumber Debugview showed as keyframes:rolleyes:

[Synth]

Im using Koepi's 16-02-2002 build.

What do you suggest for min and max iframe quant?

What impact (visually) does 'smooth quantizer fluctuations' have? In other words, whats the benefit of this function?

Is the only limitation of using the xvid decoder (as opposed to using the div one) the fact it cannot post process?

If indeed the only limitation is that, for HQ encodes where enough bits are available wouldn't it better to have the xvid do

the decoding?

Lastly, for purpose of burning onto cd / future playback...Do you suggest I burn with xvid set as the decoder, so that in the

future when it does get post processing abilities I could benefit without having to re-burn and change the header from div to

xvid, or will the issue (green & pink bugs) with div 4 decoder be resolved eventually?

[Acaila]

ChristianHJW wrote:
@koepi

What was the average bitrate of your Vorbis audio streams at q 0.01 ?

I'm not Koepi, but I just wanted to respond anyways :)

At quality 0.01 I always get 64kbps for 2ch audio and 192kbps for 6ch audio. Is this normal?
Like if I wanted a 6ch stream at for instance 128kbps it's not possible?

[Koepi]

That's the nominal rate, this is lowest: 32 kbps per channel.
2 channels result in 64kbps nominal, 6 channels in 192kbps nominal (6 channels has no channel coupling btw...).

I got (of course) 64kbps nominal, german audio was 63 kbps real, english 62...

Regards,
Koepi

[MaTTeR]

@Koepi

With your new build (XviD-16022002-1) I'm still seeing the nasty green bugs. I was thinking this bug had been fixed.

Auto Optimizations for PIII
Gknot 2ps with Luma-Masking
Used both WiMP, PowerDivX 4a7 and PD4a8.

The artifact is mostly green but I do see other colored blocks as well. Let me know if you need more information.

Thx,
MaTTeR

EDIT- I should add I used the default encoder setting motion search(5),H.263 and FourCC=XviD.

[-h]

What do you suggest for min and max iframe quant?

I'd say 2 to 3, for a 2-CD rip. Or 2 to 5 otherwise.

What impact (visually) does 'smooth quantizer fluctuations' have? In other words, whats the benefit of this function?

When you change the I-frame quantization settings from 1 to 31, the codec is no longer able to assign an I-frame exactly the quantizer that it wants to. Because we're overriding the 2-pass code (and usually making I-frames much larger by doing so), the frames immediately following the larger I-frame will be quantized heavily to keep the desired size in check. Since such quantization jumps look pretty ugly, Koepi came up with the idea to restrict jumps in subsequent frame quantizers to +- 2.

Is the only limitation of using the xvid decoder (as opposed to using the div one) the fact it cannot post process?

Yes.

If indeed the only limitation is that, for HQ encodes where enough bits are available wouldn't it better to have the xvid do the decoding?

Not at all.

Lastly, for purpose of burning onto cd / future playback...Do you suggest I burn with xvid set as the decoder, so that in the future when it does get post processing abilities I could benefit without having to re-burn and change the header from div to xvid, or will the issue (green & pink bugs) with div 4 decoder be resolved eventually?

In the future, you'll be able to play avi's that have the DIVX FourCC using the XviD decoder, so it won't really matter too much. Depends on preference I guess.

-h

[Synth]

@ -h: Thanks.

@ Matter: Using the XviD-16022002-1, I get pink bugs (okay more like purple) with my athlon and motion search=6, Quant=MPEG, NO lumi.

But it goes away when using the Xvid as the decoder.

I was so sure it was a strictly divx decoder fault...Now maybe I think its also a lumi issue? In anycase isn't the lumi function still much a work in progress, more so than the rest of the codec? Thats why I'm staying away from it for now. Plus my encodes look better without the lumi enabled.

[Synth]

What about the user being able to choose the level of "smooth quantizer fluctuations" instead of the fixed +/-2?

[MaTTeR]

@Synth

Thanks for the feedback. Maybe your right and it is the Luminance Masking. I'm going to rerun my 2nd pass with it set to off and try again.

I'm using the XviD as the decoder and still seeing the issue. Is there a way to ensure I'm using XviD? I thought I seen mention of a utility that would change the FourCC code.

EDIT- I gotta say that the overall quality of this latest build is simply superb. I hardly see any macroblocks in the shadows:)

[Synth]

I just tried WITH lumi...but xvid decoder..no bug.Course its entirely possible it could be the specific source/scene itself that helps initiates this bug or a combination of factors.

Its only a matter of time before we narrow down the exact reason and fix it.

I dare say using 2CDs with this latest build gives quality almost indistinguishable from actual dvds.

FourCC changer here:
http://www.inmatrix.com/

BTW Im also using iframes 1-3 and 'smooth quantizer fluctuations'.

/me remembers the many weeks of encoding time spent trying to get the holy grail of SBC settings in his youthful days :D

[Synth]

Just a random idea...check the "About" section in the codec..check if the date is the 16th.