Ciao! (XviD-1.0-Beta2-05122003)

[Koepi]

Aloha!

Finally, beta 2 is out - Codename Ciao.

Changelog:

XviD-1.0-Beta2-05122003:
- Fixed bitstream headers to be mpeg4-compliant again.
- Quant matrices are finally threadsafe.
- Interlaced encoding is fixed.
- VFW frontend fixes and misuses of core.
- Final tweaks to frame decision and motion estimation.
- Fixed overflow degeneration (Q31 problem with i-min-distance >1).
- Fixed CBR mode - the first frames weren't capped as wanted.

Please report further bugs (from this build) here, I closed the old thread as it was getting too huge.

Best regards
Koepi

[tiki4]

Thanks, testing...

tiki4

[raz0r]

did first test, seems to be a lot faster.
always had ~5fps, now ~10.
encoded a very dark scene at ~1mbit, MSP-6 VHQ-4 H.263 trellis quant chroma motion adaptive quant (lumi masking) no GMC/QPEL/BVOP. source was pal progressive dvd (matrix reloaded), destination 640*272. quality is very good, but thats no wonder at 1mbit.

nice build, especially because of its speed. but still doesnt play on standalone :(

[P0l1m0rph1c]

Great work guys!

I'm testing as I speak...some results in a few hours.

Keep up with the great work!

:)

[Manao]

Thanks a lot ! Right on time between my first and second pass :)

[P0l1m0rph1c]

Originally posted by Manao
Thanks a lot ! Right on time between my first and second pass :)

That's not very recomendable. You should finish your movie with beta1 and then move on to beta2. If i'm wrong, plz someone correct me.

[Manao]

Well I don't think it will have a big influence on the result. And having an mpeg4-compliant header is (imho) more important than missing the wanted size by a few KB.

[zettai]

Originally posted by Manao
Well I don't think it will have a big influence on the result. And having an mpeg4-compliant header is (imho) more important than missing the wanted size by a few KB.

This release of the codec has had "tweaks to frame decision and motion estimation" and you dont think it will matter??

[Prettz]

I have a question about GMC. Is there any way at all for GMC plus VHQ4 and MPEG quant to result in lowered quality? Or does it now either not help but not affect quality or save some bits but still not affect quality adversely?

What I'm getting at is if you don't mind the loss of speed in encoding and decoding, is there any reason NOT to always enable GMC just for the heck of it?

[Manao]

zettai : I won't be sure until a developper answers that particular point, but AFAIK frame decision are made during the second pass ( and in the first when choosing between p & b & s frames ). Motion estimation is important in both passes, but since the previous ME was already very good, I don't think the ME in the beta 2 would be completely different. It may lead to a little imprecision in the final size, but I don't think it'll be sensitive.

[HarryM]

Old good 'blooming' delay b-frame message ('WARNING: NOTHING TO OUTPUT, BFRAME DECODER LAG') at decoding (xvid.ax) is resolved/removed?

[cipher]

Great work, guys!

But since it hasn't been fixed in beta2, I'm gonna state this again :)-----LigH has found that, what should be "Global Motion Compen>>s<<ation" is now shown as "Global Motion Compenation", where an "s" is missing :)

[Koepi]

Frame type decision is done in first pass, so you should redo the first pass - though since it is written in the first pass statsfile the encoder will oblige to that :)

About GMC, well, it's very slow and doesn't help too much. To be really useful it has to be tweaked a bit more. Bt sure, you can enable it if you don't mind the additional cpu power and memory necessary.

As the GUI will maybe change later we didn't give too much importance to a missing letter ;)

Regards
Koepi

[cipher]

About GMC, well, it's very slow and doesn't help too much. To be really useful it has to be tweaked a bit more. Bt sure, you can enable it if you don't mind the additional cpu power and memory necessary.

is GMC now using 3 warping points or 4?
Thx.

[Koepi]

I never heard of 4 warppoint GMC. This is "still" 3 warppoint GMC.

Ah, and about that decoder lag message: it'll stay in there as long as we don't do special DirectShow-coding for the decoder :-/ Using ffdshow (libavcodec) for decoding solves the issue.

Regards
Koepi

[mfluder]

Originally posted by mfluder
I noticed this during the testing of my dev-api-4 compiles but I wanted to wait for the official betas. When decoding dev-api-3 streams (Koepi's latest) and the height is not multiple of 16 (it's multiple of 8, 640x360) there is a bleeding at the bottom of the clip. I don't have any webspace so I can't post a screenshot but you can spot it easily. There is no problem when the clip is encoded with dev-api-4.
Sorry guys for bringing this up again but is it possible to add some kind of workaround for this problem? For example, add a condition in decoder so when bitstream version of dev-api-3 builds is detected then apply this workaround. I know it's possible to decode this kind of files correctly as ffdshow doesn't have any problem with them.

Thanks for all your work,
mfluder

[Manao]

Ok, I'll do another first pass ( btw, I can keep both stats files if you want ). It'll only be the tenth time I made a first pass for this movie ( that's mainly why I was so reluctant :rolleyes: )

[mf]

Still I find these betas slightly early. Wasn't it supposed to be so that 1.0 had core and vfw separate, and a proper GUI (ok well we know the story about the GUI..) ?

[Koepi]

ok mf, you're right. I take back the betas and remove them from my site ;)

(Or what do you expect as answer to such a [unconstructive] post?)

Regards
Koepi

[AgeOfPanic]

Ik know now that I can't use Gordian Knot to control the proces, but can I still use it to setup my Avisynth-script?

[KpeX]

Originally posted by AgeOfPanic
Ik know now that I can't use Gordian Knot to control the proces, but can I still use it to setup my Avisynth-script?

Yes. AVS scripts are independent of the codec used.


Koepi & XviD team: thanks for a quick second beta.

[mf]

Originally posted by Koepi
ok mf, you're right. I take back the betas and remove them from my site ;)

(Or what do you expect as answer to such a [unconstructive] post?)

Regards
Koepi
Ok, it's hindsight. But it's not hard to admit faults later, is it? :D The nice thing is you don't have to do anything about it because it's already happened, you can only say "Oh that's Jim Fix isn't it? Oh what a f*cking tragedy." :p.

Edit: oh and next beta, do name it "Ahoy", I specially came to #xvid to ask that. :D

[Arhu]

Originally posted by sysKinOriginally posted by cipher
and in "zone", we could also force a range of frames to use keyframes(I frames) only;No, this means that a zone starts with a forced keyframe. You have to use it if you're starting a greyscale zone, but it was originally designed for chapters - it's good when a chapter starts with a keyframe, you have faster/more accurate seeking.
This would be nice, but it's not what I experience. If I set the keyframe modifier on the start frame of a zone and run the job, only I-frames are produced. Furthermore, if the first zone starts not at frame 0 but at a later frame, the same problem occurs starting at frame 0.
If I remove the "K" from zones, everything works fine.

By the way, could perhaps MAX_ZONES be increased? I'm doing TTT which has 68 chapters, and 64 zones are not quite enough...

Anyway, thanks for all your efforts, xvid team!

[P0l1m0rph1c]

Originally posted by AgeOfPanic
Ik know now that I can't use Gordian Knot to control the proces, but can I still use it to setup my Avisynth-script?
The answer is yes.

[eng3]

I think the "s" is missing in GMC "compenation"?

[Koepi]

Enrico:
Please read page one of this thread and find the report of the missing "s" and an answer to that.

Regards
Koepi

[eng3]

Originally posted by Koepi
Enrico:
Please read page one of this thread and find the report of the missing "s" and an answer to that.

Regards
Koepi


Oh, sorry, I thought I had finally made my first conribution :(

[Wilbert]

I guess I'm doing something stupid, but I can't get the specified target size. Source is an AVS script where a mpeg2 stream is loaded with dvd2avi.

I used the following options (2pass):

HDTV profile
trellis
VHQ4
I: 2-4
P: 2-4
target bitrate: 4000 kbps

But the resulting bitrate is always about 7000 kbps. This happens also if I choose "unrestriced + h.263 + settings above" for both passes.

If the encoding is finished I got the following error message (dunno whether it is related though):

[Manao]

What is the final quant dstribution ? Since your quants are capped at 4, if your quant distribution is composed only by quants 4, then you'll have to rise that capping value to 5 or 6 ( or you can even try not to cap ). If not, I don't know what you did wrong.

[Wilbert]

What is the final quant dstribution ? Since your quants are capped at 4, if your quant distribution is composed only by quants 4, then you'll have to rise that capping value to 5 or 6 ( or you can even try not to cap ).
Yes, you were correct :) Stupid that I didn't realise that.

[Leak]

Funny enough, the about dialog says "libxvidcore version 1.0.-127" - I've never seen a program with such a negative version number... ;)

(I guess the fix for this is right next to the other speling bug on Koepi's to-do list... :p)

np: Plaid - Marry (Spokes)

[Hoschi]

Originally posted by HarryM
Old good 'blooming' delay b-frame message ('WARNING: NOTHING TO OUTPUT, BFRAME DECODER LAG') at decoding (xvid.ax) is resolved/removed?

nope, it' still there. But this is no real problem...

As I have a large number of files encoded with xvid - I noticed that some files do not decode properly. Keyframes are OK, but all other frames are scrambled.

Is 1.0 incompatible to older releases or are these problems caused by errors of earlier releases ?

24062003 decodes properly...
I do not use ffdshow, playback using divx5 improves the image, but still some errors. What did I miss ?

[torsius]

@ Hoschi

what you missed is that the included xvid decoder is sketchy at best, not only with decoding it's matching build, but even more so with streams made from previous builds, that's why ffdshow is recommended for decoding

[Koepi]

Leak:

The version number is intentional. Before beta1 it was xvid 1.-127.0 ... it's just meant to show the progress. I agree though we should have raisen the number to 1.0.-126.

Hoschi:
Unfortunately we don't have something useful as ffdshow -automatic bitstream & workaround detection- in our xvid decoder (yet). There are plans to add something like that, but they don't seem to be high on the "urgent to do" list.

Regards
Koepi

[LigH]

Great thing!

- Interlaced encoding works fine.
- 2-pass encoding works with MPEG matrix, with or without B frames.

Now that's what I call a "beta". http://forum.gleitz.de/images/smilies/thumb.gif
__

(@ mf: Did you explain your "recommended settings" somewhere?)

[Zep]

Mf said -Why would this have to be something in XviD? Use VDub's job control and set it to first encode the VBLE, then encode the XviD first and second pass from the VBLE.


read my post that is what i said i do already and WHY I want this feature
to get rid of three passes which takes longer than 2 passes.
Oh and lets not forget you have to wait for the VBLE then set it up later
when done. You can't go to bed just yet you have to let the VBLE finish
then set it up. of course I'm sure someone could write some scripts or
something to automate this so we do not have to be around. My time
(which adds to the total time to to encode) and hassle is a huge factor
in this as well :)

.You'll find that it takes just as much time as filtering, VBLE and first xvid pass at the same time. Plus, if something goes wrong during filtering to VBLE, you'll still have something useful, which is not the case with XviD.


2 passes are always faster than 3. i have tested it myself on 1920 x 1080i .ts streams

the speed gain comes from only reading that 10 gig file in ONCE since
you are doing 2 things to the data when it is already in memory
when you have the chance.

There is a TON of time wasted having to read that 10 gig file in TWICE
(once for each pass your way)

You ALREADY have the data in memory so you should do as much as you
can to it while it is already THERE and doing the VBLE and Xvid first pass on those
pixels at the same time while you have that data is exactly what I'm talking
about. You will be surprised just how long it takes to read in a 10 gig file and
the VD/VDM overhead on all that data etc... This is where the savings come from

Your way = Filter and write once , read twice
My way = Filter and write once, read once

in all my tests on 44 minute streams at 1920 x 1080i I get about a 20%
time savings. (and this is not including my time and hassle factor)

[Sofus]

There is a small problem in the decoder, witch the new Beta2 build. Overlay (gamma, contrast, brightnes) is no longer accessible in the ATi Catalyst control panel. It was working in one of gamer´s builds from the 4th

Sofus

[mf]

Originally posted by Zep
2 passes are always faster than 3. i have tested it myself on 1920 x 1080i .ts streams

the speed gain comes from only reading that 10 gig file in ONCE since
you are doing 2 things to the data when it is already in memory
when you have the chance.

There is a TON of time wasted having to read that 10 gig file in TWICE
(once for each pass your way)

You ALREADY have the data in memory so you should do as much as you
can to it while it is already THERE and doing the VBLE and Xvid first pass on those
pixels at the same time while you have that data is exactly what I'm talking
about. You will be surprised just how long it takes to read in a 10 gig file and
the VD/VDM overhead on all that data etc... This is where the savings come from

Your way = Filter and write once , read twice
My way = Filter and write once, read once

in all my tests on 44 minute streams at 1920 x 1080i I get about a 20%
time savings. (and this is not including my time and hassle factor)
I don't know about you, but whenever I filter my PC has to throw huge frames around in memory and then dragging them through CPU processing at max speed, and doing anything beside that (encoding xvid first pass) is going to cost me more memory, bandwidth, and CPU cycles. Plus I will get a fragmented hard drive because I'm writing two things at around the same time, which will cause slow disk access on my second pass. My biggest concern here is bandwidth and CPU. Windows timesharing is not effective at having your CPU do two different things at the same time in an elegant(fast) way. By pushing your hardware that far you'll probably only create new bottlenecks to slow your processing down on.

[jarthel]

The options used are:

2 pass encoding with 273mb as target size
h.234
lumamasking on
gmc on
b-frames on (3/1.5/1) with closed GOV
motion search precision - ultra-high
VHQ wide-search
cartoon mode
default quantization with trellis on

Encoded using latest vdubmod CVS
using latest Avisynth CVS
-----------------------

1st pass is successful.
But 2nd pass always fails at the last frame.

and here's what I get (http://f1.pg.photos.yahoo.com/ph/jarthel/detail?.dir=/divx&.dnm=xvid-error.jpg).

[mf]

Originally posted by LigH
(@ mf: Did you explain your "recommended settings" somewhere?)
Nope, but there's not much explaining to do. It's just the settings that I consider best for my average XviD encode. I put it in my sig so I can update it anytime and the change would be reflected in every post I ever made, since "what's best" tends to change quite rapidly during active public XviD development (it was kinda frozen for a long time because of the halt of all api3 development and api4 being non-public).

[nanji]

Please read this thread
http://forum.doom9.org/showthread.php?s=&threadid=66208
If the 2nd-pass size is greater than 2G, xvid will keep using quant 2 value to compress. I guess it's because there are some issues about file size over 2G, which didn't happen before. It is very annoying for 4CD or 5CD files.

[jarthel]

Originally posted by nanji
Please read this thread
http://forum.doom9.org/showthread.php?s=&threadid=66208
If the 2nd-pass size is greater than 2G, xvid will keep using quant 2 value to compress. I guess it's because there are some issues about file size over 2G, which didn't happen before. It is very annoying for 4CD or 5CD files.

have you tried the latest beta? You used the older beta.

[outlyer]

Originally posted by Arhu
This would be nice, but it's not what I experience. If I set the keyframe modifier on the start frame of a zone and run the job, only I-frames are produced.My (only) frustrating test with KF modifiers showed same results.

[Manao]

nanji : in the source code of the interface ( config.h , beta 2 ), I found a "int desired_size", so effectively, if you enter a size over 2GB, it'll lead to an erratic behavior. The good news is that it should be ( I hope ) only a bug of the interface, because inside the core of the codec, the size is coded by an uint64. Anyway, it's very late here, so I may be totally wrong, and so wait for confirmation by Koepy or another developper.

[nanji]

Originally posted by jarthel
have you tried the latest beta? You used the older beta.
I read the fixed bugs, there is nothing about the error I got. And it worked in old versions of xvid.

[nanji]

Originally posted by Manao
nanji : in the source code of the interface ( config.h , beta 2 ), I found a "int desired_size", so effectively, if you enter a size over 2GB, it'll lead to an erratic behavior. The good news is that it should be ( I hope ) only a bug of the interface, because inside the core of the codec, the size is coded by an uint64. Anyway, it's very late here, so I may be totally wrong, and so wait for confirmation by Koepy or another developper.
I looked at the code and somewhat I think xvid codec addresses or at least considered about the problem, evidenced by /* Total length of each frame types (1st pass) */ uint64_t tot_length[3];. somehow this does not solve the problem.

[jarthel]

Originally posted by nanji
I read the fixed bugs, there is nothing about the error I got. And it worked in old versions of xvid.

Isn't it better to use the new beta and re-test and post back if the error still exists? It's possible that they themselves changed the code of that part and the problem was fixed.

[nanji]

Originally posted by jarthel
Isn't it better to use the new beta and re-test and post back if the error still exists? It's possible that they themselves changed the code of that part and the problem was fixed.
Yes, I have just tested with the new beta version as well. The problem still exists.

[nanji]

this might be the problem (i only have the code for beta1 version, but since beta2 still has the same problem it still applies)

/* Compute the target filesize */
if (rc->param.bitrate<0) {
/* if negative, bitrate equals the target (in kbytes) */
rc->target = (-rc->param.bitrate) * 1024;
else
...

/**********************************************/

line 4
bitrate is int\
(-rc->param.bitrate) * 1024 is over sized
a int type * 1024 represents -2G <-> +2G

[LigH]

Originally posted by mf
It's just the settings that I consider best for my average XviD encode.

Well - I cannot imagine any relation between those 3 numbers, and any of the edit fields of the XviD codec. Where may I e.g. insert numbers around 30.000?