XviD-14112002-1...

[ErMaC]

I too can corroberate the undersizing problem in the Alt CC usage. Just did a test with both targetting same filesize (230something megs, it's an anime episode) and normal CC hit it almost dead on, AltCC was undersized by 5MB. Build was Koepi's 25112002-1 and only exotic options were b-frames 3/150/100 and chroma estimation - no masking or anything else.
I look forward to a new build - so far uManiac hasn't had a new build and Koepi's last one is the 25th (which is only like yesterday, gosh that's SO OLD! :D)
Huzzah for the bleeding edge.

[kilg0r3]

@ ermac

did you make any comparisons between enabling/disabling chroma motion? What are the effects?

[ErMaC]

I haven't been testing it personally, I just went ahead and turned it on. I don't really care about the encoding speed - I only care about final quality. From previous threads about it, which have said it will improve the ME on animation, I figure it can't hurt the matter and so I turned it on. My 1st passes all look damn perfect. ^_^
I will run another first pass with Chroma ME turned off but with same settings otherwise and let you know if there's a change in filesize. From what I understood of the explanation, if the motion estimation is worse it'll take more bits to fit the same data so we would see a filesize difference.
Koepi et al: If my information is wrong please let me know. ^_^
I will set up the encode now and check it in the morning.

[serbersan]

@kilg0r3

I've made a rip of killerJoy a 70 minutes movie and in the first pass
the difference between use chroma estimation or not was 1Mb. And yes I had made 2 first passes the only difference between them in parameters was than one has enable chroma estimation and the other not.

The first pass was over 1000Mb

[kilg0r3]

that does not seem to be much of a difference regarding filesize, but, is there a gain in visual quality

[Koepi]

Yes, there is a huge improvement: it meakes blocking nearly impossible (well, with bframes and high quantization there is _of course_ blocking in i.e. smoke, but that isn't what I'm talking about here). Walls etc. look better with chroma me.

Regards
Koepi

[NoLogo]

@Koepi

Have you seen what I said about hinted vectors a few posts above ? Is it a bug or does it come from my system ?

Regards
NoLogo

[Koepi]

MVhints are broken for several weeks now :(

Maybe sysKin spends some time on that later, but it isn't top priority at all ...

Best regards
Koepi

[iago]

@Koepi and all,

After several constant quant 2 tests with the 25112002-1 binary, I still get the annoying smearing effect with "qpel" (though not in all test clips), which somehow becomes less visible when using "qpel + b-frames" (ffdshow 13/11 libavcodec for decoding).

As for the b-frames alone (4/150/100), imho it's literally great in terms of quality and compressibility gain! Also, using mpeg and h263 quantization types with b-frames gave me different results and different filesizes, which is a good sign, too, I believe ;).

I still prefer and find safer to use only b-frames at the moment.

best regards,
iago

[Koepi]

@iago:

thanks for the report.
I have a local build (maybe i'll put it online?) which even handles custom quant matrices again... Unfortunately I think I know what you mean by "smearing effect", but I don't think it looks too bad - it's just a little strange .) (like a temporal smoother on moving objects?)

Serefe dostum!

Best regards,
Koepi

[Didée]

About the smearing effect

Personally, I didn't experience that up to now with koepi's 25112002.

But, a friend tried to encode TombRaider with this build, and in the beginning sequence (spinning wheels, lots of information is coming into the frame from outside the top border, plus the sequence is pretty dimish/dark), very strong smearing appeared.
I don't know the exact settings - but Bframes 4-150-100 + qpel was used, no filters at all but cropping & resizing, processing in YUV2.

<EDIT> Deleted some misinformation the friend told me

[iago]

@Koepi

It's nice to hear that a new build (handling custom matrices too) is on the way! ;) Well, actually I don't know what to call this effect (smearing?) with "qpel", but on some sources I get it on flat surfaces such as sky, walls, etc., especially when there is also some sort of motion in the scene. Yep, it resembles the effect of (heavy?) temporal smoothing I guess.

Looking forward to the new build! ;)

take care my friend, ;)
iago


Edit: Btw, I forgot to mention that "qpel + b-frames" helps compressibility gain a lot, resulting in even more compressibility than b-frames alone and great visual quality (except for the "smearing effect" of course!). ;)

[NoLogo]

@Koepi or any developpers

After a first pass done with a given build (Umaniac's 20112002 one, in my case), using BF and only BF (3/200/150, DX5), is it possible to re-use that 1st pass with a more recent build and same options or do I have to keep the same build (or redo first pass) ?
I know it's not possible to make a second pass with different options but same build, but is it (or theorically is - or maybe if I'm lucky :)) with same options but different build ?
Because Foxer told me that any change in altCC wouldn't be a pb (as it affects on second pass only), but maybe there can be other, not viewable, changes.

PS: I didn't know for hints ME, hadn't used them for a while before my tests. But anything else (especially BF is real good: didn't know Mulholland Drive at 522kB/s could be so nice :)) is great.

Thx for everything

Regards
NoLogo

[Koepi]

Hm, there were changes in the ME routines affecting compressability, thus you need to do a full 2pass again, sorry for the bad news :-/

Regards
Koepi

PS: @iago, hehe, bframes+qpel+gmc+chroma me+lumi masking are my favorites ATM, I have 1st pass of 5th element down to 1.3GB with an amazing quality :)
(btw., i prefer 3/100/200...)

[NoLogo]

@Koepi

Ok, I hope then that I won't have any oversize... :(
One more question: do you consider that the really recent improvements (compared to the Umaniacs 20112002) really worth the try ? I mean, would you give up a 22 hours first pass, to re-make it with your new build ?

About Qpel or CM, I had tried them, but get strange "slipping colors" effects; I still consider these options as "really exotic", whereas BF... oh lovely BF !... :)

Regards
NoLogo

[HarryM]

Originally posted by NoLogo
@Koepi

Ok, I hope then that I won't have any oversize... :(
One more question: do you consider that the really recent improvements (compared to the Umaniacs 20112002) really worth the try ? I mean, would you give up a 22 hours first pass, to re-make it with your new build ?

About Qpel or CM, I had tried them, but get strange "slipping colors" effects; I still consider these options as "really exotic", whereas BF... oh lovely BF !... :)

Regards
NoLogo


I think, that you can use (without big risk) 1st pass from older build. My experiences are, that deviations are very little, practically unvisible!

[sysKin]

Originally posted by NoLogo
[B]One more question: do you consider that the really recent improvements (compared to the Umaniacs 20112002) really worth the try ? I mean, would you give up a 22 hours first pass, to re-make it with your new build ?Koepi knows how things work in 2-pass encoding, so he's very strict about using exactly the same settings etc for both passes. However, I noticed that it's even safe to change quant type from one pass to another and still have both correct size and quantizer distribution. I never tried this, but I think I'm going to make first pass with motion precision 5 and no CM, second with 6 and CM. I _bet_ it will work great.
About Qpel or CM, I had tried them, but get strange "slipping colors" effects; I still consider these options as "really exotic", whereas BF... oh lovely BF !... :)CM is a safe thing. It only extends motion estimation procedure, so it's just motion precision 7. It can't produce wrong bitstream.

Best regards,
Radek

[NoLogo]

@HarryM & sysKin

Thx guys, this reassures me :) but I guess it's a no-no tu add QPel or GMC only in the second pass, right ? Maybe I'll give a look at the new build.

@sysKin

About your .dll, are they for developpers only, or can they safely be used for testings ?
Sorry for saying CM was "exotic", actually you're right: the problem with "slipping colors" only occured with GMC; the fact is that CM, when I tried it, did nothing: nor size neither quant changes. But maybe this has changed in the new build (after 20112002).

Regards
NoLogo

[Boardlord]

@Nic

Hi Nic! I absuletely love your Dshow filter for XviD, but now it seems that there is a bug in it. I used your newest (from 11/25) filter to watch a film, but DirectVobsub (2.20) didn't show the subtitles. Whenever I seeked, I saw the subs for a split second, than they vanished again. DirectVobsub works with ffdshow, but your postprocessing method is broken in it right now.

Thanks 1000x!!

[Snakeisthestuff]

Hi i have tested koepi's 25112002 build yesterday at an anime source
with bframes (8 bframes 200/150) + qpel and got very good results, but i noticed that koepies xvid.ax, the bframes not correct decoded, there seams to be an affect like an dropped key-frame at severall times, ffdshow 13112002 decodes it well. the b-frames increases the compressabelity a lot but i noticed this smear effect iago is talking about too, i think 8 bframes are maybe too much, but for an anime this smearing is not so bad how for an other movie :)

thx alot for this great build koepi :D

[ErMaC]

OK For those interested, here's status on my encodes:

Galaxy Angel 3rd Epsiode 1-2, 24:55 long
First pass, with Chroma ME: 355,174,400
First pass, w/o Chroma ME: 362,952,704

That's 338MB vs. 346MB, a fairly substantial difference. They both subjectively look the same to me, I think of course the smaller filesize comes from more accurate motion estimation meaning less bits needed to compress the same thing.
I suspect the reason this test was so positive was because it was done on animation (very clean digital one, too).

I'm also of the opinion that now b-frames are good enough for primetime, but I'm still keeping away from Qpel and GMC until they're more mature.

[iago]

... @iago, hehe, bframes+qpel+gmc+chroma me+lumi masking are my favorites ATM, I have 1st pass of 5th element down to 1.3GB with an amazing quality ...

@Koepi,

All are OK imho, except GMC for the time being! ;) I got terrible color artifacts with it in my Indian Runner encode.

If you only knew what kinda scripts I've been trying for a while for 1CD encodes with LanczosResize and 0.400q vorbis audio! :D (Pardon, did you call me crazy?! ;))

However, I'm sure we'll achieve that soon with a reasonable resizing of minimum 576*... and "b-frames + qpel + cm + a good lumi masking algo and custom matrices + the proper use of certain avisynth filters"! ;)

Since I _know_ that LanczosResize is _the_ resizing method, ogg vorbis (at ~128 Kbps) is _the_ audio (for 1CD), and XviD is _the_ codec!

þerefe dostum! ;)

and best regards to all,
iago

[Koepi]

@iago:

i had _no_ problems watching those movies with the latest pre-p4-optimizations ffdshow :)

Ah, well, the problem may be: I've a local build that differs from the public build... i wanted to see the results with 5th Element before i put it online.

It seemed to work flawless (in a previous version I have to admit) with "kiss of the dragon", but I thought it would be nice testing the functionality before giving it to the public, it's really disturbing getting useless bug-reports for things that are known to happen.

(No offense in this, it's just something like quality-assurance).

*hicks*

To the next drinks... þerefe!

Koepi

[NoLogo]

@iago

Of course you use Convolution3d, do you ? :)
God bless highly compressible movies.

@Koepi

If you want, I can make test series on my poor computer, running 24 hours a day, 168 hours a week. :D

Regards
NoLogo

[lighty]

Originally posted by Koepi
bframes+qpel+gmc+chroma me+lumi masking are my favorites ATM, I have 1st pass of 5th element down to 1.3GB with an amazing quality :)
(btw., i prefer 3/100/200...)

Just to report that I just finished encoding with almost the same settings: bframes+qpel+chroma me+lumi masking+AltCC-Medium

The results are GREAT. I mannaged to finaly encode some movies I had problems with in the past. That and the fact that trbarry provided TomsMoCompYV12 and Marc FD mpeg2dec3YV12 makes me a very happy puppy. Thx to all ppl involved!:cool: :cool: :cool:

[kilg0r3]

um, i have got a problem here with the latest koepi dev build

i shifting blocks before keyframes. here (http://www.mynetcologne.de/~nc-allgeife8/blocks-683KB.rar) are some screen shots from a clip i encoded recently. For this one i only used bframes (3/150/0 or 100 not sure) without DX50 compatibility this phenomenon occurs before scene changes and is visible in vdub, mplayer+ffdshow, mplayer+xvidDec. it occurs in avi container as wellas ogm. the avi was a created from the ogm though.

The screen shots are taken with virtualdubmod.

i hope i am not repeating something already mentioned here. If so, please tell me.


ok stupid me, DX Vop compatibility seems to fix this. Is this a known issue?

[Rrrough]

DX Vop compatibility seems to fix this. Is this a known issue? AFAIK switching off DivX-compatibility means that you'll get b-frames which are related to the following I-frame. DivX doesn't do that, its bframes are correlated only to P-frames. I guess I-frame related bframes are ISO-compliant, but no decoder is supporting it. thus you can't play your file without these huge shifting blocks (just like here, always before keyframes) and it's recommended to always switch on DivX-compatibility for the moment.
please correct me if I'm wrong here.

cheers

[lighty]

Originally posted by Rrrough
I guess I-frame related bframes are ISO-compliant, but no decoder is supporting it. thus you can't play your file without these huge shifting blocks (just like here, always before keyframes) and it's recommended to always switch on DivX-compatibility for the moment.

I don't experience any of the probs you talk about and I don't use DX compatibility? BTW- I use FFDshow.

[Rrrough]

well, I haven't tried with the latest ffdshow version, maybe it's "solved" now, but I guess kilg0r3 tried with latest !?

[jpl]

My settings are for B-frames are

packed bitstream = on
divx compatibility = off

and I have no problems either (using ffdshow)

[kilg0r3]

the problem is not ffdshow related. as i said, it appered with ffdshow and the with the standalone xvid decoder. yet, most importantly it also shows in vdub which uses the xvid.dll and not the directshow filter (no divx installed on my system).

@jpl
i though packed bitstream was still buggy ..
:confused: i don't have enough time to keep up with the changes of status of the xvid features, it seems. :(

BTW is the 'smearing' issue with qpel resolved now?

[iago]

Movie length: ~120min.
Audio: 0.700q ogg
Video size: 558000 kb
700 mb 1CD encode

encoding parameters:
--------------------
XviD 25112002-1 / b-frames 4-150-100 - FourCC: DX50 - DX50 B-VOP Compatibility / h263 / chroma motion / lumi masking / internal linear scaling

script:
-------
LoadPlugin("C:\FILTERS-YV12\MPEG2Dec3.dll")
LoadPlugin("C:\FILTERS-YV12\DctFilter.dll")
LoadPlugin("C:\FILTERS-YV12\UnFilter.dll")
mpeg2source("D:\GHOSTDOG\GHOSTDOG.d2v",cpu2="ooooxx")
crop(2,16,-2,-16)
LanczosResize(576,304)
LumaFilter(-2)
UnFilter(-10,-10)
DctFilter(1,1,1,1,.7,.4,.4,0)

Results: Great and no A/V synch. problems!

best regards ;),
iago

[DBaT]

Iago:
Good to know your 2h movie turned out ok.

I'm trying to get a 2h 4:3 movie on one cd with decent picture. I'm not there yet but while I learn every weeks something new and new xvid builds and avisynth filters turn up every week I'll have atleast hope.

I made some short quant2 tests with b-frame to see what different b-frame settings would do to file size:

test1 no b-frame -1 21118KB
test2 b-frame 2 150,100 16630KB
test3 b-frame 2 200,100 15904KB
test4 b-frame 2 200,200 15528KB
test5 b-frame 2 200,300 15266KB
test6 b-frame 2 300,200 15082KB
test7 b-frame 2 300,300 14934KB

test8 b-frame 3 150,100 16552KB
test9 b-frame 3 200,200 15384KB
test10 b-frame 3 200,300 15098KB
test11 b-frame 3 300,200 14906KB
test12 b-frame 3 300,300 14746KB
test13 b-frame 4 300,300 14758KB

[NoLogo]

Movie length: 2h20
Audio: 2 x Ogg q1.5
Video: Xvid 537598kB (500kBs) 720x384
700 MB 1CD encode

Encoding parameters:
--------------------

XviD 20112002U / BF-3-200-150 - FourCC: XviD - DX50 B-VOP Compatibility - MSP:6 - MPEG.

Script:
-------

LoadPlugin("E:\Rippage\Progz\GORDIA~1\mpeg2dec3.dll")
LoadPlugin("E:\Rippage\Progz\GORDIA~1\Convolution3d.dll")
mpeg2source("E:\Rippage\Ripp\D2vz\muholland_drive.d2v",lumoff=20)
Convolution3D(preset="movieHQ")
crop(0,8,720,560)
trim(0,204900).BicubicResize(720,384,0,0.5)+trim(204901,0).BilinearResize(720,384)

The results are amazing at such a Bitrate, it almost do not need any post-proc...:eek:
The movie may be highly compressible, I wasn't sure I could have done it... but it's done :D

Here a few stats:
-----------------

Quantizers Analisis
---------------------

Quantizers Used For Movie :
------------------------------
Quant 2 Used : 6671 Times, Percentage Used : 10.33%
Quant 3 Used : 50599 Times, Percentage Used : 78.35%
Quant 4 Used : 7196 Times, Percentage Used : 11.14%
Quant 5 Used : 109 Times, Percentage Used : 0.17%
Quant 6 Used : 8 Times, Percentage Used : 0.01%
Quant 7 Used : 1 Times, Percentage Used : 0.00%

Average Quantizer Used for Movie : 3.012

MPEG Quantization Type Used 210856 timed, Percentage Used : 100.00%

Quantizers prevented from rising too steeply 20 times


Intra-Frame (Key-Frame) Quantizers
------------------------------------

Movie
-------
Quant 2 Used : 1038 Times, Percentage Used : 0.72%
Quant 3 Used : 45 Times, Percentage Used : 0.03%
Quant 4 Used : 28 Times, Percentage Used : 0.02%


Inter-Frame (P-Frame) Quantizers
------------------------------------

Movie
-------
Quant 2 Used : 5633 Times, Percentage Used : 8.52%
Quant 3 Used : 50554 Times, Percentage Used : 76.46%
Quant 4 Used : 7168 Times, Percentage Used : 10.84%
Quant 5 Used : 109 Times, Percentage Used : 0.16%
Quant 6 Used : 8 Times, Percentage Used : 0.01%
Quant 7 Used : 1 Times, Percentage Used : 0.00%


Frame Analisis
----------------

Number Of Intra-Frames (Key-Frames) : 144708
Number Of Inter-Frames (P-Frames) : 66141
Total Number Of Frames : 210849

68.63% of the Movie is Intra-Frames (Key-Frames)
31.37% of the Movie is Inter-Frames (P-Frames)


Size Analysis
----------------

1-Pass Size : 634188174 Bytes or 619324 KBytes or 604 MBytes
Scaled Size : 418319040 Bytes or 408514 KBytes or 398 MBytes
Actual Size : 335431105 Bytes or 327569 KBytes or 319 MBytes
Actual Size of Avi File without Sound : 340603904 Bytes or 332621 KBytes or 324 MBytes

Usefull Statistics
------------------
Compressibility : 52.89%
Relative Quality of XviD avi : 66.40%
Absolute Quality of XviD avi : 96.96%



BF are definitely not dev anymore, I think this could be a root option.

Thanx a lot to all developpers.

Regards
NoLogo

[mikeson]

Hi!

First of all I want to thank all XviD developers for their great work!

But to the point...

I've tested latest development build 25112002 from Koepi and noticed some things.

1. I-frames are not inserted as they should be while using B-frames (3/150/100). First frame is I-frame (of course :)) but next I-frame is at frame 986 (I've specified min/max I-frame interval as 1/300). This only appears when using B-frames, otherwise it is ok.

2. There is a strange artifact on the top of the movie, but it only appears while using Lumi masking, otherwise it is gone.

Maybe I'm doing something wrong... :confused:

Using Avisynth 2.5, MPEG2Dec3 0.93b, XviD 25112002 (1 Pass - quantizer 5) and VirtualDubMod 1.4.11.2.

Here is the link: http://mikeson.wz.cz/xvid_testing.rar

[NoLogo]

@mikeson

1) I've been told (can anybody confirm that point ?) that user's specifications about min and max I-frames interval are not used anymore by the codec for quite a long time ago, this could maybe explain that...

2) Sorry, don't know where it comes from... cause I've never found Lumi mask very usefull or reliable, then I do not use to check the option...

Regards
NoLogo

[mikeson]

@NoLogo

1) I've been told (can anybody confirm that point ?) that user's specifications about min and max I-frames interval are not used anymore by the codec for quite a long time ago, this could maybe explain that...

But if I turn B-frames off, I-frames are inserted as they should be (ie on max I-frame interval or scene change) :confused:

[Koepi]

Only pframes are counted for the iframe interval, that's causing this strange behaviour.

Regards
Koepi

[mikeson]

@Koepi

Only pframes are counted for the iframe interval, that's causing this strange behaviour.

Is this an intention or a bug?

[Koepi]

?

Guess!

[mikeson]

@Koepi

I'm not sure, really. Anyway thank you for answering.

[Koepi]

It's not intentional, but noone coded that yet with the new motion estimation routines.

SysKin is fixing it ATM and we're thinking about adding support for min iframe interval as well again... but i prefer consecutive keyframe treatment over hacking such bad things into xvid.

regards
Koepi

[mikeson]

@Koepi

Thank you again for explanation.

I'm just asking because I've encoded a movie and seek times are very long (much longer than usual).

[miha]

Koepi can you compile a new build please

[Koepi]

I don't have the fix for the iframe interval counter here, so it wouldn't be of any use.

I'll put up a new binary soom though (not today).


regards
Koepi

[ffroms]

Here is how you can calculate and enter correct Iframe interval.
Let say you are using 4B-frame and 300 max Iframe interval.
With this you'll get Iframe every 1500 frames.
4x300+300=1500 (or 5x300)
or for 3Bframes
3x300+300=1200 (or 4x300)

So for 4Bframes set Max I-frame interval to 60, for 3B set to 75, for 2B set to 150 and 1B let as it is.

This is only until Koepi (or someone else) fix the problem. Keep up with good work Koepi and thanks.

[sysKin]

Originally posted by ffroms
Here is how you can calculate and enter correct Iframe interval.
Let say you are using 4B-frame and 300 max Iframe interval.
With this you'll get Iframe every 1500 frames.
4x300+300=1500 (or 5x300)
or for 3Bframes
3x300+300=1200 (or 4x300)

So for 4Bframes set Max I-frame interval to 60, for 3B set to 75, for 2B set to 150 and 1B let as it is.

This is only until Koepi (or someone else) fix the problem. Keep up with good work Koepi and thanks. No it won't work, because max number of b-frames is only a max, it usually is not reached. We decide on the number dynamically.

However, it really doesn't matter - it's been fixed for a day now :D

Radek

[iago]

uManiac's site:

XviD.Alpha.29.11.2002.1100.exe

---------------------------------------------------------------------
29.11.2002 11:00:
U vfw/src/config.c (rev.1.20.) suxen_drol:
- removed EnableWindow(FALSE) for bframes widgets

28.11.2002 20:20:
U xvidcore/src/encoder.c (rev.1.76.) syskin:
- proper max keyframe interval with b-frames
U xvidcore/src/image/image.c (rev.1.20.) syskin:
- qpel interpolation code fixed (but please check it's really a fix, noone answered to my mail)
U xvidcore/src/motion/motion_est.c (rev.1.44.) syskin:
- another interpolate bug (I promise to stop producing them. really. lol); some thresholds fixed for better mode decision (in bframes)
U vfw/vfw.dsp (rev.1.8.2) suxen_drol:
- removed #ifdef BFRAMES
U vfw/src/2pass.c (rev.1.7.2) suxen_drol:
- foxers 2pass + 'packed bitstream' patch; part 2
U vfw/src/codec.c (rev.1.23.) suxen_drol:
- removed #ifdef BFRAMES
U vfw/src/config.c (rev.1.20.) suxen_drol:
- removed #ifdef BFRAMES
U vfw/src/config.h (rev.1.14.) suxen_drol:
- removed #ifdef BFRAMES

21.11.2002 11:00:
U xvidcore/src/motion/motion_est.c (rev.1.44.) syskin:
- an ugly bug squashed (bframes+qpel)
---------------------------------------------------------------------

He he, worth a try until Koepi releases a new(est) binary! ;)

[Koepi]

hehe, most of those fixes are already in my 2511-binary (!) - since we closed up in development i have some of those fixes before they appear in CVS.

I want to release the next binary when i can apply suxendrol's 2ndpass corrections for IPB-decision, and for that occasion i have statsreader2.0 waiting in the queue. it's capable of inserting keyframes (!), but since now oin 2nd pass dynamic IPB decision is still done, those keyframes are ignored :-/

Well, just wanted to give some feedback here so you people know what you're dealing with :)

Best regards
Koepi

[iago]

@Koepi,

Thanks for the explanation man! Looking forward to the new build actually! ;)

regards,
iago