XviD-22032003-1 :)

[Koepi]

Heya folks,

after testing this build for 3 days I think it's time to make it public. It contains a bframes fix from sysKin (unbelievable - it's just the old "TOO_SMALL_LIMIT" which caused troubles half a year ago. But for bframes it does 2 things: it shrinks filesize and amazingly improves image stability and quality!).

XviD-22032003-1:
- Fresh CVS checkout.
- Added syskin's experimental bframe optimizations.
- BFrames threshold still doesn't accept negative values.

All other restrictions still apply:
- VHQ doesn't work with GMC (and vice versa ;) )
- VHQ is slow, as is lumi masking (refdivx code, increasing filesize a bit for 1pass encodings, but works well with 2pass encoding quality-wise)
- this build uses simple idct (necessary for proper qpel encoding)
- ICL7 optimized.
- other things i forgot already.

I hope you enjoy this build as much as I do!

Best regards
Koepi

[Selur]

cool :)
Thx for another nice build :)

[sungey]

ooo must try .. :P

[cipher]

cool!
Thanks, koepi!

[((( atom )))]

@koepi: might i come up with my old wish for a checkbox "keep 1st pass if filesize is under VALUE"?

the other day i encoded a movies 1st pass up to 85% and then hit the 2GB-border and f***ed up! aargh!


EDIT: By Nic, Dont Swear! :)

[JagPanzer]

Koepi thanks for the new build!
I will do some tests now.

[Manao]

First, thank you a lot for this new build, it seems very promising.

Just done a little test, since I was finishing an encoding with previous build ( 080303 )

I took a Marilyn Manson concert bonus ( PAL, interlaced, and pretty hard to encode ), it went through the following avs :

MPEG2Source("C:\Ripp\courtm.d2v",cpu=0)
SeparateFields()
SelectEven()
Crop(6,4,-14,-4)
FluxSmooth(6,6)
BilinearResize(384,288)
Convolution3D(0,6,6,6,6,2.8,0)
TemporalSoften(3,3,3,10,2)
DCTFilter(1,1,1,1,1,1,0.5,0)

The settings where the same for Xvid :

Motion search 6 - H263 - VHQ 1 - CM on - Bframes 3,150,150,0, linear curve scaling. No Qpel, no GMC, no lumimasking

With 080303 build :

First pass size : 299220 Ko
Second pass size : 153220 Ko ( 153185 wanted )
Number of : I-frames : 3400 ; P-frames : 25684 ; B-frames : 15049

With 220303 build :

First pass size : 260240 Ko
Second pass size : 153230 Ko ( 153185 wanted )
Number of : I-frames : 3384 ; P-frames : 24508 ; B-frames : 16241

Visual difference : it's very hard to tell. I would tend to say the 220303 gives a less blocky result, perhaps a little more ringing ( but the frames I was comparing then may have been a P against a B frame ), and may be a little more details, but it's more than subjective, and the material ( the video is really ugly, with a lot of flashes ( look at the number of keyframes... )) is surely not a good candidate for such testing. I will try on something more classic, but later.

BTW, why when asking XviD a round second pass size ( ie : 150000 ) it always gives 150XXX with XXX roughly about 250 Ko, whereas when asking 150200 it gives almost exact size ? Is there a rationnal explanation ? It isn't disturbing, but it surely intrigues me ( I ask myself really dumb questions sometimes :rolleyes: )

[kilg0r3]

From the change.log on Umaniac's site
mbfunctions.h, mbtransquant.c (rev.1.18) <i>syskin</i>:- cleanups; dark blocks prevented in bframes; warning: <b>bframes have lower quality due to thresholding: decrease quantizer (150/0 for example)

Does this also pertain to the current build? BTW, what thresholding mean in this case?

[ssjkakaroto]

great, i didn't get any artifacts with bframes and cm on a 1 min test clip :D i'll test a bigger file later, thx koepi :cool:

[Valky]

Thank you. I was just planning to test some new build. That installation path seems little bit stranger... :)

[gldblade]

I thought a "Fresh CVS checkout" build is supposed to have today's date :D (Yes, I understand you've been testing it for three days.)

[Alxemi]

I´ve been umaniacs 21-2, and i think i will give this build a try (when i read some other posts saying "everything is fine with this build" xD )

Than you very much for your work and development

[Koepi]

@kilgore:

That's the bframe fix in CVS, it's the same that I have (theoretically at least %) ). Thresholding? In my initial post I mentioned that TOO_SMALL_LIMIT - which is nothing less than thresholding of coding (or not coding) macroblocks... not coding them in bframes doesn't give a visible quality impact (well, honestly it does: it looks better).
I think it's safe to use the "old" bframe values, you won't notice the "lower quality" bframes... it's just a notice for being on the safe side as this thresholding was producing blurry output with I/P-frames.

Regards
Koepi

[sysKin]

Originally posted by Koepi
Thresholding? In my initial post I mentioned that TOO_SMALL_LIMIT - which is nothing less than thresholding of coding (or not coding) macroblocks... not coding them in bframes doesn't give a visible quality impact (well, honestly it does: it looks better).
I think it's safe to use the "old" bframe values, you won't notice the "lower quality" bframes... it's just a notice for being on the safe side as this thresholding was producing blurry output with I/P-frames.
Yes exactly :). Because of this TOOSMALL_LIMIT, bframes are smaller and it comes at the cost of their quality (at least in theory). I can't tell if it's better to keep 150/100 and use these extra bytes for all frames, or maybe it's better to give them back to bframes - by using 150/0 or something like that :)

Ah, you might also try 100/0 - the same quantizer for bframes as for pframes. It doesn't appear to be working with anime (bigger filesize than no bframes at all) but seems to work for normal movies - 1st pass is smaller. I can't tell much about quality, it looks all the same to me (at quant 2 lol) but maybe you want to try.

happy testing,
Radek

[majerle]

@sysKin
sse2 bug on pentium 4 is resolved ?

Thanks

Andres

[jarthel]

Originally posted by Koepi

In my initial post I mentioned that TOO_SMALL_LIMIT - which is nothing less than thresholding of coding (or not coding) macroblocks...

Not everyone knows that "TOO_SMALL_INT" has something to do with threshold. :)

jayel

[kilg0r3]

1. Originally posted by sysKin
I can't tell if it's better to keep 150/100 and use these extra bytes for all frames, or maybe it's better to give them back to bframes - by using 150/0 or something like that :)

I, usually, use very moderate values e.g. 124 for BF-quant ratio and 100 for the offset. I found this to reduce the bframe flicker a bit, and, it suffices to stuff a movie into ( :)) one xcd.

2. Anything done to the chroma optimizer?

3. A bit more than just occasionally, I get stuttering playback after seeking with the included decoder. It is not too bad though. Since it ca be resolved by pushing the seekbar around a bit. The decoded file has bframes of course.

[Mel Maconoo]

Originally posted by majerle
@sysKin
sse2 bug on pentium 4 is resolved ?

Thanks

Andres

when i tried it.. Vdub shut right down on me..

[sungey]

Umaniac's page ..

20.3.2003 09:40:
U xvidcore/src/encoder.c (rev.1.96) syskin:
- some cleanups ; revised p/b decision with sensitivity control ; max iframe interval works again

good to see the max iframe interval is working again :) ....

[Bulletproof]

Originally posted by sysKin
Yes exactly :). Because of this TOOSMALL_LIMIT, bframes are smaller and it comes at the cost of their quality (at least in theory). I can't tell if it's better to keep 150/100 and use these extra bytes for all frames, or maybe it's better to give them back to bframes - by using 150/0 or something like that :)

Ah, you might also try 100/0 - the same quantizer for bframes as for pframes. It doesn't appear to be working with anime (bigger filesize than no bframes at all) but seems to work for normal movies - 1st pass is smaller. I can't tell much about quality, it looks all the same to me (at quant 2 lol) but maybe you want to try.

happy testing,
Radek

I'm a bit confused on the B-frames settings, From my understanding the offset references the last P-frame quantizer and uses the setting you specify to determine the B-frame quantizer, but I am not quite sure what the first setting (150) is for then? It sort of seems redundant to me, but I suppose it's because I don't understand the corelation between the two.

[Assault]

@Bulletproof

b-frame quantizer is calculated by average qunatizer of the last I/P-frame and the quantizer of the next I/P-frame * quantizer ratio(first setting ;))/100 + quantizer offset/100.

Regards
Assault

[kilg0r3]

Originally posted by Koepi
[B]
- this build uses simple idct (necessary for proper qpel encoding)


:stupid: This said, ...
Just want to make sure i get this right. So, please correct me if the statements below are wrong.

This line refers to the xvid.ax included right. However, checking 'use xvid in ffdshow uses the xvid.dll. Consequentially there is currently no way to get proper qpel decoding using ffdshow for playback; except, of course, applying it to the overlay by checking 'raw video'.

[mf]

Originally posted by majerle
@sysKin
sse2 bug on pentium 4 is resolved ?

Thanks

Andres
Donate a pentium 4 to one (or more) of the main devs. They all have AMDs, so they wouldn't be able to test their own code. And btw: wasn't there a workaround to disable SSE2 entirely ? If so, please don't complain.

[jarthel]

Originally posted by mf
Donate a pentium 4 to one (or more) of the main devs. They all have AMDs, so they wouldn't be able to test their own code. And btw: wasn't there a workaround to disable SSE2 entirely ? If so, please don't complain.

Hmm I don't think he was complaining. He was just asking if the P4 bug has been resolved.

[mf]

Originally posted by jarthel
Hmm I don't think he was complaining. He was just asking if the P4 bug has been resolved.
All the same. Is it in the changelog ? No.

[Nic]

Ill put VC6 on my P4 in the next week or so and run xvid till I get a crash and see if I can fix it, even if I cant at least ill be able to say exactly where and why its crashing which should hurry along any fix.

-Nic

[Sigmatador]

is it possible to have the TSL as a parameter in the vfw interface ? a higher tsl value is sometime nice for anime ^c^

[sysKin]

Originally posted by Nic
Ill put VC6 on my P4 in the next week or so and run xvid till I get a crash and see if I can fix it, even if I cant at least ill be able to say exactly where and why its crashing which should hurry along any fix.
sse2-ed fdct probably requires aligned matrix for input (created with DECLARE_ALIGNED_MATRIX). VHQ stuff doesn't create this matrix this way... But I'm not sure, I don't mind if you check :)

Radek

[kastro68]

qpel
chroma
vhq4
3 bf, 150, 100, 0 [if applicable]
everything else on default
same avisynth script for both

1st pass size for 17-2-2003 koepi's build:
The Eye, 811 megs

1st pass size for 22-03-2003 koepi's build:
The Eye, 959 megs

Can't really tell any noticeable difference at this time, closer examination still pending.

[jarthel]

Originally posted by mf
All the same. Is it in the changelog ? No.

maybe his fault is not reading the changelog. But that doesn't mean the person is complaining.

[Nic]

@mf & jarthel: Dont argue in threads please. The comments aren't constructive. Please continue it privately from now on.

-Nic

[TheXung]

Originally posted by kastro68
qpel
chroma
vhq4
3 bf, 150, 100, 0 [if applicable]
everything else on default
same avisynth script for both

1st pass size for 17-2-2003 koepi's build:
The Eye, 811 megs

1st pass size for 22-03-2003 koepi's build:
The Eye, 959 megs

Can't really tell any noticeable difference at this time, closer examination still pending.

Yea I noticed a larger 1 pass too. I was testing the Matrix. The 2nd pass was arguable better or arguably worse quality, but leaning towards worse.

[pan]

Just a quick question about the sse2 bug on the P4's.
When I switch sse2 off everything runs fine, its just that when I reopen the Xvid I have to switch it off again.

Is there any way to switch this off permanently?

By the way well done to everyone involved with Xvid, great stuff!

Pan

[Koepi]

Between 17022003 and 22032003 sysKin changed the I/P/B-decision code, so bframes are set differently. That's the reason for the bigger first pass size.

Compare this build against 08032003 (already new I/P/B decision) and see the difference, that'd be reasonable. Your comparison simply is invalid.

On the simple IDCT issue: ENcoding is _not_ DEcoding. So it doesn't affect xvid.ax directly.

Koepi

[jarthel]

Originally posted by Nic
Ill put VC6 on my P4 in the next week or so and run xvid till I get a crash and see if I can fix it, even if I cant at least ill be able to say exactly where and why its crashing which should hurry along any fix.

-Nic

Thanks Nic. I'm eagerly awaiting for your build. :)

Jayel

[kilg0r3]

Originally posted by Koepi
On the simple IDCT issue: ENcoding is _not_ DEcoding.
my bad
So it doesn't affect xvid.ax directly.
Does the use of simple idct mean then that there should no longer bee any related qpel artifacts when decoding them or do we still have to wait for a working decoder?
Sorry to get onto every body's nerves but, as you might have noticed, I am little bit confused about where the problem.

btw, does anyone have a torture clip for a motion smearing and face-speckling test?

[deXtoRious]

Some people wrote that VHQ lessens the "moving background" effet, which REALLY annoys me. So I tried... but no results :( Am I doing something wrong???

[PowerMacG4]

Argh! B-Frames will not disable no matter what I do! Sometimes I 'get lucky' and the encode goes fine, but shouldn't the default '-1' value for maximum b-frames disable them? Thats what it says it does. And it does not.

Is there a way to just completely make sure that b-frames do not get encoded? Like a check-box or something? This is getting aggrivating.

[kilg0r3]

@ deXtoRious
in my tests vhq >1 increases this effect

@ PowerMacG4
if this is os, it is a bug in the vfw interface(?).

[Blueseb]

Originally posted by kilg0r3
@ deXtoRious
in my tests vhq >1 increases this effect

in mine too

[drebel]

@PowerMacG4
You could try B-frames threshold -110 (i dont know if it will save negative values though) which prevernts placing any b-frame.

No probs with moving backgroung here with VHQ1 (even WITH QPEL).The movie came out perfect.But the avg bitrate was 1931 (85% comp) which is high.I should make some tests with low bitrate conditions.

[Gazza]

I have noticed that my encodes have appeared really quite dark (and maybe noticeably blurry) whilst using the current and last versions of Koepi's versions of xvid. I've noticed that using qpel, me, bframes (3/150/100), vhq=1 can produce very dark encodes whereas using no qpel, no me, no bframes (-1), vhq=0 the result is a lot lighter and sharper (nearer to the original).

Also, using qpel, me, bframes (3/150/100), vhq=1 means that the replay is very jerky (due to the fact that the cpu works a lot harder during replay and hits 100%). This processor problem (I have a 1ghz p3) seems to be a recent problem with the latest xvid.

[Nic]

Just to say I updated my build, http://nic.dnsalias.com
Updated the DShow along with it too,

Cheers,
-Nic

[Blueseb]

Originally posted by Gazza
Also, using qpel, me, bframes (3/150/100), vhq=1 means that the replay is very jerky (due to the fact that the cpu works a lot harder during replay and hits 100%). This processor problem (I have a 1ghz p3) seems to be a recent problem with the latest xvid.

i had that prob with last koepi's build, resolved disabling qpel, b-frames are ok for me (duron@900mhz,~50%load with a 720x304 bf@4/120/70 ffdshow playback)

[deXtoRious]

No probs with moving backgroung here with VHQ1 (even WITH QPEL).The movie came out perfect.But the avg bitrate was 1931 (85% comp) which is high.I should make some tests with low bitrate conditions.

I was using 1500kbps for one test and 800kbps for the other. In the high bitrate rip the only problem my eyes could see was the moving background... The background was quite jerky too. Gonna try with Nic's new build...

[kilg0r3]

Originally posted by Gazza
I've noticed that using qpel, me, bframes (3/150/100), vhq=1 can produce very dark encodes whereas using no qpel, no me, no bframes (-1), vhq=0 the result is a lot lighter and sharper (nearer to the original).

could post/upload some material to verify this, please? Ideally scrrenshots and two avi snippets of encoded material?

[jarthel]

Originally posted by Nic
Just to say I updated my build, http://nic.dnsalias.com
Updated the DShow along with it too,

Cheers,
-Nic

You mentioned several messages ago that you're looking into the SSE2 bug. Has this been fixed in this build?

I looked at your website and it doesn't say it as such. But then again, the website doesn't say much about changes either.

Thanks :)

Jayel

[BoNz1]

Nic, your latest build + dshow are really great, it resolves many issues for me including the SSE2 bug and the bad mosquito noise which my encodes were suffering from because of quarter pixel. I suppose it looked bad because I was using ffdshow and 1/4 pixel which now uses simple IDCT. Anyhow my encode looks great now with your dshow. I ran a short 1000 frame clip that I was having particular difficulty with before and it is far better now. So I am now encoding a couple vobs I had sitting on my hd. Once again, thanks. :)

[jarthel]

Originally posted by BoNz1
Nic, your latest build + dshow are really great, it resolves many issues for me including the SSE2 bug

GREAT NEWS!

I asked if the bug was fixed as I can't test it right now. I'm at work.

Thanks Nic.

-----------------------

Bonzi, does enabling SSE2 speed up the encoding?

Thanks

Jayel

[BoNz1]

No, I can't notice any noticable speedup :D, there probably is some but it would be pretty small I would guess.

[Didée]

Originally posted by Gazza
I've noticed that using qpel, me, bframes (3/150/100), vhq=1 can produce very dark encodes whereas using no qpel, no me, no bframes (-1), vhq=0 the result is a lot lighter and sharper (nearer to the original).
Now let me guess, Gazza:

You opened two clips at the sime time in two instances of your player, and compared them side-by-side? :D

If so: thanks, no further questions ...

[unplugged]

Originally posted by Didée
You opened two clips at the sime time in two instances of your player, and compared them side-by-side? :D
Don't do this, at least with player, because one session will start with overlay video acceleration and the other don't.
So it's likely to happen color differences between the outputs (especially with ATi cards).

Just to remind this...

[Gazza]

Unplugged & didee,

Point taken about multiple instances of a player and thanks for the tips.

If I view a test encode with a single session of the player, I still reckon the output appears darker. I would be interested to see if more people see the same. Maybe using qpel, etc puts extra stress on the processing???



Cheers

[sungey]

Originally posted by unplugged
Don't do this, at least with player, because one session will start with overlay video acceleration and the other don't.
So it's likely to happen color differences between the outputs (especially with ATi cards).

Just to remind this...

That's true especially when you are using mplayer2.exe ... you may want to try a player called Sasami2k ... this player doenst seem to cause 2 clips to have different brightness when they r opened at the same time .. :P

[kilg0r3]

@ gazza
just open the clips in vdub or open three instances of the player, one occupying the overlay and the other two for comparing your clip ..

[Gazza]

@kilg0r3

I'll give vdubmod a go and let you know if the dark problem is a real issue for me or not. Have to do this tomorrow when I get a chance.

Cheers

[ssjkakaroto]

nic do you recommend using 150/0 on bframes with your build too?

[Gazza]

Originally posted by kilg0r3
@ gazza
just open the clips in vdub or open three instances of the player, one occupying the overlay and the other two for comparing your clip ..

I used three instances of vdubmod to look at the original avs, an avi with qpel, b-frames, etc and an avi without qpel, b-frames, etc. I compared the same frames and the I think that the original source looked, to my eyes, lighter and sharper than the other two. The two encodes looked fairly similar though??? Therefore, is the latest xvids suppressing some the chroma's in an encode regardless of the choices you use in encoding. I would be very interested if others could do the same tests and report their results.

Edit - The tests above used an anime source. I repeated the same test above with a 'normal' video source (with live actors, etc) and the difference between original source and encodes was hard to pick. Therefore maybe the dark problem is related to or amplified by animes?

It depends if using vdubmod to examine before/after in this way is a reliable test? (I'm not having a go but maybe there is a more objective test to measure the video quality in this way?)

Cheers

[Koepi]

Stupid question:

do you use the line

LumaFilter()

in your avs-script?

Regards
Koepi

[Gazza]

Koepi

I've used that in the past but not this time. I have noticed that when using vdubmod to view original source and encodes, the encodes look close to the original (maybe a little blurry if you magnify the view). Using bsplayer the encodes look pretty close to the original as well. However, using media player classic, core media player and any versions of Windows media player the video is pretty dark. Any clues? Is there a generic setting that needs to be set for each player? I tend to stick to default settings.

By the way I am using nics dshow....

[Nic]

Hmmm, Ill definitely add something to my DShow's properties so you can be sure what colorspace its rendering out at (as this might help spot differences like this). @Gazza: Does the force colorspace options work on my filter for you? And if so do they have any effect when playing in MPC or similar?

Cheers,
-Nic

[Bulletproof]

If you are using an nVidia card, all of the brightness and contrast settings are forced on the video by its drivers in its control panel. When you open a video in virtualdub it doesn't use the overlay.

[HarryM]

@Nic:

I cant use 'force YV12' in your decoder. I seeing mishmash graphics. Unforced settings works O.K.

Personally I dont prefer your decoder. I think, that native decoding via 'xvid.dll' + 'simple xvid.ax' give the best quality.
Your older decoders smearing at q-pel. Your newest (30032003) decoder is O.K. in this, but I still prefer native decoding.

[JasonFly]

I also had problem with force to YV12 with the "old" build of Nic's decoder but i haven't tested the latest one.No force is ok.
I'm using Win2k

[Nic]

Using my decoder without any options selected is exactly the same as using XviD.dll, they both use the same libraries and "native" decoding.

Forcing doesn't seem to work for alot of people, all options worked fine for me, ill have to go look at it, I must be doing something odd :)

-Nic

[Gazza]

Nic, forcing colorspace (yv12) and viewing with core mediaplayer produces a black window and the video doesn't play. You can seek OK in the avi but video still doesn't play and you still get the same black view. Trying to view with bsplayer whilst forcing yv12 produces a 'Unknown file format' error message.

Hope this helps

[sungey]

probably some video cards cant output YV12 overlay ?

im using nvidia .. and after upgrading the driver it works fine with force YV12 CSP selected in Nic's latest decoder..

[Gazza]

In summary,

In Nic's dshow, when selecting no force, most players (except bsplayer) give dark video.
Forcing yv12, video will not play at all (but will seek)
Forcing yuy2, rgb16, rgb32 video plays well with none of the dark video issues.

bsplayer must force videos to yuy2 by default (I checked that 'force rgb' was not selected) therefore it appears to play video's better than others.

I guess that I started to see dark videos when Koepi included Nic's dshow in recent versions of xvid. Also, at about the same time, I must have installed and started using avisynth 2.5 and latest vdubmod, etc - I assume that the resultant avi's are yv12 (anyone know how I can check this?).

Therefore, it looks as if the latest xvid encoding is OK but the dshow filters (including ffdshow) darken the video when no force is selected (I assume that my latest tests produce yv12 and this causes the video to go dark when going through nic's dshow or ffdshow) and don't handle yv12 at all when forced. All other modes seem to produce video close to the original.

[kilg0r3]

hm, could be a color conversion issue yv12 (xvid) -> yuy2 etc. But this is going OT.

[tiki4]

Hi, I got a stupid question:

The newer builds again regard the MAX I-frame value. What I didn't find out yet, do they count the B-frames or are the B-frames still ignored?

Thank you.

tiki4

[Gazza]

Originally posted by kilg0r3
But this is going OT.

This is true. I think I have finished my little bit of testing.....

Well, at least for now.

Cheers

[Sharro]

I dare...

Would it be possible to also set quantizer type in credits?

I know it is be possible by trimm but if we could define it in the codec even better.

Why? I'm running an encode with a compressible movie for 1 cd with mpeg quant, and I've set quant 20 for credits...but still I'm using mpeg quant for credits where I would prefer to use h263.

@EDIT: If no problem for b-frames arouses!

Just an idea.

Thanks for all work.


Sharro

[Koepi]

Modulating the quantizer type isn't quite mpeg4-spec compliant. That option is still in xvid just because nobody removed it, so please forget about changing quantization types within a stream.

Best regards
Koepi