Cvs 2002-02-25

[-h]

Just a heads up - I'm going to commit a bunch of stuff, as soon as I've made sure my crappy debug code has been stripped out:

- Foxer's 2-pass curve compression (internal mode only), on by default, looks great
- Foxer's "average frame size" overflow patch (both 2-pass modes), allows for better allocation of overflow
- minimum distance between I-frames
- more debug output for errors
- changed __TIMESTAMP__ to better variable (only took what, 2 months Nic? ;))
- other code cleanups

A whole lot changed, so I expect (nay demand) bugs (check 2nd pass external mode!!). I'll be around for maybe 2 more hours, so if it's quick, I'll be able to fix.

Edit: Just committed, fingers crossed.

-h

[Nic]

Guess what? :)

Get it at

www.freewebz.com/xvid
or
xvid.stormpages.com

also, I added that line of code to disable the Luminance Masking when 2pass - 1stpass is set (I cant wait another 2 months ;)

Cheers,
-Nic

[Nic]

-h:
Could you post exactly how
1) Bitrate payback delay (frames)
2) Curve Compression

Effect the encode & what they mean.

Thanks alot,
Cheers,
-Nic

[-h]

I experimented with that additional line, but horrible things happened when other controls were included. Things wouldn't enable/disable until the dialog was closed and reopened, simply because of the mismatch between PSN_KILLACTIVE and the upload/download. I'll play some more, but it's plain bizarre.

The new options are similar to the Nandub options of the same name, though they may be completely different, seeing as I never really used Nandub and have little idea what I'm talking about.

Curve Compression flattens the distribution of bits throughout the movie. I.e., bits are taken from large frames and given to small ones - the higher the % value, the more extreme this effect becomes.

The Bitrate Payback Delay governs how many frames the codec should spend in compensating for an over-use / under-use of bits elsewhere, usually during a high action scene. In other words, if a high-action scene eats up more bits than would be expected, those bits which have "gone missing" are consumed over this number of frames. Similar to the overflow. Well very similar actually.

That's what I guess it should be anyway, though the source seems to be doing something else completely :) Hopefully this will get cleared up. I certainly can't complain about the results however, so perhaps it's not important how it works.

-h

[rui]

Well, i just did a small test using this new build.

The results are AMAZING!

I used the trailer from the movie "The Replacements", using for 1 pass H.263, motion serach precision 5, and for 2 pass H.263, no lumi, I-frame lock 2-5, with the smooth quantizer enabled, and using Foxer's new settings with the default values.
I wanted a file of 12199 and got a file of 12122. I compared it with another encoding made with the 21/2 build, and the later build is MUCH better. In fast scenes i was noticing some blocks with the 21/2 build, and now they are gone!
And, i can't be sure of this but the new build seemed faster. But i'm at work, where i have a P2-350 with 64MB Ram :(, but each pass only lasted ~13 minutes, and with the 21/2 build it used to last ~13-14 minutes.
But when in home i will make more tests with my XP.
By the way, Nic, the lumi masking not available in first pass seems to be working, at least i can't enable it when in first pass mode.

Resuming, the new build is GREAT!
Now, if it only had the modulated quantization mode... Koepi? ;)

[Nic]

Well done Rui! :) I was waiting for you (or at least someone!) to do a test :D

Cheers,
-Nic

[Ripe73]

Now, if it only had the modulated quantization mode... Koepi?


What do you mean with that?
Will it use both H:263 and MPG,i think i missed something:)

Thanks

[rui]

Yap!

But you better go to Koepi's site and download his pdf explaining that.

And about tests: the more i think about it, the more i believe that trailers are a very dificult test, because in a normal movie there are times (most of them i believe) where the picture is still.
But in a trailer, the objective is to show parts from the all movie, so the scenes are changing very fast, during all the time.
Even a trailer from a slow paced movie (you know what i mean) is always changing scenes.

[Nic]

However, I always think very low motion is a good test as well....DivX4 cant seem to reproduce a very very still picture nearly as perfectly as it should be able to (I find it very noticable, to have still frames but have macroblocks changing colour all the time)

Cheers,
-Nic

[Teegedeck]

Hm, I'll give the new algo a go, but I'll set it to 1 (you cannot set it to '0', it seems). I gotta say, I'm more than a bit sceptical about this 'nandubization'. Way back, before GKnot and DivX4, I found out that I reached the best quality in Nandub when I set the curve-compression to '0', thus disabling it - and lateron that Gordian Knot's way of bitrate-curve-scaling was superior to Nandub's (I think because flattening the curve, like Nandub does, _unneccessarily_ pushes the bitrate of small frames and takes away bits from large frames that would need them). I wish my CPU wasn't so slow :( , testing would be more fun if I finally got my upgrade assembled.

[Koepi]

I love nando's additions and will apply quant_type modulation.
But don't think that will be error free ;)

Curve smoothing/compression _always_ worked best for me :)

Regards,
Koepi

[rui]

Originally posted by Koepi
I love nando's additions and will apply quant_type modulation.
But don't think that will be error free ;)

Curve smoothing/compression _always_ worked best for me :)

Regards,
Koepi

Great! I was going to encode a full movie in home, but i think that i will wait for your build.
About the possible bugs: are you expecting trouble, or you are just giving us the "experimental stuff, can have bugs" warning? :D

[Ripe73]

I encoded a second pass with this new build and i was aiming for 615725KB and the final size was 613230KB.
The movie was Blade Runner 167817 frames
Settings
Payback delay:240
CurveC.:20%
M.Search:5
H:263
Main Q Min:2 Max:8
I.frame Min:2 Max:5
Smooth.Q:On
No L.Masking
No endcredits settings(encoded manual)

[-h]

I fixed a couple stupid mistakes, plus removed the undersizing hack I created earlier (is now redundant).

The results of the updated CVS shouldn't be all that different, but are more "correct" now.

-h

[rui]

About the tests using movie trailers.

I have been noticing that when doing tests using trailers, I always get undersized files.
Then I remembered that I have been using in my tests the I-frame lock. If I remember correctly, Koepi once said in the xvid.org forums that by default the codec locks the intra frames at 2-3 quantizers. But I use the intra frames lock 2-5, with the smooth quantizer enabled.
When I use trailers, the scenes are changing very fast (I explained the reason in a prior post in this thread), so it is natural that I have a large number of intra frames (key frames), right?
So, because I am using 2-5 I-frames lock, and the default would be 2-3, and since I have a large number of intra frames, I get undersized files.
Well, the codec should compensate this by attributing more bits to the inter frames, but maybe because we are talking about rather small files (The Replacements trailer vob has about 100MB), maybe it doesn’t compensate enough to give me the desired file size).

I only decided to post this as a warning for anybody also doing tests using trailers, so if they are getting undersized files I believe that this could explain it.

To Koepi: maybe you could use this latest code fixed by -h for applying quant_type modulation, instead of the earlier code

[MaTTeR]

The latest build looked absolutely beautiful on a Pitch Black clip that I encoded. I was shooting for a bitrate of 850 and I noticed no artifacts.

Maybe I should start testing some trailers as well. What bitrate do you guys usually shoot for? Around 800?

Great job guys!

[Koepi]

I did a fresh checkout of the sources (~22:40h CET) and tomorrow I'll try to apply quant_type modulation again (already edited the release notes ;) )

I hope I get a new working build up at ~12h CET tomorrow.

Regards,
Koepi

[Kurg]

Only my thanks,koepi and h for your latest modifications, i hope that doom9 try and compare the old xvid images from his codec comparison with the latest build can be interesting =)

[Franko30]

Hi again,

I don't want to spoil the overall happiness over the latest additions, but tonight I encoded 2 movies (Tom&Viv 576x316 and Demolition Man 576x260) from my analogue source (see signature).

I used the standard settings of the new interface (Quant 20 Credits, 240 bitrate payback delay and 25% curve compression, min keyframe setting 10 max 300) and used my usual H.263 quantization type, "normal" quant settings 1-16 and I-frame Quant settings 1-6, quantizer fluctuation smoothing enabled.

First: both movies had an overall lousy, faded look, like using very grainy film material, that is to say: too blocky.

For Tom&Viv I ran Debugview in the background and it showed me the reason:

Usually all my recordings with the DVC II card produce MPEG2 files that, when encoded with Koepi's Feb17th binaries, produce almost only Quant 3-4 and higher (up to 8-9) in action scenes.

But Nic's new Feb25th binaries produce only Quant 5-6 frames with some higher Quants for action...:(

This is very strange, as it is the worst quality I ever got out of XVID.

I'll redo both movies with Koepi's Feb17th binaries - expect results no sooner than 11 p.m. tonight (it's 9 a.m. now).

Frank

[Foxer]

How would you all feel about the addition of asymetric curve compression and maybe smart keyframe spacing in addition to my changes which were added?

BTW, I'd advise against making new compilations with the current CVS. Today's CODEC.Cs (v1.22 thru v1.24) contain a few errors and I am waiting for -h to return so he can commit the corrected codec.c I e-mailed him.

This particular form of smart keyframe spacing I've thought of is simple and is where it detects that there should've been a keyframe in the space where it was preventing new ones and will add one when min keyframe interval - 1 p-frames (from the previous keyframe) have been placed.

It won't look that different in very high action scenes but soon (average of min keyframe spacing / 2 frames) after it switches to a low action scene it'll place a keyframe whereas normal keyframe spacing might leave a little mosquito noise from the action scene but might also produce a noticable flicker effect as it gets a little blocky from the keyframe and then reses up.

[Nic]

( I was up until midnight implementing deblocking algorithms....the results weren't brilliant (the deblocking filter worked great on really low quality video, but on higher quality smoothed far too much in the wrong places)

Ill try cleaning up & re-implementing the 9-tap algo from decore tonight & start from there.

-Nic

[-h]

Bug fixes have been committed to CVS.

Smart keyframe spacing sounds fun. If it's a core mod, however, it won't be long until it's blasted by Isibaar's work.

-h

[rui]

Originally posted by Franko30

Usually all my recordings with the DVC II card produce MPEG2 files that, when encoded with Koepi's Feb17th binaries, produce almost only Quant 3-4 and higher (up to 8-9) in action scenes.

But Nic's new Feb25th binaries produce only Quant 5-6 frames with some higher Quants for action...:(

This is very strange, as it is the worst quality I ever got out of XVID.

I'll redo both movies with Koepi's Feb17th binaries - expect results no sooner than 11 p.m. tonight (it's 9 a.m. now).

Frank

Well, this is odd. The only test i have done with this new build is the trailer test i mencioned above. I am waiting for a Koepi's build with modulated quantization to do a full movie (i have ready and waiting 3 movies: Independence Day, Platoon and The Replacements (again this one)).
The problem that Franko30 speaks about is exactly where i think that the codec has improved: the blocks. In my test, there's a scene where a football player is running and he crashes against a group of other players. WIth the prior builds the exact moment when he reaches the other players and bumps with them was a very blocky image. His body was like a bunch of blocks. With this new build i almost didn't noticed the blocks, the player was very detailed.
But, like i said, this was a small movie trailer test, and Franko30 did a full movie test. Thas has to count for something.

To MaTTer: my trailer tests are done using a 600KB bitrate. I choose a low bitrate because with a high bitrate i can't notice very well the diference between builds. They are still there, but with a high bitrate they aren't so visible.
And the other reason i choose a low bitrate is because i like to do 1 cd encodings (unless i now that is going to look bad, then i move to 2 cd's). So this a more real test to me.

[-h]

@Franko30 - that would have to do with the more "accurate" quantizer decision the codec now has. If you'd like the low-action scenes to have a low quantizer again, try playing with the curve value and testing the dbgview output.

Overall, I've found the latest additions to increase quality on my analog material (DV camera input).

-h

[Baalthazaar]

Originally posted by Franko30
Hi again,
Usually all my recordings with the DVC II card produce MPEG2 files that, when encoded with Koepi's Feb17th binaries, produce almost only Quant 3-4 and higher (up to 8-9) in action scenes.
Frank

Just out of curiousity, how do you deinterlace your Dazzle DVCII movies? I only ask because I have the same card and I'm really not satisfied with the way that Avisynth is deinterlacing the movies. Should I use a Vdub filter instead? My apologies for being off-topic.:)

[Nic]

Get the bug fixed version!

www.freewebz.com/xvid
or
xvid.stormpages.com

-Nic

[Franko30]

Originally posted by -h
@Franko30 - that would have to do with the more "accurate" quantizer decision the codec now has. If you'd like the low-action scenes to have a low quantizer again, try playing with the curve value and testing the dbgview output.

-h


Well, I hate to admit it, but at the moment I'd rather like to encode movies than playing around.
I want to reach a total number of 2000 movies in my collection soon - 1100 just reached now (including VCDs)&the Start trek Voyager collection has to be completed, too. The Feb17th binaries of Koepi are sufficient for my quality needs at the moment. :p

Thanks for the tip anyway - I'll lurk around to keep up with changes etc.

Thanks

Frank

[Franko30]

Originally posted by Baalthazaar


Just out of curiousity, how do you deinterlace your Dazzle DVCII movies? I only ask because I have the same card and I'm really not satisfied with the way that Avisynth is deinterlacing the movies. Should I use a Vdub filter instead? My apologies for being off-topic.:)

Hey great! Another DVCII user!

I use SmartDeinterlace in the *.avs script GordianKnot creates - but I didn't notice any difference to the SmartDeinterlace Filter used in VirtualDub. And I tested some other Deinterlace Filters, too - all worse than SmartDeinterlace.

For more DVCII talk we could create a new thread in the "Capture" section of this forum discussing the timer program "TWNH" etc. - just PM me :)

Cheers

Frank