XviD-19062003-1 [Archive]

sysKin

24th June 2003, 06:27

Hi,
Since I'm here, instead of learning for tommorow:

About 'max bframes' - the statement "will produce only bframes" means that the decision itself will always say 'B' or 'I'. After the 'max' has been reached, B turns to P anyway. 'Max' is always respected.
But the statement is not 100% true anyway, because every next b-frame has its threashold decreased by 20. If you always want 10 bframes in a row, the threshold will have to be 90+9*20 = 270 ;)))

In any case, threshold above 40..60 will just remove any smartness of the decision. If you want to experiment, go for +/- 10 first.

About lambdas and VHQ: VHQ still decreases filesize. It wasn't the case in the last build becuase of the bugs. Lambda is still 1.00 , I don't think changing it will have any big effect (on constant-bitrate/constant-filesize. It will have effect on constant quant, but that's not what we do*) :)

I have a favour to ask - could you please check the new keyframe-decision? Does it miss any scenechangs? If yes, are the scenes dark or bright?
Does it put any unexpected keyframes, which hasn't been there in last build**?

Thanks,
Radek

* well, if you do 'compressability test', you care about constant quantizer result. Another prove that compressability value just stopped showing anything after divx3.11.

** like at the beginning, or in the middle, of any fade

Selur

24th June 2003, 06:48

@syskin: thx for clearing this up

about the keyframes, as far as I can tell form some small clips Iframe distribution is better than earlier and one doesn't have to 'activate' bframes for not ending up with only getting Iframes at the maximum Iframe intervall,..

I'll do further test later, have to hurry to university,.. ;)

CU Selur

BoNz1

24th June 2003, 07:36

Okay, I have got a test ready to go for VHQ 0-4 and I have calculated PSNR for 1-4 and it increases. However, I have been unable to calculate PSNR for VHQ 0 so I don't think I can post my little test. psnr4avi just gives back a whole bunch of garbarge letters for the last few frames and gives me a average and total PSNR values that are nonsensical letters. Sorry, I can't post it, hopefully I can figure out what is wrong. :(

Didée

24th June 2003, 08:21

Koepi,

what IDCT is this build actually using? The changelog on uManiac's site reports that simple was switched over to Walken again.
And WHY? After all the noise in the past about simple idct being "the one", I'm a little confused now.

- Didée

Morbo

24th June 2003, 08:48

Nice build Koepi!!

Could anyone else confirm that its slightly slower...

Im not complaining,but Id like to know if its just me.

Thanks again K!!

Koepi

24th June 2003, 09:46

This build is using SimpleIDCT.
Isibaar switched back to walken idct as he feels in theory it shouldn't be wrong. Well, please test umaniacs build and see if divx/ffdshow decoders produce qpel smearing again.

The speed isn't much of a difference here, can't say much about it.

Regards
Koepi

JohnMK

24th June 2003, 09:50

Thanks Koepi, I appreciate this new build very much! I have a question. When I run a compressibility test, I get a result of 86% for VHQ-0, and 71% for VHQ-4. This indicates that within a fixed filespace, VHQ-0 will have lower quantizers than VHQ-4! I thought I should be expecting the reverse.

Malevolent

24th June 2003, 10:00

@Morbo:

Hmm, actually it seems to be a bit faster.
I encoded earlier today one futurama episode i had been meaning to encode for weeks now, and it took ~2h15min. /pass.
After that i checked the forum, and noticed a new build (now why didn't i see it earlier? Doh!), and decided to encode again with the same script, just to compare.
I had a 300 frame testclip to check filters etc. , so i used it to rough out the b-frame setup (previously i used 3,150,100,200 - now i used 3,150,100,40), and the overall result was quite pleasing.
Time consumed was ~2h5min. /pass. & the result looked a bit better in my eyes.
:devil:

Lobuz

24th June 2003, 10:02

@sysKin
@Koepi

Could that b-frames quantizer decision be more fluent.
For example if there are I,P frames with q2 and there is a frame which couldn't be a B-frame with q4 cause of threshold. But there is a room for lower quantizer q3 with that threshold and it still could be smaller then p-frame.

Could it be done? Or am I wrong?

And I'm curious what is the status of VHQ for B-frames(does it work)?

Regards
Lobuz

Gaia

24th June 2003, 10:04

compressibility test

Little off the topic but i think compressibility tests are useless and don't tell much about final quality. Anyway that's just my opinion.

Wuntvor

24th June 2003, 10:10

@Morbo & Malevolent

I think I remember that the more bframes, the faster, because VHQ doesnt do any work on Bframes yet.

Or maybe this has changed?

regards
/Wuntvor

OUTPinged_

24th June 2003, 10:51

Just tested "Evil overflow" thing is still there. Expect bad results from 2pass encodes if you dont cap them.

Koepi

24th June 2003, 10:59

Outpinged,

while you're on the mission to spread your observation (which is ok) you should remember one thing: if you keep the sound of your posts like this, you're just insulting.

Just set your overflow redistribution from 60 to 30, and you're done. Or turn of bframes and you get what you want. But stop putting things in such unkind words. (Ungratefulness is a habbit of yours it seems. Always just complaining without helping with a single line of c++. What can I say? That's the spirit! That's how you have to take a gift!)

Koepi

Teegedeck

24th June 2003, 11:03

Reminds you of the Linux Router Project, doesn't it? ;)

OT: That gave me the idea we should have something like a 'XviD Hall of Fame' here with the names of everyone who contributed to the project and a special mention to active developers in it and a big THANK YOU! from all of us on the forum.

bilu

24th June 2003, 11:06

Originally posted by Gaia
Little off the topic but i think compressibility tests are useless and don't tell much about final quality. Anyway that's just my opinion.

Compressibility tests have shown in this case that a first pass done with VHQ-4 is harder to compress than VHQ-0 because it gets a bigger filesize.

VHQ used to help compressibility. :confused:

Bilu

Teegedeck

24th June 2003, 11:08

Hi Bilu,

haven't you read the whole thread? :)

sysKin:
* well, if you do 'compressability test', you care about constant quantizer result. Another prove that compressability value just stopped showing anything after divx3.11.

I mean, if sysKin says that, I believe him! How small a movie can be compressed at constant quant=2 isn't related closely to how well the movie can be compressed in 2-pass -- that's the essence of my experience with XviD, too. [Edited further, but that's the last OT thing that I write in this thread:] XviD isn't tuned to deliver optimum results at quant=2.

bilu

24th June 2003, 11:18

You're right, I didn't :o

But now that I did, He also says:

About lambdas and VHQ: VHQ still decreases filesize. It wasn't the case in the last build becuase of the bugs.

Maybe JohnMK was using this buggy build? :confused:

About compressibility: well, it's an approach just like PSNR.
Of course it's not necessarily precise but it gives a safe margin to work with :)

Bilu

Teegedeck

24th June 2003, 11:21

Please see my edited post above.

bilu

24th June 2003, 11:41

@Teegedeck

Safety margin methods are important, that's why people use PSNR and comp.tests.
But this is OT and it sure deserves another thread discussing reliable methods.

Could you start that thread? I'd like to see an intro from you :D

Best regards,
Bilu

sysKin

24th June 2003, 11:43

Originally posted by JohnMK
I have a question. When I run a compressibility test, I get a result of 86% for VHQ-0, and 71% for VHQ-4. This indicates that within a fixed filespace, VHQ-0 will have lower quantizers than VHQ-4! I thought I should be expecting the reverse. Ok this _is_ a problem. And I think I might have a theory now.
Can you tell us if you have P4?
Also, can you make a simple, short test, and encode something at fixed quantizer, with VHQ 0 and 1, and tell us if 1 had smaller filesize?

One of the problems with the previous VHQ was that one asm-ed function was not working as I expected. It was intra-block dequantization (both mpeg-type and h263-type) if I remember. It is ok for normal coding and decoding, but I had decided to be 'smart' and save us entire 128 bytes of memory, and the function stopped working. I changed it to plain C and it was ok with me.
However, perhaps there is more functions which can't work properly that way, and I just don't use them? Like P4 functions.

Anyway, if VHQ 1 results in bigger filesize, do a final test with disabled CPU optimizations. If it's ok, I'll change the whole thing ASAP.

Thanks,
Radek

PS. about iDCT - Koepi, you might have missed it, but I think there is a 'special' code in decoder which forces Walken iDCT if stream ID is below 10 (or something). This means that _de_coder from your build will use Walken, which is different than encoder uses.

This makes PSNR results deadly (I fell for this trap, it's 3-4 dB loss)

PS v2.0 : are you sure you can do PSNR calculations with psnr4avi, if your avi has bframes? The problem is the delay. If you can, please make sure that PSNR of the first frame is reasonable.

if you want, this is my PSNR code:
name = "00.avi"

a = avisource("monsters.avi").assumefps(100).converttoyuy2()
b = avisource(name).assumefps(100).converttoyuy2().trim(1,0)

compare(b, a, "", name+".txt", false)
It works like charm. Remove the "trim" if you have no bframes. Credit for "trim" thingy goes to mf.

Teegedeck

24th June 2003, 11:46

Originally posted by bilu
@Teegedeck

Safety margin methods are important, that's why people use PSNR and comp.tests.
But this is OT and it sure deserves another thread discussing reliable methods.

Could you start that thread? I'd like to see an intro from you :D

Best regards,
Bilu
Oooooh, if quant=2 should still be smaller, I was plain wrong! :p :o

ssjkakaroto

24th June 2003, 11:50

syskin you said that a threshold above 40..60 will just remove any smartness of the decision, so what would you recommend to extremely low bitrate scenarios (the old 3/150/100/255)?

tia

Koepi

24th June 2003, 12:01

Dilemma, dilemma:

I could set the bitstream version a number back.
I could switch to Walken IDCT.
I could remove the check from the decoder.

I think I'll go for the first option ;)

Regards
Koepi

sysKin

24th June 2003, 12:04

Originally posted by ssjkakaroto
syskin you said that a threshold above 40..60 will just remove any smartness of the decision, so what would you recommend to extremely low bitrate scenarios (the old 3/150/100/255)? In my humble opinion 255 was very wrong, just like 80 (100?) would be wrong here.

But as for recommendation, I don't have it: this new decision is experimantal, I haven't even checked thresholds different than 0 ;)

Try 10, 20, 30 and share your impressions with us.

/me should _really_ be learning now ;)

Isibaar

24th June 2003, 13:19

Originally posted by Koepi
Dilemma, dilemma:

I could set the bitstream version a number back.
I could switch to Walken IDCT.
I could remove the check from the decoder.

I think I'll go for the first option ;)

Regards
Koepi

Koepi, never ever switch the bitstream version back! Use higher numbers and tell us which version numbers we should reserve for you but never switch back version numbers. This destroys any chance of automatically activating workarounds for both XviD and libavcodec decoders.

Regarding idct: I posted several mails about this issue to the team mailing list which you have hopefully read. I concluded that simple idct is no good choice for encoding: Walken idct did _not_ cause qpel noise or the infamous smearing problems. These problems appeared when playing back XviD content with ffdshow and were caused by a bug in libavcodec's qpel code. I pointed this out already back in january on the XviD developer's mailing list:

http://list.xvid.org/pipermail/xvid-devel/2003-January/001762.html

This bug has been fixed in the mean-time and the most recent versions of ffdshow should decode XviD qpel flawlessly.

Different idct implementations however are indeed a problem, for qpel but for normal halfpel as well. If possible, the same idct should be used for decoding than was used for encoding. This means that Walken idct material should be played back with Walken, simple idct material should be decoded with simple idct.

The problem with using simple idct for encoding is the fact that this idct implementation isn't used in any other popular decoder besides ffdshow. This means that XviD content encoded with simple idct is _not_ correctly played back by DivX, 3ivX and EnvivioTV. However XviD Walken idct material is decoded correctly in all popular decoders besides ffdshow (because it uses simple idct by default). However ffdshow leaves the option to switch to XviD idct (Walken) which will fix potential problems. Also Walken idct could be automatically activated in ffdshow in the future depending on the XviD bitstream version.

That's why I think that using Walken idct for encoding is the best solution because it causes the least interoperability troubles and existing incompatibilities (with ffdshow) can be quite easily fixed.

kilg0r3

24th June 2003, 13:23

1.)
Originally posted by sysKin
name = "00.avi"

a = avisource("monsters.avi").assumefps(100).converttoyuy2()
b = avisource(name).assumefps(100).converttoyuy2().trim(1,0)

compare(b, a, "", name+".txt", false)

Would I have to adjust the trim value according to the number of b-frames?

2.)
some VHQ file sizes with constant quant 2 on an Athlon XP18000 (250frames)

vhq 0 2490KB (w/o cpu opt.-s 2494KB)
vhq 1 2420KB
vhq 2 2366KB
vhq 3 2348KB
vhq 4 2328KB (w/o cpu opt.-s 2332KB)

Koepi

24th June 2003, 13:30

Thanks for clearing this up isibaar,

i'll simply remove simple_idct again then with the next build.

EDIT:

LOL! Just checked my sources, and guess what: as long as you have a MMX capable processor and didn't switch off the MMX-features in the CPU-tab, Walken IDCT gets used. (note to myself: don't compile builds in a hurry.)

I'll clean that up in my next build anyways, so switching off MMX in the next build won't use MMXed simple_idct (did I eat something bad that day? That's so incredibly wrong but has no effect in "usual usage" *phew* ).

EDIT2:
Further comparing my sources with CVS showed up that on P4, fdct_sse2 gets used in my build. This maybe could explain some of the file size mismatch as well.

Regards
koepi

trbarry

24th June 2003, 15:14

Further comparing my sources with CVS showed up that on P4, fdct_sse2 gets used in my build. This maybe could explain some of the file size mismatch as well.

Koepi -

You mean it gets used on P4's when MMX features are switched off, right? Or is it used all the time on P4's?

- Tom

Koepi

24th June 2003, 15:29

Heya Tom,

no, they just got used when SSE2 optimizations were selected on the debug tab (like with autodetection).

@All:

XviD-24062003-1.exe (389kb)
Changelog:
- Fixed P4 issues and the use of SimpleIDCT when unchecking all CPU optimizations. Walken gets always used again.
Remember to switch IDCT to XviD when decoding with ffdshow.

Well, some issues are solved easily ;)

Regards
Koepi

Didée

24th June 2003, 15:38

Originally posted by kilg0r3
some VHQ file sizes with constant quant 2 on an Athlon XP18000

I want that machine, too!! :D

How many milliseconds did one pass take?

vass-iliskus

24th June 2003, 15:57

Originally posted by sysKin
I have a favour to ask - could you please check the new keyframe-decision? Does it miss any scenechangs? If yes, are the scenes dark or bright?
Does it put any unexpected keyframes, which hasn't been there in last build**?

Well, I did some checks, and here is what we have (1-pass q2):

vhq0, boff(-1): 7 keyrames (misses 2 dark scenechanges)
vhq1-4, boff(-1): 2 keyframes (misses all scenechanges)
vhq0-4, bon(0,1): 7 keyframes (all scenechanges have kf)

[1] has two groups of 2 succcessive keyframes, while [3] has only one.
The source is BW and rather dark, although case [2] happens in more lightened scenes too.

UPDATE: Checked the previous build too (koepi's 14052003).
Same results: when VHQ is in use and b-frames are OFF it misses all the scenechanges, except for the case [3]: when b-frames are ON (0 or greater), the result looks similar to [1] - both dark scenechanges are missed, one additional scenechange is missed, and one of the succcessive keyframes is missing too (just like [3] in the new build), totalling to 5 keyframes.

All in all: new build with b-frames ON has better scene detection.

Koepi

24th June 2003, 15:59

Syskin's ipb-decision is only active when bframes are set to be 0 or more. -1 uses the old frame type decision code.

regards
koepi

vass-iliskus

24th June 2003, 16:15

Originally posted by Koepi
Syskin's ipb-decision is only active when bframes are set to be 0 or more. -1 uses the old frame type decision code.

ok, then we do have some improvement over the last build :)
And still, is it not recommended to use VHQ without b-frames, since it has negative influence on ip-decision ?

JohnMK

24th June 2003, 17:35

Originally posted by sysKin
Ok this _is_ a problem. And I think I might have a theory now.
Can you tell us if you have P4?

Yep I do. Will test.

kilg0r3

24th June 2003, 17:37

Originally posted by Didée
I want that machine, too!! :D

How many milliseconds did one pass take?

Nah it's not that great, gets relatively hot compared to slower models. Sorry for the polar caps guys :)

JasonFly

24th June 2003, 17:37

Thnak you for this build.
I have tested with animatrix( some parts quite hard to encode)

I have used bf(2,150,100,50), and VHQ1.

The results is very good execpet some block in these hard scenes.

Thank you also for your humour syskin.
->CVS changelog:
14.4.2003 15:00:
U xvidcore/src/motion/motion_est.c (rev.1.66) syskin:
- "What bug could i invent today ?"

amango

24th June 2003, 20:05

I tested the new build.

If have an Athlon XP 1800. If I use "Force optimizations" and select all option without SSE2 it seems to be that encoding is faster than with "automatic detection".

"R-D quantisation" always saves some bitrate (about 8-10%) but it lacks quality. In Anime there are always "flashing pixels" in large and bright one-colored areas (faces for example). Without R-D it is fine.

kxy

24th June 2003, 20:37

Originally posted by vass-iliskus
And still, is it not recommended to use VHQ without b-frames, since it has negative influence on ip-decision ?

If we set the b-frame to 0, does it mean b-frame is activated and no b-frame will be used in the entire clip? :confused:

How is this different than disable b-frame?

Nibor

24th June 2003, 20:53

Yes, setting Max bframes to 0 -> no bframes

Originally posted by Koepi
Syskin's ipb-decision is only active when bframes are set to be 0 or more. -1 uses the old frame type decision code.

Here's your answer :D

Edit:
Forgot to thank Koepi, sysKin and all other developers for this new build! I made a few tests, and the quality is nice.... very nice!!!

kxy

24th June 2003, 21:09

Let me clarify my question, what is the difference of diabling b-frame(-1) and set b-frame to zero, besides that it activate Syskin's ipb-decision? So if I set the b-frame to 0, it mean the new ipb-decision is activated and no b-frame will be used? Why there is a need for -1 then?

Koepi

24th June 2003, 22:07

kxy:

some people prefer the old IP-decision. It respects frametypes from the statsfile - if you add them with one of the tools around (i.e. statsreader ;) ) they get inserted in the final encoding.

Koepi

bilu

24th June 2003, 22:27

@Koepi

Is there any way to force keyframes with Zulu's Keyframe Enforcer or your Stats Reader over the new I/P/B decision?

Bilu

kxy

24th June 2003, 23:15

Originally posted by Koepi
kxy:
some people prefer the old IP-decision.
Koepi

Ah, I forgot about the stat-reader, thanks for clearing it up. :)

Let me summarize then,

-1 => NO B-frame, old IPB-decision, not recommand for using it with VHQ

0 => NO B-frame, Syskin's ipb-decision(better scenechange detection), use it with VHQ

jwu42

24th June 2003, 23:44

With respect to the 24-06 build. I ran a few tests using the 19-06 build before I installed the latest. Seems the sizes are SLIGHTLY larger - beyond that, all looks great!

Test Material: The Matrix Scene 29 (The Shooting Spree)

1Pass Quant @ 2

VHQ1 and Bframes 2/150/100/0 48,096 KB with 19-06
VHQ1 and Bframes 2/150/100/0 48,142 KB with 24-06

VHQ4 and Bframes 2/150/100/0 47,080 KB with 19-06
VHQ4 and Bframes 2/150/100/0 47,154 KB with 24-06

MSP=6
H.263 Quant
Chroma Motion
Qpel
Chroma Optimizer

Morbo

25th June 2003, 02:51

Originally posted by Malevolent
@Morbo:

Hmm, actually it seems to be a bit faster.
I encoded earlier today one futurama episode i had been meaning to encode for weeks now, and it took ~2h15min. /pass.
After that i checked the forum, and noticed a new build (now why didn't i see it earlier? Doh!), and decided to encode again with the same script, just to compare.
I had a 300 frame testclip to check filters etc. , so i used it to rough out the b-frame setup (previously i used 3,150,100,200 - now i used 3,150,100,40), and the overall result was quite pleasing.
Time consumed was ~2h5min. /pass. & the result looked a bit better in my eyes.
:devil:

I was using Trells Q,but I didnt think it would be a hit,or is it?

Im using a pan-scan 704x512 test chapter from Big Trouble in little China(the last fight scene).Its a QUITE noisy source,and its full screen as well(it makes a better "test").

I was gonna say using K's suggested B frames and T's new Quant,Im getting DIVX3 type blocks @ 900k/bits(and going by SNowbeach's guide BTW).

Use 7 min test clips,anything less than 1000 frames is a compression test,IMHO.

Cheers and thanks again K!

JohnMK

25th June 2003, 07:49

What should I be expecting out of VHQx?

Compressibility test results:

DivX 5.05 + b-frames: 64%
XviD 2003-06-24 VHQ0: 66%
XviD 2003-06-24 VHQ4: 67%

My XviD settings were fairly normal, MSP6, h.263, chroma motion, chroma optimizer, Trellis-D, NO Qpel, 2/150/75/25.

Is that all VHQ4 is supposed to buy us? I mean -- it's worth using at like 5% or 10% compressibility gain, but only 1% simply isn't worth the lost time.

By the way, this test was run on an Athlon XP.

Teegedeck

25th June 2003, 07:58

Please read the whole thread before replying in it. You should always do this or we get redundancies like this one; i.e. who wants to answer the same question several times within the same thread?

JohnMK

25th June 2003, 08:05

Originally posted by Teegedeck
Please read the whole thread before replying in it. You should always do this or we get redundancies like this one; i.e. who wants to answer the same question several times within the same thread?

I am providing another data point. Apparently you're not aware of the scientific method, are you?

Teegedeck

25th June 2003, 08:14

Originally posted by JohnMK
What should I be expecting out of VHQx?
Is that all VHQ4 is supposed to buy us? I mean -- it's worth using at like 5% or 10% compressibility gain, but only 1% simply isn't worth the lost time.
Providing more data is fine with me but excuse me if I took that for a complaint that you wanted answered. And that answer has been given before. If you didn't want an answer, why did you ask?

Thank you for providing us with the results of your test, have a nice day.

JohnMK

25th June 2003, 08:27

Who says I made a complaint? Is it pejorative to make statements of fact interlaced with personal opinion, and then ask others to corroborate or disprove? I suggest you leave moderating to the moderators please. That's three posts you've spawned now that are just wasting space here.

Teegedeck

25th June 2003, 08:33

I am a moderator...

And I really don't know what I should make of your comments. I feel sorry if you found my posts to display personal preconceptions against your attitude, I might have to reflect on that.

But please read the thread. You'll find out that compressibility isn't supposed to be the only indicator of VHQ's effects - the quality of the result in a two-pass encode is what we look at in the end. And this has been said (in other words) before. In my mind I'm allowed to say that.

Shall we leave it at that or 'waste more space' with moderating? If 'wasting space' is something you are concerned about, I can even delete our conversation here and get the thread more 'on-topic', again. (Just let me know. I'll be off to work now, anyway, so it would have to wait.)

Didée

25th June 2003, 08:51

Mates, I've encountered a big problem with this build.

I was testing with some very dark material, and found the combination qpel+VHQ came out with very bad quality: flat backgrounds were "floating" again to an unacceptable amount. And, even worse, I got again ... qpel smearing! :( Plus, the image degraded noticably over time ("dirty" picture), with some shocking refresh-effect on the next keyframes.
Okay, syskin mentionend above that the decoder might fall back to walken idct, which is not good here since the encoding was done with simple idct by the core ... but even when decoded by xvid.dll itself, the floating backgrounds and qpel smearing were there.
However, when I took the clip encoded with 16-06, and played it through either 14-05's decoder or DLL, the picture quality was much better - but in no way as good as the older encodings.

But it comes worse.
I've done a lot of encodings with the (famous, for me) 14-05-2003 build. After encoding, they were all fine.
Now, when playing back 14-05 encodings with the 19-06 build, the clips that were fine before show ... QPEL SMEARING!! :eek: ...Here too, it's not only a decoder problem: with 14-06 installed, the older rips show smearing even when played in VirtualDub! Therefore, the problem seems to be in the core.

Friends, please do also some testing of how older clips look with this new build.
I'm getting my first grey hairs here!

- Didée

[Edit]
Oops, somehow I missed some posts on page 3, the fact that Koepi made a little mistake, and that a new build is up.
But now, I'm even more confused:
Originally posted by Koepi
...guess what: as long as you have a MMX capable processor and didn't switch off the MMX-features in the CPU-tab, Walken IDCT gets used.Here I understand: Since I left auto-detect on, 19-06 has used walken idct on my AthlonXP, right?
Originally posted by Koepi
Fixed the use of SimpleIDCT when unchecking all CPU optimizations. Walken gets always used again..Here I understand: Now walken idct gets used everytime.

Well ... but it was the walken idct that produced all those artefacts I described above! (I assume it was).

If I'm not mistaken,
- 14-05 yet used simple idct. This gave no problem together with qpel.
- 19-06, with default settings, used walken idct. This gave me big problems.

I would like to check how 24-06 is doing things, but ATM it seems like no testing until weekend, for me.

Please bare with me if I should have produced nonsense above ... but if I see a potential major problem, I usually start crying loud ;)

Clarifications are welcome
... if there's anything to clarify.

Tueurne

25th June 2003, 09:03

A very little bug with Matrix type

There is no difference between MPEG et MPEG-custom. both always load the personnal matrix...

yaz

25th June 2003, 09:36

Originally posted by Tueurne
A very little bug with Matrix type ...
imho, it's more than that. what happens on 'defaulting'? how can i check what matrix is actually in use? (if it's not that selected)

y

kilg0r3

25th June 2003, 10:50

@ Tueurne & yaz

this is an old bug. there are two work arounds i know of:
(a) after using the custom matrix reiinstall the codec
(b) save the standard matrices as files and improt them as custom matrix. I still don't know the values of the h.263 matrix though; only remeber that all values are the same except the 8 in the upper left intra corner.

JimiK

25th June 2003, 11:15

I am a moderator... :p
And a good one, I have to add (also I have no complaints about any other mod. You guys all do a great job).
One thing I don't understand: everytime somebody complains about your moderation you say you're sorry and you might have been to harsh. Why? You always had a valid point. Is this sarcasm? O.K., I don't want to escalate this.
@Didee
Yes, your encode should have used Walken iDCT (from what I understood). Now I'm not very experienced with QPel, never liked it too much. But Isibaar stated that there has never been a problem with Walken iDCT itself, but only with the implementation in libavcodec. Now I'm wondering how it can still produce smearing. And hasn't been there QPel smearing when decoding with XviD in former times? According to Isibaar, it should never have been there. I'm sure, as one of the main developers, he knows what he's talking about, but maybe the problem just did not show in his tests? (Waiting for XviD2 with integer arithmetics (why are you guys all waiting for XviD1.0 and 1.1 ;) ))
Best regards,
JimiK

yaz

25th June 2003, 11:55

Originally posted by kilg0r3
this is an old bug. there are two work arounds i know of:
(a) after using the custom matrix reiinstall the codec
(b) save the standard matrices as files and improt them as custom matrix. I still don't know the values of the h.263 matrix though; only remeber that all values are the same except the 8 in the upper left intra corner.
oops ... i've missed this info :-(( does it relate only to mpeg/custom changing or is it general? i mean, if custom's been ever used then no other can be, or is it possible to change back to h263? (just to make me easier to figure out why are my latest encodes so far from what i expected. now it seems to be a matrix problem :-(
btw, what's the matrix found as custom by default?

thx for the answer
y

kilg0r3

25th June 2003, 12:28

@yaz
AFAIR the problem is only that, after having used a custom matrix, it can happen that the codec will stay configured to use mpeg-custom no matter if you select h.263, mpeg, modulated or mod-hq. Loading th standard mpeg matrix from the custom-matrix dialog should definitely do the trick. And, if someone could supply us with the values for the h.263 matrix all would be warm applepie with cream (neoidiomism).

yaz

25th June 2003, 12:59

@kilg0r3:
thx again! (i'm sure that's responsible for some 'surprising' encodes of mine)

h263: a time ago, one of the devels (sorry, i can't recall who) wrote, it's not a matrix in a strickt meaning as it contains only one(some?) discrete value(s). thus using it needs simplified operations. would any of the devels drop that coeffs? pls, pls, pls :-)

thx
y

ps where can i find this 'standard mpeg' matrix? (what mistake would i make next? :-(

kilg0r3

25th June 2003, 13:56

@ yaz

to your service :D

after installing the codec and hitting load defaults, the quantization tables should contain the standard-mpeg matrix.

Tueurne

25th June 2003, 14:26

Why reinstall the codec ? load default isn't enought ?
Well don't care, I will reinstall ...

Prettz

25th June 2003, 17:24

@didee:
I'm not quite sure if I can confirm your decoder bug. I was just looking back at a 2CD encode that was made with the 5/14 build (which I remember as looking pretty good, but I can't remember perfectly what my impression back then was) that used MPEG matrix, VHQ4, QPEL, chroma motion, and b-frame max -1. There appears to be massive, grotesque artifacting around edges (like ringing with really messed up, seemingly-random chroma), and some of the classic tiny solid green blocks. I remember there being lots of seemingly-random chroma noise in all my encodes, and I just assumed it was unavoidable, but might the chroma noise I noticed back then have been a bug with the 5/14 build? Either way, I'm fairly certain there wasn't this much artifacting when I viewed the movie using the 5/14 build. I can't recheck at the moment because I'm encoding it again with the new build.

And on to the new build. I'm encoding that same clip with all the same settings as I used with the 5/14 build except for b-frame max is now 0 instead of -1 (and yes I did load the codec defaults before setting it up). Previously (for the first CD of the movie) every frame was quant 2, except for 38 quant 3 p-frames.
Now, I'm looking at the debug output for the second pass being encoded with the 6/24 build and nearly every frame is quant 3, with large numbers of quant 4 frames and a few quant 2 frames here and there. I don't know what's going on but I have a feeling this encode is not going to look good when it comes out.

edit: I mixed up b-frame threshold with b-frame max, corrected now

edit 2: disregard my comments on quantizers for the file encoded with the 6/24 build, I messed up the settings

OUTPinged_

25th June 2003, 17:28

h263 seems to be "all 24s" matrix.

Koepi

25th June 2003, 17:47

h263 matrix should be "all 32" values - except the topleft corner of course.

If you read Isibaars post again you'll see that there is a bug in libavcodec which is used in ffdshow. only newer ffdshow alpha builds have that bug fixed, so switch to one of these! And remember to switch IDCT in the misc options to XviD.

Regards
Koepi

DoW

25th June 2003, 19:30

Im slightly confused. I know that in the 14052003 Koepi build that -1 disabled bframes and used old IPB and 0 disabled bframes and used Syskins new IPB. Since the new build introduced the -40 to 90 thresholds, what are the corresponding thresholds for using old IPB and Syskin IPB in the new build. Also, will there ever be the option to add keyframes using utils under the Syskin IPB (need to be able to so I can add chapters to my ogms).

kilg0r3

25th June 2003, 19:40

Originally posted by Koepi
h263 matrix should be "all 32" values - except the topleft corner of course.

Ah o.k., now I understand why it is said to give blurrier images than the mpeg matrix at low quants ...

Here is what I gathered about the 24062003 build (a birthday build, finally!) so far by following this thread.

* build always uses walken idct now.
* 'suggested' b-frames settings: 2/150/75/0
* new b-frame threshold range -40 to 90; suggested setting is 0. Thresholds > 40 will remove the smartness of the decision algo, though.
* trellis still works only with h.263
* b-frames seem to look very good now (good enough for anime)(Teegedeck)
* syskins new i/p/(b)frame decision is active when max b-frame is set to 0 instead of -1. This results in better scene change detection. Always use it when using vhq.
* there might be something wrong with vhq, we'll see.
* former built had probs with vhq on sse2 machines
* "R-D quantisation" always saves about 8-10% but, in Anime there are always "flashing pixels" in large and bright one-colored areas .
* Some reports about strange artifacts waiting for Prettz and Didee

Nibor

25th June 2003, 20:14

Originally posted by DoW
Im slightly confused. I know that in the 14052003 Koepi build that -1 disabled bframes and used old IPB and 0 disabled bframes and used Syskins new IPB. Since the new build introduced the -40 to 90 thresholds, what are the corresponding thresholds for using old IPB and Syskin IPB in the new build.
No no no!
Its the __Maximum B-frames__ setting, that determines the IPB decision which get used!
Maximum B-frames -1 -> old decision gets used...
Maximum B-frames 0 -> sysKins new decision gets used...
Which method gets used has _nothing_ to do with the treshold values!

Get it?
Or am I wrong?

DoW

25th June 2003, 21:30

This is what I get for trying to reason out something without being able to download and install it (silly Mac OSX work machine;))

BiaTch 5.0

25th June 2003, 22:04

So every encode done with this build needs IDCT set to decode properly? Hell of a bug. If this is the case I think I’ll switch back to XviD-14052003-1.

Prettz

25th June 2003, 22:35

What would cause giant pixelated white letters to be displayed on every frame of your encode? There goes a 12-hour+ encode...

OUTPinged_

25th June 2003, 22:36

@koepi:
h263 matrix should be "all 32" values - except the topleft corner of course.
(VHQ, bframes, gmc, qpel, trellis, chroma_opt are off, me=6, quant=2)

h263: 2,494,464

all 32: 1,626,112
all 24: 1,961,984
all 20: 2,576,384
all 16: 3,702,384

"all 20s" = h263 ?

Gaia

25th June 2003, 22:49

What would cause giant pixelated white letters to be displayed on every frame of your encode?

You selected "Print debug info on each frame"...

iago

25th June 2003, 23:23

Originally posted by BiaTch 5.0
So every encode done with this build needs IDCT set to decode properly? Hell of a bug. If this is the case I think I’ll switch back to XviD-14052003-1. BiaTch 5.0,

No need to be rude, and this is not a bug! Read Isibaar's post again, and again if you still cannot understand the reason why. :angry:

Btw, you can even switch back to divx3 or 4 or 5 or something if you like...

iago

unplugged

25th June 2003, 23:37

Lately I'm using MPEG matrix, because give me smaller output than H.263 with some stuff, at least with quant > 2 when flasking from MJPEG captures.
With MPEG m., if I remember fine, "trellis" is not working (yet), are there other new features not (fully) working or not tested with MPEG matrix?

Thanks :)

Koepi

25th June 2003, 23:37

EDIT:
Bad tamper is over, I still can't help it, I feel like nearly noone is reading the FAQ, the releasenotes, this thread, the guides... but commenting about things already covered there.

(Don't use VHQ and GMC together).

Koepi

Teegedeck

25th June 2003, 23:47

Originally posted by BiaTch 5.0
So every encode done with this build needs IDCT set to decode properly? Hell of a bug. If this is the case I think I’ll switch back to XviD-14052003-1.
[*whistling all-friendly-like*] This is our private testing-ground, remember? [*singing like in the Musical-extravaganza in Buffy season 6*] We all volunteered to crash-test new and exciting stuff because we're bored to use proven and well-working solutions, bored, bored, bored. (This is 'stable' XviD.) Be happy about the excitement. Happy, happy, happy...

Alestrix

25th June 2003, 23:57

Originally posted by Nibor
Maximum B-frames -1 -> old decision gets used...
Maximum B-frames 0 -> sysKins new decision gets used...

I'm a little bit confused here... :(

I don't read the forum as often as I used to, but did a search on "syskin b-frame decision" and also found (might even be linked in this very thread) on this Thread (http://forum.doom9.org/showthread.php?s=&threadid=54729&perpage=20&highlight=syskin%20bframe%20decision&pagenumber=3) the following (written by crusty):

soemthing like -100, can't remember exactly, is the least amount of B-frames, while 255 is the most possible.
SysKin said in another thread that the 0 setting is programmed to be the most intelligent mode of decision. So it's safe to use.
And only -1 or 0 in max number of B-frames pervents their use.

and on this Tread (http://forum.doom9.org/showthread.php?s=&threadid=48443) I found (written by syskin himself):

As for positive (thresholds) - no, it's not predictable. I'd recommend 10..50..100 if you like bframes. You'd have to limit 'maximum' or you'll really make things look bad. People use 255 here, which almost always means "maximum" - and this maximum of 3 looks very bad, of 2 looks better, of 1 means just divx5.

So in one thread it's stated that a max of 0 or -1 decides which b-frame decition is used, in the second thread it sais that 0 and -1 disable it which is supported by the last thread I quoted.

Can someone please enlighten me? (pointing me to a thread I didn't find will suffice)

Thanks!
- aL

Alestrix

25th June 2003, 23:59

Originally posted by Koepi
Ok, I finally can't help it, you#re just a whining a******.

Go away, stay away, use DivX.

GMC and VHQ?

You're SO smart. (...)

My feeling was that what he meant was VHQ, bframes, gmc, qpel, trellis, chroma_opt are all off

- aL

BoNz1

26th June 2003, 00:37

Originally posted by Alestrix
Can someone please enlighten me? (pointing me to a thread I didn't find will suffice)

Thanks!
- aL

Sure, not a problem. If you read this thread specifically the posts by Koepi you will understand what to use.

Prettz

26th June 2003, 00:39

Originally posted by Gaia
You selected "Print debug info on each frame"...
Oh dear. I had always enabled that in the past because I thought that's what enabled the normal debug output :rolleyes:. It never actually put letters on my video before when I enabled it though...

kxy

26th June 2003, 01:01

Hey guys,

For those of you who can't reproduce Didée's bug report, can you try and turn the luminance up a level or two OR just turn the monitor's brightness up some more. I used the qpel + VHQ4 combination and I wasn't able to see the "bleeding/smearing" effect that he described until I adjusted the luminance gain level.

Alestrix

26th June 2003, 01:10

Sure, not a problem. If you read this thread specifically the posts by Koepi you will understand what to use.

Thanks BoNz1, must've been the "hunter syndrome" - only seeing what's far away :)
So iiuc -1 uses ip-decision from statsfile, 0 syskin's new ip-decision and anything above 0 uses syskin's ipb-decision while still adhering to the max-no.-of b-frames constraint if using threshold of 0...

Off for my next encode,
Alex

Prettz

26th June 2003, 01:16

Yes, I can confirm that decoding files encoded with the 5/14 build looks significantly different (in a bad way) between using the 5/14 build and the 6/24 build.

edit: forgot, the clip I tested this on used VHQ4 and qpel.

unmei

26th June 2003, 02:40

using (quite*) my settings as for 14-05, 24-06 is about 10% slower on K7 1.2GHz (sse,3dnow,3dnow2 by autodetect) and about 15% slower on PIII 600mhz (sse by autodetect). I think that's worth it.

Again, anime look better - although i think the big step was already done with 14-05, probably Trellis RD and/or improvements in B-frames. Just to throw in some unbased numbers: 14-05 allowed to lower the bitrate for anime by 30%, 24-06 by another 10%. Trellis has not produced any of the mentioned specks in flat areas until now, neither in 14-05 nor 24-06.

The only artifact i could think of coming from Trellis would be a very rare "block-flip" (4x4 or 8x8 pixel i guess) but that is always at edges, say clear crayon lines of a few pixel width. It appears for about 2 frames, i guess consecutive B-frames until a P is reached and only showing up once in every second or third episode of 20min. (also in 14-05, not new in 24-06)

Still anime, at extremely low bitrate (260kbit/s, avg. Quant around 9)
build 24-06 produces a picture that seems "sharper" or especially the more complex parts of a still scene retain more details, at the cost of slightly more noticeably blocking or banding in large areas with soft color gradients (clouds and trees, less on walls and floor). IMHO this is more pleasing on normal watching as you usually concentrate on the sharper more complex parts are usually there the "action" happens (a bit like RealVideo but with blocks, not smear :)

*my anime settings are:
MSP6, h263, VHQ4, QPel, chromaMotion, B 2/150/75/0, chromaOpt and Trellis R-D . i used a B-frame offset of 100 in 14-05, that being only difference.
no i live prefectly happy with qpel, bframes and trellis in anime, in fact i think those are are the very things that make it good

replaced "sky" wth "clouds" which makes clearer what i meant :)

Danzel

26th June 2003, 05:41

Just finished a 515meg encode of Final Fantasy: Spirits within.

using Lancsoz (spelling?) resize and qpel (with vhq4, 2 bframes, trellis, chroma opt, h263, etc, etc) (on a P4 Processor)

It looks Excellent, sharp and just... wow, its really good.
well.... except the credits, heh.

using 15% compression for the end credits and they looked bad, like pre-smearing, which was sorta weird... I think I probally compressed to much, or something...

@keopi
that OGMCalc looks pretty nice, but when i calculated using 3 streams (1 x 128kbit stereo ogm and 2 x 54kbit stereo ogm) and the size it suggested me to compress my video to was 14 meg too big.

im re-encoding now anyway, gotta clean up the credits and I threw away one of the audio streams (decided I dont want it now).
i've turned the credits compression to 20%, can anyone point me to something like a guide for compressing credits, or suggest some ways to compress them nicer ;)

Thanks Heaps!
Danzel.

BiaTch 5.0

26th June 2003, 08:20

Btw, you can even switch back to divx3 or 4 or 5 or something if you like...Don't worry I knw what I can & can't do :P . I don't know what your getting upset about just stated this build is not for me, what's the big deal? not like you a dev or anything :D .

Didée

26th June 2003, 08:29

Okay, I was able to test the 24-06 build very quickly (for less than 10 minutes, that's all I could grasp for now).
Couldn't spot any obvious problems like before.
Phew, waving green flag.

- Didée

Nibor

26th June 2003, 09:45

@Danzel

To your credits question...

standard black and white scrolling text credits :
I normally do a seperate 1 pass (quantizer) for that. In the AVS-File i use no filters expect LumaFilter (and maybe UnFilter)... You could use a Value like -20 for LumaFilter.. For the Xvid settings i'd propose to use as much B-frames as possible ;) try 5, or even 10... Qpel helps a lot too! The quantizer can be set to something arround 16, depends on the credits of course...

other credits :
For that I do a seperate 2 pass.. Filtering for high compressibility, 5 B-frames... Testing different settins is always good! :D

Then when I have my credits in lowest size and the acceptable quality, I encode the main movie part....

Works great for me! Just my 2 cents, and MHO :)

Have a nice day!

Isibaar

26th June 2003, 11:13

hm, when reading this thread I have the feeling that there is quite a lot of confusion about the idct topic. So I think I should clear some things up:

Originally posted by Didée
Mates, I've encountered a big problem with this build.

I was testing with some very dark material, and found the combination qpel+VHQ came out with very bad quality: flat backgrounds were "floating" again to an unacceptable amount. And, even worse, I got again ... qpel smearing! :( Plus, the image degraded noticably over time ("dirty" picture), with some shocking refresh-effect on the next keyframes.
Okay, syskin mentionend above that the decoder might fall back to walken idct, which is not good here since the encoding was done with simple idct by the core ... but even when decoded by xvid.dll itself, the floating backgrounds and qpel smearing were there.
However, when I took the clip encoded with 16-06, and played it through either 14-05's decoder or DLL, the picture quality was much better - but in no way as good as the older encodings.

This shouldn't be the case. A clip encoded with XviD and then decoded with the same build in VirtualDub should not exhibit any (unusual) smearing. I think I recall that there was some problem with Koepi's previous build. Can you recheck using his latest (24-06) build? Just encode a clip with this build, decode it in VDub using the same build and check whether you see any unnormal smearing?

Originally posted by Didée
But it comes worse.
I've done a lot of encodings with the (famous, for me) 14-05-2003 build. After encoding, they were all fine.
Now, when playing back 14-05 encodings with the 19-06 build, the clips that were fine before show ... QPEL SMEARING!! :eek: ...Here too, it's not only a decoder problem: with 14-06 installed, the older rips show smearing even when played in VirtualDub! Therefore, the problem seems to be in the core.

ok, this is not surprising: your old encodes have been made using simple idct and are now decoded with Walken idct. This creates a mismatch which causes these artifacts. However this is not a problem of neither simple idct nor Walken idct. Both are bug-free and comply to the requirements of the IEEE-1180 and ISO/IEC 14496-2 standards.

The whole issue is a general (design) problem of MPEG-4: Regardless which idct implementation we'll choose in our encoder (simple-, Walken or something else), there will _always_ be artifacts (like smearing) be introduced as soon as you use a decoder that implements a different idct.

Because of this, I think it's reasonable to implement the most popular and most widely used idct implementation into the XviD encoder in order to achieve highest possible compatibility. And the most popular idct is Walken (because it's used in DivX and 3ivX, for example). Simple idct isn't used in any MPEG-4 codec despite libavcodec.

All in all there is no perfect solution for the idct problem because as soon as decoder and encoder use different idct implementations, artifacts will appear. However, XviD Walken content is decoded quite well with all popular MPEG-4 decoders (DivX, 3ivX, EnvivioTV) despite ffdshow/libavcodec (but you have an option to switch idct here, so no problem). On the other hand, XviD simple idct content is never played back correctly with any decoder besides ffdshow.

Originally posted by Didée
[...]
Well ... but it was the walken idct that produced all those artefacts I described above! (I assume it was).

Walken did _not_ produce these artifacts. It's the mismatch between encoder and decoder idct implementations which is the problem. Walken is just as fine (or bad) as simple idct and vice versa.

Originally posted by BiaTch 5.0
So every encode done with this build needs IDCT set to decode properly?
yes.

Originally posted by BiaTch 5.0
Hell of a bug.
no bug.

Originally posted by BiaTch 5.0
If this is the case I think I’ll switch back to XviD-14052003-1.
your choice. But keep in mind that Koepi's 14-05 build uses simple idct. This means that your 14-05 encodes play in ffdshow without switching idct, yes. But they won't be decoded properly with anything else besides ffdshow (not with DivX, no 3ivX, no EnvivioTV...). Also keep in mind that DivX certified hardware players (existing and future devices) are based on the DivX decoder and also won't decode your 14-05 content properly because of idct mismatch.

I'm currently thinking about how to resolve the problem that different XviD versions use different idct implementations. You have to know that the XviD CVS version (equivalent to uManiac's builds) always used Walken idct. Simple idct has only been activated in Koepi and Nic builds as a 'special feature'. Therefore it's unfortuntely impossible to detect which XviD versions used which type of idct implementation - that's a big problem.

What we could do is to figure out starting from which version Koepi and Nic activated simple idct and simply treat all these XviD versions as simple idct versions. This will fail for clips created with uManiac's builds but I suppose Koepi/Nic builds are by far more popular, right? Then we could guarantee that clips created with Koepi or Nic builds will continue to play correctly with future XviD versions...

iago

26th June 2003, 11:30

Thanks once again for the explanations, Isibaar.

@BiaTch 5.0,

What gets on my nerves is your flip attitude, if you know what I mean. Bug and necessity are two different things, and choosing XviD IDCT in ffdshow should not be such a big inconvenience except for some.....

Anyway...

regards,
iago

O/T: Koepi, it's burning like hell here, my friend, beware! :D

U977

26th June 2003, 12:09

Hi all,

First, thank you Koepi for this new build!
As always, what I do here is mostly reading, not posting (I waited a whole year before registering on the forum :-) ).

Thank you isibaar for the explanations, too!

And for the many reasons which prevent me to read here regularly enough, I also would like to thank Killgor a lot for the summary he makes about a build. This help me much, still when I came here, I've read, but forgot three days later! lol

We are not at an Oscar ceremony, so I will stop here, altough many more have to be thanked too...

I did not get enough time to test the new build yet. Actually, I've though several times to help, but I think I'm still too new to post anything interesting. This is why I stopped myself to do yet.

Indeed, it would be great to have Xvid decoders switching automatically between Simple or Walken IDCT algorithms to get the best decoding. But from the far position I see this problem, hats off i you have an idea on how to do! Otherwise, I'll be happy to change that manually in the decoder options.
I did a bunch of encodes using Koepi's build 14052003 :-) Wish I would have been able to contribute to posts like here with results.
However, all what I can do is post stat files content, and comment on the quality... At least, I think I'm not too bad at that.

Newbie question: why is Walken IDCT more used/more popular?

Th3-S4int

26th June 2003, 15:12

Hi,
I have a short question: at my Pc xvid chose automatictly 3dnow 2, but i have a 1.2 GHZ Thunderbird and iam sure that a thunderbird cant handle 3dnow 2! i think it was firstly given in the atlon xp (i dont know the Code name)!

Koepi

26th June 2003, 15:21

I propose a checkbox in the xvid decoders:

"[ ] use SimpleIDCT (check when quality seems bad)".

Should be easy to do within the sources IIRC. That would be the best solution in terms of elegance.

Regards
Koepi

sysKin

26th June 2003, 15:23

Originally posted by Th3-S4int
I have a short question: at my Pc xvid chose automatictly 3dnow 2, but i have a 1.2 GHZ Thunderbird and iam sure that a thunderbird cant handle 3dnow 2! i think it was firstly given in the atlon xp (i dont know the Code name)! No, SSE was added to XP. 3dnow2 was added to athlon/duron (first 3dnow was added to K6-2 I think)

kxy

26th June 2003, 17:19

Originally posted by Koepi
I propose a checkbox in the xvid decoders:

"[ ] use SimpleIDCT (check when quality seems bad)".

"[ ] use SimpleIDCT (check when quality seems bad or check when smearing occurs)

I Like this idea. :)

bugsan

26th June 2003, 21:18

new PSNR-table

xvid koepi 24-06-03, 2pass: 20000ko (650kbps)
avisynth 2.52
Mpeg2Dec3 1.08
FFdshow 030103
Athlon XP 2000+
---avs------avs------avs------avs------avs------avs---
mpeg2source("E:\Ripping\lotrcutted\lotr.d2v",idct=7)
Crop(4,80,-4,-80)
BicubicResize(640,256,0,0.5)
---avs------avs------avs------avs------avs------avs---
clip1 = mpeg2source("E:\Ripping\lotrcutted\lotr.d2v",idct=7)
.Crop(4,80,-4,-80).BicubicResize(640,256,0,0.5).ConvertToYUY2()
clip2 = directshowsource("xvid_XX.avi",fps=25).ConvertToYUY2()
Compare(clip1,clip2,"YUV","psnr.log")
---avs------avs------avs------avs------avs------avs---
info:
altcc-best = "altcc-h-150-100-50"
exotic = "mpeg vhq4 bf1 cm cmop altcc-best"
pp4 = "ffdshow nic pp4 strength 256"

Builds general comparison:

|---------|---------|
| K140503 | K240603 |
|-----------------------|---------|---------|
| default | 41.9028 | 41.8955 |
| default pp4 simple | 42.2854 | 42.1851 |
| default pp4 walken | - | 42.2744 |
| default pp0 walken | - | 41.8940 |
|-----------------------|---------|---------|
| chroma motion | 41.9698 | 41.9864 |
| global motion comp | 41.8826 | 41.8799 |
| quarter pixel | 41.5587 | 41.5599 |
| lumi masking | 41.8397 | 41.8423 |
| chroma optimiser | 41.8964 | 41.9019 |
| trellis R-D 0 | 41.8684 | 41.8855 |
| mpeg quantization | 42.0297 | 42.0356 |
|-----------------------|---------|---------|
| VHQ 0 bf -1 | 41.9028 | 41.8955 |
| VHQ 1 bf -1 | 42.0766 | 42.0887 |
| VHQ 2 bf -1 | 42.1032 | 42.1782 |
| VHQ 3 bf -1 | 42.1491 | 42.2292 |
| VHQ 4 bf -1 | 42.2199 | 42.3316 |
| | | |
| VHQ 0 bf 0 | | 41.8946 |
| VHQ 1 bf 0 | | 42.1150 |
| VHQ 2 bf 0 | | 42.2359 |
| VHQ 3 bf 0 | | 42.2522 |
| VHQ 4 bf 0 | | 42.3327 |
| | | |
| VHQ 0 bf 1 | | 42.0270 |
| VHQ 1 bf 1 | | 42.2614 |
| VHQ 2 bf 1 | | 42.3151 |
| VHQ 3 bf 1 | | 42.3338 |
| VHQ 4 bf 1 | | 42.4166 |
|-----------------------|---------|---------|
| bf0 default | 41.9028 | 41.8946 |
| bf1 default | 41.8666 | 42.0270 |
| bf2 default | 41.7618 | 41.9992 |
| bf3 default | 41.7480 | 42.0043 |
| bf4 default | 41.7471 | 41.9937 |
|-----------------------|---------|---------|
| altcc-default | 42.1407 | 42.1330 |
| altcc-best | 42.3478 | 42.3417 |
|-----------------------|---------|---------|
| exotic | 42.6715 | 42.9207 |
| exotic pp4 simple | 43.0756 | 43.2172 |
| exotic pp4 walken | - | 43.2652 |
|-----------------------|---------|---------|

Koepi 240603 bframe test:

|---------|---------|---------|---------|---------|
| 1 - 150 | 1 - 190 | 2 - 150 | 2 - 190 | 3 - 150 |
|---------------|---------|---------|---------|---------|---------|
| bf thresh -40 | 41.9154 | 41.9148 | 41.9160 | | |
| bf thresh -20 | 41.9632 | 41.9542 | 41.9632 | | |
| bf thresh 0 | 42.0270 | 42.0494 | 41.9992 | | 42.0043 |
| bf thresh 10 | 42.0545 | 42.0750 | 42.0118 | | |
| bf thresh 20 | 42.0500 | 42.0745 | 41.9598 | | |
| bf thresh 30 | 42.0407 | 42.0689 | 41.9061 | | |
| bf thresh 40 | 42.0286 | 42.0655 | 41.8842 | | |
| bf thresh 60 | 42.0125 | 42.0318 | 41.8165 | | |
| bf thresh 80 | 42.0007 | 42.0225 | 41.7905 | | |
| bf thresh 90 | 41.9973 | 42.0103 | 41.7871 | | |
|---------------|---------|---------|---------|---------|---------|
| bf offset 0 | 41.9844 | | | | |
| bf offset 25 | 41.9887 | | | | |
| bf offset 50 | 42.0181 | | | | |
| bf offset 75 | 42.0270 | 42.0494 | 41.9992 | | 42.0043 |
| bf offset 100 | 42.0115 | | | | |
| bf offset 125 | 42.0104 | | | | |
|---------------|---------|---------|---------|---------|---------|

Koepi

26th June 2003, 21:42

Interesting results. Thanks for the work!

Can you check the "default PSNR" value at the top without PP and with xvid idct again please for the 2405-build? Thanks a lot :)

EDIT: is the conversion to YUV2 really necessary? Each colour space conversion comes with a little quality loss...
EDIT2: the ffdshow build you use has some issues with xvid idct and qpel. this got fixed with later builds, (I thikn the 2405-alpha-build of ffdshow has this fixed). If you could retest the qpel clip again and the default values with that build, it would be really interesting how bad the bug is mathematically :)

Best regards
Koepi

bugsan

27th June 2003, 02:00

in all tests i use xvid internal decoder, not ffdshow, except for the post processing test. i will retest with a newer ffdshow :) and i will add an internal post processing test.

Compare() doesnt accept YV12 format :( => "incompatible clips"

BoNz1

27th June 2003, 03:07

@ bugsan, thanks for the psnr table. I did a similar test recently too with similar results. I have one query though, did you by any chance do an encode with VHQ 0? The reason I say this is because this is quite important to know what the psnr is for this setting since the last test like this VHQ 0 had a higher psnr than 1-4 even though from 1-4 psnr did increase. Like I said before, I did a similar test but was unable to get the results for VHQ 0 because psnr4avi gave me an error on the last couple frames, unfortunately. Thanks.

Danzel

27th June 2003, 07:36

This build is very very nice.

Big thanks to all the crew for their work.

One worry though. I was getting really weird results when I was trying to compress the credits in 2 pass, if I set them to come out at 10meg then they would end up way oversized at 30 or 20meg (depending on bframe settings) and look REALLY bad, smearing and random noise (Checked in vdub too).
But then when i did it at constant quant 4 it came out at 12meg and looked excelent, sorta weird.(max 10 B-frames, didnt look better with 5, havent tryed with -1)

The 2 pass encode (20meg) was giving very high quantizer values for most of the clip (11 B, 9 P at the start. 31 B, 29 P for the majority)
the bits at quants around 9 had tiny smearing, and the ones around 31 had major random noisy blocks (around the scrolling text).

Settings: Qpel, H263, lumi masking, greyscale, chroma motion, Bframs: 15,150,75,50, chroma opt, trellis.

Oh, damn it. I just found the problem, trellis...

So Trellis is possibly buggy for low bitrates?

Other than that my encode of Final Fantasy: Spirits Within looks great, havent seen any bad details, and after a bit of work got the credits down to 12 (11 now!) meg looking good. (Thanks Nibor!)

Danzel.

CruNcher

27th June 2003, 12:37

@bugsan

this is a nice test but i belive it is flawed and i try to explain why, you compareing 2 differnt builds of xvid here 1 which used the whole time SimpleIdct and one which uses now the Walken idct so now if you use as in your script shown idct7 which is Simpleidct and make an VBLE avi with it it would have the characterstics of that before the encoding you used idct7 to decode the picture from the mpeg2 domain and here is the importants proccess. This will change the picture you can see (not see) this if you decode the picture with idct2 or 5 for example the filesize in the end of the VBLE avi will not be the same as compared to idct7 and so something in the picture must have changed it is not anymore the same as an idct2 decoded picture this can also bee tested if you try to PSNR compare those 2 VBLE against each other normaly if they both are still the same the PSNR has to be infinite but it wont be infinite this has also to do with the fact that you cant actually use 2 idct methods to compare only 1 will be used by the decoder and psnr wouldn't match and you doing even more worse things now in your compare you take this idct7 decoded mpeg2 and put it into the xvid encoder wich uses walken idct this would change the picture again and the final psnr as yours is showing would be lower then the one where you used the whole time the same idct.

and now if somebody says idct is a looseless procces i ask him how can it be loosless if filesize in the end differs ?????

so i think to compare idct transforms with PSNR is wrong it wont work and so also my test i did some month ago is flawed it shows only that SimpleIdct produces smaller filesize but it doesnt show that the quality of a SimpleIdct Picture is actually better or more worse then compared against a Walken transformed Picture, the only thing it shows is that the filesize differs more not, now you could speculate why it differs is SimpleIdct accurater and produces (decodes) a cleaner picture or is the Reference Idct better and produces a better picture and for that needs more space ?

Koepi

27th June 2003, 12:54

hell, where are the commata, full stops and paragraphs? Your post is for lakc of a better word unreadable (though I think I got the idea).

Please try to write a bit more reader-friendly next time :)

To the content: I think the error introduced by the decoding SimpleIDCT for the MPEG2 doesn't have an influence on how the encoding IDCT (Simple/Walken) performs. The test for this would be easy:

1) decode with default IDCT, encode with simple and with walken
2) decode with idct=7, encode with simple and with walken.

The resulting PSNRs will give you an idea of dependencies.

Regards
Koepi

Lobuz

27th June 2003, 13:40

One more thing I discovered a little bug with ffdshow postprocessing compared to NIC's decoder PP. It inserts some strange discolour.
I will test it a little more.

ps. what idct is used in NIC's decoder?

Regards
Lobuz

Sigmatador

27th June 2003, 13:55

@Koepi
I've read isibaar explanation about dct, so i supose it's a bad idea to decode the mpeg2 stream with the simple idct right ? (erf! it's the fastest on athlonxp :( ... well nevermind, it's a so small gain ;) ).
In the mpeg2dec3_1.08 thread, we've seen that the simple idct produce a PSNR gain at fix Q2 (another bugsan test table :D ) , i suppose it comes from the bit "blurring" effect of the imple idct?
so it's definitevely a bad idea to use the simple idct (except with mpeg4 stream encoded with your old build) right ? (i want to be sure ^^ )

cruncher: about your question: maybe there is a reason to be called "simple" no ?

Koepi

27th June 2003, 14:05

I use simpleidct for mpeg2 decoding in the meantime with no visual bad side effects.

Regards
Koepi

Soc

27th June 2003, 15:51

Originally posted by CruNcher
and now if somebody says idct is a looseless procces i ask him how can it be loosless if filesize in the end differs ?????
The first part of the DCT process is lossless, ie the transformation of each block into the 64 different signal coefficiants, since you could take those coefficents, multiply them by their respective blocks, add them together and get exactly the original source block. However, the loss comes from dividing each coefficiant by its number in the quant matrix, and by the frame quantizer, thus reducing "zeroing" or "dropping" some coefficents, and reducing the number of bits required to store the others. This rounding off effect will of course hurt quality when the DCT is reversed.

Sorry for going a bit OT, I tried to keep it short :rolleyes:

- Soc

wing1

27th June 2003, 15:54

wow, 24062003 build is awesome! thanks you xvid developers.

1-pass cbr + bf (2/150/75/50) + chr me + chr optz =>8Mb/min for 672x400@NTSC. 2-pass int =>4.8Mb/min for the same res above. Visual quality is simply grand :D

bugsan

27th June 2003, 16:53

I confirm that PSNRs aren't the same for default 1405 and 2406. May be the DCT process...

VHQ 0 is there, it's the default test :)

@koepi
I can't test post processing with xvid-dec : i've enabled "luma deblocking" and "chroma deblocking" in encoder control panel, but no PP appears :confused:
something is wrong ?...

EDIT: ffdshow without pp and with idct-xvid, added.

Lobuz

27th June 2003, 22:37

I've made a few decoder PSNR tests with lates 24-06-2003 xvid encode
at quant 2 and default settings

I have made an original clip compressed with vble from avisynt 2.52

LoadPlugin("mpeg2dec3.dll") version 1.08
mpeg2source("DVDsource.d2v",idct=3)
crop(2,2,-0,-2)
LanczosResize(640,352,0,0.75)

and decoded it with avisynth DirectShowSource( ) function in VDubMod from 11-06-2003 with appropriate dshow decoder
to another vble compressed file

and compared it with

psnr4avi file1.avi file2.avi 350 20 20

and started from 20-th frame to avoid that greenish lines at the beginning in ffdshow

And the results

pp divx ffdshow nic's dec
-------------------------------------------------------------------------------
5.05 24-04-2003 23-05-2003 30-03-2003
libav xvid libav xvid
w-idct s-idct dll w-idct s-idct dll
-------------------------------------------------------------------------------
0 46.25 46.22 45.31 46.25 46.22 45.31 46.25 45.31

1 X20Y40 46.27 46.24 46.24 46.26 45.35
X33Y40 45.37 46.25
mplayer 46.18 45.89

2 X20Y40 46.29 46.27 46.11 45.16 46.25 45.38
X33Y40 45.35 46.28
mplayer 46.05 45.27

3 X20Y40 46.37 46.35 46.26 45.84 45.49
X33Y40 46.28 46.30
mplayer 45.86 44.32

4 X20Y40 46.43 46.41 45.57 46.20 45.94 45.10 45.58
X33Y40 46.24 45.59 45.51
mplayer 45.45 42.86

5 X20Y40 46.40 46.15 45.52 45.50
X33Y40 44.91
mplayer

6 X20Y40 46.40 45.60 44.77 44.42 45.45
X33Y40 44.91
mplayer

With latest ffdshow there are some problems with NIC's PP but increasing X threshold from 20 to 30 helps a little.

With older ffdshow xvid dll is somhow working badly

Nic's dshow decoder presumably is using simple idct so the value isn't high

It looks like there is no one stable best solution. Maybe with DivX decoder for latest officialy approved walken idct;)

ps. after these test I found that vble conversion to RGB24 is somhow not right at the edges and the psnt4avi is asking RGB24. Doing that conversion in avisynth script could raise PSNR values but overal dependances shoul stay thesame.

edit: ffdshow version datas were switched

Regards
Lobuz

Tueurne

28th June 2003, 12:17

A question about ffds :
Simple use simple IDCT
Xvid use walken IDCT

but what kind of IDCT use normal and reference ?
this look pretty good for this last build

puschpull

28th June 2003, 14:26

Hallo!

(Sorry for my English!!)

I have needs mete exactly job time in VirtualDubMod.
Contemporary Job Control offers metering only with accuracy on minutes.
But I need accuracy on tenths seconds.

Meanwhile mete exercise by the help of DebugView, but has it many problems.
<a href="http://www.sysinternals.com/ntw2k/freeware/debugview.shtml">Sysinternals Freeware - Utilities for Windows NT and Windows 2000 - DebugView</a>
Please, isn't any possibility mete run-unit exercise in Job Control on tenths seconds?

Please, Give Me One or More Tips for exactly measurement Time Jobs in Batch Mode for my Testing

Thank you!
:-)

CruNcher

28th June 2003, 15:06

@ all
ok what i said on the previous page that was a bit as koepi said it correct unreadable, because of the lack of punctuation here is what i meant in a compare :)

XviD Encoder idct mpeg2dec3 idct Filesize
--------------------------------------------------
Original (idct=7) 64,8 MB (67.969.024 bytes)
Original (idct=2) 64,8 MB (68.002.816 bytes)

XviD Simple Idct (idct=7) 6,62 MB (6.944.256 bytes)
XviD Walken Idct (idct=2) 6,63 MB (6.960.128 bytes)

Linear parallel Compared
----------------------------------------------
Simple Idct XviD avi compared vs Original (idct=7)

Decoder XviD with SimpleIdct PSNR=50.3561
Decoder XviD with WalkenIdct PSNR=50.0742

Walken Idct XviD avi compared vs Original (idct=2)

Decoder XviD with WalkenIdct PSNR=49.9695
Decoder XviD with SimpleIdct PSNR=49.8312

Cross non parallel Compared (giving totaly meaningless results)
----------------------------------------------------
Simple Idct XviD avi compared vs Original (idct=2)

Decoder XviD with SimpleIdct PSNR=50.3309
Decoder XviD with WalkenIdct PSNR=50.0500

Walken Idct XviD avi compared vs Original (idct=7)

Decoder XviD with WalkenIdct PSNR=49.9714
Decoder XviD with SimpleIdct PSNR=49.8331

but please don't think now that only because the psnr value of idct=7 is higher then the of idct=2 means it is better that would be false cause its linear parallel compared

Koepi

28th June 2003, 17:30

Ok, let's see:

- Using SimpleIDCT in XviD on your testfile gave a PSNR increase of between 0.1-0.3dB depending on the decoding IDCT (which is quite much, didn't expect that).

- Using SimpleIDCT for decoding MPEG2 increases PSNR at least 0.1dB. If using SimpleIDCT in XviD additionally, the increase is stunning 0.5dB compared to the "standard IDCTs".

Thanks CruNcher for these tests, they could be useful. Unfortunately SimpleIDCT isn't standard except for ffdshow/libavcodec, it'll be hard to convince Isibaar to use SimpleIDCT anyways ;)

But your results show that PSNR is increased by 0.1dB even when the SimpleIDCT-XviD is decoded with Walken IDCT. So it should better in any case.

Can you take the time and do 2-3(or more) different test files, encode them with SimpleIDCT (as your test shows, using SimpleIDCT for MPEG2-decoding is the way to go) and check PSNR when decoding with Simple/Walken and make a nice table again? This can be pure luck in that sample :)

Anyways, thanks a million for those tests!

Best regards
Koepi

loni_blues

28th June 2003, 22:37

Hello,

I´m sorry, but all this seems a little confusing to me. I need to clarify this:

- encodings done with the 14-05 build should be decoded with simple IDCT.
- encodings done with the 24-06 build and so on should be decoded with Walken IDCT.

Is this correct?

CruNcher

28th June 2003, 22:47

many many tests later....

ok i came to a conclusion now, whatever you are a fan of either XviDs Walken Idct or FFMpegs Simpleidct do not mix them neither @ playback nor @ encoding, what i mean is when you encode for example mpeg2dec3->idct=7->xvid walken you will lose quality and not only that your filesize will pump up without any quality win and as said you even will lose quality.

to pretend the best quality in your encodes there is only one way stay allways with the same idct, for Koepis newest build this would mean mpeg2dec3->idct=1,2 or 5

if you still have Koepis older builds you have to encode exactly the other way so mpeg2dec3->idct=7, or as said before you will loose quality and pump up the size.

so where is basicaly the difference between XviD and SimpleIdct ok
from what i could see it is in the visual representation of the final picture, you can see this by yourself when you encode one time mpeg2dec3->idct=7->xvid simple and one time mpeg2dec3->idct=1,2 or 5->xvid walken, what you will see is that the filesize is exactly the same but the CRC of both files are differnt now take 2 pictures, but be carefull as i said before stay with the same idct when you take the pictures and i do not advise you to use ffdshow for this. Build your own xvid.dll and take these pictures then through vdubmod, now save them somewhere in a folder and slide through them what you will see is that Simple Idct blurs the picture a little and the noise is somewhere another aligned, but you wont see a reall visible difference this difference is also allot smaller if you would compare an AVS with idct=1,2 or 5 and idct=7 there you cant even see a difference but filesize when encoded to some looseless format is alot lower for idct=7 but this has nothing to do with Mpeg4.

puhh hope this could be of use :) and i hope FFMPEg is changing from SimpleIdct as standard decoder in libavcodec it woult be much better in terms off interoperability as isibaar also mentioned it before :)

Thx goes out to Syskin for a real constructive discussion hehe :D and Koepi for his kindly support :) and the whole #XviD guys and ofcourse the main XviD developers thx for this wonderfull codec.

had to edit ofcourse only idct1,2 and 5 are the same sorry

CruNcher

28th June 2003, 23:09

@ loni_blues

- encodings done with the 14-05 build should be decoded with simple IDCT.
- encodings done with the 24-06 build and so on should be decoded with Walken IDCT.

not only 14-05 all releases before that too all the builds that used Simple Idct and that where alot of builds i think Koepi started @ beginning with the Qpel smearing problem useing Simple Idct :( but it should be easy, to detect them by their bitstream and change idct accordingly this would however destroy interoperability to for example umaniacs encodes, because they have exact the same bitstream identification and the decoder would then change to Simple Idct but there where actually encoded with walken so the only way @ the moment is by checking it manualy and setting it in ffdshow :(

but it isn't that bad actually its only really bad if you used things like qpel a real visible degredation with the standard features isn't really that visible and should be not that much of a problem thats what i experienced.

loni_blues

29th June 2003, 01:15

Thanks CruNcher!

You wrote:
it should be easy, to detect them by their bitstream
Can you explain how?

Also, there may be some problem with my eyes, because I did the test of changing idct between simple and xvid in ffdshow while watching the movie I had encoded with the BSPlayer, and I can´t see any notorious difference in quality. I used the 14-05 build for encoding it. What´s more, I did the encoding with Qpel. Am I really blind or am I missing doing something?

Loni Blues

TheXung

29th June 2003, 02:37

Not all the builds used in the world come from Koepi, Nic, or uManiac. There is no easy way decide which iDCT to use when decoding.

bond

29th June 2003, 14:39

sorry to write about this, which isnt really xvid build related

Originally posted by bugsan
clip1 = mpeg2source("E:\Ripping\lotrcutted\lotr.d2v",idct=7).Crop(4,80,-4,-80).BicubicResize(640,256,0,0.5).ConvertToYUY2()
clip2 = directshowsource("xvid_XX.avi",fps=25).ConvertToYUY2()
Compare(clip1,clip2,"YUV","psnr.log")Originally posted by sysKin
if you want, this is my PSNR code:
name = "00.avi"
a = avisource("monsters.avi").assumefps(100).converttoyuy2()
b = avisource(name).assumefps(100).converttoyuy2().trim(1,0)
compare(b, a, "", name+".txt", false)It works like charm. Remove the "trim" if you have no bframes. hm i cant really figure out this two scripts:

ad bugsan)
shouldnt be the 4th line in your script:
Compare(clip2,clip1,"YUV","psnr.log")!? as the encoded file comes always first?

ad syskin)
i searched the forum but i didnt really found any hints what the sense of assumefps(100) and trim(1,0) is?

thanks for clearing this up

Soc

29th June 2003, 15:23

Originally posted by bond
ad syskin)
i searched the forum but i didnt really found any hints what the sense of assumefps(100) and trim(1,0) is?
You can find their descriptions (and a lot of other useful info) in the Avisynth manual (http://www.avisynth.org/index.php?page=AviSynthManual).

- Soc

bond

29th June 2003, 15:34

:rolleyes: of course i know what the original sense of this two filters is but i dont know why to use them for psnr calculating...

but thanks anyway

bugsan

29th June 2003, 17:07

@bond
with avisource() there is a "bframe lag" frame, so we need to skip it with trim(1,0)
and with directshowsource() there isn't this frame...

bond

29th June 2003, 17:47

thanks :)

Prettz

29th June 2003, 21:50

I've read this thread several times but I'm still not understanding where the relation between the IDCT used to decode an MPEG2 clip in an avisynth script and the IDCT used to encode the clip to Xvid comes in.
The encoder knows nothing about whether the frames it's recieving came from a DCT-encoded source, so I'm not seeing how the two are related at all.

Can someone learn me on what the deal is?

bond

29th June 2003, 22:12

i now did some heavy testing and in my test clip a quantizer offset of 100 and some higher threshold (~30) gave me visually surprisingly a sharper image than with the default b-frames settings (2-150-75-0) (compared the clips with avscompare)!

settings:
hvs_good
vhq4
qpel
2 bframes
chroma motion and optimizer

sysKin

30th June 2003, 06:02

Originally posted by Prettz
I've read this thread several times but I'm still not understanding where the relation between the IDCT used to decode an MPEG2 clip in an avisynth script and the IDCT used to encode the clip to Xvid comes in.
The encoder knows nothing about whether the frames it's recieving came from a DCT-encoded source, so I'm not seeing how the two are related at all.

Can someone learn me on what the deal is? You're absolutely right, there isn't any.
Moreover, if the same mpeg2 clip is decoded with two idcts, it creates two different clips. Encoding two different clips with xvid will always produce different psnrs - just like encoding two different movies.

Radek

iago

30th June 2003, 06:52

Hi all,

Just a humble opinion of mine based on Koepi's and uManiac's latest builds ;):

In its current status XviD (with b-frames) is really powerful, delivering 'fantastic' results even with a 1stPass/2ndPass ratio of '2/1'.

Currently my preference is 'no filtering' at all unless it is absolutely necessary as long as I can manage to stay around or even a bit under this ratio.

Keep it in mind before you take the risk of ruining/smoothing :D your encodes with excessive or unnecessary filtering while trying to reach a compressibility of ~ 60%-80% or something! ;)

(Atm, I use b-frames 2/150/75/(0->with Koepi's build), vhq1, cm, quantization type MPEG for 2CD and usually h263/hvs_good for 1CD encodes.)

Still, that's me and it's all something personal of course. Anyways...

best regards,
and thanks once again to all developers,

iago

Teegedeck

30th June 2003, 08:18

iago, you really speak my mind, again! What you say is so true, I cannot repeat it often enough.

yaz

30th June 2003, 12:27

dear devels!
24.06 is damn good.
thx a lot!!!

- i'm confused w/this simple/walken business pretty much. afais, i/we should always be aware of the type of dct used for encoding so as to assure the best decoding. it's ok with my own encodes but what to do with that of others. i mean, is there any way to figure out what type of idct does fit to a certain source the best? (besides trial&error, of course:-)

- i-frame placement seems to be ok with vhq1/mbf0 but ... max interval seems to be neglected completely. i found frame distances 250-360 when max was set to 250. is it normal? (anyway, i don't care too much about it until all scene changes are detected well. & they are!)

@iago:
how do u solve the problem of 'switching back from custom matrix' (mentioned above)? i don't really like this re-install-all-days-solution. is this problem still existing at all? (just ask cus u're a hvs-good fan:-)

the bests
y

iago

30th June 2003, 20:12

@iago:
how do u solve the problem of 'switching back from custom matrix' (mentioned above)? i don't really like this re-install-all-days-solution. is this problem still existing at all? (just ask cus u're a hvs-good fan:-)yaz,

I have never had that problem with custom-matrices fortunately. (Or is it possible that I had but I just didn't realise it! :p) However, in my recent encodes (with Koepi's 24/06 and uManiac's 26/06) I have used only MPEG and h263 so far.

regards,
iago

Assault

30th June 2003, 20:25

@ iago

Currently my preference is 'no filtering' at all unless it is absolutely necessary as long as I can manage to stay around or even a bit under this ratio.

Do you think that the quality of dark scenes(black blocks problem) got better with latest builds or did you simply mean that you don't use any filtering beside this lumafilter/unfilter combination?

Assault

Gaia

30th June 2003, 20:34

iago

30th June 2003, 21:03

Do you think that the quality of dark scenes(black blocks problem) got better with latest builds or did you simply mean that you don't use any filtering beside this lumafilter/unfilter combination?Assault,

I don't know whether it's due to eliminating unnecessary colorspace conversions as a result of working in the native colorspace of YV12 (as Gaia also points out) or due to recent builds working better on that point now, but I think there has been improvements in that respect. Though I don't view often on TV lately, I have not noticed that problem for quite a while when viewing on my 17" CRT monitor. (You know I've always been pretty allergic to this black-blocks problem and when it exists it usually doesn't escape my eyes! ;) However, that problem might also depend on the source imho, but now at least we know how to deal with it when it occurs -> blockbuster, lumafilter/unfilter, etc.)

Back to your question, since I don't see/notice this problem any more, no, I don't use lumafilter/unfilter either unless I need them for some apparent purpose.

regards,
iago

Assault

30th June 2003, 21:11

@ iago

Thanks. :)
I think I'll try my next encodes without lumafilter then and I'll report it here if I see black blocks after all. ;)

Assault

yaz

1st July 2003, 13:23

dear devels (or any1)!

can u comment my note on 'max i-frame' neglection? sometimes it can be more serious than i expected.
i'm just trying to encode some home-video for my family. most of them consist of long slowly moving scenes, no scene changes in its strict meaning. imho, here it's essential to insert keyframes regularly. however, with vhq1/mbf0 i got 2x(3x!) of the max k-frame interval. of course, i could switch off vhq, but this way that some scene changes still existing are neglected. is there any other way to use the 'new' scd together with a stricktly kept max i-frame value?

thx
y

Koepi

1st July 2003, 13:33

Current Iframe insertation works best ever for me.

(Using usual content though.)

If you need more keyframes you have to set your limit to a shorter period - i would hate to see the final working iframe-decision which puts the keyframes on spot, even in dark scenes, broken again for inserting IMO useless keyframes.

Regards
Koepi

yaz

1st July 2003, 13:48

@koepi
ok ok ok ... scd is pretty good, i agree. my problem is why 'max i-frame interval' is neglected? imho, it's a different issue. or isn't it? however, if u say it's normal, i'll accept.

thx
y

Koepi

1st July 2003, 13:54

I'm trying to check this, but I always get iframes _before_ the max iframe interval is reached. Need a different source.

EDIT: sysKin just tested "max i frame interval" = 3 and it worked like charme... must be something special with your source i fear :( We can't reproduce that behaviour.

Regards
Koepi

yaz

1st July 2003, 15:13

Originally posted by Koepi
... must be something special with your source i fear :(
definitely. these videos were made as most of these home-made stuffs. somebody grabbed the camera & moved slowly round the crowd, slow zooms here & there. in general, all 'scenes' have the same character, no sudden changes, anything. (not an action movie but a sleepy afternoon in the garden, or so :-) so i'm not surprised that scd doesn't feel the need of k-f insertion. the only scene changes are the frames of camera on/off.
imho, the only way to avoid gradual degradation here is inserting keyframes regularly (i meant 5 sec). the problem is that scd overrules it somehow pushing k-fs away, up to ~15 sec sometimes.

just to make my point clear (& cut this long story short :-)
i don't think it's a seroius problem from the viewpoint of encoding quality. it's just a bug. it makes trouble only with some clips/scenes like mine. but with such 'special' sources i have there's no need to use a sophisticated scd at all.

EDIT:
- in the meantime i've made encodes quite acceptable by switching off vhq. 'old' scd worked perfectly.
- i checked some of my late encodes & i found the same 'shifting away'. ok, i just found it cus i knew what & where to look for :-) i checked 20min from a 'regular' movie & i found only 3 'displacement' out of hundreds. good rate, isn't it?

the bests
y

kilg0r3

1st July 2003, 16:01

@yaz - you did set the b-frame max value to >=0, right?

yaz

1st July 2003, 16:18

Originally posted by kilg0r3
@yaz - you did set the b-frame max value to >=0, right?
no, i don't. i've tried so far:
vhq off / mbf -1 : i-frames in place (but some is missing) max interval kept
vhq 1 / mbf -1 : i-frames only at max & nowhere else (practically, no scd!)
vhq 1 / mbf 0 : i-frames placed well, no one seems to be missing, max neglected

the bests
y

sysKin

1st July 2003, 16:57

Originally posted by yaz
vhq 1 / mbf -1 : i-frames only at max & nowhere else (practically, no scd!)Actually there is a thing I have to tell you all: when I created VHQ, I didn't care about max bf = -1. And I still don't :P
So assume "max bf = -1" as officially not supported by VHQ ;) which doesn't mean it won't work ;) but there might be problems with scene-change detection.
It all won't matter in version 1.0.
vhq 1 / mbf 0 : i-frames placed well, no one seems to be missing, max neglectedThat's really weird, I tried (as Koepi wrote) both my build and Koepi's build and it worked... Indeed, it's difficult to find a clip when max = 300 will be reached so I tried small numbers, like 3.

Also, there is no connection between VHQ and scene change detection with bframes active. If there is, it must be some extremely nasty bug.

Regards,
Radek

bugsan

1st July 2003, 22:20

psnr-table updated (page 6) with various vhq+bf tests
vhq+bf0 gives a better psnr than vhq+bf-1 :)

sungey

11th July 2003, 17:12

wow this thread sure is long ... :P

so now it seems the smearing in QPel that we have a while ago is not because Walken idct being not as accurate as Simple idct .. but actually caused by different idct being used for encoding/decoding?

i do alot of xvid encoding and playback using DX50 for public ... i have yet to see/hear any prob with playback since Koepi 1405 and older version used Simple IDCT and DX50 filter uses Walken IDCT. (i didnt use qpel).

So now i wonder if we will get the same Qpel smearing when using this 24062003 build and play it back using DX50. Did anyone try this yet ?

frodoontop

11th July 2003, 18:08

You won't have the smearing since divx uses walken idct as well.

BoNz1

15th July 2003, 06:57

Originally posted by sungey
So now i wonder if we will get the same Qpel smearing when using this 24062003 build and play it back using DX50. Did anyone try this yet ?

You mean through ffdshow or the DXN DivX decoder? I assume you meant the later since the first wouldn't probably make any difference at all. The reason why you don't want to play qpel encoded movies in this decoder is because of a very very nasty effect, I suppose it is some sort of chroma shift much worse than any smearing. I can post some screenshots of it if you like.

U977

23rd July 2003, 07:31

Hi all,

I did my second encode using Koepi's 24062003 build.
Movie: PAL progressive, 25 FPS, A.R. 16:9, about 1h40 long.

Quick settings summary:
2 passes
Motion search precision: Ultra
Qantization type: H.263
FourCC: XviD
VHQ = wide search
Max I-frame Interval: 250
Min I-frame Interval: 1

Checkboxes: Chroma motion enabled, all the others (Qpel too) disabled.

B-frames used, with settings 2/150/75/0.

DX50 B-VOP compatibility checked.

In the debug options: Chroma optimizer is used, Trellis R-D quant not used.

All the rest was left to default, except:
Below i-frame distance: 13 (accordingly to Snowbeach guide)
i-frame bitrade reduction: 25

Also, since I was going to a 2CD rip, with a res of 704x(something I forgot), and since I had a high bitrate, I've selected "playback with bias".

The resulting AVI was of very good quality (excelllent, as usual with XviD), except the last 15 minutes. At this point, I got a very surprising smearing, and many big blocks, like if I was viewing a movie encoded at res of 320 or probably even less, on a large monitor at full sceen. However, those occured in scenes with less light, but it wasn't dark, though. In scenes with a lot of light, the quality was quite OK, altough slightly less good than usual.

Playback was made with the native XviD decoding (I installed ffdshow, but I disable it for XviD 24062003 encoded movies playback).
Players: I tried with Windows media player 9, and BSPlayer.

I will test playback using ffdshow tonight, to see what it produces.

Also, I'm at work right now, but when I will have more time, I can post stats. I kept all the files produced by GKnot and XviD after the encoding.

See you later! :-)

Teegedeck

23rd July 2003, 07:54

Did you by any chance set the starting-frame for the credits too early?

U977

23rd July 2003, 10:50

Now that you mention it....
Usually, I just strip credits, and don't even encode them.
But here, because I had plenty remaining space left on the second CD, I intended to keep them, without even encoding at lower quality...
Glad enough with that, I'm sure to remember that I simply forgot to put the slider at the very end of the movie, leaving it as it was for the previous movie...

Wow, I just earned the prize for the most stupid post of the week!
And I thought it may have been my first contribution in testing builds...
It's because of my wife disturbing me for taking dustbins out, because of the neightbourgs loudly listening at Celine Dion' songs, it's not my fault! Please believe me! :-)
Yet, I'll hide from here for a century or two, now! lol

Think twice about checking the most obvious, first.

Thanks, Teegedeck!

kxy

23rd July 2003, 19:55

smearing with credit is still not acceptable.

Danzel

25th July 2003, 07:22

from one of my encodes, i found that if you use trellis and try do your credits at very low bitrates you get smearing, but the same settings without trellis gives nice looking credits.

anyone else seen this?

Danzel.

sysKin

25th July 2003, 10:20

Originally posted by kxy
smearing with credit is still not acceptable. Use many bframes (5?) and high bframe threshold (50? 100?) and you'll get rid of any smearing.

Radek

U977

25th July 2003, 10:41

(Can't hide for long...)

It's a bit off topic, but since I hear speaking about B frames: I've read contradictory statements, at different places.

Somewhere on this forum (don't remember where. I read too many things a day to remember where I've read something), I've read that a B frame takes more disk space than a P frame. I've also read that the quality obtained when viewing a B frame is less good than the quality we have when viewing a P frame, and that when we use more B frames, it allows to save bits that can then be used to improve the overall quality. This is why B frames help to improve quality.

So, there must be something I misunderstood. Either B frames are smaller than P frames, which would allow to save bits, either there is something I really did not catch. Can someone enlighten me about B frames, please?

Thanks!

Selur

25th July 2003, 16:21

As far as I remember b-frames are only larger than p-frames if they only refer to one frame (not two, like it should be).

Cu Selur

Acaila

25th July 2003, 16:35

B-frames are larger than P-frames when you compare both at the same quantizer. Also, I believe that a B-frame that is referenced by two frames is larger than one referenced by one frame.

However B-frames are never used at the same quantizer as P-frames, but always with a higher quantizer, so that is when they are smaller and help overal quality. The fact that B-frames don't cause error propagation (like P-frames do) allow you to get away with a higher quantizer.

yingx2

25th July 2003, 17:08

Originally posted by Acaila
B-frames are larger than P-frames when you compare both at the same quantizer.

Should be smaller.
Did you test it?

A clip encoded with bframe control 3/100/0/255 should be smaller than a clip encoded without bframes when using quantizer mode.

sysKin

25th July 2003, 17:49

Originally posted by Acaila
B-frames are larger than P-frames when you compare both at the same quantizer.
Actually Bframes are still smaller, but because Pframes are bigger when Bframes are present (because they describe picture change over larger distance), the whole video is larger (not much and not always, but still).

Also, I believe that a B-frame that is referenced by two frames is larger than one referenced by one frame.Bframes are designed to reference from two frames. This always happens, but if one of the reference frames is useless (because it belongs to diferent scene, for example) Bframe doesn't work as good as expected.

Acaila

25th July 2003, 18:19

Originally posted by yingx2
A clip encoded with bframe control 3/100/0/255 should be smaller than a clip encoded without bframes when using quantizer mode.On my tests they were either equal in size or the one without B-frames was smaller.

Originally posted by sysKin
Actually Bframes are still smaller, but because Pframes are bigger when Bframes are present (because they describe picture change over larger distance), the whole video is larger (not much and not always, but still).I thought the extra referencing info always makes B-frames larger unless you up the quantizer.

kxy

25th July 2003, 19:27

Originally posted by sysKin
Use many bframes (5?) and high bframe threshold (50? 100?) and you'll get rid of any smearing.

Radek

Can this be tuned in the codec level? Instead of doing 2 seperate encodes, we can let the codec handle it in the credit encoding part.

symonjfox

25th July 2003, 21:38

Originally posted by kxy
Can this be tuned in the codec level? Instead of doing 2 seperate encodes, we can let the codec handle it in the credit encoding part.
Good IDEA! For example calling it "Advanced Credits Treatement" (or ACT :D ).
My idea is that credits will be threated as special, just in the second pass. For example it will be possible to change B Frames thresolds, number of max B frames, Quantizers (min,max, fixed), and so on; but the features that MUST NOT be changed will remain unselectable.

Good idea, I really hope it will be developed in the future :)

PS: Maybe extend this idea not only to credits, but to several frame selections.
PS2: Maybe an external program that join all files togheter in one shot should be nice. For example, I create 10 AVS scripts, each one will give 1 scene of a stuff I want to encode. I open VDubMod and I create 10 jobs (one for each part, one with its own settings). I let VDMod encode them separately. Then a simple program would be nice to rejoin everything togheter (ex. part01.avi, part02.avi ... partxx.avi ---> Alltogheter.avi)

drebel

26th July 2003, 00:44

Quote PS2:Never heard of GordianKnot's special credits treatment through trim function ? :D
Nice idea about credits.Somewhere in IRC I saw a credits part encoded by Syskin which was simply amazing!Please ,do share the knowledge with us, or ,even better ,keep it as an option somewhere in debug tab (only available space IIRC) or as a special gift with dev-api-4 ,if possible ;)

symonjfox

26th July 2003, 11:26

Originally posted by drebel
[B]Quote PS2:Never heard of GordianKnot's special credits treatment through trim function ? :D

I never used GKnot :D ... I always do it by myself ... manual scripting, manual encoding and so on ...

wannabe

5th October 2003, 23:12

Hi

I read trough the thread. I can confirm the same problem that Didée and Prettz stated about 24062003's Qpel's decode smearing. I understood that this shouldnt be a bug, just different idct: Walken and Simple.

BUT: as you advised if i playback a 24062003-Qarterpel-encoded clip in FFDSHOW (20030523 or 20030927) with "xvid iDCT" set in, it still has the same smearing problem. Yes. It only plays back flawlessly if i use 24062003 for playback. But this cant be a solution, because as yaz pointed out, i cant tell from other encodes that they made with what specific codec, so in the future these encodes would be useless if there would be no decoder that could automatically decide which iDCT to use.

And i repeat, i set IDCT to "XviD" in the Latest ffdshow, restarted the application, rebooted my machine etc, and its still bad, still has the smearing effect, altough maybe not that much as with other idct but still it looks pretty bad. And no, autoselect does not help either.

So it seems that the Walken iDCT that 24062003 uses is not compatible with ffdshow's xvid idct to me... Any ideas ?

Greetings