XviD-20072002-1 with new curve treatment

[Koepi]

Ahoy,

I decided to upload the binary since my "hard case" test succeeded with it. I hope Foxer submits this code to CVS soon (even if he dislikes my fix for "high quantized scenes after consecutive keyframes", but it works, anbd this is what counts ;) )

Please test the new 2nd pass internal curve compression (you can't switch it off ;) ) a bit and tell me your results please.
Ah, and just to be sure that consecutive keyframe sequences don't look like sh*t, better set max iframe quantizer to 5 or something in that direction.

XviD-20072002-1:
- Compiled with intel compilers & optimizations
- EPSZ(^2) Motion Estimation activated
- New BFrame code from gruel with direct mode - still buggy and ugly :-/
- New curve treatment implemented by Foxer, more stable overflow managemant

Best regards,
Koepi

[cabal]

i am just curious - what's this hardcase test ?? can you say your specs for this test ??

greetings cabal

[Nic]

@Koepi:
Can you do a compile without BFRAMES_DEC or does that interfere with the ffdshow stuff (I still havent got around to looking at ffdshow !)

If you can, then I can send you a new Dshow filter without the green bug (but with BFRAMES_DEC on it doesnt work at all :( )

-Nic

[Koepi]

@nic:

uh, i didn't define bframe_dec at all - just bframes(and this is just lazyness, i wanted to disable them but i was to eager to see results of my modifications ;) )
So it's strange you encounter prblems... should I generally #undef BFRAMEs, may that help?

@cabal:

hehe, nice to see you here as well ;)

well, my hard case is the movie I mentioned in the "new curve treatment thread". It has plenty of scenes where consecutive iframes are necessary, and with the old binaries it was har dto make an acceptable encoding even at 640x272 for 1CD... this binary makes it BETTER quality (setting b-frames to -1 i have to insist on) than everything i did before...

I hope this helps!

Regards,
Koepi

[Nic]

No BFRAMES is fine, BFRAMES_DEC is the bit that doesnt work well. I should try discussing it with chemn over the next week see if can help resolve it.

Im going to try & work on xvid a bit more again, Ive been using DivX5 recently but last night I was testing different bicubic settings. The test clip I used looked cr*p even with 2pass (at the bitrate of 628 I was using), so I thought Id try XviD. Came out excellent. So I thought "Back to XviD" it is then ! :) (& that was compared to DivX5 with Bframes)

Cheers,
-Nic

[soujir0u]

I just tested encoding a small clip (resized to 320x240) with Constant Quality 100, and I found the quality a lot better than the previous build I was using (30062002-1). There was a lot less edge noise compared to before. The difference was hardly noticeable at higher resolutions (512x384 and above) though.

[Koepi]

Uh. I didn't realize there were changes to the 1pass code - can you please do a "propper" 2pass to test the new code? :)

Thanks for your hep,

regards,
Koepi

[soujir0u]

Funny... 1-pass mode seems to have improved either intentionally or not. :D

[alanshum]

Is this build 'SMP enabled' just like the previous one?

Alan.

[Koepi]

Nope. SMP has a bad speed impact on uni-processor machines so I compile it as usual release without SMP support.

[ookzDVD]

@Koepi,

sorry.. it's stupid question...
1. how to disable the B-frame support ?
2. could I use Nic's DS filter to play the result ?

thanks.

[Swede]

1. Disable B_Frames by setting "max b-frames" to -1
2. Without B-frames it should be ok.

[Koepi]

I could release a new binary without bframes and SMP activated, so there should be less confusion.
Foxer tweaked the curve treatment a little, which from my view solves the matter of consecutive keyframes a little worse, but still produces nice output (well, now I have some frames quantized with 9+10, the old code left me alone with max 8 ;) )

If you prefer this, just drop a note.

Regards,
Koepi

P.S.: I can't remember who needed the xvid-core without any options compiled in, if someone could point me to his nick again? :)

[Nic]

(I should be releasing a new version of my filter soon without the green bug, that was easy to fix. The crashing of explorer is not easy, ive tracked it down to the colorspace code, but I cant work out why its getting corrupted, Id almost think it was a threading issue normally....)

-Nic

[Koepi]

Nic,

there definatly is some error in the mmx colour space conversion code, but I'm unable to identify it... look into the xvid forums, there someone reported it and tracked it down a little.

Regards,
Koepi

[Nic]

Thanks Koepi,

Weird thing is it happens to me if I use the C implementation too. dst2 in the yv12 to RGB function becomes a <Bad Ptr> & alot of the local variables in the function get corrupted too...

...Ill go look into the thread at the xvid.org forum now,

Cheers,
-Nic

[Koepi]

Nic,

GomGom has uploaded a patch for the problem i was referrencing to, can you test if this solves your issues?

(Well, I guess the problem must be more eval as this happens not only for multiple threads).

Regards,
Koepi

[Shayne]

@ Koepi

Hi your XviD Options Explained V1.3 PDF, link appears to be dead. I would like to read your new version.

Thank you for all your great input

@ Nic

Good to have you back with the free living

[Koepi]

Shayne,

due to massive direct linking I had to compile in referer-checking into the web server.
Try another browser, or configure it that it sends the correct referer, after that the download will succeed.

Regards,
Koepi

[Shayne]

Thanks but no thanks Redundant software is not really my thing

BP ftp return code

linked directly from ur site

URL detected in clipboard.
Connecting to roeder.goe.net port 80
Requesting http://roeder.goe.net/~koepi/XviD_Options_Explained.pdf
HTTP/1.1 403 Forbidden
Date: Tue, 23 Jul 2002 01:29:46 GMT
Server: Apache/1.3.26 (Unix) PHP/4.2.0 mod_gzip/1.3.19.1a mod_perl/1.26
Last-Modified: Sat, 13 Jul 2002 11:55:37 GMT
ETag: "122ffb-720-3d301539"
Accept-Ranges: bytes
Content-Length: 1824
Connection: close
Content-Type: text/html

Error retrieving file.
Data Connection closed

[Koepi]

Erm, boy, this is rude.

I told you that you need the right referer.

If you use a download-manager, it is your fault if it does NOT send the right referer.

No need to post your error log, you might want to actually READ the page your dl manager gets:

http://www.roeder.goe.net/403-Access_forbidden.html

Maybe you take a look at that and think again.

Regards,
Koepi

[Gazza]

In koepi's previous build (xvid-19072002-1.exe) there was reports of crashing when 'fast recompress' was checked. It looks as if this same bug is still around with the latest build (xvid-20072002-1.exe) as virtualdub crashes on my machine after the first 2 to 10 minutes? I tried both 2 pass and single (CBR) passes but same problem. I also loaded all the defaults but no good. I had b-frames set to 3/200% in all cases.

I haven't tried without using 'fast recompress' yet but will give it a go soon.

Gazza

[Edit] I tried with 'Full processing' checked but encode crashed after 10 minutes on the first pass. Is it a bug or am I doing something wrong????

[MaTTeR]

I've tried that build with and without B Frames enabled using Fast Recompress without issue. Actually, I've encoded roughly 7 movies within 48hrs without a crash.

WinXP
512MB
Dual XP 1600s (OCd @ 139FSB)
Vdub 1.4.10

If it's a bug it doesnt appear to affect SMP machines which is usually quiet the opposite:D

[manono]

Hi-

Is it a bug or am I doing something wrong????

It sounds as if it may be an .avs problem. Have you tried to load it into VDub and just scroll around to see if you can make it crash? And there's at least a chance it's a sub problem if you are using them. But just a guess.

[BiaTch 5.0]

i got blue arifacts on some frames.

BTW i used DIVX FourCC code (for mp4)maybe that's why?

[Gazza]

Originally posted by manono
Hi-

Is it a bug or am I doing something wrong????

It sounds as if it may be an .avs problem. Have you tried to load it into VDub and just scroll around to see if you can make it crash? And there's at least a chance it's a sub problem if you are using them. But just a guess.

I suspect you are right but I'm not sure how to prove it. I have loaded the avs and scrolled around but it seems OK. I used Gknot to generate the avs - maybe that has something to do with it...

If by sub you mean vobsub, I haven't used them.

I tried another encoding tool (not Virtualdub), it got through 50% of the first pass before crashing.

Gazza

[iago]

Hi everybody,

Giving another try; this time with Fight Club (139 min), 576x240, soft bicubic, with temporal smoother (2,1), aimed for 1 cd = 623MB video size :)

Though I know that this is a dark and perfectly compressible movie and I already have a good looking rip done with sbc, the reason why I choose to apply soft bicubic resize and filtering with temporal smoother is just to see if I can completely get rid of the macroblocks still slightly visible at Chapter 1 - Fear Center (the very beginning of the movie) and at Chapter 5 - Remaining Men Together (esp. on the brown-yellow walls).

Btw for the first time I decided to use filtering in one of my rips :)

I'll post the settings I try and the results including the quantizers distribution when it's finished.

Thanks to Koepi and others for this great build and to everyone in the forum sharing their own experiences and comments.

@BiaTch 5.0
Maybe because of lumi masking, if you enabled it? I got the same problem once, and without lumi masking in the 2nd pass it was OK.

Best regards...

[BiaTch 5.0]

Maybe because of lumi masking, if you enabled it? I got the same problem once, and without lumi masking in the 2nd pass it was OK.

Thank you very much, i did use lumi masking 1st&2ndpass, that must be it(i'm enocoding From Hell! whhoo scary), any way i'm a novice at video encoding so i need all the help i can get it would be nice if you experts could make a sticky best settings for Xvid 1CD rips i saw one thread but there where lots of contradictions & no clear final answer, i understand that there is no magic answer but it would be nice to have a overall for most cases good setting.

Thread could have: recommended res,resize,Xvid compile,Xvid settings.

[iago]

@ BiaTch 5.0

Also, I guess you should have used lumi masking only for the second pass and if you are going for a 1cd rip. Try again by disabling it in both passes; or at least disable it in the first pass and use in the second pass.

Just an opinion... :)

Best regards.

[rui]

For the resolution, you guys should always do a comp check.
It never fails me, i usually use the same percentages indicated for divx5 in Doom9 dixv5 guide (maybe a little higher percentages for xvid).
Like i already read in this forums, the first step for a good looking encoding is to choose the correct resolution for the size you are aiming.

[Koepi]

BFrames are highly experimental, unfinished, buggy - and, most important, ugly. Please, set them to -1 (means: disabling them) for now as you sure won't get a nice encoding with using them.

The crashes you experience are well known.

Regards,
Koepi

[int 21h]

Originally posted by BiaTch 5.0

(i'm enocoding From Hell! whhoo scary)

Compresses very well.

[iago]

Hi again Koepi and everybody,

For testing purposes, I encoded Fight Club with Koepi's new build, as I mentioned in my earlier post, to see its effect on blockiness and to see if the undersize problem in my rips continue. (I used 576*xxx resolution, soft bicubic resize and temporal smoother with a low setting to check the effect on blockiness issue.) Here are the settings and the results:

Movie: 139 min
Res: 576*240
soft bicubic
temporal smoother (2,1)
Aimed video size: 623000 KB
No B-frames


1st pass:
Motion search: 6.Ultra High
Quantizer: H.263
No lumi masking, no hinted ME
Max I-frame interval: 300
Min I-frame interval: 6

2nd pass:
Motion search: 6.Ultra High
Quantizer: Modulated
No lumi masking, no hinted ME

Min-Max I-frame: 2-5
Min-Max P-frame: 2-12

I-frame boost: 20%

Alt CC:
Aggression: Low
High: 225
Low: 75
Strength: 50
Automatic bonus bias calc. enabled

End credits: quant.31

final size: 611258 KB -> approximately 12 MB undersized again.


The debugview analyzer results are as follows:

--------------------------------------------------------------
DebugView analyzer for XviD codec v0.7 by MoonWalker
e-mail : s_ilias@gmx.net


Quantizers Analisis
---------------------

Quantizers Used For Movie :
------------------------------
Quant 2 Used : 22695 Times, Percentage Used : 11.58%
Quant 3 Used : 156114 Times, Percentage Used : 79.64%
Quant 4 Used : 15854 Times, Percentage Used : 8.09%
Quant 5 Used : 981 Times, Percentage Used : 0.50%
Quant 6 Used : 307 Times, Percentage Used : 0.16%
Quant 7 Used : 57 Times, Percentage Used : 0.03%
Quant 8 Used : 6 Times, Percentage Used : 0.00%

Average Quantizer Used for Movie : 2.981

Quantizers Used For Credits :
--------------------------------
Quant 31 Used : 4187 Times.

MPEG Quantization Type Used 182996 timed, Percentage Used : 91.41%

H.263 Quantization Type Used 17205 timed, Percentage Used : 8.59%

Quantizers prevented from rising too steeply 103 times


Intra-Frame (Key-Frame) Quantizers
------------------------------------

Movie
-------
Quant 2 Used : 2407 Times, Percentage Used : 77.85%
Quant 4 Used : 67 Times, Percentage Used : 2.17%

Credits
---------
Quant 31 Used : 618 Times, Percentage Used : 19.99%

Inter-Frame (P-Frame) Quantizers
------------------------------------

Movie
-------
Quant 2 Used : 20288 Times, Percentage Used : 10.29%
Quant 3 Used : 156114 Times, Percentage Used : 79.20%
Quant 4 Used : 15787 Times, Percentage Used : 8.01%
Quant 5 Used : 981 Times, Percentage Used : 0.50%
Quant 6 Used : 307 Times, Percentage Used : 0.16%
Quant 7 Used : 57 Times, Percentage Used : 0.03%
Quant 8 Used : 6 Times, Percentage Used : 0.00%

Credits
---------
Quant 31 Used : 3569 Times, Percentage Used : 1.81%



Frame Analisis
----------------

Number Of Intra-Frames (Key-Frames) : 3092
Number Of Inter-Frames (P-Frames) : 197109
Total Number Of Frames : 200201

1.54% of the Movie is Intra-Frames (Key-Frames)
98.46% of the Movie is Inter-Frames (P-Frames)



Size Analysis
----------------

1-Pass Size : 880955357 Bytes or 860307 KBytes
Scaled Size : 633136007 Bytes or 618296 KBytes
Actual Size : 621007882 Bytes or 606453 KBytes
Compressibility : 70.49%
---------------------------------------------------------------

The overall quality of the rip is OK concerning especially blocks problem, though the image is a bit soft as expected.

Finally, what can be the reason for getting undersized files with the above settings, while the average quantizer used for the movie is 2.981, which shows that it's not a saturation-of-the-codec problem imo?

Should I increase the I-frame boost, or change the min-max I-frame intervals, or cap the quantizers with different settings, or does the problem lie in the Alt CC settings I use, etc? :) Where do I go wrong?

Any comments pls?

Thanks in advance. Best regards.

[Koepi]

The reason for the undersize problem is my "consecutive keyframe treatment".
I'll make a new build available where this doesn't happen and the codec hits spot-on kbyte again ;) (well, i told you it's experimental, right? ;) )
Still, the results are way better than they were before IMHO.

Thanks for testing!

Regards,
Koepi

[iago]

@Koepi

Thanks a lot for your reply.

I regained my self-confidence :)

Best regards.

[Koepi]

Ok, new binary available, without this workaround for consecutive keyframes but with correct file size ;)

Results in some high quantized frames after consecutive keyframes occur, but it's not too bad.

Regards,
Koepi

EDIT: sorry for the short offline time of my page, I had to upgrade the webserver to use php4.2.2 and forgot to compile in mod_userdir :rolleyes: well, now it's fixed and everything works as expected.

[BiaTch 5.0]

Ok, new binary available, without this workaround for consecutive keyframes

Any setting/way to help fix this?

[Koepi]

I'm currently trying to set min keyframe distance to 4, I'll report later if this works.

Theoratically this _should_ help it.

Regards,
koepi

[dread]

And it helped to me.
But I've got another, little problem.
With new build (2307), in the preferences of xvid I cannot set brightness, etc. everything disappered :)
Is that normal ?

[ookzDVD]

@dread,

sure it's normal ;)
the latest Koepi's build is not using the Nic's DS filter.

[rui]

I believe Koepi stop including Nic's ds filter because it couldn't handle b-frames well.

But since the latest build doesn't have b-frames enabled anymore, maybe Koepi could again start including Nic's ds filter (?).

For the meantime, use ffdshow filter.

[The People's Elbow]

I encoded 3 movies with the newest build using a minKF distance of 12... no undersize problem anymore and quality was fine, either! (Though it depends of the source, if minKF 12 hurts or not!)

greetz, Elbow!