Adaptive Quantization vs CQMs and other questions

[amgeex]

Hello all. I'm quite confortable using x264 and most of it's settings, but I was never quite at ease with xvid. Now, I understand that Adaptive Quantization was "backported" from x264 and that it should be the preferred method when encoding. Now, my question is if it is still alright to use CQMs nowadays and how correct is it to turn off AQ when using them (some do not work with it enabled).

Other questions:

Where can I find recent builds of xvid_encraw or it's source code?

Where can I find recent builds of xvid itself to use in combination with xvid_encraw?

Is there any development going on at all on xvid 1.3?

That is all! :)

P.S. The reason I'm using xvid now is that I got a netbook as a present today and want to encode some movies to watch on it. I already tried some x264 encoded ones (without cabac and deblocking), but wasn't satisfied at all with the quality, as the bitrate needs to be too low to be decoded properly (my netbook doesn't have a dxva capable video card) and thus the image is not that great. I thought maybe xvid could give me better results in this specific case.

[henryho_hk]

The best xvid_encraw is available here (http://members.optusnet.com.au/squid_80/xvid_encraw.zip). One of the places we can find latest xvid compile is here (http://www.xvidvideo.ru/xvid-video-codec/). Simply put a copy of xvidcore.dll in the same directory as xvid_encraw.exe and it will get used.

Since the new VAQ replaces the old lumimasking feature, it is triggered by the same "-lumimasking" option.

[SilaSurfer]

Be sure to use VAQ with H263 or a high bitrate matrix like EqmV3HR.

[amgeex]

Thanks a lot for the answers guys, I'll do some test encodes with those builds. I'm currently using VAQ+H263, I'll see how it goes!

[henryho_hk]

xvid_encraw is suitable for batch script coding because it exposes nearly all XviD options on the command line and it can give a stats file output even for CQ encodes. Also, we dun need to install the VFW stuffs for it to work.

I refer Squid_80's xvid_encraw as the best because he fixed bugs in the original version, added threaded input and improved the AVI output feature.

BTW, Jawor also offers his XviD compiles here (http://jawormat.republika.pl/xvid.html).

[amgeex]

Thanks for clearing that out henryho. I now have some more questions after encoding a few files, as I'm getting one massive undersizing and I already read the undersizing thread and with this particular file it hasn't helped matters much. I'm getting a 650 MB undersize. Right now I'm using these settings:

START /WAIT xvid_encraw.exe -i video.avs -type 2 -pass1 xvid.stats -max_bframes 0 -bitrate 1460 -max_key_interval 250 -vhqmode 4 -bvhq -qpel -gmc -qtype 0 -lumimasking -imin 1 -bmin 1 -pmin 1 -threads 4 -progress 32 -stats -o preliminary.m4v

START /WAIT xvid_encraw.exe -i video.avs -type 2 -pass2 xvid.stats -max_bframes 0 -bitrate 1460 -max_key_interval 250 -vhqmode 4 -bvhq -qpel -gmc -qtype 0 -lumimasking -imin 1 -bmin 1 -pmin 1 -threads 4 -progress 32 -stats -o target.m4v

Even setting -max_bframes to 0 didn't help much, so I guess the only thing left to do is increas the resolution, so more bits get used, right? And, does setting -max_bframes to 0 means there won't be any consecutive bframes, or that there won't be any bframes at all? I suppose its the later.

Also, I see a lot of people set -kboost to 100, but I do not know what it does exactly. I already did a search on it, but didn't find much useful information. Also, could anyone explain to me what -bquant_ratio and -bquant_offset do, and how they affect the final size? I see a lot of people set -bquant_offset to 1, but I don't understand why.

:thanks:

[henryho_hk]

1) "-gmc" is not particularly useful
2) "-imin", "-bmin" and "-pmin" should be 2 or 3 (or even 4 if u are using certain custom matrices such as SixOfNine).
3) If your video is properly denoised, "-qpel" saves bitrate.
4) By "-lumimasking", I suppose u are using an updated version of xvidcore.dll and hence it is actually VAQ. Again, it saves bitrate.
5) "-kboost" and "-max_bframes" are not really intended for fixing undersize situations.
6) Since u are getting undersized files with "-qtype 0" (H.263), download a high bitrate custom matrix (e.g. EQM V2, EQM V3HR, or even SixOfNine) and use the options "-qtype 1 -qmatrix xxxx.cqm". Do not use a low bitrate custom matrix with VAQ.
7) Besides custom matrices and bigger resolution (or less downsizing), u can also reduce the level of noise reduction in your AVS.

"-max_bframes 0" means no B-frames at all. "-bquant_ratio" and "-bquant_offset" are used to calculated the quantizer of B-frame based on that of the referenced frames (something like floor(Avg-Ref-Q * B-ratio + B-offset)). Default is (1.5, 1.0) and Teegedeck recommends (1.62,0). As long as your quantizers stay below 12, Teegedeck's recommendation means lower B-frame quantizers, and hence bigger size and better quality.

[amgeex]

I thought VAQ didn't play well with CQM's? And if it wouldn't be much trouble, could you explain why is gmc not very useful? And what does kboost do? I'll do an encode with a CQM to test. Thanks a lot!

[henryho_hk]

I think others know much more (http://www.gromkov.com/faq/conversion/xvid_options.html) than I do.

[Sharktooth]

GMC is so useless it was not "ported" to h.264. the reason is: B-Frames.
for your undersize, try using a high bitrate matrix (test it with VAQ first and if it produces corrupted frames, disable VAQ).
also, remember when you get undersizes you SATURATED the encoder. by forcing it to produce a bigger file, you wont gain any quality (using the same quantization matrix).

[henryho_hk]

Sharktooth's great "EQM AutoGK Sharpmatrix" and "EQM V3HR" matrices are reported to be working great with VAQ.

PS: "EQM V3UHR" and "EQM V3EHR" are terrific for archive or edit purposes.

[amgeex]

Thanks for the links and explanations, I'll try doing some more encodes. I know the encoder is saturated, but I'm trying to un-saturate it Xd, if that's possible. Henryho, thanks A LOT for that link, it cleared some things up. Also, turning off b-frames can't hurt qualilty, or can it? And if setting 0 b-frames, it doesn't make sense to keep bvhq on, right?

[Blue_MiSfit]

Also, turning off b-frames can't hurt qualilty, or can it? And if setting 0 b-frames, it doesn't make sense to keep bvhq on, right?


Of course turning off b-frames can hurt quality, unless you're targeting obnoxiously high bitrates.

If your laptop is too slow to decode H.264, then I'd suggest trying VC-1 as an intermediate. Microsoft Expression Encoder is free, and does a very nice job with VC-1 encoding.

VC-1 is a lot more light-weight than H.264 to decode, and typically does better than MPEG-4 ASP in compression. It's worth a try anyway!

Derek

[henryho_hk]

Indeed, amgeex, what's ur resolution and target bitrate?

[amgeex]

Right now I'm targetting 816x336 at 1460 kbps.

I'll give the expression encoder a go, but can it accept avisynth scripts as input?

[Sharktooth]

IMHO Microsoft's VC-1 is not better (compressionwise or qualitywise) than xvid with an adequate quantization matrix.

[henryho_hk]

Being a fans of Teegedeck presets, I support your view.

[henryho_hk]

Jawor's XviD 1.3 RC1 (http://jawormat.republika.pl/xvid.html) SSSE3 compile seem to produce corrupted output with H.263 VAQ.

[jethro]

IMHO Microsoft's VC-1 is not better (compressionwise or qualitywise) than xvid with an adequate quantization matrix.

Recently I tied Expression Encoder 4 VC-1 and it actually has balanced quality and pleasing picture. Xvid turned out to be slightly but noticeably worse.

[amgeex]

I'm actually gonna try EE4, now that I have installed it and see it can support avs input. I'll give it a whirl and see what happens. Thanks again y'all! =D

Actually, how can I get raw VC-1 output from EE4? Can it be done with the free version? Or am I stuck with WMV? And how can I get EE to encode and mux an audio track?

Oh, and how can I get that stats file from xvid_encraw?

[Jawor]

Jawor's XviD 1.3 RC1 (http://jawormat.republika.pl/xvid.html) SSSE3 compile seem to produce corrupted output with H.263 VAQ.

Xvid4PSP users noticed such problems with my xvidcore.dll and xvid_encraw from xvidvideo.ru, so I added my own xvid_encraw.exe to the most recent build. That seems to solve the VAQ issue.

[henryho_hk]

My compressibility test script needs parsing the stats file output (sample attached) of squid_80's xvid_encraw to get the size of each frame. Is there any way to get his version works with your XviD compile?

[Jawor]

Squid_80's last xvid_encraw is from 31.08.2007, isn't it? That's really old... There have been lots of changes in the CVS since then (VAQ, PSNR-HVS-M, slices, VBV peakrate etc.). No wonder it has problems with xvidcore.dll from 2011.

The only good solution would be to update Squid's version. Adapting (crippling?) the latest Xvid to work with an old application (if it's even possible) wouldn't make much sense ;)

[henryho_hk]

No, it is the CVS version being "ancient". See this (http://forum.doom9.org/showthread.php?p=1242214#post1242214).

As it is said, squid_80's 31.08.2007 version did, e.g.
1) fix >2GB AVI output
2) add threaded input
3) add detailed per-frame stats file output
4) add the option to enable/disable chroma optimizer
5) fix fps timing
6) add plugh's experimental HVS

All these are very useful, and the CVS version seems to be lacking them.

[Jawor]

All these are very useful, and the CVS version seems to be lacking them.
Yes, they are useful... but Squid's xvid_encraw simply needs some adjustments to work with 1.3.0 RC1 and I don't know what should be adjusted.

Even vanilla builds of 1.3.0 RC1 (both MSVC 2008 and MingW32-GCC, default CFLAGS) produce corrupted or low quality output with Squid's xvid_encraw, so it's got nothing to do with compile-time optimizations or my patches.

Well, there's always 1.2.2 to fall back on...

[amgeex]

The following makes your build crash instantaneously Jawor:

START /WAIT xvid_encraw.exe -i video.avs -type 2 -pass1 xvid.stats -bitrate 1583 -max_key_interval 240 -vhqmode 4 -bvhq -qpel -masking 2 -nopacked -threads 3 -progress 32 -stats -o preliminary.m4v

START /WAIT xvid_encraw.exe -i video.avs -type 2 -pass2 xvid.stats -bitrate 1583 -max_key_interval 240 -vhqmode 4 -bvhq -qpel -masking 2 -nopacked -threads 3 -progress 32 -stats -o target.m4v

I tried using the sse2, sse3 and ssse3 xvidcore versions, but all with the same result. I'm on a AMD Phenom X3 8750. :confused:

And I have another question: Which frames would benefit from having a lower quantizer? I, P or B?

[Jawor]

The following makes your build crash instantaneously Jawor:

START /WAIT xvid_encraw.exe -i video.avs -type 2 -pass1 xvid.stats -bitrate 1583 -max_key_interval 240 -vhqmode 4 -bvhq -qpel -masking 2 -nopacked -threads 3 -progress 32 -stats -o preliminary.m4v

START /WAIT xvid_encraw.exe -i video.avs -type 2 -pass2 xvid.stats -bitrate 1583 -max_key_interval 240 -vhqmode 4 -bvhq -qpel -masking 2 -nopacked -threads 3 -progress 32 -stats -o target.m4v

I tried using the sse2, sse3 and ssse3 xvidcore versions, but all with the same result. I'm on a AMD Phenom X3 8750. :confused:
Does the same happen with the “SSE” build (named xvidcore.dll in the 7-Zip file)?

Is your AviSynth 32-bit or 64-bit? A 32-bit AviSynth is required to interact with 32-bit applications (it's a Windows limitation). A few hours ago I uploaded a 64-bit build of encraw and xvidcore.dll, try those if you use 64-bit AviSynth.

And I have another question: Which frames would benefit from having a lower quantizer? I, P or B?

B-frames can handle higher quantizers a little better than P-frames and I-frames. Usuallly it's beneficial to give some more bits to I-frames (hence the “I-frame boost” option) to lower their quantizers (the idea is that a better-looking I-frame will cause the next P-frame to look better and that will help the next P/B-frame and so on...).

[amgeex]

Does the same happen with the “SSE” build (named xvidcore.dll in the 7-Zip file)?

Is your AviSynth 32-bit or 64-bit? A 32-bit AviSynth is required to interact with 32-bit applications (it's a Windows limitation). A few hours ago I uploaded a 64-bit build of encraw and xvidcore.dll, try those if you use 64-bit AviSynth.Yes, it happens with that version too. I'm currently using 32-bit avisynth, although it's an alpha version (2.60). It works ok with squid's xvid_encraw and xvidcore from the xvidvideo.ru, but maybe that's the problem? Thanks for answering the question about the frames!

[Jawor]

Yes, it happens with that version too. I'm currently using 32-bit avisynth, although it's an alpha version (2.60). It works ok with squid's xvid_encraw and xvidcore from the xvidvideo.ru, but maybe that's the problem?
Maybe... Although maybe it's just GCC 4.x.x acting up ;) Just when I thought this compiler made real progress...

I just whipped up an MSVC 2008 build (32-bit and 64-bit). Try it.
http://jawormat.republika.pl/msvc.7z

If it works, probably I'll just dump GCC... at least the 4.x.x version.

[henryho_hk]

but Squid's xvid_encraw simply needs some adjustments to work with 1.3.0 RC1 and I don't know what should be adjusted.


I see. Did you simply recompile xvid_encraw to make it work with your xvidcore.dll compiles?

[Jawor]

I see. Did you simply recompile xvid_encraw to make it work with your xvidcore.dll compiles?

Basically, yes. I used the xvid_encraw source code that came with 1.3.0 RC1 and applied 3 patches (supplied by the Xvid4PSP Team). But these patches are only simple fixes for the exit code, width/height values in the AVI header and FPS auto-detection, so they make no difference when it comes to corrupted VAQ output (builds from xvidvideo.ru also use these patches).

[amgeex]

Maybe... Although maybe it's just GCC 4.x.x acting up Just when I thought this compiler made real progress...

I just whipped up an MSVC 2008 build (32-bit and 64-bit). Try it.Jawor, I'm sorry to report that the MSVC build crashes instantaneously, just as the GCC builds... Maybe someone else (or you? Oo) could verify if it's avisynth 2.60 that's breaking these builds?

[Jawor]

And what happens if you use an *.avi file as input?

[henryho_hk]

VHQ4 + QPel has been crashing since XviD 1.2

[Jawor]

VHQ4 + QPel has been crashing since XviD 1.2

In all builds of 1.2.2?

Didée reported this issue with my 1.3.0 RC1 from 02.01.2011, but it's fixed in 1.3.0 RC1 from 08.01.2011... or so it would seem.

[amgeex]

And what happens if you use an *.avi file as input?I don't have an .avi to try that right now (believe it or not... that container sucks), but I will get one.

VHQ4 + QPel has been crashing since XviD 1.2In all builds of 1.2.2?

Didée reported this issue with my 1.3.0 RC1 from 02.01.2011, but it's fixed in 1.3.0 RC1 from 08.01.2011... or so it would seem.I can use VHQ4 + QPel without problems with Squid's xvid_encraw plus the xvidcore.dll from the xvidvideo.ru cvs builds.

[henryho_hk]

In all builds of 1.2.2?


In some builds.... and I'd say late 1.2..... something like 1.2.127

[Jawor]

I don't have an .avi to try that right now (believe it or not... that container sucks), but I will get one.


That would tell us if it's AviSynth's fault.

If this unpatched build (http://jawormat.republika.pl/xvidcore.7z) of xvidcore.dll won't work, the problem is probably somewhere in the CVS source code.

In some builds.... and I'd say late 1.2..... something like 1.2.127
I've had no issues with encoding using Qpel + VHQ=4 in 1.2.x versions... which means it's another strange bug (not necessarily in Xvid) that I can't reproduce.

[jmac698]

Hi,
I've always used autogk but it fails sometimes with sync issues. I'm afraid to switch now because it has this VAQ patch which I thought would improve quality. I also always use mpg matrix because the default looks horrible. Finally I force 4:3 and use mpg4modifier to patch to 16:9 for anamorphic.
However I'd like to find an alternative. Is there some easy way that's gonna ivtc, with good sound sync, with vaq, and a matrix that doesn't suck? specifically not the xvid default, and that's easy to use and has a queue? Just wondering how you guys do it.

The problem is, there's so many builds, and advanced options with players not always working, and different bugs, it's too confusing for me.

ps I always use the max 2GB filesize and ac3. I've tried the default matrix at the highest quality and it just looks horrible. What is wrong with it?

[henryho_hk]

Yes, what's wrong with the default (H.263 is not a matrix actually)? In what way is it horrible to you at a file size of 2GB? How long is your movie?

[jmac698]

It's banded. Is this a bug? I'll make a sample.