MEncoder Xvid 2-pass generates undersized file

[Vict0r Meldrew]

This is a question about undersized video files and a discrepancy between the file size produced by Mencoder Xvid 2-pass versus Xvid VfW 2-pass. I'm trying to make a 2CD .avi backup of a film called Walk The Line, desired video filesize 1260Mb.

I'm encoding directly from the original VOB file, using the same bitrate, crop and resize parameters for both Mencoder and XviD VfW (Koepi's build 1.1.3). For Mencoder, I have put together a small batch file which takes care of both passes - see below:

@echo off

REM Change the variable values below.
REM crop[=w:h:x:y]
set volpath=C:\WTL
set infile=VTS_04_PGC_01_1.VOB
set outfile=Walk-The-Line-xvid-vidonly.avi
set crop=0:424:0:76
set res=688:288
set bitrate=1351

echo Encoding "%volpath%\%infile%"...(first pass started at %time%).
START /W /MIN /LOW C:\PROGRA~1\MPlayer-1.0rc2\mencoder.exe "%volpath%\%infile%" -passlogfile "%volpath%\mencoder.2passlog" -vf crop=%crop%,scale=%res% -nosound -ovc xvid -xvidencopts bitrate=%bitrate%:me_quality=6:nointerlacing:max_bframes=1:noqpel:trellis:quant_type=mpeg:chroma_opt:vhq=4:pass=1 -ffourcc XVID -o "%volpath%\%outfile%"
echo First pass finished at %time%
echo Encoding "%volpath%\%infile%"...(second pass started at %time%)
echo Second pass start time is %time%
START /W /MIN /LOW C:\PROGRA~1\MPlayer-1.0rc2\mencoder.exe "%volpath%\%infile%" -passlogfile "%volpath%\mencoder.2passlog" -vf crop=%crop%,scale=%res% -nosound -ovc xvid -xvidencopts bitrate=%bitrate%:me_quality=6:nointerlacing:max_bframes=1:noqpel:trellis:quant_type=mpeg:chroma_opt:vhq=4:pass=2 -ffourcc XVID -o "%volpath%\%outfile%"
echo Second pass finished at %time%
if exist "%volpath%\mencoder.2passlog" del /q "%volpath%\mencoder.2passlog"
pause


For Xvid VfW, I am opening the VOB file in VirtualDub 1.8.4 with help from fccHandler's MPEG-2 plugin, and using the native crop/resize filters present in VirtualDub.

When I encode the video with mencoder, and specify the bitrate as 1351, I always end up with an undersized file (1020Mb). However, when I encode using Xvid VfW and 2-pass "Home" profile, specifying the same bitrate value, the desired filesize is correctly achieved (1260Mb). I have tried using two different mencoder builds - MEncoder 1.0rc2-4.2.1 and MEncoder Sherpya-SVN-r27323-4.2.4 (supplied with Automen), but still an undersized file is generated.

I have also tried feeding mencoder an AVS file (frameserved via DGDecode), but the file is still undersized.

Is it likely that mencoder is maxed out on bitrate? ..and that the way Xvid is implemented within mencoder differs from VfW - which would explain the file size differential? Or could it be that the mencoder mpeg2 decoder decodes the source differently (which doesn't go very far to explain why mencoder undersized the AVS script). Maybe my mencoder "-xvidencopts"'s need adjustment?

The quality of both files appears indiscernible, at least to my eyes. I've ready the sticky on Oversized/Undersized explanations, but I'm not sure if it applies here..

Thanks in advance.

[kinematic]

First thing I would try is no resizing. Just crop to a multiple of 16 and try eqm_v3lr matrix.

[Vict0r Meldrew]

@kinematic: Thank you for you reply. Are you suggesting that, with mencoder, by resizing I'm limiting the bitrate applied to the video, hence a smaller output file? In short, I'm asking why the bitrate isn't being honoured in MEncoder's XviD implementation, yet it is honoured in XviD VfW. I'd like to avoid using a CQM, in order to retain standalone compatibility with my Philips DVD/DivX player.

[kinematic]

I don't know why XviD VfW delivers the target filesize because I've never used it. I do know that if mencoder delivers a smaller file size than you expected it's maxed out on quality for that given file size wich is likely due to the fact that you downsized the resolution (lower resolution means smaller frames so you reach a given target quality faster). If the quality seems exactly the same to you there's no reason the encode it again unless you absolutely want to reach the filesize you specified (I don't think in terms of filesize but in terms of quality because I don't care about filesize). If that's the case not resizing+ a cqm designed for anamorphic encoding will put the extra space to good use and you will probably end up with the file size you want (I'd go with eqm_v3lr.cqm and do a 10 min encode to test). You also don't have to worry about standalone compatability with Sharktooth's eqm_v3 matrices, I also have a Philips DVD/DivX player and it plays files encoded with those matrices just fine.

[Gusar]

One thing comes to mind: container frame overhead. Mencoder sets it to 24 by default. Add container_frame_overhead=0 to xvidencopts and report what happens.

[Vict0r Meldrew]

@kinematic:
I don't know why XviD VfW delivers the target filesize because I've never used it. I do know that if mencoder delivers a smaller file size than you expected it's maxed out on quality for that given file size wich is likely due to the fact that you downsized the resolution (lower resolution means smaller frames so you reach a given target quality faster).


..so if that's the case, why doesn't XviD VfW also max out on quality? - I resized the video there too using the same values, just like I did with MEncoder.

If the quality seems exactly the same to you there's no reason the encode it again unless you absolutely want to reach the filesize you specified (I don't think in terms of filesize but in terms of quality because I don't care about filesize).


I agree with what you're saying. Really, I just want to understand why MEncoder's doing what it's doing.

@Gusar: Thank you for your reply. I've tried adding container_frame_overhead=0 to -xvidencopts, but it really hasn't made much of a difference in the size of the encoded file. The file's 362KBs less than the previous MEncoder file, going from 1044733KB to 1044371KB. The differential between the VfW file and MEncoder file is 240Mb, so it strikes me that there must be significant option that's causing this.

One thing I forgot to mention is that the first time I encoded the file using MEncoder, I used LAVC instead of XviD, the filesize was even less (959Mb) even when the bitrate specified was 1351 (as specified with XviD). I don't know if this provides a clue.

I might try the xvid dxnhtpal profile in MEconder and see what file size results.

[kinematic]

Are the min quantizers set to 1 in XviD VfW? That might explain why it reaches the file size you specified (big fat Q1 frames). And just as a test who don't you do a short encode with no resizing and Sharktooth's eqm_v3lr matrix and see what happens.

[Vict0r Meldrew]

Indeed the minimum quantizers are set to 1. This is the default for the "General purpose" quality preset in XviD VfW.

In MEncoder, the default for minimum quantizer (vqmin) is 2. According to the documentation, 1 is "Not recommended (much larger file, little quality difference and weird side effects)"

This probably explains the undersizing. Thank you kinematic!:thanks:

[kinematic]

It explains the oversizing :p Mencoder is doing what it's supposed to do with the settings you gave it :)

[Vict0r Meldrew]

Just to confirm that min_iquant/min_pquant/min_bquant were the offending MEncoder XviD options (not vqmin as this relates to LAVC codec). When these were set to 1, the expected file size is achieved. Thanks all.

[kinematic]

Ofcourse it's your choice to do it this way but it's really not the way to achieve what you want. You're not doing anything usefull with the extra space and you're certainly not getting a higher quality file, just a fatter file. Try to understand the difference.

[Vict0r Meldrew]

I do understand the difference and my mencoder min quantizers are set to 2 (i.e. the default). My tests using min quantizer 1 were simply to prove one way or the other why MEncoder XviD was creating a smaller file in comparison with XviD VfW.

On a related point, presumably the XviD VfW default min quantizers default is 1 for a reason?

[kinematic]

I understand :) but I can't tell why they're set to 1 in XviD VfW because I've never used it and since it's a Windows app I'm never going to use it.

Have you also done an encoding at full res using one of Sharktooth's cqm's like I suggested?

[Vict0r Meldrew]

No I haven't tried Sharktooth's cqm matrix yet, but I will try it out some time soon. Thanks for the tip. If I'm trying to achieve maximal quality/fidelity from my DVD backups, without regard for file size, is this the way to go?

[kinematic]

Well, no resizing (also called anamorphic encoding) is the only way to go if you want to achieve maximum quality (you want to retain as much information as possible wich can only be done by not resizing). The reason I recommended Sharktooth's cqm's is because they're top notch and designed for anamorphic encoding and deliver a sharp and crisp picture. Alternatively you could try XviD with the VAQ patch (you have to enable AQ aka lumi masking) using h263 as the matrix wich according to some on this forum is now the "all purpose" cqm.