Encodes always too small!

[theboywonderuk]

Hi,

I've recently started using GKnot. Brilliant program.

However, when encoding (divx 5 2 pass) my movies always end up ~675-685mb rather than 695-700mb.

I select my source mp3 audio file correctly (though it never muxes but I can do this after no problem).

Any reasons why this might be happening?

Thankyou,
Robin

[theboywonderuk]

Ok this time I unticked recalculate bitrate and it muxed the audio...... but ended up with a file size of 612mb !!! :(

I don't know if my GKnot.log file will be of use but here it is just incase:

18/01/2003 20:55:19: Job "Movie 1" started.

o DivX5-First Pass: D:\se7en\cd1\cd1.avs
o DivX5-Second Pass: D:\se7en\cd1\cd1.avs
o Mux Audio.
End of Job 1 (Movie 1).
_______________________________________________________________________________________________________________________

Gordian Knot 0.27.0.13
Encoding Job Data:

Type: EncD5_Both
Number: 1
Name: Movie 1
Platform: Win2000 (5.0.2195).2

Files:
VirtualDubExe: C:\Program Files\GordianKnot\VirtualDub\VirtualDub.exe
NandubExe: C:\Program Files\GordianKnot\Nandub\Nandub.exe
AviFinal: D:\se7en\cd1\cd1.avi
AviMovie: D:\se7en\cd1\cd1_Movie.avi
AviCredits: D:\se7en\cd1\cd1_Credits.avi
MovieFS: D:\se7en\cd1\cd1.avs
CreditsFS:
Log : D:\se7en\cd1\cd1.log
Stats: D:\se7en\cd1\cd1.stats
Ecf : D:\se7en\cd1\cd1.ecf

Options:
Quality/DRF: 0
CreditsStart: 0
UseEcf: 1
CreditsIVTC: 0
CreditsAppend: 1
DeleteInterm.: 0
EnforceBitrate:1
AntiShit: 0

Calculator:
VideoSize: 567335 kbyte.
Audio1Size: 138748 kbyte.
Audio2Size: 0 kbyte.
Overhead: 5596 kbyte.
AviSize: 711680 kbyte (695 mbyte).
FilesSize: 0 kbyte.
TotalSize: 711680 kbyte (695 mbyte).
Interleaving: vbrmp3
Bitrate: 1309 k(=1000)Bits/s
Frames: 88800
FPS: 25.000
Duration: 0:59:12
Quality: 0.256 bits/(pixel*frame).
CompCheck3: 0.000 bits/(pixel*frame).
CompCheck5: 0.000 bits/(pixel*frame).
DivX5BiDirect: 1
DivX5GMC: 1
DivX5QuartPel: 0
DivX5PsyVis: 0
Resolution: 672 x 304

Audio 1:
mode: Just Mux Source File.
sourceFile: D:\se7en\cd1\cd1.mp3
wavFile: D:\se7en\cd1\cd1.wav
mp3File: D:\se7en\cd1\cd1_1.mp3
azid: ""
lame: ""
delay: 0
interval: 2
preload: 500
mux: 1
recalc: 1
deleteWav: 1

Audio 2:
none.

20:55:19: Audio 1, determined: 138748 kb
20:55:19: Audio 1, calc: 138748 kb
20:55:19: NOT recalculating video-bitrate, audio size ignored, bitrate enforced! (You wanted that...)

Adding this script to C:\Program Files\GordianKnot\VirtualDub\VirtualDub.jobs:

// $job "DivX4 First Pass"
// $input "D:\se7en\cd1\cd1.avs"
// $output "D:\se7en\cd1\cd1_Movie.avi"
// $state 0
// $start_time 0 0
// $end_time 0 0
// $script

VirtualDub.Open("D:\\se7en\\cd1\\cd1.avs",0,0);
VirtualDub.audio.SetSource(0);
VirtualDub.audio.SetMode(0);
VirtualDub.audio.SetInterleave(1,0,0,0,0);
VirtualDub.audio.SetClipMode(1,1);
VirtualDub.audio.SetConversion(0,0,0,0,0);
VirtualDub.audio.SetVolume();
VirtualDub.audio.SetCompression();
VirtualDub.video.SetDepth(24,24);
VirtualDub.video.SetMode(1);
VirtualDub.video.SetFrameRate(0,1);
VirtualDub.video.SetIVTC(0,0,-1,0);
VirtualDub.video.SetRange(0,0);
VirtualDub.video.SetCompression(0x78766964,0,10000,0);
VirtualDub.video.SetCompData(334,"GAEAAEj5EwDQBwAACgAAABQAAAAsAQAADAAAAAIAAAAFAAAAAgAAAJABAAAAAAAAAAAAAB4AAABkAAAAAAAAAAAAAAABAAAAAAAAAAAAAAAAAAAAAAAAAAAgAAAyAAAAAAAAAAAQAACamZmZmZnJP5qZmZmZmck/QAAAAAEAAACAAAAAAAAAAAAAAAAAAAAAAAAAAIACAADgAQAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAOA/AAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAABAAAAAQAAAAEAAAABAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAABAAAAAAAAAAEAAAD/////BAAAAEQ6XHNlN2VuXGNkMVxjZDEubG9nAGM6XHRlc3QuZGl2eABEOlxzZTdlblxjZDFcY2QxLm12AG0=");
VirtualDub.video.filters.Clear();
VirtualDub.subset.Delete();
VirtualDub.SaveAVI("D:\\se7en\\cd1\\cd1_Movie.avi");
VirtualDub.Close();

// $endjob
//
//--------------------------------------------------
// $done

20:55:19: Started DivX5-First Pass: D:\se7en\cd1\cd1.avs
22:01:41: Finished DivX5-First Pass. Duration: 1 hour, 6 minutes, 22 seconds.
22:01:41: Trying to open Log-file.
22:01:43: Success: Log-file open.
22:01:43: Encoded: 88799 Frames.
22:01:43: Speed: 22.297 Frames per Second.

Adding this script to C:\Program Files\GordianKnot\VirtualDub\VirtualDub.jobs:

// $job "DivX4 Second Pass"
// $input "D:\se7en\cd1\cd1.avs"
// $output "D:\se7en\cd1\cd1_Movie.avi"
// $state 0
// $start_time 0 0
// $end_time 0 0
// $script

VirtualDub.Open("D:\\se7en\\cd1\\cd1.avs",0,0);
VirtualDub.audio.SetSource(0);
VirtualDub.audio.SetMode(0);
VirtualDub.audio.SetInterleave(1,0,0,0,0);
VirtualDub.audio.SetClipMode(1,1);
VirtualDub.audio.SetConversion(0,0,0,0,0);
VirtualDub.audio.SetVolume();
VirtualDub.audio.SetCompression();
VirtualDub.video.SetDepth(24,24);
VirtualDub.video.SetMode(1);
VirtualDub.video.SetFrameRate(0,1);
VirtualDub.video.SetIVTC(0,0,-1,0);
VirtualDub.video.SetRange(0,0);
VirtualDub.video.SetCompression(0x78766964,0,10000,0);
VirtualDub.video.SetCompData(334,"GAEAAEj5EwDQBwAACgAAABQAAAAsAQAADAAAAAIAAAAFAAAAAwAAAJABAAAAAAAAAAAAAB4AAABkAAAAAAAAAAAAAAABAAAAAAAAAAAAAAAAAAAAAAAAAAAgAAAyAAAAAAAAAAAQAACamZmZmZnJP5qZmZmZmck/QAAAAAEAAACAAAAAAAAAAAAAAAAAAAAAAAAAAIACAADgAQAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAOA/AAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAABAAAAAQAAAAEAAAABAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAABAAAAAAAAAAEAAAD/////BAAAAEQ6XHNlN2VuXGNkMVxjZDEubG9nAGM6XHRlc3QuZGl2eABEOlxzZTdlblxjZDFcY2QxLm12AG0=");
VirtualDub.video.filters.Clear();
VirtualDub.subset.Delete();
VirtualDub.SaveAVI("D:\\se7en\\cd1\\cd1_Movie.avi");
VirtualDub.Close();

// $endjob
//
//--------------------------------------------------
// $done

22:01:43: Started DivX5-Second Pass: D:\se7en\cd1\cd1.avs
22:51:45: Finished DivX5-Second Pass. Duration: 50 minutes, 1 second.
22:51:45: Speed: 29.580 Frames per Second.

Adding this script to C:\Program Files\GordianKnot\Nandub\VirtualDub.jobs:

// $job "DivX3 Append"
// $input "D:\se7en\cd1\cd1_Movie.avi"
// $output "D:\se7en\cd1\cd1.avi"
// $state 0
// $start_time 0 0
// $end_time 0 0
// $script

VirtualDub.Open("D:\\se7en\\cd1\\cd1_Movie.avi",0,0);
VirtualDub.audio.SetSource("D:\\se7en\\cd1\\cd1.mp3");
VirtualDub.audio.SetMode(0);
VirtualDub.audio.SetInterleave(1,500,2,0,0);
VirtualDub.audio.SetClipMode(1,1);
VirtualDub.audio.SetConversion(0,0,0,0,0);
VirtualDub.audio.SetVolume();
VirtualDub.audio.SetCompression();
VirtualDub.audio2.SetSource(0);
VirtualDub.audio2.SetMode(0);
VirtualDub.audio2.SetInterleave(1,0,0,0,0);
VirtualDub.audio2.SetClipMode(1,1);
VirtualDub.audio2.SetConversion(0,0,0,0,0);
VirtualDub.audio2.SetVolume();
VirtualDub.audio2.SetCompression();
VirtualDub.video.SetDepth(24,24);
VirtualDub.video.SetMode(0);
VirtualDub.video.SetFrameRate(0,1);
VirtualDub.video.SetIVTC(0,0,-1,0);
VirtualDub.video.SetRange(0,0);
VirtualDub.video.SetCompression();
VirtualDub.video.filters.Clear();
VirtualDub.subset.Delete();
VirtualDub.SaveAVI("D:\\se7en\\cd1\\cd1.avi");
VirtualDub.Close();

// $endjob
//
//--------------------------------------------------
// $done

22:51:45: Started Appending Credits and Muxing Audio.: D:\se7en\cd1\cd1_Movie.avi
22:52:47: Finished Appending Credits and Muxing Audio.. Duration: 1 minute, 2 seconds.

22:52:47: Done.
22:52:47: Movie = D:\se7en\cd1\cd1.avi
Total Encoding Time: 1 hour, 57 minutes, 28 seconds.
18/01/2003 22:52:47: Job "Movie 1" finished.
_______________________________________________________________________________________________________________________

[N_F]

Originally posted by theboywonderuk
However, when encoding (divx 5 2 pass) my movies always end up ~675-685mb rather than 695-700mb.
Always as in for 1-2 movies or 20?

Are you running compression tests? What values are you settling for in that case?

[theboywonderuk]

Tried:

In the Mood For Love - 3 times
Hannibal
Se7en

All have been too small!

I'm not using compressibility test although I did use it once on 'In the Mood For Love' and that didn't help either - it was set to the default value "Use 5%".

:confused:

[N_F]

After running a comp. test you should get a percentage value returned which is likely to be between 40-80% depending on what resolution you use, bitrate, etc...

Recommended values here are ~60-75%. If you go higher than 75% you're likely to get undersized movies.


Edit: Removed a phrase that could easily be misunderstood.

[theboywonderuk]

Ok, thanx, I'll give it a go.

I'll post back with the results!

[theboywonderuk]

Hmmm. Well I did the compressibility test with 5% of the movie (I'm guessing that that 5% stands for)...

The result I got in GKnot at the bottom of the resolution tab was:

115.9 % of 0.221 CompressibilityTest

What does this mean? If the test returns the result, how am I supposed to change this / ensure I don't get undersized movies?!

[TheWEF]

read this (http://forum.doom9.org/showthread.php?s=&threadid=24584)

[manono]

Hi-

That could explain why you're undersized. 115% means that you can't reach your desired file size for the given resolution, filter combination, audio, etc. The obvious answer is to increase the resolution and run the compress test again. Or maybe you're trying to put a movie on 2 CDs that can easily go on 1. As N_F says, a good percentage to shoot for would be 60-75%. Another thing you can try to boost the file size (and the quality) is to leave off the B-Frames.

For example, to use a movie you did recently, I did In The Mood For Love last week. At 640x368 resolution, it came out to 699 MB, and a quality percentage of 76%. It compresses very well.

Also, you might find Acaila's Sticky on Undersized Files (http://forum.doom9.org/showthread.php?s=&threadid=24584) helpful.

Edit: TheWEF beat me to the link. :)

[jggimi]

You are getting undersized files because you are asking for a bitrate that is 112% of the maximum possible, from a test of 5% of your complete video stream (14 frames, every 280 frames).

You want your encoding to be between 60-75% loading factor. If the factor is too large (as it is in your case), you have a number of choices; here are just a few:[list=1] Increase resolution
Increase audio bitrate
Use a sharper resizing method[/list=1] If you were below 60%, you also have lots of choices, such as:[list=1] Decrease resolution
Decrease audio bitrate
Increase output size (e.g.: 2-CDs instead of 1-CD)
Use a less sharp resizer
Use temporal smoothers (noise reduction)[/list=1] For more information on compressibility checks, see this Sticky thread from the DivX 5 forum: Undersized Files. Why do I get them, and what do I do about it? (http://forum.doom9.org/showthread.php?s=&threadid=24584)

[TheWEF]

lol :)

[theboywonderuk]

Hey guys,

Wow all replied within a few mins of each other! Wierd!

Ok now I kinda understand. However, a few new questions.

Se7en is just over 2 hours. Now, I wasn't overly chuffed with an encode of Pretty Woman @ just under 2 hours on 1 cd....
so I wanted to make a 2 disc set for Se7en: 1 hour per disc - which is what I tried earlier.

Even with 320kbps audio and a resolution of 672 x 304 it gives a 115% compressibility test result. I can't increase anything else! (Increasing res to 704x320 only lowers the result to 105% and I get a W-Zoom error of 102%).

Does this mean that I have to put more than 1 hour onto a cd if I'm to get the whole 695/699mb available? I'm guessing it does...! and if you want ever want to make a 2cd set and use up the whole cdrom the (average action / colour / brightness etc) movie would have to be about 2.5-3 hours?

By the way Manono how many discs was your rip of In The Mood For Love? What did you think of it by the way. I thought it was quite clever! The director is supposed to be quite good! Happy Times (http://www.dvdboxoffice.com/description.asp?id=BC4F974C-61BD-40A0-9543-B3983C30D3A1&Class=mvi&wh=MAIN&s=gg&rate=GBP++%2D2%2E460+++++&pid=90027297)

[jggimi]

You are misleading yourself by thinking of "time period per cd." A 2-cd encode will end up being 1.4GB (if your cd size is 700MB). After you split, you may find one CD has a longer amount of content than the other. A 2-pass encoding has a variable bitrate, which is why the value used is an average. By default, DivX 5 will attempt to apply that average across 2000 frames. If interested, there's a thread or two in the DivX 5 forum on "RC Averaging."

In addition, don't be fooled into assuming there is any length of content per CD rule. There isn't one. How much "fits" will depend on many things beyond length alone, such as aspect ratio, motion, shading, and compressibility. This is one reason why compress tests are important if you're trying to hit a particular size.

Of course, acceptable quality is subjective. If you don't like the results at 1-CD, you could opt for 2, even if you can't fill two CDs completely.

[manono]

Hi-

Manono how many discs was your rip of In The Mood For Love?

I said it was 699 MB, so, 1 CD.

What did you think of it by the way. I thought it was quite clever! The director is supposed to be quite good! Happy Times

Happy Times has a different director. Maybe you meant to say Happy Together (http://us.imdb.com/Title?0118845) (which I haven't seen). If you like slow moving romance stories (and I do), and if you like Maggie Cheung (and I really do!), then they don't come any better than In The Mood For Love. Other Wong Kar-Wai films to check out and which I loved include Chungking Express (http://us.imdb.com/Title?0109424) and its sequel Fallen Angels (http://us.imdb.com/Title?0112913).

And as jggimi said, the length of the movie is no indication at all of how many CDs you'll need. Many 2 hour movies fit nicely on 1 CD. On the other hand, I did a 109 minute movie the other day that I decided needed 3 CDs(!), and at only 512x384 resolution. It's Alexander Nevsky (http://us.imdb.com/DVD?0029850) and the first pass was 5.5 GB in size (XviD). So, the Compress Test is an imperative.

[theboywonderuk]

Eeek! Sorry Manono hehe.. someone told me Happy Times was the same director but I didnt actually look hehe. The trailer for that film looks excellent however. I'll try and check out some of the other films you recommended. However, I think it's harder to get hold of them in the UK - Happy Times is due for release in Feb and play.com didn't bring results for Chongking Express or Fallen Angels. I'll have to go to a huge Blockbuster if there is one near here...

I know that 2 pass = variable which is why I did say "average" movie (taking into account the things you mentioned / motion / brightness etc) ;) Perhaps the variation in compressibility is bigger than I thought then - I guessed that would be able to make a rough estimate but obviously not! Thanx Jggimi - I'll have a look at that thread.

I'm very surprised that any 109 minute film could require 3 cds! Ouch! May I ask why you made 3 cds? Are you a quality freak or are you watching with one of those players that play avi and that you can hook up to your tv? Although I did think they were divx not xvid...

Thankyou for all the help people - much appreciated :)

[manono]

Hi-

Yeah, you're right. Chungking Express and Fallen Angels haven't been released to R2. That's strange, as they are both major movies.

Alexander Nevsky was very noisy and didn't compress well. Since I wanted the Audio and the Commentary track (both Mono), that left about 1260 MB for the video if done for 2 CDs. So the Compress test was less than 25% for 2 CDs. That was for XviD with the MPEG Matrix, Neutral Bicubic resizer, and 512x384 resolution. I switched to BiLinear Resizing, the softer H.263 matrix (which is what DivX5 uses), and added Temp and Spatial Smoothers. Then I actually ran the 2 passes for 2 CDs. The final average quant was over 5 (under 40%), and it just didn't look so good. So, I did it again for 3 CDs and the final average quant was 3.492 with a quality percentage of 57.27%. Those aren't very good figures, but were the best I could get out of it. People usually list Saving Private Ryan, The Abyss and Waterworld as among the least compressible movies (along with lesser known films like Pi, Clerks or Tigerland). But badly preserved older films such as Seven Samurai or Alexander Nevsky, and particularly many silent films, can be much less compressible. I defy anyone to encode Seven Samurai to 3 CDs and make it look good.

Although I haven't seen Happy Times, other Zhang Yimou films you might enjoy, and which I have seen and which are available in Britain are Not One Less (http://www.amazon.co.uk/exec/obidos/ASIN/B000055Z8Z/qid=1042981294/sr=1-2/ref=sr_1_2_2/202-7127531-3968623) and The Road Home (http://www.amazon.co.uk/exec/obidos/ASIN/B00005B1N8/qid=1042981294/sr=1-1/ref=sr_1_2_1/202-7127531-3968623). The Road Home stars Zhang Ziyi (the young beauty in Crouching Tiger, and a baddie in Rush Hour 2) in her first film.

[theboywonderuk]

Thankyou for all the info there :) Might just check out the last two films you mentioned. Play.com them :)

I want to try doing "In The Mood For Love" again tonight (1 disc rip) - i've just ran the compressibility test and am faced with a dilemma!

At a resolution of 640 x 384 with a 101mb audio file and sharp bicubic filter, the loading factor (is that the correct term?) was 97.7% ... and the bits/pixel is 0.143 :(

If I increase the resolution / audio filesize to decrease the loading factor.... the bits/pixel will decrease even more !

Hopefully you can help Manono since you did this dvd yourself!

[TheWEF]

Originally posted by theboywonderuk
...the loading factor (is that the correct term?)...

i think jggimi introduced this term. the value is called that way because it's diplayed next to the "load" button in gknot. but actually this button is labeled "load" because you can manually load and analyze a stats/log - file with it.

i guess the correct term would be "compression ratio". of course it's not relative to the uncompressed filesize, but relative to the minimum compression possible with a certain codec.

this should make it obvious that you can forget about the absolute bits/pixel value once you check compressibility. if the compression ratio is above 60% the quality of the avi will be fine, no matter how low the absolute value is.

think of a movie-clip that just shows a black sceen all the time. do you think you need a lot of bits/pixel to store that information? every source is different it it depends on content, filters,duration and framesize what compression ratio is possible at a certain filesize.

wef.

[manono]

Hi boywonderuk-

Leave off B-Frames (bi-directional encoding), Psychovisuals, GMC and Q-Pel if you have them checked in the Options Tab. If you have the first 2 of those checked, by unchecking them, you should get the percentage down. You could even take the resolution up to 704*xxx and get it down for sure. Sometimes you don't need B-Frames to help with the compression, and In The Mood For Love is one such movie. It ranks up with Panic Room for compressibility.

By the way, check your Aspect Error in the Resolution Tab, I'm not saying 640x384 is wrong, but you may have cropped more than necessary from the left and right. I came out with 640x368. But I no longer have the vobs on the hard drive to go back and check.

[theboywonderuk]

Hi,

Thanx. My copy is region 2 (pal) so I suspect that is why we have different resolutions!? The aspect error ratio is 0.1% - I always check this :)

Whenever I up the res to 704x... i the W-Zoom box goes red and the value is over 100% - is this a bad thing? I saw some people saying they were encoding at 800x... and higher ... ?

I'll try leaving B-Frames, Psychovisuals, GMC and Q-Pel off anyway! :)

I did do another last night using a 192kbit (CBR) mp3 audio source (129mb) and got 672mb combined. Still, it's really good so it's tempting to keep that hehe... but i will give it another go. Only takes just over 3 hours anyway :D

Robin

[manono]

Yes, of course. I forgot that they were from different regions. Sorry.

Personally I don't see anything wrong with going over 100% in the W-Zoom by a little bit. I've done it a few times.

[theboywonderuk]

Yes! 704x416 resolution and it turned out fantastic!! 700mb in size - the compression ratio was 78% so I guess thats why it came out perfect!

I love divx 5 !

Thankyou for all your help, much appreciated!