Undersized file

[genseb]

hi everybody,

First of all i would like to express my appreciation for this nice piece of software, namely GKnot.
I already tried it a couple of times and i find it very useful.
I was previously encoding using flask, but i now prefer GKnot.
It's truly a great job:)
Of course all the softwares related to GKnot ( DVD2AVI, SMARTRIPPER, BESWEET, etc...) are included in the above acknowledgment.
Thank you all for your work:)

Being a new user i still have a question, and i'm sure that you can help me.
I never succeeded in having a file the size i wanted when using the append credits option.
The file is always smaller.
Let's take an example.
I tried to encode the movie "Xmen" recently.
My purpose was to make it fit on a 700Mb CD.
The resulting file when i do not check the append credits option is a 699Mb file, which is almost prefect.
But when i check this option the size of the resulting file is only 665Mb.
I don't understand where the problem is.
It is most probably a problem with my setup, but i don't know where.
It si not really the fact that the file is smaller that bothers me.
The quality of the encoding is really good even with this difference.
It is just that i'm trying to understand what is going on.
I don't like lose ends in a certain way:)
So please could you help me?
Thank you in advance.
I hope you will have everything you need.

My log:

2002-05-22 14:00:38: Job "Movie 1" started.

o DivX5-Single, Quality 20%: C:\DVD\dvd_video\XMen\XMen_Credits.avs
o DivX5-First Pass: C:\DVD\dvd_video\XMen\XMen.avs
o DivX5-Second Pass: C:\DVD\dvd_video\XMen\XMen.avs
o Append Credits.
x Delete Intermediate Files (disabled).
End of Job 1 (Movie 1).
_______________________________________________________________________________________________________________________

Gordian Knot 0.26.0.6
Encoding Job Data:

Type: EncD5_Both_CreditsSep
Number: 1
Name: Movie 1
Platform: WinXP (5.1.2600).2

Files:
VirtualDubExe: C:\PROGRA~1\GORDIA~1\virtualdub\Virtualdub.exe
NandubExe: C:\PROGRA~1\GORDIA~1\nandub\Nandub.exe
AviFinal: C:\DVD\dvd_video\XMen\XMen.avi
AviMovie: C:\DVD\dvd_video\XMen\XMen_Movie.avi
AviCredits: C:\DVD\dvd_video\XMen\XMen_Credits.avi
MovieFS: C:\DVD\dvd_video\XMen\XMen.avs
CreditsFS: C:\DVD\dvd_video\XMen\XMen_Credits.avs
Log : C:\DVD\dvd_video\XMen\XMen.log
Stats: C:\DVD\dvd_video\XMen\XMen.stats
Ecf : C:\DVD\dvd_video\XMen\XMen.ecf

Options:
Quality/DRF: 20
CreditsStart: 171041
UseEcf: 1
CreditsIVTC: 1
CreditsAppend: 1
DeleteInterm.: 0
EnforceBitrate:0
AntiShit: 0

Calculator:
VideoSize: 608282 kbyte.
Audio1Size: 96701 kbyte.
Audio2Size: 0 kbyte.
Overhead: 11815 kbyte.
AviSize: 716800 kbyte (700 mbyte).
FilesSize: 0 kbyte.
TotalSize: 716800 kbyte (700 mbyte).
Interleaving: vbrmp3
Bitrate: 797 k(=1000)Bits/s
Frames: 187542
FPS: 29.970
Duration: 1:44:18
Quality: 0.171 bits/(pixel*frame).
CompCheck3: 0.000 bits/(pixel*frame).
CompCheck5: 0.375 bits/(pixel*frame).
DivX5BiDirect: 1
DivX5GMC: 1
DivX5QuartPel: 0
DivX5PsyVis: 0
Resolution: 608 x 256

Audio 1:
mode: Just Mux Source File.
sourceFile: C:\DVD\dvd_video\XMen\XMen AC3 T03 2_0ch 192Kbps DELAY 6ms.mp3
wavFile: C:\DVD\dvd_video\XMen\XMen AC3 T03 2_0ch 192Kbps DELAY 6ms.wav
mp3File: C:\DVD\dvd_video\XMen\XMen AC3 T03 2_0ch 192Kbps DELAY 6ms_1.mp3
azid: ""
lame: ""
delay: 6
interval: 2
preload: 500
mux: 1
recalc: 1
deleteWav: 1

Audio 2:
none.

14:00:38: IVTC: Re-Calculating Frame Count.
14:00:38: IVTC: FrameRate was 29.970 fps
14:00:38: IVTC: FrameRate now is 23.976 fps
14:00:38: IVTC: 150042 Frames.
14:00:38: Audio 1, determined: 96701 kb
14:00:38: Audio 1, calc: 96701 kb
14:00:38: New Bitrate for Movie = 800 k(=1000)Bits/s

Adding this script to C:\PROGRA~1\GORDIA~1\virtualdub\VirtualDub.jobs:

// $job "DivX4 Single Pass"
// $input "C:\DVD\dvd_video\XMen\XMen_Credits.avs"
// $output "C:\DVD\dvd_video\XMen\XMen_Credits.avi"
// $state 0
// $start_time 0 0
// $end_time 0 0
// $script

VirtualDub.Open("C:\\DVD\\dvd_video\\XMen\\XMen_Credits.avs",0,0);
VirtualDub.audio.SetSource(0);
VirtualDub.audio.SetMode(0);
VirtualDub.audio.SetInterleave(1,0,0,0,0);
VirtualDub.audio.SetClipMode(1,1);
VirtualDub.audio.SetConversion(0,0,0,0,0);
VirtualDub.audio.SetVolume();
VirtualDub.audio.SetCompression();
VirtualDub.video.SetDepth(24,24);
VirtualDub.video.SetMode(1);
VirtualDub.video.SetFrameRate(0,1);
VirtualDub.video.SetIVTC(0,0,-1,0);
VirtualDub.video.SetRange(0,0);
VirtualDub.video.SetCompression(0x78766964,0,10000,0);
VirtualDub.video.SetCompData(308,"GAEAAACgDwDQBwAACgAAABQAAAAsAQAADAAAAAIAAAAFAAAAAQAAAMgAAAAAAAAAAAAAAB4AAABkAAAAAAAAAAAAAAABAAAAAAAAAAAAAAAAAAAAAAAAAAAgAAAyAAAAAAAAAAAQAACamZmZmZnJP5qZmZmZmck/QAAAAAEAAACAAAAAAAAAAAAAAAAAAAAAAAAAAIACAADgAQAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAOA/AAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAABAAAAAQAAAAEAAAABAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAABAAAAAAAAAAEAAAD/////BAAAAGVtcHR5AGM6XHRlc3QuZGl2eABlbXB0eS5tdgBt");
VirtualDub.video.filters.Clear();
VirtualDub.subset.Delete();
VirtualDub.SaveAVI("C:\\DVD\\dvd_video\\XMen\\XMen_Credits.avi");
VirtualDub.Close();

// $endjob
//
//--------------------------------------------------
// $done

14:00:39: Started DivX5-Single, Quality 20%: C:\DVD\dvd_video\XMen\XMen_Credits.avs
14:16:39: Finished DivX5-Single, Quality 20%. Duration: 16 minutes, 0 seconds.
14:16:39: Assumed: Main Movie has 136833 Frames.
14:16:39: Assumed: Credits Part has 13209 Frames.
14:16:39: Credits Size = 12618Kb
14:16:39: New Bitrate for Movie = 858 k(=1000)Bits/s

Adding this script to C:\PROGRA~1\GORDIA~1\virtualdub\VirtualDub.jobs:

// $job "DivX4 First Pass"
// $input "C:\DVD\dvd_video\XMen\XMen.avs"
// $output "C:\DVD\dvd_video\XMen\XMen_Movie.avi"
// $state 0
// $start_time 0 0
// $end_time 0 0
// $script

VirtualDub.Open("C:\\DVD\\dvd_video\\XMen\\XMen.avs",0,0);
VirtualDub.audio.SetSource(0);
VirtualDub.audio.SetMode(0);
VirtualDub.audio.SetInterleave(1,0,0,0,0);
VirtualDub.audio.SetClipMode(1,1);
VirtualDub.audio.SetConversion(0,0,0,0,0);
VirtualDub.audio.SetVolume();
VirtualDub.audio.SetCompression();
VirtualDub.video.SetDepth(24,24);
VirtualDub.video.SetMode(1);
VirtualDub.video.SetFrameRate(0,1);
VirtualDub.video.SetIVTC(0,0,-1,0);
VirtualDub.video.SetRange(0,0);
VirtualDub.video.SetCompression(0x78766964,0,10000,0);
VirtualDub.video.SetCompData(354,"GAEAAJAXDQDQBwAACgAAABQAAAAsAQAADAAAAAIAAAAFAAAAAgAAAJABAAAAAAAAAAAAAB4AAABkAAAAAAAAAAAAAAABAAAAAAAAAAAAAAAAAAAAAAAAAAAgAAAyAAAAAAAAAAAQAACamZmZmZnJP5qZmZmZmck/QAAAAAEAAACAAAAAAAAAAAAAAAAAAAAAAAAAAIACAADgAQAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAOA/AAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAABAAAAAQAAAAEAAAABAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAABAAAAAAAAAAEAAAD/////BAAAAEM6XERWRFxkdmRfdmlkZW9cWE1lblxYTWVuLmxvZwBjOlx0ZXN0LmRpdngAQzpcRFZEXGR2ZF92aWRlb1xYTWVuXFhNZW4ubXYAbQ==");
VirtualDub.video.filters.Clear();
VirtualDub.subset.Delete();
VirtualDub.SaveAVI("C:\\DVD\\dvd_video\\XMen\\XMen_Movie.avi");
VirtualDub.Close();

// $endjob
//
//--------------------------------------------------
// $done

14:16:39: Started DivX5-First Pass: C:\DVD\dvd_video\XMen\XMen.avs
17:22:19: Finished DivX5-First Pass. Duration: 3 hours, 5 minutes, 40 seconds.
17:22:19: Trying to open Log-file.
17:22:34: Success: Log-file open.
17:22:34: Encoded: 136831 Frames.
17:22:34: Speed: 12.283 Frames per Second.

Adding this script to C:\PROGRA~1\GORDIA~1\virtualdub\VirtualDub.jobs:

// $job "DivX4 Second Pass"
// $input "C:\DVD\dvd_video\XMen\XMen.avs"
// $output "C:\DVD\dvd_video\XMen\XMen_Movie.avi"
// $state 0
// $start_time 0 0
// $end_time 0 0
// $script

VirtualDub.Open("C:\\DVD\\dvd_video\\XMen\\XMen.avs",0,0);
VirtualDub.audio.SetSource(0);
VirtualDub.audio.SetMode(0);
VirtualDub.audio.SetInterleave(1,0,0,0,0);
VirtualDub.audio.SetClipMode(1,1);
VirtualDub.audio.SetConversion(0,0,0,0,0);
VirtualDub.audio.SetVolume();
VirtualDub.audio.SetCompression();
VirtualDub.video.SetDepth(24,24);
VirtualDub.video.SetMode(1);
VirtualDub.video.SetFrameRate(0,1);
VirtualDub.video.SetIVTC(0,0,-1,0);
VirtualDub.video.SetRange(0,0);
VirtualDub.video.SetCompression(0x78766964,0,10000,0);
VirtualDub.video.SetCompData(354,"GAEAAJAXDQDQBwAACgAAABQAAAAsAQAADAAAAAIAAAAFAAAAAwAAAJABAAAAAAAAAAAAAB4AAABkAAAAAAAAAAAAAAABAAAAAAAAAAAAAAAAAAAAAAAAAAAgAAAyAAAAAAAAAAAQAACamZmZmZnJP5qZmZmZmck/QAAAAAEAAACAAAAAAAAAAAAAAAAAAAAAAAAAAIACAADgAQAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAOA/AAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAABAAAAAQAAAAEAAAABAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAABAAAAAAAAAAEAAAD/////BAAAAEM6XERWRFxkdmRfdmlkZW9cWE1lblxYTWVuLmxvZwBjOlx0ZXN0LmRpdngAQzpcRFZEXGR2ZF92aWRlb1xYTWVuXFhNZW4ubXYAbQ==");
VirtualDub.video.filters.Clear();
VirtualDub.subset.Delete();
VirtualDub.SaveAVI("C:\\DVD\\dvd_video\\XMen\\XMen_Movie.avi");
VirtualDub.Close();

// $endjob
//
//--------------------------------------------------
// $done

17:22:34: Started DivX5-Second Pass: C:\DVD\dvd_video\XMen\XMen.avs
19:44:47: Finished DivX5-Second Pass. Duration: 2 hours, 22 minutes, 13 seconds.
19:44:47: Speed: 16.035 Frames per Second.

Adding this script to C:\PROGRA~1\GORDIA~1\nandub\VirtualDub.jobs:

// $job "DivX3 Append"
// $input "C:\DVD\dvd_video\XMen\XMen_Movie.avi"
// $output "C:\DVD\dvd_video\XMen\XMen.avi"
// $state 0
// $start_time 0 0
// $end_time 0 0
// $script

VirtualDub.Open("C:\\DVD\\dvd_video\\XMen\\XMen_Movie.avi",0,0);
VirtualDub.Append("C:\\DVD\\dvd_video\\XMen\\XMen_Credits.avi");
VirtualDub.audio.SetSource("C:\\DVD\\dvd_video\\XMen\\XMen AC3 T03 2_0ch 192Kbps DELAY 6ms.mp3");
VirtualDub.audio.SetMode(0);
VirtualDub.audio.SetInterleave(1,500,2,0,6);
VirtualDub.audio.SetClipMode(1,1);
VirtualDub.audio.SetConversion(0,0,0,0,0);
VirtualDub.audio.SetVolume();
VirtualDub.audio.SetCompression();
VirtualDub.audio2.SetSource(0);
VirtualDub.audio2.SetMode(0);
VirtualDub.audio2.SetInterleave(1,0,0,0,0);
VirtualDub.audio2.SetClipMode(1,1);
VirtualDub.audio2.SetConversion(0,0,0,0,0);
VirtualDub.audio2.SetVolume();
VirtualDub.audio2.SetCompression();
VirtualDub.video.SetDepth(24,24);
VirtualDub.video.SetMode(0);
VirtualDub.video.SetFrameRate(0,1);
VirtualDub.video.SetIVTC(0,0,-1,0);
VirtualDub.video.SetRange(0,0);
VirtualDub.video.SetCompression();
VirtualDub.video.filters.Clear();
VirtualDub.subset.Delete();
VirtualDub.SaveAVI("C:\\DVD\\dvd_video\\XMen\\XMen.avi");
VirtualDub.Close();

// $endjob
//
//--------------------------------------------------
// $done

19:44:47: Started Appending Credits and Muxing Audio.: C:\DVD\dvd_video\XMen\XMen_Movie.avi
19:47:23: Finished Appending Credits and Muxing Audio.. Duration: 2 minutes, 35 seconds.

19:47:23: Done.
19:47:23: Movie = C:\DVD\dvd_video\XMen\XMen.avi
Total Encoding Time: 5 hours, 46 minutes, 44 seconds.
2002-05-22 19:47:23: Job "Movie 1" finished.

I'm also including a screenshot of my settings.
Thank you once again for your help!:)
And sorry for my english:)

[genseb]

On the image we could see that i had run a compressibility test.
The result was:

Bits/(Pixel*Frame): 0.171
percentage: 45.6% of 0.375

The resolution was 608x256

Thanks:)

[llemor]

Credits is encoded with 20% quality at 1-pass quality based mode. It is encoded with higher quantizer of 25 (lesser bitrate), i guess, so the credits file size is smaller. That's why you get smaller file size when you activate append credits option.

[genseb]

Thank you very much for your help Ilemor:)
One more question if i may.
I thought that when choosing the append credits option GKnot was recalculating the bitrate for the movie, thus taking all the extra place available.
If it's the case, how is it that the size is smaller?
Does it mean that in a certain way the movie has reached its maximum size?
Or is it that GKnot behave differently?

Thanks once again:)

[TheWEF]

Originally posted by genseb
I thought that when choosing the append credits option GKnot was recalculating the bitrate for the movie, thus taking all the extra place available.

and your thoughts were correct.
open analyze.log and post the last line, then we know what went wrong.

wef.

[genseb]

Thank you for your help TheWEF, and congratulation for your software:)

Here are the last lines of my analyse.log.

Frame 136829: PRESENT (B), complexity 0, quant multiplier 0.500000, texture 25, total 750
Progress: expected 4896437827, achieved 4615641143, dq 0.600000, new quant 2
Frame 136830: PRESENT (P), complexity 3149, quant multiplier 0.500000, texture 1395, total 5968
Progress: expected 4896560871, achieved 4615647111

There's a clear difference between the expected and the achieved.
What does it mean?

Genseb.

[tempe]

Hi Wef and everybody,

first, thank you very much for the new release!! Also thanks to all the writers of the freeware programs that are involved.

I have the very same problem with the conversion of "Along Came A Spider". The file size is supposed to be 700 megs, but is only 664.

Here are the last lines of my analyse.log:

Frame 140605: PRESENT (B), complexity 122312, quant multiplier 1.500000, texture 28612, total 32616
Progress: expected 4904234207, achieved 4665028360, dq 0.600000, new quant 2
Frame 140606: PRESENT (P), complexity 2371656, quant multiplier 1.500000, texture 204541, total 208536
Progress: expected 4904372160, achieved 4665236896

Any help and solution is greatly appreciated!! Thank you very much!

Tempe

[jonny]

@tempe:

http://forum.doom9.org/showthread.php?s=&threadid=24584
Have you read this?

[tempe]

Hi Johnny,

I did read it beforehand, but I don't think it applies to my case, since I only use 1 CD. So I don't think the movie's maxed out.
Also, the result of the compressibility test was 55.7% of 0.310. So it's far below 80% (referring to Doom 9's guide) and it's only 35 megs and not >50 below anticipated size.
Still, thanks for answering.

I cropped the picture on the left and right in order not to get a too wide screen, so the W-Zoom value went red and up to 102%. Could this have something to do with it?

Tempe

[jonny]

55.7% is ok... it is strange you get undersized


I cropped the picture on the left and right in order not to get a too wide screen, so the W-Zoom value went red and up to 102%. Could this have something to do with it?


This have no influence on getting undersized



Progress: expected 4904372160, achieved 4665236896


4.665.236.896 / 8 = 583.154.612 = 569.487 KB = 556 MB

Please verify if you have this values: 569.487 KB - 556 MB in "Video Size" in the Bitrate tab of GKnot (values in GKnot should differ by 3-4MB because of the AVI overhead)


Can you post the first line of your analyse.log?


Cheers
jonny

[tempe]

No... the video size in the bitrate tab is 607948 kb or 594 mb...

Here's the beginning of my analyse.log:

Total frames: 140607 Non-zero I & P decisions: 71329 Avg quantizer: 3.289344
Expecting 4904372160 bits, including 554527618 bits from B frames
Frame 0: PRESENT (I), complexity 209125, quant multiplier 1.250000, texture 16730, total 17080
Progress: expected 73628, achieved 17080, dq 1.000000, new quant 4

So what does that mean?
Please help!!

Tempe

[jonny]

The video seems to be not saturated... but the final size is undersized... strange thing...

any idea? Wef?

[TheWEF]

Originally posted by genseb
Progress: expected 4896560871, achieved 4615647111

There's a clear difference between the expected and the achieved.
What does it mean?

4896560871 / (1024*1024*8) = 583.7 mb
this is what gknot calculated for video size.

4615647111 / (1024*1024*8) = 550.2 mb
this is what the codec ended up with.

583.7 - 550.2 = 33.5 mb
that's what your final avi is too small.

all i can say is, that this is not gknot's fault. the calculations are correct. either the movie is saturated at 550.2 mb (i doubt that because of the res you used. also comp-check told you that the limit is much higher) or the codecs filesize predictibility still is not as good as some might think.

you can:

1) use divx3 (and get 100% exact filesize every time).

2) go to www.divx.com and complain in the forum over there. ;)

wef.

[genseb]

Thank you very much for your answer TheWEF:)

I encoded a second movie this week end and interestingly all went very well.
The only difference between this movie and the previous one is that for the "XMen" i used the ivtc option.

Do you think there could be a connection?

Tempe did you also use ivtc?

Cheers

Genseb

[PeterTheMaster]

well seems like filesize predictability is not yet perfect in divx502.
i just got 667mb (with audio, instead of 700mb)
some movies before were around 685 or 690, i just ignored it then because it didnt seem worth the effort.
now i try to encode the movie with quant=2 and see if it was already at its limits but i dont think so since its 125min and 704x... resolution. (its "enemy at the gates")

but it dont think this is a reason to go back to divx311a (which by the way sometimes also had like 2mb undersized files). because a 311a encode which is 10% larger will by no means look as good as the 502 encode.

i use gmc, b-frames and psyvis strong, does any one of them make the filesize predictability worse?

maybe this thread should be moved to the divx5 section so someone who should see it can find it. (are there any divx5 developers reading this forum?)

[hakko504]

Originally posted by PeterTheMaster
i use gmc, b-frames and psyvis strong, does any one of them make the filesize predictability worse?

B-frames will throw filesize prediction to a warm place. Or rather, it will guarantee that you never get a larger file than you wanted. It seems the first-pass stats are based on useing quant=2. But b-frames are always encoded with a higher quant then a normal frame would. This sticky thread (http://forum.doom9.org/showthread.php?s=&threadid=24584) in the DivX 5.0 forum (http://forum.doom9.org/forumdisplay.php?s=&forumid=56) can explain it better.

[genseb]

Hi everybody,

In my case i'm always using b-frames and gmc with a very good result(the size of the resulting file is always between 699mb and 701mb).
That's the reason why i don't understand where the problem is.
Once again the only difference is that this time i used the ivtc.
I don't have to use it generally.
Does ivtc have an influence on the size predictability?

Cheers

Sebastien