Wierd Filesize Problem!! [Archive]

imfaizzi

18th April 2008, 23:18

I've been using AGK for DVD ripping using DivX with auto resolution and auto mp3 settings and predefined size of 700MB. It has always resulted in a 700MB rip. Now when I tried to rip the same DVD using Xvid with moreorless the same settings then it produced an avi of 459MB. I tried it couple of times, using variations in predefined and custom size of 700MB and 1400MB, but the resulting AVI remained sizing in a range between 450MB to 460MB for 700MB and 920MB to 980MB for 1400MB etc.
Can anyone tell me what configuration is required to produce the same 700MB AVI using Xvid which the Divx easily produces?

unskinnyboy

19th April 2008, 02:44

Did you install Xvid over a previous version? Sounds like it. Can you try uninstalling Xvid, reinstall the latest build again and try your encode again? Also, post your AGK log file. Your post doesn't contain all the information needed to troubleshoot this issue.

imfaizzi

21st April 2008, 07:45

I had uninstalled Xvid 1.12 and installed the latest build 1.13 then tried another rip but it came out a no use. The rip sized 459MB instead of 700MB.

Although the log file is attached but for convenience sake, following is another link

http://www.4shared.com/file/44831670/797cdd31/AP3_agk.html

imfaizzi

24th April 2008, 09:52

Please help me!

unskinnyboy

24th April 2008, 13:52

I don't see anything out of the ordinary in the log. There was some undersizing issues sometime back with the ESS option, but that shouldn't apply to you since you are using MTK/Sigma. I did notice, however, that you are using AutoGK v2.40. Can you upgrade to v2.48 and retry this? Be sure to uninstall any Xvid builds you already have, before upgrading.

weaver4

24th April 2008, 15:36

I believe that autogk uses an unofficial XviD build version 1.2.

Reinstall Autogk again, with the XviD that came with it. That should help.

manono

24th April 2008, 16:47

Yep, first uninstall XviD, and then upgrade to 2.48 and the XviD included with it.

yetanotherid

25th April 2008, 06:55

I wonder if your problem is related to mine?

I've written a post here entitled "This target file size thing....." which involves a similar problem. I've tried lots of different versions of the XviD codec, and all but one fail to hit the target file size every time. They either give me the target size, or a file around 60% of the target size, which seems to be similar to what you're experiencing.
The only difference is that while I was trying different codecs I was testing with a smaller conversion (100MB output file) and the result was always pretty close to being the same (60MB) if it didn't hit the 100MB target. However I just looked at a larger conversion which I'd kept, which was supposed to result in a 350MB file but the output file was 272MB, which is a different percentage, and I can't remember how much variation I was getting when converting larger files.

I'm wondering though if we're experiencing the same problem. Whatever is causing it, I find it hard to believe that it's not common, given that I'm having the same problem on two different computers. So far nobody's replied in my thread which has surprised me. I thought someone here would be able to have a guess at the cause of the problem. Using the ESS option was mentioned earlier here, and I've always used it, so when I get a chance I'll do some test conversions again with that option turned off and see what happens.
The only thing which made me suspect the codec rather than AutoGK was early on I tried a different MPEG4 ripper (can't remember which one) and had the same problem, so I figured it the problem must be with the 'engine' rather than the software driving the car, but I guess I'll just keep playing around until I come up with another theory.

I can't remember where I found it, but I'm still using Koepi's XviD version 1.2.127 Alpha (25/02/2006) as it's the only Xvid version which still, after many more days of converting, still hasn't missed the target file size for me. Maybe if all else fails, try that version (if you can find it via Google) and see what happens. It'd be interesting to see if as with me, that version of Xvid makes the problem go away for you.

imfaizzi

25th April 2008, 09:12

I guess you're right. Two people can have a same experience but know what...this thing happens only with Xvid conversions. DivX is still sailing smoothly. Its been converting to 700MBs without any problem and hassles.

yetanotherid, wondering whether you've tried out the conversion using DivX??

As for uninstalling is concerned, I believe I must wait a couple of days for it. Since DivX is providing me with my desired results therefore I must wait, before upgrading, until the entire stack of movies are ripped.

Can anybody provide me with the most authentic Xvid version?

manono

25th April 2008, 10:33

Can anybody provide me with the most authentic Xvid version?
Reinstall Autogk again, with the XviD that came with it.
...then upgrade to 2.48 and the XviD included with it.
What's with all the boldfaced writing?

imfaizzi

25th April 2008, 14:25

What's with all the boldfaced writing?

Well I had read the rules before posting on forum and read them again (after your post ofcourse) but couldn't find anything saying about the "Boldfaced" writings. Hence, I presumed there would be no harm in using it. Like I believed, what is wrong in using a feature provided by the forum itself? However, if it bothers to anybody (including mods and admins) then I sincerely apologize in this regard.

Also, those quotations were unnecessary for I had known what had been recommended to me. All I asked for was the "version" of such particular codec. Thats all.

unskinnyboy

25th April 2008, 15:54

If you just let AutoGK install it's bundled version of Xvid, then you don't need to worry about the versioning yourself. But, if you must know, it's the CVS build from 12/08/06, the same one which was bundled with AutoGK v2.40. Then why didn't the same build work for v2.40? Well, the build works better with v2.48.

yetanotherid

26th April 2008, 08:32

yetanotherid, wondering whether you've tried out the conversion using DivX??
To be honest, no I haven't. I did a bit of 'internet' reading before I started my first rip, and the consensus seemed to be 'Xvid' so I haven't thought about DivX since. When I get through the rips I'm doing at the moment, I'll install it and give it a try.

I did, last night, try disabling ESS compatibility for a rip, but the resulting file didn't play well in my standalone player. I guess it's got an ESS chip. So I didn't do any more experimenting with other XviD versions and ESS compatibility disabled, as even if it solved the file size problem I can't use the files anyway.

I'm still amazed that this file size issue isn't a common problem (assuming we're experiencing the same one). I installed Auto GK on two different computers (one of which had been reformatted recently and never seen another codec) and both have the same issue with all but one version of XviD. Can I be that unlucky while everyone else is hitting the target file size? And in both those cases I started out with AutoGK 2.5, applied the 2.8 upgrade, and used the XviD version that came with AutoGK. Sigh....

Mind you even after all the problems, now I've found a version of Xvid that hits the target file size each time I'm more than happy. AutoGK rocks! I really only started my thread and jumped in on this one because I've just never been able to leave well enough alone. :-)

yetanotherid

26th April 2008, 08:52

What's with all the boldfaced writing?
I'm going to go out on a limb here, especially being a newbie and all, and say I sorta like the boldfaced posts from the original poster. It sort of says, this is the original post, and here's the replies in this thread by that person.
I'm not trying to start a debate on the subject... just thinking out loud. Although maybe a different color text rather than using bold would be more appropriate. ;-)

jeffy

26th April 2008, 14:32

I'm going to go out on a limb here, especially being a newbie and all, and say I sorta like the boldfaced posts from the original poster. It sort of says, this is the original post, and here's the replies in this thread by that person.
I'm not trying to start a debate on the subject... just thinking out loud. Although maybe a different color text rather than using bold would be more appropriate. ;-)

IMHO: don't use bold text unless necessary. Use it for highlighting the important details in a longer text block when you want to save readers' time when skimming through a longer paragraph.

What is the original post and what are the replies is clear for everyone, isn't it?
IMHO: The name of the poster on the left is enough.

imfaizzi

26th April 2008, 20:04

now I've found a version of Xvid that hits the target file size each time I'm more than happy.

Good for you my friend.:)

imfaizzi

27th April 2008, 13:11

Irakli

27th April 2008, 22:06

I uninstalled Xvid along with AutoGK 2.40. Then installed AutoGK 2.5 (I also let it installed its bundled Xvid) then upgraded it to 2.48b. Tried another Xvid rip and here Im complaining for undersizing and asking for help, yet again.:confused:

Filesize: 429 MB

http://www.4shared.com/file/45538694/87736346/RD1_agk.html

I understand why you have undersize problem with this source (btw it has nothing to do with incorrect xvid install etc., so keep your setup as it is). I will briefly explain the reason below.

Looking at your .log file I can see that the resolution of your source is 352x480.
Now, AutoGK has to resize to 352x208 in order to obtain a proper aspect ratio. This basically means loosing more than a half of original vertical resolution (but AutoGK has to resize like this because it simply doesn't support any upsizing).
Because the resolution is only 352x208 now, the video you are encoding becomes very compressible (as you can see from the log file - 182%!). So the filesize of 428MB just corresponds to the maximum bitrate XviD can encode with in these circumstances (i.e. given resolution of 352x208, compressibility of the source, etc.). Bitrate (and, hence, filesize) just can't go any higher than this. This is a typical 'codec saturation' situation.

[4/27/2008 2:10:24 PM] Source resolution: 352x480

[4/27/2008 2:31:40 PM] Duration was: 2 minutes 33 seconds
[4/27/2008 2:31:40 PM] Speed was: 42.29 fps.
[4/27/2008 2:31:40 PM] Compressibility percentage is: 324.17
[4/27/2008 2:31:40 PM] Using sharper matrix
[4/27/2008 2:31:40 PM] Switching b-frames off
[4/27/2008 2:31:40 PM] Chosen resolution is: 352x208 ( AR: 1.69 )
[4/27/2008 2:31:40 PM] Predicted comptest value is: 182.10%

unskinnyboy

28th April 2008, 00:40

imfaizzi

28th April 2008, 16:43

Indeed. The second source (RD1) is way different from the first one (AP3), and the undersizing of RD1 is exactly due to the reason Irakli explained. imfaizzi, you should take the time to look at the logs before reporting something as a bug. In the second log, AutoGK clearly says:

[4/27/2008 3:01:26 PM] Warning: final AVI will likely be undersized.

You should do the AP3 one again to compare, not this.

I don't think different DVDs are of any matter here. Its just whatever movie I have tried so far, has failed to achieve the desired predefined size using Xvid. However, the situation is exactly opposite in DivX.

Although there's a hell lot of difference in both the logs, yet the striking similarity within the two outputs is something beyond. All I wanna know is how can this "warning" thing be fixed?

unskinnyboy

28th April 2008, 16:57

I don't think different DVDs are of any matter here. Its just whatever movie I have tried so far, has failed to achieve the desired predefined size using Xvid. However, the situation is exactly opposite in DivX.

Although there's a hell lot of difference in both the logs, yet the striking similarity within the two outputs makes me think of any other reason besides the one you've suggested.

All I wanna know is how can this "warning" thing be fixed?You need to understand the basics of compression first, which I don't think you do now. Of course, the source matters! With a source resolution of 352x480 and a big target file size, of course, Xvid will max out trying to reach it. Are you saying that with DivX, this same source (RD1) gave you the exact file size? If so, please post the log file for that. I ain't believing unless I see it myself.

Now that you have upgraded AutoGK since your first post, why aren't you encoding your first source (AP3) again to do a fair comparison? That's what I am asking you to do.

imfaizzi

28th April 2008, 16:59

I understand why you have undersize problem with this source (btw it has nothing to do with incorrect xvid install etc., so keep your setup as it is). I will briefly explain the reason below.

Looking at your .log file I can see that the resolution of your source is 352x480.
Now, AutoGK has to resize to 352x208 in order to obtain a proper aspect ratio. This basically means loosing more than a half of original vertical resolution (but AutoGK has to resize like this because it simply doesn't support any upsizing).
Because the resolution is only 352x208 now, the video you are encoding becomes very compressible (as you can see from the log file - 182%!). So the filesize of 428MB just corresponds to the maximum bitrate XviD can encode with in these circumstances (i.e. given resolution of 352x208, compressibility of the source, etc.). Bitrate (and, hence, filesize) just can't go any higher than this. This is a typical 'codec saturation' situation.

[4/27/2008 2:10:24 PM] Source resolution: 352x480

[4/27/2008 2:31:40 PM] Duration was: 2 minutes 33 seconds
[4/27/2008 2:31:40 PM] Speed was: 42.29 fps.
[4/27/2008 2:31:40 PM] Compressibility percentage is: 324.17
[4/27/2008 2:31:40 PM] Using sharper matrix
[4/27/2008 2:31:40 PM] Switching b-frames off
[4/27/2008 2:31:40 PM] Chosen resolution is: 352x208 ( AR: 1.69 )
[4/27/2008 2:31:40 PM] Predicted comptest value is: 182.10%

hmmm rite....I've understand what you've said so far but could you please highlight me what to do now in order to avoid the warning?

imfaizzi

28th April 2008, 17:23

You need to understand the basics of compression first, which I don't think you do now. Of course, the source matters! With a source resolution of 352x480 and a big target file size, of course, Xvid will max out trying to reach it. Are you saying that with DivX, this same source (RD1) gave you the exact file size? If so, please post the log file for that. I ain't believing unless I see it myself.

Now that you have upgraded AutoGK since your first post, why aren't you encoding your first source (AP3) again to do a fair comparison? That's what I am asking you to do.

When Xvid failed, I've deleted the original AP3 source after when it had been converted to 700 MB AVI using DivX.

Yes, believe it or not, but it has been happening, RD1 gave 429 MB in Xvid and 700 MB in DivX (both were predefined size 700 MB). I had deleted the DivX log file for I always delete the agk folder and log files, after successful and desired DivX conversions.

Wait for sometime, Whenever I'll encode any other movie. I'll certainly share both, the Xvid and the DivX versions of the log file.

For now, How can this Xvid-maxing-out thing could be cured??

manono

29th April 2008, 01:59

For now, How can this Xvid-maxing-out thing could be cured??
Try either going into the Advanced Settings and choosing a Fixed Width (maybe try 720), or in the main screen choosing a Custom Size (maybe try 350 MB).
Yes, believe it or not, but it has been happening, RD1 gave 429 MB in Xvid and 700 MB in DivX (both were predefined size 700 MB).
Both for the same resolution, without any intervention by you? Open them both in GSpot (or VDubMod, File->File Information) and compare the resolutions.

unskinnyboy

29th April 2008, 02:13

Try either going into the Advanced Settings and choosing a Fixed Width (maybe try 720), or in the main screen choosing a Custom Size (maybe try 350 MB).This isn't a good idea. His source resolution is 352x480 (CVD resolution). Blowing that up to 720x??? would create lot of artifacts in his video, nothing else. His width shouldn't go above 352 (as is the case now).

imfaizzi, for RD1, there's only two things you can do which would make any sense - keep the original AC3 audio (it's being encoded to MP3 now) or to reduce the target file size (which manono already suggested).

manono

29th April 2008, 03:36

Hi unskinnyboy,

Thanks for trying to help with this one. You have more patience than do I. :)
His source resolution is 352x480 (CVD resolution). Blowing that up to 720x??? would create lot of artifacts in his video, nothing else. His width shouldn't go above 352 (as is the case now).
When being resized at playback as the original VCD, doesn't it get resized to 640x480? I'm not sure I agree that the width shouldn't ever be increased. Doesn't a 16:9 NTSC DVD get resized to ~854x480 at playback? In any event, I think we can agree that his source is garbage to begin with, and that the best solution would be to cut the file size (or as you also suggest, pad the size with the original audio). I suggested increasing the width because it would get the filesize up, and imfaizzi seems to be married to 1 CD filesizes for some reason.

unskinnyboy

29th April 2008, 04:51

When being resized at playback as the original VCD, doesn't it get resized to 640x480? I'm not sure I agree that the width shouldn't ever be increased. Doesn't a 16:9 NTSC DVD get resized to ~854x480 at playback? In any event, I think we can agree that his source is garbage to begin with, and that the best solution would be to cut the file size (or as you also suggest, pad the size with the original audio). I suggested increasing the width because it would get the filesize up, and imfaizzi seems to be married to 1 CD filesizes for some reason.It's not VCD, it's CVD (China Video Disc) or Half D1 (352x480 for NTSC and 352x576 for PAL). :)

OK, so we don't know the DAR of his source, so if he were to play it back, it would be resized to 854x480 (16:9) or 640x480 (4:3), that's right. But, doesn't mean he should upconvert to 720x??? in this case (unless he wants to do it just for the sole purpose of hitting the target file size). The only way upconverting would even make sense is if it's done properly using a good resizer (EEDI2, NNEDI or something such) and supplemented by a lot of bitrate. AutoGK, with its limited repertoire of downsizers, isn't the ideal tool for this. It's better to let the player resize, IMO.

Ideally, he should crop, anamorphically encode without resizing, resize during playback using the appropriate filters in ffdshow, use a Matroska container and all that, but that's beyond the scope of this thread. :)

manono

29th April 2008, 06:30

Hi again,

After doing some research on CVD, you're right (as you well knew), and thanks for the correction. It's not VCD, of course, as it would have to be 352x240. And it doesn't have to be 4:3, as I was also assuming, but can be 16:9. But his is 4:3 (from the log):
Source aspect ratio: 4:3
If I'm not mistaken, CVD requires MP2 audio. I can't find much about the CVD specs, but my source for that statement is here:

http://forum.videohelp.com/topic98177.html?sid=1347a879052058538f1764b7dc1cd35e

And this thing has AC3 audio. Half-D1 DVD can have AC3 audio, so I suspect the source is just a Half-D1 DVD. Or maybe I'm wrong again. In any event, since these things get resized at playback time (the source MPEGs, I mean) then I don't really understand why letting AutoGK upsize will be any worse. Although LanczosResize upsizing might not be optimal, it's not as bad as you seem to make it out to be (in my opinion). Personally, I wouldn't upsize anything greater than the original height, and ignore the width all together.
It's better to let the player resize, IMO.
Maybe, if for computer playback only. There aren't that many standalones that will resize (he has the MTK/Sigma option turned on).
Ideally, he should crop, anamorphically encode without resizing, resize during playback using the appropriate filters in ffdshow, use a Matroska container and all that, but that's beyond the scope of this thread.
And also useless for standalone playback.

unskinnyboy

29th April 2008, 15:16

After doing some research on CVD, you're right (as you well knew), and thanks for the correction. It's not VCD, of course, as it would have to be 352x240. And it doesn't have to be 4:3, as I was also assuming, but can be 16:9. But his is 4:3.Yes, you are right. I didn't think of looking at the log to find out the DAR of his source. Doh. If I'm not mistaken, CVD requires MP2 audio. I can't find much about the CVD specs, but my source for that statement is here:

http://forum.videohelp.com/topic98177.html?sid=1347a879052058538f1764b7dc1cd35e

And this thing has AC3 audio. Half-D1 DVD can have AC3 audio, so I suspect the source is just a Half-D1 DVD. Or maybe I'm wrong again.No, you are right. I didn't phrase my statement properly - when I said CVD, I meant just the resolution part of it. The other aspects you researched and brought up are correct - CVD can have only MP2 audio, and just to add, the audio won't play on most players (at least older ones) because of the frequency difference - CVDs' 44.1 kHz Vs. DVD standards' 48 kHz. So to be precise, he has a Half D1 with an AC3 track. Agreed.In any event, since these things get resized at playback time (the source MPEGs, I mean) then I don't really understand why letting AutoGK upsize will be any worse. Although LanczosResize upsizing might not be optimal, it's not as bad as you seem to make it out to be (in my opinion). Personally, I wouldn't upsize anything greater than the original height, and ignore the width all together.Even without considering the merits and demerits of Lanczos, unless you are encoding at the full screen resolution width (i.e. the resolution of the screen where you are viewing this in), even if you resize to 720x during encoding, isn't it going to be resized again by the player to fit the screen (unless you are watching it at 720x itself)? If you are encoding it at 352x, then it just needs to be resized once by the player. More the number of resizes, the more lossy it is. This is my theory anyway. If you think it's OK to upsize this way during the encode, with something like Lanczos (like you said), why aren't you doing it on your encodes? Isn't that because you have the feeling that it's sub-optimal?

Maybe, if for computer playback only. There aren't that many standalones that will resize (he has the MTK/Sigma option turned on).

And also useless for standalone playback.Since we were talking about resize or not, I was just mentioning the ideal way of doing it. Won't work on standalones, of course, that's why I said it was beyond the scope of this thread. Sorry about the confusion.

manono

29th April 2008, 15:48

Hi again,
...isn't it going to be resized again by the player to fit the screen (unless you are watching it at 720x itself)? If you are encoding it at 352x, then it just needs to be resized once by the player. More the number of resizes, the more lossy it is.
You seem to be saying that 352 * xxx isn't a resize. Why is increasing the width a resize but decreasing the original height not a resize? Once resized to a square-pixel 1:1 AVI, then it will just get scaled or upconverted to fit the display. No way around that (unless I start encoding everything for 1080p and then face the problem that I haven't any standalone that will be able to play it).
If you think it's OK to upsize this way during the encode, with something like Lanczos (like you said), why aren't you doing it on your encodes? Isn't that because you have the feeling that it's sub-optimal?
Hehe, I'm not sure I understand. How do you know what I do with my encodes? To be honest, though, I do very little resizing up, and when I do I use Lanczos4Resize. But I've seen only 2 352x480 DVDs in my life, both bootlegs I think, even though rented to me by a reputable on-line rental service. If I wanted to use the full 480 height I wouldn't hesitate to make it 640x480, but I usually make 512x384 XviDs from 4:3 DVDs. imfaizzi's source is a bit unusual, to say the least.

unskinnyboy

30th April 2008, 03:47

You seem to be saying that 352 * xxx isn't a resize. Why is increasing the width a resize but decreasing the original height not a resize? Once resized to a square-pixel 1:1 AVI, then it will just get scaled or upconverted to fit the display. No way around that (unless I start encoding everything for 1080p and then face the problem that I haven't any standalone that will be able to play it).Yes, 352x??? is also a resize, but hopefully of not a big magnitude. If a lot of black was cropped, and just a minimal resize was done to minimize the AR error, then I'd think that this resize is still OK. But if not a lot of black was cropped, or worse - if there weren't any black bars, then the resize AutoGK did, 352x208, is quite bad too, because now it has thrown away more than half the vertical resolution. Not having the actual source in hand to look at, I'm just speculating...

The thing that bothers me about just plain upsizing using a linear method like Lanczos, especially when trying to hit a target size is that, we are most probably wasting bits there. An upsizing by more than 2x, if he were to upsize to 720x, is sure to create some aliasing and artifacting and we are helpless to use any other filters to alleviate it. What a player/video card does is a Bilinear resize, which ain't anything dandy either, but at least we aren't wasting bits there. But now, I'd sure like imfaizzi to try this though, if he can. If he is indeed married to a 1 CD size, then heck, he has enough wiggle room to afford some blow up.

So, manono, I think I'm halfway there, agreeing with you. Upsize, I think he can, but probably not more than 2x. :)

Hehe, I'm not sure I understand. How do you know what I do with my encodes?A safe guess. Somehow I can't imagine you upsizing, hehe.

imfaizzi, can you provide a VOB sample to us? I want to see for myself what this thing looks like.