Undersized Files. Why do I get them, and what do I do about it? Discussion [Archive]

snake-boy

22nd June 2002, 21:27

Hi,

Thanks for the GREAT explanation guys. I think I am suffering from this problem. So much so in fact that I am getting almost 100meg less than the specified size.

The explanation above makes a LOT of sence, but could anyone give a few suggestions of ways to increase the filesize (other than upping the audio rate, from say 128 to 192 for example)..?

Please have a look at my thread here for further info:

http://forum.doom9.org/showthread.php?s=&postid=144978#post144978

Thanks guys

-snake

Acaila

22nd June 2002, 23:55

Have you tried all the ways I mentioned in the first post?

snake-boy

23rd June 2002, 07:29

Hi,

When trying various combinations of:

- Increasing audio size
- Increasing resolution
- Disabling bu-directional enc.
- Using Sharp Bicubic as my resize filter

I keep getting an error message, as the encoding will just not start...

My .avs file:
----
LoadPlugin("F:\PROGRA~1\GORDIA~1\mpeg2dec.dll")
LoadPlugin("D:\WINDOWS\System32\vobsub.dll")
mpeg2source("F:\esca\esca.d2v")
crop(13,0,701,478)
VobSub("F:\esca\vts_07_0")
BicubicResize(640,480,0,0.75)
----

And my logfile can be viewed here:

http://www.sputnik.za.net/test/esca_GKnot.log

I am following the guide, and my settings are:

Bits /PixelxFrame = 0.025
Compressibility test = 55.0 % (0.374)

I have NO idea why this won't work :( Anyone have any ideas?

-snake

Acaila

23rd June 2002, 10:05

08:18:22 AM: Error: Could not open F:\esca\esca.log
Please recheck your logfile settings. Name, path, extension, etc. Something went wrong there.

snake-boy

23rd June 2002, 17:39

Yep, I KNOW something went wrong, and I did notice the line complaining that it cannot find the .log file, but I checked, and the file IS if the specified folder.
I even went as far as to delete all the files and re-create the entire project from scratch, but I get EXACTLY the same result at this point.

I really just don't know anymore :(

-snake

Zalbar

27th June 2002, 09:23

This is interesting and good information, but one thing confuses me utterly.

Why would someone want to shoot for a larger filesize? If the quality is already "maxed out," then that means it can't get any better - so all you would be doing is making it bigger.. why?

I can understand using the extra space to increase the audio/video quality by, i.e. increasing audio bitrate, or not downsizing the resolution as much, or using a higher-quality resize filter, or disabling apparently unnecessary noise/preprocessing filters.

What I don't understand is why some people seem to want to sacrifice all in the name of getting a bigger filesize for no gain in quality - such as turning off B-frames and GMC, psychovisuals (OK granted - shutting down psychovisuals *can* improve quality - a teeny amount,) or playing with the max quantizer.

I don't understand - are there some special CD-Rs out there that require a minimum filesize in order to burn or what? Isn't maxed out just what it says? Can't we be happy with having a max-quality encode which as a bonus weighed in under expectations? Use the extra space on the disc to pack away a codec install and bsplayer. At least when you are calculating, you know the filesize will never come in too big! And that's what's really important... right?

Can someone explain this odd behavior in humans to me? Or else school me in divx quality. I have only been at this a month or so.

Thanks!
-R

Acaila

27th June 2002, 10:39

What I don't understand is why some people seem to want to sacrifice all in the name of getting a bigger filesize for no gain in quality - such as turning off B-frames and GMC, psychovisuals
Having B-frames turned on decreases quality because the frames encoded as B-frame get twice the quantizer they would have gotten if they had been P-frames. It's just that this quality decrease is hardly visible that many people use it. If you have the space available, turning off B-frames WILL give you higher quality (or you can mess with the registry to get the advantage of a B-frame's greater accuracy without the higher quantizers :)).
And like you said Psycovisuals also decrease quality in certain parts of the picture. So turning that off leaves you with higher quality (but not a bigger file though).
Although I'm not sure about GMC...

Playing with the max quantizer has absolutely no effect if your movie is maxed out, but then where did we ever mention that?

There's no need in wasting space for no increase in quality, but it's just that almost everything you can think of does either in- or decrease quality.

snake-boy

27th June 2002, 18:01

Hi,

Thanks to this thread, my problem is now sorted. :)

By disabling the use of B-Frames, using a higher quality resize filter, encoding the audio at a higher rate, 160-192-224 etc , my filesize has gone from being almost 100meg UNDERsized, to being within 2meg (below) of my target size.

And of course the quality is SWEEEET! :)

Thanks guys, for all the helpful advice!

-snake

Zalbar

28th June 2002, 04:25

Originally posted by Acaila

If you have the space available, turning off B-frames WILL give you higher quality (or you can mess with the registry to get the advantage of a B-frame's greater accuracy without the higher quantizers :))

Ahh. That's interesting, I didn't know that B-frames decreased quality.

And like you said Psycovisuals also decrease quality in certain parts of the picture. So turning that off leaves you with higher quality (but not a bigger file though).

My tests show that if I have a maxed out undersized movie, and turn on psychovisuals, it does make the filesize smaller.

Playing with the max quantizer has absolutely no effect if your movie is maxed out, but then where did we ever mention that?

One person above did mention forcing the max quantizer to 4.

There's no need in wasting space for no increase in quality, but it's just that almost everything you can think of does either in- or decrease quality.

It seemed to me that the thrust of the discussion was a bit off the mark in some of the posts, and I wanted to know if we were on the same page - getting max quality within the available space. Thanks for mentioning the b-frames as I didn't that before. I'm going to see if I can search up a discussion on the registry trick you mentioned. Thanks,
-R

jonny

28th June 2002, 08:53

>One person above did mention forcing the max quantizer to 4.

It's me :D

On my tests this help, but only if you run a compressibility test and you are in the range 80%-95%.

Cheers
jonny

snake-boy

11th July 2002, 17:13

Originally posted by Zalbar
Why would someone want to shoot for a larger filesize? If the quality is already "maxed out," then that means it can't get any better - so all you would be doing is making it bigger.. why?
-R

Seems nobody has answered your question specifically, so let me give you my reason(s):

Encoding anime episodes (escaflowne), I like to have two (encoded) eps per CD-R. One english, and the other in japanese, with subs.
So, 700meg CD, 2 files, 350megs each (per episode).

So if getting a maxed file at say 200 meg, then 2x200=400megs, so 300megs on the CD is "wasted".

Sure, one may argue that who cares, since the quality is max ALREADY. But my opinion is that if I can tweak for a bigger filesize (b-frames, audio quality, resolution (larger) etc.)then why not? Sure, the gains may be insignificantly little, but still there are gains.

For those of you who aren't perfectionists, this will no doubt seem "illogical", but for those of us who inderstand this sentiment, it makes perfect sence.
Besides, if there is free space on the CD-R to fill, then the question should not be "why fill it". The question should be "why not fill it?" (Especially if there is a TINY quality gain to be had in the process.)

Why do we do it?,.. well, because we CAN and we WANT TO. It's as simple as that. - Well, that's my opinion anyway ;) -

Cheerios

-snake

cordraconis

25th July 2002, 08:54

and since we are now talking about "what shall we fill the extra space in with", I might suggest putting there another AC-3 language stream ....

I made a 2 CD "backup" of X-men, and it ended up undersized by about 25%. I tried upping the bitrate by 26% (I know, I know, ... but I *had* to try :D ), and it was still *way* too small. (only 10MB increase).
Since I already had an english AC-3, I decided to put in the French AC-3, because I remembered the filesize, and it just fitted :) )

After subripping and splitting, the whole thing fitted quite nicely.
(It's Kick-ass for brushing up my french!) :D

Maybe that's a tip: Multilingual Rips?

otoh, Saving Private Ryan, .... help! I finally decided to make a 3 CD-rip, but after muxing the AC-3, I got sync probs. (because of the 2GB limit in the windows AviSplitter [or something].) anyway, after splitting that file in 3 parts, sync was back ok.

cheers,

Cordraconis

tempe

15th August 2002, 11:43

I guess this is pretty much a matter of personal opinion, but...

what would you suggest to do with a maxed out 1-CD-movie: Which method will improve the picture quality the most?
Not using B-frames?
Higher resolution?
A combination of both?
Anything else?
Does it depend on how much unersized?

I know that's a question but it might help to make the right decision.
Bye
Tempe

Acaila

15th August 2002, 12:45

I think I would use this order:

- disable Psycovisuals
- increase resolution to 640x..
- mux original AC3 audio instead of MP3/Vorbis/Whatever
- increase resolution to 720x..
- Disable B-frames

Egrimm

26th September 2002, 20:11

I tried the compressibility check, but there wouldn't appear any values in GKnot... anyone know how this happened?

I'm ripping Anime episodes, which I'd like to be 170MB each, instead they're more like 100 mb :S

jggimi

26th September 2002, 20:30

Something went wrong. My guess: a .log file was not produced.

The test does the first pass, normally producing a .log file which is then loaded into GKnot. If your folder doesn't have a .log file with "D5" and the resolution within the file name, one was not produced.

The most likely reason for a log file not to be produced is a codec failure. There may be many causes, the most likely is that DivX 5 is either not installed, or not installed properly.

You can check it. Start Virtual Dub, then check that DivX 5 is available(Alt...Video....Compression). If not, re-install DivX5.

If it's installed, then the second most likely problem is having something improper in the .avs script, such as invalid cropping or resizing values.

Egrimm

26th September 2002, 20:35

Omg, Divx5 is actually not there. Then how did it encode the movie?

And, somehow, when I press preview, I get good quality.. when I have encoded the movie, the quality is horrible =\

*reinstalls Divx-5*

You'll hear from me again :)

jggimi

26th September 2002, 20:39

"Preview" just runs the .avs into an .avi player; it doesn't use the encoder at all. It's great for making sure that the .avs script work, or for playing with filter values within the scripts.

But it has nothing to do with DivX, per se. AviSynth .avs files are text files you can edit (or Gknot can), that act just like media files in an .avi player.

Egrimm

26th September 2002, 21:33

Alright... this time I got a 5MB file =\

jggimi, could you contact me on ICQ / MSN please? (27989063 / egrimm@dds.nl)

Agent-X

8th December 2002, 18:21

I had a similar issue with the Panic Room dvd
I aimed to get a 720 mb file but it turned out to be 499mb..huge difference

I cant believe movies can differ that much but the quality is very acceptable, too bad I deleted the vobs by now.

What I learn from the posts here is that the resolution amounts to more detail as long as the bitrate is around .18-.24?

I was always very focused on the bits and less about the resolution (never lower then 5xx X ...)

Another question, how big a size percentage will disabling the b-frame produce?..and if its accaptable WILL it show? (quality)

Hi all btw, first post here :)

Acaila

8th December 2002, 20:45

Hi, and welcome to this forum (always great to see another dutch person posting here :)).

What I learn from the posts here is that the resolution amounts to more detail as long as the bitrate is around .18-.24?Higher resolution means you get more detail at the cost of quality. Which means that with a high bitrate you can use a higher resolution to get a more detailed picture that doesn't exhibit compression artefacts.
If the movie gets undersized then for that detail level the movie has reached maximum quality. Logically, increasing the detail will increase the filesize for maximum quality. Panic Room has been widely agreed upom as a very compressible movie (low motion, dark movie), so just increase the resolution.

Another question, how big a size percentage will disabling the b-frame produce?..and if its accaptable WILL it show? (quality)Generally, B-frames increase compressibility by about 20-25%. This varies with every movie though, so it's difficult to predict. Disabling this setting on an undersized movie theoretically increases the quality, because B-frames get compressed more than other frame types. Whether or not the effect is visible...you can be the only judge of that. Some people can see it clearly, others can't. Most people say that the effect of B-frames can be most clearly seen on high motion scenes, so those should have better quality with B-frames turned off (but since Panic Room hardly has any of those I doubt you'd be able to notice a difference, short of a filesize increase).

Agent-X

8th December 2002, 21:57

Thank you for the quick reply, it would seem you certainly know what you`re talking about. Nice to see I am not the only flatlander around here as well ;)

What I still find confusing is the difference between detail and quality, could you enlighten me on that please?

Also, I use the compressibility test mostly but to be honest, even if the rate is 50 or 70, I can`t really tell what it`ll do with the output. Is it that simple that the lower it gets, the higher I can set the resolution/bits?

Sorry if these are basic questions but I`m not sure about them.

Acaila

9th December 2002, 08:44

What I still find confusing is the difference between detail and quality, could you enlighten me on that please?Quality is how well the compression progress went. High compression will show artifacts like blocks and noise surrounding edges (=ringing).
Detail, is how much you can see in a picture, and has nothing to do with quality.

e.g. take a frame resize it to 100x100 and compress it. You'll most probably have very high quality, because it's very easy to compress such a small frame without compression artifacts. However the amount of detail that remain in this picture will be very low because it's been rsized to such a small size.

That's how DivX5 compression works as well, and why we have the compresibility test: the smaller the resolution the fewer artifacts, BUT you'll also decrease detail. So always it is a struggle to find a balance between artifacts and level of detail.

Is it that simple that the lower it gets, the higher I can set the resolution/bits?The compressibility test meaures what the quality will be at a given resolution. So if you get a low percentage then that means you get a low quality movie. To INcrease the quality you should DEcrease the resolution (thereby DEcreasing the level of detail as well, that's the price you have to pay).
General guidelines are to aim for a comp. test result of around 60%, preferably higher. That way your movie will be high enough quality that it will hardly have any artifacts. Noise filters increase this percentage artificially by selectively reducing noise which counts as a form of detail (but one you DON'T want to have in your final product).

BoNz1

13th December 2002, 01:24

I think that most people already know this, but just for the benefit of those who do not I will say this because I don't think anyone has yet. If the rc averaging period is set to half the # of frames you will get much better filesize predictability and will have less problems with undersized files. I have been doing this for a long time now and I can always get very close to the filesize I want like within about 3-4MB.

crusty

18th April 2003, 21:07

Panic Room has been widely agreed upom as a very compressible movie (low motion, dark movie), so just increase the resolution.
I can agree with this totally. The point is that turning off B-frames and Psychovisuals will have almost NO effect on increasing the quality of a dark movie, they just increase filesize.

Some other things you could use:
--try a sharpen filter in your avs file, like Asharp, Msharp or Unfilter and use it to SLIGHTY sharpen the movie. I find that many DVD sources tend to be a bit unsharp and I like crispness very much.
You will add bits to make the movie look much less washed out.
Sharpening filters are much more potent than using another resize filter.
--Removing noise filters will decrease compressibility. While some noise filters are very intelligent many of them will not only remove noise but some fine details as well. So without the noise filters you well get a bigger file size AND finer detail.
--Use a filter like Blockbuster to add EXTRA noise to the movie. This will increase file size, but will get rid of any darkblocks in your movie.
--Increase brightness and/or contrast juuust a bit. Increasing brightness tends to washout your movie but if you apply only light brightening the dark parts of your movie *might* look better.

With all these options the key is: Don't overdo it!
Consider it like the use of make-up: Your aim is to enhance the movie without making it *look* enhanced.

So in general if you get an undersized file you can start playing with the image to make it look cleaner, less blocky in dark scenes and sharper. But I really have to agree that the first thing you want is a bigger resolution. But do NOT go over the original resolution.
Also imagine that you might be viewing your encoding now on your 15" monitor but when someday you get a HDTV you will really appreciate the extra resolution.

CdinT

2nd June 2003, 01:11

Don't know if it's a known fact but here it is :)

With Divx 5.05 and 3-passes you can reach the EXACT desired size even if the comp. test it's up to 99%!

Archivist

5th November 2005, 05:10

I think I'm running up against this problem, but is there a sure way to tell?
Is there some message I can look for? I'm using 733 KBit/S for my bitrate
and coming out 130MB short. The quality is not very good.

But if the codec is "maxing" out at 733 KBitS and the quality
is worse than the DVD original, then does this imply that the codec
is not using a VBR algorithm? Is there a VBR options for mpeg4?

Or could this be due to some frame-dropping problem?
It seems that every other line of my mencoder output gives me this warning:
1 duplicate frame(s)!
Maybe that is normal.

I'm running MPlayer 1.0pre7try2-3.4.5 on Debian Linux.

Yours,
Arch

P.S.
I'm trying to encode "Logan's Run" to 700MB CDR.
I'm using "mpeg4" vcodec. The resulting file was 130MB under my estimate!
The quality was bad, so I upped the bitrate and got a much better video.
For example, I'm trying to fill a 700MB CDR -- officially 737280000 bytes.
My audio stream takes up 84145644 bytes, so that leaves 653134356 bytes.
I figured the formula for estimating bitrate like this:
bitrate = target_size / (125 * length_seconds)
That gives me a bitrate of 733 KBit/S. I then run mencoder like this
using 733 for vbitrate:
mencoder dvd://1 -o video.avi -nosound -ovc lavc -lavcopts vcodec=mpeg4:vbitrate=733:mbd=2:vpass=1:keyint=12
mencoder dvd://1 -o video.avi -nosound -ovc lavc -lavcopts vcodec=mpeg4:vbitrate=733:mbd=2:vpass=2:keyint=12

Your_Idol

8th September 2006, 14:37

dvd5 are the same price as cds
why does anyone still use cds?

Trahald

18th October 2006, 01:22

i use them for audio cds...

per video... i suppose there are some people who dont have dvd burners. also since xvid/divx movies can routinely fit on one or 2 cds, with dvd either you have wasted space or have to put multiple videos on 1 disk. so cataloging is an issue.

on average dvd5 are about 20-33% more than equivelant quality cds. thats not much more but in large quantities it makes a difference