crusty
29th April 2003, 23:35
I originately wanted to reply to the thread about encoding 2.30+ hour movies on 1 CD, but it sort of got a bit out of hand, so I decided to post it as a new thread. Please don't bash me if I say something wrong, let's try to be constructive here...
RIPS: 1 CD OR 2 CD? THAT'S THE QUESTION...
First: Perception is everything
-you have to make a choice wheter you're going to watch this rip only on TV, on your monitor from far away, or on your monitor from close up. This alone makes a huge difference in perception. What is ok on TV (which you usually watch from a distance anyway) can be totally crap on your monitor(or on my 19", that's for sure:D ).
If you choose to rip only for TV you can get away with just about anything, so that's hardly an effort. And I don't do that, so I'll stop talking about that...right ....NOW.
;)
If you're going for, let's say, a long-distance monitor rip, you can get away with some blocking, especially in fast-motion scenes, because you will not notice it. You can also get away with some notable colour-gradients that you would find irritating on close-up, and some posterizing (that is when, at slight panning of the scene, some objects that are supposed to move along with the panning don't do that properly but have a tendency to stay put or move in shocky or snappy way).
Because you're to far away to see them you can get away with these artifacts, and you could use heavy filtering, improving compressibility alot and making more 1 CD rips possible.
I personally don't go for both options and I throw away any TV-only rips that I come accross, unless it's an EXTREME oddity.
Offcourse this is only the video part of your movie.
You also have to make a choice about the quality of your audio.
If your ears are crap from going to all those metal concerts and standing to close to the speakers, only clean your ears every other year, or you just have the habit of banging your head against the wall now and then, then I suggest a low bitrate and a lowpass filter to cut off any high frequencies you can't hear anyway.
Cutting off high frequencies (which is what a lowpass filter does, somewhat contra-intuitive to the name) will give the frequencies you CAN hear more bits, ergo more quality.
Then again for 1 CD Rips you should always go for ogg vorbis sound because it's always smaller at same quality as mp3 and 1 CD ogm's work very well, while 2 cd ogm's can have some issues (atleast, in my experience). But we're not talking about 2 CD Rips here.
Also, Ogm's (which is what avi video and ogg audio are muxed into) have smaller file overhead than avi and mp3, which will gain you up to 8 MB for your content.
If you like AC3, then 1 CD rips are hardly an option for you.
Another audio-tip: check the original AC3 if it is mono or not. Encoding in mono will save you bits...lots of bits. Check the audio forum for how to do that.
So, let's say we've decided we want to try to make a good 1 CD Rip, i.e. one that will look good on your monitor. I'll assume here you have a 17" monitor. I have a 19" so my standards are a bit higher. If you have a laptop with a 14", you should always aim for future monitor growth...otherwise you're gonna slap yourself when you get that funky new 21"tft screen in three years time.
(if you do, please slap yourself hard then for not listening and pm me with a description of the effect :D )
Now that we've got that out of the way we still have to look at your psyche...well the visual part at least. Visual distortion that's irritating to you could be perfectly acceptable to someone else, and vice versa. So if you encode something and want to make sure it looks good, let someone else look at the results as well. Then again you could be overcritical of yourself and your work and get irritated about artifacts that nobody IRL gives a crap about.:)
Since we don't have that under any objective control, that's basically up to you and your friends to decide.
Now we're finally getting to the source, always look at the source first, fully and complete. Even if it is a movie you've seen dozens of time (or hundreds of times like some daylight-deprived geeks I know) the DVD you've just bought can have some peculiarities which you should know about.
Basically three important things spring to mind:
1, Noise 2, telecining/interlaced/hybrid material and 3, movie-to-DVD issues.
1)-Noise
Noise is ugly, kills compressibility and in general ruins your day.
Some people say they actually LIKE the noise, but you can add that easily while watching with a so-called postprocessing filter like ffdshow (search for it in the forum) or the 'film effect' in Divx 5.0x. Some people actually ADD noise while encoding to mpeg4 to kill any 'darkblocks', but in general, unless you have a specific need for noise, you should get rid of it.
2)-The funky telecining and interlacing stuff.
This actually a rather difficult issue and I do not pretend to fully comprehend it either. What I can say is that if you're source is Interlaced and you do not deinterlace properly, you will end up with a bigger and worse looking divx than if you did do it. Why?
Because not deinterlacing will introduce extra combing effects that need to be encoded AND are visible and irritating. So you need more bits and the movie will look worse.
And then you have the telecined and hybrid DVD's making life even more difficult.
For a better understanding of this you should take a look at the following two links:
The IVTC and Deinterlacing sticky in the DVD2AVI forum:
http://forum.doom9.org/showthread.php?s=&threadid=34696
And 100FPS.com for lots of info and lots of funny pictures:
http://www.100fps.com
3)-DVD mastering stuff.
I know of two issues that have an impact on compressibility:
Is the DVD source sharp, detailed, or is it on the soft, blurry side of things?
Like resize and sharpen filters, this has a serious impact on compressibility.
The next one is a bit difficult to describe. It's when the whole movie sort of 'dances' around on your screen. It's usually especially notable at start credits. You will see the start credits come up on your screen, en then it 'wavers' around a bit, like as if it was on a ship on a rolling sea. This unwanted motion has to be encoded as well, so it just wastes bits.
OK, now that we've got that stuff out of the way, we should look at the TYPE of movie.
-Is it a dark or a light movie?
Darkness compresses very well because our eyes will see less detail in it and you can easily fit longer dark movies on CD than light movies.
-Is it a fast motion movie or a low motion movie?
In a fast motion scene you will see less detail than in a low motion scene. The codecs know that and will make them blockier than low motion scenes. It also influences the choice of filters to use, because many filters work with some form of motion estimation, and like you in traffic, they can sometimes get it wrong, and the result is just as horrible. In the case of divx 3 (which I don't recommend using) it would also influence the choice of codec.
-Has the movie a lot of scenes with the 'classic mpeg4 torture test material' in it?
This is mighty important, because it can mean the difference between a 1 CD or 2 CD encode more than many other things.
Classic mpeg4 torture test material consists of stuff that's usually very vague and very fast moving, yet needs a lot of bits to conserve detail to the human eye. Fog, Smoke, Dust, Rain, Explosions (especially those with lots and lost of bits and pieces flying around) are extremely hard for mpeg4 to get right. They also suffer very quickly from filtering because you will notice the loss of fine details much earlier than the codec will. So the codec will think you won't see it compressing certain stuff that you really do. (codecs should have a 'slap!' option just for this) :)
That's a major part of the reason why Doom9 uses Saving Private Ryan in his codec comparisons. It's got major amounts of this type of content, so any flaws in the codecs will quickly show up.
-Has the movie a lot of panning, zooming and scanning in it?
This is important to know if you planning to use this funky little feature called GMC or Global Motion Estimation. GMC will help compression on any scene that has a lot of stuff moving in some mathematical correlated way. In plain broken internet-english that means it's all moving in one direction, like a pan, or it's zooming in or out of your field of view.
Typical examples of a pan are (of the top of my head)...erm can't think of one right now...but you get the idea :D
While a good example of a zooming scene is the part in 2001: A Space Odysee where the astronaut has his freaky transdimensional trip.
In some cases, turning on GMC will have a positive effect, while in other cases it will just eat up bits. If you have a static movie you could probably live without GMC.
-Is it a real movie, a cartoon, a combination (or just plain weird)?
Mpeg4 is heavily optimized for real world movies, i.e. movies with real people in real places in them. Cartoons and other animated stuff like Anime have completely different properties. They often have very solid looking coloured parts combined with very sharply contrasting black borders all over a single frame. Mpeg4 doesn't like this one bit and has a significant tendency to produce artifacts just for the heck of it. Also, because there is such sharp contrast you will see artifacts created by lowering the resolution much quicker than with real life movies. In general this is considered to be one of the most difficult type of movie to encode. Fortunately, there are plenty of filters around that are specifically designed to enhance cartoons and animes, and the results can be quite stunning.
Unfortunately you also have the combination of both which can be even harder to encode. Typical examples of this are movies like 'Space Jam', 'Tron', and 'Who Framed Roger Rabbit?'. Filtering will be much more limited because filters that are designed for anime >might< create distortion on everything in the movie that's real world footage. And offcourse vice versa.
Note: special effects are considered real-world footage for this article. Unless they are computer generated they will usually have the same noise, colourdepth and contrast as real-world footage.
-Has it a lot of fine detail in it's content?
If the movie has got footage which contains a lot of fine detail then it will be harder to encode then when it hasn't.
Say for instance 'Platoon' which has a lot of jungle in it. This consists of very fine curved leaves which waver around a lot, generally not all in the same direction.
Another example is footage of walls with very intricate details, like wallpaper covered with delicate repeating flowers.
-Is it black or white, or is it a colour movie?
Black and white movies are supposed to have no colour information, so in theory it should be much less information to store, especially since some codecs can encode in grayscale (xvid for instance).
You will however see many B/W movies that have a kind of yellowish tint to it, because they're old films and the original celluloid either was that colour or has become yellowish from aging.
You then have to decide wheter you want to save that colour information or if you want remove it altogether. Also, B/W movies often suffer from mastering errors and lots of noise as well.
-Has it a lot of scene changes?
You might think it's not such a big issue, but wait until you get a movie that has about 10 to 15 scene changes per MINUTE, and you will quickly learn the hard way.
----------------------------------------------------------
By now you should have an idea of whether or not the movie you'll want to rip will fit on 1 CD or not.
If your movie has a lot of the difficult to encode content, you will have to fit less of the movie on 1 CD, lower the resolution, or live with a worse looking video.
Let's go to some examples:
Dark City:
Low motion, very dark, pure real-world content. 100 minutes.
Will easily fit on one CD unfiltered at high resolution
Terminator:
Dark movie, with plenty of high action scenes. Still, will look very good at good resolutions.
The Return of the Living Dead:
Dark movie, low motion, VERY clean celluloid and good DVD-mastering.
About 90 minutes. Mono audio.
With the use of the avisynth filters Undot and Unfilter , and encoding to mono audio, this rip looks both sharper and cleaner than the original DVD at 640 resolution.
El Mariachi:
Bright movie, lots of action scenes, about 90 minutes, lots of scene changes, and a huge amount of noise . 81 minutes.
Will not look good on 1 CD unless you do a lot of filtering.
Tron:
Reasonably dark movie, but a combination of computer graphics and digitally edited real world footage. 96 minutes.
Even though this is a relativily short movie, it's difficult to fit on 1 CD with good quality. The sharp contrast and generally artifialness of the movie will not compress well.
Mad Max 2:
Very bright movie, 70-80% fast motion scenes, lots of dust, smoke, explosions, car pursuits, only about 3 minutes dark scenes, about 90 minutes. Very noisy footage. And did I mention scene changes?
I dare you to fit this on 1 CD. Took me 2 months, latest xvid builds, Ogg vorbis sound, heavy filtering and an 800 MB file. But it looks GOOD! :cool:
Well that's about it for now, coz' I'm frigging tired of three hours typing. Feel free to post suggestions, corrections or some of your own experiences. :)
RIPS: 1 CD OR 2 CD? THAT'S THE QUESTION...
First: Perception is everything
-you have to make a choice wheter you're going to watch this rip only on TV, on your monitor from far away, or on your monitor from close up. This alone makes a huge difference in perception. What is ok on TV (which you usually watch from a distance anyway) can be totally crap on your monitor(or on my 19", that's for sure:D ).
If you choose to rip only for TV you can get away with just about anything, so that's hardly an effort. And I don't do that, so I'll stop talking about that...right ....NOW.
;)
If you're going for, let's say, a long-distance monitor rip, you can get away with some blocking, especially in fast-motion scenes, because you will not notice it. You can also get away with some notable colour-gradients that you would find irritating on close-up, and some posterizing (that is when, at slight panning of the scene, some objects that are supposed to move along with the panning don't do that properly but have a tendency to stay put or move in shocky or snappy way).
Because you're to far away to see them you can get away with these artifacts, and you could use heavy filtering, improving compressibility alot and making more 1 CD rips possible.
I personally don't go for both options and I throw away any TV-only rips that I come accross, unless it's an EXTREME oddity.
Offcourse this is only the video part of your movie.
You also have to make a choice about the quality of your audio.
If your ears are crap from going to all those metal concerts and standing to close to the speakers, only clean your ears every other year, or you just have the habit of banging your head against the wall now and then, then I suggest a low bitrate and a lowpass filter to cut off any high frequencies you can't hear anyway.
Cutting off high frequencies (which is what a lowpass filter does, somewhat contra-intuitive to the name) will give the frequencies you CAN hear more bits, ergo more quality.
Then again for 1 CD Rips you should always go for ogg vorbis sound because it's always smaller at same quality as mp3 and 1 CD ogm's work very well, while 2 cd ogm's can have some issues (atleast, in my experience). But we're not talking about 2 CD Rips here.
Also, Ogm's (which is what avi video and ogg audio are muxed into) have smaller file overhead than avi and mp3, which will gain you up to 8 MB for your content.
If you like AC3, then 1 CD rips are hardly an option for you.
Another audio-tip: check the original AC3 if it is mono or not. Encoding in mono will save you bits...lots of bits. Check the audio forum for how to do that.
So, let's say we've decided we want to try to make a good 1 CD Rip, i.e. one that will look good on your monitor. I'll assume here you have a 17" monitor. I have a 19" so my standards are a bit higher. If you have a laptop with a 14", you should always aim for future monitor growth...otherwise you're gonna slap yourself when you get that funky new 21"tft screen in three years time.
(if you do, please slap yourself hard then for not listening and pm me with a description of the effect :D )
Now that we've got that out of the way we still have to look at your psyche...well the visual part at least. Visual distortion that's irritating to you could be perfectly acceptable to someone else, and vice versa. So if you encode something and want to make sure it looks good, let someone else look at the results as well. Then again you could be overcritical of yourself and your work and get irritated about artifacts that nobody IRL gives a crap about.:)
Since we don't have that under any objective control, that's basically up to you and your friends to decide.
Now we're finally getting to the source, always look at the source first, fully and complete. Even if it is a movie you've seen dozens of time (or hundreds of times like some daylight-deprived geeks I know) the DVD you've just bought can have some peculiarities which you should know about.
Basically three important things spring to mind:
1, Noise 2, telecining/interlaced/hybrid material and 3, movie-to-DVD issues.
1)-Noise
Noise is ugly, kills compressibility and in general ruins your day.
Some people say they actually LIKE the noise, but you can add that easily while watching with a so-called postprocessing filter like ffdshow (search for it in the forum) or the 'film effect' in Divx 5.0x. Some people actually ADD noise while encoding to mpeg4 to kill any 'darkblocks', but in general, unless you have a specific need for noise, you should get rid of it.
2)-The funky telecining and interlacing stuff.
This actually a rather difficult issue and I do not pretend to fully comprehend it either. What I can say is that if you're source is Interlaced and you do not deinterlace properly, you will end up with a bigger and worse looking divx than if you did do it. Why?
Because not deinterlacing will introduce extra combing effects that need to be encoded AND are visible and irritating. So you need more bits and the movie will look worse.
And then you have the telecined and hybrid DVD's making life even more difficult.
For a better understanding of this you should take a look at the following two links:
The IVTC and Deinterlacing sticky in the DVD2AVI forum:
http://forum.doom9.org/showthread.php?s=&threadid=34696
And 100FPS.com for lots of info and lots of funny pictures:
http://www.100fps.com
3)-DVD mastering stuff.
I know of two issues that have an impact on compressibility:
Is the DVD source sharp, detailed, or is it on the soft, blurry side of things?
Like resize and sharpen filters, this has a serious impact on compressibility.
The next one is a bit difficult to describe. It's when the whole movie sort of 'dances' around on your screen. It's usually especially notable at start credits. You will see the start credits come up on your screen, en then it 'wavers' around a bit, like as if it was on a ship on a rolling sea. This unwanted motion has to be encoded as well, so it just wastes bits.
OK, now that we've got that stuff out of the way, we should look at the TYPE of movie.
-Is it a dark or a light movie?
Darkness compresses very well because our eyes will see less detail in it and you can easily fit longer dark movies on CD than light movies.
-Is it a fast motion movie or a low motion movie?
In a fast motion scene you will see less detail than in a low motion scene. The codecs know that and will make them blockier than low motion scenes. It also influences the choice of filters to use, because many filters work with some form of motion estimation, and like you in traffic, they can sometimes get it wrong, and the result is just as horrible. In the case of divx 3 (which I don't recommend using) it would also influence the choice of codec.
-Has the movie a lot of scenes with the 'classic mpeg4 torture test material' in it?
This is mighty important, because it can mean the difference between a 1 CD or 2 CD encode more than many other things.
Classic mpeg4 torture test material consists of stuff that's usually very vague and very fast moving, yet needs a lot of bits to conserve detail to the human eye. Fog, Smoke, Dust, Rain, Explosions (especially those with lots and lost of bits and pieces flying around) are extremely hard for mpeg4 to get right. They also suffer very quickly from filtering because you will notice the loss of fine details much earlier than the codec will. So the codec will think you won't see it compressing certain stuff that you really do. (codecs should have a 'slap!' option just for this) :)
That's a major part of the reason why Doom9 uses Saving Private Ryan in his codec comparisons. It's got major amounts of this type of content, so any flaws in the codecs will quickly show up.
-Has the movie a lot of panning, zooming and scanning in it?
This is important to know if you planning to use this funky little feature called GMC or Global Motion Estimation. GMC will help compression on any scene that has a lot of stuff moving in some mathematical correlated way. In plain broken internet-english that means it's all moving in one direction, like a pan, or it's zooming in or out of your field of view.
Typical examples of a pan are (of the top of my head)...erm can't think of one right now...but you get the idea :D
While a good example of a zooming scene is the part in 2001: A Space Odysee where the astronaut has his freaky transdimensional trip.
In some cases, turning on GMC will have a positive effect, while in other cases it will just eat up bits. If you have a static movie you could probably live without GMC.
-Is it a real movie, a cartoon, a combination (or just plain weird)?
Mpeg4 is heavily optimized for real world movies, i.e. movies with real people in real places in them. Cartoons and other animated stuff like Anime have completely different properties. They often have very solid looking coloured parts combined with very sharply contrasting black borders all over a single frame. Mpeg4 doesn't like this one bit and has a significant tendency to produce artifacts just for the heck of it. Also, because there is such sharp contrast you will see artifacts created by lowering the resolution much quicker than with real life movies. In general this is considered to be one of the most difficult type of movie to encode. Fortunately, there are plenty of filters around that are specifically designed to enhance cartoons and animes, and the results can be quite stunning.
Unfortunately you also have the combination of both which can be even harder to encode. Typical examples of this are movies like 'Space Jam', 'Tron', and 'Who Framed Roger Rabbit?'. Filtering will be much more limited because filters that are designed for anime >might< create distortion on everything in the movie that's real world footage. And offcourse vice versa.
Note: special effects are considered real-world footage for this article. Unless they are computer generated they will usually have the same noise, colourdepth and contrast as real-world footage.
-Has it a lot of fine detail in it's content?
If the movie has got footage which contains a lot of fine detail then it will be harder to encode then when it hasn't.
Say for instance 'Platoon' which has a lot of jungle in it. This consists of very fine curved leaves which waver around a lot, generally not all in the same direction.
Another example is footage of walls with very intricate details, like wallpaper covered with delicate repeating flowers.
-Is it black or white, or is it a colour movie?
Black and white movies are supposed to have no colour information, so in theory it should be much less information to store, especially since some codecs can encode in grayscale (xvid for instance).
You will however see many B/W movies that have a kind of yellowish tint to it, because they're old films and the original celluloid either was that colour or has become yellowish from aging.
You then have to decide wheter you want to save that colour information or if you want remove it altogether. Also, B/W movies often suffer from mastering errors and lots of noise as well.
-Has it a lot of scene changes?
You might think it's not such a big issue, but wait until you get a movie that has about 10 to 15 scene changes per MINUTE, and you will quickly learn the hard way.
----------------------------------------------------------
By now you should have an idea of whether or not the movie you'll want to rip will fit on 1 CD or not.
If your movie has a lot of the difficult to encode content, you will have to fit less of the movie on 1 CD, lower the resolution, or live with a worse looking video.
Let's go to some examples:
Dark City:
Low motion, very dark, pure real-world content. 100 minutes.
Will easily fit on one CD unfiltered at high resolution
Terminator:
Dark movie, with plenty of high action scenes. Still, will look very good at good resolutions.
The Return of the Living Dead:
Dark movie, low motion, VERY clean celluloid and good DVD-mastering.
About 90 minutes. Mono audio.
With the use of the avisynth filters Undot and Unfilter , and encoding to mono audio, this rip looks both sharper and cleaner than the original DVD at 640 resolution.
El Mariachi:
Bright movie, lots of action scenes, about 90 minutes, lots of scene changes, and a huge amount of noise . 81 minutes.
Will not look good on 1 CD unless you do a lot of filtering.
Tron:
Reasonably dark movie, but a combination of computer graphics and digitally edited real world footage. 96 minutes.
Even though this is a relativily short movie, it's difficult to fit on 1 CD with good quality. The sharp contrast and generally artifialness of the movie will not compress well.
Mad Max 2:
Very bright movie, 70-80% fast motion scenes, lots of dust, smoke, explosions, car pursuits, only about 3 minutes dark scenes, about 90 minutes. Very noisy footage. And did I mention scene changes?
I dare you to fit this on 1 CD. Took me 2 months, latest xvid builds, Ogg vorbis sound, heavy filtering and an 800 MB file. But it looks GOOD! :cool:
Well that's about it for now, coz' I'm frigging tired of three hours typing. Feel free to post suggestions, corrections or some of your own experiences. :)