x264 File Larger Than Original MPEG-2

[Danette]

I'm starting with a cartoon that is telecined. I'm applying IVTC to it (DVD source), which leaves it with a smooth 23.976 (no duplicates). Each episode I compress turns out reverse compressed, being about 15% larger than the original MPEG-2. Although I am using an x264 quality level of 18, it should still be smaller, perhaps significantly smaller.

Can anyone explain why this would happen?

[mkver]

The encoder that produced the video on your DVD was allowed to make errors and create distortions; by using a crf value of 18 you have essentially told x264 that it should faithfully preserve the input, including any encoding artifacts and other noise that it might have. And so x264's output can be bigger than the input even when the codec used for the source is inferior to the codec used by you.

[Danette]

The encoder that produced the video on your DVD was allowed to make errors and create distortions; by using a crf value of 18 you have essentially told x264 that it should faithfully preserve the input, including any encoding artifacts and other noise that it might have. And so x264's output can be bigger than the input even when the codec used for the source is inferior to the codec used by you.

Thanks. To test this, I tried applying QTGMC to clean it up and, sure enough, the resulting file was about half the size at the same crf value of 18.

However, this causes a new problem (perhaps better posted elsewhere). The resulting file, from using QTGMC use, is very dark. I use QTGMC in nearly every film-type conversion and have never encountered this odd behavior before with non-anime source. Can this be corrected?

[kuchikirukia]

If noise is the issue, try smdegrain. QTGMC is a deinterlacer, not a denoiser, though if you have a terrible source QTGMC can clean it up. But QTMC will usually introduce ghosting on anime content.

For light noise you can start with a thsad=130, and go up from there.

[Danette]

Smdegrain didn't do much. The QTGMC package does a very good job of improving a lot of issues.

Thanks to all for your efforts. I think that the darkening problem will need to be posted elsewhere.

[benwaggoner]

And MPEG-2 doesn't have any in-loop deblocking, and has pretty characteristic patterns of those. There are filters that are really good at removing MPEG-2 artifacts which will make it easier to reencode. But telecined content gets more challenging, since field/frame encoding gives different artifact patterns. The simplest thing you could try is use DGDecode with its luma and chroma deblocking mode on. Integration into the decoder lets it handle field/frame issues, and it can weight the deblocking based on the QP of adjoining blocks, which is more efficient. It can lose some detail as well, proportionally to how heavily the MPEG-2 was encoded. There's a deringing mode as well, but that REALLY softens the image up.