Noise preserving in x264

[bigcat]

Hi people,
I have lots of cartoons ("Looney Tunes") recorded from satellite in mpeg2 544x576 format. I didn't reencode them, but I need to shrink them at least 2x without noticeable quality loss. I thought h264 would help me, but one thing I can't get rid of is noise reduction. "Looney tunes" is quite old cartoon and it was recorded on tape. So there's lots of "old tape" noise in this cartoon. When this noise is reduced, the cartoon looks awful. I tried lots of combinations of x264 settings but couldn't find the proper one that wouldn't reduce the noise so much. In comparsion, xvid preserves all the noise but the resulting picture doesn't look very well.
So what settings (I mean cmd arguments) should I try to preserve the noise and achieve good compression level?

[Sharktooth]

look at the deadzones settings and lower them a bit. use a low subme value (preferably 1) and keep the bitrate high.
another possibility is to try the new Dark Shikari's adaptive quant (0.3). you will find in this forum.

[bigcat]

Thnaks for quick reply, I'll try something of those.

[cogman]

I don't know if you are using avisynth. If you are, you might want to browse some of the filters available. If the some noise is grainy, you might want to try Dark Shakira's (sp?) GrainOptimizer. I have also seen some good results from dfttest and fft3dfilter.

So go give some filters a run and see if any of them make a difference, use AVSP to see what they are doing to the video. Hope that helps, It took me some time to find that the type of filter you are using on your film is almost as important as the codec :)

[audyovydeo]

So what settings (I mean cmd arguments) should I try to preserve the noise and achieve good compression level?

1. what results do you get if you disable noise reduction altogether in x264 ? That doesn't stop it compressing the video.

2. MSU Cartoon Restore Filter ?

"Filter for quality improvement of compressed animated video
MSU Graphics & Media Lab (Video Group)

One of the most prominent coding artifacts in highly compressed animated video are the ringing artifacts. Ringing noise preferably manifest itself around edges due to coarse quantization. The filter is designed for quality improvement of compressed cartoons like video by deleting ringing artifacts."

http://www.compression.ru/video/cartoon_restore/index_en.htm

for VirtualDub, like most of MSU's excellent filters.


cheers
audyovydeo

[Dark Shikari]

Try my AQ. You won't be able to avoid eliminating noise without a high bitrate, but you should be able to do it without going much above 2 megabits/second.

[Chainmax]

I seem to recall that not using Trellis also helps in this regard, is that correct?

[Dark Shikari]

I seem to recall that not using Trellis also helps in this regard, is that correct?I've found it isn't so much not using trellis that helps; moreso, using low deadzones can help.

[Chainmax]

But, for example, using no-dct+Trellis0+MSP5 should preserve more grain than no-dct+Trellis2+MSP7, right?

[Dark Shikari]

But, for example, using no-dct+Trellis0+MSP5 should preserve more grain than no-dct+Trellis2+MSP7, right?No, because No-dct+Trellis0+subme5 will simply waste more bits than no-dct+Trellis2+subme7. If it does preserve more grain, I'd say its likely just because of the trellis2.

Wasted bits doesn't necessarily mean better grain retention.

[Chainmax]

So the only one of those settings that might result in slightly better detail preservation is Trellis = 1 then?

[bigcat]

I'm sorry i didn't reply for so long, it's because i couldn't log on to the forum (no PING reply either... did someone ban my IP?).
Well lowering deadzones helps, but whether I use AQ or not i still get too big bitrate (about 1900k), which gives me 1.5-1.3x compression (comparing to the recorded mp2 source). This makes such compression useless, and I decided to leave everything as is.
MSU Cartoon Filter is not intended for my problem, it's actually for deringing animation. I tried it on some other sources: 2 cartoons and 1 TV show. It works fine with cartoons, does some deringing, and when the settings are improperly tuned, it causes detail loss. And this filter doesn't work with non-animation at all. Plus it's extremely slow. But it works anyway...
//offtopic: Is there any way to decode interlaced h264 (not with CoreAVC) ?

[Sagekilla]

Lower deadzones is exactly why your bitrate is so huge: Lower deadzone settings means that it's allocating more bits to preserve high frequency detail (like noise or grain.)


Personally, I don't see a reason why anyone would want to preserve noise unless if denoising it causes an exorbitant amount of detail loss. Still, there's some quite good denoising filters and chains that you can use to avoid this. In any case, good luck.

[nm]

//offtopic: Is there any way to decode interlaced h264 (not with CoreAVC) ?
libavcodec (in ffdshow, MPlayer and VLC among others) works with most encodings. Spatial direct mode is not supported though, so there are artifacts in some cases.

The JM reference decoder should also work (but it's slow). Most commercial decoders have full support for interlaced video.