Retaining strong noise on a DVD encode

[Morte66]

Normally, I don't like noise and I denoise my DVD rips with fft3dgpu() before feeding them into x264 which drops a lot of noise itself. But I have some NTSC TV DVDs ("The Shield") with loads of noise and in this particular case I'd like to keep it. This is new territory for me, and I'm a bit stuck, so I'd appreciate some general advice on what to try.

Here's a clip: http://rapidshare.com/files/13153422/shield301clip.m2v.html

Here's what I've tried so far...

- I deinterlaced it and ran it straight into x264 crf 22 deblocking -2-2 (pretty sharp). Most of the noise disappeared, except odd clumps floating around incongruously. I turned it up to crf18, and it retained more of the noise but still had this odd half-smooth/half-noisy look.

- I used some of the Xvid profiles in MeGUI and I eventually got a good, transparent-ish encode but I needed to make the output 80% bigger than the input. [I rarely use Xvid, I don't have much of a feel for it.]

- I can just copy the VOBs or remux the MPEG2 from the DVDs, but my assorted software players/decoders all glitch on scene changes. I think it's "wrong field first" (i.e not whatever the player expected) at the cuts. [I don't especially understand interlacing, I usually let MeGUI worry about removing it.]

So I wonder if anyone has general suggestions on how to proceed. I'm looking to produce mkv files which will go on a media server. I'm prepared to spend far more bitrate than normal, but it would be nice to at least get it smaller than the original. I was wondering whether a x264 CQM might help, or there are some Xvid settings I should try, or maybe it's worth trying vc-1.

Does anyone have relevant experience they're prepared to share? Thanks in advance for any help.

[Sharktooth]

use the deadzones settings to keep the noise (not yet supported by megui but you can add them into the custom commandline textbox)

[Morte66]

use the deadzones settings to keep the noise (not yet supported by megui but you can add them into the custom commandline textbox)

*blink*

Well, that one snuck under the radar. Thanks for the tip.

*tries stuff*

Hmm, it certainly encodes more noise.

--crf 22 --deadzone-intra 6 --deadzone-inter 6 is still splotches of smooth and noisy. Completely wrong.

--crf 20 --deadzone-intra 4 --deadzone-inter 4 is about right structurally, i.e. there is noise wherever there should be noise, but it's no way transparent. It looks like different noise, and doesn't quite convey proper creative intent.

At --crf 18 --deadzone-intra 0 --deadzone-inter 0 it's starting to look right, I need to A/B to find flaws. That uses about 85% of the original mpeg2 bitrate. And there's some blocking introduced. It may or may not be as good as DVD Shrink on 85%.

I could make probably this satisfactory at something like original size, if I keep going at it. OTOH it might be easier to fix the playback glitches on the MPEG2. Hmm...

[shon3i]

Maybe try to use some matrix like M4G High Detail V3.1, or try with xvid @ q 2~4

I haved similar problem long time ago, with "War of The Worlds" movie, i never saw that much amount of noise in one original movie, and i must say that xvid shows much better results.

But aslo shrinked mpeg2 is look aslo good.

[Morte66]

Maybe try to use some matrix like M4G High Detail V3.1, or try with xvid @ q 2~4

I tried x264 with M4G High Detail v3.1 and --crf 18 --deadzone-intra 0 --deadzone inter 0 --direct-none. Whilst it wasn't perfectly transparent under rigorous examination, it was good enough. I certainly preferred it to looking at the deinterlacer glitches on the DVD. But it was 130% the size of the original.

Xvid q2, same story but 180% size.

This is more bitrate than I normally use on 1080p, to get a NTSC TV DVD to look right. :scared:

I haved similar problem long time ago, with "War of The Worlds" movie, i never saw that much amount of noise in one original movie, and i must say that xvid shows much better results.

Until recently, I've thought Xvid was better for this sort of thing. But with the new deadzone settings in x264 I'm not so sure. I wish I'd known about them before I did all those encodes in the "is BluRay necessary" thread.


Anyhow, I seem to have solved this one -- I found that the ffmpeg deinterlacer in ffdshow will play the MPEG2 without problems. So I'm just going to stick with that.

[Sharktooth]

well, you can add parametric noise during playback with FFDShow though...
my preferred noise settings for SD movies are:
http://www.webalice.it/f.corriga/temp/noise.png

[Morte66]

well, you can add parametric noise during playback with FFDShow though...[/img]

I use that a lot on my normal denoised encodes, but I hit a couple of problems this time:

- If I make a denoised encode to save space and add noise later on playback, the encode is horribly blocky. I had to do spatial-temporal deblocking to get a reasonable denoised encode and that takes about 16 hours per episode. I'm looking to do five seasons of this show...

- I couldn't get ffdshow to generate noise that looked exactly like the original. It's close, but this one time I'd like to get it exact. [I'd love to use an automated grain retention tool, that analyses the source and gives parameters for a noise generator. Does anybody know of a program/filter like that?]

[Soulhunter]

I haved similar problem long time ago, with "War of The Worlds" movie, i never saw that much amount of noise in one original movie...

True, this flick was the reason I made UberSmooth back then!

http://img258.imageshack.us/img258/8232/ubersmoothwotw3c12tr.th.png (http://img258.imageshack.us/my.php?image=ubersmoothwotw3c12tr.png) http://img258.imageshack.us/img258/5466/ubersmoothwotw3c29er.th.png (http://img258.imageshack.us/my.php?image=ubersmoothwotw3c29er.png)

Here's a short clip (http://soulhunter.chronocrossdev.com/data/WOTW_Sample.avi) of the output... ^^


Bye

[shon3i]

Soulhunter, this is fantastic, can you remember what filters you use to get this?

[*.mp4 guy]

Could someone please post a clip of one of the noisy sources from the original vob, I would like to see what I can get X264 to do with it. I don't have any DVD's with noise that strong to test with myself.

[edit] soulhunter posted the script.

[edit2]

I completely missed the clip in the first post, heh, guess this post is completely pointless now.

[Soulhunter]

Soulhunter, this is fantastic, can you remember what filters you use to get this?

UberSmooth(3,3,3,3,25) (http://mf.creations.nl/avs/functions/soulhunter/UberSmooth%20v.0.4c%20%5bDeen%5d.avs) it was iirc... ^^


Bye

[raziel666]

So I tried various Ubersmooth versions (with deen and dctfun) but all of them make my vdub crash. Any ideas?

[Soulhunter]

Hummm, here it works...

- Make sure you have the right plugins (listed in the script)
- Try to play it in MPC or so n check what error message you get


Bye

[Jay Bee]

The clip you posted plays fine with both Nvidia Purevideo decoder and DScaler5. What problems were you having? And for fixing MPEG-2 streams VideoReDo is very good.

[Didée]

Hummm, here it works...

- Make sure you have the right plugins (listed in the script)

Seems that it's related to Deen. In lines 113 & 116 of UberSmooth04c, Deen is called with a radius of 0. The version of Deen I've lying around (v1.0 beta 1) doesn't like that, a minimum radius of 1 is needed. OTOH, using Deen with a radius of 0 doesn't make any sense anyway (would be a stupid no-op), so just guessing:

Perhaps UberSmooth requires a not-so-well-known newer version of Deen (where Deen's 1st parameter is interpreted differently), and you forget to tell that?

***

Quick results on my usual stress test, with your settings:

original - UberSmooth(3,3,3,3,25):

http://img410.imageshack.us/img410/4113/ubersmooth1origpn5.th.png (http://img410.imageshack.us/my.php?image=ubersmooth1origpn5.png) http://img107.imageshack.us/img107/4120/ubersmooth11proclk1.th.png (http://img107.imageshack.us/my.php?image=ubersmooth11proclk1.png)


original - UberSmooth(3,3,3,3,25):

http://img252.imageshack.us/img252/5218/ubersmooth2origbu0.th.png (http://img252.imageshack.us/my.php?image=ubersmooth2origbu0.png) http://img443.imageshack.us/img443/7427/ubersmooth22procqo7.th.png (http://img443.imageshack.us/my.php?image=ubersmooth22procqo7.png)


Not that I had a clue how to tweak it ... but while in the 1st UberSmooth is standing by it's name ;) , in the 2nd it's still overwhelmed by the grain, so tweaking seems difficult.

[Soulhunter]

The version of Deen I've lying around (v1.0 beta 1) doesn't like that...
Probably thats the reason I wrote "Deen v.1 beta2" (http://mf.creations.nl/avs/functions/soulhunter/UberSmooth%20v.0.4c%20%5bBloated%5d.avs) and not "Deen v.1 beta1" huh?

And Deen v.1 beta2 is in mfs dumpster (http://mf.creations.nl/avs/filters/) for more than 3 years now... ;P


For ya sample: I dunnu if UberSmooth fits its characteristics! I wrote it for War of the Worlds back then [double fft3d blend with different block sizes n overlapping to reduce the grid effect (http://forum.doom9.org/showthread.php?p=684321#post684321) + Deen to reduce ringing (http://forum.doom9.org/showthread.php?p=774553#post774553)] and later I extended it just for fun to improve my avs skills a little bit [you remember, the time when i bugged you with all this questions, hehe]. Thats also the reason I never released it here in the forum... However, if you give me a link to ya sample Ill look what I can do! :]


And for lines 113+116: I use the C3D mode there...

So it should use the 121 matrix, not a spatial radius of 0, huh?


From the Readme:

mode :
with "c3d", it's the weight matrix.
- 0 <121> matrix (default)
- 1 <111> matrix (softer)
with other methods, it's the spatial raduis.
default : 0


Bye

[Didée]

Oh yes, my wrong! It's that I use Deen very rarely, and even less on this rig here I'm actually sitting at - it slipped me that I had the outdated beta in the plug dir. Sorry. :o

Screens are updated, too ... looks less horrid now.

BTW, thanks for the offer, but I don't need help on that sample. I know how to treat with it. I just like to use that one, because it's really difficult: in the grainy scenes, the grain is really pretty strong & nasty. And those scenes change with those where there is little or no grain at all. You need a filter that can deal with strong grain, but does no harm when there is none. Tricky. ;)

'Nuff of babbling - we're in the AVC forum here, not Avisynth.

[Soulhunter]

I just like to use that one, because it's really difficult: in the grainy scenes, the grain is really pretty strong & nasty. And those scenes change with those where there is little or no grain at all. You need a filter that can deal with strong grain, but does no harm when there is none. Tricky. ;)

Maybe have a look at this... (http://forum.doom9.org/showthread.php?p=960228#post960228) ^^


Bye