Not so grainy source, too much encoding time

[Zyklon B]

Hi,

During the past few weeks I've been trying to rip some DVDs I had at home using x264 & megui just for fun. Most of them are/were really old movies with crappy quality, so I had to "clean" them a lot using Avisynth.

But now I'm trying to rip a Chinatown DVD, which is grainy but not so terrible. The problem is my Avisynth Script takes way too long to encode (~48h encoding time for a 2pass). The script I am using to clean, which I found here at doom9, uses MVTools and is the following:

SetMemoryMax(1024)
AssumeTFF()
source=sEPARATEfIELDS()
backward_vec1=source.MVAnalyse(isb = true, delta = 2, pel = 2, blksize=8, overlap=4, sharp=1, idx = 1, dct=0, chroma=false)
forward_vec1=source.MVAnalyse(isb = false, delta = 2, pel = 2, blksize=8, overlap=4, sharp=1, idx = 1, dct=0, chroma=false)
backward_vec2=source.MVAnalyse(isb = true, delta = 4, pel = 2, blksize=8, overlap=4, sharp=1, idx = 1, dct=0, chroma=false)
forward_vec2=source.MVAnalyse(isb = false, delta = 4, pel = 2, blksize=8, overlap=4, sharp=1, idx = 1, dct=0, chroma=false)
backward_vec3=source.MVAnalyse(isb = true, delta = 6, pel = 2, blksize=8, overlap=4, sharp=1, idx = 1, dct=0, chroma=false)
forward_vec3=source.MVAnalyse(isb = false, delta = 6, pel = 2, blksize=8, overlap=4, sharp=1, idx = 1, dct=0, chroma=false)
chroma=source.MVDegrain3(backward_vec3,backward_vec2,backward_vec1,forward_vec1,forward_vec2,forward_vec3,thSAD=1200,idx=1,plane=3)# try tune thSAD
source.MVDegrain3(backward_vec3,backward_vec2,backward_vec1,forward_vec1,forward_vec2,forward_vec3,thSAD=800,idx=1, plane=0)# try tune thSAD
MergeChroma(chroma)
Weave()


Here is how the original source looks like:

http://xs233.xs.to/xs233/08473/vlcsnap-4685999715.png


And here is how it would look like after the encoding process using said script:

http://xs233.xs.to/xs233/08473/no_noise730.png


Full script I'm using this time is:

DGDecode_mpeg2source("D:\CHINATOWN\VIDEO_TS\VTS_01_1.d2v",info=3)
ColorMatrix(hints=true)
#deinterlace
crop( 2, 70, -2, -68)

Lanczos4Resize(720,304) # Lanczos4 (Sharp)
Undot() # Minimal Noise (for compressibility)

SetMemoryMax(1024)
AssumeTFF()
source=sEPARATEfIELDS()
backward_vec1=source.MVAnalyse(isb = true, delta = 2, pel = 2, blksize=8, overlap=4, sharp=1, idx = 1, dct=0, chroma=false)
forward_vec1=source.MVAnalyse(isb = false, delta = 2, pel = 2, blksize=8, overlap=4, sharp=1, idx = 1, dct=0, chroma=false)
backward_vec2=source.MVAnalyse(isb = true, delta = 4, pel = 2, blksize=8, overlap=4, sharp=1, idx = 1, dct=0, chroma=false)
forward_vec2=source.MVAnalyse(isb = false, delta = 4, pel = 2, blksize=8, overlap=4, sharp=1, idx = 1, dct=0, chroma=false)
backward_vec3=source.MVAnalyse(isb = true, delta = 6, pel = 2, blksize=8, overlap=4, sharp=1, idx = 1, dct=0, chroma=false)
forward_vec3=source.MVAnalyse(isb = false, delta = 6, pel = 2, blksize=8, overlap=4, sharp=1, idx = 1, dct=0, chroma=false)
chroma=source.MVDegrain3(backward_vec3,backward_vec2,backward_vec1,forward_vec1,forward_vec2,forward_vec3,thSAD=1200,idx=1,plane=3)# try tune thSAD
source.MVDegrain3(backward_vec3,backward_vec2,backward_vec1,forward_vec1,forward_vec2,forward_vec3,thSAD=800,idx=1, plane=0)# try tune thSAD
MergeChroma(chroma)
Weave()

I tried to use Undot to remove the noise but it fails completely. Also tried using mc_spuds, but didn't work (anyway I'd read that it's also quite slow). If you need it, I can upload a sample of the source, of course.

Any thoughts on this? Thanks in advance. And sorry for my crappy english :( .

[Blue_MiSfit]

For dog slow filter chains like this, it's often helpful to "render" to a lossless codec like HuffYUV (ffdshow's YV12 variant, preferably) - or Lagarith.

Then, do a vanilla 2 pass encode from that source. It usually ends up being faster than having to do the filters twice ;)

~MiSfit

[Dark Shikari]

fft3dgpu is a nice fast denoising filter. Not as good quality as your chain though.

[Blue_MiSfit]

Let's see a sample.

Undot will almost never produce visible results, and usually just slightly improves compressibility.

MC_Spuds is an uber MVDegrain script IIRC.

I've had fantastic luck with the new MT enabled build of MVTools2. MVDegrain2 runs quite quickly on this, and for SD material I can get 8fps encoding into CRF mode with high settings on my Q6600. This produces very nice results. Strangely, I've also noticed that I have to tweak the thsad down a lot from the documentation's default of 400.

fft3dgpu is my go-to guy for standard filtering :) Adjusting sigma values is mandatory!

~MiSfit

[Zyklon B]

Here's a 5min clip of the source "as is", with no editing whatsoever, ~900kbps bitrate xvid. As you can see, I'm using Lanczos4 as a resizing filter because the source is kinda blurry at times. You can also see that the grain is noticeable but doesn't really hurt a lot. Oh BTW, forgot to say that the 1.Xfps encoding speed with the script is using a 3.2GHz Core2Duo / NV9800GT.

http://www.mediafire.com/?lzokow2umdt


@Blue_MiSfit: would you mind going a bit further in your explanation about rendering the video to HuffYUV first? I'm not sure if I'm understanding it correctly. Should I encode the video using HuffYUV and then use that rendered file as the source for the x264 encoding? Then there would be no need to add the filter chain at the x264 encoding, would it?

@all: Thanks for the tip, I'll have a look at the documentation of fft3dgpu and see if I don't get too lost :P. Any tips on how to start with it?

Thanks again.

[Blue_MiSfit]

You've got it :)

Encode to HuffYUV or Lagarith using VirtualDub or whatever, then use AVISource on that AVI, and run your x264 2pass. No filtering required now!

Also, your source is Xvid?

~MiSfit

[ash925]

or just click on add a pre-rendering pass in megui.file size will be humongous.

[Zyklon B]

No, my source is .VOB, but 5min of a .VOB file is really lots of megabytes to upload :P.

[Blue_MiSfit]

5 minutes might be excessive. Send a minute or so. Just trim the M2V with DGIndex
~MiSfit

[Zyklon B]

As you wish:

http://www.mediafire.com/?zt3nwnw2wm4

:)

[Blue_MiSfit]

I think:

fft3dgpu(plane=4, sigma=2, sigma2=1, sigma3=1, sigma4=3)
gradfun2db(1.51)


Looks good to me!!

Compare like this:


DSS2("C:\Test\VTS_01_1.demuxed.m2v")

a=last
fft3dgpu(plane=4, sigma=2, sigma2=1, sigma3=1, sigma4=3)
gradfun2db(1.51)
interleave(a,last)

spline36resize(852,480)
histogram(mode="luma")


You can comment out the histogram to see the true video..

~MiSfit

[Zyklon B]

Thanks a lot for your tips. Using the histogram I've come to realize that my chain filter script creates more artifacts than fft3dgpu (at least in this case) and tends to give some "blurry" or "smooth" aspect to the pictures (that's why I was using Lanczos4 in the first place, but it's still quite noticeable).

BTW I hadn't ever heard about gradfun2db, looks quite nice! Thanks for that, too.

I'll get back to you as soon as I try 2 encodes: one with your suggestion for the .avs and one with my script. 2-3 minutes for each one, see which one looks better.

[Blue_MiSfit]

The histogram is an invaluable tool for looking at the effects of a filterchain. I use it constantly!

~MiSfit

[Sagekilla]

Is there a reason you're separating fields then weaving again on a.. progressive source?