DVD to H264. Recommended NR filters to reduce sizes of crf18 encodes

[LeXXuz]

Hey People.

I got an H264 capable hdd player recently (Tvix 6500 with Sigma 8635) and now I want to backup parts of my dvd collection to h264 to save some space. :)

I've decided to go with anamorphic encodes and my encoding parameters currently look like these:

program --crf 18.0 --level 4 --ref 4 --mixed-refs --no-fast-pskip --bframes 3 --b-pyramid --b-rdo --bime --weightb --direct auto --subme 6 --trellis 2 --partitions p8x8,b8x8,i4x4,i8x8 --8x8dct --qpmin 6 --qpmax 42 --ipratio 1.1 --pbratio 1.1 --vbv-bufsize 9000 --vbv-maxrate 24000 --qcomp 0.5 --me umh --threads auto --thread-input --progress --no-dct-decimate --no-psnr --no-ssim --output "output" "input" --sar 16:11

It's slightly adapted from Sharktooth presets for PS3/X360.

Okay whats the point of all this? On newer movies the results and filesizes are fine (between 1,5-2,5GB).

But on older movies filesizes start to explode and sometimes end up even higher than the dvd. So I need some filtering for sure.

What noise filters do you use for HQ DVD backups? Are there any noise filters you could always use without worrying that they affect visual quality?

I thought about using
RemoveGrain(mode=1)
or
DeGrainMedian(limitY=5,limitUV=5,mode=3)


What's your opinion on these? I dont want to loose any visual quality or getting ghosts in moving areas or something like that. Just want to get rid of noise the encoder wastes bits on but I won't probably see on my TV-set. :)

Should I use noise filters at all or should I better raise crf values to reduce filesizes?

[Dark Shikari]

You could try a low-strength mvdegrain or dfttest.

Undot apparently also helps compression.

[LeXXuz]

Undot apparently also helps compression.

I thought about undot as well. But when I used forum search, I always ended up in threads about Anime movies. So I did not give it some more thought.

[noee]

I use RemoveGrain(mode=2) and Convolution3d (preset="movieHQ") or sometimes both together:

RemoveGrain(mode=2).Convolution3d (preset="movieHQ")

[fields_g]

RemoveGrain(mode=1) is identical to undot(), but faster

[LeXXuz]

I use RemoveGrain(mode=2) and Convolution3d (preset="movieHQ") or sometimes both together:

RemoveGrain(mode=2).Convolution3d (preset="movieHQ")

Do you always use these filters or only on real noisy sources?


RemoveGrain(mode=1) is identical to undot(), but faster

Again something learned. :) :thanks:

[Ranguvar]

I usually do an FFT3DGPU with my own sigmas, adjusted for each encode. If the video is very bad, I'll do a TTempSmooth(F) too.

[lcx]

I prefer NeatVideo but it costs $$s
The results speak for themselves though.

[Avenger007]

I usually do an FFT3DGPU with my own sigmas, adjusted for each encode.
Do you have any advice about what you look for in a source and how then to determine or narrow down the range of sigma values to try?

[fields_g]

I played around with fft3dgpu a while, and got tired of the slow performance on my mobile GPU in my laptop. So I started using a filter called hqdn3d() by some guy named Loren Merritt :) I have been satisfied with hqdn3d(2), but sometimes adjusted up or down one due to the source. It is fast and doesn't eat the details. A good noise filter can be an encoder's best friend. Others listed above are really good also.

[fibbingbear]

Do you have any advice about what you look for in a source and how then to determine or narrow down the range of sigma values to try?

As a generality for FFT3DFilter, I think sigmas should be somewhere between 0.5-3, depending on the level of noise. I think rolling sigmas are popular, where sigma1 and sigma4 are higher, while sigma2 and sigma3 are lower, like (completely made up example):

sigma1=1.5, sigma2=0.5, sigma3=0.5, sigma4=1.0

See http://forum.doom9.org/showthread.php?t=139012 for an example/discussion.

Generally speaking, 0.5 is very lightweight removal, 1 is lightweight, 2 is medium, and 3+ is heavy (imo).

[kemuri-_9]

as with filters like FFT3D, hqdn3d, and FrFun, you will smooth away the details if you keep ramping it up to remove more grain/noise.
so just as a general warning, be careful when using these.

[Ranguvar]

Mmm... I don't have my scripts anymore, but if I remember right, the first sigma was typically around 1.8, and the fourth one around 0.5, with 2 and 3 in the middle. Increasing sigma1 tends to kill a lot of *easy* noise without hurting detail, but 2 and 3 are very difficult to set right to kill the right amount of noise, if you have *hard* noise (easy meaning obviously noise, hard meaning much more grain-like).

I typically set it up as a curve from sigma1 at the highest and sigma4 at the lowest, like I said, and then look at the video. Too much of a certain type of noise? Increase each sigma a LOT, one at a time, and find which kills the noise the best. Then, lower it until you get to an acceptable level of noise killed vs. quality lost. Rinse and repeat. Then, at the end, try decreasing or increasing each value you didn't adjust before and decide on a look.

Of course, one could use FFT3DFilter if you don't have a fast GPU - but IMO, filters like HQDN3d, etc. are better than FFT3DFilter in most scenarios. FFT3DGPU just runs blazing fast on a good video card (easily 4x faster on my 9600GT 512MB vs. my quad-core Q6600 and 4GB of RAM, and that's even without recognizing that encoding will eat into some of that CPU).

Have fun :)

[LeXXuz]

Of course, one could use FFT3DFilter if you don't have a fast GPU - but IMO, filters like HQDN3d, etc. are better than FFT3DFilter in most scenarios. FFT3DGPU just runs blazing fast on a good video card (easily 4x faster on my 9600GT 512MB vs. my quad-core Q6600 and 4GB of RAM, and that's even without recognizing that encoding will eat into some of that CPU).


I guess I have to stick with CPU-Power then (Q9650). My VGA-card is very slow (ATI x1550) and was the cheapest I could find when assembling my pc. :D

as with filters like FFT3D, hqdn3d, and FrFun, you will smooth away the details if you keep ramping it up to remove more grain/noise.
so just as a general warning, be careful when using these.

That is exactly what I'm afraid of when using filters. Primary goal is just to get rid of noise I wont be ably to see on my TV anyway.

I've read a lot of threads about filtering in the capture forum and most of the results there wouldn't satisfy me. A lot of people use such strong filter settings that there is a great loss of detail and the actors start looking like wax dolls. That's absolutely a no go.

That's why I asked for filtering solutions that reduce file size but won't filter the image that much that all the noise is gone.

Uhm, well I guess its hard to describe what I want. :p

Thank you guys for your input. I guess I have a lot of filter settings to play with the next couple of days. :) :thanks:

[Ranguvar]

I guess I have to stick with CPU-Power then (Q9650). My VGA-card is very slow (ATI x1550) and was the cheapest I could find when assembling my pc. :D

Try and see :)

And I do use FFT3DGPU to help the encoder :)

[LeXXuz]

Try and see :)

And I do use FFT3DGPU to help the encoder :)

Does it work on any card? Which cards might be preferable to gain a good speed advantage out of this?

[fields_g]

With my crappy GPU, while using fft3dgpu, I was able to "explore" more insane settings in x264 with my Core2 T7200 because the system was waiting for the GPU so much!

Comparing no noise reduction to hqdn3d, I found an appreciable speed increase in encoding with hqdn3d, even though the cpu was doing both NR and encoding. This is due to the simple fact that the encoder was less distracted by the excess garbage. It also created much smaller files while in CRF mode. This will not be the case with all filters. The whole trick is to find the balance between details and reduction for your specific source material.

[Ranguvar]

Should, any DX9 card really (my old GeForce 5200 FX 128MB was pathetic though). And pretty much any fast card is good, preferably with lots of RAM. If you don't have enough, you can't use that high of resolution, or you have to lower some of the settings.

Even if it's a bit slower than FFT3DFilter, that's OK, because now your CPU is just idling, which means you won't lose nearly as much speed during encode as you would if FFT3DFilter or another CPU filter is hogging the CPU.

[Blue_MiSfit]

As a generality for FFT3DFilter, I think sigmas should be somewhere between 0.5-3, depending on the level of noise. I think rolling sigmas are popular, where sigma1 and sigma4 are higher, while sigma2 and sigma3 are lower, like (completely made up example):

sigma1=1.5, sigma2=0.5, sigma3=0.5, sigma4=1.0

See http://forum.doom9.org/showthread.php?t=139012 for an example/discussion.

Generally speaking, 0.5 is very lightweight removal, 1 is lightweight, 2 is medium, and 3+ is heavy (imo).

+10

I love this filter so much :)

Tweaking sigma values is essential - I usually do it with the aid of Histogram(mode="luma"), and UtoY() or VtoY().

Also, fft3dfilter/gpu can reveal / introduce banding, so I always dither with gradfun2db or gradfunkmirror.

For fft3dgpu, I would really suggest a GeForce 7800gt or higher. This is what the developer used as his benchmark, and it's perfectly fast for SD. It will run on any card that supports Pixel Shaders 2.0, so that means anything newer than a Radeon 9600 pro on the ATi/AMD camp, and anything newer than a GeForce 6, on the nVidia camp. I know it works on the FX 5xxx series, but it's hideously slow.

I run HD encodes on an 8800gt at home, and a 9600gt at work. They're also perfectly fast, but with only 512mb of ram, they choke when I try to run two instances on 1080p content. Vista's improved GPU memory management makes this workable at home :)

If looking for a sweet card for fft3dgpu, I would steer you towards a Radeon 4850 or faster, with 1GB of RAM if you're going to process 1080p.

~MiSfit