Need suggestion to denoise this video

[huang_ch]

I'm encoding Robotech from one DVDrip source, but the source itself is not in good quality, it has many noise. I've tried FFT3DGPU/FILTER, VagueDenoiser, but neither give a satisfactory result, so I'm asking for some extra suggestions here. Please help, thanks.
One preview is like below:
http://img133.imageshack.us/img133/6194/robotechrs3.png

[dbzgundam]

Get "Robotech" remastered, if you must have Robotech... Sadly though, the "remastered" title doesn't say much.

[Chainmax]

This seems to have blocking and haloing as well. I'd use DeBlock_QED, DeGrainMedian, DeHalo_Alpha, LimitedSharpenFaster and aWarpSharp. In any case, we will be better equipped to help you if you can cut a vob sample with something like ChopperXP (http://www.inchwest.com/digital-miner.com/chopperxp.html) and upload it somewhere.

[huang_ch]

Get "Robotech" remastered, if you must have Robotech... Sadly though, the "remastered" title doesn't say much.

Indeed, fft3dgpu does a not bad job when encoding the first ~20 collection, but after those videos, the noise becomes more and more,:devil: that makes fft3dgpu not enough to deal with those noises, so I have to ask for help here. :)

[huang_ch]

This seems to have blocking and haloing as well. I'd use DeBlock_QED, DeGrainMedian, DeHalo_Alpha, LimitedSharpenFaster and aWarpSharp. In any case, we will be better equipped to help you if you can cut a vob sample with something like ChopperXP (http://www.inchwest.com/digital-miner.com/chopperxp.html) and upload it somewhere.
Thanks for offering help. I'm not familiar with so much filters...
Below are two sample cut from the original video, and please note that the noise in these two samples are above the average:
http://rapidshare.com/files/5153378/1.avi.html
http://rapidshare.com/files/5154130/2.avi.html

[Chainmax]

Those are AVI files, it's better if you post actual VOB samples and reencoding alters the source.

[DarkZell666]

For the haloing, isn't there a plugin that shifts the selected plane by N pixels in any direction ? I've always been wondering about this ... :/

Forgetting about the halo, I would use a strong hqdn3d on this, or FrFun7(). I'd even consider using jim casaburi's 2dcleaner vdub filter (either of the k7 or p4 optimized ones). I still can't get over the source being such a mess ^^"

[huang_ch]

Those are AVI files, it's better if you post actual VOB samples and reencoding alters the source.
Sorry, but what I get in hand is the DVDrip AVI files, not the DVD itself.

[huang_ch]

For the haloing, isn't there a plugin that shifts the selected plane by N pixels in any direction ? I've always been wondering about this ... :/

Forgetting about the halo, I would use a strong hqdn3d on this, or FrFun7(). I'd even consider using jim casaburi's 2dcleaner vdub filter (either of the k7 or p4 optimized ones). I still can't get over the source being such a mess ^^"

For the haloing, I'm wondering whether its reasonable to use something like shifts the selected plane by N pixels, because the N may be different from frame to frame....

About hqdn3d/frfun7 or others, I can try them individually, but I'm lacking of knowledge to use multiple filters in combination to achieve a better result, indeed, I don't know which filter is good at filtering which kind of noise, I've read one guide at doom9.org about some basic knowledge of chrome/luma noise or halo or others, but still got confusing when deal with real videos.........

[Chainmax]

Sorry, but what I get in hand is the DVDrip AVI files, not the DVD itself.

Ok. I'm in the middle of a heavy encoding job (~4h for a ~5min short, LOL :)), but I'll give those a try tomorrow.

[huang_ch]

Ok. I'm in the middle of a heavy encoding job (~4h for a ~5min short, LOL :)), but I'll give those a try tomorrow.

Thanks~~~ But... how could you make a 5min short video encoding at ~4h.... too terrible... a lot of filter?:eek:

[Chainmax]

Yup, the filterchain includes a quite CPU intensive bobber (double framerate deinterlacing) and heavy filtering.

[Chainmax]

Check these encodes out:

http://rapidshare.com/files/5308514/Clip_1.mp4.html
http://rapidshare.com/files/5308350/Clip_2.mp4.html

if you like the results, I'll post the filterchain.

[huang_ch]

Check these encodes out:

http://rapidshare.com/files/5308514/Clip_1.mp4.html
http://rapidshare.com/files/5308350/Clip_2.mp4.html

if you like the results, I'll post the filterchain.

Thanks, the result is really impressive, I've never expected such a noiseless result, until now I realize the power of denoisers....
To the result itself, below are some feedback:
1. The denoise of Clip_1 is perfect.
2. The denoise of the background of Clip_2's beginning scene(a boy running) still contains many noises though already much better than the original one. Don't know if it could be improved.
3. The background of both Clip1 & 2, especially Clip_2's beginning scene contains some large blocky areas, that's hurt the visual quality I think, can it be improved by deblock filter? or by lighter the denoise level?
4. Most of the edges seems to be a little over smooth/blurred

So I'm guessing that you may apply a pretty strong denoise to get a noiseless video, but I think I may prefer a video which contains more detail even with a bit more noise. But if you could explain how to tune the denoise level with your script, I can tune it myself.

Anyway, thanks again for your help.

[Chainmax]

Here's the script:

Source Loading
FFT3DFilter(sigma=6,plane=3,bw=32,bh=32,bt=3,ow=16,oh=16)
DeBlock(quant=35)
Crop(whatever,align=true)
DeGrainMedian(LimitY=5,LimitUV=7,mode=0)
DeHalo_Alpha()
BicubicResize(whatever,0,0.5)
aWarpSharp(depth=16,cm=1)
dull=last
sharp=dull.LimitedSharpenFaster(SMode=4,Strength=150)
Soothe(sharp,dull,25)
Tweak(sat=1.2)
Levels(0,1,255,16,235)

[DarkZell666]

Ehm, is it me or do those caracters look like they've been oil-painted ? :p
awarpsharp was set a bit strong imho ^^

I'll have a quick try later on :)

[huang_ch]

Here's the script:

Thanks for the script, I'll try it this night.
BTW, is there any experiences that how do you usually combine these filters?

[huang_ch]

Ehm, is it me or do those caracters look like they've been oil-painted ? :p
awarpsharp was set a bit strong imho ^^

I'll have a quick try later on :)

Suggestion welcomed.:thanks:

[Chainmax]

Ehm, is it me or do those caracters look like they've been oil-painted ? :p
awarpsharp was set a bit strong imho ^^

I'll have a quick try later on :)

Yeah, whatever little details there were are of course lost, but if you look closely blocking is not completely removed even at such settings. I could have used less destructive settings but leftover blocking would have been much more evident. Also, maybe a high bitrate MPEG2 encoding would look better. As for aWarpSharp, it's there as a line thinner and the seting are the default one except for the changing in the cm setting.


Thanks for the script, I'll try it this night.
BTW, is there any experiences that how do you usually combine these filters?

Not really. I play it by ear initially, then tweak it by changing the filter order in VDubMod's script editor, then refresh to see the changes. As a rule of thumb, I do color corrections first, then deblocking, then denoising, then resizing, then line thinning, then sharpening and finally grain adding.

[Serbianboss]

Maybe offtopic, which filter for deblocking to use?

[Chainmax]

I use DeBlock_QED for light-to-medium blocking as it does a great job and is designed to keep as much detail as possible. If stronger deblocking and/or more speed are needed I switch to DGDecode's DeBlock which is inspired by H.264's deblocking and was originally a part of Manao's MVTools.

[DarkZell666]

huang_ch: Is it possible for you to upload those files somewhere else ? egoshare.com aren't bad. Since I'm behind a http proxy some thousands of employees are sharing, I can hardly download anything from rapidshare ... I was lucky enough to download Chainmax's samples earlier on :)

[huang_ch]

huang_ch: Is it possible for you to upload those files somewhere else ? egoshare.com aren't bad. Since I'm behind a http proxy some thousands of employees are sharing, I can hardly download anything from rapidshare ... I was lucky enough to download Chainmax's samples earlier on :)
Uploaded for your convenience. :)
http://www.egoshare.com/7c588f6ac0cc4975d8d4ab1827874191/1avi.html
http://www.egoshare.com/b53515852d0f7fed7530279d3d629e92/2avi.html

[huang_ch]

Here's the script:


Chainmax, I've played with your script tonight, the result is pretty good and I'm still tuning for a more common script because not all video needs such a strong denoise, and I have below questions:
1. What's the goal for Crop and BicubicResize? Just for a size purpose?
2. I've seen that by using plane=4 instead of plane=3 it gets more noise removed in many scene, so I want to ask why you just use plane=3? Any special reason?

[Chainmax]

The cropping was done to remove those black borders on the side, and resizing was set to neutral bicubic (i.é, the 0,0.5 switches) instead of sharper resizer like Lanczos4 because your source has very strong haloing and there's already a sharpener in the chain. Do you have an HDTV or a common TV, By the way?

As for the FFT3DFilter line, it was set to process only the chroma planes (i.é: the plane=3 switch) in order to get rid of some slight rainbowing I saw. plane=4 processes both luma and chroma, which means it denoises and eliminates rainbows at the same time.
You might ask why not just using it for both tasks instead, the answer is that DeGrainMedian is about as strong as FFT3DFilter's sigma=3 for denoising, but has a nice side effect of slight block removal (and also slight dotcrawl removal, although your source doesn't have that), which for this source was perfect.

Finally, this chain was made for this source and I don't think you'll be able to transform it into a general purpose chain. Each source requires different treatment, even when dealing with the same kind of artifacting. It's not a good idea to limit yourself to just one denoiser, one deblocker, one comb remover, etc. The best way to create a filter library that suits you is to practice a lot and ask here when in doubt :).

[huang_ch]

The cropping was done to remove those black borders on the side, and resizing was set to neutral bicubic (i.é, the 0,0.5 switches) instead of sharper resizer like Lanczos4 because your source has very strong haloing and there's already a sharpener in the chain.
But I've seen there's already a DeHalo_Alpha in the chain to remove halos, why another bicubic resizing is needed? The halo is to strong?


Do you have an HDTV or a common TV, By the way?

I only have a common TV, but hope to get one these years....:D


You might ask why not just using it for both tasks instead, the answer is that DeGrainMedian is about as strong as FFT3DFilter's sigma=3 for denoising, but has a nice side effect of slight block removal (and also slight dotcrawl removal, although your source doesn't have that), which for this source was perfect.

Oh? A simple question here is that I usually apply a light weight FFT3DFilter or VagueDenoiser with some pretty good DVDrip sources to just get it cleaner by removing some artifacts brought by the encoder when rip it from DVD. Usually I use sigma=1, sigma2=1.5, sigma3=2, sigma4=2, plane=0(just luma, no chroma), or a VagueDenoiser with threshold=1.5, method=1, nsteps=6, chromaT=-1, so in this case, will you suggest replace it with DeGrainMedian? I just want to replace it with a faster filter with no quality loss.


Finally, this chain was made for this source and I don't think you'll be able to transform it into a general purpose chain. Each source requires different treatment, even when dealing with the same kind of artifacting. It's not a good idea to limit yourself to just one denoiser, one deblocker, one comb remover, etc. The best way to create a filter library that suits you is to practice a lot and ask here when in doubt :).
What I mean a more common purpose just means common to this full video source, not everything:p

[Chainmax]

The halos are very strong. As a matter of fact, I was not sure wether to add another halo remover or not.
Since you have a common TV, it will likely have big overscan, meaning that a significant portion of the picture would be hidden upon display on it. I'd recommend you to set the resizer to BicubicResize(656,448,0,0.5) (the source was 29.97fps, right?) and to include AddBorders(32,16,32,16) right before the Levels(0,1,255,16,235) line. The downsizing will have the double benefit of further hiding artifacts and making sure you see as much of the picture as possible when playing it on your TV.
As for the rest of your reply, my recommendations stand, but do experiment with alternatives and see what you like best. That's the best way to learn :).

[huang_ch]

Chainmax, I'm still studying the filter by changing/disabling/enabling one by one to see the result to get a better understanding of them. One thing I found is that, I'm getting a slightly better result with DeHalo->Deblock->fft3dfilter instead of fft3dfilter->Deblock->DeHalo, any comments on this?

[Chainmax]

I haven't tried them in that order, but if it looks better for you then that's all that needs to be said. Filter order can indeed make a huge difference in the quality. Just a couple of days ago, I was treating a video with some extremely ugly aliasing, and using AAA before EEDI2 cleaned them up much better than using EEDI2 before AAA.

It's interesting that the place FFT3DFilter is in causes a difference though, as it's only there for some minor chroma (color) denoising. Can you post a before-after couple of screenshots showing this improvement you speak of?

[huang_ch]

Chainmax, about FFT3DFilter, I've modified its plane to 4 instead of 3 which also give some improvements as I've mentioned in a previous post, and after that I tried to remove the DeGrainMedian which seems to be not contribute much now, would this be the reason that there's a bit improvement when put it after halo & deblock?
And about what I said a slightly improvement, is that, in many cases, I need to zoom it to 150% or 200% to see the noise difference, so I've though a way that put a Sharpen(1).Sharpen(1) at the end of script, which will obviously over sharp the frame, so the noises will also be over sharped and much clearer to see without zoom. With this method, the difference between put FFT3DFilter at the beginning or after dehalo/deblock is pretty obvious, but I don't know whether it is a correct method when tuning the filters.
And also, when I encoded the video with these two different sequence filters with X264, the one with Dehalo->Deblock->FFT3DFilter gives better compression (~5% less in size) and a higher PSNR(>0.2).
I'll post some screenshots when I get home tonight. :)

[huang_ch]

Chainmax, some screenshot for the different filter sequence:
1. FFT->DeBlock->DeHalo
http://img148.imageshack.us/img148/9296/fftblockhalolv8.png

2. DeHalo->DeBlock->FFT (didn't cut well, left side cropped...)
http://img88.imageshack.us/img88/9736/haloblockfftts6.png

Please notice the clouds to the left of the helmet, the 2) is a bit more cleaner/smoother than 1).

[Chainmax]

Well, luma denoising of sigma=3 should be roughly on par with a default DeGrainMedian call (which means limitY=4,limitUV=6), so it's reasonable that you see less removed detail. And I definitely like the second picture better. As for the cropping:

Since you have a common TV, it will likely have big overscan, meaning that a significant portion of the picture would be hidden upon display on it. I'd recommend you to set the resizer to BicubicResize(656,448,0,0.5) (the source was 29.97fps, right?) and to include AddBorders(32,16,32,16) right before the Levels(0,1,255,16,235) line. The downsizing will have the double benefit of further hiding artifacts and making sure you see as much of the picture as possible when playing it on your TV.

[huang_ch]

I see. Thanks again for your help, Chainmax~~~ I think I can start my encoding now, and keep learning the filters. :)