Dali Lama
21st May 2003, 03:42
Hi,
With some free time on my hands, I decided to try to improve my anime encoding. Knowing all too well that a clean source leads to clean output, I decided to re-evaluate some spatial and temporal cleaning techniques.
***************************************************************
Note: The following information has been gained by experimentation and reading posts from users like you! I hope we will continue to share our knowledge and improve our encoding and filtering techniques.
***************************************************************
__________________________________________________________________
After reading posts by MasterYoshidino concerning the use of spatial following temporal filters, I began to experiment...
My previous belief was that using temporal filtering would remove as much noise as possible. Then a spatial filter could smooth the remaining noise away. And it does work well, I'm sure as you all know. However, I tried to use spatial first and then temporal afterwards. To my surprise the noise removal was considerably better. Thinking about it and looking at what the spatial filter did, I realized that using a spatial filter first actually helps to smooth out mosquito noise into flat beveled blocks, enabling the temporal smoother to work better.
(good work MasterYoshidino!, maybe others have a different opinion?)
_________________________________________________________________
Then I began to experiment with the order of the filters. There are basically two ways. Apply both filters after resize or before. Now, logic would entail that applying the filters before resizing would give the filter more data (higher resolution) and noise to work with, which is good. In addition, resizing afterwards allows any noise that has not been removed to be somewhat disgused by the resizer.
However, I found the opposite to be true in my trials. I have a few reasons for this. One, the common resizing algorithms (bicubic, bilinear, lanczos) all use anti-alisizing/spatial filtering on resize (please correct me if I am wrong). Therefore, it goes back to the first discussion on applying spatial before temporal. So, I think if you believe that spatial should be applied before temporal filters, then maybe you should also try filtering after resize?
Secondly, I have read before on the forums that applying filters after resize results in more heavy filtering. This may also play a role in my subjective observations. However, I am usually not impressed by over filtering and the strengths/thresholds used for my tests were low. This should also be a consideration.
_______________________________________________________________
The last issue I'd like to bring for (at least for this initial post) is the number of sequential spatial filtering. I was thinking that if spatial filters blend pixels that are adjecent to each other, then using more than one instance of the filter sequentially should not lead to rapid image degredation, if the thresholds are LOW. Instead it could enhance filtering and preserve sharpness/detail of the edges especially.
Very quickly, I realized that for the use of sequential spatial filtering to be useful there would have to be increasingly wide/large filter radii. This is needed, because once one spatial filter is used and averages some pixels at for example 1x1 radius, then a larger radius should be used the second time to average slightly more pixels.
So basically, the spatial filtering can look something like this:
Name of Spatial Cleaner, threshold, x radius, y radius
2DCleaner 1,1,1
2DCleaner 2,2,2
2DCleaner 3,3,3
2DCleaner 4,4,4
2DCleaner 5,5,5
So the thresholds and radii increase after each incidence of the spatial filter. The thresholds can stay low (5 and under is ver low for 2D cleaner) the radii go very high (5x5 = 11x11 in 2DCleaner), which makes the speed very slow, but the noise reduction is substaintial and detail level high, IMO.
_________________________________________________________________
A few questions:
1. Do others get nice results using this technique?
2. Can this work on non-anime or film type sources? (I better check)
3. Is the speed loss due to the multiple sequential instances of the spatial filtering? IMO, probably not. You could just use two instances or just one. However, one may have to use higher thresholds to acheive the same noise reduction at the sacrifice of detail.
___________________________________________________________________
Next time, I will report back on the use of various types of spatial and temporal cleaners. Sh0dan just unveiled MipSmooth! I will have to try it out. But you can probably guess which spatial "cleaner" I like from my example ;)
___________________________________________________________________
Here are some example snapshots using this filtering technique:
Source: Rurouni Kenshin TV (of course! ;))
[list=1]
Saito's Face Original (http://avisynth.scripterz.org/SaitoFaceOrig.PNG)
Saito's Face Filtered (http://avisynth.scripterz.org/SaitoFace.PNG)
Kenshin Trees Original (http://avisynth.scripterz.org/KenshinTreesOrig.PNG)
Kenshin Trees Filtered (http://avisynth.scripterz.org/KenshinTrees.PNG)
Kenshin's Face Original (http://avisynth.scripterz.org/KenshinFaceOrig.PNG)
Kenshin's Face Filtered (http://avisynth.scripterz.org/KenshinFace.PNG)
Kaoru Trees Original (http://avisynth.scripterz.org/KaoruTreesOrig.PNG)
Kaoru Trees Filtered (http://avisynth.scripterz.org/KaoruTrees.PNG)
[/list=1]
Note: You have to highlight the url and hit 'enter' for image to load. :confused:
Update: I just discovered...if you look at the two pictures right after each other, the filtered image seems to be stretched. This is a side effect of a light (threshold=15) warpsharp that I used. I did another sample with no filtering but warpsharp and it exhibited the same thing! nueron2 mentioned a while ago that warpsharp was not recommended!?! I guess I should have believed him (not that I think its useless, really, but its not practical for keeping the original lines of the anime).
Have a great summer!
Dali
With some free time on my hands, I decided to try to improve my anime encoding. Knowing all too well that a clean source leads to clean output, I decided to re-evaluate some spatial and temporal cleaning techniques.
***************************************************************
Note: The following information has been gained by experimentation and reading posts from users like you! I hope we will continue to share our knowledge and improve our encoding and filtering techniques.
***************************************************************
__________________________________________________________________
After reading posts by MasterYoshidino concerning the use of spatial following temporal filters, I began to experiment...
My previous belief was that using temporal filtering would remove as much noise as possible. Then a spatial filter could smooth the remaining noise away. And it does work well, I'm sure as you all know. However, I tried to use spatial first and then temporal afterwards. To my surprise the noise removal was considerably better. Thinking about it and looking at what the spatial filter did, I realized that using a spatial filter first actually helps to smooth out mosquito noise into flat beveled blocks, enabling the temporal smoother to work better.
(good work MasterYoshidino!, maybe others have a different opinion?)
_________________________________________________________________
Then I began to experiment with the order of the filters. There are basically two ways. Apply both filters after resize or before. Now, logic would entail that applying the filters before resizing would give the filter more data (higher resolution) and noise to work with, which is good. In addition, resizing afterwards allows any noise that has not been removed to be somewhat disgused by the resizer.
However, I found the opposite to be true in my trials. I have a few reasons for this. One, the common resizing algorithms (bicubic, bilinear, lanczos) all use anti-alisizing/spatial filtering on resize (please correct me if I am wrong). Therefore, it goes back to the first discussion on applying spatial before temporal. So, I think if you believe that spatial should be applied before temporal filters, then maybe you should also try filtering after resize?
Secondly, I have read before on the forums that applying filters after resize results in more heavy filtering. This may also play a role in my subjective observations. However, I am usually not impressed by over filtering and the strengths/thresholds used for my tests were low. This should also be a consideration.
_______________________________________________________________
The last issue I'd like to bring for (at least for this initial post) is the number of sequential spatial filtering. I was thinking that if spatial filters blend pixels that are adjecent to each other, then using more than one instance of the filter sequentially should not lead to rapid image degredation, if the thresholds are LOW. Instead it could enhance filtering and preserve sharpness/detail of the edges especially.
Very quickly, I realized that for the use of sequential spatial filtering to be useful there would have to be increasingly wide/large filter radii. This is needed, because once one spatial filter is used and averages some pixels at for example 1x1 radius, then a larger radius should be used the second time to average slightly more pixels.
So basically, the spatial filtering can look something like this:
Name of Spatial Cleaner, threshold, x radius, y radius
2DCleaner 1,1,1
2DCleaner 2,2,2
2DCleaner 3,3,3
2DCleaner 4,4,4
2DCleaner 5,5,5
So the thresholds and radii increase after each incidence of the spatial filter. The thresholds can stay low (5 and under is ver low for 2D cleaner) the radii go very high (5x5 = 11x11 in 2DCleaner), which makes the speed very slow, but the noise reduction is substaintial and detail level high, IMO.
_________________________________________________________________
A few questions:
1. Do others get nice results using this technique?
2. Can this work on non-anime or film type sources? (I better check)
3. Is the speed loss due to the multiple sequential instances of the spatial filtering? IMO, probably not. You could just use two instances or just one. However, one may have to use higher thresholds to acheive the same noise reduction at the sacrifice of detail.
___________________________________________________________________
Next time, I will report back on the use of various types of spatial and temporal cleaners. Sh0dan just unveiled MipSmooth! I will have to try it out. But you can probably guess which spatial "cleaner" I like from my example ;)
___________________________________________________________________
Here are some example snapshots using this filtering technique:
Source: Rurouni Kenshin TV (of course! ;))
[list=1]
Saito's Face Original (http://avisynth.scripterz.org/SaitoFaceOrig.PNG)
Saito's Face Filtered (http://avisynth.scripterz.org/SaitoFace.PNG)
Kenshin Trees Original (http://avisynth.scripterz.org/KenshinTreesOrig.PNG)
Kenshin Trees Filtered (http://avisynth.scripterz.org/KenshinTrees.PNG)
Kenshin's Face Original (http://avisynth.scripterz.org/KenshinFaceOrig.PNG)
Kenshin's Face Filtered (http://avisynth.scripterz.org/KenshinFace.PNG)
Kaoru Trees Original (http://avisynth.scripterz.org/KaoruTreesOrig.PNG)
Kaoru Trees Filtered (http://avisynth.scripterz.org/KaoruTrees.PNG)
[/list=1]
Note: You have to highlight the url and hit 'enter' for image to load. :confused:
Update: I just discovered...if you look at the two pictures right after each other, the filtered image seems to be stretched. This is a side effect of a light (threshold=15) warpsharp that I used. I did another sample with no filtering but warpsharp and it exhibited the same thing! nueron2 mentioned a while ago that warpsharp was not recommended!?! I guess I should have believed him (not that I think its useless, really, but its not practical for keeping the original lines of the anime).
Have a great summer!
Dali