Original image source filtering

[Dali Lama]

Good day,

I was browsing the official Rurouni Kenshin site and I found high quality cell scans of the TV Series. I looked at it compared to the dvd source and its incredible how much the color changed and noise increased in the conversion process.

So anyways, the idea came to me, that if you have 1-3 original source images, would it be possible to filter the dvd source baised upon that clean image?

Am I making sense?

Instead of using a filter to "replicate" the original clean source, couldn't a filter use the clean source as a template to filter the DVD source?

I think there is an image noise removal program that does this, called "NeatImage"

bye,

Dali

[sh0dan]

Programmingwise something like Neatimage is quite complex, and VERY slow. Furthermore neatimage needs much manual adjustment to get good results. I don't, if was even done it would be of much use.

Furthermore - having a "clean" image is not something many of get the chance for.

Search it has been discussed before.

[Dali Lama]

Hi Sh0dan,

I understand what you are saying and learned of NeatImage from that post you probably are suggesting.

Actually, my idea is not so much like NeatImage.

Its more like the filter analyzes the "clean" image and compares it to the "dirty" image from the video source. Then makes some sort of customized noise removal thresholds.

It is my belief that most noise in many types of sources are random, but common in how they appear.

Another filter I am testing out recently is called Wavelet 3DNR Type G. Its a filter for Aviutl that comes with a helper filter. This is a square box that you set on a part of the image of the source that has noise but should be completely flat/clear (like a white piece of paper that has noise). Then it automatically adjusts the thresholds to get rid of all noise in the source that has those characteristics.

Something like that has been proven useful, but is not exactly what I imagine would be better. Instead of analyzing the noise in the source, why not compare the clean image to the source and find its noise in the whole frame.

I hope I was able to clarify my ideas.

If anyone needs the link to that filter or help using it, look in the General Discussion forum, where I asked for a translation of the command box.

Regards,

Dali

[sh0dan]

Yes it could be interesting, if any good method of "noise quantification" could be found. I've been thinking about it for some time, but I've decided to let the more 'math-minded' doing it.

A temporal "intelligent" filter should be doable, since it is "easier" to measure. Spatial filtering is mostly pixel to pixel, without larger patterns to detect. Basicly it would give "semi-intelligent" filters, that could give suggestions for settings - unfortunately I think it'll require just as much tweaking as setting thresholds.

On a slightly similar issue, I have finally had an idea on how to improve MipSmooth - so when I've done some work I'll hopefully return with an improved version.

[Kurosu]

Originally posted by Dali Lama
Another filter I am testing out recently is called Wavelet 3DNR Type G. Its a filter for Aviutl that comes with a helper filter. This is a square box that you set on a part of the image of the source that has noise but should be completely flat/clear (like a white piece of paper that has noise). Then it automatically adjusts the thresholds to get rid of all noise in the source that has those characteristics.

Does it really adapt thresholds, or does it do a spectral analysis (like in audio noise removers, by establishing a pattern of the noise)? I guess it's the later one, and therefore, it only removes some typical kinds of noise (generic noise ["white"] shouldn't be describable by a simple set of spatial frequencies): wavy patterns, or more or less repetitive. It can remove noise anyway, but it would act as any smoother: cut off high spatial frequencies where the signal is *mostly* noise.

[Dali Lama]

@kurosu

I believe it does "adapt thresholds." First you set it over an area where you want to preserve edges, and it sets the edge detection threshold, then you set it over an area where you want the noise to be completely removed (smoothed) and it sets the wavelet thresholds.

To be honest, I don't know what a spectral analysis is, so it may actually be that. Give it a try if you have time.

@Sh0dan

That's good news, I really like MipSmooth :)

Also, you said it would be mathematically difficult, but couldn't one just use a simple difference filter (some exist now i believe) and use that to detect the noise difference between the two images?

I know of a filter that actually detects the noise in a clip, maybe its opensource, let me double check.

Dali

[sh0dan]

Unfortunately a difference filter doesn't tell that much.

Assuming to evenly aligned sources (actually hard in the real world) - the differences you get is still not very useable for denoising, as noise (per definition) changes on every frame.

So - you need to get a "noise pattern" - their size, their variance, their relation to the original material - and then use these informations to be able to detect similar noise, and exclude material that is part of the original signal. A sound analogy is actually more fitting here:

When you have a sound signal, with a tone at a certain frequency throughout the entire piece of music, what you do is:

1) Find a silent part in the music.
2) Detect the tone there.
3) Filter out this "exact" frequency throughout the entire piece of music.

This works quite well, since there probably isn't much of the music actually suffering even if some of the original material contained some frequencies at this tone.

Now consider a piece of music with some white noise present. White noise is noise which operates at all frequencies. So what the audio program does is that it detects the level of the white noise at all frequencies, and then lowers the volume at exact the same level as this.

Now the degradation of the music becomes obvious - typically low volume sounds are lost - mostly in the high frequency area.

Consider the same with video. Detecting static "noise" is possible - and can be removed - this is shown by several "delogo" filters.

However due to the nature of video noise - noise is quite often non-static, and is present at the same frequency (at least spatially) - so you have to use the same thresholds throughout the entire frame - this will give an effect like SmartSmooth, C3D, etc. TemporalSoften is the same algorithm, with slight modification for temporal use. The "amplitude/volume" of the frequency is what you set as threshold - this usually doesn't differ much throughout material.

Slightly different (and more adaptive) algorithms can be used for temporal smoothing - this is mostly seen in filters like peach, flux (name more).

Programs like NeatImage tries to detect very distinct features of the material - mostly based on the flaws of the camera. This includes color inaccuracy, luma flux on a spatial basis. Basicly it builds a "noise profile" like the audio program, and then uses this to remove these "frequencies" throughout the image. So in essence it is an adaptive spatial smoother.

Instead of giving a "flat" area you have to give thresholds to the smoother, which just makes life simpler for the filter - an adaptive filter would still need guidance, so my point is - you don't gain much - you just have to specify other parameters.

[This became way more complex than intended - beware of nonsense - I can recommend playing with CoolEdit to try out some of these things - it is great inspiration].