Welcome to Doom9's Forum, THE in-place to be for everyone interested in DVD conversion. Before you start posting please read the forum rules. By posting to this forum you agree to abide by the rules. |
23rd September 2011, 04:18 | #21 | Link |
Registered User
Join Date: Jan 2006
Posts: 1,867
|
I could suggest that you use 120 frames from tnlmeans, that would take ~3 hours, but at least I can look through frame-by-frame for artefacts, but also I want to take some still frames and do objective measurements metrics (blur+noise).
Also, I went looking for some very high quality frames to test, and finally found them. Here ya go: http://media.xiph.org/sintel/sintel-1k-png16/ So the question then becomes, how to load 16bit png into Avisynth? Well, that lead to ImageMagick, which finally lead me to write a high bit-depth image import for Avisynth Now we can have extremely high quality noiseless images to test with, but also I know how to add highly accurate modelled film grain. The final result will be the ultimate reference quality test images, and also measurements of the different tools with objective measurements. Gonna be a lot of work though, but thanks for your efforts so far, we'll have to share some efforts. Update: Well I got this far, Code:
convert %08d.png[8970] -evaluate and 65280 -set filename:f msb%t.bmp +adjoin %[filename:f] convert %08d.png[8970] -evaluate and 255 -evaluate leftshift 8 -set filename:f lsb%t.bmp +adjoin %[filename:f] Last edited by jmac698; 23rd September 2011 at 08:03. |
23rd September 2011, 09:12 | #22 | Link |
typo lover
Join Date: May 2009
Posts: 595
|
Code:
cd /d d:\sintel_rgb48 ffmpeg -f image2 -i %08d.png -pix_fmt yuv444p16le -vcodec rawvideo -an -f rawvideo d:\sintel_raw_yuv\yuv444p16le.yuv #read_16bit_yuv.avs LoadPlugin("RawSource26.dll") LoadPlugin("flash3kyuu_deband.dll") RawSource("d:\sintel_raw_yuv\yuv444p16le.yuv", width=2048, height=436, fpsnum=24, fpsden=1, pixel_type="i444") f3kdb_dither(stacked=false)
__________________
my repositories Last edited by Chikuzen; 23rd September 2011 at 09:15. |
24th September 2011, 18:43 | #26 | Link |
Registered User
Join Date: Nov 2009
Posts: 2,352
|
smdegrain(tr=6,thSAD=600,contrasharp=true,lsb=true,refinemotion=true,prefilter=2)
http://www.mediafire.com/?81o4zz0in25muwr I also think these tests have no real meaning. You are mixing spatial denoisers, with temporal, and each of them in different strengths. Quality has also many broad interpretations.
__________________
i7-4790K@Stock::GTX 1070] AviSynth+ filters and mods on GitHub + Discussion thread |
24th September 2011, 19:14 | #27 | Link |
Quality Freak
Join Date: Jun 2007
Location: Area 52
Posts: 597
|
1) I never tested smdegrain because it is the same thing as mdegrain or mvdegrain.
2) If they have no real meaning, then why are you posting here? I'm glad that you think I'm wasting many hours of my time, but keep it to yourself next time. 3) I am simply showing others what the different options out there do so that they can see what they do. Granted, anyone could do this by themselves, but not everyone is going to try to test every option (or several) to see what fits the bill. This is a quick, or somewhat quicker way of determining what may be the right option for them. Also, what is wrong with mixing different denoisers? I did that on purpose. There is no one option, and many functions deliver different results. (As this should be painfully obvious) |
24th September 2011, 19:39 | #28 | Link |
Registered User
Join Date: Nov 2009
Posts: 2,352
|
calm down. Indeed smdegrain is a wrapper of mdegrain, but it can use 16 bit and higher radius with the cretindesalpes mod. Also it adds refinemotion(MRecalculate) which not many wrappers do. Most of your tests involves mdegrain so I add my 2 cents here, but its strange when you also add dfttest, tnlmeans, removegrain, etc
Different sources need different denoisers, here source has a lot of grain, so it's only one example to test denoisers against. I also found mctemporaldenoise to perform the best in this example, mainly because its two-pass option, and fizz noise reduction. I just don't use it much because it requires many plugins, and is slow. I'm not throwing down your work, I only wasn't agree from the point of view, but its only my opinion. I also did some tests a few months ago.
__________________
i7-4790K@Stock::GTX 1070] AviSynth+ filters and mods on GitHub + Discussion thread |
24th September 2011, 19:59 | #29 | Link | ||
Registered User
Join Date: Apr 2002
Location: Germany
Posts: 5,389
|
Quote:
Quote:
Add to that that you are e.g. using FFT3dFilter in 4 different (!) configurations, but all of them are only spatial (albeit the bigger strength in grain removal comes from temporal filtering). In comparison, dfttest (being a "mightier relative" of FFT3D) is used only in spatio-temporal mode, never in spatial-only. That's not really a "fair" comparison of what either filter possibly can do. It smells more like "random settings". Again: there's big respect for the effort, but the strategy is lacking.
__________________
- We´re at the beginning of the end of mankind´s childhood - My little flickr gallery. (Yes indeed, I do have hobbies other than digital video!) |
||
24th September 2011, 20:28 | #30 | Link |
Quality Freak
Join Date: Jun 2007
Location: Area 52
Posts: 597
|
Unfortunately, I cannot possibly post every option there is. I understand where you are coming from, but I do not exactly have oodles of time available to me, and so I chose the options that are, for the most part, used most often in the operation of removing film noise.
For instance, I only used the rather simple calls for dfttest because that is what is used most often, both from my experience and what I have been able to find by searching the forum. The same goes for fft3d. More often than not, the different sigma's are set and very few other options are every messed with, at least from what I could tell. I know that I haven't possibly searched every post or even most of them, so I just never saw many uses of fft3d in any other function other than spatial, and the same for dfttest. I suppose one of the reasons I chose these particular settings was to show off the more popular settings basic users might be prone to messing with and what they might more commonly use. I know that you know far more than I do, Didee, about the different options that can be used. So, I would like to request that if you have time, would you mind considering providing me some settings that you would find preferable to see? I will still continue to upload the other samples from 300 and I will also attempt to get the TNLmeans samples up. Judging from these recent posts, I will also try to find out other syntax's for using fft3d in temporal mode and dfttest in spatial mode. As to the lack of a strategy, I quite agree. I was inspired to do this because of the recent thread for the MeGUI filter revival. But I currently work in the military, and I was only able to conduct my tests at the end of the day where I only have a few hours to myself. My main job takes up most of my time, so I didn't really have the time to try to flesh out more thoughtfulness. I started doing the tests the same night I saw the thread. But I'd still like to press on and continue until it does work Another edit: @Dogway I do apologize for the offensive way I responded, but to me, your short comment of "these tests have no real meaning" came off to me as "therefore, this thread has no meaning" and I took it a little insultingly because of the amount of time and effort I had put into this. I don't think I ever saw a thread before about denoiser comparisons like this (as large or trying to be as comprehensive) and so I have nothing to go off of, as I've also never done anything like this before. Last edited by Nightshiver; 24th September 2011 at 20:33. |
24th September 2011, 20:55 | #31 | Link |
Registered User
Join Date: Jan 2006
Posts: 1,867
|
@Night
I, for one, congratulate our new denoising overlord! I agree, I've never seen such a comparison before. I was already happy to see some screenshots, but now videos? I can immediately see that mctemporaldenoise is superior than other tested settings, and that's good enough for me! However I'm still working on my own objective tests, in a more scientific way. I have some very high quality, noiseless video, which is also copyright free, and I want to test artefacting and filmgrain. @all About the new tests, starting with a very high quality 16bit, noiseless source, I'd like to run denoisers on it with/without added noise, to measure artefacting/denoising. I can provide usual error measures, ssim, mse, psnr, msu noise estimation, msu blur estimation. I want to focus on one filter at a time and it's settings, to determine which affect denoising. Assuming this leads me to find the best settings for each tool (that the settings form a linear trend in performance), I can then compare tools. Data will be categorized by spatial, temporal, or both. Any other ideas for methodology? Please note I'm researching my own questions, I can't necessarily take every request. |
24th September 2011, 21:48 | #32 | Link | |
Registered User
Join Date: Apr 2002
Location: Germany
Posts: 5,389
|
Quote:
- It is a very different ballgame if you have a high-bitrate BR source (with inherently very "clean" grain reproduction), or if you're starting with an oldschool DVD source (or even an SD DVB capture). In a sense, the latter is more difficult, because grain reproduction of bitrate limited Mpeg-2 is not clean. Mpeg-2 compression of (stronger) grain usually introduces lots of DCT noise, and more often than not even DCT flicker (i.e. the low frequencies, or even the DC component, are not temporally stable). In addition to this, using just "AddGrain(x)" is not a overly good approximation of realworld film grain. These artificial grain generators aim for zero-mean in rather small localisations. Film grain usually is much more "rough" than that - it might be reasonably close to zero-mean too, but only in a (much) bigger spatio-temporal window. It's a bit like if you can make some zero-to-hundred sprint tests with a new car, but what you really want to find out is if the new car is suited for city traffic or holiday journey: The test is simple and scientific, but practice is much more complex.
__________________
- We´re at the beginning of the end of mankind´s childhood - My little flickr gallery. (Yes indeed, I do have hobbies other than digital video!) |
|
24th September 2011, 22:37 | #33 | Link |
Registered User
Join Date: Jan 2006
Posts: 1,867
|
@didée
Good point - however, starting with my source I can compress to mpeg2. Anyhow, this makes me think that we don't want denoisers per se, but 'DCT denoisers', siince the properties could be quite different. As for noise, I'm not using addnoise - I'm using a carefully measured noise model for specific films. You are right that they have different properties to guassian noise. Now at least I know why you say 'it depends on the source' - you could at least, be referring to the ability to 'DCT denoise' different compressed noise. I could compress pure noise on a flat frame to see what happens? |
25th September 2011, 01:23 | #34 | Link |
Registered User
Join Date: Nov 2009
Posts: 2,352
|
http://en.wikipedia.org/wiki/Image_noise
http://en.wikipedia.org/wiki/Compression_artifact http://en.wikipedia.org/wiki/Noise_%28electronics%29 @jmac698: For some time now I have been more interested in noise itself than the denoisers. A more pictural example of different types of noise and ways to combat it. From the inherent image grain up to the compressed grain artifacts, through analog electronic devices artifacts. Code:
lossless | | digital________|_________analog | | heavy compressed Plus a 3rd axis indicating clean-------------grainy
__________________
i7-4790K@Stock::GTX 1070] AviSynth+ filters and mods on GitHub + Discussion thread |
25th September 2011, 03:03 | #36 | Link |
Registered User
Join Date: Jan 2006
Posts: 1,867
|
I'd have to say in some cases noise is noise - don't get too carried away about there being different 'types' of noise. I would say temporal denoisers for example reduce *anything* which is not consistent with the image, including specks etc. It's possible for even spatial denoisers to be adaptive in some sense; there certainly are tools which gather a 'noise profile'.
I do have some VHS noise available however, so perhaps I can start by discussing the various types of noise and see how they differ. Off the top of my head, there's about 3 kinds of electronic noise which is based in physics and they have different frequency responses (like pink noise, brown noise etc.). And by DCT-noise we mean normal noise which was compressed; thus it's not the original noise anymore and will have different properties. As for digital noise I think quantization errors (round-off noise) is generally white noise. Noise isn't always bad either, it's useful in different situations. It probably helps with banding without having to dither? Last edited by jmac698; 25th September 2011 at 03:05. |
25th September 2011, 07:11 | #37 | Link |
Registered User
Join Date: Jan 2006
Posts: 1,867
|
@Chikuzen,
Code:
RawSource("d:\sintel_raw_yuv\yuv444p16le.yuv", width=2048, height=436, fpsnum=24, fpsden=1, pixel_type="i444") Do you have any suggestions as a workaround? Thanks |
25th September 2011, 07:54 | #38 | Link | |
typo lover
Join Date: May 2009
Posts: 595
|
Quote:
I must already have changed MAX_WIDTH into 4096. http://forum.doom9.org/showthread.ph...59#post1504259
__________________
my repositories Last edited by Chikuzen; 25th September 2011 at 07:58. |
|
25th September 2011, 08:04 | #39 | Link |
Registered User
Join Date: Jan 2006
Posts: 1,867
|
@Chikuzen,
Thank you for your speedy reply. I am referring to this file: http://avisynth.org/warpenterprises/...l_20060728.zip I see the line you are referring to: Code:
#define maxwidth 2880 Do you see any reason to restrict the width? In fact we need even higher to support all samples. There are even some 4k files available. It's only gonna get bigger too. |
25th September 2011, 08:30 | #40 | Link |
typo lover
Join Date: May 2009
Posts: 595
|
__________________
my repositories |
Thread Tools | Search this Thread |
Display Modes | |
|
|