Need Suggestions for VERY GRAINY source

[Terranigma]

Originally Posted by Sagekilla

Will do I'm really curious as to how much of an improve MVDegrain 3 will be making, and it seems like (from what I can infer) it uses an extra forward and backward motion vector.

Well, I guess you could compare frames 3 to frames 2 from spuds' mc denoiser. Not sure it'd be that much of a difference like that script though. For now, we must speculate. :P

The thing I'm most interested in, is the faster mvanalyze.
That's why I made this post.

[Zep]

Originally Posted by Zanejin

Grain is apparently becoming more and more popular and is not as undesired as one may think. It may give the illusion of detail and improve perceived color depth by dithering, or its abundance may simply appear more attractive to the general public. Much of 300's grain was added after filming.

yes but they added dancing grain and that is what really bugs me.


an example. The Spartan queen is very beautiful and in one close up shot of her face there was so much dancing grain her cute little birth mark could not be seen cause the dancing pixels were all over it to the point it to looked like grain. Same thing with her eye lashes. Total blasted out because of the dancing grain which makes it hard to see any detail that happens to be about the size of the grain. very distracting to me

[Sagekilla]

Originally Posted by Zep

yes but they added dancing grain and that is what really bugs me.


an example. The Spartan queen is very beautiful and in one close up shot of her face there was so much dancing grain her cute little birth mark could not be seen cause the dancing pixels were all over it to the point it to looked like grain. Same thing with her eye lashes. Total blasted out because of the dancing grain which makes it hard to see any detail that happens to be about the size of the grain. very distracting to me

Indeed, I'm willing to settle with the level of grain present in 300 if it was not dancing grain. But, thanks to Didee I'll be using this script to see how well it degrains other noisy material and if it does work just as well, grain may very well disappear from my x264 encodes.

[Terranigma]

Originally Posted by Zep

yes but they added dancing grain and that is what really bugs me.

I've noticed this dancing grain in newer dvd's as well. Looks like the only thing that can be done about it is to encode as mpeg-2, since it tends to hold grain easily, and banding? Not a problem with mpeg-2. :P

[Sagekilla]

Originally Posted by Terranigma

I've noticed this dancing grain in newer dvd's as well. Looks like the only thing that can be done about it is to encode as mpeg-2, since it tends to hold grain easily, and banding? Not a problem with mpeg-2. :P

I've noticed that as well. MPEG-2 seems to handle video very well at a decent bitrate, whereas x264 you need to play around with the settings a bit to get it to work nicely, even with high bitrates. Kind of feels like we've gone two steps forward in terms of compression and a few more backwards in terms of grain retention/banding/other artifacts

[Zep]

Originally Posted by Didée

To not get accused of only babbling & critizising, without demonstrating something better, I gave it a go. Note it was quite an effort, since my PC is sub- (sub-sub-sub-...)-par for using demanding filters on & H264-encoding of HD content. You guys have machines that crunch 10 times faster than mine ...

Sample: MegaUpload -or- RapidShare (67MB)
[edit: now with working links...]

Bitrate came out a tad higher than that of Zep's sample (his: 4992 kbit, mine: 5066 kbit). No color correction was done. The grain was processed strictly temporal (spatial processing was only used for probability checks, but not applied in the denoising chain), and straightforward: no kind of area/detail/whatever masking was used.
Also, no arbitrary "pimp-it-up" sharpening was used; what has been used is a "dont-add-more-than-what-was-removed-previously" kind of sharpening. The goal was to keep the result looking the same as the original does ... just without the grain.

The filterchain definetly could benefit from some additional twists, but my pre-Christian Celeron doesn't allow for more. (That's why the result is named "easy".) The x264 encoding surely could be done better too - I'm an old-fashioned Xvid user, tweaking x264 is too complicated for me. (truley, I'm not fully satisfied with what I managed to get from x264. Is someone giving lessons?)


Some screenshots:

(middle= original / left= Zep's / right= my try)

Frame 50: <= =>

Frame 686: <= =>

Frame 910: <= =>

Frame 1274: <= =>

Frame 2100: <= =>


[post break because of image limit]

I'm downloading the example now. Will check it out shortly.

Please delete some stuff in your inbox so I can reply to your PM.
I wrote back but am unable to send it to you as I get a "Can not send inbox is full error"

note I did not temporal really. mine was spatial removable mostly.

the single frames look good though that you did now to see if
the dancing is gone :P

[Zep]

Originally Posted by Didée

Note#2: The script uses MVDegrain3. If you don't have MVTools v1.9.0, you can either use MVDegrain2 instead (with slightly worse results), or make a donation to Fizick (recommended!) to get the goodies more early.

I will give your script a go and yes though dancing grains is my main concern I do prefer at least 50% reduction in grain straight up. I truly feel they added too much heavy grain to this movie.


MVDegrain3?????? No fair!!!! access to tools we (I) do not have hahahahaha

[Sagekilla]

Personally, I think your spatial degrain looks a lot smoother and has an airbrushed look, Zep. Didee's looks like it retains more of the fine nuances in skin detail.

[Zep]

Originally Posted by Sagekilla

Very nicely done Didee.. Your script looks like it retains much more detail than Zep's does, with that said I have to hand it to both of you.. Both are an amazing improvement over the original, well done

I'd really love to see the filter chain you used to degrain the movie like that Didee! Also, it seems like the reason why yours was a slightly higher bitrate could because of the more detail yours retains. I noticed in the close up of faces yours had more skin detail than Zep's.

yeah I like them both a lot. I do want to pump up the selective sat more in Didee's and add in a some more selective masking to try and better hit only the areas that annoy ME but seem to be fine for Didee.


oh and hey slight advantage to didee for getting to work
FROM my better than original but I admitted could be improved starting point Now that I have his scripts I think I can do even better than either of ours by merging some of mine with his. Then didee will get to a new starting point to work from hahahaha

at this rate we may make it better looking than the I-MAX release

[Zep]

Originally Posted by Terranigma

Perhaps because zep's was oversharpened? Sure, it may look great while playing, but when zoomed, you can see the detail that was destroyed. It's never good to oversharpen; there's a detail analysis out there that talks about edge-enhancement; maybe you could find it.

maybe. I used Didee SEESAW and it was the very first time I ever use it. I went with settings that pokie posted in a thread than brought out /sharpened and episode of "Lost"

I never tweaked it because I do not know enough about SEESAW
to know what does what. The problem I am still having is getting the perfect masks. some scene I get a perfect one then other it is way off (even more so in a fast cut scene change the first frame or 2 the masks suck)

[Zep]

Originally Posted by Sagekilla

Mmm, yes it is quite a bit faster than an HD source, a blazing 0.87 fps! Only settings not maxed are refs (6) and ME (umh)

But, back on topic though, was your script (generally speaking) about the same methodology that Didee was using for degraining?

some yes some no. His is more temporal, mine more spatial.

the thing that I MUST make clear is that Didee's and my goals on the final are not the same. I want all dancing grain removed
and more non dancing grain removed.

Didee is ok with leaving a good amount of grain in and looking for more of a balance to save detail and maybe give it a more natural look etc...


Me am leaning more towards more grain removal at the expense of slight detail loss because for me it is all about MOTION. His frame to frame compares to mine will always be better. heck I like them better also. But when watching the clips I lean towards my latest test clips I did today. Granted I now am using Didee's script tweak added with my own.

MY WHOLE POINT of that RANT?


this stuff is very subjective! I thought I was ok with a fair amount of grain left in as long as it was not dancing grain but it appears I hate any grain more than I thought

[Zep]

Originally Posted by Terranigma

I've noticed this dancing grain in newer dvd's as well. Looks like the only thing that can be done about it is to encode as mpeg-2, since it tends to hold grain easily, and banding? Not a problem with mpeg-2. :P

yes! and I want to know WHY? damn it they should leave it clear and crisp. If we want grain it is much easier to ADD then to remove. :P

[Terranigma]

Originally Posted by Zep

yes! and I want to know WHY?

To annoy consumers

[Zep]

Originally Posted by Didée

Zep!

Didee!


clean out your PM box I can't send my reply to you because your PM space is all used up

let me know in PM when I can try to send my reply again. yes I answered your questions and yes you will find it interesting
(your first sentence was correct on so many levels haha)


thanks,


Zep

[Zep]

Originally Posted by Terranigma

To annoy consumers

LOL yeah :P


seriously though I wonder why they added so much to 300. Grain is one thing but all that *dancing* grain is another and really does take away from the enjoyment (For me anyway) of the movie.

[Zep]

Originally Posted by Didée

Hope this is sufficient to make my point: removing grain is okay if it annoys, but I'd want my result to keep the "natural" look. Sharp soup I like to eat, but not to watch.

So you are saying pookie's settings to your SEESAW added too much sharpening to my final huh

yes I used SEESAW :P and it was the very first time ever I used it and since I had no idea what settings to use I used one pookie posted somewhere showing sharpening to an episode of "Lost". yeah now that I think about it I should have turned it down some as I see looking at the pookie post he really brought out the detail/sharpen a lot in his example still frame compare. I probably should have cut the sharpening in SEESAW by about half and went more temporal for the grain removal/dancing. I do like the added rich colors I did though. I think the original was a little bland also. I feel now that with your script and mine and the tweaks I did today it is really rocking.

only down side the final encode is gonna take a week

[Zep]

Originally Posted by Didée

The biggest difference to Zep's denoising is - most probably, since we didn't see his script up to now - the usage of fft3dfilter. It seems that he used to much of it, or at to strong settings. All the ringing and texture echoing is a typical side-effect fft3dfilter, when not used with enough caution. Judging from the overall look, I'd guess that Zep first did an initial filtering with fft3dfilter, and then continued to further process the result with MV-denoising. (Some places look like fft3d-banding that has been temporally averaged, but it's hard to judge after x264-compression.)

In comparison, my script never applies fft3dfilter in the chain that produces the output clip. Instead, it is only used as a "brake" to keep the 1st temporal filter within reasonable bounds. (Which is essential in the bigger scripts where the 1st stage is done with median filtering.)

basically correct. I on purpose went overkill on fft3dfilter with sigma 8 on luma only to get rid of more than just dancing grain but as much grain as I could without totally destroying the detail. Some mush came from this and why I had to add masking and then later I decided to add SEESAW but now that I look back at the final encode I feel the settings were too strong on BOTH fft3dfilter and seesaw. If I went sigma 6 then I could have lowered the SEESAW strength a lot and got more balanced encode I guess.

Still I do like mine a lot than the original and up until just 2 weeks ago I never ever did grain removal before. This was my very first foray into this area.


I have learned much in this thread over the last 2 weeks and now with your script and some of mine added and tweaked I think my next final encode it gonna just be mega awesome! ( for my tastes anyway haha)

(you may still think I blasted out too much grain)

[Zep]

@ Didee,

I like very much your final test. (I just watched it) Even though it leaves a little grain it is more than ok even for me because the detail is more even across the whole clip (I clearly overdid removal which then forced me to over sharpen)

Now the hard part for you. That scene is on the LOW END of the grain dancing and brutal grain scale. I picked that scene because it is a favorite scene of mine and also why I left the audio in when I made the clips

I ran your script and it is now having a harder time on the night scenes and the scenes where the grain is really really bad. My original final made them great and better than the clips I uploaded. My guess is than some of the cons were hidden (like ringing or halo) because they are dark night scenes and the eye can't spot them so easy. The grain looks even worse then because dark night sky with WHITE grain dancing stands out even more etc...

So now it has me wondering if you would be able to add some way to determine darker scenes and/or more grain so that your script will adjust the strengths during those scenes.

I can UL some clips for you to test on if you think it is possible and worth adding to your script.

PM me so we can talk more


Huge thanks!


Zep

[Terranigma]

Originally Posted by Zep

I ran your script and it is now having a harder time on the night scenes and the scenes where the grain is really really bad.

Hmm, I thought that was just me.

After looking at the script, I don't think it's designed for dark noise filtering, but I could be wrong.

[Zep]

Originally Posted by Terranigma

Hmm, I thought that was just me.

After looking at the script, I don't think it's designed for dark noise filtering, but I could be wrong.

Well to be fair Didee only had a very bright day scene to work with and he did an amazing job with it. I like it better than mine on grain and detail but I like mine better in night scenes and mega grain scenes.

if he could add a way to detect and change strengths based on darkness and grain amount. I would not even bother with my script anymore and I would only add some color masking to the chain so I could pick and choose sat boost / color corrections because I feel the original color wise is bland and washed out some.