Welcome to Doom9's Forum, THE in-place to be for everyone interested in DVD conversion.

Before you start posting please read the forum rules. By posting to this forum you agree to abide by the rules.

 

Go Back   Doom9's Forum > Capturing and Editing Video > Avisynth Usage

Reply
 
Thread Tools Search this Thread Display Modes
Old 17th June 2006, 04:15   #1  |  Link
Audionut
Registered User
 
Join Date: Nov 2003
Posts: 1,281
Saving grain for later use

Is there any way (perhaps with masks and such) to create a copy of the grain and perhaps fine details of video, and save that information in to a small file, with the ability to add that information back to the video at playback?

It's only an idea, I would not be able to attempt it myself, nor have I really any idea how to do it.

But I know you guy's use masks and such for denoising/sharpening/etc. Perhaps is there was a way to save that information into say a smaller than 1mb file, then use ffdshow's avisynth to insert it back in at playback.

Would be great for x264!!
Audionut is offline   Reply With Quote
Old 17th June 2006, 04:19   #2  |  Link
Guest
Guest
 
Join Date: Jan 2002
Posts: 21,901
My company's silicon can do something like this (STMicroelectronics' 7109 MPEG/AVC/VC1 decoder). It doesn't save the grain, but it regenerates random grain and adds it to the display frames. It is very effective at eliminating that plastic look. There's no need to save the actual grain because it is random.

I also noticed Sony's big LCOS TVs have a slightly grainy display panel surface that achieves the same effect!

Last edited by Guest; 17th June 2006 at 04:22.
Guest is offline   Reply With Quote
Old 17th June 2006, 04:53   #3  |  Link
FredThompson
Registered User
 
FredThompson's Avatar
 
Join Date: Feb 2002
Location: Charlotte, NC USA
Posts: 1,984
ffdshow can add some grain during playback. It IS a good way to reduce excess smoothness.

Hmm...I wonder if harshly removing grain and using a slight application of MSU's cartoon filter would give a semi-CGI appearance. That would be kind of funny, emulating CGI with live action.
__________________
Reclusive fart.
Collecting Military, Trains, Cooking, Woodworking, Fighting Illini, Auburn Tigers
FredThompson is offline   Reply With Quote
Old 17th June 2006, 04:59   #4  |  Link
Mug Funky
interlace this!
 
Mug Funky's Avatar
 
Join Date: Jun 2003
Location: i'm in ur transfers, addin noise
Posts: 4,555
do you mean something similar to wavpack's dualstream feature, where a lossy track is stored, and a residual is also stored losslessly so they can be combined later?

easiest way would be to run your favourite denoiser, then subtract the result from the original.

then encode both versions (maybe the noisy one with a different codec?) and subtract them again on playback using an avisynth script.

i can't see it being terribly useful though - random noise doesn't compress very well at all, and will tend to swim around just like it would if it were left in the original (perhaps it'll swim in different directions which could be an improvement). this means you'll have twice (or more) the filesize to do what an addgrain on playback would do.

as neuron alluded to, newer encoding technologies can add grain on playback. i think h.264 (though no current software implementations that i know of) will store grain paramerically and reproduce it on playback. it wont be the same grain, but will have the same properties and look the same. quite clever actually, considering how much grain is removed by h.264 even at relatively low quantisers (~17-20).
__________________
sucking the life out of your videos since 2004
Mug Funky is offline   Reply With Quote
Old 17th June 2006, 05:50   #5  |  Link
foxyshadis
ангел смерти
 
foxyshadis's Avatar
 
Join Date: Nov 2004
Location: Lost
Posts: 9,556
Use this prior to denoising or encoding and save it:
Code:
	grain=default(grain,10)
	gthresh=default(gthresh,1.0)
	gmask=fft3dfilter(sigma=gthresh*10,sigma2=gthresh*4,sigma3=gthresh,sigma4=0,plane=4) \
		.mt_lutxy(last,expr="x y - abs",chroma="process") \
		.BilinearResize(m(4,width/4),m(4,height/4)).mt_lut(expr="x 2 ^",chroma="process")
and on playback:
Code:
mt_merge(last,addgrainc(grain*10,grain*2),gmask.bilinearresize(width,height),chroma="process")
This is an extremely avisynth-specific implementation of the general idea: form a mask of noisy areas from the main picture, shrink it to a blobby mess, then add random noise back to the picture based on the (enlarged) mask. It'll definitely compress extremely well. I believe that's exactly what AVC FGS does. You'd have to hack ffdshow to support playback merging of an external stream this way, probably so much that it'd be easier to add FGS to lavc instead.

A smart enough grain filter could probably determine the best areas to apply it back to without an external reference, which would be an interesting area of research someday. Probably too mathematical for me though!

Last edited by foxyshadis; 17th June 2006 at 06:02.
foxyshadis is offline   Reply With Quote
Old 17th June 2006, 09:20   #6  |  Link
Boulder
Pig on the wing
 
Boulder's Avatar
 
Join Date: Mar 2002
Location: Finland
Posts: 5,717
A grain factory created by Didée, needs MaskTools v1.5.x and AddGrain:
Code:
function GrainFactory(clip last,int "grain1_strength",int "grain2_strength",int "grain1_sharpness",int "grain2_sharpness")
{

grain1_strength=default(grain1_strength,5)
grain2_strength=default(grain2_strength,2)
grain1_sharpness=default(grain1_sharpness,75)
grain2_sharpness=default(grain2_sharpness,75)

#grain1_strength  = 7      # [  0 - ???]  strength of grain / for mid-tone areas
grain1_size      = 1.15    # [0.5 - 4.0]  size of grain     / for mid-tone areas 

#grain2_strength  = 0       # [  0 - ???]  strength of grain / for dark & bright areas
grain2_size      = 0.8     # [0.5 - 4.0]  size of grain     / for dark & bright areas 

#grain1_sharpness = 75     # [  0 - 100]  very soft (0) to very sharp(100) 
#grain2_sharpness = 75     # [  0 - 100]                     grain definition 

grain1_texture   = 0     # [  0 - 100]  strength of "material texture" appearance
grain2_texture   = 0      # [  0 - 100]                   (in fact: 'bump' effect)

ontop_grain      = 0       # [  0 - ???]  additional grain to put on top of prev. generated grain 

temp_avg         = 20      # [  0 - 100]  percentage of noise's temporal averaging 

brightness_bias  = 0.96    # [0.5 - 2.0]  bias for  dark|bright --> midtone  fading 

#----------------------------------------------------------------------------------- 
o    = last
ox   = o.width
oy   = o.height
sx1  = m4(ox/grain1_size)
sy1  = m4(oy/grain1_size)
sx1a = m4((ox+sx1)/2.0)
sy1a = m4((oy+sy1)/2.0)
sx2  = m4(ox/grain2_size)
sy2  = m4(oy/grain2_size)
sx2a = m4((ox+sx2)/2.0)
sy2a = m4((oy+sy2)/2.0)
b1   = grain1_sharpness/(-50.0) + 1.0
b2   = grain2_sharpness/(-50.0) + 1.0
b1a  = b1/2.0
b2a  = b2/2.0
c1   = (1.0-b1)/2.0
c2   = (1.0-b2)/2.0
c1a  = c1/2.0
c2a  = c2/2.0
t1a  = string(grain1_texture)
t1b  = string(100-grain1_texture)
t2a  = string(grain2_texture)
t2b  = string(100-grain2_texture)
tmpavg = temp_avg/100.0

#-----------------------------------------------------------------------------------

ADD_DIFF    = "x y - 128 +"
BUMPKERNEL1 = T1a + " 0 0 0 " + T1b + " 0 0 0 0"
BUMPKERNEL2 = T2a + " 0 0 0 " + T2b + " 0 0 0 0"
CENTER2MAX_BIAS = "x 120 - abs "+string(brightness_bias)+" ^ 2.5 *"

#-----------------------------------------------------------------------------------

grainlayer1 = blankclip(o, width=sx1, height=sy1, color=$808080) .AddGrain(grain1_strength, 0, 0)

grainlayer1 = (grain1_texture==0)
 \           ? grainlayer1
 \           : yv12lutxy(grainlayer1,grainlayer1.DEdgeMask(0,255,0,255, BUMPKERNEL1), ADD_DIFF)

grainlayer1 = (grain1_size == 1.0) ? grainlayer1
 \          : (grain1_size < 0.75) ? grainlayer1.bicubicresize(sx1a,sy1a, b1a, c1a).bicubicresize(ox,oy, b1a,c1a)
 \          :                        grainlayer1.bicubicresize(ox,oy, b1,c1)

#-----------------------------------------------------------------------------------

grainlayer2 = blankclip(o, width=sx2, height=sy2, color=$808080) .AddGrain(grain2_strength, 0, 0)

grainlayer2 = (grain2_texture==0)
 \           ? grainlayer2
 \           : yv12lutxy(grainlayer2,grainlayer2.DEdgeMask(0,255,0,255, BUMPKERNEL1), ADD_DIFF)

grainlayer2 = (grain2_size == 1.0) ? grainlayer2
 \          : (grain2_size < 0.75) ? grainlayer2.bicubicresize(sx2a,sy2a, b2a, c2a).bicubicresize(ox,oy, b2a,c2a)
 \          :                        grainlayer2.bicubicresize(ox,oy, b2,c2)

#-----------------------------------------------------------------------------------

grainlayer = MaskedMerge(grainlayer1, grainlayer2, o.yv12lut(CENTER2MAX_BIAS), U=1,V=1)

grainlayer = (temp_avg==0) ? grainlayer
 \                         : grainlayer.mergeluma(grainlayer.temporalsoften(1,255,255,255,2), tmpavg)
grainlayer = (ontop_grain==0) ? grainlayer
 \                            : grainlayer.AddGrain(ontop_grain,0,0)

#-----------------------------------------------------------------------------------

result = yv12lutxy(o, grainlayer, ADD_DIFF, U=2,V=2)

return result
}
#-----------------------------------------------------------------------------------
function m4(float x) {return( x<16?16:int(round(x/4.0)*4)) }
#===================================================================================
It would be nice if someone could modify it for MaskTools v2, I recently tried it but I'm not sure if it worked just like this older version.

The function can be used to create a very nice film grain effect and add fake details to the image.
__________________
And if the band you're in starts playing different tunes
I'll see you on the dark side of the Moon...
Boulder is offline   Reply With Quote
Old 17th June 2006, 09:55   #7  |  Link
Audionut
Registered User
 
Join Date: Nov 2003
Posts: 1,281
I've never really liked any of the addgrain implementations of ffdshow for one reason or the other.

X264 removes grain at lower bitrates.

I was thinking that if it was possible to take the "original" grain from the video and add it back at playback, that would be better than adding artificial grain.
Audionut is offline   Reply With Quote
Old 17th June 2006, 11:33   #8  |  Link
Guest
Guest
 
Join Date: Jan 2002
Posts: 21,901
Compression is achieved by throwing away grain and small detail (among other things). Keeping it is equivalent to using a higher bitrate. As they say, you can't have your cake and eat it too.
Guest is offline   Reply With Quote
Old 17th June 2006, 11:48   #9  |  Link
Alain2
Registered User
 
Join Date: May 2005
Posts: 236
Quote:
Originally Posted by neuron2
My company's silicon can do something like this (STMicroelectronics' 7109 MPEG/AVC/VC1 decoder). It doesn't save the grain, but it regenerates random grain and adds it to the display frames. It is very effective at eliminating that plastic look. There's no need to save the actual grain because it is random.

I also noticed Sony's big LCOS TVs have a slightly grainy display panel surface that achieves the same effect!
Can someone explain to me why it is bad to remove noise (not details)? In the real world, my eyes aren't seeing random noise when I look at someone, so why should we expect noise on films?

You all say that x264 is removing too much noise and gives a plastic look, but to me it's getting closer to reality... When I filter the noise but keep details, and then encode with --filter -3:-1 or so to limit the blur from inloop, this preserves quite well the textures, and I don't have the "plastic" impression that I had with the rv10 codec for instance..
Alain2 is offline   Reply With Quote
Old 17th June 2006, 15:54   #10  |  Link
MrTroy
Registered User
 
Join Date: Apr 2005
Posts: 224
Quote:
Originally Posted by Alain2
Can someone explain to me why it is bad to remove noise (not details)?
Another example of 'you can't have one without the other'. Removing noise removes details. Too much detail removing creates a plastic-y look. Adding grain masks this bad effect.
__________________
PAL FACTS
* 20% more horizontal scan lines than NTSC
* No telecine judder
MrTroy is offline   Reply With Quote
Old 17th June 2006, 16:55   #11  |  Link
Alain2
Registered User
 
Join Date: May 2005
Posts: 236
Yes, but you can greatly preserve details with motion compensation tools, and remove mainly noise ; I do plenty of comparison between my source and my filtered clip, I hardly see what detail is altered, only the noise ; I think the noise is a way to compensate the small pixel resolution we have, that makes all small movements obvious and unnatural on a noiseless pictures, but as we increase the resolution of the clips, I think it will be less and less needed to do this..
Alain2 is offline   Reply With Quote
Old 17th June 2006, 23:16   #12  |  Link
Soulhunter
Bored...
 
Soulhunter's Avatar
 
Join Date: Apr 2003
Location: Unknown
Posts: 2,812
New: Now with 500% more words!

Quote:
Originally Posted by neuron2
There's no need to save the actual grain because it is random.
Wrong! Filmgrain isnt random... While noise has mostly fixed characteristics, the characteristics of filmgrain depend on many things like brightness, processing or the film material... To quote myself: "Basically film grain is nothing more than a lot silver particle clusters... Most motion-picture film material is made of 3 layers (one for each color -> RGB) and they contain dye + silver salts. In the dark parts of the image the salts get transformed into metallic silver (which blocks the light) when the film gets exposed, this is also the reason grain is usually much rougher in dark areas than in bright ones! Why are some flicks more grainy than others? Well, you can get celluloid in different granularity grades! Why? Celluloid with low granularity has a lower photo-sensitivity (lower ISO speed) and celluloid with high granularity has a higher photo-sensitivity (higher ISO speed) Means: For celluloid with low granularity you need much more illumination than youd need for celluloid with a high granularity! The film format makes a difference as well, Super8 film for example looks usually more grainy than 35mm film (8mm = smaller than 35mm = needs more upsizing = grain becomes more visible). And, last but not least there is also postprocessing/filtering which can be used to remove or enhance the grain! And what is noise? Noise could be anything, depends on the context... For video its basically everything that distorts the signal, and its mostly a unwanted by-product (analog interferences, compression artefacts etc.)" Another thing to consider is the color... Each layer of the film contains a different amount of grain, the blue layer for example is usually much more grainy than the red/green one! So, to reproduce grain accurately youd need to analyze the luma and the underlying chroma of each pixel and add different amounts and sizes of grain depending on this!

Anyway, here is another filter to add noise via avisynth... ^^
__________________

Visit my IRC channel

Last edited by Soulhunter; 17th June 2006 at 23:23.
Soulhunter is offline   Reply With Quote
Old 17th June 2006, 23:39   #13  |  Link
Guest
Guest
 
Join Date: Jan 2002
Posts: 21,901
Well, I oversimplified. Yes, the grain that is recreated is parameterized from local image content. What I meant was it is not necessary to save the exact grain from the source.
Guest is offline   Reply With Quote
Old 18th June 2006, 01:16   #14  |  Link
FredThompson
Registered User
 
FredThompson's Avatar
 
Join Date: Feb 2002
Location: Charlotte, NC USA
Posts: 1,984
@soulhunter,

You're vague, again. Would it be too much to ask for you to be specific in your posts?
A little attention to detail and some background information never hurt anyone.
__________________
Reclusive fart.
Collecting Military, Trains, Cooking, Woodworking, Fighting Illini, Auburn Tigers

Last edited by FredThompson; 18th June 2006 at 01:19.
FredThompson is offline   Reply With Quote
Old 18th June 2006, 04:11   #15  |  Link
Soulhunter
Bored...
 
Soulhunter's Avatar
 
Join Date: Apr 2003
Location: Unknown
Posts: 2,812
Quote:
Originally Posted by FredThompson
You're vague, again. Would it be too much to ask for you to be specific in your posts?
What do you mean? Why grain in dark areas looks rougher? The dark areas are dark coz silver transforms and blocks the light, so this area becomes less transparent, therefore the grain becomes more visible... The bright areas stay more transparent (let more light through) so you see less grain! Why grain is usually more visible in blue areas? Coz the grain in the blue emulsion has usually a higher diameter (up to 1/3 higher iirc) than in the red / green one! However, all this things can vary a lot depending on the exact purpose the film material was made / developed / thought / tweaked for -> in-door shooting, outdoor shooting, low light condition, etc... Or you meant that you want some more general info about film / photography? Some random links: 1, 2, 3, 4, 5. But theres much much more out there, google is your friend! ;)


Bye
__________________

Visit my IRC channel

Last edited by Soulhunter; 18th June 2006 at 05:35.
Soulhunter is offline   Reply With Quote
Old 18th June 2006, 04:59   #16  |  Link
FredThompson
Registered User
 
FredThompson's Avatar
 
Join Date: Feb 2002
Location: Charlotte, NC USA
Posts: 1,984
Sorry, I still don't understand what you're trying to explain. Can you give more details?

It's obvious you're trying to be helpful. I appreciate that. It's still a little...to...vague...
__________________
Reclusive fart.
Collecting Military, Trains, Cooking, Woodworking, Fighting Illini, Auburn Tigers
FredThompson is offline   Reply With Quote
Old 18th June 2006, 05:16   #17  |  Link
Guest
Guest
 
Join Date: Jan 2002
Posts: 21,901
Fred, please knock it off. A rule 11 strike is imminent. Thank you.

http://forum.doom9.org/forum-rules.htm

If you have any questions, please follow up in PM, not here.
Guest is offline   Reply With Quote
Old 18th June 2006, 06:25   #18  |  Link
FredThompson
Registered User
 
FredThompson's Avatar
 
Join Date: Feb 2002
Location: Charlotte, NC USA
Posts: 1,984
Geez, lighten up. The smiley should make the first reply obvious. He's got thick enough skin to recognize the backhanded comment and the humor.

edit: That's supposed to read "compliment" - should be obvious ...
__________________
Reclusive fart.
Collecting Military, Trains, Cooking, Woodworking, Fighting Illini, Auburn Tigers

Last edited by FredThompson; 18th June 2006 at 09:07.
FredThompson is offline   Reply With Quote
Old 18th June 2006, 08:28   #19  |  Link
Guest
Guest
 
Join Date: Jan 2002
Posts: 21,901
Quote:
Originally Posted by FredThompson
Geez, lighten up. The smiley should make the first reply obvious. He's got thick enough skin to recognize the backhanded comment and the humor.
I asked you to take discussion to PM. Therefore, struck for rule 16.
Guest is offline   Reply With Quote
Old 18th June 2006, 10:36   #20  |  Link
bond
Registered User
 
Join Date: Nov 2001
Posts: 9,770
Quote:
Originally Posted by Mug Funky
as neuron alluded to, newer encoding technologies can add grain on playback. i think h.264 (though no current software implementations that i know of) will store grain paramerically and reproduce it on playback. it wont be the same grain, but will have the same properties and look the same. quite clever actually, considering how much grain is removed by h.264 even at relatively low quantisers (~17-20).
are you sure that it wont be the same grain?

actually i think if someone would be willing to write an avs plugin that outputs the grain info into a file aku will add support for the avc filmgrain feature to x264
__________________
Between the weak and the strong one it is the freedom which oppresses and the law that liberates (Jean Jacques Rousseau)
I know, that I know nothing (Socrates)

MPEG-4 ASP FAQ | AVC/H.264 FAQ | AAC FAQ | MP4 FAQ | MP4Menu stores DVD Menus in MP4 (guide)
Ogg Theora | Ogg Vorbis
use WM9 today and get Micro$oft controlling the A/V market tomorrow for free
bond is offline   Reply With Quote
Reply

Thread Tools Search this Thread
Search this Thread:

Advanced Search
Display Modes

Posting Rules
You may not post new threads
You may not post replies
You may not post attachments
You may not edit your posts

BB code is On
Smilies are On
[IMG] code is On
HTML code is Off

Forum Jump


All times are GMT +1. The time now is 12:45.


Powered by vBulletin® Version 3.8.11
Copyright ©2000 - 2024, vBulletin Solutions Inc.