basic repeatability question

[BabaG]

i'm terrible at scripting and programming and have a pretty basic question. if i have a script, and i'm thinking along the lines of add/subtract noise and grain here, and i run filters for said processing, is the result always precisely repeatable or does it vary if the script is rerun without changes?

thanks,
babag

[poisondeathray]

Some functions have a "randomness" to them - such as some types of dithering, and some types of adding noise and grain - those will not be repeatable

[BabaG]

thanks pdr. is there any easy way to know which filter is or isn't repeatable?

[poisondeathray]

is there any easy way to know which filter is or isn't repeatable?

I'm not aware of any official list, but you can test it pretty easily .

(I guess it depends on your definition of "easy" )

[BabaG]

reason i asked is that i've been playing with cleaning up some old and incredibly grainy film transfers of late. removing the grain has been only slightly successful because of its size. some of this material was shot on tri-x, a grainy 16mm stock, that expired years before it was shot. the dmax had fallen a lot.

the grain in this material seems so severe that the filters i've tried seem to get confused between actual image detail and the grain itself. it occurred to me that i might be able to remedy that by sprinkling some finer noise over the grainy image before degraining. that actually seems to have helped.

now i'm wondering if layering multiple copies of the degrained files might diffuse the remaining grain, similarly to how the astronomical photography folks seem to get quality from layering their images.

it would be highly time/storage intensive but i though it might be interesting to try. only works, though, if the layered copies are different from each other as far as their individual degraining goes.

thanks,
babag

[poisondeathray]

reason i asked is that i've been playing with cleaning up some old and incredibly grainy film transfers of late. removing the grain has been only slightly successful because of its size. some of this material was shot on tri-x, a grainy 16mm stock, that expired years before it was shot. the dmax had fallen a lot.

the grain in this material seems so severe that the filters i've tried seem to get confused between actual image detail and the grain itself. it occurred to me that i might be able to remedy that by sprinkling some finer noise over the grainy image before degraining. that actually seems to have helped.

now i'm wondering if layering multiple copies of the degrained files might diffuse the remaining grain, similarly to how the astronomical photography folks seem to get quality from layering their images.

it would be highly time/storage intensive but i though it might be interesting to try. only works, though, if the layered copies are different from each other as far as their individual degraining goes.

thanks,
babag



Unlikely to help much - because it's not a "random" distribution of grain in each version of your transfer - The difference for your scenario is you have 1 version, 1 snapshot in time . That is different from astronomical layering of images - or any related techniques such as photoshop stacking, or mean/median of aligned images - where there are multiple versions, and it's the noise between the versions that is roughly "randomly" distributed

Also most of the "degrain" filters in avsiynth should be "repeatable" or deterministic - you get the same results every time. (If you work at higher depth, and go to a lower depth - that might include a dithering step which might include some "randomness" ,but dithering differences would not conceivably help much with your scenario)

It's the add grain/add noise type that generally have a "random seed", not the remove grain/noise type of filters - and that generally won't be beneficial. You mentioned adding some noise "seemed to have helped", but I doubt it helped much.

If you post a sample, someone might have some suggestions for other approaches

[BabaG]

hmm. ok. well, that should save a lot of work!

i'll look at posting a sample. is a still image good enough or does it need to be running footage? the original files for this are either 2048x1536 or 2880x2160.

thanks,
babag

[poisondeathray]

hmm. ok. well, that should save a lot of work!

i'll look at posting a sample. is a still image good enough or does it need to be running footage? the original files for this are either 2048x1536 or 2880x2160.

thanks,
babag


A still image cannot represent temporal grain/noise fluctuations or patterns

Needs to be footage

[BabaG]

never done this before so hopefully it works:

https://mega.nz/file/yRZx2Y7K#D2e0bhpE4Awmg4XHYQalNEnZiN7ejds3QADA_E-RkOI

i hope 1 sec is enough to evaluate.

thanks,
babag

[poisondeathray]

never done this before so hopefully it works:

https://mega.nz/file/yRZx2Y7K#D2e0bhpE4Awmg4XHYQalNEnZiN7ejds3QADA_E-RkOI

i hope 1 sec is enough to evaluate.

thanks,
babag

Thanks for the sample, it's in rough shape

For UHD you can try downscaling and/or use increased blksize if using mvtools2 based degrainers . There is no way this scan has actual UHD resolution (as in resolvable details) . And the Prores compression artifacts don't help

Personally I would use Neat Video for this. Fully adjustable, fairly fast if you have a decent GPU

[BabaG]

that's among the worst of the images. not really looking for perfection, here, just better. thanks for the neat video tip.

babag

[anton_foy]

Thanks for the sample, it's in rough shape

For UHD you can try downscaling and/or use increased blksize if using mvtools2 based degrainers . There is no way this scan has actual UHD resolution (as in resolvable details) . And the Prores compression artifacts don't help

Personally I would use Neat Video for this. Fully adjustable, fairly fast if you have a decent GPU

Sorry if off topic but which codec and compression would be more suitable for this kind of footage in your opinion?

@BabaG
You could try my script called "Clay". I had really bad flickering and blotchy noise that I eliminated and could even see more details after degraining than before.

[poisondeathray]

Sorry if off topic but which codec and compression would be more suitable for this kind of footage in your opinion?

Lossless compression - sources like this are very "taxing" for any type of lossy compression



You could try my script called "Clay". I had really bad flickering and blotchy noise that I eliminated and could even see more details after degraining than before.

Can you recommend some settings for this source ?

[anton_foy]

Can you recommend some settings for this source ?

I used the defaults for my very noisy clip but keep in mind the heavier denoising is affecting the darker parts and the lighter degraining on the brighter parts. I am not at home until later this week to try it out on my comp. Try first default settings and then play with the masking (lo, mid, hi, mix), thsad and tr.

[BabaG]

thanks anton_foy. where do i find "Clay?"

babag

[anton_foy]

thanks anton_foy. where do i find "Clay?"

babag

https://forum.doom9.org/showthread.php?t=184938

[BabaG]

thanks. being dumb at this stuff, it seems to be in two parts. do i make the first one a Clay.avsi file and put it in plugins folder? second one save as .avs and run from wherever my material is? anxious to try it.

thanks again,
babag

[anton_foy]

thanks. being dumb at this stuff, it seems to be in two parts. do i make the first one a Clay.avsi file and put it in plugins folder? second one save as .avs and run from wherever my material is? anxious to try it.

thanks again,
babag

Yes you got it correct :)

[BabaG]

great! off to set it up.

thanks again,
babag

[BabaG]

Script error: There is no function named 'ex_bs'.

[poisondeathray]

Script error: There is no function named 'ex_bs'.

Dogway's ExTools.avsi
https://github.com/Dogway/Avisynth-Scripts

[BabaG]

thanks, pdr. actually just found that myself so i was able to get past it. however, now getting:Script error: There is no function named 'ex_sbr'.
continuing to investigate.

found it! in dogway's smdegrain.

thanks again,
babag

[BabaG]

finally got it loaded and looked at the uploaded clip with anton_foy's script at default settings. it seems to have little to no affect. i'd echo poisondeathray's question:Can you recommend some settings for this source ?

thanks,
babag

[anton_foy]

finally got it loaded and looked at the uploaded clip with anton_foy's script at default settings. it seems to have little to no affect. i'd echo poisondeathray's question:

thanks,
babag

Yes please read post 14, I replied. Try to raise thsad to maybe 600 or more until something happens. Chroma=false, mix=10 maybe.

[BabaG]

thanks, anton_foy! will try that.

babag

[BabaG]

thanks, anton_foy. i couldn't really get anything from this. tried several different settings but nothing. be curious if you can get anything out of it if you get a chance to try.

thanks again,
babag

[poisondeathray]

Same here - I tried many settings for clay , DthSAD and BthSAD to very high numbers. I also tried changing blksize, downscaling clip . It starts to do something if you use a very strong prefilter. But the results aren't great, perhaps anton can suggest better settings and/or better prefilter.

Similar results with other common avs denoisers MCTD, SMDegrain , KNLMeansCL, stacking them, each with many different settings. If you use a very strong prefilter clip (e.g. for SMDegrain), they start to work

Neat Video works on this. Easy to tune. About 10-20x faster processing speed than doing the stuff above

But if you want it super clean (TM) , yet retain details, you probably need to use advanced settings - multiple Neat video instances with layers, composite, masks. And/or add some other stuff like machine learning filters , because strong Neat video settings with high temporal radius cause a lot of blur