Welcome to Doom9's Forum, THE in-place to be for everyone interested in DVD conversion.

Before you start posting please read the forum rules. By posting to this forum you agree to abide by the rules.

 

Go Back   Doom9's Forum > Capturing and Editing Video > Avisynth Development

Reply
 
Thread Tools Search this Thread Display Modes
Old 17th August 2020, 21:53   #601  |  Link
StainlessS
HeartlessS Usurer
 
StainlessS's Avatar
 
Join Date: Dec 2009
Location: Over the rainbow
Posts: 10,266
Quote:
Originally Posted by FranceBB View Post
Hi there.
I was going to perform temporal degrain as usual when I noticed something weird:

Code:
source=FFMpegSource2("Cricket.mov", atrack=-1)
soccer=trim(source, 0, 1534)
soccer_fields=AssumeTFF(soccer)
soccer_bob=Bob(soccer_fields)
soccer_cropped=Crop(soccer_bob, 64, 18, -32, -16)
soccer_super=MSuper(soccer_cropped, pel=2, sharp=1)         # MSuper aint got no clip, and no Last defined !!!
soccer_bv1=MAnalyse(soccer_super, isb = true, delta = 1, overlap=4)
soccer_fv1=MAnalyse(soccer_super, isb = false, delta = 1, overlap=4)
soccer_bv2=MAnalyse(soccer_super, isb = true, delta = 2, overlap=4)
soccer_fv2=MAnalyse(soccer_super, isb = false, delta = 2, overlap=4)
soccer_degrain=MDegrain2(soccer_super, soccer_bv1, soccer_fv1, soccer_bv2, soccer_fv2,thSADC=200, thSAD=200)
soccer_SD=DebilinearResizeMT(soccer_degrain, 720, 576)
This doesn't work as MSuper doesn't recognize "soccer_cropped" as "clip", therefore MDegrain2 at the end complains about an invalid input.
On the other hand, this works:

Code:
source=FFMpegSource2("Cricket.mov", atrack=-1)
soccer=trim(source, 0, 1534)
soccer_fields=AssumeTFF(soccer)
soccer_bob=Bob(soccer_fields)
Crop(soccer_bob, 64, 18, -32, -16)      # Assigns to implicit Last, so below MSuper gets from implicit Last and works OK.
soccer_super=MSuper(pel=2, sharp=1)
soccer_bv1=MAnalyse(soccer_super, isb = true, delta = 1, overlap=4)
soccer_fv1=MAnalyse(soccer_super, isb = false, delta = 1, overlap=4)
soccer_bv2=MAnalyse(soccer_super, isb = true, delta = 2, overlap=4)
soccer_fv2=MAnalyse(soccer_super, isb = false, delta = 2, overlap=4)
soccer_degrain=MDegrain2(soccer_super, soccer_bv1, soccer_fv1, soccer_bv2, soccer_fv2,thSADC=200, thSAD=200)
soccer_SD=DebilinearResizeMT(soccer_degrain, 720, 576)
Is it intended?
I mean, the first parameter of MSuper is supposed to be clip, right? Why doesn't it accept my custom name?
1st code block, you dont give a clip to MSuper, and Last is undefined.
2nd one, works because crop supplies implicit last with a clip, so it works, your script was at fault.
__________________
I sometimes post sober.
StainlessS@MediaFire ::: AND/OR ::: StainlessS@SendSpace

"Some infinities are bigger than other infinities", but how many of them are infinitely bigger ???

Last edited by StainlessS; 17th August 2020 at 21:59.
StainlessS is offline   Reply With Quote
Old 17th August 2020, 22:03   #602  |  Link
FranceBB
Broadcast Encoder
 
FranceBB's Avatar
 
Join Date: Nov 2013
Location: Metropolitan City of Milan, Italy
Posts: 2,261
Quote:
Originally Posted by StainlessS View Post
1st code block, you dont give a clip to MSuper, and Last is undefined.
But I gave "soccer_cropped" as first argument to MSuper which is the output from "Crop" and of course, since it's the very first argument, I expect it to be used as a clip. Why isn't it used as a clip?

Code:
soccer_super=MSuper(soccer_cropped, pel=2, sharp=1)
FranceBB is offline   Reply With Quote
Old 17th August 2020, 22:05   #603  |  Link
StainlessS
HeartlessS Usurer
 
StainlessS's Avatar
 
Join Date: Dec 2009
Location: Over the rainbow
Posts: 10,266
Oops sorry, I was looking for soccer_cropped.MSuper() not Msuper(soccer_cropped)
NOT your script fault. Senility, you gotta luv it, always good for a laugh.

Wonkey is correct, funny happenings of late.
__________________
I sometimes post sober.
StainlessS@MediaFire ::: AND/OR ::: StainlessS@SendSpace

"Some infinities are bigger than other infinities", but how many of them are infinitely bigger ???

Last edited by StainlessS; 17th August 2020 at 22:09.
StainlessS is offline   Reply With Quote
Old 17th August 2020, 22:19   #604  |  Link
FranceBB
Broadcast Encoder
 
FranceBB's Avatar
 
Join Date: Nov 2013
Location: Metropolitan City of Milan, Italy
Posts: 2,261
Quote:
Originally Posted by StainlessS View Post
Senility, you gotta luv it, always good for a laugh.
Nah, not your fault, the script is a bit messy to read and it may seem "stupid" given that I could have left everything as it was, but it's actually part of a bigger, more complicated script, hence the need to assign those names... I've basically got a series of events in the same tape, but whoever dubbed the thing through a VTR at the time clearly didn't care about things like inactive lines and reverse fields and levels and many other things. Heck, in some points I even had to make a mask by hand to make sure the pitch looked almost the same from every camera... (and I don't have the cameras, I had to TRIM every single part... It was a hell of a job, but apparently my boss likes that game particularly... I don't know why, though, as his favorite team lost in the Champion's League final, so to me it would be something to forget, not remember xD I shouldn't comment any further, though, as he might be lurking on Doom9, especially since he knows that I spend much time here...)


Anyway, the thing is that David is right, there have been some weird things going on in the latest AVS+ version that affect a series of filters on the "last" argument, so it's not bonded to MVTools...

(Side Note: by "David" I mean Wonkey Monkey, although I never understood why he changed his nickname a while ago xD)

Last edited by FranceBB; 17th August 2020 at 22:26.
FranceBB is offline   Reply With Quote
Old 19th August 2020, 07:05   #605  |  Link
feisty2
I'm Siri
 
feisty2's Avatar
 
Join Date: Oct 2012
Location: void
Posts: 2,633
@pinterf

there's something fishy with MRecalculate, it seems we need to scale up searchparam when refining vectors to a smaller block size to get the correct result.
theoretically, this
Code:
vec = sup.MAnalyze(blksize=64, overlap=32, search=3, searchparam=2, badrange=-24)
vec = sup.MRecalculate(vec, blksize=2, overlap=1, search=3, searchparam=2, thsad=0)
MDegrainN(clp, sup, vec, thsad=2000)
should be identical to simply
Code:
vec = sup.MAnalyze(blksize=2, overlap=1, search=3, searchparam=2, badrange=-24)
MDegrainN(clp, sup, vec, thsad=2000)
but the results are far from being identical, the one using MRecalculate has A LOT of residual ghosting artifacts from blksize=64. To produce something similar to the one without MRecalculate (I'm not sure if it's bit-identical tho), you have to scale searchparam up to 64 for MRecalculate
Code:
vec = sup.MAnalyze(blksize=64, overlap=32, search=3, searchparam=2, badrange=-24)
vec = sup.MRecalculate(vec, blksize=2, overlap=1, search=3, searchparam=64, thsad=0)
MDegrainN(clp, sup, vec, thsad=2000)
have you checked if this is a bug or something?
feisty2 is offline   Reply With Quote
Old 18th October 2020, 12:06   #606  |  Link
Boulder
Pig on the wing
 
Boulder's Avatar
 
Join Date: Mar 2002
Location: Finland
Posts: 5,428
I posted an issue to the MVTools2 Github tracker, but I'll post it here as well in case someone can verify my findings.

This is a rather strange issue and I don't know if MVTools, Avisynth+, AVS2YUV or x265 is the real culprit.. nevertheless, I was able to determine that MVTools can be used to trigger it.

Using this kind of a script (don't worry about the supermdg bit being "unneeded", I used my denoising function as a base), I get jerky movement in the beginning of the encoded clip where the Enterprise appears on the screen. This happens if Prefetch is used - if I comment it out, no problems. The defect is not consistent between encodes, for example these are two consecutive test encodes with the same script and encoder parameters:

encoded 198 frames in 37.80s (5.24 fps), 1742.24 kb/s, Avg QP:24.60
encoded 198 frames in 36.98s (5.35 fps), 1721.16 kb/s, Avg QP:24.59

dgsource("c:\x265\tng\tngjerk.dgi")
convertbits(16)
superanalyse = msuper(pel=2, sharp=2, rfilter=4, chroma=true)
supermdg = msuper(pel=2, levels=1, sharp=2, rfilter=4, chroma=true)
fv1 = manalyse(superanalyse, isb=false, delta=1, blksize=16, overlap=8, search=5, searchparam=16, pelsearch=8, truemotion=false)
bv1 = manalyse(superanalyse, isb=true, delta=1, blksize=16, overlap=8, search=5, searchparam=16, pelsearch=8, truemotion=false)
mdegrain1(supermdg, bv1, fv1, thsad=100, thsadc=100)
Prefetch(threads=24, frames=24)

The encoder command line is this:
c:\x265\avs2yuv64.exe -no-mt -depth 16 "c:\x265\tng\tngjerk.avs" - | c:\x265\x265.exe -F 2 --input - --y4m --input-depth 16 --dither --sar 1:1 --profile main10 --ctu 32 --preset slower --merange 58 --crf 19 --output "c:\x265\tng\tng.hevc"

The cache settings are these:
SetMemoryMax(20480)
SetCacheMode(1)
Cache mode 2 didn't change things, still jerky.

I just tested removing truemotion=false from the script and the defect is gone. However, in my earlier tests this was also not very consistent and seemed to depend also on the x265 parameters.

Some samples:
https://drive.google.com/file/d/1e3c...ew?usp=sharing (original)
https://drive.google.com/file/d/1OdN...ew?usp=sharing (with Prefetch, jerky)
https://drive.google.com/file/d/1NpT...ew?usp=sharing (without Prefetch, no issues)


EDIT: just remembered this issue: https://forum.doom9.org/showthread.p...02#post1698102
Seems that adding RequestLinear(clim=100) after DGSource fixes the issue.
__________________
And if the band you're in starts playing different tunes
I'll see you on the dark side of the Moon...

Last edited by Boulder; 18th October 2020 at 12:24.
Boulder is offline   Reply With Quote
Old 14th December 2020, 19:03   #607  |  Link
pinterf
Registered User
 
Join Date: Jan 2014
Posts: 2,181
New MvTools 2.7.44
Code:
- 2.7.44 (20201214)
  - MAnalyze: fix motion vector generation inconsistencies across multiple runs.
    Note: when internal multithreading is used (avstp + mt=true), inconsistencies will still occur by design.
pinterf is offline   Reply With Quote
Old 14th December 2020, 20:20   #608  |  Link
StainlessS
HeartlessS Usurer
 
StainlessS's Avatar
 
Join Date: Dec 2009
Location: Over the rainbow
Posts: 10,266
Cheers P.
__________________
I sometimes post sober.
StainlessS@MediaFire ::: AND/OR ::: StainlessS@SendSpace

"Some infinities are bigger than other infinities", but how many of them are infinitely bigger ???
StainlessS is offline   Reply With Quote
Old 18th December 2020, 09:21   #609  |  Link
FranceBB
Broadcast Encoder
 
FranceBB's Avatar
 
Join Date: Nov 2013
Location: Metropolitan City of Milan, Italy
Posts: 2,261
Quote:
x86 and x64 builds
Windows XP is still supported, special DLLs with v141_xp toolset
ain't that sweet? Thanks!
FranceBB is offline   Reply With Quote
Old 5th January 2021, 04:28   #610  |  Link
takla
Registered User
 
Join Date: May 2018
Posts: 135
Is it known that large thSAD values (even the default 400) produces artefacts?
You can use this 25 seconds lossless video to test filters on yourself

I'll use frame 480 in the examples below.
Raw, no filters.
https://i.ibb.co/XCHfPLD/RAW.png

MDegrainNL(original from hello_hello, somewhat modified by me)
No artefacts of any kind and decent denoising.
https://i.ibb.co/175QJmg/MDegrain-NL-th-SAD150.png

MDegrainNL thSAD=400 (check the characters head for black pixels, also faded outlines)
https://i.ibb.co/30ZJrk0/MDegrain-NL-th-SAD400.png

The amount of artefacts are even worse with something like MCDegrainSharp to the point I'd never use or recommend it to anyone

MCDegrainSharp(truemotion=false) (check for the massive artefact on the wall under the characters head)
https://i.ibb.co/16fPMGz/MCDegrain-Sharp-TM-False.png

MCDegrainSharp(truemotion=true) (less obvious artefact with truemotion enabled, but it is still there, almost the same spot, just slightly shifted to the right)
https://i.ibb.co/gdxhG4P/MCDegrain-Sharp-TM-True.png

I also used ConvertBits(16) before the filter and ConvertBits(bits=10, dither=1) after.

Last edited by takla; 5th January 2021 at 04:52.
takla is offline   Reply With Quote
Old 5th January 2021, 06:11   #611  |  Link
real.finder
Registered User
 
Join Date: Jan 2012
Location: Mesopotamia
Posts: 2,541
yes it's known thing, it's all about the motion vectors will be wrong in cases like fades or dark frames like this, in your case (dark frames) you can use Dither_Luma_Rebuild to feed super clip for motion analysis to get better motion vectors
__________________
See My Avisynth Stuff
real.finder is offline   Reply With Quote
Old 5th January 2021, 11:33   #612  |  Link
Boulder
Pig on the wing
 
Boulder's Avatar
 
Join Date: Mar 2002
Location: Finland
Posts: 5,428
It would be interesting to see how the vectors look (MShow) and if the issue can be helped by using Dither_Luma_Rebuild or MRecalculate or dct=1 or dct=5.
__________________
And if the band you're in starts playing different tunes
I'll see you on the dark side of the Moon...
Boulder is offline   Reply With Quote
Old 5th January 2021, 21:14   #613  |  Link
takla
Registered User
 
Join Date: May 2018
Posts: 135
MDegrainNL_v1 (16-235, old)
MDegrainNL_v2 (0-255, new, more accurate)

v1
https://i.ibb.co/175QJmg/MDegrain-NL-th-SAD150.png

v2
https://i.ibb.co/3yJc9xg/NEW.png

If you zoom in, and look very hard you can see the characters nose hole slightly better in the v2 version. At basically no performance penalty, this is a nice improvement. I will do some more tests with higher thSAD later.

In the meantime, if someone could give me some help on how to add MRecalculate i'd appreciate it.
takla is offline   Reply With Quote
Old 5th January 2021, 21:18   #614  |  Link
ChaosKing
Registered User
 
Join Date: Dec 2005
Location: Germany
Posts: 1,706
From the SMDegrain wiki http://avisynth.nl/index.php/SMDegrain

Str float (0.0-8.0, default 1.0)
Gamma correction. With this parameter you control the strength of the brightening of the prefilter clip, good for when problems with dark areas arise. Using this internally instead of externally creates less quantization artifacts, since it's done in the same stage as the TV->PC range conversion.
__________________
AVSRepoGUI // VSRepoGUI - Package Manager for AviSynth // VapourSynth
VapourSynth Portable FATPACK || VapourSynth Database || https://github.com/avisynth-repository
ChaosKing is offline   Reply With Quote
Old 6th January 2021, 00:04   #615  |  Link
takla
Registered User
 
Join Date: May 2018
Posts: 135
MDegrainNL_v3

Added (pre)blur to suppress initial noise, for better motion vectors.

(I did play around with pre-amping gamma, which indeed got rid of the smaller black artefacts at higher thSAD. You can add gamma_y=128 to ColorYUV if you need a higher thSAD. For me though the sweetspot is 150, as increasing it to 200 didn't really change filesize but slightly dimmed the nose hole again)

Overall the v3 provided better denoising, performance and no artefacts, when compared to SMDegrain(defaults) and MCDegrainSharp(defaults) on the test clip posted above.

Last edited by takla; 6th January 2021 at 00:08.
takla is offline   Reply With Quote
Old 10th January 2021, 07:01   #616  |  Link
takla
Registered User
 
Join Date: May 2018
Posts: 135
One more update to this script:

Quote:
function MDegrainNext(clip Input, int "thSAD", int "thSADC", int "TR", int "BLKSize", int "Overlap", float "Sharpen", float "Blur")
{
thSAD = default(thSAD, 150)
thSADC = default(thSADC, thSAD)
TR = default(TR, 3)
BLKSize = default(BLKSize, 16)
Overlap = default(Overlap, BLKSize/2)
Sharpen = default(Sharpen, 0.2)
Blur = default(Blur, 1.0)

Super_Sharp = Input.Sharpen(Sharpen).MSuper(pel=1)
Super_Blur = Input.Blur(Blur).MSuper(pel=1)
Multi_Vector = Super_Blur.MAnalyse(Multi=True, Delta=TR, BLKSize=BLKSize, Overlap=Overlap)

Input.MDegrainN(Super_Sharp, Multi_Vector, TR, thSAD=thSAD, thSAD2=thSAD/2, thSADC=thSADC, thSADC2=thSADC/2)
}
Info:
Quote:
MDegrainNext v1 Documentation
Original by hello_hello https://forum.doom9.org/showpost.php...6&postcount=10
Modified by Takla

Why did I mod this: Needed something easy to read that only makes use of internal functions (excluding MVTools2)
This script is intended to be set-and-forget, with a superb quality-to-speed ratio and no artefacts.

thSAD & thSADC = Luma & Chroma Noise sensitivity.
Leave as is.
Values too low can cause blockiness and too high can cause black spot artefacts.
Default is 150 & thSADC=thSAD.

thSAD2 & thSADC2 = Noise sensitivity falloff.
Leave as is.
Default is thSAD/2 & thSADC/2.

TR = Temporal Radius.
The main setting you should be tweaking.
Higher values allow for more noise to be removed, at the cost of speed.
Higher values might also require you to lower thSAD to avoid black spot artefacts.
Default is 3.

BLKSize = Block size in pixels.
Lower means slower and more accurate but not necessary visible better.
Default is 16.
Recommendation: 540p=8, 1080p=16, 2160p=32

Overlap = Modifies Block size behaviour.
Leave as is.
Use BLKSize/4 for the best quality-to-speed ratio.
Use BLKSize/2 to preserve very fine detail. Slower.
Default is BLKSize/2.

Sharpen = Increases sharpness.
Much more accurate when used with Super, instead of calling it as a standalone after MDegrainNext.
You only want to do a little bit of sharpening, as it will increase bitrate requirements.
You can always sharpen more via media player (mpv or mpc).
Good values are 0.1 to 0.5.
Default is 0.2.

Blur = (Pre)blurs the input to avoid false motion vectors.
Allows for more accurate motion vectors.
Good values are 0.5 to 1.0, or 1.58 for very noisy sources.
Default is 1.0.
And maybe change pel back to 2 for SD videos.

Last edited by takla; 10th January 2021 at 07:08.
takla is offline   Reply With Quote
Old 2nd February 2021, 15:38   #617  |  Link
tormento
Acid fr0g
 
tormento's Avatar
 
Join Date: May 2002
Location: Italy
Posts: 2,241
Quote:
Originally Posted by pinterf View Post
MDegrain: out16 parameter, which outputs 16 bit result from 8 bit input
Would be possible to have out10 parameter, as it is the most used bit depth with x265, and out12, for HDR, or the 16 number is given from a stacked image and so only 8 multiples can be used?
__________________
@turment on Telegram
tormento is offline   Reply With Quote
Old 2nd February 2021, 16:11   #618  |  Link
DJATOM
Registered User
 
DJATOM's Avatar
 
Join Date: Sep 2010
Location: Ukraine, Bohuslav
Posts: 376
But that can be achieved by simply put ConvertBits(10/12/14) after MDegrain
__________________
Me on GitHub
PC Specs: Ryzen 5950X, 64 GB RAM, RTX 2070
DJATOM is offline   Reply With Quote
Old 2nd February 2021, 16:13   #619  |  Link
tormento
Acid fr0g
 
tormento's Avatar
 
Join Date: May 2002
Location: Italy
Posts: 2,241
Quote:
Originally Posted by DJATOM View Post
But that can be achieved by simply put ConvertBits(10/12/14) after MDegrain
I am doing various benchmarks and out16 outperforms any other kind of script with ConvertBits. I will post results very soon.
__________________
@turment on Telegram
tormento is offline   Reply With Quote
Old 2nd February 2021, 16:31   #620  |  Link
Boulder
Pig on the wing
 
Boulder's Avatar
 
Join Date: Mar 2002
Location: Finland
Posts: 5,428
It's very odd if it does that. I would expect it to be internally like ConvertBits(16) at the end of the whole processing chain.
__________________
And if the band you're in starts playing different tunes
I'll see you on the dark side of the Moon...
Boulder is offline   Reply With Quote
Reply

Thread Tools Search this Thread
Search this Thread:

Advanced Search
Display Modes

Posting Rules
You may not post new threads
You may not post replies
You may not post attachments
You may not edit your posts

BB code is On
Smilies are On
[IMG] code is On
HTML code is Off

Forum Jump


All times are GMT +1. The time now is 05:20.


Powered by vBulletin® Version 3.8.11
Copyright ©2000 - 2022, vBulletin Solutions Inc.