Dogway's Filters Packs

[Reel.Deel]

Well that really doesn't make sense for deep_resize to be changing the frame properties to 709 colorspace when it isn't. It should either assume the colorspace by the input frame resolution or leave it unspecified.

I looked at the script and the only place that I can guess where the frame properties are being set are in ConvertFormat. There's a few calls to that function withing deep_resize. I'm not sure what color format it converts to, it's hard to follow along.

ColorBars(pixel_type="YV12")
propClearAll()
deep_resize(width*2, height*2, grain=0)
propShow(size=24)

At least 4 properties are being set:
https://i.ibb.co/VCQ8fD8/deep-fp.png



avsresize would look like


z_convertformat(pixel_type=last.pixeltype, colorspace_op="170m:709:709:l=>709:709:709:l")


You no longer recommend full=>full (https://forum.doom9.org/showthread.php?p=1906754&highlight=full#post1906754)?

[poisondeathray]

You no longer recommend full=>full (https://forum.doom9.org/showthread.php?p=1906754&highlight=full#post1906754)?

Good point - Full is probably still better, I don't think anything has changed.

I rarely use colormatrix (or related) anymore for upscaling, because almost everything SD I'm upscaling uses machine learning at some point, so it's converted to RGB early on

[simple_simon]

z_convertformat(pixel_type=last.pixeltype, colorspace_op="170m:709:709:l=>709:709:709:l")


Why wouldn't this be the correct way to do it?

z_ConvertFormat(colorspace_op="470bg:601:170m:full=>709:709:709:full")

And does it matter where in the script chain I do the color conversion? Should it be early in the script or after all the cleanup and resizing is done?

At least 4 properties are being set:
https://i.ibb.co/VCQ8fD8/deep-fp.png

That's what I'm seeing too.

[poisondeathray]

Why wouldn't this be the correct way to do it?

z_ConvertFormat(colorspace_op="470bg:601:170m:full=>709:709:709:full")



Matrices 470bg and 170m have the same values Kr = 0.299, Kb = 0.114, so it's the same mathematically

Transfer also have the same values for "601" and "709" so, it's the same mathematically

Primaries are debatable - you actually shift the colors if you go from 170m to 709. Technically, you're supposed to adjust the primaries, but >95% of the time, it's not actually done upstream for the transfer to the DVD in the first place - so you end up with slightly shifted colors if you adjust them in my experience. Primaries are typically not adjusted for in broadcast, BD, DVD, web streaming either (except for HDR) You only get the "correct" colors, if the primaries were handled correctly all the way through, upstream and downstream and the playback method.


And does it matter where in the script chain I do the color conversion? Should it be early in the script or after all the cleanup and resizing is done?


It probably doesn't matter too much, but I'd generally do it near the end.

OTOH, some types of artifacts related to colormatrix conversion can occur with some types of animation (there are some threads and examples on this on this forum and others), so depending on how picky you are , you might want to filter them farther, so earlier could be better in some cases

[poisondeathray]

Sorry, me again :(

I thought I'd give this a try, and as per usual (for me), I'm getting an error with AVSMeter..

https://github.com/Dogway/Avisynth-Scripts/blob/master/GradePack.avsi#L401

Import("%AVISYNTHPLUGINS%\PD_TOOLS\EXTOOLS\ExTools.avs")
Import("%AVISYNTHPLUGINS%\PD_TOOLS\GRADE-PACK\Masks Pack.avs")
Import("%AVISYNTHPLUGINS%\PD_TOOLS\MASK-PACK\Grade Pack.avs")
Import("%AVISYNTHPLUGINS%\PD_TOOLS\RESIZERS-PACK\Resizers Pack.avs")
Import("%AVISYNTHPLUGINS%\PD_TOOLS\SCENES-PACK\Scenes Pack.avs")
Import("%AVISYNTHPLUGINS%\PD_TOOLS\TRANSFORM-PACK\Transforms Pack.avs")
Import("%AVISYNTHPLUGINS%\PD_TOOLS\TRANSFORM-PACK\Transforms Pack-Models.avs")
Import("%AVISYNTHPLUGINS%\PD_TOOLS\TRANSFORM-PACK\Transforms Pack-Transfers.avs")
video=ex_autolevels(video,interval=0.5,th=0.1)



You need to run SceneStats before ex_autolevels . 2pass is probably better with a log file, but might be difficult to do in ripbot alone


video = SceneStats(video, mode="Range+Stats",Y=true,UV=false) #1pass , no log file written
video = ex_autolevels(video, interval=0.5,th=0.1)

[poisondeathray]

Try with a small short test video first to see if it "works" at all on your setup - because it can take a while while calculating

[poisondeathray]

Thanks again, PDR, that got it to work...

But I'd hate to think how long it would take to process a 1080p or 4K movie length :( :confused: :(


It was just a quick test to rule out configuration problems (if any)

Actually you can run the 2pass method, since avsmeter is available in ripbot. The analysis will not seem to "hang" and it will "bake" the analysis results into the text file. ie. use avsmeter to run the 1st pass, change the script for the encoding pass (or passes) . Make sure you run the analysis pass single threaded (no prefetch, or prefetch (1), also source filter with threads=1, and/or use a very robust indexed method like DGSource. You also have to edit the log file, there is a repeated section at the top that you have to delete (open in a text editor, and delete all the lines up to the 2nd "Type string" line)

dogway posts this warning in ScenesPack.avsi - that's what it means

### BEWARE!! For some reason in the exported log the first scene has repeated stats so manually remove those lines before loading the file.


analysis pass

video = SceneStats(video, mode="Range+Stats",Y=true,UV=false, path="path\source_stats.log")


encode pass/passes

video = ReadStats(video, "path\source_stats.log")
video = ex_autolevels(video, interval=0.5,th=0.1)



There are other "auto" leveling filters and methods that you can try out too...

[poisondeathray]

What are the other's ?? ;)


What kind of "auto leveling" are you wanting ?

Is it more "general" brightness/darkness/contrast issues, or to deal with "luminance flicker", or "white balance" / color balance/color cast , or something else?

For the general category, you can try AutoLevels, or AutoAdjust

[poisondeathray]

Well, I have tried several white balance types, and they don't work with RipBot (well they do, but they don't).

I used to use a white balance filter when I was using Adobe Premier, and that worked well, I've even tried 3DLUT's to varying degrees of success.

I find I'm tweaking the colours in most of my encodes, mainly taming down "red's".



For avisynth white balance, I find RGBAdapt_AWB_Process is quite good and customizable - results are close to commercial plugins in some cases - but IIRC you had issues running it. It works in ripbot, loading the video then copy/pasting - I just tried it.

And playing with ripbot - I think I figured out the "no error message" black screen MPCHC ripbot preview issue that you sometimes have. AWB_Func didn't work, at first no error message. It turns out I had a typo, which the now visible error message conveyed. What you need to do is move the seekbar in MPCHC, and the error message will pop up

You can also try coloryuv(autowhite=true) , it' s not as advanced but can do the job adequately in some cases

If you prefer the PP filter's, you can use PP . One option is to frameserve in (Premiere AVS Import plugin) / frameserve out (Advanced Frameserver, or Debugmode Frameserver - now called "Cumincode Framserver") . Or use intermediates. I personally prefer intermediates when using PP

[poisondeathray]

I recall that it DID work that way, BUT it did not invoke the distributed encoding function, which is the basically the whole point of RipBot (as far as encoding is concerned)

Even if you got it working - the results would not be ideal distributed anyways. Any distributed, or chunked method will have potential issues with temporal filters "at the seams". Temporal auto leveling is probably the worst type of filter you could use with chunks - it would predispose you to flicker at the joins. You can reduce some of the issues if you overlap the chunks and discard the overlaps before joining, but I don't think ripbot had that feature the last time I checked



Don't use Premiere anymore, no mkv support :(

You can indirectly import MKV's through an .avs. The AVS import plugin doesn't work for many people, but AVFS (avisynth virtual file system) works for everyone and with just about every program (other NLE's too) . Vapoursynth version of avfs is better - because it has fourcc emulation such as UYVY for 8bit422, v210 for 10bit422 - these are treated as "YUV" in NLE's with YUV capable timelines like PP, instead of being "mishandled" as RGB (many lossless YUV codecs and uncompressed YUV formats are mishandled by most NLE's, and get converted to RGB) . "RGB" is usually not a big issue if you're using some auto leveling/auto white balance plugin - because most will be working in RGB anyways

Another option is if you re-wrap (streamcopy) into MP4, PP can import AVC, HEVC too. Native file import such as MP4 colorspace is handled properly

[Dogway]

Thanks DTL and Dogway for your input. Really appreciate it. :thanks:
Something new to fiddle around with the next couple of days. :D

I updated ex_autolevels(), now it's really robust but as I said it's a global operator, I might look into local in the future.

I also updated ex_luma_rebuild() to allow for ex_levels() type of normalization with 'lo' and 'hi' args, but optimized without any performance penalty.

Also updated StabPlus() with ex_autolevels(). Actually you could hack open the StabPlus() function to reuse some code with the below script.

ffvideosource("noisy_original.mkv")
propSet("_FieldBased",0)

ConvertBits(16)
ex_Luma_Rebuild(15.0, 0.004, 21, tv_out=true)
ConvertBits(8,dither=1)
StabPlus()
crop(2,2,-6,-2)
SceneStats("Range+Stats")

pref = ex_autolevels(hi=true,tv_out=false)
temp = Pref.TemporalSoften(5,255,0,25,2)
rep = temp.Repair(Pref.TemporalSoften(1,255,0,25,2))

pref = ex_merge(rep, pref, MotionMask(rep,dir=0,scale=3,sharpness=2))
SMDegrain(2, 600, mode="TemporalSoften",str=2,prefilter=pref,RefineMotion=true)


ex_unsharp(0.3)
gr = 1
size = (720/2133.) * 3.3
GrainFactory3mod(g1str=7*gr,g2str=9*gr,g3str=7*gr,g1size=1.7*size,g2size=1.4*size,g3size=1.2*size,g1cstr=0.5*gr,g2cstr=0.3*gr,g3cstr=0.1*gr,temp_avg=1,tv_range=true)


For the new SMDegrain update, I'm still missing a bugfix in ex_retinex() for lvl=1 and highlights=false. Just was burnt out with the latest fixes.

This is the new output: link (https://www.mediafire.com/file/xx19gf0e6pavv21/New_File_%2528328%2529_Dog4.mp4/file)

For comparison I was curious to test videoFred's script, it does a good job on stabilization and luma flicker but there's still some wobble: link (https://www.mediafire.com/file/prmza39fdno27bq/New_File_%2528331%2529_VF.mp4/file)

[anton_foy]

I updated ex_autolevels(), now it's really robust but as I said it's a global operator, I might look into local in the future.

I also updated ex_luma_rebuild() to allow for ex_levels() type of normalization with 'lo' and 'hi' args, but optimized without any performance penalty.

Also updated StabPlus() with ex_autolevels(). Actually you could hack open the StabPlus() function to reuse some code with the below script.

ffvideosource("noisy_original.mkv")
propSet("_FieldBased",0)

ConvertBits(16)
ex_Luma_Rebuild(15.0, 0.004, 21, tv_out=true)
ConvertBits(8,dither=1)
StabPlus()
crop(2,2,-6,-2)
SceneStats("Range+Stats")

pref = ex_autolevels(hi=true,tv_out=false)
temp = Pref.TemporalSoften(5,255,0,25,2)
rep = temp.Repair(Pref.TemporalSoften(1,255,0,25,2))

pref = ex_merge(rep, pref,MotionMask(rep,scale=10,sharpness=5))
SMDegrain(2, 600, mode="TemporalSoften",str=2,prefilter=pref,RefineMotion=true)


ex_unsharp(0.3)
gr = 1
size = (720/2133.) * 3.3
GrainFactory3mod(g1str=7*gr,g2str=9*gr,g3str=7*gr,g1size=1.7*size,g2size=1.4*size,g3size=1.2*size,g1cstr=0.5*gr,g2cstr=0.3*gr,g3cstr=0.1*gr,temp_avg=1,tv_range=true)


For the new SMDegrain update, I'm still missing a bugfix in ex_retinex() for lvl=1 and highlights=false. Just was burnt out with the latest fixes.

This is the new output: link (https://www.mediafire.com/file/xx19gf0e6pavv21/New_File_%2528328%2529_Dog4.mp4/file)

For comparison I was curious to test videoFred's script, it does a good job on stabilization and luma flicker but there's still some wobble: link (https://www.mediafire.com/file/prmza39fdno27bq/New_File_%2528331%2529_VF.mp4/file)

Looking really good! But something is happening on the right in the bushes when the chef is passing by it. Warping kind of.

[Dogway]

Sure, I said before it's not perfect. For these kind of borked sources the best solution is not a surgeon fix, but call TemporalSoften and fix several issues at once.
You can play with the temporal radius of TemporalSoften and SMDegrain to balance the effect, or use a pyramidal approach where you refine motion vectors further and further.
Maybe applying BM3D with high radius to 'rep' frame can improve some stabilization.

[anton_foy]

Sure, I said before it's not perfect. For these kind of borked sources the best solution is not a surgeon fix, but call TemporalSoften and fix several issues at once.
You can play with the temporal radius of TemporalSoften and SMDegrain to balance the effect, or use a pyramidal approach where you refine motion vectors further and further.
Maybe applying BM3D with high radius to 'rep' frame can improve some stabilization.

Actually this is to me very interesting as I am experimenting with wobble and jitter locally to stab. Not quite there yet though but hundreds of ideas.

[Dogway]

I was curious to test the performance of ex_autolevels() on a synthetic test so I made this gradient patch with alternating lifted blacks and gamma.
https://i.imgur.com/wMEhR2q.gif
This is ex_autolevels:
https://i.imgur.com/nUByuUM.gif
Out of curiosity I also tested Autolevels(), I don't know if I'm doing something wrong here, the call is:
Autolevels(filterRadius=10, output_low=0, output_high=255, ignore=0.05, sceneChgThresh=255)
I had to raise sceneChgThresh as it was detecting flicker as SC
https://i.imgur.com/RY2L8XD.gif

Probably if I run a second pass of ex_autolevels() with original stats it can be stabilized further. Will check that.
EDIT: Yes, it's more stable with a second run but 'lo' and 'hi' have to be disabled to avoid a re-normalization:
ex_autolevels(true ,true,false,reStats=false,tv_in=false)
ex_autolevels(false,true,false,reStats=false,tv_in=false)

[LeXXuz]

Looks very interesting! Where would you suggest to put this in a (double) SMDegrain call?


Out of curiosity I took a much closer look on other transfers I did over the last couple of month.

Now that I developed a certain eye for this texture wobble, I see it in a lot of films I did in the past. SD and HD. Even films where you would not expect this because you can hardly see any luma flickering in the source.

I start to believe this texture wobble is some kind of general side effect of the way SMDegrain removes noise or from the motion compensation. I still understand too little on how the entire process works to judge this.

SMDegrain does a terrific job on films that had a very good AD transfer and where filmgrain really is the only kind of noise present. Same goes for modern digital productions where grain is added in post-processing for some arguable reasons.

But on sources where other types of noise are involved, escpecially these 'unstable' sources, the side effects become clearly superficial when you try to go for a clean overall look.

I just had a SD film here where in one scene almost the whole picture wobbled. Right now I'm playing around with other denoisers on this one to see what they can do.

[anton_foy]

@Dogway I think I found what is causing that wobble in the bushes. MotionMask(rep,dir=0,scale=3,sharpness=2) with the temporalsoften lines above it.
Try and see if you change the temporalsoften params lower if the wobble disappear.

[Dogway]

I've been refining ex_autolevels() and now I get it almost perfectly flicker free.
First I had a bug in ArrayDelVal() in ExTools which I'll update soon. Then I added 'Deflicker' arg to ex_autolevels() to make the normalization per scene (autolevels) or per frame (deflicker).

Effectively running 2 passes makes it balanced enough for the gamma flicker.
ex_autolevels(true ,true,false,Deflicker=true,tv_in=false)
ex_autolevels(false,true,false,Deflicker=true,tv_in=false)
https://i.imgur.com/CDYa8L1.gif

This is how you make a flickering patch:
GradientLinear(zoom=1, positive=true, precision=4)
trim(0,32)
SceneStats("Range")

ScriptClip(function [] () {
lo = rand(30)
gm = (rand(100)-50)/200.+1
ex_levels(0,gm,255,lo,255,tv_range=false)
} )

SceneStats("Stats")

...Deflickering here...

ConvertBits(8,dither=1)

For comparison the next is a double call of Deflicker(scene=255,percent=100)
https://i.imgur.com/GooKnPK.gif


@Dogway I think I found what is causing that wobble in the bushes. MotionMask(rep,dir=0,scale=3,sharpness=2) with the temporalsoften lines above it.
Try and see if you change the temporalsoften params lower if the wobble disappear.

Yes I know, but hadn't time to test a new approach. Want to try with my heavy grain prefilter preset like in 21 grams sample, using pyramidal temporal denoising.

Looks very interesting! Where would you suggest to put this in a (double) SMDegrain call?

I start to believe this texture wobble is some kind of general side effect of the way SMDegrain removes noise or from the motion compensation. I still understand too little on how the entire process works to judge this.

SMDegrain does a terrific job on films that had a very good AD transfer and where filmgrain really is the only kind of noise present. Same goes for modern digital productions where grain is added in post-processing for some arguable reasons.

It's DCT flicker, or in other words low frequency temporal "noise", motion and luma flicker just makes it worse. You can take a lowpassed version, temporally denoise it (tricky) and add it back to source. In any case using a radical approach like with temporalsoften prefilter works, I'm going to test with other forms of hardcore prefilters. For best results stabilize your clip motion and luma flicker beforehand then fix DCT flicker.

EDIT: Here an updated script with less wobble but also less background stabilization. Link (https://www.mediafire.com/file/kw7r6wm8tixyvz5)
I tested with ex_reduceflicker() as the clip presented bob-like jitter and worked very nice, also reduced luma flicker a bit further despite not being designed for that.
ffvideosource("noisy_original.mkv")
propSet("_FieldBased",0)

ConvertBits(16)
ex_Luma_Rebuild(15.0, 0.004, 21, tv_out=true)
ConvertBits(8,dither=1)

StabPlus()
crop(2,2,-6,-2)
SceneStats("Range+Stats")

pref = ex_autolevels( true ,true, true,Deflicker=true,tv_out=false)
pref = ex_autolevels(pref,false,true,false,Deflicker=true,tv_out=false)
pref2 = ex_reduceflicker(pref,3,true)
pref3 = SMDegrain(tr=3,mode="temporalsoften",blksize=32,thSAD=900,prefilter=pref2,str=2,contrasharp=false,refinemotion=false)

bk = SMDegrain(pref3,tr=3,mode="MDegrain",blksize=16,thSAD=400,refinemotion=true)
ft = SMDegrain( tr=2,mode="MDegrain",blksize=16,thSAD=300,refinemotion=true,prefilter=bk)
ex_merge(bk, ft, MotionMask(pref3,scale=10,sharpness=5))

ex_unsharp(0.3)
gr = 1
size = (720/2133.) * 3.3
GrainFactory3mod(g1str=7*gr,g2str=9*gr,g3str=7*gr,g1size=1.7*size,g2size=1.4*size,g3size=1.2*size,g1cstr=0.5*gr,g2cstr=0.3*gr,g3cstr=0.1*gr,temp_avg=1,tv_range=true)

EDIT2: And another variation (https://www.mediafire.com/file/1a5lxv2w0wpwim9), less smearing and wobble but again a bit more nervous background. I also replaced StabPlus() with the next videoFred's suggestion (https://forum.doom9.org/showthread.php?p=1981048#post1981048), so much better that I removed ex_reduceflicker().

ffvideosource("noisy_original.mkv")
propSet("_FieldBased",0)

ConvertBits(16)
ex_Luma_Rebuild(15.0, 0.004, 21, tv_out=true)
ConvertBits(8,dither=1)

StabPlus2()
crop(2,2,-6,-2)
SceneStats("Range+Stats")

pref = ex_autolevels( true ,true, true,Deflicker=true,tv_out=false)
pref = ex_autolevels(pref,false,true,false,Deflicker=true,tv_out=false)
pref3 = SMDegrain(tr=3,mode="temporalsoften",blksize=32,thSAD=900,prefilter=pref,str=2,contrasharp=false,refinemotion=false)

bk=SMDegrain(tr=2,mode="temporalsoften",blksize=16,thSAD=400,LFR=false,refinemotion=true,prefilter=pref3)
ft=SMDegrain(tr=2,mode="MDegrain" ,blksize=16,thSAD=300,LFR=false,refinemotion=true,prefilter=bk)
ex_merge(bk, ft, MotionMask(pref3,scale=10,sharpness=5))

ex_unsharp(0.3)
gr = 1
size = (720/2133.) * 3.3
GrainFactory3mod(g1str=7*gr,g2str=9*gr,g3str=7*gr,g1size=1.7*size,g2size=1.4*size,g3size=1.2*size,g1cstr=0.5*gr,g2cstr=0.3*gr,g3cstr=0.1*gr,temp_avg=1,tv_range=true)

[simple_simon]

Anybody have an idea what might be causing the horizontal chroma lines in these two screenshots? They don't exist in the source video.

https://send.cm/d/KEeM
https://send.cm/d/KEeP

This is the script I'm using:
z_convertformat(pixel_type=last.pixeltype, colorspace_op="170m:709:709:full=>709:709:709:full").propSet("_ColorRange",1)
tfm(mode=1, pp=7)
tdecimate(mode=0, dupthresh=1.4)
lsfplus(strength=60,ss_x=1.5,ss_y=1.5,soft=30,overshoot=1)
removegrain(mode=5)
propSet("_FieldBased",0).deep_resize(960,720,grain=0,qual=2,gpuid=-1,show=false)
prefetch(2)

I thought I had my upscaling routine nailed down but I guess I'm going to have to go back and redo everything again.

[poisondeathray]

Anybody have an idea what might be causing the horizontal chroma lines in these two screenshots? They don't exist in the source video.


Move z_convertformat after IVTC. "colormatrix" or similar applied on fields can cause similar issue, because it's applying a progressive transform while still organized in fields. There is an interlaced=true/false switch , but I believe it's non functional. Better just to move it

[simple_simon]

Move z_convertformat after IVTC. "colormatrix" or similar applied on fields can cause similar issue, because it's applying a progressive transform while still organized in fields. There is an interlaced=true/false switch , but I believe it's non functional. Better just to move it

(slaps forehead) I should have realized that would cause a problem converting colorspace before detelecine. Thanks for pointing out my mistake. That fixed it.

[Tempter57]

Dogway

I am sorry, but you could not edit Stabilization Tools Pack.avsi and enter there function StabPlus2 ()

[Arx1meD]

Dogway, ex_autolevels for Avisint 3.7.3 only?

[Dogway]

I am sorry, but you could not edit Stabilization Tools Pack.avsi and enter there function StabPlus2 ()
It was a proxy name, there's nothing special about it so you can name it how you want.
# https://forum.doom9.org/showthread.php?p=1981048#post1981048
function Stabilize(clip c, int "offset", bool "info") {

in = Default(info, false)
of = Default(offset, 40)

c
vectors = MSuper().MAnalyse(isb=false)
mdata = MDepan(vectors, rot=false, zoom=false, error= 65)

DePanStabilize(data=mdata,dxmax=of, dymax=of, zoommax=0, rotmax=0, method=1, mirror=15, prev=1, next=1, info=in)
}

ex_autolevels for Avisint 3.7.3 only?
Yes because it uses SceneStats() which itself requires PlaneMinMaxStats() with 'average' support which was a late addition.


-------------------------

In conversations in Github I recommended an edge masking for anime alternative to FlatMask() and more similar to VapourSynth's detailmask() sometimes surpassing it. It uses a combination of lumamask() and ex_edge() with final mask merging.


LC = ex_unsharp(2,Fc=width()/4) # increase local contrast
LM = lumamask(LC,30,70,invert=true) # luma mask

ex_edge("frei-chen",lo=4,hi=25)

ex_expand(mode="disk")
ex_deflate()
ex_smooth(2,mode="SG")

ex_blend(LM,mode="overlay") # Improve edge masking with luma mask reinforcement

[Tempter57]

Dogway

Thank you, but in my opinion it would be possible simply in StabPlus to set thanks dxmax=20, dymax=20, to refine image stabilising on a bush

[poisondeathray]

I have a 1920 x 960 clip I would like to downscale to 1280 (by whatever it is supposed to be), and if I use this call :-

video=deep_resize(video,1280,flat="nnedi3",grain=0,qual=2)

It errors with a Line 259 error in Resizers Pack.


https://github.com/Dogway/Avisynth-Scripts/blob/master/ResizersPack.avsi#L259


What is the exact error message?


But if I change it to :-

video=deep_resize(video,1920,flat="nnedi3",grain=0,qual=2)

It works...


It should work, because nothing is being resized at 1920 width - a no-op ie. Filter is bypassed. It's the same as not including it.



This works for me, resulting in 1280x640 (2:1)


video=blankclip(width=1920, height=960, pixel_type="YV12")
video=deep_resize(video,1280,flat="nnedi3",grain=0,qual=2)
return video


So deep_resize script is ok. It means the issue is with your specific video or setup - maybe some props issues in the src video (you can try clearing them), also make sure scripts/dependencies are up to date

[kedautinh12]

I was said you need add height for correct result you expect. If not, result won't 1280x720 if your source not 16:9

[poisondeathray]

Hello again, PDR....

I don't want to have to manually add the width & height....



It's not "manually adding width and height" per se - It's a test script making a blank clip with the same dimensions as your example to debug the script. To test if the deep_resize script itself works - It does, so that's ruled out as the problem

I can encode the blank clip, then load it without entering anything - and it works too. So again, the problem is not deep_resize - it's something either with your setup or your video

When you debug something you break it down into parts ,examine each step



But this seems to work:-

video=ConvertBits(video,16)
video=deep_resize(video,1280,edge="SSIM2")[/CODE]

I added these (in red), to get the last line to do it's thing :)



Obviously this is a different script. Do you no longer care about the old one ?

And you still didn't answer the question about exact error message, but if you don't care, then ignore


So let me ask you this, if I load this 1920 x 960 clip into RipBot and ask it to do a resize down to 1280 x 720, it changes it to 1280 x 692, when it should actually be 1280 x 640 ???

Not sure what Ripbot does, I don't really use it.

You can examine the temp files and log to see what it's really doing



Would that be an Avisynth problem, or the way the resizing has been written into RipBot. ??

Not an avs problem

[poisondeathray]

No error msg, other than referencing line 259, and that's where I got the idea for those extra calls & dependencies ;)



Did you try moving the seekbar in mpchc ?



No, it's still that script from the previous page, I just needed to add those calls...


Not really, do these look the same to you?


video=deep_resize(video,1280,flat="nnedi3",grain=0,qual=2)


vs

video=ConvertBits(video,16)
video=deep_resize(video,1280,edge="SSIM2")






Well, it must be the way Atak wrote the resizing functions, then.

It is. It says so in the brackets. "1280x720 [1,85:1 AR]"

1280x720 is achieved by resizing to 1280x692 (1280/692 = 1.8497 - close enough to 1.85 AR), letterbox borders added to get 1280x720 - this is in the script when you select that preset

[poisondeathray]

Come on PDR, one didn't downscale, and the other does...simple, TMWOT.


If it's the same source video, both should be downscaling



It is. It says so in the brackets. "1280x720 [1,85:1 AR]"

1280x720 is achieved by resizing to 1280x692 (1280/692 = 1.8497 - close enough to 1.85 AR), letterbox borders added to get 1280x720 - this is in the script when you select that preset


But not when it should be 1280 x 640 !!!

The preset is doing what it says it would do.

1280x720 [2:1 AR] would take 1920x960 and resize to 1280x640 with letterbox borders 40px top & bottom each to achieve 1280x720

Anyways, it has nothing to do with Dogways filters or avisynth. You should discuss in ripbot thread

[kedautinh12]

You can read document about AddBorders, this code will add black boredrs to change 1280x640 to 1280x720
video=deep_resize(video,1280,640,flat="nnedi3",grain=0,qual=2)
video=AddBorders(video,0,40,0,40)
http://avisynth.nl/index.php/AddBorders

[poisondeathray]

video=deep_resize(video,1280,flat="nnedi3",grain=0,qual=2)

If I change that 1280 to 1920, it does work, as it's a 1920 source....wasn't this the problem I had with deep_resize a couple of weeks ago ??

If you change it to 1920, it's a no-op - so the line is by passed. So it does not really "work"

Yes I think it's a similar issue to what you 've had in the past, but you never answered dogway's questions directly either...

When you ask for help, usually you should make an attempt to answer questions...otherwise people will be less inclined to help


But this call does NOT work for me.


That's what I was trying to help debug...


What preset are you referring to ??

The hypothetical preset that would solve your specific video situation . The preset you want should say 2:1 AR - but there is none with that AR included in the default ripbot. The one you selected gives you the expected result (not what you want, because it's the wrong preset). Maybe you can hack it to include others, or maybe you can add a custom option

[poisondeathray]

I don't want to start adding borders.


There is a custom preset [x,xx:1] included in ripbot. You can just enter the values such as 1280 , 640 . This would have no borders and the correct DAR of 2:1 .

[LeXXuz]

It's DCT flicker, or in other words low frequency temporal "noise", motion and luma flicker just makes it worse. You can take a lowpassed version, temporally denoise it (tricky) and add it back to source. In any case using a radical approach like with temporalsoften prefilter works, I'm going to test with other forms of hardcore prefilters. For best results stabilize your clip motion and luma flicker beforehand then fix DCT flicker.



Thank you very much for those scripts and the explanation. :)

[poisondeathray]

I was playing around with trying to get MPC-BE to work in RipBot, and I played the preview of the clip I was having problems with, and it displayed a pretty comprehensive error message :)


error messages should be the 1st step to solve all your avs problems. Nearly impossible to debug without error messages

"There is no function named 'SSIM_downsample'"

Just "google" the error message, you don't even have to ask or post unless google doesn't have an answer

https://github.com/Asd-g/AviSynthPlus-Scripts/blob/master/SSIM_downsample.avsi





When using this call on a 1920 clip:-

video=deep_resize(video,1280,flat="nnedi3",grain=0,qual=2)

But change that 1280 to 1920, no more error message :)


Of course, this has been mentioned several times, but several different people - the 1920 is a no-op. It's like deleting the line, or commenting out the line

1280 gives you the error message, because you're missing the SSIM_downsample function. (ie. check your prerequisites )

[kedautinh12]

@TDS Or you can use Dogway's SSIM_downsample
https://github.com/Dogway/Avisynth-Scripts/blob/master/EX%20mods/SimilarityMetrics.avsi

[Dogway]

@TDS, could you open your own thread? The last 8 pages or so have been completely yours with mostly RipBot and basic AVS usage questions, going in rounds like a dog trying to catch his own tail, double posting and whatnot. It's going to be suffering for future readers.

For anyone reading, proper discussion ended in first half of page 96 (URL="https://forum.doom9.org/showthread.php?t=182881&page=104), with some break in previous page (page 104 (https://forum.doom9.org/showthread.php?t=182881&page=104)) for luma deflicker.

On another note, I'm finishing SMDegrain documentation, will probably promote it to v4.0.0 and call it "finished".

[guest]

@TDS, could you open your own thread? The last 8 pages or so have been completely yours with mostly RipBot and basic AVS usage questions, going in rounds like a dog trying to catch his own tail, double posting and whatnot. It's going to be suffering for future readers.

For anyone reading, proper discussion ended in first half of page 96 (URL="https://forum.doom9.org/showthread.php?t=182881&page=104), with some break in previous page (page 104 (https://forum.doom9.org/showthread.php?t=182881&page=104)) for luma deflicker.

Hi Dogway,

I think you're being quite unfair at singling me out for all these "non related" post's, it takes 2 (or more) to tango !!!

It started out as a legit question about deep-resize, and it just went on & on from there.

I do recall mentioning that this should go to the RipBot264 thread, but it didn't.

I'm not going to throw anyone under the bus, but it should be very obvious who the main protagonist's were.

I'm sorry I'm not as experienced or as knowledgable as the ppl involved (and yourself), so it takes me a little while for it to sink in.

So to save you further embarrassment of my posts, I have removed some 62 that were "off topic".

Unfortunately by doing so, it leaves a LOT of posts that are now pointless & irrelevant, so maybe they need to be removed, as well.

I honestly appreciate the endless development of your tool's, and they have been a game changer for me.

But I would doubt you will see anymore posts from me, in this thread, in the future.


https://forum.doom9.org/showthread.php?p=1981416#post1981416 first WARN !!!

https://forum.doom9.org/showthread.php?p=1981903#post1981903 second WARN !!!

https://forum.doom9.org/showthread.php?p=1981904#post1981904 legit question about deep-resize

[Boulder]

You could have asked for a moderator to move them to a separate thread, which they will happily do if requested.

[johnmeyer]

The guy hijacks this thread, complains about being singled out and treated unfairly but, oh by the way, he then deletes 62 posts.

Everyone has their own definition of what it means to hijack a thread, but 62 posts about things not related directly to Dogway's amazing filter packs is pretty much the definition of hijacking.

Dogway did the right thing by asking him to start his own thread. That is precisely what should be done when someone goes off topic. This is especially true of this thread which, because his filter packs are so popular, is now over 100 pages in length. We most definitely do not want to have to wade through several pages of posts that are irrelevant to the topic when we want to find out something about Dogway's software.

[Dogway]

Well, just updated SMDegrain mod to 4.0.0 with fully interlaced and YUY2 direct support for all clip type arguments (input, CClip, mfilter and prefilter).
I'm wrapping up my packs so unless something is really borked these will be final. SMDegrain requires now latest versions of the main packs specially LSFplus and SharpenersPack if using sharpening specially with interlaced content.

I also fixed the remaining issues in ex_retinex() hopefully for the good and a few more updates and cosmetics, most important of all the Documentation is finished, so you can have a glance on how things work now although it's pretty much what I've been posting around here.
It's labeled RC (Release Candidate), if I didn't brake anything in a week or two I edit it out.

It's been long so this is my last presentation card for the release.

http://i.imgur.com/pCiSZdhh.png (https://i.imgur.com/pCiSZdh.png)


I also made a heavy refactor of GrainFactory3mod called FilmGrain, I wanted to leverage the option that AddGrainC has to scale up "grain", although it didn't end too well since the grain shows up as big squares instead of round grain, so it's a proof of concept and still need to play with the bias between AddGrainC scaling and blur strength (scaling so to speak). Will upload soon.

Now, I will be installing Windows 10, having a bit of a rest and from time to time improving TransformsPack.
Also I have almost finished my research on HVS (Human Visual System) and surround environment for viewing conditions. It's a two part study with some final suggestions on my part.
It's not AviSynth related but more like HTPC, so probably will post at General Discussion.

[LeXXuz]

Well, just updated SMDegrain mod to 4.0.0 with fully interlaced and YUY2 direct support for all clip type arguments (input, CClip, mfilter and prefilter).
I'm wrapping up my packs so unless something is really borked these will be final.

Thanks a lot for all your hard work. It's very much appreciated! :):thanks:

Btw I see you re-sorted prefilters again. You'd better give a heads up when doing that or people may wonder why their scripts behave strangely different. :D

ex_Median(mode="IQMV", UV=Chr)

That one looks new to me. When would you use that prefilter instead of Minblur or Flux?

[Guest]

Thanks a lot for all your hard work. It's very much appreciated! :):thanks:

Btw I see you re-sorted prefilters again. You'd better give a heads up when doing that or people may wonder why their scripts behave strangely different. :D

ex_Median(mode="IQMV", UV=Chr)

That one looks new to me. When would you use that prefilter instead of Minblur or Flux?

There has been so many updates in the passed days, it can be frustrating.

As for SMDegrain, the pre-filter order was changed back at build 3.5.8.

3.5.8

pref = !GlobalR ? preclip ? UHDhalf ? pref : prefilter : \
(prefilter==-1) ? pref : \
(prefilter== 0) ? pref.ex_MinBlur(0,Chr) : \
(prefilter== 1) ? pref.ex_MinBlur(1,Chr) : \
(prefilter== 2) ? pref.ex_MinBlur(2,Chr) : \
(prefilter== 3) ? pref.ex_Median(mode="IQMV",UV=Chr) : \
(prefilter== 4) ? pref.ex_FluxSmoothST(2,1,255,0,false,UV=Chr) : \
(prefilter== 5) ? ex_merge(dfttest(pref,sstring="0.0:4.0 0.2:9.0 1.0:15.0",tbsize=1,U=chroma,V=chroma,dither=1,threads=1,sbsize=12,sosize=6,swin=2),pref, \
pref.ex_lut(Format("range_max range_max {Lthres} ymin - / x ymin - * -"),UV=1,clamp_float=true), luma=chroma, UV=Chr) : \
(prefilter== 6) ? pref.ex_KNLMeansCL(a=2,s=2,d=1,h=7.0,wmode=1,chroma=chroma,gpuid=gpuid,LFR=600*(nw/1920.)).ex_boxblur(0.5, mode="weighted", UV=Chr) : \
(prefilter== 7) ? pref.ex_DGDenoise(str=0.10,LFR=nw/2.,UV=Chr). ex_boxblur(0.5, mode="weighted", UV=Chr) : \
(prefilter== 8) ? pref.ex_BM3D(10,1,"normal",UV=Chr,gpuid=gpuid,tv_range=!fsp) : \
Assert(false, "SMDegrain: Prefilter must be between -1~8: "+string(prefilter)) : \
pref

3.5.7

pref = !GlobalR ? preclip ? UHDhalf ? pref : prefilter : \
(prefilter==-1) ? pref : \
(prefilter== 0) ? pref.ex_MinBlur(0,Chr) : \
(prefilter== 1) ? pref.ex_MinBlur(1,Chr) : \
(prefilter== 2) ? pref.ex_MinBlur(2,Chr) : \
(prefilter== 3) ? pref.ex_FluxSmoothST(2,2,255,0,false,UV=Chr) : \
(prefilter== 4) ? ex_merge(dfttest(pref,sstring="0.0:4.0 0.2:9.0 1.0:15.0",tbsize=1,U=chroma,V=chroma,dither=1,threads=1,sbsize=12,sosize=6,swin=2),pref, \
pref.ex_lut(Format("range_max range_max {Lthres} ymin - / x ymin - * -"),UV=1,clamp_float=true), luma=chroma, UV=Chr) : \
(prefilter== 5) ? pref.ex_KNLMeansCL(a=2,s=2,d=1,h=7.0,wmode=1,chroma=chroma,gpuid=gpuid,LFR=600*(nw/1920.)).ex_boxblur(0.5, mode="weighted", UV=Chr) : \
(prefilter== 6) ? pref.ex_DGDenoise(str=0.10,LFR=nw/2.,UV=Chr). ex_boxblur(0.5, mode="weighted", UV=Chr) : \
(prefilter== 7) ? pref.ex_BM3D(10,1,"normal",UV=Chr,gpuid=gpuid,tv_range=!fsp) : \
Assert(false, "SMDegrain: Prefilter must be between -1~7: "+string(prefilter)) : \
pref

I need to apologize for this post being out of sync with the discussion, I had to wait for it to be approved..
And in that time, it has changed a couple of times.

[Dogway]

Btw I see you re-sorted prefilters again. You'd better give a heads up when doing that or people may wonder why their scripts behave strangely different. :D

ex_Median(mode="IQMV", UV=Chr)

That one looks new to me. When would you use that prefilter instead of Minblur or Flux?
Yes, it's explained in the docs now (https://raw.githack.com/Dogway/Avisynth-Scripts/master/SMDegrain/SMDegrain.html#prefilter2), it was changed a month ago in v3.5.8. I was thinking to also create a string type for 'prefilter' so you can define the prefilter preset with absolute terms like "DFTTest", instead of trying to remember the number. Will update with that.

[kedautinh12]

Yes, it's explained in the docs now (https://raw.githack.com/Dogway/Avisynth-Scripts/master/SMDegrain/SMDegrain.html#prefilter2), it was changed a month ago in v3.5.8. I was thinking to also create a string type for 'prefilter' so you can define the prefilter preset with absolute terms like "DFTTest", instead of trying to remember the number. Will update with that.

Are you test with RIFE-denoise and avs_mlrt??

[DTL]

RIFE-denoise is still sort of RIFE-MC only. It still require complex enough blending engine and simplest Average() may create significant blurring and very bad blends where RIFE fail to do ideal motion compensation. For better denoise activity it is require to set new task to developers of its neural-network to create as best possible motion compensated frames with 'current' frame as target. Currently it only hack-like usage of inbetween frame interpolator as motion-conpensation to 'current' timestamp using +N and -N frames only (a pair of frames only). For best results it need to analyse large sets of frames around current frame including current frame (and it will be even much more slower and may take much more memory).

So it is required to go to RIFE core developers, present current state of RIFE-denoise system and its current issues and ask for may be new RIFE core mode for denoise action specifically.


Now, I will be installing Windows 10,

Finally you can try DX12-ME with post-2.7.45 mvtools builds and your GTX1070 card.

[LeXXuz]

Finally you can try DX12-ME with post-2.7.45 mvtools builds and your GTX1070 card.

I'd love to have that in an upcoming build of SMDegrain someday. :rolleyes: :D

[Dogway]

@LeXXuz: Updated with absolute string preset names. Now you can do prefilter="DFTTest". IQMV is very similar to ex_MinBlur(3) but faster, and FluxSmoothST is a spatio-temporal version of MinBlur (only spatial).

I also added "RIFE" as 'mode' denoiser, but I didn't see much value in it, it doesn't denoise much, and warps content a little bit.

EDIT: I'm not convinced enough, so if nobody opposes I remove it in the next revision.

Finally you can try DX12-ME with post-2.7.45 mvtools builds and your GTX1070 card.

Maybe if I have time, I also want to run some AI scaling and delogo on personal projects. My card is not that much but nvidia releases have been a train-wreck lately so...

[LeXXuz]

@LeXXuz: Updated with absolute string preset names. Now you can do prefilter="DFTTest". IQMV is very similar to ex_MinBlur(3) but faster, and FluxSmoothST is a spatio-temporal version of MinBlur (only spatial).


Thanks for that update and the updated docs. The new parts about prefiltering were very interesting.

Because of that I'm fiddling around with a custom prefilter an I was wondering what I have to set for tv_range (true or false) when calling ex.BM3D()?
In SMDegrain it's set to tv_range=!fsp but I couldn't make out the syntax how the fsp variable is set. My Avisynth knowledge is too elementary. :o

[kedautinh12]

You set SMDegrain=true or false, ex_bm3d will set like that