Color banding and noise removal

[cretindesalpes]

Oh I can just make the message more explicit.

[cybersharky]

QTGMC( Preset="Slow",FPSdivisor=2)
o=last
super = MSuper(pel=2,sharp=2)
backward_vec2 = MAnalyse(super, isb = true, delta = 2, overlap=4)
backward_vec1 = MAnalyse(super, isb = true, delta = 1, overlap=4)
forward_vec1 = MAnalyse(super, isb = false, delta = 1, overlap=4)
forward_vec2 = MAnalyse(super, isb = false, delta = 2, overlap=4)
bc1  = o.MCompensate(super,backward_vec1)
fc1  = o.MCompensate(super,forward_vec1 )

Interleave(fc1,o,bc1)
MDegrain2(super, backward_vec1,forward_vec1,backward_vec2,forward_vec2,thSAD=400)
SelectEvery(3,1)

How do I use MDegrain2 as the denoiser with MCompensate?

[StainlessS]

You would (I think) need a new Super & set of vectors on the interleaved clip as the ones already calculated
relate to the original (o) clip and not the interleaved one.
Could well be wrong but think that there is little to be gained by doing motion compensation twice.
Suggest that you just use instead, the function linked below after QTGMC.
http://forum.doom9.org/showthread.ph...89#post1508289 # By Didee, MCDegrain

EDIT: Changed above link to point to specific post rather than thread.

EDIT: Ref the code in prev post, if you throw away the vec2 vectors, then you could use as is
using some denoiser that takes a single frame either side of the frames to denoise eg FFT3DFilter,
instead of the call to MDegrain2 (which requires two frames either side of frame to denoise).

EDIT: If this is for B5 Ivana's head spots, then perhaps QTGMC has already applied temporal denoising
and partially migrated some of the spots into adjacent frames making it difficult to remove the spots later.
Despotting really should be done before any temporal processing, perhaps on even/odd separated fields,
separately.

[GMJCZP]

Continued instability in MVTools (see my previous post).

PD: Or is it a incompatibildad with the latest version of avisynth?

[cretindesalpes]

QMJCZP:

Please be more specific. I would need your script, as simple as possible to make it crash, and an excerpt of your source clip that exhibits the problem.

[GMJCZP]

Here the sample and script:

Here

Thanks.

[ajp_anton]

I feel like I missed something trivial, but is there an easy way to convert RGB24 to RGB48?

[cretindesalpes]

QMJCZP:

Thank you. I'll check your sample.

ajp_anton:

To convert from RGB24 to the format I call RGB48Y:

Interleave (
\ ShowRed (pixel_type="YV12"),
\ ShowGreen (pixel_type="YV12"),
\ ShowBlue (pixel_type="YV12"))
Dither_convert_8_to_16 ()

[ajp_anton]

That's what I ended up using. I just thought there'd be an easier way. Why not make dither_convert_8_to_16 support RGB also?
BTW, any reason not to have RGB48 as stacked RGB instead of 3x stacked Y-planes?

[cretindesalpes]

Because these tools are designed to work with planar data, which RGB is not. I could make them support any possible colorspace but this is a lot of work I don't want to do. My main goal for the version 1 of this package is to bring new possibilities in Avisynth processing, not to achieve production-class tools. That's why everything looks like clumsy hacks ; I just don't care about elegant design because I didn't plan any specification or anything, I just go where it looks interesting at the moment.

[cybersharky]

It appears I'm also encountering MVTools issues.

SetMemoryMax(512)
SetMTMode(2,0)
LoadPlugin("D:\MeGUI_OneClick_Preview\tools\avisynth_plugin\mvtools2.dll")
LoadPlugin("D:\MeGUI_OneClick_Preview\tools\avisynth_plugin\avstp.dll")
LoadPlugin("D:\MeGUI_OneClick_Preview\tools\avisynth_plugin\dither.dll")
Import("D:\MeGUI_OneClick_Preview\tools\avisynth_plugin\QTGMC-3.33.avsi")
Import("D:\MeGUI_OneClick_Preview\tools\avisynth_plugin\dither.avsi")
Import("D:\MeGUI_OneClick_Preview\tools\avisynth_plugin\mt_xxpand_multi.avsi")
LoadPlugin("D:\MeGUI_OneClick_Preview\tools\avisynth_plugin\mt_masktools-25.dll")
LoadPlugin("D:\MeGUI_OneClick_Preview\tools\avisynth_plugin\RemoveGrainSSE2.dll")
LoadPlugin("D:\MeGUI_OneClick_Preview\tools\avisynth_plugin\NNEDI3.dll")
LoadPlugin("D:\MeGUI_OneClick_Preview\tools\avisynth_plugin\TTempSmooth.dll")
LoadPlugin("D:\MeGUI_OneClick_Preview\tools\avisynth_plugin\FFT3DFilter.dll")
Import("D:\MeGUI_OneClick_Preview\tools\avisynth_plugin\McDegrainSharp.avsi")
LoadPlugin("D:\MeGUI_OneClick_Preview\tools\dgindex\DGDecode.dll")
mpeg2source("C:\Users\me\Documents\DVDrips\BABYLON5_S4_DISC4_E15\VIDEO_TS\VTS_01_1.d2v")
QTGMC( Preset="Slow",FPSdivisor=2)
LanczosResize(640,368)
McDegrainSharp()
GradFun3 ()

Faulting application name: x264.exe, version: 0.124.2197.0, time stamp: 0x4f97d104
Faulting module name: mvtools2.dll, version: 2.6.0.3, time stamp: 0x4faf77d0
Exception code: 0xc0000005
Fault offset: 0x000a2a61
Faulting process id: 0x1a0c
Faulting application start time: 0x01cd39855f35be4d
Faulting application path: D:\MeGUI_OneClick_Preview\tools\x264\x264.exe
Faulting module path: D:\MeGUI_OneClick_Preview\tools\avisynth_plugin\mvtools2.dll
Report Id: a725c16f-a578-11e1-8831-1c6f65d777b9
Event Xml:
<Event xmlns="http://schemas.microsoft.com/win/2004/08/events/event">
  <System>
    <Provider Name="Application Error" />
    <EventID Qualifiers="0">1000</EventID>
    <Level>2</Level>
    <Task>100</Task>
    <Keywords>0x80000000000000</Keywords>
    <TimeCreated SystemTime="2012-05-24T08:15:42.000000000Z" />
    <EventRecordID>27978</EventRecordID>
    <Channel>Application</Channel>
    <Computer>my-PC</Computer>
    <Security />
  </System>
  <EventData>
    <Data>x264.exe</Data>
    <Data>0.124.2197.0</Data>
    <Data>4f97d104</Data>
    <Data>mvtools2.dll</Data>
    <Data>2.6.0.3</Data>
    <Data>4faf77d0</Data>
    <Data>c0000005</Data>
    <Data>000a2a61</Data>
    <Data>1a0c</Data>
    <Data>01cd39855f35be4d</Data>
    <Data>D:\MeGUI_OneClick_Preview\tools\x264\x264.exe</Data>
    <Data>D:\MeGUI_OneClick_Preview\tools\avisynth_plugin\mvtools2.dll</Data>
    <Data>a725c16f-a578-11e1-8831-1c6f65d777b9</Data>
  </EventData>
</Event>

[Dogway]

Just wanted to add my 2cents. I got a thread deadlock while encoding to FFV1, I don't know if it's relevant since I don't encode as often but I used:
setmtmode(2,2)
smdegrain(...)
I wonder if that's right, or avstp.dll has crashes with MT, I chose to take out avstp.dll for that specific encode. Hope it helps...

[Boulder]

I had issues with MVTools internal multithreading and SetMTMode, disabling internal multithreading fixed them.

[Boulotaur2024]

Yeah, I've always had troubles using anything else than SetMtMode(5) or no SetMtMode at all iirc, the recommended SetMtMode(3) MySource() SetMtMode(2) SMDegrain always crashed eventually even though it first looked like it was about to work.

By the way a huge thanks to CretinDesAlpes (chapeau l'artiste si tu me permets), it really felt like night and day between not using your tools at all and learning how to finally properly use them (it's not so easy though). The added value is like 200% on the final encode, I've always wanted to thank you but couldn't for some reason. You have all my respect... and more

[cretindesalpes]

Boulotaur2024:

Thanks for your support!

Dogway:

I couldn't reproduce the deadlock. I used smdegrain() with default settings, and encoded ~10000 PAL frames with FFV1 in CLI FFmpeg.

EDIT: I tried with tr=4 and it freezed after 4500 frames. Now I have something to investigate!

cybersharky:

I couldn't reproduce the crash. I tried encoding about 20000 frames. However:
– I don't know the number of threads you used, as you just specified 0 in the SetMTMode parameter. I tried with 4 threads (my native configuration).
– It's strongly advised to run the source filter (mpeg2source here) in MT mode 5 and to switch to mode 2 afterwards.

QMJCZP:

I couldn't make your script crash. But I have a few more questions:
– Could you confirm that the script crashes with the exact source sample you provided in the zip file? It is very short, and the script loaded a different file.
– The SetMTMode() functions were commented in your script, and I have no idea what was the number of threads it actually used. This matters because of the related memory usage, the cache memory being fixed to 512 KB.
– I couldn't find any function called PreF_MDeGrain2.avsi (apparently from John Meyer), so I had to bypass it. I would need it to check the script.
– While looking for a link to this function, I stumbled across a Didée's post regarding a Clense instability in MT mode. And your script uses Clense via RemoveDirt2. So you could try to replace it with the Didée's suggestion.

[GMJCZP]

Originally Posted by cretindesalpes

Thanks for your support!

QMJCZP:

I couldn't make your script crash. But I have a few more questions:
– Could you confirm that the script crashes with the exact source sample you provided in the zip file? It is very short, and the script loaded a different file.
– The SetMTMode() functions were commented in your script, and I have no idea what was the number of threads it actually used. This matters because of the related memory usage, the cache memory being fixed to 512 KB.
– I couldn't find any function called PreF_MDeGrain2.avsi (apparently from John Meyer), so I had to bypass it. I would need it to check the script.
– While looking for a link to this function, I stumbled across a Didée's post regarding a Clense instability in MT mode. And your script uses Clense via RemoveDirt2. So you could try to replace it with the Didée's suggestion.

Thanks cretindesalpes:

I will answer in order:

- The source is the same, what happens is that as I have no internet itself I could not upload a file larger. The script is generic, because it is for a series of DVDs.
- Indeed, I use SetMTMode (), what happens is grayed out to prevent the crash, for your testing simply remove the # to the left of the command. My PC is a Core 2 Duo E4400 with 2GB RAM.
- Sorry for not having included PreF_MDeGrain2 function, here the script:

function PreF_MDeGrain2(clip src)
{

preNR = src.unfilter(-6,-6)

preNR_super = preNR.MSuper(pel=2, sharp=1)
src_super = src.MSuper(pel=2, sharp=1, levels=1)

backward_vec4 = MAnalyse(preNR_super, isb = true, delta = 4, overlap=4)
backward_vec2 = MAnalyse(preNR_super, isb = true, delta = 2, overlap=4)
forward_vec2 = MAnalyse(preNR_super, isb = false, delta = 2, overlap=4)
forward_vec4 = MAnalyse(preNR_super, isb = false, delta = 4, overlap=4)

src.MDegrain2(src_super, backward_vec2,forward_vec2,backward_vec4,forward_vec4,thSAD=400)
}

- I'll try the Didée's suggestion. Thanks for the tip.

Whatever else you need let me know.
Good luck, and thanks again.

[cretindesalpes]

QMJCZP:

Thank you. But I still couldn't make the script crash.

Dogway:

OK, I think I found the problem. Please update avstp.

[Lenchik]

Someone may be interested in implementation of such conversion through Dither plugin: http://forum.doom9.org/showthread.ph...432#post639432

[nhope]

I'd be interested in suggestions for dealing with the banding that you see starting at about 03:00 in my test video on YouTube, as the blue opening of the cave fades up from black.

The clip was shot in HDV (1440x1080-60i). There is no significant banding in the original clip. There is some noise. For YouTube I bobbed to 30p as follows, then wrote an RGB UT Video Codec lossless intermediate for further editing in Vegas, then encoded it to x264 in MeGUI at crf 18.

#Clip frameserved in RGB from Sony Vegas Pro
AviSource("d:\fs.avi")
ConvertToYV12(interlaced=true)
AssumeTFF
QTGMC( Preset="Slower", EdiThreads=2 )
Spline36Resize(1920,1080)
AssumeFPS(29.97)

I can provide source if it helps. If so, the original .m2t HDV file, or the lossless file from later in the process?

[Reel.Deel]

Hi nhope, I'm assuming your original HDV source is YUV 4:2:0. Correct? Why not deinterlace in it's original color space and save to a lossless file and then import to Vegas? It saves you from having extra unnecessary color space conversions.

BTW the link to your video on youtube does not work.