Welcome to Doom9's Forum, THE in-place to be for everyone interested in DVD conversion.

Before you start posting please read the forum rules. By posting to this forum you agree to abide by the rules.

 

Go Back   Doom9's Forum > Capturing and Editing Video > Avisynth Usage

Reply
 
Thread Tools Search this Thread Display Modes
Old 22nd November 2017, 14:51   #1  |  Link
monohouse
Registered User
 
Join Date: Jul 2007
Posts: 106
I am new and I have a few general questions

I like avisynth, I am using avisynth+ on windows x64.
I am using it to clean up an interlaced MPEG2.
for now I am using daa3 for removing interlacing
Code:
(nn      = c.nnedi3(field=-2,nsize=6,nns=4,qual=2,pscrn=4,fapprox=0))
but I am not yet sure if it is better to use QTGMC (I also tested mcdaa3 by using his daa3mod function but his results were not so good)
I am also doing the best I can to do noise reduction, after some testing
Code:
MCTemporalDenoise(p=pre,search=3, searchparam=4, pel=4, pelsearch=4, bwbh=512, owoh=256, blksize=4, overlap=2, edgeclean=false, radius=5, twopass=true, useTTmpSm=true,limit=1,limit2=1,stabilize=true,maxr=1,TTstr=1,sigma=0,sharp=false,protect=false,AA=false,deblock=false,post=0,chroma=true,interlaced=false,GPU=true,precision=2,bt=4,enhance=false)
gives the best quality (I also tried knlCL but he makes stains)
I may use some sharpening later but I haven't decided that yet (3 of 4 sharpeners I tested increase noise, the only one who doesn't is MSharpen but has some other small problems)
the result is to be encoded into lossless x264 with:
Code:
x264.2851.x86_64.exe --qp 0 --level 6.
2 --ref 16 --bframes 16 --b-adapt 2 --8x8dct --mixed-refs --direct auto --slow-f
irstpass --no-fast-pskip --me tesa --merange 24 --partitions all --rc-lookahead
60 --subme 11 --trellis 2
1. when MCTD is running twopass=true, is it recommended/benefitial to use p=pre ? or is it that twopass already does that ?
2. I know that x264 lossless is lossless regardless of --pass 2/--pass 3 (multipass x264 encoding) and I know that many use --pass 2 to control file size. if I don't care about variances in file size but do care about reducing file size, should I use --pass 2/3 ?
3. is MCTD() (the mod) better than MCTemporalDenoise() ?
4. is QTGMC() better than daa3() ? the source has not only interlacing but also aliasing and flicker (flicker I couldn't find a way to remove without causing other problems

Last edited by monohouse; 22nd November 2017 at 15:00.
monohouse is offline   Reply With Quote
Old 22nd November 2017, 15:02   #2  |  Link
StainlessS
HeartlessS Usurer
 
StainlessS's Avatar
 
Join Date: Dec 2009
Location: Over the rainbow
Posts: 10,980
This provides a little denoising, + sharpen, I use it constantly with Frames=1.
MCDegrainSharp()
Quote:
# From:- http://forum.doom9.org/showthread.ph...45#post1737045
# Based on MCDegrain By Didee, http://forum.doom9.org/showthread.php?t=161594
# Also based on DiDee observations in this thread: http://forum.doom9.org/showthread.php?t=161580
# "Denoise with MDegrainX, do slight sharpening where motionmatch is good, do slight blurring where motionmatch is bad"
# In areas where MAnalyse cannot find good matches, the blur() will be dominant.
# In areas where good matches are found, the sharpen()'ed pixels will overweight the blur()'ed pixels
# when the pixel averaging is performed.
https://forum.doom9.org/showthread.p...45#post1737045
__________________
I sometimes post sober.
StainlessS@MediaFire ::: AND/OR ::: StainlessS@SendSpace

"Some infinities are bigger than other infinities", but how many of them are infinitely bigger ???

Last edited by StainlessS; 22nd November 2017 at 15:06.
StainlessS is offline   Reply With Quote
Old 22nd November 2017, 15:13   #3  |  Link
monohouse
Registered User
 
Join Date: Jul 2007
Posts: 106
is nice StainlessS, I like it
thank !
but which is which ? I understand top right is the mask but what is the other 3 ?
monohouse is offline   Reply With Quote
Old 22nd November 2017, 15:22   #4  |  Link
StainlessS
HeartlessS Usurer
 
StainlessS's Avatar
 
Join Date: Dec 2009
Location: Over the rainbow
Posts: 10,980
Quote:
Originally Posted by monohouse View Post
iI understand top right is the mask but what is the other 3 ?
what mask ???, not a single image in entire linked thread.
__________________
I sometimes post sober.
StainlessS@MediaFire ::: AND/OR ::: StainlessS@SendSpace

"Some infinities are bigger than other infinities", but how many of them are infinitely bigger ???
StainlessS is offline   Reply With Quote
Old 22nd November 2017, 15:31   #5  |  Link
monohouse
Registered User
 
Join Date: Jul 2007
Posts: 106
I ran your code and looked at results
I understand bottom right image is the output and top left is the input, but what are the other 2 ?
monohouse is offline   Reply With Quote
Old 22nd November 2017, 15:36   #6  |  Link
StainlessS
HeartlessS Usurer
 
StainlessS's Avatar
 
Join Date: Dec 2009
Location: Over the rainbow
Posts: 10,980
Oops, sorry, you meant result of the client script.
That script was just in question to Feisty2, top right is amplified difference between the two results both with and without Precise=True.
Set Amp=False for non amp'd diff.

EDIT: Top Left=Source, BotLeft=Precise=False, BotRight=Precise=True.

EDIT: A=MCDegrainSharp(2,bsrch=BSRCH,Precise=False)
Above has Frames=2 for both function calls, I usually use 1, but sometimes up to 3.

EDIT: The Amp'ed difference just to make obvious where there are differences, so you can examine those areas more closely.
You could change eg Frames=1 in A and Frames=2 in B and set Precise = False/true *same for both)
to view difference due to only Frames arg. (Below untested)
Code:
BSRCH=true
PRECISE1=True
PRECISE2=True
FRAMES1=1
FRAMES2=2
AMP=True
#AviSourcE("E:\V\FlashTest.avi")
ColorBars.convertToYV12.BilinearResize(320,240) # just to test

O=Last
A=MCDegrainSharp(Frames=FRAMES1,bsrch=BSRCH,Precise=PRECISE1)
B=MCDegrainSharp(Frames=FRAMES2,bsrch=BSRCH,Precise=PRECISE2)
D=ClipDelta(A,B,Amp=AMP)
O=Sub(O,"Source")
D=Sub(D,"D) Difference : Amp="+String(AMP))
A=Sub(A,String(FRAMES1,"A) MCDS : Frames=%.0f : Precise=")+String(PRECISE1))
B=Sub(B,String(FRAMES2,"B) MCDS : Frames=%.0f : Precise=")+String(PRECISE2))
Return StackVertical(StackHorizontal(O,D),StackHorizontal(A,B))

# Return Clip Difference of input clips (amp==true = Amplified, show==true = show background)
Function ClipDelta(clip clip1,clip clip2,bool "amp",bool "show") {
    amp=Default(amp,false)
    show=Default(show,false)
    c2=clip1.levels(128-32,1.0,128+32,128-32,128+32).greyscale()
    c1=clip1.subtract(clip2)
    c1=(amp)?c1.levels(127,1.0,129,0,255):c1
    return (show)?c1.Merge(c2):c1
}

Function Sub(clip c,string Tit,Bool "ShowFrameNo",int "first_frame", int "last_frame",string "font",float "size",int "text_color",
    \ int "halo_color",int "align",int "spc",float "font_width",float "font_angle",Int "BackColor") {
    /*
        Stack Overhead Subtitle Text, with optional FrameNumber shown
        http://forum.doom9.org/showthread.php?p=1813402#post1813402
        Title bar is Round(size+2) pixels hi (default 20).
        Dont use align=4,5,6, better use 1,2,3,7,8,or 9.
    */
    ShowFrameNo=Default(ShowFrameNo,False)      first_frame=Default(first_frame,0)          last_frame=Default(last_frame,c.FrameCount-1)
    font=default(font,"Ariel")                  size=Default(size,18.0)                     text_color=Default(text_color,$00FFFF00)
    halo_color=Default(halo_color,$00000000)    align=default(align,7)                      spc=Default(spc,0)
    font_width=Default(font_width,0)            font_angle=Default(font_angle,0.0)          BackColor=Default(BackColor,$00000000)
    c.BlankClip(height=round(size+2.0),Color=BackColor)
    (ShowFrameNo)
        \ ? ScriptClip("""Subtitle(String(current_frame,"%.0f] ")+"""+Chr(34)+Tit+Chr(34)+String(first_frame,",first_frame=%.0f")+
        \       String(last_frame,",last_frame=%.0f,font=")+Chr(34)+font+Chr(34)+String(size,",size=%.3f")+String(text_color,",text_color=%.0f")+
        \       String(halo_color,",halo_color=%.0f")+String(align,",align=%.0f")+String(spc,",spc=%.0f")+String(font_width,",font_width=%.3f")+
        \       String(font_angle,",font_angle=%.3f)"))
        \ : Subtitle(Tit,first_frame=first_frame,last_frame=last_frame,font=font,size=size,text_color=text_color,halo_color=halo_color,align=align,
        \       spc=spc,font_width=font_width,font_angle=font_angle)
    Return StackVertical(c).AudioDubEx(c)
}
EDIT: Above updated, tested for non crash.
EDIT: Oops, fixed bug.

__________________
I sometimes post sober.
StainlessS@MediaFire ::: AND/OR ::: StainlessS@SendSpace

"Some infinities are bigger than other infinities", but how many of them are infinitely bigger ???

Last edited by StainlessS; 22nd November 2017 at 16:37.
StainlessS is offline   Reply With Quote
Old 22nd November 2017, 16:58   #7  |  Link
monohouse
Registered User
 
Join Date: Jul 2007
Posts: 106
will do, thank

also what would be the recommended order for the operations ? antialiasing+deinterlacing and noise reduction and sharpening ?
I see it like this:
Code:
AA/DEINT
NR
Sharp
but I suppose it's right to do Sharp before NR to compensate for increased noise from sharp ? QTGMC with noiseprocess can also add noise right so AA/DEINT is always at the top ? I suppose if you sharp before AA/DEINT it could affect for better or worse the AA/DEINT ?

Last edited by monohouse; 22nd November 2017 at 17:18.
monohouse is offline   Reply With Quote
Old 22nd November 2017, 20:38   #8  |  Link
Asmodian
Registered User
 
Join Date: Feb 2002
Location: San Jose, California
Posts: 4,406
Deinterlacing should always be first, sharpening before will affect deint for the worse.
__________________
madVR options explained
Asmodian is offline   Reply With Quote
Old 22nd November 2017, 23:13   #9  |  Link
real.finder
Registered User
 
Join Date: Jan 2012
Location: Mesopotamia
Posts: 2,587
Quote:
4. is QTGMC() better than daa3() ? the source has not only interlacing but also aliasing and flicker (flicker I couldn't find a way to remove without causing other problems
daa things are not really antialiasing, daa did almost same thing as Vinverse(), The comparison is not valid between De-interlacing function and combing removal or even antialiasing

post sample, cuz most antialiasing artifact come from bad/inappropriate Deinterlaced
__________________
See My Avisynth Stuff
real.finder is offline   Reply With Quote
Old 23rd November 2017, 02:06   #10  |  Link
monohouse
Registered User
 
Join Date: Jul 2007
Posts: 106
I did many tests, I used QTGMC, AAAmod, sangnom2, nnedi3, maa2, daa3 and mcdaa3: none of them reach remotely close to the artifact reduction/sharpness/interlacing elimination qualities of daa3 for this source anyways. the one that did give very good result is TIVTC, very effective, even better in some areas with horizontal lines.

I have a small problem, the process is very long, so I have idea to run the process on another computer, but the computers are not the same, the computer that runs right now has multimedia instructions AVX+2, SSE 1,2,3,3S,4.1,4.2, but the other computer has SSE 1,2,3,4A without AVX, I am asking because I am afraid to loose precision in the process not important for me speed, does AVX/2 give more precision in registers than SSE registers ?

Last edited by monohouse; 24th November 2017 at 17:30.
monohouse is offline   Reply With Quote
Old 28th November 2017, 14:48   #11  |  Link
magiblot
Eurobeat Fan
 
Join Date: Sep 2014
Posts: 108
Quote:
Originally Posted by monohouse View Post
the one that did give very good result is TIVTC, very effective, even better in some areas with horizontal lines.
Could your source be telecined (23.976 fps -> 29.970 fps conversion)? If the interlacing is due to telecine pulldown all you need to do is match the fields and remove duplicate frames (no heavy processing at all). This is what you call inverse telecine (IVTC). I would say most used IVTC plugins are Decomb (Telecide, Decimate) and TIVTC (TFM, TDecimate). An example:

Code:
TFM(pp=0) #aligns matching fields (picture stops looking interlaced)
TDecimate() #removes duplicate frames and restores original framerate 23.976 fps. mode=1 is suggested for animation
If the picture still looks interlaced after applying TFM then it's not telecined. If it looks like frames are not in the right order then you should try assuming another field order by calling either AssumeTFF() or AssumeBFF() before TFM, but this shouldn't be necessary if you are importing a .d2v file with MPEG2Source. If after applying the code above interlacing disappears except for only some parts (titles, intros, etc.) then your source is hybrid.
magiblot is offline   Reply With Quote
Old 3rd December 2017, 02:38   #12  |  Link
manono
Moderator
 
Join Date: Oct 2001
Location: Hawaii
Posts: 7,406
Perhaps you can provide a short 10-second sample from the source? One with steady movement? You can cut samples using DGIndex.
manono is offline   Reply With Quote
Old 3rd December 2017, 12:37   #13  |  Link
monohouse
Registered User
 
Join Date: Jul 2007
Posts: 106
there is significant improvement with the first set of code, then I added TFF but it had little effect, BFF looked the same as without assume
with this code the horizontal lines are significantly reduced and are visible only in fast movements, good catch
I will see what I can do about sample
the problem with daa3 is not his effectiveness at deinterlace/antialias at this he is the best but he leaves some flickering in some areas and if someone can help figure out how to fix it I will give you half of the box office
as you will see soon in the samples, the movie has many problems:
interlacing, aliasing, flickering, noise, 16-235 range which I can't extend to full range, blurry not realy sharp enough
the blurry problem I can live with since I know there is no solution to that, the range problem have no choice but to live with since that also have no solution, but I would like to do anything I can to fix the other problems

im working on a VA panel with 5000 static contrast and standard 80% gamut with less than 5% color error ratio, is nice but is not externally calibrated, he is full 8-bit native panel

there you go 600 frames of lossless 264 goodness xD
Source
TFM(),TDecimate()
daa3

tfm(), Decimate() made some stutter, I don't know whay at first I thought I had MT enable but then I checked and it was off during the encoding

Last edited by monohouse; 3rd December 2017 at 13:40.
monohouse is offline   Reply With Quote
Old 3rd December 2017, 20:04   #14  |  Link
manono
Moderator
 
Join Date: Oct 2001
Location: Hawaii
Posts: 7,406
Why lossless? Didn't I suggest cutting from the MPG using DGIndex? Much smaller file; much less time to download. I don't much want to wait an hour to download from whatever that place is. There are much better filesharing sites.

Open the MPG in DGindex. Use the [ and ] buttons to isolate a short section. Then File->Save Project and Demux Video.
manono is offline   Reply With Quote
Old 6th December 2017, 15:37   #15  |  Link
kuchikirukia
Registered User
 
Join Date: Oct 2014
Posts: 476
The live action just looks telecined to me. The CGI is borked. If you're not going to go through and filter it separately, just leave it as it is. It's not worth ruining everything else just to fix it. You can do 2-pass to make a VFR file so you don't decimate those.

Run an analysis pass with:
tfm(d2v="D:\video.d2v",output="tfm.txt").tdecimate(mode=4,output="tdecimate.txt")

Then encode with:
tfm(d2v="D:\video.d2v",input="tfm.txt").tdecimate(mode=5,hybrid=2,vfrDec=1,input="tdecimate.txt",tfmIn="tfm.txt",mkvOut="mkv-timecodesfile.txt")

And mux with the time code file.

Last edited by kuchikirukia; 6th December 2017 at 15:41.
kuchikirukia is offline   Reply With Quote
Old 6th December 2017, 23:16   #16  |  Link
monohouse
Registered User
 
Join Date: Jul 2007
Posts: 106
i did all this but the output is stuttering, how to fix it ?
does daa3 ruining the other things while fixing the CGI ?
the timecode file is only 300 bytes and have this in it:
Assume 29.970030
# TDecimate v1.0.3 by tritical
# Mode 5 - Auto-generated mkv timecodes file
0,7999,23.976024

and nothing else

I don't have d2v files i used ffmpegsource is that a problem ?

I created d2v files with dgindex but now the avs file when it's running is leaking memory
he also says error: TFM: d2v frame count does not match filter frame count (79465 vs 63572)!

Last edited by monohouse; 7th December 2017 at 09:04.
monohouse is offline   Reply With Quote
Old 7th December 2017, 09:10   #17  |  Link
hello_hello
Registered User
 
Join Date: Mar 2011
Posts: 4,823
Quote:
Originally Posted by kuchikirukia View Post
Run an analysis pass with:
tfm(d2v="D:\video.d2v",output="tfm.txt").tdecimate(mode=4,output="tdecimate.txt")

Then encode with:
tfm(d2v="D:\video.d2v",input="tfm.txt").tdecimate(mode=5,hybrid=2,vfrDec=1,input="tdecimate.txt",tfmIn="tfm.txt",mkvOut="mkv-timecodesfile.txt")

And mux with the time code file.
Assuming the output is VFR, would it be better to encode in VFR mode? I kind of recall x264 distributes the bits a bit differently in VFR mode (CRF encoding) as it's frame rate aware, or an I remembering wrong?

The x264 encoder can encode in VFR mode by specifying something like
--tcfile-in "D:\timecodesfile.txt" in the command line.

I use MeGUI for VFR encoding now and then. The timecodes file doesn't need to exist when you add it to the command line and add the encoding job to the queue. It'll be created as soon as the second pass script is run. You just need to plan ahead in respect to naming the timecodes files if you're creating multiple encoding jobs. If the encoder is writing directly to MKV there's no need to worry about muxing with the timecodes file.
Batch=true is an option to be aware of when running both passes in some sort of automated job queue.

Last edited by hello_hello; 7th December 2017 at 09:13.
hello_hello is offline   Reply With Quote
Old 7th December 2017, 10:03   #18  |  Link
monohouse
Registered User
 
Join Date: Jul 2007
Posts: 106
I just had another look at the output of daa3, you are right, it does add noise, but plain and simple needi3 looks better than tfm, unlike tfm it doesn't miss some of the interlacing, this explains whay I needed to use such high noise reduction settings, without the noise that is added from daa3 I can cut the noise reduction quantity by half and might as well set twopass to false, this will double the rendering framerate and at the same time increase the quality

I tried to modify the daa3 code and remove the sharpening stage from it:
Code:
    shrpD   = mt_makediff(dbl,dbl,U=3,V=3)
I think it worked good, but now when comparing to nnedi3, needi3 looks like it preserves many details that daa3 removes
with the reduced noise reduction requirements I set to twopass=false and reduced more the temporal radius to 1, even with the optimistic settings I am using it went up from 0.00/0.01 fps to 0.60 fps, 60 times more speed

Last edited by monohouse; 8th December 2017 at 07:56.
monohouse is offline   Reply With Quote
Old 9th December 2017, 04:16   #19  |  Link
kuchikirukia
Registered User
 
Join Date: Oct 2014
Posts: 476
It's not interlaced.
kuchikirukia is offline   Reply With Quote
Old 16th December 2017, 06:45   #20  |  Link
monohouse
Registered User
 
Join Date: Jul 2007
Posts: 106
this is a small improvement for daa3 with controllable settings:
Code:
    #controllable Anti-aliasing with contra-sharpening by Didée
    FUNCTION daa3(clip c, int "nthreads", int "nsharp", int "neurons", int "q", int "size" ) {
	nthreads    = default(nthreads, 8)
	nsharp      = default(nsharp, 25)
	neurons		= default(neurons, 4)
	q			= default(q, 5)
	size		= default(size, 3)
	Assert((nthreads>=1)        	 ? true : false, chr(10) + "'nthreads' must be at least 1" + chr(10))
	Assert((nsharp>=-1&&nsharp<=25)  ? true : false, chr(10) + "'nsharp' must be between -1 and 25" + chr(10))
	Assert((q>=0&&q<=5)      	     ? true : false, chr(10) + "'q' must be between 0 and 5" + chr(10))
	Assert((neurons>=0&&neurons<=4)  ? true : false, chr(10) + "'neurons' must be between 0 and 4" + chr(10))
	Assert((size>=1&&size<=6)        ? true : false, chr(10) + "'size' must be between 1 and 6" + chr(10))
	nn = (q == 0)  ? c.nnedi3(field=-2,nsize=size,nns=neurons,qual=1,pscrn=2,fapprox=0,threads=nthreads) :
	\	(q == 1)  ? c.nnedi3(field=-2,nsize=size,nns=neurons,qual=1,pscrn=1,fapprox=0,threads=nthreads) :
	\	(q == 2)  ? c.nnedi3(field=-2,nsize=size,nns=neurons,qual=1,pscrn=0,fapprox=0,threads=nthreads) :
	\	(q == 3)  ? c.nnedi3(field=-2,nsize=size,nns=neurons,qual=2,pscrn=2,fapprox=0,threads=nthreads) :
	\	(q == 4)  ? c.nnedi3(field=-2,nsize=size,nns=neurons,qual=2,pscrn=1,fapprox=0,threads=nthreads) :
	\	c.nnedi3(field=-2,nsize=size,nns=neurons,qual=2,pscrn=0,fapprox=0,threads=nthreads)
    dbl     = mt_average(selecteven(nn),selectodd(nn),U=3,V=3)
    dblD    = mt_makediff(c,dbl,U=3,V=3)
	shrpD   = nsharp==25 ? mt_makediff(dbl,dbl,U=3,V=3) : mt_makediff(dbl,dbl.removegrain(nsharp),U=3,V=3)
    DD      = shrpD.repair(dblD,13)
    return dbl.mt_adddiff(DD,U=3,V=3) }
everything is on highest by default and sharpening is default to off
monohouse is offline   Reply With Quote
Reply

Thread Tools Search this Thread
Search this Thread:

Advanced Search
Display Modes

Posting Rules
You may not post new threads
You may not post replies
You may not post attachments
You may not edit your posts

BB code is On
Smilies are On
[IMG] code is On
HTML code is Off

Forum Jump


All times are GMT +1. The time now is 14:16.


Powered by vBulletin® Version 3.8.11
Copyright ©2000 - 2024, vBulletin Solutions Inc.