Welcome to Doom9's Forum, THE in-place to be for everyone interested in DVD conversion.

Before you start posting please read the forum rules. By posting to this forum you agree to abide by the rules.

 

Go Back   Doom9's Forum > Capturing and Editing Video > Avisynth Usage

Reply
 
Thread Tools Search this Thread Display Modes
Old 17th June 2015, 10:26   #1  |  Link
LouieChuckyMerry
Registered User
 
LouieChuckyMerry's Avatar
 
Join Date: Feb 2014
Posts: 230
Seeking Advice To Improve An AviSynth Script For NTSC Simpsons DVDs

Hello and thanks in advance for any help . I'm in the process of encoding my NTSC DVD's of the first ten seasons of the Simpsons and, after seemingly endless trial-and-error, dumb luck, and kind help from various people in various places, I've come to the following script:

Code:
# Set DAR in encoder to 6480 : 4739. The following line is for automatic signalling
global MeGUI_darx = 6480
global MeGUI_dary = 4739
SetMemoryMax(256)
SetMTMode(3,3)
LoadPlugin("F:\[0]StandAloneApps\MeGUI-2500(core)2443(data)0.3.5(libs)[Portable]\tools\ffms\ffms2.dll")
FFVideoSource("SourcePath", fpsnum=30000, fpsden=1001, threads=1)
### Deinterlace ###
SetMTMode(5)
LoadPlugin("F:\[0]StandAloneApps\MeGUI-2500(core)2443(data)0.3.5(libs)[Portable]\tools\avisynth_plugin\TIVTC.dll")
TFM(Order=-1,Slow=2,PP=0).TDecimate(Mode=1)
SetMTMode(2)
Vinverse()
### Deshaker ###
Stab(Mirror=15)
### Crop ###
Crop(8,0,-8,0)
### Resize ###
RatioResize(10/11.0,"PAR")
### Gibbs Noise Block ###
Edge=MT_Edge("prewitt",thY1=20,thY2=40).RemoveGrain(17)
Mask=MT_Logic(Edge.MT_Expand().MT_Expand().MT_Expand().MT_Expand(),Edge.MT_Inflate().MT_Inpand(),"xor")
MT_Merge(dfttest(),Mask,Luma=True)
### Overall Temporal Denoise ###
SMDegrain(tr=2,thSAD=600,ContraSharp=True,RefineMotion=True,Plane=0,Lsb=True,Lsb_Out=True,PreFilter=2)
### Debanding ###
GradFun3(thR=0.55,Radius=12,Mask=2,SMode=1,Lsb=True,Lsb_In=True, StaticNoise=True,Y=3,U=3,V=3)
DitherPost(Stacked=True,Prot=False,Mode=0)
### Line Darkener And Thinner ###
FastLineDarkenMod(Strength=20,Prot=6,Thinning=0)
aWarpSharp2(Blur=4,Type=1,Depth=3,Chroma=2)
The video from Season 1 through most of Season 7 is like this: S2.E1-TestClip, and the video from the end of Season 7 through Season 10 is like this: S8.E1-TestClip. I'm encoding with 10-bit x264 with --preset veryslow --tune animation --crf 13.0 --deblock 2:1 --ref 12 --rc-lookahead 120 --merange 32 --no-fast-pskip --stitchable, looking to the future when I hopefully have a proper large screen TV instead of my current 14" laptop monitor . With my i5-3320M, 8GB RAM setup I'm achieving ~2.4 Fps. I've some questions before I attack:

1) Is there any reason quality-wise to use DGIndex-M2v instead of FFMSIndex-Mkv? I'm using FFMSIndex because it allows me to utilize AviSynth 2.6 MT; however, I'd switch to DGIndex (and run simultaneous encodes for more production) if it would output higher quality video.

2) Speed not being the main issue is there any way to streamline or amend the above script, any redundancies to rectify, any fat to trim? I'm able to run the script and understand the gist of it, but the finer details are still beyond me (someday ).

3) Looking to that large screen TV, are the x264 settings reasonable or are they overkill? Can they be improved?

I can't think of anything else, but I'm sure I will. Thank you for your time, all well-intentioned feedback is much appreciated.

Last edited by LouieChuckyMerry; 17th June 2015 at 10:29. Reason: Clarity
LouieChuckyMerry is offline   Reply With Quote
Old 17th June 2015, 11:12   #2  |  Link
kuchikirukia
Registered User
 
Join Date: Oct 2014
Posts: 396
What's RatioResize?

Good chance all that's going to be using more than 256MB.

And you should probably put the high color depth stuff last and Dither_out() to x264 10 bit.
kuchikirukia is offline   Reply With Quote
Old 17th June 2015, 13:29   #3  |  Link
videoh
Registered User
 
Join Date: Jul 2014
Posts: 475
Quote:
Originally Posted by LouieChuckyMerry View Post
1) Is there any reason quality-wise to use DGIndex-M2v instead of FFMSIndex-Mkv? I'm using FFMSIndex because it allows me to utilize AviSynth 2.6 MT; however, I'd switch to DGIndex (and run simultaneous encodes for more production) if it would output higher quality video.
No. The only possible difference would be due to different IDCT algorithms but that would be insignificant, or a difference in random access handling, but your script appears to be linear.
videoh is offline   Reply With Quote
Old 17th June 2015, 13:55   #4  |  Link
LouieChuckyMerry
Registered User
 
LouieChuckyMerry's Avatar
 
Join Date: Feb 2014
Posts: 230
Quote:
Originally Posted by kuchikirukia View Post
What's RatioResize?

RatioResize


Quote:
Originally Posted by kuchikirukia View Post
Good chance all that's going to be using more than 256MB.
I spent a week testing with AVSMeter (thanks Groucho2004!) and those MT settings are batch-stable and give me the highest fps.

Quote:
Originally Posted by kuchikirukia View Post
And you should probably put the high color depth stuff last and Dither_out() to x264 10 bit.
Please, would you explain that in more detail? I'm a primary school student when it comes to AviSynth . Also, I just decided to go with the x264 10-bit, after receiving all the previous help on the script, so this is something new to me.

Quote:
Originally Posted by videoh View Post
No. The only possible difference would be due to different IDCT algorithms but that would be insignificant, or a difference in random access handling, but your script appears to be linear.
Thank you .

Last edited by LouieChuckyMerry; 17th June 2015 at 13:59.
LouieChuckyMerry is offline   Reply With Quote
Old 17th June 2015, 14:35   #5  |  Link
Groucho2004
 
Groucho2004's Avatar
 
Join Date: Mar 2006
Posts: 3,786
When you're doing this:
Quote:
Originally Posted by LouieChuckyMerry View Post
Code:
RatioResize(10/11.0,"PAR")
you probably also want to flag the video stream with "--sar 10:11".


You could probably remove a lot of stuff after TDecimate by just using SRestore() (unless you tried this already).

Since you are going to encode a lot of episodes I would avoid the AVS MT hassle and potential frustration and encode 2 - 4 (depending on the number of cores of your CPU) episodes simultaneously. Any decent "gui" should offer that feature. If not, use batch files.

Last edited by Groucho2004; 17th June 2015 at 14:44.
Groucho2004 is offline   Reply With Quote
Old 17th June 2015, 15:12   #6  |  Link
LouieChuckyMerry
Registered User
 
LouieChuckyMerry's Avatar
 
Join Date: Feb 2014
Posts: 230
Quote:
Originally Posted by Groucho2004 View Post
When you're doing this:

you probably also want to flag the video stream with "--sar 10:11".
Thanks, does it matter where in line I add "--sar 10:11"?

Quote:
Originally Posted by Groucho2004 View Post
You could probably remove a lot of stuff after TDecimate by just using SRestore() (unless you tried this already).
I've not tried SRestore, but have it available. Would it replace Vinverse and Stab? Please, could you suggest a usage given the situation?

Quote:
Originally Posted by Groucho2004 View Post
Since you are going to encode a lot of episodes I would avoid the AVS MT hassle and potential frustration and encode 2 - 4 (depending on the number of cores of your CPU) episodes simultaneously. Any decent "gui" should offer that feature. If not, use batch files.
Actually, figuring out the settings for AviSynth 2.6 MT has been the easiest part of this project thus far, thanks to AVSMeter . Really, I can queue episodes for days in MeGUI without any issues and at a higher rate of production than simultaneously running episodes. Now if you'd only come up with an app to help me test and improve scripts...
LouieChuckyMerry is offline   Reply With Quote
Old 17th June 2015, 15:27   #7  |  Link
Groucho2004
 
Groucho2004's Avatar
 
Join Date: Mar 2006
Posts: 3,786
Quote:
Originally Posted by LouieChuckyMerry View Post
Thanks, does it matter where in line I add "--sar 10:11"?
No.

Quote:
Originally Posted by LouieChuckyMerry View Post
I've not tried SRestore, but have it available. Would it replace Vinverse and Stab? Please, could you suggest a usage given the situation?
I have almost zero experience with these, I just know that SRestore has been used successfully by others to cure nasty field blending. Search the forum.


Quote:
Originally Posted by LouieChuckyMerry View Post
Actually, figuring out the settings for AviSynth 2.6 MT has been the easiest part of this project thus far
Having such a complex process run and not crash is one thing. I stopped using AVSMT for "serious" work when I discovered random artefacts at various places in a BD backup. The script only had MDegrain + LSFMod, I used AVSMT to speed it up. I don't know if it was a problem with AVSMT or one of the filters, point is that I only discovered this months later when I watched the backup. Nowadays MT may be more reliable but it's something you have to consider.
Groucho2004 is offline   Reply With Quote
Old 17th June 2015, 22:08   #8  |  Link
kuchikirukia
Registered User
 
Join Date: Oct 2014
Posts: 396
Quote:
Originally Posted by LouieChuckyMerry View Post
Please, would you explain that in more detail? I'm a primary school student when it comes to AviSynth . Also, I just decided to go with the x264 10-bit, after receiving all the previous help on the script, so this is something new to me.
The problem with 8 bit is it doesn't have enough colors to avoid banding. So to avoid obvious banding in 8 bit requires some magic -- dithering. So if you deband and then want to encode 8 bit you want to dither.
10 bit doesn't need dithering.
You are converting it to 16 bit color, debanding, using LSB_out to tell gradfun3 not to dither and keep it 16 bit, but then using DitherPost which dithers and converts it to 8 bit... before sending it off to x264 10 bit. Just do your high bit depth color processing last and send it out in 16 bit color with dither_out() and tell x264 what's going on.
Add these to your x264 command line with appropriate resolution and FPS:
Code:
--demuxer raw --input-depth 16 --input-res 1280x720 --fps 59.97
kuchikirukia is offline   Reply With Quote
Old 17th June 2015, 22:12   #9  |  Link
creaothceann
Registered User
 
Join Date: Jul 2010
Location: Germany
Posts: 360
But... is there going to be banding in the Simpsons?
creaothceann is offline   Reply With Quote
Old 18th June 2015, 04:09   #10  |  Link
LouieChuckyMerry
Registered User
 
LouieChuckyMerry's Avatar
 
Join Date: Feb 2014
Posts: 230
Quote:
Originally Posted by Groucho2004 View Post
No.
Thanks, I added it.

Quote:
Originally Posted by Groucho2004 View Post
I have almost zero experience with these, I just know that SRestore has been used successfully by others to cure nasty field blending. Search the forum.
OK, I'll do some research on SRestore.

Quote:
Originally Posted by Groucho2004 View Post
Having such a complex process run and not crash is one thing. I stopped using AVSMT for "serious" work when I discovered random artefacts at various places in a BD backup. The script only had MDegrain + LSFMod, I used AVSMT to speed it up. I don't know if it was a problem with AVSMT or one of the filters, point is that I only discovered this months later when I watched the backup. Nowadays MT may be more reliable but it's something you have to consider.
Ahhh, thanks for that. I never would've considered this.

Quote:
Originally Posted by kuchikirukia View Post
The problem with 8 bit is it doesn't have enough colors to avoid banding. So to avoid obvious banding in 8 bit requires some magic -- dithering. So if you deband and then want to encode 8 bit you want to dither.
10 bit doesn't need dithering.
You are converting it to 16 bit color, debanding, using LSB_out to tell gradfun3 not to dither and keep it 16 bit, but then using DitherPost which dithers and converts it to 8 bit... before sending it off to x264 10 bit. Just do your high bit depth color processing last and send it out in 16 bit color with dither_out() and tell x264 what's going on.
Add these to your x264 command line with appropriate resolution and FPS:
Code:
--demuxer raw --input-depth 16 --input-res 1280x720 --fps 59.97
If 10-bit x264 doesn't need dithering, then should I then eliminate the entire "### Debanding ###" section (the GradFun3 and DitherPost lines) in the script? And which lines are the "high bit depth color processing"? Sorry for my ignorance. The source resolution is 720x480 and original frame rate is 29.970, so would I add:

Code:
--demuxer raw --input-depth 16 --input-res 720x480 --fps 29.970
to the x264 command line? Thanks.

Last edited by LouieChuckyMerry; 18th June 2015 at 06:08. Reason: Spelling
LouieChuckyMerry is offline   Reply With Quote
Old 18th June 2015, 04:50   #11  |  Link
kuchikirukia
Registered User
 
Join Date: Oct 2014
Posts: 396
Quote:
Originally Posted by LouieChuckyMerry View Post
If 10-bit x264 doesn't need dithering, then should I then eliminate the entire "### Debanding ###" section (the GradFun3 and DitherPost lines) in the script? And which lines are the "high bit depth color porcessing"? Sorry for my ignorance. The source resolution is 720x480 and original frame rate is 29.970, so would I add:

Code:
--demuxer raw --input-depth 16 --input-res 720x480 --fps 29.970
to the x264 command line? Thanks.
If it has banding you should deband. Gradfun3 is a debander, the "adding dither" part is only needed for 8 bit.
I don't exactly know where you're converting to 16 bit, though a quick look says it's probably SMDegrain. So you'd end with SMDegrain --> Gradfun3 --> Dither_out()

I wouldn't resize back to 720x480 after cropping. There's really no need. Just set the appropriate DAR. Though I don't know how to do that with 16 bit out. I don't know if the MeGUI lines get passed or not. Try it and see.
Oh, and it should be 23.97fps after IVTC, no?

Last edited by kuchikirukia; 18th June 2015 at 04:53.
kuchikirukia is offline   Reply With Quote
Old 18th June 2015, 06:33   #12  |  Link
LouieChuckyMerry
Registered User
 
LouieChuckyMerry's Avatar
 
Join Date: Feb 2014
Posts: 230
Quote:
Originally Posted by kuchikirukia View Post
I wouldn't resize back to 720x480 after cropping. There's really no need. Just set the appropriate DAR. Though I don't know how to do that with 16 bit out. I don't know if the MeGUI lines get passed or not. Try it and see.
Oh, and it should be 23.97fps after IVTC, no?
Ahhh, my mistake, I took "--input-res" to mean the source's resolution (and fps). So, I've added:

Code:
--demuxer raw --input-depth 16 --input-res 640x480 --fps 23.976
to the command line. Is this correct?

Quote:
Originally Posted by kuchikirukia View Post
If it has banding you should deband. Gradfun3 is a debander, the "adding dither" part is only needed for 8 bit.
I don't exactly know where you're converting to 16 bit, though a quick look says it's probably SMDegrain. So you'd end with SMDegrain --> Gradfun3 --> Dither_out()
It indeed has banding and you're right, it's SMDegrain that's converting to 16-bit (Lsb=True) then sending this (Lsb_Out=True) to GradFun3 (Lsb_In=True). So, with your dither fix would the end of the script change to:

Code:
### Overall Temporal Denoise ###
SMDegrain(tr=2,thSAD=600,ContraSharp=True,RefineMotion=True,Plane=0,Lsb=True,Lsb_Out=True,PreFilter=2)
### Debanding ###
GradFun3(thR=0.55,Radius=12,Mask=2,SMode=1,Lsb=True,Lsb_In=True, StaticNoise=True,Y=3,U=3,V=3)
Dither_Out()
### Line Darkener And Thinner ###
FastLineDarkenMod(Strength=20,Prot=6,Thinning=0)
aWarpSharp2(Blur=4,Type=1,Depth=3,Chroma=2)
from the original "DitherPost(Stacked=True,Prot=False,Mode=0)"?

Edit: the above change caused serious flashbacks with MeGUI's preview window, with stacked frames and odd color spaces. Removing "Lsb=True" from GradFun3 eliminated the stacked frames but the color space was still trippy; removing the "Lsb=True" line from GradFun3 and deleting the entire "Dither_Out() resulted in normal looking output, however.

EditEdit: seems I typed too soon. The preview window in MeGUI shows normal video with the proper aspect ratio, but the output of a short test clip produces a file with the proper overall aspect ratio but four copies of the video, with the wrong color space, in a grid. Something to do with the command line additions, maybe?

Last edited by LouieChuckyMerry; 18th June 2015 at 08:44.
LouieChuckyMerry is offline   Reply With Quote
Old 18th June 2015, 16:08   #13  |  Link
kuchikirukia
Registered User
 
Join Date: Oct 2014
Posts: 396
Put them last.
(edit: fixed to address the 4-in-one problem below)
Code:
# Set DAR in encoder
global MeGUI_darx = 628
global MeGUI_dary = 480
SetMemoryMax(256)
LoadPlugin("F:\[0]StandAloneApps\MeGUI-2500(core)2443(data)0.3.5(libs)[Portable]\tools\ffms\ffms2.dll")
FFVideoSource("SourcePath", fpsnum=30000, fpsden=1001, threads=1)
### Deinterlace ###
LoadPlugin("F:\[0]StandAloneApps\MeGUI-2500(core)2443(data)0.3.5(libs)[Portable]\tools\avisynth_plugin\TIVTC.dll")
TFM(Order=-1,Slow=2,PP=0).TDecimate(Mode=1)
Vinverse()
### Deshaker ###
Stab(Mirror=15)
### Crop ###
Crop(8,0,-8,0)
### Resize ###
### Gibbs Noise Block ###
Edge=MT_Edge("prewitt",thY1=20,thY2=40).RemoveGrain(17)
Mask=MT_Logic(Edge.MT_Expand().MT_Expand().MT_Expand().MT_Expand(),Edge.MT_Inflate().MT_Inpand(),"xor")
MT_Merge(dfttest(),Mask,Luma=True)
### Line Darkener And Thinner ###
FastLineDarkenMod(Strength=20,Prot=6,Thinning=0)
aWarpSharp2(Blur=4,Type=1,Depth=3,Chroma=2)
### Overall Temporal Denoise ###
SMDegrain(tr=2,thSAD=600,ContraSharp=True,RefineMotion=True,Plane=0,Lsb=True,Lsb_Out=True,PreFilter=2)
### Debanding ###
GradFun3(thR=0.55,Radius=12,Mask=2,SMode=1,Lsb=True,Lsb_In=True, StaticNoise=True,Y=3,U=3,V=3)
Dither_out()
Code:
--demuxer raw --input-depth 16 --input-res 704x480 --fps 23.976
MeGUI preview will be borked since it doesn't know what to do with 16 bit. Use DitherPost to convert to 8 bit if you want to use the preview.

Last edited by kuchikirukia; 19th June 2015 at 21:13.
kuchikirukia is offline   Reply With Quote
Old 19th June 2015, 04:58   #14  |  Link
LouieChuckyMerry
Registered User
 
LouieChuckyMerry's Avatar
 
Join Date: Feb 2014
Posts: 230
Thanks for your reply, kuchikirukia. Running a test clip with the suggested script and the suggested command line addition resulted in an output video suffering from four-pictures-in-a-grid-with-trippy-color-space. However, running a test clip with the suggested scrpt but no command line additions resulted in a proper video output. So, it seems that something's wrong with the command line additions. Ahhh, Happy Friday :beer: .
LouieChuckyMerry is offline   Reply With Quote
Old 19th June 2015, 07:00   #15  |  Link
Scythe42
Registered User
 
Join Date: Jun 2009
Posts: 21
Quote:
Originally Posted by LouieChuckyMerry View Post
I'm in the process of encoding my NTSC DVD's of the first ten seasons of the Simpsons
I am doing exactly the same as you right now. These DVDs are really horrible in quality.

That's what I use currently (still experimenting a bit over various disc and seasons before a long batch job):

First I load stuff. Loading from an remuxed MKV works nicely. No need at all for DGIndex, provides the same result for me. So I went with my selected and configured DirectShow filters here.
Code:
DirectShowSource("E:\Encode\S01E01.mkv",audio=false).ConvertToYV12()
There's a ton of dot crawl all over the scenes, hard to remove. Checkmate() did the best for me. Still some left but it's in an acceptable range now. This is clearly an analog tape transfer, and a bad one. Need to check if I can do better here with other filters or stack filters. Especially visible on the the credits.
Code:
Checkmate(thr=10, max=20, tthr2=0)
Next deinterlacing. These DVD have strange pulldown patterns, similar to Futurama. You can do various TFMs and TDecimate cycles, where you have to remove like every x frame in addition to a normal IVTC. You can also use AnimeIVTC, but didn't bother to try. The pattern changes now and then and I am not going through every frame to find some blended frames or adjust stuff per episode. So I went for an QTGMC/Strestore combo here (Yadif doesn't work very, worked better on Futurama then QTGMC). That takes care of nearly everything for me so far.

Code:
QTGMC(Preset="medium") #fast should work as well, going slower will produce blended artifacts.
SRestore(frate=23.976)
Next 4:3 resizing from the anamorph DVD encode to square pixels. Optional and could be done over x264.
Code:
Crop(12,0,-4,-0) # not ITU compliant, more on the left then on the right. 
Spline36Resize(640,480)
Certain scenes have Gibbs artifacts, I remove them the same way you do (we both probably found the same post when googling)
Code:
Edge=MT_Edge("prewitt", thY1=20, thY2=40).RemoveGrain(17)
Mask=MT_Logic(Edge.MT_Expand().MT_Expand().MT_Expand().MT_Expand(),Edge.MT_Inflate().MT_Inpand(),"xor")
MT_Merge(dfttest(), Mask, Luma=True)
There were still some stuff left around the edges and some halos now and then. Still a lot of noise and no clear edges. With that noise they cannot be really enhanced later on. So let's add some not so agressive stuff. Can always remove later if not not needed.
Code:
Edgecleaner(strength=50)
Dehalo_alpha(darkstr=0.5, brightstr=0.5, rx=1, ry=1)
Next, because there is so much noise and everything is very blocky. I went with deen() here as you really need to be aggressive to remove all the artifacts, blocking and other crap. And after the spatial smoothing I applied a temporal smoothing as well as the one in Deen doesn't do anything for me. Yes, I am very aggressive here.

The problem is that even after these filters there is noise in the corners left near the black lines and in small areas, that are smudgy on the DVD to begin with. So I stacked the smoothers and added an FFT3D on top. That finally removed all the crap.

Now the picture is of course too smooth. So I need to adjust the parameters probably a bit and see if I can get some details out around the edges, areas in general are fine now. Though some "smuge" is left. FFT3D cannot cleanup the picture alone, and neither can Deen. The nice thing is that these stacked combo also took care of the leftover of the Dot Crawl.
Code:
FFT3DFilter(sigma=3, bw=32, bh=32, ow=16, oh=16, plane=4)
Deen(mode="a2d", rad=4, thrY=7, thrUV=9, min=0.5)
FluxSmoothST()
Next up is line darkening. I tried various plugins and Toon gave me the best overall result, given that the lines were never sharp and are interrupted on the DVD to begin with.
Code:
Toon(strength=0.75)
Then as usual sharpening follows. I didn't overdo it or stuff I smoothed out will be visible again. Just trying to get the edges crisper again. But if you want more you could add a LimitedSharpenFaster() in addition to the warp sharpening as well or just for with it.
Code:
aWarpSharp2(blur=4, type=1, depth=11, chroma=2)
And because the smoothing could not remove all blocking artifacts and created some color banding and the sharpening makes them way more visible now, I use a deband filter to smooth these areas out . In some scenes it is not possible anymore as they are so bad in source that nothing can fix them.
Code:
GradFun3(smode=2)
Now the problem is that certain scenes have a wrong white balance, colors are general oversaturated or understaturated. Not really something you can solve automatically here. So, I live with it.

Yes, the edges could be a bit more defined and crisper and the picture is very smoothed. But even if you scale it up on a TV to 1080p it is still better then the cropped HD upscales that air for old episodes since some time. If not scaled up the result is more or less like the DVDs should have been in the first place.

Will work a bit on the smoothing to lower it. Maybe I'll find a sweet spot. Right now it's a little too much of course, since I stacked filters to get all the noise out.

If someone has a suggestion or sees a mistake pls. let me know.

And here are some screenshots on a frame with not too much noise on the DVD scaled up to 1080p in the player (so don't mind the jaggies), so you can easily see all the noise. As you see I need still some adjustment with the lines on which I have not yet focussed.
http://screenshotcomparison.com/comparison/131783

Edit: Update post

Last edited by Scythe42; 19th June 2015 at 22:11. Reason: Updated post. Added Screenshots
Scythe42 is offline   Reply With Quote
Old 19th June 2015, 21:12   #16  |  Link
kuchikirukia
Registered User
 
Join Date: Oct 2014
Posts: 396
Quote:
Originally Posted by LouieChuckyMerry View Post
Running a test clip with the suggested script and the suggested command line addition resulted in an output video suffering from four-pictures-in-a-grid-with-trippy-color-space. However, running a test clip with the suggested scrpt but no command line additions resulted in a proper video output. So, it seems that something's wrong with the command line additions. Ahhh, Happy Friday :beer: .
It was something wrong with my script. When I cut and pasted yours to change the order I left in the DitherPost instead of changing it to Dither_Out. So it was sending 8 bit to x264 which was now expecting 16 bit.

It's fixed now.

Last edited by kuchikirukia; 19th June 2015 at 21:14.
kuchikirukia is offline   Reply With Quote
Old 20th June 2015, 03:06   #17  |  Link
Scythe42
Registered User
 
Join Date: Jun 2009
Posts: 21
So, as the tests for first thin dark lines to get a sharper impression and later thicken them didn't really work out in general as it was too far away from the source. I made some changes:
  • switching from Toon to ToonLite as the thin lines really didn't work out and introduce too much aliasing if scale on a TV to 1080p.
  • Removal of additional filters I throw in that are not needed anymore. Little bit of unwanted noise remains in a few scenes, but not worth the effort.
  • Re-Ordered some filters
  • Using UnFilter for sharpening now. aWarpSharp created to much halos with the changes. PSharpen or LimitedSharpenFaster should work as well. Settled on this one for now.

Here is what is running now over the first seasons and then I check my results and see if something goes really wrong somewhere.

Picture is now a bit more smooth compared to before but closer to the original thicker blurry lines.

Code:
#Video Source
DirectShowSource("E:\Encode\S01E01.mkv",audio=false).ConvertToYV12()

#Dot Crawl removal
Checkmate(video, thr=10, max=20, tthr2=0)

#Deinterlace and Restore
QTGMC(Preset="Medium")
SRestore(frate=23.976)

#Crop and Resize
Crop(12,0,-4,-0)
Spline36Resize(640,480)

#Gibbs Removal
Edge=MT_Edge("prewitt", thY1=20, thY2=40).RemoveGrain(17)
Mask=MT_Logic(Edge.MT_Expand().MT_Expand().MT_Expand().MT_Expand(),Edge.MT_Inflate().MT_Inpand(),"xor")
MT_Merge(dfttest(), Mask, Luma=True)

#General Noise Removal
FFT3DFilter(sigma=3, bw=32, bh=32, ow=16, oh=16, plane=4)

#Edge Cleaning as there is additional noise and halo leftovers
Edgecleaner(strength=20)

#Temporal Smoothing
FluxSmoothST()

#Main Smoothing
Deen(mode="a2d", rad=4, thrY=7, thrUV=9, min=0.5) 

#Darken Lines without sharpening to avoid thinning
ToonLite(strength=0.75)

#Sharpen
UnFilter(50, 50)

#Color Debanding to get better gradients on larged deblocked areas.
GradFun3(video, smode=2)
Scythe42 is offline   Reply With Quote
Old 20th June 2015, 09:55   #18  |  Link
LouieChuckyMerry
Registered User
 
LouieChuckyMerry's Avatar
 
Join Date: Feb 2014
Posts: 230
Quote:
Originally Posted by kuchikirukia View Post
It was something wrong with my script. When I cut and pasted yours to change the order I left in the DitherPost instead of changing it to Dither_Out. So it was sending 8 bit to x264 which was now expecting 16 bit.

It's fixed now.
Thanks for that. Running the script with the command line additions output a file with proper color space but a DAR of 0.683. Very thin, tall everything! Returning the RatioResize line and editing the command line output a file with proper color space and a proper DAR (although MediaInfo shows the aspect ratio as 1.212). Also, I cleaned up the MT settings in case someone finds this useful some day. The script is now:

Code:
# Set DAR in encoder to 6480 : 4739. The following line is for automatic signalling
global MeGUI_darx = 6480
global MeGUI_dary = 4739
SetMemoryMax(256)
SetMTMode(3,3)
LoadPlugin("F:\[0]StandAloneApps\MeGUI-2500(core)2443(data)0.3.5(libs)[Portable]\tools\ffms\ffms2.dll")
FFVideoSource("D:\Temp\zSimpTemp\S1.E1-[I-4028].mkv", fpsnum=30000, fpsden=1001, threads=1)
### Deinterlace ###
SetMTMode(5)
LoadPlugin("F:\[0]StandAloneApps\MeGUI-2500(core)2443(data)0.3.5(libs)[Portable]\tools\avisynth_plugin\TIVTC.dll")
TFM(Order=-1,Slow=2,PP=0).TDecimate(Mode=1)
SetMTMode(2)
Vinverse()
### Deshaker ###
Stab(Mirror=15)
### Crop ###
Crop(8,0,-8,0)
### Resize ###
RatioResize(10/11.0,"PAR")
### Gibbs Noise Block ###
Edge=MT_Edge("prewitt",thY1=20,thY2=40).RemoveGrain(17)
Mask=MT_Logic(Edge.MT_Expand().MT_Expand().MT_Expand().MT_Expand(),Edge.MT_Inflate().MT_Inpand(),"xor")
MT_Merge(dfttest(),Mask,Luma=True)
### Line Darkener And Thinner ###
FastLineDarkenMod(Strength=20,Prot=6,Thinning=0)
aWarpSharp2(Blur=4,Type=1,Depth=3,Chroma=2)
### Overall Temporal Denoise ###
SMDegrain(tr=2,thSAD=600,ContraSharp=True,RefineMotion=True,Plane=0,Lsb=True,Lsb_Out=True,PreFilter=2)
### Debanding ###
GradFun3(thR=0.55,Radius=12,Mask=2,SMode=1,Lsb=True,Lsb_In=True, StaticNoise=True,Y=3,U=3,V=3)
Dither_Out()
with the command line additions:

Code:
--demuxer raw --input-depth 16 --input-res 640x480 --fps 23.976 --sar 10:11
I think that I'll try to improve the deinterlacing section of the script, so it's due to change again.

Thanks for your kind help, kuchikirukia, and a couple more questions if you've time . I remember that the line darkening and thinning was last because that's the usual way (to my understanding). Do you think that this line is now pointless, given that it comes before the denoising-debanding? Or should it be edited or moved?

Also, any idea why the aspect ratio according to MediaInfo is 1.212? As I typed above, it displays properly so I reckon that it doesn't rally matter; I just find myself curious. Ahhh, and did I "fix" the 0.683 aspect ratio in a "correct" fashion, or is there a better way? Thanks.

Edit: it seems that these lines from MeGUI's "Clever (TM) anamorphic encoding:" setting:

Code:
global MeGUI_darx = 6480
global MeGUI_dary = 4739
were buggering the works. With them removed all is well.

Scythe42: you may want to reconsider using QTGMC for deinterlacing. When I began this project (seemingly forever ago, ha ha), I asked some questions on the QTGMC thread at Doom9 and was informed that QTGMC is not recommended for animation. Read starting here for details.

Also, I take little credit for my script but the results are really good (to my eyes, at least). Start reading here for more information (my Gibb's noise line was born there).

Ahhh, thanks for the idea to change my cropping from 8,0,-8,0 to 12,0,-4,0; you're right, the left side is consistently much worse than the right side .

Last edited by LouieChuckyMerry; 30th June 2015 at 06:01. Reason: Information, Information
LouieChuckyMerry is offline   Reply With Quote
Old 20th June 2015, 13:06   #19  |  Link
Scythe42
Registered User
 
Join Date: Jun 2009
Posts: 21
Quote:
Originally Posted by LouieChuckyMerry View Post
Scythe42: you may want to reconsider using QTGMC for deinterlacing. When I began this project (seemingly forever ago, ha ha), I asked some questions on the QTGMC thread at Doom9 and was informed that QTGMC is not recommended for animation.
Will read the thread, thanks for pointing that one out. I used it because the pattern of blended frames is so highly irregular and QTGMC/Srestore was the only way of getting rid of them quick and dirty and the result is actually quite good. But proper field matching as far as possible should be nicer for this kind of source.

Will dive into the fields again and see if I can find an pattern beside a standard IVTC that deals with these frames without making pans choppy by replacing too much or having too much duplicates.

Last edited by Scythe42; 21st June 2015 at 06:12.
Scythe42 is offline   Reply With Quote
Old 21st June 2015, 09:08   #20  |  Link
LouieChuckyMerry
Registered User
 
LouieChuckyMerry's Avatar
 
Join Date: Feb 2014
Posts: 230
Quote:
Originally Posted by Scythe42 View Post
Will read the thread, thanks for pointing that one out. I used it because the pattern of blended frames is so highly irregular and QTGMC/Srestore was the only way of getting rid of them so far - or the only one I knew of to deal with it.

Will dive into the fields again and see if I can find an overall pattern that beside a standard IVTC deals with these frames without making pans choppy by replacing too much. Well I'll give it a shot. Not using QTGMC would speed up my encodes a lot.
I know what you mean about the poor quality of the source video. I was actually into Season 9 when I decided to upgrade to 10-bit x264, so I figured I might as well try to improve my script before starting over. The first four seasons went well using TIVTC with the settings in the above posted script, but starting with Season 5 the source material began suffering from occasional bits of field blending and other irregularities. Eventually I posted here and received some good suggestions, but there are some source episodes in seasons 5,6,7,and 8 so bad that they require a frame-by-frame inspection then the use of ReplaceFrameNext or ReplaceFramePrev which, unfortunately, isn't automated (you have to find the frame numbers yourself). If you ever figure out the pattern, please let me know .
LouieChuckyMerry is offline   Reply With Quote
Reply

Thread Tools Search this Thread
Search this Thread:

Advanced Search
Display Modes

Posting Rules
You may not post new threads
You may not post replies
You may not post attachments
You may not edit your posts

BB code is On
Smilies are On
[IMG] code is On
HTML code is Off

Forum Jump


All times are GMT +1. The time now is 05:24.


Powered by vBulletin® Version 3.8.11
Copyright ©2000 - 2018, vBulletin Solutions Inc.