Welcome to Doom9's Forum, THE in-place to be for everyone interested in DVD conversion. Before you start posting please read the forum rules. By posting to this forum you agree to abide by the rules. |
25th January 2011, 09:55 | #161 | Link |
Registered User
Join Date: Apr 2002
Location: Germany
Posts: 5,389
|
Wow. That surely was lots of work. Respect!
Any thoughts about making the searchclip/processing better suited for mixed progressive/interlaced sources? * * * * * BTW, now that the script is over 1000 lines and comes closer to 100 kB script size, I can't resist to post this - just for fun: Code:
i = last b = i.bob(.0,.5) t = b.temporalsoften(1,255,255,25,2).merge(b,0.25) s1 = t.blur(1.4).blur(1.4).merge(t,0.25).msuper() s2 = b.msuper(levels=1) b.mdegrain1(s2,s1.manalyse(isb=true),s1.manalyse(isb=false)).merge(b,0.25) sharpen(.2,.8).repair(b,1)
__________________
- We´re at the beginning of the end of mankind´s childhood - My little flickr gallery. (Yes indeed, I do have hobbies other than digital video!) |
25th January 2011, 14:28 | #162 | Link | |
Registered User
Join Date: Jul 2010
Posts: 448
|
This version only started as a little experiment a few months ago, but then it kinda snowballed... 1000 lines and half of them are comments! And still to add YUY2 and chroma speed-ups
Quote:
Shall I add it...? Preset="Cute" ____ BTW: Just noticed that I've broken the "Draft" preset. That's not too important so I'll wait to see if there are any other issues before posting a fix. Last edited by -Vit-; 25th January 2011 at 19:39. |
|
25th January 2011, 15:54 | #163 | Link |
Pig on the wing
Join Date: Mar 2002
Location: Finland
Posts: 5,718
|
It would also be interesting to know whether there are good settings to use with fieldblended sources to get the good frames (output from SRestore, that is) as detailed as possible. The detection clip most likely needs to be treated in a different way from QTGMC.
Thanks for the new version
__________________
And if the band you're in starts playing different tunes I'll see you on the dark side of the Moon... |
25th January 2011, 19:09 | #165 | Link | |
Registered User
Join Date: Jul 2010
Posts: 448
|
Quote:
____ From some questions coming in I realize that despite that mass of detail earlier, I didn't properly address the practical issues of speed and memory. The new features require more memory and CPU time. They require you know how to run demanding scripts in avisynth. The defaults are tweaked for a particular balance of performance and quality, but you can tweak in either direction Speed Source-match performs 1 or 2 extra interpolations. So there are up to three interpolations per frame. The interpolators are determined by: EdiMode="XXX" : the main interpolation, always occurs - nothing new here MatchEdi="XXX" : Basic source match - for all source-match modes. Defaults to same as main interpolation, change if using a slow choice like "EEDI3+NNEDI3" as your main MatchEdi2="XXX" : Refined source match - SourceMatch=2,3. Defaults to high speed NNEDI3. Can use "Bob" here with little penalty Using "Bob" with SourceMatch=2,3 is a good performance boost for those modes: Code:
QTGMC( Preset="Slower", SourceMatch=3, MatchEdi2="Bob" ) MatchPreset="XXX" - typically "Fast" or quicker still MatchPreset2="XXX" - typically "Very Fast" or quicker ____ Memory + Multi-Threading Multiple NNEDI3s and complex QTGMC settings like noise bypass use memory. If you run out of memory you often get a crash. Use SetMemoryMax(XXX) at the start of the script to help. You might need higher values, or maybe lower, it depends on the source resolution, settings, SetMTMode use etc. - try 400,600,700,800,1000,1200... Using multi-threading (SetMTMode) will help speed things up, but if you're on 32-bit avisynth you have a good chance of running out of memory if you go for very high settings, even on SD material, certainly on HD material. Again use SetMemoryMax, limit the number of threads in SetMTMode, and also use the EdiThreads setting of QTGMC - this last point is very important for source-match. 64-bit avisynth makes things easier - you still need to consider the same points, but you will have more latitude (depending on your machine). An example multi-threading tweak to limit threads and allow a bit more memory, this is just an example, each case is different: Code:
SetMemoryMax(700) SetMTMode(5,4) # 4 threads for avisynth WhateverSource(...) SetMTMode(2) # Three interpolations when using SourceMatch=2,3. Two defaulting to NNEDI3 here, plus a Bob. Set 2 threads for *each* NNEDI3 QTGMC( Preset="Slower", SourceMatch=3, Lossless=2, MatchEdi2="Bob", EdiThreads=2 ) Last edited by -Vit-; 25th January 2011 at 19:36. |
|
25th January 2011, 22:04 | #166 | Link |
Registered User
Join Date: Apr 2009
Location: Martin, Slovakia
Posts: 79
|
I'm trying to figure out how to use "ShowNoise". Whatever parameters I use, I get only plain grey picture, no detail, no noise. What am I doing wrong?
Code:
# Set DAR in encoder to 31 : 17. The following line is for automatic signalling global MeGUI_darx = 31 global MeGUI_dary = 17 LoadPlugin("D:\Install\meGUIx64\tools\dgindex\DGDecode.dll") DGDecode_mpeg2source("D:\Download\VTS_01_1.d2v", info=3) LoadPlugin("D:\Install\meGUIx64\tools\avisynth_plugin\ColorMatrix.dll") ColorMatrix(hints=true, interlaced=true, threads=0) QTGMC( Preset="Slow", ShowNoise=8.0, sigma=2) |
26th January 2011, 00:44 | #167 | Link |
Pig on the wing
Join Date: Mar 2002
Location: Finland
Posts: 5,718
|
Would you like to have a sample to look at? The problem with fieldblended material IMO is that the motion is not "real motion". The underlying source is basically always progressive.
__________________
And if the band you're in starts playing different tunes I'll see you on the dark side of the Moon... |
26th January 2011, 01:46 | #168 | Link |
Registered User
Join Date: Jul 2010
Posts: 448
|
You need to enable noise bypass for this to work:
Code:
# I suggest you explicitly set NoiseDeint - it defaults to "DoubleWeave", "Generate" is best, you can also set "Bob" QTGMC( Preset="Slow", NoiseBypass=1, ShowNoise=8.0, sigma=2, NoiseDeint="Generate" ) ____ It would be good to have a typical example of the problem from someone who deals with it. Still I don't expect to be able to look at this issue for some time... Last edited by -Vit-; 26th January 2011 at 01:50. |
26th January 2011, 04:30 | #169 | Link |
Registered User
Join Date: Apr 2009
Location: Martin, Slovakia
Posts: 79
|
Heh I was assuming that it can set up all needed parameters automatically. Thanks.
Now I have another problem. I was trying to find appropriate sigma but I have found that I need to denoise also chroma planes to get good results. How it is with "plane=4" support? Are there needed any other changes in your script to get "plane=4" support? |
26th January 2011, 04:54 | #170 | Link | |
Registered User
Join Date: Jul 2010
Posts: 448
|
Quote:
Last edited by -Vit-; 26th January 2011 at 04:58. |
|
26th January 2011, 19:31 | #172 | Link | |
Registered User
Join Date: Nov 2009
Posts: 110
|
Quote:
I use something like this for converting NTSC Torchwood/Dr Who/Robin Hood/etc DVDs: Code:
setmtmode(5,0) loadcplugin("C:\program files\avisynth 2.5\plugins\yadif.dll") o = MPEG2Source("Torchwood.S1E01-clip.d2v",cpu=3) setmtmode(2) y = o.yadif(mode=1,order=1) qtgmc = o.QTGMC(Preset="Ultra Fast",rep2=4,SLmode=1,SLrad=1) qtgmc.Srestore(frate=25,dclip=y) |
|
26th January 2011, 20:05 | #173 | Link |
Registered User
Join Date: Nov 2007
Posts: 114
|
I'm currently trying to repair some material that was sloppily converted from interlaced to progressive using what seems to be weave deinterlacing (lots of combing artifacts).
So far, I'm using this: QTGMC( Preset="Slower", InputType=2, ProgSADMask = 10.0, SourceMatch=3, Sharpness=0.4, TR2=3 ) This makes substantial repairs, but there are still some combing artifacts in few frames. Are there any other variables that I should experiment with? I've read the documentation but embarrasingly most of it is just way over my head. |
26th January 2011, 21:20 | #174 | Link | |
Registered User
Join Date: Jul 2010
Posts: 448
|
Quote:
InputType=2 tries to rebuild a shimmer-free result from the even lines (fields), InputType=3 does the same using the odd lines. Each loses detail from the other lines. The following script runs both modes. It means some redundant processing, but it does improve detail and reduce artefacts: Code:
c = WhateverSource("Your.src") # Two versions of fixed source (use TR2 and ProgSADMask also if you wish) t = c.QTGMC( Preset="Slower", InputType=2, SourceMatch=3, Sharpness=0.4 ) b = c.QTGMC( Preset="Slower", InputType=3, SourceMatch=3, Sharpness=0.4 ) # Now merge the results in some way. Choose ONLY ONE of the lines below and delete the other Merge( t, b ) # Smooth result, may need more sharpening t.Repair( Merge( t, b ), 1 ) # Sharper result, could try 2 at end of this line One of the minor fixes I'm sitting on at the moment in version 3.01 is allowing Lossless modes for InputType=2,3. That helps with detail, but I guess that's not your problem. ____ You can do some serious script abuse by chaining together progressive QTGMCs. Generally it's not a good idea because the repeated motion compensations are both redundant and drift in accuracy which causes smearing. You also get gradual over-sharpening. [Although sensible uses like the one above makes me think a toolbox approach might be beneficial] So another approach for your situation, that I don't recommend but you might want to try, is to run a QTGMC( InputType=1 ) on the result of your QTGMC( InputType=2 ). It will certainly clean up the artefacts, but it may cause smearing / oversharpening: Code:
QTGMC( Preset="Slower", InputType=2, SourceMatch=3, Sharpness=0.4 ) # Plus any other settings QTGMC( Preset="Slower", InputType=1, Sharpness=0.1 ) # Don't use source-match here Last edited by -Vit-; 27th January 2011 at 00:23. |
|
27th January 2011, 01:43 | #175 | Link |
Registered User
Join Date: Jan 2011
Posts: 6
|
Excuse me for the n00b question but, do I loose any vertical resolution if I decide to keep the framerate of the interlaced source? In example, I have a 1080i60 (60 fields/s | 59.94 frames/s playback) bluray source, and I want to encode it to 1080p60 (59.94 frames/s). In my experiments I haven't seen any noticeable decrease in quality after deinterlacing, but I just wanted to know of the side-effects.
Thank you in advance. |
27th January 2011, 01:51 | #176 | Link |
Registered User
Join Date: Feb 2009
Location: USA
Posts: 676
|
one field is thrown away, and the data from the other is used /interpolated to keep the same resolution, if I recall correctly. That's how it usually works. What you see is what you get basically... So if you are previewing and choose same rate deinterlacing, and you see the same image size as the source, then that is what you get
|
27th January 2011, 02:39 | #177 | Link |
Registered User
Join Date: Jul 2010
Posts: 448
|
No, you won't really lose any resolution. Deinterlacing effectively increases vertical resolution.
Put simply: your 1080i60 source is really a 60fps stream of 540 pixel high images (fields). Each pair of fields is interlaced (weaved) into a single 1080 pixel high frame, resulting in a 30fps stream of interlaced frames. Deinterlacing separates the fields again and doubles their height to give a 60fps stream of true 1080 pixel high frames. Compare with the starting point: the vertical resolution is increased. The new pixels are created by intelligent interpolation of other pixels in the same field and pixels in neighboring fields. No fields are thrown away during this process. If you decide to drop half the frames after deinterlacing to keep the original 30fps rate, that's fine - no quality is lost on those frames you keep. The original fields in your source are sometimes altered a little in the deinterlaced output to ensure that the result is smooth and shimmer-free. But there should be very little detail loss (especially with the latest features). In fact, the newest developments mean the doubled height output can make a good estimate of lines that are not actually stored in the source. That's what the Source_Comparison example was showing in my earlier post. This compares a source recorded with 720 pixels of vertical resolution against a QTGMC deinterlacing of 360 pixel high fields (i.e. I deliberately threw away half the pixels and got QTGMC to try and work out what they were) Last edited by -Vit-; 27th January 2011 at 02:49. |
27th January 2011, 02:56 | #178 | Link | |
Registered User
Join Date: Jan 2011
Posts: 6
|
Thank you for the input guys. Much appreciated.
Quote:
|
|
27th January 2011, 03:28 | #179 | Link |
Registered User
Join Date: Jan 2011
Posts: 6
|
Hey -Vit-, I just installed the new 3.0 version. I am still playing around with it, but one thing I noticed was that when I set SourceMatch=3 the CPU utilization drops significantly. Without it I'm getting ~98% utilization, with SourceMatch I'm getting ~70%. Am I doing something wrong or is that expected with the new enchantments?
|
27th January 2011, 04:00 | #180 | Link |
Registered User
Join Date: Jul 2010
Posts: 448
|
CPU under-utilization often means that there are cores idle on a multi-core machine. E.g. One core is busy running a key step in the script, another core is trying to run a task that needs the result - if there are no other tasks to do then the second core idles (it's rather more intricate than that, but you get the idea). Run more threads, give your cores more to do. Run multi-threaded using SetMTMode, or increase the thread count if you're using it already. I posted some tips on that a few posts ago in this thread. Also NNEDI3 runs two or three times with source-match and NNEDI3 defaults to running multi-threaded. You can tweak its thread usage with EdiThreads=x. Try tweaking x, it might help (or not).
Last edited by -Vit-; 28th January 2011 at 20:53. |
Thread Tools | Search this Thread |
Display Modes | |
|
|