Welcome to Doom9's Forum, THE in-place to be for everyone interested in DVD conversion.

Before you start posting please read the forum rules. By posting to this forum you agree to abide by the rules.

 

Go Back   Doom9's Forum > Capturing and Editing Video > Avisynth Usage

Reply
 
Thread Tools Search this Thread Display Modes
Old 29th May 2013, 18:45   #21  |  Link
Gavino
Avisynth language lover
 
Join Date: Dec 2007
Location: Spain
Posts: 3,431
That's strange, since the change you made makes no difference to the order in which the functions are actually called internally by Avisynth.

Perhaps the underlying problem is really one of random memory corruption, so changing the script slightly affects whether it works or not.

Incidentally, have you tried it without loading avisynth_c.dll at all (and deleting it from the plugins folder)? (I don't know enough about SmartDecimate to know whether or not it should work just using the built-in LoadCPlugin().)
__________________
GScript and GRunT - complex Avisynth scripting made easier
Gavino is offline   Reply With Quote
Old 30th May 2013, 06:11   #22  |  Link
DoctorM
Registered User
 
Join Date: Sep 2005
Posts: 558
No, SmartDecimate needs the author's own avisynth_c to work. It's a shame the filter is so dated since apparently it still has its uses.

As an aside, the audio pitch seems correct so I tried to time expand it from 25 to 23.976, but neglected to remember the actual framerate is 24.975.
So for future interest, the correct adjustment using BeSweet appears to be: -soundtouch( -tempo -4.0 )

Edit: While it might appear the source video was PAL poorly converted to NTSC, I'm more convinced this was some interesting "Time Compression" method used to make it fit a time slot.

Last edited by DoctorM; 30th May 2013 at 06:42.
DoctorM is offline   Reply With Quote
Old 31st May 2013, 04:10   #23  |  Link
DoctorM
Registered User
 
Join Date: Sep 2005
Posts: 558
Thanks to everyone that helped with this. The final video, while not fantastic is tolerably good.

I do see the occasional stutter from a dropped frame about once every 5 minutes. I suspect this is a rounding error from the frame rate becoming 24.975 instead of 25.
Considering it would be outrageous to do something like: TDecimate(Mode=0,Cycle=2997,CycleR=497) to get it bang on, it'll do.
DoctorM is offline   Reply With Quote
Old 31st May 2013, 04:18   #24  |  Link
ChiDragon
Registered User
 
ChiDragon's Avatar
 
Join Date: Sep 2005
Location: Vancouver
Posts: 600
Source looks like VHS. What is the lineage of this sample?

I would need several minutes of the source to provide better advice, but I have solved this clip: WeaveMan result (XviD)

Quote:
Originally Posted by DoctorM View Post
It would be quicker and produce better quality to IVTC directly, but I know of no filter that is tunable to match fields for frame rates besides 29.97.
It never hurts to be familiar with the brute-force method of standard IVTC: http://avisynth.org/mediawiki/ShowFiveVersions

If there were no dropped fields and you had infinite time and patience, some combination of DoubleWeave().Pulldown(x,y) would remove the pulldown pattern. But here's how I used to use my WeaveMan plugin.

Code:
MPEG2Source("Iclip.d2v")

TFM(mode=0,slow=2,micmatching=0,PP=1,output="Iclip.TFM.txt") # Run a video analysis pass in VDub
Code:
MPEG2Source("Iclip.d2v")

#TFM(mode=0,slow=2,micmatching=0,PP=1,output="Iclip.TFM.txt")
WeaveMan(order=1,input="Iclip.TFM.txt",output="Iclip.WMout.txt") # Just open and close the script to create the output text file
My plugin apparently has a bug with the final group of stats but here is the resulting text file up to the portion that is garbage.

Code:
3 cc-pcXpcc-pN = 3 in 16
19 ppN = 1 in 7
26 ccc-XXpN = 2 in 12
38 ccNN = 2 in 12
50 pXNN = 2 in 12
62 ppN = 1 in 7
69 ccRN = 2 in 12
81 pcNN = 2 in 12
93 cXpcc-pN = 2 in 12
105 ppN = 1 in 7
112 ccR = 1 in 7
We can ignore the first line, and try to group the others.

Code:
19 ppN = 1 in 7
26 ccc-XXpN = 2 in 12
38 ccNN = 2 in 12
50 pXNN = 2 in 12

62 ppN = 1 in 7
69 ccRN = 2 in 12
81 pcNN = 2 in 12
93 cXpcc-pN = 2 in 12
To start with, focus only on the "X in Y" numbers. In terms of the number of duplicates determined to exist, we have found a repeating cycle: 7 in 43. For a frame rate, that gives us (43-7)/43*30000/1001 = 25.0912. Of course I have no clue if this actually holds up for the entirety of your source.

Here is a script showing what it would look like if you stop here rather than going for all-out manual control.

Code:
MPEG2Source("Iclip.d2v")

Bob() # For speed. Replace with a smart bobber that first attempts to match fields. One is: TDeint(mode=1,tryWeave=true,slow=2)
LanczosResize(960,720)
TDecimate(cycleR=25,cycle=43,display=true) # Frame 92 (source 220) is a dupe
I forget the math involved in why cycleR becomes 25 instead of 7 for a bobbed input... I just cheated and guessed until I found the matching frame rate.

For a manual pattern that perfectly undoes the cadence in this sample, you need to copy and paste one of the patterns from the WMout.txt file into a WMin.txt "ovr" file and play around with it. Your goal is to reduce it down to as many N's and c's as possible, using p, X, Y, R, etc. only when absolutely necessary. Eventually I came up with this:

Code:
-19,0 ppNccRNccNNcXNN
30 X                  # inframe 25 looks interlaced otherwise, due to an AGC freakout
"-19" just means that the pattern was determined using frame 19 as the starting point and we want it repeated all the way back to frame 0.

Last edited by ChiDragon; 31st May 2013 at 05:24. Reason: Updates
ChiDragon is offline   Reply With Quote
Old 31st May 2013, 06:32   #25  |  Link
ChiDragon
Registered User
 
ChiDragon's Avatar
 
Join Date: Sep 2005
Location: Vancouver
Posts: 600
Whoops, I should have responded to these posts first.

Quote:
Originally Posted by manono View Post
Anything that doesn't return 24.975fps is wrong.
As mentioned earlier, time compression means that isn't true. There are also real 25.000 -> 29.970 pulldowns.

Quote:
Originally Posted by johnmeyer View Post
There is something in the TFM logic that I don't quite understand in how it post-processes the individual fields that have no match. If you set "order=1" TFM will correctly post-process the unmatched odd fields. For these single fields (shown by the "1" in the pattern above) it will give you a frame that contains the original unmatchable field plus a synthesized field, created using whatever technique you choose using the "pp" parameter. However, it skips the even fields. If you instead set "order=0," it then does the even fields but skips the odd fields.
It's because the orphan exists in the top field in the first instance, and the bottom field in the second. The field matcher would need to compare to surrounding frames to know that in the second instance it should keep the bottom field and throw away the top.

(Disclaimer: there's probably something I'm missing since tritical is obviously considerably smarter than me and invested more time in this than I ever did.)

I've never understood why, but TFM does everything it possibly can to create a field match, even when the matches it generates very obviously cause extra duplicate frames. You have to dig deep into the documentation to disable some of the features that don't seem to serve any purpose (even theoretically) if smooth motion is a goal for the output and not just complete frames.

It defaults to mode 1 even though the documentation notes that it creates a jerky output, and from v0.9.10.0 the micmatching feature has to be disabled for mode 0 to always function how you would expect it to.

Code:
Plain 2-way matching (option 0) is the safest of all the options in the sense
that it wont risk creating jerkiness due to duplicate frames when possible,
but if there are bad edits or blended fields it will end up outputting combed
frames when a good match might actually exist.
In the case of a bad edit, this "good match" will either (1) be dropped by TDecimate at the edit point anyway or (2) stay as an extra duplicate. How can this ever be considered a good thing? What you actually want to do for what tritical describes as this 3rd match is interpolate the opposite-order field, since it contains temporal information that is thrown away by mode=1.

Last edited by ChiDragon; 31st May 2013 at 06:39.
ChiDragon is offline   Reply With Quote
Old 31st May 2013, 09:04   #26  |  Link
manono
Moderator
 
Join Date: Oct 2001
Location: Hawaii
Posts: 7,406
Quote:
Originally Posted by ChiDragon View Post
I would need several minutes of the source to provide better advice, but I have solved this clip
So did we, and a helluva lot more easily than you did, if the explanation in your post is any indication. My main problem was in getting SmartDecimate to run properly, not in figuring out the true framerate. All we had to work with was the sample and I agree that with the entire video maybe a better effort could have been made.

Quote:
Originally Posted by ChiDragon View Post
As mentioned earlier, time compression means that isn't true.
Having the 'real' framerate being either 24.975 or 25fps is too much of a coincidence for it to be time compression for a modern TV show. Time compressed videos have all kinds of weird-ass framerates. As I also think, the original source looks a lot like a VHS tape which pretty much rules out time compression I would think. But I'd also be curious as to where this thing came from. I don't think that ordinarily a TV network would use a VHS tape as a source for broadcast, and they'd be even less likely to then perform time compression on it. I've never heard of time compression being used for a DVD, if that's what this is, but there's always a first time, I suppose. But whatever fly-by-night company made this thing wouldn't even know what time compression was, much less purposely use it for no good reason at all. You're thinking the source before the supposed time compression was progressive 23.976fps hard telecined to 29.97fps?
Quote:
Originally Posted by ChiDragon View Post
There are also real 25.000 -> 29.970 pulldowns.
That may be, but not often and not here. At least the sample doesn't give any indication of that. And even if DoctorM thinks it should be 25fps based on having the complete video, he can make it so without any great difficulty. Which doesn't negate the value of using SmartDecimate to keep the bobbed orphaned fields in the final output.
manono is offline   Reply With Quote
Old 31st May 2013, 17:36   #27  |  Link
DoctorM
Registered User
 
Join Date: Sep 2005
Posts: 558
I can provide a larger clip if you'd all like to take another look. I don't recall anyone asking. How big a clip is ideal?

The source is an old VHS recording from TV. That's why I considered time compression to be a possibility.

Edit: ChiDragon: Yikes. I don't even have an opinion.
DoctorM is offline   Reply With Quote
Old 31st May 2013, 17:41   #28  |  Link
ChiDragon
Registered User
 
ChiDragon's Avatar
 
Join Date: Sep 2005
Location: Vancouver
Posts: 600
Quote:
Originally Posted by manono View Post
So did we, and a helluva lot more easily than you did, if the explanation in your post is any indication.
I thought your comment about a jump on frame 15 was with your SmartDecimate solution, but I see it was above the EDIT. In case you're wondering why I went to all the trouble, I had been looking for a test case to re-learn how to use the filter that I posted in Development recently. I feel that using it is faster than manually counting fields as johnmeyer did, but you may disagree.

Quote:
Having the 'real' framerate being either 24.975 or 25fps is too much of a coincidence for it to be time compression for a modern TV show.
He did say there is a stutter every ~5 minutes with that frame rate. Since I calculated the frame rate as 25.0912, the difference between that and 25 can't be seen with a clip shorter than 11 seconds.

Quote:
As I also think, the original source looks a lot like a VHS tape which pretty much rules out time compression I would think. But I'd also be curious as to where this thing came from.
After hanging around the originaltrilogy.com forums one learns that LaserDisc releases used time compression all over the place because of length-per-side issues. Similarly, a lot of retail VHS movies were time compressed since not every movie can fit on a 2-hour SP mode tape or justify two tapes.

Quote:
You're thinking the source before the supposed time compression was progressive 23.976fps hard telecined to 29.97fps?
My thinking was progressive with a speedup and then pulldown.

@DoctorM: It's not as complicated as it first appears, since most of the work is automated. How about a minute clip for now?

Last edited by ChiDragon; 31st May 2013 at 17:48.
ChiDragon is offline   Reply With Quote
Old 31st May 2013, 19:11   #29  |  Link
johnmeyer
Registered User
 
Join Date: Feb 2002
Location: California
Posts: 2,685
Chiman,

Thank you for your posts. You made me realize that my posts above were not as stupid as I was lead to believe.

I also thank you for your WeaveMan plugin. It may help me reduce the workflow in transferring movie film using my shutterless movie projector. I described that invention here:

High Speed Film to Video Transfer Machine -- Need IVTC Help

I currently use TFM in two-pass mode, as you describe above, but I manipulate the metrics inside of an Excel spreadsheet in order to identify the candidates for decimation. It is a pathetic kludge, and I've been meaning to create something like WeaveMan for a long time. I'm hoping I can adapt your work to my situation, so thank you very much for posting it.
johnmeyer is online now   Reply With Quote
Old 31st May 2013, 20:10   #30  |  Link
manono
Moderator
 
Join Date: Oct 2001
Location: Hawaii
Posts: 7,406
Quote:
Originally Posted by ChiDragon View Post
He did say there is a stutter every ~5 minutes with that frame rate.
Yes he did. But there's lots we don't know. If he used his modified script (modified from the one I proposed), perhaps there was too long of an interval before the cycle repeated sometimes and he got missing frames. If so, there should also be duplicate frames to compensate if it's 24.975fps.

And as you also know, the difference between 24.975 and 25fps is one frame about every 40 seconds, not every 5 minutes. This gets tricky, of course, because sometimes they may be hidden by static scenes. But even so, I'd suspect imperfect decimating over time compression here. If he's right about the frequency of missing frames at 24.975fps, if it's really 25fps not only would the missing frames be added back in, but a bunch of duplicate frames would be in there as well.
Quote:
Since I calculated the frame rate as 25.0912, the difference between that and 25 can't be seen with a clip shorter than 11 seconds.
Yes, a much longer sample might (and might not) help solve the dilemma. If it's really 25.0912fps then, of course, time compression is the answer. Except for the odd nature of that sample though, I see no evidence at all of that being the case. I chalk it up to an unusual kind of standards conversion.

He also just said it's a VHS tape of a TV show. So, if the tape is ... say ... 20 years old, do you think TV broadcasts used time compression back then?

Quote:
Originally Posted by johnmeyer View Post
Thank you for your posts. You made me realize that my posts above were not as stupid as I was lead to believe.
Nowhere did I say your post was stupid, just wrong. You presented results that included missing frames. Even you, in your qualifications, apparently realized your results weren't very good. If so, why post then?
manono is offline   Reply With Quote
Old 31st May 2013, 20:44   #31  |  Link
ChiDragon
Registered User
 
ChiDragon's Avatar
 
Join Date: Sep 2005
Location: Vancouver
Posts: 600
Quote:
Originally Posted by manono View Post
He also just said it's a VHS tape of a TV show. So, if the tape is ... say ... 20 years old, do you think TV broadcasts used time compression back then?
If it's a TV show then I agree with you, but we'll have to wait for him to chime in again since he didn't actually say it's a series. A movie shown on TBS in 1993 with time compression? I'd buy it.

15 years ago: 'Tristar decided to proceed over Apted's objection and hired an editor who made 270 separate cuts totaling 22 minutes, sped up the credits to steal two more minutes, and compressed the rest of the film electronically to gain another four.'

I found that by searching for the phrase "edited to run in the time allotted" which you would commonly see on TV airings of movies in the '90s. Of course they don't disclose in that short disclaimer whether the editing is trimming or speeding-up.
ChiDragon is offline   Reply With Quote
Old 31st May 2013, 20:59   #32  |  Link
johnmeyer
Registered User
 
Join Date: Feb 2002
Location: California
Posts: 2,685
Quote:
Originally Posted by manono View Post
Even you, in your qualifications, apparently realized your results weren't very good. If so, why post then?
Because I was trying to help! If perfection is a requirement for posting, then none of us should bother.
johnmeyer is online now   Reply With Quote
Old 31st May 2013, 22:18   #33  |  Link
manono
Moderator
 
Join Date: Oct 2001
Location: Hawaii
Posts: 7,406
Quote:
Originally Posted by ChiDragon View Post
If it's a TV show then I agree with you, but we'll have to wait for him to chime in again since he didn't actually say it's a series. A movie shown on TBS in 1993 with time compression?
My fault. I wasn't clear. By 'TV show' I meant anything shown on TV, including movies. Maybe the year the tape was made is important. I didn't know time compression was used beginning more than a few years back but apparently your link proves it's been around much longer than I had thought.
Quote:
...which you would commonly see on TV airings of movies in the '90s.
Right, I always assumed it meant cutting out sections of the movies. Maybe I was wrong and it might have meant more than that.
manono is offline   Reply With Quote
Old 1st June 2013, 16:36   #34  |  Link
DoctorM
Registered User
 
Join Date: Sep 2005
Posts: 558
It's quite correct, this is a VHS recording from a cable broadcast (Family Channel) from the 90's (not sure of the exact year).

I'll have to check the tape again, but I don't even remember there being a notice that it was time compressed or anything... but that doesn't mean it wasn't. I assume they meant to hit a target run time of 2 hours and still fit plenty of commercials.

As request, a 1+ minute clip.

Also here was my full script:
Code:
loadplugin("C:\AviSynth\plugins\dgdecode.dll")
import("C:\AviSynth\plugins\limitedsharpenfaster.avs")
loadcplugin("C:\AviSynth\plugins\yadif.dll")
loadplugin("C:\AviSynth\plugins\fft3dgpu.dll")
loadplugin("C:\AviSynth\plugins\tivtc.dll")
loadcplugin("C:\AviSynth\plugins\yadif.dll")
loadplugin("C:\AviSynth\plugins\avisynth_c.dll")
loadcplugin("C:\AviSynth\plugins\SmartDecimate.dll")

mpeg2source("Source2.d2v",cpu=1)

crop(4,0,-10,-6).addborders(4,0,10,6)
yada=yadif(mode=1)
SmartDecimate(30,60,bob=yada)
TDecimate(Mode=0,Cycle=12,CycleR=2)
assumefps(23.976)

crop(4,0,-10,-6)
spline36resize(716,480).addborders(2,0,2,0)

fft3dgpu(sigma=1.5)
limitedsharpenfaster().limitedsharpenfaster()
converttoyuy2()
DoctorM is offline   Reply With Quote
Old 1st June 2013, 18:22   #35  |  Link
ChiDragon
Registered User
 
ChiDragon's Avatar
 
Join Date: Sep 2005
Location: Vancouver
Posts: 600
For simplicity this is my quick test version of your script.

Code:
loadcplugin("C:\Program Files (x86)\AviSynth 2.5\plugins\SmartDecimate.dll")

MPEG2Source("RAW IRBR Clip2(3).d2v")

SmartDecimate(30,60)
TDecimate(Mode=0,Cycle=12,CycleR=2,display=true)
You can quickly see that these aren't the correct cycle/cycleR values, as every few cycles the layout of the uniques and dupes changes. Input frame 206 is dropped despite being unique, as there is only one dupe during that cycle. Same with 727, 1009, 1501, 2064 (probably more between those last two but it's a low-motion scene). 5 in 30 incorrectly drops 210, and 7 in 43 isn't correct either.

I don't suppose there is a full minute of big movements and camera pans anywhere...

EDIT: 21 in 129 (which is just three 43-frame cycles) gets closer but the pulldown pattern is actually hundreds of frames long.

Code:
1308 ppN = 1 in 7
1315 ccRN = 2 in 12
1327 ccN = 1 in 7
1334 pcc-ppXNN = 3 in 17
= 43

1351 ppN = 1 in 7
1358 ccR = 1 in 7
1365 pcc-pccNN = 3 in 17
1382 cXNN = 2 in 12
= 43

1394 ppN = 1 in 7
1401 ccR = 1 in 7
1408 pcc-pccNN = 3 in 17
1425 pXNN = 2 in 12
= 43

1437 ppN = 1 in 7
1444 ccRN = 2 in 12
1456 ccNN = 2 in 12
= 31

1468 cXN = 1 in 7
1475 pcc-pppN = 2 in 12
1487 pcc-c-ppcc-pccNN = 5 in 24
1511 ppNN = 2 in 12
= 55

1523 ppN = 1 in 7
1530 ccNN = 2 in 12
1542 ccNN = 2 in 12
1554 ppNN = 2 in 12
= 43

1566 ppN = 1 in 7
1573 ccRN = 2 in 12
1585 ccN = 1 in 7
1592 pcc-pcXNN = 3 in 17
= 43

1609 ppN = 1 in 7
1616 ccRN = 2 in 12
1628 ccNN = 2 in 12
= 31

1640 ppN = 1 in 7
1647 pcc-pppN = 2 in 12
1659 ccNN = 2 in 12
1671 cXNN = 2 in 12
= 43

1683 ppN = 1 in 7
1690 ccRN = 2 in 12
1702 ccNN = 2 in 12
1714 cXpcc-pN = 2 in 12
= 43

1726 ppN = 1 in 7
1733 ccRN = 2 in 12
1745 ccNN = 2 in 12
= 31

1757 ccN = 1 in 7
1764 pcc-pppN = 2 in 12
1776 ccRN = 2 in 12
1788 ccNN = 2 in 12
1800 cXNN = 2 in 12
= 55

1812 ppN = 1 in 7
1819 ccR = 1 in 7
1826 pcc-pccN = 2 in 12
1838 pcc-pcXNN = 3 in 17
= 43

1855 ppN = 1 in 7
1862 ccRN = 2 in 12
1874 ccNN = 2 in 12
1886 cXNN = 2 in 12
= 43
31+55 = 86 but there is that 31 in the middle of two 43's that throws a spanner into the mix. Now you see why I gave up on my TNT captures. Why not just leave it at 60Hz, by the way?

EDIT2: This appears to do the trick for the sample.

Code:
LoadCplugin("C:\Program Files (x86)\AviSynth 2.5\plugins\yadif.dll")
loadplugin("C:\Program Files (x86)\AviSynth 2.5\plugins - old C\avisynth_c.dll")
loadcplugin("C:\Program Files (x86)\AviSynth 2.5\plugins - old C\SmartDecimate.dll")

MPEG2Source("RAW IRBR Clip2(3).d2v")

yada=yadif(mode=1)
SmartDecimate(30,60,bob=yada)
#TDecimate(output="IClip.TDec.txt",mode=4) #Run a video analysis pass in VDub
LanczosResize(1440,1080)
TDecimate(input="IClip.TDec.txt",batch=true,mode=0,sdlim=4,cycle=289,cycleR=47,display=true)

#Field match errors at 242 (289) and 1472 (1758)
Note the sdlim parameter, which you definitely want to use for a cycle this long (and is a good idea to use anyway).

This gives 25.096 fps. A sample longer than 208 seconds would be the next step.

Last edited by ChiDragon; 1st June 2013 at 20:18.
ChiDragon is offline   Reply With Quote
Reply

Thread Tools Search this Thread
Search this Thread:

Advanced Search
Display Modes

Posting Rules
You may not post new threads
You may not post replies
You may not post attachments
You may not edit your posts

BB code is On
Smilies are On
[IMG] code is On
HTML code is Off

Forum Jump


All times are GMT +1. The time now is 03:42.


Powered by vBulletin® Version 3.8.11
Copyright ©2000 - 2024, vBulletin Solutions Inc.