Welcome to Doom9's Forum, THE in-place to be for everyone interested in DVD conversion.

Before you start posting please read the forum rules. By posting to this forum you agree to abide by the rules.

 

Go Back   Doom9's Forum > Capturing and Editing Video > Avisynth Usage

Reply
 
Thread Tools Search this Thread Display Modes
Old 21st January 2011, 19:30   #1  |  Link
OvejaNegra
ekTOMBE STUDIOS
 
OvejaNegra's Avatar
 
Join Date: Dec 2005
Location: Cuba
Posts: 253
create 120 fps video -> dealing with vfr + interlaced

Let's say i have a vfr video with 29fps progressive sections, 29.97fps interlaced sections and 23.97fps pulldown sections.

Usually on animes i have interlaced credits, spliting the episode from the credits and procesing each part with tivtc and a good bober and merging again on mkv with edited time codes is not a big deal. But sometimes i find sources with many interlaced parts like 29.97i fades, pans, or wierd fx like smoke or particles or light that was created with CG and overlayed on the static scenes of the episode. Those sections look more fluid with a bob filter.

Using a solution like this one is not possible becase of the many interlaced sections ramdonly placed on the clip:
http://forum.doom9.org/showthread.ph...create+120+fps

Any solution to automatize the process?

My goal is to create a 120fps clip with the all the sections using frame duplication (since 24/30/60 are mults of 120) and after that use dedup with a timecode to delete the duplicated frames.

My las resource is analyze the source using TFM and create an override file for tdeint based on the output of TFM (the progressive sections passed, and the interlaced frames touched by tdeint) split each section on and AVS script, duplicate frames up to 120, merge the segments and encode. The TFM output stats have all the information a need but maybe there is more automatized way to do this.


thanks.
__________________
So, it works or not???
OvejaNegra is offline   Reply With Quote
Old 23rd January 2011, 10:06   #2  |  Link
TheRyuu
warpsharpened
 
Join Date: Feb 2007
Posts: 787
Why can't you just use tfm/tdecimate in the hybrid modes (2 pass I would guess would be preferable) and supply the timecodes file when muxing?

Seems pretty automated to me although it may not give ideal output in some cases (I don't think it can bob either)

Last edited by TheRyuu; 23rd January 2011 at 10:10.
TheRyuu is offline   Reply With Quote
Old 24th January 2011, 19:13   #3  |  Link
osgZach
Registered User
 
Join Date: Feb 2009
Location: USA
Posts: 680
You can use tfm/tdecimate and lets it handle that for you. But personally I don't like the timecodes format it spits out (I prefer timecode V1) because its not always compatible with external players, such as my WDTV Live. But V1's run fine.

You -could- manually write an overrides file, or you could just use YATTA. It lets you do the same thing but with a GUI and lets you also mark frames for post processing, etc. It will also calculate and spit out V1 Timecodes for your project file and generate the Avisynth script to get your deinterlaced/decimated video stream. I wrote a Python script a while ago to speed that process up, but it has its drawbacks. It marks all your 30FPS sections for you in advance, although the process of getting the data moved around is somewhat manual. And re-opening your project can sometimes take a few minutes (i.e go eat a sandwich)

http://forum.doom9.org/showthread.php?t=153102

Getting YATTA and specific filters can sometimes be a pain. If you are interested in giving this a go let me know and I'll just setup a working install for you and give you a download link.
osgZach is offline   Reply With Quote
Old 25th January 2011, 01:43   #4  |  Link
sneaker_ger
Registered User
 
Join Date: Dec 2002
Posts: 5,563
Quote:
Originally Posted by osgZach View Post
But personally I don't like the timecodes format it spits out (I prefer timecode V1) because its not always compatible with external players, such as my WDTV Live. But V1's run fine.
I don't quite get you, since "timecodes v1" is just for import/export into a muxer and nothing mp4 or mkv would use internally, there should be no player that "supports timecodes v1".
sneaker_ger is offline   Reply With Quote
Old 25th January 2011, 05:38   #5  |  Link
osgZach
Registered User
 
Join Date: Feb 2009
Location: USA
Posts: 680
Quote:
Originally Posted by sneaker_ger View Post
I don't quite get you, since "timecodes v1" is just for import/export into a muxer and nothing mp4 or mkv would use internally, there should be no player that "supports timecodes v1".


See this thread

Matroska VFR Proof of Concept

Quote:
Originally Posted by gizmotech

# timecode format v1
Assume 29.970
# Entries Start Here
# 29.970 entries do not need to be entered.
# If frames are not mention they are assumed
# to be the fps above
769,829,23.976
# (843+1-769)*.8 = 60 frames
Last time I used it, TFM's automatic method output a different format. Whatever Matroska is doing with that data internally I do not know, and is beside the point. Muxing with V1 timecodes does not give me problems with my WDTVLive, whereas TFM's auto method would produce unplayable files. Huge corruption/blocking/pauses/skipping.

In any case. I've given him a solution that is just as valid all the same, and IMHO more compatible if he has concerns about set-top media players.

Last edited by osgZach; 25th January 2011 at 05:45.
osgZach is offline   Reply With Quote
Old 25th January 2011, 09:48   #6  |  Link
JEEB
もこたんインしたお!
 
JEEB's Avatar
 
Join Date: Jan 2008
Location: Finland / Japan
Posts: 514
Quote:
Originally Posted by osgZach View Post
See this thread

Matroska VFR Proof of Concept



Last time I used it, TFM's automatic method output a different format. Whatever Matroska is doing with that data internally I do not know, and is beside the point. Muxing with V1 timecodes does not give me problems with my WDTVLive, whereas TFM's auto method would produce unplayable files. Huge corruption/blocking/pauses/skipping.

In any case. I've given him a solution that is just as valid all the same, and IMHO more compatible if he has concerns about set-top media players.
OK, no problem with YATTA (which, albeit being a gigantic PITA, is a relatively useful tool). But those statements are just... quite illogical.
  • MKVMerge takes in V1 and V2 timecodes
  • Matroska has an internal timescale, that by default is 1 millisecond, and all frames have a timecode of this precision
  • With V1 timecodes the timecodes are calculated by the muxer from the framerates marked
  • With V2 timecodes the timecodes are calculated from the per-frame timecodes, which are basically "the amount of milliseconds passed" (and thus in case of integer precision only used, can be thought to be used practically as-is with the default timescale)
  • Thus, Matroska has no real "framerates" at all, and in most cases the output of V1 and V2 timecodes in the case of same source is exactly the same on the output side
Are you sure you are not making false links between things? Are you sure it's not, say, header compression, that your playback device is having problems with (which is well-known, and was fixed for some players lately in a firmware update)?

Also, it could of course be a bug in the player's matroska parsing code in case of a non-standard timescale, f.ex., but that wouldn't be a case of "V2 timecodes is the problem!", to be honest, since you'd have to set that one by yourself. Also, you might just want to use mkvinfo and compare the internal timecodes of the video track in the final cases (mkvinfo -s filename.mkv outputs the timecodes of every frame for the given matroska file's tracks [yes, audio included]). Also, old mkvmerge versions had problems with timecode files (old bugs).

Bringing this to a conclusion:

You problem is most probably:
  1. Header compression (mkvmerge 3.4 was the last version without it being on automatically, and from 4.4 you can now set it to be off by default for certain types of tracks from the settings)
  2. Old mkvmerge (older than 2.2?)
  3. Timescale bug in the player if you are using a non-default timescale (?)
  4. Weird timecode bug in the player (?)

All I am asking is that before you start spouting out things that might stick as a "user myth" even if they are not true, please make sure you understand the things at hand and know the exact cause of the problem. Thank you.
__________________
[I'm human, no debug]
JEEB is offline   Reply With Quote
Old 25th January 2011, 23:00   #7  |  Link
osgZach
Registered User
 
Join Date: Feb 2009
Location: USA
Posts: 680
Where can I specifically look to verify header compression on or off? Though chances are its probably not the issue as I don't recall ever using it... Be that as it may. A particular file I looked at was made back in 2004 with mkvmerge 0.9.2, IIRC.

I will do the following when I have some proper free time. (I have just done a case transfer and things are a mess).
I still have some old files that give me specific problems, so I will take one, re-rip and make a VFR with TFM's auto method, mux it in with the latest release and see if it plays correctly or not.
I'll also try and do some kind of thorough compare between the old and new mkvs.

So we'll see what happens, or if any problems are resolved. Other than that, I don't see the need to be so dismissive. I outlined a specific problem that may be a special case; or may not. Time will tell

Last edited by osgZach; 25th January 2011 at 23:03.
osgZach is offline   Reply With Quote
Old 26th January 2011, 20:48   #8  |  Link
osgZach
Registered User
 
Join Date: Feb 2009
Location: USA
Posts: 680
Alright. I ran a 2-pass mkv create (mode 4/ mode 5) with TFM/TDec and muxed two mkvs from it.

One with no header compression, and one with. The one with header compression would not even play (it would start the play screen, then exit immediately). And from what I could gleen from brief Googling, the issue with the WDTV and header compression is people complaining about audio files not playing (I'm guessing compression was not used on the video tracks). So in either case I think we can rule out header compression as the culprit.

The one I made without compression enabled, played pretty much flawlessly from what I could tell. However I did not compare different timecode format ouputs, so I won't make any accusations either way about that, as I don't think its all that important anyway. I can stick with V1 if I want to be superstitious.

I guess the problem with some of my older files, was the age of the tools? In either case now I'm at a different crossroads of sorts.

I can't decide whether to continue my current programming project (very early testing, but assuming I don't lack the skill, it would be a tool like YATTA only much easier to use and with a half decent documentation), or to scrap that and instead focus on writing a simple GUI to ease the manual script creation process (I will be doing lots of batch encodes) of a 2-pass VFR MKV using TIVTC...lol I guess its a more preferable problem to files that won't play right.

Last edited by osgZach; 26th January 2011 at 20:51.
osgZach is offline   Reply With Quote
Old 26th January 2011, 21:56   #9  |  Link
JEEB
もこたんインしたお!
 
JEEB's Avatar
 
Join Date: Jan 2008
Location: Finland / Japan
Posts: 514
Quote:
Originally Posted by osgZach View Post
One with no header compression, and one with. The one with header compression would not even play (it would start the play screen, then exit immediately). And from what I could gleen from brief Googling, the issue with the WDTV and header compression is people complaining about audio files not playing (I'm guessing compression was not used on the video tracks). So in either case I think we can rule out header compression as the culprit.
I remember header compression getting used by default on all tracks at one point. This was retracted somewhat quickly though on certain a/v formats IIRC.

Quote:
Originally Posted by osgZach View Post
The one I made without compression enabled, played pretty much flawlessly from what I could tell. However I did not compare different timecode format ouputs, so I won't make any accusations either way about that, as I don't think its all that important anyway. I can stick with V1 if I want to be superstitious.
Kind of as suspected, and sure -- I use V1 timecodes when I exactly know where certain frame rates are used and so on (24/1.001fps, 30/1.001fps mostly). Not to mention that V1 timecodes are usually much, much easier to edit by hand .

V2 timecodes usually come up when there can be a lot of different frame rates and the output is usually meant only to be handled by the computer (hence per-frame timecodes format). Albeit conversion applications do exist to convert between v1 and v2 timecode formats.

Different tools for different usage scenarios.

Quote:
Originally Posted by osgZach View Post
I guess the problem with some of my older files, was the age of the tools? In either case now I'm at a different crossroads of sorts.
I would probably blame the tools, as there are bug reports of mkvmerge working weirdly with timecode files on the tracker etc.

Quote:
Originally Posted by osgZach View Post
I can't decide whether to continue my current programming project (very early testing, but assuming I don't lack the skill, it would be a tool like YATTA only much easier to use and with a half decent documentation), or to scrap that and instead focus on writing a simple GUI to ease the manual script creation process (I will be doing lots of batch encodes) of a 2-pass VFR MKV using TIVTC...lol I guess its a more preferable problem to files that won't play right.
Personally I would love a tool like YATTA to be "re-born" in a way (And Myrs was hinting at maybe getting to it at some point using Qt -- although this was months ago, before he headed towards Chinaland).

Although nowadays it is indeed true that we mostly have content that doesn't need as much detail and manual work with the process of IVTC itself. On the other hand YATTA's pattern guidance as well as area-based filtering features etc. would be spectacularly useful in a more user-friendly package (or at least with more "user-friendly" documentation).
__________________
[I'm human, no debug]
JEEB is offline   Reply With Quote
Old 26th January 2011, 22:55   #10  |  Link
osgZach
Registered User
 
Join Date: Feb 2009
Location: USA
Posts: 680
Yeah I was just thinking about that myself. Most days I work directly off DVD sources from more modern Anime, although I also own old and notorious (Lodoss War) or just troublesome (Gall Force, Dirty Pair Flash in particular) sets too. I really don't know a lot of stuff I am gonna have to try any understand at some point.. In terms of properly implementing/cloning some of YATTA's features (I never really used nor understood pattern guidance I guess. I did play around with switching patterns on a per scene basis though but I usually did more harm than good). Like for handling frame cutting and being able to cut accompanying audio files in the proper place without screwing it up - much less how to cut the files. Although in a lot of cases with stuff I don't know about, I am hoping I can find pre-existing tools that I could integrate into the program in some way. It will involve a lot of research for sure..

I think for basic goals... Basic VFR creation for non-troublesome sources will be a major step for this novice programmer. Having an "automatic mode" that just lets you setup some files, setup options for TFM/Tdecimate, setup a Clip2 external deinterlacer if you want, etc.. But then let you review it manually if you want to, so you can mark specific frames for decimation or keeping, deinterlacing, maybe even let you compare different field matching combinations and write those out to an OVR if you want, etc.

The biggest feature I would hope to implement if it all comes together, would be a "Project Database System". I'm likely not going the text file route YATTA went, but will use an integrated SQLite database engine, and have all the data saved there, pretty much everything to do with file edits and other processing settings.. That way users could distribute project databases among each other, be it a single file or maybe a set of files from a series.. So if one person does all the hard work and wants to save everyone else the trouble, as long as they are working from the same source file/DVD/whatever all they have to do is load the project, run it, and hopefully the results will be identical!

If I could get that paltry amount done, I would be very proud of myself.. Just getting an AVI to open and allow frame-based navigation has already involved heavy trial and error on my part, that is a long ways from finished.. I'm actually going against common practice and not manipulating frames with Avisynth directly - because I lack the skill to write a wrapper, much less understand the PureBasic Wrapper that Inc. wrote - on top of learning the Avisynth API. Since this will be a Windows only deal in the end (tied down to Windows by Avisynth obviously) I have been taking baby steps with the AVIFIL32.DLL and API. So far I can at least open an AVI file, get a frame count and the resolution. But I've yet to implement displaying it or working with a stream / frames directly. It does seem to load AVS scripts too, though!


Wish me luck...lol. I'm sorely in need of it

Last edited by osgZach; 26th January 2011 at 23:08.
osgZach is offline   Reply With Quote
Old 4th February 2011, 06:04   #11  |  Link
OvejaNegra
ekTOMBE STUDIOS
 
OvejaNegra's Avatar
 
Join Date: Dec 2005
Location: Cuba
Posts: 253
sorry for the delay:
osgZach: If Yatta can detect->mark->bob()->write correct timecode for interlaced sections (and of course ivtc the film sections and not-touch the 29.97 sections can you give me a quick tutorial?

Thanks in advance (actually never found a comprensible tutorial about how to do certain tasks in yatta)

PD: in fact im making a multi-use tool for compression , and im going to include a IVTC asistant and something like a comprensible YATTA. I have a working code that can open and seek AVS scripts on vb.net (and present the frame), if you want it, ill give it to you.
I think StaxRip does the same (but more well written that mine of course).
The tool will be like VisualStudio but with X264/Xvid and Avisynth + a wierd AVS editor
If i can get it to work i will release it (maybe someone find it usefull)
Rigt now im with the audio encoding tools.
I think that you will end quickly if you write your own wrapper for avisynth using managed C++ cli.

............maybe someone here can write a .net windows forms control in c++ cli that can load and present AVS frames, it will be usefull for tools like megui or for VisualBasic users (like me).

.........please


wish you luck

Megui has an avisynth wrapper but i cant get it to work
__________________
So, it works or not???
OvejaNegra is offline   Reply With Quote
Old 7th February 2011, 18:00   #12  |  Link
osgZach
Registered User
 
Join Date: Feb 2009
Location: USA
Posts: 680
Well it's not so much that YATTA can detect Interlaced vs Progressive and all that. It collects metrics from TFM or Telecide (or both) and the big part about YATTA is that it can be a very intensive manual process.. Although it does have tools for trying to match patterns, etc I think most people tend to work it manually and make the needed changes to frames individually, or by marking frame ranges (0 - 100, 150 - 300, etc).

I'm probably not the best person to teach you how to use YATTA though That and I haven't done any work with it for close to a year now.
The big thing is learning to spot pattern changes and mark them accordingly. If you want to be extremely thorough, this means checking every frame in the source file with your own eyes. There really isn't anything that YATTA does that you couldn't do with Avisynth + Virtualdub as a visual aid for writing your script. YATTA just makes it a heck of a lot easier by automating some really boring repetitive tasks

The problem with YATTA is that it has poor /possibly out of date documentation and does not appear to be actively developed. One of the reasons I decided to try writing my own tool. If you are already working on a tool as well, you may do just as well to continue doing that and program the functionality that you specifically want/need. That's what I intend to do at any rate, and why I haven't really used YATTA since.

For what its worth, you can head over to Tritical's web site at http://web.missouri.edu/~kes25c/ and download his TIVTC package. It comes with example files on how to do a 2-pass VFR encode with Timecodes output and the results are usually pretty good. There are also other examples, such as how to call your own deinterlacer of choice, and a few other things. I have never worked with true interlaced sources, so I can't offer any advice on that. But if you are just IVTC'ing some Anime, assuming it hasn't been butchered by edits after Telecining, etc. Than the 2-pass VFR method will likely get you good results. It's dead simple as well, and goes quite fast as long as you are doing heavy filtering. I usually save to a lossless file using the hacked HuffYUV codec that supports YV12 color space. Then do my filtering, then final encode which gets muxed to MKV with the Timecodes TIVTC yields.

Last edited by osgZach; 7th February 2011 at 18:03.
osgZach is offline   Reply With Quote
Reply

Tags
120 fps, tivtc, vfr

Thread Tools Search this Thread
Search this Thread:

Advanced Search
Display Modes

Posting Rules
You may not post new threads
You may not post replies
You may not post attachments
You may not edit your posts

BB code is On
Smilies are On
[IMG] code is On
HTML code is Off

Forum Jump


All times are GMT +1. The time now is 06:55.


Powered by vBulletin® Version 3.8.11
Copyright ©2000 - 2020, vBulletin Solutions Inc.