@VFW/DShow experts: VFR in Avisynth could be good news for Matroska

[bilu]

Hi,

I've been trying to push the variable framerate subject in Avisynth and we're still trying to figure out if it's doable.

How about an Avisynth fork or extender for Variable Framerate?
http://forum.doom9.org/showthread.php?s=&threadid=63290

Containers like MP4, Matroska and Real Video all support variable framerate, but nobody seems to be trying to work out the subject, except Real Video on a anime filter.

Advantages for VFR:
- Perfect hybrid (30 fps Video + 24 fps Film) movies, without blending 24 fps->30 fps or decimating 30 fps->24fps;
- Much less frames on anime, speeds encoding and gains bitrate;

Disadvantages for VFR:
- AVI and OGM unsupported;
- Questionable video editing capacity?


This post is to request help pushing Avisynth in this useful direction by providing some of your VFW/DShow knowledge. As a non-programmer it is my duty to beg :)

Only then will this be more than experimental in containers.


Best regards,
Bilu

[Koepi]

Well, it's not necessary to have VFR in my oppinion.

In xvid we have a feature called "frame drop" - try it. Encode your video at i.e. 60 fps, set frame drop ratio somewhere to 50%... you should prefilter the sources a bit so that the frames are really a little close.

So there you have something very similar to vfr without breaking compatibility, editibility and all tools we are used to.

Regards
Koepi

[bilu]

@Koepi

:confused:

But how do you know which frames to drop?

There are two main goals for VFR, and if framedrop works it's only for one of them.

1) Anime. You drop a duplicate but double the play time of the original frame, keeping the framerate.

2) Hybrid movies. You don't drop frames in the video parts (pure 30 fps) but drop duplicates in the film part (telecined 24fps -> 30 fps) to achieve pure 24 fps, and on this segment you DON'T want to keep the original framerate.

Conclusion: XVID's framedropping can probably work for anime. To be something more it would have to include some sort of IVTC filtering to know Telecine parts and decide to drop without keeping framerate.

I'm trying your method over the Star Trek Hybrid clip just to prove it.

That's why VFR in Avisynth would benefit all VFR supporting encoders.
XVID is MPEG-4 and would benefit under Matroska and MP4 containers (AFAIK OGM doesn't support VFR).

EDIT: The result sucked :/ Framedrop was dropping much more than just duplicates. I used frame drop ratio 50 as you said.

Bilu

[bilu]

Originally posted by Koepi
So there you have something very similar to vfr without breaking compatibility, editibility and all tools we are used to.


You don't have to break compatibility :rolleyes:
Inserting a special mode into decombers and an internal flag into Avisynth doesn't break compatibility. You're not forced to use this new modes and flags.

But you give users a free choice that can work as good as MotionPerfect on hybrid encodes ;)


Bilu

[Koepi]

Wow. you tested a single suggested value and tell us it sucks?

I expected a little more from you, try some lower values, that was just an example. It should work quite well. Try 10% or something alike.

regards
Koepi

[Neo Neko]

Well the lower the values the less noticable it is. I'll give you that. I have used this feature on more than one occasion though myself. And I wonder if there is not something that could be done to improve the placement of drop frames. Best I can tell even on a 2-pass encode Xvid decides on the fly what to drop. And if it is trying to match a percentage it could drop important frames. :P Perhaps would it be possible to tie it into a 2-pass encode for better placement after the stream has been anylised. Or am I mistaken and this already done?

[Koepi]

Nah, framedrop is very (pre)alpha, it's just a proof-of-concept. The error you encounter on 2pass should be corrected with dev-api-4 as there the full control over the 2nd pass is given unlike in current dev-api-3 with bframes != -1.

Best regards
Koepi

[Atamido]

I've always been a big fan of modularity myself. I think that you could get the frame dropping in XviD to be pretty good. But, as a standalone filter, it would be much more useful. You could use it with any codec, and it could be updated seperately.

I think that adding a VFR ability to something like AVISynth would be VERY useful, especially when dealing with hybrid clips.

[bilu]

Originally posted by Koepi
Wow. you tested a single suggested value and tell us it sucks?

I expected a little more from you, try some lower values, that was just an example. It should work quite well. Try 10% or something alike.

regards
Koepi

I will, sorry for the rushed answer :rolleyes: , but my goal really is enjoying VFR on every codec I want to test that supports that feature. But I promiss I'll do serious testing next time.

BTW, do you know is this feature is being revised for dev-api-4?
I would think developers could look to it as less important... :sly:

EDIT: Another question: how would you know the percentage of dups to drop on a hybrid film? Would it be worth knowing?

Sorry about that again. :P

Bilu

[Neo Neko]

Well it would almost not be worth it trying to calculate the dupe percentage. So many things could effect it and even mess it up. Bad sceen transitions from telecined to interlaced. Or tiny interlaced overlays on large telecined backdrops. What would be ideal is instead of setting a percentage of frames to be dropped which is as I understand how the current frame drop works is to set a threshold at which it should drop frames. And it will drop all frames that are either below that threshold or match other search criteria. This is somewhat how I believe it already works. But I think it would be better to let the algo simply search for positives than force false positives with a frame percentage. But as Keopi says it is a proof of concept. And dev4-api performs quite nicely BTW. Though I must say that recent ffvfw builds are giving everyone a run for the money. :D

[Neo Neko]

Oh BTW. IIRC doesn't the dupe filter for AVS (By Marc FD IIRC) do more or less what you are describing bilu? Replacing nearly duplicate frames with an exact duplicate according to it's threshold? It is more up to the codec and container involved how VFR would be handled. So tying it to AVS while it sounds nice does not make perfect sense since avisynth is not geared to a single codec or output format. Which I think is more or less what koepi is trying to get at.

[bill_baroud]

Dupe is a filter by Donald Graft, which copy frames which are below a defined threshold (diff from the previous frame). The xvid frame drop feature seems to be something like this too, but it's not what Bilu want.

[bilu]

This is a practical case of a VFR filter:

RV9 Animation DropDupe Pre-filter
http://forum.doom9.org/showthread.php?s=&threadid=56564

It's not Avisynth, but you get the general idea :)

@Neo Neko

Quoting myself:

There are two main goals for VFR, and if framedrop works it's only for one of them.

1) Anime. You drop a duplicate but double the play time of the original frame, keeping the framerate.

2) Hybrid movies. You don't drop frames in the video parts (pure 30 fps) but drop duplicates in the film part (telecined 24fps -> 30 fps) to achieve pure 24 fps, and on this segment you DON'T want to keep the original framerate.

So as you see there are two different objectives for VFR and to include such decision in a codec means inserting an IVTC filter into it.

Moreover, if you check that RV9 Dropdupe filter thread I linked to, you'll see they recommend Donald Graft's Dup to help their own filter, complementary and not replacement. Advantages still exist for their filter: faster encode, smaller bitrate.


Bilu

[Neo Neko]

Originally posted by bill_baroud
Dupe is a filter by Donald Graft, which copy frames which are below a defined threshold (diff from the previous frame).

DOH! Sleep is good. Brain funktion impared without slep.

[Neo Neko]

Originally posted by bilu

1) Anime. You drop a duplicate but double the play time of the original frame, keeping the framerate.


The dupe filter is as close as you are going to get on this in AVIsynth. Any functionality beyond that should be handled by the codec itself.

Originally posted by bilu

2) Hybrid movies. You don't drop frames in the video parts (pure 30 fps) but drop duplicates in the film part (telecined 24fps -> 30 fps) to achieve pure 24 fps, and on this segment you DON'T want to keep the original framerate.


Again that is more something that is going to have to be done at the codec level. AVIsynth synthesizes a fake AVI. AVI can't have varriable frame rates. If this were to work And I don't know what current software would if it did Avisynth would have to be forked into a more general videosynth program that outputs to it's own format so as not to be limited in immatating a limited format. This would instantly break support in all AVS supporting software for that fork.

Originally posted by bilu

Conclusion: XVID's framedropping can probably work for anime. To be something more it would have to include some sort of IVTC filtering to know Telecine parts and decide to drop without keeping framerate.


What's wrong with using nearly null pading frames as placeholders to keep constant frame rate with fewer unique frames? It takes slightly more space than nothing at all. But it would have to be like a 12 hour movie for the null frames to eat up significant bitrate that even then would be nothing in relation to the final size of the film.

[bilu]

The dupe filter is as close as you are going to get on this in AVIsynth.
Any functionality beyond that should be handled by the codec itself.
Look at the RV9 example: gains bitrate, speeds encoding a lot.
Again that is more something that is going to have to be done at the codec level.
AVIsynth synthesizes a fake AVI. AVI can't have varriable frame rates. If this were to
work And I don't know what current software would if it did Avisynth would have to be
forked into a more general videosynth program that outputs to it's own format so as not
to be limited in immatating a limited format. This would instantly break support in all
AVS supporting software for that fork.
Then why don't you explain my what does OutputDebugString used by Telecide(debug=true)
has to do with VFW?? Has it broken compatibilities?

What I'm trying to say, and have already discussed the issue with Bidoche before ( see my
1st post ) is that Avisynth exports data through the VFW interface ( not allways, as I've
shown you, not to mention text logs) and I'm trying to find out if frame duration could be
sent in another stream (just like you send audio through Avisynth) and then a DShow Splitter
would act on the Video stream based on the information of this frame duration stream.

Could be done easier by using text logs, do a XVID null encode with an AVS script with
VFR-prepared filters that would export such info, and then load the source script and log
into a VFR prepared encoder.

But that is a bit primitive, don't you think? And the DShow splitter option would bring VFR
power to every app that supports DShow sources and is VFR capable.
What's wrong with using nearly null pading frames as placeholders to keep constant
frame rate with fewer unique frames? It takes slightly more space than nothing at all. But
it would have to be like a 12 hour movie for the null frames to eat up significant bitrate that
even then would be nothing in relation to the final size of the film.
It's not just about bitrate. It's mainly about speed gain in anime encodes and ***PERFECT***
( as in better than MotionPerfect ) hybrid movie encodes.

I'm trying to find ways that don't break compatibilities. As Bidoche said, frame duration
flags are easy. And IMHO create an IVTC filter that modifies duration flags instead of blending
frames is easier to develop than traditional IVTC filters and faster.

VFR: AVS -> VFW + frame duration stream -> VFR DShow splitter -> VFR rendering

CFR: AVS -> VFW + frame duration stream(a) -> CFR DShow splitter(b) -> CFR rendering

(a) frame duration stream would be optional and activated by a flag in Avisynth. If not available,
even the VFR DShow splitter would do CFR rendering;

(b) if the CFR DShow splitter received such a stream it would ignore it;

Still think it's undoable, breaks compatibility and not worth it? :sly:


EDIT: Older discussions I found on this subject

Variable frame rate support
http://forum.doom9.org/showthread.php?s=&threadid=47203

VFR in Avisynth 3.0?
http://forum.doom9.org/showthread.php?s=&threadid=53028

3.0 Wip
http://forum.doom9.org/showthread.php?s=&threadid=48940


Bilu

[Neo Neko]

In short yes. But that is just my oppinion.

The biggest thing AVS does for me is bring support to VFW based programs for codecs and formats that only Directshow supports or reading in the odd DVD. So having to use a directshow based program to make use of it already limits the features usability and if I understand correctly would indeed break compatability.

If I understand your sped up encoding claim you are claiming that changing a frames duration means the frame would only have to be filtered once and not twice if only 3:2 pulldown and no decimation etc was done. All I can say is you would have to be doing some heavy duty filtering for it to make a large difference. And it would not speed up encoding that much especially in comparrison with Xvid's frame drop. Actualy in conjunction with Xvids frame drop if there were not some communication between AVS and the codec itself about which frames were already calculated to drop the calculations would have to be done again slowing down the encoding even more. And how in VFW and directshow are you going to acomplish that?!

If someone were going to do this. And I mean do this right. Then an end to end AVS solution would be best. Then external program issues would be relatively trivial. But before that can happen a few things would need to be done first.

1. Have directshow source and other source plugins handle audio uniformly much like AVISOURCE. Which I hear is in the works anyway.

2. Have codecs reworked as plugins for AVS itself. You would not be able to divx with it unless by some twist of fate they supported it or there were some wrapper shenanegins. But Xvid, ffmpeg, and even VP3 since the source code is avalible could be created as loadable modules

3. Finally include muxing filters whether via directshow or loadable filters and a file writer.

Then you could do scripts like this.

loadplugin("c:\avs\mpeg2dec3.dll")
loadplugin("c:\avs\vfr.dll")
loadplugin("c:\avs\ffmpeg.dll")
loadplugin("c:\avs\lame.dll")
loadplugin("c:\avs\mkvmuxer.dll")
movie = mpeg2source("c:\path to\some.d2v")
movie.vfr(8)
movie.lanczosresize(480x480)
movie.ffmpeg(3,2,31,2,31,620000,MPEG,"c:\path to\movie.stats",4:3)
movie.lame(alt-preset-medium)
movie.mkvmuxer(4:3, "c:\path to\chapters.ogg", Title="My Movie", Author="Me")
filesink("c:\path to\movie.mkv")
return null


But even that would extend the software far beyond the scope it currently occupies. Not that it would be a bad thing. Keep in mind it would not be an easy thing. And it's benefit would be limited and would not justify a rewrite of almost everything in Avisynth to handle it. It sounds great in theory. But I think logistics could be killer.

[Atamido]

I am a little split on how to accomplish this.

VFR support in some sort of complex encoding system like AVISynth is needed because there are VFR sources in the wild right now. Obviously, VFW applications could not support that feature, which limits you to DirectShow applications.

Also, while I do believe that the codec should be able to alter the framerate, drop or add frames, I also believe that handling this outside of the codec would be better as it would make it possible for any codec to make use of it.

If so much of AVISynth would have to be edited, then it would probably be much more feasible to write an AVISynth compatible system from scratch that was designed to handle those extra features. An AVISynth plugin wrapper for DS shouldn't be that hard to produce, but you would loose all of the advanced scripting features of AVISynth.

[Neo Neko]

Frame dropping and altering frame durration outside the codec is utterly useless if the codec itself does not support it. If the codec does not support it the best you will ever get is telecining and then running the dupe filter. Most codecs require constant framerate. So even if AVS could flag a frame to have a durration 10 times longer than that of an average frame pretty much every VFW and directshow codec in existance is gonna encode 10 identical frames in place of the 1 longer one. Useless. The speed up on filtering would also be negligable. And in order to get any speed increase on filtering you are gonna have to make the filters both internal and external VFR aware at least in some small way.

It sounds like a good idea. But I am having troubles seeing the real benefit at the moment and a justification that could convince the AVS developers to re-write large portions of their code.

[Atamido]

Originally posted by Neo Neko
Most codecs require constant framerate. Most of that post is nonsense, and here is the root of the problem with it. I can't think of a codec that DOES know about framerate. Codecs today are entirely independant of framerate or the timecodes of a frame*.

In VFW, a frame is passed to the codec and data for that frame is passed out the other end. The only thing the codec can know in this case is how many frames it has been passed. It doesn't even know if there is a dropped frame in the stream because that data isn't passed to it. To quote the developer of VirtualDubMod:**

[13:43] <Cyrius> well at least VirtualDub doesn't send the dropped frame to the codec :p
...and...
[13:51] <Cyrius> no frame number is sent with VfW
[13:51] <Cyrius> so the codec doesn't know anything
[13:51] <Cyrius> except that you are giving a frame :p

DirectShow is a little different in that frame(s) are passed to the codec and data's are passesd out. Also, there is a timecode attached to each frames. And while it is theoretically possible for a codec to change a timecode, or not output one of the data, none do. All current codecs simply pass the timecode information through with each frame.

So, assuming you did the same thing in AVISynth, filters could either be aware of the timecode or not. If they are then they can drop out a frame or change the timecode. If they aren't then they just ignore it and pass on the extra frame data. Like the codecs in DS, most filters would never need to know what the timecode is.

*(The only exceptions that I can think of would be MPEG-2 but that is a different story, and FLASH as it is technically a CODEC that renders VFR where VFR=however fast it can render on a particular CPU.) **(The entire IRC conversation on this very topic is available upon request.)

[ChristianHJW]

Originally posted by Pamel In VFW, a frame is passed to the codec and data for that frame is passed out the other end. The only thing the codec can know in this case is how many frames it has been passed. It doesn't even know if there is a dropped frame in the stream because that data isn't passed to it.

That may be true, still, won't the motion vector calculation be completely screwed if i feed it with frames that have a non-linear timescale underlying ? I dont know about RV9, Karl to comment, but i guess sysKin will confirm that XviD's ME ( Motion Estimation ) had problems here, at least with linear ( = constant speed ) movements ...

[Atamido]

Originally posted by ChristianHJW
That may be true, still, won't the motion vector calculation be completely screwed if i feed it with frames that have a non-linear timescale underlying ? I dont know about RV9, Karl to comment, but i guess sysKin will confirm that XviD's ME ( Motion Estimation ) had problems here, at least with linear ( = constant speed ) movements ... No.

While it is true that XviD can store the framerate in it's video bitstream, changing the framerate that they are displayed at would have no effect on the decoding. The ME in XviD, and other codecs, work on a frame basis, not timecodes. As an example, here (http://www.jkfanclub.com:81/3blocks-vfr.mkv) is a clip in Xvid that changes framerates 4 times between 3 framerates and there are no problems with it. (500KB)

[Neo Neko]

Originally posted by Pamel
Most of that post is nonsense, and here is the root of the problem with it. I can't think of a codec that DOES know about framerate. Codecs today are entirely independant of framerate or the timecodes of a frame*.


Yes you are correct the codec just encodes or decodes. But for all intents and purposes they may as well be cfr for the simple fact that the framework it is trying to opperate in and the software it is trying to work with often have little or no concept of anything outside cfr. At least in my usage. Directshow does support VFR. ASF for instance does not contain any framerate constraints as such the frame rate is not stored anywhere in the file. But still how is one to pass the info from AVS via faux avi to a directshow program? Does AVI support VFR?

Originally posted by Pamel
So, assuming you did the same thing in AVISynth, filters could either be aware of the timecode or not. If they are then they can drop out a frame or change the timecode. If they aren't then they just ignore it and pass on the extra frame data. Like the codecs in DS, most filters would never need to know what the timecode is.


Yes but how are you going to do this via frameserving in faux AVI? Can this information be passed in this manner and if it could how are current programs to pick it up?

[bilu]

Originally posted by Neo Neko
The biggest thing AVS does for me is bring support to VFW based programs for codecs and formats that only Directshow supports or reading in the odd DVD. So having to use a directshow based program to make use of it already limits the features usability and if I understand correctly would indeed break compatability.


You would only use DShow splitter as a complement to an AVS VFR mode set by an internal AVS flag.
Output through the VFW stream would allways be CFR but the other stream would send the frame duration info.
That frame duration stream would send info to a DShow splitter ***OR a VFW codec modified to support that info***.
The DShow splitter is the only alternative for codecs not specifically modified to support that stream.

Alternatively you could do pure CFR encoding and then send the frame duration info to a file that would be used when muxing that stream to a VFR container.
As an example, you would encode an hybrid movie without the Telecine dups, with the video parts at 30fps and the film parts SPEEDED UP from 24fps to 30 fps.
Then the frame duration log file would be used when muxing to a VFR container such as MP4, MKV or RMVB.

Now that I think of it, this would be the easiest method to get VFR working :)

And for encoding anime, you could encode only the original frames (if a 30 fps anime is really only 8fps rendered, it would be almost 4 times faster to encode - it would look like an 8fps anime speeded up to 30 fps) but in the log stream define all the duplicates as "framenr 1234 is referenced to framenr 1233, duration 0 ms", and the original frame as "framenr 1233, duration ((30/8) x Y) ms" where Y is the standard frame duration. Since only frame 1233 was encoded it would have a duration = duration of frame + all duplicates.

On a hybrid movie it would look like "framenr 1234 is referenced to framenr 1233, duration 0 ms" and "framenr 1233, duration Y ms" since we don't want to replicate the frame duration of the dupped frame.

This system would allow transcoding an hybrid movie back to CFR. It would also be helpful doing the same thing in anime, but not necessarily - anime CFR transcodes could be done by simply dividing (Frame duration / Min Frame Duration). Hybrids would be the only case where VFR frame duration != CFR frame+dup duration.

It wouldn't be breaking compatibility EVEN in VFR mode (both hybrids and animes would encode only one original frame, 24 fps would look speeded to 30 fps WHEN encoding in VFR mode) because to CFR filters this is just a normal frame, they don't care about timestamps anyway. Should be careful with temporal smoothers though.

It's nice to talk to skeptics about this coz my ideas are improving! :D Thanks.

So now I think it as:

VFR flag, CFR Anime source -> 8 fps speeded up to 30 fps + VFR log -> VFW encode with (30/8) less frames -> mux VFW encode + VFR log into VFR container -> VFR anime stream

VFR flag, CFR Hybrid source -> 24fps parts speeded up to 30 fps + VFR log -> VFR encode with (nr of Telecide() dups) less frames -> mux VFW encode + VFR log into VFR container -> VFR hybrid stream

So this got easier and more compatible:

Avisynth: a VFR IVTC filter, a VFR Dup filter, an internal AVS VFR flag, an internal AVS VFR timestamp that can be changed by VFR filters, and a log writer.
VFR Muxer: A log reader and timestamps adjustments in the container. If log is kept then movie can be transcoded back to CFR.


See? Much easier to think, to develop and more compatible. :)

@all
Thanks for the criticism until now, it has been very helpful. Keep doing the good job :D


Best regards,
Bilu

[gizmotech]

Avisynth: a VFR IVTC filter, a VFR Dup filter, an internal AVS VFR flag, an internal AVS VFR timestamp that can be changed by VFR filters, and a log writer.
VFR Muxer: A log reader and timestamps adjustments in the container. If log is kept then movie can be transcoded back to CFR.


Bilu you're moving DANGEROUSLY close to some ideas I was trying to develop before.

My ideas was as such.

Patern restoration through regular means.
Decimation on 24fps content in AVS (selective decimation if you will. Most likely manual).
And assuming AVS can just keep streaming to VFW like I think it does this would provide you with a completely useable AVI, which is completely timecode INCORRECT. <-- Meaning no mods to avs, just mods to decimation.

Now the real issue comes in the remux into a vfr container.

We have a file which is now composed of either 30fps content at 30fps and 24fs content at 30 fps (sped up). Now all we need to do is specify to the container muxer which parts need to be slowed down.

We can determine the time paterns required for a 24fps encode, they are quite easy to produce (I've done it before) which would give you the appropriate time codes to feed the muxer for the container.

See my idea was this:
MKV (the only VFR container I'm aware of that is FREE to use *I don't consider rv9 free*) stores a time code onto ever block as it writes the frame. Now if you could modify the block writing proceedure to modify the time codes on the blocks (frames) appropriately to return 24fps content back to 24fps (from the 30 fps speedup) then you should have a vfr encode.

See the biggest thing here is you're assuming that there is a NEED for AVS to do anything relating to the VFR proceedure. When infact the encoder will know the frame numbers where the 30fps and 24fs content starts, as they had to make the distinction. I'm suggesting that AVS need not know anything about the VFR proceedure at all, and continue to stream a fixed frame rate content to VFW and that the corrections take place afterwords through manual file settings.

My Strategy would be like this (and could be implemented by someone who understand the code pertainnin to libmatroska):
Standard video stream input --> AVS w/ manual decimation --> VFR container specific muxer w/ override file --> vfr container.

I see this as quite possibly the easiest way to do it, and even tried to code it, however my programming skills are almost non existant so all I succeeded in doing was b0rking up the libs and apps I downloaded to modify.

Could I get some feed back from libmatroska people as to how possible it would be to manuall specify the time codes on a block during muxing?

Gizmo

[bilu]

See the biggest thing here is you're assuming that there is a NEED for AVS to do anything relating to the VFR proceedure. When infact the encoder will know the frame numbers where the 30fps and 24fs content starts, as they had to make the distinction. I'm suggesting that AVS need not know anything about the VFR proceedure at all, and continue to stream a fixed frame rate content to VFW and that the corrections take place afterwords through manual file settings.

What I'm suggesting is that Avisynth streams a fixed frame rate content to VFW, same thing.
But the encoder DOESN'T need to know anything but Maximum Framerate, already provided by VFW.

The Avisynth script would render the timestamp into a file log and the VFR Muxer (using the "VFR-style" CFR AVI and the timestamp log) would then modify the timestamp info for each frame it keeps. No need to modify the codecs to pass other info ( the encoder itself never know when there is 24fps or 30fps content, framerate is defined in the stream beginning) and this also means you can use every (or almost every) codec muxable into VFR containers such as Matroska.

Implementing a flag, a timestamp and a log writer into Avisynth shouldn't be too hard. And modifying a Matroska muxer or any other VFR muxer to read timestamps from a file shouldn't be to hard either. Having a DShow filter that grabs the timestamp file and the VFR'ed AVI to provide VFR streams into RV9 if they can't modify the container and want to support VFR Avisynth sources in their container should also be easy.

EDIT: A scheme.

CFR source -> VFRed AVS + timestamp log -> VFW -> VFRed AVI (VFRed = CFR with speedups)

and then

VFRed AVI + timestamp log -> VFR Muxer -> VFR stream (timestamps in the VFR container)


Bilu

[bilu]

Feedback, anyone? Neo Neko? :)

@GizmoTech


Could I get some feed back from libmatroska people as to how possible it would be to manuall specify the time codes on a block during muxing?


This can help:

http://forum.doom9.org/showthread.php?s=&postid=270117#post270117


Bilu

[gizmotech]

Ok,

But I still don't understand why you need to pass anything through avs. As far as I can see there is no reason to modify AVS itself to do anything special. Most of the emphasis here is on a modified decimation and a muxing application which can change the the frame rates to compensate for speedup.

I really don't see a need to modify avs, as what part of this is required in it? It doesn't need to know timestamps, it doesn't need to modify time stamps. Because all it does is stream single frames into VFW as if it were an avi being accessed it doesn't need to do a thing differently, just the decimation filter needs to be modified for range decimation.

Either way, I'm waitting to hear input from the muxing side of this as this is where I see all the required work.

Gizmo.

[bilu]

AVS already has great IVTC and Dup filters which can be modified by adding them a special VFR mode.
I was thinking about changing a timestamp to 0 ms instead of dropping frames until the moment of writing the file log, it would help reconstruct a CFR stream if needed.

If you had a Telecined anime you would use the IVTC filter to remove the Telecine dup, and the filter wouldn't add time on the original frame timestamp.

Like this "frame 1233, X ms" and "frame 1234 referenced to frame 1233, 0 ms" and the next frame "frame 1235, X ms"

Because the filter would do IVTC only, frame 1234 would be marked as dup to drop by putting the timestamp to 0 ms.
No frames would be dropped until the moment of writing the file log, but filters would skip processing frames with timestamp=0ms.
The idea is to keep the total number of frames on the log, to make VFR mux but also to reconstruct VFR later if needed.

Then it would run through the Dup filter, which would drop every other dup frame, but add the dup frame timestamp to the original timestamp.
Timestamps would have to be each frame duration for this to work, not the film stamp of when the frame is displayed.

Like this:

"frame 1233, 2x X ms" - because you added frame 1235 timestamp to it
"frame 1234 referenced to frame 1233, 0 ms" - would be skipped
"frame 1235, 0 ms" - marked for dropping

At the end the timestamp log would be written and the frames with 0ms would be dropped from exporting to VFW.
The VFR muxer would also skip durations=0 ms.

This is my original concept, but it may be difficult to provide a timestamp log that provides both VFR muxing and CFR rebuilding, given that I also don't know how hard CFR rebuilding would be - maybe it would only work when Avisynth would use its own generated VFR streams as source! :D So it's a good thing after all.


An easier concept would be really dropping the frames. As in the former concept, IVTC would mark the timestamps as 24 fps duration, 30 fps duration or drop the frames.
Then Dup would drop the rest of the dupes and multiply the timestamp of the original frame. Timestamp log would only include frames that would be exported to VFW.

The only easier part about this is that you avoid marking frames as 0 ms and filter skipping on those frames. The filter skipping method most probably is not compatible with actual filters since it would be only used by VFR prepared filters, which means the rest of the filters would really process them, making the AVS rendering take more time, memory and probably other problems, I don't know. Filter skipping on timestamp=0ms would probably only work with VFR prepared filters.

So IMHO we need two VFR AVS flag states, one for total AVS backward compatibility and other to be able to do CFR reconstruct and VFR reuse in Avisynth.
So it needs three VFR flag states (including "no VFR"), a timestamp, a frame skipping process (probably already exists, but now associated to the timestamp) and frame dropping in the end, and a log writer.

In the VFR muxer changes wouldn't need to be made to support both modes: just ignore timestamps= 0ms when they happen, because in one of the modes they don't happen.


Avisynth changes are needed, of course. Because frame duration should be calculated by both Dup and IVTC filters - on hybrid movies besides dropping dups it has to define which frames are 24 fps and which are 30 fps. If it would be only for anime you would be right, no need to put it in Avisynth, a codec could multiply the original timestamp as much as the frames it drops. But we wouldn't do hybrid IVTC in the codec itself, would we? ;)

Besides this way we get the opportunity to use this in every codec supported in Matroska! :D


Best regards,
Bilu

[bilu]

A Summary until now:

VFR DShow filter
==========
- reads frames and timestamps and increases the frame duration as needed;
- for unmodified VFR muxers;
- VFR AVS preview;

VFR Muxer
=======
- add support for timestamp log reading;
- ignore timestamps = 0ms when they exist;

Avisynth VFR producing
===============
- add a VFR flag with three states: no VFR, compatibility (a), complete timestamp (a);
- a timestamp flag with the frame duration for each frame;
- a frame skipping process when timestamp=0ms (a);
- frame dropping in the end of the script when timestamp=0 (a);
- a timestamp log writer;

Avisynth VFR reading (especially to convert VFR -> CFR)
=============
- add support to read timestamp logs;
- ability to copy same frame for the number of times defined in the original frame;
- ability to copy same frame again if "framenr 1235 referenced to frame 1233, duration 0 ms" but not included in the duration of the original frame in the VFR timestamp - this means it is a IVTC duplicate;

if log not available:
- calc max framerate;
- divide (frame duration/max framerate) and copy frame that number of times;

it would work on anime, but never in hybrid IVTC. Log is allways necessary in this case.


(a) The Avisynth VFR process
==================
1) Complete Timestamp mode: used to generate a timestamp log that includes the dropped frames (0 ms).
Useful for VFR -> CFR conversions and VFR reading in Avisynth (needed for VFR -> CFR).

This method should be compatible with VFR-prepared filters only.
IVTC filters mark frames with 24fps duration, 30fps duration or 0ms. No frame is dropped.
Dup filters mark frames as (nr dups + 1) frame duration or 0ms. No frame is dropped.

All the VFR-compatible filters skip frames with 0ms, but don't drop them. This is to keep the complete timestamp integrity.

In the end of the script the complete timestamp log is written and frames are dropped before exporting to VFW.

2) Compatibility mode: used to generate a timestamp log without the dropped frames(0ms).
Useful whenever you need to use non VFR-compatible filters in the script.

This method should be compatible with all filters.
IVTC filters mark frames with 24fps duration, 30fps duration and drops duplicates.
Dup filters mark frames as (nr dups + 1) frame duration and drops duplicates.

Filters don't need to skip any frames based on timestamp, hence the increased compatibility.
Timestamp log only contains non-dropped frames.

In the end of the script the timestamp log is written and all frames are exported to VFW.


Note: the "framenr 1235 referenced to frame 1233, duration 0 ms" notation in the timestamp log serves to define the original frame, so that when reading for VFR->CFR in Avisynth you know which frame to copy.



Best regards,
Bilu

[Bidoche]

add a VFR flag with three states: no VFR, compatibility (a), complete timestamp (a); 3.0 properties don't work like that.
It's either you have the property (and its attached values) or not.
What do you mean by compatibilty/complete timestamp ?

I was thinking at that :class FrameDurationProperty : public Property
{
public:
FrameDurationProperty(unsigned _duration)
: duration( _duration ) { }

virtual const PropertyKey& GetKey() const { static Property key, retun key; }

unsigned duration;

};Instances of this class being either attached to a frame (or not).

- a timestamp flag with the frame duration for each frame; What do you mean ? Frames not concerned won't get the flag...

- a frame skipping process when timestamp=0ms (a); Affect too much filters to be realistic. You must know what you are doing when using VFR unaware filters.


- frame dropping in the end of the script when timestamp=0 (a); Please explain that .

Avisynth VFR reading (especially to convert VFR -> CFR)
=============
- add support to read timestamp logs;
- ability to copy same frame for the number of times defined in the original frame;
- ability to copy same frame again if "framenr 1235 referenced to frame 1233, duration 0 ms" but not included in the duration of the original frame in the VFR timestamp - this means it is a IVTC duplicate;

if log not available:
- calc max framerate;
- divide (frame duration/max framerate) and copy frame that number of times;

it would work on anime, but never in hybrid IVTC. Log is allways necessary in this case.Effect of a source filter to be defined... in a plugin, like the rest of this anyway.

[bilu]

3.0 properties don't work like that.
It's either you have the property (and its attached values) or not.
What do you mean by compatibilty/complete timestamp ?--------------------------------------------------------------------------------
- a frame skipping process when timestamp=0ms (a);
--------------------------------------------------------------------------------

Affect too much filters to be realistic. You must know what you are doing when using VFR unaware filters.--------------------------------------------------------------------------------
- frame dropping in the end of the script when timestamp=0 (a);
--------------------------------------------------------------------------------
Please explain that .
The whole point of skipping frames with timestamp=0ms until the end of the script was to generate a timestamp log file that would include the dropped frames.
That would help rebuild a CFR stream if you ever want to transcode back into a CFR container.

For anime a timestamp log is not necessary, as (frame duration/(max framerate duration)) - 1 = number of duplicates, so you can generate frame copies in Avisynth when you serve the script back as input. This is because anime dups duration will allways be added to the original frame duration in the VFR stream.

But for hybrid movies you don't add the Telecide() dup frame duration to the original frame, so there is no way to know if a part of the hybrid movie is 24fps or 30 fps.

HEY!!! You can get the frame duration from the stream itself when using VFR streams as source! That's it! No need for a complete timestamp mode! :D
So you can still recreate a dup by deliverying another dup at each 5 frames when timestamp of the last 4 frames was 24fps and max framerate is 30 fps.

It would even work in IVTC anime, since you can recreate the dups, then check if the 4 last frames are 24fps and max framerate is 30 fps, and recreate another dupe.

So after all, there is no need for frame dropping at the end of the script or frame skipping. :)
--------------------------------------------------------------------------------
- a timestamp flag with the frame duration for each frame;
--------------------------------------------------------------------------------

What do you mean ? Frames not concerned won't get the flag...
On hybrid movies, FILM frames should be marked with 24 fps duration and VIDEO frames with 30 fps duration.
On anime, original frames get dup duration added.

The only not concerned frames would be those with the max framerate.
I think there should be a max framerate flag exported to VFW (already there) that the VFR container also grabs.
It's the only way to avoid flagging every non-dropped frame and let the VFR muxer itself attaches frame duration to every frame's properties.

So after all on a hybrid movie you would drop dups and mark only the 24 fps frames. In the VFR container all should be marked as 24 fps or 30 fps, because the non-marked frames got the max framerate on the VFR muxing from the VFW header.
I was thinking at that :
code:
--------------------------------------------------------------------------------
class FrameDurationProperty : public Property
{
public:
FrameDurationProperty(unsigned _duration)
: duration( _duration ) { }

virtual const PropertyKey& GetKey() const { static Property key, retun key; }

unsigned duration;

};
--------------------------------------------------------------------------------
Instances of this class being either attached to a frame (or not).
They're only useful attached to a frame. Max framerate gets defined in VFW so you can skip those as seen in the middle of this post, so you won't decrease performance on pure video non-anime encodes, NTSC or PAL. But it's the only way to deal with both hybrid clips and anime, I think. It may be easier to VFR muxers this way.

I don't know if using a timecode SMTPE-like through the whole script, generating a timecode similar to most subtitle formats instead of using each frame duration, would bring any advantage.

@all

Frame 700: 00:00:35.527
Frame 701: 00:00:35.531

or

Frame 700: 41,666 ms
Frame 701: 41,666 ms

Which timestamp format would be the best for both editing and muxing?

@Bidoche
Now that I think of it, itmay be easier to edit and to synchronize with the "00:00:35.531" timestamp associated with each frame.
But for doing VFR -> CFR conversions the "41,666 ms" format would be easier, because Avisynth wouldn't have to do time calculations for each frame. The decision would be most probably performance dependent. :sly:

Best regards,
Bilu

[Atamido]

Originally posted by Bidoche
3.0 properties don't work like that.
It's either you have the property (and its attached values) or not.
What do you mean by compatibilty/complete timestamp ?Would you explain the 3.0 properties section a little better? I'm afraid that I've never understood it very well. If you have a property on one frame, do you need it on all of them? Can the value of one property affect the value of another property?

I think it would be best to just assume that all clips are VFR. If you know they aren't, then it won't matter. If your destination won't support them, then you just avoid plugins/filters that would make the source VFR.I was thinking at that :
....
Instances of this class being either attached to a frame (or not). Using durations can be fun, but I would urge you to use timecodes instead. For instance, if the video has a delay to it, there would be no way to indicate this if the frames only had durations. Also, I fear for this situation:

"You have 3 frames: A, B, and C. AVISynth processes A and it goes out. Then it processes B and decides to drop it as a duplicate frame. When you process frame C it will end up in frame B's old place because frame A's duration wasn't changed. So your entire video after frame A is offset by one frame. With timecodes this would not have been an issue."

Now if you decided to have a property for both timecodes AND durations, then you could do all sorts of delicious things like having gaps in the stream. In this case AVISynth could then be used to process subtitle streams also. :D

In Matroska, the duration is assumed to be from one timecode to the start of the next timecode UNLESS specified otherwise. This might be a handy way to do things in AVISynth also.Affect too much filters to be realistic. You must know what you are doing when using VFR unaware filters. I was under the impression that if a filter didn't know what a property was, it would just ignore it. Obviously, using a timecode of zero would be bad because that is a real timecode. What would be the best way to indicate that a frame should not be used, in other words that a frame has been dropped? I'm not sure if AVISynth can decide to drop out a frame in the middle of a filter chain. Effect of a source filter to be defined... in a plugin, like the rest of this anyway. I don't think that I see why you would want to do frame references. I think thats something that a plugin would just handle automatically with either creating or deleting frames.

[bilu]

I vote for timecodes instead of frame durations. It seems to have fewer drawbacks.

@Pamel
Have you read my last post?

EDIT: Changed my mind, have no opinion on frame duration vs timecodes now until this gets better discussed. It has been largely discussed in the thread below with no conclusive results:

Variable frame rate support
http://forum.doom9.org/showthread.php?s=&threadid=47203


Frame duration:
(+) ability to change frame duration manually on video editor or delete frame without reencoding;
(+) ability to do better VFR->CFR conversions, framerates are easier to calculate;

Timecodes:
(+) ability to seek;
(+) ability to synchronize;

So frame durations are better for editing while timecodes are better for playback. If we need to use BOTH, I suggest we keep trace of the last modified property (durations or timecodes) and synchronize the other one. Synchronization should be made only after finished editing.


Bilu

[Atamido]

Originally posted by bilu
The whole point of skipping frames with timestamp=0ms until the end of the script was to generate a timestamp log file that would include the dropped frames.
That would help rebuild a CFR stream if you ever want to transcode back into a CFR container. I don't know if there would be enough benefit to this to merit the work it would require to implement. I suppose that someone might want to make the clip CFR again, but either they would have the original source and the log, or neither. In both cases the log doesn't seem like it would be very useful.

If you were JUST dropping frames, then it wouldn't be a problem to do. VDMod currently creates VFR files when it encounters and AVI with dropped frames. (Dropped frames in AVI are frames where the frame was lost, like in recording and the computer isn't fast enough and misses a frame. It writes a "dropped frame" so that the file can stay in synch, but the reader knows not to use that frame.) VDMod simply doesn't write those frames in MKV because there isn't a way to do it(it would be pointless). But, when parsing the file, it figures out what the framerate of the file 'should' be and if it finds a spot where there is not a frame at the right timecode, it creates a "dropped frame" internaly so that it can keep synch. (VDMod cannot handle VFR internally). So, if you used AVISynth to create VFR matroska clips where you took a CFR clip and ONLY dropped frames, VDMod could converth them back to a CFR container.In the VFR container all should be marked as 24 fps or 30 fps, because the non-marked frames got the max framerate on the VFR muxing from the VFW header. They're only useful attached to a frame.VFR containers aren't going to mark clips as 24fps or 30fps because there is no fps, just timecodes. The exception to the are MPEG containers, but their methods are screwy and part of the problem.

[bilu]

quote:Originally posted by bilu
The whole point of skipping frames with timestamp=0ms until the end of the script was to generate a timestamp log file that would include the dropped frames.
That would help rebuild a CFR stream if you ever want to transcode back into a CFR container.

I don't know if there would be enough benefit to this to merit the work it would require to implement. I suppose that someone might want to make the clip CFR again, but either they would have the original source and the log, or neither. In both cases the log doesn't seem like it would be very useful.
You're right about the log not being very useful to transcode back to CFR, since you can get frame duration from the VFR streams, even if calculated from timecodes.
But you would need a Max Framerate info kept on the VFR container so that AVS would know it would have to add a dup after four 24 fps frames on VFR hybrid encodes.If you were JUST dropping frames, then it wouldn't be a problem to do. VDMod currently creates VFR files when it encounters and AVI with dropped frames. (Dropped frames in AVI are frames where the frame was lost, like in recording and the computer isn't fast enough and misses a frame. It writes a "dropped frame" so that the file can stay in synch, but the reader knows not to use that frame.) VDMod simply doesn't write those frames in MKV because there isn't a way to do it(it would be pointless). But, when parsing the file, it figures out what the framerate of the file 'should' be and if it finds a spot where there is not a frame at the right timecode, it creates a "dropped frame" internaly so that it can keep synch. (VDMod cannot handle VFR internally). So, if you used AVISynth to create VFR matroska clips where you took a CFR clip and ONLY dropped frames, VDMod could converth them back to a CFR container.That's not VFR, that's something like Donald's Dup filter triggered by dropped frames. When it sees no frame, it copies the former frame again to stay in synch.
At least this is what I understood from your explanation.
quote:In the VFR container all should be marked as 24 fps or 30 fps, because the non-marked frames got the max framerate on the VFR muxing from the VFW header. They're only useful attached to a frame.

VFR containers aren't going to mark clips as 24fps or 30fps because there is no fps, just timecodes. The exception to the are MPEG containers, but their methods are screwy and part of the problem.As you can now see in the end of my last post (you were probably writing your last one while I edited it) I agree with you that timecodes are enough info, provided that you also keep some sort of Max Framerate info on the container - this seems to be REALLY needed to convert back to CFR and to spare Avisynth from using a time duration flag internally for each frame, just for the modified duration ones. Time duration would be better inside Avisynth than timecodes, since frame processing is occuring.


A question: if Matroska is entirely timecode based how does it deal it video editing, like manual frame removing?

Bilu

[Atamido]

Originally posted by bilu
Frame duration:
(+) ability to change frame duration manually on video editor or delete frame without reencoding;
(+) ability to do better VFR->CFR conversions, framerates are easier to calculate;To clarify, you would have to "reencode" just the timecode. That shouldn't be that hard as you are going to handle all of them anyway.

The only time that VFR->CFR would be easier with only duration as opposed to only timecodes, would be if you only had one frame. Once you have two frames, you know the duration.Timecodes:
(+) ability to seek;
(+) ability to synchronize; This is a little confusing because you aren't defining what exactly the case is for. So frame durations are better for editing while timecodes are better for playback. If we need to use BOTH, I suggest we keep trace of the last modified property (durations or timecodes) and synchronize the other one. Synchronization should be made only after finished editing. All of this is only partially accurate. Durations are better for certain cases of editing. How do you know if the whole stream is supposed to be offset by 200ms using only durations? What about if you want a gap in the stream?

I would still go with timecodes, but like I said before, ALSO being able to store durations would be very nice. I'm not sure why you wouldn't synchronize all of the timecodes as you are processing the stream.

[bilu]

quote:Originally posted by bilu
Frame duration:
(+) ability to change frame duration manually on video editor or delete frame without reencoding;
(+) ability to do better VFR->CFR conversions, framerates are easier to calculate;

To clarify, you would have to "reencode" just the timecode. That shouldn't be that hard as you are going to handle all of them anyway.I still think that frame durations are nicer INSIDE Avisynth, as there is a lot of frame-dropping and duration changing when creating a VFR stream. So it's better NOT to rebuild the timecode again and again. It's better for the VFR muxer to convert frame durations into timestamps because the whole stream was already processed and we can assume that there will be less editing than during it's creation.

Timecodes would also work with VFR hybrids as source if you also have a Max Framerate info available. That's the only way to know if you have to create another dup after four 24 fps frames, since the original frame didn't kept the dup duration as opposed to VFR anime encodes. The Max Framerate is available in the VFW header before the VFR muxing, but the VFR container needs to keep that information.
The only time that VFR->CFR would be easier with only duration as opposed to only timecodes, would be if you only had one frame. Once you have two frames, you know the duration.
duh! :) Timecodes or frame durations, doesn't matter outside Avisynth. But don't forget the Max Framerate info! :D
quote:Timecodes:
(+) ability to seek;
(+) ability to synchronize;

This is a little confusing because you aren't defining what exactly the case is for.
Humm... Now you confused me :D I was hoping you would agree with this ;)
quote:So frame durations are better for editing while timecodes are better for playback. If we need to use BOTH, I suggest we keep trace of the last modified property (durations or timecodes) and synchronize the other one. Synchronization should be made only after finished editing.

All of this is only partially accurate. Durations are better for certain cases of editing. How do you know if the whole stream is supposed to be offset by 200ms using only durations? What about if you want a gap in the stream?

I would still go with timecodes, but like I said before, ALSO being able to store durations would be very nice. I'm not sure why you wouldn't synchronize all of the timecodes as you are processing the stream.Better to use timecodes outside Avisynth and frame durations inside, and the conversion to be made by the VFR muxer.
But I think delays would be even easier with durations - a fake frame with 200 ms at beginning :D

EDIT: Sorry, misunderstood. You wouldn't know with durations.
But wouldn't you have to re-render all timecodes when you remove a frame in VDMod ? Does it re-render timecodes from the frame before the removed one or from the beginning?


Why would you want a gap in the stream ????

Bilu

[Atamido]

Originally posted by bilu
You're right about the log not being very useful to transcode back to CFR, since you can get frame duration from the VFR streams, even if calculated from timecodes.
But you would need a Max Framerate info kept on the VFR container so that AVS would know it would have to add a dup after four 24 fps frames on VFR hybrid encodes. Or, you could create a filter or script to do that automatically very easily. There is no reason to keep that info if you know what you want to do. And as I said before, VDMod can usually do that automatically for storing in AVI.That's not VFR, that's something like Donald's Dup filter triggered by dropped frames. When it sees no frame, it copies the former frame again to stay in synch.
At least this is what I understood from your explanation.In Matroska it is VFR because the framerate halves anywhere there is a dropped frame. But, it only happens for one frame.

The "dropped frame" in AVI is not a copy like Donald's Dup filter. It is a recorded frame that means, "the data for this frame is missing". I think that when AVISynth reads a dropped frame from AVI it does duplicate it just like Donald's Dup filter, but VDMod just stores a "Dropped frame" in memory and writes that to the AVI. The codec never even sees it.As you can now see in the end of my last post (you were probably writing your last one while I edited it) I agree with you that timecodes are enough info, provided that you also keep some sort of Max Framerate info on the container - this seems to be REALLY needed to convert back to CFR and to spare Avisynth from using a time duration flag internally for each frame, just for the modified duration ones. Time duration would be better inside Avisynth than timecodes, since frame processing is occuring. What about just using the old Framerate property for Max Framerate? I don't think its really needed to convert back to CFR, but it seems that Framerate would need to be stored in a property somewhere in AVISynth.A question: if Matroska is entirely timecode based how does it deal it video editing, like manual frame removing? I'm not sure I understand your question.

If you want the same total length for a stream after removing a frame, then you just leave all of the timecodes the same for the other frames.

If you want the total length for a stream to be one frame shorter, than you subtract the duration of the frame(s) that you removed from the timecode of every frame after the frame(s) that you removed.

[bilu]

quote:Originally posted by bilu
You're right about the log not being very useful to transcode back to CFR, since you can get frame duration from the VFR streams, even if calculated from timecodes.
But you would need a Max Framerate info kept on the VFR container so that AVS would know it would have to add a dup after four 24 fps frames on VFR hybrid encodes.

Or, you could create a filter or script to do that automatically very easily. There is no reason to keep that info if you know what you want to do. And as I said before, VDMod can usually do that automatically for storing in AVI. Well, you're somewhat right: just had a dup after every four 24 fps frames and then make your own Decimate() decisions. But what about anime encodes? How do you know what was the original framerate to recreate the dups? Do you really think that a filter that would scan the entire VFR stream looking for the maximum framerate is worth it? quote:That's not VFR, that's something like Donald's Dup filter triggered by dropped frames. When it sees no frame, it copies the former frame again to stay in synch.
At least this is what I understood from your explanation.

In Matroska it is VFR because the framerate halves anywhere there is a dropped frame. But, it only happens for one frame.

The "dropped frame" in AVI is not a copy like Donald's Dup filter. It is a recorded frame that means, "the data for this frame is missing". I think that when AVISynth reads a dropped frame from AVI it does duplicate it just like Donald's Dup filter, but VDMod just stores a "Dropped frame" in memory and writes that to the AVI. The codec never even sees it. Not much use for that though. :) quote:As you can now see in the end of my last post (you were probably writing your last one while I edited it) I agree with you that timecodes are enough info, provided that you also keep some sort of Max Framerate info on the container - this seems to be REALLY needed to convert back to CFR and to spare Avisynth from using a time duration flag internally for each frame, just for the modified duration ones. Time duration would be better inside Avisynth than timecodes, since frame processing is occuring.

What about just using the old Framerate property for Max Framerate? I don't think its really needed to convert back to CFR, but it seems that Framerate would need to be stored in a property somewhere in AVISynth. Answered above. Framerate in Avisynth is retrieved from source headers, and I hope it still is on the new VFR sources ;)
Avisynth Framerate property = Max Framerate retrieved from container. quote:A question: if Matroska is entirely timecode based how does it deal it video editing, like manual frame removing?

I'm not sure I understand your question.

If you want the same total length for a stream after removing a frame, then you just leave all of the timecodes the same for the other frames.

If you want the total length for a stream to be one frame shorter, than you subtract the duration of the frame(s) that you removed from the timecode of every frame after the frame(s) that you removed. That's what I thought, rebuild the whole thing, or almost. That's why it's better outside Avisynth, it already spends lots of memory as it is. :D
But a good thing on the container though.


Bilu

[Atamido]

Originally posted by bilu
I still think that frame durations are nicer INSIDE Avisynth, as there is a lot of frame-dropping and duration changing when creating a VFR stream. So it's better NOT to rebuild the timecode again and again. If a filter does something that will affect all of the timecodes afterwards, it shouldn't have a problem altering all of the other timecodes.

Besides, if a filter wants to rearrage the order of frames, it is suddenly as easy as changing the timecodes.But I think delays would be even easier with durations - a fake frame with 200 ms at beginning That is practically blasphemy. Fake frames is nonsense. :devil:Why would you want a gap in the stream ???? Why would you want to prevent them? They are possible in existing containers, and being able to do subs in AVISynth would be fascinating for such a simple addition.

[Neo Neko]

Well some of the ideas could work but this is no small task and stands to create a number of hedaches in alot of areas. It will make synchronising audio and video after the fact a bigger pain than it already is. Any filter using temporal filtering like say oh temporal smoother perhaps will have to be re-worked. Clip layering will also have to be re-worked. The fade-in fade-out filters will give unpredictable results. And that is only the blatantly obvious stuff. Then there are the un-known possible ungodly number of hidden bugs waiting to crop up. And IMHO there are more important things to be done that stand to break less. Things like directshowsource with audio etc. Perhaps it should be put in the pipeline for a future version. Likely it already is though. ;)

[bilu]

--------------------------------------------------------------------------------
Originally posted by bilu
I still think that frame durations are nicer INSIDE Avisynth, as there is a lot of frame-dropping and duration changing when creating a VFR stream. So it's better NOT to rebuild the timecode again and again.
--------------------------------------------------------------------------------

If a filter does something that will affect all of the timecodes afterwards, it shouldn't have a problem altering all of the other timecodes.

Besides, if a filter wants to rearrage the order of frames, it is suddenly as easy as changing the timecodes.This won't work with random seeking inside Avisynth because the frames are still being processed. But this also happens with others filters right now. So changing timecodes for the next frames to be processed can be easily done in a normal encoding process.You're right.--------------------------------------------------------------------------------
Why would you want a gap in the stream ????
--------------------------------------------------------------------------------

Why would you want to prevent them? They are possible in existing containers, and being able to do subs in AVISynth would be fascinating for such a simple addition.Please explain how gaps can be used for subs and how that would work with existing filters. This looks like my former "complete timestamp mode" suggestion where filters would do frame skipping when timestamps=0ms.
As Bidoche said, it can't be done. As we found out later, it wasn't needed. ;)Well some of the ideas could work but this is no small task and stands to create a number of hedaches in alot of areas. It will make synchronising audio and video after the fact a bigger pain than it already is. Any filter using temporal filtering like say oh temporal smoother perhaps will have to be re-worked. Clip layering will also have to be re-worked. The fade-in fade-out filters will give unpredictable results. And that is only the blatantly obvious stuff. Then there are the un-known possible ungodly number of hidden bugs waiting to crop up. And IMHO there are more important things to be done that stand to break less. Things like directshowsource with audio etc. Perhaps it should be put in the pipeline for a future version. Likely it already is though. Any kind of temporal filtering should be avoided with anime VFR encoding. All frames will be different and no equal frames should be encoded. Never worked with clip layering, but if it is temporal same rule applies.
Fade in - fade out is another temporal filtering.

Just to remind you that this is mostly for DVD backup purposes and not to treat your own videos. CFR will still be good enough for them IMHO. DShow with audio is more important to you but less important to me.We all have different purposes. And I'd rather see a perfect hybrid movie encode than to use temporal filtering.

Best regards,
Bilu

[Bidoche]

Well, well, you left lots of things for me to answer...
One at a time.

@Pamel

Would you explain the 3.0 properties section a little better? I'm afraid that I've never understood it very well. If you have a property on one frame, do you need it on all of them? Can the value of one property affect the value of another property?
The word property may be confusing, but when I am talking about properties here, I mean extra properties, not the regular ones (width, height, color space..)
Each frame holds a property set to keep track of those extra properties, and couldn't care less about what the frame next door knows.
They cannot affect other properties directly, but one could achieve that by making obligatory Attach and Detach functions for his property.

I think it would be best to just assume that all clips are VFR. If you know they aren't, then it won't matter. If your destination won't support them, then you just avoid plugins/filters that would make the source VFR.Would break all filters who expect an underlying linear speed and that is many.

Using durations can be fun, but I would urge you to use timecodes instead. Impossible, frames exist out of context.
Timecodes do suppose a context (the containing clip), but the same frame can be a part of many clip. It has to be durations.

For instance, if the video has a delay to it, there would be no way to indicate this if the frames only had durations. Well, at the time being, there is already no way to indicate this, why should it be made possible ?

You have 3 frames: A, B, and C. AVISynth processes A and it goes out. Then it processes B and decides to drop it as a duplicate frame. When you process frame C it will end up in frame B's old place because frame A's duration wasn't changed. So your entire video after frame A is offset by one frame. With timecodes this would not have been an issue."Don't worry it can't happen, or at least it shouldn't unless a filter coder has screwed up.
Clips publish the frames count they intend to output, they can't decide to drop a frame in the middle. That frame is expected.
What they can do is mark it to be dropped later (the duration=0 thing), not drop it.

In this case AVISynth could then be used to process subtitle streams also. You have to develop that better.

I was under the impression that if a filter didn't know what a property was, it would just ignore it. And that's exactly how it works. Though the property could be discarded as a side effect. (Unless I can figure a smart way to prevent this (but sometimes it is just the good thing to do :( ))

I don't think that I see why you would want to do frame references. I am not the one talking about frame refs, bilu is.

[bilu]

Impossible, frames exist out of context.
Timecodes do suppose a context (the containing clip), but the same frame can be a part of many clip. It has to be durations.It's decided then: frame duration inside Avisynth, timecode in the VFR container, VFR muxer makes the translation. :) quote:You have 3 frames: A, B, and C. AVISynth processes A and it goes out. Then it processes B and decides to drop it as a duplicate frame. When you process frame C it will end up in frame B's old place because frame A's duration wasn't changed. So your entire video after frame A is offset by one frame. With timecodes this would not have been an issue."

Don't worry it can't happen, or at least it shouldn't unless a filter coder has screwed up.
Clips publish the frames count they intend to output, they can't decide to drop a frame in the middle. That frame is expected.
What they can do is mark it to be dropped later (the duration=0 thing), not drop it.
Humm... so frames can't be skipped but can be marked for dropping later... wouldn't conditional filtering provide the "frame-skipping" functionality to frames with duration=0ms if needed? We came to the conclusion that it isn't needed, but still it seems possible (I'm just curious if it would be possible, don't want to see it implemented - we agreed on a unique VFR flag).quote:I don't think that I see why you would want to do frame references.

I am not the one talking about frame refs, bilu is.It was only needed for the now obsolete "complete timestamp mode", to keep the dups in the timestamp log.
Already agreed we don't need that.


Bilu

[Bidoche]

Humm... so frames can't be skipped but can be marked for dropping later... wouldn't conditional filtering provide the "frame-skipping" functionality to frames with duration=0ms if needed? We came to the conclusion that it isn't needed, but still it seems possible (I'm just curious if it would be possible, don't want to see it implemented - we agreed on a unique VFR flag). Well, I guess you can skip frames, but :
- since frame count is fixed, you will have to come up with replacement frames sometimes.
- it's complicated : frame access is totally random, not streamed, so you must keep track of all your skip decisions.
It's must be doable to replace all frames to skip by black frames pushed at end of clip, but to gain what ?
Filters may not process those black frames faster.

A solution would be to recognize an already processed frame and reuse the result.
That may be done by attaching filter result to the source frame (as a property), so if it's come again, you have the result direclty..
But it can cause ugly reference cycles problems :
Memory leak if you attach a frame at itself, and with the size of frame (buffer) you can't really afford to leak them.

[bilu]

Then I'm glad we won't be needing it :D

EDIT: We shouldn't forget about the process to use on AVS source filters when using VFR sources. Every filter should be able to to get the timecode from the VFR container (Matroska or others) and translate it into the frame duration property.

I don't know if we're able to use MKVs as source on Avisynth, we probably have to demux them first. Then the best method should be a new filter that grabs the timecode from beginning and end of a frame, calculates duration and inserts it in frame property. This new filter should be run immediatly after the source filter.

When we're able to use MKVs as source ( I don't know if we can now ) then we can calculate the frame duration directly from the container data with the same filter.


Bilu

[gizmotech]

Ok,

Who here has heard of the chicken before the egg scenario?

I have... and in this case were trying to invent the object which will invent the chicken which will produce the egg.

Why don't we SIMPLIFY the process.

FIRST: Create the Problem. (VFR Files)
Second: Solve the Problem. (Reading VFR streams into AVS)
Third: Improve the Solution.

The biggest problem I see at the moment is you don't have the first problem to deal with. You're currently talking about how to use Avisynth on vfr sources when guess what, you don't have em to play with (and 120fps dropped frame avi's are not vfr atm, they're 120fps).

So as far as i can see it, lets first go about making VFR files from any source, and refine that part, THEN we can worry about all these other little oddities. Because at the moment for me to make 1 true VFR encode of say, Excel Saga, it will take me almost a week of cutting/muxing/encoding/merging/muxing to even create 1 30 minute section of VFR material. First refine the problem, then create a solution.

So really unless there is some productive discussion on the CREATION element of vfr files then this whole conversation is just a waste of time. I don't mean to be rude, but this discussion really seems immensely pointless until you've got a problem to solve. It's like figuring out how you're gonna do diplomacy with aliens... When you don't even have the aliens to do diplomacy with anyways.

Gizmo.

For the record: My current process to make true vfr encodes is as follows:
a) encode one version in 24fps
b) encode second version in 30fps
c) cut into ittybitty little pieces which correspond to the different framerate sections
d) adjust timecodes on one of the 2 frame rates to match the other
e) merge adjusted files
f) encode whole file (to achieve distributed bitrate over all frames)
g) split file back into itty bitty pieces
h) readjust frame rates back to normal
i) merge w/ avi-mux-gui into mkv
Result: True VFR 24/30fps hybrid mpeg4 encode. no modifications to avs.

[Atamido]

Originally posted by gizmotech
The biggest problem I see at the moment is you don't have the first problem to deal with. You're currently talking about how to use Avisynth on vfr sources when guess what, you don't have em to play with (and 120fps dropped frame avi's are not vfr atm, they're 120fps). I think you must have missed something. The original MPEG is VFR. People will use the trick of speeding up the video to 120fps and adding dropped frames to make make the 30fps and 24fps video segments match up in AVI. But, if you worked directly with the MPEG-2, then you have a VFR source. I want to work directly with the MPEG-2 and not worry about the goofy 120fps hack. And both DVDs and Digital broadcasts are excellent sources for VFR MPEG-2.

This brings us to the second problem, how to read these VFR files into AVS. Thats not hard, you just need to modify the mpeg-2 source plugin. You could also do it with a modified DirectShowSource plugin.

All anyone was really going for was a way to use the AVISynth filtering/scripting system with a VFR source. In the end you may end up making DS input and output plugins.

[gizmotech]

Ok,

I don't see how you can say the original mpeg-2 is VFR, because it isn't VFR until you perform the IVTC on it. Unless of course you've got actual streamed mpeg2 which contains true 30fps and 24fps footage that has not been IVTC'd.

Either that or you're talking about streaming already selectively IVTC'd mpeg2 into avs...

And as for using the scripting system, Yes it will be a shame that a number of filters will be unable to work now, however I must be confused as to which part of avs will be broken by this, because I don't know of any filters that I currently use which require knowledge of time, just knowledge of the next and previous frames. (Could these be audio components which will be broken?)

Gizmo