Welcome to Doom9's Forum, THE in-place to be for everyone interested in DVD conversion.

Before you start posting please read the forum rules. By posting to this forum you agree to abide by the rules.

 

Go Back   Doom9's Forum > Capturing and Editing Video > New and alternative a/v containers

Reply
 
Thread Tools Search this Thread Display Modes
Old 19th October 2003, 02:51   #1  |  Link
bilu
Registered User
 
bilu's Avatar
 
Join Date: Oct 2002
Location: Portugal
Posts: 1,182
@VFW/DShow experts: VFR in Avisynth could be good news for Matroska

Hi,

I've been trying to push the variable framerate subject in Avisynth and we're still trying to figure out if it's doable.

How about an Avisynth fork or extender for Variable Framerate?
http://forum.doom9.org/showthread.php?s=&threadid=63290

Containers like MP4, Matroska and Real Video all support variable framerate, but nobody seems to be trying to work out the subject, except Real Video on a anime filter.

Advantages for VFR:
- Perfect hybrid (30 fps Video + 24 fps Film) movies, without blending 24 fps->30 fps or decimating 30 fps->24fps;
- Much less frames on anime, speeds encoding and gains bitrate;

Disadvantages for VFR:
- AVI and OGM unsupported;
- Questionable video editing capacity?


This post is to request help pushing Avisynth in this useful direction by providing some of your VFW/DShow knowledge. As a non-programmer it is my duty to beg

Only then will this be more than experimental in containers.


Best regards,
Bilu

Last edited by bilu; 19th October 2003 at 02:58.
bilu is offline   Reply With Quote
Old 19th October 2003, 08:27   #2  |  Link
Koepi
Moderator
 
Koepi's Avatar
 
Join Date: Oct 2001
Location: Germany
Posts: 4,454
Well, it's not necessary to have VFR in my oppinion.

In xvid we have a feature called "frame drop" - try it. Encode your video at i.e. 60 fps, set frame drop ratio somewhere to 50%... you should prefilter the sources a bit so that the frames are really a little close.

So there you have something very similar to vfr without breaking compatibility, editibility and all tools we are used to.

Regards
Koepi
Koepi is offline   Reply With Quote
Old 19th October 2003, 11:49   #3  |  Link
bilu
Registered User
 
bilu's Avatar
 
Join Date: Oct 2002
Location: Portugal
Posts: 1,182
@Koepi



But how do you know which frames to drop?

There are two main goals for VFR, and if framedrop works it's only for one of them.

1) Anime. You drop a duplicate but double the play time of the original frame, keeping the framerate.

2) Hybrid movies. You don't drop frames in the video parts (pure 30 fps) but drop duplicates in the film part (telecined 24fps -> 30 fps) to achieve pure 24 fps, and on this segment you DON'T want to keep the original framerate.

Conclusion: XVID's framedropping can probably work for anime. To be something more it would have to include some sort of IVTC filtering to know Telecine parts and decide to drop without keeping framerate.

I'm trying your method over the Star Trek Hybrid clip just to prove it.

That's why VFR in Avisynth would benefit all VFR supporting encoders.
XVID is MPEG-4 and would benefit under Matroska and MP4 containers (AFAIK OGM doesn't support VFR).

EDIT: The result sucked :/ Framedrop was dropping much more than just duplicates. I used frame drop ratio 50 as you said.

Bilu

Last edited by bilu; 19th October 2003 at 12:09.
bilu is offline   Reply With Quote
Old 19th October 2003, 12:13   #4  |  Link
bilu
Registered User
 
bilu's Avatar
 
Join Date: Oct 2002
Location: Portugal
Posts: 1,182
Quote:
Originally posted by Koepi
So there you have something very similar to vfr without breaking compatibility, editibility and all tools we are used to.

You don't have to break compatibility
Inserting a special mode into decombers and an internal flag into Avisynth doesn't break compatibility. You're not forced to use this new modes and flags.

But you give users a free choice that can work as good as MotionPerfect on hybrid encodes


Bilu
bilu is offline   Reply With Quote
Old 19th October 2003, 13:38   #5  |  Link
Koepi
Moderator
 
Koepi's Avatar
 
Join Date: Oct 2001
Location: Germany
Posts: 4,454
Wow. you tested a single suggested value and tell us it sucks?

I expected a little more from you, try some lower values, that was just an example. It should work quite well. Try 10% or something alike.

regards
Koepi
Koepi is offline   Reply With Quote
Old 19th October 2003, 13:50   #6  |  Link
Neo Neko
Registered User
 
Neo Neko's Avatar
 
Join Date: Mar 2002
Location: Kansas City, Missouri
Posts: 1,812
Well the lower the values the less noticable it is. I'll give you that. I have used this feature on more than one occasion though myself. And I wonder if there is not something that could be done to improve the placement of drop frames. Best I can tell even on a 2-pass encode Xvid decides on the fly what to drop. And if it is trying to match a percentage it could drop important frames. :P Perhaps would it be possible to tie it into a 2-pass encode for better placement after the stream has been anylised. Or am I mistaken and this already done?
__________________
Opensource will not take over the world. But it will sure improve the lives of most of it!
_______________________________________________
Inkscape - Scallable Vector Graphics for everyone.
The GIMP - Free raster graphics and photo editing software.
Planeshift - Free 3D MMORPG. Cause everyone needs some fun.
Neo Neko is offline   Reply With Quote
Old 19th October 2003, 14:58   #7  |  Link
Koepi
Moderator
 
Koepi's Avatar
 
Join Date: Oct 2001
Location: Germany
Posts: 4,454
Nah, framedrop is very (pre)alpha, it's just a proof-of-concept. The error you encounter on 2pass should be corrected with dev-api-4 as there the full control over the 2nd pass is given unlike in current dev-api-3 with bframes != -1.

Best regards
Koepi
Koepi is offline   Reply With Quote
Old 19th October 2003, 15:21   #8  |  Link
Atamido
Seņor Member
 
Atamido's Avatar
 
Join Date: May 2002
Location: Austin, Texas
Posts: 915
I've always been a big fan of modularity myself. I think that you could get the frame dropping in XviD to be pretty good. But, as a standalone filter, it would be much more useful. You could use it with any codec, and it could be updated seperately.

I think that adding a VFR ability to something like AVISynth would be VERY useful, especially when dealing with hybrid clips.
Atamido is offline   Reply With Quote
Old 20th October 2003, 00:05   #9  |  Link
bilu
Registered User
 
bilu's Avatar
 
Join Date: Oct 2002
Location: Portugal
Posts: 1,182
Quote:
Originally posted by Koepi
Wow. you tested a single suggested value and tell us it sucks?

I expected a little more from you, try some lower values, that was just an example. It should work quite well. Try 10% or something alike.

regards
Koepi
I will, sorry for the rushed answer , but my goal really is enjoying VFR on every codec I want to test that supports that feature. But I promiss I'll do serious testing next time.

BTW, do you know is this feature is being revised for dev-api-4?
I would think developers could look to it as less important...

EDIT: Another question: how would you know the percentage of dups to drop on a hybrid film? Would it be worth knowing?

Sorry about that again. :P

Bilu

Last edited by bilu; 20th October 2003 at 01:06.
bilu is offline   Reply With Quote
Old 20th October 2003, 03:24   #10  |  Link
Neo Neko
Registered User
 
Neo Neko's Avatar
 
Join Date: Mar 2002
Location: Kansas City, Missouri
Posts: 1,812
Well it would almost not be worth it trying to calculate the dupe percentage. So many things could effect it and even mess it up. Bad sceen transitions from telecined to interlaced. Or tiny interlaced overlays on large telecined backdrops. What would be ideal is instead of setting a percentage of frames to be dropped which is as I understand how the current frame drop works is to set a threshold at which it should drop frames. And it will drop all frames that are either below that threshold or match other search criteria. This is somewhat how I believe it already works. But I think it would be better to let the algo simply search for positives than force false positives with a frame percentage. But as Keopi says it is a proof of concept. And dev4-api performs quite nicely BTW. Though I must say that recent ffvfw builds are giving everyone a run for the money.
__________________
Opensource will not take over the world. But it will sure improve the lives of most of it!
_______________________________________________
Inkscape - Scallable Vector Graphics for everyone.
The GIMP - Free raster graphics and photo editing software.
Planeshift - Free 3D MMORPG. Cause everyone needs some fun.
Neo Neko is offline   Reply With Quote
Old 20th October 2003, 03:40   #11  |  Link
Neo Neko
Registered User
 
Neo Neko's Avatar
 
Join Date: Mar 2002
Location: Kansas City, Missouri
Posts: 1,812
Oh BTW. IIRC doesn't the dupe filter for AVS (By Marc FD IIRC) do more or less what you are describing bilu? Replacing nearly duplicate frames with an exact duplicate according to it's threshold? It is more up to the codec and container involved how VFR would be handled. So tying it to AVS while it sounds nice does not make perfect sense since avisynth is not geared to a single codec or output format. Which I think is more or less what koepi is trying to get at.
__________________
Opensource will not take over the world. But it will sure improve the lives of most of it!
_______________________________________________
Inkscape - Scallable Vector Graphics for everyone.
The GIMP - Free raster graphics and photo editing software.
Planeshift - Free 3D MMORPG. Cause everyone needs some fun.
Neo Neko is offline   Reply With Quote
Old 20th October 2003, 09:46   #12  |  Link
bill_baroud
Registered User
 
Join Date: Feb 2002
Posts: 407
Dupe is a filter by Donald Graft, which copy frames which are below a defined threshold (diff from the previous frame). The xvid frame drop feature seems to be something like this too, but it's not what Bilu want.
bill_baroud is offline   Reply With Quote
Old 20th October 2003, 10:29   #13  |  Link
bilu
Registered User
 
bilu's Avatar
 
Join Date: Oct 2002
Location: Portugal
Posts: 1,182
a practical case of a VFR filter

This is a practical case of a VFR filter:

RV9 Animation DropDupe Pre-filter
http://forum.doom9.org/showthread.php?s=&threadid=56564

It's not Avisynth, but you get the general idea

@Neo Neko

Quoting myself:
Quote:
There are two main goals for VFR, and if framedrop works it's only for one of them.

1) Anime. You drop a duplicate but double the play time of the original frame, keeping the framerate.

2) Hybrid movies. You don't drop frames in the video parts (pure 30 fps) but drop duplicates in the film part (telecined 24fps -> 30 fps) to achieve pure 24 fps, and on this segment you DON'T want to keep the original framerate.
So as you see there are two different objectives for VFR and to include such decision in a codec means inserting an IVTC filter into it.

Moreover, if you check that RV9 Dropdupe filter thread I linked to, you'll see they recommend Donald Graft's Dup to help their own filter, complementary and not replacement. Advantages still exist for their filter: faster encode, smaller bitrate.


Bilu
bilu is offline   Reply With Quote
Old 20th October 2003, 20:44   #14  |  Link
Neo Neko
Registered User
 
Neo Neko's Avatar
 
Join Date: Mar 2002
Location: Kansas City, Missouri
Posts: 1,812
Quote:
Originally posted by bill_baroud
Dupe is a filter by Donald Graft, which copy frames which are below a defined threshold (diff from the previous frame).
DOH! Sleep is good. Brain funktion impared without slep.
__________________
Opensource will not take over the world. But it will sure improve the lives of most of it!
_______________________________________________
Inkscape - Scallable Vector Graphics for everyone.
The GIMP - Free raster graphics and photo editing software.
Planeshift - Free 3D MMORPG. Cause everyone needs some fun.
Neo Neko is offline   Reply With Quote
Old 20th October 2003, 20:59   #15  |  Link
Neo Neko
Registered User
 
Neo Neko's Avatar
 
Join Date: Mar 2002
Location: Kansas City, Missouri
Posts: 1,812
Quote:
Originally posted by bilu

1) Anime. You drop a duplicate but double the play time of the original frame, keeping the framerate.
The dupe filter is as close as you are going to get on this in AVIsynth. Any functionality beyond that should be handled by the codec itself.

Quote:
Originally posted by bilu

2) Hybrid movies. You don't drop frames in the video parts (pure 30 fps) but drop duplicates in the film part (telecined 24fps -> 30 fps) to achieve pure 24 fps, and on this segment you DON'T want to keep the original framerate.
Again that is more something that is going to have to be done at the codec level. AVIsynth synthesizes a fake AVI. AVI can't have varriable frame rates. If this were to work And I don't know what current software would if it did Avisynth would have to be forked into a more general videosynth program that outputs to it's own format so as not to be limited in immatating a limited format. This would instantly break support in all AVS supporting software for that fork.

Quote:
Originally posted by bilu

Conclusion: XVID's framedropping can probably work for anime. To be something more it would have to include some sort of IVTC filtering to know Telecine parts and decide to drop without keeping framerate.
What's wrong with using nearly null pading frames as placeholders to keep constant frame rate with fewer unique frames? It takes slightly more space than nothing at all. But it would have to be like a 12 hour movie for the null frames to eat up significant bitrate that even then would be nothing in relation to the final size of the film.
__________________
Opensource will not take over the world. But it will sure improve the lives of most of it!
_______________________________________________
Inkscape - Scallable Vector Graphics for everyone.
The GIMP - Free raster graphics and photo editing software.
Planeshift - Free 3D MMORPG. Cause everyone needs some fun.
Neo Neko is offline   Reply With Quote
Old 20th October 2003, 22:24   #16  |  Link
bilu
Registered User
 
bilu's Avatar
 
Join Date: Oct 2002
Location: Portugal
Posts: 1,182
Quote:
The dupe filter is as close as you are going to get on this in AVIsynth.
Any functionality beyond that should be handled by the codec itself.
Look at the RV9 example: gains bitrate, speeds encoding a lot.
Quote:
Again that is more something that is going to have to be done at the codec level.
AVIsynth synthesizes a fake AVI. AVI can't have varriable frame rates. If this were to
work And I don't know what current software would if it did Avisynth would have to be
forked into a more general videosynth program that outputs to it's own format so as not
to be limited in immatating a limited format. This would instantly break support in all
AVS supporting software for that fork.
Then why don't you explain my what does OutputDebugString used by Telecide(debug=true)
has to do with VFW?? Has it broken compatibilities?

What I'm trying to say, and have already discussed the issue with Bidoche before ( see my
1st post ) is that Avisynth exports data through the VFW interface ( not allways, as I've
shown you, not to mention text logs) and I'm trying to find out if frame duration could be
sent in another stream (just like you send audio through Avisynth) and then a DShow Splitter
would act on the Video stream based on the information of this frame duration stream.

Could be done easier by using text logs, do a XVID null encode with an AVS script with
VFR-prepared filters that would export such info, and then load the source script and log
into a VFR prepared encoder.

But that is a bit primitive, don't you think? And the DShow splitter option would bring VFR
power to every app that supports DShow sources and is VFR capable.
Quote:
What's wrong with using nearly null pading frames as placeholders to keep constant
frame rate with fewer unique frames? It takes slightly more space than nothing at all. But
it would have to be like a 12 hour movie for the null frames to eat up significant bitrate that
even then would be nothing in relation to the final size of the film.
It's not just about bitrate. It's mainly about speed gain in anime encodes and ***PERFECT***
( as in better than MotionPerfect ) hybrid movie encodes.

I'm trying to find ways that don't break compatibilities. As Bidoche said, frame duration
flags are easy. And IMHO create an IVTC filter that modifies duration flags instead of blending
frames is easier to develop than traditional IVTC filters and faster.
Quote:
VFR: AVS -> VFW + frame duration stream -> VFR DShow splitter -> VFR rendering

CFR: AVS -> VFW + frame duration stream(a) -> CFR DShow splitter(b) -> CFR rendering

(a) frame duration stream would be optional and activated by a flag in Avisynth. If not available,
even the VFR DShow splitter would do CFR rendering;

(b) if the CFR DShow splitter received such a stream it would ignore it;
Still think it's undoable, breaks compatibility and not worth it?


EDIT: Older discussions I found on this subject

Variable frame rate support
http://forum.doom9.org/showthread.php?s=&threadid=47203

VFR in Avisynth 3.0?
http://forum.doom9.org/showthread.php?s=&threadid=53028

3.0 Wip
http://forum.doom9.org/showthread.php?s=&threadid=48940


Bilu

Last edited by bilu; 20th October 2003 at 22:46.
bilu is offline   Reply With Quote
Old 21st October 2003, 07:16   #17  |  Link
Neo Neko
Registered User
 
Neo Neko's Avatar
 
Join Date: Mar 2002
Location: Kansas City, Missouri
Posts: 1,812
In short yes. But that is just my oppinion.

The biggest thing AVS does for me is bring support to VFW based programs for codecs and formats that only Directshow supports or reading in the odd DVD. So having to use a directshow based program to make use of it already limits the features usability and if I understand correctly would indeed break compatability.

If I understand your sped up encoding claim you are claiming that changing a frames duration means the frame would only have to be filtered once and not twice if only 3:2 pulldown and no decimation etc was done. All I can say is you would have to be doing some heavy duty filtering for it to make a large difference. And it would not speed up encoding that much especially in comparrison with Xvid's frame drop. Actualy in conjunction with Xvids frame drop if there were not some communication between AVS and the codec itself about which frames were already calculated to drop the calculations would have to be done again slowing down the encoding even more. And how in VFW and directshow are you going to acomplish that?!

If someone were going to do this. And I mean do this right. Then an end to end AVS solution would be best. Then external program issues would be relatively trivial. But before that can happen a few things would need to be done first.

1. Have directshow source and other source plugins handle audio uniformly much like AVISOURCE. Which I hear is in the works anyway.

2. Have codecs reworked as plugins for AVS itself. You would not be able to divx with it unless by some twist of fate they supported it or there were some wrapper shenanegins. But Xvid, ffmpeg, and even VP3 since the source code is avalible could be created as loadable modules

3. Finally include muxing filters whether via directshow or loadable filters and a file writer.

Then you could do scripts like this.
Quote:
loadplugin("c:\avs\mpeg2dec3.dll")
loadplugin("c:\avs\vfr.dll")
loadplugin("c:\avs\ffmpeg.dll")
loadplugin("c:\avs\lame.dll")
loadplugin("c:\avs\mkvmuxer.dll")
movie = mpeg2source("c:\path to\some.d2v")
movie.vfr(8)
movie.lanczosresize(480x480)
movie.ffmpeg(3,2,31,2,31,620000,MPEG,"c:\path to\movie.stats",4:3)
movie.lame(alt-preset-medium)
movie.mkvmuxer(4:3, "c:\path to\chapters.ogg", Title="My Movie", Author="Me")
filesink("c:\path to\movie.mkv")
return null
But even that would extend the software far beyond the scope it currently occupies. Not that it would be a bad thing. Keep in mind it would not be an easy thing. And it's benefit would be limited and would not justify a rewrite of almost everything in Avisynth to handle it. It sounds great in theory. But I think logistics could be killer.
__________________
Opensource will not take over the world. But it will sure improve the lives of most of it!
_______________________________________________
Inkscape - Scallable Vector Graphics for everyone.
The GIMP - Free raster graphics and photo editing software.
Planeshift - Free 3D MMORPG. Cause everyone needs some fun.
Neo Neko is offline   Reply With Quote
Old 21st October 2003, 18:39   #18  |  Link
Atamido
Seņor Member
 
Atamido's Avatar
 
Join Date: May 2002
Location: Austin, Texas
Posts: 915
I am a little split on how to accomplish this.

VFR support in some sort of complex encoding system like AVISynth is needed because there are VFR sources in the wild right now. Obviously, VFW applications could not support that feature, which limits you to DirectShow applications.

Also, while I do believe that the codec should be able to alter the framerate, drop or add frames, I also believe that handling this outside of the codec would be better as it would make it possible for any codec to make use of it.

If so much of AVISynth would have to be edited, then it would probably be much more feasible to write an AVISynth compatible system from scratch that was designed to handle those extra features. An AVISynth plugin wrapper for DS shouldn't be that hard to produce, but you would loose all of the advanced scripting features of AVISynth.
Atamido is offline   Reply With Quote
Old 21st October 2003, 21:15   #19  |  Link
Neo Neko
Registered User
 
Neo Neko's Avatar
 
Join Date: Mar 2002
Location: Kansas City, Missouri
Posts: 1,812
Frame dropping and altering frame durration outside the codec is utterly useless if the codec itself does not support it. If the codec does not support it the best you will ever get is telecining and then running the dupe filter. Most codecs require constant framerate. So even if AVS could flag a frame to have a durration 10 times longer than that of an average frame pretty much every VFW and directshow codec in existance is gonna encode 10 identical frames in place of the 1 longer one. Useless. The speed up on filtering would also be negligable. And in order to get any speed increase on filtering you are gonna have to make the filters both internal and external VFR aware at least in some small way.

It sounds like a good idea. But I am having troubles seeing the real benefit at the moment and a justification that could convince the AVS developers to re-write large portions of their code.
__________________
Opensource will not take over the world. But it will sure improve the lives of most of it!
_______________________________________________
Inkscape - Scallable Vector Graphics for everyone.
The GIMP - Free raster graphics and photo editing software.
Planeshift - Free 3D MMORPG. Cause everyone needs some fun.
Neo Neko is offline   Reply With Quote
Old 22nd October 2003, 01:02   #20  |  Link
Atamido
Seņor Member
 
Atamido's Avatar
 
Join Date: May 2002
Location: Austin, Texas
Posts: 915
Quote:
Originally posted by Neo Neko
Most codecs require constant framerate.
Most of that post is nonsense, and here is the root of the problem with it. I can't think of a codec that DOES know about framerate. Codecs today are entirely independant of framerate or the timecodes of a frame*.

In VFW, a frame is passed to the codec and data for that frame is passed out the other end. The only thing the codec can know in this case is how many frames it has been passed. It doesn't even know if there is a dropped frame in the stream because that data isn't passed to it. To quote the developer of VirtualDubMod:**

[13:43] <Cyrius> well at least VirtualDub doesn't send the dropped frame to the codec
...and...
[13:51] <Cyrius> no frame number is sent with VfW
[13:51] <Cyrius> so the codec doesn't know anything
[13:51] <Cyrius> except that you are giving a frame

DirectShow is a little different in that frame(s) are passed to the codec and data's are passesd out. Also, there is a timecode attached to each frames. And while it is theoretically possible for a codec to change a timecode, or not output one of the data, none do. All current codecs simply pass the timecode information through with each frame.

So, assuming you did the same thing in AVISynth, filters could either be aware of the timecode or not. If they are then they can drop out a frame or change the timecode. If they aren't then they just ignore it and pass on the extra frame data. Like the codecs in DS, most filters would never need to know what the timecode is.

*(The only exceptions that I can think of would be MPEG-2 but that is a different story, and FLASH as it is technically a CODEC that renders VFR where VFR=however fast it can render on a particular CPU.) **(The entire IRC conversation on this very topic is available upon request.)
Atamido is offline   Reply With Quote
Reply

Thread Tools Search this Thread
Search this Thread:

Advanced Search
Display Modes

Posting Rules
You may not post new threads
You may not post replies
You may not post attachments
You may not edit your posts

BB code is On
Smilies are On
[IMG] code is On
HTML code is Off

Forum Jump


All times are GMT +1. The time now is 14:44.


Powered by vBulletin® Version 3.8.11
Copyright ©2000 - 2024, vBulletin Solutions Inc.