Welcome to Doom9's Forum, THE in-place to be for everyone interested in DVD conversion.

Before you start posting please read the forum rules. By posting to this forum you agree to abide by the rules.

 

Go Back   Doom9's Forum > Capturing and Editing Video > Avisynth Development

Reply
 
Thread Tools Search this Thread Display Modes
Old 28th May 2012, 15:54   #1  |  Link
Aktan
Registered User
 
Join Date: Feb 2002
Posts: 303
VFRtoCFR

Hi all! From my testing, it seems FFMS2 and Dss2 were slightly inaccurate when converting variable frame rate to constant frame rate videos in MKV. So I decided to write my own plugin. It still requires something like FFMS2 to retrieve all the frames without dups/drops and to grab the timecodes (though mkvextract could also do this). I hope this will be useful to someone.

Code:
VFRtoCFR

Converts a variable frame rate (VFR) video to a constant frame rate (CFR) video with the help of Matroska Version 2 Timecodes.

Usage:

VFRtoCFR(clip c, string times="times.txt", int numfps=30000, int denfps=1001, bool dropped=false)

Where:

c	The clip to convert from VFR to CFR
times	The path to  Matroska timecodes (v2)
numfps	The numerator of the CFR
denfps	The denominator of the CFR
dropped	If true, it will throws an error if there are frames dropped in the conversion.  Good to figure out if the CFR is set too low.

Changelog:

2012/07/30:
	-Rewrote the whole algorithm to be a lot smarter

2012/05/28:
	- Initial release
Download: http://www.mediafire.com/download.php?q6zfgpo5dhh50si

Edit: I'm not sure I posted this in the right forum. If not could some mod move it to the correct place?
Edit 2: Clarified that the inaccuracies of FFMS2 and Dss2 is only slight.

Last edited by Aktan; 30th July 2012 at 20:07. Reason: Updated release
Aktan is offline   Reply With Quote
Old 28th May 2012, 19:31   #2  |  Link
TheFluff
Excessively jovial fellow
 
Join Date: Jun 2004
Location: rude
Posts: 1,100
in what way was ffms2 "inaccurate"? would it really have been too hard to file a bug report or at least mention the issue to its developers?
TheFluff is offline   Reply With Quote
Old 29th May 2012, 01:35   #3  |  Link
Aktan
Registered User
 
Join Date: Feb 2002
Posts: 303
FFMS2 is only slightly inaccurate in that it is off by half a frame, but depending on the current framerate, the delay may be huge. Take the following example:

Code:
frame number	timecode (in ms)
0		0
1		1000
2		2000
3		3000
If you set the fpsnum to 1000 and fpsden to 1 (basically 1000 FPS), frame 3 would show up at time 2500 ms. While this is an extreme example, it is still off by 500 ms. Dss2 is off by 1 frame meaning frame 3 would get a time of 2000 ms.

As to why not a bug report, well I didn't want to wait for it to be fixed, but I can write one if you want. I also had fun programming this, so might as well. Another thing was I couldn't tell if the framerate I choose in FFMS2 dropped frames or not.

Last edited by Aktan; 29th May 2012 at 02:12.
Aktan is offline   Reply With Quote
Old 29th May 2012, 12:11   #4  |  Link
natt
Registered User
 
Join Date: Mar 2012
Posts: 10
Quote:
Originally Posted by TheFluff View Post
in what way was ffms2 "inaccurate"? would it really have been too hard to file a bug report or at least mention the issue to its developers?
http://doom10.org/index.php?topic=25.msg9997#msg9997

Covers the specific issue mentioned here; that FFMS treats the timecode as a sort of 'centerpoint' in a frame, instead of the beginning of the frame.

Aktan: Guess you didn't like my TimecodeFPS plugin?
natt is offline   Reply With Quote
Old 29th May 2012, 12:31   #5  |  Link
Aktan
Registered User
 
Join Date: Feb 2002
Posts: 303
I had no idea it existed really. I was more interested in having fun writing a plugin, lol.

Edit: Oh wait, I did know, oops, I forgot, sorry.

Last edited by Aktan; 29th May 2012 at 12:34.
Aktan is offline   Reply With Quote
Old 29th May 2012, 13:14   #6  |  Link
Aktan
Registered User
 
Join Date: Feb 2002
Posts: 303
Also, while FFMS2 is only off by half a frame that will go pretty much unnoticed in higher FPS, it's still off. One real example it will be very obvious is a VFR slideshow video changing frames depending on the audio.
Aktan is offline   Reply With Quote
Old 29th May 2012, 15:00   #7  |  Link
TheFluff
Excessively jovial fellow
 
Join Date: Jun 2004
Location: rude
Posts: 1,100
I'd argue that it's not "off", it's just a different way of converting. For each frame in the output, FFMS2 uses the source frame that starts closest to the target timestamp. You want it to use the first frame that has a timestamp that is greater than or equal to the requested timestamp instead. In my opinion that's a feature request, since I don't see the current implementation as buggy. It might not do what you want, but it's not strictly wrong either.

For many cases, the FFMS2 implementation will almost always pick frames that are much less "off" time-wise than the one you suggest will. The issue, I guess, is that while the strictly equal-or-greater algorithm may pick frames that are at worst "off" by (almost) one output frame duration, the closest-source-timestamp algorithm will at worst pick frames that are 0.5 input frame durations "off". If the maximum input frame duration * 0.5 is smaller than the output frame duration, closest-source-timestamp will always choose better.

I use "off" within quotation marks above because the term is arguable. I guess since you chose the algorithm you did and because of your slideshow example that you consider any frame that is displayed before its source timestamp says it should be displayed to be off, while a frame that is displayed later than its source timestamp might not necessarily be considered "off" to you. I'm not sure if that's the best way to approach framerate conversions in general, though. It works better for your example, sure, but as stated above it has problems with other funny inputs (more specifically, if the output frame duration is longer than the maximum input frame duration, i.e. you're dropping frames).

Last edited by TheFluff; 29th May 2012 at 15:02.
TheFluff is offline   Reply With Quote
Old 29th May 2012, 15:05   #8  |  Link
sneaker_ger
Registered User
 
Join Date: Dec 2002
Posts: 5,565
But 500ms (or in other words 500(!) output-frames) off is wrong however you look at it. Whether strict equal-or-greater is better or not is a different question.
sneaker_ger is offline   Reply With Quote
Old 29th May 2012, 16:37   #9  |  Link
Aktan
Registered User
 
Join Date: Feb 2002
Posts: 303
I still think that choosing a frame under the timestamp is still incorrect as the original playback would not have the chosen frame shown at that time. Take for example a source area of 30000/1001 FPS. Lets say the timestamps of frames in ms were:

Code:
...
29963
29997
30030
30063
30097
...
Let's say the extreme case of setting the CFR to be 1 FPS so there would be a frame at timestamp 30000ms. With FFMS2, the frame at timestamp 29997ms would be chosen, but I say any frame from timestamp 30000ms - 39999ms would be better off since at 1 FPS the frame at timestamp 30000ms is shown from 30000ms - 39999ms. Currently, VFRtoCFR unfortunately chooses the frame at 30998ms, which is a pretty bad choice when the frame at 30030ms is clearly a better choice (I mention this in the limitations above and may fix this in the future). Though there are a ton of dropped frames in this example, so really practically in terms of view, as long as it is consistent, probably all are okay (lol).

I think I can see how this is of opinion which is why, TheFluff, since you consider this a feature request, instead of a bug, it was a good thing I wrote my own then, right?

Last edited by Aktan; 29th May 2012 at 16:43.
Aktan is offline   Reply With Quote
Old 29th May 2012, 18:21   #10  |  Link
vdcrim
Registered User
 
Join Date: Dec 2011
Posts: 192
Quote:
Originally Posted by TheFluff View Post
If the maximum input frame duration * 0.5 is smaller than the output frame duration, closest-source-timestamp will always choose better.
So why not just choose between the closest and the previous frame using this? Though it would be better to decide between the two approaches in a per frame basis instead of just using the max VFR duration.
vdcrim is offline   Reply With Quote
Old 29th May 2012, 18:42   #11  |  Link
Aktan
Registered User
 
Join Date: Feb 2002
Posts: 303
I think I have a better solution, why not best of both worlds? When dropping, use timestamp as center, when duping, use timestamp as start.
Aktan is offline   Reply With Quote
Old 29th May 2012, 19:26   #12  |  Link
Aktan
Registered User
 
Join Date: Feb 2002
Posts: 303
Wow, I must have had a brainfart when writing the post with dropping frames. Take the example again with frames timestamp in ms:

Code:
...
29963
29997
30030
30063
30097
...
VFRtoCFR would choose 29997 like FFMS2 because at time 30000ms the frame at 29997ms would still be showing. Now take this example (again frames timestamp in ms):

Code:
...
968
1001
...
For 1 CFR with a frame at time 1000ms VFRtoCRF would choose the frame at 968ms since it would normally still be showing at that time, but FFMS2 would choose the frame at 1001 ms, which I deem incorrect (opinion of course).

In general, what VFRtoCFR does is this:

Given a CFR frame timestamp:

1. Find the first VFR frame with a timestamp greater than the CFR fame.
2. Take the previous VFR frame as the output for that CFR frame.
Aktan is offline   Reply With Quote
Old 29th May 2012, 19:55   #13  |  Link
Aktan
Registered User
 
Join Date: Feb 2002
Posts: 303
I have another proposed solution that I may implement in VFRtoCFR instead:

Given a CFR frame:

If the frame time fits completely inside a VFR frame time, choose that VFR frame. Example:

A CFR frame has a timestamp playback from 1 ms to 2 ms which fits completely inside a VFR frame that has a timestamp playback from 0 ms to 1000 ms

Else if there is a VFR frame time that takes most of the CFR frame time, choose that VFR frame. Example:

A CFR has a timestamp playback from 1000 ms to 2000 ms with 3 VFR frames to choose from. The 3 VFR frames have the following timestamps:

1. 999 ms to 1010 ms
2. 1010 ms to 1900 ms
3. 1900 ms to 2001 ms

VFR frame choice 2 is taken.

Else if there are multiple largest VFR frame that takes most of the CFR frame time, take your pick, but be consistent. Example:

A CFR has a timestamp playback from 1000 ms to 2000 ms with 2 VFR frames to choose from. The 2 VFR frames have the following timestamps:

1. 999 ms to 1500 ms
2. 1500 ms to 2001 ms

Either VFR frame should be fine, but maybe there are other factors as to why one is chosen over the other (like to avoid dropping or dupping frames).

I think this solution would fixed the problems I originally had with FFMS2 and the drop frame problems TheFluff mentioned.

Last edited by Aktan; 29th May 2012 at 19:58.
Aktan is offline   Reply With Quote
Old 29th May 2012, 20:44   #14  |  Link
Ghitulescu
Registered User
 
Ghitulescu's Avatar
 
Join Date: Mar 2009
Location: Germany
Posts: 5,769
Have you considered how the VFR frames have been achieved?
It may be that a frame was taken just before/after a certain TC during recording (BTW, what recorder other than screen caps do record in VFR?), and thus, due to 2 approximations, be substantially displaced from its place.
__________________
Born in the USB (not USA)
Ghitulescu is offline   Reply With Quote
Old 29th May 2012, 21:38   #15  |  Link
Aktan
Registered User
 
Join Date: Feb 2002
Posts: 303
I have not thought of that, but I do create VFR MP4s and MKVs regularly. I make encodes of console emulator games being played back via an input files a lot, and those may have a ton of real duplicate frames. I capture and then I run it through something like DeDup, or lately, ExactDedup to create a timecode file and remove duplicate frames. It is because I know what the original CFR source looks like that I noticed the problems I deem incorrect with Dss2 and FFMS2. I should probably read what the MKV timecodes stand for, but I believe it does mean the start time of a frame, in which case, in your example, the program generating the timecodes are doing it wrong and nothing I can do to fix it.
Aktan is offline   Reply With Quote
Old 29th May 2012, 21:47   #16  |  Link
sneaker_ger
Registered User
 
Join Date: Dec 2002
Posts: 5,565
Quote:
Originally Posted by Aktan View Post
I should probably read what the MKV timecodes stand for, but I believe it does mean the start time of a frame, in which case, in your example, the program generating the timecodes are doing it wrong and nothing I can do to fix it.
^ Should be correct on both accounts.
Ghitulescu's objection is irrelevant to the problem at hand, since we have no way of knowing when a frame "really" should've been displayed.
sneaker_ger is offline   Reply With Quote
Old 30th May 2012, 07:42   #17  |  Link
Ghitulescu
Registered User
 
Ghitulescu's Avatar
 
Join Date: Mar 2009
Location: Germany
Posts: 5,769
It was not an objection, just a thought. AFAIK screen capturing programs (it includes here PC games played on a PC) lay the TC at the beginning of a frame change, while console games should yield CFR videos (they must be compatible with CFR display devices (eg TVs), if my definition of console games coincides with that of the OP , so a capturing SW that provides VFR videos from CFR ones is wrongly set, used or designed).
__________________
Born in the USB (not USA)
Ghitulescu is offline   Reply With Quote
Old 30th May 2012, 12:00   #18  |  Link
Aktan
Registered User
 
Join Date: Feb 2002
Posts: 303
If capturing programs lay the timecode at the beginning of a frame change, wouldn't our definition of a timecode still be the same? A timecode lists the start time of a frame (or a frame change).
Aktan is offline   Reply With Quote
Old 30th May 2012, 13:28   #19  |  Link
Ghitulescu
Registered User
 
Ghitulescu's Avatar
 
Join Date: Mar 2009
Location: Germany
Posts: 5,769
I would consider logic to move all the visual information of the current VFR frame into the current CBR one, if the VFR-TC did not get over half of the CBR-framerate, or the next VFR frame if over. In case a VFR frame is shorter than a CBR one, it should be maintained nevertheless. But I assume this case is rather rare, VFR appeared to save space due to longer frame durations, not shorter.
__________________
Born in the USB (not USA)
Ghitulescu is offline   Reply With Quote
Old 30th May 2012, 13:51   #20  |  Link
Aktan
Registered User
 
Join Date: Feb 2002
Posts: 303
In your example graphic, wouldn't my proposed solution above fit it perfectly?

If the CFR frame fits in the VFR frame, take that VFR frame. CFR frames 0 and 1 fits in VFR frame 0. CFR frames 4 and 5 fit in VFR frame 3.

Else if there are multiple VFR frames to choose from, take the largest. CFR frame 2 can choose from VFR frame 0 and 1, but since VFR frame 0 takes most of the time in the CFR frame, VFR frame 0 is chosen. Similarly, CFR frame 3 can choose from VFR frames 1 and 2, but since VFR frame 1 takes most of the time in the CFR frame, VFR frame 1 is chosen.
Aktan is offline   Reply With Quote
Reply

Thread Tools Search this Thread
Search this Thread:

Advanced Search
Display Modes

Posting Rules
You may not post new threads
You may not post replies
You may not post attachments
You may not edit your posts

BB code is On
Smilies are On
[IMG] code is On
HTML code is Off

Forum Jump


All times are GMT +1. The time now is 09:18.


Powered by vBulletin® Version 3.8.11
Copyright ©2000 - 2024, vBulletin Solutions Inc.