Welcome to Doom9's Forum, THE in-place to be for everyone interested in DVD conversion.

Before you start posting please read the forum rules. By posting to this forum you agree to abide by the rules.

 

Go Back   Doom9's Forum > Capturing and Editing Video > Avisynth Usage

Reply
 
Thread Tools Search this Thread Display Modes
Old 1st February 2009, 02:38   #1  |  Link
halsboss
likes to tinker
 
Join Date: Jan 2004
Location: girt by sea
Posts: 635
how to call ffmpegsource V2 ?

Had a look at the FFmpegSource2 Documentation and wasn't sure how to apply specifically for re-indexing if necessary. I wasn't clear on how to get FFVideoSource() to accept an index from ffmsindex.exe for example.

The issue: Over at http://forum.videohelp.com/topic363395.html (since the doom9 forums were down for a bit) an issue is outlined with a xvid .AVI clip. Briefly, a problem .AVI something to do with badly indexed keyframes, or more particularly non packed bitstream .AVI returning frames out of order under certain circumstances.

That's unusual and very very unhelpful when trying to use temporal filters and/or MT, as the output becomes jerky.

jagabo and poisondeathray kindly diagnosed it and mentioned that ffmpegsource to open the .avi may deal with it.

So, can anyone please clarify (a tad more than the doco) with an example if possible, how to use FFVideoSource() (on such a problem .avi) and then how to separately use the external ffmsindex.exe and feed it's result into FFVideoSource() in case I need to do that ?

Lastly, how does FFIndex() fit into that picture (ie how/when to use it) ?

Hopefully this clarification will be of use to other less technically inclined avisynth users too.

Thanks.

Last edited by halsboss; 1st February 2009 at 02:41.
halsboss is offline   Reply With Quote
Old 1st February 2009, 02:59   #2  |  Link
halsboss
likes to tinker
 
Join Date: Jan 2004
Location: girt by sea
Posts: 635
Interestingly FFVideoSource seems to work on that source inconsistently "after a fashion". If you open the .avs in Virtualdub
Code:
loadplugin("C:\Program Files\AviSynth 2.5\plugins-zzz\FFMS2.dll")
FFVideoSource("G:\DVD\problem\zzz-problem-frames-34-35.avi")
AssumeFPS(25)
and single-step through it all the way past frames 32,33,34,35,36,37 then you'll see the "new scene" begin on frame 33 for the duration of that session.

If you close Virtualdub and open the same .avs again and then click directly on frame 42 and single-step backwards you'll see the "new scene" begin on frame 34 for the duration of that session.

So, If I knew how to apply the indexing maybe I'd get consistent results ?

Last edited by halsboss; 1st February 2009 at 06:49.
halsboss is offline   Reply With Quote
Old 1st February 2009, 06:40   #3  |  Link
halsboss
likes to tinker
 
Join Date: Jan 2004
Location: girt by sea
Posts: 635
Worked it out. Hopefully correctly. Finally, something gives a consistent result, right or wrong FFMpegSource V2.00beta4 from http://forum.doom9.org/showthread.php?t=127037 is consistent and usable on the odd source exihibiting the aforementioned problem in post #1.
Code:
loadplugin("C:\Program Files\AviSynth 2.5\plugins-zzz\FFMS2.dll")
FFIndex("G:\DVD\problem\problem-full.avi", indexmask=7, dumpmask=0, overwrite=true)
FFVideoSource("G:\DVD\problem\problem-full.avi")
AssumeFPS(25)
The FFIndex(..., overwrite=true) forces re-creation of a full index with the same filename/extension plus appended extension ".ffindex". In this case a 500Mb xvid .avi results in only about an 8Mb ".ffindex" file. The FFVideoSource(...) seems to detect the ".ffindex" and use it. It takes about a minute for the FFIndex(...) to do it's thing which is acceptable in the scheme of things to get a reliable source for avisynth to use.

The original problem, of out-of-sequence frames being returned, goes away and you get a repeatable result whether stepping forward or backward through the frames in the non-bitstream-packed xvid .AVI.

I guess a thing to be aware of is a 2-pass HCEnc may force re-creation of a new index. If you use HCEnc, I guess the "lossless" function is handy - it's better than great for slow scripts in any case !

A HUGE thankyou to poisondeathray and jagabo without whose direction a usable solution would never have arisen.

Last edited by halsboss; 1st February 2009 at 07:30.
halsboss is offline   Reply With Quote
Old 1st February 2009, 20:11   #4  |  Link
Myrsloik
Professional Code Monkey
 
Myrsloik's Avatar
 
Join Date: Jun 2003
Location: Ikea Chair
Posts: 1,628
This shouldn't happen or work. Is it possible that you first created an index using beta 3?
__________________
VapourSynth - proving that scripting languages and video processing isn't dead yet
Myrsloik is offline   Reply With Quote
Old 2nd February 2009, 06:08   #5  |  Link
halsboss
likes to tinker
 
Join Date: Jan 2004
Location: girt by sea
Posts: 635
Quote:
Originally Posted by Myrsloik View Post
This shouldn't happen or work.
Oh dear, I'm sorry to read that it shouldn't happen or work. That means I must have done something wrong and may be back to square 1 with the issue. I'll check my re-encodes tonight and see if the jerkiness at scene changes, which highlights the underlying issue, has gone away. It seemed to in the initial tests.

Edit: yes the jerkiness in the result seemed to disappear implying the out-of-sequence frames issue went away

Quote:
Originally Posted by Myrsloik View Post
Is it possible that you first created an index using beta 3?
Unfortunately, I've only downloaded it for the first time just the other day and used it, and it was the latest V2beta4.

I did notice that all 4 cores are now being flogged during encodes whereas they used to run at about 60% (implying waiting for non-cpu resources like disk). I'll check a test cpu/encode time of avisource vs FFVideoSource over the next day or so.

Last edited by halsboss; 2nd February 2009 at 10:10. Reason: added result
halsboss is offline   Reply With Quote
Old 2nd February 2009, 09:44   #6  |  Link
halsboss
likes to tinker
 
Join Date: Jan 2004
Location: girt by sea
Posts: 635
Results for HCEnc023 1-pass , 7500 frames identical script/source excepting the xxxSource statements
Code:
*PROFILE BEST 
*AUTOGOP 15 
*CLOSEDGOPS 
*DC_PREC 10 
*MATRIX FOX1 
*BITRATE 9200 
*MAXBITRATE 9400 
*CQ_MAXBITRATE 2 

SetMTmode(mode=5,threads=4) # start with mode=5 forAVIsource http://forum.doom9.org/showthread.php?p=1067216#post1067216 
SetMemoryMax(256)                    
... source ...
SetMTmode(mode=2,threads=4) 
ConvertToYV12(interlaced=FALSE) # for Deblock_QED_MT2 and DeHalo_alpha 
Deblock_QED_MT2() #default quant1=20 
ConvertToYUY2(interlaced=FALSE) # Ensure YUY2 for the rest including Convolution3D  
Convolution3D(0, 3, 4, 3, 4, 2.8, 0) 
lanczos4resize(resizeWidth,resizeHeight).LimitedSharpenFaster(smode=4,strength=100) 
Converttoyv12() 
SetPlanarLegacyAlignment(True) 
Distributor() # use this when using HC and SetMTmode, per http://forum.doom9.org/showthread.php?p=1063622#post1063622
HCEnc023 results cut'n'pasted from the log (the times don't look right, but it IS cut'n'paste I assure you. The elapsed time was about 3 mins 10 secs and all 4 cores of the Q9450 belted along at 85%-100%) -

FFIndex(...,indexmask=7,dumpmask=0,overwrite=true)
FFVideoSource(...)
Code:
pass 1 encoding time:            0:03:08 (188.45 s)
fps:                             39.8
nr. of gops:          555
nr. of frames:        7500
nr. of I-frames:      555
nr. of P-frames:      2666
nr. of B-frames:      4279
average bitrate:      8138
minimum bitrate:      1163
maximum bitrate:      9361
bytes in bitstream:   305254021
bits in bitstream:    2442032168
average Quantizer:    3.588
total CPU time:       0:11:20 (188.45 s)
total elapsed time:   0:03:36 (216.22 s)
AviSource(...)
Code:
pass 1 encoding time:            0:03:11 (191.09 s)
fps:                             39.2
nr. of gops:          556
nr. of frames:        7500
nr. of I-frames:      556
nr. of P-frames:      2664
nr. of B-frames:      4280
average bitrate:      8143
minimum bitrate:      1163
maximum bitrate:      9361
bytes in bitstream:   305448191
bits in bitstream:    2443585528
average Quantizer:    3.586
total CPU time:       0:10:58 (191.09 s)
total elapsed time:   0:03:13 (192.75 s)
Conclusion ? Not a huge difference given the FF indexing time is probably included in the elapsed time. Well acceptable (to me, at least) if the issue in post 1 goes away.

Last edited by halsboss; 2nd February 2009 at 10:15. Reason: ran them again, just to be sure, posted new results
halsboss is offline   Reply With Quote
Old 3rd February 2009, 14:28   #7  |  Link
halsboss
likes to tinker
 
Join Date: Jan 2004
Location: girt by sea
Posts: 635
Well, it was too good to be true. It did something all funny to some frames. It DID say it was a beta.

Refer clipped source 200k http://rapidshare.com/files/19334862...d-original.avi and funny output from HC (recompressed to avi) 130k http://rapidshare.com/files/19334627...-destroyed.avi also clipped from a full encode.

Last edited by halsboss; 4th February 2009 at 08:54.
halsboss is offline   Reply With Quote
Old 3rd February 2009, 18:39   #8  |  Link
Myrsloik
Professional Code Monkey
 
Myrsloik's Avatar
 
Join Date: Jun 2003
Location: Ikea Chair
Posts: 1,628
Both links point to the same file. Is this intentional?
__________________
VapourSynth - proving that scripting languages and video processing isn't dead yet
Myrsloik is offline   Reply With Quote
Old 4th February 2009, 08:51   #9  |  Link
halsboss
likes to tinker
 
Join Date: Jan 2004
Location: girt by sea
Posts: 635
Nope. Sorry about that. Link corrected.

Last edited by halsboss; 4th February 2009 at 12:19.
halsboss is offline   Reply With Quote
Old 6th February 2009, 11:12   #10  |  Link
halsboss
likes to tinker
 
Join Date: Jan 2004
Location: girt by sea
Posts: 635
jagabo really helped with the mpeg2modifier recommendation.

Used mpeg4modifier to make the AVIs packed bitstream and all problems went away opening the avis with avisynth with avisource. Even an avi I had problems with which was supposedly packed bitstream I tried to unpack then pack again and hey presto it worked just fine - go figure.

Is there any other ways to open AVIs the "non-VFW" way ? I'd have thought problems with xvid AVIs and avisynth would have been more common. Seems like a bit of a "hole" in avisynth in the modern environment and I'd strongly prefer avisynth remain relevant
halsboss is offline   Reply With Quote
Old 24th February 2009, 14:22   #11  |  Link
halsboss
likes to tinker
 
Join Date: Jan 2004
Location: girt by sea
Posts: 635
why are XVIDs in AVI so unreliable as AVIsources - Bframes & packed bitstreams etc

mpeg4modifier of itself was a good idea while it lasted. I saw a packed bitstream xvid in .AVI and thought "that'll be OK" but it turned out to be a little odd. Unpacked with mpeg4modifier and repacked with it, and still had problems with it.

The only reliable way I found to get around it was via a combination of mpeg4modifier and the latest ffmpeg -
(1) unpack it with mpeg4modifier
(2) convert to a lossless ffv1 .AVI (ffmpeg handles xvid's b-frames in AVI OK by the looks, from non-packed-bitstream AVIs)
(3) then process the ffv1 AVI with avisynth.

Command lines :-
Code:
"C:\SOFTWARE\MPEG4Modifier\mmcl.exe" --unpack --always-write "input.avi" "input-unpacked.avi"   
"C:\SOFTWARE\ffmpeg\ffmpeg.exe" -y -i "input-unpacked.avi" -f avi -vcodec ffv1 -acodec copy "input-ffv1.avi"
halsboss is offline   Reply With Quote
Old 25th February 2009, 23:07   #12  |  Link
halsboss
likes to tinker
 
Join Date: Jan 2004
Location: girt by sea
Posts: 635
ANSWER:
The Packed Bitstream AVI fix ONLY works if you use the latest XVID codec and not ffdshow's xvid decoder. It works then.

From the other thread http://forum.doom9.org/showthread.php?t=145262
Quote:
Always use packed bitstream with avi and use xvid to decode instead of ffdshow. Unticking decoding xvid in ffdshow (both vfw and non-vfw) and installing the latest xvid makes the problem go away.
halsboss is offline   Reply With Quote
Old 26th February 2009, 02:28   #13  |  Link
henryho_hk
Registered User
 
Join Date: Mar 2004
Posts: 889
Kinda losing track..... is ffmpegsource working or not?
henryho_hk is offline   Reply With Quote
Old 26th February 2009, 08:29   #14  |  Link
Myrsloik
Professional Code Monkey
 
Myrsloik's Avatar
 
Join Date: Jun 2003
Location: Ikea Chair
Posts: 1,628
No idea but this sounds like a very odd bug. (if it is a bug)
__________________
VapourSynth - proving that scripting languages and video processing isn't dead yet
Myrsloik is offline   Reply With Quote
Old 26th February 2009, 09:47   #15  |  Link
halsboss
likes to tinker
 
Join Date: Jan 2004
Location: girt by sea
Posts: 635
I haven't tested it since I had issues. If it's based on ffmpeg, the recent ffmpeg commandline gave a message about illegal packed bitstream when I fed a packed bitstream AVI into it, so I'm not sure. I fed the ffmpeg commandline an unpacked AVI (after mpeg4modifier) and it didn't spit, so I venture a guess ffmpegsource may be similar ?
halsboss is offline   Reply With Quote
Reply

Thread Tools Search this Thread
Search this Thread:

Advanced Search
Display Modes

Posting Rules
You may not post new threads
You may not post replies
You may not post attachments
You may not edit your posts

BB code is On
Smilies are On
[IMG] code is On
HTML code is Off

Forum Jump


All times are GMT +1. The time now is 01:06.


Powered by vBulletin® Version 3.8.11
Copyright ©2000 - 2017, vBulletin Solutions Inc.