Welcome to Doom9's Forum, THE in-place to be for everyone interested in DVD conversion.

Before you start posting please read the forum rules. By posting to this forum you agree to abide by the rules.

 

Go Back   Doom9's Forum > Capturing and Editing Video > Avisynth Usage

Reply
 
Thread Tools Search this Thread Display Modes
Old 12th September 2015, 21:33   #1  |  Link
fvisagie
Registered User
 
Join Date: Aug 2008
Location: Isle of Man
Posts: 586
Freaky question of the decade: frame-serving from ffmpeg to Avisynth

Is there a way of frame-serving ffmpeg output to Avisynth?

I foresee doing some processing in ffmpeg, and then continuing in Avisynth. This project is huge and where possible I'd like to avoid intermediate files. Is Avisynth's TCPSource() compatible with any of the ffmpeg output protocols, and if so, in this case what ffmpeg command line would be needed for that protocol, especially '-f format'?

Thanks,
François
fvisagie is offline   Reply With Quote
Old 13th September 2015, 12:29   #2  |  Link
raffriff42
Retried Guesser
 
raffriff42's Avatar
 
Join Date: Jun 2012
Posts: 1,377
After a few tries, I got ffmpeg to stream to VLC Player:
Quote:
ffmpeg -i %file% -c:v mpeg2video -q:v 1 -qmin 1 -pix_fmt yuv420p -c:a ac3 -q:a 0.35 -f mpegts "tcp://127.0.0.1:2000?listen"
(EDIT Regarding your format question, -f mpegts seems to be mandatory here)

But TCPSource...
Quote:
TCPSource("tcp://127.0.0.1", port=2000, compression="none")
...always failed with "Access Violation." I tried deleting the compression argument, but it made no difference.
(EDIT I just realized compression here refers to over-the-network compression, not video compression.)

Checking the source code, I have to conclude that TCPSource is designed to work with TCPServer only - they are a matched pair.
Code:
  SendRequest(CLIENT_CHECK_VERSION, &ccv, sizeof(ccv));
  GetReply();

  if (reply->last_reply_type != REQUEST_CONNECTIONACCEPTED) {
    env->ThrowError("TCPSource: Version Check failed! (Ensure Client and Server are same version)");
  }
EDIT I can't even get the matched pair to work. "Access Violation."
Now I'm wondering if it's a Windows 8.1 compatibility issue.
EDIT about that...scroll down to post #19.


...If you want to do ffmpeg processing of some kind (what does it do that Avisynth filters can't?) with frameserving, you need some kind of ^DirectShow wrapper for ffmpeg.^ Fortunately a very good one exists, namely, ffdshow tryouts.

Last edited by raffriff42; 8th January 2016 at 02:55. Reason: post 19
raffriff42 is offline   Reply With Quote
Old 13th September 2015, 18:16   #3  |  Link
fvisagie
Registered User
 
Join Date: Aug 2008
Location: Isle of Man
Posts: 586
Thanks for your trouble, much appreciated.

Quote:
Originally Posted by raffriff42 View Post
EDIT I can't even get the matched pair to work. "Access Violation."
Now I'm wondering if it's a Windows 8.1 compatibility issue.
On my Windows 7 SP1 machine TCPServer() and TCPSource() work together correctly. IIRC, initially I had an issue with the firewall popping up an authorisation request for the host application, so that might play a role in your case too.

Regrettably TCPSource()'s success did not extend to working with this particular ffmpeg command. I got the default unacceptable response:
Quote:
TCPSource: Version Check failed! (Ensure Client and Server are same version)
Quote:
...If you want to do ffmpeg processing of some kind (what does it do that Avisynth filters can't?)
Works with hundreds of sources without crashing, and produces audio ;-).

Quote:
with frameserving, you need some kind of ^DirectShow wrapper for ffmpeg.^ Fortunately a very good one exists, namely, ffdshow tryouts.
I'm not sure DirectShow support is necessarily required, in the sense that the frame-serving host application interface would probably determine that (e.g. VirtualDub uses the VfW interface)? Or does ffdshow-tryouts provide its own frame-serving host? Its documentation is better than most but still sparse!

In any event, I'm looking into ffdshow-tryouts also for its FFV1 support.
fvisagie is offline   Reply With Quote
Old 13th September 2015, 19:13   #4  |  Link
raffriff42
Retried Guesser
 
raffriff42's Avatar
 
Join Date: Jun 2012
Posts: 1,377
Quote:
Originally Posted by fvisagie View Post
(e.g. VirtualDub uses the VfW interface)? Or does ffdshow-tryouts provide its own frame-serving host?
You can use VirtualDub's decoders and frameserving, with the ffdshow plugin for video effects:
(source) >> Vdub >> "ffvdub" plugin >> Vdub >> frameserve
...but it's easier to access ffdshow's decoders and effects in one step:
(source) >> DirectShowSource calling ffdshow >> Avisynth >> frameserve
You just need to enable the appropriate decoders in ffdshow options. If you have problems with this ("DirectShow video decoders are not required to support frame-accurate seeking"), you can fall back to the VirtualDub method.

EDIT I was thinking in terms of video effects filters. I wasn't thinking of audio processing at all, so this might not help you.

Last edited by raffriff42; 13th September 2015 at 19:20.
raffriff42 is offline   Reply With Quote
Old 14th September 2015, 12:30   #5  |  Link
fvisagie
Registered User
 
Join Date: Aug 2008
Location: Isle of Man
Posts: 586
Quote:
Originally Posted by raffriff42 View Post
You can use VirtualDub's decoders and frameserving
Thanks for explaining that. I was hoping that I could frame-serve to Avisynth from ffmpeg instead, because the latter can concatenate directly from non-AVI files without requiring intermediate conversion (and storage). However, reality seems to dictate otherwise...
fvisagie is offline   Reply With Quote
Old 15th September 2015, 13:18   #6  |  Link
fvisagie
Registered User
 
Join Date: Aug 2008
Location: Isle of Man
Posts: 586
Quote:
Originally Posted by fvisagie View Post
Regrettably TCPSource()'s success did not extend to working with this particular ffmpeg command.
On the premise that Avisynth would output via TCPServer() and input via TCPSource() data in similar format than it produces via frame-serving, I noted its video and audio formats when serving to ffmpeg and specified those codecs for the most likely-looking ffmpeg protocol, tcp.

Then came the issue of which format to specify for that protocol. ffmpeg reports Avisynth's input format as 'avisynth', which the former does not support as an output format. I tried a couple of likely-looking candidates, mostly the 'raw' ones. With some, e.g. 'rawvideo' or 'dnxhd':
Code:
ffmpeg -y -i 00000.MTS -c:v rawvideo -c:a pcm_s16le -f dnxhd tcp://127.0.0.1:22050?listen
ffmpeg and Avisynth would connect and ffmpeg would in fact start serving. After a short while, however, Avisynth would crash out and ffmpeg would terminate with
Quote:
av_interleaved_write_frame(): Unknown error
This seems to indicate at least format incompatibility, and perhaps also initial connection set-up as raffriff42's source extract above would indicate.

If only I had the time, I would get into coding and adapt TCPSource() to FFTCPSource() (or whichever ffmpeg protocol most suitable in terms of seeking etc.)...
fvisagie is offline   Reply With Quote
Old 15th September 2015, 13:25   #7  |  Link
Wilbert
Moderator
 
Join Date: Nov 2001
Location: Netherlands
Posts: 6,301
What happens if you frameserve a video only clip? Does that work?
Wilbert is offline   Reply With Quote
Old 15th September 2015, 14:06   #8  |  Link
fvisagie
Registered User
 
Join Date: Aug 2008
Location: Isle of Man
Posts: 586
Nothing exciting to note. Without audio, Avisynth seems to exit with the "TCPSource: Version Check failed! (Ensure Client and Server are same version)" message more often instead of crashing, compared to before. And for what it's worth, with two protocols Avisynth exited with the error message without aborting, AND the ffmpeg server continued running now. The protocols are 'mpegts' and 'rtsp'. With audio the ffmpeg server would exit with an error condition each time.
fvisagie is offline   Reply With Quote
Old 15th September 2015, 14:34   #9  |  Link
raffriff42
Retried Guesser
 
raffriff42's Avatar
 
Join Date: Jun 2012
Posts: 1,377
I've been Googling for alternatives and gotten nowhere...
Quote:
MPC-HC Frameserver [cccp-project.net, 2013
http://www.cccp-project.net/forums/i...p?topic=6853.0

VLC backend server questions [2012]
http://forum.doom9.org/showthread.php?t=166460

Pipe data to VLC [videolan.org, 2010]
https://wiki.videolan.org/Uncommon_uses/

Mencoder as realtime frameserver [MPlayer-users, 2009]
https://lists.mplayerhq.hu/pipermail...er/078407.html

IDEA: A new universal way for frameserving [virtual drive emulation, 2006]
http://forum.doom9.org/showthread.php?t=108653
Quote:
Originally Posted by fvisagie View Post
This project is huge and where possible I'd like to avoid intermediate files.
I'd say it's very likely not possible. Just get a huge RAID array or whatever, and start working on your actual project, instead of this side project...
raffriff42 is offline   Reply With Quote
Old 15th September 2015, 15:15   #10  |  Link
fvisagie
Registered User
 
Join Date: Aug 2008
Location: Isle of Man
Posts: 586
Quote:
Originally Posted by raffriff42 View Post
Just get a huge RAID array or whatever, and start working on your actual project, instead of this side project...
Hehe, point taken, but in my defence, I put high value on some measure of preparation. If there's one thing I dislike more than working hard, it's working unnecessarily hard .

Thanks for your trouble, much appreciated.
fvisagie is offline   Reply With Quote
Old 15th September 2015, 15:19   #11  |  Link
feisty2
I'm Siri
 
feisty2's Avatar
 
Join Date: Oct 2012
Location: Los Angeles, California
Posts: 2,119
write a special source filter that reads raw videos directly from RAM
__________________
If I got new ideas, will post here: https://github.com/IFeelBloated
feisty2 is offline   Reply With Quote
Old 15th September 2015, 17:55   #12  |  Link
jmartinr
Registered User
 
jmartinr's Avatar
 
Join Date: Dec 2007
Location: Enschede, NL
Posts: 285
It would require some footwork, but you might try this: http://forum.doom9.org/showthread.php?t=170590
__________________
Roelofs Coaching
jmartinr is offline   Reply With Quote
Old 15th September 2015, 20:01   #13  |  Link
Wilbert
Moderator
 
Join Date: Nov 2001
Location: Netherlands
Posts: 6,301
Quote:
Originally Posted by jmartinr View Post
It would require some footwork, but you might try this: http://forum.doom9.org/showthread.php?t=170590
Looks like great stuff. Did anyone play with this? Does it support audio too?
Wilbert is offline   Reply With Quote
Old 15th September 2015, 20:11   #14  |  Link
poisondeathray
Registered User
 
Join Date: Sep 2007
Posts: 3,412
Could you maybe modify the workflow so some of the ffmpeg processing is done in avisynth or vice-versa, or even another program ? What types of operations are being performed ?
poisondeathray is offline   Reply With Quote
Old 16th September 2015, 10:07   #15  |  Link
fvisagie
Registered User
 
Join Date: Aug 2008
Location: Isle of Man
Posts: 586
Quote:
Originally Posted by jmartinr View Post
It would require some footwork, but you might try this: http://forum.doom9.org/showthread.php?t=170590
Wow! Well spotted, thanks.
fvisagie is offline   Reply With Quote
Old 16th September 2015, 10:51   #16  |  Link
fvisagie
Registered User
 
Join Date: Aug 2008
Location: Isle of Man
Posts: 586
Quote:
Originally Posted by poisondeathray View Post
Could you maybe modify the workflow so some of the ffmpeg processing is done in avisynth or vice-versa, or even another program ? What types of operations are being performed ?
Assuming your question refers to the first post, concatenation of M2TS input files at least, followed by transcoding to FFV1 (which seems to be the lossless codec most easily added to my existing workflow) for further improvement and editing in Avisynth. Transcoding to intermediates for lack of a known reliable way of frame-serving from ffmpeg to Avisynth, and FFV1 for ffmpeg's lack of Lagarith encoding.

However, following an analysis of M2TS and FFV1 support in the tools concerned it appears that that approach has issues of its own which require work-arounds that at least cancel the benefits. Therefore, for this project I'll stick with Lagarith intermediate AVIs and concatenating those in VirtualDub.

What I found when looking into M2TS and FFV1 support:
  • ffmpeg has a bug with concatenating M2TS (H.264 in general it seems) files
  • Avisynth intermediate files can overcome that, but at the expense of added complexity and processing
  • On my system Lagarith decodes (read: editing use) around 3 times faster than intra-frame-only encoded FFV1. The newer version 3 FFV1 codec creates files that decode marginally faster than those created with the older version 1 codec.
  • ffdshow-tryout's DirectShow, VfW and Virtualdbug plugin FFV1 interfaces all only read and write the FFV1 version 1 codec
  • The only way (with these tools) to write FFV1 codec version 3 files is with recent ffmpeg versions, which have the buggy M2TS support above
  • The only way for Avisynth and VirtualDub to read FFV1 codec version 3 files is with LWLibavVideoSource(), which has FFV1 issues of its own
  • Lagarith files used in the above comparison are only ~10% larger than their FFV1 counterparts
fvisagie is offline   Reply With Quote
Old 16th September 2015, 14:14   #17  |  Link
poisondeathray
Registered User
 
Join Date: Sep 2007
Posts: 3,412
Quote:
Originally Posted by fvisagie View Post
Assuming your question refers to the first post, concatenation of M2TS input files at least, followed by transcoding to FFV1 (which seems to be the lossless codec most easily added to my existing workflow) for further improvement and editing in Avisynth. Transcoding to intermediates for lack of a known reliable way of frame-serving from ffmpeg to Avisynth, and FFV1 for ffmpeg's lack of Lagarith encoding.

However, following an analysis of M2TS and FFV1 support in the tools concerned it appears that that approach has issues of its own which require work-arounds that at least cancel the benefits. Therefore, for this project I'll stick with Lagarith intermediate AVIs and concatenating those in VirtualDub.

What I found when looking into M2TS and FFV1 support:
  • ffmpeg has a bug with concatenating M2TS (H.264 in general it seems) files
  • Avisynth intermediate files can overcome that, but at the expense of added complexity and processing
  • On my system Lagarith decodes (read: editing use) around 3 times faster than intra-frame-only encoded FFV1. The newer version 3 FFV1 codec creates files that decode marginally faster than those created with the older version 1 codec.
  • ffdshow-tryout's DirectShow, VfW and Virtualdbug plugin FFV1 interfaces all only read and write the FFV1 version 1 codec
  • The only way (with these tools) to write FFV1 codec version 3 files is with recent ffmpeg versions, which have the buggy M2TS support above
  • The only way for Avisynth and VirtualDub to read FFV1 codec version 3 files is with LWLibavVideoSource(), which has FFV1 issues of its own
  • Lagarith files used in the above comparison are only ~10% larger than their FFV1 counterparts



If these are camcorder files, you run a higher risk of "glitches" , or sometimes AV sync , or crackle issues at the seams when appending individual file spanned clips when using avisynth, or ffmpeg or tsmuxer. The most reliable way is to copy the entire folder structure over, because there is accessory meta data that some programs use to seamlessly join the the clips. Usually the camera comes with crappy (but reliable) software to do this, or look at the manufacturer website. Most decent NLE's also do it that way with the entire folder structure - but they decode the files , not concatenate or stream copy

UT Video and x264 are other possibilities for lossless intermediates, that are also available in ffmpeg. You can adjust x264 parameters so it's has temporal compression like ffv1; you can tweak the parameters so it can decode faster or slower; higher or lower compression. UT usually is less compressed than lagarith, but decodes much faster. So if editing speed is a concern, it's probably the better choice. UT is also very compatible in NLE, x264 lossless isn't

Be aware Lagarith can have it's own issues for some people with random corrupt frames. A few threads discuss this at doom9. I've never personally had it happen to me
poisondeathray is offline   Reply With Quote
Old 16th September 2015, 20:05   #18  |  Link
fvisagie
Registered User
 
Join Date: Aug 2008
Location: Isle of Man
Posts: 586
Thanks for that advice. Fortunately the clips as input to this workflow have already been joined by the camera's export software .

As potential back-ups should I run into issues with Lagarith (which I've been using for a long time) I've been keeping UT Video and H.264 in mind; I appreciate your inputs on those too.
fvisagie is offline   Reply With Quote
Old 20th September 2015, 19:10   #19  |  Link
Wilbert
Moderator
 
Join Date: Nov 2001
Location: Netherlands
Posts: 6,301
Quote:
Originally Posted by raffriff42 View Post
After a few tries, I got ffmpeg to stream to VLC Player: (EDIT Regarding your format question, -f mpegts seems to be mandatory here)

But TCPSource...

...always failed with "Access Violation." I tried deleting the compression argument, but it made no difference.
(EDIT I just realized compression here refers to over-the-network compression, not video compression.)

Checking the source code, I have to conclude that TCPSource is designed to work with TCPServer only - they are a matched pair.
Code:
  SendRequest(CLIENT_CHECK_VERSION, &ccv, sizeof(ccv));
  GetReply();

  if (reply->last_reply_type != REQUEST_CONNECTIONACCEPTED) {
    env->ThrowError("TCPSource: Version Check failed! (Ensure Client and Server are same version)");
  }
EDIT I can't even get the matched pair to work. "Access Violation."
Now I'm wondering if it's a Windows 8.1 compatibility issue.


...If you want to do ffmpeg processing of some kind (what does it do that Avisynth filters can't?) with frameserving, you need some kind of ^DirectShow wrapper for ffmpeg.^ Fortunately a very good one exists, namely, ffdshow tryouts.
For future reference. raffriff42 should have used (without the tcp:// in front of the ip number):
Code:
TCPSource("127.0.0.1", port=2000, compression="none")

Last edited by Wilbert; 20th September 2015 at 19:24.
Wilbert is offline   Reply With Quote
Old 20th September 2015, 19:23   #20  |  Link
raffriff42
Retried Guesser
 
raffriff42's Avatar
 
Join Date: Jun 2012
Posts: 1,377
Thanks Wilbert, my use of "tcp://" (copied from the ffmpeg line I guess) was the problem.
raffriff42 is offline   Reply With Quote
Reply

Thread Tools Search this Thread
Search this Thread:

Advanced Search
Display Modes

Posting Rules
You may not post new threads
You may not post replies
You may not post attachments
You may not edit your posts

BB code is On
Smilies are On
[IMG] code is On
HTML code is Off

Forum Jump


All times are GMT +1. The time now is 03:11.


Powered by vBulletin® Version 3.8.11
Copyright ©2000 - 2018, vBulletin Solutions Inc.