Avisynth MVC decoder plugin ?

[Nico8583]

Hi ;)
Does anyone could make an Avisynth plugin for MVC decoding based on Intel Media SDK 2013 R2 ?
We are talking about this in this topic : http://forum.doom9.org/showthread.php?p=1659500#post1659500
This plugin already exists but it is not free to redistribute (MVCsource.dll from BDtoAVCHD software).
In this topic (http://forum.doom9.org/showthread.php?t=169651&page=22), an existing decoder / encoder exists also but it needs .yuv files to work.
Thanks if anyone could help us !

[Nico8583]

Intel Media SDK 2013 R2 is available here : http://software.intel.com/en-us/vcsource/tools/media-sdk

[r0lZ]

I second that request. We desperately need a good MVC decoder plugin for avisynth. It seems that currently, only the Intel MVC decoder can properly decode the MVC streams, but there is no avisynth plugin able to use it, except MVCSource, but its author imposes too many limitations, and we cannot use it freely.

Thanks in advance to anybody interested to help us. :-)

[Wilbert]

Your best option is to ask videofan3d to make FRIMDecode open source so someone (perhaps him?) can modify it to a plugin.

[Cedvano]

And use it directly with SSIF most great !!!

[Nico8583]

Your best option is to ask videofan3d to make FRIMDecode open source so someone (perhaps him?) can modify it to a plugin.
Yes we have already ask him if he could transform his decoder to an Avisynth plugin but it seems he can't do that so we are searching a pro to help us :)

[slavanap]

If this MVCSource requires only one file, that sample_decode.exe requires, I can write a simple workaround.
Just wrap sample_decode as Avisynth plugin and force it to use pipes.

I don't know, what exactly mvcsource does, so I can't be sure about what you want.

ADD:
can you explain how now the pipeline is looks like? I can join almost any software for using pipes, if it does not require seeking, as I did for ssifSource1.

[r0lZ]

Thanks for the proposition. :-)

The current MVCSource takes 2 files as input: the AVC and the MVC demuxed streams. It has a third parameter to specify what it must return: the AVC frames (value of the parameter is 0), the MVC frames (value 1), or the AVC + MVC streams, interleaved (value 2 or -2, where 2 returns the AVC (odd frames) + MVC (even frames), and -2 returns the MVC (odd) +AVC (even). It is not possible to seek, and it's why it is important to have the possibility to select the order of the AVC and MVC frames. It doesn't use pipes, and calls the external Intel DLL.

MvcSource("AVC.h264", "MVC.h264", output)
Where output is:
0 : base view (AVC)
1 : dependent view (MVC)
2 : interleaved base-dependent (base first)
-2 : interleaved dependent-base (dependent first)

Of course, we don't need exactly this syntax. It is however much better to accept the 2 input streams instead of a single AVC+MVC combined stream, as combining the 2 streams would require an additional long pass, and more disc space. Of course, the best solution would be to accept a MPLS file as input, but I suppose it's too complex. Currently, I think the best option is to demux the 2 video streams with tsMuxeR or eac3to, and to use them directly in the Source command. I don't know if the Intel decoder requires a single combined file as input, but I suppose it can accept two streams, even if sample_decode.exe requires a single combined input.

Using the pipes to communicate the data to the decoder is of course possible, but since the decoder is open source and some examples exist, I wonder if it is much more difficult to integrate it completely in the avisynth plugin. Currently, the open source examples accept only plain files as input, but afaik the FRIM decoder accept pipes. Unfortunately, FRIM is currently closed source. It should be usable without modifications with the pipes, but I haven't studied how FIRM works, and I may be wrong.

Anyway, thanks again for considering to help us.

[Nico8583]

Thanks slavanap to try to help us ! :)

[Nico8583]

A new info if it can help you :)
I've installed Intel Media SDK 2013 and there are :
- "sample_decode.exe" (and source code) : permit to decode input to .yuv files
- "h264_dec_filter.dll" (and source code) : an Intel DirectShow Filter to decode h264
- "mvc_dec_filter.dll" (and source code) : an Intel DirectShow Filter to decode MVC
Perhaps it is easier to adapt DirectShow source filter to Avisynth plugin :??:

[r0lZ]

Yes, it's probably the best solution, but I don't know if it's easy. What do you think?

I have tested sample_decode.exe with a small clip, and it works well, but it creates huge files on HDD. It is therefore not usable in practical situations. So, it should either be adapted to write to pipes, or the mvc_dec_filter.dll should be adapted as an avisynth plugin. Of course, I prefer largely the second solution. Pipes are usually not easy to handle, and require much memory. An avisynth plugin similar to DirectShowMVCSource but without its bugs would be perfect.

[Nico8583]

I'm not competent to make that but I think it's possible.
Slavanap (or another dev) could tell us if he thinks it's possible ;)

[Nico8583]

I hope anyone could help us :)

The most unfortunate thing is that this plugin already exists (in this forum) but it is not free to use and redistribute :/

[Guest]

Can't users download BDtoAVCHD and take out the DLL and install it themselves? Maybe not fully convenient I agree but one does what one has to do. Or have I missed something?

[Sparktank]

Can't users download BDtoAVCHD and take out the DLL and install it themselves? Maybe not fully convenient I agree but one does what one has to do. Or have I missed something?

I believe they're looking for one that can be freely redistributeable (without depending on installing a software bundle).

One that can also be maintained separately than any other versions currently available (with additional support for .yuv files and/other future support?).

Between the several threads, that's what I'm gathering.
I would like to throw in a vote for a stand-alone build, if that's indeed the goal.

[r0lZ]

I agree with Sparktank. The DLL can't be installed without installing the whole program. It doesn't work if it is installed alone. And, of course, we would prefer a standalone dll, freely re-distributable.

[Nico8583]

With BDtoAVCHD 1.9.4, DLL can work in standalone mode but since BDtoAVCHD 1.9.5, the author added a protection to force users to install his soft if they want to use the DLL. But the soft installs freewares who can interfere with already installed freewares, and his soft contains a Google Ad-sense code so it could be considered as an Ad-aware...
So yes users can take out DLL but no it doesn't work :(

Edit : Sorry, I have not refresh page before my response so I have not seen Sparktank and r0lZ have already responded :D

[Guest]

Thanks for the clarification, guys.

[frencher]

NOP dll is possible.
Sharing unrestricted Pistacho always be the welcome.

[pistacho]

EDIT: Since frencher cleaned his message (with the criticism towards BDtoAVCHD). I also prefer clean this.

[Sharc]

.... I have my website hosted in a professional hosting (since year 2000) and have developed other programs besides BDtoAVCHD. All this costs money. Or is someone plan to pay me if I remove the advertising?
Well, you may think about options like:
- Removing the ads for donators
- License your .dll for a reasonable fee

It's of course all up to you.

[r0lZ]

We have just replied to a simple and legitimate question: why we cannot use your dll in our programs. You have decided to protect it. It's not our decision. We can't use your plugin, especially because we cannot force our users to install an adware program, just to have access to an avisynth plugin based on an open source library. OK, but that means that we need an alternative. It's the subject of this thread. If you re-read my post above, you will see that there is no criticism of your software. Just an objective description of a fact. So, please, stop to complain, and to criticise our work, and our ability to program.

[frencher]

As you take ownership of our work and that of other recently Developer "Frim MVC Decoder" for hardware acceleration if i'm not mistaken, you go through the other so I concluded that you do not have to search by level yourself.
Being banned here I find so low that I'd rather not go down to your level.
I propose solutions to remember Pistacho there very long I've already warned.
You have yourself created this image and now please leave this topic because you do not have your space, you do not make progress.

[Nico8583]

I honestly think this kind of post should I be banned. Nico, frencher, r0lz, etc.. Worry about your programs and projects but stop criticizing the work of others

We have just replied to a simple and legitimate question: why we cannot use your dll in our programs. You have decided to protect it. It's not our decision. We can't use your plugin, especially because we cannot force our users to install an adware program, just to have access to an avisynth plugin based on an open source library. OK, but that means that we need an alternative. It's the subject of this thread. If you re-read my post above, you will see that there is no criticism of your software. Just an objective description of a fact. So, please, stop to complain, and to criticise our work, and our ability to program.

Rolz said it all. This topic is devoted to creating an Avisynth plugin that will probably do the same thing as yours but with "freedom".
There was no critical if you reread the posts, just observations. You do not want to share your DLL, no problem, but you can not forbid us to want to do the same to yours as the base is the same (Intel Media SDK).
[] do not found our posts so critical, they are just real informations. So please help up or stay out of this topic and respect others ! Thanks !

[Guest]

Spirited discussion is fine but let's keep it civil guys.

Also, we don't censor/ban discussions unless they violate our forum rules. So far, that does not appear to be the case. pistacho, you have your thread and good prominence for your work here at the valuable Doom9 forum, and Doom9 appreciates your contributions and your offering additional options for working, even if they may not be suitable for all. Let others have their threads. Of course feel free to rebut anything you feel is unfair but do so in a civil way. And others, please don't condemn pistacho for the way he chooses to make his work available. I'm sure you are aware that several other significant applications are not totally free.

Thanks guys.

[slavanap]

As I ivestigated,
Intel Media SDK 2013 has MVC decoder bundled into the libmfxsw32.dll (or libmfxsw64.dll for 64bit apps). This decoder accepts only combined MVC streams.
I can write AviSynth plugin that accepts combined stream and produses both views (as ssifSource1 did). But it is not the solution.

Now I need to know, how to produce combined stream. I know that you used MVCCombine to combine 2 raw h264 streams into the stream accepted by Intel MVC decoder.
How did you get these 2 raw streams, that MVCCombine accepts? The method based on MPEG Splitter does not work, unfortunately.

[r0lZ]

MVCCombine was only the first step. The output of MVCCombine had to be muxed to M2TS and then fed to the decoder. That was not a practical solution. Now, the new tsMuxeR can combine and mux to M2TS in one operation, but it's still a step that should be avoided if possible.

[Nico8583]

As I ivestigated,
Intel Media SDK 2013 has MVC decoder bundled into the libmfxsw32.dll (or libmfxsw64.dll for 64bit apps). This decoder accepts only combined MVC streams.
I can write AviSynth plugin that accepts combined stream and produses both views (as ssifSource1 did). But it is not the solution.

Now I need to know, how to produce combined stream. I know that you used MVCCombine to combine 2 raw h264 streams into the stream accepted by Intel MVC decoder.
How did you get these 2 raw streams, that MVCCombine accepts? The method based on MPEG Splitter does not work, unfortunately.
Have you seen Direct Show H264 and MVC decoder filter include in Intel Media SDK sample ? Perhaps it can help you.
Or have you seen sample_decode with Intel Media SDK tutorial ? H264 sample decoder accept .264 input but output is .yuv

[frencher]

For help maybe slavanap could contact videofan3d

[Thalyn]

Now I need to know, how to produce combined stream. I know that you used MVCCombine to combine 2 raw h264 streams into the stream accepted by Intel MVC decoder.
How did you get these 2 raw streams, that MVCCombine accepts? The method based on MPEG Splitter does not work, unfortunately.

Originally, you got the two raw streams using EAC3to to extract them as AVC and MVC, which would then be combined using MVCCombine and finally put into a .ts package using TSMuxer.

Since then, TSMuxer has been updated to a point where it is capable of understanding MVC, making it capable of performing all three operations (extraction, combination and repackaging) by itself. However I don't believe that will work directly with the Intel package as I don't think it understands any containers - only raw streams - and you need to revert to an older TSMuxer (one which isn't MVC-aware) to extract the combined stream in one part.

I also believe the results of MakeMKV (currently in free beta), extracted using MKVExtract, are almost if not exactly the same as the previous MVCCombine results. At least, they can be fed directly into ssifSource2 (with the appropriate MKV .dll) and give appropriate results.

[videofan3d]

Hi,

just for fun I started few days ago similar plugin

LoadPlugin ("some_path\FRIMSource.dll")
FRIMSource (codec="mvc", file="some_path\SRC_L.h264", dependent="some_path\SRC_R.h264", layout="sbs", cache=24, num_frames=250)

which is based on kernel of already tested FRIMDecode.

Since scrolling through elementary stream is not supported, I added there at least a cache for last N frames which can help a bit in case of subsequent temporal filters.

If anyone is interested - you can download this beta 1 from FRIMSource_v1.zip (https://drive.google.com/file/d/0BymRNDHq74DEQVdTN2RiUXNyQXM).

Remark: to be used together with libmfxhw32.dll or libmfxsw32.dll from Intel Media SDK or FRIM Package (http://forum.doom9.org/showthread.php?p=1650646#post1650646)

[Nico8583]

Thanks videofan3d, I'll try your plugin this afternoon also ;-)
We are going from 0 plugin to 2 :-D

[Sharc]

Nice, as it already seems to support 2 elementary streams rather than a combinedMVC as input, if I read this correctly .....

[Nico8583]

DGMVCSource should support separated elementary streams soon :)

[Nico8583]

@videofan3d : how about left or right base view ? Can we choose first view or is it automatic like neuron2's plugin ?

[videofan3d]

@videofan3d : how about left or right base view ? Can we choose first view or is it automatic like neuron2's plugin ?

Do you mean two "base" views: for L and R eye, i.e. like two completely separated .h264 files from two cameras?
(they will be anyway likely encoded independently)

Than I guess you don't need extra plugin, just
L=DirectShowSource("L.h264")
R=DirectShowSource("R.h264")
StackHorizontal(L,R)

Or do you mean something else?

[r0lZ]

I guess Nico asks how to swap the two views. As you know probably, the left view is usually in the AVC stream, and the right view in the MVC. But it's not always the case, and when the views are inverted, we must have a way to swap them, to build a correct SBS or T&B combined view, with the left view first.
With pistacho's plugin, this can be done by specifying a parameter. With DGMVCSource, it's in the AVS script itself that you swap the views, because it supports a limited possibility to seek to the next or previous frame. I suppose that your plugin can do the same thing if the cache argument is set to at least 2. Right?

See here (http://forum.doom9.org/showthread.php?p=1662554#post1662554) for more info.

[videofan3d]

I guess Nico asks how to swap the two views. As you know probably, the left view is usually in the AVC stream, and the right view in the MVC. But it's not always the case, and when the views are inverted, we must have a way to swap them, to build a correct SBS or T&B combined view, with the left view first.
With pistacho's plugin, this can be done by specifying a parameter. With DGMVCSource, it's in the AVS script itself that you swap the views, because it supports a limited possibility to seek to the next or previous frame. I suppose that your plugin can do the same thing if the cache argument is set to at least 2. Right?

See here (http://forum.doom9.org/showthread.php?p=1662554#post1662554) for more info.

I see.

Parameters are always
file="base_view.h264"
dependent="dependent_view.h264"

in SBS/TAB format the left/top portion corresponds to base view (regardless it is L- or R-eye)

Is they need to be swapped, then Avisynth functions Crop(), StackHorizontal() will do the job.

[videofan3d]

To explain cache parameter

Plugin allocates a buffer of size "cache=N" for last N frames.
Which means that you can partially scroll in "Whatever"-Media Player - unless you are in cache.

You can test it like

FRIMSource (..., cache=24, num_frames=120)

when playing in MediaPlayerClassic or Stereoscopic Player, you can scroll through last 1 second (and scrolling before cache-size will display first frame in cache).

I assumed it will be necessary for some temporal filters in Avisynth to be able to retrieve -N frames (from current position).

Be reasonable - it is impossible to allocate cache for whole movie!

[jdobbs]

@videofan3d

I posted a note in the encoder thread (http://forum.doom9.org/showthread.php?p=1662918#post1662918) -- I can't seem to get FRIMEncode to accept the AVISYNTH script using FRIMSource. Can you point out what I'm doing wrong?

[r0lZ]

@videofan3d: Yes, the cache concept is simple to understand. But I forgot to take into account that your plugin returns already a video with the stacked frames (horizontally or vertically, depending of the "layout" parameter). Therefore, the two frames (corresponding to the left and right views) are served at the same time, and there is no need to buffer anything. However, IMO, the lack of an option to swap the 2 views automatically when it's necessary is an important limitation. Having to use Crop two times and StackHorizontal (or Vertical) just to restore the correct order of the views is a pity. It's possible, but relatively slow, and that kills the advantage of the Stack being done in the plugin itself. Are you sure that it is not possible to tell the FRIM decoder to swap the two views?

[videofan3d]

@videofan3d

I posted a note in the encoder thread (http://forum.doom9.org/showthread.php?p=1662918#post1662918) -- I can't seem to get FRIMEncode to accept the AVISYNTH script using FRIMSource. Can you point out what I'm doing wrong?

Please check setting of VFW in you system (ffdshow setup) - it seems that VFW does some extra conversion and does not provide data in YV12 format.

[videofan3d]

@videofan3d: Yes, the cache concept is simple to understand. But I forgot to take into account that your plugin returns already a video with the stacked frames (horizontally or vertically, depending of the "layout" parameter). Therefore, the two frames (corresponding to the left and right views) are served at the same time, and there is no need to buffer anything. However, IMO, the lack of an option to swap the 2 views automatically when it's necessary is an important limitation. Having to use Crop two times and StackHorizontal (or Vertical) just to restore the correct order of the views is a pity. It's possible, but relatively slow, and that kills the advantage of the Stack being done in the plugin itself. Are you sure that it is not possible to tell the FRIM decoder to swap the two views?

I understand your point.

Intel Media doesn't provide information if the base-view is L- or R-eye (I didn't find it in documentation, probably it is even not stored in the elementary stream). I guess that also Bluray3D keeps this info somewhere in clip-info or playlist file. So anyway user need to set it up manually :(.

I will put it to my plans to add it to FRIMDecode kernel.

Fortunately, my Panny Z10000 stores L-eye in base view, and most of BD3D use also this natural order :).

[r0lZ]

... and most of BD3D use also this natural order :).Right. But the "most" stinks.

@Neuron2: Sorry to have squatted your thread.

[Guest]

Right. But the "most" stinks.

@Neuron2: Sorry to have squatted your thread. It's not my thread. But I have split my stuff off to its own thread now as things are getting difficult to follow in here after Frank joined in with his independent development.

http://forum.doom9.org/showthread.php?t=170107

[videofan3d]

@WBL:
.. the intel decoder, that accepts only the AVC+MVC combined stream in a very specific format.

I don't think that combined AVC+MVC it is "very specific format".
Moreover, I think it is some kind of standard format defined by a Motion Picture Engineering Group (MPEG).
Because it is single unified 3D/MVC elementary stream, that can be multiplexed with audio and possibly transported via streaming devices (network, cable, satellite ...). Unfortunately not supported by end-user devices and therefore appearing like "very specific".
Pity, it would bring significant streaming benefit to 3D/MVC.

On the contrary, SSIF is very specific, and in my opinion very ugly "format" derived from Bluray physical medium layout.
It is some kind of virtual sector-multiplex, because it physically doesn't exist on BD3D disk. There are only physical .m2ts files (always a pair containing base and dependent elementary streams), which are virtually sector-multiplexed into SSIF file - some kind of "symbolic link".
Reason behind this was probably technological:
due to backward compatibility with older BD players it was necessary to store dependent view into separate file, but both views need to be read and decoded together.
And because laser head cannot swiftly swing and read two different physical files, engineers had to invent virtual SSIF sector-multiplexed file which is read by laser head continuously and feeds decoder with information from both base and dependent view.

Technically is SSIF interesting, but conceptually very ugly "format".

[r0lZ]

I fully agree. I just wanted to highlight that the combined file is at the same time somewhat "muxed" since there are two streams in one file, but also "elementary", because it is not in a container. Also, there are only a few tools able to build such combined files, and for that reason, I don't like it much. But I agree that SSIF is ugly, but there are many tools that can handle it, because it is very frequent.
Anyway, even if the terms I have used are not perfect, I wanted to explain why the Intel decoder cannot be fed directly with complex files such as SSIF.

[Thalyn]

Hi,

just for fun I started few days ago similar plugin

LoadPlugin ("some_path\FRIMSource.dll")
FRIMSource (codec="mvc", file="some_path\SRC_L.h264", dependent="some_path\SRC_R.h264", layout="sbs", cache=24, num_frames=250)

which is based on kernel of already tested FRIMDecode.

Since scrolling through elementary stream is not supported, I added there at least a cache for last N frames which can help a bit in case of subsequent temporal filters.

If anyone is interested - you can download this beta 1 from FRIMSource_v1.zip (https://drive.google.com/file/d/0BymRNDHq74DEQVdTN2RiUXNyQXM).

Remark: to be used together with libmfxhw32.dll or libmfxsw32.dll from Intel Media SDK or FRIM Package (http://forum.doom9.org/showthread.php?p=1650646#post1650646)

It seems to work fine (yes, I made sure I checked the right files this time!). However, there's two minor issues, at least when using sequential output (no "layout" value):

1) The framerate is halved. It accepts that the video is 24000/1001, but the combined stream has the same so splitting them results in 12000/1001. This can be corrected with AssumeFPS, hence minor.

2) The frame count is effectively halved. The value given to Num_Frames is the total number of frames you get back - which means if you're alternating back and forth you only get half as many of each (AVC/MVC), resulting in half the framecount if you use OU or SBS. Doubling the frame count corrects it, hence minor.

Otherwise, no issues to speak of.

[videofan3d]

It seems to work fine (yes, I made sure I checked the right files this time!). However, there's two minor issues, at least when using sequential output (no "layout" value):

1) The framerate is halved. It accepts that the video is 24000/1001, but the combined stream has the same so splitting them results in 12000/1001. This can be corrected with AssumeFPS, hence minor.

2) The frame count is effectively halved. The value given to Num_Frames is the total number of frames you get back - which means if you're alternating back and forth you only get half as many of each (AVC/MVC), resulting in half the framecount if you use OU or SBS. Doubling the frame count corrects it, hence minor.

Otherwise, no issues to speak of.


(Points for improvement/correction for next version ;) )
Thanks for feedback!

[r0lZ]

I have tested FRIMSource too, and indeed, it works fine, with the exception of the two minor problems reported by Thalyn, and the lack of the possibility to swap the views in SBS or TAB mode, as reported by Nico8583 and me. (It is however possible to use the interleaved mode and SelectEven() and SelectOdd() in any order, so again it's a relatively minor problem.)

But I have been impressed by its (partial) seek support. It is possible to seek in the buffer, forward and backward, as explained in the doc. But it is also possible to seek forward to any point anyway (even when the buffer is null). It's slow, as obviously all intermediate frames must be decoded, but it works. Can you confirm that it's really supported? For example, can we be sure that the correct frame is returned after a Trim() ?