Avisynth MVC decoder plugin ?

[videofan3d]

I have tested FRIMSource too, and indeed, it works fine, with the exception of the two minor problems reported by Thalyn, and the lack of the possibility to swap the views in SBS or TAB mode, as reported by Nico8583 and me. (It is however possible to use the interleaved mode and SelectEven() and SelectOdd() in any order, so again it's a relatively minor problem.)

But I have been impressed by its (partial) seek support. It is possible to seek in the buffer, forward and backward, as explained in the doc. But it is also possible to seek forward to any point anyway (even when the buffer is null). It's slow, as obviously all intermediate frames must be decoded, but it works. Can you confirm that it's really supported? For example, can we be sure that the correct frame is returned after a Trim() ?

I plan to add bool "swap" for sbs|tab in next release, just patience, please :)

Internally, FRIMSource reads only forward (via cache), from 0 till num_frames. When jumping forward, as you noticed, all frames to the jump-point need to be read and decoded (via cache). If - by chance - you set num_frames bigger than real frames in the stream, then black frames are added at the end.

[r0lZ]

OK, so that confirms that the decoder can seek (although slowly) to any frame, as long as the target frame number is higher than the current frame number. It's good to know, as that permits to trim the beginning of a movie.

Don't worry, I am patient. I just wanted to summarize the current (and minor) problems of your decoder. Also, I wanted to write that that swap problem is not relevant if you use the interleaved output mode, just like with DGMVCSource.

Now, my biggest problem is that I have to decide which filter to use in my BD3D2MK3D GUI. FRIMSource, or DGMVCSource? That is the question! :p
I think I will add an option to let the user decide... :cool:

[videofan3d]

OK, so that confirms that the decoder can seek (although slowly) to any frame, as long as the target frame number is higher than the current frame number. It's good to know, as that permits to trim the beginning of a movie.

Don't worry, I am patient. I just wanted to summarize the current (and minor) problems of your decoder. Also, I wanted to write that that swap problem is not relevant if you use the interleaved output mode, just like with DGMVCSource.

Now, my biggest problem is that I have to decide which filter to use in my BD3D2MK3D GUI. FRIMSource, or DGMVCSource? That is the question! :p
I think I will add an option to let the user decide... :cool:

Just to add: if you jump to some previous frame, FRIMSource will return it from cache. If this frame is not in the cache anymore, it returns the first (=oldest) frame from the cache.
(I believe it is better behavior than returning black frame, because cache is primarily intended to support consequent temporal filters, not for scrolling)

(I guess this all you already realized during testing :) )

[r0lZ]

(I guess this all you already realized during testing :) )
Yes, but thanks for the precision.

[jdobbs]

This is related to Thalyn's report. When I am encoding a non-3D source with X264 (using FRIMSource as an AVISYNTH input), I am seeing the framerate as half what it should be, the number of frames reported to be encoded is also halved. So it doesn't seem to be related only to using MVC -- but to the count itself.

To be fair, though, I am using an interlaced source, which the documentation says hasn't been tested. I'll try it again with a progressive source.

It seems to be working correctly on a progressive source. So... uhhh... forget everything I've just said. ;)

[Guest]

Now, my biggest problem is that I have to decide which filter to use in my BD3D2MK3D GUI. FRIMSource, or DGMVCSource? That is the question! :p
I think I will add an option to let the user decide... :cool: Well, mine doesn't work yet, so there's no choice yet. :)

[Nico8583]

Well, mine doesn't work yet, so there's no choice yet. :)
I don't doubt about your plugin ;)
I think both are good plugins :)

[videofan3d]

Hi,

FRIMSource beta 2 is available - download from FRIM Package (http://forum.doom9.org/showthread.php?p=1650646#post1650646) page.

Changes:
2014-01-21:
- added parameter "swaplr"
- renamed parameters "filename" and "filename_dep" - to be aligned with other standard functions in Avisynth documentation
- fixed (internally doubled) frame rate and num_frames for frame sequential output
- created simple manual FRIMSource_readme.pdf

[r0lZ]

Thanks! I'll check it tomorrow...

[Sharc]

Thanks videofan3d. First quick test looks good..... :)

[Thalyn]

So far I've put it up against Escape from Planet Earth and the big test: The Croods. Both came back without any flaws on a quick viewing but I won't know for certain until I watch them through from start to finish.

[Nico8583]

So far I've put it up against Escape from Planet Earth and the big test: The Croods. Both came back without any flaws on a quick viewing but I won't know for certain until I watch them through from start to finish.
Could you post your Avisynth script please ?

[Thalyn]

Since they're working this is the full script I use without any modifications to make sure my extra crap isn't causing problems (like the cut-down one I was trying with DGMVCSource). It's a "cover all" script I knocked up to save time editing other scripts and making mistakes in the process.

## *Load plugins
LoadPlugin ("T:\BRD\FRIMSource.dll")
LoadPlugin ("T:\BRD\SupTitle.dll")
LoadPlugin ("T:\BRD\VSFilter.dll")
## *Source files
SP = "T:\BRD\" # Source path (must end in "\")
VF = "Croods" # AVC & MVC file names (raw streams only)
FC = 141865 # Frame count
SFL = "Subs.sup" # Left subtitle file (SUP, SUB/IDX, SSA, ASS, SRT, SUB, SMI, PSB, USF or SSF)
SFR = 12 # Subtitle horizontal offset (.SUP only) or right subtitle file (non-SUP)
# Good offsets are 6-12. Can be set to SFL for 0 offset non-.SUP
LC = 0 # Left cropping (typically 0)
TC = 132 # Top cropping (typically 0, 132, 138 or 140)
RC = LC # Right cropping (typically the same as LC)
BC = TC # Bottom cropping (typically but not always the same as TC)
## *Load video file - default assumes AVC is left, change 2 to -2 for AVC right.
Source = FRIMSource (codec="mvc", filename=SP + VF + ".264", filename_dep=SP + VF + ".MVC", \
cache=4, num_frames=FC)
## *Crop stream
Source = Source.Crop (LC, TC, -RC, -BC)
## *Split interleaved frames
Left = SelectEven (Source)
Right = SelectOdd (Source)
## *Hard subtitles - add/change as necessary.
Try { # Attempt to add subtitle file as offset PGS
Left = Left.SupTitle (SP + SFL, forcedOnly=false, relocate=true, \
relocOffset="0, " + String (TC) + ", " + String (SFR) + ", 0")
Right = Right.SupTitle (SP + SFL, forcedOnly=false, relocate=true, \
relocOffset=String (SFR) + ", " + String (TC) + ", 0, 0")
}
Catch (errPGS) { # Try to read subtitles with VSFilter if not PGS format
Try { # First attempt to read as IDX/VOB
Left = Left.VobSub (SP + SFL)
Right = Right.VobSub (SP + SFR)
}
Catch (errVOB) { # If not IDX/VOB, attempt to read as text subs
Try {
Left = Left.TextSub (SP + SFL) # Recommend SSA/ASS for text subs
Right = Right.TextSub (SP + SFR)
}
Catch (errTXT) { NOP } # If subtitle file not found or not compatible, do nothing at all.
}
}
## *Merge to HOU.
Try { # Attempt to merge frames normally
HOU = StackVertical (Left.BilinearResize (1920 - LC - RC, (1080 - TC - BC) / 2), \
Right.BilinearResize (1920 - LC - RC, (1080 - TC - BC) / 2))
}
Catch (errCrop) { # If reduction fails, convert to RGB for the merge then back to YV12
Left = ConvertToRGB (Left, matrix = "rec709")
Right = ConvertToRGB (Right, matrix = "rec709")
HOU = ConvertToYV12 (StackVertical (Left.BilinearResize (1920 - LC - RC, (1080 - TC - BC) / 2), \
Right.BilinearResize (1920 - LC - RC, (1080 - TC - BC) / 2)), matrix = "rec709")
}
## *Output - adjust trim as necessary.
return HOU.Trim (0, FC-1)
As you can see, adjusting those 9 variables near the start lets me pretty quickly alter it to accept just about anything I could want to plug into it. For example, Escape from Planet Earth would have me changing VF to "Earth", FC to "128348" and TC to "128" - everything else remains the same.

[frencher]

As I ivestigated,
Intel Media SDK 2013 has MVC decoder bundled into the libmfxsw32.dll (or libmfxsw64.dll for 64bit apps). This decoder accepts only combined MVC streams.
I can write AviSynth plugin that accepts combined stream and produses both views (as ssifSource1 did). But it is not the solution.

Here (http://forum.doom9.org/showthread.php?p=1663432#post1663432) from neuron2 ;)
Thanks to all ;)

[Guest]

Use this to extract the two streams:

eac3to 00001.ssif -demux

This shows how the streams are muxed and frencher gave you the link to the code:

http://forum.doom9.org/showthread.php?p=1662762#post1662762

[videofan3d]

FRIM 1.21

http://forum.doom9.org/showthread.php?p=1664207#post1664207

FRIM Source included

@everybody who is dealing with his/her own video from 3D camcorder (Panasonic Z10000, Sony TDx series)
FRIM Exporter for Adobe Premiere CS6

[allanlee]

FRIM 1.21

FRIM Exporter for Adobe Premiere CS6

WoW!!! Excellent work! Thank you soooo much!
:p

[videofan3d]

FRIM package 1.22 available: support reading transport stream .ts/.mts/.m2ts directly

see info FRIM package 1.22 - transport stream (http://forum.doom9.org/showthread.php?p=1665615#post1665615)


Avisynth example for Panasonic 3D-Z10000 (with audio):

FILE3D="Z10000.m2ts"
V=FRIMSource(codec="mvc", filename=FILE3D, filename_dep=FILE3D, ts=true, layout="sbs", cache=24, num_frames=300)
A=DirectShowSource(FILE3D).KillVideo()
AudioDub(V,A)

[jdobbs]

FRIM package 1.22 available: support reading transport stream .ts/.mts/.m2ts directly

see info FRIM package 1.22 - transport stream (http://forum.doom9.org/showthread.php?p=1665615#post1665615)


Avisynth example for Panasonic 3D-Z10000 (with audio):

FILE3D="Z10000.m2ts"
V=FRIMSource(codec="mvc", filename=FILE3D, filename_dep=FILE3D, ts=true, layout="sbs", cache=24, num_frames=300)
A=DirectShowSource(FILE3D).KillVideo()
AudioDub(V,A)
Cool, cool. This saves a lot of time and trouble.

Thanks!

[Guest]

Hi Frank, great work!

I wonder if you would like to comment on my speculations here in regards to some bad streams that are handled OK by your combiner but not mine. I'm curious as to whether it is serendipitous or you designed things to accept malformed streams somehow.

http://forum.doom9.org/showthread.php?p=1665680#post1665680

And hey, why not go ahead and implement full frame accuracy. It would save me a lot of work. :)

[videofan3d]

And hey, why not go ahead and implement full frame accuracy. It would save me a lot of work. :)

Having frame accuracy would be great! But...

Intel provided us SDK/API and in their samples described framework for sequential processing.
This is what FRIM does, and for most tasks - 3D encoding and transcoding - it is well sufficient.
Also sequential transport stream to elementary stream demuxing is straightforward routine...
So the only matter was to put all these things together and after ~3-4 months (occasionally - evenings, weekends) to get working sequential decoding/encoding solution.

Frame accuracy seeking - I'd like to have it.
But I'm afraid it far above my current knowledge of H.264 format as well as TS-container. (H.264 is not simple, MVC is even more complicated, and documentation is not very much available, difficult to read/understand)
And it is definitely far far above my time capacity to dive into it. :D
Just imagine: Adobe took over 3 full versions of their Premiere CSx to implement Mercury Engine which supports relatively reliable frame accurate seeking over H.264 in M2TS container.

[jdobbs]

Having frame accuracy would be great! But...

Intel provided us SDK/API and in their samples described framework for sequential processing.
This is what FRIM does, and for most tasks - 3D encoding and transcoding - it is well sufficient.
Also sequential transport stream to elementary stream demuxing is straightforward routine...
So the only matter was to put all these things together and after ~3-4 months (occasionally - evenings, weekends) to get working sequential decoding/encoding solution.

Frame accuracy seeking - I'd like to have it.
But I'm afraid it far above my current knowledge of H.264 format as well as TS-container. (H.264 is not simple, MVC is even more complicated, and documentation is not very much available, difficult to read/understand)
And it is definitely far far above my time capacity to dive into it. :D
Just imagine: Adobe took over 3 full versions of their Premiere CSx to implement Mercury Engine which supports relatively reliable frame accurate seeking over H.264 in M2TS container.Neuron2 has a lot of experience in this area. He has implemented several frame-accurate AVISYNTH plugins -- and others have tried and failed. ;) I don't think anyone is more qualified to get it done.

[videofan3d]

Neuron2 has a lot of experience in this area. He has implemented several frame-accurate AVISYNTH plugins -- and others have tried and failed. ;) I don't think anyone is more qualified to get it done.

I have no doubt about him (I also use his DGMPGDec and DGAVCDec :) )...

[Guest]

Thanks for the kind words guys!

Frank, the Intel documents also talk about how to perform repositioning in the bitstream, aka seeking. You really only need to know about a few NALU types that must be injected before seeking. And if you seek to a non-anchor frame, you back off to the previous anchor, decode the right number of frames, and deliver the last one. That's it! It's like your "fake" trim handling where you silently decode frames at the start and deliver only the last requested frame. So instead of being always starting from the start of the stream, you start from the nearest anchor frame. The tricky part, and it's not too difficult, is to manage the injection of the right SPS and PPS. Then, of course, it should all work correctly for all video types and the major containers, and pulldown must be properly honored, so careful modular design is needed. For extra credit support multiple decode engines in one application. Hopefully it isn't biting off more than can be chewed.

Good luck, we're waiting. :D

[KreuzBlick]

We are able to compute the number of frames from the length of the audiostream. So I propose:

FILE3D="Z10000.m2ts"
A=DirectShowSource(FILE3D).KillVideo()
count=Floor(1.0*Audiolength(A)/AudioRate(A)*25)
V=FRIMSource(codec="mvc", filename=FILE3D, filename_dep=FILE3D, ts=true, layout="sbs", cache=24, num_frames=count)
AudioDub(V,A)

[Sharc]

@videofan3d
Perhaps you have followed the discussion in the DGMVCsource thread. For some 3D-BD sources both DGMVC and FRIM fail at same scenes, and there are indications that the root cause of the problem may be with the Intel SDK. Do you have some ties to Intel which could help?
Thanks.

[Guest]

Answering for myself, AFAIK, the only tie is through the Intel forum. But we need a small clip(s) that we can provide to demonstrate the issue.

[Sharc]

Answering for myself, AFAIK, the only tie is through the Intel forum. But we need a small clip(s) that we can provide to demonstrate the issue.
Small clip is here. See the glitches close to the end of the Clip. Hope it helps!
http://www.mediafire.com/download/bib2za0uxzcctfm/pr.zip

[videofan3d]

Small clip is here. See the glitches close to the end of the Clip. Hope it helps!
http://www.mediafire.com/download/bib2za0uxzcctfm/pr.zip

This example pr.zip seems to be corrupted source.

FRIM Decode somehow decoded it but frame 138-144 are corrupted.

I tried to decode it also using "ldecod" - the JM H.264/AVC Reference Decoder, and it even crashed on frame 138 (with error message "AllocPartition: Memory allocation for streamBuffer failed").

[Sharc]

This example pr.zip seems to be corrupted source.

FRIM Decode somehow decoded it but frame 138-144 are corrupted.

I tried to decode it also using "ldecod" - the JM H.264/AVC Reference Decoder, and it even crashed on frame 138 (with error message "AllocPartition: Memory allocation for streamBuffer failed").
Hmmm... interesting.
I wonder now where the corruption/crash has it's origin:
a) the source disc
b) ripping
c) demuxing the .ssif
d) cutting and creating the clip

Several people using different media independently have reported these glitches, and similar glitches have been reported from other discs as well using FRIM or DGMVC. The base view of the clip (pr.264) is playable without any anomalies with SW-Players here. Does "Idecod" crash on the base view pr.264?

[videofan3d]

Hmmm... interesting.
I wonder now where the corruption/crash has it's origin:
a) the source disc
b) ripping
c) demuxing the .ssif
d) cutting and creating the clip

Several people using different media independently have reported these glitches, and similar glitches have been reported from other discs as well using FRIM or DGMVC. The base view of the clip (pr.264) is playable without any anomalies with SW-Players here. Does "Idecod" crash on the base view pr.264?

Yes, exactly - I passed only base view pr.264 to "ldecod" - and it crashed after frame 138

Some other SW players are probably more resistant to errors in original stream while ldecod and Intel Media not.

Btw. some time ago I compared yuv-output from IntelMedia (FRIMDecode) and ldecod. And they were bit-to-bit identical.
(Which is good, it shows that IntelMedia decoder is quite precise)

[r0lZ]

That's strange IMO. In all test I did with the Intel library, the problem was present, but I did also a lot of tests with other demuxers, libraries or players and NONE have had a problem. (I haven't tried with ldecod, but I will do a quick test soon.) Also, that BD comes from a big company, and it should have been correctly encoded. So, as I wrote in the DGMVCSource thread, I'm almost sure the Intel Library has a bug, or at least, it has a weakness that no other AVC decoders have (except perhaps ldecod).

[slavanap]

Here is the possible solution for this problem:
http://forum.doom9.org/showthread.php?p=1667073#post1667073

[Guest]

Here is the possible solution for this problem:
http://forum.doom9.org/showthread.php?p=1667073#post1667073 How can that be a solution?

[slavanap]

Hi, neuron2,
the task was stated like this:
Does anyone could make an Avisynth plugin for MVC decoding based on Intel Media SDK 2013 R2 ?

First of all, the plugin I developed uses only free available software. Second, it uses a pipeline: ssif-file => MpegSplitter => MVCCombine => Intel Decoder => output
As I known MVCSource uses MVCCombine-like algorithm to combine two streams. As next release I plan to add a parameters to be able to start from any step of the pipeline. Say, you need to specify both base & dependent h264-files, or one h264-muxed file of them, or several ssif-files to join. It will be easy, because all pipeline has been built already.

And I apologize, I haven't noticed the publication of FRIMSource plugin that deals (I guess) with this problem.
Sorry for any inconvenience.

[kypec]

Sorry for any inconvenience.
Looks like you totally missed the point - there are streams from genuine Bluray disks that exhibit issues when decoding certain frame sequences through Intel Media SDK components. This findings make us (tool users) and also developers to believe that there are bug(s) in Intel Media decoding library and need to be verified by Intel support. ;)

[Nico8583]

I hope Intel support will respond (quickly) to neuron2 :)

[slavanap]

Here (http://forum.doom9.org/showthread.php?p=1663432#post1663432) from neuron2 ;)
Thanks to all ;)

Oh, sorry. You guys had so many posts in different topics that January, so I haven't read them all before posting.

Maybe this MPEGSplitter I used to demux the ssif-file (instead eac3to & tsMuxer) can fix this issue with the films.
The same MPEGSplitter used with CoveAVCDecoder for ssifSource2 & DirectShowMVCSource, though it delivers both (main & dependent) streams independently right into CoreAVCDecoder.

[allanlee]

Maybe this MPEGSplitter I used to demux the ssif-file (instead eac3to & tsMuxer) can fix this issue with the films.
The same MPEGSplitter used with CoveAVCDecoder for ssifSource2 & DirectShowMVCSource, though it delivers both (main & dependent) streams independently right into CoreAVCDecoder.

I posted my test results here (http://forum.doom9.org/showthread.php?p=1667914#post1667914).
Just wondering is there anything special with your sample_decode.exe?

You did a good job ;) , but I'm really confused.

[Selur]

Small question: When using FRIMSource on 2D VC-1 interlaced content, is there a way to keep FRIMSource from deinterlacing the content (wanted to use QTGMC for deinterlacing)?