DGMVCSource

[r0lZ]

Tested with your files and the problem is clear. The dependent stream runs out of frames when the credits start, presumably because the credits are not in 3D. The EOF on the dependent stream currently stops things. Fixing...Oh, I was far to imagine that it is possible to mix 2D and 3D in the same MPLS! The fix is welcome. :-)

[Guest]

That was just a guess. :) Now I am back from the pool I will find out what is really happening.

[Guest]

My end of stream handling was defective and could lose up to 3MB of the end of the stream. I wasn't flushing the combined stream buffer properly. I have fixed it and offer you beta 8:

http://neuron2.net/misc/dgmvcsource_b8.zip

I have removed the sample files to make downloads faster. If you need them, get them from an earlier beta, or use your own source files.

Also note that I have updated DGMVCCombine to version 1.0.1 with the same fixes:

http://neuron2.net/dgmvccombine/dgmvccombine101.zip

[Sharc]

Thank you for the new beta.
No issues so far with beta 8 and separate streams.
Anything we should pay special attention to?

Is the speed bottleneck given by the decoding process?

For View=0: Do the even frames always present the base view, and the odd frames the dependent view?

[r0lZ]

For View=0: Do the even frames always present the base view, and the odd frames the dependent view?Yes. Swap the "left" and "right" clip names in the SelectEven and SelectOdd commands to restore the common order (left first) if the order is inverted in the original BD.

[Sharc]

Yes. Swap the "left" and "right" clip names in the SelectEven and SelectOdd commands to restore the common order (left first) if the order is inverted in the original BD.
ok, that's what I assumed. No problem as it is.
I was just in doubt if the decoder would eventually read from the stream (flag or similar) which one is left or right and make the assignment accordingly.

[r0lZ]

That info is not in the stream. Only in the MPLS.

[r0lZ]

Finally got some time to test b8. It's better, but the problem is not totally fixed.
I have encoded Puppy again. The end credits are now partially visible, but they are cut at approx 4:25, about 8 seconds before the end of the movie. So, it seems that the eof fix is not sufficient. I can do other tests if you need so...

[Guest]

Third time is the charm! I hope. Anyway, for me beta 9 now delivers all the frames from your streams:

http://neuron2.net/misc/dgmvcsource_b9.zip

[r0lZ]

Indeed, this one is OK. I'll do some other tests in the next days, but I think you can go on with the next thing in your todo list. :-)

[Guest]

Indeed, this one is OK. I'll do some other tests in the next days, but I think you can go on with the next thing in your todo list. :-) Sweet, thank you for your testing. Your provision of this short really sped things up for me because I didn't have to wait over an hour each time to hit EOF.
:thanks:

I want to make a source code drop as I promised and then I'll move on with my to-do list. Please let me know if you need anything for immediate usability.

BTW, DGMVCCombine didn't require an update. It was a problem only in the source filter; it wasn't signaling the decoder to take the last partial buffer of combined data (after EOF). Things should be robust now.

[r0lZ]

No, afaik, I can use the plugin as it is.

Can I assume that the parameters will not change? Or at least that, if it is necessary to add new parameters, that they will be added after the current ones, for compatibility reasons?

[Nico8583]

I have made a quick test with hardware acceleration enable (with videofan3d plugin) and result is very smooth on my i5-3360M. Without hardware acceleration, display is slow, I don't have 23.976fps, all frames don't appear on screen but with hardware acceleration, video play like a charm, very smooth ;) so I think hardware acceleration will be a good improvment :)
I don't have tried x264 encode for the moment...

[Guest]

Can I assume that the parameters will not change? Or at least that, if it is necessary to add new parameters, that they will be added after the current ones, for compatibility reasons? Yes, I will follow that protocol.

[Guest]

I think hardware acceleration will be a good improvement. HW acceleration is the next item on my to-do list, but thank you for reminding me about it. :p

[Nico8583]

HW acceleration is the next item on my to-do list, but thank you for reminding me about it. :p
It was only to tell you that is not useless :D

[Sharc]

beta9 / separate files input: It appears that the last few frames (10....20) of the source don't get encoded. The black filler frames at the end seem to kick in too early.
I am using DGIndexNV for playback.

[r0lZ]

Damn! I haven't noticed that with my test, because the last frames of the original "Puppy" movie are black anyway. I will test with other streams...

[Sharc]

Hmmm.... strange. I can't reproduce it now.
It got missing end frames in repeated tests when I posted, but now running the same script and cmd, all seems to be good.
Unless someone else discovers something similar, let's assume that the issue was just a local one here....

Edit:
Ugggh ... it appears that my all-clear came to early. Now it happened again - even worse. At about 80% of the file the black filler frames start to kick in. It seems not to be reproducible though :confused:

[Guest]

That's surprising. I will make a debug build for you and post it in a little while.

[Guest]

Here is a debug load:

http://neuron2.net/misc/dgmvcsource_b10.zip

To debug with it first get the DebugView utility here:

http://neuron2.net/dbgv.zip

Open that and do Edit/Filter and then put this in the include field: DGMVCSource
Then do OK.

Now start your decode. You should see a message immediately that DGMVCSource has been instantiated. At the completion of the encode, wait 15 seconds before doing anything else. If you see a timeout message tell me what it says (it gives a timeout number and a frame number). Also give your script. Thank you.

[Sharc]

[2592] DGMVCSource: timeout 3 at frame 3218

The encode looks ok, no missing frames at the end.

Script:
LoadPlugin("c:\Program Files Video\AviSynth 2.5\plugins\DGMVCDecode.dll")
# View Parameter:
# 0 : interleaved
# 1 : base view (AVC)
# 2 : dependent view (MVC)

Video = DGMVCsource("c:\Program Files Video\FRIM\panama.264","c:\Program Files Video\FRIM\panama.mvc",frames=1650,view=0)
V1 = SelectEven(video)
V2 = SelectOdd(video)
return StackHorizontal(horizontalreduceby2(V1),horizontalreduceby2(V2)) # for half SBS x264 input


Edit:
Now I continue with repetitions of the encode until I catch one which fails. Maybe after 10 or 20 attempts ...

[Guest]

It may not fail anymore. :) I lengthened the timeout. Note that the timeout should kick in only when you specify the frames parameter with more frames than the source contains. It may have been triggering falsely before I increased the timeout. And when proper random access is implemented, this whole issue goes away because you won't need to specify frames and the filter will never get requests for frames that don't exist.

[Sharc]

Great. Thanks!

[Guest]

I think I found a way to stop properly without needing a timeout. Testing it...

[Guest]

OK, here is beta 11. Decoding will switch immediately to black frames after the end of stream without needing a timeout. If you ever see a timeout, then we need to investigate that, assuming the stream is not hosed.

http://neuron2.net/misc/dgmvcsource_b11.zip

[Sharc]

beta 11:
[12244] DGMVCSource: timeout 3 at frame 596

No luck ....

Added:
The next attempt went well:
[5624] DGMVCSource: Decoder has delivered all available frames. Setting end of stream EOS.

Added 2:
Failed again:
[6584] DGMVCSource: timeout 3 at frame 2420

I'll PM you a link.

[Guest]

Are you transcoding? I can't believe the encoder would hold up the decoder for 10 seconds but I suppose it is possible. Can you tell me how you are encoding, your command line, etc., so I can try to duplicate this? Thanks.

Maybe I'll just remove all the timeouts.

[Sharc]

Yes, I transcode.
Command:
x264.exe "c:\Program Files Video\DGMVCSource_.avs" --bluray-compat --crf 22 --vbv-bufsize 15000 --vbv-maxrate 15000 --sar 1:1 --fps 23.976 --output "out_DGMVCSource-to-x264.h264"
pause
The script is same as posted earlier.

[Guest]

OK, I have your streams you sent in PM. Will try to duplicate your timeout...

[Sharc]

OK, thanks.
I'll be away for about 2 hours now.

[Guest]

I transcoded with your source files and encode line five times without issues. Maybe you have a network drive involved?

[r0lZ]

Are you transcoding? I can't believe the encoder would hold up the decoder for 10 seconds but I suppose it is possible.That can happen due to external conditions, such as heavy disc access during the decoding. In that case, the bottleneck is the speed of the disc, not the transcoding. I have such problems easily with my current hardware and Win8.

Maybe it's time to add a "timeout" parameter? Or at least while the plugin is still under heavy development, so that we can test with other values.

[Guest]

@r0lZ

Have you seen any timeouts?

[r0lZ]

No, but currently, I can't do tests. I hope I'll be able to do a few tests tomorrow...

[Guest]

I'll just remove the timeouts. They serve no purpose now anyway, because I get the end of stream signal from the decode pipeline.

[Guest]

Here is beta 12 with timeouts of 120 seconds. :sly:

http://neuron2.net/misc/dgmvcsource_b12.zip

[r0lZ]

... I get end the of stream signal from the decode pipeline.Does it mean that you know for sure when the movie finishes? If yes, then it might be a good idea to stop adding black frames, even if the user has specified more frames in the parameters. But I'm not sure it's a good idea.

And I wonder what your decoder does if the two streams do not have exactly the same number of frames. Will it truncate the additional frames, or try to decode them anyway? (It's purely an academical question. My GUI passes the lower number as the argument anyway.)

[EDIT] Thanks for b12. :-)

[Guest]

Does it mean that you know for sure when the movie finishes? If yes, then it might be a good idea to stop adding black frames, even if the user has specified more frames in the parameters. But I'm not sure it's a good idea. Good question. Yes, I know for sure when there are no more frames. But you have to return something to Avisynth's GetFrame() call. You can't just block in there. A proper source filter will set vi.num_frames to the correct number, without needing a frames parameter and without requests ever being made for frames that do not exist.

And I wonder what your decoder does if the two streams do not have exactly the same number of frames. Will it truncate the additional frames, or try to decode them anyway? (It's purely an academical question. My GUI passes the lower number as the argument anyway.)
Our behaviors match up, because I end the stream when either the base or dependent streams run out of frames.

[Sharc]

beta 12:
Same again. It seems to stall at some frame, waiting for something. CPU load is almost Nil.
[6572] DGMVCSource: timeout 3 at frame 2420

I seem not to get this with FRIM plugin.

Doing more tests ...

[Guest]

Really strange. I've run your scenario many times. How often does it happen?

[Sharc]

I can have up to say 20 runs without issues. I have also seen it happening ever 2nd or 3rd run. I can't spot a regular or specific pattern. It's weird.

Edit:
I wonder why the timeouts refer to frames >> total number of the source frames (1608) or frames=1650 parameter.
How are the frames counted/numbered?

Edit2:
After 12 successful runs:
[6540] DGMVCSource: timeout 3 at frame 2822
CPU load 0%, very low I/O disc activity.

Yawn .... time for me to go to sleep.

[Sharc]

I can't produce a timeout when running the script in MPC-HC, so far. It seems to produce these random hangers only when frameserving to x264.
I'll try with an different version of x264.

Edit:
No joy. With the latest build of x264 r2389 I got after 12 successful runs
[6100] DGMVCSource: timeout 3 at frame 2950
(I assume that a stereo pair counts as 2 frames?)

[r0lZ]

I have a big problem with b12, when trying to decode The Blue Umbrella with this script:

LoadPlugin("DGMVCDecode.dll")
DGMVCSource("00261.track_4113.264", "00261.track_4114.mvc", 9748, 0)

The script doesn't work when launched from the command prompt. When I try to preview it with AvsPmod, it displays this error message, after its traceback:

AttributeError: AvsClip instance has no attribute 'display_pitch'
And it displays -1 as the current frame number in its GUI. It seems that the plugin cannot decode any more that movie, that I have already decoded without problem.

FRIMSource works fine.

Any idea?

[EDIT] Hum, really strange. I can't get b12 to work with another movie too. So, I've tried b11, then b10 and b9, and I have the same behaviour with all versions! The Source command cannot decode any frame.
I don't understand. Something must have changed in my computer that makes all DGMVCDecode plugins to fail. But I wonder what! I have not changed or installed anything important recently.

[Guest]

@Sharc

In interleaved mode, the printed frame number is doubled. For the timeout, I suppose somehow the two threads are getting into deadlock. I will review the design and add some code to detect deadlocks and give you a new debug version to help us understand what is happening. BTW, I did your encode 200 times straight without issues.

@r0lZ

Well that is baffling. Have you tried rebooting? What happens if you just load the script in VirtualDub?

[Sharc]

I have a big problem with b12, when trying to decode The Blue Umbrella with this script:

LoadPlugin("DGMVCDecode.dll")
DGMVCSource("00261.track_4113.264", "00261.track_4114.mvc", 9748, 0)

The script doesn't work when launched from the command prompt. When I try to preview it with AvsPmod, it displays this error message, after its traceback:

AttributeError: AvsClip instance has no attribute 'display_pitch'
And it displays -1 as the current frame number in its GUI. It seems that the plugin cannot decode any more that movie, that I have already decoded without problem.

FRIMSource works fine.

Any idea?

[EDIT] Hum, really strange. I can't get b12 to work with another movie too. So, I've tried b11, then b10 and b9, and I have the same behaviour with all versions! The Source command cannot decode any frame.
I don't understand. Something must have changed in my computer that makes all DGMVCDecode plugins to fail. But I wonder what! I have not changed or installed anything important recently.
Re-install Avisynth?

[r0lZ]

Damn, that was my fault! :o
I have copied, pasted and modified a command from another script, and I have accidently changed the order of the frames and views parameters. DGMVCSource has just encoded the requested number of frames: 0.
Sorry for the false alarm!

So, now, I am able to do some new tests, and I have tested b12 with The Blue Umbrella. Unfortunately, it seems that there are still a few frames missing at the end of the movie. I am currently investigating the problem, and I will come back here if I find why.

[r0lZ]

Forget my previous bug report. I just did a new test, after having rebooted my PC, and this time, it went OK. The prev encode may have failed because I had still b10 in memory. Anyway, if I encounter the problem again, I will report it here.

[Guest]

@sharc

Would you please re-download beta 12? I have added full debug. If you can run DbgView and when a timeout occurs give me the contents of the DbgView capture buffer. That should tell me what I need to see what is happening.

Thanks!

[Sharc]

I resumed the testing with the re-downloaded beta 12. Surprisingly no timeout/deadlock after about 20 runs.
I'll do more testing later this week.