DGMVCSource

[r0lZ]

Yes, a buffer overflow is a good candidate for that kind of erratic errors.
I'll post a sample from Paperman for you, but without guarantee that you'll see anything wrong.

[Guest]

Thanks for the sample in PM. Right now I am ripping Escape from Planet Earth and will test with that initially.

[Sharc]

In order to speed up the plugin+encoder testing time significantly I think one could reduce the output resolution of the encodes, e.g. to 1280x720 or even lower, rather than keeping it at full HD. This should not adversely affect the validity of the test regarding glitches. Correct or am I missing something?

[Guest]

In order to speed up the plugin+encoder testing time significantly I think one could reduce the output resolution of the encodes, e.g. to 1280x720 or even lower, rather than keeping it at full HD. This should not adversely affect the validity of the test regarding glitches, correct? Correct. But I would ask if encoding is even required to see the issue. I am testing right now.

[Sharc]

oooops! I just run into an issue:
- BD: Man of Steel, Base view garbled about 30 minutes into the movie
- Decoder: DGMVCDecode.dll beta 4 (I missed your update)
- Decoder Input: separate elementary streams
- Encoder: FRIM mvc

I'll try with beta 5 now.

Edit:
Same problem with beta 5 (at slightly different position it seems).
As I did this movie successfully before with FRIM (pipe), I tend to believe that the problem is with DGMVCdecode ...:o

[Guest]

I found an underrun condition on the reader/writer that I have fixed. Testing further locally before pushing a new beta.

[frencher]

I created a video with AVC / MVC flow and it seems that the MVC view is not dependent on the stroke order in any case we can put 2 different views, very strange. :stupid:
Someone can tell me why?
AVC+MVC - Not dependent view.m2ts (http://ul.to/79mi47e2)

[Guest]

I created a video with AVC / MVC flow and it seems that the MVC view is not dependent on the stroke order in any case we can put 2 different views, very strange. Sorry, I don't understand your report.

[r0lZ]

I suppose he wants to say that we can use the AVC stream of one BD with the MVC stream of another one. Indeed, it's not a good idea, but nothing can stop us to do strange things like that.

(@Frencher: Si je n'ai pas compris ton post, dis-le en français, et je traduirai.)

[frencher]

I suppose he wants to say that we can use the AVC stream of one BD with the MVC stream of another one. Indeed, it's not a good idea, but nothing can stop us to do strange things like that.

(@Frencher: Si je n'ai pas compris ton post, dis-le en français, et je traduirai.)
@r0lZ ;)

Yes

What I mean is that the AVC standard / MVC was created to slow the development of software.
Is that we expect in the near future (Ultra HD 4k 8k) to see the same flower x265/UltraMVC system with a folder like "/ SSIFUltra/00001.ussif"
In brief the MVC view is not so dependent upon ???

[Guest]

@frencher

I'm sorry but again I find your post to be unintelligible. Can someone please explain what he's talking about?

If it's OT, take it elsewhere!

[frencher]

@frencher

I'm sorry but again I find your post to be unintelligible. Can someone please explain what he's talking about?

If it's OT, take it elsewhere!

You understand the source code Intel SDK and see how it works.
I wanted to know if it uses very basic view for decoding dependent view knowing that we just use two different videos in AVC / MVC stream?
It is true that this is a little off topic but it is to understand the principle of decoding.
It was also a sample for your test to improve DGMVCSource.
You can delete my posts relating my question if you really find out about and apologize.

In french
Vous comprenez le code source Intel SDK et vous voyez comment il travaille.
C'était pour savoir si celui-ci utilise bien la vue de base pour décoder la vue dépendante sachant que l'on peu utiliser deux videos différentes dans un flux AVC/MVC ?
C'est vrai que c'est un peu hors sujet mais c'est pour comprendre le principe du décodage.
C'était aussi un échantillon pour que vous puissiez le tester pour améliorer DGMVCSource.
Vous pouvez supprimer mes posts relatant ma question si vous les trouvez vraiment hors sujet et m'en excuse.

[Guest]

I still don't understand. I just don't know what you mean by "very basic view". Anyway, the decoder doesn't know we have two files, because I combine them before passing to the decoder.

[Thalyn]

Have you tried with pistacho plugin or videofan3d plugin ? It's to determine if its cause this problem too

I'm a little reluctant to mention other decoders (the whole, "But X does it like Y - why don't you do it that way!?" whining) and though I've already done it, yes I have tested with FRIMSource. No issue presented using the two separate input streams (cache set to 4, in case it's relevant in any way, shape or form).

Interestingly, the transcode I ran last night bombed out as soon as it ticked to 5:34, or what should be frame 8,008. MeGUI is apparently less tolerant than VirtualDub of whatever is going AWOL.

[Tom Keller]

@neuron2

I know, there are more mandatory things to do and (maybe more... but hopefully no) problems to solve... but just out of curiosity:

Are you planning on adding direct support for *.mts files (created by MPEG4-MVC/H.264 AVCHD 3D camcorders... like Sony's HDR-TD10, or like Panasonic's Z10000) in a future release of DGMVCDecode?

Btw... seems like 'frencher' uses Google to translate his native French text into English. Since automatic translations are far from being perfect (or even 'good'), it's no big surprise, that he's hard to understand ;) . Maybe 'r0lZ' can come to the rescue... again?

[Guest]

As I said MANY TIMES I don't want to discuss future stuff in this thread. Is it so hard to understand? :devil:

[Guest]

I'm a little reluctant to mention other decoders (the whole, "But X does it like Y - why don't you do it that way!?" whining) and though I've already done it, yes I have tested with FRIMSource. No issue presented using the two separate input streams (cache set to 4, in case it's relevant in any way, shape or form).

Interestingly, the transcode I ran last night bombed out as soon as it ticked to 5:34, or what should be frame 8,008. MeGUI is apparently less tolerant than VirtualDub of whatever is going AWOL. It's broken, OK. I am fixing it. I know what the problem is and I am working on it. Please be patient or use another filter if you have immediate needs.

[Guest]

I just found and fixed the bug. Expect a new beta tomorrow morning.

[Selur]

Nice!
Btw. will DGIndexMVC also have an option to check if the system supports it?

[r0lZ]

In french
Vous comprenez le code source Intel SDK et vous voyez comment il travaille.
C'était pour savoir si celui-ci utilise bien la vue de base pour décoder la vue dépendante sachant que l'on peu utiliser deux videos différentes dans un flux AVC/MVC ?
C'est vrai que c'est un peu hors sujet mais c'est pour comprendre le principe du décodage.
C'était aussi un échantillon pour que vous puissiez le tester pour améliorer DGMVCSource.
Vous pouvez supprimer mes posts relatant ma question si vous les trouvez vraiment hors sujet et m'en excuse.
Here is my (approximative) translation of his message:
You [Neuron2] understand well how the Intel decoder works.
I want to know if it uses really the base view to decode the dependent view, given the fact that it is [theoretically] possible to put two totally different videos in the two streams.
It is true that this question is somewhat out of topic, but I want to understand the principle of the decoding.
[The next sentence is very obscure for me, even in French:] It's also a sample [or idea?] for you to improve DGMVCSource.
You can delete my posts related to this question if you think they are really OT. In that case I'm sorry.

I can already reply that yes, in the 3D BDs, the dep view is really dependent of the base view. They did that that way because it's a good way to gain disc space. There is only a few differences between the base and dep views, and it is necessary to encode that differences only in the MVC stream. It's why the demuxed MVC file is always much smaller than the h264 file of the base view.
And yes, IMO, that question is out of topic, as it is not related directly to the development of DGMVCSource.

[r0lZ]

I just found and fixed the bug. Expect a new beta tomorrow morning.
Might be too late, but I've finished to examine the test encodes I did during the night.

I have encoded 10 times Paperman with the combined.264 file created with MVCCombine: No problem at all.

I have also encoded 10 times the same movie but this time with the 2 ES and the internal combiner. There are errors in 3 files. (It is easy to find them, because the size of the files are different.) Two times, some garbage appears during one GOP or two, and one time, the movie stops abruptly and is followed by totally black frames.

Given the fact that these errors are different at each encode, I guess that the buffer bug you have fixed is indeed the cause of the problem. Is it in the code you have added to combine the two streams? It it's the case, you have probably fixed the right bug. :-)

I will repeat that test when the new beta will be available...

[Tom Keller]

As I said MANY TIMES I don't want to discuss future stuff in this thread. Is it so hard to understand? :devil:
I must have skipped those passages - sorry :( . I didn't want to bother you. Keep up the good work!

[Thalyn]

It's broken, OK. I am fixing it. I know what the problem is and I am working on it. Please be patient or use another filter if you have immediate needs.

Better too much information than not enough. Please don't confuse that for impatience - I understand how long these things can take, especially to do it right.

I'm pretty sure everyone else here also understands that, and we all appreciate your efforts. You're helping us out and we're just doing the best we can to return the favour; albeit without actually being able to program.

I just found and fixed the bug. Expect a new beta tomorrow morning.

Looking forward to giving it a run through. I've kept the streams to make proper comparisons.

[allanlee]

@neuron2

I know, there are more mandatory things to do and (maybe more... but hopefully no) problems to solve... but just out of curiosity:

Are you planning on adding direct support for *.mts files (created by MPEG4-MVC/H.264 AVCHD 3D camcorders... like Sony's HDR-TD10, or like Panasonic's Z10000) in a future release of DGMVCDecode?

Btw... seems like 'frencher' uses Google to translate his native French text into English. Since automatic translations are far from being perfect (or even 'good'), it's no big surprise, that he's hard to understand ;) . Maybe 'r0lZ' can come to the rescue... again?

To my knowledge, MTS is pretty similar to m2ts or ssif stream, TSMuxer and eac3to can already deal with such format. So all you need to do is demux into elementary streams and feed them into either MVCCombine or directly DGMVCSource.
I tested that TSMuxer and eac3to work pefectly with the 2D MTS generated by Sony and JVC cameras. But since I don't have a 3D MTS sample file -- maybe you can have a test? :p

[Guest]

Might be too late, but I've finished to examine the test encodes I did during the night.

I have encoded 10 times Paperman with the combined.264 file created with MVCCombine: No problem at all.

I have also encoded 10 times the same movie but this time with the 2 ES and the internal combiner. There are errors in 3 files. (It is easy to find them, because the size of the files are different.) Two times, some garbage appears during one GOP or two, and one time, the movie stops abruptly and is followed by totally black frames.

Given the fact that these errors are different at each encode, I guess that the buffer bug you have fixed is indeed the cause of the problem. Is it in the code you have added to combine the two streams? It it's the case, you have probably fixed the right bug. :-)

I will repeat that test when the new beta will be available... My fix stopped one problem, as it started decoding past the point where it was going nuts before. But later in the stream the decoder threw an exception and crashed. I am making the combined file and will compare the streams delivered in the two cases to see what is going wrong in the combiner.

Yes, I do plan to add container support.

[Guest]

Oy, fseek() does not work for files >16GB. Replacing with _fseeki64(). Waiting for 70,000 frames each test run is inconvenient but there is no other way. I am hopeful...

[Guest]

Seems to be working fine now. I have to teach a swimming lesson and when I get back I'll post a new beta.

[r0lZ]

Good news! But it is time to sleep here. I'll test your beta tomorrow...

[Guest]

Here is beta 6 with bug fixes and performance improvements for the combiner.

http://neuron2.net/misc/dgmvcsource_b6.zip

If you think Franks's pseudo seeking is useful, that would be easy to add. I'm thinking more in terms of unlimited full frame-accurate random access without long delays when seeking. Isn't that a requirement for a correct source filter?

[Thalyn]

Using exactly the same "Escape from Planet Earth" source, I have good news and bad news.

The good news is that it appears all the errors I was encountering previously are gone and the transcode went through to the end. Previewing with VirtualDub was also happening in constant real-time, so viewing without transcoding was much easier on the brain.

The bad news is that a brief glitch (on both AVC and MVC frames, starting with the MVC) hit at frame 14,100 (~6,000 frames past where it previously stalled) and affected 24 frames, followed by the output going completely blank at frame 22,895 - and that's how it stayed for the rest of the movie. What makes these two occurrences particularly unusual, at least to me, is that the glitching at frame 14,100 appeared in both the x264 encode and when previewing with VirtualDub, however only the transcode went blank while VirtualDub continued playing as normal. I'm running the transcode again to see if that's an isolated occurrence and will update shortly when I know more.

Update:
Second transcode attempt has passed frame 22,895 without speeding up to over 8x realtime, so I can only assume it's not compressing blank frames this time. I'll let it run to the end (approximately 80 minutes) and see what else, if anything, has changed.

Update 2:
Dangit. Glitch at 14,100 still occurred (as I suspected), but unfortunately so did the blanking. This time at frame 26,066. I may not know much about modern programming, but I know inconsistency is a pain to diagnose... sorry.

[Guest]

Make sure you clear your cache and re-download beta 6 as I had a bad version up for quite a while. I don't see any issues with that disk on my side.

I may not know much about modern programming, but I know inconsistency is a pain to diagnose... sorry. You could make it a lot easier by giving your full script.

There is some simulated blocking drawn by the artist at some scene changes. I hope you aren't confusing them with glitches.

Video goes black if your frames parameter is too low. You may have to double the frames parameter in interleaved mode depending on your script.

[Thalyn]

Re-downloading now just in case - I would have originally grabbed it shortly after your edit so it could be the bad version. I'll run it again when I get an idle moment on my rig as I like to test speed as well, but that may not be for a couple hours yet. Will keep you posted. I'll also include the script I used at exactly the time I started it, just to make sure I'm not doing anything daft.

I'm fairly confidant that they're not deliberate "glitches" in the output. My connection is happy today (touch wood!) so these are exactly what I was seeing: Frame 14,099 (fine) (http://s204.photobucket.com/user/Lord_Thalyn/media/Fine_zps821a987d.png.html), Frame 14,100 (going) (http://s204.photobucket.com/user/Lord_Thalyn/media/Going_zps6251ccdb.png.html), Frame 14,101 (gone!) (http://s204.photobucket.com/user/Lord_Thalyn/media/Gone_zps8903bb3c.png.html)

[r0lZ]

If you think Franks's pseudo seeking is useful, that would be easy to add. I'm thinking more in terms of unlimited full frame-accurate random access without long delays when seeking. Isn't that a requirement for a correct source filter?Ideally yes, but that's not necessary if you just want to transcode. But it's very useful to preview the video or to split a movie in several independent parts.
I suppose that will require to build an index, and if it's time consuming, I propose to add an option to skip the index creation and, of course, keep the current behaviour.

Now, I will test b6...

[Thalyn]

Well, I'm at a loss. I keep getting that exact same error in the exact same place, and every time I run it I get slightly further through the transcode before it goes black (up to 57,116 now!). If you [Neuron2] aren't having an issue than it's probably safe to say that the issue is at my end. I'm not exactly sure what it is that's causing it but that's all I can come up with.

That being the case, it's possible that further feedback from me won't necessarily be useful if I'm unable to figure out what the problem here is and correct it.

The exact script I'm using is as follows:
## *Load plugins
LoadPlugin ("T:\BRD\DGMVCDecode.dll")
## *Load video files.
Source = DGMVCSource ("T:\BRD\Escape.264", "T:\BRD\Escape.MVC", view = 0, frames = 256696) # Frames is doubled
## *Crop stream
Source = Source.Crop (0, 128, 0, -128)
## *Split interleaved frames
Left = SelectEven (Source)
Right = SelectOdd (Source)
## *Merge to HOU.
HOU = StackVertical (Left.BilinearResize (1920, 412), Right.BilinearResize (1920, 412))
## *Output.
return HOU
A fully-updated MeGUI is being used to do the transcode.

[allanlee]

I'm thinking more in terms of unlimited full frame-accurate random access without long delays when seeking. Isn't that a requirement for a correct source filter?

Adding the frame-accurate seeking is fantastic! And I agree that if indexing (like DGNV) is required, having a method to skip it (and remain in sequencial access mode) is also very useful.
Thanks for the great work! Tested b6 on several sample clips and so far no probem. :-)

[r0lZ]

Tests done. I have again transcoded 10 times the Paperman short movie, with the 2 ES as input, and everything went fine. So, for me, the bug is fixed. :-)

Well, I'm at a loss. I keep getting that exact same error in the exact same place, and every time I run it I get slightly further through the transcode before it goes black (up to 57,116 now!). If you [Neuron2] aren't having an issue than it's probably safe to say that the issue is at my end. I'm not exactly sure what it is that's causing it but that's all I can come up with.Could it be a bad rip?

[Thalyn]

It's possible but unlikely. I've used the same files with other source filters without issue, and even as the input for MVCCombine which I then fed through beta 3 without a problem.

Guess it won't hurt to rip it again and try, though, just in case.

[Nico8583]

Have you tried without crop and with a resize filter other than bilinearresize ?

[Thalyn]

I really should do a "essentials" run just to make absolutely sure.

That having been said, re-ripping it is proving awkward - I'm trying to ensure I use a different method to what I used previously (first rip simply used TSMuxer to extract from the SSIF with AnyDVD HD active), and in doing so I found that MakeMKV would fail every time regardless of which of my drives I used. Given that, I can't say for certain that it wasn't a bad rip, so I can't say with any certainty that my feedback using this particular title is of any actual value. I'm trying working from the M2TS files to see if there's any change.

However, I think I might step back from this thread. Given I don't trust the value of my own feedback I can't reasonably expect it to be of any value to Neuron2. I do intend to keep following and testing, though - just quietly.

[Guest]

@Thalyn

You're always welcome here to post your feedback and I will certainly pay attention and try to help you. At the moment it seems you may have some system issues, as I experience no issues with your title and no-one else has reported further issues. When you have evidence of an issue in DGMVCSource(), or course, I encourage you to bring it up.

@all

I have released my stand-alone MVC streams combiner (open source equivalent of MVCCombine) with source code:

http://neuron2.net/dgmvccombine/dgmvccombine100.zip

[r0lZ]

Just did a new short test with Puppy (a short animated movie from Despicable Me 2), and... the last part of the movie, corresponding to the end credits, is totally blank! I'm encoding it again with FRIMSource, to verify if it works, and I'll report the result later. I may have to do a lot of tests to verify, so don't expect a confirmation today...

I have released my stand-alone MVC streams combiner (open source equivalent of MVCCombine) with source code:

http://neuron2.net/dgmvccombine/dgmvccombine100.zip
Is it exactly identical to the one embedded in DGMVCDecode? I can perhaps combine several movies with your version and the original MVCCombine.exe, and compare them to see if there are differences.

[Guest]

Just did a new short test with Puppy (a short animated movie from Despicable Me 2), and... the last part of the movie, corresponding to the end credits, is totally blank! I'm encoding it again with FRIMSource, to verify if it works, and I'll report the result later. I may have to do a lot of tests to verify, so don't expect a confirmation today... OK, please keep me informed. And if you can give me the stream for testing I can look at it. Make sure you specify the correct number of frames, and see the new beta linked below, which changes how the frames parameter is handled.

Is it exactly identical to the one embedded in DGMVCDecode? I can perhaps combine several movies with your version and the original MVCCombine.exe, and compare them to see if there are differences. Yes, they use an identical algorithm.

Here is beta 7, which internally doubles the frames parameter when delivering interleaved (view= 0).

http://neuron2.net/misc/dgmvcsource_b7.zip

[Guest]

Please re-download beta 7 if you already got it. I increased some timeouts that might have explained a clip going prematurely black.

[r0lZ]

404 Not found!

Unfortunately, I can confirm the problem: It is OK when encoded with FRIM, but the end credits are blank for the two tries I did with DGMVCSource. Luckilly, this time, the problem seems reproducible. I will try to upload the 2 ES soemwhere, but be patient. I have to upload around 1GB, and I have to go now...

Small request: Can you add the version of the build in the files properties of the DLL? Currently, it is always 1.0.0.0. And, afaik, the file properties are the only way to verify if we use really the latest version.

[Sharc]

Can't access the beta 7. 404 file not found.

Edit: r0IZ was faster, sorry.

[r0lZ]

Please re-download beta 7 if you already got it.Can you upload it? The link is still dead.
[EDIT] Shark was faster.

[Guest]

404 Not found! Sorry, fixed.

Unfortunately, I can confirm the problem: It is OK when encoded with FRIM, but the end credits are blank for the two tries I did with DGMVCSource. Luckilly, this time, the problem seems reproducible. I will try to upload the 2 ES soemwhere, but be patient. I have to upload around 1GB, and I have to go now... OK, with the streams I can fix it if it still happens with the b7 redownloaded.

Small request: Can you add the version of the build in the files properties of the DLL? Currently, it is always 1.0.0.0. And, afaik, the file properties are the only way to verify if we use really the latest version. Yes, for future betas.

[r0lZ]

The problem persists with b7. I will upload the two ES for you when I'm not at home, and I should give you the link when I'm back, in 5 hours or so.

[Sharc]

beta 7: 51'000 frames transcoded with DGMVCSource beta 7 + FRIMEncode.
No issues so far, way past the position where I had the former failure with beta 5.
Ongoing ......

Edit:
And oh yes, true seeking would be great, option for trim, perhaps even cutting a snippet ....

[Guest]

The problem persists with b7. I will upload the two ES for you when I'm not at home, and I should give you the link when I'm back, in 5 hours or so. Tested with your files and the problem is clear. The dependent stream runs out of frames when the credits start, presumably because the credits are not in 3D. The EOF on the dependent stream currently stops things. Fixing...