LAV Filters - DirectShow Media Splitter and Decoders [Archive] - Page 428

nevcairiel

3rd December 2016, 00:31

On my end everything looks fine. I didn't check on a 3D screen yet, but LAV Splitter reads 5 different offset ids for the 5 PGS subtitle streams (ids 1 to 5 for the 5 streams), and the decoder reads per-frame offsets for those 5 ids and reports them to the renderer (in total it reads 32 offsets, but only the first 5 are used for subtitles, which is pretty common for Blu-rays to just declare all 32 offset slots even if most are zero)

There is about 30 seconds of subtitles at the end of the samples, can you confirm the issue already exhibits right there? The sample goes until around 04:40 (until "I was named after a swimming pool"). And if so, with which subtitle stream specifically?

madshi

3rd December 2016, 01:05

Yep, would be good to know which exact subtitle (runtime and exact text) is shown incorrectly, and which subtitle track.

BTW, do you guys also have problems quoting someone? If I try, I get server error 500.

Kotik

3rd December 2016, 02:47

I am getting server error 500 half the day:-) i will play the movie with my glasses on to tell u exactly the moment it looks wrong. But i can tell u already that during the lizard scene (towards the end of the file) the movie cuts through the subtitles.
I noticed something very interesting. Subtitles in 3D movies have various depth based on the current scene, i have no idea how this depth is instructed, but i know for sure that their depth is never static, to see this u need to watch the movie without glasses on.
Whether these depth instructions are stored in the subtitle file itself or the .mpls file is beyond me. But it should never be static, this is where the lizard scene comes, the lizard has a pop out of the screen effect and the subtitles should also get this effect, but in this particular scene they remain unaffected, by noticing this i started paying attention to the rest of my movies and i indeed can confirm that in the working movies the subtitles are constantly changing their depth.
I hope this information is of some help. Meanwhile i am trying to obtain another copy of life of pi disc. But yes @nevcairiel, towards the end of the file, the lizard scene is totally off for me and most of the subs that follow that scene are off too.
Let me know if u need further info.

Kotik

3rd December 2016, 03:03

Ok let me be more specific: minute 04:13, subtitle line "who was there to check on the Bengal monitor lizard."

This is absolutely an eye killer:-) the lizard pops out of the screen while the subtitles look as if they are behind the lizard.

Sarasa

3rd December 2016, 12:26

@madshi

Yeah seem the forum as caught a cold, in my case some thread show also as empty (for example xy-VSFilter & ASSFilterMod)

Aleksoid1978

3rd December 2016, 14:00

nevcairiel

3rd December 2016, 15:08

Kotik is right. MadVR set "3D depth" static for any subtitles - 4 pixel offset when call subtitles redraw. But in SEI can stored different values for different frames/timestamp. LAV Video Decoder read it and store in IMediaSideData interface. I think MadVR just ignore it.

This entire interface was designed for madVR, so I would be surprised if it just ignores it.

Another problem - that "3D depth" may be different for different subtitles tracks for one frame/timestamp. But - madVR don't know what subtitles track is rendering right now. I think need get biggest value for "3D depth"(pixel offset) and use it.

LAV Splitter can tell it which subtitle track is currently active, and therefor which offset index to use (ask IPropertyBag for "stereo_subtitle_offset_id", thats the index for the currently active subtitle track)

Kotik

3rd December 2016, 15:28

Kotik is right. MadVR set "3D depth" static for any subtitles - 4 pixel offset when call subtitles redraw. But in SEI can stored different values for different frames/timestamp. LAV Video Decoder read it and store in IMediaSideData interface. I think MadVR just ignore it.

Another problem - that "3D depth" may be different for different subtitles tracks for one frame/timestamp. But - madVR don't know what subtitles track is rendering right now. I think need get biggest value for "3D depth"(pixel offset) and use it.

I am pretty sure that madvr properly reads dynamic subtitle depth for most movies, how else could we explain that it works with some other 3D movies? I tried yesterday Despicable Me 2 3D and this one works perfectly fine by either opening the .mpls file or after repacking it with makemkv. And i can confirm that subtitle are dynamic during the whole movie in this case. It is just Life Of Pi that has this issue, and this is why i am trying to obtain another disc of this movie. At this point i am not sure of anything, it is a very strange issue and since i am unable to reproduce it with my other movies.

Kotik

3rd December 2016, 15:48

Nevcairiel could explain please if the offset index is present in the .mpls or is it located in the .m2ts?
Why i am asking, during the tests with other movies i noticed that subtitle 3D depth is properly working even with external subtitles which is a nice feature, but if i completely remove the 3D subtitle from the .mkv then the external subtitles do not get any dynamic depth.
This led me to the conclusion that the depth index is not stored in the subtitle itself but rather somewhere else and this is activated only when an internal su title track is present. Is that a correct theory or i am talking bs:-)?

nevcairiel

3rd December 2016, 15:50

The m2ts has a list of different offsets per video frame, up to 32 different "slots" that subtitles and interactive elements can use, for each single video frame. The mpls assigns these "slots" to subtitle tracks and other elements. So you really need both for it to function properly. The mpls to tell you "subtitle track #1 uses offset slot #1", and the m2ts info that tells you "offset slot #1 uses an offset of 5 pixels for this video frame".

Kotik

3rd December 2016, 16:11

Thanx for the info, I think now I understand why external 2D subtitles .idx/.sub actually work if u play a 3D .mkv movie. I think what happens in this case is that external subtitles are assigned the same offset slot with the internal one and then they behave the same way with the internal one during playback. This also explains why it doesn't work if I remove the internal subtitle track from the .mkv, there is no offset slot present for madvr/lav to tie the external sub to.

CruNcher

3rd December 2016, 19:20

@VictorLS

Did you get my PM ?

VictorLS

3rd December 2016, 21:55

@VictorLS

Did you get my PM ?

Yes, I've answered you.

Aleksoid1978

4th December 2016, 05:05

Currently MadVR always call draw subtitles with 4 pixel offset.

CruNcher

4th December 2016, 05:24

@Aleksoid1978

With Aleksoids Patch if by default HQ DXVA Processing is checked then your RussiaHD.ts plays with block distortions (errors) with CUVID if unchecked it plays without block distortions,
but when HQ DXVA Processing is unchecked then LG_4K_View-the-Feeling.ts is falling back to avcodec if checked again it uses CUVID but then again RussiaHD.ts gets block distortions.

Player is MPC-BE EVR-CP, nothing restarted just on the fly uncheck load play look at the status.

So this means currently if i want to play RussiaHD.ts without errors (block distortions) i have to uncheck HQ DXVA Processing but then i lose CUVID for LG_4K_View-the-Feeling.ts unless i check it again then it works via CUVID and the Hybrid HEVC Decoder as it should.

This is a relative small usage problem but still not convenient.

So for me it's currently impossible to playback RussiaHD.ts and LG_4K_View-the-Feeling.ts with the same setup via CUVID error free either i chose HQ DXVA Processing on or off one file is going to fail, both stable via CUVID isn't possible with the same HQ DXVA Processing off i have to check/uncheck it based on which file i want to playback accelerated via CUVID, that's a usage problem.

Any Idea why your Patch behaves like that with the LG_4K_View-the-Feeling.ts Stream when HQ DXVA Processing is unchecked and why it then falls back to avcodec instead of using CUVID as with HQ DXVA Processing checked ?

Aleksoid1978

4th December 2016, 07:42

@Aleksoid1978

Any Idea why your Patch behaves like that with the LG_4K_View-the-Feeling.ts Stream when HQ DXVA Processing is unchecked and why it then falls back to avcodec instead of using CUVID as with HQ DXVA Processing checked ?
I'm not interesting to support it :)

NikosD

4th December 2016, 08:50

@NikosD

Yes i tried MPDN Renderer it works ok with DXVA Copy Back and Lav Video for the CUDA Decoder but it has major issues with stable Multithreaded CPU Playback with Lav Video (decoding queue stalls very fast) compared to EVR-CP in MPC-BE

Also Potplayer works rather ok with its D3D Renderer but jitters to much with DXVA Copy Back on EVR-CP compared to CUVID with the CUDA Decoder as well.

I was trying to reply since yesterday but obviously the forum had some problems.

I was referring to test Cyberlink's and Potplayer's decoders, not renderers.

Test different hybrid HEVC decoders using same EVR-CP on your same platform (OS, drivers etc)

nevcairiel

4th December 2016, 10:54

Please move discussions about other decoders, players or renderers to other threads, thanks. :)

NikosD

4th December 2016, 11:14

I was mentioned by CruNcher in his post and I just wanted to be polite and reply to him in order to clarify my suggestion.

There is no discussion actually.

As I said before, it is no so meaningful to insist on hybrid HEVC decoders as they are actually an intermediate stage from SW to fixed-fuction HW decoders.

Also, I have clearly understood that you don't invest so much time in optimizing further your decoders as long as they work fine.

But you have to understand that sometimes we have to refer or mention other decoders than yours, in order to make comparisons, find bugs or find the best solution/ implementation for the situation described by the user.

CruNcher

4th December 2016, 13:59

Your suggestion would mean investing into new Hardware for some target that still can be achieved on the current one absolute no go for me, especialy seeing this is a core stability issue it needs to be fixed in some way ;)

Cyberlinks decoder is pure crap except of its very nice decoding tricks layer especially in combination with sliced bitstreams the rest is out of date.

It's performance is still inferior to a decoder of the 2015 SOA time period and that's ridiculous.

Which might be suggesting that Cyberlink is about to giving up to Hardware Decoders fully in their targets.

nevcairiel

4th December 2016, 14:11

Also, I have clearly understood that you don't invest so much time in optimizing further your decoders as long as they work fine.

What you have "understood" is far from reality, through.
I am always open to optimizations within reason (ie. a decent advantage to be gained)

What you have to understand however is that hardware decoding is a huge black box. You give the video to the hardware and get images back. All I can do to optimize this is one of two things - make sure everything on my side is as efficient as possible, which brought you things like dxva2cb "direct" mode, and secondly try to tweak the very few knobs that the hardware interface gives you - which are very little, and the ones I do have (or know about) are already tuned differently for different hardware based on testing.

So "optimizing" hardware decoding is mostly a big guessing game, try random changes and see if something changes. Clearly this has no guarantees of success and without any insight from the respective GPU manufacturers also quite unproductive.

If anyone has any suggestions on how to make hybrid decoding work more smoothly, do speak up.

CruNcher

4th December 2016, 14:20

It's surely not easy especially with all the different combinations of Hardware Firmwares and self made Bios versions and Driver as well.

But anyway i wouldn't suggest implementing Aleksoids patch as it is currently is it will endup in a lot of negative Feedback and support needs as switching off DXVA HQ Processing has negative issues here on the Hybrid CUDA Decoder (unless explicitly advising that it could endup in CUVID not working on specific streams in maybe certain hardware OS and render combinations if DXVA HQ Processing is turned off and in it's worst case fallback to avcodec then) im not sure why as i don't understand yet the relation of it and what is going wrong here or if anything at all is going wrong or if it's even expected to behave like this from Nvidias side for some reason (uknown).

Surely it could be my System stability as well i can't guarantee that's not the case at all, but it looks more like something going wrong on the Software interop layer of things.

So as it stands now it really fixes the Playback Render error behavior for RussiaHD.ts but it has side effects, that can overshadow it using it as default especially (the fallback to avcodec where it's not supposed to fallback with DXVA HQ Processing enabled).

But it really is a viable workaround currently for fixing these Render issues no doubt about that arguable though if it's the right way todo it, but it could be especially for Low Latency Broadcast users with specific Bitstream Decoding needs (as VictorLS clearly is Hardcore) the salvation for their Problems as RussiaHD.ts clearly shows.

Maybe it would be even time to get Nvidia Engineers already into the Discussion.

Though i didn't test so much Bitstreams Container combinations myself yet to see if their is maybe a detectable pattern about this issue or if it's a general interop issue (maybe even a issue on the specific bitstream signal layer).

Better would be much more Feedback from different Systems with Aleksoids patch applied and by default HQ DXVA Processing disabled to maybe find a useful pattern :)

VHT

5th December 2016, 09:39

It is just Life Of Pi that has this issue, and this is why i am trying to obtain another disc of this movie. At this point i am not sure of anything, it is a very strange issue and since i am unable to reproduce it with my other movies.

I've had the same 3D subtitle problem (finnish subs.) with Jungle Book and new Ghostbusters using LAV+madVR.

Kotik

8th December 2016, 21:38

Good evening, i think i got some extra info regarding the 3D subtitles issue, i already ordered from amazon.de a german version of the Life Of Pi disc, meanwhile yesterday i started troubleshooting a bit deeper and i made an interesting discovery!!

So we know that there are movies out there that are using different eye information for base view, i did an experiment, i started the movie and reversed the 3D view on my TV settings, guess what!! the 3D subtitles this way look fine but the movie now looks wrong, as if these two elements exchanged the issue, now i wonder, do we respect the base view (which eye is used as base view) information for subtitles as well? i know we do it for the movie, but are we using this information for subtitles as well?

nevcairiel

8th December 2016, 21:47

I don't think subtitles have this information, but I can check. Maybe madVR accidentally applied that information wrong to the subtitles.

Kotik

8th December 2016, 21:57

Would be nice to know, cause it is a huge coincidence that if i reverse the 3D base view, suddenly the non working 3D subtitles start working perfectly fine, of course the video looks crazy this way but it is an indication that something is wrong around that area (which eye is used for base view), it could also explain why most people don't notice this issue, it is possible that the CEE release of Life Of Pi has reversed eye for base view? it is a possibility and this is why i made sure to order a completely different version of the movie this time.

Kotik

8th December 2016, 22:17

@nevcairiel is there an easy way to see whether a movie has reversed base view? and if there is could u point me to the right direction?

Thanx

nevcairiel

8th December 2016, 22:19

I checked the specs, and subtitles don't care what order of views the video uses. Left eye should always use "+offset" and Right eye always "-offset". madshi would have to say if thats how he implemented it in madVR.

For the base view, LAV tells you in the stream listing. "lr" at the end means "left-right" (ie. normal), and "rl" the opposite.

madshi

8th December 2016, 22:22

@Kotik, please don't use PM and forum at the same time. I read this thread, so if you post here, you don't have to PM me, as well.

@nevcairiel, how did we solve the 3D eye swapping for those weird movies where it's the unusual order? Are you doing the swapping or am I doing that? Couldn't find the swapping in my code, on a quick check. If you do swap the eyes in the decoder, then maybe you need to negate the subtitle depth information? Because in that case I simply don't know that any swapping took place.

Kotik

8th December 2016, 22:29

@Kotik, please don't use PM and forum at the same time. I read this thread, so if you post here, you don't have to PM me, as well.

Sorry, i was just too excited with the discovery that i went ahead with it without thinking too much :(

Kotik

8th December 2016, 22:36

madshi

8th December 2016, 22:42

I think it's likely that the eye swapping is the cause of the issue, because it would explain the problem oh so nicely. It's probably my fault. I'll see what I can do during the following weekend.

nevcairiel

8th December 2016, 22:43

Seems reasonable, the offset going in the wrong direction looks totally weird.

Kotik

8th December 2016, 22:47

I think it's likely that the eye swapping is the cause of the issue, because it would explain the problem oh so nicely. It's probably my fault. I'll see what I can do during the following weekend.

I don't think that it has to be someones fault, i am glad that both u and nevcairiel contribute to this community.

I for one am glad that it might be a bug, if it indeed is then eventually it will be one bug less.

Keep up the good work guys, both of u. I really appreciate ur work.

Thanx.

LigH

9th December 2016, 09:00

Authoring studios can make mistakes too. I remember some DVDs which contain falsely flagged video streams. Flagging 3D BD subtitles with the wrong eye preference is merely an accidental check in a box in their authoring software, I guess?

Aleksoid1978

9th December 2016, 09:46

I think renderer must reverse subtitle offset based on flag from splitter.

nevcairiel

9th December 2016, 11:13

Flagging 3D BD subtitles with the wrong eye preference is merely an accidental check in a box in their authoring software, I guess?

Like I explained above, subtitles don't have an eye preference, only the video stream does - and we already established that the video stream is flagged properly in these cases.

The easiest solution for the renderer would be to flip the video views to normal L/R order before applying subtitles to them, and everything should be fine - otherwise if the flow doesn't allow flipping first, the subtitle offset needs to be inverted based on the video view order flag.

red5goahead

9th December 2016, 13:42

to anyone interested VisualSubSync-Enhanced there are the Lav codec Delphi/Pascal interfaces in use in that project

LavAudioSettingsInterface.pas
LavSplitterSettingsInterface.pas
LavVideoSettingsInterface.pas

Would be useful in developer_info folder on github code section

nevcairiel

11th December 2016, 16:53

LAV Filters 0.69

LAV Splitter
- NEW: Support for extracting HDR metadata from YouTube HDR VP9 streams
- Fixed: Reading MKV files could crash when encountering tags for unknown or disabled streams

LAV Video
- NEW: Support for 10-bit UtVideo
- NEW: MagicYUV decoding support
- NEW: Experimental support for CineformHD decoding
- Fixed: Converting 12-bit 4:4:4 YCbCr to RGB32 would result in a garbled image
- Fixed: Decoding certain H.264 streams could drop a few frames at the start of playback

LAV Audio
- NEW: DTS Express (LBR) decoding support

Download: Installer (both x86/x64) (https://files.1f0.de/lavf/LAVFilters-0.69.exe) -- Zips: 32-bit (https://files.1f0.de/lavf/LAVFilters-0.69.zip) & 64-bit (https://files.1f0.de/lavf/LAVFilters-0.69-x64.zip)

Its been a long time since the last release, and some of these features have been in the nightly builds for a long time. Nevertheless, LAV Filters are far from dead, even if slowing down a bit as the list of important features still in need of development shrinks.

YouTube HDR
This version introduces support for YouTube HDR streams, which utilize VP9 Profile2 10-bit streams, and HDR metadata in the container.
For this metadata to be properly passed to the video renderer, you'll need LAV Splitter and LAV Video, as well as a video renderer that can understand it - at this time only madVR.

DTS Express support
DTS Express was the last missing piece of full DTS support, and eventhough its only typically used for commentary tracks on Blu-rays and not primary audio tracks, its still noteworthy for people trying to archive and potentially transcode all the available audio.

More!
In addition to the major points listed above, there is a whole list of small fixes and improvements mainly on the side of FFmpeg that this build makes available to LAV Filters users.

As always, please report any regressions or bugs with the appropriate information to fully analyze it, even better with a sample file to reproduce it!
Thanks, and have fun.

PS:
I'll be on vacation in Australia all through January, so don't be alarmed when there is no responses.

madshi

11th December 2016, 16:59

Thanks! :)

I'll be on vacation in Australia all through January
Sounds amazing! You may need to prepare for a cold weather shock, though, when you come back to Germany in February!! :scared:

pirlouy

11th December 2016, 17:38

Arf, I guess I don't come here the right day. I have a feature request (hoping I don't say something wrong).
In France, several of our Internet providers use RTSP streams for IPTV. Unfortunately, with LAV splitter, it often results in errors when trying to read the channel. You have to try several time hoping it will work at some point. I've tried to change "Stream Analysis Duration" values, but without success.
With VLC (which is recommended by provider), there's no problem at all (except VLC quality).

From what I understood, LAV filters use FFMPEG for RTSP, whereas VLC Uses live555.com RTSP implementation. Do you think you can do something about it ? I know it's not really motivating, I don't have links to test, since the RTSP content is between computer and the Provider router.