LAV Filters - DirectShow Media Splitter and Decoders

[SamuriHL]

Nev, I'm confused, those deinterlacing changes only apply to the CUVID code or have you added software deinterlacing support? And people please pay attention to whatever answer we get and don't ask this 100 times. Thanks!

[glorp]

Originally Posted by sneaker_ger

It's not that I think that it is important. It's just that the probability of an "und" track being in a language known to the user is higher, than for a track in a language the user hasn't defined in the selector. It's the strawberry on top of the cake, not really vital.

But there is also the possiblility that someone specifically marks tracks as "und" in cases where he does not want those tracks to enter into the selection process at all. I'll give you an example where that's useful:

- You only understand English and need English subs for all other languages.
- A film has a French main audio track and an English commentary track.

There currently are no options to prioritze language tracks by flags (i.e., forced) so if you mux this film with both language tracks and tag them correctly for language you will always get the commentary track with subs turned off. Not at all what you want.

The only way I've been able to keep the commentray track in a mux is to label it "und" knowing that then it would not be included in the selection criteria. I can't see any other way to handle this except to choose a completely unrealistic language tag for the commentary track.

So I don't want to see "und" become a priority or I'd like to see a way to handle that scenario by flags or language logic.

Note: Haali didn't/doesn't handle this any better. The "und" trick was the only way I could find to do it there too.

[sneaker_ger]

Well, nevcairiel does not want to apply any advanced logic to the audio selection at all, so my (unfinished) proposal wouldn't change your situation. You could still mark your commentary tracks as "und" and be happy.

Remember, my proposed prioritization when using the *-symbol:
1.) languages in the audio field
2.) "und"
3.) the rest

Nothing would change for you, the proposal of prioritizing "und" over "the rest" is only for the *-character.

Though your situation really shows one of the downsides of using Matroska files instead of DVDs/Blu-Rays with menus. Makes me even stronger think about searching the track names for things like "forced" or "commentary", instead of just looking at the Matroska flags.

[glorp]

Originally Posted by sneaker_ger

Remember, my proposed prioritization when using the *-symbol:
1.) languages in the audio field
2.) "und"
3.) the rest

Yes, you're right. I agree. I misunderstood what you were proposing but your clarification is clear

Originally Posted by sneaker_ger

Though your situation really shows one of the downsides of using Matroska files instead of DVDs/Blu-Rays with menus. Makes me even stronger think about searching the track names for things like "forced" or "commentary", instead of just looking at the Matroska flags.

Yes, agreed again, although in my case simply honoring the forced flag for audio selection too (or a way to include it in the advanced string) would have been enough for mkv's. Sadly that wasn't part of the new option so I still need the "tricks".

[sneaker_ger]

Originally Posted by glorp

Yes, agreed again, although in my case simply honoring the forced flag for audio selection too (or a way to include it in the advanced string) would have been enough for mkv's. Sadly that wasn't part of the new option so I still need the "tricks".

Yes, nev's decision to make the audio selection (1) not "forced" and "default" aware and (2) be separated from the subtitle selection imposes certain limitations.

I'm currently writing down a few basic assumptions for a new proposal, but with that decision, certain limitations will remain, sadly.

[rica]

Thanks for the new build but no fix for LavSplitter > raw pcm mkv > ReClock issue yet. Hoping for 040 build.
Best.

[pankov]

Originally Posted by SamuriHL

Nev, I'm confused, those deinterlacing changes only apply to the CUVID code or have you added software deinterlacing support? And people please pay attention to whatever answer we get and don't ask this 100 times. Thanks!

SamuriHL,
I think these changes are meant to control the deinterlacer in madVR ... or any other renderer that reads the correct flags and obeys them
...
and in the future probably for the software deinterlacer in LAV itself

[SamuriHL]

Originally Posted by pankov

SamuriHL,
I think these changes are meant to control the deinterlacer in madVR ... or any other renderer that reads the correct flags and obeys them
...
and in the future probably for the software deinterlacer in LAV itself

That's fine (and good as I wanted to test madVR deinterlacing vs the CUVID deinterlacing like you've been doing) but I just wanted to get clarification. Cause if I'm confused, others likely will be, too. But that's a good thing if that's what this change is for. I'll have to build myself a copy and then update to the latest nVidia beta driver since I BELIEVE they fixed that annoying black screen nonsense in 7MC in the latest driver.

[nevcairiel]

Originally Posted by sneaker_ger

1.) Is it possible to know beforehand (without parsing the complete file) whether a PGS track includes forced subtitles? I presume it's not.

Its not. There is no global header, its on a per-frame basis.

Originally Posted by sneaker_ger

2.) Why isn't there the same possibility for idx/sub tracks?

I suppose they do have a similar mechanic (a forced flags in the frame headers), but i don't have that much media with DVD-style subtitles.

Originally Posted by sneaker_ger

3.) Is it possible to know beforehand (without parsing the complete file) whether an idx/sub track includes forced subtitles?

Same goes as for PGS.

Originally Posted by SamuriHL

Nev, I'm confused, those deinterlacing changes only apply to the CUVID code or have you added software deinterlacing support? And people please pay attention to whatever answer we get and don't ask this 100 times. Thanks!

They apply to all possible kinds of deinterlacing, be it CUVID, or in the Renderer (EVR/madVR), or through a software post-processor, or for any future software deinterlacers in LAV Video itself.

Originally Posted by sneaker_ger

Yes, nev's decision to make the audio selection (1) not "forced" and "default" aware and (2) be separated from the subtitle selection imposes certain limitations.

It is at least "default" aware, however the language is a stronger match then the default flag. I understand the problem with commentary tracks that have a "better" language match then the main feature, however i'm unsure how to fix that properly.

Regarding the advanced selection - i'm not sure why you really need the audio selection in there. It seems far more logical that people have a preferred audio language, and then want subtitles that go with that audio language, instead of the other way around. What kinds of rules are you aiming to achieve with that? "If english subs are present, use japanese audio" seems like a silly rule to me - its far more likely people want "If japanese audio is used, use english subs".

[PeQuE]

Originally Posted by nevcairiel

Here is a test build with some new deinterlacing changes:

http://files.1f0.de/lavf/LAVFilters-...5-g905db7d.zip

Bulletpoint changes:
- Changed "Field Order" and "Force Deinterlacing" to global options
- Added "Aggressive Deinterlacing" option

That means that all those options not only apply to the CUVID deinterlacer now, but also to deinterlacing when its done in the renderer.

Aggressive deinterlacing is similar to CoreAVCs aggressive checkbox - it forces deinterlacing if the stream indicates interlaced coding is used, even if not all frames are coded as such. Alot of broadcasts seem to be broken that way.

Force Deinterlacing foces deinterlacing on all streams, even if the whole stream is marked progressive. Not sure that option is still needed, i have no sample which would require it.

@PeQuE:
Aggressive Deinterlacing should probably make your stuff play properly.

Thanks so much for attending my needs nev.

I'll test this ASAP and report back.

[psymed]

Is 22.2 surround sound planned for future versions?

[nevcairiel]

Originally Posted by psymed

Is 22.2 surround sound planned for future versions?

Once there actually are main stream codecs and files that use it, sure. Until then, no. (So not for some years)

[SamuelMaki]

Originally Posted by psymed

Is 22.2 surround sound planned for future versions?

Use jRiver MC internal codecs...

[HeadlessCow]

Originally Posted by nevcairiel

Regarding the advanced selection - i'm not sure why you really need the audio selection in there. It seems far more logical that people have a preferred audio language, and then want subtitles that go with that audio language, instead of the other way around. What kinds of rules are you aiming to achieve with that? "If english subs are present, use japanese audio" seems like a silly rule to me - its far more likely people want "If japanese audio is used, use english subs".

I think the real problem here is that there's no concept of an "original" audio track. In an ideal world, one audio track (or multiple for something like "The Perils of Gwendoline in the Land of the Yik Yak" where the cast was half-English and half-French speaking) would be marked as "original" and playback should (optionally) prefer that rather than the language of the viewer as long as there are subtitles that match a language that the viewer understands. In the event of multiple "original" audio tracks, the selected "original" audio track should follow the preference of the viewer and when there are no tracks marked as "original", the preference should just follow the list that the user configures.

Examples:
Configured language: eng, fre

Harry Potter:
Audio: eng:"original", fre, esl
Subs: eng, fre, esl
Selected on playback:
Audio: eng
Sub: None

Excel Saga:
Audio: jpn:"original", eng
Subs: eng
Selected on playback:
Audio: jpn
Sub: eng

Gwendoline:
Audio: eng:"original", fre:"original"
Subtitles: eng, fre
Selected on playback:
Audio: eng
Sub: None

Cheesy (Dubbed) Kung Fu Movie
Audio: zho:"original", eng
Subs: zho
Selected on Playback
Audio: eng
Subs: None

If that all makes sense... does it make sense to treat the default flag for audio tracks as the "original" flag in my suggestion?

[sneaker_ger]

Originally Posted by nevcairiel

Regarding the advanced selection - i'm not sure why you really need the audio selection in there. It seems far more logical that people have a preferred audio language, and then want subtitles that go with that audio language, instead of the other way around. What kinds of rules are you aiming to achieve with that? "If english subs are present, use japanese audio" seems like a silly rule to me - its far more likely people want "If japanese audio is used, use english subs".

Ok, I wrote a wall of text yesterday, which I was hesitating to post at all, because today I didn't feel as confident about it as I felt yesterday. But it includes examples that show exactly what I mean. It is not simply a question of "more logical", but the current logic makes it straight out impossible to do certain things. I will make two separate posts for them.

[sneaker_ger]

I have thought about an improved advanced selection logic, based on the following assumptions:

1. The audio track selection is not part of the advanced logic. This is by nev's choice and will not be changed.
2. The languages in the audio selection field are languages the user prefers over other languages. If possible, these should take priority over any quasi random selection.
3. "Undefined" tracks are more likely to be known by the user, than defined tracks that are not listed by the user as known languages.
4. Users might want to prioritize certain audio languages they do not understand. They need subtitles in a language they DO understand for these, though.
5. The selection fields should be both human readable and fillable, without having to resort to some kind of batch system.
6. The advanced logic should not make it necessary for the user to write down each and every possible combiniation. The more languages you understand, the more complicated it currently gets, because you basically have to write at least one pair for each possible combination. Wildcards are a solution to make the options faster to write and easier to read and grasp.
7. The new logic should for the most part be downwards compatible to the old logic. Exceptions are quasi random selections, which may be superceded by selections based on known languages.
8. The new logic shall be easy to implement, by substituting wildcards through combinations already known by the old logic.
9. People might want to select the subtitle language not based on the general order of known languages, but according to the currently selected audio language.


The biggest problem I see with my new proposal, is the limitation imposed by assumption #1. This means that assumption #4 can not be satisfied in its entirety.

The first change I deduce from assumption #4 and #1 is: a) the audio selection field has to be split into two separate groups:
- [A] = the current audio selection field. LAV splitter will select the audio track according to this list, its meaning will stay the same and it is thus backwards compatible. If [A] is empty, then [A]=[B]
- [B] = a new group with languages the user actually understands. If [B] is empty, then [B]=[A]

b) Now we already have our two new wildcards: [A] and [B]. Naturally, LAV shall obey the order given by the user when selecting a track with these wildcards.

c) We will also introduce a new operator: "!" (exclamation point)
This operator allows the user to easily select only languages that are not part of a group (or isn't a specific language).

d) To meet assumption #9 we need to introduce yet another wildcard: "[?]" (interrogation mark, in brackets as to stay consisted with the other wildcards)
This wildcard means "same as selected audio language". This allows to select multiple audio/subtitle pairs, without having to write a new pair for every language. This wildcard can only be used for the subtitle, not for the audio.

e) to account for assumptions 2#, #3 and #4, we need to redefine the "*"-wildcard. From now on it shall use the following order for the subtitle track:
- select [B], incl. [?]
- select "und", incl. [?]
- select [A], incl. [?]
- select the rest (e.g. by track ID, ascending)

This order will increase the probability of a "lucky hit", i.e. selecting a language that is understood by the user. "[?]" shall NOT be prioritized within any group, for this could disobey the language order setup by the user. If he wants to prioritize "[?]", he has to create a separate pair.


I will now give an example on how to use this new logic.
We have an American that only speaks English, but likes to watch Korean and Japanese movies in their original audio. Of course he needs subtitles for that.
In the "audio field" (group [A]):
kor,jpn,eng
In the "languages understood" (group [B]):
eng
In the "subtitle selection" field:
[B]:[?]|f;[B]:[B]|f;[B]:off;![B]:*

Let's not forget, because of assumption #1, the audio tracks is selected before anything else:
1.) tries to select a Korean audio track
2.) tries to select a Japanese audio track
3.) tries to select an English audio track

Only after that comes the subtitle selection, which consists of four parts:
1.) if the user understands the selected audio track, it will try to select the forced subtitle track of the same language. If not found, try:
2.) if the user understands the selected audio track, it will select a subtitle track with a language understood by the user. If not found, try:
3.) if the user understands the selected audio track, subtitles will be deactivated. If not found, try:
4.) if the user does not understand the selected audio track, it will select any subtitle track, in the order [B], "und", [A], rest

Some parts of the last pair can be exchanged with others and will probabably still lead to the desired result. Most users won't even need [A].

(The example shows why #1 and #4 can not be combined, btw. If a file has only a Korean and an English audio track, but no subtitles, the Korean audio track gets selected, even though the American can not understand a single word, because of the missing subtitles. A more advanced logic would be able to switch to the English audio track in this specific case, although the user usually prefers the Korean track.)

Now this is a very simple example, which shouldn't be too complicated, even with the current logic. But if a user understands two, three or even more languages, the old logic gets very complicated. For the new logic, it doesn't really matter, you just enter the additional languages in field [A] and/or [B].

Now let's check if the new logic meets assumption #8, "easy to implement":
The above example would translate to:
eng:eng|f;eng:eng|f;eng:off;...
(The fourth part of the selection would be too long to write out, but it doesn't pose any problems.)
Basically all pairs with the new symbols can be subsituted by a number of pairs of the old logic.

I'd say the remaining assumptions are also respected:
#7: check, it does not redifine any existing setting, users can simply update. The only exception is the *-wildcard, but it wasn't predictable to begin with.
#5: check, the new wildcards and the new operator are easily writeable and readable. In fact, the wildcards can significatly reduce the amount of characters, improving the user experience.


Now for the summary:
- Introduction of new wildcards [A] (audio track selection group), [B] (languages actually understood group), [?] (currently selected audio language)
- Introduction of a new operator "!" (all languages BUT the group/language behind the operator)
- Redefinition of the "*"-wildcard, new order: [B], "und", [A], rest

My new proposal would make things shorter and easier, but sadly, it cannot solve #4 under the current conditions.

[sneaker_ger]

(Sorry, the style got lost a bit while copying over from my text editor. Got to go, right now, I hope it's readable.)

Proposal for a combined subtitle and audio track selection. (I know it's insane, just a bit of brainstorming.)

A combined subtitle and audio track selection could solve #4 shown in the example in my other proposal. The downside: assumptions #1 and #7 are false.
Summary:
- Introduction of new wildcards [A] (Group A), [B] (Group B), [?] (currently selected audio/subtitle language)
- Introduction of a new operator "!" (all languages BUT the group/language behind the operator)
- Redefinition of the "*"-wildcard,
>new order for subtitles: [B], "und", [A], rest, off
>new order for audio : [A], [B], "und", rest, off
- Syntax extensions: combination of elements:
>*:[A][B]
>*:![A]![B] = *:!([A][B])
(more parts of the set theory supposable, of course)
-several track types
>[A]|f|d|
-explicit definition of "|!n" = not forced, "|!d" = not default, "|!fd" = neither forced nor default etc.:
[A]|!f = not forced of Group A

The syntax extensions could of course also be used for the first proposal, but I wanted to keep that one sane. Regard them as "optional"!


Example of the old proposal solved:
Group [A]:
kor,jpn (NOT eng)
Group [B]:
eng
In the combined selection field:
[A]:[B];[B]:[?]|f;[B]:[B]|f;[B]:off;*:*

reads:
1.) Try to select an audio track of Group A and a subtitle track of Group B. If not found, try:
2.) Try to select an audio track of Group B and a forced subtitle track of the same language. If not found, try:
3.) Try to select an audio track of Group B and a forced subtitle track of Group B. If not found, try:
4.) Try to select an audio track of Group B and disable all subtitles. If not found, try:
5.) Select an audio track in the order [A], [B], "und", rest, off, and a subtitle track in the order [B], "und", [A], rest, off.

We see that the problem of the old mode has been solved. With Korean and English audio and no subtitles, the English audio gets selected. With Korean and English audio, and an English subtitle track, the Korean audio and the English subtitles get selected.

[sneaker_ger]

Ok, wall of text posted. I hope you can see the problem of the current mode, even if none of my proposals seem convincing to you.

[nevcairiel]

Originally Posted by sneaker_ger

In the "subtitle selection" field:
[B]:[?]|f;[B]:[B]|f;[B]:off;![B]:*

I just read this and just had to correct it.

"[B]:off;![B]:*" is equal to "[B]:off;*:*" because [B] will never reach the last selection anyway.

Also, instead of having a separate "group" for [B], maybe some kind of special symbol on the language tags could be used, to keep it in one box. I would propose "!" for "important", but it would be confusing with "!" meaning "not" in the advanced box. Maybe "~kor,~jpn,eng" ? :d

I'll read it in detail later.

[sneaker_ger]

Originally Posted by nevcairiel

I just read this and just had to correct it.

"[B]: off;![B]:*" is equal to "[B]: off;*:*" because [B] will never reach the last selection anyway.

In this specific case. Yes, I know.

But there's a reason we have "for()" in programming languages, even though we could as well combine "while()", "if()" and "i++". It is easier to write, read and grasp.

I wouldn't regard my proposals as the solution. It's more like a base for further discussions.