LAV Filters - DirectShow Media Splitter and Decoders

[sirhaden]

If you're certain the video is fine, then you may still be delivering it to the decoder wrong. I assume you use your own source filter, and not LAV Splitter?
My guess would be that the media type doesn't match the video contents.

I am my own source and splitter filters.

I made changes to the MP4 parsing to remove all NALUs from non IDR video frames and updated the output media sample size. For IDR frames, changes were made to only send the PPS NALU followed by the IDR NALU with an updated output media sample size.

The output media sample type is:

MEDIASUBTYPE_AVC1
FORMAT_MPEG2Video

with:

pVIH2->bmiHeader.biCompression = MKTAG('a', 'v', 'c', '1');

Which is the same as when delivered from the proprietary format container.

Unfortunately, the issue is still occurring.

[ca]

I tested two nightlies: 0.69.0-43 and 0.69.0-73. Everything seems to work fine except that i can not playback Blu-Rays anymore using the DLLs libaacs.dll and libbdplus.dll. When media playback should start it says "Can not render Video/Output pin, error: 0x80004005 unknown error".

With the last official LAV release 0.69 this still works.

Aacs still doesn't seem to be working a year later. Can anyone else confirm this?

[clsid]

ProcessMonitor might reveal if it still tries to load those DLLs.

[sirhaden]

Resolved the issue related to a different DirectShow filter, not the LAV filters. :thanks:

[DTL]

Addition posts of VictorLS:
If feeding LAV Splitter with interlace HEVC stream we see HEVC VUI-> field_seq_flag = 1 and frame_field_info_present_flag = 1, but VIDEOINFOHEADER2 -> dwInterlaceFlags = 0x00000000 that confuses renderer. Also at the output of LAV decoder also VIDEOINFOHEADER2 -> dwInterlaceFlags = 0x00000000 that indicates progressive frames. I think if LAV splitter-decoder operates with such HEVC bitstream it must set dwInterlaceFlags |= AMINTERLACE_IsInterlaced | AMINTERLACE_1FieldPerSample in the VIDEOINFOHEADER2
of output decoded samples and feed it to the renderer. Also for each sample the LAV Decoder must pass through field placement data and set dwTypeSpecificFlags member of AM_SAMPLE2_PROPERTIES: if pic_struct member of picture timing SEI = 1, then dwTypeSpecificFlags |= AM_VIDEO_FLAG_FIELD1; if if pic_struct member of picture timing SEI = 2, then dwTypeSpecificFlags |= AM_VIDEO_FLAG_FIELD2. May this helps.

Some info about streaming interlaced content using h.265 as sequence of fields: http://www.ramugedia.com/interlace-support-in-hevc

"But implementing the logic to detect incorrectly interlaced HEVC may be. "

The input h.265 bitstream to LAV Filters do properly marked as field-based in the VUI. So to detect this the LAV Filters must process VUI data in the input h.265 bitstream and use it in the internal processing or pass this data to renderer for correct visualization on progressive electro-optical devices.

To show VUI data in graphstudio the source .ts was remuxed to .mp4 using ffdshow.

[nevcairiel]

There will be no support for interlaced HEVC unless FFmpeg properly supports decoding them like any other interlaced format. I recommend inquiring there.

[DTL]

But ffmpeg Do properly decode sequence of pictures that handle interlaced h.265 field-based video. Generally users do not ask ffdshow and LAV to output weaved frames or perform deinterlace for such h.265 streams. But LAV must parse h.265 stream VUI data and mark output DirectShow stream samples properly at least.

Now it marks them as non-interlaced so all further renderers fails to display it properly.

Also field-based stream is completely valid DirectShow stream marked with flags AMINTERLACE_IsInterlaced | AMINTERLACE_1FieldPerSample in the VIDEOINFOHEADER2. As well as more common interlaced-frames stream. So I hope if LAV splitter-decoder will correctly pass-through the metadata to the next to decoder renderer it may do all required work for displaying it correctly.

Ofcourse if LAV-software is too high-level wrapper around ffmpeg that it can not go low-level into VUI of h.265 data processed so it really no way to do it. Sadly in this case.

[nevcairiel]

But ffmpeg Do properly decode sequence of pictures that handle interlaced h.265 field-based video

No, it does not. If it did, there would be no discussion because it would just work. It decodes it like a progressive half-height video. It decodes any other interlaced format as weaved fields into one frame and exports metadata about interlaced status and field order, which is required for both software and hardware deinterlacing. It does not do that for HEVC. Hence, it is not supported. There will be no manual hackery for HEVC interlaced, or trying to get separate fields to work. Either it adopts standard decoding, or it won't be supported.

PS:
Just because something is theoretically valid in DirectShow, it does not mean that it would actually be supported by whatever renderer one cares about.

[mclingo]

Hi, @nevcairiel - i'm using some Playstation Platinum wireless headphones for late night viewing but finding i'm having to turn off bitstreaming to get sound, shouldn't these be working now RE changelog: LAV Audio will fallback to audio decoding if bitstreaming is not supported by the audio renderer/hardware

[nevcairiel]

That depends if the driver actually refuses bitstreaming. Maybe it doesn't.

[mclingo]

any way to test whats going on here, I also have a pair of arctis 7's plugged in, they also work over a wireless dongle, they work, however i'm getting too much latency on those so i'm probably going to send them back and keep the Sony's as I dont want to constantly have to change latency settings.

!!! caution, off topic !!!! - finding some wireless headphones just for movies is a total nightmare, none of them seem to be able to do everything well :(

[mclingo]

Hi, I might have solved the latency issue on my artis 7's so dont bother with platinum issue. Thanks.

[VictorLS]

No, it does not.
Why you so sure?
I'm not sure you've tried ffplay.exe -vf weave,yadif=1 "Zee TV20180716-203544.ts" because only blind man not see the difference between ffplay and any players on ZEE TV logo - with ffplay it's normal but in players vertically stretched x2 it flickers like old Bob deinterlacing on ordinary SD MPEG2 or H264 interlaced streams.
There will be no manual hackery for HEVC interlaced, or trying to get separate fields to work. Either it adopts standard decoding, or it won't be supported.
I repeat - many modern TVsets already support streams like Zee TV20180716-203544.ts well so it's de facto standard for H265 interlaced video even if you don't know and/or it isn't on paper - so when such (mostly SD) videos will be often in Internet you'll have to return to this question in a future (like it was with "bad streams for nVIDIA DXVA" and you did CUVID for that working as I said but for Win7 and newer only) so why don't you do it now and question i.e. http://trac.ffmpeg.org/ticket/4141 (thanks clsid) will be closed?

[nevcairiel]

I'm so sure because I actually know how LAV and FFmpeg works. I could entirely repeat the previous post, but it would not make a difference. FFmpeg needs to properly support HEVC interlaced without custom filtering to fix it, there is nothing for LAV to do. I will most definitely not implement work-arounds that are slow and also just don't work with hardware decoding.

[VictorLS]

This is more clear - as I understand correctly you don't want do anything with it while ffmpeg not improved for correctly Zee TV20180716-203544.ts decoding without any "external" filters like weave?
I will most definitely not implement work-arounds that are slow and also just don't work with hardware decoding.
But interlaced are just SD and HD (as minimum from SATs) so even my old three-core AMD AthlonII x3 460 decodes them without hardware acceleration easily in MPEG2, H264 and H265.
May be implementing -vf function with automatic turning off hardware acceleration [i.e. rectangle where people can write any working ffmpeg filter (or comma-separated filters) in with check near it to activate and prefer it over other settings] in LAV Video Decoder will solve present (and may be many future various) issue?

[foxyshadis]

This is more clear - as I understand correctly you don't want do anything with it while ffmpeg not improved for correctly Zee TV20180716-203544.ts decoding without any "external" filters like weave?

But interlaced are just SD and HD (as minimum from SATs) so even my old three-core AMD AthlonII x3 460 decodes them without hardware acceleration easily in MPEG2, H264 and H265.
May be implementing -vf function with automatic turning off hardware acceleration [i.e. rectangle where people can write any working ffmpeg filter (or comma-separated filters) in with check near it to activate and prefer it over other settings] in LAV Video Decoder will solve present (and may be many future various) issue?

Dude, it's not going to happen, how many times do you need to be told it's not going to happen? Use ffplay with custom filtering to watch these and beg ffmpeg to support it, not nev, because he doesn't care. And for the most part, neither does anyone else outside of India; ZeeTV chose to use a backward format versus the entire rest of the world, so it won't get fixed until someone personally affected by it fixes it.

[VictorLS]

foxyshadis
How much time I have to say it's from SAT so I and many SAT amateurs need decoder (not player) - to watch such streams in maximum possible quality directly from SAT - not record from SAT (as it was with "bad streams")? And so h265 becomes standard de facto (SAT providers use it for ~ two times less bandwidth with comparable quality in h264) and not support interlaced streams in right way like h264 but SD and HD from SATs still interlaced for a long time ( so support such Zee TV streams (there is more than 20 channels from several Russian SAT providers and it grows up day by day) is question of the time. Besides it's very easy (compared to NVIDIA CUVID 10bit support was implemented too as I said but for Win7 and newer) to implement - "history is just spiral" )
As I told give people choice and they choose best.

[lvqcl]

How much time I have to say it's from SAT so I and many SAT amateurs need decoder (not player)

How much time people have to say to you that it doesn't matter.

[el Filou]

Victor, you just don't understand nev's position, do you?
LAV has to 'just work', and this exceptional case would not be compatible with that without a lot of work that is beyond LAV development philosophy, which is to just use ffmpeg 'as is' for software decoding.
While a lot of stuff used interlacing with AVC, including Blu-rays, interlaced HEVC is just too rare and for a reason because it's less efficient as it cannot use the interlace optimised coding tools AVC had. This is a mistake from your SAT provider, just because something is technically possible does not mean it is recommended, sometimes it's just an inefficient dirty trick, which this is. I doubt they're really getting the same quality for half the bandwidth with those interlaced-as-progressive HEVC streams.
So go ask ffmpeg to make the changes so that their HEVC software decoder outputs this particular case of HEVC as 1080i50 instead of 540p50, and then it will 'just work' with LAV in software decoding.
The ideal way to broadcast material that originated in 1080i50 with HEVC is to deinterlace it to 1080p50 before broadcasting it. You can ask your SAT provider to fix this.

Edit: here's the ticket you're interested in: https://trac.ffmpeg.org/ticket/5514

[YGPMOLE]

Hi folks!

Maybe I'm writing something stupid but, or I found a bug in LAV Video decoder and/or Pin Info, or I didn't understand how it works and I'll be glad if someone would explain to me.

On Windows 10 64 Bit I'm using MPC-BE with LAV Splitter/Video/Audio Decoder, madVR and (only for SD and HD resolution) FFDshow by SmoothVideo Project for frame rate conversion. The output resolution is 1920x1080@59.970 that madVR automatically change to 3840x2160 in case of UHD files.

Playing an HEVC 3840x2160@23.976 file with DXVA D3D11 in hardware decoding, the Pin Info shows the correct resolution both for input (from LAV Splitter) and output (to madVR). Changing to software decoding, the output Pin Info shows a resolution of 4096x2160@23.976 (that means 4x1024 and not 2x1920) to madVR.

I made the same try with a 1920x1080@23.976: Pin Info shows the same correct input/output resolution in hardware decoding, in software decoding the output file passed to FFDshow (using the frame Rate conversion of SmoothVideo Project) become 2048x1080@23.976 (2x1024 and not 1920x1080@59.970).
Without the frame rate conversion, the Pin Info shows an output file resolution of 1024x1080@23.976.

Seems that in software decoding the output file is based on multiplies of 1024 instead of 1920 (or the Pin info shows it). Am I wrong?

P.S. Sorry for the bad english and the possible mistakes (I'm not translating with Google, but learning to write).

Best Regards. Leo!

[huhn]

you are missing an important part of info here is the file displayed correctly or not.

having a bigger resolution (usually multiplier of 1024)shown in outpin is totally normal.

[YGPMOLE]

The images on video are ok, I have not problems.

But I was wondering (if what outpin shown it's true) why madVR have to do an extra unwanted scaling for everything that comes at 1024 (or multiplier) to 1920 (or multiplier), expecially when sources and display matches (i.e. blu-rays at 1920 and UHD at 3840). I though that LAV should leave the resolution untouched…


Another question, if you can help me: assuming that madVR does the resize and color space conversion in the best way it's possible (at least that's what I understood from madVR thread), what color space output I should set in LAV Filters to avoid multiplies conversions? I mean, I would like that everything goes to madVR untouched (both resolution and color space) and it does it's magic!

[nevcairiel]

There is no scaling. Its just a technical detail on how the image is transported to madVR.

[huhn]

Another question, if you can help me: assuming that madVR does the resize and color space conversion in the best way it's possible (at least that's what I understood from madVR thread), what color space output I should set in LAV Filters to avoid multiplies conversions? I mean, I would like that everything goes to madVR untouched (both resolution and color space) and it does it's magic!

defaults are fine you can tick AYUV for rare files that are decoded to that format and risk other renderer not working correctly. but 8 bit 4:4:4 is 8 bit 4.4:4 so YV24 should be perfectly fine.

[Megalith]

What are the benefits of "headless mode"?

Also, is it just me, or does deinterlacing not work with D3D11?

[huhn]

madVR doesn't support deinterlancing with d3d11 native and that's why i personally say don't use it.

the benefit of headless mode is you can use a GPU that is not correct to a display(headless) like a iGPU this is not possible with dxva.

[mclingo]

do we think D3D11 will ever support it?

[Snowknight26]

Pausing bitstreaming is always a bit of an issue, because the AVR loses the bitstream sync no matter what you do, and specifically in DirectShow as well, because the audio decoder doesn't really control any of that. The renderer might still be buffering data, so even if the decoder tries to reset the bitstream on un-pause, there is no guarantee that it would work.

Any possibility of debugging this? The latest release version of LAV Filters pretty much guarantees that when I pause the content, bitstreaming cuts out even after resuming. Sometimes takes up to 10 tries pause/resumes for my receiver to pick up the audio again.

[nevcairiel]

Any possibility of debugging this? The latest release version of LAV Filters pretty much guarantees that when I pause the content, bitstreaming cuts out even after resuming. Sometimes takes up to 10 tries pause/resumes for my receiver to pick up the audio again.

I can't reproduce that, and anything I tested that might theoretically make a difference just broke it here.

I would recommend trying a different audio renderer.

[SamuriHL]

do we think D3D11 will ever support it?

It's on madshi's list of things to do for madvr.

[Siso]

I'm sorry if it has been asked before, but when using reclock should the "Auto A/V sync correction" be on or off?

[nevcairiel]

You should basically never turn that off.

[Siso]

You should basically never turn that off.

Thank you, sir.

[mzso]

Hello!

I have long standing choppy playback, framedrop issues with specific channels using different DVB software (ProgDVB and SmartDVB) the same streams are affected. In SmartDVB's case changing to Microsoft's decoder fully fixes the problems. When I use LAV the problem is only avoided when I use CUVID, with every other decoder option I get the butchered video. (But I want to use cpu decoding)
Playing back recorded streams with MPC or Potplayer doesn't show any issues, but playing back with SmartDVB does. (here are some samples: https://drive.google.com/open?id=122xP_XrNB5zAewKrqbIl-IZi_qOFzF6H )

I'm at a loss as to why this happens. And so far the SmartDVB dev seems to be also. Any ideas?

[mkver]

The two sports videos switch between frame-MBAFF and individiual fields; two fields are in one PES packet in the transport stream and share the same timecode (in other words, only every second field has a timecode of its own (and ffmpeg is unable to make timecodes up)). The National Geographic video meanwhile uses field coded video. Two fields share the same timecode, too, but the bitstream contains dpb_output_delay information so that ffmpeg can infer the missing timestamps. Given that LAV is based upon ffmpeg, it, too, can probably infer the timestamps. So it seems that the only thing that these three files have in common is that part of them is field-encoded which is the likely suspect.
The sports channels use MBAFF-frames when the video is quite static and switch to field-based encoding when not. Given that you said (http://www.smartdvb.net/bb/viewtopic.php?f=3&t=1673) that this only happens when the picture is un-static this confirms my above hypothesis that it is somehow the field-based nature of the source.

[el Filou]

When I use LAV the problem is only avoided when I use CUVID, with every other decoder option I get the butchered video. (But I want to use cpu decoding)What GPU?
This is strange because that would imply it's a video decoding problem, but you say using other splitter filters like MPC's or Potplayer's fix it.

I tried your samples with MediaPortal (which uses its own splitter for MPEG TS, not LAV) and an NVIDIA Pascal in DXVA2 copy-back, and playback is perfect.

Can you try forcing the use of SmartDVB's or ProgDVB's MPEG TS splitter filters in MPC or Potplayer to see if the problem happens too in that scenario?

[mzso]

What GPU?
This is strange because that would imply it's a video decoding problem, but you say using other splitter filters like MPC's or Potplayer's fix it.

I tried your samples with MediaPortal (which uses its own splitter for MPEG TS, not LAV) and an NVIDIA Pascal in DXVA2 copy-back, and playback is perfect.

Can you try forcing the use of SmartDVB's or ProgDVB's MPEG TS splitter filters in MPC or Potplayer to see if the problem happens too in that scenario?

Actually I meant that I use LAV (splitter, video, audio) with both mediaplayers and that way the playback is fine.
But now I realize that SmartDVB doesn't use LAV splitter (the icon doesn't appear) so maybe it's the issue with pdvb/sdvb splitter and LAV.

(Not sure if I can use their splitters from other apps, but I'll have a look.)

So it seems that the only thing that these three files have in common is that part of them is field-encoded which is the likely suspect.
The sports channels use MBAFF-frames when the video is quite static and switch to field-based encoding when not. Given that you said (http://www.smartdvb.net/bb/viewtopic.php?f=3&t=1673) that this only happens when the picture is un-static this confirms my above hypothesis that it is somehow the field-based nature of the source.

Interesting. Seems like it, that switching to MBAFF borks something.
(So, if you monitor that forum you might have commented there then.)

[mkver]

Actually, both your sports samples are always MBAFF. MBAFF allows two types of coded pictures: Coded fields and coded frames; the latter can be coded with field or frame macroblocks on a per-macroblock basis. If your content is interlaced, but fairly static, then it is advantageous to use MBAFF frames. So there is no switch to MBAFF. And in case of the National Geographic documentary, there are no coded frames at all, just fields. So if this sample is also acting weird, then it is not really the switch that is the culprit, but just the usage of coded fields.
PS: I don't monitor said forum, I just searched and read the thread you refered to.

[Edit]: If you don't use LAV Splitter then it might be that the splitter keeps the two fields in the same packet, while LAV Decoder expects the two fields to be in separate packets. You could test this by remuxing your National Geographic video with ffmpeg to another transport stream.
[Edit2]: But actually, if one uses a splitter different than LAV Splitter, then the decoder is supposed (https://github.com/Nevcairiel/LAVFilters/blob/master/decoder/LAVVideo/decoders/avcodec.cpp#L320) to parse the packets again.

[mclingo]

hi, could do with some help, just lost surround sound on my arctis 7 headset and I cant work out why, the odd thing is its only affecting LAV audio. If I play a surround test file I only get left and right audio front and front height, i get silence from all other channels, center, surround left and right, happens with DTS and Dolby.

if I play the same file with VLC it summs everything fine to stereo as it should.

happens in both MPC and MPC-BE regardless of whether i'm bitreaming or not, cant work it out at all.

[mclingo]

probably an issue with the latest firmware, i've logged a ticket with steelseries, just odd its a LAV issue only, windows media player and VLC are working fine.

[huhn]

can you make a screen from lav audio status.

[mzso]

[Edit]: If you don't use LAV Splitter then it might be that the splitter keeps the two fields in the same packet, while LAV Decoder expects the two fields to be in separate packets. You could test this by remuxing your National Geographic video with ffmpeg to another transport stream.
[Edit2]: But actually, if one uses a splitter different than LAV Splitter, then the decoder is supposed (https://github.com/Nevcairiel/LAVFilters/blob/master/decoder/LAVVideo/decoders/avcodec.cpp#L320) to parse the packets again.

Hi!

So the conclusion of this is what?

[el Filou]

You need to keep trying combinations of splitters and video decoders.
Did you try adding the SmartDVB/ProgDVB splitter in Preferred mode to MPC's external filters (you need to uncheck the MPEG TS internal source filter first) to see if the same happens?

As I said, your samples play just fine with MediaPortal's own TS splitter and LAV, so if they don't with SmartDVB/ProgDVB it may be a problem with their splitters.
The source for MediaPortal's TS splitter is here, if it's of any interest to the developers of SmartDVB/ProgDVB (open source license terms apply of course...)
https://github.com/MediaPortal/MediaPortal-1/tree/master/DirectShowFilters/TsReader/source
(Edit: I tried attaching the .ax to the post so you can try it but it exceeds the file size limit even compressed in 7z. You can install MediaPortal 1 to try it, it registers the filter with the OS)

[mclingo]

can you make a screen from lav audio status.

Hi, what do you mean by lav audio status, which screen is this you want a shot of?

I think whats happening here is that windows is seeing the headphones as a 7.1 device correctly, the engineer3 sofware is then supposed to sum these channels into virtual 7.1 and then them to the headphones as a stereo feed as there are only two drivers, this bit is failing so i'm only getting the non summed audio, front stereo channels.

[huhn]

you play a file with 7.1 audio.
you open the filter lav audio
you go to status.

[mclingo]

ok, I hadnt realised. I was using MPC BE with MPC audio so no LAV status, this is what threw me, this also has the same problem so its not a LAV issue, again its odd though that VLC works fine.

I played it through MPC with external LAV filters and turned off bitstreaming so you can see the channels are full, just the headphones arent receiving them.


https://1drv.ms/u/s!AgvFafeelEBigP5VZ9WlXROVpqJ62g

[huhn]

it can now be assumed lavfilter has nothing todo with that.

so i will now point you out to the next part that can be the issue the audio renderer.

so feel free to make a thread about that.

[mclingo]

it can now be assumed lavfilter has nothing todo with that.

so i will now point you out to the next part that can be the issue the audio renderer.

so feel free to make a thread about that.

yeah, i'm going to wait to here back from steelseries first, hoping i can rollback the firmware or something, there are other issues with this headset and DTS mentioned on their reddit forum too so it must be connected to that.

cheers for your help.

[mclingo]

found a workaround, if I enable mixing and set LAV to stereo LAV sums everything, not ideal but it will do for now.

[mzso]

You need to keep trying combinations of splitters and video decoders.
Did you try adding the SmartDVB/ProgDVB splitter in Preferred mode to MPC's external filters (you need to uncheck the MPEG TS internal source filter first) to see if the same happens?

As I said, your samples play just fine with MediaPortal's own TS splitter and LAV, so if they don't with SmartDVB/ProgDVB it may be a problem with their splitters.
The source for MediaPortal's TS splitter is here, if it's of any interest to the developers of SmartDVB/ProgDVB (open source license terms apply of course...)
https://github.com/MediaPortal/MediaPortal-1/tree/master/DirectShowFilters/TsReader/source
(Edit: I tried attaching the .ax to the post so you can try it but it exceeds the file size limit even compressed in 7z. You can install MediaPortal 1 to try it, it registers the filter with the OS)

I can't seem to get MPC to use SmartDVB's filter (my best guess is that it's Smart AV Source Filter), but it would just keep on using the internal filter. (all source filters are unchecked in MPC)