Welcome to Doom9's Forum, THE in-place to be for everyone interested in DVD conversion.

Before you start posting please read the forum rules. By posting to this forum you agree to abide by the rules.

 

Go Back   Doom9's Forum > Capturing and Editing Video > New and alternative a/v containers

Reply
 
Thread Tools Search this Thread Display Modes
Old 10th July 2019, 10:32   #23561  |  Link
kolak
Registered User
 
Join Date: Nov 2004
Location: UK
Posts: 2,376
Quote:
Originally Posted by manolito View Post
I can confirm your issues, LAV Filters for me always had such problems when dealing with HD transport streams. It looks like DirectShow is simply not good at seeking within transport streams...

Just to make sure that your clip was not corrupted I ran it through TSDoctor, and it came out fine without any complaints. My usual workaround for HD transport streams is to repack them into an MKV container. I either use MKVMerge or ffmpeg (using the dmMediaConverter GUI) for this, and the resulting MKVs play with just about any player and have no problems with seeking.


Cheers
manolito

I can also confirm TS seeking issues. I also know that those TS streams are 100% correct.
kolak is offline   Reply With Quote
Old 10th July 2019, 11:33   #23562  |  Link
Aleksoid1978
Registered User
 
Aleksoid1978's Avatar
 
Join Date: Apr 2008
Location: Russia, Vladivostok
Posts: 2,187
Quote:
Originally Posted by kolak View Post
I can also confirm TS seeking issues. I also know that those TS streams are 100% correct.
Can you upload .ts sample with seeking porblem ?
__________________
I7 2600K@4.2 /Asrock P67 Extreme4 Gen 3 /Kingston HyperX 8Gb 1866 (4x2) Kit /OCZ Vertex 3 256Gb /Gigabyte GTX 960 /BenQ EW2430 /LG 47LM620T /Yamaha RX-V471 + NS-555 + NS-C444 + NS-333 + YST-SW215
Aleksoid1978 is offline   Reply With Quote
Old 10th July 2019, 12:21   #23563  |  Link
el Filou
Registered User
 
el Filou's Avatar
 
Join Date: Oct 2016
Posts: 467
Quote:
Originally Posted by Sunspark View Post
I just now tested with a still image 16 bit gradient in a window. Set the display to 2 bit, and turned off change dither pattern every frame. As I apply the selection through the ordered, ED 1 and ED 2 options, there is no change in the dither pattern.
You can check by creating a file called ShowRenderSteps (no extension, and delete the file after testing) in madVR's folder, and comparing the 'FinalStep' rendering step in the OSD. On my system, ED2 takes ~3x the rendering time as Ordered (first make sure that the other steps are the same time to eliminate GPU frequency variation).
__________________
HTPC: W10 1809, E7400, 1050 Ti, DVB-C, Denon 2310, Panasonic GT60 | Desktop: W10 1809, 4690K, HD 7870, Dell U2713HM | MediaPortal 1/MPC-HC, LAV Filters, ReClock, madVR

Last edited by el Filou; 10th July 2019 at 13:27.
el Filou is offline   Reply With Quote
Old 10th July 2019, 16:40   #23564  |  Link
kolak
Registered User
 
Join Date: Nov 2004
Location: UK
Posts: 2,376
Quote:
Originally Posted by Aleksoid1978 View Post
Can you upload .ts sample with seeking porblem ?
I need to make one, as none of my files can be made public.
kolak is offline   Reply With Quote
Old 11th July 2019, 00:59   #23565  |  Link
manolito
Registered User
 
manolito's Avatar
 
Join Date: Sep 2003
Location: Berlin, Germany
Posts: 2,513
I cut out a 100MB sample of a standard DVB-T2 broadcast from German public television, get it here:
https://www.sendspace.com/file/51jlvy
//EDIT//
SendSpace has become painfully slow, here is an alternative link:
https://www.zeta-uploader.com/2126706227

It shows corrupted frames with every seek operation using DirectShow based players (which all employ LAV Filters). It plays and seeks nicely using VLC.

Using the preview in an older 32-bit StaxRip it behaves like this with different 32-bit source filters:

1. DSS2Mod with current stable LAVVideosource:
Corrupt frames

2. FFVideoSource version 2.23.1 (latest stable version by Myrsloik):
Major disaster. Preview is totally unusable. Stepping through frames not possible.

3. LWLibavVideoSource (from lsmash-r929-170224-32)
Works without problems. Seeking does not display any corrupt frames.


Frankly I have no idea what I should make out of this. If LSMASH can do it, why can't LAVVideo and FFMS2 ?


Cheers
manolito

Last edited by manolito; 13th July 2019 at 00:37.
manolito is offline   Reply With Quote
Old 11th July 2019, 01:18   #23566  |  Link
nevcairiel
Registered Developer
 
Join Date: Mar 2010
Location: Hamburg/Germany
Posts: 9,733
As mentioned a few weeks ago as this topic came up, this has nothing to do with the file being TS (although it doesn't help), but because of the codec being HEVC. Noone yet implemented a function to properly suppress corrupt frames after seeking, which are basically always going to be present, in the ffmpeg HEVC decoder like it was done for H.264. The problem is of course that as codecs get more complex, this logic also does.

This is not something I do in LAV itself, so it needs to happen in FFmpeg. Feel free to contribute if this is a topic of importance for you.

As a general rule, TS as a container just does not offer any proper seeking functionality. It barely has usable timestamps, and thats it. No keyframe flags, no index, no nothing. As such, if you want "artifact free" seeking in TS, this has to rely on deep knowledge of the codec inside the TS container. And as mentioned above, noone has really bothered with HEVC for that, because its not a very popular codec in the open-source world due to the patent nonsense.
__________________
LAV Filters - open source ffmpeg based media splitter and decoders

Last edited by nevcairiel; 11th July 2019 at 08:59.
nevcairiel is offline   Reply With Quote
Old 12th July 2019, 08:18   #23567  |  Link
ryrynz
Registered User
 
ryrynz's Avatar
 
Join Date: Mar 2009
Posts: 3,140
Any chance of an update to latest GIT of dav1d? Thanks.
ryrynz is offline   Reply With Quote
Old 12th July 2019, 10:42   #23568  |  Link
nevcairiel
Registered Developer
 
Join Date: Mar 2010
Location: Hamburg/Germany
Posts: 9,733
I currently plan to update it to 0.4.0 once thats available, which should be not that much longer.
__________________
LAV Filters - open source ffmpeg based media splitter and decoders
nevcairiel is offline   Reply With Quote
Old 12th July 2019, 11:42   #23569  |  Link
kolak
Registered User
 
Join Date: Nov 2004
Location: UK
Posts: 2,376
Quote:
Originally Posted by nevcairiel View Post
As mentioned a few weeks ago as this topic came up, this has nothing to do with the file being TS (although it doesn't help), but because of the codec being HEVC. Noone yet implemented a function to properly suppress corrupt frames after seeking, which are basically always going to be present, in the ffmpeg HEVC decoder like it was done for H.264. The problem is of course that as codecs get more complex, this logic also does.

This is not something I do in LAV itself, so it needs to happen in FFmpeg. Feel free to contribute if this is a topic of importance for you.

As a general rule, TS as a container just does not offer any proper seeking functionality. It barely has usable timestamps, and thats it. No keyframe flags, no index, no nothing. As such, if you want "artifact free" seeking in TS, this has to rely on deep knowledge of the codec inside the TS container. And as mentioned above, noone has really bothered with HEVC for that, because its not a very popular codec in the open-source world due to the patent nonsense.
Seeking problems (long/frozen seek) happens for H264 as well. Files are coming from Manzanita muxer (which is by many treated as reference).
kolak is offline   Reply With Quote
Old 12th July 2019, 11:59   #23570  |  Link
nevcairiel
Registered Developer
 
Join Date: Mar 2010
Location: Hamburg/Germany
Posts: 9,733
TS, as a format, is not designed for seeking. Its designed for broadcast and streaming. If you want local file playback with perfect seeking, use a format that is designed for these tasks. I have said that repeatedly over the years. I will not spend excessive amount of time trying to improve TS seeking, because its fundamentally not a concept helped by the container. There are perfect formats/tools for each job, use the right ones.

Seeking in a format like MPEG-TS is always just guess work. You "randomly" (with some educated guesses) look around in the file for a timestamp that matches the one you are interested in, because there is no index or any other clues where your target time is. This is a balance between the amount of file IO you perform to find it, obviously I could just read from the start and stop when I find it, but that would be slow, and the accuracy. Sometimes it finds one a bit too early, or a bit too late, which then takes the playback chain a bit to "catch up", or it just missed the previous key frame (and due to container inadequacy, it also has no clue where those are), hence "slow seeking".

There is a reason Blu-ray, for example, which uses MPEG-TS carries an external seeking index in a seperate file, so it can actually do seeking using that information.

LAV uses the FFmpeg MPEG-TS demuxer, so you know where to send improvements.

One final note, for example the cited "LWLibavVideoSource" in an earlier post which does "perfect seeking" - you know how it does that? It reads the entire TS file and builds a complete seeking index. A video processing tool can certainly afford to be a bit slow to do this, but something focused on playback like LAV is not in a position to be that slow on file opening or first seek.
__________________
LAV Filters - open source ffmpeg based media splitter and decoders

Last edited by nevcairiel; 12th July 2019 at 12:14.
nevcairiel is offline   Reply With Quote
Old 12th July 2019, 14:24   #23571  |  Link
el Filou
Registered User
 
el Filou's Avatar
 
Join Date: Oct 2016
Posts: 467
Personally never encountered any seeking issue in my H.264 TS files (muxed by MediaPortal's TSWriter, but even when opening a raw multi-program TS direct from the cable provider it's fine) when I open them in MPC-HC with LAV.
I don't consider glitches when seeking in broadcast stream an issue. You're seeking, the very operation you're doing is breaking normal playback. It's much less disturbing than audio crackles and pops.
__________________
HTPC: W10 1809, E7400, 1050 Ti, DVB-C, Denon 2310, Panasonic GT60 | Desktop: W10 1809, 4690K, HD 7870, Dell U2713HM | MediaPortal 1/MPC-HC, LAV Filters, ReClock, madVR
el Filou is offline   Reply With Quote
Old 12th July 2019, 14:45   #23572  |  Link
manolito
Registered User
 
manolito's Avatar
 
Join Date: Sep 2003
Location: Berlin, Germany
Posts: 2,513
My main requirement for TS streams is to reencode them (editing out commercials and get rid of station logos). Finding the correct cut points in the preview does require a lot of seeking, and always getting corrupt frames is very annoying.

So LAV Filters is simply the wrong source filter in this case, I should use a filter like LWLibavVideoSource which indexes the whole stream first. OK, I got that. But then how is it that FFVideoSource which also indexes the stream gives such catastrophic results in StaxRip when trying to seek? Is it its default seek mode (I did use the defaults)? Or is this a bug in v. 2.23.1 which has been fixed in later versions?
manolito is offline   Reply With Quote
Old 12th July 2019, 15:41   #23573  |  Link
nevcairiel
Registered Developer
 
Join Date: Mar 2010
Location: Hamburg/Germany
Posts: 9,733
Quote:
Originally Posted by manolito View Post
But then how is it that FFVideoSource which also indexes the stream gives such catastrophic results in StaxRip when trying to seek? Is it its default seek mode (I did use the defaults)? Or is this a bug in v. 2.23.1 which has been fixed in later versions?
That I can't answer. It might just be broken, if it does indexing.

If I could realistically index a file from start to finish, seeking would be perfect every time, but alas I cannot, since people want their files to open instantly, and not in 10-20 seconds.
__________________
LAV Filters - open source ffmpeg based media splitter and decoders
nevcairiel is offline   Reply With Quote
Old 12th July 2019, 16:32   #23574  |  Link
el Filou
Registered User
 
el Filou's Avatar
 
Join Date: Oct 2016
Posts: 467
Quote:
Originally Posted by manolito View Post
My main requirement for TS streams is to reencode them (editing out commercials and get rid of station logos). Finding the correct cut points in the preview does require a lot of seeking, and always getting corrupt frames is very annoying.
Sorry for the off-topic but have you tried TS-Doctor? It has configurable settings for automatically choosing the previous/next I-Frame for cutting, or you can also seek to I-Frames in a preview window. It's not freeware but I find the price well worth it.
__________________
HTPC: W10 1809, E7400, 1050 Ti, DVB-C, Denon 2310, Panasonic GT60 | Desktop: W10 1809, 4690K, HD 7870, Dell U2713HM | MediaPortal 1/MPC-HC, LAV Filters, ReClock, madVR
el Filou is offline   Reply With Quote
Old 12th July 2019, 17:02   #23575  |  Link
manolito
Registered User
 
manolito's Avatar
 
Join Date: Sep 2003
Location: Berlin, Germany
Posts: 2,513
TSDoctor is one of my standard tools for dealing with TS files. But cutting at I-Frame borders will not do it for editing out commercials (especially not with HEVC streams where the key frames can be spead apart very widely).

Since I reencode anyways I do the cutting in AVISynth, and this is frame accurate, no matter what the source format is.
manolito is offline   Reply With Quote
Old 12th July 2019, 22:06   #23576  |  Link
manolito
Registered User
 
manolito's Avatar
 
Join Date: Sep 2003
Location: Berlin, Germany
Posts: 2,513
Quote:
Originally Posted by nevcairiel View Post
That I can't answer. It might just be broken, if it does indexing.
Yes, you are right, v. 2.23.1 is broken...

After replacing it with this version
https://forum.doom9.org/showthread.p...03#post1876703

(I believe it is v. 2.31.0) seeking works perfectly using the default seek mode 1.
manolito is offline   Reply With Quote
Old 12th July 2019, 22:31   #23577  |  Link
Asmodian
Registered User
 
Join Date: Feb 2002
Location: San Jose, California
Posts: 3,530
Quote:
Originally Posted by nevcairiel View Post
If I could realistically index a file from start to finish, seeking would be perfect every time, but alas I cannot, since people want their files to open instantly, and not in 10-20 seconds.
A cool feature would be an option to index or use a separate index file, for those who want better seeking in ts or other bad files. I do encounter broken files which are unable to seek more often than I would like (incomplete mp4 is my most common, due to canceling encodes). The way FFMS2 works has always struck me as very convenient and functional. A way to tell LAV that I wouldn't mind indexing the file, or even automatic loading of a previously created index file I have to generate manually would be nice. At least it doesn't seem crazy.

I do have to admit that only a small percentage of the user base would benefit, so it may not be worth the effort, but it would still be cool for me.
__________________
madVR options explained
Asmodian is offline   Reply With Quote
Old 13th July 2019, 07:03   #23578  |  Link
huhn
Registered User
 
Join Date: Oct 2012
Posts: 5,649
even very advanced user get confused by the decoder names and what that means just imagine what will happen with that setting when user start wondering why it sometimes takes so long to start playback.
huhn is offline   Reply With Quote
Old 13th July 2019, 21:18   #23579  |  Link
Asmodian
Registered User
 
Join Date: Feb 2002
Location: San Jose, California
Posts: 3,530
Yeah, it would probably generate a lot of extra questions.

This is a reason simply detecting an index you have to create manually would be good.. but probably not too important anyway. I can always use Vapoursynth.
__________________
madVR options explained
Asmodian is offline   Reply With Quote
Reply

Tags
decoders, directshow, filters, splitter

Thread Tools Search this Thread
Search this Thread:

Advanced Search
Display Modes

Posting Rules
You may not post new threads
You may not post replies
You may not post attachments
You may not edit your posts

BB code is On
Smilies are On
[IMG] code is On
HTML code is Off

Forum Jump


All times are GMT +1. The time now is 16:20.


Powered by vBulletin® Version 3.8.11
Copyright ©2000 - 2019, vBulletin Solutions Inc.