Welcome to Doom9's Forum, THE in-place to be for everyone interested in DVD conversion.

Before you start posting please read the forum rules. By posting to this forum you agree to abide by the rules.

 

Go Back   Doom9's Forum > Capturing and Editing Video > HDTV / DVB / TiVo

Reply
 
Thread Tools Search this Thread Display Modes
Old 23rd May 2018, 11:18   #1  |  Link
Music Fan
Registered User
 
Join Date: May 2009
Location: Belgium
Posts: 1,647
How to repair this TS (h264) ?

Hi,
I often make AVCHDs with tsMuxer for movies I recorded on tv with my VU+ decoder.
I noticed recently that the default setting for video in tsMuxer ("Insert SEI and VUI data if absent") could create some artifacts with some recordings, thus now I select "Do not change SEI and VUI data" and it's ok.

But it was too late for one of my recordings and I don't have the original file anymore. I don't remember if the original file had these problems but I'm pretty sure it was ok and eac3to did not detect any bug when I used it to demux the original TS (I did it mostly to remove dialog norm in the ac3 track and thus I demuxed video too).
The demux with eac3to seems to have the same effect than the remux with tsMuxer (with its default setting) on some videos.

When I look at the new ts on my pc, there are some sort of pixelation. Here is a cap (click for full screen);



When I look at it on my standalone player (Oppo), the problem is different, there are freezes.

Is there a way to repair it, possibly without re-encoding ?

I of course tried to remux it with the other settings but they don't have effect.

Thanks.
Music Fan is offline   Reply With Quote
Old 24th May 2018, 08:39   #2  |  Link
FranceBB
Broadcast Encoder
 
FranceBB's Avatar
 
Join Date: Nov 2013
Location: Germany
Posts: 285
Does it happen in just one scene?
If it doesn't happen very often, you could trim out the corrupted frames and use interpolation to replace them via Avisynth.
That's the first thing that popped up in my mind, 'cause you said that the original is lost and looking at the picture I see that blocking is so severe that a deblocking filter wouldn't work.
__________________
Broadcast Encoder
FranceBB is offline   Reply With Quote
Old 24th May 2018, 08:52   #3  |  Link
Music Fan
Registered User
 
Join Date: May 2009
Location: Belgium
Posts: 1,647
It happens in several scenes.
I'm not gonna try to repair it with Avisynth, and as I said, there are freezes and no pixelation on my standalone player, thus the problem is quite special.
I only want to recreate the SEI and VUI data if possible.
Music Fan is offline   Reply With Quote
Old 24th May 2018, 17:30   #4  |  Link
mkver
Registered User
 
Join Date: May 2016
Posts: 85
Can you upload a sample?
mkver is offline   Reply With Quote
Old 24th May 2018, 20:11   #5  |  Link
Music Fan
Registered User
 
Join Date: May 2009
Location: Belgium
Posts: 1,647
Yes, but before, can you tell me if what I wanna do is possible ?
Music Fan is offline   Reply With Quote
Old 24th May 2018, 21:29   #6  |  Link
mkver
Registered User
 
Join Date: May 2016
Posts: 85
How should we know when we don't have a sample to analyze? Do you have a (nondefective) file that gets screwed by tsmuxer with the "Insert SEI and VUI data if absent" option? Then upload it so that we can see what ill tsmuxer created. Such a sample would be even more valueable than a sample of an already defective tv recording.
mkver is offline   Reply With Quote
Old 24th May 2018, 21:44   #7  |  Link
Music Fan
Registered User
 
Join Date: May 2009
Location: Belgium
Posts: 1,647
Quote:
Originally Posted by mkver View Post
How should we know when we don't have a sample to analyze?
I gave detailed explanations, it can help to know if tools exist to repair lost (or modified) datas.
I guess it's not completely h264 related because tsMuxer does not recode, it looks like a container problem.

Quote:
Originally Posted by mkver View Post
Do you have a (nondefective) file that gets screwed by tsmuxer with the "Insert SEI and VUI data if absent" option? Then upload it so that we can see what ill tsmuxer created. Such a sample would be even more valueable than a sample of an already defective tv recording.
Ok, I'm gonna make a research, I believe I have it.
Music Fan is offline   Reply With Quote
Old 24th May 2018, 23:42   #8  |  Link
Music Fan
Registered User
 
Join Date: May 2009
Location: Belgium
Posts: 1,647
Here is a sample cut with tsMuxer and saved with the setting "Do not change SEI and VUI data" ;
https://www.sendspace.com/file/prknwt

Open it in tsMuxer and save it with the setting "Insert SEI and VUI data if absent", you will see what happens.
Music Fan is offline   Reply With Quote
Old 25th May 2018, 09:21   #9  |  Link
mkver
Registered User
 
Join Date: May 2016
Posts: 85
I see what the issue is. But first some information: The H.264 standard knows two kinds of "packets" (if you want to call them) called SPS (Sequence Parameter Set) and PPS (Picture Parameter Set). They contain information that is essential for the decoding process (like how many reference frames there are, what the picture dimensions are). They can be several of them simultaneously (each PPS has a PPS id, each SPS a SPS id) and every picture uses exactly one of them during decoding (which one is written in the header of the picture (or actually "slice")). If the bitstream contains a PPS with a PPS id that is already known, the new PPS instantaneously overwrites the old one (with the same PPS id) and is used thereon.
Your file contains one constant SPS and PPS with id in the range [0..3]. And they change in the middle of a GOP (GOP = Group Of Pictures; a keyframe by definition starts a GOP and end the last one (if there was one) so that a GOP consists of the frames between two keyframes (the last keyframe is part of the GOP it starts, not of the GOP of the preceding keyframe)). If you instruct tsmuxer to insert SEI and VUI data, then it seems to strip the SPS and PPS that were in the bitstream away and replaces them with new ones. Said new ones are prepended to keyframes (not to the frames they were originally in) and they contain all the SPS and PPS that existed at the time of said keyframe. When a PPS is overwritten mid-GOP, the file produced by tsmuxer doesn't contain the new PPS when it is needed, because the last keyframe only contains the old, non-overwritten PPS.
Can this be fixed? Hypothetically, yes. Practically, no: Whenever you get decoding errors, you can try another PPS. Where to get the other PPS? Encoder that change SPS/PPS midstream usually use only a few values for each PPS (in your case, I counted four PPS with id 0, five with id 1, four with id 2 and two with id 3) and change between these values. The best guess for the PPS is probably the PPS that is prepended to the next keyframe. But honestly, it's probably not worth the effort.
PS: There is actually a second bug: recovery_point SEI messages are discarded by tsmuxer which means that a lot of keyframes of the input end up not being keyframes of the output (but tsmuxer still prepends SPS and PPS in front of them).
mkver is offline   Reply With Quote
Old 25th May 2018, 10:11   #10  |  Link
Music Fan
Registered User
 
Join Date: May 2009
Location: Belgium
Posts: 1,647
Thanks, very interesting.
Does it mean that the option "Insert SEI and VUI data if absent" doesn't really work in tsMuxer ? Or it depends on the encoding ?

Quote:
Originally Posted by mkver View Post
in your case, I counted four PPS with id 0, five with id 1, four with id 2 and two with id 3) and change between these values.
How did you do this ?
Is there a way to compare both versions ("Insert ..." and "Do not change ...") ?
This may help to see how tsMuxer works (and to find a sort of sequence in the SPS and PPS order) to make a kind of reverse engineering, even when we don't have the original file anymore.
But I'm probably dreaming

Last edited by Music Fan; 25th May 2018 at 10:36.
Music Fan is offline   Reply With Quote
Old 25th May 2018, 15:32   #11  |  Link
mkver
Registered User
 
Join Date: May 2016
Posts: 85
It means that this option is unsafe.
I used two tools for this: First, ffmpeg's trace_headers bitstream filter and secondly mkvmerge with the option --debug avc_sps_pps_changes.
mkver is offline   Reply With Quote
Old 25th May 2018, 20:52   #12  |  Link
Music Fan
Registered User
 
Join Date: May 2009
Location: Belgium
Posts: 1,647
Ok thanks.
Music Fan is offline   Reply With Quote
Reply

Thread Tools Search this Thread
Search this Thread:

Advanced Search
Display Modes

Posting Rules
You may not post new threads
You may not post replies
You may not post attachments
You may not edit your posts

BB code is On
Smilies are On
[IMG] code is On
HTML code is Off

Forum Jump


All times are GMT +1. The time now is 15:29.


Powered by vBulletin® Version 3.8.11
Copyright ©2000 - 2018, vBulletin Solutions Inc.