Welcome to Doom9's Forum, THE in-place to be for everyone interested in DVD conversion.

Before you start posting please read the forum rules. By posting to this forum you agree to abide by the rules.

 

Go Back   Doom9's Forum > Video Encoding > MPEG-4 ASP

Reply
 
Thread Tools Search this Thread Display Modes
Old 8th December 2005, 01:12   #281  |  Link
eXistenZ_69
Registered User
 
Join Date: Mar 2003
Posts: 87
Quote:
Originally Posted by SeeMoreDigital
Okay....

As a cross-reference, use YAMB again to de-mux the MPEG-4 in MP4 stream back to MPEG-4 in AVI. Then play this new .AVI file to see if it displays any artefacts!


Cheers
The 'Extract Visual Track [ 1] to an AVI file' option on the extraction tab is grayed out :-/
eXistenZ_69 is offline   Reply With Quote
Old 8th December 2005, 11:12   #282  |  Link
SeeMoreDigital
Life's clearer in 4K UHD
 
SeeMoreDigital's Avatar
 
Join Date: Jun 2003
Location: Notts, UK
Posts: 12,219
Quote:
Originally Posted by eXistenZ_69
The 'Extract Visual Track [ 1] to an AVI file' option on the extraction tab is grayed out :-/
Sounds like you are using an older versions of YAMB.... Here's an example of what you should be seeing: -




Cheers
__________________
| I've been testing hardware media playback devices and software A/V encoders and decoders since 2001 | My Network Layout & A/V Gear |
SeeMoreDigital is offline   Reply With Quote
Old 8th December 2005, 18:11   #283  |  Link
eXistenZ_69
Registered User
 
Join Date: Mar 2003
Posts: 87
Ah, yes indeed! I was using v1.3.0 which I got from one of the Doom9 guides. I got the YAMB 1.3.2 installer (containing MP4Box) from another forum thread now and it worked.

The from mp4 extracted avi has no artefacts!

So what now? I can cut out a small part where artefacting occurs after unpacking with MPEG4Modifier and upload it so that someone can analyse the the cause maybe...


Regards,

eXistenZ_69
eXistenZ_69 is offline   Reply With Quote
Old 8th December 2005, 23:23   #284  |  Link
Moitah
Registered User
 
Join Date: Feb 2004
Location: Virginia, USA
Posts: 334
Quote:
Originally Posted by eXistenZ_69
So what now? I can cut out a small part where artefacting occurs after unpacking with MPEG4Modifier and upload it so that someone can analyse the the cause maybe...
Yes, please do (from the original video of course, don't unpack it).
__________________
moitah.net
Moitah is offline   Reply With Quote
Old 8th December 2005, 23:27   #285  |  Link
Moitah
Registered User
 
Join Date: Feb 2004
Location: Virginia, USA
Posts: 334
Quote:
Originally Posted by celtic_druid
As far as I know it produces bit identical output and has when I have tested. Encode with XviD and pb enabled, run MPEG4Modifer to unpack, compare that to XviD with pb disabled and the output is the same.
Correct, I have tested it this way before.
__________________
moitah.net
Moitah is offline   Reply With Quote
Old 8th December 2005, 23:30   #286  |  Link
SeeMoreDigital
Life's clearer in 4K UHD
 
SeeMoreDigital's Avatar
 
Join Date: Jun 2003
Location: Notts, UK
Posts: 12,219
Quote:
Originally Posted by eXistenZ_69
So what now?

I can cut out a small part where artefacting occurs after unpacking with MPEG4Modifier and upload it so that someone can analyse the the cause maybe...
Yes please....

That said, are you 100% sure you did not fart about with the "User Data" settings in MPEG4 Modifier, prior to removing the packed bit-stream
__________________
| I've been testing hardware media playback devices and software A/V encoders and decoders since 2001 | My Network Layout & A/V Gear |

Last edited by SeeMoreDigital; 8th December 2005 at 23:34.
SeeMoreDigital is offline   Reply With Quote
Old 9th December 2005, 08:49   #287  |  Link
eXistenZ_69
Registered User
 
Join Date: Mar 2003
Posts: 87
Quote:
Originally Posted by SeeMoreDigital
Yes please....

That said, are you 100% sure you did not fart about with the "User Data" settings in MPEG4 Modifier, prior to removing the packed bit-stream
Well, define 'fart about'

It's on 'auto' by default and I don't touch that. I just load the file, check unpack and save it...


Greetz,

eXistenZ_69
eXistenZ_69 is offline   Reply With Quote
Old 9th December 2005, 10:08   #288  |  Link
SeeMoreDigital
Life's clearer in 4K UHD
 
SeeMoreDigital's Avatar
 
Join Date: Jun 2003
Location: Notts, UK
Posts: 12,219
I'm sure any information will be useful to Moitah
__________________
| I've been testing hardware media playback devices and software A/V encoders and decoders since 2001 | My Network Layout & A/V Gear |
SeeMoreDigital is offline   Reply With Quote
Old 10th December 2005, 17:31   #289  |  Link
eXistenZ_69
Registered User
 
Join Date: Mar 2003
Posts: 87
I uploaded a file named 'problem_sample.rar' to a free online file hosting service (see link below). It contains a small fragment from the file that has the artefacting after unpacking with MPEG4Modifier. I included the unpacked versions by MPEG4Modifier and MP4Box as well. Size of the rar is 1.31MB.

http://www.freefileupload.net/file.p...lem_sample.rar

I compared the info MPEG4Modifier shows for these three files and noticed the following: the unpacked version by MPEG4Modifier loses 1 P-VOP! The unpacked version by MP4Box still has all original VOP's. Another difference is that the MP4Box one has DivX999b000n, XviD0038 as user data, the MPEG4Modifier one only XviD0038 (auto setting was enabled during unpacking so this is as expected).

I hope this enables Moitah to find and fix the bug!


Regards,

eXistenZ_69
eXistenZ_69 is offline   Reply With Quote
Old 10th December 2005, 21:00   #290  |  Link
Moitah
Registered User
 
Join Date: Feb 2004
Location: Virginia, USA
Posts: 334
Thanks for the sample files. I have found the problem. This video was encoded in segments and joined together. You can see this by using the "Write Frame List" button inside "Video Info", at frame 90 the timestamps jump backwards. The problem is that frame 89 is "packed" (it contains a P-VOP and a B-VOP), but there is no corresponding N-VOP placeholder to use for unpacking. In this case, I used to stop and show the error message "No dummy N-VOP found for packed frame", but in 1.3.0 I removed the error message and ignored the problem, without realizing more needs to be done.

I think I can fix this easily but I'm not sure yet, I don't have Visual Studio installed because I reinstalled Windows a few days ago. Hopefully I will have time tonight to do it.
__________________
moitah.net
Moitah is offline   Reply With Quote
Old 10th December 2005, 22:24   #291  |  Link
eXistenZ_69
Registered User
 
Join Date: Mar 2003
Posts: 87
Great you found the problem! I'm glad I could be of assistance. Should have mentioned that it concerns an encoding of a tv series...

I got a few questions regarding your explanation:

1) Is the fact that the timestamps jump backwards normal?
2) Does the problem occur on the join points of the segments? Strange, cause the point where artefacting occurs doesn't look like a moment where commercials where cut out (it's in the middle of a scene).
3) Is the artefacting because of a packed frame that didn't get unpacked or just left out?
4) Is it possible to detect affected unpacked files?
4) Is it possible to fix affected unpacked files? I don't have the original packed files of all files I've unpacked until now :-/

Take your time for the fix. No need to rush out a new version of MPEG4Modifier with a solution that hasn't been properly thought through

Regards,

eXistenZ_69

Last edited by eXistenZ_69; 10th December 2005 at 22:26.
eXistenZ_69 is offline   Reply With Quote
Old 10th December 2005, 22:36   #292  |  Link
eXistenZ_69
Registered User
 
Join Date: Mar 2003
Posts: 87
Found a related forum thread:

http://forum.doom9.org/showthread.php?postid=518857
eXistenZ_69 is offline   Reply With Quote
Old 11th December 2005, 00:46   #293  |  Link
Moitah
Registered User
 
Join Date: Feb 2004
Location: Virginia, USA
Posts: 334
Quote:
1) Is the fact that the timestamps jump backwards normal?
Normal? Not for a regular encode, but when you join several encodes then yes. I'm not sure if any decoders have a problem with this or not. But it's not what's causing the problem in this clip.

Quote:
2) Does the problem occur on the join points of the segments? Strange, cause the point where artefacting occurs doesn't look like a moment where commercials where cut out (it's in the middle of a scene).
Yes, it appears to have been joined on frame 90, which seems rather strange to me also. I can't think of anything else that would cause a missing N-VOP and a change in timestamps.

Quote:
3) Is the artefacting because of a packed frame that didn't get unpacked or just left out?
It happens because the first P-VOP of the next scene got removed. The code is holding a VOP in memory looking for somewhere to put it (an N-VOP). It never finds anywhere to put it, so the next time it encounters a packed frame, the frame it was holding gets overwritten.

Quote:
4) Is it possible to detect affected unpacked files?
It would be possible.

Quote:
4) Is it possible to fix affected unpacked files? I don't have the original packed files of all files I've unpacked until now :-/
Unfortunately, no. Sorry .
__________________
moitah.net
Moitah is offline   Reply With Quote
Old 11th December 2005, 03:50   #294  |  Link
Moitah
Registered User
 
Join Date: Feb 2004
Location: Virginia, USA
Posts: 334
I have fixed the problem. There's really no perfect way to handle this. If you make room for the extra VOP by shifting all the other VOPs forward (which I cannot do because of the way the code is currently structured), you will mess up the audio synch. The other way it to just remove the extra VOP, which is what I did. The latter is what would have happened anyway if the files were encoded with packed bitstream off before they were joined. Basically, the VOP sequence in this file isn't really valid because of the way it was cut. For anyone cutting/joining MPEG-4 ASP, I would recommend A) making all cuts on I-VOP boundaries (not only the beginning, but the end as well), and B) not using the frames after and including the last I-VOP, because XviD doesn't always end cleanly (it makes the last frame a B-VOP, which doesn't make any sense).

Also, I removed the "smart unpacking" code, which was supposed to handle files that contained real N-VOPs along with packed bitstream. I don't think any encoder does this, so it was just needlessly complicating the code.

I don't know when I'll get around to releasing it, tomorrow if I have time.
__________________
moitah.net
Moitah is offline   Reply With Quote
Old 11th December 2005, 04:14   #295  |  Link
charleski
Registered User
 
charleski's Avatar
 
Join Date: Jul 2004
Posts: 383
Quote:
Originally Posted by celtic_druid
There aren't still standalones that can't handle packed bitstream + multiple bframes are there though? Shouldn't they all have recieved firmware updates by now?
That would imply that the companies selling these standalone mpeg-4 players weren't running on razor-thin margins and could afford to provide more than a couple of months' worth of updates.
charleski is offline   Reply With Quote
Old 11th December 2005, 12:48   #296  |  Link
bond
Registered User
 
Join Date: Nov 2001
Posts: 9,770
missing n-vops occur at the end of encodes, when the last frame is a packed one?
__________________
Between the weak and the strong one it is the freedom which oppresses and the law that liberates (Jean Jacques Rousseau)
I know, that I know nothing (Socrates)

MPEG-4 ASP FAQ | AVC/H.264 FAQ | AAC FAQ | MP4 FAQ | MP4Menu stores DVD Menus in MP4 (guide)
Ogg Theora | Ogg Vorbis
use WM9 today and get Micro$oft controlling the A/V market tomorrow for free
bond is offline   Reply With Quote
Old 11th December 2005, 14:07   #297  |  Link
eXistenZ_69
Registered User
 
Join Date: Mar 2003
Posts: 87
Quote:
Originally Posted by Moitah
I have fixed the problem. There's really no perfect way to handle this. If you make room for the extra VOP by shifting all the other VOPs forward (which I cannot do because of the way the code is currently structured), you will mess up the audio synch.
With extra VOP you mean the missing N-VOP that should have been frame 90, right?

Quote:
Originally Posted by Moitah
The other way it to just remove the extra VOP, which is what I did.
And here you mean the packed frame at position 89, correct? Won't removing it cause audio sync issues as well?


How does MP4Box handle this situation?

How come playback of the packed bitstream doesn't show artefacting? Isn't the missing N-VOP needed there as well??

Regards,

eXistenZ_69

Last edited by eXistenZ_69; 11th December 2005 at 14:19.
eXistenZ_69 is offline   Reply With Quote
Old 11th December 2005, 14:17   #298  |  Link
eXistenZ_69
Registered User
 
Join Date: Mar 2003
Posts: 87
Quote:
Originally Posted by Moitah
It happens because the first P-VOP of the next scene got removed. The code is holding a VOP in memory looking for somewhere to put it (an N-VOP). It never finds anywhere to put it, so the next time it encounters a packed frame, the frame it was holding gets overwritten.
I'm trying to understand your explanation here, but it's a bit confusing. Isn't the first sentence here contradictory with the rest? Doesn't frame 89 get overwritten with frame 91? The first sentence says it's the other way round...
eXistenZ_69 is offline   Reply With Quote
Old 11th December 2005, 20:32   #299  |  Link
Moitah
Registered User
 
Join Date: Feb 2004
Location: Virginia, USA
Posts: 334
@bond: Yes, or it can add some B-VOPs afterwards. Ideally the end of the video should look something like:

Code:
(Frame#: VOPType)
57: P-VOP
    B-VOP
58: B-VOP
59: N-VOP
However, XviD (maybe others too, I haven't tried) sometimes will end with:

Code:
58: P-VOP
    B-VOP
59: B-VOP
or
Code:
59: P-VOP
    B-VOP
__________________
moitah.net
Moitah is offline   Reply With Quote
Old 11th December 2005, 21:07   #300  |  Link
Moitah
Registered User
 
Join Date: Feb 2004
Location: Virginia, USA
Posts: 334
eXistenZ_69: I think it's a lot easier to understand if you use the "Write Frame List" button and inspect the output.

Original packed video:
Code:
    89: P-VOP    (0:00:03.840)
        B-VOP    (0:00:03.756)
    90: I-VOP    (0:00:03.000)
    91: P-VOP    (0:00:03.125)
        B-VOP    (0:00:03.041)
    92: B-VOP    (0:00:03.083)
    93: N-VOP(D) (0:00:03.125)
Unpacked in 1.3.2:
Code:
    89: P-VOP    (0:00:03.840)
    90: B-VOP    (0:00:03.756)
    91: I-VOP    (0:00:03.000)
    92: B-VOP    (0:00:03.041)
    93: B-VOP    (0:00:03.083)
Notice the B-VOP previously in frame 89 was shifted forward one, and the I-VOP also shifted forward. The P-VOP previously in frame 91 was removed, and then the two B-VOPs were shifted forward. This doesn't really make sense of course, but that's just how it turned out with the way the code was written.


Unpacked in 1.3.3:
Code:
    89: P-VOP    (0:00:03.840)
    90: I-VOP    (0:00:03.000)
    91: P-VOP    (0:00:03.125)
    92: B-VOP    (0:00:03.041)
    93: B-VOP    (0:00:03.083)
Now the code is smarter and only shifts B-VOPs forward. When it encounters the I-VOP, it realizes it needs to stop shifting, and discards the B-VOP from frame 89 that it was trying to shift forward.


Unpacked in MP4Box (frame numbers adjusted to compensate for MP4Box adding 2 delay frames near the beginning):
Code:
    89: P-VOP    (0:00:03.840)
    90: B-VOP    (0:00:03.756)
    91: I-VOP    (0:00:03.000)
    92: P-VOP    (0:00:03.125)
    93: B-VOP    (0:00:03.041)
    94: B-VOP    (0:00:03.083)
All the frames are kept, which unless I am understanding something wrong, would offset all frames after 90 with respect to the audio.
__________________
moitah.net

Last edited by Moitah; 11th December 2005 at 21:09.
Moitah is offline   Reply With Quote
Reply

Thread Tools Search this Thread
Search this Thread:

Advanced Search
Display Modes

Posting Rules
You may not post new threads
You may not post replies
You may not post attachments
You may not edit your posts

BB code is On
Smilies are On
[IMG] code is On
HTML code is Off

Forum Jump


All times are GMT +1. The time now is 07:19.


Powered by vBulletin® Version 3.8.11
Copyright ©2000 - 2024, vBulletin Solutions Inc.