open-gop patch for x264 (committed)

[Trahald]

@me7
What format is it (.ts m2ts mp4 avi etc is it) and how did you mux it (x264 was the only tool you used, yamb(mp4box), tsmuxer, etc..) then we can now if its the muxer or the player.

[Dark Shikari]

CoreAVC with CUDA doesn't seem to like it too, picture is sometimes messed up after seeking. When I skip back a few seconds and watched the previously corrupted part again (without seeking into the trouble area), everything is fine.That sounds more likely to be a muxer or demuxer issue.

[me7]

I encoded it with Lord_Mulder's GUI and selected mkv as output type (don't know which muxer it uses), played it with MPC-HC and Haali's Media Splitter.
x264 command line ("x264_x64" is the folder for Lord_Mulder's GUI): C:\Tools\x264_x64\pipebuf.exe C:\Tools\x264_x64\avs2yuv.exe C:\casino\video.avs - : C:\Tools\x264_x64\x264_x64.exe --preset veryslow --tune film --crf 18.0 --bframes 16 --open-gop display --output C:\casino\video.mkv --frames 256547 --demuxer y4m --stdin y4m - : 4

Here is a sample: http://forum.doom9.org/showthread.php?p=1412260#post1412260
The sample was created with mkvtoolnix 4.0, I chose to remux the mkv and told mkvtoolnix to split the output into 2 min chunks. One of them served as that sample.

[Guest]

My testing indicates it has nothing to do with x264. It's not x264's fault if demuxers/decoders can't handle open GOPs properly.

[Atak_Snajpera]

Here is a sample: http://forum.doom9.org/showthread.ph...60#post1412260
I tested on my PC and I couldn't force that sample to show any corruption.
ffdshow r3474 + Haali Media Splitter + MPC-HC (internal filters disabled)

[me7]

I didn't mean to imply that x264 was to blame. I just wanted to add that Flash isn't the only decoder/demuxer that has problems.

[moviefan]

CoreAVC with CUDA doesn't seem to like it too, picture is sometimes messed up after seeking. When I skip back a few seconds and watched the previously corrupted part again (without seeking into the trouble area), everything is fine.

What revision is your x264 binary? I've had the same issue a few weeks ago, but it seems that the patch had a typo accidentally introduced by rack04. My last encodes were all successful with a much more up to date revision of the patch.

[me7]

I used the "official" 1659 build, no patch ;)
It really seems to be related to CoreAVC, some other guy in the CoreAVC thread reproduced it but didn't have any issues with the DivX decoder.

[sneaker_ger]

I used the "official" 1659 build, no patch ;)
It really seems to be related to CoreAVC, some other guy in the CoreAVC thread reproduced it but didn't have any issues with the DivX decoder.

I think he was talking about the DivX Splitter and not the DivX Decoder.
/edit: my bad, he edited his post yet again. He is talking about the DivX decoder.

I tested with DiAVC: no problems.

[shon3i]

@Trahald let's back on jpsdr second problem which is related to Open-GOP. I don't know what you changed from rev 26 to rev 40 (or commited rev), but something not same i suppose that coded mode should be same as rev26 since display mode not pass muigenerator with GOP exceeding error.

Scenarist on mux create large number of warnings with message "Illegal EPMAP". This not happen if keyint 24 used or opengop is disabled, only with keyint 48, and not with your experimental buld which contain rev26 patch.

[jpsdr]

Unhappy for the unfortunate problem, but happy problem confirmed...

[Maccara]

I tested with DiAVC: no problems.

Surprisingly (after reading about all the trouble people have had with ATI), MPC-HC + Haali + ATI DXVA (internal filters) works perfectly with "display" mode too (with my limited tests).

[Trahald]

@Trahald let's back on jpsdr second problem which is related to Open-GOP. I don't know what you changed from rev 26 to rev 40 (or commited rev), but something not same i suppose that coded mode should be same as rev26 since display mode not pass muigenerator with GOP exceeding error.

Scenarist on mux create large number of warnings with message "Illegal EPMAP". This not happen if keyint 24 used or opengop is disabled, only with keyint 48, and not with your experimental buld which contain rev26 patch.

what settings get you the error (full command line). how small of a sample can be made that does it? (im up to 5000 frames unable to duplicate) my pc is slow so i need to narrow that down. meanwhile i'll keep trying

[shon3i]

what settings get you the error (full command line). how small of a sample can be made that does it? (im up to 5000 frames unable to duplicate) my pc is slow so i need to narrow that down. meanwhile i'll keep trying
--level 40 --qpmin 0 --vbv-maxrate 15000 --vbv-bufsize 15000 --keyint 48 --min-keyint 2 --ref 6 --bframe 3 --b-pyramid "strict" --aq-mode 2 --aud --nal-hrd "vbr" --colorprim "bt709" --transfer "bt709" --colormatrix "bt709" --videoformat "ntsc" --sar 1:1 --threads 0 --thread-input --open-gop "coded"

first source is jpsdr "problematic" sample 786 frames long.

second source is mine, some test clip from blu-ray, around 5000 frames long.

both reproduce same warnings.

btw this warnings are show on muxing stage. passes normaly muigenerator.

[jpsdr]

I've tested on another sample than my problematic, of around 1000 frames, it reproduce.

[Trahald]

try using 48 - b-frames for your max key-int. So if you have b-frames = 3 then use 45. and see if you get the epmap warnings. (try a few different encodes)

[shon3i]

I confirm on 2 samples with Keyint 45 and 3 bframes, warnings are gone.

[Trahald]

With this particular warning, my inclination is not to code against it. If it were fatal, I'm sure scenarist would ERROR out and cancel the mux (as opposed to just a warning). So I'll leave it up to the user to truncate the gop length by b-frames value in order to avoid the warning. The reason the older patch wouldnt create the warning is because that patch limited both coded AND display length of gop. The current one only limits coded order when in coded mode. I need someone that knows for sure to clarify what bluray is looking for before I change it again. Unless of course a core developer demands it.

[kemuri-_9]

If it was really the bframes being a problem, then a gop size of 24 (for >15mbps material) should exhibit similar problems with --opengop coded.
But it was already stated to not.

So I'm inclined to say that something else is the problem and it's just being aggravated by the long gop.

[jpsdr]

Maybe should test with a GOP of 47 to see if there is still the warnings. Will try this evening to see, unless shon3i can test that before.

[Trahald]

If it was really the bframes being a problem, then a gop size of 24 (for >15mbps material) should exhibit similar problems with --opengop coded.
But it was already stated to not.

So I'm inclined to say that something else is the problem and it's just being aggravated by the long gop.

The thing is, i dont think scenarist checks for 1 second gop the way it checks for 2 seconds. I just muxed a video at 17/30 and 35 frames and it muxed no errors or warnings

[shon3i]

So I'll leave it up to the user to truncate the gop length by b-frames value in order to avoid the warning.It's ok for me, but i think is not good solution, since for example Blu-Code and Cinevision aslo reduce gop size according b-frames, ref frames, b-pyramid, open-gop combinations, for example GOP is 22 if all is options is used, in x264 i think like other things should automaticly corrected, at least on coded since is Blu-Ray specific.

If it was really the bframes being a problem, then a gop size of 24 (for >15mbps material) should exhibit similar problems with --opengop coded.Well not all combinations allowed, aslo you can use always GOP 24, totaly independent of max bitrate. I think that nothing has been violated with long gop, just exceeding because is 3 bframes used.

Maybe should test with a GOP of 47 to see if there is still the warnings. Will try this evening to see, unless shon3i can test that before. It still show warning but, with GOP 48 i get 3 warnings, with 47 i get one.

[jpsdr]

Ok, the solution is to use 45 GOP with open GOP and b-frame 3. Fine with me.

[shon3i]

@Trahald, can i send you blu-code stream that use OpenGOP together with LongGOP. I examine stream and i am suprised that stream contain few IDR frames, while x264 only first frame is IDR, other is non-IDR.

whether this could be explanation? or it's just one of implementation of open-gop?

btw when i selected max bitrate to 15mbps, encoder reccomend me to use gop 45 but max is 47. I will test that with blu-code, maybe is something with scenarist

[Trahald]

Ok.. this is what i gathered... the gop size is being determined by coded distance (leading b to leading b) in open gop. what the EP_MAP is is a guide for fast-forward/rewinding the disc. it contains whats called fine entries that point to the start of every GOP. These fine entries point to the IDR/i-frames not the leading b-frames. the time between these entries cannot exceed 2 seconds. since our display times exceed 2 seconds. You end up with a stream that will mux but the complaints of the spec violation. So.. that means .. i'll code a patch that limits coded mode to both display and coded limits and see if it gets in. i'll change options to none/display/both.

unless someone says otherwise and and honestly, comes up with a stream that proves otherwise (48 frames not 24) its the way i'll believe

[Trahald]

@shoni
sure pm me.

[shon3i]

Ok, after playing with Blu-Code, i have clear picture.

Encoder allow to set GOP to 47 after reduce bframes from 3 to 2, otherwise encoder don't allow to set nothing higher than 45. There is aslo one more parameter which explain IRD frames in OpenGOP, that is "IDR Interval Factor" which have default value of 4, and can't be lower. From their manual:

IDR (Instantaneous Decoder Refresh: Instantaneous refresh
of decoder recovery operation) Specify the picture interval.

[Trahald]

That encoder benefits from a GUI. I cant force someone to type in 45. ;) I think truncating for display and coded will do the trick. i'll make up my mind after seeing the stream

[Dark Shikari]

That encoder benefits from a GUI. I cant force someone to type in 45. ;) I think truncating for display and coded will do the trick. i'll make up my mind after seeing the streamIf we're going to truncate, why do we need display/coded?

[Trahald]

the display stream would have fewer i-frames (if the stream were long enough) as truncating both will make the effective average gop length shorter. honestly im ok with one mode, but i was ok with one mode before :)

[Dark Shikari]

the display stream would have fewer i-frames (if the stream were long enough) as truncating both will make the effective average gop length shorter. honestly im ok with one mode, but i was ok with one mode before :)But I thought the whole point of "coded" mode to begin with was that it didn't need truncation? :rolleyes:

i.e. previously we had two options for Blu-ray:

Coded without truncation
Display with truncation

Now you're saying we need Coded with truncation? That's worse than both!

[shon3i]

That encoder benefits from a GUI. I cant force someone to type in 45No but you can use typed value and subtract by number of bframes :)

But truncating display and coded is better solution, since working in previous revisions.

Please check when is qpfile used aslo.

[Trahald]

http://pastebin.org/364480 Can someone make a binary for testing. The option on this patch is --open-GOP "bluray"

[Trahald]

http://www.mediafire.com/?mny22nymrim is a test binary.. please test against anything bluray you have. its a very basic build.

[shon3i]

It's ok now. Muxing passes without "Illegal EPMAP" warnings, verification is passed. On all sources.

x264.exe --tune film --bitrate 4507 --pass 2 --stats Output.264.x264-stats --level 4.1 --ref 4 --bframes 3 --aud --nal-hrd vbr --b-pyramid strict --open-gop bluray --keyint 48 --vbv-maxrate 15000 --vbv-bufsize 30000 --sar 1:1 --slices 4 --output Output.264 Input.avs

[Emulgator]

Good work !

[sneaker_ger]

I have a few questions concerning open-gop. Apparently CoreAVC has problems seeking on streams created with the new --open-gop option. Neuron2 said that it would probably have to request the preceding gop, but I do not understand why. Here's what I thought was true (until now):
1.) x264 has been using non-IDR-i-frames for quite some time now but I see no reports of CoreAVC having seeking problems on those streams. But aren't information of preceding gops necessary for those? Probably not only a single gop but also multiple gops when the gops are very short? (at least in theory?)
2.) the new open-gop option deletes the reference list after the last b-frame, so shouldn't it be unnecessary to have access to the preceding gop? I thought that was the reason the i frames are flagged as recovery point to begin with?

[LoRd_MuldeR]

With OpenGOP you have B-Frames right before the I(DR) Frame. In display order that looks like:
IBBBPBBBPBBBIBBBPBBBP...

But in decoding order, which is how the frames are stored in the file, that would look like this:
IPBBBPBBBIBBBPBBBPBBB...

So those B-Frames that are located after the I-Frame (in decoding order), but they make reference to the I-Frame and to the P-Frame from the previous GOP.

What I don't understand is:
In display order those B-Frames are before the I-Frame. Hence if we start playback at the I-Frame, they are "obsolete" already. Why not skip them entirely?

[sneaker_ger]

Ok, if he meant previous gop in coded order it totally makes sense. Totally overlooked that. But on such a sequence:
PPPPPiPPPPP ("i" as non-IDR-i-frame)
wouldn't it also be necessary to decode frames from the previous gop first? Aren't such streams created even without open-gop?

As to the "obsolete" question: that's also what I am thinking and I guess we are correct. That's why they get flagged as recovery points.

[LoRd_MuldeR]

Ok, if he meant previous gop in coded order it totally makes sense. Totally overlooked that. But on such a sequence:
PPPPPiPPPPP ("i" as non-IDR-i-frame)
wouldn't it also be necessary to decode frames from the previous gop first? Aren't such streams created even without open-gop?

Well, if the I-Frame is a Non-IDR I-Frame, then we have to decode the frames before the I-Frame, as the None-IDR I-Frame doesn't flush the reference buffer and thus frames after the I-Frame (in display order) may reference to frames before the I-Frame. But that's nothing specific to OpenGOP. The same applies to all Non-IDR I-Frames, even when OpenGOP is not used. Also, in my understanding, a GOP goes from an IDR-Frame to an IDR-Frame, i.e. None-IDR I-Frames are not GOP delimiters. Not sure what the "official" definition is here.

BTW: As far as I know, x264 always used to put a Non-IDR when a scene-cut is detected, but the minimum key-frame interval isn't reached yet...

[sneaker_ger]

Well, if the I-Frame is a Non-IDR I-Frame, then we have to decode the frames before the I-Frame, as the None-IDR I-Frame doesn't flush the reference buffer and thus frames after the I-Frame (in display order) may reference to frames before the I-Frame. But that's nothing specific to OpenGOP. The same applies to all Non-IDR I-Frames, even when OpenGOP is not used.

That's why it was bugging me that some devs were talking about reading the previous gop as something special why from my understanding it was necessary all along.

Also, in my understanding, a GOP goes from an IDR-Frame to an IDR-Frame, i.e. None-IDR I-Frames are not GOP delimiters. Not sure what the "official" definition is here.

Then open-gop wouldn't really be open gop, would it? But ok, the term might just be a "tradition" coming from older standards.

[Guest]

There are cases other than IDR where you do not have to go back a GOP. E.g.:

Recovery point SEI
non-IDR I picture
...

The sample posted earlier in the thread places recovery points before the non-IDR I pictures as above. In fact, in the very first post Trahald states that is what the "open GOP" patch does. I do not consider those to be open GOPs! For me a GOP is open if you have to go back a GOP to decode without errors.

The decision whether to go back is quite tricky. I see it like this (coding order):

IDR starts closed GOP.
RPS+I starts closed GOP.
I P B starts closed GOP. (This is the controversial one. Theoretically not so, but practically yes.)
I B P starts open GOP.

For me a GOP is just a sequence of pictures starting with IDR/I and it is seekable without going back if it is closed.

[sneaker_ger]

The decision whether to go back is quite tricky. I see it like this (coding order):

RPS+I starts closed GOP.

But the open-gop option of x264 uses recovery points on the open gops, so if you seek to a b-frame in a gop that starts with RPS+I we will still need information from the previous gop.

This hole debate is a real mess when we do not have a common opinion on the definition of a gop (and/or forget to mention whether we're talking coded or display order). Is there any definition in the h.264 standard?

[Guest]

But the open-gop option of x264 uses recovery points on the open gops, so if you seek to a b-frame in a gop that starts with RPS+I we will still need information from the previous gop. The RPS declares that you do not need previous information.

[Trahald]

blurays definition of open-gop says this
1. the first frame in decode order must be a non-idr i frame and the frames that proceed it in display order are allowed to reference forward and backward. these frames cannot be correctly decoded during a random seek and are skipped.
2. frames after the non idr i-frame in display order must NOT reference frames prior to the i-frame in display order.

recovery points are not mentioned. (and there is plenty of room for a 'shall not' if that were the intent)

the frames in 1. are always b frames in x264. p frames that preceed the i-frame in display order (yet follow in coded order) are allowed but x264 doesnt generate p-frames in this condition. if b-frame decision decides that b-frames arent needed following the iframe in display order, that gop will essentially be closed (again based on blurays definition). The command open gop is ignored when --bframes == none.



blurays definition of open gop is the only one i know of. I modeled its behavior after AVC blurays.

[RBO]

I'm going to go way out on a unfounded claim, and say it's an open-gop related issue.
FYI, h264 open-GOPs feature withing MP4 (MPEG4 part 12 spec) is not specified enough. There have been propositions at MPEG in 2010 to provide a solution. The problem comes from the collusion between RAP and sync points.

The compliant solution is implemented in GPAC. However it is not usable as seeking big files takes a lot of time. x264 (with GPAC MP4 support) marks all key frames as sync points, which is less compliant but practically allows you to seek within the file flawlessly.

Practically, I would advise you to create your MP4 files using x264, and then adding your audio with:
mp4box -add my_audio.xxx my_h264_muxed_with_x264.mp4

We are waiting the normalization decision from MPEG.

Romain

[kypec]

Practically, I would advise you to create your MP4 files using x264, and then adding your audio with:
mp4box -add my_audio.xxx my_h264_muxed_with_x264.mp4

Are you suggesting to create raw H264 streams in x264? :confused:Because that's what I was doing since I started to work with x264 CLI.

[kieranrk]

The compliant solution is implemented in GPAC. However it is not usable as seeking big files takes a lot of time. x264 (with GPAC MP4 support) marks all key frames as sync points, which is less compliant but practically allows you to seek within the file flawlessly.


Why should the container care about the underlying gop structure? Surely it's a job for the decoder to ignore b-frames from the previous gop.

[VFR maniac]

According to 14496-15, sync samples are just IDR-pictures, so treating a I-picture as a sync sample is out-of-spec.
(Current x264's mp4 muxer module sets key-frame (IDR or I) as sync sample.)
I think it is the better solution using sdtp box (Independent and Disposable Samples Box) for marking random access point when Open-GOP is available.

[Selur]

Are you suggesting to create raw H264 streams in x264?
No, he suggested to:
1st use x264s internal muxer to pack the videostream into a .mp4 file
2nd use mp4box to add additional streams

----

*snip*

Cu Selur