Welcome to Doom9's Forum, THE in-place to be for everyone interested in DVD conversion.

Before you start posting please read the forum rules. By posting to this forum you agree to abide by the rules.

 

Go Back   Doom9's Forum > Video Encoding > MPEG-4 AVC / H.264

Reply
 
Thread Tools Search this Thread Display Modes
Old 30th August 2014, 18:54   #1  |  Link
asarian
Registered User
 
Join Date: May 2005
Posts: 1,173
H264 splitter?

I wonder, do H264 video/file splitters exist? Generally video splitter split across I-Frames, which sounds okay. But what about B-Frames then? What if they look back to before the last I-Frame? (Or is that not allowed?). Maybe something that splits on IDR frames?

I'm also looking into going the AviSynth route, with EDLmaker, to cut on scene changes (although that looks even riskier).
__________________
Gorgeous, delicious, deculture!
asarian is offline   Reply With Quote
Old 30th August 2014, 21:35   #2  |  Link
videoh
Registered User
 
Join Date: Jul 2014
Posts: 562
Yes, of course such splitters exist. If you have open GOPs, you'll need something that re-encodes the GOP. VideoRedo is often used for AVC editing. Source filters, such as DGSource(), will also let you serve arbitrary clips via an Avisynth script, avoiding having to actually split source streams.
videoh is offline   Reply With Quote
Old 30th August 2014, 21:46   #3  |  Link
asarian
Registered User
 
Join Date: May 2005
Posts: 1,173
Quote:
Originally Posted by videoh View Post
Yes, of course such splitters exist. If you have open GOPs, you'll need something that re-encodes the GOP. VideoRedo is often used for AVC editing. Source filters, such as DGSource(), will also let you serve arbitrary clips via an Avisynth script, avoiding having to actually split source streams.
Thx. I'll look into VideoRedo. Already found it mentioned several times, and it comes highly recommended.

Yeah, definitely don't want things split on open GoP's, as that would be horrible.
__________________
Gorgeous, delicious, deculture!
asarian is offline   Reply With Quote
Old 31st August 2014, 07:41   #4  |  Link
asarian
Registered User
 
Join Date: May 2005
Posts: 1,173
Quote:
Originally Posted by videoh View Post
Yes, of course such splitters exist. If you have open GOPs, you'll need something that re-encodes the GOP. VideoRedo is often used for AVC editing. Source filters, such as DGSource(), will also let you serve arbitrary clips via an Avisynth script, avoiding having to actually split source streams.
Turns out VideoRedo is an absolute disaster! First of all, it can't handle lossless H264 (really); and then it bumped out on my lowly mpeg-1 source, saying it had no audio (which it didn't, of course, but who cares?!).

All I want is a h264 splitter than can 'GoP-safe' split an input file into multiple parts, LOL. So, "of course such splitters exist." does not feel so natural to me any more as I thought it would.
__________________
Gorgeous, delicious, deculture!
asarian is offline   Reply With Quote
Old 31st August 2014, 08:10   #5  |  Link
vivan
/人 ◕ ‿‿ ◕ 人\
 
Join Date: May 2011
Location: Russia
Posts: 649
Quote:
Originally Posted by asarian View Post
Turns out VideoRedo is an absolute disaster! First of all, it can't handle lossless H264 (really);
Lossless H.264 is different from "regular" H.264 (High Profile I mean), so it's not that surprising.

Splitting not on IDR is a bad idea, so tools should avoid that (and instead of splitting on selected frame they split on the nearest IDR).
Try mkvtoolnix, mp4box/yamb or ffmpeg...
vivan is offline   Reply With Quote
Old 31st August 2014, 08:16   #6  |  Link
sneaker_ger
Registered User
 
Join Date: Dec 2002
Posts: 5,120
Mkvtoolnix splitting does not work correctly with open GOPs.
sneaker_ger is offline   Reply With Quote
Old 31st August 2014, 09:09   #7  |  Link
Dust Signs
Registered User
 
Join Date: Jun 2004
Location: Salzburg, Austria
Posts: 219
I wrote a H.264 bit stream splitter for a project at work once, but I assume that it won't be of much help for you since it
1. Does not allow you to split at open GOPs, only closed GOPs with IDRs
2. Splits the bit stream into all of its GOPs at once and puts them in an output directory (so splitting is not limited to a certain frame or point in time)
Although one could trivially extend this and merge the files back together (which costs an unnecessary amount of time and disk space), I assume that this is not what you are looking for.

If you are looking for a splitting tool, you could try the following workaround which worked for me at times: Put your H.264 bit stream into an MP4 container (using mp4box), then cut it (using mp4box) and re-extract it (using mp4box). Of coure, this will only cut at the nearest IDR frame, but if this is what you are looking for, this may be a workaround to get what you want.

Dust Signs
__________________
The number you dialed is imaginary. Please turn your phone by 90 and try again
Dust Signs is offline   Reply With Quote
Old 31st August 2014, 09:42   #8  |  Link
asarian
Registered User
 
Join Date: May 2005
Posts: 1,173
Quote:
Originally Posted by Dust Signs View Post
1. Does not allow you to split at open GOPs, only closed GOPs with IDRs
No, no, that's a GOOD thing! Splitting inside an open GoP invariably means you're gonna get glitches. In fact, I was looking for precisely a tool like yours, LOL, that just doesn't blindly split (like every 1,000th frame or something), but does so without ruining the project.

Quote:
2. Splits the bit stream into all of its GOPs at once and puts them in an output directory (so splitting is not limited to a certain frame or point in time)
Although one could trivially extend this and merge the files back together (which costs an unnecessary amount of time and disk space), I assume that this is not what you are looking for.
Again, LOL, this is precisely what I'm looking for! It will be trivial to splice a multitude of GoP sets into AviSynth. And, since they're spit on GoP already, it will be incredibly easy to encode with overlap!

Tl;dr: please, post it! And thank you!
__________________
Gorgeous, delicious, deculture!
asarian is offline   Reply With Quote
Old 31st August 2014, 09:51   #9  |  Link
sneaker_ger
Registered User
 
Join Date: Dec 2002
Posts: 5,120
If you want to process in AviSynth that implies complete re-encoding. Why would you need to split files on GOP borders if you can split/trim on arbitrary frames in AviSynth anyways?
sneaker_ger is offline   Reply With Quote
Old 31st August 2014, 09:59   #10  |  Link
asarian
Registered User
 
Join Date: May 2005
Posts: 1,173
Quote:
Originally Posted by sneaker_ger View Post
If you want to process in AviSynth that implies complete re-encoding. Why would you need to split files on GOP borders if you can split/trim on arbitrary frames in AviSynth anyways?
The idea is to split up my encoding into several standalone parts, so all parts need to be split on closed GoP's. I can then use AviSynth to encode, say, 1,000 GoP's at a time, instead of having one wickedly long, 200+ hours encoding project, only to have it crash at 90% or something.

These parts will be encoded losslessly, and then will undergo a second x264 pass, encoded with a small overlap, in a process that only does the x264 encoding (and not hefty other AviSynth stuff).
__________________
Gorgeous, delicious, deculture!
asarian is offline   Reply With Quote
Old 31st August 2014, 10:05   #11  |  Link
sneaker_ger
Registered User
 
Join Date: Dec 2002
Posts: 5,120
Why don't you resume your lossless (or lossy) encode from the point it crashed (minus a few GOPs at the end that might be broken because of the crash)?
sneaker_ger is offline   Reply With Quote
Old 31st August 2014, 10:11   #12  |  Link
asarian
Registered User
 
Join Date: May 2005
Posts: 1,173
Quote:
Originally Posted by sneaker_ger View Post
Why don't you resume your lossless (or lossy) encode from the point it crashed (minus a few GOPs at the end that might be broken because of the crash)?
I suppose I could. But I rather just have a tool that will make it easier, and cleaner, just splitting up a project into several parts automagically.

Besides, few progs there be that even take a lossless h264 stream for me to determine the right split-point to begin with.
__________________
Gorgeous, delicious, deculture!
asarian is offline   Reply With Quote
Old 31st August 2014, 11:33   #13  |  Link
Dust Signs
Registered User
 
Join Date: Jun 2004
Location: Salzburg, Austria
Posts: 219
Quote:
Originally Posted by asarian View Post
Tl;dr: please, post it! And thank you!
I'll have to clarify about the license I release it under with my employer first, so this might take a bit.

Dust Signs
__________________
The number you dialed is imaginary. Please turn your phone by 90 and try again
Dust Signs is offline   Reply With Quote
Old 31st August 2014, 13:55   #14  |  Link
videoh
Registered User
 
Join Date: Jul 2014
Posts: 562
When splitting AVC files you have more to worry about than just whether the GOP is open. You also must ensure that each split you make is given copies of the applicable SPS/PPS. They may not exist before every IDR.
videoh is offline   Reply With Quote
Old 31st August 2014, 14:57   #15  |  Link
Dust Signs
Registered User
 
Join Date: Jun 2004
Location: Salzburg, Austria
Posts: 219
Quote:
Originally Posted by videoh View Post
When splitting AVC files you have more to worry about than just whether the GOP is open. You also must ensure that each split you make is given copies of the applicable SPS/PPS. They may not exist before every IDR.
It very much depends on the types of streams that you are dealing with. If you want to be able to split every possible H.264-compliant stream there is, a lot of complex cases have to be considered.
If you limit yourself to some types of bit streams, things become a lot easier. The tool I wrote, for example, used to be applied to demuxed H.264 bit streams from Blu-rays or bit streams of very similar structure. In this case, every closed GOP has an SPS, a PPS and an IDR picture, while open GOPs have additional IDR pictures, which makes it quite easy to detect them and to split the bit stream accordingly (open GOPs are combined with the next closed GOP).
For other types of bit streams, it would of course be trivial to copy the SPS and PPS to those GOPs which don't have them, but it would be a lot harder to detect open GOPs in the first place.

Dust Signs
__________________
The number you dialed is imaginary. Please turn your phone by 90 and try again
Dust Signs is offline   Reply With Quote
Old 31st August 2014, 15:29   #16  |  Link
videoh
Registered User
 
Join Date: Jul 2014
Posts: 562
Quote:
Originally Posted by Dust Signs View Post
open GOPs have additional IDR pictures
That makes no sense, because an IDR is not open.

Quote:
it would of course be trivial to copy the SPS and PPS to those GOPs which don't have them
I suppose you'll think it's trivial until you try to do it.
videoh is offline   Reply With Quote
Old 31st August 2014, 16:16   #17  |  Link
STaRGaZeR
4:2:0 hater
 
Join Date: Apr 2008
Posts: 1,307
Unless I haven't understood the OP, he only wants to split at IDR frames, nothing more and nothing less. Mkvtoolnix is just perfect for the job, no need to mess with anything else. Even more so if the resulting parts are to be encoded to something else afterwards.
__________________
Specs, GTX970 - PLS 1440p@96Hz
Quote:
Originally Posted by Manao View Post
That way, you have xxxx[p|i]yyy, where xxxx is the vertical resolution, yyy is the temporal resolution, and 'i' says the image has been irremediably destroyed.
STaRGaZeR is offline   Reply With Quote
Old 31st August 2014, 16:20   #18  |  Link
sneaker_ger
Registered User
 
Join Date: Dec 2002
Posts: 5,120
No, mkvtoolnix does split on non-IDR-I-frames for open GOP files resulting in broken files. It's only safe for non-open GOP.
sneaker_ger is offline   Reply With Quote
Old 31st August 2014, 16:35   #19  |  Link
Dust Signs
Registered User
 
Join Date: Jun 2004
Location: Salzburg, Austria
Posts: 219
Quote:
Originally Posted by videoh View Post
That makes no sense, because an IDR is not open.
I am sorry, but this is not true. There are several Blu-rays out there which use open GOPs with IDRs (instead of I frames). It seems really strange (and is a pain to process), but it is not as uncommon as one might think.

Dust Signs
__________________
The number you dialed is imaginary. Please turn your phone by 90 and try again
Dust Signs is offline   Reply With Quote
Old 31st August 2014, 17:36   #20  |  Link
STaRGaZeR
4:2:0 hater
 
Join Date: Apr 2008
Posts: 1,307
Quote:
Originally Posted by sneaker_ger View Post
No, mkvtoolnix does split on non-IDR-I-frames for open GOP files resulting in broken files. It's only safe for non-open GOP.
Why do you suppose his stream is encoded with open GOPs? The OP doesn't specify anything about it, and the probability of his stream having them is small. He should try mkvtoolnix, and if that doesn't work, then use other tools. It's absurd to start working with the most difficult tools if the easy ones work.
__________________
Specs, GTX970 - PLS 1440p@96Hz
Quote:
Originally Posted by Manao View Post
That way, you have xxxx[p|i]yyy, where xxxx is the vertical resolution, yyy is the temporal resolution, and 'i' says the image has been irremediably destroyed.
STaRGaZeR is offline   Reply With Quote
Reply

Thread Tools Search this Thread
Search this Thread:

Advanced Search
Display Modes

Posting Rules
You may not post new threads
You may not post replies
You may not post attachments
You may not edit your posts

BB code is On
Smilies are On
[IMG] code is On
HTML code is Off

Forum Jump


All times are GMT +1. The time now is 00:04.


Powered by vBulletin® Version 3.8.11
Copyright ©2000 - 2018, vBulletin Solutions Inc.