Stuttery seeking of x264-encoded video in QuickTime Player

[chaynik]

This may have been posted before, but I was unable to find anything in my searches. I've noticed that x264 encoded h.264 video does not seek as "fluidly" in QuickTime Player as h.264 video encoded with other encoders such as Apple Compressor and MainConcept. That is, clicking around with the playhead takes a few seconds to display the frame with x264 video, while the others appear almost instantly. Also, "scanning" around the video (clicking the playhead and dragging) exhibits major lag as well with x264 vs others. Does this occur due to the complexity of the video (B-frames, etc) or does it have to do with something else?

If this is a muxing issue, I apologize and will repost in the corresponding section.

Thanks in advance!

[Dark Shikari]

Quicktime is not a spec-compliant H.264 decoder. You need to set x264 settings just so to make it happy.

[chaynik]

Thanks for the response. What settings would those be?

[Dark Shikari]

Thanks for the response. What settings would those be?Last I heard (not necessarily accurate as of now):

No 8x8dct (or any other high profile features)
1 B-frame max

Also note that muxing with mp4box might help as well.

[chaynik]

Thanks, but if QT is able to play back the video perfectly, just having problems with seeking, are those settings relevant? Here are the lines I use to do my encoding? Do you see anything in it that might be problematic?


x264.exe "temp.avs" --bitrate 3000 --pass 1 --level 3 --ref 3 --mixed-refs --bframes 3 --weightb --subme 7 --me umh --analyse p8x8,b8x8,i4x4 --no-fast-pskip --thread-input --progress --no-psnr --no-ssim --threads auto -o "video.mp4"

x264.exe "temp.avs" --bitrate 3000 --pass 2 --level 3 --ref 3 --mixed-refs --bframes 3 --weightb --subme 7 --me umh --analyse p8x8,b8x8,i4x4 --no-fast-pskip --thread-input --progress --no-psnr --no-ssim --threads auto -o "video.mp4"


I realize I have --bframes 3, contrary to what you said. Would decreasing them to 1 have a noticeable impact on quality?

Thanks!

[Dark Shikari]

Yes, it will result in a significant drop in quality.

But if you're using Quicktime, it isn't as if you care about quality to begin with... ;)

[chaynik]

Yes, it will result in a significant drop in quality.

But if you're using Quicktime, it isn't as if you care about quality to begin with... ;)
It really saddens me to see the hostility towards QuickTime in this community. While not as versatile of a player as others, QuickTime's architecture is behind some of the biggest editing and effects software packages used in post production. Chances are the TV show or movie or music video you are watching was edited in Final Cut or Avid, both of which rely on QuickTime to do all of their rendering.

Now having said that, I have a slight idea of where that hostility is coming from, and that is the Windows version of QuickTime, which is slow, bloated, buggy and unstable. However, if you ever have a chance to work with QuickTime on OS X, your opinions of it will hopefully change.

As much as I care about quality, I also care about compatibility. A large portion of people who will be playing my encodes are OS X users, so I must have QuickTime support nailed to the tee.

Now going back to the original topic, I will try encoding with only 1 b-frame and report back with the results. And on another note, what is the maximum amount of b-frames one can use and what is the maximum QT supports?

[Dark Shikari]

It really saddens me to see the hostility towards QuickTime in this community.How is that hostility? It's a fact. If you're encoding for Quicktime, you are making many quality sacrifices due to the limitations on its playback abilities. Of course, you can bypass this with plugins like Perian.

Why do you assume that all negative statements about Quicktime are inherently attacks on it? Of course, now that you made such statements, you've asked for it...While not as versatile of a player as others, QuickTime's architecture is behind some of the biggest editing and effects software packages used in post production. Chances are the TV show or movie or music video you are watching was edited in Final Cut or Avid, both of which rely on QuickTime to do all of their rendering.And this is a good thing how? As someone who has repeatedly had to deal with the absolutely atrocious Mac video ecosystem--the proprietary formats nothing will decode--the inability of FCP to export to any lossless format supported by anything other than FCP--the utter inability to work reliably with any platform other than OS X--I will celebrate the day when it all disappears and I no longer have to waste days of my life dealing with such crap.

Also, I have an inherent dislike towards any company that arbitrarily selects subsets of (or slightly modifies) standardized video formats for the purpose of being subtly incompatible with everyone else. See Quicktime H.264, SVQ3, etc.Now having said that, I have a slight idea of where that hostility is coming from, and that is the Windows version of QuickTime, which is slow, bloated, buggy and unstable. However, if you ever have a chance to work with QuickTime on OS X, your opinions of it will hopefully change.Having to use Quicktime on OS X is what made me dislike it. If I had only used it on Windows, I would have merely written it off as just another crappy Real-like player that crashed a lot, rather than a pox on the video world that has led to an inordinate waste of my time.As much as I care about quality, I also care about compatibility. A large portion of people who will be playing my encodes are OS X users, so I must have QuickTime support nailed to the tee.What about the fact that Windows users probably won't even have H.264 support out of the box at all?Now going back to the original topic, I will try encoding with only 1 b-frame and report back with the results. And on another note, what is the maximum amount of b-frames one can use and what is the maximum QT supports?There is no limit to B-frames in the standard. x264 limits to 16, max. Rarely is more than 3 useful. QT limits to 1.

P.S. As I hope you can tell, posting flames will get flame responses. Posting questions will get answers. Even if they're both in the same post ;)

[LoRd_MuldeR]

It's hard to have a "good" opinion about QuickTime, because...

1. As you already stated, it's not a big secret that the Windows port of QuickTime is an almost usable piece of software. Even the Apple engineers should be able to notice that. I wonder: Do they have a Windows machine at Apple to test their Windows stuff? Still they neither put any effort into fixing it (it's in that alpha-version-like state for several years now) nor do they show the honesty to officially drop Windows support for QuickTime. It's not like we don't have any alternatives on the Windows platform.

2. First Apple tries to push H.264 as their fancy new video format, then it turns out their own implementation is totally crippled and not spec-compliant.

[chaynik]

How is that hostility? It's a fact. If you're encoding for Quicktime, you are making many quality sacrifices due to the limitations on its playback abilities. Of course, you can bypass this with plugins like Perian.

I have tried Perian, but it doesn't seem to fix the seeking issue. Does Perian take over H.264 decoding capabilities by default or does it only do so if QT's internal decoder fail to do so?

Why do you assume that all negative statements about Quicktime are inherently attacks on it? Of course, now that you made such statements, you've asked for it...
I don't assume so at all, it just seems most people's experience with QuickTime is for playback of end-user delivery formats such as h.264. There is a lot more to QuickTime than just a media player, and its wide use in professinal video environments should be an indication...

And this is a good thing how? As someone who has repeatedly had to deal with the absolutely atrocious Mac video ecosystem--the proprietary formats nothing will decode--the inability of FCP to export to any lossless format supported by anything other than FCP--the utter inability to work reliably with any platform other than OS X--I will celebrate the day when it all disappears and I no longer have to waste days of my life dealing with such crap.

What proprietary formats are you talking about? DV? Uncompressed 4:2:2? MPEG-IMX? Anything will decode these... About the only thing proprietary is ProRes422. Remember, we are talking about its professional use, during editing and finishing. And for that purpose, I've never had a problem with it...

As far as lossless, I have to agree with you, it would be nice to have a cross-platform solution. So far, the closest I've come was this: http://www.bitjazz.com/sheervideo/ Although it's been more than half a year since they promised their AVI port of it...

There is no limit to B-frames in the standard. x264 limits to 16, max. Rarely is more than 3 useful. QT limits to 1.
I see, thank you. When you say QT limits to 1, you mean during encoding stage, right? On playback it supports many, just with possible side effects of slow seeking, etc..

[Dark Shikari]

I have tried Perian, but it doesn't seem to fix the seeking issue. Does Perian take over H.264 decoding capabilities by default or does it only do so if QT's internal decoder fail to do so?I'm not sure--I assume it takes over, but I'm not sure whether it takes over splitting. As I don't use OS X for media playback, I don't have much experience with Perian. Maybe someone else knows the specifics.What proprietary formats are you talking about? DV? Uncompressed 4:2:2? MPEG-IMX? Anything will decode these... About the only thing proprietary is ProRes422. Remember, we are talking about its professional use, during editing and finishing. And for that purpose, I've never had a problem with it...I have a hard disk full of Sheer YUV. Do you realize how hard it was to get them open? It took me over a day and I still only managed to get 5 minutes of PNG sequences out of it.As far as lossless, I have to agree with you, it would be nice to have a cross-platform solution.You mean HuffYUV or FFV1? We have plenty of open-source lossless video formats with crossplatform implementations: we don't need more.I see, thank you. When you say QT limits to 1, you mean during encoding stage, right?No, I mean on playback. That's where the phrase "non-compliant decoder" comes from.On playback it supports many, just with possible side effects of slow seeking, etc..Large keyframe distance (--keyint) affects seeking times. B-frames have nothing to do with it.

[LoRd_MuldeR]

You mean HuffYUV or FFV1? We have plenty of open-source lossless video formats with crossplatform implementations: we don't need more.

And not to forget: We have lossless mode of H.264. Now that libavcodec supports x264's lossless mode, playback is possible on all platforms (e.g. via MPlayer or VLC Player).

[yuvi]

Last time I tried, QuickTime's decoder handled everything except CQM (and 64-bit mp4 timecodes, but it'll simply refuse to open such files.) If you actually have problems make sure to upgrade to at least 7.5. Of course, it is a good deal slower than libavcodec on x86 so I don't use it much.

As for the actual problem, it has absolutely nothing to do with how much of H264 QuickTime supports. QuickTime will seek to the exact frame rather than the nearest keyframe, so if you have a large GOP of course seeking is going to be slow. x264 by default uses a max of 250, which means that potentially 10 seconds of video will have to be decoded when you seek (24 fps of course.)

And as of 1.1.3 Perian only handles high profile by default since too many people complained for one reason or another.

[chaynik]

I have a hard disk full of Sheer YUV. Do you realize how hard it was to get them open? It took me over a day and I still only managed to get 5 minutes of PNG sequences out of it.
Get them open in Windows? I've had great results with tateu's AviSynth plugin QTInput and sheer...

You mean HuffYUV or FFV1? We have plenty of open-source lossless video formats with crossplatform implementations: we don't need more.
I mean something I can export with out of with Final Cut or After Effects ;) Or in a perfect world, even edit with.

No, I mean on playback. That's where the phrase "non-compliant decoder" comes from.
Soo... if it doesn't support more than 1 b-frame, how is it playing my encodes with 3? And yes, I have Perian disabled.

Large keyframe distance (--keyint) affects seeking times. B-frames have nothing to do with it.
YES! That is the answer I was looking for, how did I overlook x264's keyframe setting before?! Thank you!

Last time I tried, QuickTime's decoder handled everything except CQM (and 64-bit mp4 timecodes, but it'll simply refuse to open such files.) If you actually have problems make sure to upgrade to at least 7.5. Of course, it is a good deal slower than libavcodec on x86 so I don't use it much.

As for the actual problem, it has absolutely nothing to do with how much of H264 QuickTime supports. QuickTime will seek to the exact frame rather than the nearest keyframe, so if you have a large GOP of course seeking is going to be slow. x264 by default uses a max of 250, which means that potentially 10 seconds of video will have to be decoded when you seek (24 fps of course.)

And as of 1.1.3 Perian only handles high profile by default since too many people complained for one reason or another.
Today 17:29

The timescales is another issue I had to battle with. Somehow, 2 very similar PCs I have were feeding x264 input with different timescales. Using the same source files, the same scripts and the same versions of everything. One produced files that played back perfectly in QT the other did not. I ended up using NiceFPS() in one of them to compensate but I really wish I could understand what caused this bizarre behavior...

So x264's --keyint setting directly affects QTs seeking behavior then? If I do --keyint 20, how much will my quality suffer?

As far as Perian goes, if I understand the behavior is like so: (open mp4 file)-->(perian checks if its baseline or high)-->(if baseline goto qt decoder, if high goto perian decoder). What about main profile?

Thanks to all for your insight!

[yuvi]

The timescales is another issue I had to battle with. Somehow, 2 very similar PCs I have were feeding x264 input with different timescales. Using the same source files, the same scripts and the same versions of everything. One produced files that played back perfectly in QT the other did not. I ended up using NiceFPS() in one of them to compensate but I really wish I could understand what caused this bizarre behavior...
Dunno if you were running into the 64-bit issue (if the files opened you weren't) but the most common reason 64-bit timestamps are used is because e.g. a framerate of 23.98 is converted to 10000000/417083 somewhere during encoding rather than 24000/1001.

So x264's --keyint setting directly affects QTs seeking behavior then? If I do --keyint 20, how much will my quality suffer?
Quite a bit.

As far as Perian goes, if I understand the behavior is like so: (open mp4 file)-->(perian checks if its baseline or high)-->(if baseline goto qt decoder, if high goto perian decoder). What about main profile?
Main is handled by QT by default.
EDIT: also all interlaced is passed off to QuickTime.

[LoRd_MuldeR]

Soo... if it doesn't support more than 1 b-frame, how is it playing my encodes with 3? And yes, I have Perian disabled.

If you open a H.264 stream in QuickTime, usually one of three things happens:
* It shows an error message with an extremely detailed error description like "Error: -1"
* It consumes an enormous amount of CPU time and hangs until forever, so you have to kill the process
* It starts playback with "green" screen, so you don't see the video and only hear the sound

[Dark Shikari]

Last time I tried, QuickTime's decoder handled everything except CQM (and 64-bit mp4 timecodes, but it'll simply refuse to open such files.) If you actually have problems make sure to upgrade to at least 7.5. Of course, it is a good deal slower than libavcodec on x86 so I don't use it much.Seems they've improved it a lot since most of the guides I've seen were written then.

[cacepi]

Last time I tried, QuickTime's decoder handled everything except CQM
Nope, CQM works just fine (http://rapidshare.com/files/175957394/QuickTime.mp4.html); has since 7.4(?) I believe.

Command line:

--crf 18 --level 3.1 --deblock -1,0 --merange 24 --ref 3 --mixed-refs --fullrange on --deadzone-inter 4 --deadzone-intra 6 --bframes 16 --b-pyramid --subme 9 --b-adapt 2 --qpmin 8 --me umh --partitions all --8x8dct --no-fast-pskip --trellis 0 --cqmfile m4g_hrm_v2.cfg

--Edit: on the Mac. QuickTime is still a POS on Windows.

[chaynik]

Dunno if you were running into the 64-bit issue (if the files opened you weren't) but the most common reason 64-bit timestamps are used is because e.g. a framerate of 23.98 is converted to 10000000/417083 somewhere during encoding rather than 24000/1001.
Yes, I believe that's what it was. But how did the two machines come up with such a discrepancy? They both used identical settings.

Main is handled by QT by default.
EDIT: also all interlaced is passed off to QuickTime.
I thought QT didn't support interlaced h.264, when was it added?

If you open a H.264 stream in QuickTime, usually one of three things happens:
* It shows an error message with an extremely detailed error description like "Error: -1"
* It consumes an enormous amount of CPU time and hangs until forever, so you have to kill the process
* It starts playback with "green" screen, so you don't see the video and only hear the sound
Well in my case it plays back perfectly :) yuvi, can you confirm recent versions of QT being able to play back 1+ bframes?

Nope, CQM works just fine; has since 7.4(?) I believe.

--Edit: on the Mac. QuickTime is still a POS on Windows.
Plays back on Windows here too, QT 7.5.5 (249.13). Now may I ask, what exactly is CQM and what does it do?

[LoRd_MuldeR]

Well in my case it plays back perfectly :)

Just tried with this file and version v.5.5, but only got "green" video output:

General
Complete name : E:\HD\palvik-Clip003 - Copy_001.mp4
Format : MPEG-4
Format profile : Base Media
Codec ID : isom
File size : 84.6 MiB
Duration : 13s 930ms
Overall bit rate : 50.9 Mbps
Encoded date : UTC 2008-11-19 14:59:26
Tagged date : UTC 2008-11-19 14:59:26

Video
Format : AVC
Format/Info : Advanced Video Codec
Format profile : High@L4.1
Format settings, CABAC : Yes
Format settings, ReFrames : 3 frames
Codec ID : avc1
Codec ID/Info : Advanced Video Coding
Duration : 13s 913ms
Bit rate mode : Variable
Bit rate : 49.5 Mbps
Maximum bit rate : 56.5 Mbps
Width : 1 920 pixels
Height : 1 080 pixels
Display aspect ratio : 16/9
Frame rate mode : Constant
Frame rate : 29.970 fps
Resolution : 24 bits
Colorimetry : 4:2:0
Scan type : Progressive
Bits/(Pixel*Frame) : 0.796
Stream size : 84.4 MiB (100%)
Encoded date : UTC 2008-11-19 13:29:14
Tagged date : UTC 2008-11-19 14:59:30

Audio
Format : AAC
Format/Info : Advanced Audio Codec
Format version : Version 4
Format profile : LC
Format settings, SBR : No
Codec ID : 40
Duration : 13s 930ms
Bit rate mode : Variable
Bit rate : 129 Kbps
Maximum bit rate : 135 Kbps
Channel(s) : 2 channels
Channel positions : L R
Sampling rate : 48.0 KHz
Resolution : 16 bits
Stream size : 219 KiB (0%)
Encoded date : UTC 2008-11-19 13:29:17
Tagged date : UTC 2008-11-19 14:59:30

Not to mention that the entire system becomes unresponsive for ~15 seconds before QuickTime finally does do anything :rolleyes:

[Mr VacBob]

I thought QT didn't support interlaced h.264, when was it added?

It was added whenever AVCHD editing started working. (it's disabled in Perian just because we don't have an AVCHD camera to test with, and ffh264 used to crash/decode wrongly much more often)


Well in my case it plays back perfectly :) yuvi, can you confirm recent versions of QT being able to play back 1+ bframes?


Works with multiple B-frames and B-pyramid too, but on x86 it randomly becomes incredibly slow.

[cacepi]

Plays back on Windows here too, QT 7.5.5 (249.13).
It does? Better fire up Parallels...

I'll be, it does! Still doesn't respect the colr atom on Windows, though. Boo.
Now may I ask, what exactly is CQM and what does it do?
CQM stands for Custom Quantization Matrix; in general terms it's a way to increase compression while maintaining detail by modifying how the encoder performs DCT calculation. You can massage the block steps for each specfic macroblock type and smooth or sharpen how DCT coefficients are derived.

In other words, it's a serious Egghead option for squeezing more quality out of an encode for fewer bits. Check out this thread on custom matrices for AVC (http://forum.doom9.org/showthread.php?t=117041) for more details.

[chaynik]

Works with multiple B-frames and B-pyramid too, but on x86 it randomly becomes incredibly slow.
So should I still only use 1 b-frame to prevent these random slowdowns? And if that's the behavior of x86, does that mean it's fine on PPC?

[chaynik]

Still doesn't respect the colr atom on Windows, though. Boo.
Ah, the infamous colr atom! While I have you here, cacepi, how are you setting the colr atom in your encodes? I've been searching far and wide for a way to get QT to display my TV-scale encodes as TV-scale instead of as PC-scale that it defaults to.

[cacepi]

Ah, the infamous colr atom! While I have you here, cacepi, how are you setting the colr atom in your encodes? I've been searching far and wide for a way to get QT to display my TV-scale encodes as TV-scale instead of as PC-scale that it defaults to.
I think you mean the other way around, as most TV scale encodes aren't expanded to PC scale on both Windows and the Mac. You have to fiddle with ffmpeg or use CoreAVC to get full range decoding on Windows; with the Mac it's the colr atom and proper Color parameter, Primaries, Transfer function and Matrix values.

There's a little cli app called mp4track that's part of the mp4v2 (http://code.google.com/p/mp4v2/) library that can do what you need. I uploaded a Universal binary (http://rapidshare.com/files/175983075/mp4track.zip.html) to Rapidshare for you. Run 'mp4track --help' for its proper usage.

Have fun!

[refulgentis]

The Quicktime H.264 decoder is slow (except on the newest machines because it runs on the GPU) and supports everything I think except 64-bit time codes. I'm not surprised it isn't playing a 50 Mbps stream -- its slow. Very slow, maybe 60-70% the speed of libavcodec.

[audyovydeo]

It really saddens me to see the hostility towards QuickTime in this community. While not as versatile of a player as others, QuickTime's architecture is behind some of the biggest editing and effects software packages used in post production.

Now having said that, I have a slight idea of where that hostility is coming from, and that is the Windows version of QuickTime, which is slow, bloated, buggy and unstable. However, if you ever have a chance to work with QuickTime on OS X, your opinions of it will hopefully change.

As much as I care about quality, I also care about compatibility.


Yo. I've been a QuickTime (for Win) user since v 2.0, but I'm afraid I've been pretty much forced to abandon it as a player ever since I switched to x264. There *are* some settings which will help playback (mainly --no-cabac), but I don't see why I must adapt to the player, to such an extent. QT for Win (7.x) will simply not play back high profile (I'm talking PAL/SD rez here) stuff with an acceptable quality, while almost every tom dick & harry, garage-developed windoze player will. I recently tried one of my clips on a friend's iBook (10.4.x ?). Playback was better than under windoze, but only marginally.
And that was a 352x288 clip rez.

my 2 €€€ cents...
audyovydeo

[Ranguvar]

If you need compatibility, just point people to VLC. It's available on pretty much every platform possible, free as in freedom and cost, and plays pretty much every available media type through internal codecs.

[Mr VacBob]

So should I still only use 1 b-frame to prevent these random slowdowns?


Any number should be fine, B-pyramid is a seperate option. And they may have fixed it, too; I'd recommend testing first.


And if that's the behavior of x86, does that mean it's fine on PPC?

I'd be surprised if so, it's not the kind of thing that would depend on CPU type. But I've only tested it on OS X/x86.

[TheRyuu]

If you need compatibility, just point people to VLC. It's available on pretty much every platform possible, free as in freedom and cost, and plays pretty much every available media type through internal codecs.

My sarcasm detector must be broken today.

[vmrsss]

Hi. I think part of the misunderstanding here comes from talking cross-platform: recent versions of QT player on a Mac are not bad at all, although in my opinion not as good as libavcodec (and derivatives, eg mplayer and Plex) and VLC.


Dark Shikari, I read in this thread you have experience with the Final Cut suite. I am burning to ask you a question, though it is quite lateral to the present discussion. I'll try: do you happen know how to make DVD Studio accept an x264-generated file? (According to the Apple discussion-lists, it will only accept files from apple's own Compressor; only problem is that Compressor's H264 is seriously, seriously crap.)

[Dark Shikari]

Dark Shikari, I read in this thread you have experience with the Final Cut suite. I am burning to ask you a question, though it is quite lateral to the present discussion. I'll try: do you happen know how to make DVD Studio accept an x264-generated file?No idea. I'd start with encoding H.264 that uses the same settings as Compressor and going up from there until it stops working.