PAR, SAR, pasp, and QuickTime

[chaynik]

I am trying to encode non square-pixel video using x264 to be properly displayed on playback. QuickTime compatibility is a must for me (certain software these files are to be played back is based on QuickTime and is OS X only).

I am having trouble understanding the differences between PAR and SAR since they both seem to use the same ratios (10:11, 40:33, 8:9, 32:27, etc). I've had success with using the --sar switch in x264, the produced encodes are played back by QT at the correct aspect ratio, but with the vertical res upsampled (so 720x540 instead of 640x480) for 4:3 video. 16x9 video is displayed at 853x480. Using the -par switch in mp4box does not seem to have any effect on QT (video is displayed at 720x480) but is properly honored by VLC. I have also read in a different thread here that mp4box implemented 'pasp' atom support in their recent build (back in November 2007), but I do not know how to take advantage of it. (http://forum.doom9.org/showthread.php?t=125552)

Is SAR just a flag on the stream level while PAR on the container level? Is there any way to edit the SAR in an existing stream without reencoding? And ideally, which one should be used?

Lastly, I am having a hard time understand the differences between the ratios 10:11 & 8:9 for 4*3 video and 40:33 & 32:27 for 16*9 video. I've seen them used interchangeably in other threads, guides, etc but they are definately not the same. So, is the PAR of D1/DV NTSC video 10:11 or 8:9? I believe some of these numbers have something to do with overscan and nominal analog blanking (http://en.wikipedia.org/wiki/Nominal_analogue_blanking). That is, 720x480 converted to square pixels is somewhere close to 655x480, not a true 4:3 resolution. 8 pixels must be cropped off each side of the image to obtain 704x480 and then resize to 640x480. So, the 'pasp' entry would set the correct pixel ratio, while the 'clap' entry would crop off the extra padding on the side restoring the true 4:3 res, all done during playback. Am I wrong? Some references: http://www.mir.com/DMG/aspect.html, http://lurkertech.com/lg/pixelaspect/, http://lipas.uwasa.fi/~f76998/video/conversion/, http://standards.iso.org/ittf/PubliclyAvailableStandards/c043689_ISO_IEC_14496-12_2005_Amd_1_2007(E).zip

Thanks in advance!

[SeeMoreDigital]

Is SAR just a flag on the stream level while PAR on the container level? SAR and PAR perform the same tasks. They both set "aspect ratio signalling" flags at "Video Stream Level".

Is there any way to edit the SAR in an existing stream without reencoding? And ideally, which one should be used?Yes you can... YAMB (with MP4Box) can do this.

However, if you require compatibility with QuickTime player... You'll need to use MP4muxer. As this application can alter the resolution of .MP4/.MOV contained encodes at the "Container Level" (aka: PASP/Transformation Level).

Lastly, I am having a hard time understand the differences between the ratios 10:11 & 8:9 for 4*3 video and 40:33 & 32:27 for 16*9 video. I've seen them used interchangeably in other threads, guides, etc but they are definately not the same. So, is the PAR of D1/DV NTSC video 10:11 or 8:9? I believe some of these numbers have something to do with overscan and nominal analog blanking (http://en.wikipedia.org/wiki/Nominal_analogue_blanking). That is, 720x480 converted to square pixels is somewhere close to 655x480, not a true 4:3 resolution. 8 pixels must be cropped off each side of the image to obtain 704x480 and then resize to 640x480. So, the 'pasp' entry would set the correct pixel ratio, while the 'clap' entry would crop off the extra padding on the side restoring the true 4:3 res, all done during playback. Am I wrong? Some references: http://www.mir.com/DMG/aspect.html, http://lurkertech.com/lg/pixelaspect/, http://lipas.uwasa.fi/~f76998/video/conversion/, http://standards.iso.org/ittf/PubliclyAvailableStandards/c043689_ISO_IEC_14496-12_2005_Amd_1_2007(E).zipI'll leave this for somebody else to explain ;)

[Inventive Software]

Put it this way. All I know is this (from studying 2 R1 DVDs), and it's simple enough.

720x480 @ 4:3 Aspect Ratio square pixeled is 640x480 EXACTLY. A SAR of 8:9 is correct at 720x480.
720x480 @ 16:9 Aspect Ratio square pixeled is 853.3333(recurring)x480 EXACTLY. A SAR of 32:27 is correct to get this to display at 720x480.

Hope this helps.

[kemuri-_9]

it's best to differentiate them all using the proper terms
DAR - Display Aspect Ratio - the end ratio, common ones - 4:3 & 16:9

SAR - Sample Aspect Ratio
PAR - Pixel Aspect Ratio
these two are used relatively interchangeable, they specify the stretch of the sample or pixels applied horizontally on the video to generate a different DAR than it is originally.
the mathematical relationship between them is
DAR = video AR * [SAR,PAR]
i.e.
width 720 x 8:9 SAR (height 480)= 640x480 -> 4:3 DAR
width 704 x 10:11 SAR (height 480) = 640x480 -> 4:3 DAR

width 720 x 32:27 SAR (height 480) = 853+1/3 x480 -> 16:9 DAR
width 704 x 40:33 SAR (height 480) = 853+1/3 x480 -> 16:9 DAR

[Ritsuka]

pasp box: a box where the pixel aspect ratio is stored. QuickTime can read in from mp4 files but it does not apply it automatically, you can enable it manually by enabling the "clean aperture" from the property windows.

In the end, you only need to change the video track width in the mp4 container if you want the right size, and add a pasp box for future compatibility.

[CruNcher]

as SeeMoreDigital already said if you want Aspect Ratio interoperability with Quicktime you need to use Mp4muxer.

[poisondeathray]

Doesn't it depend on the software? Most display it like that anyway?

e.g. 720x480 frame size with 4:3 DAR (thus 8:9 SAR) gets displayed as 720x540.

This is true for MPC, QT player, VLC, Mplayer

You can superimpose the players to verify this is the case.

KMplayer is the only one that displays @ 640x480, and gives you the option to take a screenshot as what you see (640x480), or as original size (720x480)

Interestingly, VLC takes screenshots @ 640x480, but displays @ 720x540.

Or am I missing something here?

[SeeMoreDigital]

Doesn't it depend on the software? Most display it like that anyway?

e.g. 720x480 frame size with 4:3 DAR (thus 8:9 SAR) gets displayed as 720x540. Actually, no it does not.... It will be displayed at 640x480. The native vertical resolution is not altered!


Cheers

[poisondeathray]

Actually, no it does not.... It will be displayed at 640x480. The native vertical resolution is not altered!


Cheers

Are you sure? I just tested it on all 5 players, superimposed them...on each other. I used default display settings, so maybe something is different between our configurations?

Verified the frame aspect ratio, pixel aspect ratio, and display aspect ratio with avinaptic

[SeeMoreDigital]

Yep... I'm sure.

It sounds like your software player is "re-scaling" the image and not displaying it at its native resolution plus aspect ratio signalling.

[poisondeathray]

Yep... I'm sure.

It sounds like your software player is "re-scaling" the image and not displaying it at its native resolution plus aspect ratio signalling.

Then I guess this is the default behaviour of VLC, Mplayer, MPC, QT ? (not KMPlayer).

This occurs on all files for me, VOB, MPEG2, x264 in .mp4 or .mkv ....

May I ask what software player you use?

Test file 720x480 DAR 4:3 (<4MB)
http://www.mediafire.com/download.php?uqhojwrrhim

avinaptic
Resolution: 720 x 480
Frame aspect ratio: 3:2 = 1.5
Pixel aspect ratio: 8:9 = 0.888888
Display aspect ratio: 4:3 = 1.333333

http://i34.tinypic.com/20ucoxu.jpg


Am I doing something wrong? Have I incorrectly set the flags? Do I need to change some settings in the players?

[SeeMoreDigital]

Yes, there appears to be something wrong in with the way you've muxed your encodes. When I run your file thru' MP4Box it reports this: -

Track # 1 Info - TrackID 1 - TimeScale 10000000 - Duration 00:00:05.972
Media Info: Language "Undetermined" - Type "vide:avc1" - 179 samples
MPEG-4 Config: Visual Stream - ObjectTypeIndication 0x21
AVC/H264 Video - Visual Size 720 x 480 - Profile High @ Level 3
NAL Unit length bits: 32
Pixel Aspect Ratio 8:9 - Indicated track size 720 x 540
Self-synchronized


When it should be reporting this: -
Track # 1 Info - TrackID 1 - TimeScale 30000 - Duration 00:00:05.972
Media Info: Language "Undetermined" - Type "vide:avc1" - 179 samples
MPEG-4 Config: Visual Stream - ObjectTypeIndication 0x21
AVC/H264 Video - Visual Size 720 x 480 - Profile High @ Level 3
NAL Unit length bits: 32
Pixel Aspect Ratio 8:9 - Indicated track size 640 x 480
Self-synchronized

Notice that your "TimeScale" data is incorrect too!

[poisondeathray]

Thanks SeeMoreDigital, I'll look into it and post if I have any new findings

I used MeGUI and the native wrapper (I think it's mp4box as well), I'll re-do it and see if it makes a difference

If it's not too much to ask, can you provide a small sample that is done "correctly"?

[chaynik]

Thank you all for the replies, my results are very similar to those of poisondeathray. If I use the the --sar switch in x264, I get the following result:

Track # 1 Info - TrackID 1 - TimeScale 24000 - Duration 00:03:33.171
Media Info: Language "Undetermined" - Type "vide:avc1" - 5111 samples
MPEG-4 Config: Visual Stream - ObjectTypeIndication 0x21
AVC/H264 Video - Visual Size 720 x 480 - Profile Main @ Level 3
NAL Unit length bits: 32
Pixel Aspect Ratio 8:9 - Indicated track size 720 x 540
Self-synchronized

If I mux in mp4box with the '-par' switch, however, running info on it produced the following result:


Track # 1 Info - TrackID 1 - TimeScale 24000 - Duration 00:03:33.171
Media Info: Language "Undetermined" - Type "vide:avc1" - 5111 samples
MPEG-4 Config: Visual Stream - ObjectTypeIndication 0x21
AVC/H264 Video - Visual Size 720 x 480 - Profile Main @ Level 3
NAL Unit length bits: 32
Pixel Aspect Ratio 8:9 - Indicated track size 720 x 480
Self-synchronized


Notice how the PAR stayed the same (8:9) but the Indicated track size changed from 720x540 to 720x480. So my question is, what exactly controls this 'Indicated track size' value? It seems it is what QT uses to correctly display the video. Using the QuickTime friendly MP4Muxer many suggested did cause QT to display the video at whatever the W and H were set, however did not have any effect on the Indicated track size when --info is ran in mp4box. Furthermore, the software I use mentioned in the original post that relies on QT for rendering completely ignores the values set by Mp4muxer (but respects the values set by the x264 -sar switch.)

And lastly, Apple's own Compressor application (http://www.apple.com/finalcutstudio/compressor/) now has the option to do anamorphic encoding. The files it produced are properly displayed by QuickTime, VLC, MPC and pretty much every other player tried. Running --info in mp4box on these files produced the following results:

Track # 1 Info - TrackID 1 - TimeScale 2997 - Duration 00:03:33.254
Media Info: Language "English" - Type "vide:avc1" - 5113 samples
MPEG-4 Config: Visual Stream - ObjectTypeIndication 0x21
AVC/H264 Video - Visual Size 720 x 480 - Profile Main @ Level 3
NAL Unit length bits: 32
Pixel Aspect Ratio 10:11 - Indicated track size 640 x 480
Self-synchronized

What's mind boggling here is that the PAR is set at 10:11 yet the track size is 640x480! There is no way to modify the PAR value in Compressor, you just tell it whether you want 4x3 or 16x9 output.

After all of these tests, it seems like the PAR and "Indicated track size" are two different values and do not seem to have any kind of sane relationship (as seen above, 720x480 @ 10:11 par has indicated size of 640x480, which I do not believe to be true.)

pasp box: a box where the pixel aspect ratio is stored. QuickTime can read in from mp4 files but it does not apply it automatically, you can enable it manually by enabling the "clean aperture" from the property windows.

In the end, you only need to change the video track width in the mp4 container if you want the right size, and add a pasp box for future compatibility.
Thanks for that info, Ritsuka. But what tool and what command line can be used to do that? And, if you apply the 'pasp' box by selecting "clean aperture" in QT, what is the point of the 'clap' box?

[poisondeathray]

OK I redid my encode, using --SAR 8:9



Track # 1 Info - TrackID 1 - TimeScale 30000 - Duration 00:00:05.972
Media Info: Language "Undetermined" - Type "vide:avc1" - 179 samples
MPEG-4 Config: Visual Stream - ObjectTypeIndication 0x21
AVC/H264 Video - Visual Size 720 x 480 - Profile High @ Level 3
NAL Unit length bits: 32
Pixel Aspect Ratio 8:9 - Indicated track size 720 x 540



Then I redid it without the --SAR, no surprise



Track # 1 Info - TrackID 1 - TimeScale 30000 - Duration 00:00:05.972
Media Info: Language "Undetermined" - Type "vide:avc1" - 179 samples
MPEG-4 Config: Visual Stream - ObjectTypeIndication 0x21
AVC/H264 Video - Visual Size 720 x 480 - Profile High @ Level 4
NAL Unit length bits: 32
Pixel Aspect Ratio 1:1 - Indicated track size 720 x 480



But when I took the 1:1 PAR file and used YAMB specifying 8:9 pixel aspect ratio



Track # 1 Info - TrackID 1 - TimeScale 30000 - Duration 00:00:05.972
Media Info: Language "Undetermined" - Type "vide:avc1" - 179 samples
MPEG-4 Config: Visual Stream - ObjectTypeIndication 0x21
AVC/H264 Video - Visual Size 720 x 480 - Profile High @ Level 4
NAL Unit length bits: 32
Pixel Aspect Ratio 8:9 - Indicated track size 640 x 480



Now this last file plays 640x480 in MPC if overlay mixer is used, not VMR9 renderless. The 1st and 3rd play 640x480 in MPC if VMR9 mixer mode box is unchecked. VLC doesn't play any of them in 640x480. So it really seems to depend on the player's settings and the file's characteristics

QT seems to play the 3rd one correctly at 640x480. The frame size is correct, but I forgot to use a QT compatible profile. So I redid it 1) square pixels in MeGUI using QT for Windows settings 2) 8:9 pixel aspect ratio in YAMB remuxed into new .mp4

(1.44MB)
http://www.mediafire.com/download.php?owwrm61m1gr

And it works for some reason. Now maybe someone can explain why remuxing with YAMB works in this fashion?

Another question after searching & reading posts, some people argue that 720x540 is preferred over 640x480 because of no loss of information? Is there any validity behind this? Intuitively it makes sense, but the "upscaling" might give weird artifact as a "con".

(I'd like to say thanks too all who contributed here I've learned a lot today)

[kemuri-_9]

yes i've also noticed some players and muxers actually do the following interesting thing:
if( SAR > 1) dispaly_width = actual_width * SAR
else display_height = actual_height / SAR

meaning they stretch the video's height by the reverse of SAR to get the DAR rather than shrink the video's width.

it's all a matter of preference, i would imagine the argument for the height stretching case would be the following:
'Why did you encode it with a width bigger than it displays as? waste of bitrate/data!'

[chaynik]

QT seems to play the 3rd one correctly at 640x480. The frame size is correct, but I forgot to use a QT compatible profile. So I redid it 1) square pixels in MeGUI using QT for Windows settings 2) 8:9 pixel aspect ratio in YAMB remuxed into new .mp4
I can confirm that your file is displayed correctly in QuickTime on Mac OS X:

http://img222.imageshack.us/img222/1527/qtyambsx8.png

Also very exciting is QuickTime reporting the correct file size and allowing to "pass through" the video when exporting. Both of these were usually a problem to mp4box and spawned "QuickTime friendly" tools such as 'mp4muxer'.

Unfortunately, I am unable, to replicate your success! I've tried setting the pixel aspect ratio in YAMB to 8:9 and 10:11, QuickTime ignores both muxes and incorrectly displays them at 720x480. What version of YAMB and mp4box are you using? Maybe it's a version thing... (I hope)

Now this last file plays 640x480 in MPC if overlay mixer is used, not VMR9 renderless. The 1st and 3rd play 640x480 in MPC if VMR9 mixer mode box is unchecked. VLC doesn't play any of them in 640x480. So it really seems to depend on the player's settings and the file's characteristics
I've noticed too that VLC likes to display 4:3 video at 720x540, while QuickTime at 640x480. This behavior is true for 4:3 MPEG2 streams as well. There's nothing wrong with this behavior IMO, and only shows the power of PAR, i.e, you have a choice of displaying your 720x480 encoded video at either 720x540 or 640x480. Which leads us to your next question...

Another question after searching & reading posts, some people argue that 720x540 is preferred over 640x480 because of no loss of information? Is there any validity behind this? Intuitively it makes sense, but the "upscaling" might give weird artifact as a "con".
This is the classic argument whether to upscale the vertical res to take advantage of the higher horizonal res or downres the horizontal res to keep the vertical res the NTSC standard of 480. The best thing? If your input uses rectangular pixels, keep the pixels in your output rectangular as well and take advantage of the PAR flag in the mp4 spec like you are doing now! It seems your method is compatible with every major player including QuickTime, so I think what you're doing now is the absolute best practice.

[poisondeathray]

What version of YAMB and mp4box are you using? Maybe it's a version thing... (I hope)




The GUI is 2.0.0.8, mp4box 0.4.5-dev_20070914 . I actually changed to using the installer version right before, not sure if that had anything to do with it
http://yamb.unite-video.com/download.html




This is the classic argument whether to upscale the vertical res to take advantage of the higher horizonal res or downres the horizontal res to keep the vertical res the NTSC standard of 480. The best thing? If your input uses rectangular pixels, keep the pixels in your output rectangular as well and take advantage of the PAR flag in the mp4 spec like you are doing now! It seems your method is compatible with every major player including QuickTime, so I think what you're doing now is the absolute best practice.




1) I know there is no "Best" @ Doom9, but if you wanted 640x480, then why not resize it to 1:1 pixels @ 640x480 frame size?

I'm guessing you might lose a bit of quality at the expense of saving some bitrate/filesize? (Encoding @ 640x480 1:1pixels should be ~12.5%smaller than 720x480 DAR 4:3 if my math is correct)

The thing is that it's NOT "compatible" with every player - it depends on what settings are used, and it NEVER displays @640x480 in VLC despite playing with the settings, and only in MPC after changing the default settings ...you can't guarantee that all people will have their settings adjusted...So the only sure way to display at 640x480 is 1:1 pixels with resize

2) It didn't matter what settings I used in terms of --SAR X:Y in MeGUI which is using mp4box 2008.02.01->2008.09.14 (no version number) - the end result is identical after setting the flag in the YAMB.

If that is true, what exactly does --SAR do? (chaynik had a similar question in his OP) Is it at the container level? or video stream level? These observations would suggest it only affects the container?

It's slightly slower encode using --SAR compared to none specified. So why should you ever use --SAR in MeGUI? Why not encode and "fix" everything after? (it's actually faster) I don't know if it's a mp4box version difference either. Is this a variant of the aspect ratio bug in MeGUI that everyone talks about?

[kemuri-_9]

It's slightly slower encode using --SAR compared to none specified. So why should you ever use --SAR in MeGUI? Why not encode and "fix" everything after? (it's actually faster) I don't know if it's a mp4box version difference either. Is this a variant of the aspect ratio bug in MeGUI that everyone talks about?

what? the --sar parameter in x264 only affects the VUI parameters which are static for the stream, it has no effect otherwise,
so where are you getting this '--sar makes it slower' claim?

and the aspect ratio bug in megui is (iirc) that it doesn't calculate the sar properly when given a desired DAR and a video that'll need a SAR to have such a DAR.

[poisondeathray]

what? the --sar parameter in x264 only affects the VUI parameters which are static for the stream, it has no effect otherwise,
so where are you getting this '--sar makes it slower' claim?

and the aspect ratio bug in megui is (iirc) that it doesn't calculate the sar properly when given a desired DAR and a video that'll need a SAR to have such a DAR.

Thanks, but us normal folk might not understand what "VUI parameters" are? Can you explain in simple terms what the --sar is doing?

My "--sar makes it slower observation" is just that, an observation. It was observed on a few short test clips, so it's not a universal truth :) I'll retest it on some longer clips and let you know...

[kemuri-_9]

Thanks, but us normal folk might not understand what "VUI parameters" are? Can you explain in simple terms what the --sar is doing?.



Annex E
Video usability information
(This annex forms an integral part of this Recommendation | International Standard)
This Annex specifies syntax and semantics of the VUI parameters of the sequence parameter sets.
VUI parameters are not required for constructing the luma or chroma samples by the decoding process. Conforming
decoders are not required to process this information for output order conformance to this
Recommendation | International Standard (see Annex C for the specification of conformance). Some VUI parameters
are required to check bitstream conformance and for output timing decoder conformance.


things that are in the VUI are aspect ratio, overscan, video format (NTSC, PAL, etc.), Color Primaries (BT.609, BT.701, etc.)
and other information used for interpreting the stream from the encoded data.

[SeeMoreDigital]

If that is true, what exactly does --SAR do? (chaynik had a similar question in his OP) Is it at the container level? or video stream level? These observations would suggest it only affects the container?As I mentioned in my earlier post, SAR and PAR (as well as DAR) perform the same tasks. They set the "aspect ratio signalling" flags at "Video Stream Level".

PASP sets the "aspect ratio signalling" flags at "Container Level".

By-the-way (and this is important). If an .MP4 or .MOV file happens to contain both types of signalling, QuickTime player will "always" read and use the "Container Level" signalling. When most other software players will read and use the "Stream Level" signalling!


Cheers

[poisondeathray]

Thanks for the explanations guys! I appreciate it

[chaynik]

The GUI is 2.0.0.8, mp4box 0.4.5-dev_20070914 . I actually changed to using the installer version right before, not sure if that had anything to do with it
http://yamb.unite-video.com/download.html
Success!! I changed to the mp4box bundled with YAMB 2.0.0.8 (0.4.5-DEV - compiled by Kurtnoise - Sep 14 2007) and now the PAR settings are correctly honored by QuickTime!! It seems the newer version of mp4box I was using (0.4.5-DEV (build 19 - Apr 13 2008) broke QT support (possible bug - where to report?)

Now my 720x480 encodes with 10:11 PAR play back as 654x480 in QT while the ones with 8:9 PAR play back as 640x480. I assume the 654 number is due to overscan, so the next question is how to set the "clean aperture" flag? I tried using the "--overscan crop" switch in x264, but that seemed to have no effect on QT...

So for now, I am cropping 8 pixels off each side of my source files, bringing them down to 704x480, encoding them with x264 and setting the 10:11 PAR in YAMB/mp4box. Seems to be the best solution for not having overscan and avoiding resizing while having the player display the correct 4:3 DAR. Interestingly enough, my 720x480 encodes done with Apple Compressor report back as having a PAR of 10:11 and an "Indicated track size" of 640x480... I still don't understand how that could be possible...

Now onto the next question, does the MP4 container allow for "soft cropping" like the MKV one does?

Thanks to all for your help!!

[chaynik]

Almost forgot, as great as all of this is, my mp4box muxes are still not fully compatible with QT, I believe this is a well known issue (data size is reported as 0 and passthrough fails). So how do i combine the PAR compatibility of mp4box with the data size/pass thru compatibility of mp4muxer? In a perfect world, both would work... :)

And poisondeathray, the sample you provided DID in fact report the correct data size in QT and allowed for passthrough. I'm still unable to replicate your success 100%.

[poisondeathray]

chaynik - I'd like to help but I honestly don't even know what the "data size" and "passthrough" are. I rarely deal with QT, maybe the odd Apple Movie Trailer. Is it indicated when you right click properties in QT Player (there is a "data size" column there)?

EDIT: do you mean export movie to mpeg-4, and using "passthrough" in the video format dropdown menu in the options? If so, I can't get it to work properly either (QT Pro 7.5.5)

@kemuri-_9 - you were completely right about the --sar encoding time - thanks for pointing it out. I must have confused something like changed encoding parameters when doing all the testing.