Get h264 encode information from a video

[Kouta]

hey pll, i wanna know if there is a way to know what resources were used in a h264 video, because i'm dying to know how that guy did this, got a 40mb video with nice quality. The link is down here. if anyone knows how to do something like that, or know encode cfgs to let the video like this one, please tell me. Thx
video link : http://www.megaupload.com/?d=5PEZ9F2X

[buzzqw]

please edit the title . follow rules 9) Use a title that describes the content of your post. Don't use all caps or special characters to draw attention.

BHH

[nm]

From the file:
x264 - core 47 svn-525 - H.264/MPEG-4 AVC codec - Copyleft 2005 - http://www.videolan.org/x264.html -
options:
cabac=1 ref=8 deblock=1:1:1 analyse=0x3:0x133 me=hex subme=7 brdo=1
mixed_ref=1 me_range=16 chroma_me=1 trellis=2 8x8dct=1 cqm=0 chroma_qp_offset=0
slices=2 nr=0 decimate=1 bframes=16 b_pyramid=1 b_adapt=1 b_bias=0 direct=3 wpredb=1
bime=1 keyint=250 keyint_min=25 scenecut=40 rc=2pass bitrate=530 ratetol=10
rceq='blurCplx^(1-qComp)' qcomp=0.80 qpmin=10 qpmax=51 qpstep=4 cplxblur=20.0
qblur=0.5 ip_ratio=1.40 pb_ratio=1.30

[DarkZell666]

downloading right now, but how much should I bet that it hardly has any action ?

Just guessing : he used crf and resized to 512*384 or 576*432 (that's what I do to get naruto and bleach episodes to fit onto 60mb, and they are action-packed :p).

You are unlucky it's mkv, if it had been mp4 encoded with x264 you would have had the nearly-complete command-line available in the mp4 headers :)
Edit : nm how did you get that info out of the mkv ? oO
Edit2 : Bitrate = 530kbps ? I get nice quality with < 350kbps sorry xD

[Guest]

@Kouta

Remove the profanity from your thread title immediately. Hit Edit and Go Advanced on your first posting.

If this is not done promptly, you will be struck for rule 4. Thank you.

[nm]

Edit : nm how did you get that info out of the mkv ? oO
x264 puts the info to the H.264 stream, so it can be extracted no matter what container is used. I used vim to read it, but almost any decent text editor will do :)

[Kouta]

sorry ppl, for the titlte, i dindt knew the rules before, now its fixed =(. im still a newbie on h264 encoding, what is CRF? and if i understood it right, he used 16 bframes??? thx ppl

[Guest]

sorry ppl, for the titlte, i dindt knew the rules before, now its fixed =(. Thank you. Now please edit this post as well. Thank you.

http://forum.doom9.org/showthread.php?p=750132#post750132

[foxyshadis]

You are unlucky it's mkv, if it had been mp4 encoded with x264 you would have had the nearly-complete command-line available in the mp4 headers :)
Edit : nm how did you get that info out of the mkv ? oO
Edit2 : Bitrate = 530kbps ? I get nice quality with < 350kbps sorry xD
They're stored a header in the AVC bitstream itself, not in the container. Even vfw x264 has that information, I believe.

To the other dude: Yes, 16 b-frames, but adaptive. Usually there aren't more than 5-6 in a row except in extremely static scenes.

[Kouta]

hey, i couldn't understando just one thing: how a video can have bitrate 530 and be 40mb-sized? i keeped trying to understando, but i couldn't. Can someone tell me?

[nm]

Good question. The average bitrate is about 230 kbps for 24 minutes of video and that file size. Zones could explain this, but x264 should report that in the info too. Of course the bitrate information in the bitstream is only the parameter given to x264, not the actual average bitrate (unless I'm reading the code incorrectly). So the only odd thing is why the actual bitrate is so much lower.

[yanyani989]

maybe they used sth like DeDup?
by remuxing the extracted raw avc to mp4, i found that the total number of frame is 15448, assuming they used 23.976fps for the original DeDupped script, the actual length would be 00:10:44 @ 23.976fps, which gives a bitrate around 520k, still a bit off though

[check]

I've been trying similar things lately, although a bit smaller, I'm trying to get anime episodes (sans the outro, which I'm planning to include via Matroska ordered chapters) into 25mb.
For me, the breakdown of bits is 24kbits audio (via Nero, although I am considering increasing this up to 32 or so) and around 120kbits video. At this level, the container actually becomes an important choice, so you should use MP4 with srt subtitles.

The main things to remember when trying to get insanely low filesizes like here are to:
-have a blurry source. You want to insanely blur the chroma to get rid of as much change as possible (chroma offset helps as well, but not that much), and minimise sharp lines. If you compare that encode to a normal one you will see the entire show is blurry. Filters I'd recommend are hqdn3d(), aaa() (a function that includes antialiasing, resizing and unfilter functionality) and good ol' blur(). At these bitrates, anamorphic encoding is useless, as are normal resolutions (the video you linked is very static, it's not a 'normal' source)
-pick a good resolution. I'm using 480x272 atm for a medium-high motion source. The general rule of thumb is the more motion the lower the res, but if you go too low it will cause blocking when you go full screen. 320 horizontal is definately too low, 480 is probably getting near the bottom.
-use good x264 settings. In other words, the best :>. These are what I'm using atm:
C:\Program Files\megui\tools/x264\x264.exe --pass 3 --bitrate 700 --stats ".stats" --ref 16 --mixed-refs --no-fast-pskip --bframes 16 --b-pyramid --b-rdo --bime --weightb --direct temporal --filter 4,4 --subme 7 --trellis 2 --analyse all --8x8dct --chroma-qp-offset 3 --qcomp 0.8 --b-bias 15 --me esa --merange 32 --progress --no-psnr --output "" ""
You should play around with the deblocking settings, the chroma qp offset and B-frame mode in particular, I'm still experimenting with those values.

[foxyshadis]

VFR. Extreme use of Dedup. :p I only noticed when I turned quantizer OSD on and it got super jerky. It was probably 530 kbps @ 11 minutes after decimation and when inflated to 24 minutes it became 230 kbps. With so little motion and so much filtering it came out encoded quite nicely.

Too much loli though.

Edit: Natch, yanyani989 answered first. :p

Check, mkv with simpleblock would have less container overhead than mp4. Not much, but a bit. And why don't you just use direct auto?

[nm]

Ah, that explains it.

[check]

foxyshadis, I've only encoded one file so far to test, and temporal is coming out last. It was just my latest test used temporal, and I grabbed the CLI without checking that. Good spot though. Another thing I should add is that when encoding something this small, your encoding time can never be too long :D

[yanyani989]

do u mean 'dedup, extreme use of vfr' ;) ?

[Caroliano]

--me esa --merange 32This is overkill. --merange higher than 24 will most likely decrease the quality very slight, and --me esa is a dumb motion estimation method that is a waste of time for too little gain. So, with --me umh --merange 24 you can get an much faster encoding at the same quality (or maybe 0.001 PSNR less, aproximatedly). If you realy don't care about the time, then enable --esa, as it will not hurt the quality.

And 4:4 deblocking? Are you sure that this isn't too much? I use 1:1 with normal anime, even at very low bitrates. See this tread for an extreme low bitrate encoding with 1:1 deblocking: http://forum.doom9.org/showthread.php?t=111169

[Kouta]

what is dedup? dunno what's this plx sume1 explain me (yeah, im just a noob, but what can i do? just keep asking)

[nm]

You could use Google and/or the forum search.

See: http://students.washington.edu/lorenm/src/avisynth/dedup/

[Kurth]

http://www.avisynth.org/mediawiki/wiki/VFR
I always use VariableFrameRate for low bitrate anime, with less frames to encode you can distribute more kbps for the usefull frames and you will use less time to finish the encode.
I like to use those settings to encode anime.
http://img471.imageshack.us/img471/1359/main0pw.jpg
http://img471.imageshack.us/img471/8233/rcme3vq.jpg
http://img210.imageshack.us/img210/839/advanced1wt.jpg
Deblocking if you use more than 3,3 the video start to lose details you should use the max of 3,3.
M.E.range 32 is good you will not lose quality with 32, I dont saw any problems using 32. The PSNR always come a little bit better.
I always use subme 7 and Umh I dont care if this is slow it always give better quality.
Reference frames always give a little better quality but using 10 mixed is really slow, if you dont want to wait a lot of time you can use 5 mixed or just a simple 1 reference frame.
Something that I really like is the matrice made by Mp4Guy you can find the matrice file on the forums.
I think that 3-Pass Encoding dont really worth too much using because give a lot more time to encode and you dont really see any diference on the image soo I always use automated 2-Pass.
I use this slow encode configuration for anime episodes that will have less than 100MB file size.

[foxyshadis]

I think with extreme low rate it's important to check carefully what inloop is the best, I usually use -2,0 or it turns into an enormous mass of blurring.

Of course by extreme I mean 568x320 @ 25 kbps... 120 is enormous. xD This is average q ~45.

Otherwise seconding Carolino's, along with ref 16 being too high (10 is already squeezing blood out of a turnip, even that low). But if you really are testing all different combinations, then you've probably noticed how useless some options are by now, right?

Also when I wrote the VFR page I added a trick to enable you to use the same average bitrate before and after timecodes. It's in the mkv section. Otherwise you have to mess around with recalculating everything after filtering. VFR is not entirely necessary, you can just use Dup, as each skipped frame is only 4 bytes coded + 4 bytes container, but that can add up if you have enough frames and mess with motion estimation anyway.

[check]

25kbits? That's a bit high if you are looking to sqeeze into 5mb :P. I did mess around with those, but they look bad even on *anything*. I tended to cut right down to 320x240 or so and just watch in a window / on my analogue TV. Of course, when you have a total available bitrate of 33kbits, it's hard to get anything good out :D

The animes I'm currently messing around with are Planetes and Infinite Ryvius. Ryvius especially needs some hueg deblocking because I'm finding it hard to get a decent quant value. Planetes still has more motion than normal (even with the ending cut out and added in via ordered chapters) but the real problem is that neither of these animes have lines that blur well, they always smear, no matter what deblocking I use. I'm still messing around with deblocking, my setting is also personal preference, I sit further back than most when watching stuff on my monitor, so I have a higher tolerance for blurredness ;-) -2,0 would be far too low surely?? I'm messing around with 0,0 2,2 2,4 and 4,4.

This is overkill. --merange higher than 24 will most likely decrease the quality very slight, and --me esa is a dumb motion estimation method that is a waste of time for too little gain. So, with --me umh --merange 24 you can get an much faster encoding at the same quality (or maybe 0.001 PSNR less, aproximatedly). If you realy don't care about the time, then enable --esa, as it will not hurt the quality.

And 4:4 deblocking? Are you sure that this isn't too much? I use 1:1 with normal anime, even at very low bitrates. See this tread for an extreme low bitrate encoding with 1:1 deblocking: http://forum.doom9.org/showthread.php?t=111169
I'm well aware that exhaustive is a monumental waste of time, but I'm really not that fussed. A three pass no turbo encode only takes me 24hrs at these settings, which is nothing at all compared to a movie length encode.

Edit: I forgot to mention I did some testing on reference frame bitrate savings for crf anime encodes. The graphs and text I wrote up are lost in the mists of time, but from memory going from 1 ref frame to 16 mixed reduced the crf size from (relative) 1.00 to 0.92. I don't remember the exact figures, but the law of diminishing returns was definately in (strong) force. I got 8 ref frames as the max I would use (I think the reduction in bitrate for the 9th ref frame was <0.25). Once again, for a 30 minute episode of TV, i'm quite happy to havea 24hr encode, but if I was planning to encode a whole series all in a row (which sends you to QC hell), I would drop ref frames to 8 and use umh. Also I don't bother for 'normal' encodes unless I don't think I'll be doing anything on the computer for a few days :D

[imcold]

Why someone should use exhaustive search, is quite beyond my capacity (except for experiments with code ;) ) http://forum.doom9.org/showpost.php?p=660079&postcount=6
Slower doesn't automatically mean better (and no, so that negligible increase in psnr cannot be spotted).

[DarkZell666]

Way off topic dudes ;)

... but ESA gives worse PSNR ! Someone did some comparisons 2-3 months ago and we discovered that ESA is counter-productive :)

I mean, even in real life, when you do too much you usually ruin the hole thing right ? ;)

Still, I do wonder what such tiny bitrates look like O_O *will try that tonite xD*

[check]

Hehe, but you should know you can't trust PSNR with a 5c coin ;)
To get back to nearly the original topic, my latest attempt at a 25mb encode is here:http://www.project357.com/Misc/Videocrud/source_nofilters_480-272_0-0.mp4 , audio here: http://www.project357.com/Misc/Videocrud/source_nofilters_480-272-AUDIO.m4a
My settings used for it were:
AVS:
hqdn3d(3,4,7,9)
AAA(480,272,-30,-30,1,0,chroma=true)
blur(0.1)
undot()
dup(threshold=0.5,chroma=true)
x264:

"C:\Program Files\megui\tools/x264\x264.exe" --pass 3 --bitrate 120 --stats "source_nofilters_480-272.stats" --ref 16 --mixed-refs --no-fast-pskip --bframes 16 --b-pyramid --b-rdo --bime --weightb --direct auto --subme 7 --trellis 2 --analyse all --8x8dct --me umh --merange 28 --progress --no-psnr --output "source_nofilters_480-272_1-1.mp4" "source_nofilters_480-272.avs"
Look ma, no -esa! :D I'm encoding a few variants with different deblocking values, and although only 0,0 has been encoded so far, i'm either very impressed with it, or with my new aaa() function :)

[imcold]

Hehe, but you should know you can't trust PSNR with a 5c coin ;)


Need to measure invisible differences somehow :D

[futurex]

From the file:
x264 - core 47 svn-525 - H.264/MPEG-4 AVC codec - Copyleft 2005 - http://www.videolan.org/x264.html -
options:
cabac=1 ref=8 deblock=1:1:1 analyse=0x3:0x133 me=hex subme=7 brdo=1
mixed_ref=1 me_range=16 chroma_me=1 trellis=2 8x8dct=1 cqm=0 chroma_qp_offset=0
slices=2 nr=0 decimate=1 bframes=16 b_pyramid=1 b_adapt=1 b_bias=0 direct=3 wpredb=1
bime=1 keyint=250 keyint_min=25 scenecut=40 rc=2pass bitrate=530 ratetol=10
rceq='blurCplx^(1-qComp)' qcomp=0.80 qpmin=10 qpmax=51 qpstep=4 cplxblur=20.0
qblur=0.5 ip_ratio=1.40 pb_ratio=1.30


sorry if it was made clear in this thread, but i didn't catch it. how did you get that info? can i do the same with MP4 files? yamb give some info but not as detailed

[buzzqw]

open with a text editor (as pspad or ultraedit) your mp4 file, and look at start

BHH

[lazyn00b]

open with a text editor (as pspad or ultraedit) your mp4 file, and look at start

BHH

Does that work for ALL h.264 files, or only for ones encoded with x264?

[buzzqw]

x264 for sure, other (like quicktime...) don't know :o

BHH

[foxyshadis]

Only x264, vfw and cli, as well as mencoder and lavc. Anything generated with lib264 basically. Pretty sure nero doesn't store its config in the file.

(Text editors are pretty bad for loading videos in, while hex editors usually don't suck up all your memory preloading the file...)

[wata]

possible to know if the video is done in 1 or 2 pass?

[foxyshadis]

From prior example:
x264 - core 47 svn-525 - H.264/MPEG-4 AVC codec - Copyleft 2005 - http://www.videolan.org/x264.html -
options:
cabac=1 ref=8 deblock=1:1:1 analyse=0x3:0x133 me=hex subme=7 brdo=1
mixed_ref=1 me_range=16 chroma_me=1 trellis=2 8x8dct=1 cqm=0 chroma_qp_offset=0
slices=2 nr=0 decimate=1 bframes=16 b_pyramid=1 b_adapt=1 b_bias=0 direct=3 wpredb=1
bime=1 keyint=250 keyint_min=25 scenecut=40 rc=2pass bitrate=530 ratetol=10
rceq='blurCplx^(1-qComp)' qcomp=0.80 qpmin=10 qpmax=51 qpstep=4 cplxblur=20.0
qblur=0.5 ip_ratio=1.40 pb_ratio=1.30
The modes are 2pass, cbr, abr, crf, and cqp.

[bkman]

I wonder is it possible to discover things like average quantiser, etc?

[foxyshadis]

Not without parsing the file. Getting it out of the x264 post-encode stats is the only sane way, everything else requires demuxing, parsing text dumps, or writing your own custom parser. Ouch. Same thing for bitrate distribution graphs, the tools aren't there, unlike asp-in-avi. (Okay, Elecard Streameye sort of gives you the latter.)

[bond]

independant of what x264, and x264 only atm, writes in the userdata sei, i would use h264_parse over the raw avc stream for reading out the headers

[futurex]

i used YAMB to split a small segment and the encode information isn't there anymore. is there a way to keep it?

[bond]

i used YAMB to split a small segment and the encode information isn't there anymore. is there a way to keep it?no, its only stored once at the beginning of the stream

also strictly speaking you dont need this userdata info written only by x264, because the plain avc headers tell you everything you need to know about how the stream is set up (what avc options are used, cabac, loop or whatever)