x264 and high profile: 8x8 intra prediction and spatial transform is here

[bond]

in case that someone didnt realise it till now: thanks to akupenguin/pengvado x264 now offers, with revision 251, support for the avc high profile :)

it seems that with this step, x264 is the first publically available avc encoder that offers high profile encoding (except the reference of course)!

currently it does the following high profile features:
- 8x8 dct spatial transform, which you can enable in x264cli with --8x8dct
- 8x8 intra prediction, which you can enable with --analyse i8x8 (or --analyse all). it also needs 8x8dct enabled

enjoy :)

[berrinam]

Has libavcodec also been updated to handle HP streams now? I seem to remember that the HP encodes from the Ateme beta testing thread were unplayable with libavcodec.

[bond]

Has libavcodec also been updated to handle HP streams now? I seem to remember that the HP encodes from the Ateme beta testing thread were unplayable with libavcodec.
should be updated too, and i could play the hp samples bobololo posted with libavcodec

[Sirber]

When will it be avalible in VFW? Many thanks for the good work :D x264 became a pretty strong codec!

[Sagittaire]

this --8x8dct option don't work with x264cli

"unknow option" ... ???

[bond]

well make sure you get a build that is uptodate (so not sharktooths latest)

[Doom9]

does anybody how the corresponding mencoder options are called? I don't see the update in the CSV documentation yet, and as I'm currently synching the x264 options in mencoder with MeGUI I'd like to include the new options

[DeathTheSheep]

Holy CRAP. This is nearly inconcievable to me-- has this thing progressed so quickly as to do such a thing at so early a stage? When exactly did the x264 project begin again?

[Sagittaire]

yes ... work only with rev251 ... :D


------------------------------------------------------------------------
r251 | pengvado | 2005-06-05 13:07:28 +0200 (Sun, 05 Jun 2005) | 4 lines

SATD-based decision for 8x8 transform in inter-MBs.
Enable 8x8 intra.
CLI options: --8x8dct, --analyse i8x8.

[DeathTheSheep]

Isn't it true that in the vfw, you HAVE TO enable the 8x8 in order to enable the others (4x4, etc)?

[celtic_druid]

mencoder hasn't been updated as of yet.
http://www1.mplayerhq.hu/cgi-bin/cvsweb.cgi/main/libmpcodecs/ve_x264.c
Second it is you can see the options there.

[Yong]

ffdshow(latest build) crash when decoding x264 encoded with --8x8dct --analyse i8x8 enable.
moonlight h264 decoder cannot decode it(blank output)...
only the latest CVS mplayer is able to play it back ;)

[bond]

ffdshow(latest build) crash when decoding x264 encoded with --8x8dct --analyse i8x8 enable.of course, you have to use a build that uses latest libavcodec (no such build exists till now)

Isn't it true that in the vfw, you HAVE TO enable the 8x8 in order to enable the others (4x4, etc)?
vfw doesnt offer high profile

[IgorC]

Untill now it can be opened by the last Nero decoder.

[hpn]

Nice work! Now waiting for both HP enabled vfw and ffdshow to start some real testing. Hope the next x264 builds will gradually start narrowing (or completely obliterating) the quality gap between the Ateme's HP AVC samples posted recenly by Bobololo (this encoder is not yet publicly available however) and my recent x264 encodes at the same bitrate.

[easyfab]

Thanks for that
Newbie questions : - how much more quality can we expect with high profile ?
- how much more cpu will be need by this compare to main profile ?
- Will high profile be supported by the futur standalone player ?

[bond]

- how much more quality can we expect with high profile ?depends ;)

- how much more cpu will be need by this compare to main profile ?depends :D

- Will high profile be supported by the futur standalone player ? hd-dvd and bluray both include hp in their specs

[virus]

Looks like the new additions don't work very well yet (as of rev. 251).

Using --subme 5 --ref 3 (2nd pass) x264 uses too much i8x8 blocks, and that lowers PSNR by a few cents compared with a 4x4-only encode.
The 1st pass run with --subme 3 --ref 2 gave a much more normal number of intra 8x8 blocks (just for reference: I got 5% vs 21% i8x8 blocks in the 2 passes). Other people are experiencing the same problems.

Issue worth a look I suppose.

[bond]

i got the following values for intra for p-frames when encoding the classic matrix1 clip:

i4x4: 1.6%, i8x8: 44.8%, i16x16: 0.5%

i also used 3 b-frames (pyramid), mref 5, cabac, loop, subme 5 and weightb

[easyfab]

Ok I answer myself with a little test a made.

Clip : 1500 frames 1088*608

x264 "main" : x264 --progress --bitrate 800 --subme 5 --ref 2 --filter 0:0 --keyint 250 --min-keyint 25 --direct temporal --sar 1:1 --bframes 2 --weightb --b-pyramid --ipratio 1.40 --pbratio 1.30 --qcomp 0.60 --analyse b8x8,p8x8,p4x4,i4x4 -o ...

x264 "high" : x264 --progress --bitrate 800 --subme 5 --ref 2 --filter 0:0 --keyint 250 --min-keyint 25 --direct temporal --sar 1:1 --bframes 2 --weightb --b-pyramid --ipratio 1.40 --pbratio 1.30 --qcomp 0.60 --8x8dct --analyse all -o ( I hope all is correct)



For information the files size are ~the same (under 1% difference) and decoder is nero (ffdshow doesn't work with HP)

SSIM :

x264 "main" : 60.23

x264 "high" : 59.13


Same but with --bitrate 4000

x264 "main" : 83.52

x264 "high" : 83.67

It seems that new options not improve in all situations for the moment.
Let's wait futur releases for a better conclusion

Thanks to the x264 team

[CREXbzh]

does anybody how the corresponding mencoder options are called? I don't see the update in the CSV documentation yet, and as I'm currently synching the x264 options in mencoder with MeGUI I'd like to include the new options
Try the attached patch (option name "i8x8mv")

[Doom9]

And what about the 8x8 transform?
is that implied by
+ if(i8x8mv)
+ mod->param.analyse.inter |= X264_ANALYSE_I8x8;

[akupenguin]

Updated mencoder.
Fixed the i8x8 decision: it was comparing luma cost of i8x8 against luma+chroma cost of other modes.

[bob0r]

Holy CRAP. This is nearly inconcievable to me-- has this thing progressed so quickly as to do such a thing at so early a stage? When exactly did the x264 project begin again?

Hmm (changelog of x264):
------------------------------------------------------------------------
r1 | fenrir | 2004-06-03 21:29:33 +0200 (Thu, 03 Jun 2004) | 2 lines

* all: re-import of the CVS.

------------------------------------------------------------------------

So around begin 2004 i guess!
I liked x264 from the first day i used it.
The Command Line Interface really blows my mind... and now the start of high profile, slowly learning what it all means, means x264 has a great future ahead ( :devil: ), and i hope many people will keep supporting it!

[Sirber]

I'm adding CLI version to realanime...

http://www.detritus.qc.ca/files/realanime/screens/300pRC3.jpg

[berrinam]

Any comments on quality yet? Unfortunately, my computer is too slow to do much testing on.

[Sagittaire]

x264 HP is very good (for OPSNR): x264 is better than NDAVC MP with best setting ...

... but I have playback problem with x264 HP and Nero decoder: seem to be problem with playback frame order ???

... same problem with x264 MP and bref + <8 pref !!!

[TheUnforgiven]

I tested r253 with matrix reload trailer 640 pixel width 57.9 mb quicktime compressed
http://progressive.warnerbros.com/thematrix/us/med/trailer_final_640_dl.zip

here is the avs script i used:
----------------------------
directshowsource("a.mov")
crop(0,5,-0,-5)
converttoyv12()
killaudio()
----------------------------

i did two encodes with the following two commands:
encode1 without -8:
x264 -B 1000 -b3 -r5 --b-pyramid -w -f0:0 --ipratio 1.1 --pbratio 1.4 --qcomp 0.75 -A all --me umh --progress -o without.mp4 src.avs

encode2 with -8:
x264 -B 1000 -b3 -r5 --b-pyramid -w -f0:0 --ipratio 1.1 --pbratio 1.4 --qcomp 0.75 -A all --me umh -8 --progress -o with.mp4 src.avs

results:
encode1 gave: Avg:46.92 Global:42.81 kb/s:1040
encode2 gave: Avg:47.12 Global:43.01 kb/s:1041

-this is very good improvement indeed thanx akupenguin.

-i could not playback the the encode with -8 in ffdshow-20050527

Two more things:
-this line in the output
encoded 3626 frames, 2.79 fps, -1583668.83 kb/s
could somebody fix the bitrate display.
-although i set -pbratio to 1.4 i get better average QP for B slices that for P slices? as u notice below



here is the complete test output:
##########################################
C:\>x264 -B 1000 -b3 -r5 --b-pyramid -w -f0:0 --ipratio 1.1 --pbratio 1.4 --qcom
p 0.75 -A all --me umh --progress -o without.mp4 C:\src.avs
avis [info]: 640x336 @ 24.00 fps (3626 frames)
x264 [info]: using cpu capabilities MMX MMXEXT SSE SSE2
mp4 [info]: initial delay 833334 (scale 10000000)
x264 [info]: slice I:239 Avg QP:21.98 Avg size: 7823 PSNR Mean Y:58.71 U:61.66
V:62.88 Avg:59.10 Global:45.31
x264 [info]: slice P:2158 Avg QP:24.22 Avg size: 6925 PSNR Mean Y:43.94 U:46.76
V:47.78 Avg:44.72 Global:42.60
x264 [info]: slice B:1229 Avg QP:24.03 Avg size: 2306 PSNR Mean Y:47.72 U:50.51
V:51.46 Avg:48.42 Global:42.82
x264 [info]: slice I Avg I4x4:32.2% I8x8:0.0% I16x16:67.8%
x264 [info]: slice P Avg I4x4:12.9% I8x8:0.0% I16x16:16.4% P:36.4% P8x8:9.
3% PSKIP:25.0%
x264 [info]: slice B Avg I4x4:1.6% I8x8:0.0% I16x16:1.2% P:18.4% B:6.1% B
8x8:4.2% DIRECT:2.9% BSKIP:65.6%
x264 [info]: PSNR Mean Y:46.19 U:49.01 V:50.03 Avg:46.92 Global:42.81 kb/s:1040.
3

encoded 3626 frames, 2.71 fps, -1581660.43 kb/s

C:\>x264 -B 1000 -b3 -r5 --b-pyramid -w -f0:0 --ipratio 1.1 --pbratio 1.4 --qcom
p 0.75 -A all --me umh -8 --progress -o with.mp4 C:\src.avs
avis [info]: 640x336 @ 24.00 fps (3626 frames)
x264 [info]: using cpu capabilities MMX MMXEXT SSE SSE2
mp4 [info]: initial delay 833334 (scale 10000000)
x264 [info]: slice I:248 Avg QP:21.90 Avg size: 7691 PSNR Mean Y:58.60 U:61.41
V:62.56 Avg:58.98 Global:45.56
x264 [info]: slice P:2149 Avg QP:24.02 Avg size: 6932 PSNR Mean Y:44.17 U:46.85
V:47.90 Avg:44.93 Global:42.80
x264 [info]: slice B:1229 Avg QP:23.90 Avg size: 2334 PSNR Mean Y:47.87 U:50.56
V:51.56 Avg:48.56 Global:42.99
x264 [info]: slice I Avg I4x4:19.0% I8x8:30.4% I16x16:50.6%
x264 [info]: slice P Avg I4x4:7.9% I8x8:14.5% I16x16:10.6% P:32.9% P8x8:9.
3% PSKIP:24.8%
x264 [info]: slice B Avg I4x4:1.1% I8x8:1.6% I16x16:0.9% P:17.6% B:6.2% B
8x8:4.1% DIRECT:2.9% BSKIP:65.5%
x264 [info]: PSNR Mean Y:46.41 U:49.10 V:50.14 Avg:47.12 Global:43.01 kb/s:1041.
7

encoded 3626 frames, 2.79 fps, -1583668.83 kb/s
############################################

[akupenguin]

i could not playback the the encode with -8 in ffdshow-20050527Requires >= 2005-06-02.

encoded 3626 frames, 2.79 fps, -1583668.83 kb/s Can't reproduce.

although i set -pbratio to 1.4 i get better average QP for B slices that for P slices? as u notice below Not an error. See, the adaptive B-frame placement algo puts more B-frames in low-motion scenes than in high-motion. And the ratecontol algo gives lower quants to low-motion scenes. So the average B-quant may be lower than the average P even though each B-frame has higher quant than the neighboring P.

[TheUnforgiven]

thanx for the response

i can't find any recent build of ffdshow. but i guess sooner or later a build will be available at the usual places.

about the wrong display of bitrate, have anybody encountered this problem too?

one can try the same source and command to reproduce.

[TheUnforgiven]

Not an error. See, the adaptive B-frame placement algo puts more B-frames in low-motion scenes than in high-motion. And the ratecontol algo gives lower quants to low-motion scenes. So the average B-quant may be lower than the average P even though each B-frame has higher quant than the neighboring P.

thanks for the explanation. but in xvid we notice good difference in average quants between P and B. should i increase the pbratio to get similar results or i am comparing different things?

[berrinam]

Wow! So x264 is now already better than NDAVC MP just with the addition of 8x8 transform*. How is it that the codec becomes so much more powerful because of this? Sorry if this is a stupid question, but it only seems to be enabling one more mode.

*I know that OPSNR isn't a good measure of actual visual quality, but it's the best comparison as of yet.

[bond]

i can't find any recent build of ffdshow. but i guess sooner or later a build will be available at the usual places.publically there is not ffdshow compile available able to handle HP

about the wrong display of bitrate, have anybody encountered this problem too?nope

... but I have playback problem with x264 HP and Nero decoder: seem to be problem with playback frame order ???

... same problem with x264 MP and bref + <8 pref !!!can you be more specific what error you get?

note that haali's parser + nero decoder dont work together for HP

also when you have playback problems after doing raw avc -> .mp4, make sure you have uptodate compiles of mp4creator and mp4box (yeah, the ones on rarewares are outdated)

[Sharktooth]

publically there is not ffdshow compile available able to handle HP
celtic druid will publish a new build very soon.

[bond]

Fixed the i8x8 decision: it was comparing luma cost of i8x8 against luma+chroma cost of other modes.
ok now i got the following values for intra on the matrix clip:

i-vops: i4x4: 48.5%, i8x8: 36.3%, i16x16: 15.2%
p-vops: i4x4: 4.6%, i8x8: 12.7%, i16x16: 8.2%
b-vops: i4x4: 0.4%, i8x8: 1.9%, i16x16: 1.4%

so i8x8 gets indeed often used, but also not that much as before

[akupenguin]

thanks for the explanation. but in xvid we notice good difference in average quants between P and B. should i increase the pbratio to get similar results or i am comparing different things?
XviD defaults are (approximately) pbratio=2. I think that's way too high, but you can try it if you want.

[Revgen]

celtic druid will publish a new build very soon.
He just did. His download page (http://www.aziendeassociate.it/cd.asp?dir=/ffdshow) lists a new build dated 06-06-2005.

[bob0r]

Yup

New ffdshow-20050607.exe works great, i have mirrored it too.

This --8x8dct is a great addition, my test video now really looks awesome FULL screen on my 21" monitor :)

[dinolib2]

What kind of mp4 splitter ar you using? I've trouble with haali: I can open the movie, but it seems have no frames (only with HP options)

thank you

[bond]

note that haali's parser + nero decoder dont work together for HP
haali + latest ffdshow doesnt work either with HP

[dinolib2]

thanks.
so, what's a good splitter? (nero, I think...)

[bond]

thanks.
so, what's a good splitter? (nero, I think...)
indeed, maybe moonlight's filter handle it too

apart from HP (which surely will get fixed) haalis splitter is definitely one of the best

[akupenguin]

haali + latest ffdshow doesnt work either with HP
I wonder why. MPlayer's MP4 demuxer didn't need any changes, and it has no special provisions for HP. Only the codec and the muxer need to know.

[DeathTheSheep]

By very definition, "High Profile" is meant for large-size video, right? This 8x8 transform likely wouldn't do very much for 320*240 stuff, or at least not as much as it would for 2048*2048 HD stuff, right?

Just makin' sure so I won't screw myself in the long run.

Cheers, baa

[bond]

I wonder why. MPlayer's MP4 demuxer didn't need any changes, and it has no special provisions for HP. Only the codec and the muxer need to know.
well i dont know, i can only guess that nero's filter do some strange things (wouldnt be the first time), passing some extra info or so to the decoder...
this would explain why haali and nero's decoder doesnt work together for HP

than it might be that milan (ffdshow dev) tried to make his filter working with nero's parser, not realising that it borked functioning with haali
this would explain why nero's parser and ffdshow works

lot of ifs and whens...
maybe there is just a bug in haali :D

facts are that nero's parser + ffdshow and nero's parser + nero's decoder work with HP

[Revgen]

It wasn't too long ago that people were spelling doom and gloom for x264 after some of the programmers were hired by Nero.

It doesn't looks that way now. ;)

Keep up the good work. :thanks:

[DeathTheSheep]

It looks like they're still hired but still workin' on it. Yeah, it's been like, what? 60 revisions ago when we all "spelled out gloom." Which means, since not too long ago, there were 60 revisions worth of work in addition to the codec's working all the way up to HP level. Wow.

Hm... I wonder what XviD is up to these days... ;-)
Cheers, baa

[bond]

I wonder why. MPlayer's MP4 demuxer didn't need any changes, and it has no special provisions for HP. Only the codec and the muxer need to know.
ok bob0r found the reason for this: its a bug in haali's parser

haalis version from 14th april works fine (with both ffdshow and nero's decoder), version from 17th april doesnt anymore

the change back than (which later should have been reversed already for .mp4) was
"[General] Patch avc sps to always use AR info from container, not from bitstream (fixes ffdshow ignoring container AR). "

which seemed to have f*ed things up

[DeathTheSheep]

Well, I tried it out. 320*240, anime source, 32 quant, 24fps, 1:30. I got 2.09MB with I8x8 enabled and 1.96MB without it. Hm... fishy?

Other specs: No b-frames, every other analyzer flag enabled. Slower with I8x8 (a little).

Oh, and as for the MP4 splitter, I don't have one, it seems, but everything works fine. GraphEdit reports that the MP4 goes directly into the ffdshow decoder... how does this work? Or could I have installed one and just not known (and it hid itself from view)?

[bob0r]

I put version MatroskaSplitter.exe 14/04/2005 on my website, so people can test their x264 8x8dct (HP) encoded files.

I hope the next version of MatroskaSplitter.exe will have this "bug" fixed... enjoy!

[DeathTheSheep]

Cool. ;-)

But what's wrong with the vfw (today's new ffdshow)? Wouldn't that give you the same thing but in AVI?

[bob0r]

Cool. ;-)

But what's wrong with the vfw (today's new ffdshow)? Wouldn't that give you the same thing but in AVI?

VFW does not support this option yet (8x8dct). We have to be patient for that.
If you want it quick, donate Sharktooth a huge amount of money :p

Sharktooth did himself patch threads for VFW, which is not in the official x264 revisions. I use the original x264 revisions source.

[Mutant_Fruit]

I'm going to put up a mirror for the latest FFDShow on my FTP. Its only got 2meg upload, and it'll only be up for at most a few days, so get it while its hot. The main site seems to be going very slowly.

Its up now:
Download Link (ftp://)

EDIT: Link taken down, there are 6 high speed mirrors on: http://x264.nl

[DeathTheSheep]

Oh. I see. Ah, well, patience is a virtue, I hear ;-).

Anyone know if this is new or not? That new i8x8 in the picture?
http://img145.echo.cx/img145/8185/ffdshow7cj.jpg

[celtic_druid]

Actually I was the one who added threads to the VfW. It was just quick and dirty though. No where to put it so I extended the main GUI to make room and shifted the buttons at the bottom down, I might add some more options tonight. No one seems to like my idea to resize the GUI to 1000x500, put all of the options on it (no advanced or other tabs) and make it look more like K.I.T.T's interior though.

I'm sure I could find some room for AR and 8x8dct in the advanced tab.

[Kostarum Rex Persia]

Fantastic job Celtic_druid,keep on working.I like your idea,it's a very smart.

[DeathTheSheep]

Wow, that IS a darned good idea. But else would you expect of c_d? ;-)

[bob0r]

@Mutant_Fruit
Thanks, but we have enough mirrors for it.
http://x264.nl even has 6 already! :)

@celtic_druid
Ah yes sorry, sharktooth just added it.
... but the K.I.T.T's interior idea is SUPER COOL.
x264 sponsored by Knight Rider (ofcourse 1982 knight rider)

I think you should just create the VFWgui options the way you think is best.
At least don't let the size be its limitation.
But you should go on holiday first, worry about creating the perfect vfwgui once akupenguin/pengvado is going for this summer time off.
Then we should have a version that can be used for several weeks/months, that can also be used for some up to date tests between different encoders.

[bond]

Actually I was the one who added threads to the VfW. It was just quick and dirty though. No where to put it so I extended the main GUI to make room and shifted the buttons at the bottom down, I might add some more options tonight. No one seems to like my idea to resize the GUI to 1000x500, put all of the options on it (no advanced or other tabs) and make it look more like K.I.T.T's interior though.

I'm sure I could find some room for AR and 8x8dct in the advanced tab.
actually celtic_druid, when you are patching the vfw gui, can you also patch the x264gui (http://forum.doom9.org/showthread.php?t=95415)
its the vfw gui code ported to a standalone gui app useable with the cli and should therefore be pretty much the same code

[Mutant_Fruit]

@Mutant_Fruit
Thanks, but we have enough mirrors for it.
http://x264.nl even has 6 already! :)
Rightio, didn't even know that x264.nl mirrored it. Thought there was just the one site, and it took me a good 15 attempts to get the file downloaded :p I'll take my mirror down then.

*goes off to edit previous post*.

[Mc Onyx]

I dont know what i would do without celtic_druid's builds :), @celtic_druid about X264VFW, i too don't think, that size of the window is important, as long as i have all the options. Keep up the good work!

[bob0r]

Let me quote myself

I put version MatroskaSplitter.exe 14/04/2005 on my website, so people can test their x264 8x8dct (HP) encoded files.

I hope the next version of MatroskaSplitter.exe will have this "bug" fixed... enjoy!

:stupid:

MatroskaSplitter.exe 07/06/2005 added


Changes
07/06/2005 [General] Fixed avc sps parser to understand high profile.


Good job Haali!

[hpn]

MatroskaSplitter.exe 07/06/2005 added
Good job Haali!

Thanks, Haali and bob0r! I'm now able to play flawlessly my CLI encoded HP stuff with the good old MPC instead of using mplayer or Nero Showtime :)
let's repost the links again:
http://mirror02.x264.nl/x264/MatroskaSplitter.exe
or
http://mirror01.x264.nl/x264/MatroskaSplitter.exe
or more mirrors at
http://x264.nl/
and of course at Haali's home page
http://haali.cs.msu.ru/mkv/

[Kostarum Rex Persia]

I can't wait celtic_druid new x264 build,I think that new build will be fantastic.
celtic_druid,did you finished working on the new build,and can you tell us more detail.