x264 beta when?

[COREiP]

how close are we from a beta version of x264?

[bob0r]

http://x264.nl/jab.gif

[Sirber]

we are close to alpha. We are near of the end of "experimental fiasco" stage :D

[Adub]

Man, I could have sworn we were done! Ha, this codec is so stable I could completely live without further advancement happily! Just kidding. SSPPEEEEDDDD!!!!!

[Dark Eiri]

x264 still in alpha? It's so stable and so great-quality o.o
Just needs a LOT of speed improvement.

[nfm]

Maybe someboody would implement better cpu's instruction support. To convert simple mpeg to mp4 using x264's command line encoder takes me like 30min. x264 is very slow :(

@echo off

title H.264 Encoding

rem --interlaced; interlace switch

x264.exe --pass 1:2 --bitrate 1000 --stats ".stats" --ref 16 --mixed-refs --no-fast-pskip --bframes 3 --b-pyramid --b-rdo --bime --weightb --direct auto --filter -2,-1 --subme 7 --trellis 2 --analyse all --8x8dct --me umh --threads 2 --thread-input --progress --no-psnr --output output.mp4 input.avs

pause exit

Same mpeg to divx using virualdubmod and divx pro 6.2.37 takes a mere 4min with everything maxed out. Why isn't gcc 4.1.1 used? Any word on 64-bit version?, or I'll learn asm and C and will rewrite code by myself.

[Sagittaire]

x264 is not slow ... in fact x264 is faster than XviD/DivX for example and by far.

It's not an obligation to use the slowest setting. If you use the best setting all the good H264 implementation (Ateme, x264, Elecard ...) will be always very slow. If you want high speed or good quality/speed don't use umh ME, RDO, Trellis, brdo, 16 ref, no-fast-skip ... lol

With expirimental version of DivX (Helium), we can use very slow setting and make encoding at 0.01 fps. DivX Network desactived these really slow setting in final version. The x264's developper could do the same thing with umh and esa, 5 ref limitation, always fast skip, fast trellis, fast RDO ... etc etc

[GodofaGap]

nfm: if you want x264 to be faster don't use such ridiculous settings. :p

[Manao]

nfm : Sagittaire is completely right, but that doesn't prevent you from learning C & asm and participate to the project. And since with the next revision, the slowest settings will most certainly be 20x times as slow as the current slowest, you definitively have no excuses not to code for x264.

[foxyshadis]

Win64 compiles exist as well, of a very outdated revision. It has full x64 asm optimizations. But it requires 64-bit avisynth (or else hits a major speed penalty); squid_80 largely has that under control, but most external plugins won't be available. He's sort of the one-man band when it comes to 64-bit development, since very few devs have 64-bit windows. (64-bit *nix is another story.) If you're willing to update it, I'm sure it'd be appreciated. (Especially by KRP.)

[popper]

Win64 compiles exist as well, of a very outdated revision. It has full x64 asm optimizations. But it requires 64-bit avisynth (or else hits a major speed penalty); squid_80 largely has that under control, but most external plugins won't be available. He's sort of the one-man band when it comes to 64-bit development, since very few devs have 64-bit windows. (64-bit *nix is another story.) If you're willing to update it, I'm sure it'd be appreciated. (Especially by KRP.)

it sounds like several AVC and other projects might benifit from setting up a bounty type option like the AROS OS project to encurage and reward coders to optimise and add options.

see:http://thenostromo.com/teamaros2/index.php

[Kostarum Rex Persia]

He's sort of the one-man band when it comes to 64-bit development, since very few devs have 64-bit windows. (64-bit *nix is another story.) If you're willing to update it, I'm sure it'd be appreciated. (Especially by KRP.)

That's right. I can't wait for proper 64-bit build, with all necessary optimizations(like SSE,SSE2,SSE3).

[Sirber]

That's right. I can't wait for proper 64-bit build, with all necessary optimizations(like SSE,SSE2,SSE3).no use if the rest of the filter chain is 32bit, like avisynth, ffdshow, etc.

[COREiP]

Is x264 optimized for sse4? The new core 2 duo processors have sse4. :) more speeeeed!!!

[akupenguin]

no use if the rest of the filter chain is 32bit, like avisynth, ffdshow, etc.
Sure it is: you can pipe video from a 32bit avs to a 64bit x264. There's a little overhead involved in the pipe, but it's negligible compared to making the encoder 10% faster.

Is x264 optimized for sse4?
x264 will use sse4 when I have a cpu with sse4, and not before.

[soresu]

This may sound like a fairly stupid question, but does anyone here know of some good example docs on converting MMX/unoptimised code to SSE?

Perhaps also a good idea of areas of x264 that would benefit from the SSE optimisations.

Both Intel Core 2 and AMD K8L CPU architectures seem geared to running the SSE instruction sets faster, so I would like to start coding on this as soon as I learn the layout of x264 + C.

[foxyshadis]

Is x264 optimized for sse4? The new core 2 duo processors have sse4. :) more speeeeed!!!
It doesn't even have SSE2/SSE3 optimizations, unless something's changed in the last couple of months, partly because they're pointless on current AMD chips. (Which internally translates them back into SSE.)

Soresu, both intel and amd publish optimization guides, but I'm in no position to say if they're any good or not. ^^;

[Manao]

It does have SSE2/3 optimizations. And they are usefull for P4 / PM ( and even more for duo cores ). As for SSE4, I failed to find an official documentation stating what instruction does what ( though what they do can be guessed from their name ). That was a week ago, perhaps intel updated its website meanwhile.

soresu : most obvious & not so obvious MMX/SSE optimizations have already been made for x264.

@all : x264 alleged slow speed isn't due to a lack of optimization. It's due to users using insanely slow & useless settings a codec intrinsicly slower than ASP because it gives more work to the encoder ( more choices, deblocking, cabac ).There are still some optimizations possible, but nothing that will double the speed, except - perhaps - in the multithread area. For example, SSE4 would perhaps speed up the SSD ( ~5% ) / fDCT (~20% ) / iDCT(~20%) ( thanks to horizontal sums ) which would give an overall speed up of perhaps ~5% ( that's completely guessed numbers ), depending on the quality settings.

[Sirber]

so our best option is dual / quad core?

[nfm]

This is what my cmd prints
x264 [info]: using cpu capabilities MMX MMXEXT SSE SSE2 3DNow!

I don't see sse3. So I lowered to ref 5, subme 6, trellis 1, disabled me uhm, brdo and no fastskip. I cannot find much documentation about those, maybe somebody has a link to a guide or any kind of info (I got deaththesheepavcguide.pdf, so maybe I'll review that ;) ) . So I'm getting now 9.75fps and before I got ~5fps on HQ Insane, will above settings lower my quality?

no use if the rest of the filter chain is 32bit, like avisynth, ffdshow, etc.
and a 64-bit direcshow player.

Edit: nevermind, I'll try using higher values and stay close to ~10fps.

[Manao]

After checking, it doesn't seem to have any SSE3 functions. It would be easy however to convert SSE2 functions to SSE3 ( since it's only a matter of replacing movdqu by lddqu ). Of course, the speed gain to expect is alas proportionnal to the complexity of the conversion, which means that it's rather small.

nfm : --subme 1 --me dia --ref 1. There will be a definite speed up. I consider your settings still insane ( especially ref 5 ). Put back no fastpskip and brdo instead, keep subme 5, use either me hex or umh. Remove trellis. That should be a bit more balanced.

[akupenguin]

There are still some optimizations possible, but nothing that will double the speed, except - perhaps - in the multithread area. For example, SSE4 would perhaps speed up the SSD ( ~5% ) / fDCT (~20% ) / iDCT(~20%) ( thanks to horizontal sums ) which would give an overall speed up of perhaps ~5% ( that's completely guessed numbers ), depending on the quality settings.
DCT doesn't use horizontal sums. SSD, SATD, and SSIM;) do.

And unlike sse3, it looks like there's more than one instruction in sse4.
PSHUFB: yes. Too bad it still only allows an immediate for the mapping, though. Altivec's shuffle is better.
PHADD*: yes.
PALIGNR: yes.
PABS*: yes.
PMULHRSW, PMADDUBSW: dunno if there's a use for these in x264.
PSIGN*: no point. Can't these be trivially implemented as PSAR*?
PHSUB*: no.

Put back no fastpskip and brdo instead, keep subme 5, use either me hex or umh.
brdo requires subme 6.

[nfm]

You guys are awesome ;), this where I've settled:

@echo off

title H.264 Encoding

rem --interlaced; interlace switch

x264.exe --pass 1:2 --bitrate 1000 --stats ".stats" --ref 5 --mixed-refs --no-fast-pskip --bframes 3 --b-pyramid --b-rdo --bime --weightb --direct auto --filter -2,-1 --subme 6 --trellis 2 --analyse all --8x8dct --threads 2 --thread-input --progress --no-psnr --output output.mp4 input.avs

pause exit

no much changed from hq insane quality :D ref went to 5 and subme to 6 and me hex. That's it, now I get ~15fps, which is 300% better than a last time. I don't need any more help. Let's discuss about x264 :)

@akupenguin, so sse3 can be implemented? (I don't know who is x264 developer here)

[Sirber]

@nfm

what is your CPU? Smells like ~8FPS on my 3000+.

[nfm]

I'm an overclocker :D but I a have a very weak cpu, at time I was testing that it was on 2.50Ghz AMD X2 3800+ and ram memory at ddr500 with 1.5-2-2-0 timigs as well as with trfc 1 and other , it's comlicated, but they are very very tight. I can run that cpu at 2.7Ghz with default voltage but since it's very hot now, over the summer I run at 2.50Ghz

Overclocking is a lottery, I was hoping for a very good stepping that can reach 3.00Ghz+ but I wasn't that lucky.

[Manao]

PSIGN : I thought it would return -1, 0 and 1, while PSAR gives -1 or 0.

I hoped PHSUB / PHADD could be used to avoid the transposition step in the DCT.

Finally, I have searched info on PALIGNR but couldn't find any. Do you have any link / insight ?

[popper]

This may sound like a fairly stupid question, but does anyone here know of some good example docs on converting MMX/unoptimised code to SSE?

Perhaps also a good idea of areas of x264 that would benefit from the SSE optimisations.

Both Intel Core 2 and AMD K8L CPU architectures seem geared to running the SSE instruction sets faster, so I would like to start coding on this as soon as I learn the layout of x264 + C.

i thought id go looking for whats available, and it turns out it seems its relativly easy to find working tested current example code and librarys for the Altivec and even talk to the vectorising experts over there on that PPC CPU.

it seems they are having site problems or moved over to password entry now
http://www.simdtech.org/home

when it comes to Vectorising for the x86 however the story seems very different and lacking in so many ways.

i couldnt find any current tryed and tested code or libs with comparisons and explanations of how and why a peace of code is better for a given operation, and it seems everyone relys on a/the x86 compiler to produce faster code or blame the same for lack of improvements unlike the Altivec people that get down to just do it.....

while not about vectorising(infact after checking again there is a section on just that:) ), this seems like a place to at least start to find the right people that take the time to test assumptions and code to improve things.

http://www.agner.org/optimize/optimizing_cpp.pdf
http://www.agner.org/optimize/

[akupenguin]

I hoped PHSUB / PHADD could be used to avoid the transposition step in the DCT.
col+transpose+col is quite a lot faster than row+col. I don't think PHADD is enough to change that.
(Note: this applies to mpeg* DCT also, except that the common implementations (i.e. xvid, divx, and lavc) use row+col, and I can't switch to my faster version without changing rounding. I suppose I could enable it for mpeg2 playback, when we have no idea which DCT the encoder used anyway.)

As for why col+tanspose+col is faster:
Generic matrix multiplication is O(N^3) scalar ops (= O(N^2) vector ops).
DCT is not a generic matrix. It can be factored into O(N^2*log(N)) scalar ops. But that factoring conflicts with vectorization of a row DCT, so only the column DCT becomes O(N*log(N)) vector ops.
Granted, the difference between N and log(N) is not so huge when N=8, but it's still more than the cost of a transpose.

Additionally, the H.264 DCT was designed so as not to require any multiplies, but a row implementation still needs some.


Finally, I have searched info on PALIGNR but couldn't find any. Do you have any link / insight ?
I think it's the same as the WirelessMMX WALIGN, i.e. takes two mmregs and extracts a block of bytes that was split between them.

e.g.
input: [a0 a1 a2 a3 a4 a5 a6 a7], [b0 b1 b2 b3 b4 b5 b6 b7], immediate=3
output: [a3 a4 a5 a6 a7 b0 b1 b2]

The name comes from the fact that it's used to implement unaligned memory accesses on platfiorms that don't natively support them. Even though that's not strictly necessary on x86, it's still useful in places where we want to load multiple overlapping memory locations, or if we want to avoid dumping the intermediate values to memory at all.

[popper]

PSIGN : I thought it would return -1, 0 and 1, while PSAR gives -1 or 0.

I hoped PHSUB / PHADD could be used to avoid the transposition step in the DCT.

Finally, I have searched info on PALIGNR but couldn't find any. Do you have any link / insight ?

i just searched on "vector PALIGNR" and got this, dont know if theres anything useful there as iv not checked to closely,seems there might be!.

http://download.intel.com/design/Pentium4/specupdt/25204616.pdf

http://search.ntlworld.com/ntlworld/search.php?q=vector+PALIGNR&cr=

[akupenguin]

i just searched on "vector PALIGNR" and got this, dont know if theres anything useful there as iv not checked to closely,seems there might be!.
No, Intel has only officially published the opcodes, not what the instructions actually do.

[squid_80]

and a 64-bit direcshow player.

Um... Why? If your current CPU is not fast enough to play back a stream using a 32-bit player, chances are the slim benefit provided by a 64-bit player won't make any difference.

[nfm]

Hey squid_80 good to see you, how about 64-bit x264? I'll rewrite ASM and you'll rewrite C, deal? :D

[squid_80]

The C code doesn't need to be rewritten. If you want to fix the assembly code for Win64, be my guest. Unlike KRP says, it is already optimized but just needs a bit of tweaking to fit windows rather than linux.

[baer999]

I have three little questions about x264:
- what feature will be developed in future or what will be improved?
- when will the development starts or when is the summer break over?
- what do you think the quality and / or speed will be improved in % until the end of x264?

thx

[foxyshadis]

http://students.washington.edu/lorenm/src/x264/todo.txt

Most of these are (or at least might be) quality or speed improvements, so that should answer two questions.

[virus]

- what do you think the quality and / or speed will be improved in % until the end of x264?
x264 will never end.

[Sirber]

nothing ever end... In my case for every new version of RealAnime I say it's the final one and still, RA5 is not out yet and RA6 is in planning :D

[baer999]

Oh i thought there is not as much features left, but it's great that the codec has much improvements left... Now that Nero will release a new codec, x264 is not at the same level as Neros codec?

[Romario]

Now that Nero will release a new codec, x264 is not at the same level as Neros codec?

Who knows, new H.264 codec testing starting in August, I think. New Nero(Ateme) H.264 codec will be High Profile(finaly), and I think that Ateme H.264 has very good chance to beat x264 in quality.

About 64-bit x264, it's really sad to see that stable version stll doesn't exist.

Guys, Windows XP x64 came out before one and a half year. What are you waiting for:devil:

[ChronoCross]

because like 12 people have x64 XP. 64-bit apps will gain popularity once Vista Comes out.

[Sirber]

because like 12 people have x64 XP. 64-bit apps will gain popularity once Vista Comes out.Linux will gain popularity once Vista comes out :D

[squid_80]

About 64-bit x264, it's really sad to see that stable version stll doesn't exist.

Guys, Windows XP x64 came out before one and a half year. What are you waiting for:devil:
Why do you guys keep saying there's no win64 version? There is! It uses assembly! It's ~8% faster than 32-bit! What more do you want?

[Sharktooth]

They want updated daily builds in the sticky... but since i havent got winxp 64 i cant test them.

[popper]

No, Intel has only officially published the opcodes, not what the instructions actually do.

in the interests of trying to help push things along, perhaps this might help work out some of that problem
(thinks to hammer elsewere for the pointer)

a Disassembler with SSE4 support
20th, May 2006 - Ver 1.3.9 - diStorm now supports the VMX and SSE4 instructions sets!
http://www.ragestorm.net/distorm/

perhaps some of the japanese readers could translate this
SSE4(Merom New Instructions) unofficial reference page to english
http://dango.kousaku.in/hiki/SSE4.html


as a side note this paper while above my understanding right now, looks to have some interesting
thoughts
http://www.stanford.edu/~bschumit/Schumitsch_SPIE_05.pdf

[celtic_druid]

I could probably manage a new 64bit build. Have to reboot to X64 first though.

[Kostarum Rex Persia]

That will be very nice, celtic_druid.

[akupenguin]

perhaps some of the japanese readers could translate this
SSE4(Merom New Instructions) unofficial reference page to english
http://dango.kousaku.in/hiki/SSE4.html

Thanks. Yes, I can read Japanese, though one could probably decipher the asm examples even without that. Maybe I'll translate the whole page later.

(Keeping in mind that that page isn't certain either...)
eww, PHADD doesn't do what I thought it would. If one call to PHADD only adds pairs, then it reduces 3x(PSHUF,PADD) to 3x(PHADD), which is ok I guess, but not as good as a single instruction to add the whole width.
PADDUBSW would almost make SSD a bunch faster, but Words aren't quite enough precision.
PSHUFB is programmable, not immediate. cool.
PSIGN toggles the sign of one operand based on the sign of the other? might be useful in deblocking.

[Manao]

Thanks a lot for the link.

PSIGN in conjunction with PABS would help quantization. PSHUFB is programmable & seems to break the boundary between the low and high part of the 128bits registers, which means no more PSHUFHW & PSHUFLW. PABS would be a killer for deblocking, but I fail to see where PSIGN would help in deblocking.

PHADDW doesn't sum the whole vector, but three consecutive PHADDW will sum two whole vectors. It's not that bad, and you have more liberties than with a PHADDW that does sum the whole vector. I'd say it's a fair tradeoff.

[akupenguin]

I fail to see where PSIGN would help in deblocking.
I was thinking of the lines:
p0 += clip(delta, -tc0, tc0)
q0 -= clip(delta, -tc0, tc0)

I wonder if I can optimize the zigzag scan to some sequence of PUNPCK/PSHUFB/PALIGNR...

[Bluedan]

I could probably manage a new 64bit build. Have to reboot to X64 first though.

Any news on this attempt?

[burfadel]

I'm not going to open an old can of worms here, I am making a valid point so don't jump to conclusions!

Admittedly the CLI version of x264 seems to produce better picture quality that the VFW version, but both have their weaknesses and strengths.

CLI:
Strengths:
- Produces high quality video
- Full range of options available

Weaknesses:
- Not the easiest to use and set up, whether is with MEGUI, Staxrip or other. That is, not user friendly
- Requires many separate programs to compete the one task
- Requires large amounts of disk space, that is, for the original source, the indexed source, the direct audio extraction, the encoded video file, the encoded audio file, and the output file. If the source is a movie off TV and is 10GB, thats easily up to 25GB worth of stuff.
- Takes a short while to set up each encode
- Batch option available but with the above disk space use its not always practical.
- Difficult to set up the encoding of TV episodes off DVD.

VFW

Strengths:
- Using a programme like XMPEG:
- very fast to set up to encode
- Comes in under 3MB, all that is required is the VFW x264 and audio encoder
- Very user friendly
- Doesn't take up excessive hard disk space, only the space of the original file and the output file (files if using direct stream copy for audio)
- Encoding episodes of a Tv series is as easy as opening the 'Movie file' IFO, then right hand clicking on the slide at the bottom of the screen and each episode is listed as a title. Just click on the title you want, its as simple as that!

Weaknesses:
- Doesn't produce as good of a quality as the CLI version
- Some people hate the VFW version now
- Some of the options aren't available, but they could be if written for it

Now my point is, a means of combining the advantages of both into one package would be great! An encoder with all the features of XMPEG (ease of use, episodes encoding etc) all in one (even if its 5mb installed) would be great. One idea if it were possible, would be to index and extract audio 'just in time', that is on the fly, into memory then encode it. Just a thought.

Simple fact is, more people would use x264 if there was an easy way like that to use it, otherwise they're just use Nero digital or some other encoder!

[GodofaGap]

Well, I'd say: stop wasting time and start writing it. :)

You are making a valid point, until one realises it is a free-time project, not a commercial product. If you'd rather use Nero, no one will try to stop you.

[KoD]

And the coders will benefit from all this what ? Lots of time spent because it's hard for some people to learn how to use the CLI version ? How about taking yourself some time and read why vfw is so much hated when it comes to new video formats, or is your time more valuable than theirs ?

[burfadel]

No, its more the fact that most people are amateurs when it comes to using computer out there, most would not be able to work out how to use a programme thats not user friendly.

My other point about episode encoding is probably more to the point, the references to each title (episode), which couldn't be that difficult to incorporate...

Also maybe the automatic deletion of the source vob's etc should be an option after it is indexed to save disk space, then the option to skip the indexing and go straight to encoding if for some reason you have to start over again.

That ^^ would be particularly useful for episode encoding. An option in MEGUI for example to select the titles you want to encode (that is, just by simple clicking on a box for title 1, title 2, etc. (and have the length of the title next to it like in xmpeg etc), then once you set it up for one title, it automatically encodes all the titles without you having to worry about setting each one up manually, nor having it indexed on each subsequent title.

My last post was more about if there was an easy way to do things many more people would be interested in using this great codec rather than relying on the inferior but easier to use programmes and codecs that they currently use!

The people working on x264, MEGUI etc are doing a great job, I didn't mean to sound like I was criticising at all, I was just trying to hint about would would be a great advantage to it without sounding too critical, and open some ideas to them about how to make it better.

And yes, about time, if you try encoding say, 9 seasons of Stargate SG-1 and 2 seasons of Atlantis (over 210 episodes), the CLI route is extremely slow in its current form due to the indexing (and reindexing), whereas the automation and only the space required for the DVD's etc required for the XMPEG route was a great advantage to me, especially since I had to return them to a friend before he went back home (which is a long way from here). Even for a few episodes the ease and practicality of the VFW XMPEG is a great advantage, and there's no reason why that ease of use can't be incorporated into MEGUI, Staxrip etc. And yes I did use xmpeg, it was a relief to just click on 'title' 1, 'title' 2 etc and name it and encode it, instead of selecting the source range of each episode, having it index etc. Also the 60+ DVD's takes considerable amount of disk space, having it automated with indexing and the final would have required about 900gb or so, which I for some reason don't have...!

[GodofaGap]

I'm sorry, I have no idea what you mean with indexing... if you mean the creation of a d2v file it hardly requires extra space (on a DVD scale) and doesn't take much time (compared with encoding the file).

and there's no reason why that ease of use can't be incorporated into MEGUI, Staxrip etc.
So do it.

What point are you trying to make? Do you think developers are not aware that the current software is not an optimal 1 click solution? Do you think developers, if everything you say is so easy and straightforward to implement, wouldn't have done it already?

I'm not sure what you are trying to achieve with your post. If something esle works better for you, go ahead and use that. What is the trouble?

[emmel]

Also the 60+ DVD's takes considerable amount of disk space, having it automated with indexing and the final would have required about 900gb or so, which I for some reason don't have...!

I wonder, where that 900G came from?

Let's say you need a 25G workspace to transcode one title. Ok. But once it is done, you just delete the temporary files, releasing your work space, and move on with the next title, right? I'd say you need about max. 100G for 60+ dvds with x264.

Besides, by using x264, you save 25G in a moment, you don't even have to spend the 7,5$ (30 cents/G nowadays) to purchase a temp disk.

[burfadel]

The main thing still comes back to the episode encoding, for movies the current way of doing things is fine! but encoding several episodes like I have been with Stargate the process became a bit excessive!

I didn't mean to offend anyone! Its just that I wanted to run the programme in batch mode which if I did it for all episodes of stargate off the DVD's then it would equal 900gb in total! I actually resorted to XMPEG which was quick, so I thought there may have been others who have experienced the same thing that would come out of their shell when I moaned and griped (usually if one person starts others will follow!) So those developing MEGUI would incorporate it in a future version.

My idea would be for it to read the IFO file, then tick the checkboxes of each title wanted (the length of the title is included with with IFO so that can be displayed), and below that have an input line for each title so the output file can be named. Only one d2v file needs to be created, and each episode can be automatically selected by the programme setting the source range according to the ifo. This would be a handy feature for anyone wanting to do episode recording.

I guess the whole idea of forums is to help each other out, and to give possible suggestions for improvements! I do apologise for my griping but I became frustrated with it :o you know how people can go overboard when they're frustrated with something :D

Anyways, I'd still love to hear what others think of my suggestion, I think it would be a very beneficial feature, and appreciated by many!