x264 development

[snacky]

No, but you can run "svn log" to see the log notes that appear with each revision.

[ChronoReverse]

http://img86.exs.cx/img86/162/pframe0ag.png
http://img86.exs.cx/img86/5433/bframe17nn.png
http://img86.exs.cx/img86/4302/bframe29vs.png
http://img86.exs.cx/img86/7670/bframe30up.png

The flashing blocks that sometimes come with 3 bframes is occuring. Pframe looks fine. Bframe following is blocked. Bframe following that is fine. 3rd bframe is blocked.

C_D's x264 revision 113 build
2-pass encode targetted at 612kbps (I was aiming for 1/5 CDs for 26 minutes to see how x264 would do)
Load Default Settings
No Fast first pass
Max reference frames: 12
Max B-frames: 3
Max IDR-keyframe interval: 240
Min IDR-keyframe interval: 24

640x480 anime DVD source @ 23.97FPS


I'm making encodes with 0-2 bframes to see what happens.

[akupenguin]

As for why it's blocking in some frames but not others: Changing brightness combined with non-adaptive B-frames and no weighted prediction means that some blocks are predicted as a simple average of the two reference frames. The middle B-frame is exactly half-way between the references, so that's correct. The others are not halfway, but they still use the same prediction.
As for why it's blocking at all: dunno, will do more testing.

For this and other reasons, I do not consider multiple B-frames to be usable in x264.

[Mug Funky]

forgive me if this has been asked before (or if it's just a plain annoying question), but is interlace support on the to do list? i have a buttload of DV footage and also lots of field-blended anime.

i realise a lot of people can't stand interlacing (myself included), but it's going to be a legacy thing for quite a few years to come, and deinterlacing is pointless - if you're using high-quality encoders there's no point ditching up to half your information beforehand.

btw, x264 is kicking severe arse. i love what you guys are doing.

[Manao]

Mug Funky : interlacing is a real pain to support in AVC/h264 ( both in encoder and decoder ). And, even if x264 supported the tools that allows better compression with interlaced content, i think you'd still get a better quality by deinterlacing the video, and encode it at the same bitrate ( you lose some motion, but you gain a lot in picture quality after encoding, at least on a progressive display ). Finally, h264 ( and mpeg-4 ) handle relatively well interlaced content, even without the interlacing tools ( you using these tools correctly makes the codec so slow that you don't want to, anyway... )

[akupenguin]

Interlacing is not planned, and patches for interlacing will be accepted only if they do not increase the code complexity of the rest of the codec. (Which at best means pure interlacing only, no PAFF/MBAFF).

Interlacing is inherently ugly (and interlaced displays only make progressive content ugly, they don't make interlaced content any prettier). The only solution is fix the source, not the codec. Anime is always originally progressive; find a better copy. If you captured your own footage in interlaced DV, then the damage is already done, and you'll just have to find a good motion-adaptive 3D deinterlacer (not that I'm saying such a filter exists). IMNSHO, even simply throwing away one field is an improvement over watching interlaced video, let alone compressing it.

Similarly, I don't plan to add interlacing support to libavcodec's H.264. Though it's already partially present, and there are other people working on lavc, so support might appear anyway.

And that's just my reasons for not wanting to touch interlaced video. I haven't even gone into implementation details.

Originally posted by Manao
Finally, h264 ( and mpeg-4 ) handle relatively well interlaced content, even without the interlacing tools
Interlaced content in a non-interlaced codec adds lots of high-frequencies, and makes vertical motion compensation completely impossible.
I just ran a test using current x264, comparing compression of 30p content decimated by either interlacing or by dropping frames. Result: 15i took 1.8x the bitrate of 15p at a given qp, and 2.0x the bitrate at a given psnr. 30p at half height (deinterlaced) took 1.1x the bitrate of 15p.

[gotaserena]

Sorry for the OT, but there's something I've never quite got in this interlacing business:

Suppose the source material is pure interlaced -- like TV captures or smthg similar. What is wrong about the idea of having the codec separate the fields before encoding and having the interlacing done "on the fly" during decoding (assuming you don't want it bobbed or blended or something else)?

[akupenguin]

Originally posted by gotaserena
What is wrong about the idea of having the codec separate the fields before encoding and having the interlacing done "on the fly" during decoding? That's exactly what interlaced H.264 is. (Assuming you're not using PAFF or MBAFF.)

[gotaserena]

I had to look up those words! :) I've read somewhere that XviD's interlaced mode consisted of "not spending too many bits on combing effects" (I'm paraphrasing here), and I assumed that you had the same thing in mind for x264. It would be nice to have this simple algorithm, anyways.

[Yong]

Originally posted by Yong
Can someone please explain, how the nthpass encoding process work?
Is it will increase the quality with nthpass by using the lower bitrate?
I only tried until 3th pass, with 400kbps.
Any difference with 2pass encoding with 400kbps?

sorry for my bad english...

It seems my question too hard for answer:D
ok, i make it simple:
anyone try to compare video quality 2pass encoding at 400kbps(other bitrate also can);)
with Npass encoding at 400kbps?

[Mug Funky]

I just ran a test using current x264, comparing compression of 30p content decimated by either interlacing or by dropping frames. Result: 15i took 1.8x the bitrate of 15p at a given qp, and 2.0x the bitrate at a given psnr. 30p at half height (deinterlaced) took 1.1x the bitrate of 15p.

those are some interesting numbers. so i'd be better off full-rate deinterlacing and encoding in twice the frame rate? i guess my machine's overdue for an upgrade anyway...

it's just a pity that so much of the footage out there is interlaced. progressive films are actually the exception rather than the rule.

i can live with xvid for now, at least for my interlaced stuff :)

[akupenguin]

Originally posted by gotaserena
I've read somewhere that XviD's interlaced mode consisted of "not spending too many bits on combing effects" (I'm paraphrasing here), and I assumed that you had the same thing in mind for x264. No, ASP has the same ideas of field-based MC and DCT, PAFF, and MBAFF. The only difference is that for the AFF types, ASP just splits and joins fields, while H.264 downscales or deinterlaces as needed, and also scales MVs for temporal prediction.

[Manao]

There's no PAFF in ASP, only a kind of MBAFF ( interlaced DCT and interlaced motion compensation ). XviD only handles interlacing with interlaced DCT, which doesn't help that much ( in comparison to VHQ4 without interlacing )

When I said h264 / mpeg4 handles relatively well interlaced content, it was in comparison to h264 / mpeg4 with PAFF / MBAFF, with roughly the same speed settings.

[708145]

Originally posted by Mug Funky
those are some interesting numbers. so i'd be better off full-rate deinterlacing and encoding in twice the frame rate? i guess my machine's overdue for an upgrade anyway...


It might be a bit OT but:
1) Is there a good smart deinterlacer that produces 60p from 60i content at full resolution? Not just blend field or such!
2) How much more bitrate would x264 consume with 60p compared to 30p (both full resolution). I guess B-frames are _very_ important there.

edit: spelling error fixed.

bis besser,
Tobias

[temporance]

Does there exist a working H.264 VfW decoder anywhere?

Here's what I've already tried:

- x264 : doesn't decode
- VSS decode set: only DShow decoder supplied
- VSS 5-day trial: buggy decode with block artifacts (looks like PP because errors don't propagate). Also looks like B-frames are not reordered.
- FFVFW: haven't been able to get this working yet.

I really need a native VfW decode: can anyone help?

[Sharktooth]

latest ffdshow (that includes ffvfw too).
http://www.aziendeassociate.it/cd.asp?dir=/ffdshow

[Tommy Carrot]

Yes, the latest ffdshow, but you have to enable the h.264 decoding on the VFW codec configuration/decoder/codecs tab, because it's disabled by default.

[Sharktooth]

exactly, you should enable this 4cc in ffvfw configuration (set it to libavcodec):
http://www.aziendeassociate.com/ffdshow-vfw.jpg

result:
http://www.aziendeassociate.com/x264.jpg
Note: the picture image quality does not represent the codec image quality.

[hpn]

Something a little off-topic for this thread, but please use png instead of jpg format (or at least higher quality jpg) when making screenshots, otherwise some people may start thinking that the low quality of the shots they see has something to do with the x264 encoder, which is completely untrue.

[Sharktooth]

well that was not representing the codec quality... ok, i will add a disclaimer...

[Blue_MiSfit]

Akupenguin:
As for why it's blocking in some frames but not others: Changing brightness combined with non-adaptive B-frames and no weighted prediction means that some blocks are predicted as a simple average of the two reference frames. The middle B-frame is exactly half-way between the references, so that's correct. The others are not halfway, but they still use the same prediction.

So, ATM using 1 b-frame is all good, but with more than 1 we might see the evil blocking errors like I was getting with my Family Guy encode awhile back?

Is there any point in having more than 1 reference frame if you are only using 1 b-frame?

Just trying to get things clarified... I am very impressed BTW, this is developing faster than XviD was :)

~misfit

[akupenguin]

Originally posted by Blue_MiSfit
So, ATM using 1 b-frame is all good, but with more than 1 we might see the evil blocking errors like I was getting with my Family Guy encode awhile back? yes. (The blocking is still a bug, I was just explaining why it occurred only some of the time.)

Is there any point in having more than 1 reference frame if you are only using 1 b-frame? # of reference frames is completely independent from # of B-frames. Multiple references always help, with or without B-frames. (But I haven't tuned it for B-frames. I might be able to improve total quality by spending more time on the reference search for P-frames and less time for B-frames.)

[ChronoReverse]

Strange bug I'm encountering. Very reproducible on both my AthlonXP boxes.

Take one avisynth script (640x480 anime DVD source @ 23.97FPS)
Add vdubmod 1.5.10.1
Fast Recompress mode
Stir in x264vfw rev. 113
*2-pass encode targetted at 612kbps
*Load Default Settings
*No Fast first pass
*Max reference frames: 12
*Max B-frames: 1
*Max IDR-keyframe interval: 240
*Min IDR-keyframe interval: 24

Set to simmer and encode using the Job Control.



Somewhere during the first pass, vdubmod invariably exits completely without warning. It doesn't happen with 3 bframes and 0 frames (with all other settings the same).

After I finish off this 1 bframe encode, I'll be trying the same settings with revision 112.

I'm currently also doing an encode with all default settings exception 1 bframe.


I tried it without job control and got this error dialogue:
http://img215.exs.cx/img215/5994/error1rn.gif



While I was comparing x264 to XviD, I noticed some things.

Source: http://img202.exs.cx/img202/1957/source7rf.png
XviD 1.1.0b: http://img202.exs.cx/img202/6515/xvid7wz.png
x264vfw rev.113: http://img202.exs.cx/img202/5663/x264r1135cn.png

First off, there's a line that only exists in the x264vfw. Weird, but I'm sure it's just one of those bugs that'll be fixed.

Secondly, x264 looks quite a bit better in this particularly frame. However, for most of the frames the XviD encode only has marginally more ringing than x264 and often more detail (it may be ffdshow, but the x264 encodes are slightly more blurry).
To be fair, the XviD encode was done using 3 bframes while the x264 encode was done using 0 bframes.

[akupenguin]

That assertion has been reported several times, and each time that I could verify anything at all, it turned out that the first pass logfile had been truncated. Maybe whatever the bug is, it's fatal if using jobcontrol, but silently ignores the error in a single encode?

[temporance]

I'm encountering a crash in x264 which, from the disassembly looks like it's an access violation in a MMX SAD routine somewhere.

I've built the binary using cygwin, so when I debug in MSVC I have no debug info so can't trace the bug.

Which is easier: have the cygwin build env generate debug info for MSVC, or build x264 using MSVC?

[akupenguin]

I refuse to have anything to do with MSVC, so I can't say which is easier. My choice would be to debug with GDB.
Have you determined whether it's a problem with MMX, and not bad inputs to the SAD function?

[Yong]

My question probably too stupid, :p
so i try myself comparing 2pass encoding with 3pass encoding,
the result is quite disappointed,
i found nothing is gain or lost in the 3pass mode, comapare with 2pass mode.

Here's how:
(example only, forgot to copy the encoding batch script :p, may be something wrong with my encoding step, if you don't thrust me, please try it out yourself, with mencoder or x264vfw)
2pass encoding
i use fast first-pass,
the use slow second pass, 800kbps.

3pass encoding
also fast first pass
slow second pass with 1600kbps,
slow third pass(same as second pass, but only the bitrate is change),800kbps.

the result is,
2pass encoded video is bit-identical to 3pass encoded video,
:eek: , please correct me if i'm do something wrong:D

[temporance]

Originally posted by akupenguin
I refuse to have anything to do with MSVC, so I can't say which is easier. My choice would be to debug with GDB.
Have you determined whether it's a problem with MMX, and not bad inputs to the SAD function? It's not a problem with MMX, as I expected. The crash seems to originate here:

/* try extra predictors if provided */
if (0) for( i_iter = 0; i_iter < i_mvc; i_iter++ )
{
const int mx = x264_clip3( ( mvc[i_iter][0] + 2 ) >> 2, mv_x_min, mv_x_max );
const int my = x264_clip3( ( mvc[i_iter][1] + 2 ) >> 2, mv_y_min, mv_y_max );
if( mx != bmx || my != bmy )
COST_MV( mx, my );
}
My hack-fix is in bold.

When i_iter=1 there are some extreme MVs. Hope this helps: if you want me to take a look further I can.

[akupenguin]

2pass encoded video is bit-identical to 3pass encoded video
You didn't enable "Update statsfile". (Either that or the vfw interface for it is broken, but I doubt it.)

And changing the bitrate between 2nd and 3rd pass completely defeats the purpose of 3pass encoding. It's suppose to refine the bit distribution, not start over from scratch.

[Yong]

Originally posted by akupenguin
You didn't enable "Update statsfile". (Either that or the vfw interface for it is broken, but I doubt it.)

And changing the bitrate between 2nd and 3rd pass completely defeats the purpose of 3pass encoding.

No test that much with x264vfw, but i'm sure the option "Update statsfile" is selected.

The test is done using mencoder 2005 FEB 01 compiled by celtic druid,
so how to use the 3pass encoding?
It's suppose to refine the bit distribution, not start over from scratch.
Which options to change?
Sorry i'm dumb and please theach me:D

[akupenguin]

Originally posted by temporance
When i_iter=1 there are some extreme MVs. Hope this helps: if you want me to take a look further I can. Are the MVs really extreme (i.e. uninitialized data, which would be a separate bug), or just point a little bit outside the picture?
Either way, I think I have fixed the crash.

[akupenguin]

Originally posted by Yong
The test is done using mencoder 2005 FEB 01 compiled by celtic druid,
so how to use the 3pass encoding? In that case, it's simple: "pass=2" means 2 pass encoding. "pass=3" means N pass encoding.
A 3 pass encode consists of one encode with fast settings and pass=1, followed by two encodes with identical slow settings and pass=3. (You don't have to stop at 3 passes, but any further is pointless.)
Which options to change? Don't change anything between the 2nd and 3rd passes. (In the interest of speed, you should be able to get away with a slight change in subq or frameref, but I haven't done any rigorous tests of that.)

For an example (which does tweak subq and frameref), see the shell scripts in virus' x264 pack for cygwin (http://forum.doom9.org/showthread.php?s=&postid=603606#post603606)

[Yong]

Originally posted by akupenguin
In that case, it's simple: "pass=2" means 2 pass encoding. "pass=3" means N pass encoding.
A 3 pass encode consists of one encode with fast settings and pass=1, followed by two encodes with identical slow settings and pass=3. (You don't have to stop at 3 passes, but any further is pointless.)
Don't change anything between the 2nd and 3rd passes. (In the interest of speed, you should be able to get away with a slight change in subq or frameref, but I haven't done any rigorous tests of that.)

For an example (which does tweak subq and frameref), see the shell scripts in virus' x264 pack for cygwin (http://forum.doom9.org/showthread.php?s=&postid=603606#post603606)

Thanks for the replies,
i'll try out later:D

[temporance]

Originally posted by akupenguin
Are the MVs really extreme (i.e. uninitialized data, which would be a separate bug), or just point a little bit outside the picture?
Either way, I think I have fixed the crash. I think they were just a little bit outside the picture.

[Sharktooth]

Rev 114 binaries: outdated

Rev 114 WFV installer: outdated

This is not celtic_druid build. There are both the VFW dll and commandline encoder. Compiled with gcc and nasm with default asm and c flags.
It may not work since it's the first time i compile x264. However, no, it won't make your computer explode... :)

EDIT: updated with NDEBUG=1 option.
EDIT2: added nsis installer version.

[hpn]

Just did a fast 2-pass encode (quality settings maxed up) and your rev114 build works fine, although the x264vfw.dll is now twice as big as the one in build 113 (972K compared to 406K). The encoding speed is the same, so I hope the dll size doesn't matter too much and only depends on the way one compiles it.

Edit: tested again with the new build (dll is now 362K only!) and it works fine again. A bit by bit comparison proves that both encodes are identical, not that I expected something else, but just to make sure :)

[virus]

@Sharktooth:
better if you use "make NDEBUG=1" for both the core and the vfw. You'll end up with much smaller files.

I'll update my pack very soon with rev. 114. You'll be able to find it both at the usual address as well as Tobias' mirror (http://www.iti.uni-stuttgart.de/~bergmats/video_mirror/videotools/) (which is very fast btw).

I hope to be able to add other goodies to it in the near future - spyder sent me some scripts/tools for automated AVS2MP3 conversion, which can be tweaked for AAC too. Also nls is working on a commandline calculator (based on XviD code) which may be worth inclusion. Maybe I can manage to do a full rip with a single command... ;)

EDIT: find it here (http://www.webalice.it/riccardo.stievano/video_stuff/x264-encpack-r114.zip).

[gotaserena]

Funny, my cross-compiled version has 419kb... I wonder how fast it is, when compared to celtic_druid's...

Better spend more time in my win box! :D

[Sharktooth]

Originally posted by virus
@Sharktooth:
better if you use "make NDEBUG=1" for both the core and the vfw. You'll end up with much smaller files.
i thought i did it...
... re-making

EDIT: udpdated. filesizes are ok... now.

[ChronoReverse]

Load Defaults button doesn't visibly do anything.

@Akupenguin

That assertion has been reported several times, and each time that I could verify anything at all, it turned out that the first pass logfile had been truncated. Maybe whatever the bug is, it's fatal if using jobcontrol, but silently ignores the error in a single encode?
I have no idea, but the crash also occurs in the first pass too in some situations (again perfectly reproducible).

[Sharktooth]

load defaults should affect advanced options only.

[JoeBG]

Originally posted by virus
@Sharktooth:
better if you use "make NDEBUG=1" for both the core and the vfw. You'll end up with much smaller files.

I'll update my pack very soon with rev. 114. You'll be able to find it both at the usual address as well as Tobias' mirror (http://www.iti.uni-stuttgart.de/~bergmats/video_mirror/videotools/) (which is very fast btw).

I hope to be able to add other goodies to it in the near future - spyder sent me some scripts/tools for automated AVS2MP3 conversion, which can be tweaked for AAC too. Also nls is working on a commandline calculator (based on XviD code) which may be worth inclusion. Maybe I can manage to do a full rip with a single command... ;)

EDIT: find it here (http://www.webalice.it/riccardo.stievano/video_stuff/x264-encpack-r114.zip).

And it´s great. Testet it already. Thank you very much.

[Sharktooth]

Ok, celtic_druid built the rev114 too.
It's at the usual place (already mirrored)
Next builds will have an installer :)

[virus]

thx JoeBG :)

Since I plan to keep the mencoder pack updated (and even expand it with additions), I thought about setting up a small webpage to keep you informed, without the need to spam this forum for each tiny change/update :D

I won't do builds for all revisions. I've already skipped revs 111 and 113 because they were purely cosmetic, and didn't change the encoder at all. So I'm going to do meaningful updates only ;)

I'm a total n00b at HTML, but anyway here (http://www.webalice.it/riccardo.stievano/video_stuff/x264.html) you can find my own x264 page (with a link to the encpack). I hope to expand it in future... I also have some plans of writing some decent documentation for x264 ("decent" means "TeX" for me, not "Word", just to make it clear): how to get the sources, how to compile, what encoding options to choose and so on. Someday maybe I'll find the time...

virus

[akupenguin]

Originally posted by ChronoReverse
I have no idea, but the crash also occurs in the first pass too in some situations (again perfectly reproducible). Not possible. Ratecontrol doesn't even run on the 1st pass. (I'm not saying there aren't other possible crashes, like the one I just fixed.)

[ChronoReverse]

I'm not referring to the Assert Error, if that's what you meant. It's the other crash where vdubmod just disappears.


In any case, it didn't seem to happen with revision 114 so it's a moot point.

[Sharktooth]

x.264 VFW rev.115 win32 binary with NSIS installer: outdated

Changes:
r114 didn't completely fix the problem, trying again.

Notes:
compiled with gcc and nasm and default c and asm flags.

[DLMunson]

WOW, i just tried x264(115) and DAMN. i couldnt tell difference from the original dvd, sure i use high bitrates(1500-1700 for video) but it looked perfect, while xvid (1.0.3, 1.1.0beta) had some blocking, artifacts or whatever you call them, im a complete n00b to videoencoding :(

for being so early in development this codec was flawless, no crashes and quality was sick. mad props to all the people workin on this. too bad its 1/30 the encoding speed. but oh well its worth it.

[Yong]

@akupenguin:
You say don't change the options between 2pass and 3pass,
i tried with x264vfw and mencoder,
still same result,
2pass encoded video file size are bit-identical to 3pass encoded
:(

[Sharktooth]

That means the ratecontrol just did it's job in the second pass.
You should be happy with that :)