x264 development

[Doom9]

This is a partial continuation of the former x264 development thread (http://forum.doom9.org/showthread.php?t=80910). This thread is meant purely for development purposes, which does not include making any feature requests, especially not for features to be added to the VfW interface!. Any VfW relates issues have to go here (http://forum.doom9.org/showthread.php?t=108569) instead. Any bugreports also do not belong to this thread but either in the VfW thread, or the x264 cli bugreport thread (http://forum.doom9.org/showthread.php?p=798509#post798509).

[macher]

I am new to x264 code
The decoder testing part in x264.c was removed some months ago and it has not returned since then. The exact release was a snapshot of 24-08-2005.
I want to work on the decoder for a project of my own. In that old snapshot the only thing in the coments was something like "The decoder test part is not working and we would appreciate some help in this direction". I would like to know what eactly were the problems in this regard and how much work i would have to put in to get it working again ?

[akupenguin]

The decoder supposedly supports baseline profile only. I have no idea how close to working it was in 2004, since I have never compiled it. In addition to just making it work, much will need to be updated to match changes in the functions shared with the encoder, which have been modified since then.

[macher]

from the compilation that i have i cannot get a clue as to which profiles it is supporting because i didnt see any comments about it(may b i ddint look hard enough). but i think it should be a bit more advanced.

[akupenguin]

No B-frames, no cabac, no wpred, no interlacing, and it was written before high profile existed. That makes it baseline.

[macher]

and nobody ever mad eany changes to it ?

[akupenguin]

Fenrir converted from cvs to svn in 2004-06-03, at which point earlier revision history was lost. The decoder has not been modified since then.

[macher]

is there any other open source decoder which supports the main profile or at least compatible with common output formats?

[akupenguin]

http://forum.doom9.org/showthread.php?t=108570

[macher]

thanks

[lspbeyond]

I have implemented the x264 decoder supported baseline profile several weeks ago. Unexpected, the current x264 program structure is not suitable for decoder. Especially of the interpolation module and the macroblock context cache load/save module. The experimental results show that it's half lower than ffmpeg libavcodec.
However, I won't stop improving my decoder. For I think the structure of h.264 decoder of ffmpeg libavcodec is so bad. And it's decoding speed is not fast enough.

[Leo 69]

I think the structure of h.264 decoder of ffmpeg libavcodec is so bad. And it's decoding speed is not fast enough.

That's right. Kick some CoreAVC's asses out there too :D

And now my question to the developers. What is to be done to the X264 encoder in order to improve visual quality and compressibility (i.e. what features are yet to be implemented) ?
Is there a roadmap for improving or anything else ?
It is always interesting to know in advance about what we may expect in the future, isn't it ?

Thanks

[akupenguin]

There is no plan, do not ask about future features, I will not discuss anything I'm writing until it's ready to test.

[Kostarum Rex Persia]

Well, Leo 69, try to read TODO list from http://students.washington.edu/lorenm/src/x264/todo.txt

[Tommy Carrot]

The RD subpel motion estimation (--subme 7) added in rev.476 is the old patch finally committed to the svn, or is it a completely new approach?

[Romario]

I don't know for sure, but I think that RD subpel motion estimation(--subme 7) is a old patch, with necessary changes.

[ChronoCross]

it's the old patch updated to work with the new svn. I think he may have made some updates to it in other ways as well.

[CEC]

Is the in-loop filter adaptive??? I made a high bitrate encode (720x576 2500 kbps) once with in-loop and once without! I can't see a difference between them!!!!!!

[Sharktooth]

yes it is.

[CEC]

Thanks for making it clear to me!!!:thanks:

In the Sticky: "MPEG-4 AVC/H.264 Information" bond doesn't say anything about adaptive loop in x264!!!

[IgorC]

I hope this link will be considirated as corelative (not like crossposting) http://forum.doom9.org/showthread.php?p=810697#post810697

[akupenguin]

In the Sticky: "MPEG-4 AVC/H.264 Information" bond doesn't say anything about adaptive loop in x264
Becuase all h264 codecs have adaptive loopfilter. It's impossible to make one that isn't adaptive, and still comply with the standard.

[7zeal]

I am trying to compress (2 pass, sharktooth's x264 530 - on the HQ-Insane profile, and 2000 kbps) a HD 720p movie (1280x528 @ 23.976 - a little cropped, it's widescreen) from wmv content.
I am running the first pass, as so:

H:\Movies>x264.exe --f
ps 23.976 --pass 1 --bitrate 2000 --stats "H:\Movies\[......] 16x9 72
0p WM9 HD-DVD.stats" --bframes 3 --b-pyramid --filter -2,-1 --subme 1 --analyse
none --direct auto --me dia --cqmfile "C:\Codecs\x264\Matrices\eqm_avc_hr.cfg" --progress --no-psnr --output NUL "H:\Movies\[.........] 16x9 720p WM9 HD-DVD.avs"
x264 [info]: file name gives 16x9
avis [info]: 1280x528 @ 23.98 fps (181032 frames)
x264 [info]: using cpu capabilities MMX MMXEXT SSE SSE2 3DNow!

and it finally ends up writing this (you can see, not ALL the frames were written to the .stats file)

encoded frames: 180851/181032 (99.9%), 10.24 fps, eta 0:00:17
H:\Movies>

Now, as I call pass 2 with all the other options enabled, it throws out this error:
x264 [error]: 2nd pass has more frames than 1st pass (181032 vs 180999)
x264_encoder_open failed

I am using the same x264 commandline as the latest MeGui or StaxRip 0.9.9.2, in fact this first occured while trying to compress through MeGui, x264 HQ-Insane, then I wrote the commands on my own into the console, trying to figure out what I should change - no success. StaxRip 0.9.9.0 had the same difficulty (with Sharktooth's x264 build 467 if I remember well)

I tried to trim the movie from the avs script, cutting until that last frame encoded into the .stats file , so pass 2 worked.
However, though x264 said "23.98 fps", as I took a part of the encoded movie and muxed it into mp4 (latest mp4box), BSPlayer recognized it as 25 fps. (?)
The aac encoded is perfectly synched with the source wav from the wmv, BUT the encoded x264 movie (even from minute 3) seems to be earlier I think than the wmv. (???) why..
This happened before I ran x264 with --fps 23.976 - but I don't think it made a difference since it recognized it anyway as 23.98.

[daverc]

encoded frames: 180851/181032 (99.9%), 10.24 fps, eta 0:00:17

It happens to me when the source is directshowsource in the avs script and ffdshow does audio filtering, especially channel mixing.

Try to disable every filtering before launching your avs script.

[Sharktooth]

this is the dev thread and it has nothing to do with those kind of questions...

[7zeal]

I thought it was an x264 bug, doesn't that qualify?

[Sharktooth]

yes, sorry, i was a bit too drastic.

[7zeal]

It happens to me when the source is directshowsource in the avs script and ffdshow does audio filtering, especially channel mixing.

Try to disable every filtering before launching your avs script.

Thanks, seems like I may have a lead here..:P
(note: have lastest ffdshow-2546-gcc4.0.3-sse2-x264)
I had "Dolby Surround Downmix" on at audio processing plugins to be loaded automatically, but don't know if it's to blame, unchecked it anyway. (oh, and "subtitles" was also ticked for automatic loading in ffdshow at video decoder config).
I also have 'Avisynth' ticked at the video decoder config... hmm
It's suppose to bring ffdshow filters into avisynth. I left it on.
Is there anything else I should be doing ? I'll get back to you after running the first pass.

Thanks

[7zeal]

Same behaviour .. All ffdshow filters out, and the first pass still stops at 99% , 24 sec remaining.
What's wrong?

[ChronoCross]

Same behaviour .. All ffdshow filters out, and the first pass still stops at 99% , 24 sec remaining.
What's wrong?

This is the development thread. Bug-reports do not go here. Open a new thread or find the bug-thread. Thanks

[Xesdeeni]

Please forgive me (and be kind) if these are braindead questions (consider the source):

I'm trying to determine what the output file format is for the various apps wrapping the codec, specifically under Windows. Obviously the DirectShow AVI filter wraps the data in an AVI file format. But does the command line application use another file format? Which one(s)? And is it true that the h.264 standard file format is Quicktime (MOV)?

Xesdeeni

[GodofaGap]

CLI can output to raw h264, Matroska and mp4 (if compiled with GPAC library). There is not one standard container for h264.

[bond]

There is not one standard container for h264.indeed, there are three:
1) .mp4
2) .mpg
3) .ts

[Sergey A. Sablin]

indeed, there are three:
1) .mp4
2) .mpg
3) .ts

what is .mpg here? PS? or AVC Annex B format?

[RBF]

Sergey
.mpg -> PS
AVC Annex B is the byte stream format, not the container format, неправда ли?

[Sergey A. Sablin]

Sergey
.mpg -> PS
AVC Annex B is the byte stream format, not the container format, неправда ли?

right, just want to clarify things - I saw many .mpg files with TS/PS/VES/AnnexB inside. Sometimes even just audio.

[bond]

yep, i meant PS with .mpg

[Romario]

Look at this, can anyone confirm that devs read this.

http://www3.intel.com/cd/software/products/asmo-na/eng/219766.htm

[KoD]

Intel has those libraries on sale for quite some time. But it's not like they're free for use.

[Romario]

I don't know why Intel want money for this.

If this package become freeware, then devs can optimize their programs and codecs for Intel Core 2 architecture MUCH EASIER.

[Koti]

I don't know why Intel want money for this.

If this package become freeware, then devs can optimize their programs and codecs for Intel Core 2 architecture MUCH EASIER.

you could always buy the libraries for x264 dev's ;)

[Romario]

Yes, but I can't.

[KoD]

Because Intel is not making money only from selling Core2 Duo processors. They are actually producing more hardware and software than just cpus for personal computers.

[Romario]

Certainly, but...

If someone decide to make pressure on Intel software division, who knows what can happen? :)

[Sharktooth]

It's like asking M$ to make Windows OpenSource...
and however x264 has ASM optimizations that maybe are even faster than intel libs...

[Adub]

faster, really? is that because they are done by hand? But I thought that x264 hadn't been all the way optimized yet.

[Sharktooth]

Hand optimizations are always better than third party libs (that work well only on certain scenarios), also qualitywise x264 is as fast if not faster than commercial h.264 encoders... so how can you say it's not optimized?
Maybe there could be further speed optimizations but it's already quite fast.

[Adub]

I thought that Ateme blew it out of the water, maybe not in terms of quality, but speed wise. Maybe I am just hearing voices again.

Also, I ask about hand optimizations because I am interesting in learning to program and speed is a fetish. Ha! :)

[imcold]

so, learn about programming. It's fun and it doesn't hurt - allright, debugging can be very frustrating, but the bigger will be your joy after you finnaly manage to make the code behave the way you want.
Large programs are broken into smaller parts: functions. When you want to make your program run faster, you search for the functions that are most computationally intensive and then you find a better (faster) algorithm for it, you rewrite it in assembly, or very often both. Compiler may not generate the best assembly code for your function -it's just a program, not thinking being- in contrast with creator of the code, who is perfectly aware of what the function is supposed to do. Optimalizations (most of the time?) take advantage of SIMD (single instruction - multiple data) instructions: MMX, 3DNow!, SSE1/2/3 aso. They are kind of... parallel processing instructions, they provide huge speedup if properly used. X264 has plenty of optimized functions. It's kind of complicated for compiler to rewrite your function into SIMD instructions. So is for humans... but humans are intelligent, thinking beings, and creative too. You can find out an algorithm, that would work much better in SIMD. Compiler sort of... can't. That's why I believe humans will always (or for a very long time) be better for optimizing.

Also, about optimizing for some CPU - under this I understand using instructions that suit the CPU is best at: be it 3DNow for (old?) amds, SSE2 for core2, 64-bit instructions for 64bit processors on 64-bit OS etc. So in my eyes x264 is pretty much optimized for core2. Also I admire the programmers that take part in developing our beloved oss encoders and find them very skilled and inteligent (and I am most thankful for their work); I doubt that Intel's optimized code would be significantly faster - IF it would be faster at all.

sorry if this was kind of boring reading :D (also corrections welcomed). I also think that x264 is very fast taking it's complexicity (that makes me dizzy) into consideration (hey, it got faster by >30% during 2006 according to my tests). Big thankyou for the devs.

[pgb]

Hear, hear, imcold!

Anyone wanting to learn more about real programming and squeezing the most out of those clock cycles could do worse than read Don Knuth's magnum opus:

http://www-cs-faculty.stanford.edu/~knuth/taocp.html

And I'd like to add my thanks to the X264 developers too.

Happy holidays, everyone!

[Adub]

Thanks guys! I will give them a read when I have the time.

[dancho]

if you wanna learn more check this links:

Software optimization resources :http://www.agner.org/optimize/

Assembly Optimization Tips : http://www.mark.masmcode.com/

[Inventive Software]

Got a suggestion. Bear with me. :)

The progress indicator only updates every (total frames / 1000), thus for a 100000 frame movie, it updates every 100 frames. This is a little slow with some resolutions, so why not have it update the progress every frame and not by percentage? The current percentage could then go down to 2dp or 3dp instead of 1dp to accomodate the longer encodes, and be calculated each frame.

[check]

You can get something like that if you enable verbose mode.

[chadamir]

Sorry if this crosses over into feature request territory, but in the advanced dvd authoring forum we've been discussing codecs that are hd-dvd compliant. X264 is not. Some minor header changes need to be made to conform to spec. Is this being worked on? Fantastic work otherwise, certainly my choice for pc backups.

Heres the thread: http://forum.doom9.org/showthread.php?t=119392&page=4

[akupenguin]

No. I am not interested in HD-DVD or any other application that adds requirements beyond the H.264 standard for no good reason.

[chadamir]

But aren't the two data structures in the reference encoder?

[akupenguin]

You tell me. Can you make an HD-DVD compliant stream using JM? Does anyone even know what the constraints are?

[chadamir]

I guess the people on the forum who have the specs know. I do know JM supports those data structures which were said to be needed by those who have the specs. I thought JM only outputs avi and I have a feeling demuxing would ruin it.

[Manao]

I guess the people on the forum who have the specs knowNo. They have only a vague idea of what some of the constraints may be. Those who effectively have the specs are most probably under NDA and can't disclose them ( CF : http://forum.doom9.org/showthread.php?p=882932#post882932 )

[Sagittaire]

No. I am not interested in HD-DVD or any other application that adds requirements beyond the H.264 standard for no good reason.

Well it's really strange ... ???

It's like say "I am not interested in DVD or any other application that adds requirements beyond the MPEG2 standard for no good reason"

Actually all the MPEG2 encoder without DVD compliant support are completely useless. Tomorrow all the AVC encoder without HDDVD/BD compliant support will be completely useless ...

[bond]

in the end we simply need an avc stream that is hddvd compliant and analyse it in detail

i till now didnt see such a stream

[Manao]

Trying to second guess Loren : you want it, do it yourself, and he'll commit it.

Adding HDDVD support to x264 is boring, dull, and technically potentially illegal(yet) since the specifications don't seem to be public. There are much more interesting things to do in x264 than that ( for one, interlacing support ( half ironic *cough* ) ). Personnal motivation is still the principal incentive when coding for open source software.

Anyway, Loren must feel a bit lonely lately, so if somebody who was actually interested in HD DVD were to add such a fonctionnality, I think he'd be more than welcome.

Actually all the MPEG2 encoder without DVD compliant support are completely uselessDVD compliancy has nothing to do with the mpeg2 bitstream, so any encoder that respect the VBV buffer is 'DVD compliant'. Requirements put by HD DVD on the h264 bitstream are much more invasive and complicated, and are best handled in the encoder itself.

[vsv]

in the end we simply need an avc stream that is hddvd compliant and analyse it in detail

i till now didnt see such a stream

Just decrypt any HD-DVD released in Japan by backupHDDVD
http://forum.doom9.org/showthread.php?t=119871

and demux streams with HD DVD Demuxer
http://dvd-logic.com/hddemuxer.htm

[chadamir]

I will post a short hd-dvd image made with an avc for all to enjoy.

The following is a sample video file and an authored disc using it.
It was encoded with mainconcept H264 encoder version 2.1 and
sonic scenarist 4. It is 1920x1080i encoded as interlaced.


http://download.yousendit.com/6F2F833F3F48859A
Only 100 downloads so don't send it around kthnx.

[chadamir]

Using Inlet semaphore I've gotten the header. I'll now encode the same file using x264 at 4.1 profile and see how it differs

Summary
Source file : C:\Target\Untitled\0\emotion1.mpv
Width : 1920
Height : 1088
Frame rate : 59.9402 fps
FourCC :
Total frames : 0
File duration : 00:00:00:00
File bit rate : 0 bps
Video bit rate : 0 bps
Encode : CBR
Target bit rate : 8,000,000 bps
Buffer window : 2500 ms
Max key frame : 0 ms
Min quant : 0
Max quant : 0

Sequence Parameter Set
Field Value Description
profile_idc 100 High Profile
constraint_set0_flag 0
constraint_set1_flag 0
constraint_set2_flag 0
constraint_set3_flag 0
reserved_zero_4bits 0
level_idc 41 Level 4.1
seq_parameter_set_id 0
chroma_format_idc 1
bit_depth_luma_minus8 0
bit_depth_chroma_minus8 0
lossless_qpprime_y_zero_flag 0
seq_scaling_matrix_present_flag 0
log2_max_frame_num_minus4 4
pic_order_cnt_type 0
...log2_max_pic_order_cnt_lsb_minus4 4
num_ref_frames 4 Maximum number of reference frames for inter-prediction
gaps_in_frame_num_value_allowed_flag 0
pic_width_in_mbs_minus1 119 (1920)
pic_height_in_map_units_minus1 33 (544)
frame_mbs_only_flag 0 Frame or field macroblocks
...mb_adaptive_frame_field_flag 0
direct_8x8_inference_flag 1
frame_cropping_flag 1
...frame_crop_left_offset 0
...frame_crop_right_offset 0
...frame_crop_top_offset 0
...frame_crop_bottom_offset 2 (8)
vui_parameters_present_flag 1
...aspect_ratio_info_present_flag 1
.......aspect_ratio_idc 1 1:1
...overscan_info_present_flag 0
...video_signal_type_present_flag 1
.......video_format 2 NTSC
.......video_full_range_flag 0 luma/chroma range = 219/224
.......colour_description_present_flag 1
...........colour_primaries 1 ITU-R BT.709
...........transfer_characteristics 1 ITU-R BT.709
...........matrix_coefficients 1 ITU-R BT.709
...chroma_loc_info_present_flag 0
...timing_info_present_flag 1
.......num_units_in_tick 1001
.......time_scale 60000
.......fixed_frame_rate_flag 1
...nal_hrd_parameters_present_flag 1
.......cpb_cnt_minus1 0
.......bit_rate_scale 1
.......cpb_size_scale 3
.......bit_rate_value_minus1 1 62499 (8000000 bits/sec)
.......cpb_size_value_minus1 1 43399 (5555200 bits)
.......cbr_flag 1 0
.......initial_cpb_removal_delay_length_minus1 31
.......cpb_removal_delay_length_minus1 17
.......dpb_output_delay_length_minus1 17
.......time_offset_length 24
...vcl_hrd_parameters_present_flag 0
...low_delay_hrd_flag 0
...pic_struct_present_flag 1
...bitstream_restriction_flag 1
.......motion_vectors_over_pic_boundaries_flag 1 Motion vectors may exceed picture boundary
.......max_bytes_per_pic_denom 0
.......max_bits_per_mb_denom 0
.......log2_max_mv_length_horizontal 10 +/- 256
.......log2_max_mv_length_vertical 10 +/- 256
.......num_reorder_frames 1
.......max_dec_frame_buffering 4 Maximum frame decode buffers required

Edit:

The same file then demuxed from the EVO and put through semaphore

Source file : C:\0\VTS_001\Titles\t001_v001c001.m4v
Width : 1920
Height : 1088
Frame rate : 59.9402 fps
FourCC :
Total frames : 0
File duration : 00:00:00:00
File bit rate : 0 bps
Video bit rate : 0 bps
Encode : CBR
Target bit rate : 8,000,000 bps
Buffer window : 2500 ms
Max key frame : 0 ms
Min quant : 0
Max quant : 0

Sequence Parameter Set
Field Value Description
profile_idc 100 High Profile
constraint_set0_flag 0
constraint_set1_flag 0
constraint_set2_flag 0
constraint_set3_flag 0
reserved_zero_4bits 0
level_idc 41 Level 4.1
seq_parameter_set_id 0
chroma_format_idc 1
bit_depth_luma_minus8 0
bit_depth_chroma_minus8 0
lossless_qpprime_y_zero_flag 0
seq_scaling_matrix_present_flag 0
log2_max_frame_num_minus4 4
pic_order_cnt_type 0
...log2_max_pic_order_cnt_lsb_minus4 4
num_ref_frames 4 Maximum number of reference frames for inter-prediction
gaps_in_frame_num_value_allowed_flag 0
pic_width_in_mbs_minus1 119 (1920)
pic_height_in_map_units_minus1 33 (544)
frame_mbs_only_flag 0 Frame or field macroblocks
...mb_adaptive_frame_field_flag 0
direct_8x8_inference_flag 1
frame_cropping_flag 1
...frame_crop_left_offset 0
...frame_crop_right_offset 0
...frame_crop_top_offset 0
...frame_crop_bottom_offset 2 (8)
vui_parameters_present_flag 1
...aspect_ratio_info_present_flag 1
.......aspect_ratio_idc 1 1:1
...overscan_info_present_flag 0
...video_signal_type_present_flag 1
.......video_format 2 NTSC
.......video_full_range_flag 0 luma/chroma range = 219/224
.......colour_description_present_flag 1
...........colour_primaries 1 ITU-R BT.709
...........transfer_characteristics 1 ITU-R BT.709
...........matrix_coefficients 1 ITU-R BT.709
...chroma_loc_info_present_flag 0
...timing_info_present_flag 1
.......num_units_in_tick 1001
.......time_scale 60000
.......fixed_frame_rate_flag 1
...nal_hrd_parameters_present_flag 1
.......cpb_cnt_minus1 0
.......bit_rate_scale 1
.......cpb_size_scale 3
.......bit_rate_value_minus1 1 62499 (8000000 bits/sec)
.......cpb_size_value_minus1 1 43399 (5555200 bits)
.......cbr_flag 1 0
.......initial_cpb_removal_delay_length_minus1 31
.......cpb_removal_delay_length_minus1 17
.......dpb_output_delay_length_minus1 17
.......time_offset_length 24
...vcl_hrd_parameters_present_flag 0
...low_delay_hrd_flag 0
...pic_struct_present_flag 1
...bitstream_restriction_flag 1
.......motion_vectors_over_pic_boundaries_flag 1 Motion vectors may exceed picture boundary
.......max_bytes_per_pic_denom 0
.......max_bits_per_mb_denom 0
.......log2_max_mv_length_horizontal 10 +/- 256
.......log2_max_mv_length_vertical 10 +/- 256
.......num_reorder_frames 1
.......max_dec_frame_buffering 4 Maximum frame decode buffers required

[chadamir]

--pass 2 --bitrate 8000 --stats ".stats" --level 4.1 --ref 4 --bframes 2 --no-b-adapt --direct temporal --analyse p8x8,b8x8,i4x4 --vbv-maxrate 768 --threads auto --thread-input --progress --no-psnr --no-ssim --interlaced --output "C:\Documents and Settings\Chad\Desktop\x264header2.264" "C:\Documents and Settings\Chad\Desktop\x264header.avs"

Source file : C:\Documents and Settings\Chad\Desktop\x264header2.264
Width : 1920
Height : 1088
Frame rate : 59.9402 fps
FourCC :
Total frames : 249
File duration : 00:00:04:08
File bit rate : 31,670,915 bps
Video bit rate : 17,685,959 bps
Encode : CBR
Target bit rate : 4,000,000 bps
Buffer window : 2500 ms
Max key frame : 0 ms
Min quant : 17
Max quant : 27

Sequence Parameter Set
Field Value Description
profile_idc 77 Main Profile
constraint_set0_flag 0
constraint_set1_flag 1 Main profile constraints (clause A.2.2)
constraint_set2_flag 0
constraint_set3_flag 0
reserved_zero_4bits 0
level_idc 41 Level 4.1
seq_parameter_set_id 0
log2_max_frame_num_minus4 5
pic_order_cnt_type 0
...log2_max_pic_order_cnt_lsb_minus4 6
num_ref_frames 5 Maximum number of reference frames for inter-prediction
gaps_in_frame_num_value_allowed_flag 0
pic_width_in_mbs_minus1 119 (1920)
pic_height_in_map_units_minus1 33 (544)
frame_mbs_only_flag 0 Frame or field macroblocks
...mb_adaptive_frame_field_flag 1
direct_8x8_inference_flag 1
frame_cropping_flag 1
...frame_crop_left_offset 0
...frame_crop_right_offset 0
...frame_crop_top_offset 0
...frame_crop_bottom_offset 2 (8)
vui_parameters_present_flag 1
...aspect_ratio_info_present_flag 0
...overscan_info_present_flag 0
...video_signal_type_present_flag 0
...chroma_loc_info_present_flag 0
...timing_info_present_flag 1
.......num_units_in_tick 1001
.......time_scale 60000
.......fixed_frame_rate_flag 1
...nal_hrd_parameters_present_flag 0
...vcl_hrd_parameters_present_flag 0
...pic_struct_present_flag 0
...bitstream_restriction_flag 1
.......motion_vectors_over_pic_boundaries_flag 1 Motion vectors may exceed picture boundary
.......max_bytes_per_pic_denom 0
.......max_bits_per_mb_denom 0
.......log2_max_mv_length_horizontal 11 +/- 512
.......log2_max_mv_length_vertical 11 +/- 512
.......num_reorder_frames 1
.......max_dec_frame_buffering 5 Maximum frame decode buffers required

There are some discrepancies between my settings and the header. I ended up with 5 reference frames instead of 4. And I chose high profile in megui. Also there's all the header info that's simply not there.

[chadamir]

HD-DVD profile from megui

Sequence Parameter Set
Field Value Description
profile_idc 77 Main Profile
constraint_set0_flag 0
constraint_set1_flag 1 Main profile constraints (clause A.2.2)
constraint_set2_flag 0
constraint_set3_flag 0
reserved_zero_4bits 0
level_idc 41 Level 4.1
seq_parameter_set_id 0
log2_max_frame_num_minus4 5
pic_order_cnt_type 0
...log2_max_pic_order_cnt_lsb_minus4 6
num_ref_frames 5 Maximum number of reference frames for inter-prediction
gaps_in_frame_num_value_allowed_flag 0
pic_width_in_mbs_minus1 119 (1920)
pic_height_in_map_units_minus1 33 (544)
frame_mbs_only_flag 0 Frame or field macroblocks
...mb_adaptive_frame_field_flag 1
direct_8x8_inference_flag 1
frame_cropping_flag 1
...frame_crop_left_offset 0
...frame_crop_right_offset 0
...frame_crop_top_offset 0
...frame_crop_bottom_offset 2 (8)
vui_parameters_present_flag 1
...aspect_ratio_info_present_flag 0
...overscan_info_present_flag 0
...video_signal_type_present_flag 0
...chroma_loc_info_present_flag 0
...timing_info_present_flag 1
.......num_units_in_tick 1001
.......time_scale 60000
.......fixed_frame_rate_flag 1
...nal_hrd_parameters_present_flag 0
...vcl_hrd_parameters_present_flag 0
...pic_struct_present_flag 0
...bitstream_restriction_flag 1
.......motion_vectors_over_pic_boundaries_flag 1 Motion vectors may exceed picture boundary
.......max_bytes_per_pic_denom 0
.......max_bits_per_mb_denom 0
.......log2_max_mv_length_horizontal 11 +/- 512
.......log2_max_mv_length_vertical 11 +/- 512
.......num_reorder_frames 1
.......max_dec_frame_buffering 5 Maximum frame decode buffers required

[chadamir]

Ok thanks to bond we got the header to this point

profile_idc 100 High Profile
constraint_set0_flag 0
constraint_set1_flag 0
constraint_set2_flag 0
constraint_set3_flag 0
reserved_zero_4bits 0
level_idc 41 Level 4.1
seq_parameter_set_id 0
chroma_format_idc 1
bit_depth_luma_minus8 0
bit_depth_chroma_minus8 0
lossless_qpprime_y_zero_flag 0
seq_scaling_matrix_present_flag 0
log2_max_frame_num_minus4 1
pic_order_cnt_type 0
...log2_max_pic_order_cnt_lsb_minus4 2
num_ref_frames 4 Maximum number of reference frames for inter-prediction
gaps_in_frame_num_value_allowed_flag 0
pic_width_in_mbs_minus1 119 (1920)
pic_height_in_map_units_minus1 67 (1088)
frame_mbs_only_flag 1 Frame macroblocks only (no field)
direct_8x8_inference_flag 1
frame_cropping_flag 1
...frame_crop_left_offset 0
...frame_crop_right_offset 0
...frame_crop_top_offset 0
...frame_crop_bottom_offset 4 (8)
vui_parameters_present_flag 1
...aspect_ratio_info_present_flag 1
.......aspect_ratio_idc 1 1:1
...overscan_info_present_flag 0
...video_signal_type_present_flag 1
.......video_format 2 NTSC
.......video_full_range_flag 0 luma/chroma range = 219/224
.......colour_description_present_flag 1
...........colour_primaries 1 ITU-R BT.709
...........transfer_characteristics 1 ITU-R BT.709
...........matrix_coefficients 1 ITU-R BT.709
...chroma_loc_info_present_flag 0
...timing_info_present_flag 1
.......num_units_in_tick 1001
.......time_scale 60000
.......fixed_frame_rate_flag 1
...nal_hrd_parameters_present_flag 0
...vcl_hrd_parameters_present_flag 0
...pic_struct_present_flag 0
...bitstream_restriction_flag 1
.......motion_vectors_over_pic_boundaries_flag 1 Motion vectors may exceed picture boundary
.......max_bytes_per_pic_denom 0
.......max_bits_per_mb_denom 0
.......log2_max_mv_length_horizontal 11 +/- 512
.......log2_max_mv_length_vertical 11 +/- 512
.......num_reorder_frames 1
.......max_dec_frame_buffering 4 Maximum frame decode buffers required

But no NAL HRD.

[Inventive Software]

You can get something like that if you enable verbose mode.

Tried that, didn't give what I wanted. Essentially, a better progress indicator is what I'd like, if that's not too much to ask. I'd do it myself, but my coding's non-existant (practically).

[akupenguin]

It's like say "I am not interested in DVD or any other application that adds requirements beyond the MPEG2 standard for no good reason"
Exactly. If I were writing an MPEG2 encoding library, I wouldn't put anything DVD-specific in it. Of course, profile/level/VBV/GOP/etc are all configurable, but it's up to the application or user to configure them appropriately.

So: If you determine that HDDVD needs some H.264 feature like maybe a specific SEI message or one of the optional fields in the SPS, then feel free to implement it and send a patch. But I will reject any application-specific kludges, just like I rejected magiK's PSP-mode patch.

There are some discrepancies between my settings and the header. I ended up with 5 reference frames instead of 4. And I chose high profile in megui. Also there's all the header info that's simply not there.
x264's --ref selects the number of L0 references used by P-frames, not necessarily the DPB size. (Maybe someday I'll add a mode to optimize for max quality per DBP size instead of per encoding time/complexity.)
x264 signals the lowest profile that allows all your enabled features. You can't select high profile without enabling some feature that needs high profile.

[hpn]

For those who like patches I just got around to adding something small I've always thought was missing while encoding - total time on the progress indicator: patch (http://x264.net/x264_clock1-614.diff)

The progress indicator only updates every (total frames / 1000), thus for a 100000 frame movie, it updates every 100 frames. This is a little slow with some resolutions
Also in the patch - up to 10'000 times per input file seems more enjoyable with negligible speed penalty (a second/hour or something).

[Malow]

if someone need another hd-dvd compilant h.264 ES, go here:

http://forum.doom9.org/showthread.php?p=919921#post919921

[pyrates]

came across this strange error, here's the output:

C:\>x264.exe --pass 2 --bitrate 4665 --stats "L:\movie\108
0i.stats" --ref 16 --mixed-refs --no-fast-pskip --bframes 3 --b-pyramid --b-rdo
--bime --weightb --direct auto --direct-8x8 -1 --filter -2,-1 --subme 7 --trelli
s 1 --analyse all --8x8dct --vbv-maxrate 25000 --me umh --threads 6 --non-determ
inistic --cqmfile "E:\eqm_avc_hr.cfg" --progress -
-no-psnr --output "L:\movie\1080i.264" "L:\movie\1080i.a
vs"
avis [info]: 1920x1080 @ 23.98 fps (172975 frames)
x264 [warning]: width or height not divisible by 16 (1920x1080), compression wil
l suffer.
x264 [info]: using cpu capabilities MMX MMXEXT SSE SSE2 3DNow!
x264 [warning]: VBV maxrate specified, but no bufsize.
mb type: 7 mes: 10552/172975 (6.1%), 0.50 fps, eta 89:44:29
mv: l1r0 (115,407)
limit: 352
mb_xy: 1,11
completed: 264
Assertion failed: 0, file encoder/analyse.c, line 2714

This application has requested the Runtime to terminate it in an unusual way.
Please contact the application's support team for more information.

I am using the MT plugin for avisynth 2.5.6a with the following avisynth script:

SetMTMode(6,0)
# Modify the path so that it finds your decomb dll file #
LoadPlugin("E:\DeComb\decomb521.dll")

# Modify the path so that it finds your MPEG2DEC dll file #
LoadPlugin("E:\DGMPGDec\DGDecode.dll")


LoadPlugin("E:\MKVMagic\exe\filter\Convolution3D.dll")
LoadPlugin("E:\MKVMagic\exe\filter\deen.dll")
LoadPlugin("E:\MKVMagic\exe\filter\MSharpen.dll")
LoadPlugin("E:\MKVMagic\exe\filter\tdeint.dll")
LoadPlugin("E:\MKVMagic\exe\filter\UnFilter.dll")
LoadPlugin("E:\MKVMagic\exe\filter\undot.dll")

import("E:\MvBob\mvbob.avs")
import("E:\AVS Scripts\convert60ito24p.avsi")
Import("E:\MKVMagic\exe\filter\HybridFuPP.avsi")

# Modify the path so that it finds the d2v file you created using dvd2avi #
MPEG2Source("L:\movie\movie.d2v")
#audio=wavSource("E:\24.1.wav")
#audiodub(video,audio)
#Crop(left, top, -right, -bottom)
Crop(0, 0, -0, -0)

# Assume Top Field
#AssumeTFF().SeparateFields()
#Telecide(order=1)
# Assume Bottom Field
#AssumeBFF().SeparateFields()
Telecide(order=0)

# Reduce down to 23.976 fps
Decimate()

Undot()
Convolution3d(preset="movieHQ")

HybridFuPP(preset="High")

#assumefps(23976,1000)
assumefps(24000,1001)

I'm using build 614 of x264. The error occurs quite randomly I find but it is always the same error referencing the same line. Anyone know what it means?

[akupenguin]

It means that somehow the motion estimation failed to limit itself to the available range, and ended up using pixels that might not yet have been encoded by another thread.

[bkman]

Akupenguin, are you aware of the problem with direct B-frame modes and AQ or certain custom matrices? They can currently cause artifacts in some scenes unless the direct mode is set to "none".

Hopefully you can fix it, as disabling the direct B-frame modes wastes bitrate.

[pyrates]

It means that somehow the motion estimation failed to limit itself to the available range, and ended up using pixels that might not yet have been encoded by another thread.

So does this mean it is a bug and can be fixed?

[ChronoCross]

Akupenguin, are you aware of the problem with direct B-frame modes and AQ or certain custom matrices? They can currently cause artifacts in some scenes unless the direct mode is set to "none".

Hopefully you can fix it, as disabling the direct B-frame modes wastes bitrate.

AQ is an unsupported patch. As for custom matirces........any examples?

[bkman]

*.mp4 guy mentions that the problem occurs with some of his matrices in his Custom Matrix topic.

[akupenguin]

Akupenguin, are you aware of the problem with direct B-frame modes and AQ or certain custom matrices? They can currently cause artifacts in some scenes unless the direct mode is set to "none".
No, I am not aware of any such. I have heard some people suggest disabling B-direct as a last-ditch attempt to remove some artifacts, but I don't recall any cases where it actually worked. I always figured it was similar to people not using B-frames in XviD: a mistaken assumption that anything that improves compression ratio shouldn't be used if you want very high quality.

Artifacts related to cqm (assuming they're not a direct consequence of the matrix's frequency distribution) are probably due to the cqm deviating too far from the bitrate-per-qp values of the flat matrix. x264 makes some assumptions about the quality level of a given qp, and I'm not likely to work very hard at generalizing them for all cqms when you can always scale a cqm to match the flat matrix. (That and I'm not convinced that cqms are useful in the first place.)

[*.mp4 guy]

To be fair X264 does have trouble with b direct at times even with the standard matrix (though I haven't had any trouble in this reguard in a long time, so it may have been fixed). I'm also not sure that the problems are completely related to quality/qp, I think X264 may have trouble with cqm's that have a low dc coeficient even if they are similar in quality/qp to the flat matrix(just a guess though). I probably should mention that I have nothing against Bframes, or space saving features, I always use bframes in Xvid and X264, and whenevere possible I use B direct.

B Direct usualy only has trouble on smooth low contrast sources like the new starwars movies and you need a good crt (or any lcd with high contrast) to notice the problems, since they usually manifest as jittery flat blocks in almost flat areas.

As to matrices usefulness, all I have to say to that is banding, and low level detail loss (I don't care about noise loss personally). The flat matrix just isn't good at preventing banding and its reliance on inloop filtering to remove blocks removes small details aswell, though it preserves high frequencies amazingly well. Until the flat matrix stops banding and stops needing high inloop settings to have normal blocking levels (which might happen) matrices will be usefull to people who dislike banding and the "washed out" look inloop can cause more then ringing and slight blurring of prominent details.

I would like to have band free encodes with the standard matrix, but right now it isn't happening and cqm's are the best solution I'm aware of.

[*.mp4 guy]

Update: here is an example (http://www.megaupload.com/?d=1KGU9XJR)
This example is a lot more blatant then usual, the clouds in the beginning really seam to throw something off, though there are some (much less obvious) artifacts in other parts of the clip.

heres the matrix used:

INTRA4X4_LUMA =
15,6,15,22,
6,7,25,30,
15,25,38,48,
22,30,48,96

INTRA4X4_CHROMAU =
16,6,24,72,
6,8,41,144,
24,41,96,200,
72,144,200,255

INTRA4X4_CHROMAV =
16,6,24,72,
6,8,41,144,
24,41,96,200,
72,144,200,255

INTER4X4_LUMA =
4,11,15,22,
11,11,25,30,
15,25,38,48,
22,30,48,96

INTER4X4_CHROMAU =
5,16,16,16,
16,16,16,56,
16,16,72,96,
16,56,96,128

INTER4X4_CHROMAV =
5,16,16,16,
16,16,16,56,
16,16,72,96,
16,56,96,128

INTRA8X8_LUMA =
24,7,7,12,15,20,21,23
8,9,14,15,18,21,23,21
10,12,15,17,17,21,26,20
14,15,17,17,18,33,30,23
15,17,17,21,26,39,39,29
18,18,20,24,33,39,41,35
21,24,29,34,39,45,45,38
27,35,36,39,38,38,38,38

INTER8X8_LUMA =
6,10,13,13,15,16,18,22
10,10,12,15,15,16,21,24
13,12,15,16,19,21,25,27
13,15,16,21,24,28,30,34
15,15,19,24,30,36,37,40
16,16,21,28,36,42,45,52
18,21,25,30,37,45,55,72
22,24,27,34,40,52,72,96
I was actually trying to make the matrix more "compatible" to x264 by getting it closer to the compressability of the standard matrix... Doesn't seem to have worked. Its not spot on with the standard matrice's compressability, but I don't think it is far enough off to justify these results.

I'm currently making an encode without b predict to see if that fixes the problem as it has for me in the past.

[akupenguin]

also try with and without b-rdo

[*.mp4 guy]

The problem clip used b rdo, when I tested the clip with b predict disabled the problem went away. Right now I am working on getting a lossless file of a subset of the clip to exhibit the same behavior, after that I will test the clip with b predict enabled but without b rdo. I'll update this post when I am done.

[edit]
Quick summary of results
938KB B-rdo+B=predict enabled: prominent noticible artifacts
1210KB B-rdo disabled, B predict enabled: slight jerkiness of motion in low contrast areas, probably justified by space savings, ie everything is fine
1321KB B-rdo enabled, B predict disabled: everything is fine, however worse quality/space ratio then with b prediction enabled and b-rdo disabled

Here is a link (http://www.megaupload.com/?d=GYJ2Y6XH) with everything you need to reproduce the results.

[elguaxo]

I would be interested in doing a similar test like yours, but targeting the same filesize for all combinations. However I can't download anything from megaupload: All download slots assigned to your country (Argentina) are currently in use.

Could you upload the files somewhere else? Thanks!

[*.mp4 guy]

Files uploaded somewhere else (http://www.mytempdir.com/1146936).

[elguaxo]

Thanks, the new link works fine!

My eyes are not so fine-tuned like yours, I can spot the artifacts, but it was difficult.

M4G HRM V1.5 is the matrix some posts above?

[*.mp4 guy]

M4G HRM V1.5 is the matrix some posts above?
Yes, I probably should have more clear on that or included it in the archive to make things easier. I still have to do more testing to see how it compares to the old version, and I have to adjust the 4x4 luma matrices before it will be finnished.



My eyes are not so fine-tuned like yours, I can spot the artifacts, but it was difficult.
Its probably your monitor, not your eyes that made the problem hard to spot. The monitor I'm using is very unforgiving of artifacts compared to most crts.

[noclip]

I'm not a big fan of x264's logo (the firey one). It makes the best (IMHO) AVC encoder out there look second rate to somebody not well-versed in such things. Here's a slightly cleaner version which, if you want to use it, you can feel free to as it is in the public domain.

http://img340.imageshack.us/img340/5628/x264largedr4.png

[akupenguin]

The firey one is not x264's logo, it's x264.nl's logo. There is no official x264 logo.
Of the logos proposed so far, my favorite is the old vfw icon, though I never saw a large render of it.
http://img128.imageshack.us/img128/1286/x264icontb3.png

[noclip]

The firey one is not x264's logo, it's x264.nl's logo. There is no official x264 logo.

Ah. Well as I said, public domain, so if you ever feel like you need one...

[bob0r]

The VFW icon is just the same logo noclip posted, but with the 3 colors added.

So making a big render is quite easy.



... i have a whole week all for myself, i was thinking of a new design for x264.nl, but as usual i lack any graphical insight what so ever! :o

I was thinking a more dark style like:
http://x264.nl/x264_blue.png
But this is just fooling around.

[poyupay]

The VFW icon is just the same logo noclip posted, but with the 3 colors added.

yeah right! looks like that:

http://img329.imageshack.us/img329/2084/draft1fi5.png

or maybe like that...:

http://img329.imageshack.us/img329/4840/draft2er4.png

only drafts though. but if you're interested... :D

[AGDenton]

Hi,

[631] introduces a few SSE3-optimized functions for the x86_64 branch. I'm on a 386 right now, so I can't test them, but could you provide an estimate of their performance impact?

Also, there seems to be a few processors out there (Yonah, for instance) which support SSE3 but not EM64T : would SSE3 optimizations in the i386 branch be feasible, or benefitial ?

Best regards,

AG

[Manao]

rev631 introduces SSSE3 optimized functions, not SSE3. AFAIK, all SSSE3 processors are 64bits. Still, some people might want to use a 32 bits OS and a SSSE3 version would be appreciated for them.

It would be feasible and benefitial, you just need to find the guy to do it.

[Cef]

Hi,

I made x264 ssse3 builds since i couldn't find any, sharing if anyone interested
http://cef.neuf.fr/x264/x264_x86_r637.7z
http://cef.neuf.fr/x264/x264_x64_r637.7z

[Adub]

Thanks mate!

Yo, Sharktooth!
Are you making builds that support ssse3 as well, or are you sticking with the lesser optimizations?

[ChronoCross]

Thanks mate!

Yo, Sharktooth!
Are you making builds that support ssse3 as well, or are you sticking with the lesser optimizations?


ssse3 only works on intel processors.

[Eretria-chan]

ssse3 only works on intel processors.

Does it? AMD Athlon 64 X2 supports SSE3. Cpu-Z confirms this. SSE4 is Intel-only for the moment, though, AFAIK.

[Manao]

Electria-chan : SSE4 has been renamed SSSE3. So ChronoCross is right.

ChronoCross: strange, I was getting the impression that SharkTooth was an Intel fanboy :p

[Adub]

No, he is an AMD fanboy.
Shucks, does anyone know where we can get patched ssse3 builds?

[Romario]

Electria-chan : SSE4 has been renamed SSSE3. So ChronoCross is right.

Noo, SSE3 is SSE3, SSE4 is future instruction set designed for Penryn arhitecture (45 nm shrink of Core 2 Duo and Core 2 Quad).

[Manao]

Romario : count the number of Ss and check on the internet. But don't confuse users anymore than necessary when you're wrong - and wrong you are. MMX < SSE < SSE2 < SSE3 < SSSE3 < SSE4, and SSE4 doesn't yet exist, period.

[Adub]

Okay, now that that has been settled, does anyone know if their will be other ssse3 builds besides the two provided by our good man Cef? I am hoping for the patches that Sharktooth usually includes as well.

[Bigmango]

Hi,

I made x264 ssse3 builds since i couldn't find any, sharing if anyone interested
http://cef.neuf.fr/x264/x264_x86_r637.7z
http://cef.neuf.fr/x264/x264_x64_r637.7z

Wow thanks !!

Please keep them coming ;)

[Cef]

Please keep them coming ;)
Here goes
x264_x86_r644.7z (http://cef.neuf.fr/x264/x264_x86_r644.7z)
x264_x64_r644.7z (http://cef.neuf.fr/x264/x264_x64_r644.7z)
and source (http://cef.neuf.fr/x264/x264_src_r644.7z)

Okay, now that that has been settled, does anyone know if their will be other ssse3 builds besides the two provided by our good man Cef? I am hoping for the patches that Sharktooth usually includes as well.

I guess I could add them, if he doesn't mind.

[Adub]

That would be great!

Edit: Although, now that I think about it, are the patches really necessary? I mean a majority of them have already been incorporated into x264. Does anyone know if the patches compiled by Sharktooth actually make that big of a difference in quality?


Thanks a lot Cef, and know that you are always appreciated here.

[foxyshadis]

The signature patch is trivial, to increase MeGUI compatibility iirc, but the AQ patch is quite important to a lot of people.

An SSSE3 build will only make sense on x64, since the SSSE3 code is only currently enabled there. For optimizing the C code, well, it's not worth a special compile just to print the options list with SSSE3.

[Cef]

I enabled ssse3 in 32bit too, basically copy/paste with few passing parameters change. Except for x264_pixel_sa8d_8x8_ssse3/x264_pixel_sa8d_16x16_ssse3, which use way too many xmm registers to be converted to 32bit.

[Adub]

Thanks alot Cef!
I have yet to use your latest build, but your older one actually fixed a problem I was having with the previous X264 builds. And I think (it may have been my imagination) it was 2 fps faster.
Double thanks!

[Cef]

np at all. I made r644 builds with aq patch, just take previous links and replace 640 with 644. Both my 32 and 64bit builds are faster than anything I could find, on my core 2 duo. So if you have a C2D it's probably not your imagination. I think we're off-topic though, sorry about the spam >.<

[foxyshadis]

Nah, it's on-topic enough for a development thread. If you enabled all the satd & quant functions, did you submit a patch to akupenguin or the list? I'm sure he'd commit it. (You can edit new links into your old posts, if you want to, btw.)

[Cef]

did you submit a patch to akupenguin or the list?
Do you mean x264-devel (http://www.videolan.org/developers/lists.html) ? Quant changed in rev642 and is both 32 and 64bit now anyway, I could do that for satd I guess.

[Adub]

Yeah, I am using a core 2 duo, and I can definetly see a slight speed increase. I am downloading you latest build now.

[squid_80]

The 64-bit build doesn't run at all, it crashes immediately on startup. Did you check the code to see if it was win64 compatible?

[Cef]

which one? 644 runs just fine here. There was a bug in previous builds, coming from you actually :p movdqa rsp+16 with stack misaligned in x264_pixel_ssim_end4_sse2. Btw much props for your work on win64 asm :thanks:

[Bigmango]

The 645 build from x264.nl crashes every time after about 5 frames. Does anyone have the same experience ?

I am back to Cef's 644 build.

[LoRd_MuldeR]

Maybe it's related to this?
http://forum.doom9.org/showthread.php?p=979857#post979857

[Bigmango]

Maybe it's related to this?
http://forum.doom9.org/showthread.php?p=979857#post979857

Hmmm... I don't know. I am having this problem with the CLI version; the crash happens with the official x264.nl 645 exe.

But Cef's 644 exe works fine, both are using sse2... :confused:

[akupenguin]

The crash (http://forum.doom9.org/showthread.php?p=979857#post979857) is due to a bug in gcc. Cef built a win64 version, so he can't have been using gcc.

[Bigmango]

The crash (http://forum.doom9.org/showthread.php?p=979857#post979857) is due to a bug in gcc. Cef built a win64 version, so he can't have been using gcc.

Ok, thx for the info.

Btw, I am using Cef's x86 version.

[LoRd_MuldeR]

Bigmango, try Gruntster's rev645 build with the new fix included:
http://www.razorbyte.com.au/x264

[Bigmango]

Bigmango, try Gruntster's rev645 build with the new fix included:
http://www.razorbyte.com.au/x264

Oh, it's the lib; so I can try avidemux again :)

:thanks:

[squid_80]

which one? 644 runs just fine here. There was a bug in previous builds, coming from you actually :p movdqa rsp+16 with stack misaligned in x264_pixel_ssim_end4_sse2. Btw much props for your work on win64 asm :thanks:
I had r637, it was crashing in exactly that spot. 644 runs good. Not sure how I came up with [rsp+16] which presumably would never have worked.
Also the handle leak I was getting when using multiple threads seems to be gone; did you do something to fix it, has pthreads been updated since I made a build, or did the problem just mysteriously go away?

[bob0r]

When the patch is applied to the svn, i will remove the notes and everything should be ok again.

So hopefully in the next revision its all ok again.

Keep testing, using and reporting! :thanks:

[Cef]

Not sure how I came up with [rsp+16] which presumably would never have worked.
Maybe you never noticed it, with my usual settings the bug wasn't triggered, I happened to find it while running x264 with no options for a quick test.

Also the handle leak I was getting when using multiple threads seems to be gone; did you do something to fix it, has pthreads been updated since I made a build, or did the problem just mysteriously go away?
Actually it's not using pthread, just some #define's falling back on win32 threads, cause I had trouble getting pthread to work, but now I got it to work, please check x264_x64_r645.7z (http://cef.neuf.fr/x264/x264_x64_r645.7z)

[squid_80]

Can't open that .7z file: winrar 3.62 reports unknown format or damaged. Same with 7-zip 4.44 beta.

[Cef]

Sorry about that, try again.

[squid_80]

Got it, shows the same handle leak that I was seeing with before. :(

[Cef]

I think I found the problem, some inline asm in ptw32_InterlockedCompareExchange.c was ignored when compiling pthread for amd64. Should be gone now : x264_x64_r648 (http://cef.neuf.fr/x264/x264_x64_r648.7z)

[Terranigma]

I hope it works. x264, for me, seems to have been b0rken after revision 644 :(

---
Nope, doesn't even start up. =/

[Adub]

Cef, are you going to do a x86 build?

[Terranigma]

yes cef, when you do builds, could you do an x86 build as well? :D

[Bigmango]

I hope it works. x264, for me, seems to have been b0rken after revision 644 :(

---
Nope, doesn't even start up. =/

The latest 648 build works great for me. It seems the problem was fixed since build 646.

Did you try it ?

Here:
http://x264.nl/
http://mirror01.x264.nl/x264/revision648/x264.exe

[Terranigma]

The latest 648 build works great for me. It seems the problem was fixed since build 646.

Did you try it ?

Here:
http://x264.nl/
http://mirror01.x264.nl/x264/revision648/x264.exe


It works, but unfortunately I can't use adaptive quantization for dark scenes. Perhaps the commandline for aq changed? :confused:
I've made profiles using aq. =P


Anyways, i'd like to say that It seems x264's now twice as fast as before :D

[burfadel]

Adaptive quantisation is only an experimental patch apparently! The builds at x264.nl are pure svn builds. Although adaptive quantiser is experimental I find it works very well! it mustn't be quite ready though...

You have to search for the adaptive quantiser builds. Sharktooth makes them but he doesn't always have the time to stay updated, he's still with revision 635. CEF on this thread actually has the latest adaptive quantiser build... rev 645 for x86 and rev 648 for x64. Both work great! I'm sure he'll put up a rev 648 for x86 at some stage as people have asked him for it... He did a good job of implementing SSSE3 instructions! :)

It works, but unfortunately I can't use adaptive quantization for dark scenes. Perhaps the commandline for aq changed? :confused:
I've made profiles using aq. =P


Anyways, i'd like to say that It seems x264's now twice as fast as before :D

[Terranigma]

Thanks burfadel for the prompt reply. Look like i'll have to look forward to a build by cef :cool:

[Selur]

What about the one here:
http://forum.doom9.org/showpost.php?p=980523&postcount=130

[Terranigma]

that's for x64.

[Selur]

oh, you are right overlooked that sorry :)

[Romario]

About SSE3 implementation in recent x264 builds, can someone explain to me what parts of the code is changed?

Speed-ups?

[Manao]

SSS3, and, yes, it results in speed up.

[akupenguin]

There are only 2 places I found where SSSE3 could be used: satd uses PABS, and quantization uses PSIGN and PABS.
PHADD/PHSUB looked interesting, and they do reduce the code size, but at least on a current Core2 they're so much slower than PADD/PSUB that it's faster to transpose, column sum, transpose than to row sum.

[Manao]

Did you do deblocking already ? Or do you consider it not as important (speedwise) in an encoder as in a decoder.

[akupenguin]

Both: I didn't see anything in deblocking that would change, and there's already several deblocking optimizations in ffmpeg that I haven't bothered to port. Deblocking spends more time in C code (computing the filter strengths) than in asm (munging pixels).
There are some absolute value computations, but they can't be done with PABS because the intermediate signed value would take 9 bits, whereas the current code can get away without ever unpacking the pixels from 8 bits.

The horizontal 6tap filter would benefit from PALIGNR, but then it would probably benefit even more from SSE2 (currently it's only MMX). And again that's not in the inner loop when encoding.

[Cef]

Cef, are you going to do a x86 build?
Sorry, focused on this handle leakage which was 64-bit specific then I been busy till now. x264_x86_r648.7z (http://cef.neuf.fr/x264/x264_x86_r648.7z)

[Terranigma]

Sorry, focused on this handle leakage which was 64-bit specific then I been busy till now. x264_x86_r648.7z (http://cef.neuf.fr/x264/x264_x86_r648.7z)

Thanks a lot Cef. :devil:

[akapuma]

Sorry, focused on this handle leakage which was 64-bit specific then I been busy till now. x264_x86_r648.7z (http://cef.neuf.fr/x264/x264_x86_r648.7z)
Thank you very much.

Best regards

akapuma

[Bigmango]

Sorry, focused on this handle leakage which was 64-bit specific then I been busy till now. x264_x86_r648.7z (http://cef.neuf.fr/x264/x264_x86_r648.7z)

Your link does not work ?


Multiple Choices
The document name you requested (/x264/x264_x86_r648.7z) could not be found on this server. However, we found documents with names similar to the one you requested.

Available documents:

* /x264/x264_x86_r640.7z (mistyped character)
* /x264/x264_x86_r644.7z (mistyped character)
* /x264/x264_x86_r645.7z (mistyped character)

[burfadel]

Works for me! he must have fixed the link :)

Thanks Cef for making these builds, its much appreciated!

[squid_80]

I think I found the problem, some inline asm in ptw32_InterlockedCompareExchange.c was ignored when compiling pthread for amd64. Should be gone now : x264_x64_r648 (http://cef.neuf.fr/x264/x264_x64_r648.7z)
Looks perfect. Congrats on fixing it and thanks for doing x64 builds so I don't feel guilty about not keeping mine up-to-date.

[burfadel]

He also include the AQ patch which is great! :)

[Dannyboy007]

What's the differense if you rip a DVD with x264 in a MKV container or in a MP4 container? Is the quality better in MP4?

[bob0r]

What's the differense if you rip a DVD with x264 in a MKV container or in a MP4 container? Is the quality better in MP4?
http://x264.nl/jab.gif

[Bigmango]

What's the differense if you rip a DVD with x264 in a MKV container or in a MP4 container? Is the quality better in MP4?

The container has nothing to do with quality. It's only what it is... a "container".

The advantage with mkv, is that you can keep all the DVD sound tracks and subtitles. The disadvantage is that there is almost no hardware player support for mkv.

MP4 has more hardware player support, but compared to mkv, it has a lot of restrictions. You can't put DTS or AC3 tracks in MP4, and if I remember well, it is also limited to max 2 subtitles.

MKV is the better container.

[Dannyboy007]

I see. Thanks a lot Bigmango!

[bond]

in MP4, and if I remember well, it is also limited to max 2 subtitles.not true, you can place as much subs as you want in mp4

[Selur]

little feature request: option to stop x264 from exporting all the x264 encoding settings into the userdata.

[Terranigma]

little feature request: option to stop x264 from exporting all the x264 encoding settings into the userdata.

x264 Author's Response (http://forum.doom9.org/showthread.php?p=989015#post989015)

[Bigmango]

not true, you can place as much subs as you want in mp4

Ok, I was not sure on that one. Thx for correcting this.

[Sagittaire]

Possible to implement pulldown flags in x264 elementary stream ... ?

[Terranigma]

cef, any chance for a rev 651 build?
r651 | pengvado | 2007-04-21 13:32:34 +0200 (Sat, 21 Apr 2007) | 3 lines

cabac: use bytestream instead of bitstream.
35% faster cabac, 20% faster overall lossless, ~1% faster overall at normal bitrates.

[burfadel]

cef, any chance for a rev 651 build?

Make that 652 build :)

The aq patch is great! I wonder why it hasn't been included in the final build yet!

The thread patch sounds interesting, I wonder when that will be ready for inclusion!

Oh, and Cef you should let Sharktooth etc know what settings you use for your x264 compilation, because it seems to be way faster (same quality of course). Its not just a few percent either, it flies in comparison on my Core 2 Duo :)

[burfadel]

Cef's builds can be found on his x264 page here:

http://cef.neuf.fr/x264/

Hopefully he keeps up the good work and updates it soon with the latest build! He's currently got 650 up.

[e-Pawel]

The aq patch is great! I wonder why it hasn't been included in the final build yet!


What is that aq patch? What is its advantage???

[Terranigma]

What is that aq patch? What is its advantage???
Quote:
One part of the settings for Adaptive Quantisation. Adaptive Quantisation allows different parts of a frame to use different quantisers, as opposed to the whole frame using the same quantizer. The common use for this in XviD was that it would compress light and dark areas more, thus freeing bits for parts that were more noticable. Adaptive Quantisation works differently in x264 because it was added by Haali to address a problem that large blue areas could sometimes look blocky. Adaptive Quantisation in x264 assigns more bits to dark and blue areas to help stop them blocking up. This particular setting instructs the amount to adjust QP per Macroblock. Setting 0.0 is no Adaptive Quantisation, and 1.1 is considered strong. It is recommended that you only use this setting if your source is particularly blocky in blue areas only wheras the rest of the image looks fine.

[Selur]

little question about all the patches:

What has to happen for a patch to become part of the main branch?

Cu Selur

[Cef]

Cef's builds can be found on his x264 page here:

http://cef.neuf.fr/x264/

Hopefully he keeps up the good work and updates it soon with the latest build! He's currently got 650 up.

Thanks for noticing :) I updated to 654 today, sorry about the delay.

[LoRd_MuldeR]

little question about all the patches:

What has to happen for a patch to become part of the main branch?

Cu Selur

I guess the developers must be willing to accept and maintain the patch.
Just adding the patch to the SVN isn't enough, as future changes might break the patch.

[Terranigma]

Thanks for noticing :) I updated to 654 today, sorry about the delay.

Welcome back Cef, we missed you :D

[burfadel]

New build of x264 out, 656. Very small changes in regards to speed apparently, but still its the first update in a while!

[fight2win]

how to update x264 in megui to latest version???

[Terranigma]

how to update x264 in megui to latest version???

By copying the x264.exe file to megui\tools\x264

[burfadel]

Long wait, 658 is out :) very small changes but its an update! :D

[mroz]

I'm trying to find out what the current state of support is for interlaced encoding (in respect of future pal dv encodes).

I've spent a good three nights searching the web including this forum & can't find any mention more recent than last year, none of which were definitive but did make it clear interlaced support is still young.

DeathTheSheep's guide http://gabextreme.googlepages.com/DeathTheSheep_x264_VfW_guide.html states trellis & b frames have to be disabled. Is this still the case or is it maybe specific to the VfW version (seems unlikely)? Disabling b frames seems so extreme as to make me question whether I ought to consider x264 yet for this application. I think in such a case I'd have to restrict x264 to dv with only low motion & then deinterlace prior to encoding, which would be a shame.

Yet when I look at various third party front ends several seem to allow selection of interlaced functionality without any caveats either built into their code or docs - eg Megui iirc & certainly h264enc.

Is there any information available elsewhere?

I hope it was okay to pose this question in the dev thread. :thanks: for an excellent codec btw.

[akupenguin]

Trellis and B-frames work with interlacing.
B-pyramid might not, there's a bug I still have to track down.

[mroz]

:thanks: for the lightening quick response. I'll give interlaced encoding a go when I next have a batch of miniDV tapes to edit - quite curious so might have to do some testing soon.

[simonhowson]

I'm wondering if the motion estimation in next generation intel CPUs could help x264? Intel have sample code on their webpage here (http://softwarecommunity.intel.com/articles/eng/1246.htm) explaining how the new SIMD instruction is designed specifically to help speedup modern video encoders.

I was considering buying a Core 2 Quad CPU when the prices are cut on July 22nd. But if these new instructions are going to make their next gneration CPUs even better for video encoding, then I may wait for those.

Does x264 rely on these sort of motion estimation? Or is it unlikely that this feature will help with the way x264 works?

I mainly encode to the iPod 5.5 profile in MeGUI, does this even use motion estimation? Or is it only used in higher profiles?

[Terranigma]

I think someone, maybe even myself, should consider compiling patched builds from the sources in Cef's absence. Now if only there was a package that contained everything I need to compile......

[Manao]

simonhowson : http://forum.doom9.org/showthread.php?p=993986#post993986

So SSE4 isn't that usefull for the motion estimation. It will speed up the codec, but don't expect miracles.

[simonhowson]

simonhowson : http://forum.doom9.org/showthread.php?p=993986#post993986

So SSE4 isn't that usefull for the motion estimation. It will speed up the codec, but don't expect miracles.
Ahh, thanks for the reply, so it looks like the main improvement will be the bigger caches (6 MB rather than 4 MB).

[mroz]

How important is L2 cache to x264? I'm currently using a bunch of old Athlon XP machines & have always built AMD in the past as I like value for money. Probably be building a new machine in the next three months & will obviously be switching to Intel Core 2.

The cpu will either be the price slashed Quad 6600 or an overclocked budget E21xx. As x264 encoding will be a common task for the box I just wanted to check how big an impact the cache size will have.

Obviously I'd rather have the quad, but it'll depend on exact prices, how many pennies I have in the jar & expected relative performance.

[burfadel]

There is a massive difference metween the Q6600 and the ultra budget one you suggested. If you are thinking of an E6300 or E6400, you should pass those as they have been superceded by the E6320 and the E6420. The E6320 and E6420 have 4mb of L2 cache compared to the E6300 and E6400's 2mb, but run at the same speed. They are also the same price, so going for the older 6300's and 6400's would be silly (unless they cut the price in half or something).

[mroz]

There is a massive difference between the Q6600 and the ultra budget one you suggested.

I realise that. I just wanted to be able to make a judgement based on performance as one component only, but to do that I need to know how important cache size is to x264. How important is it?

Thanks for the other info.

Any c2d is going to massively outperform my old machines. A quad would be nice but even slashed it will still cost significantly more than an E21xx & cost is a major concern, especially as over the next year I'd like to replace all three of our existing desktop machines.

Normally I go with value first (ie minimal price : performance) & keep to the budget end of the spectrum, but if the price cut to the Q6600 makes it /much/ better value I might aim for it. I'm not likely to go for anything mid range unless they look to be at least as good value as the Q6600. I don't want to go into further detail about my reasoning as that would take this off thread.

Can anyone answer the cache question?

[akupenguin]

I have only one data point on the cache question, so take this with a grain of salt:
An Athlon64 w/ 1MB L2 cache is 5% faster per clock cycle than an Athlon64 w/ 512KB L2 cache when running x264.

[mroz]

:thanks:

A small data set indeed, but if cache size was the only difference it still seems potentially significant. Ta for the info.

Reminds me, I seem to recall reading elsewhere about the idea to bundle some performance analysis & reporting code with standard x264 builds. As I like stats that'd be a cool thing to see. I hope that idea gets off the ground. Must search for that thread.

Thanks again.

[Manao]

That would be : http://forum.doom9.org/showthread.php?p=1011128#post1011128

[mroz]

Ooh, lovely data :) Now I do want a quad.

Wasn't the thread I stumbled over before though - that talked of x264 incorporating code to gather stats automatically.

[Terranigma]

Update: x264 revision 659 by Cef (http://cef.neuf.fr/x264/) :)

[JarrettH]

rev 662

Add vertical and horizontal luma deblocking accelerated with Altivec,
based on Graham Booker's code written for FFmpeg with slight modifications
to re-use x264's macros

can anyone explain more about this change?

[Manao]

It speeds up the deblocking on power PC

[burfadel]

Altivec is the PowerPC (older Macintosh) version of SSE (thankyou Wikipedia!). Therefore it will provide no benefit for a x86/x64 based comptuer.

Here's the next question... is deblocking fully SSE/SSE2/SSE3/SSSE3 optimised? ...

[Manao]

For x86, it's mmx/isse only.
For x64, it's mmx/isse & sse2 only.

Deblocking will gain some speed with SSE2 on p4/conroe in x86, and with SSSE3 in both x86/x64. However, on the whole encoding process, that speed gain is negligible, especially when subme is high.

[JarrettH]

thanks, i've been looking for a reason to update from 656, guess that isn't it :p

[burfadel]

thanks, i've been looking for a reason to update from 656, guess that isn't it :p

Update to 661 though...!

Oh, and use cef's build, they're noticeably faster (several percent) on Core 2's anyway.

Plus he has the thread pool patch, aq patch etc so with the extra settings to can maximise the cpu use, leading to a little extra speed.

If someone knew what they were doing, they could update the x86 code to maximise the use of sse/sse2/sse3/ssse3, as there's many parts that aren't using the fastest implementation if available.

[rbt01]

how to use the C version instead of asm version of the module in codec?

I want to use the dct.c instead of the dct-a.asm.

[burfadel]

Here's a good question:...
Media Player Classic has the options to use pixel shaders, and you can select the pixel shader version through the editor. These do things such as sharpening etc.

I was wondering whether its possible to port it into x264, and have certain functions completed with the pixel shaders (preferable 3 or 4 due to their speed, but with the ability to select shader 2 on older cards)? I'm sure Gabest would be ok with that, as long as you get prior permission.

Even if you also have to port the (preferably) EVR mode (as this is the preferred mode in Vista for the future, and is available on XP through Net framework 3).

Is this at all possible? I realise it would require a bit of work, but at least there's a base to start from. This could give a very significant boost to encoding performance (in theory)...

[Inventive Software]

how to use the C version instead of asm version of the module in codec?

I want to use the dct.c instead of the dct-a.asm.

If you mean when you compile, I can't remember cause I've not done it in years, but if you mean when encoding, then add:
--no-asm
to the command line.

[Terranigma]

akupenguin, could you comment on what the change is for revision 665? I just seen a new version posted at x264.nl, but nothing of a changelog. Thanks in advance :)

[Bigmango]

akupenguin, could you comment on what the change is for revision 665? I just seen a new version posted at x264.nl, but nothing of a changelog. Thanks in advance :)

r665 | pengvado | 2007-07-13 01:48:23 +0200 (Fri, 13 Jul 2007) | 2 lines

extend zones to support (some) encoding parameters in addition to ratecontrol.

[celtic_druid]

"extend zones to support (some) encoding parameters in addition to ratecontrol."

[bob0r]

gcc -o x264.exe x264.o matroska.o muxers.o libx264.a -lpthreadGC2 -lwsock32 -lgpac_static -lwinmm -lvfw32 -s -fprofile-generate
libx264.a(ratecontrol.o):ratecontrol.c:(.text+0x4077): undefined reference to `strtok_r'
libx264.a(ratecontrol.o):ratecontrol.c:(.text+0x422d): undefined reference to `strtok_r'
make[1]: *** [x264.exe] Error 1
make[1]: Leaving directory `/home/user/x264'
make: *** [fprofiled] Error 2

Any clue anyone? (i see Cef compiled it)

[Sharktooth]

are you sure you did a correct svn checkout?
i cant check what's going on at the moment, but it seems you could have some old files on your local copy (or maybe a typo in the sources??!? check the changes on the SVN)

[bob0r]

Trax has the same issue, only x264 changed, nothing on my system.

[Sharktooth]

try updating glibc. if it still fails then HAVE_STRTOK_R / NO_STRTOK_R must be implemented ...

[Selur]

"extend zones to support (some) encoding parameters in addition to ratecontrol."
some additional infos would be nice, since --longhelp doesn't show any additional infos :)

[bob0r]

try updating glibc. if it still fails then HAVE_STRTOK_R / NO_STRTOK_R must be implemented ...

Might you help me with telling me what to update?
lib and include files? If so which ones... i dont want to mess up my current system :)

Compiling whines about some port thingy....

[Sharktooth]

the GNU C Library... http://www.gnu.org/software/libc/

[bob0r]

Temp solution found here:
http://forum.doom9.org/showthread.php?p=1024677#post1024677

Note, i will not update x264.nl with a patched version, ill wait for this to be fixed in SVN.

Thanks for the help!

[MetalPhreak]

"extend zones to support (some) encoding parameters in addition to ratecontrol."
some additional infos would be nice, since --longhelp doesn't show any additional infos :)

I'd also like to know what parameters can be used with zones, usage examples?

[akupenguin]

8x8dct, b-bias, b-pyramid, bime, brdo, chrome-me, dct-decimate, deblock, direct, fast-pskip, me, merange, mixed-ref, nr, partitions, ref, scenecut, subme, trellis.

example:
x264 in.avs -o out.264 --crf 20 --deblock -2:-2 --zones 1000,2000,deblock=0:0/2000,3000,nodeblock


Regarding strtok_r: I just remembered there's a reason strtok_r is necessary and strtok doesn't work. It's not just making x264 as a whole re-entrant, but even within one call to zones it depends on strtok_r being re-entrant. So if you have multiple zones and any of them use the new parameters, parsing will fail on windows. I'm not interested in re-implementing strtok (not that it's hard, just I don't care), so it will stay broken until someone submits a patch.

[Selur]

@akupenguin: any plans to extend zones, that one can force an IDR Frame at it's beginning? (sometimes one might want cutpoints at specific frames)

[Cef]

I found a small bug in ratecontrol.c, h->rc->zones[i].param are allocated with regular malloc and freed with x264_free. It probably goes unnoticed on most platforms, but it's actually crashing in my build.

[Trahald]

Regarding strtok_r: I just remembered there's a reason strtok_r is necessary and strtok doesn't work. It's not just making x264 as a whole re-entrant, but even within one call to zones it depends on strtok_r being re-entrant. So if you have multiple zones and any of them use the new parameters, parsing will fail on windows. I'm not interested in re-implementing strtok (not that it's hard, just I don't care), so it will stay broken until someone submits a patch.

im just using
#ifdef _MSC_VER
#define strtok_r strtok_s
#endif

in ratecontrol.h

from what i read the problem is solely on a gnu compile. (i compile in msvc)

[TheRyuu]

I've built x264 using a newer GCC 4.2.1.

The reason I don't like GCC 3.4.x is because when you use -O3, it turns on -frename-registers (or whatever it's called :p). That option is useless (IMO) (but it can be useful on x86_64), not to mention the newer GCC versions are better at optimizations.

You can get it here (rev667b x86). (http://www.esnips.com/doc/27a5b2cc-5b04-4d82-bb7a-ae6a200e1d39/x264_x86_r667b)
(if you want to try it :))

It was compiled with the rev. 667b source code from Cef's build site. I'm finding that this version is faster then the current Cef builds. (if only by a little bit, 34 sec. compared to 36 sec. on a fast 1:30 file with a fast 1 pass encode, probably a greater speed difference would be noticeable on a larger encode, but I'll take any more speed I can get no matter how little on x264).

It was also compiled with march=i686 (for all you people who don't have an sse cpu) and with mtune=pentium3 (which would enable sse for people who have it) which pretty much make the build all around compatible with almost any cpu and allows newer cpu's to take advantage of sse.

Like I said, I just like building things with the newer GCC since I find things faster and sometimes more stable.

I can't see any downsides to using the newer GCC 4.2.1 (thanks to cc979 ;)) I find it more stable, optimizations are better (no -frename-registers), etc..

Thanks and Enjoy, cya :)

P.S.
Cflags were:
-ffast-math -fomit-frame-pointer -march=i686 -mtune=pentium3 -mfpmath=387 -and any other standard flags
additional ldflags:
-Wl,-O1 -Wl,--sort-common

[akupenguin]

The reason I don't like GCC 3.4.x is because when you use -O3, it turns on -frename-registers (or whatever it's called :p). That option is useless (IMO) (but it can be useful on x86_64), not to mention the newer GCC versions are better at optimizations.
Why should you care whether any given option is enabled? The only question is which compiler generates faster code.
Anyway, if it really bothers you, then the obvious solution is -O3 -fno-rename-registers.

It was also compiled with march=i686 (for all you people who don't have an sse cpu) and with mtune=pentium3 (which would enable sse for people who have it) which pretty much make the build all around compatible with almost any cpu and allows newer cpu's to take advantage of sse.
Either the binary contains sse or it doesn't. gcc doesn't generate runtime cpu detection, so any binary that contains sse won't run on pre-sse computers. -march determines what instructions are available, -mtune only determines scheduling.

[TheRyuu]

Why should you care whether any given option is enabled? The only question is which compiler generates faster code.
Anyway, if it really bothers you, then the obvious solution is -O3 -fno-rename-registers.

I care what I compile my programs with, don't you?
I also care because -frename-registers is pretty much useless on anything but x86_64, and it bloats code IMO (would be about the same speed, sometimes there might be a slight gain, but still, IMO, not worth it and could be counter productive too)
It was eventually taken out of -O3 also although thats besides the point. :p

The real purpose of the post wasn't to tell my personal opinions on compiler options though, it was to show that a newer GCC is better at optimizing x264 then the older (mingw 3.4.4) one.

Either the binary contains sse or it doesn't. gcc doesn't generate runtime cpu detection, so any binary that contains sse won't run on pre-sse computers. -march determines what instructions are available, -mtune only determines scheduling.

Thats what I meant I guess. It was build for i686 but is optimized for the pentium3, correct?
That should allow someone with an i686 cpu to run it, but also allow someone with a pentium3 or above to use the extra optimizations, right? (I don't fully understand how -march+-mtune really work together yet I guess)

Just sharing some experience building it thats all. :)

[akupenguin]

It was build for i686 but is optimized for the pentium3, correct?
That should allow someone with an i686 cpu to run it, but also allow someone with a pentium3 or above to use the extra optimizations, right?
Right.
A given instruction takes different numbers of cycles on different cpus. A given computation can be expressed many ways, using different combinations of instructions.
Of all the alternatives that would run on the cpu specified by -march, gcc picks the alternative that's fastest for the cpu specified by -mtune.

Example of how -march can help: -march=pentium4 goes through contortions to avoid register-to-register mov instructions, because mov is really slow on pentium4. So if it needs a value in 2 registers, it loads it twice from memory, whereas most other cpus would load it once and then mov to the other register.

The point I was correcting is: -march=i686 -mtune=pentium3 doesn't use sse because i686 can't run sse, even if sse would be the fastest way to implement something on pentium3.

Nitpicking your cflags: -mfpmath=387 is redundant. Not only is it the default, it's also the only possibility given that sse is disallowed.

[burfadel]

I've built x264 using a newer GCC 4.2.1.

The reason I don't like GCC 3.4.x is because when you use -O3, it turns on -frename-registers (or whatever it's called :p). That option is useless (IMO) (but it can be useful on x86_64), not to mention the newer GCC versions are better at optimizations.

You can get it here (rev667b x86). (http://www.esnips.com/doc/27a5b2cc-5b04-4d82-bb7a-ae6a200e1d39/x264_x86_r667b)
(if you want to try it :))

It was compiled with the rev. 667b source code from Cef's build site. I'm finding that this version is faster then the current Cef builds. (if only by a little bit, 34 sec. compared to 36 sec. on a fast 1:30 file with a fast 1 pass encode, probably a greater speed difference would be noticeable on a larger encode, but I'll take any more speed I can get no matter how little on x264).

It was also compiled with march=i686 (for all you people who don't have an sse cpu) and with mtune=pentium3 (which would enable sse for people who have it) which pretty much make the build all around compatible with almost any cpu and allows newer cpu's to take advantage of sse.

Like I said, I just like building things with the newer GCC since I find things faster and sometimes more stable.

I can't see any downsides to using the newer GCC 4.2.1 (thanks to cc979 ;)) I find it more stable, optimizations are better (no -frename-registers), etc..

Thanks and Enjoy, cya :)

P.S.
Cflags were:
-ffast-math -fomit-frame-pointer -march=i686 -mtune=pentium3 -mfpmath=387 -and any other standard flags
additional ldflags:
-Wl,-O1 -Wl,--sort-common

I tried it, it was slightly slower on my computer by a few percent than the 667b build made by Cef. I have a Core 2 Duo E6600.

I believe Cef optimised the builds for the Core 2 Duo himself, it just goes to show different compiling optimisations work better for different CPU types :)

[TheRyuu]

Right.
A given instruction takes different numbers of cycles on different cpus. A given computation can be expressed many ways, using different combinations of instructions.
Of all the alternatives that would run on the cpu specified by -march, gcc picks the alternative that's fastest for the cpu specified by -mtune.

Example of how -march can help: -march=pentium4 goes through contortions to avoid register-to-register mov instructions, because mov is really slow on pentium4. So if it needs a value in 2 registers, it loads it twice from memory, whereas most other cpus would load it once and then mov to the other register.

The point I was correcting is: -march=i686 -mtune=pentium3 doesn't use sse because i686 can't run sse, even if sse would be the fastest way to implement something on pentium3.

Nitpicking your cflags: -mfpmath=387 is redundant. Not only is it the default, it's also the only possibility given that sse is disallowed.

Thanks.
Adding sse wouldn't really matter anyway since most of it is in asm though right?

Whatever though.
New build here. (http://www.esnips.com/doc/a1d6e08a-0dc8-4baa-8d9b-ee0f3a1cee07/x264_x86_667b_2)

[sampath]

is there a bug in recent vfw build of cefs. It gives error messeage : second pass has more frames than first pass 331>328 .but i didn't change frames at passes?$$

x86 vfw r677 on virtualdub1.6.19

i didn' encounter this on the snow_xmas vfw build

[akupenguin]

Symptom of the fact that vfw is incompatible with B-frames or x264's threads, whichever of those you're using. No, it can't be fixed.

[daWsOn_s]

Hi I'm now trying the new 667 Cef release and I've discovered an acceleration in enconding of about 40% , is that general?

[Terranigma]

Cef, do you plan on updating to rev.671, or do you not find the current crop of changes significant enough?

[morph166955]

the only changes since r665 have to do with the configure script and a few small build options, nothing that should change the binary in any way (theoretically). So what I won't speak for Cef, I doubt hes going to update it since there really is no reason based on the changes that were done.

[Sharktooth]

yep, 667 and 671 bins would be identical. however this is x264 dev thread where ppl should talk about development, not about bugs, not about vfw (that's no longer mantained) and not about builds.
thanks.

[rbt01]

can anybody give me a suggestion, i can't use the long option, such as : --pass 1 --bitrate 2000

[JarrettH]

maybe look at the megui x264 profiles for command line examples

[Vesi]

I have downloaded megui from sourceforge and got all the updates, i want to do xvid rips and x264, so my question is that i have read this thread http://forum.doom9.org/showthread.php?t=89979.
and i'm using HP 3200+ (2,20 GHz) with 1GB DDR memory. do i need those pathes and file which made by sharktooth?

[Sharktooth]

no. x264 and xvid are already included in megui updates.

[Vesi]

no. x264 and xvid are already included in megui updates.

Thanks alot Sharktooth. I really appreciate your work.
@ one thing more what i was thinking that H264 give the highest quality.would it be a good idea to rip with A/R 720*320 for 1CD 650MB with good quality source? thanks

[akupenguin]

Initial review of the instructions to be included in SSE5 (http://developer.amd.com/sse5.jsp). Note that I'm only discussing the functionality, I have no idea of the speeds of the instructions yet. There were some SSSE3 instructions that looked interesting, and reduced the instruction count of some functions, but when run on a Core2 turned out to be much slower than the equivalent multiple SSE2 instructions.

First, discard all the floating-point instructions, because video codecs don't use floating-point.

PCMOV: generally useful anywhere the scalar version of the algorithm has branches, e.g. deblocking.
PCOM*: might save a MOV or a negation over PCMP*, if one wants a predicate other than EQ or LE.
PPERM: zigzag scan in 2 instructions. (finally equivalent to altivec)
PHADD*: useful in all the pixel comparison functions, especially satd. (note that these aren't the same as SSSE3's PHADD*)
PMADC*: SSD and SSIM.
PMAC*: DCT (but not HCT).
PSHLB: I have occasionally wanted this. the per-element shift amount isn't important though, only that there wasn't previously any byte shift instruction.

[SpAwN_gUy]

apparently.. i've found whole bunch of patches at Cef's place on x264.nl...

can anybody give an advice.. which should be applied, and which are not so usefull?
(i'm going to supreme quality ;) ... speed does not matter)

[Audionut]

can anybody give an advice.. which should be applied, and which are not so usefull?
(i'm going to supreme quality ;) ... speed does not matter)


Current Patches, Where to get them, How they affect speed/output
http://forum.doom9.org/showthread.php?t=130364

[LigH]

Excuse my non-developer based question...

A user in the german doom9/Gleitz board spotted news about the evision 683, implementing quant_2x2_dc for Altivec; despite the fact that Altivec is irrellevant for (intel compatible) PC owners, he just wondered why he never saw any "2x2" option in MeGUI...

Could you briefly explain how far the distance is between quant_NxN[_dc] functions and macroblock partitions (with or without DCT)?

[Manao]

In intra macroblock, once the 4x4 DCTs are made, the DC of each of the DCT are retransformed with another DCT. For luma, there are 16 blocks so it's another 4x4 DCT. For chroma, there are only 4 4x4 blocks, so it's a 2x2 DCT. It's done only on DCs coefficients.

[LigH]

This abbreviation is too short to search for, sorry...

DCT = Discrete Cosine Transform;
DC = ?

[Manao]

The DC is the DCT coefficient that represents the average of the untransformed block. It is usually opposed to the ACs, which are the remaining coefficients (those that represent the variations in the block).

If I am not mistaken, the names come from electricity (Alternative Current / Direct Current).

[Guest]

This abbreviation is too short to search for, sorry... Use google and put site:forum.doom9.org after the search term(s). E.g.:

dc coefficients site:forum.doom9.org

@Manao

You are not mistaken.

[LigH]

Wonderful... Indeed, I remember AC/DC, just didn't expect such a similarity. ;) - Now I understand quite well.

BTW, I was quite surprised by the rating of AltiVec in the german Wikipedia article ("ignored and underestimated"). Nice to see that a few developers try to use all means of optimizations, even if only few people may get an advantage.

Fun and success! :cool:

[mroz]

Actually you can get useful info from just searching on dc coefficients (at least with Google), which is odd as I thought it discarded two character words from the search string.

[Adub]

No, just the common ones like "to" or "of".

[Inventive Software]

Feature request that'll get me shot at.... implement interlaced and esa and interlaced and direct=temporal, as per the errors I got when I tried to encode interlaced to interlaced with x264.

[Dark Shikari]

Feature request that'll get me shot at.... implement interlaced and esa and interlaced and direct=temporal, as per the errors I got when I tried to encode interlaced to interlaced with x264.I could easily implement ESA and Interlaced simply by having it swap to the standard (and mindbogglingly slow) ESA algorithm instead of the SEA algorithm that x264 currently uses :)

[Inventive Software]

You could keep the current one for progressive input, and switch it when the user flags interlaced. :) I just thought I'd mention it as it flagged up when I encoded interlaced content.

It's just as well it doesn't work as I'd like it to... interlaced decoding seems to be broken with ffdshow and interlaced x264 encodes. :(

[nm]

It's just as well it doesn't work as I'd like it to... interlaced decoding seems to be broken with ffdshow and interlaced x264 encodes. :(
You'll need to use mod-32 height. That's a limitation in libavcodec's decoder.

[Inventive Software]

Bugger... did wonder what was wrong with it. Thanks.

EDIT: Just done some quick calcs, and I am using mod-32 height... 720x576. You sure that's not mod-32 width as well as height?

[Sagekilla]

Bugger... did wonder what was wrong with it. Thanks.

EDIT: Just done some quick calcs, and I am using mod-32 height... 720x576. You sure that's not mod-32 width as well as height?

720 / 32 = 22.5, it's the width that's the problem.

Edit: Whoa, just realized it's not necessarily mod32. 720 / 16 = 45, works fine.

[Inventive Software]

Sussed that one, and I tested with a binary of ffplay, and it's ffdshow that's the problem.

EDIT: It's also my idiotness that didn't update ffdshow from rev 1620 to rev 1656! Cos that fixed the problem! Doh! :o

[akupenguin]

Just so that anyone reading doesn't get funny ideas: width does not need to be mod32.

[Inventive Software]

For the record, interlacing appears to be fine and dandy. Have had no problems since updating ffdshow. :D

Suggestion: what about a switch to make x264 adapt between SAD and SATD for frames? SATD is more costly in terms of time, but from what people have shown, it seems to be not worth the enormous speed loss. Easy frames can be SAD, complex ones SATD, or however would be most efficient to implement it. Ideally, it'd be another option in the fpel-cmp switch, on top of "sad" and "satd", you'd have "adaptive". If Hadamard was included, it could use all 3 if you weren't picky! The only comparison I can draw is the one used in WMV9 AP. This can use an adaptive motion search method to determine motion matching, and when enabled, it switches between SAD and Hadamard. To quote the help from WMV Powertoy 1.2.5.0:

SAD (Sum of Absolute Differences) is a faster but less accurate method than the Hadamard transform. Because Hadamard transforms are very computationally intensive, an automatic (macroblock-adaptive) mode exists which can dynamically choose between the two methods, providing a reasonable compromise between Hadamard quality and SAD performance.

Just my 2 US cents, worth less than 1 penny Sterling ATM. :p Hopefully this suggestion is more practical and it'd make it to the trunk! ;)

[Dark Shikari]

If Hadamard was included, it could use all 3 if you weren't picky!SATD is Hadamard.

I've tried this and in my experience it is not very worthwhile because of predictors; SATD predictors aren't very good for SAD, and vice versa. So using one after another generally results in worse results than just sticking with one for everything.

And SATD is a lot slower. Think 400-450 clock cycles for a 16x16 block, while SAD is 48.

[XolocoTuxmaster]

Hi everybody, I'm in OpenSUSE 10.3 Linux and I have just installed x264 via YaST. It's alright, but I have no AVIS input support, so I installed also via YaST x264-devel and libx264-57. I'm new in programming and I don't know what to do now (I'm used to have a compiling readme). These are the installed packages file list

x264-devel-0.57.702-7.1 - x264 Headers and library

/usr/include/x264.h
/usr/include/x264_gtk.h
/usr/include/x264_gtk_enum.h
/usr/lib/libx264.a
/usr/lib/libx264.so
/usr/lib/libx264gtk.a
/usr/lib/libx264gtk.so


libx264-57-0.57.702-7.1 - x264 Share library, Major Version $57

/usr/lib/libx264.so.57
/usr/lib/libx264gtk.so

Can anyone tell me please how to compile x264 using this files with AVIS support?

[Dark Shikari]

Hi everybody, I'm in OpenSUSE 10.3 Linux and I have just installed x264 via YaST. It's alright, but I have no AVIS input support, so I installed also via YaST x264-devel and libx264-57. I'm new in programming and I don't know what to do now (I'm used to have a compiling readme). These are the installed packages file list

x264-devel-0.57.702-7.1 - x264 Headers and library

/usr/include/x264.h
/usr/include/x264_gtk.h
/usr/include/x264_gtk_enum.h
/usr/lib/libx264.a
/usr/lib/libx264.so
/usr/lib/libx264gtk.a
/usr/lib/libx264gtk.so


libx264-57-0.57.702-7.1 - x264 Share library, Major Version $57

/usr/lib/libx264.so.57
/usr/lib/libx264gtk.so

Can anyone tell me please how to compile x264 using this files with AVIS support?Avisynth is Windows/WINE-only.

[XolocoTuxmaster]

In sourceforge.net they say I can compile it for linux

http://avisynth2.cvs.sourceforge.net/*checkout*/avisynth2/avisynth/docs/html/index.html?revision=1.1.2.4&pathrev=avisynth_3_0

By they way, is avisynth the only way to have avis support in Linux? I've heard there were other ways like MPJPEGtools or something...

[J_Darnley]

I am no expert but as far as I know, x264 cannot use AviSynth 3 on any OS. You can only access the avisynth 2 from Windows or Wine as it is a dll which needs to be loaded. You can use any cmd line program in Wine and then pipe the output into a native x264. Some you could use are avs2yuv (which was made just for this), ffmpeg and mencoder if you can pipe its output (I've never used it).

[XolocoTuxmaster]

I'm very intrested in ffmpeg output piping. If anyone knows how to do it please tell me

[Dark Shikari]

You can also use a FIFO... output mplayer/mencoder/ffmpeg in raw YV12 to a FIFO, and then use that as the input for x264.

[XolocoTuxmaster]

yes J_Darnley has already suggested using a pipe/fifo, but I don't know how to do it. What I've tried is

mkfifo video.yuv

ffmpeg video.264 video.yuv & x264 -options video.yuv

Someone in this forum used these lines and said they worked

mkfifo -m 660 dvd.y4m
ffmpeg -i vts_01_0.vob -f yuv4mpegpipe - > dvd.y4m &
x264 -o dvd1.264 dvd.y4m

but when I use that I get the the pipe and the x264 file both with 0 Kb size, and ffmpeg just says

Input #0, h264, from 'video.264':
Duration: N/A, bitrate: N/A
Stream #0.0: Video: h264, yuv420p, 1920x1080 [PAR 0:1 DAR 0:1], 23.98 fps(r)
Output #0, yuv4mpegpipe, to 'pipe:':
Stream #0.0: Video: rawvideo, yuv420p, 1920x1080 [PAR 0:1 DAR 0:1], q=2-31, 200 kb/s, 23.98 fps(c)
Stream mapping:
Stream #0.0 -> #0.0
Press [q] to stop encoding

But CPU doenst seem to be working

My output file is video.y4m while ffmpeg says "pipe", I don't know if that matters

I would thank If someone gave me the lines I have to use to pipe with ffmpeg

[J_Darnley]

If you use raw YV12 (yuv420) in a file or a pipe/fifo then you need to specify the frame dimensions. If you use a y4m file or fifo then x264 will read the frame dimensions from from the file/fifo.

What are the exact command lines you are using? I guess you can omite the exact encoding options if you provide the input and output parts. Also, your PAR/DAR seems to very wrong. Have you tried avs2yuv? The couple of times I have used x264 and a pipe have used:

ffmpeg -benchmark -threads 2 -i "D:\AviSynth\fotr\fotr_final.avs" -vcodec copy -f rawvideo - -acodec vorbis -aq 50 -y "D:\AviSynth\fotr\fotr_audio.ogg" | x264 --encoding-options --output "D:\AviSynth\fotr\fotr_video.mkv" - 704x288

[XolocoTuxmaster]

Said nothing, all solved.

[dbmaxpayne]

hey, is there any way to use my ATI HD3870 to hardware accelerate the encoding process of x264?

[Ranguvar]

Not so far, AFAIK. If you use FFT3DFilter in your Avisynth script, you can swap that out for FFT3DGPU, which runs off the graphics card. Also, nVidia's PureVideo decoder will decode MPEG-2 video and possibly others off the GPU. 99% positive it will work with other cards besides nVidia's. But no x264 GPU encoding.

[nm]

Also, nVidia's PureVideo decoder will decode MPEG-2 video and possibly others off the GPU. 99% positive it will work with other cards besides nVidia's.
No need to buy PureVideo decoder for that. These cards provide hardware decoding of MPEG-2, VC-1 and H.264 through DXVA(2) with any compatible DirectShow filter/player (such as MPC Home Cinema).

[Ranguvar]

No need to buy PureVideo decoder for that. These cards provide hardware decoding of MPEG-2, VC-1 and H.264 through DXVA(2) with any compatible DirectShow filter/player (such as MPC Home Cinema).
Ah, really? Could you please enlighten me on how to do that? Getting a 9600GT tomorrow, and even though I'll also have a quad core, it's nice to take the load off the CPU.

[nm]

Sorry, can't tell you much more since I don't use Windows myself. I guess it should "just work" with Cyberlink's decoder (from latest PowerDVD versions). MPC HC's DXVA support is under development, but I've heard it already works with some cards and streams.

[burfadel]

A month without a new build! A well deserved holiday for akupenguin/dark shikari or hard at work making big interesting changes?!

[Ranguvar]

Well, I know DS has been working :p Check out Xvid and x264's variance adaptive quantization, among other things.

[Ranguvar]

Sorry, can't tell you much more since I don't use Windows myself. I guess it should "just work" with Cyberlink's decoder (from latest PowerDVD versions). MPC HC's DXVA support is under development, but I've heard it already works with some cards and streams.
Alright, thanks... just use it in GraphEdit right? I wonder if PureVideo has one I can use that's even better.

[gizmo27]

Who are the of x264 cli codec authors ?

[nm]

Take a look at the AUTHORS file from x264 source code tree:
http://git.videolan.org/?p=x264.git;a=blob_plain;f=AUTHORS;hb=HEAD

[gizmo27]

Thanks a lot

[SpAwN_gUy]

Who are the of x264 cli codec authors ? they are :)

[Rodger]

May I ask where to find 64bit realeases?

I just got vista64bit installed and everything is working nice and smoothly, but MeGui....but that´s another thing.
Now I´m collection all my applications in 64bit "editions".

[Henrikx]

Till x264 core:58 r798 17e5dac all was ok. I was able to compile all versions. But today : /usr/bin/ld: soname: No such file: No such file or directory
collect2: ld gab 1 als Ende-Status zurück
make: *** [libx264.so.59] Fehler 1

Ubuntu 7.10.

[Audionut]

May I ask where to find 64bit realeases?

I just got vista64bit installed and everything is working nice and smoothly, but MeGui....but that´s another thing.
Now I´m collection all my applications in 64bit "editions".

http://members.optusnet.com.au/squid_80/

[Henrikx]

Now with x264 core:59 r806 457ce6a i can compile again. THX

[akupenguin]

http://members.optusnet.com.au/squid_80/
But that x264 build is more than a year old. The latest version will not build on win64 until someone updates the asm, because I don't have a win64 box and thus didn't support that when reorganizing everything to merge 32bit and 64bit versions.

[squid_80]

I stopped building when Cef started. If someone sends me the .asm(s) that won't build/run on win64 I'll try and get it working.

[Rodger]

But that x264 build is more than a year old. The latest version will not build on win64 until someone updates the asm, because I don't have a win64 box and thus didn't support that when reorganizing everything to merge 32bit and 64bit versions.

By the way...I compared some "old" 64bit versions with newest 32bit versions. You can´t stop developement ;)
The newer 32bit versions pulverized the older 64bit versions :rolleyes:

Correct me if I´m wrong...the main work is done by x264 so it would help to have everything in 64bit but there would be no big bottleneck if there is missing something "in the line".

But as much as I can see....damn...the world is sleeping on 64bit Code! Seems the world is STILL not ready for 64bit.
There is still a long way until 64bit is preferred to 32bit.

[Prettz]

Initial review of the instructions to be included in SSE5 (http://developer.amd.com/sse5.jsp). Note that I'm only discussing the functionality, I have no idea of the speeds of the instructions yet. There were some SSSE3 instructions that looked interesting, and reduced the instruction count of some functions, but when run on a Core2 turned out to be much slower than the equivalent multiple SSE2 instructions.

First, discard all the floating-point instructions, because video codecs don't use floating-point.

PCMOV: generally useful anywhere the scalar version of the algorithm has branches, e.g. deblocking.
PCOM*: might save a MOV or a negation over PCMP*, if one wants a predicate other than EQ or LE.
PPERM: zigzag scan in 1 instruction.
PHADD*: useful in all the pixel comparison functions. (but I said the same of SSSE3's PHADD* ...)
PMADCSSWD: SSD and SSIM.
That's pretty cool. But will Intel accept it? They never implemented 3DNow!, after all.

aku, do you believe any of x264's asm routines would benefit noticeably from the extra XMM registers you get in 64-bit mode?

[Dark Shikari]

aku, do you believe any of x264's asm routines would benefit noticeably from the extra XMM registers you get in 64-bit mode?"Would"?

Have you looked at common/x86/dct-64.asm lately? ;)

[Prettz]

"Would"?

Have you looked at common/x86/dct-64.asm lately? ;)
Nope. I take it then that they can really put those extra registers to good use?

[burfadel]

By the way...I compared some "old" 64bit versions with newest 32bit versions. You can´t stop developement ;)
The newer 32bit versions pulverized the older 64bit versions :rolleyes:

Correct me if I´m wrong...the main work is done by x264 so it would help to have everything in 64bit but there would be no big bottleneck if there is missing something "in the line".

But as much as I can see....damn...the world is sleeping on 64bit Code! Seems the world is STILL not ready for 64bit.
There is still a long way until 64bit is preferred to 32bit.

Well everything in the line has to be 64 bit, Microsoft did this as a security measure I believe. I guess you mean missed as in development wise?

[lexor]

But as much as I can see....damn...the world is sleeping on 64bit Code! Seems the world is STILL not ready for 64bit.
There is still a long way until 64bit is preferred to 32bit.
I think it's hard to justify a large investment of time into writing 64bit x264 code, simply because as you can see from change logs not only is 32bit enough, they keep periodically going over stable code to reduce variables from 32bit to 16 and 8 bit integers.

[Dark Shikari]

I think it's hard to justify a large investment of time into writing 64bit x264 code, simply because as you can see from change logs not only is 32bit enough, they keep periodically going over stable code to reduce variables from 32bit to 16 and 8 bit integers.Actually, reducing those variable sizes makes the code faster on 64-bit because you can more easily copy many variables at once by using 64-bit registers. That's the main reason its done (other than to save L1 cache size).

[akupenguin]

I think it's hard to justify a large investment of time into writing 64bit x264 code.
There is no such thing as 64bit C code. Assembly can be x86_32 or x86_64 or whatever, but C can only be portable or not. The compiler does the arch-specific optimizing.

[Pyth0n]

Trying to convert a 1280x544 video to 1280x720 video by adding 88 borders up and down with megui and avs isn't working here.


When x264 does it's second pass I get this warning.
The video encodes but with a much lower bitrate.


Job commandline: "C:\Program Files (x86)\megui\tools\x264\x264.exe" --pass 2 --bitrate 4518 --stats "E:\TEMP\Rambo.stats" --level 4.1 --keyint 24 --min-keyint 1 --ref 3 --mixed-refs --bframes 3 --b-rdo --bime --weightb --direct auto --subme 6 --partitions p8x8,b8x8,i4x4,i8x8 --8x8dct --qpmin 1 --ipratio 1.1 --pbratio 1.1 --vbv-bufsize 30000 --vbv-maxrate 38000 --qcomp 0.5 --me umh --threads auto --thread-input --sar 1:1 --progress --no-psnr --no-ssim --output "E:\TEMP\Rambo.264" "E:\TEMP\Rambo.avs" --mvrange 511 --aud --nal-hrd --sar 1:1
avis [info]: 1280x720 @ 23.98 fps (133995 frames)
x264 [info]: using SAR=1/1
x264 [info]: using cpu capabilities: MMX MMX2 SSE SSE2 SSE3 3DNow
x264 [warning]: target: 4518.00 kbit/s, expected: 149.21 kbit/s, avg QP: 1.0000
x264 [warning]: try reducing target bitrate or reducing qp_min (currently 1)
x264 [info]: slice I:5584 Avg QP:31.33 size: 7997
x264 [info]: slice P:33499 Avg QP:34.82 size: 2293
x264 [info]: slice B:94912 Avg QP:36.57 size: 935
x264 [info]: mb I I16..4: 43.5% 54.6% 1.9%
x264 [info]: mb P I16..4: 7.5% 6.4% 0.2% P16..4: 13.1% 1.0% 0.2% 0.0% 0.0% skip:71.8%
x264 [info]: mb B I16..4: 0.9% 0.9% 0.0% B16..8: 18.2% 0.1% 0.1% direct: 0.1% skip:79.7%
x264 [info]: 8x8 transform intra:50.1% inter:67.8%
x264 [info]: direct mvs spatial:0.1% temporal:99.9%
x264 [info]: ref P 77.2% 15.2% 7.6%
x264 [info]: ref B 85.9% 14.1%
x264 [info]: kb/s:300.9

encoded 133995 frames, 13.56 fps, 305.39 kb/s
desired video bitrate of this job: 4518 kbit/s - obtained video bitrate (approximate): 305 kbit/s



Job commandline: "C:\Program Files (x86)\megui\tools\x264\x264.exe" --pass 2 --bitrate 6165 --stats "E:\TEMP\Terminator 2.stats" --level 4.1 --keyint 24 --min-keyint 1 --ref 3 --mixed-refs --bframes 3 --b-rdo --bime --weightb --direct auto --subme 6 --partitions p8x8,b8x8,i4x4,i8x8 --8x8dct --qpmin 1 --ipratio 1.1 --pbratio 1.1 --vbv-bufsize 30000 --vbv-maxrate 38000 --qcomp 0.5 --me umh --threads auto --thread-input --sar 1:1 --progress --no-psnr --no-ssim --output "E:\TEMP\Terminator 2.264" "E:\TEMP\Terminator 2.avs" --mvrange 511 --aud --nal-hrd --sar 1:1
avis [info]: 1280x720 @ 23.98 fps (196499 frames)
x264 [info]: using SAR=1/1
x264 [info]: using cpu capabilities: MMX MMX2 SSE SSE2 SSE3 3DNow
x264 [warning]: target: 6165.00 kbit/s, expected: 149.01 kbit/s, avg QP: 1.0000
x264 [warning]: try reducing target bitrate or reducing qp_min (currently 1)
x264 [info]: slice I:8188 Avg QP:34.65 size: 6095
x264 [info]: slice P:49127 Avg QP:37.97 size: 3158
x264 [info]: slice B:139184 Avg QP:39.88 size: 1563
x264 [info]: mb I I16..4: 45.5% 53.5% 0.9%
x264 [info]: mb P I16..4: 14.2% 14.0% 0.2% P16..4: 13.3% 1.1% 0.1% 0.0% 0.0% skip:57.1%
x264 [info]: mb B I16..4: 2.4% 2.6% 0.0% B16..8: 25.3% 0.2% 0.2% direct: 0.1% skip:69.2%
x264 [info]: 8x8 transform intra:51.0% inter:79.9%
x264 [info]: direct mvs spatial:0.1% temporal:99.9%
x264 [info]: ref P 77.7% 15.3% 7.0%
x264 [info]: ref B 87.2% 12.8%
x264 [info]: kb/s:412.6

encoded 196499 frames, 11.92 fps, 417.08 kb/s
desired video bitrate of this job: 6165 kbit/s - obtained video bitrate (approximate): 417 kbit/s

Just to check the default qp value is 10 and I tried that value too. Tried 16 thats the x264 default and them tried 1.
I also tried 7 wich worked with the sample file but not the whole movie.
Isn't and auto value for that?

It's encoding the file but with a much lower bitrate. All my other searchs the guys where trying absurd values and the program didn't even encode it.
Just can't find the solution anywhere else.

This is the first and second pass log of a sample encode.
But I get the exactly same error with the whole movie.

Job commandline: "C:\Program Files (x86)\megui\tools\x264\x264.exe" --pass 1 --bitrate 4518 --stats "E:\TEMP\sample.stats" --level 4.1 --keyint 24 --min-keyint 1 --bframes 3 --direct auto --subme 1 --partitions none --ipratio 1.1 --pbratio 1.1 --vbv-bufsize 30000 --vbv-maxrate 38000 --qcomp 0.5 --me dia --threads auto --thread-input --sar 1:1 --progress --no-psnr --no-ssim --output NUL "E:\TEMP\sample.avs" --mvrange 511 --aud --nal-hrd --sar 1:1
avis [info]: 1280x720 @ 23.98 fps (1487 frames)
x264 [info]: using SAR=1/1
x264 [info]: using cpu capabilities: MMX MMX2 SSE SSE2 SSE3 3DNow
x264 [info]: slice I:63 Avg QP:10.33 size: 12302
x264 [info]: slice P:372 Avg QP:16.10 size: 254
x264 [info]: slice B:1052 Avg QP:17.01 size: 80
x264 [info]: mb I I16..4: 96.2% 0.0% 3.8%
x264 [info]: mb P I16..4: 0.0% 0.0% 0.0% P16..4: 1.0% 0.0% 0.0% 0.0% 0.0% skip:99.0%
x264 [info]: mb B I16..4: 0.0% 0.0% 0.0% B16..8: 0.2% 0.0% 0.0% direct: 0.2% skip:99.6%
x264 [info]: final ratefactor: -17.86
x264 [info]: direct mvs spatial:11.3% temporal:88.7%
x264 [info]: kb/s:123.1

encoded 1487 frames, 46.83 fps, 128.94 kb/s

Job commandline: "C:\Program Files (x86)\megui\tools\x264\x264.exe" --pass 2 --bitrate 4518 --stats "E:\TEMP\sample.stats" --level 4.1 --keyint 24 --min-keyint 1 --ref 3 --mixed-refs --bframes 3 --b-rdo --bime --weightb --direct auto --subme 6 --partitions p8x8,b8x8,i4x4,i8x8 --8x8dct --ipratio 1.1 --pbratio 1.1 --vbv-bufsize 30000 --vbv-maxrate 38000 --qcomp 0.5 --me umh --threads auto --thread-input --sar 1:1 --progress --no-psnr --no-ssim --output "E:\TEMP\sample.264" "E:\TEMP\sample.avs" --mvrange 511 --aud --nal-hrd --sar 1:1
avis [info]: 1280x720 @ 23.98 fps (1487 frames)
x264 [info]: using SAR=1/1
x264 [info]: using cpu capabilities: MMX MMX2 SSE SSE2 SSE3 3DNow
x264 [warning]: target: 4518.00 kbit/s, expected: 176.69 kbit/s, avg QP: 10.0000
x264 [warning]: try reducing target bitrate or reducing qp_min (currently 10)
x264 [info]: slice I:63 Avg QP:28.49 size: 20660
x264 [info]: slice P:372 Avg QP:32.34 size: 4600
x264 [info]: slice B:1052 Avg QP:34.09 size: 1962
x264 [info]: mb I I16..4: 35.9% 57.3% 6.9%
x264 [info]: mb P I16..4: 3.4% 4.6% 0.4% P16..4: 22.6% 3.3% 0.9% 0.0% 0.0% skip:64.9%
x264 [info]: mb B I16..4: 0.2% 0.3% 0.0% B16..8: 24.9% 0.5% 0.8% direct: 0.3% skip:73.0%
x264 [info]: 8x8 transform intra:56.4% inter:63.1%
x264 [info]: direct mvs spatial:11.6% temporal:88.4%
x264 [info]: ref P 82.4% 11.8% 5.8%
x264 [info]: ref B 88.8% 11.2%
x264 [info]: kb/s:654.8


When I reduce qpmin from 10 to 7 the sample works. But not the movie.
And sometimes the sample even with 7 gives the same erro.

Tried using different's builds of x264. The one from x264.nl, the one from this site and the default megui build.

[LigH]

Now that you switched from SVN revision numbers to GIT date-time IDs -- how do you still keep track of the revision numbers? By counting up from a known start? I see hardly any relation anymore, reading the changelog today... Would be great to have revisions included in the changelogs too.

[burfadel]

I believe there are a few ways to get around the 32/64 bit problem. One way is to have the filters etc run in 32 bit avisynth which serves the output to a buffer, possibly by using a filter. At the beginning lf the 32 bit avisynth script, have another filter that spawns the 64 bit modified avisynth that can load from a .cmd file (its kind of like a .bat file). The filter can then terminate. The 64 bit avisynth then serves the data from the buffer to x264 (and possibly run 64 bit filters along the way). The 64 bit avisynth script would be unique to the x64 version, using a special source filter to load the source from the buffer instead of a file. Never said it was practical, just possible!

[akupenguin]

Now that you switched from SVN revision numbers to GIT date-time IDs -- how do you still keep track of the revision numbers?
git-rev-list HEAD | wc -l
or perhaps
git log | perl -e '$/="\ncommit"; $i=@t=<>; for(@t){s/\n/ r$i\n/m; $i--; print}'

I have not, however, found a suitable replacement for revision ids as git commandline arguments.

[LigH]

:eek: You assume that I am a developer using Linux?

:o No, I just wanted to report the changes to the community... I do not usually use SVN or GIT from a command line.
__

Furthermore: Is there an official request list where people can see if a specific request was already implemented? One user of the german board just reminds on a "keyframe to IDR cut list". As long as this thread is the closest to a "wish list", it is not easy to keep tracks of wishes and solutions.

[squid_80]

I believe there are a few ways to get around the 32/64 bit problem.
...
Never said it was practical, just possible!
I started to implement something like this, but never finished it off. The basic idea was the same, a separate process was spawned and a shared memory pool was used to pass video frame buffers between them. I never got around to finishing it, and besides the new avsfs project makes things a lot easier - just mount the script and the raw avi file is visible to any program, 32-bit or 64-bit.

[lexor]

Hey guys, a while back when GSoC started, the the guy who picked up the bframes project (I believe) said he'll be keeping people up to date either on mailing list or blog (I forget which one he said). Is that happening anywhere and if yes, does anyone have direct links?

[akupenguin]

You assume that I am a developer using Linux?
I assume that either you're a developer using Linux or Cygwin or OSX or some other sane commandline envireonment, or you're not a developer at all and depend on someone else to provide the changelog (http://mirror01.x264.nl/x264/changelog.txt).

[Inventive Software]

@devs: Came across this whilst looking for some info about new GPUs, would the instructions here: http://en.wikipedia.org/wiki/ARB_%28GPU_assembly_language%29 be of any use? My programming knowledge isn't great, but you'd probably be able to see whether this'd help in porting the search routines et al to GPUs.

[LigH]

Wonderful! This changelog now contains revision numbers. I read the same source earlier today (when it reported only r880) and missed them...

:thanks: for adding them!

[Ratio2]

I encoding i-frame only
x264.exe --ref 0 --bitrate 3000 --verbose --interlaced --keyint 1 --level 4.1 --progress -o dolphins_i2.mkv dolphins_i2.avs

MediaInfo show:
Format settings, ReFrames : 1 frame
Encoding settings : cabac=1 / ref=1 / deblock=0:0:0 / analyse=0x1:0x111 / me=hex / subme=5 / rdcmp=psy / brdo=0 / mixed_ref=0 / me_range=16 / chroma_me=1 / trellis=0 / 8x8dct=0 / cqm=0 / deadzone=21,11 / chroma_qp_offset=0 / threads=1 / nr=0 / decimate=1 / mbaff=1 / bframes=0 / keyint=1 / keyint_min=1 / scenecut=40 / rc=abr / bitrate=3000 / ratetol=1.0 / rceq='blurCplx^(1-qComp)' / qcomp=1.00 / qpmin=10 / qpmax=51 / qpstep=4 / ip_ratio=1.40 / aq=2:1.00

1) Why after encoding ref = 1 if theoretically ref frames not used? (each frame will be decoded directly to destination)
2) Why interlaced flag not set in mkv container?

[Ranguvar]

Got two guesses, might as well share them.

1.) Either it just happens to be what x264 writes in that case, perhaps some readers of this don't understand ref=0, or maybe x264 doesn't write anything at all there, and MediaInfo is just guessing. Yes, I'm grasping at straws.
2.) Perhaps there are two places an interlaced flag can be - in the container or in the H.264 stream? And x264 writes the flag to the bitstream. Or you could not set that flag and set it when using a separate muxer to Matroska.

[Dark Eiri]

Correct me if i'm wrong, but I guess x264 can't use 0 ref frames.

[Dark Shikari]

Correct me if i'm wrong, but I guess x264 can't use 0 ref frames.Technically it can by not using any P-frames at all, but I'm not sure if such a thing can be written in the stream (i.e. whether the "number of reference frames" value can be zero in the h264 frame header).

[Ratio2]

according spec (http://www.itu.int/rec/T-REC-H.264):
num_ref_frames specifies the maximum number of short-term and long-term reference frames, complementary
reference field pairs, and non-paired reference fields that may be used by the decoding process for inter prediction of
any picture in the sequence. num_ref_frames also determines the size of the sliding window operation as specified in
subclause 8.2.5.3. The value of num_ref_frames shall be in the range of 0 to MaxDpbSize (as specified in subclause
A.3.1 or A.3.2), inclusive.

[kandrey89]

according spec (http://www.itu.int/rec/T-REC-H.264):

Beh, sounds like a lawyer wrote it :devil:

[burfadel]

I've got a question relating to thread allocation. When x264 is encoding, I know that avisynth runs in it own thread, but in a multicore environment, does the multithreading nature of x264 take thread cycles away from the avisynth thread in order to encode, thus actually possibly resulting in a reduction of encoding speed because of the reduction in data flow? It wouldn't affect simple avisynth scripts unless you have say 8 cores or more, but it would affect the more complex scripts. x264 should only run in the spare cycles left from avisynth, not delay the avisynth thread in order to process its own (through priority). In fact, x264 should give all thread priority to the avisynth chain to maximise speed. Also, since other threads are taking place on the computer, when they take a certain amount of cycles away from the avisynth thread, x264 should sacrifice its own threads in another core to continue running the avisynth script. It is possible to change the affinity (which core it uses) on the fly. It can be done through the task manager by right clicking on the process, you can select which cores it can run on :) The reason why the latter would be important is since the cpu load is towards 100 percent, instead of the other system processes using the core with the least load, since the load would be pretty even they'll use any core, which on a quad core leaves a 1 in 4 chance of slowing down avisynth (and hence x264), and on a dual core 1 in 2! Since most people generally run with a reasonably complex script, and avisynth isn't multithreaded, this would be beneficial for encoding speed in most cases.

[saint-francis]

Since most people generally run with a reasonably complex script, and avisynth isn't multithreaded, this would be beneficial for encoding speed in most cases.

But don't most people who use complicated scripts use MT?

[Dark Shikari]

I've got a question relating to thread allocation. When x264 is encoding, I know that avisynth runs in it own thread, but in a multicore environment, does the multithreading nature of x264 take thread cycles away from the avisynth thread in order to encode, thus actually possibly resulting in a reduction of encoding speed because of the reduction in data flow?If a thread is blocking, the operating system will allocate it time until it no longer blocks other threads that depend on it. This will ensure that Avisynth will get all the time it needs.

[Manao]

No, avisynth won't get all the time it needs when it runs in its own thread, if the encoder is configured to use at least as much threads as CPUs. So if avisynth is the bottleneck, reducing the number of threads for x264 to #CPUs - 1 should help.

[Shinigami-Sama]

No, avisynth won't get all the time it needs when it runs in its own thread, if the encoder is configured to use at least as much threads as CPUs. So if avisynth is the bottleneck, reducing the number of threads for x264 to #CPUs - 1 should help.

or just boost avisynth priority a notch or two

[Trahald]

diff --git a/common/bs.h b/common/bs.h
index c8edbdc..3979d2f 100644
--- a/common/bs.h
+++ b/common/bs.h
@@ -141,6 +141,17 @@ static inline void bs_align_1( bs_t *s )
bs_flush( s );
}

+static inline void bs_align_10( bs_t *s )
+{
+ if( s->i_left&7 )
+ {
+ s->cur_bits <<= s->i_left&7;
+ s->cur_bits |= 1 << ((s->i_left&7) - 1);
+ s->i_left &= ~7;
+ }
+ bs_flush( s );
+}
+
/* golomb functions */

static const uint8_t x264_ue_size_tab[256] =
diff --git a/encoder/set.c b/encoder/set.c
index f47954d..aee4790 100644
--- a/encoder/set.c
+++ b/encoder/set.c
@@ -499,7 +499,7 @@ void x264_sei_version_write( x264_t *h, bs_t *s )
for( i = 0; i < length-16; i++ )
bs_write( s, 8, version[i] );

- bs_rbsp_trailing( s );
+ bs_align_10( s );

x264_free( opts );
x264_free( version );the reason why is...SEI Payload syntax
*snip*
if( !byte_aligned( ) ) {
bit_equal_to_one /* equal to 1 */ 5 f(1)
while( !byte_aligned( ) )
bit_equal_to_zero /* equal to 0 */ 5 f(1)
}is supposed be used at the end of an SEI message syntax... where as rbsp_trailing is currently used which works since the message is over but is technically wrong. (writes a 1 bit that shouldnt be there)

[akupenguin]

+ s->cur_bits <<= s->i_left&7;
+ s->cur_bits |= 1 << ((s->i_left&7) - 1);
+ s->i_left &= ~7;

You can still use bs_write1, the only thing added is the if.

Now, can anyone explain the reasoning for this rule? Normal rbsp_trailing allows one to identify the exact bit that the NAL ends on, without parsing the whole thing. But the SEI version doesn't allow that; you can't tell whether the last byte is full of data, or whether it contains some trailing bits.

[Trahald]

i was wrong.. (sorta) x264 is writing the output correctly.. within the scheme of how its layed out its technically wrong (rbsp_trailing_bits probably shouldnt be in the function , but outside of the function. a sei nalu should have rbsp_trailing bits at the end... and since user data is the only thing x264 writes in the nalu the output is correct..

(btw.. bs_align_10 as it is in regaurds to a user data sei message is a NULL result because user_data always ends aligned but i like it being there because its called that way in the spec) ..
leaving x264 code as is... if someone where to write a function that appended a second message in the same nalu it would result in incorrect data. but anyways since noone does i guess its moot.

regards to ' You can still use bs_write1, the only thing added is the if.' im not sure what you mean

[Trahald]

here is a proper layout of nal w/seisei_rbsp( ) { C Descriptor
do
sei_message( ) 5
while( more_rbsp_data( ) )
rbsp_trailing_bits( ) 5
}

sei_message() {
*snip*
if( !byte_aligned( ) ) {
bit_equal_to_one /* equal to 1 */ 5 f(1)
while( !byte_aligned( ) )
bit_equal_to_zero /* equal to 0 */ 5 f(1)
}
}

[akupenguin]

regards to ' You can still use bs_write1, the only thing added is the if.' im not sure what you mean
if( s->i_left&7 )
bs_write1( s, 1 );
instead of
if( s->i_left&7 )
{
s->cur_bits <<= s->i_left&7;
s->cur_bits |= 1 << ((s->i_left&7) - 1);
s->i_left &= ~7;
}

[Sharktooth]

possible bug.
when specifying to high values for the --sar option, x264 completey ignores and apply no SAR correction at all.

[kemuri-_9]

possible bug.
when specifying to high values for the --sar option, x264 completey ignores and apply no SAR correction at all.

what are these 'too high' values? the official h264 spec only goes as far as to define *EDIT* originally said 2:1, it is correctly 32:11 */EDIT* outside of extended mode, which makes sense for me as anything higher would really start showing the stretch *EDIT* it would probably be fairly obvious at 32:11 (nearly 3x stretch) and whoever does it should have a really good reason for doing so */EDIT*
...
looked briefly at the x264 code for it and found code that effectively reduces the sar parameters to relative primes between 0 and 65535 if possible; if either value reduces to 0 in the process, the sar is invalidated.... (which is a huge range)
...
also noticed that x264 doesn't treat the h264 spec sars of 4:3, 3:2, and 2:1 as their idc values of 14,15,16 respectively but would treat them as extended sar (idc 255) if occurred.

[Sharktooth]

ok. got it.

[Sharktooth]

what will happen with something like: --sar 3040878027465:2513169434243

[kemuri-_9]

they both overflow the int type and will likely ensue in invalidation,
since it seems that the function is accepting (default signed) ints, so the maximum positive value would be 2147483647 (2^31 -1)
so pick numbers lower than that.

assuming it will modulate into a 32 bit number, the resulting numbers would be 41181897:613566083;
the gcd is 1 so unable to reduce further and maintain ratio,
so therefore the x264 code will divide them continually by 2 (integer style) until they are less than 65535:

2513:37449
would be the result in this assumptive case.

[Sharktooth]

ok, maybe i found the AR problem in megui then
thanx.

[Selur]

small question :)
Why is
// {"1b", 1485, 99, 152064, 128, 350, 64, 64, 0, 0, 0, 1 },
commented out in the x264 sourcecode (set.c -> const x264_level_t x264_levels[]) ?

Cu Selur

[akupenguin]

Why is "1b" commented out?
Because x264's level variable is a number, there's no way to ask for "1b".

[easy2Bcheesy]

May I suggest support for slices in x264? I'm attempting to decode 1080p60 x264 on PlayStation 3 and it jerks and judders badly. However, the 1080p video (http://www.wipeouthd.com/en_GB/index.html) here decodes beautifully with no problem. The only difference in the encoding profiles between my clip and that one is that the Sony clip has eight slices, which the machine could well be decoding in parallel.

[Sagekilla]

I believe x264 actually used slice based parallelization at one point for encoding, on the same token decoders that supported this could decode it as well. I don't think there's much you can do on that end, other than -MAYBE- disabling CABAC and deblock and hope it decodes a bit faster, since they've abandoned slices for parallel encoding.

[Dark Shikari]

I believe x264 actually used slice based parallelization at one point for encoding, on the same token decoders that supported this could decode it as well. I don't think there's much you can do on that end, other than -MAYBE- disabling CABAC and deblock and hope it decodes a bit faster, since they've abandoned slices for parallel encoding.Abandoning slices for parallel encoding doesn't mean you can't still make it an option.

[Sagekilla]

True, I just find it a shame that the PS3 doesn't feature frame based parallelization.

[Sharktooth]

i thrown my PS3 outta window (well... not properly... i brought it back...).
however im noticing also some other decoders use slice based parallelism (quicktime for first...).
so, i back up the request for slices support.

[bob0r]

Hardware This. Hardware That.

Is it THAT HARD for ANY H.264 decoding device maker to make it support FULL H.264 specs (at least what x264 and all satellite providers can produce)????

To me it sounds so pathetic that Windows + CoreAVC can play all, yet all the devices(with enough cpu/chip power) can not. Therefor i hope pengvado will NEVER adjust to this kind of requests, based on pathetic support from the BIG companies!

[Sharktooth]

satellite streams are usually sliced too.. :P

[Sulik]

Hardware This. Hardware That.

Is it THAT HARD for ANY H.264 decoding device maker to make it support FULL H.264 specs (at least what x264 and all satellite providers can produce)????

To me it sounds so pathetic that Windows + CoreAVC can play all, yet all the devices(with enough cpu/chip power) can not. Therefor i hope pengvado will NEVER adjust to this kind of requests, based on pathetic support from the BIG companies!

Is this that hard for open-source SW to follow the same specs as well ?
Satellite transmission, Blu-ray media all have requirements that adds additional restrictions on top of the standard, as well as enforcing level restrictions -> there is a reason why levels are present.
Available power & memory is virtually infinite for SW decoders, but it is certainly not the case for HW.

Specifications like BD/ATSC/DVB have explicit requirements, such as that there shall at most XXX MBs per slice, level shall be 4.1, a P-frame shall not use a B-frame as a reference, etc. If you do not respect these specifications, your stream won't play on the device, it's as simple as that. All these requirements are usually there for a very good reason.

The real issue is that unlike the 14496-10 standard, these specifications are usually more difficult to obtain, or the OSS developers are just not aware of them.

[CruNcher]

Very well spoken Sulik, tough X264 target never was Hardware in the first place but this changed with main developers being more active for IPTV companies and getting confrontized with it now :)
Some people even say to give X264 a very efficient and stable VBV it would need almost a complete rewrite of it but i can't say if that is true as im no Engineer here (and some Engineers also for sure don't see every way that might be possible to go with less work but almost same result).

[Manao]

Some people even say to give X264 a very efficient and stable VBV it would need almost a complete rewrite of it That's complete bullshit. It would need some work, but mostly on the rate control, which is only a single .c file in x264.

[Selur]

@akupenguin:
Because x264's level variable is a number, there's no way to ask for "1b".
okay, I had that one coming ;)
i rephrase:
Why not switch to char/string as variables or use a not used int (e.g. 14) instead of leaving out one level?

[Shinigami-Sama]

Hardware This. Hardware That.

Is it THAT HARD for ANY H.264 decoding device maker to make it support FULL H.264 specs (at least what x264 and all satellite providers can produce)????

To me it sounds so pathetic that Windows + CoreAVC can play all, yet all the devices(with enough cpu/chip power) can not. Therefor i hope pengvado will NEVER adjust to this kind of requests, based on pathetic support from the BIG companies!

+1
if these big companies want support then they can chip in 0.001% of their yearly profit to get someone to add it...

[kemuri-_9]

hmm.... has anyone else been getting some severe performance hits from the latest revision (r955)?

from my profile logs, the new revision is encoding at severely slower speeds (the opposite of handwritten asm's purpose) on the average.

--- x264-old\x264_profile.i686.gcc-3.4.5.athlon-xp.log 2008-08-30 15:34:18.7217 (r953)
50000 -0400
+++ x264\x264_profile.i686.gcc-3.4.5.athlon-xp.log 2008-09-01 04:55:33.0156 (r955)
25000 -0400
-encoded 4350 frames, 11.48 fps, 1322.67 kb/s
+encoded 4350 frames, 9.15 fps, 1322.67 kb/s

-encoded 4350 frames, 23.02 fps, 5231.54 kb/s
+encoded 4350 frames, 11.61 fps, 5231.54 kb/s

-encoded 4350 frames, 18.90 fps, 1520.46 kb/s
+encoded 4350 frames, 11.21 fps, 1520.46 kb/s

-encoded 4350 frames, 4.18 fps, 2479.76 kb/s
+encoded 4350 frames, 4.33 fps, 2479.76 kb/s

-encoded 50 frames, 8.42 fps, 445.20 kb/s
+encoded 50 frames, 7.55 fps, 445.20 kb/s

-encoded 50 frames, 14.81 fps, 14018.51 kb/s
+encoded 50 frames, 15.61 fps, 14018.51 kb/s

-encoded 50 frames, 46.34 fps, 17216.05 kb/s
+encoded 50 frames, 50.81 fps, 17216.05 kb/s

--- x264-old\x264_profile.i686.gcc-3.4.5.pentium2.log 2008-08-30 12:47:37.9092 (r953)
50000 -0400
+++ x264\x264_profile.i686.gcc-3.4.5.pentium2.log 2008-09-01 04:00:32.8281 (r955)
25000 -0400

-encoded 4350 frames, 11.15 fps, 1322.67 kb/s
+encoded 4350 frames, 9.02 fps, 1322.67 kb/s

-encoded 4350 frames, 22.39 fps, 5231.54 kb/s
+encoded 4350 frames, 10.87 fps, 5231.54 kb/s

-encoded 4350 frames, 18.75 fps, 1520.46 kb/s
+encoded 4350 frames, 10.64 fps, 1520.46 kb/s

-encoded 4350 frames, 5.35 fps, 3119.76 kb/s
+encoded 4350 frames, 5.48 fps, 3119.76 kb/s

-encoded 4350 frames, 4.17 fps, 2479.76 kb/s
+encoded 4350 frames, 4.13 fps, 2479.76 kb/s

-encoded 50 frames, 8.33 fps, 445.20 kb/s
+encoded 50 frames, 8.49 fps, 445.20 kb/s

-encoded 50 frames, 14.61 fps, 14018.51 kb/s
+encoded 50 frames, 15.53 fps, 14018.51 kb/s

-encoded 50 frames, 45.05 fps, 17216.05 kb/s
+encoded 50 frames, 50.00 fps, 17216.05 kb/s


the 50 frame sections can generally be ignored as they haven't ran long enough to have the speed stabilize.

the only times the speeds are reasonably close is on profile runs
4: --crf 18 -b3 -m7 -r5 --thread-input --psy-rd 0.0:0.0 --me umh -8 -t1 -A all --mixed-refs --b-rdo -w --b-pyramid --direct auto --bime --no-fast-pskip
5: --crf 22 -b3 -m6 -r4 --thread-input --psy-rd 1.0:1.0 --me esa -8 -t2 -A all --mixed-refs --b-rdo --bime

[audyovydeo]

hmm.... has anyone else been getting some severe performance hits from the latest revision (r955)?



well I found r953 coming back from vacation, and ran my usual speed tests, I got about 3% slower than r928, which was the last I'd tested before leaving. I tested r938 and decided to encode my holiday vids with it, as a consequence.

my settings are even simpler than yours :

--crf 25 -r 3 --me hex -m 5 -A all -8 -t 1 --aq-mode 0 --threads auto

was going to gather more data & ask here sometime.
I figure the "slowdown" is between r935 and r953.

cheers
a/v

[Quark.Fusion]

I notice that x264 raises input thread priority, but I manage to get better cpu usage and fps (+20%) by separating processing to another process, linking with TcpServer() and lowering priority of x264 process. Also in multithreaded AviSynth new threads created with default priority.
Maybe it will be better idea to lower all x264 threads by 1 instead of raising input thread priority?

P.S. my AviSynth script is GPU-bound.

[burfadel]

Some (if not all) of the slowdown is attributed to quality improvements! :)

Some modded builds of rev 953 contained a bug that slowed it down quite considerable when --b-adapt 2 was optioned.

Before going by speed drop alone, check the quality of the output files, both visually (most important) and by the SSIM etc for comparative purposes. Its only really an issue if quality has dropped along with the speed!

[kemuri-_9]

what i posted are diff -U 0 logs generated from the fprofiling. there's no difference in the log files at all in them except the speeds:
that is all frame decisions, sizes, QPs, SSIMs, PSNRs, b-frame usage, mb decisions, ref usage, transform decisions, and bitrates ARE EXACTLY THE SAME.
the profiles are deterministic being single threaded.

same patches applied, the only difference in builds is the r955 asm.

and as another note, i don't profile from .avs scripts but from .y4m files.
so anything related to avs doesn't matter

[bob0r]

Random speedtest 900/910/920/930/940/950/955 2 runs:
http://x264.nl/x264-900-955-random-speedtest.txt

speed.bat settings:
--threads auto -B5000 -m6 -r5 --direct=temporal --me=hex -b2 -w --qcomp=0.10 -A"p8x8,i8x8,i4x4" -8 --fps=25 --progress --output NUL 720p50_mobcal_ter.yuv 1280x720

Note: Rev 940 was a regression, doesn't really count.
Note: Rev 950 > 955 indeed some difference.

Simply a basic test, do with it what you like :D

Edit:
The speedloss comes from Rev 951, our heroes are on the case!
And they most likely already solved it!

[Quark.Fusion]

Just for note: my previous post about AVS isn't related to current discussion about r955 slowdown, but to x264 in general.

[audyovydeo]

well I found r953 coming back from vacation, and ran my usual speed tests, I got about 3% slower than r928

hello Dark Shikari

r956's speed test results are coherent with the speed improvement curve you've been spoiling us with (I get 0.82% faster than r928).

thanks as usual
cheers
audyovydeo

[juGGaKNot]

r965 only uses 256 MB of ram

older versions used up to 1.2 GB of ram on my pc

[Ranguvar]

Um. Well, that's odd, and almost certainly something's amiss with your system, but isn't that a good thing if speed didn't drop too?

[Manao]

juGGaKNot : memory takes into account all the memory allocated by the avisynth script, which is often far more than the memory allocated by x264

[juGGaKNot]

Um. Well, that's odd, and almost certainly something's amiss with your system, but isn't that a good thing if speed didn't drop too?

well changed lots of stuff now so i don't realy know

moved on to crf 18 with trellis2 + subme7 + merange 32 so its realy slow, 1.8 FPS

before 2 pass with trelis 1 + subme6 + merange 16 was about 5-7 FPS for 1024x576 30 FPS

juGGaKNot : memory takes into account all the memory allocated by the avisynth script, which is often far more than the memory allocated by x264

i have 4 GB so i could care less, its just that encoding time is huge now.

i only render footage at 1024x576 <10 min now so its k i guess

[Quark.Fusion]

i have 4 GB so i could care less, its just that encoding time is huge now.
No, you can't care less because you have 4GB, instead you should care more if not using 2.5.8 AVS. This is because 32-bit apps have limit for 2GB of virtual memory — if you break limit app will crash/freeze/bug. Note that virtual memory is all allocated (not necessary used) memory and virtual memory fragmentation also counts.

[Octo-puss]

I might be posting in wrong thread, but I wanna say it anyway:
No idea what you recently changed in the code, but in 2pass encoding, the first one I am doing about 30-40FPS faster than before lately. 2nd by up to 10. Excellent work on the whole thing, keep up :) Cheers.

[juGGaKNot]

No, you can't care less because you have 4GB, instead you should care more if not using 2.5.8 AVS. This is because 32-bit apps have limit for 2GB of virtual memory — if you break limit app will crash/freeze/bug. Note that virtual memory is all allocated (not necessary used) memory and virtual memory fragmentation also counts.

I use 2.5.8 RC4 with the new b-frame, techouse r965

"%mypath%\bin\x264.exe" --crf 18 --stats "%mypath%\temp\%mymovie%.stats" --level 4.1 --ref 3 --mixed-refs --no-fast-pskip --bframes 3 --b-adapt 2 --b-pyramid --b-rdo --bime --weightb --direct auto --filter -3,-3 --subme 7 --trellis 2 --analyse all --8x8dct --me esa --merange 32 --threads auto --thread-input --cqmfile "%mypath%\bin\eqm_avc_hr.cfg" --progress --no-dct-decimate --no-psnr --no-ssim --output "%mypath%\temp\%mymovie%.mkv" "%mypath%\temp\%mymovie%.avs"

[Esc]

Versions 994 and 995 with following parameters
program --pass 2 --bitrate 1400 --stats ".stats" --ref 5 --mixed-refs --bframes 3 --b-adapt 2 --b-pyramid --b-rdo --bime --weightb --direct temporal --trellis 2 --psy-rd 0.0:0 --partitions p8x8,b8x8,i4x4 --ratetol 4.0 --qcomp 0.8 --me umh --threads auto --thread-input --progress --no-psnr --no-ssim --output "output" "input"
created through MeGUI 0.3.0.2018 both produce black video and encoding FPS is 83.02 on second pass vs 2.24 on first pass.

[martino]

r999
rm gtk

I don't remember why I allowed a gui into the repository in the first place. There's nothing that makes this one special relative to all the other x264 guis.

I'd like the devs to rethink this action. While there are many guis for win32, you don't have that many for Mac/Linux (or at least not that I'm aware of), and I think it'd be nice to have something around that works "right from the box". Why would someone need to get extra piece of software to do some very simple task (let's just say no cmd guru, or just someone that wants to do a quick encode) when you could have it in the package already. Just my opinions, any feedback is welcome.

[nm]

I'd like the devs to rethink this action. While there are many guis for win32, you don't have that many for Mac/Linux (or at least not that I'm aware of), and I think it'd be nice to have something around that works "right from the box". Why would someone need to get extra piece of software to do some very simple task (let's just say no cmd guru, or just someone that wants to do a quick encode) when you could have it in the package already. Just my opinions, any feedback is welcome.
Well, I doubt many people were actually using the GTK GUI anyway. Like with the command-line x264, the input needs to be raw video, so the user would need to use a separate tool (MEncoder/MPlayer, avs2yuv+Wine+AviSynth, ...) for decoding the source.

It's much easier to use Avidemux or Handbrake (even the CLI version). Avidemux is already included in most of the big Linux distributions.

[Kurtnoise]

iirc, you have the avs support as input on windows only...

[lexor]

iirc, you have the avs support as input on windows only...

Hence the mention of wine. According to aku, 64bit version is faster under linux through wine with avisynth than the 32bit version is under windows.

[Coolpplse]

On r998 with the MeGui Update I received x264.exe now uses 1.3GB of of 4gb's of ram in Vista 64Bit rather than the usual 600-800MB encoding 1080i MPEG2 -> 1080i x264

On some ocassions with this build even regular SD videos -> x264 use the same 1.3GB's of my ram... Is there a massive memory leak with the latest build or something????

I use Standalone-Blu-ray for 1080i and DXVA-SD-HQ for SD stuff

[Kurtnoise]

Hence the mention of wine. According to aku, 64bit version is faster under linux through wine with avisynth than the 32bit version is under windows.
I spoke for the GTK UI not in general...

[Ramir Gonzales]

Just want to ask,

I installed latest Staxrip beta (StaxRip.beta.1002.1443.exe) and after choosing "HQ Insane" profile and changing subpixel refinement and M.E.Algorithm to the highest level, so Command-Line shows :
--pass 2 --bitrate 1087 --stats ".stats" --ref 16 --mixed-refs --no-fast-pskip --bframes 4 --b-adapt 2 --b-pyramid
--weightb --direct auto --filter -1:-1 --subme 9 --trellis 1 --partitions p8x8,b8x8,i4x4,i8x8 --8x8dct --me tesa
--merange 32 --threads auto --progress --no-psnr --no-ssim --output "_EncoderOutput.264_EncoderOutput.264" ""

but when I run x264.exe (x264 core:65 r996 c89bc90) --help it says

me <string> Integer pixel motion estimation method ["hex"]
- dia, hex, umh

In other words, Staxrip proposes to use --me tesa, while x264
tells me it doesn't know --me tesa...:confused:

So my questions are, Is staxrip wrong in it's command-line, and what about these settings, are they as best as possible for a 2-pass encode targeting 700MB output ?

[akupenguin]

--help excludes options that we expect most people not to use. try --longhelp.

[nm]

and what about these settings, are they as best as possible for a 2-pass encode targeting 700MB output ?
You might want to use a higher trellis setting and enable psy-trellis:
--trellis 2 --psy-rd 1.0:1.0

[Ramir Gonzales]

You might want to use a higher trellis setting and enable psy-trellis:
--trellis 2 --psy-rd 1.0:1.0

Thanks nm ! :)
Thanks Akupenguin ! :)

[JarrettH]

Good luck on revision 1000 :devil:

[zyrill]

Hello x264-devs,

I was wondering about the cpu instruction sets being used in the latest (998 - Skystrife) x264 compile: on my Athlon64 x2 (Windsor), when I start an encoding-run, x264 reports
x264 [info]: using cpu capabilities: MMX2 SSE2Slow
and I was wondering whether it did not activate (or show?) all the other sets that could possibly be used, such as - but not only - SSE3!

[J_Darnley]

x264 only uses the features of your CPU which are fastest. You can force it with the --asm switch.

[Ramir Gonzales]

Sorry for another slightly OT question, but are there some special settings to be taken care of when encoding BLACK&WHITE movies ?

I know the "Greyscale Encoding" in xVid, but what about x264 ???

[kemuri-_9]

SSE3 doesn't activate by default on AMD chips because it's not faster than SSE2 and they don't have SSSE3 which would be faster (like Intel counterparts), so it's completely not used at all.

MMX2 and SSE2Slow flags consist of
MMX, MMXEXT, and SSE2 while avoiding SSE2 instructions that are slower on Athlon64s
(There is no SSE1 functions)

[akupenguin]

--no-chroma-me to save a little time. There is no greyscale compression mode because empty chroma compresses to nothing anyway. (There is a greyscale mode in the spec, but it just doesn't matter.)

[zyrill]

Thanks for clearing this up, J_Darnley and kemuri-_9!

Didn't know SSE3 was slower on AMDs in that particular application. Thanks again!

[Ramir Gonzales]

--no-chroma-me to save a little time. There is no greyscale compression mode because empty chroma compresses to nothing anyway. (There is a greyscale mode in the spec, but it just doesn't matter.)

So for B&W movies, --no-chroma-me .
Thanks a lot akupenguin !

[Ranguvar]

And try running the source through a Greyscale() in AviSynth, to effectively remove any rainbows or crap in the otherwise greyscale source :)

[Dark Shikari]

SSE3 doesn't activate by default on AMD chips because it's not faster than SSE2 and they don't have SSSE3 which would be faster (like Intel counterparts), so it's completely not used at all.

MMX2 and SSE2Slow flags consist of
MMX, MMXEXT, and SSE2 while avoiding SSE2 instructions that are slower on Athlon64s
(There is no SSE1 functions)More precisely, SSE3 just isn't useful for anything except working around the cacheline split bug--which only exists on Intel CPUs.

[Ramir Gonzales]

And try running the source through a Greyscale() in AviSynth, to effectively remove any rainbows or crap in the otherwise greyscale source :)
Thanks, I use Staxrip to encode, I have no idea how to do that in the processes of DGIndex, Compressibility Test, 2 x264 passes, and Muxing...:eek:

[zyrill]

I feel a bit dolty asking this since it is a feature only vaguely related to video-encoding. Anyway: is it planned - at any time in the future - to include a "stop and resume"-mode for the encoder? I ask since I thought of a handful of possibilities to stop encoding and resume it at a later point but none is really satisfactory. Two scenarios come to mind:
a) I want to encode during the day when the computer is running anyway but switch it off over night (win hibernation and standby are not to be trusted, I'm afraid - learned so just now the hard way)
b) I want to encode over night when the computer is not in use anyway but want to work during the day having full capacity of my ressources - including RAM - so I want to stop the encoding process completely during the day and not just renice it.
I would guess that it's relatively low on the priority list but then it shouldn't be so hard to implement - if one can code properly, that is (which excludes me) and it'd be a real handy feature, I'm sure.

[Dark Shikari]

I feel a bit dolty asking this since it is a feature only vaguely related to video-encoding. Anyway: is it planned - at any time in the future - to include a "stop and resume"-mode for the encoder? ctrl-s

(and standby and hibernate both work fine for me, no idea what you're talking about)

[zyrill]

ctrl-s

(and standby and hibernate both work fine for me, no idea what you're talking about)

Yesterday evening I hibernated my computer and this morning when I switched it on again, it had a hickup booting windows and the hibernation-state was lost. :s That's why I am a bit vexed with windows hibernation. And that's why I asked about a resume-mode where it wouldn't matter whether the process was dead intermittently.
Thanks for the tip with Ctrl+S didn't know about that...

[Atak_Snajpera]

Yesterday evening I hibernated my computer and this morning when I switched it on again, it had a hickup booting windows and the hibernation-state was lost. :s That's why I am a bit vexed with windows hibernation. And that's why I asked about a resume-mode where it wouldn't matter whether the process was dead intermittently.
Thanks for the tip with Ctrl+S didn't know about that...
If you had Vista SP1 you would not have any problems with hibernation :)

[burfadel]

Depends whether it was hibernation or 'suspend to ram'. A lot of people call the latter hibernation. The difference is hibernation saves the content of the RAM in to a file, which can take some time to shutdown and load up on large memory systems (still loads quicker than normal though), and the computer turns off once the file is saved. Suspend to ram (STR) works by shutting down the computer but keeping a small amount of power to the RAM, keeping the data in RAM instead of writing it to a file. This is almost instantaneous to shut down and load up. Its a feature that has to be supported by the motherboard and hardware (which all do now), and has been around since at least early XP days - just wasn't a very known feature until recently. Which method is your computer currently using? Since they work on a different principle one may be causing a problem whereas the other may not.

[Ranguvar]

If you had Vista SP1 you would not have any problems with hibernation :)

*cough*linux*cough*

[Sagekilla]

Thanks for the tip Dark Shikari. I'd love to go try this now, since I do encodes across 2 computers (the encoding computer being my laptop) and I can't leave my laptop at home since I need to bring it in to take notes when I have class.

I'm just curious if I can do something like: In the middle of an encode, ctrl+s the x264 instance, put laptop in sleep mode, then shut down desktop. When I get back home, start up the desktop and open the AvsP instance of my TCPServer(), and resume x264 on my laptop.

[Atak_Snajpera]

Which method is your computer currently using?
Both methods work.

[Inventive Software]

*cough*linux*cough*

Hibernation's broken with my Ubuntu install. :p

[LoRd_MuldeR]

One could run x264 in a virtual machine via free VirtualBox (http://www.virtualbox.org/wiki/Downloads) tool. It can save and restore the machine state...

[Quark.Fusion]

Built-in save and resume will be better, imho. Installing virtual machine just for one function is too complicated :) Checkpoints also can be useful feature if something going wrong during long encode (power outage or system crash). Sure, virtual machine can do all that, but at very high overhead (more memory, slower speed, time to setup and extra space on disk). Second, if you prefer to save encode over network you risking to broke encode during timeout or other network failure (I'm belive that x264 isn't smart enough to retry write operation).

[burfadel]

Both methods work.

I was commenting to Zyrill :) but its good to know that it doesn't affect it with the 2 different methods!

Due to the time it takes, especially with 2gb or more of RAM for hibernation, hibernation probably is really only beneficial for laptops in battery mode that shut down when the battery is low so no work is lost. STR still uses power because it keeps the RAM charged, hibernation is doesn't. The battery life is quite long in STR mode, but it still uses power and eventually it will shut off losing your work (unless is plugged in to power). For those wondering about power surges and thunderstorms etc, STR is as safe as having the computer off by shutdown (so the power is still connected), as there is always power running through the motherboard :)

[Shinigami-Sama]

One could run x264 in a virtual machine via free VirtualBox (http://www.virtualbox.org/wiki/Downloads) tool. It can save and restore the machine state...

dam beat me to it
<3 VMs

[Ranguvar]

Hibernation's broken with my Ubuntu install. :p

The problem would be that you're using Ubuntu :p

Go with a distro like Gentoo, Arch, Slackware/Slamd64, or even Debian. More minimalist, usually, and easier to get the the root of such problems. Ubuntu IMO hides too much. Certain kernel options need to be enabled to support hibernation.

Trust me, in one day with Arch, you'll learn more than a month with Ubuntu. Such was it with me, at least :) Anyone willing to read and ask questions can learn quite fast, and the feeling of power over your computer that you get is one of the best parts (I'm a control freak over my PC xD).

Then you get to wow Windows users with the likes of the text-editor-on-steroids, Vim, the lean speed of a custom kernel, and quick bash scripts that can do things in seconds that would take a very long time with a GUI.

Anyways, I agree with Quark... VMs seem cumbersome for a task so 'simple' and often-used as encoding. Plus, the overhead.

[Sagekilla]

Actually I think if you're running something as intense as a video encode, the overhead should be very tiny. I don't think x264 makes a lot of kernel calls that would necessitate translation (which would eat up cpu time)

[Ranguvar]

Actually I think if you're running something as intense as a video encode, the overhead should be very tiny. I don't think x264 makes a lot of kernel calls that would necessitate translation (which would eat up cpu time)

/me == efficiency freak :)

But your point is valid. I need to think more :p

[Quark.Fusion]

There is also usability and memory overhead.

[Ranguvar]

Are there any VMs that will let you allocate a dynamic amount of RAM? Actually, Windows would probably hate that... too used to memory hotplugging in Linux.

[Sagekilla]

If you have a nice amount of RAM (4 GB) you can just give an instance of a WinXP VM 1 - 1.5 GB of memory and not worry. Personally I don't think there any VMs that let you dynamically allocate RAM though.

/me == efficiency freak :)

But your point is valid. I need to think more :p

I am too, but most people don't realize how efficient VMs (especially with the proper hardware virtualization) are today. It would've been a very noticeable difference between native vs virtual a few years ago, but it's probably even less when you're dealing with something like x264.

[LoRd_MuldeR]

If you give ~1 GB of RAM to the VirtualBox, but the guest system doesn't use all of that yet, will it still occupy 1 GB of physical memory of the host system?

I guess Windows wouldn't keep those unused pages in the physical RAM, would it?

[zyrill]

One could run x264 in a virtual machine via free VirtualBox (http://www.virtualbox.org/wiki/Downloads) tool. It can save and restore the machine state...
Yeah, one possibly could - but apart from the speed issue, the same applies as for changing os: stupid. Why would anyone with half a mind set up a VM or another OS just so one app runs a fraction of a percent more smoothly?
burfadel: When I said hibernation, I ment it. So to clear things up: suspend to disk. :) And all that isn't even the reason I posted my initial question: I am quite versed with computers and I do not seek help to investigate why hibernation isn't working. It just isn't and I won't do anything about it since I don't need it anyway. I don't even care - I just wanted to know whether a handy stop-and-resume feature was somehow planned.
And I didn't mean something like "Ctrl+S" neither, because although I didn't know about that one, it doesn't really change the situation now, does it? I could just as well use PsSuspend!
Atak_Snajpera: And it does NOT work with my machine (apparently, else I wouldn't have brought up the issue). Also, bringing up OS-choice in this discussion is a bit stupid, really, because no one is going to change his system just because one application might run a bit smoother on that particular system. It's not like the x264-encoder would be the killer app on linux/(input your favourite system here) compared to win 32/xp/whatever.
#########################
Kinda stupid - you'll have to assent - having properly thought it through now yourself?

[Sagekilla]

Zyrill, it's not a question of how "smooth" things run, it's more of being able to start and stop an encode at will. Any one of the various choices will give roughly the same speed and all will be very smooth. The only variation really is memory usage (in the case of a VM) and how well it handles starting and stopping.


Still, I think it would be extremely useful if x264 could save it's state to a file in case it ends before the encode is finish. I could imagine something that would work well would be:

pseduocode:
save current state
Begin encoding of next frame
save current state
save encoded frame
....



I'd imagine the state file could be something as simple as "Last encoded frame: 24000," and that way you could just call it somehow like --state "encoding.state"

[DarkZell666]

You are in a particular situation in which you have few possiblities, and using a VM is one of the few (unless you're ready to drop one of your other requirements because you think it "costs" you less to drop it rather than to set up a VM; which is "only" a 1-hour job afterall).

Another solution would be to investigate the hibernation issue, but you said you don't want that, so you're stuck with "ctrl+s" and no hibernation.

It's your decision, and your decision won't change our lives.
The solutions that have been suggested aren't stupid, they're just not convenient for you :)

[Shinigami-Sama]

Are there any VMs that will let you allocate a dynamic amount of RAM? Actually, Windows would probably hate that... too used to memory hotplugging in Linux.

vmware only uses as much as it need at any given time so...

also if the VM is only doing IO theres very little overhead

also I've played many a 3d games in VMs and they work just as well in a VM as they do on a 'real' box
only thing would be using threads, you have to tell VMware how many threads to use oh noes~
*VM-iphile*

[Sagekilla]

There you go :) The only applications where VM's would be heavily constrained by a CPU intensive activity is one where it's doing a lot of read/writes. Video encoding does read writes, but it's in huge chunks, not millions of tiny ones.. That's the kind of load that a VM can virtualize with little loss in speed.

[akupenguin]

I just wanted to know whether a handy stop-and-resume feature was somehow planned.
There will never be a pause command in x264, because SIGSTOP (or OS-specific equivalent) is the correct way to do that.
There will never be a priority command in x264, because `nice` is the correct way to do that (and if your OS doesn't have a nice-equivalent, too bad).
In order to accept a suspend-to-disk command, someone would have to convince me that there's some use case where neither hibernate nor pause is the correct solution. Furthermore, checkpointing at every frame would be way more trouble than it's worth. Checkpoint at keyframes, then only ratecontrol state needs to be saved.

[zyrill]

akupenguin: I apologize for being a bit fuzzy in my statements again. I was not asking for a SIGSTOP feature in x264 - as I said, there's plenty of ways to do that in nearly every OS. Neither was I looking for a "renice"-feature since apparently, both features are quite superfluous really. I don't know a single OS that doesn't include them. What I was asking is whether something like the checkpoint feature that you mentioned would be implemented. There's a thousand reasons, really - the prominent ones like process crash, power outage, system crash, accidental user error etc. are just a short list. Anyway: it's really annoying if 50 hours or so of raw cpu time just go to waste because x264 just isn't aware that it already encoded a fraction of a clip and starts again from zero.
I don't know if this is the way to go here but the first though that comes to mind is: why not just write the output from keyframe to keyframe saving the info needed intermittently in a separate file so if the encoder process is somehow disturbed in between keyframes i can always pick up on the last keyframe and just continue encoding from there.

[akupenguin]

Ooh, there's a linux kernel patch out which supports checkpointing of arbitrary processes. The previous user-space coredump based solutions were somewhat limited. I may soon get to reject checkpointing too from x264 as having a standard solution. ;)

[Quark.Fusion]

But there is no checkpointing of arbitrary processes on windows :) (If there is please point to me)

Checkpointing at keyframes is acceptable solution if it will not allow cases when termination at saving new checkpoint will destroy also previous one :). Some stop-and-continue button will be also very useful, like Ctrl-C to encode to next checkpoint and exit and double Ctrl-C to terminate at last saved checkpoint. It will make possible multi-day encodes with handy tray gui and/or scheduler.

Avisynth isn't relible and I have situations when it crashes at near 90% of source clip. Usual case is when it run over 2gb userspace limit.

[akupenguin]

Some stop-and-continue button will be also very useful, like Ctrl-C to encode to next checkpoint and exit and double Ctrl-C to terminate at last saved checkpoint.
If the duplicate work of encoding 100 frames bothers you, you need to buy a CPU less than 10 years old.

[Quark.Fusion]

I just thinking about some form of background encoding (managed by frontend) when PC isn't used. As you can't predict at which intervals encode can run, imho, it's better to finish previous task that abort. But if it bothering to code — your right not to do it. Anyway if checkpoint feature will be implemented it's just question of small patch to allow such choice.

[kemuri-_9]

If i really need to pause x264 for some reason, I just use Process Explorer (procexp)'s pause and resume feature.

since it does have the ability to suspend a process indefinitely like that.

[laserfan]

If i really need to pause x264 for some reason, I just use Process Explorer (procexp)'s pause and resume feature.

since it does have the ability to suspend a process indefinitely like that.
Wow, I did not know that about procexp--thanks for the tip!

[LoRd_MuldeR]

In fact you can't "pause" a process on Windows. What you can do (and what ProcExp does) is: Enumerate and suspend all threads of a process.
In order to resurrect that process, you enumerate and resume all the threads.

This seems to work, but there's no guarantee that it will work properly for you every time...

[kemuri-_9]

I use suspend+resume quite frequently and have had no problems as of yet.

[Dark Shikari]

In fact you can't "pause" a process on Windows.Isn't that just a SIGSTOP?

[LoRd_MuldeR]

Isn't that just a SIGSTOP?

I know that signals are used on POSIX systems, but AFAIK windows doesn't use any signals.
If at all, they are emulated for POSIX apps using the Win32 API. And the Win32 API doesn't have a function to pause a process.
Only functions to suspend and resume threads...

http://www.codeproject.com/KB/threads/pausep.aspx

[akupenguin]

If you just don't allocate any more cpu-time to any of the threads in a process, that's equivalent to a SIGSTOP. (Unlike all other POSIX signals except SIGKILL, you can't catch SIGSTOP, so the fact that it's implemented as a signal doesn't really matter.)
If, as suggested in the linked article, you can create deadlocks by pause/resuming individual threads, then the threaded app was buggy already, and you should deadlock it so that we can find and fix the bugs.

[ACrowley]

Im want to encode some Blurays to 1080 x264 for private purposes,

Ive nothing changed. Not my System, not my AVS, and its the same Source.
HDDVD 1080p VC1 to x264 1920x800 9.2Mbps

Ive done the same encode in April with a older x264 Build in megui

1st Pass 33,28fps
2nd PAss 5,56 fps

Now i use the latest techhouse Build x264_x86_r999_techouse
and megui 0.3.0.3004 ,with standard DXVA HD HQ Profile 9.2Mbps 1920x800

program --pass 2 --bitrate 9231 --stats ".stats" --level 4.1 --ref 4 --mixed-refs --bframes 3 --b-adapt 2 --weightb --direct auto --filter -3:-3 --subme 7 --trellis 1 --partitions p8x8,b8x8,i4x4,i8x8 --8x8dct --vbv-bufsize 50000 --vbv-maxrate 50000 --me umh --threads auto --thread-input --progress --no-psnr --no-ssim --output "output" "input"

1st Pass 11,99 fps
2nd Pass 5.58 fps

So, you can see theres nothing changed in 2nd Pass. its the same Perfomance.
But the 1st Pass ? Only ~12fps in 1 pass and i have a CPU Load around 60%. 2nd Pas 100% Cpu Load.

So, what can be the Reason ? I know there a lot of complex new changes in 264 (psyRdo,Ada.Bframe etc etc). As i say its the same System. nothing changed

Why is the Perfomance so much lower in 1st Pass ?

[nurbs]

--b-adapt 2 slows down first pass considerably. Frametype decision(?) is done in first pass, but isn't threaded.

[J_Darnley]

--b-adapt 2 didn't exist in April and frame type decision is only run on the first pass.

[ACrowley]

Thx.. i found out that its adaptive b-adapt 2 what slows down the 1pass so heavy. The fps are again ~ 33 fps without b-adapt 2 in 1st pass

As i said, the 2nd Pass was as fast as with older builds q ~5.88fps

[burfadel]

It should be pointed out that the benefits of the new b-frame decision make that drop in speed worthwhile, if the drop in speed is acceptable for you!

[toyotabedzrock]

There you go :) The only applications where VM's would be heavily constrained by a CPU intensive activity is one where it's doing a lot of read/writes. Video encoding does read writes, but it's in huge chunks, not millions of tiny ones.. That's the kind of load that a VM can virtualize with little loss in speed.

You can created a second drive with the "Use Physical Disk" option in VMware. Then use the second drive for all x264 related files, it will cut down on the overhead. Also windows XP can be trimmed to run with just 20-30 processes, disabled networking etc... and you will have a very fast VM.

Also place the page file for the VM to the directly mapped disk and set it to a static size.

If you have enough memory then you can also use the "Disabling Memory Trimming" and "Fit all virtual machine memory into reserved host RAM" settings.

There will never be a pause command in x264, because SIGSTOP (or OS-specific equivalent) is the correct way to do that.
There will never be a priority command in x264, because `nice` is the correct way to do that (and if your OS doesn't have a nice-equivalent, too bad).
In order to accept a suspend-to-disk command, someone would have to convince me that there's some use case where neither hibernate nor pause is the correct solution. Furthermore, checkpointing at every frame would be way more trouble than it's worth. Checkpoint at keyframes, then only ratecontrol state needs to be saved.

Perhaps check-pointing after 4 or more keyframes. I'm not sure how x264 handles writes, but if it writes to often and the power goes out the checkpoint file could be corrupted.

[Sagekilla]

Indeed, but like I said there's no realistic situation I can imagine where something like video encoding would suffer from a big performance hit under a VM. x264 is generally compute bound, not IO bound, and IO bound apps tend to perform worse than compute bound apps in a VM environment.

Still, I wish we could one day somehow add in start/stop to x264. I might go dive into the code though and make a patch to learn a thing or two, so we'll see if that ever turns out being successful ;)

[MrCommunistGen]

I don't know if this is relevant here, but Skystrife's 1016 build (the default DL for MeGUI) crashes immediately for me when encoding starts, but the (vanilla?) build from x264.nl works fine.
My command line:
program --pass 2 --bitrate 950 --stats ".stats" --ref 7 --mixed-refs --no-fast-pskip --bframes 5 --b-pyramid --weightb --direct auto --filter -1:-2 --subme 9 --trellis 2 --partitions all --8x8dct --me umh --threads auto --thread-input --progress --no-dct-decimate --no-psnr --no-ssim --output "output" "input"
I also tried a few changes but with no change in end result. I tried getting rid of P4x4 (my only change from previous settings) because after reading the changelog I was going to give it a try. The MeGUI log simply states that encoding starts, then finishes 4 seconds later.

-mcg

[XhmikosR]

Read here (http://forum.doom9.org/showthread.php?p=1208804#post1208804).:)

[Sagekilla]

Currently running a r1016 vs r999 test on an extremely difficult clip with loads of motion @ 60 fps. No frames are dupes, each one has genuine motion, including lots of small scale (perfect for testing sub 8x8) and large scale motion. None of the settings used were particularly "sane" because the source requires extremely slow settings to get small file sizes for minimal lossy encoding.

Results for r1016 @ 1264x688:

C:\Users\Mike\Desktop\Test Files\Metamorphosis>x264 --crf 14 --ref 9 --mixed-
refs --bframes 16 --b-adapt 2 --weightb --b-pyramid --subme 9 --me umh --partiti
ons all --8x8dct --trellis 2 --psy-rd 0.7:0.7 --aq-strength 0.7 --threads auto -
-progress --output "video_1016.264" --stats "stats_1016.log" "source.avs"
avis [info]: 1264x688 @ 60.00 fps (1200 frames)
x264 [info]: using cpu capabilities: MMX2 SSE2Fast SSSE3 Cache64
x264 [info]: profile High, level 4.0
x264 [info]: slice I:5 Avg QP:13.88 size:245787 PSNR Mean Y:50.82 U:54.07
V:54.88 Avg:51.73 Global:51.57
x264 [info]: slice P:750 Avg QP:16.67 size: 70676 PSNR Mean Y:45.39 U:51.12
V:52.47 Avg:46.67 Global:46.57
x264 [info]: slice B:445 Avg QP:19.56 size: 22327 PSNR Mean Y:45.18 U:50.63
V:50.24 Avg:46.33 Global:46.20
x264 [info]: consecutive B-frames: 35.3% 35.3% 29.4% 0.0% 0.0% 0.0% 0.0% 0.
0% 0.0% 0.0% 0.0% 0.0% 0.0% 0.0% 0.0% 0.0% 0.0%
x264 [info]: mb I I16..4: 49.1% 38.3% 12.6%
x264 [info]: mb P I16..4: 0.6% 0.0% 0.0% P16..4: 28.0% 4.4% 17.4% 5.0% 9
.6% skip:34.9%
x264 [info]: mb B I16..4: 0.2% 0.0% 0.0% B16..8: 10.2% 2.3% 5.2% direct:
10.6% skip:71.5% L0:32.7% L1:21.3% BI:46.0%
x264 [info]: 8x8 transform intra:19.8% inter:16.3%
x264 [info]: ref P L0 53.3% 5.1% 1.9% 1.9% 1.3% 2.7% 7.1% 8.2% 18.4%
x264 [info]: ref B L0 78.5% 5.4% 2.0% 2.3% 2.2% 1.1% 4.8% 3.7%
x264 [info]: ref B L1 98.2% 1.8%
x264 [info]: SSIM Mean Y:0.9945513
x264 [info]: PSNR Mean Y:45.333 U:50.953 V:51.650 Avg:46.565 Global:46.443 kb/s:
25668.65

Results for r999 @ 1264x688:
C:\Users\Mike\Desktop\Test Files\Metamorphosis>x264r999 --crf 14 --ref 9 --mi
xed-refs --bframes 16 --b-adapt 2 --weightb --b-pyramid --subme 9 --me umh --par
titions all --8x8dct --trellis 2 --psy-rd 0.7:0.7 --aq-strength 0.7 --threads au
to --progress --output "video_999.264" --stats "stats_1016.log" "source.avs"
avis [info]: 1264x688 @ 60.00 fps (1200 frames)
x264 [info]: using cpu capabilities: MMX2 SSE2Fast SSSE3 Cache64
x264 [info]: slice I:5 Avg QP:13.89 size:248864 PSNR Mean Y:50.79 U:54.24
V:54.89 Avg:51.72 Global:51.56
x264 [info]: slice P:750 Avg QP:16.67 size: 69913 PSNR Mean Y:45.20 U:50.85
V:52.05 Avg:46.47 Global:46.37
x264 [info]: slice B:445 Avg QP:19.56 size: 22458 PSNR Mean Y:45.14 U:50.39
V:49.94 Avg:46.26 Global:46.13
x264 [info]: consecutive B-frames: 35.3% 35.3% 29.4% 0.0% 0.0% 0.0% 0.0% 0.
0% 0.0% 0.0% 0.0% 0.0% 0.0% 0.0% 0.0% 0.0% 0.0%
x264 [info]: mb I I16..4: 48.9% 38.3% 12.7%
x264 [info]: mb P I16..4: 0.6% 0.0% 0.0% P16..4: 28.3% 6.9% 20.8% 3.3% 5
.3% skip:34.9%
x264 [info]: mb B I16..4: 0.2% 0.0% 0.0% B16..8: 10.5% 2.3% 5.3% direct:
10.7% skip:71.0% L0:32.4% L1:21.4% BI:46.2%
x264 [info]: 8x8 transform intra:19.7% inter:20.0%
x264 [info]: ref P L0 53.2% 5.2% 1.9% 1.8% 1.3% 2.7% 7.1% 8.3% 18.4%
x264 [info]: ref B L0 78.2% 5.6% 2.1% 2.4% 2.2% 1.1% 4.8% 3.6%
x264 [info]: ref B L1 98.2% 1.8%
x264 [info]: SSIM Mean Y:0.9943849
x264 [info]: PSNR Mean Y:45.204 U:50.694 V:51.283 Avg:46.413 Global:46.294 kb/s:
25469.25

encoded 1200 frames, 0.84 fps, 25469.88 kb/s

I'm curious how it'll perform on a more sane source :)

[Adub]

What kind of encoding speed did you get with 1016?

[Adub]

No, that is with 999. I asked about 1016.

[Sagekilla]

Oops, I forgot to comment properly. The second one is with r999. It doesn't help any that I was doing some tasks with x264 running in the background, so I'll have to re-do the test to get a more accurate idea of the speed.

Edit: Crap, just noticed what went wrong. IIRC, 1016 was slightly (about .02 fps) slower than 999, but like I said I was doing lots of work while it was encoding so it's not a perfect image of how well x264 performed.

[Adub]

Okay, thanks for the test.

[Dark Shikari]

Oops, I forgot to comment properly. The second one is with r999. It doesn't help any that I was doing some tasks with x264 running in the background, so I'll have to re-do the test to get a more accurate idea of the speed.

Edit: Crap, just noticed what went wrong. IIRC, 1016 was slightly (about .02 fps) slower than 999, but like I said I was doing lots of work while it was encoding so it's not a perfect image of how well x264 performed.I would be surprised if it wasn't slower; the p4x4 improvements do cost speed.

[Sagekilla]

If I tested with a completely idle environment (Me not browsing the internets and unpacking some data in the background) then I expect the two to be -very- close to each other. Most of the settings I use are so ridiculously slow (esp --ref 9 and --bframes 16 --b-adapt 2) that it probably wouldn't make a huge difference anyway.

[kemuri-_9]

here was a quick test, i just did it on what i profile with, which is pretty crazy in motion and fairly complex
with my somewhat standard (mentally insane) SD material settings.

r999:

x264_r999_athlon --crf 18 --min-keyint 1 --keyint 500 --scenecut 50 --ref 16 --mixed-refs --deblock 0:0
--no-fast-pskip --b-adapt 2 --bframes 16 --b-bias 5 --b-pyramid --weightb --direct auto --subme 9 --direct-8x8 -1
--trellis 2 --analyse p8x8,p4x4,b8x8,i8x8,i4x4 --8x8dct --qpmin 0 --qpmax 51 --qpstep 50 --pbratio 1.4 --qcomp 0.65 --me
tesa --merange 24 --psy-rd 0.8:0.0 --thread-input --threads auto --progress --no-dct-decimate --aq-strength 0.8 --level
4.1 --log-file r999.x264.crf.log --output NUL x264_build1.y4m
yuv4mpeg: 640x480@30/1fps, 0:0
x264 [info]: using cpu capabilities: MMX2 SSE2Fast
x264 [info]: profile High, level 4.1
x264 [info]: slice I:19 Avg QP:13.67 size: 13837 PSNR Mean Y:62.56 U:64.49 V:62.77 Avg:62.77 Global:51.67
x264 [info]: slice P:424 Avg QP:20.54 size: 22735 PSNR Mean Y:45.24 U:48.23 V:45.90 Avg:45.62 Global:42.96
x264 [info]: slice B:537 Avg QP:24.58 size: 11225 PSNR Mean Y:41.57 U:45.50 V:42.14 Avg:42.05 Global:41.13
x264 [info]: consecutive B-frames: 12.2% 19.6% 61.5% 6.2% 0.5% 0.0% 0.0% 0.0% 0.0% 0.0% 0.0% 0.0% 0.0% 0.0% 0.0% 0.0% 0.0%
x264 [info]: mb I I16..4: 48.5% 49.2% 2.3%
x264 [info]: mb P I16..4: 5.5% 32.7% 2.0% P16..4: 25.8% 14.7% 10.0% 0.6% 0.2% skip: 8.5%
x264 [info]: mb B I16..4: 1.0% 9.2% 0.4% B16..8: 47.3% 5.0% 4.2% direct:12.2% skip:20.6% L0:37.0% L1:40.0% BI:23.0%
x264 [info]: 8x8 transform intra:79.9% inter:84.1%
x264 [info]: direct mvs spatial:99.4% temporal:0.6%
x264 [info]: ref P L0 62.2% 14.5% 6.9% 3.3% 2.4% 2.1% 2.0% 1.2% 1.0% 0.8% 0.7% 0.6% 0.6% 0.6% 0.6% 0.4%
x264 [info]: ref B L0 72.5% 12.2% 4.0% 2.6% 1.9% 1.4% 1.1% 0.9% 0.7% 0.6% 0.5% 0.5% 0.4% 0.4% 0.3%
x264 [info]: ref B L1 86.9% 13.1%
x264 [info]: SSIM Mean Y:0.9874417
x264 [info]: PSNR Mean Y:43.566 U:47.051 V:44.166 Avg:43.996 Global:41.921 kb/s:3901.35
x264 [info]: encoded 980 frames, 2.23 fps, 3901.35 kb/s


r1016

x264_r1016_athlon --crf 18 --min-keyint 1 --keyint 500 --scenecut 50 --ref 16 --mixed-refs --deblock 0:0
--no-fast-pskip --b-adapt 2 --bframes 16 --b-bias 5 --b-pyramid --weightb --direct auto --subme 9 --direct-8x8 -1
--trellis 2 --analyse p8x8,p4x4,b8x8,i8x8,i4x4 --8x8dct --qpmin 0 --qpmax 51 --qpstep 50 --pbratio 1.4 --qcomp 0.65 --me
tesa --merange 24 --psy-rd 0.8:0.0 --thread-input --threads auto --progress --no-dct-decimate --aq-strength 0.8 --level
4.1 --log-file r1016.x264.crf.log --output NUL x264_build1.y4m
yuv4mpeg: 640x480@30/1fps, 0:0
x264 [info]: using cpu capabilities: MMX2 SSE2Fast
x264 [info]: profile High, level 4.1
x264 [info]: slice I:19 Avg QP:13.64 size: 13689 PSNR Mean Y:62.56 U:64.51 V:62.75 Avg:62.77 Global:51.67
x264 [info]: slice P:424 Avg QP:20.54 size: 22607 PSNR Mean Y:45.23 U:48.31 V:45.95 Avg:45.63 Global:42.96
x264 [info]: slice B:537 Avg QP:24.54 size: 11304 PSNR Mean Y:41.56 U:45.70 V:42.29 Avg:42.08 Global:41.16
x264 [info]: consecutive B-frames: 12.2% 19.6% 61.5% 6.2% 0.5% 0.0% 0.0% 0.0% 0.0% 0.0% 0.0% 0.0% 0.0% 0.0% 0.0% 0.0% 0.0%
x264 [info]: mb I I16..4: 49.8% 47.8% 2.3%
x264 [info]: mb P I16..4: 5.6% 32.8% 2.0% P16..4: 26.1% 15.4% 8.1% 0.9% 0.6% skip: 8.5%
x264 [info]: mb B I16..4: 1.0% 9.2% 0.4% B16..8: 47.0% 5.1% 4.1% direct:12.1% skip:21.0% L0:37.2% L1:39.5% BI:23.3%
x264 [info]: 8x8 transform intra:79.8% inter:83.0%
x264 [info]: direct mvs spatial:98.7% temporal:1.3%
x264 [info]: ref P L0 62.2% 14.3% 6.9% 3.3% 2.4% 2.1% 2.0% 1.2% 1.0% 0.8% 0.8% 0.6% 0.7% 0.6% 0.6% 0.4%
x264 [info]: ref B L0 72.4% 12.2% 4.0% 2.7% 1.9% 1.4% 1.1% 0.9% 0.7% 0.6% 0.5% 0.5% 0.4% 0.4% 0.3%
x264 [info]: ref B L1 86.9% 13.1%
x264 [info]: SSIM Mean Y:0.9874128
x264 [info]: PSNR Mean Y:43.555 U:47.195 V:44.267 Avg:44.018 Global:41.943 kb/s:3897.71
x264 [info]: encoded 980 frames, 2.21 fps, 3897.71 kb/s


lower QPs @ lower bitrate -> win in my book

[Snowknight26]

--log-file
Custom build or is it just not mentioned in --longhelp?

[kemuri-_9]

custom build... w/ komisar's log file patch.
i do lots of batch encoding it and it comes in handy for writing the x264 output to a file, instead of getting lost to the shell/command prompt foreverz

[Dark Shikari]

custom build... w/ komisar's log file patch.
i do lots of batch encoding it and it comes in handy for writing the x264 output to a file, instead of getting lost to the shell/command prompt foreverzWhy, because you're incapable of doing 2> filename?

[kemuri-_9]

Why, because you're incapable of doing 2> filename?

well, w/ --progress, it totally bloats it all up.
since i do often like to frequently look at how the encode is progressing.

i used to do 2>&1 | tee *.log but again, the progress bloat came around.

[Snowknight26]

Why, because you're incapable of doing 2> filename?

It was bad enough before you switched from updating the output every .1%, but now every that its every 10 frames would make that file huge. Very handy patch.

[Dark Shikari]

It was bad enough before you switched from updating the output every .1%, but now every that its every 10 frames would make that file huge. Very handy patch. 2>&1 | grep -v "frames, " > filename

Same thing, no needless modification of x264.

[kemuri-_9]

there's no way possible with tee and/or grep to have what you want to write to file and have it display the progress to the screen for when you want to occasionally want to check up on it.

if you do
2>&1 | tee file
you get to see everything as normal, but then the file gets the progress bloat

if you do
2>&1 | grep -v " eta " | tee file
the file gets what you want (besides the I slice statistics, since they're removed with the last progress line), but you don't see jack.

the purpose of --progress and then the purpose of what you would use 2> for are counters to each other. they just don't work together well
with the log file patch you can get --progress without some drawback.

[Sagekilla]

Kemuri, which processor do you have? It might just me forgetting what it's like to encode SD material (HD + MVDegrain3 = slooow!) but I was a bit surprised it was so fast @ 2 fps.

[kemuri-_9]

it's listed in my specs tag in my signature, but it's a phenom 9850.
why i have -march=athlon builds, since it's faster on my phenom than -march=pentium2 (though i make both + fprofiled)

this was also from a y4m as indicated from the command line, which is practically raw yv12 data just wrapped in a header specifying its properties (why the 980 frames at 640x480 pixels file is 430 MB).
so any avisynth overhead is not included in this.

[crypto]

@Dark Shikari
I normally do fast HD encodes @ fps>25 and find the speed penalty of "--b-adapt 2" too high compared to the PQ gain. What do you think? Is it possible to dynamically switch to "--b-adapt 2" during fades and dissolves.

[LoRd_MuldeR]

@Dark Shikari
I normally do fast HD encodes @ fps>25 and find the speed penalty of "--b-adapt 2" too high compared to the PQ gain. What do you think? Is it possible to dynamically switch to "--b-adapt 2" during fades and dissolves.

Lower "--bframes" to a sane value, if you use "--b-adapt 2" ;)

Also I doubt that you can use "--b-adapt 2" only in places where it is better than "--b-adapt 1".
How do you want to decide, without actually running "--b-adapt 2" on the all frames?

[Shinigami-Sama]

Lower "--bframes" to a sane value, if you use "--b-adapt 2" ;)

Also I doubt that you can use "--b-adapt 2" only in places where it is better than "--b-adapt 1".
How do you want to decide, without actually running "--b-adapt 2" on the all frames?

zones?

also yes sane b-adapt 2 <= 5
even 5 is a bit much

[Sagekilla]

6 is about the upper limit I've seen used. I even tried an extremely predictable source (A 24 mp image that has a 1280x720 cropping rectangle scroll across it @ 1 pixel/s) and it never used more than even 6 B's. For most content I've never seen a source (except that artificial one) use 6 B's more than 1% of the time. For 5 B's, it was 2-4% at best. If you need speed, 4 should give you all the speed you need with very negligible decrease in quality/compression.

[LoRd_MuldeR]

Lower "--bframes" to a sane value, if you use "--b-adapt 2" ;)

Also I doubt that you can use "--b-adapt 2" only in places where it is better than "--b-adapt 1".
How do you want to decide, without actually running "--b-adapt 2" on the all frames?

zones?

That would mean the user has to decide manually for each frame whether it's a fade or not.
Having to tune the encoder for individual frames is not exactly what you want from a good encoder...

[kemuri-_9]

That would mean the user has to decide manually for each frame whether it's a fade or not.
Having to tune the encoder for individual frames is not exactly what you want from a good encoder...

if implemented it would be following the 'more rope to hang yourself with' principle,
which is what akupenguin has stated in the past as reasoning for x264's design iirc.
zones are manual tweaking of the encoder as it is, and can be applied to single frames as is.

[Sagekilla]

IIRC, zones let you fine tune a number of different parameters for that specific area, but I think it's only specific options that are exposed in some portion of the code. I remember asking Dark Shikari about this at one point and he said you could do stuff like using different settings beyond rate control for specific zones, except I never really figured out how to do this.

If anyone actually knows how you would specify something like different AQ settings or <arbitrary setting name here> I'd appreciate if anyone would tell me ;)

[check]

There's support to change some encoding settings on the fly, but it's not exposed via the CLI beyond the bitrate/qp option.

[Sagekilla]

Hm. Any chance that would ever be supported in the near future?

[Snowknight26]

[14:41:03] <Snowknight26> so what was it that you cant change in zones?
[14:41:12] <Dark_Shikari> psy-rd, which will be fixed in r10xx
[14:41:18] <Dark_Shikari> aq also
[14:41:46] <Dark_Shikari> aq can't be changed for some technical reasons: I could allow it but the timing would be very off (i.e. changing it at frame 50 might result in the change being delayed to 70)
[14:41:57] <Dark_Shikari> or I could try messing with zones to get that to wokr
[14:42:01] <Dark_Shikari> its possible, but would require effort

When I asked him if b-frames could be changed in zones, he replied 'why not', but its not clear whether he meant the number of b-frames or the adaptive b-frame decision method.

[kemuri-_9]

iirc, possible parameters that can be designated within zones would be in the x264_encoder_reconfig section, which currently has (looking at the code)

reference frames
bframe bias
scenecut threshold
use deblocking
deblocking alpha & beta
intra and inter luma partitions
direct mode
me method
me range
noise reduction
subpel refinement
trellis quantization
chroma me
dct decimate
fast pskip
mixed refs
psy rd
psy trellis
8x8 transform (if enabled to begin with)
bframe pyramid (if enabled to begin with)


so it would be to overhaul the zone parser to handle these, as it seems it can already handle being able to do it within the code base.
and this would be fairly nontrivial with trying to come up with a intuitive scheme that is able to be parsed within code... with all the possible options and combinations of how it could be, might be why it hasn't been done yet.

[Dark Shikari]

intra and inter luma partitionsfixed

[kemuri-_9]

ah thanks for pointing that out.

since people are speculating away on why these options can't be used within the zone option as is, what is the reason, if i may so ask?

[Dark Shikari]

ah thanks for pointing that out.

since people are speculating away on why these options can't be used within the zone option as is, what is the reason, if i may so ask?AQ is done in lookahead, so if I allowed AQ to be changed in zones, I would have to do one of the following:

1. Simply allow zones to be non-frame-accurate and potentially start/stop at locations up to dozens of frames off (for AQ, at least).
or
2. Restructure zones to also check the lookahead frames to see if any one of them is the start of an AQ zone.

I really don't want to do either. Also, Fallout 3 is awesome.

[kemuri-_9]

ha ha, i guess i wasn't specific enough:
i was referring as to why the current parameters within the x264_encoder_reconfig block are not able to be used with the current zone parser...

similarly, from what you just said, it does sound like AQ should not be reconfigurable to maintain zone determinism, i think people could live without having it reconfigurable.

ha ha fallout 3 eh, that does seem to be the current rave...

[Dark Shikari]

ha ha, i guess i wasn't specific enough:
i was referring as to why the current parameters within the x264_encoder_reconfig block are not able to be used with the current zone parser.They aren't? That would be news to me. Last I tried, it works fine.

[kemuri-_9]

ah...
it turns out i was misreading some of the code.

it didn't help that what can be used within the zone parser isn't specifically documented anywhere from what i can remember
(longhelp doesn't even mention anything outside of force QP and bitrate multiplier).
so i guess this compounded the misread.

so the proper zones would be
start_frame,end_frame,b=X,options
start_frame,end_frame,q=X,options
start_frame,end_frame,options

'options' being supported ones that were listed above in a option=value format if they take values or the name of the parameter if they don't ... i.e.
direct=none
nr=100
me=umh
merange=32
no-deblock
scenecut=60
no-fast-pskip
etc...

like
0,100,psy-rd=0.0:0.0,trellis=1,no-fast-pskip,me=dia

trying it out a bit now...
i seem to be getting a crash if me=t/esa is used in a zone when --me is < esa
would need to make a debug build to nail it down further...
is this a known issue?

[Dark Shikari]

trying it out a bit now...
i seem to be getting a crash if me=t/esa is used in a zone when --me is < esa
would need to make a debug build to nail it down further...
is this a known issue?Yes, TESA/ESA cannot be enabled unless the frames have the "integral" data allocated, so I'll add in code to ensure that you cannot enable them if they were off to begin with.

[crypto]

zones?

also yes sane b-adapt 2 <= 5
even 5 is a bit much
I know that >3 b-frames have a performance hit with b-adapt 2. My idea is about automatically detecting fades. VC-1 encoders can do that and can even signal the fade to the decoder which results in smooth fades like butter.

[Dark Shikari]

I know that >3 b-frames have a performance hit with b-adapt 2. My idea is about automatically detecting fades. VC-1 encoders can do that and can even signal the fade to the decoder which results in smooth fades like butter.That's explicit weighted prediction, which x264 doesn't support [yet]. Patches welcome.

[crypto]

That's explicit weighted prediction, which x264 doesn't support [yet]. Patches welcome.
Ok, I see.

About the zones, is that what's called segmented encoding in other encoders?

After the 2 pass encode is ready, the result is displayed and you can mark several segments for reencoding.

[Dark Shikari]

After the 2 pass encode is ready, the result is displayed and you can mark several segments for reencoding.Yes, it can be used for that purpose. No, I do not endorse the concept that companies can compensate for awfully-designed encoders by simply telling the user to fix every mistake their encoder made.

[crypto]

Yes, it can be used for that purpose. No, I do not endorse the concept that companies can compensate for awfully-designed encoders by simply telling the user to fix every mistake their encoder made.
LOL, thanks for the explanation.

And even if it might sounded so, I am in no way criticizing the results of x264. I tried many encoders and x264 is my choice because of its superior results.

[Adub]

Wow, okay, so i just thought I would run some tests recently, and see if I should be one to jump on the psy trellis band wagon.

Okay, so I encoded the same source twice (Heroes Season 1 Episode 11, DVD NTSC R1) once with out psy trellis, once with. It was with MeGUI, and Sharktooth's unrestricted 1 pass const. quality Extra Q modded for psy rd = 0.7 and psy rd = 0.7, along with aq = 0.6 (I stay safe in my usage), and crf = 19.0. I compared the output quality, and yes psy trellis is just that much sharper, albeit with increased grain.

But then I looked at the file sizes:
WithOUT psy trellis = 445mb
With psy trellis = 581mb

Difference = 136mb or 30.561797752808988764044943820225% (edited) larger.

Is this right?! I didn't think that enabling psy trellis would suck up so much bitrate. Hell if I wanted a perfect encode, I would have used a higher crf, or even just done a two pass encode at 581 mb target file size.

DS, is this the expected behavior? Or is this because I used CRF?

[Dark Shikari]

But then I looked at the file sizes:
WithOUT psy trellis = 445mb
With psy trellis = 581mbYes, psy-trellis lowers the effective lambda, raising bitrate at the same QP.
Difference = 136mb or 76.592082616179001721170395869191% larger.Might I suggest you check your math? :p

[DarkZell666]

You can't keep more details/grain and expect the filesize to be the same in quality-based mode, it's just not compatible by design.

Here's how I interpret it : Psy-Whatever enhances grain, but seems to leave the rest of the picture untouched, so the filesize simply goes up by the same amount. It's the bitrate-based modes' job to balance the other frequencies out, unlike in the case of AQ that was designed to balance the details out by itself.

Of course, the bitrate only goes up by the same amount as grain retained, so if the source is already clean enough, the bitrate won't go up as much ... but since there's no grain to keep in that case, why use psy in the first place ? ;)

I suppose we could agree on a rule of thumb about psy+CRF, like "CRF 18 w/o psy = CRF 21 w/ psy" bitrate-wise ? (random numbers just to illustrate).

(Edit: lol, it took me half an hour to finish my post 'coz I kept being disturbed by my boss :p)

[ACoolie]

Yes, TESA/ESA cannot be enabled unless the frames have the "integral" data allocated, so I'll add in code to ensure that you cannot enable them if they were off to begin with.

IMO x264 should automagically transform
x264 --me umh --zones 400,500,me=tesa
to
x264 --me tesa --zones 0,400,me=umh --zones 500,END_FRAME,me=umh

[Adub]

Yes, psy-trellis lowers the effective lambda, raising bitrate at the same QP.
Might I suggest you check your math? :p


That's what I get for doing math at 3 am. But, okay, thanks for your reply. I will have to run further tests, with varying crf's and possibly 2 pass settings. Right now I am not convinced quite yet.

[kemuri-_9]

IMO x264 should automagically transform
x264 --me umh --zones 400,500,me=tesa
to
x264 --me tesa --zones 0,400,me=umh --zones 500,END_FRAME,me=umh

imo, that would be more work in the code base than it's worth, when you can just do it yourself easier.

[foxyshadis]

Can you just always allocate the data, if it's not too large, whether it's originally used or not?

[akupenguin]

Can you just always allocate the data, if it's not too large, whether it's originally used or not?
Assume 1080p, 4 refs, 3 B-frames, 6 threads.
edit, now with actual measurement
Then x264 with ESA takes 800MB vmem, and without ESA takes 500MB. If you're on linux with overcommitted memory, there's no difference: as long as the ESA arrays aren't used, they never really get allocated. But if your OS expects virtual memory to be backed up by physical memory, IMO that's too much waste.

[lexor]

Assume 1080p, 4 refs, 3 B-frames, 6 threads. Then the ESA arrays take 200MB. Is that not too large?

If there is only one of those at a time, and the rest of the x264 stuff in RAM doesn't increase the total by more than 2x that value, I don't think it's too large. Assuming that it has practical usefulness to begin with, of course.

[Sagekilla]

@akupenguin: How do you calculate how large an esa array would be? Just out of curiosity.

[akupenguin]

It's 4 bytes per pixel, and extends 32 pixels off the edge of the visible image in each direction.
The only tricky part is calculating how many frames x264 will allocate.

[Sagekilla]

So (width + 64) * (height + 64) * references * threads * 4 bytes is how big you would expect the array to be.

[akupenguin]

(width + 64) * (height + 64) * 4 * (references + threads + other frames that don't really need ESA but allocate it anyway)

[kemuri-_9]

hmm....
how viable would it be to have the zone parser trigger the T/ESA memory allocation if it sees a me=t/esa, if it is not already allocated (say the --me is <= UMH)?
that currently seems to be the easiest solution for the issue from what i know of the situation, compared to rewriting and swapping zones and allowing it to work...

[Dark Shikari]

hmm....
how viable would it be to have the zone parser trigger the T/ESA memory allocation if it sees a me=t/esa, if it is not already allocated (say the --me is <= UMH)?
that currently seems to be the easiest solution for the issue from what i know of the situation, compared to rewriting and swapping zones and allowing it to work...That would require a massive amount of code--going through all the frames, allocating new data for each frame, and then calling the various functions to create the data... not worth it.

[kemuri-_9]

no no, I'm not talking about at the time of encoding the frame/on demand, but at initializer time.

though this would be dependent on when the frame memory allocation and zone parsing events happen in the initializer.

the ideal situation the code could be in is that the zone parser comes before the frame memory allocation,
so then the zone parser could add to the switch of which allocation to do (esa based allocation vs <=umh allocation), if a zone has t/esa specified.
so when it comes time to do the zone's t/esa, the memory is already allocated and there would be no crashes.

horribly oversimplified (object-based) psuedocode of this would be similar to

zones.parse(char *); // parse the zone string
if (zones.contains(me,ESA | TESA) || (me == ESA || me == TESA)) // determine which memory allocation type we need
esa_memory_allocation();
else
non_esa_memory_allocation();


if the memory allocation comes before the zone parsing, the situation then becomes 'how struggling would it be to move the zone parsing before the memory allocation', so above situation could be realized.

and I'm at work, so i don't have any code in front of me when i speculated this up.

[Quark.Fusion]

Assume 1080p, 4 refs, 3 B-frames, 6 threads.
edit, now with actual measurement
Then x264 with ESA takes 800MB vmem, and without ESA takes 500MB. If you're on linux with overcommitted memory, there's no difference: as long as the ESA arrays aren't used, they never really get allocated. But if your OS expects virtual memory to be backed up by physical memory, IMO that's too much waste.

Isn't windows also don't physically allocate that memory? But you can hit 2GB limit on 32bit apps (already problem for avisynth plugins).

[pcordes]

there's no way possible with tee and/or grep to have what you want to write to file and have it display the progress to the screen for when you want to occasionally want to check up on it.


Never underestimate what you can do with the Unix shell, young padawan.


if you do
2>&1 | tee file
you get to see everything as normal, but then the file gets the progress bloat

if you do
2>&1 | grep -v " eta " | tee file
the file gets what you want (besides the I slice statistics, since they're removed with the last progress line), but you don't see jack.


So you need to send output to the terminal before you filter it. So try this:
x264 2>&1 | tee /dev/tty | tr '\r' '\n' | LANG=C grep -v '^\[' > log

Or do it all in one command with awk. (At first I though I had a simpler awk program, but it didn't get the output quite right).
x264 2>&1 | LANG=C gawk '{if(/^\[/){ print $0"\r";fflush() }else{print $0"\n"; print $0"\n" >"logfile"}}' ORS='' RS='[\r\n]'

I timed these (with time cat x264.log x264.log x264.log ... | tee /dev/null | ... > log), cat(1)ing a total of 108MB of x264 logs to make it take at least a second. This is probably way overkill because x264 cuts back its logging rate on longer files, so this only matters when it's going to be really fast anyway.

On my C2D E6600, DDR2-800, running AMD64 Linux 2.6.27 (Ubuntu Intrepid):
real 0m0.940s, user 0m0.672s, sys 0m0.288s for the tee /dev/null| tr | grep pipeline.
real 0m0.747s, user 0m0.420s, sys 0m0.424s for the tee /dev/null| tr | LANG=C grep pipeline.
real 0m3.547s, user 0m2.872s, sys 0m0.636s for LANG=C gawk
real 0m10.089s,user 0m9.345s, sys 0m0.700s for LANG=en_CA.utf8 gawk
real 0m1.753s, user 0m1.172s, sys 0m0.572s for mawk

mawk is fast, but it isn't usable for this, because it always waits until it has 4kB of input before breaking it into lines with RS. gawk considers each block of input to see if it's a whole line.

UTF-8 makes grep and gawk's regexes slow, so use LANG=C. The tee|tr|grep combination uses less CPU when that's the limiting factor, but probably the one-process gawk way is best, to minimize context switches and cache pollution. Being woken once for each line is totally different from being CPU-limited by grep or gawk. OTOH, gawk must be doing something to use all that CPU time... Again, x264 doesn't log that often on longer files, and I went nuts figuring out how to benchmark these things before I realized that.

I think I once had a program lying around that processed input containing carriage returns, etc. and output what would be left on the screen of a terminal emulator. I think it came from a CS prof who wrote it for people to run their assignments through before submitting the output. I can't find it now, though. That would be much less overhead than either of my suggestions.

the purpose of --progress and then the purpose of what you would use 2> for are counters to each other. they just don't work together well
with the log file patch you can get --progress without some drawback.

x264 doesn't normally use stdout for anything (right?), so why not put messages that people want to redirect on stdout, and progress on stderr?
Then it's
x264 | tee log

Perhaps a --log-interval option would be good (or make --progress take an optional argument). Some terminal emulators aren't that fast, and the log messages could use a few % CPU if they're actually getting rendered onto video memory. I guess --progress isn't the default, though, so just leave it off on fast encodes where logging is a significant overhead.

Ok, currently x264 logs 1000 times for each input file, or once per 10 frames if it doesn't know the input frame count. It already knows what time it is (for fps calc), so you could also have it log at most once per second. Or just do that instead of the aim for 1000 per file trick.

Actual error messages could go to both, to make sure they're not lost, except when stdout and stderr are the same file (e.g. if fstat(2) is available, use it and see if st_dev and st_ino are equal.)


To test overhead of different pipe methods, I did this to
get a giant log file: x264 &> output, then I copy/pasted the progress lines to inflate it.

[kemuri-_9]

Never underestimate what you can do with the Unix shell, young padawan.

this isn't unix shell, but windows command prompt.
a different ball game.

so just drop it.

[Shinigami-Sama]

this isn't unix shell, but windows command prompt.
a different ball game.

so just drop it.

cygwin

[LoRd_MuldeR]

cygwin

... or MSYS

[J_Darnley]

... or change x264 (http://users.telenet.be/darnley/x264/x264_double-print_03.diff).

[kemuri-_9]

try all those solutions with --quiet -o - and see how well they work.
(why? because i didn't want to continue this discussion, it's OT)

@devs
however, on the subject of --quiet...
why was the output in muxers.c not made to respect --quiet?
-
referring to the lack of p_param->i_log_level comparisons where able...
this (http://kemuri9.net/dev/x264/patches/x264.muxers.respect.log.level.diff) should explain it better

*figures there might be some angry/irritated ppl after this and is apologizing now*

[Dark Shikari]

referring to the lack of p_param->i_log_level comparisons where able...
this (http://kemuri9.net/dev/x264/patches/x264.muxers.respect.log.level.diff) should explain it better< pengvado> macro, or call x264_log_default

[Emp3r0r]

-[NoImage] Job commandline: "C:\Programs\x264\x264.exe" --pass 2 --bitrate 6059 --stats "H:\BD\RD\rd.stats" --level 4.1 --ref 4 --mixed-refs --bframes 3 --b-adapt 2 --weightb --direct auto --deblock -1:-1 --subme 7 --trellis 2 --partitions p8x8,b8x8,i4x4,i8x8 --8x8dct --vbv-bufsize 50000 --vbv-maxrate 50000 --me umh --threads auto --thread-input --zones 174400,180359,q=40 --sar 1:1 --progress --no-psnr --no-ssim --output "H:\BD\RD\rd.mkv" "H:\BD\RD\rd.avs"

1st pass worked but at beginning of 2nd pass it crashes x264.

Problem signature:
Problem Event Name: APPCRASH
Application Name: x264.exe
Application Version: 0.0.0.0
Application Timestamp: 491c385f
Fault Module Name: ntdll.dll
Fault Module Version: 6.0.6001.18000
Fault Module Timestamp: 4791a783
Exception Code: c0000005
Exception Offset: 0002f217
OS Version: 6.0.6001.2.1.0.256.6
Locale ID: 1033
Additional Information 1: fd00
Additional Information 2: ea6f5fe8924aaa756324d57f87834160
Additional Information 3: fd00
Additional Information 4: ea6f5fe8924aaa756324d57f87834160

I'm on a Q9450 x64 VistaSP1.

[Dark Shikari]

-Problem signature:
Problem Event Name: APPCRASH
Application Name: x264.exe
Application Version: 0.0.0.0
Application Timestamp: 491c385f
Fault Module Name: ntdll.dll
Fault Module Version: 6.0.6001.18000
Fault Module Timestamp: 4791a783
Exception Code: c0000005
Exception Offset: 0002f217
OS Version: 6.0.6001.2.1.0.256.6
Locale ID: 1033
Additional Information 1: fd00
Additional Information 2: ea6f5fe8924aaa756324d57f87834160
Additional Information 3: fd00
Additional Information 4: ea6f5fe8924aaa756324d57f87834160There's no useful information in there: we can't do any debugging unless you have a backtrace from gdb, at a minimum.

[Kurtnoise]

:script:

[Emp3r0r]

[4304] DllMain: hModule=0x06a30000, ulReason=1, lpReserved=0x00000000, gRefCnt = 0
[4304] DllMain: hModule=0x06a30000, ulReason=0, lpReserved=0x00000000, gRefCnt = 0
[4584] DllMain: hModule=0x10000000, ulReason=1, lpReserved=0x00000000, gRefCnt = 0
[4584] DllGetClassObject() CLSID: CAVIFileSynth
[4584] 002D9C90->CAVIFileSynth::CAVIFileSynth()
[4584] 002D9C90->CAVIFileSynth::AddRef() gRefCnt=1, m_refs=1
[4584] 002D9C90->CAVIFileSynth::QueryInterface() {00000001-0000-0000-c000-000000000046} (IClassFactory)
[4584] 002D9C90->CAVIFileSynth::AddRef() gRefCnt=2, m_refs=2
[4584] 002D9C90->CAVIFileSynth::Release() gRefCnt=1, m_refs=1
[4584] DllGetClassObject() result=0x0, object=002D9C98
[4584] 002D9C90->CAVIFileSynth::CreateInstance()
[4584] 002D9CD8->CAVIFileSynth::CAVIFileSynth()
[4584] 002D9CD8->CAVIFileSynth::AddRef() gRefCnt=2, m_refs=1
[4584] 002D9CD8->CAVIFileSynth::QueryInterface() {00000000-0000-0000-c000-000000000046} (IUnknown)
[4584] 002D9CD8->CAVIFileSynth::AddRef() gRefCnt=3, m_refs=2
[4584] 002D9CD8->CAVIFileSynth::Release() gRefCnt=2, m_refs=1
[4584] 002D9C90->CAVIFileSynth::CreateInstance() result=0x0, object=002D9CD8
[4584] 002D9CD8->CAVIFileSynth::AddRef() gRefCnt=3, m_refs=2
[4584] 002D9CD8->CAVIFileSynth::Release() gRefCnt=2, m_refs=1
[4584] 002D9C90->CAVIFileSynth::Release() gRefCnt=1, m_refs=0
[4584] 002D9C90->CAVIFileSynth::~CAVIFileSynth(), gRefCnt = 1
[4584] 002D9CD8->CAVIFileSynth::QueryInterface() {00000000-0000-0000-c000-000000000046} (IUnknown)
[4584] 002D9CD8->CAVIFileSynth::AddRef() gRefCnt=2, m_refs=2
[4584] 002D9CD8->CAVIFileSynth::Release() gRefCnt=1, m_refs=1
[4584] 002D9CD8->CAVIFileSynth::QueryInterface() {00020025-0000-0000-c000-000000000046} (unsupported!)
[4584] 002D9CD8->CAVIFileSynth::QueryInterface() {0000010b-0000-0000-c000-000000000046} (IPersistFile)
[4584] 002D9CD8->CAVIFileSynth::AddRef() gRefCnt=2, m_refs=2
[4584] 002D9CD8->CAVIFileSynth::QueryInterface() {00020020-0000-0000-c000-000000000046} (IAVIFile)
[4584] 002D9CD8->CAVIFileSynth::AddRef() gRefCnt=3, m_refs=3
[4584] 002D9CD8->CAVIFileSynth::Load("H:\BD\RD\rd.avs", 0x0)
[4584] 002D9CD8->CAVIFileSynth::Release() gRefCnt=2, m_refs=2
[4584] 002D9CD8->CAVIFileSynth::Release() gRefCnt=1, m_refs=1
[4584] 002D9CD8->CAVIFileSynth::GetStream(*, 73646976(vids), 0)
[4584] 02676428->CAVIStreamSynth(video)
[4584] 02676428->CAVIStreamSynth::AddRef() (video) gRefCnt=2, m_refs=1
[4584] 002D9CD8->CAVIFileSynth::AddRef() gRefCnt=3, m_refs=2
[4584] 002D9CD8->CAVIFileSynth::Release() gRefCnt=2, m_refs=1
[4584] 02676428->CAVIStreamSynth::Info(0027F6C8, 204) (video)
[4584] 02676428->CAVIStreamSynth::Info(0027F6D8, 204) (video)
[4584] DllMain: hModule=0x10000000, ulReason=0, lpReserved=0x00000001, gRefCnt = 2

LoadPlugin("C:\Program Files (x86)\AviSynth\plugins\DGAVCDecode.dll")
AVCSource("H:\BD\RD\00025.dga")
crop( 0, 16, 0, -24)
Lanczos4Resize(1280,960)

If someone else can confirm that it is working on their Yorkfield then it is probably just my machine. Also, I'll try rolling back to an earlier version to see if the problem persists.

[Dark Shikari]

[4304] DllMain: hModule=0x06a30000, ulReason=1, lpReserved=0x00000000, gRefCnt = 0
[4304] DllMain: hModule=0x06a30000, ulReason=0, lpReserved=0x00000000, gRefCnt = 0
[4584] DllMain: hModule=0x10000000, ulReason=1, lpReserved=0x00000000, gRefCnt = 0
[4584] DllGetClassObject() CLSID: CAVIFileSynth
[4584] 002D9C90->CAVIFileSynth::CAVIFileSynth()
[4584] 002D9C90->CAVIFileSynth::AddRef() gRefCnt=1, m_refs=1
[4584] 002D9C90->CAVIFileSynth::QueryInterface() {00000001-0000-0000-c000-000000000046} (IClassFactory)
[4584] 002D9C90->CAVIFileSynth::AddRef() gRefCnt=2, m_refs=2
[4584] 002D9C90->CAVIFileSynth::Release() gRefCnt=1, m_refs=1
[4584] DllGetClassObject() result=0x0, object=002D9C98
[4584] 002D9C90->CAVIFileSynth::CreateInstance()
[4584] 002D9CD8->CAVIFileSynth::CAVIFileSynth()
[4584] 002D9CD8->CAVIFileSynth::AddRef() gRefCnt=2, m_refs=1
[4584] 002D9CD8->CAVIFileSynth::QueryInterface() {00000000-0000-0000-c000-000000000046} (IUnknown)
[4584] 002D9CD8->CAVIFileSynth::AddRef() gRefCnt=3, m_refs=2
[4584] 002D9CD8->CAVIFileSynth::Release() gRefCnt=2, m_refs=1
[4584] 002D9C90->CAVIFileSynth::CreateInstance() result=0x0, object=002D9CD8
[4584] 002D9CD8->CAVIFileSynth::AddRef() gRefCnt=3, m_refs=2
[4584] 002D9CD8->CAVIFileSynth::Release() gRefCnt=2, m_refs=1
[4584] 002D9C90->CAVIFileSynth::Release() gRefCnt=1, m_refs=0
[4584] 002D9C90->CAVIFileSynth::~CAVIFileSynth(), gRefCnt = 1
[4584] 002D9CD8->CAVIFileSynth::QueryInterface() {00000000-0000-0000-c000-000000000046} (IUnknown)
[4584] 002D9CD8->CAVIFileSynth::AddRef() gRefCnt=2, m_refs=2
[4584] 002D9CD8->CAVIFileSynth::Release() gRefCnt=1, m_refs=1
[4584] 002D9CD8->CAVIFileSynth::QueryInterface() {00020025-0000-0000-c000-000000000046} (unsupported!)
[4584] 002D9CD8->CAVIFileSynth::QueryInterface() {0000010b-0000-0000-c000-000000000046} (IPersistFile)
[4584] 002D9CD8->CAVIFileSynth::AddRef() gRefCnt=2, m_refs=2
[4584] 002D9CD8->CAVIFileSynth::QueryInterface() {00020020-0000-0000-c000-000000000046} (IAVIFile)
[4584] 002D9CD8->CAVIFileSynth::AddRef() gRefCnt=3, m_refs=3
[4584] 002D9CD8->CAVIFileSynth::Load("H:\BD\RD\rd.avs", 0x0)
[4584] 002D9CD8->CAVIFileSynth::Release() gRefCnt=2, m_refs=2
[4584] 002D9CD8->CAVIFileSynth::Release() gRefCnt=1, m_refs=1
[4584] 002D9CD8->CAVIFileSynth::GetStream(*, 73646976(vids), 0)
[4584] 02676428->CAVIStreamSynth(video)
[4584] 02676428->CAVIStreamSynth::AddRef() (video) gRefCnt=2, m_refs=1
[4584] 002D9CD8->CAVIFileSynth::AddRef() gRefCnt=3, m_refs=2
[4584] 002D9CD8->CAVIFileSynth::Release() gRefCnt=2, m_refs=1
[4584] 02676428->CAVIStreamSynth::Info(0027F6C8, 204) (video)
[4584] 02676428->CAVIStreamSynth::Info(0027F6D8, 204) (video)
[4584] DllMain: hModule=0x10000000, ulReason=0, lpReserved=0x00000001, gRefCnt = 2

LoadPlugin("C:\Program Files (x86)\AviSynth\plugins\DGAVCDecode.dll")
AVCSource("H:\BD\RD\00025.dga")
crop( 0, 16, 0, -24)
Lanczos4Resize(1280,960)That's an Avisynth backtrace, not an x264 backtrace.

[Emp3r0r]

I rolled back to version x264.1016.modified.02.exe and the 2nd pass is now running without crash. I'll try 1024 next.

That's all DebugView is giving me.... downloading GDB.

[New thread 4552.0x1138]
(no debugging symbols found)
Error: dll starting at 0x76f90000 not found.
Error: dll starting at 0x75510000 not found.
Error: dll starting at 0x76f90000 not found.
Error: dll starting at 0x76ec0000 not found.
(no debugging symbols found)
(no debugging symbols found)
(no debugging symbols found)
(no debugging symbols found)
(no debugging symbols found)
(no debugging symbols found)
(no debugging symbols found)
(no debugging symbols found)
(no debugging symbols found)
(no debugging symbols found)
(no debugging symbols found)
(no debugging symbols found)
(no debugging symbols found)
(no debugging symbols found)
(no debugging symbols found)
(no debugging symbols found)
(no debugging symbols found)
(no debugging symbols found)
(no debugging symbols found)
(no debugging symbols found)
(no debugging symbols found)
(no debugging symbols found)
(no debugging symbols found)
(no debugging symbols found)
(no debugging symbols found)
(no debugging symbols found)
(no debugging symbols found)
(no debugging symbols found)
(no debugging symbols found)
(no debugging symbols found)
warning: DllMain: hModule=0x10000000, ulReason=1, lpReserved=0x00000000, gRefC
= 0

warning: DllGetClassObject() CLSID: CAVIFileSynth

warning: 00B43B28->CAVIFileSynth::CAVIFileSynth()

warning: 00B43B28->CAVIFileSynth::AddRef() gRefCnt=1, m_refs=1

warning: 00B43B28->CAVIFileSynth::QueryInterface() {00000001-0000-0000-c000-00
00000046} (IClassFactory)

warning: 00B43B28->CAVIFileSynth::AddRef() gRefCnt=2, m_refs=2

warning: 00B43B28->CAVIFileSynth::Release() gRefCnt=1, m_refs=1

warning: DllGetClassObject() result=0x0, object=00B43B30

warning: 00B43B28->CAVIFileSynth::CreateInstance()

warning: 00B43B80->CAVIFileSynth::CAVIFileSynth()

warning: 00B43B80->CAVIFileSynth::AddRef() gRefCnt=2, m_refs=1

warning: 00B43B80->CAVIFileSynth::QueryInterface() {00000000-0000-0000-c000-00
00000046} (IUnknown)

warning: 00B43B80->CAVIFileSynth::AddRef() gRefCnt=3, m_refs=2

warning: 00B43B80->CAVIFileSynth::Release() gRefCnt=2, m_refs=1

warning: 00B43B28->CAVIFileSynth::CreateInstance() result=0x0, object=00B43B80

warning: 00B43B80->CAVIFileSynth::AddRef() gRefCnt=3, m_refs=2

warning: 00B43B80->CAVIFileSynth::Release() gRefCnt=2, m_refs=1

warning: 00B43B28->CAVIFileSynth::Release() gRefCnt=1, m_refs=0

warning: 00B43B28->CAVIFileSynth::~CAVIFileSynth(), gRefCnt = 1

warning: 00B43B80->CAVIFileSynth::QueryInterface() {00000000-0000-0000-c000-00
00000046} (IUnknown)

warning: 00B43B80->CAVIFileSynth::AddRef() gRefCnt=2, m_refs=2

warning: 00B43B80->CAVIFileSynth::Release() gRefCnt=1, m_refs=1

warning: 00B43B80->CAVIFileSynth::QueryInterface() {00020025-0000-0000-c000-00
00000046} (unsupported!)

warning: 00B43B80->CAVIFileSynth::QueryInterface() {0000010b-0000-0000-c000-00
00000046} (IPersistFile)

warning: 00B43B80->CAVIFileSynth::AddRef() gRefCnt=2, m_refs=2

warning: 00B43B80->CAVIFileSynth::QueryInterface() {00020020-0000-0000-c000-00
00000046} (IAVIFile)

warning: 00B43B80->CAVIFileSynth::AddRef() gRefCnt=3, m_refs=3

warning: 00B43B80->CAVIFileSynth::Load("H:\BD\RD\rd.avs", 0x0)

warning: 00B43B80->CAVIFileSynth::Release() gRefCnt=2, m_refs=2

warning: 00B43B80->CAVIFileSynth::Release() gRefCnt=1, m_refs=1

warning: 00B43B80->CAVIFileSynth::GetStream(*, 73646976(vids), 0)

(no debugging symbols found)
(no debugging symbols found)
(no debugging symbols found)
(no debugging symbols found)
(no debugging symbols found)
(no debugging symbols found)
(no debugging symbols found)
(no debugging symbols found)
(no debugging symbols found)
(no debugging symbols found)
(no debugging symbols found)
(no debugging symbols found)
(no debugging symbols found)
(no debugging symbols found)
warning: 00B43B28->CAVIStreamSynth(video)

warning: 00B43B28->CAVIStreamSynth::AddRef() (video) gRefCnt=2, m_refs=1

warning: 00B43B80->CAVIFileSynth::AddRef() gRefCnt=3, m_refs=2

warning: 00B43B80->CAVIFileSynth::Release() gRefCnt=2, m_refs=1

warning: 00B43B28->CAVIStreamSynth::Info(0027F6C8, 204) (video)

avis [info]: 1280x960 @ 23.98 fps (180360 frames)
warning: 00B43B28->CAVIStreamSynth::Info(0027F6D8, 204) (video)

x264 [info]: using SAR=1/1
x264 [info]: using cpu capabilities: MMX2 SSE2Fast SSSE3 PHADD SSE4.1 Cache64

Program received signal SIGSEGV, Segmentation fault.
0x7728f217 in ntdll!RtlDecompressBuffer () from C:\Windows\system32\ntdll.dll
(gdb) info stack
#0 0x7728f217 in ntdll!RtlDecompressBuffer ()
from C:\Windows\system32\ntdll.dll
#1 0x90909090 in ?? ()
#2 0x00000012 in ?? ()
#3 0x0bb36230 in ?? ()
#4 0x0027f26c in ?? ()
#5 0x75593593 in KERNEL32!GetNumaAvailableMemoryNode ()
from C:\Windows\syswow64\kernel32.dll
#6 0x00b40000 in ?? ()
#7 0x00000000 in ?? ()
(gdb)

[kemuri-_9]

does r1028 have problems for you as well?

[cbarroso]

Here 1028 crashes in CRF mode and 1024 works fine on Vista, core2duo.

[Emp3r0r]

yea just tried 1028, it crashed:

x264 [info]: using cpu capabilities: MMX2 SSE2Fast SSSE3 PHADD SSE4.1 Cache64

Program received signal SIGSEGV, Segmentation fault.
0x7728f217 in ntdll!RtlDecompressBuffer () from C:\Windows\system32\ntdll.dll

#0 0x7728f217 in ntdll!RtlDecompressBuffer ()
from C:\Windows\system32\ntdll.dll
#1 0x90909090 in ?? ()
#2 0x00000012 in ?? ()
#3 0x0be16230 in ?? ()
#4 0x0027f26c in ?? ()
#5 0x75593593 in KERNEL32!GetNumaAvailableMemoryNode ()
from C:\Windows\syswow64\kernel32.dll
#6 0x00020000 in ?? ()
#7 0x00000000 in ?? ()

[kemuri-_9]

try gdb again with this debug build of x264, should get more information out of it.
x264_debug.exe (http://kemuri9.net/dev/x264/x264_debug.exe)

[Emp3r0r]

here it is from debug
x264 [info]: using SAR=1/1
x264 [info]: using cpu capabilities: MMX2 SSE2Fast SSSE3 PHADD SSE4.1 Cache64
x264 [error]: ratecontrol_init: can't open stats file
x264 [error]: x264_encoder_open failed
warning: 00C43B10->CAVIStreamSynth::Release() (video) gRefCnt=1, m_refs=0

warning: 00C43B10->~CAVIStreamSynth() (video), gRefCnt = 1

warning: 00C43B68->CAVIFileSynth::Release() gRefCnt=0, m_refs=0

warning: 00C43B68->CAVIFileSynth::~CAVIFileSynth(), gRefCnt = 0

warning: DllCanUnloadNow(): gRefCnt = 0

warning: DllMain: hModule=0x10000000, ulReason=0, lpReserved=0x00000000, gRefCnt
= 0


Program exited with code 037777777777.
(gdb) info stack
No stack.

[kemuri-_9]

calling bt full when it stops working in gdb prints lots of helpful information.

but the one you just posted says it doesn't have the .stats file for the 2nd pass, so what was the arguments you used?

[Emp3r0r]

woops, your right, the debug version does not crash. Yay! I'm not sure is what wrong with 1024 and 1028.
Reading symbols from c:\programs\x264/x264.1028.exe...
(no debugging symbols found)...done.
<o-ssim --output "H:\BD\RD\rd.mkv" "H:\BD\RD\rd.avs"
The program being debugged has been started already.
Start it from the beginning? (y or n) y

Starting program: c:\programs\x264/x264.1028.exe --pass 2 --bitrate 6059 --stats
"H:\BD\RD\rd.stats" --level 4.1 --ref 4 --mixed-refs --bframes 3 --b-adapt 2 --
weightb --direct auto --deblock -1:-1 --subme 7 --trellis 2 --partitions p8x8,b8
x8,i4x4,i8x8 --8x8dct --vbv-bufsize 50000 --vbv-maxrate 50000 --me umh --threads
auto --thread-input --zones 174400,180359,q=40 --sar 1:1 --progress --no-psnr -
-no-ssim --output "H:\BD\RD\rd.mkv" "H:\BD\RD\rd.avs"
[New thread 4540.0x724]
(no debugging symbols found)
Error: dll starting at 0x76f90000 not found.
Error: dll starting at 0x75510000 not found.
Error: dll starting at 0x76f90000 not found.
Error: dll starting at 0x76ec0000 not found.
(no debugging symbols found)
(no debugging symbols found)
(no debugging symbols found)
(no debugging symbols found)
(no debugging symbols found)
(no debugging symbols found)
(no debugging symbols found)
(no debugging symbols found)
(no debugging symbols found)
(no debugging symbols found)
(no debugging symbols found)
(no debugging symbols found)
(no debugging symbols found)
(no debugging symbols found)
(no debugging symbols found)
(no debugging symbols found)
(no debugging symbols found)
(no debugging symbols found)
(no debugging symbols found)
(no debugging symbols found)
(no debugging symbols found)
(no debugging symbols found)
(no debugging symbols found)
(no debugging symbols found)
(no debugging symbols found)
(no debugging symbols found)
(no debugging symbols found)
(no debugging symbols found)
(no debugging symbols found)
(no debugging symbols found)
warning: DllMain: hModule=0x10000000, ulReason=1, lpReserved=0x00000000, gRefCnt
= 0

warning: DllGetClassObject() CLSID: CAVIFileSynth

warning: 00B73AF0->CAVIFileSynth::CAVIFileSynth()

warning: 00B73AF0->CAVIFileSynth::AddRef() gRefCnt=1, m_refs=1

warning: 00B73AF0->CAVIFileSynth::QueryInterface() {00000001-0000-0000-c000-0000
00000046} (IClassFactory)

warning: 00B73AF0->CAVIFileSynth::AddRef() gRefCnt=2, m_refs=2

warning: 00B73AF0->CAVIFileSynth::Release() gRefCnt=1, m_refs=1

warning: DllGetClassObject() result=0x0, object=00B73AF8

warning: 00B73AF0->CAVIFileSynth::CreateInstance()

warning: 00B73B48->CAVIFileSynth::CAVIFileSynth()

warning: 00B73B48->CAVIFileSynth::AddRef() gRefCnt=2, m_refs=1

warning: 00B73B48->CAVIFileSynth::QueryInterface() {00000000-0000-0000-c000-0000
00000046} (IUnknown)

warning: 00B73B48->CAVIFileSynth::AddRef() gRefCnt=3, m_refs=2

warning: 00B73B48->CAVIFileSynth::Release() gRefCnt=2, m_refs=1

warning: 00B73AF0->CAVIFileSynth::CreateInstance() result=0x0, object=00B73B48

warning: 00B73B48->CAVIFileSynth::AddRef() gRefCnt=3, m_refs=2

warning: 00B73B48->CAVIFileSynth::Release() gRefCnt=2, m_refs=1

warning: 00B73AF0->CAVIFileSynth::Release() gRefCnt=1, m_refs=0

warning: 00B73AF0->CAVIFileSynth::~CAVIFileSynth(), gRefCnt = 1

warning: 00B73B48->CAVIFileSynth::QueryInterface() {00000000-0000-0000-c000-0000
00000046} (IUnknown)

warning: 00B73B48->CAVIFileSynth::AddRef() gRefCnt=2, m_refs=2

warning: 00B73B48->CAVIFileSynth::Release() gRefCnt=1, m_refs=1

warning: 00B73B48->CAVIFileSynth::QueryInterface() {00020025-0000-0000-c000-0000
00000046} (unsupported!)

warning: 00B73B48->CAVIFileSynth::QueryInterface() {0000010b-0000-0000-c000-0000
00000046} (IPersistFile)

warning: 00B73B48->CAVIFileSynth::AddRef() gRefCnt=2, m_refs=2

warning: 00B73B48->CAVIFileSynth::QueryInterface() {00020020-0000-0000-c000-0000
00000046} (IAVIFile)

warning: 00B73B48->CAVIFileSynth::AddRef() gRefCnt=3, m_refs=3

warning: 00B73B48->CAVIFileSynth::Load("H:\BD\RD\rd.avs", 0x0)

warning: 00B73B48->CAVIFileSynth::Release() gRefCnt=2, m_refs=2

warning: 00B73B48->CAVIFileSynth::Release() gRefCnt=1, m_refs=1

warning: 00B73B48->CAVIFileSynth::GetStream(*, 73646976(vids), 0)

(no debugging symbols found)
(no debugging symbols found)
(no debugging symbols found)
(no debugging symbols found)
(no debugging symbols found)
(no debugging symbols found)
(no debugging symbols found)
(no debugging symbols found)
(no debugging symbols found)
(no debugging symbols found)
(no debugging symbols found)
(no debugging symbols found)
(no debugging symbols found)
(no debugging symbols found)
warning: 02955308->CAVIStreamSynth(video)

warning: 02955308->CAVIStreamSynth::AddRef() (video) gRefCnt=2, m_refs=1

warning: 00B73B48->CAVIFileSynth::AddRef() gRefCnt=3, m_refs=2

warning: 00B73B48->CAVIFileSynth::Release() gRefCnt=2, m_refs=1

warning: 02955308->CAVIStreamSynth::Info(0027F6C8, 204) (video)

avis [info]: 1280x960 @ 23.98 fps (180360 frames)
warning: 02955308->CAVIStreamSynth::Info(0027F6D8, 204) (video)

x264 [info]: using SAR=1/1
x264 [info]: using cpu capabilities: MMX2 SSE2Fast SSSE3 PHADD SSE4.1 Cache64

Program received signal SIGSEGV, Segmentation fault.
0x7728f217 in ntdll!RtlDecompressBuffer () from C:\Windows\system32\ntdll.dll
(gdb) bt full
#0 0x7728f217 in ntdll!RtlDecompressBuffer ()
from C:\Windows\system32\ntdll.dll
No symbol table info available.
#1 0x90909090 in ?? ()
No symbol table info available.
#2 0x00000012 in ?? ()
No symbol table info available.
#3 0x0bad6230 in ?? ()
No symbol table info available.
#4 0x0027f26c in ?? ()
No symbol table info available.
#5 0x75593593 in KERNEL32!GetNumaAvailableMemoryNode ()
from C:\Windows\syswow64\kernel32.dll
No symbol table info available.
#6 0x00b70000 in ?? ()
No symbol table info available.
#7 0x00000000 in ?? ()
No symbol table info available.
(gdb) file x264_debug.exe
A program is being debugged already.
Are you sure you want to change the file? (y or n) y
Reading symbols from c:\programs\x264/x264_debug.exe...done.
(gdb) file x264.1028.exe
A program is being debugged already.
Are you sure you want to change the file? (y or n) y

Load new symbol table from "c:\programs\x264/x264.1028.exe"? (y or n) n
Not confirmed.
<o-ssim --output "H:\BD\RD\rd.mkv" "H:\BD\RD\rd.avs"
The program being debugged has been started already.
Start it from the beginning? (y or n) y

Starting program: c:\programs\x264/x264.1028.exe --pass 2 --bitrate 6059 --stats
"H:\BD\RD\rd.stats" --level 4.1 --ref 4 --mixed-refs --bframes 3 --b-adapt 2 --
weightb --direct auto --deblock -1:-1 --subme 7 --trellis 2 --partitions p8x8,b8
x8,i4x4,i8x8 --8x8dct --vbv-bufsize 50000 --vbv-maxrate 50000 --me umh --threads
auto --thread-input --zones 174400,180359,q=40 --sar 1:1 --progress --no-psnr -
-no-ssim --output "H:\BD\RD\rd.mkv" "H:\BD\RD\rd.avs"
[New thread 952.0x137c]
Error: dll starting at 0x76f90000 not found.
Error: dll starting at 0x75510000 not found.
Error: dll starting at 0x76f90000 not found.
Error: dll starting at 0x76ec0000 not found.
warning: DllMain: hModule=0x10000000, ulReason=1, lpReserved=0x00000000, gRefCnt
= 0

warning: DllGetClassObject() CLSID: CAVIFileSynth

warning: 003E3AF0->CAVIFileSynth::CAVIFileSynth()

warning: 003E3AF0->CAVIFileSynth::AddRef() gRefCnt=1, m_refs=1

warning: 003E3AF0->CAVIFileSynth::QueryInterface() {00000001-0000-0000-c000-0000
00000046} (IClassFactory)

warning: 003E3AF0->CAVIFileSynth::AddRef() gRefCnt=2, m_refs=2

warning: 003E3AF0->CAVIFileSynth::Release() gRefCnt=1, m_refs=1

warning: DllGetClassObject() result=0x0, object=003E3AF8

warning: 003E3AF0->CAVIFileSynth::CreateInstance()

warning: 003E3B48->CAVIFileSynth::CAVIFileSynth()

warning: 003E3B48->CAVIFileSynth::AddRef() gRefCnt=2, m_refs=1

warning: 003E3B48->CAVIFileSynth::QueryInterface() {00000000-0000-0000-c000-0000
00000046} (IUnknown)

warning: 003E3B48->CAVIFileSynth::AddRef() gRefCnt=3, m_refs=2

warning: 003E3B48->CAVIFileSynth::Release() gRefCnt=2, m_refs=1

warning: 003E3AF0->CAVIFileSynth::CreateInstance() result=0x0, object=003E3B48

warning: 003E3B48->CAVIFileSynth::AddRef() gRefCnt=3, m_refs=2

warning: 003E3B48->CAVIFileSynth::Release() gRefCnt=2, m_refs=1

warning: 003E3AF0->CAVIFileSynth::Release() gRefCnt=1, m_refs=0

warning: 003E3AF0->CAVIFileSynth::~CAVIFileSynth(), gRefCnt = 1

warning: 003E3B48->CAVIFileSynth::QueryInterface() {00000000-0000-0000-c000-0000
00000046} (IUnknown)

warning: 003E3B48->CAVIFileSynth::AddRef() gRefCnt=2, m_refs=2

warning: 003E3B48->CAVIFileSynth::Release() gRefCnt=1, m_refs=1

warning: 003E3B48->CAVIFileSynth::QueryInterface() {00020025-0000-0000-c000-0000
00000046} (unsupported!)

warning: 003E3B48->CAVIFileSynth::QueryInterface() {0000010b-0000-0000-c000-0000
00000046} (IPersistFile)

warning: 003E3B48->CAVIFileSynth::AddRef() gRefCnt=2, m_refs=2

warning: 003E3B48->CAVIFileSynth::QueryInterface() {00020020-0000-0000-c000-0000
00000046} (IAVIFile)

warning: 003E3B48->CAVIFileSynth::AddRef() gRefCnt=3, m_refs=3

warning: 003E3B48->CAVIFileSynth::Load("H:\BD\RD\rd.avs", 0x0)

warning: 003E3B48->CAVIFileSynth::Release() gRefCnt=2, m_refs=2

warning: 003E3B48->CAVIFileSynth::Release() gRefCnt=1, m_refs=1

warning: 003E3B48->CAVIFileSynth::GetStream(*, 73646976(vids), 0)

warning: 02A15308->CAVIStreamSynth(video)

warning: 02A15308->CAVIStreamSynth::AddRef() (video) gRefCnt=2, m_refs=1

warning: 003E3B48->CAVIFileSynth::AddRef() gRefCnt=3, m_refs=2

warning: 003E3B48->CAVIFileSynth::Release() gRefCnt=2, m_refs=1

warning: 02A15308->CAVIStreamSynth::Info(0027F6C8, 204) (video)

avis [info]: 1280x960 @ 23.98 fps (180360 frames)
warning: 02A15308->CAVIStreamSynth::Info(0027F6D8, 204) (video)

x264 [info]: using SAR=1/1
x264 [info]: using cpu capabilities: MMX2 SSE2Fast SSSE3 PHADD SSE4.1 Cache64

Program received signal SIGSEGV, Segmentation fault.
0x7728f217 in ntdll!RtlDecompressBuffer () from C:\Windows\system32\ntdll.dll
(gdb) bt full
#0 0x7728f217 in ntdll!RtlDecompressBuffer ()
from C:\Windows\system32\ntdll.dll
No symbol table info available.
#1 0x90909090 in ?? ()
No symbol table info available.
#2 0x00000012 in ?? ()
No symbol table info available.
#3 0x0bc36230 in ?? ()
No symbol table info available.
#4 0x0027f26c in ?? ()
No symbol table info available.
#5 0x75593593 in KERNEL32!GetNumaAvailableMemoryNode ()
from C:\Windows\syswow64\kernel32.dll
No symbol table info available.
#6 0x003e0000 in ?? ()
No symbol table info available.
#7 0x00000000 in ?? ()
No symbol table info available.
(gdb) See Above (http://forum.doom9.org/showthread.php?p=1213197#post1213197) for arguments

[kemuri-_9]

all right, well this is getting more and more confusing...
the debug i provided was just r1028 straight from the repository with debug information & unprofiled,
guess it's time to try one with the same patches that skystrife used at least.

x264_mod_debug.exe (http://kemuri9.net/dev/x264/x264_mod_debug.exe)

see how it goes with this.

[Sharktooth]

or try profiling it...

[kemuri-_9]

that would be the next step if the + patches one works fine.

[Audionut]

Here 1028 crashes in CRF mode and 1024 works fine on Vista, core2duo.

My 1028 build that has no patches and is profiled works fine for me. I'd say it is a problem with one of the patches skystrife is using.

I use gcc 3.4.5 aswell.

[Dark Shikari]

If it turns out CFLAGS cause the crash (e.g. -O3, etc) you can compile a working debug version with --extra-cflags="-ffast-math -O3" and --enable-debug as well.

[Emp3r0r]

tried mod_debug

x264 [info]: using SAR=1/1
x264 [info]: using cpu capabilities: MMX2 SSE2Fast SSSE3 PHADD SSE4.1 Cache64
≡¡║ε½½½½½½½½■ε■ε■ε■ = (null)aram: ¡║
x264 [error]: failed to parse zones
x264 [error]: x264_encoder_open failed
warning: 02A55308->CAVIStreamSynth::Release() (video) gRefCnt=1, m_refs=0

warning: 02A55308->~CAVIStreamSynth() (video), gRefCnt = 1

warning: 010D3B60->CAVIFileSynth::Release() gRefCnt=0, m_refs=0

warning: 010D3B60->CAVIFileSynth::~CAVIFileSynth(), gRefCnt = 0

warning: DllCanUnloadNow(): gRefCnt = 0

warning: DllMain: hModule=0x10000000, ulReason=0, lpReserved=0x00000000, gRefCnt
= 0


Program exited with code 037777777777.

[kemuri-_9]

seems it was what i was beginning to suspect:
the zone parser patch is apparently breaking on vista (but works fine on my XP)
guess i'll have to look at it some more.

Edit:
i think i found the problem with the original version.
here is a build with the updated version, give it a try please.
x264_mod2_debug.exe (http://kemuri9.net/dev/x264/x264_mod2_debug.exe)

if this doesn't work, i'll just have to crack at it some more, debugging ftw...

[Emp3r0r]

that fixed it! thanks

[kemuri-_9]

< pengvado> macro, or call x264_log_default

like this?
x264.muxers.respect.log.level.diff (http://kemuri9.net/dev/x264/patches/x264.muxers.respect.log.level.diff)
(same url, updated file)

[Japhsoncross]

lossless encoding is broken with rev999+. Rev987 is OK.

[Adub]

Nope, it isn't. They changed their lossless format to a better one, meaning much more compression, it's just that there is only one decoder that supports it so far. There will hopefully be a patch to ffmpeg/mplayer soon.

[nm]

lossless encoding is broken with rev999+. Rev987 is OK.
It's not broken but replaced with a different kind of lossless profile that is not yet supported by most decoders. Try latest CoreAVC.

commit b35a044b95c7eab2c91f55a3ac4100ca26a29d92 r994
Author: Jason Garrett-Glaser <darkshikari@gmail.com>
Date: Sat Sep 27 16:37:27 2008 -0700

Replace High 4:4:4 profile lossless with High 4:4:4 Predictive.
This improves lossless compression by about 4-25% depending on source.
The benefit is generally higher for intra-only compression.
Also add support for 8x8dct and i8x8 blocks in lossless mode; this improves compression very slightly.
In some rare cases 8x8dct can hurt compression in lossless mode, but its usually helpful, albeit marginally.
Note that 8x8dct is only available with CABAC as it is never useful with CAVLC.
High 4:4:4 Predictive replaced the previous profile in a 2007 revision to the H.264 standard.
The only known compliant decoder for this profile is the latest version of CoreAVC.
As I write this, JM does not actually correctly decode this profile.
Hopefully this lack of support will soon change with this commit, as x264 will be (to my knowledge) the first compliant encoder.

[Selur]

Note that 8x8dct is only available with CABAC as it is never useful with CAVLC.
is this generally true or just with lossless encoding ?

[Dark Shikari]

is this generally true or just with lossless encoding ?Only true with lossless.

[Selur]

Okay, thanks. :)

[Japhsoncross]

Sorry, it's my bad :P

[kemuri-_9]

Hey Dark Shikari, I'm getting some crashes on x264 when running SSEMisalign w/ threads >1
--threads 1 and --thread-input still work fine, but --threads >1 with SSEMisalign crashes pretty quickly after startup.


(gdb) file x264_debug
Reading symbols from C:\MinGW\projects\x264/x264_debug.exe...done.
(gdb) run --threads 2 -B 1000 ../x264_build1.y4m -o test.mp4
[New thread 4516.0x19b0]
[New thread 4516.0x3d1c]
[New thread 4516.0x1acc]
[New thread 4516.0x231c]
[New thread 4516.0x1024]

Program received signal SIGSEGV, Segmentation fault.

bt full
#0 0x0048cb44 in x264_hpel_filter_c_sse2_misalign ()
No symbol table info available.
#1 0x004738db in x264_hpel_filter_sse2_misalign (dsth=0x2971a40 "",
dstv=0x29cf240 'ë' <repeats 200 times>..., dstc=0x2a2ca40 "",
src=0x2914248 'ë' <repeats 200 times>..., stride=704, width=656,
height=16) at common/x86/mc-c.c:232
buf = <incomplete type>
realign = 44223704
src = (uint8_t *) 0x2914240 'ë' <repeats 200 times>...
width = 664
height = 15
#2 0x0044d52e in x264_frame_filter (h=<incomplete type>,
frame=<incomplete type>, mb_y=0, b_end=0) at common/mc.c:380
b_interlaced = 0
stride = 704
width = 640
start = -8
height = 8
offs = -5640
x = -5640
y = 0
mb_y = -664
...


any ideas? Oh and let me know if you need some more info

[Dark Shikari]

... don't tell me you're serious.

The misalign mask has to be set once per thread? :rolleyes:

Try this in encoder.c (find the function, add the two lines):

static int x264_slices_write( x264_t *h )
{
int i_frame_size;
if(h->param.cpu&X264_CPU_SSE_MISALIGN)
x264_cpu_mask_misalign_sse();

[kemuri-_9]

that did the trick, working fine now. :)

[Mad_Martha]

Hi , have had a continuing issue with later versions of x264 (since about v750 , ish , onwards in fact)

Did a search here to see if it was already mentioned but came up blank , so I ask here...

Apologies if this has been covered elsewhere.

I'm using the AutoMKV front end , while updating (i.e. replacing) the x264 cli encoder , currently using v1062.

For some time now I've had hellishly slow encoding.
That said , yes , I'm using VERY slow settings BUT from what I'm seeing it is looking like x264 is NOT firing up all my CPU's functions (like SSE3 for example) - can't help but feel something isn't right...

I include below 3 things - a pic of the encoder working , a pic from CPU-Z describing my CPU and the input script being generated by AutoMKV.

http://homepage.ntlworld.com/mad.martha/x264encoding.png


http://homepage.ntlworld.com/mad.martha/cpu.png

Command Line 1' Pass X264: F:\BACKUPS\AutoMKV\AutoMKV0981\exe\encoder\x264.exe --pass 1 --bitrate 1274 --stats "D:\X264 Tryout Area\0981 - Standard - v1062\temp\.stats" --progress --keyint 250 --bframes 8 --qpmin 10 --qpmax 51 --no-psnr --no-fast-pskip --mixed-refs --trellis 2 --ref 12 --filter -2,-1 --subme 8 --direct auto --vbv-bufsize 14000 --vbv-maxrate 25000 --me tesa --pre-scenecut --no-ssim --level 4.1 --merange 32 --weightb --b-adapt 2 --b-pyramid --partitions p8x8,b8x8,i4x4,i8x8,p4x4 --8x8dct --threads auto --thread-input --aq-mode 1 --aq-strength 1 --psy-rd 1.0:1.0 --sar 15787623:11119680 --output NUL "D:\X264 Tryout Area\0981 - Standard - v1062\temp\movie.avs"


Any ideas anyone ?

Thanks ,

[MM]

[Dark Shikari]

Hi , have had a continuing issue with later versions of x264 (since about v750 , ish , onwards in fact)

Did a search here to see if it was already mentioned but came up blank , so I ask here...

Apologies if this has been covered elsewhere.

I'm using the AutoMKV front end , while updating (i.e. replacing) the x264 cli encoder , currently using v1062.

For some time now I've had hellishly slow encoding.
That said , yes , I'm using VERY slow settings BUT from what I'm seeing it is looking like x264 is NOT firing up all my CPU's functions (like SSE3 for example) - can't help but feel something isn't right...SSE3 provides no useful functionality for x264 except on the Pentium 4D. We're not going to be dishonest and display capabilities that aren't being used.

[Mad_Martha]

Thanks for the quick reply - interesting that SSE3 don't help much..

OK , follow on question : Given the rather "intense" settings I'm using - any suggestions on speeding up processing , while maintaining quality ?
(while also remembering I can't afford a qx9775)
I encode at very low bitrates and would rather use as many features (that aid in compression) as possible..

[MM]

[LoRd_MuldeR]

First of all "--aq-mode" never existed in official builds and "--aq-strength 1" is useless, as it's the default value. Also "--bframes 8" is overkill, more than "--bframes 3" doesn't help much and is only significant slower with "--b-adapt 2". I'd also say that "--ref 12" is overkill, except for anime maybe. The same applies to "--merage 32", as more than "--merage 16" doesn't help much. Last but not least you should leave "--partitions" out and use the defaults instead. The default is "all" partitions, except for 4x4 (as 4x4 isn't helpful, except for very low resolutions maybe). Oh, and Psy-Trellis 1.0 may be a bit too strong...

[Dark Shikari]

First of all "--aq-mode" never existed in official buildswhat.

[LoRd_MuldeR]

First of all "--aq-mode" never existed in official buildswhat.

Whoops, seems I thought of "--aq-sensitivity" :p

But "--aq-mode 1" is useless anyways, as it's just the default value. And I forgot that "--me tesa" is overkill too, unless you really don't care about speed ;)

[Mad_Martha]

And I forgot that "--me tesa" is overkill too, unless you really don't care about speed ;)

Yeah , my main concern is quality/bit rate rather than speed - I push x264's compression to it's limits (and sometimes a bit beyond the knife edge).

Unfortunately (for me) there are quite a few people out there that *literally* go over my encodes frame by frame looking for what they percieve as "errors". :(

...and oh boy do they moan about it...

The "redundant" settings were in because I am currently playing with things like AQ and the PSY-trellis settings.
(they were folded back for the example).

Thanks for the info tho - gives me summat to play with over the next few days.

[MM]

[D734]

Yeah , my main concern is quality/bit rate rather than speed - I push x264's compression to it's limits (and sometimes a bit beyond the knife edge).

Unfortunately (for me) there are quite a few people out there that *literally* go over my encodes frame by frame looking for what they percieve as "errors". :(

...and oh boy do they moan about it...

The "redundant" settings were in because I am currently playing with things like AQ and the PSY-trellis settings.
(they were folded back for the example).

Thanks for the info tho - gives me summat to play with over the next few days.

[MM]
indeed like the quality over speed mentality :)
one more thing to tweak would be subme 9 instead of 8 (barely slows things down)
something a little more specific about the ref frame is that i've heard any more then 5-6 really wont help unless anime/cartoon content.
some people trying to do max quality use 3pass, but according to the x264 developers 3pass does not increase the quality.

i've personally encoded and compared 720x576 encodes at 1mbit and the command line below worked a lot better then the old automkv "insane_quality" you've used before.

"c:\x264.exe" --pass 1 --bitrate #### --stats "c:\x264.stats" --level 4.1 --ref 5 --bframes 3 --subme 9 --me tesa --trellis 2 --b-adapt 2 --mixed-refs --no-fast-pskip --b-pyramid --weightb --direct auto --filter -2,-1 --partitions p8x8,b8x8,i4x4,i8x8 --8x8dct --threads auto --thread-input --sar 1:1 --cqmfile "C:\Prestige CQM.cfg" --progress --no-dct-decimate --no-psnr --no-ssim --output NUL "x:\some.avs"

"c:\x264.exe" --pass 2 --bitrate #### --stats "c:\x264.stats" --level 4.1 --ref 5 --bframes 3 --subme 9 --me tesa --trellis 2 --b-adapt 2 --mixed-refs --no-fast-pskip --b-pyramid --weightb --direct auto --filter -2,-1 --partitions p8x8,b8x8,i4x4,i8x8 --8x8dct --threads auto --thread-input --sar 1:1 --cqmfile "C:\Prestige CQM.cfg" --progress --no-dct-decimate --no-psnr --no-ssim --output x:\file.mkv "x:\some.avs"

not sure if it can be improved or not as far as quality (cqm matrice is obviously up to the encoder to use or not, i've just heard good stuff about prestige cqm and seen non scene hd encoders use it)
overall the speed was around 33% longer then automkv 2_pass_insane_quality profile (with equal 1 and 2 pass settings) because the automkv profile used ref 16 (vs 5or6) which doubled its encoding time.

[Snowknight26]

Non flat cqm and no --psy-rd. /facepalm

[Dark Shikari]

Non flat cqm and no --psy-rd. /facepalmBecause it isn't like psy-rd is on by default, is it? :sly:

[Snowknight26]

My mistake, switch my 'and' with 'or.' ;)

[check]

Unfortunately (for me) there are quite a few people out there that *literally* go over my encodes frame by frame looking for what they percieve as "errors". :(
Hint: --crf :)

[tal.aloni]

Hey Guys,
Amazing work on x264, Dark Shikari especially,
There is a feature I'm really missing, and I'm willing to get my hands dirty (been involved with ffdshow too)

good x264 encode is taking about a week on my Phenom 8450, and since all my computer fans are auto-adjusted to temperature, I can't have anybody over for a movie during the encoding week... if I could just hit 'spacebar' and pause the encoding for two hours (memory usage can stay as is), it would be terrific.

I would appreciate if you could guide me through how this can be implemented the best way. (if at all)

Thanks!
Tal Aloni

Edit:
I'm guessing it should be done right before updating the status line (x264.c),
and if 'spacebar' has been pressed, just print "[Paused] [58.5%] 128170/218980 frames. (press spacebar to continue)", and enter a sleep loop, untill 'spacebar' has been pressed again, is this ok? (fps statistics should be resetted somehow)

[nurbs]

I think process explorer can do that.

[J_Darnley]

If you are using a frontend then you will need to use nurbs suggestion, but if you are using the x264 binary from a command prompt then just use the pause key (and spacebar to resume).

[tal.aloni]

I think process explorer can do that.

Great, I wasn't aware of that, Thanks!

if you are using the x264 binary from a command prompt then just use the pause key (and spacebar to resume).
I do, I wasn't aware of that either! Thanks!

[kemuri-_9]

I'm guessing it should be done right before updating the status line (x264.c),
and if 'spacebar' has been pressed, just print "[Paused] [58.5%] 128170/218980 frames. (press spacebar to continue)", and enter a sleep loop, untill 'spacebar' has been pressed again, is this ok? (fps statistics should be resetted somehow)

there's not much particular interest for this, especially since the main devs don't use windows!

i believe there was a patch floating around to somewhat handle this...

but overall, if you pause it, your fps encoding rate will become junk,
as x264 currently doesn't keep track of when it's "paused" to subtract from the total time the process has been alive.

[Dark Shikari]

Hey Guys,
Amazing work on x264, Dark Shikari especially,
There is a feature I'm really missing, and I'm willing to get my hands dirty (been involved with ffdshow too)

good x264 encode is taking about a week on my Phenom 8450, and since all my computer fans are auto-adjusted to temperature, I can't have anybody over for a movie during the encoding week... if I could just hit 'spacebar' and pause the encoding for two hours (memory usage can stay as is), it would be terrific.It's the operating system's job to pause a process, not x264's.

It's trivial to pause both on Windows and Linux (Ctrl-S, iirc).the main devs don't use windows!¯\(º o)/¯

[ajp_anton]

but overall, if you pause it, your fps encoding rate will become junk,
as x264 currently doesn't keep track of when it's "paused" to subtract from the total time the process has been alive.I often have this "problem" when I use the CPU for other things with the encode running in the background.
What about changing "--progress" into "--progress n" where "n" is the number of frames used to average the fps/eta, defaulting to 0 which means everything encoded so far?

[J_Darnley]

If I recall correctly there are, or were, a couple of patches that do this last time people started banging the pause/suspend/resume drum. Even I wrote one, so perhaps I should clean it up and post it.

[professor_desty_nova]

I just noticed that H.264 (2007) Corrigendum 1 (01/09) was released last week in http://www.itu.int/rec/T-REC-H.264. Does it have any implication on x264?

[akupenguin]

Does it have any implication on x264?
Dunno. It was published as a whole new document, and PDFs aren't very convenient for taking diffs. I don't plan to reread it just to see if anything interesting was added, so if there are any new features people want implemented, they had better speak up.

[Dark Shikari]

It has SVC in it, which officially makes the spec longer than MPEG-4 Part 2.

[Raptus]

[...]I don't do releases or "big iterations". I just do features.[...]
[...]if there are any new features people want implemented, they had better speak up.
Since there are no plans or roadmaps for x264, by which criteria do you devs choose what features to implement? What people ask for the most and/or what seems reasonable and/or doable?

[Dark Shikari]

Since there are no plans or roadmaps for x264, by which criteria do you devs choose what features to implement?I have an idea. It sounds like a good idea. So I try it. My development is idea-bottlenecked, generally, not by coding time or anything.

Also, money is a good motivator.

[Chengbin]

I have an idea. It sounds like a good idea. So I try it. My development is idea-bottlenecked, generally, not by coding time or anything.

Also, money is a good motivator.

There is only so much you can improve a 1MB program. After almost 1100 revisions the room for improvement certainly is running dry.

I heard x264 focus on main and high profile. Some features in simple profile has not yet been added. Maybe you can waste some time doing that.

I would donate, but my parents won't let me for obvious reasons. So the least I can say is thank you, Dark Shikari, and other people who contributed to x264.

[akupenguin]

There is only so much you can improve a 1MB program
You say that as if the size is a constant, unaffected by our improvements.

[Dark Shikari]

You say that as if the size is a constant, unaffected by our improvements.Indeed, we make it consistently smaller over time ;)

[Chengbin]

What I mean is that 1MB is a small program. Not a gazillion codes. It has gone through over 1000 revisions. I think Windows XP only had about 2000 fixes. (I know I'm comparing apples to oranges)

Or maybe I'm naive and a 1MB program can be extremely complicated as well.

[Sagittaire]

Or maybe I'm naive and a 1MB program can be extremely complicated as well.

Well you are naive here because size of program and complexity of developpement are not same thing. There are many possible improvement for x264:
- HRD SEI support
- Complete H264 interlacing support
- FGM support (really complexe)

And not exclusive part of H264 with HVS improvement: In this domain there are virtualy infinite possibility with AQ, Custom Matrix, temporal stability ... etc etc

[cogman]

What I mean is that 1MB is a small program. Not a gazillion codes. It has gone through over 1000 revisions. I think Windows XP only had about 2000 fixes. (I know I'm comparing apples to oranges)

Or maybe I'm naive and a 1MB program can be extremely complicated as well.

Sometimes we just don't know how big 1MB really is. I know our ideas are dwarfed by the GB we deal with today, however take note.

1 MByte = 1024 kByte ~= 1024000 bytes ~= 8192000 bits. There are two possibilities for each of those bits, 1, or 0. so that is 2^81920000 possible combinations. Now, I realize that not all of those combinations actually mean something, however even if 1/1,000,000,000th of those combination are valid H.264 encoders, we can see that we have a lot of possibilities available to us.

Given that x264 is starting to use asm on an increasingly large scale, I have little doubt that we will keep seeing quality / speed improvements for a good while (until some other standard MPEG 8 perhaps comes and knocks it off its pedistal, even then, my bet is that a fair amount of x264 code will still be valid for a new MPEG standard).

Lots of wiggle room when you think about it. Not to mention the fact that it isn't beyond reason for x264 to hit 1.5 mb *gasp* dare I speak it!

[Guest]

I would donate, but my parents won't let me for obvious reasons. It's not obvious to me. What is the reason?

[Chengbin]

It's not obvious to me. What is the reason?

Mom, can I use your credit card to donate to this guy who wrote this really nice software for free?

I have to have insanely rich parents to do that.

[Shinigami-Sama]

the room for improvement certainly is running dry.

theres still C-code in it theres still room for improvment
not to mention there always ALWAYS microfixes, 1cycle here, 3 cycles there, couple dozen of those on things called a few times a second add up ;)



edit
Mom, can I use your credit card to donate to this guy who wrote this really nice software for free?

I have to have insanely rich parents to do that.

lern 2 paypal?

[Guest]

I have to have insanely rich parents to do that. Your parents have to be "insanely rich" to let you make a ten buck donation?!

[Dark Shikari]

You know, when I was a kid, and I wanted to buy something or donate money or whatever, I would go get the cash out of my own pocket and do it myself.

But apparently that's out of fashion these days.

[Chengbin]

Your parents have to be "insanely rich" to let you make a ten buck donation?!

Wouldn't you be embarassing yourself if you donate $10?

You know, when I was a kid, and I wanted to buy something or donate money or whatever, I would go get the cash out of my own pocket and do it myself.

But apparently that's out of fashion these days.

I would, but my parents hold all my money, and I'm not exactly free to use that money on anything I like. Especially when my dad lost a job, not exactly in a position to ask. Maybe if Dark Shikari is still around in 5 years doing this, I'll donate, since I'm in university by then, and I'll be free to use money on whatever I want. But then I probably not have time to waste on encoding and learning how to use x265.

[Rodger]

Woa guys!

STOP IT! That´s not part of this threat!
Please return to the topic and stop personal stuff....It just won´t work out well.

[Chengbin]

Going back on topic.

Is x264 at a point where there can be no more quality improvement (except the grain feature) and only speed improvements can be done?

[Guest]

Is x264 at a point where there can be no more quality improvement (except the grain feature) and only speed improvements can be done? Of course not. Where did you get such a silly idea? Do you even know how encoders work?

[burfadel]

VAQ could be changed, depending on how the VAQ 2.0 patch is going (if its still being modified)...?

[Chengbin]

Of course not. Where did you get such a silly idea? Do you even know how encoders work?

Not exactly unforunately. I've just touched x264 lately.

I just thought that the last 200ish revisions are all either speed improvements and/or bug fixes and Dark Shikari said that he's idea bottlenecked.

Is there a link that explains how the encoder works, and hopefully help me understand more about x264 and the improvements that can be made.

[Guest]

There are lots of resources which you can easily find with Google.

Here are some areas to consider when thinking about possible evolution of x264:

motion estimation
frame type selection
macroblock coding type selection
quantization
adaptive MBAFF support
pulldown flags
VBV compliance

I am sure there are many more.

[burfadel]

A few weeks back Dark Shikari said they were working on a massive rework of CABAC (I think it was) which should give a 'large' increase in performance... wonder how thats coming along?

[Dark Shikari]

A few weeks back Dark Shikari said they were working on a massive rework of CABAC (I think it was) which should give a 'large' increase in performance... wonder how thats coming along?You mean this (http://git.videolan.org/?p=x264.git;a=commit;h=84a1ca6ce70fe7bad4922ddc5a72c2e9cd73703b)?

[burfadel]

Oops! no :) I checked back through my post and I was referring to this:

http://akuvian.org/src/x264/satd.diff

http://akuvian.org/src/x264/sea.diff

Where you wrote:
The first is the first part of an evil plan to make an obnoxiously fast SATD function. Current estimations say that we might be able to beat the current one by over 60% on Conroe Core 2. Once all the parts of the evil plan come together, they may result in the largest speed gain in a single patch in at least a year.

The second one is a prototype of an attempt to make exhaustive a little bit faster.

[Dark Shikari]

Oops! no :) I checked back through my post and I was referring to this:

http://akuvian.org/src/x264/satd.diff

http://akuvian.org/src/x264/sea.diff

Where you wrote:Talk to holger, he is the one currently in charge of the evil plan. Blame him for it taking bloody ages.

[Sagittaire]

Going back on topic.

Is x264 at a point where there can be no more quality improvement (except the grain feature) and only speed improvements can be done?

Well It will be certainely difficult to have big improvement for PSNR but HVS improvement are virtualy infinite.

[Esurnir]

Well It will be certainely difficult to have big improvement for PSNR but HVS improvement are virtualy infinite.

Talking about this (http://en.wikipedia.org/wiki/Human_Visual_System_Model) ?

[Sharktooth]

yep. HVS optimizations on digital video are ways to optimize the picture so the human eye can perceive a better picture "quality".

[Esurnir]

yep. HVS optimizations on digital video are ways to optimize the picture so the human eye can perceive a better picture "quality".

Or "It look better to me, screw the stupid metric"?

[IgorC]

I think LAME is an excellent example of long time development. It's 10 years old and there are still improvements of quality and performance.
Even old technology MPEG-2 encoder (HCenc) obtained very important HVS improvement (VAQ) in the past year.

[LoRd_MuldeR]

I think LAME is an excellent example of long time development. It's 10 years old and there are still improvements of quality and performance.

Well, if you look at the LAME history (http://lame.cvs.sourceforge.net/*checkout*/lame/lame/doc/html/history.html) then you'll see "features and bug fixes which affect quality" and "features and bug fixes which affect speed" are pretty rare for the past two years...

[IgorC]

Settings for LAME encoders:
3.98.2 -V5 (vbr new by default)
3.97 -V5 --vbr-new
3.96 -V5 --vbr-new
3.90.3 -V5 (vbr new wasn't still recommended)

One flac album (13 files)

Results:
3.98.2 - 47.47x
3.97 - 41.3x
3.96 - 35.0x
3.90.3 - 18.3x

The quality was also imrpoved enough if you visit HA forum.

[IgorC]

Well, if you look at the LAME history (http://lame.cvs.sourceforge.net/*checkout*/lame/lame/doc/html/history.html) then you'll see "features and bug fixes which affect quality" and "features and bug fixes which affect speed" are pretty rare for the past two years...
:rolleyes:
It's full of quality and speed omptimizations.

[Dark Shikari]

Settings for LAME encoders:
3.98.2 -V5 (vbr new by default)
3.97 -V5 --vbr-new
3.96 -V5 --vbr-new
3.90.3 -V5 (vbr new wasn't still recommended)

One flac album (13 files)

Results:
3.98.2 - 47.47x
3.97 - 41.3x
3.96 - 35.0x
3.90.3 - 18.3x

The quality was also imrpoved enough if you visit HA forum.This assumes that -V5 is always the same quality no matter what happens to the underlying code, which is rather similar to the same assumption with CRF (bad idea).

[LoRd_MuldeR]

:rolleyes:
It's full of quality and speed omptimizations.

I said "for the past two years". Well, I should have said "this year and the previous one". If you look at it carefully, only two bugs that possibly effect quality were fixed in entire 2008 ;)

The last significant changes for quality/speed were made back in Dec 2007, more than a year ago...

[IgorC]

It's different for audio encoding. Yes, it's not ideal, but for practical purpose it's more than suitable. People who used 3.96 -V5 jumped into 3.97 -V5. Maybe there some little difference 1-2 kbps but it doesn't matter for them. It actually happens on HA forum.
Maybe it will be more correct to compare 3.98 -V5.7 and 3.97 -V5 but it doesn't actually affect a lot results (+/- 1%) for speed.

[IgorC]

I said "for the past two years". And if you look at it carefully, only two bugs that may effect quality were fixed in 2008 ;)

The last significant changes for quality/speed were made back in Dec 2007, more than a year ago...
It's not improvement for you +15% speed that happens 1-2 years ago since 3.98beta?
Well, develoment of LAME is different from x264. There are no revisions but alpha/beta versions. Right now there is no "public" alpha/beta. But it will be here anytime. I don't know actually plans of LAME devs.

[LoRd_MuldeR]

It's not improvement for you +15% speed that happens 1-2 years ago since 3.98beta?
Well, develoment of LAME is different from x264. There are no revisions but alpha/beta versions. Right now there is no "public" alpha/beta. But it will be here anytime. I don't know actually plans of LAME devs.

The last change in LAME that effects speed was back in 2007. No such change in 2008 and none in 2009 so far, according to the Changelog (http://lame.cvs.sourceforge.net/*checkout*/lame/lame/doc/html/history.html). That's what I said...

(The changelog also lists changes that have not been released yet, but again none of them is marked as "features and bug fixes which affect quality" or "features and bug fixes which affect speed")

[IgorC]

Read another changelog. A more detailed one.
http://sourceforge.net/mailarchive/forum.php?forum_name=lame-cvs
And I already said LAME IS NOT X264. There is no daily revisions. Last speed/quality changes was in 2007 (1.5 years ago) since then there were work around bugs and final version. There will be new version 3.99 alpha/beta with speed/quality optimizations.

[Sagittaire]

Well I think that audio and video are nor comparable. 3D video matrix is by far more complexe to compress. HVS is by far more complexe than human psychoaudio system.

[Esurnir]

Well I think that audio and video are nor comparable. 3D video matrix is by far more complexe to compress. HVS is by far more complexe than human psychoaudio system.

And it's not like Lame is arround since about twice as long as x264, with a spec that didn't changed a coma since 1991. While High profile is there since 2005.

[Chengbin]

x264 is at r1097 now, but when you type "x264 --version", it says r1092.

EDIT: I think it was a misprint. It says r1092, but it says built on Feb 1 2009

[kemuri-_9]

x264 is at r1097 now, but when you type "x264 --version", it says r1092.

seems to be an error on x264.nl, as all the dl links for x264 since and including r1092 are all the same revision.

[LoRd_MuldeR]

x264 is at r1097 now, but when you type "x264 --version", it says r1092.

My r1096 build tells its version properly:

C:\TEMP\x264\x264>x264.exe --version
x264 0.65.1096M 4c171c3
built on Jan 30 2009, gcc: 4.3.2

Check your build environment. The version is defined in "config.h" and updated when "./configure" is executed ;)

[burfadel]

Yep x264.nl has the wrong version information:
x264 0.66.1092 60f4cd8

built on Feb 1 2009, gcc: 3.4.6


The website states 1097.

Checking through the mirrors at the old revisions, 1097 and 1096 are the same (by comparing file size, as you would expect, 1097 doesn't affect Windows), likewise 1092 through 1095 are the same due to the revisions not affecting Windows.

Looks like its just a version numbering issue, and that it doesn't? affect the build.

[Dark Shikari]

Yep x264.nl has the wrong version:

x264 0.66.1092 60f4cd8

built on Feb 1 2009, gcc: 3.4.6


The website states 1097 however.The Wrong Version (http://meta.wikimedia.org/wiki/The_Wrong_Version)?

[burfadel]

Lol, well it looks like it is the correct build on the website, just the wrong internal numbering :)

[Gabriel_Bouvigne]

Well I think that audio and video are nor comparable. 3D video matrix is by far more complex to compress.
There is way more data to process in video, but I don't think that 3D video matrix would be more complex to compress than multichannel audio.

HVS is by far more complex than human psychoaudio system.
May I disagree? To me psychovisuals are not more complex than psychoacoustics. It's just that psychoacoustics have been widely used in lossy compression for significantly more time than psychovisuals, mainly because during the last 15 years brute force was providing enough compression benefits to not bother going the psychovisual way.

(Aren't we a bit off-topic there?)

[Raptus]

(Aren't we a bit off-topic there?)
Yes, but I find the discussion rather interesting :)

May I disagree? To me psychovisuals are not more complex than psychoacoustics.
If you are talking about the complexity or the refinement of currently implemented mechanisms to exploit psy-vis or psy-acoustic effects in lossy compression I tend to agree. But i wouldn't dare to say that the human visual system is less complex then the auditory.

Also, in the reproduction of video there are more unknown variables then with audio, which I think will add some difficulty during the further development of psy-vis schemes.

[techouse]

My build shows it correctly:
x264 --version
x264 0.66.1097M e404f35
built by techouse on Feb 2 2009, gcc: 4.3.2

[Chengbin]

Here is an idea for x264.

Can you make x264 use the GPU to decode the frames? If you're using a Bluray source, or using very fast settings, the amount of CPU decoding the frames can be significant.

[Blue_MiSfit]

x264 has nothing to do with decoding.

If you want to use your GPU for decoding, there are plenty of ways to do that - either via your software BluRay player, or through Media Player Classic HC's DXVA decoders. If you want to accelerate decoding source frames during encoding, neuron2's CUDA enabled applications are for you. Check out his website, there's a small fee involved.

~MiSfit

[burfadel]

Not to mention CUDA is Nvidia only... probably not going to be around forever since you now have OpenCL and the upcoming? Directx equivalent, both of which are compatible across all recent cards and not just a specific brand

[Esurnir]

Here is an idea for x264.

Can you make x264 use the GPU to decode the frames? If you're using a Bluray source, or using very fast settings, the amount of CPU decoding the frames can be significant.

x264 don't do any decoding at all, it's just linked to avisynth which handle the decoding or use .y4m or yuv data piped in that in the end are raw pictures, the avisynth library usage handle the .avi and .avs sources and convert them to yv12

[DiKey]

Is the "--interlaced" option working? I dont think so... Is any way to use it?
I tried to use it with top field first, with bottom fields first original file... But my Sony PS3 plays it like progressive video.
(sorry for my bad English)

[Dark Shikari]

Is the "--interlaced" option working? I dont think so... Is any way to use it?
I tried to use it with top field first, with bottom fields first original file... But my Sony PS3 plays it like progressive video.
(sorry for my bad English)You need to use the NAL-HRD patch and specify --tff explicitly.

--interlaced is a coding option, not a signaling option.

[DiKey]

Thank you. Is anywhere only NAL-HRD patched version of coder? (As i see: many patches = many bugs; one patch = more stability).

[LoRd_MuldeR]

Thank you. Is anywhere only NAL-HRD patched version of coder? (As i see: many patches = many bugs; one patch = more stability).

I'd go with that one:
http://forum.doom9.org/showpost.php?p=1255240&postcount=1701

[jult]

Is there really nobody maintaining a wiki-page with all the commands that can be used with x264.exe ? If there is one, I can't find the link for it.

Please, the world is in desperate need of a commandline reference for x264, and one that states WHICH version is compatible with it too. It's honestly getting waaaay too confusing with all the versions and patches that are out there these days.

Thanks in advance.

[DarkZell666]

Is there really nobody maintaining a wiki-page with all the commands that can be used with x264.exe ? If there is one, I can't find the link for it.

Please, the world is in desperate need of a commandline reference for x264, and one that states WHICH version is compatible with it too. It's honestly getting waaaay too confusing with all the versions and patches that are out there these days.

Thanks in advance.

What do you need that isn't in the x264.exe --long-help output ?

[video_magic]

Better than Longhelp, is a page that gives more detailed and clear explanations, and advice for someone who is new, or just trying to learn and make informed choices on which options to use and adjustment of the settings.

I have found this page helpful:
http://mewiki.project357.com/wiki/X264_Settings

[DarkZell666]

Better than Longhelp, is a page that gives more detailed and clear explanations, and advice for someone who is new, or just trying to learn and make informed choices on which options to use and adjustment of the settings.

I have found this page helpful:
http://mewiki.project357.com/wiki/X264_Settings

True, the default help isn't very enlightening for a beginner.
However, a command-line reference and a user-friendly wiki isn't exactly the same thing ... :p (hence my question ;)).

[jult]

Well, I also found that mewiki page before, but it doesn't have all settings included, and the settings aren't in any clear logical order.

What annoys me a little is the fact that its writers assume we will read it all, even when we're looking for just that one parameter. Nobody I know has the time to read the entire longhelp before he/she starts using it. Just not a very realistic assumption (or request, even). Also, searching inside a cmd.exe window isn't exactly easy.

It now seems there is a 'somewhat supported' XP x64 version of x264.exe, but when I tried using it, it seems to not work with .avs files the same way, or maybe that's caused by Avisynth not being x64. It gets so tiring when its documentation is not structured or organised in a logical way.

[burfadel]

Why not someone set up a definitive page, then just add the line at the end of the help in x264 'refer to (web address) for etc ...

[nm]

Well, I also found that mewiki page before, but it doesn't have all settings included, and the settings aren't in any clear logical order.
The order in mewiki is the same as in x264 --longhelp. It also looks pretty logical to me and the section titles help users locate options that they are interested in. Are there some specific improvements or ideas that you have on your mind?

What annoys me a little is the fact that its writers assume we will read it all, even when we're looking for just that one parameter. Nobody I know has the time to read the entire longhelp before he/she starts using it. Just not a very realistic assumption (or request, even). Also, searching inside a cmd.exe window isn't exactly easy.
The writers assume that people who use the command-line interface have a decent set of tools to view and search such listings: grep and less for instance. If you don't have them, just direct the output to a file and open it in some text editor.

It now seems there is a 'somewhat supported' XP x64 version of x264.exe, but when I tried using it, it seems to not work with .avs files the same way, or maybe that's caused by Avisynth not being x64. It gets so tiring when its documentation is not structured or organised in a logical way.
I don't think any amount of documentation on the command-line parameters would help with this issue. What you need is general understanding on how x86-64 and a mixed 32/64-bit environment works. As you thought, the problem is that 64-bit x264 can't communicate directly with 32-bit AviSynth. So, get a 64-bit AviSynth build (http://members.optusnet.com.au/squid_80/) or use Avs2YUV to pipe from a 32-bit AviSynth process to x264 (this may be difficult if you aren't familiar with command-line interfaces and shell scripts).

[Atak_Snajpera]

or use Avs2YUV to pipe from a 32-bit AviSynth process to x264 (this may be difficult if you aren't familiar with command-line interfaces and shell scripts).
Ripbot264 will automatically use this method on x64 OS. Just my 3 cents if you are to lazy to do it manually :)

[LoRd_MuldeR]

or use Avs2YUV to pipe from a 32-bit AviSynth process to x264 (this may be difficult if you aren't familiar with command-line interfaces and shell scripts).

I even put together a simple tool especially for that purpose:
http://forum.doom9.org/showthread.php?t=144140

[JarrettH]

It's getting super fast :devil:

Approximately 5hrs to encode a 2hr movie using rev1125 with Unrestricted 1-Pass Constant Quality HQ. Only using a Core 2 Duo E6600 (2.40 GHz) here. Swell job! :cool:
I bet you could knock that down to 3hrs with say an E8500 (3.16 GHz). No SSE4 on the E6600 model.

[Sagekilla]

Grab a Core i7 920 (Or better yet, 965) and that would be done in real time or faster ;)

[puffpio]

yes the latest batch of commits has me salivating...
dark shikari: any change of updating your x264 developer blog w/ insights into the latest round of commits that speed things up so? of course, i understand you have your own priorities and time constraints

[Dark Shikari]

yes the latest batch of commits has me salivating...
dark shikari: any change of updating your x264 developer blog w/ insights into the latest round of commits that speed things up so? of course, i understand you have your own priorities and time constraintsI probably should, and while I have a lot of stuff in the realm of midterms and such at the moment, I have no excuse for not updating it for so long.

... on the other hand, why should I write a simple, readable public explanation of how all our competitors could fix their assembly to not suck? We did a little objdump trip the other day and discovered that most of our proprietary competitors' asm code is still utterly atrociously terrible... ;)

[Chengbin]

Maybe you should update your blog when the 5th codec comparison finishes, and you can explain the comparison.

BTW, what revision of x264 was used for the codec comparison? The recent few days brought massive speed up, and considering the test machine was a Q6600, it will give x264 even more of an advantage.

I still can't believe they didn't use an i7 rig.

[Dark Shikari]

Maybe you should update your blog when the 5th codec comparison finishes, and you can explain the comparison.

BTW, what revision of x264 was used for the codec comparison? The recent few days brought massive speed up, and considering the test machine was a Q6600, it will give x264 even more of an advantage.

I still can't believe they didn't use an i7 rig.I'm not expecting too much from the comparison; the speed targets they used were unbelievably low for the test system, which may hurt x264 (as x264 is primarily optimized for somewhat-sane speed levels, not ultra-slow cases). I added my own "subme 11" algorithm for the purpose of wasting more time to bring x264 down to their speed targets (and getting a tiny bit more quality). It was literally written in about 2 minutes though, so if I actually took a few weeks to sit down and find ways to optimize more for the absurdly-slow speed case, I could probably do much better, especially since MSU is measuring nothing but SSIM (so I wouldn't have to worry about visual quality...)

[LoRd_MuldeR]

Hmmm, subme 11 http://www.soadfans.de/forum/images/armageddon/smilies/sabber.gif

[burfadel]

Hmmm subme 10? lol

I realise its probably meant to indicate an 'all frames' case, but if subme 11 features really do slow it down that much a targeted subme 10 may be of benefit?...! (hypothetically)

[Dark Shikari]

Hmmm subme 10? lol

I realise its probably meant to indicate an 'all frames' case, but if subme 11 features really do slow it down that much a targeted subme 10 may be of benefit?...! (hypothetically)If you can't figure out why I called it subme 11 (http://www.youtube.com/watch?v=d54UU-fPIsY) instead of subme 10... ;)

[Sagekilla]

x264: These go to 11 ;)

Any chance of seeing subme 11 in public builds?

[Dark Shikari]

x264: These go to 11 ;)

Any chance of seeing subme 11 in public builds?Only if someone comes up with an algorithm more reasonable than an 11x11 exhaustive qpel-RD search... :devil:

[Chengbin]

so if I actually took a few weeks to sit down and find ways to optimize more for the absurdly-slow speed case, I could probably do much better

Do you plan doing that?

subme11 sounds interesting. How much time does it waste?

[Dark Shikari]

Do you plan doing that?No.subme11 sounds interesting. How much time does it waste?A lot.

[LoRd_MuldeR]

commit 682b54d6175f98dfa14fec4d951f4b3b6e686b95 r1128
Author: Jason Garrett-Glaser <darkshikari@gmail.com>
Date: Tue Mar 17 11:01:57 2009 -0700

SSE2 zigzag_interleave
Replace PHADD with FastShuffle (more accurate naming).
This flag represents asm functions that rely on fast SSE2 shuffle units, and thus are only faster on Phenom, Nehalem, and Penryn CPUs.

I wonder, does this apply to "Kentsfield" Core 2 Quads (e.g Q6600) too?

(Note: x264 does display the "SSE2Fast" capability for my Q6600, although it doesn't equal any of the CPU types mentioned in the changelog)

[Dark Shikari]

commit 682b54d6175f98dfa14fec4d951f4b3b6e686b95 r1128
Author: Jason Garrett-Glaser <darkshikari@gmail.com>
Date: Tue Mar 17 11:01:57 2009 -0700

SSE2 zigzag_interleave
Replace PHADD with FastShuffle (more accurate naming).
This flag represents asm functions that rely on fast SSE2 shuffle units, and thus are only faster on Phenom, Nehalem, and Penryn CPUs.

I wonder, does this apply to "Kentsfield" Core 2 Quads (e.g Q6600) too?

(Note: x264 does display the "SSE2Fast" capability for my Q6600, although it doesn't equal any of the CPU types mentioned in the changelog)I said "FastShuffle" not "SSE2Fast". The entire point of "FastShuffle" is to explicitly leave out the Conroe.

[LoRd_MuldeR]

I said "FastShuffle" not "SSE2Fast". The entire point of "FastShuffle" is to explicitly leave out the Conroe.

I understand, thanks.

[burfadel]

Penryn is a type of Conroe, but we know what you mean :) (and yes it shows on a Q9400). Glad I got the Q9400 instead of the Q6600 (which many people seem to favour)!...

[Dark Shikari]

Penryn is a type of Conroe, but we know what you mean :) (and yes it shows on a Q9400). Glad I got the Q9400 instead of the Q6600 (which many people seem to favour)!...Conroe is the generic term for 65nm, Penryn for 45nm.

Perhaps you meant "Woodcrest is a type of Penryn" or something of the sort?

[burfadel]

Ah sorry! my mistake :)

Just realised I was probably thinking of Core 2, not Conroe :S

[burfadel]

Although this isn't entirely the right place to ask, but didn't think it was worth its own thread in itself. I have a very valid (in my opinion) question about the inbuilt noise reduction function in x264. The function I am referring to is the --nr xxx (where xxx is the noise reduction amount). I have noticed that is can result in quite a decent bitrate drop in CRF mode (10 percent) without visually impacting the output. Now, the question relates to the actual quality difference and whether it is perceptible to people. I have found --nr 500 to be a good amount.

The problem with working out the quality is its purely, and can only be, personal preference! I realised SSIM and PSNR are completely inaccurate means of determining the output as its a comparison to the avisynth output, comparing it to other denoisers in terms of SSIM etc can't be done as they occur before the avisynth output.

In terms of bitrate mode, I made a comparison a while ago, and for lower bitrate I thought the --nr 500 output one looked better, but this is simply because more bitrate can be given to the encode in general and not spent on noise. I'm pretty sure the noise reduction works slightly differently to avisynth based noise reduction (DS may be able to clarify that statement).

I'd love to know others opinions on this. x264 nr does reduce encoding speed, the higher the nr amount the slower (but smaller the resultant file). It does work on a diminishing return basis, but still returns a bitrate reduction in CRF mode at higher NR nonetheless.

Again, SSIM is a pointless comparison for this, the comparison can be visually only :) I think this was covered at some point but can't find it on search. It was a while ago so new comparisons may be a good thing anyway!

[Ranguvar]

IMHO using a light AviSynth denoiser would be better -- I don't know exactly how x264's denoiser works, but unless a lot of overhead is eliminated by doing the denoising in the encoder, the sheer number of denoisers available for AviSynth and their configurability means that there's always something better for any particular job.

If you have a capable GPU, FFT3DGPU can do light or heavy denoising FAR faster than the competition :)

[LoRd_MuldeR]

I think the advantage of x264's built-in denoiser is that the motion vectors, which are calculated anyway, can be re-used for the denoising...

[burfadel]

The difference is I saw a noticeable bitrate difference without a noticeable picture difference, something that in avisynth you just don't see... Good point about the motion vectors though, if its the case they're not re-used it would be be good if it were changed so they were! The NR in x264 could probably be advantageous if its code were optimised too, I don't believe it has been optimised much since its inclusion. The main point was to see if other people found it useful, even on top of avisynth noise reduction, to the extent that it provides benefits without affecting the picture noticeably. If it did then maybe it could be an area of focus for improvement.

[akupenguin]

The difference is I saw a noticeable bitrate difference without a noticeable picture difference, something that in avisynth you just don't see...
x264's nr operates directly on the inter residual, and quantizes dct coefficients more strongly when it estimates that they're probably noise. So it directly reduces bitrate in a way that you don't get if you just remove noise any old way and then ask the encoder to encode the resulting image at full precision.
The reuse of mvs is not just a speed optimization, it's part of the reason the denoiser works. Same for the use of dct, and in particular the same dct as x264 will choose for the encoding of that block.
The NR in x264 could probably be advantageous if its code were optimised too, I don't believe it has been optimised much since its inclusion.
You mean besides the ssse3 asm?

[burfadel]

Ah ok! I wasn' taking in to account things that indirectly affect its speed! I knew it worked different to avisynth based noise reduction, and it does seem advantageous even if there is Avisynth NR! It doesn't seem a well used function though, so I was wondering how others found it. As I said before, its something you can really take the SSIM of because you are changing the output, avisynth based NR changes the input the x264. You can have a filter that completely changes the input, but still have an almost perfect SSIM because you are only comparing the output of avisynth (input of x264) to the output of x264 :) hope that makes sense!

[LoRd_MuldeR]

I wonder how NR and Psy-RDO work together. Don't they work into opposite directions somehow?

[Dark Shikari]

I wonder how NR and Psy-RDO work together. Don't they work into opposite directions somehow?NR isn't used in mode decision where psy-RD is used, so they have no real effect on each other.

[PelPix]

Working well so far on my processor

[burfadel]

Thats kind of what I'm curious about, the subjective results people have about using it. I think in the past people were simple comparing the SSIM, which you can't do with using this NR. Its not a replacement for avisynth NR, it can be used in conjunction with. Actually using lighter NR in avisynth and strong NR in x264 seems to be more beneficial in many cases bitrate and picture quality wise than being heavy handed with avisynth NR. I'd love to see others perspectives of this. Again it is a visual test, SSIM is invalid because yoy are removing noise on the x264 output which ultimately lowers its 'sameness' for SSIM calculation. In certain circumstances such as bitrate mode, I can imagine it could actuallly raise SSIM as you are removing noise, reducing the encoding complexity and therefore the bitrate can be spent on improving the output of actual picture detail! NR scales up to 1000, an NR of 2 or 3 would do almost nothing! I use 500.

I take it this works on only Luma, is it possible and would it be beneficial to also work on chroma?...

[Selur]

little question: Does trellis (in x264) require cabac?

[J_Darnley]

little question: Does trellis (in x264) require cabac?
x264 --help
-t, --trellis <integer> Trellis RD quantization. Requires CABAC.

[Selur]

okay, thanks. this is more a practical problem (as in nobody wrote it) isn't? it iirc trellis+cavlc is possible but a pain to implement, right?

[Dark Shikari]

Thats kind of what I'm curious about, the subjective results people have about using it. I think in the past people were simple comparing the SSIM, which you can't do with using this NR. Its not a replacement for avisynth NR, it can be used in conjunction with. Actually using lighter NR in avisynth and strong NR in x264 seems to be more beneficial in many cases bitrate and picture quality wise than being heavy handed with avisynth NR. I'd love to see others perspectives of this. Again it is a visual test, SSIM is invalid because yoy are removing noise on the x264 output which ultimately lowers its 'sameness' for SSIM calculation. In certain circumstances such as bitrate mode, I can imagine it could actuallly raise SSIM as you are removing noise, reducing the encoding complexity and therefore the bitrate can be spent on improving the output of actual picture detail! NR scales up to 1000, an NR of 2 or 3 would do almost nothing! I use 500.I have seen very significant artifacts in anime encoding with nr=1000. IMO high values can be risky on some types of content.I take it this works on only Luma, is it possible and would it be beneficial to also work on chroma?...Probably not, and yes, it only works on luma.okay, thanks. this is more a practical problem (as in nobody wrote it) isn't? it iirc trellis+cavlc is possible but a pain to implement, right?Yes, it's possible, and there are some papers implementing it. It would likely be harder than CABAC. Patches implementing it are quite welcome.

When I got QNS working a while back it showed a significant improvement on CAVLC (but was exponentially too slow to be practical), so I suspect CAVLC trellis would help similarly.

[burfadel]

I wouldn't select 1000 either! I have been using 500, I was curious about the consensus of other people. Of course, they could select 273 if they so wish, even 200 has benefit bitrate wise. I guess its about balance, afterall an aq of 1 is more benefiical than 0, but an aq of 2 may be the exact opposite. I think its the same with NR, is just about finding that 'AQ 1' spot (comparison to AQ was purely for example, I know its not related - but the reasoning is!).

[jult]

The order in mewiki is the same as in x264 --longhelp. It also looks pretty logical to meNot to anyone I know. Alphabetical would be smarter. Or at least an option to change the order that way.

The writers assume that people who use the command-line interface have a decent set of tools to view and search such listings: grep and less for instance. If you don't have them, just direct the output to a file and open it in some text editor.Resting my case: Takes a lot of extra unnecessary time. Isn't as easy or logical as it can be.

I don't think any amount of documentation on the command-line parameters would help with this issue.With what issue? Of course it will, and I have already given the reasons and pointers towards that. By the way, I've been writing about it for years. It seems to become just as bad as linux dev in general has become: Messy. Badly documented. Badly linked online. Assuming users like to do things as if they still live in 1980; Typing and typing only. We copy paste and (right-)click these days.
I have no problems whatsoever using commandline interfaces, but having to mess in CLI more than is needed is tedious, overly complex and time-consuming. Denying that is like saying we don't like a good GUI.

[nm]

Not to anyone I know. Alphabetical would be smarter. Or at least an option to change the order that way.
So you want an alphabetical reference for options that you already know by name? It doesn't take that much work to contribute an ordered list of links to the Wiki page. Still, it would be easier to just press CTRL-F and type in the phrase you want to find.

The writers assume that people who use the command-line interface have a decent set of tools to view and search such listings: grep and less for instance. If you don't have them, just direct the output to a file and open it in some text editor.Resting my case: Takes a lot of extra unnecessary time. Isn't as easy or logical as it can be.
That is the standard way -h/--help works in command-line interfaces. I don't see any reason to change that behaviour.

don't think any amount of documentation on the command-line parameters would help with this issue.With what issue? Of course it will, and I have already given the reasons and pointers towards that.
You complained about failing to use 32-bit AviSynth with 64-bit x264. I don't think a command-line reference is the right place to list external dependencies (such as 64-bit x264 builds requiring 64-bit AviSynth). So you would never find an answer from --longhelp or any alphabetical option reference. A traditional place for such information would be a README or INSTALL file in the x264 source code tree, but since most people wouldn't notice them, a Wiki-based user's manual would be in order.

By the way, I've been writing about it for years. It seems to become just as bad as linux dev in general has become: Messy. Badly documented. Badly linked online.
I think the main reason for that is the lack of a high-profile official site (x264.org or x264.com). The devs were working on that (http://forum.doom9.org/showthread.php?t=139634), but apparently that project hasn't moved forward lately.

Assuming users like to do things as if they still live in 1980; Typing and typing only. We copy paste and (right-)click these days.
I like typing and copy&paste works much better with command lines than GUIs. x264 is a command-line tool. It does not have an official GUI and it may never get one.

I have no problems whatsoever using commandline interfaces, but having to mess in CLI more than is needed is tedious, overly complex and time-consuming. Denying that is like saying we don't like a good GUI.
How about using one of the numerous nice GUIs people have created and contribute documentation and tooltip texts to it if they are missing. Once the frontend gets better, other people start using it and it will be maintained actively along with x264.

[burfadel]

I was wondering whether gradfun2db could essentially be added to x264? I know it can be done through avisynth but I was thinking that there may be some way to incorporate it in to x264 and use some results of the x264 functions?

I know people say its pointless doing it before encoding, as the encoding itself will produce banding, but I think the exact opposite. The last thing you want to do is encode chroma banding and small artifacts and waste bitrate doing it. Remember, if there is banding and other colour abberrations, x264 has to spend bitrate actually encoding the banding, and it does a good job of it with VAQ! My idea would be to have a chroma smoother that has edge detection on the input source (negating the need for gradfun2db), and also maybe include the chroma optimiser that was trialled, if it can be tweaked and shown to be beneficial (they may work together). The chroma smoother with edge detection in my opinion, should be enabled by default as the most common sources for video is already compressed video, or raw video derived from compressed video (I know the decoding part is mostly done through avisynth). Even if you have a chroma smoother, it wouldn't hurt having the proposed method as well as edge detection will negate chroma blurring where its not needed (such as transition of colours and 'significant' chroma variation), and yet provide a good effect. This would be more beneficial to visual quality than bitrate (I presume). Hope that makes sense, the effect would be at most subtle and yet visually pleasing :)

[gizzin]

I just wanted to say thanks for this amazing encoder.

[Sagekilla]

@burfadel: Nice idea but that goes beyond x264's job, since that's pre-processing. If you want to fix your source so it's not screwed up, do it in avisynth. x264's job isn't crap in -> good out, it's to compress your input and keep it as close to the input as possible.

[lexor]

@burfadel: Nice idea but that goes beyond x264's job, since that's pre-processing. If you want to fix your source so it's not screwed up, do it in avisynth. x264's job isn't crap in -> good out, it's to compress your input and keep it as close to the input as possible.

I think he was mostly aiming at preventing x264 from creating banding when bitrate starved (or whatever other reason it has). Though I don't think gradfun2db's algorithm is applicable in that capacity either.

[Sagekilla]

x264 already has something to prevent banding: AQ.

Note: If you're bit starved you should either downscale or increase your bitrate. Either way, there's very little that could be done to stop artifacting once you're bitrate starved.

[burfadel]

x264 already has something to prevent banding: AQ.

Note: If you're bit starved you should either downscale or increase your bitrate. Either way, there's very little that could be done to stop artifacting once you're bitrate starved.

AQ works very well too! The main thing I was pointing out is any distortions (such as banding) on the input source is seen by x264 as detail, and hence the very good AQ encodes it! My point was to just provide an option (maybe as default) for subtle chroma smoothing and optimisation. This was already partly covered with the chroma optimiser tested earlier as an avisynth plugin (from XVID?) but extends on the concept. The main reason why I suggested it is most people don't use a deband filter etc before x264.

Like the comment above, 'x264 already has something to prevent banding: AQ', that reduces banding due to x264, but there's the banding in the original source that x264 actually spend bitrate to encode, as its detail! <--AQ does NOT remove source banding, it would actually help to encode it! There would also be other instances where AQ etc would use bitrate on chroma information (non visible in general unlike banding) where it wouldn't otherwise be needed.

[Dark Shikari]

AQ works very well too! The main thing I was pointing out is any distortions (such as banding) on the input source is seen by x264 as detail, and hence the very good AQ encodes it! My point was to just provide an option (maybe as default) for subtle chroma smoothing and optimisation. This was already partly covered with the chroma optimiser tested earlier as an avisynth plugin (from XVID?) but extends on the concept. The main reason why I suggested it is most people don't use a deband filter etc before x264.

Like the comment above, 'x264 already has something to prevent banding: AQ', that reduces banding due to x264, but there's the banding in the original source that x264 actually spend bitrate to encode, as its detail! <--AQ does NOT remove source banding, it would actually help to encode it! There would also be other instances where AQ etc would use bitrate on chroma information (non visible in general unlike banding) where it wouldn't otherwise be needed."Chroma smoothing" and "banding" have nothing to do with each other.

Banding is primarily a luma problem, not chroma. And smoothing creates banding, you need to dither to get rid of banding.

[Sagekilla]

Plus, what you're describing is nothing more than preprocessing. As I said before, x264's job is not preprocessing. That's for avisynth.

[burfadel]

Ah ok! that makes sense about the banding, I apologise for my ignorance! (hey, I saved someone else looking stupid) :D

[b66pak]

hi,

it is possible to output the x264 final statistic to a file? (.log)

thanks for this great tool...
_

[LoRd_MuldeR]

hi,

it is possible to output the x264 final statistic to a file? (.log)

thanks for this great tool...
_

x264.exe options > logfile.txt

And of course you can write your own front-end, capture STDOUT/STDERR and extract whatever information you need...

[kemuri-_9]

x264.exe options > logfile.txt

And of course you can write your own front-end, capture STDOUT/STDERR and extract whatever information you need...

as x264 outputs to stderr and not stdout the above would be 2> and not >

also there's a patch flying around from komisar that adds in a log file capability iirc.
though that would require building your own version with it....

[b66pak]

i tried this of course...it generate an empty file...check the dos output!!!

my line:
x264.exe "hfyu_VTS_03_1-001_Track1.avi" --bitrate 1000 --keyint 25 --level 3
--min-keyint 1 --deblock 1:1 --no-cabac --psy-rd 0.6:0 --partitions p8x8,b8x8,i4x4
--vbv-bufsize 10000 --vbv-maxrate 10000 --threads auto --thread-input --aq-mode 0
--sar 1:1 --aud --progress --no-psnr --no-ssim --deldup minfps 1
--timecode "hfyu_VTS_03_1-001_Track1.avi.ABR.VFR.mkv_timecodes_v2.txt"
--output "hfyu_VTS_03_1-001_Track1.avi.ABR.VFR.mkv"
>"hfyu_VTS_03_1-001_Track1.avi.ABR.VFR.mkv_log.log"


dos window:
F:\Tools_Video\direct264_20090320>x264.exe "hfyu_VTS_03_1-001_Track1.avi" --bitr
ate 1000 --keyint 25 --level 3 --min-keyint 1 --deblock 1:1 --no-cabac --psy-rd
0.6:0 --partitions p8x8,b8x8,i4x4 --vbv-bufsize 10000 --vbv-maxrate 10000 --thre
ads auto --thread-input --aq-mode 0 --sar 1:1 --aud --progress --no-psnr --no-ss
im --deldup minfps 1 --timecode "hfyu_VTS_03_1-001_Track1.avi.ABR.VFR.mkv_timeco
des_v2.txt" --output "hfyu_VTS_03_1-001_Track1.avi.ABR.VFR.mkv" 1>"hfyu_VTS_03
_1-001_Track1.avi.ABR.VFR.mkv_log.log"
x264 [info]: using SAR=1/1
x264 [info]: using cpu capabilities: MMX2 SSE2Slow Slow_mod4_stack
x264 [info]: profile Baseline, level 3.0
x264 [info]: slice I:24 Avg QP:25.08 size: 31682
x264 [info]: slice P:576 Avg QP:27.55 size: 3036
x264 [info]: mb I I16..4: 13.2% 0.0% 86.8%
x264 [info]: mb P I16..4: 1.5% 0.0% 2.0% P16..4: 41.8% 13.7% 1.8% 0.0% 0
.0% skip:39.2%
x264 [info]: final ratefactor: 23.95
x264 [info]: kb/s:1002.9

encoded 600 frames, 22.78 fps, 1006.30 kb/s
_

[LoRd_MuldeR]

kemuri-_9 is right. That also explains why your file was empty.

BTW: A console window on a WinNT-based operating system is not related to DOS at all ;)

[b66pak]

thanks a lot...
_

[Hyrax]

Hello- I'm having problems with x264 crashing lately in Windows XP... it either dies silently or comes up with a memory exception. The problem seems to come mainly from squeezing very large HD videos (28 - 45 GB) to 8 GB. I'm going to do some memtests and other things to confirm it isn't a hardware related issue, but I'm relatively sure it isn't. What else can I do to pinpoint the problem?

Edit: I'm real new to using x264 and will appreciate any guidance. Thanks.

[Sagekilla]

It could be possible youre running out of memory. How much memory do you have and what's your script look like?

[kemuri-_9]

yes it does sound like you're running out of memory...
are you running the x64 or x86 version?
if the x86 one, you should answer Sagekilla's questions from above.

[Hyrax]

I'm using the x86 version and have 3 GB.
I'm doing a 2 pass encode. The first pass works fine. Here's the script for the 2nd pass (I added cr/lf to make it easier for you to read):
"C:\Program Files\x264\x264.exe" "C:\temp\Video\video1.avs" --pass 2 --bitrate 7623
--stats "C:\temp\Video\video1.stats" --sar 1:1 --level 4.0
--aud --nal-hrd --vbv-bufsize 25000 --vbv-maxrate 25000 --filter 0,0
--ref 3 --mixed-refs --bframes 3 --b-adapt 1 --weightb
--direct auto --subme 7 --trellis 1 --partitions all --8x8dct --me umh
--threads auto --thread-input --progress --no-psnr --no-ssim --output "C:\temp\video\video1.264"

I'll run it again and monitor memory. Thanks for the suggestion.

edit: as a side note, it takes approximately 2 hours for the first pass to complete nad it produces a 7 GB file from a 28 GB source.

[Sagekilla]

Please post your avisynth script too. We need to see the whole chain to see if anything is memory starved.

[Dark Shikari]

x264 could not possibly crash from a malloc failure; all mallocs are checked, and no memory allocation happens outside of the start of the program.

Of course, a decoder or Avisynth could be the culprit as well.

You're not going to get anywhere without some gdb.

[MasterNobody]

x264 could not possibly crash from a malloc failure; all mallocs are checked
Probably it would crash before this check because of:
void *x264_malloc( int i_size )
{
#ifdef SYS_MACOSX
/* Mac OS X always returns 16 bytes aligned memory */
return malloc( i_size );
#elif defined( HAVE_MALLOC_H )
return memalign( 16, i_size );
#else
uint8_t * buf;
uint8_t * align_buf;
buf = (uint8_t *) malloc( i_size + 15 + sizeof( void ** ) +
sizeof( int ) );
align_buf = buf + 15 + sizeof( void ** ) + sizeof( int );
align_buf -= (intptr_t) align_buf & 15;
*( (void **) ( align_buf - sizeof( void ** ) ) ) = buf;
*( (int *) ( align_buf - sizeof( void ** ) - sizeof( int ) ) ) = i_size;
return align_buf;
#endif
}

[Dark Shikari]

Probably it would crash before this check because of:Oh indeed, that, the memalign emulation hack for crappy systems like MinGW.

[kemuri-_9]

Oh indeed, that, the memalign emulation hack for crappy systems like MinGW.

heh, i still personally use my patch to utilize mingw's aligned memory methods, so i never see that emulation.

but it still would be wise to fix the issue....

[Hyrax]

Thanks for all your help. It is greatly appreciated.

Here's my Avisynth script:

#Video
video=DirectShowSource("C:\temp\video\video1.mkv",audio=false).ConvertToYV12()

#Audio
LoadPlugin("C:\Program Files\AviSynth plugins\NicAudio\NicAudio.dll")
audio=NicAC3Source("C:\temp\video1\audio1.ac3")
audio=ResampleAudio(audio,48000)



And, so you don't need to look it up, here's a copy of my x264
"C:\Program Files\x264\x264.exe" "C:\temp\Video\video1.avs" --pass 2 --bitrate 7623
--stats "C:\temp\Video\video1.stats" --sar 1:1 --level 4.0
--aud --nal-hrd --vbv-bufsize 25000 --vbv-maxrate 25000 --filter 0,0
--ref 3 --mixed-refs --bframes 3 --b-adapt 1 --weightb
--direct auto --subme 7 --trellis 1 --partitions all --8x8dct --me umh
--threads auto --thread-input --progress --no-psnr --no-ssim --output "C:\temp\video\video1.264"


I don't think I'm running out of memory; while watching usasge in Task Manager, x264 seems to be staying at ~820,000K and physical memory remains at ~1,690,000K.


CPU usage is high (90% or more), but temperatures for all four physical cores remain good at 65 degrees. I've got an i7 CPU, so it looks like I'm running 8 cores - is it possible that x264 doesn't like hyper-threading?

In the back of my mind I have a (unreasonable?) fear that Intel's new memory controller could be the trouble here. I'm going to try replacing the memory just in casae

I've got an older, slower computer that I use as a server. Just as a sanity check I'll try to see if I can reproduce this problem on that system, but the encoding will take about 24 hours.

[Hyrax]

x264 is now working for me and I may never know what the exact problem was. On a whim I decided to uninstall my ATI video drivers and then install their latest release. I tried to re-encode immediately after starting my computer and it worked. Usually I have my computer on for hours - watching and editing videos - before I start encoding. So I don't know if something gets left in memory that causes problems, or if the video drivers I was using were somehow interfering, or if I've got a bad memory chip that occasionally causes problems.

Whatever was wrong earlier, it does seem apparent that x264 is not causing the problem.

Thanks again for your assistance.

[LoRd_MuldeR]

Video drivers can't affect x264, as x264 doesn't use the GPU in any way (yet). Of course a borked driver may crash the entire system, but that's a different story.

Also it's impossible that "something left in memory" causes the problem, because all modern operating systems are running in Protected Mode (http://en.wikipedia.org/wiki/Protected_mode) ;)

But an overheating CPU or a borked memory module may cause problems indeed! I highly recommend that you check your system with Memtest86+ (http://www.memtest.org/#downiso) as well as Prime95 (http://www.freewarefiles.com/Prime95_program_19638.html).

[laserfan]

x264.exe options > logfile.txt...

as x264 outputs to stderr and not stdout the above would be 2> and not >....Thanks fellas, something else I didn't know (and I don't think this is in the --longhelp either). I suppose most .exes will do this?

Anyway, I've been PAUSEing my .cmd files then copy/pasting to a textfile. This will save me some fiddling! :)

[LoRd_MuldeR]

Thanks fellas, something else I didn't know (and I don't think this is in the --longhelp either). I suppose most .exes will do this?

It's not documented for x264, because it has nothing to do with x264 itself!

The ">" operator will redirect STDOUT to a file, just like the "2>" operator will redirect STDERR to a file, the "<" operator will feed a file into STDIN and the "|" operator will connect two applications.

These are shell operators and they work with any application that uses standard handles...

[laserfan]

The ">" operator will redirect STDOUT to a file, just like the "2>" operator will redirect STDERR to a file, the "<" operator will feed a file into STDIN and the "|" operator will connect two applications.

These are shell operators and they work with any application that uses standard handles...I guess since I'm being stupid I might ask why x264 outputs "info" (not errors) to STDERR. but doesn't to STDOUT, but no doubt I've already pushed my luck in this advanced thread! Thanks LM! :o ;)

[Dark Shikari]

I guess since I'm being stupid I might ask why x264 outputs "info" (not errors) to STDERR. but doesn't to STDOUT, but no doubt I've already pushed my luck in this advanced thread! Thanks LM! :o ;)All sane programs output info to stderr. If they outputted info to stdout, it would make it impossible to actually output data to stdout, because data and info would end up mixed.

[akupenguin]

All sane programs output info to stderr. If they outputted info to stdout, it would make it impossible to actually output data to stdout, because data and info would end up mixed.
That is indeed why x264 uses stderr. Though x264cli has no option to output video to stdout, it only applies with ffmpeg/mencoder/etc.

[laserfan]

Ouch. I should have figured that. Thanks for your indulgences. :)

[kemuri-_9]

That is indeed why x264 uses stderr. Though x264cli has no option to output video to stdout, it only applies with ffmpeg/mencoder/etc.

you must have forgotten that x264cli does output video to stdout with -o - (or --output -) as in

$ x264 foreman_cif_352x288.yuv -o - --crf 18 --quiet 1>video.h264

encoded 300 frames, 88.47 fps, 988.47 kb/s

$ mp4box -add video.h264 -new video.mp4
AVC-H264 import - frame size 352 x 288 at 25.000 FPS
Import results: 300 samples - Slices: 2 I 298 P 0 B - 1 SEI - 2 IDR
Saving video.mp4: 0.500 secs Interleaving

[vmrsss]

x264's nr operates directly on the inter residual, and quantizes dct coefficients more strongly when it estimates that they're probably noise. So it directly reduces bitrate in a way that you don't get if you just remove noise any old way and then ask the encoder to encode the resulting image at full precision.
The reuse of mvs is not just a speed optimization, it's part of the reason the denoiser works. Same for the use of dct, and in particular the same dct as x264 will choose for the encoding of that block.

so how would that compete/compare with avisynth filters? I was surprised to read your post, because --nr isn't something that is recommended nor publicized, I always had the impression that was because it is of pretty low quality. (I'm having a hard time to find a nice degrainer/noiser which does more than 2fps and doesn't completely blur the picture.)

[vmrsss]

Its not a replacement for avisynth NR, it can be used in conjunction with. Actually using lighter NR in avisynth and strong NR in x264 seems to be more beneficial in many cases bitrate and picture quality wise than being heavy handed with avisynth NR.

why would you use a light denoiser in avisynth in addition to a strong --nr? the former is dead slow, the latter is optimised, don't they perform the same task? I would be very happy to discover that --nr can serve my purpose of removing background noise from DVB captures...

[burfadel]

I find the benefit of the --nr is more picture optimisation than physical noise reduction, although there would have to be some noise reduction! I've tested it several times with different avisynth denoisers, its more bandwidth friendly, or seems that way, to use an --nr of say around 500 (seems to be a good value for the sources I tested), and lower avisynth filtering strength to keep the encoded size down whilst keeping really good picture quality and structure rather than the perceived benefits of scrubbing out detail using avisynth noise reduction and having roughly the same end bitrate used and a smeary picture without it.

The main thing is, give it a try with and without, inspect the picture quality and file size (say using CRF) as you go. The scale of NR is from 0 to 1000, try at least 300! Don't look too much in to SSIM etc, as NR is an internal x264 function, and NR is a function that removes noise, the SSIM of the output will naturally generally be lower as its not identical to the avisynth input! If you used an avisynth filter, you can smear and brutalise the picture as much as you want, but since x264 acts on the output from avisynth (thats the typical use so thats why I'm using that for the example), the SSIM can almost be perfect from input to output, as SSIM is a measure of x264 input vs output. The picture would still look botched though. So check it with your eyes, not by an input/output SSIM comparison, and choose which one you prefer :)

[akupenguin]

so how would that compete/compare with avisynth filters? I was surprised to read your post, because --nr isn't something that is recommended nor publicized, I always had the impression that was because it is of pretty low quality.
I didn't say nr was particularly good at removing noise without harming content, I just said it's better at reducing bitrate than an external filter of the same strength. If you're encoding at a high enough bitrate that the damage done by inaccurate denoising is worse than the damage done by extra quantization, then pick an avisynth filter that's optimized for quality rather than compression. And of course there's room for compromise too.

IOW, if you have less precise knowledge about some features of your input than others (because some are covered with noise), then you should quantize the former more than the latter, so as to maximize the similarity between the encoded stream and the unknown original input within any given bit budget. But that's no substitute for your best attempt at removing the noise too.

The scale of NR is from 0 to 1000
The scale of nr is 0 to 65536. 1000 is just a reasonable guess at what you might want to try so as not to completely destroy the content.

[Blue_MiSfit]

Correct me if I'm wrong, but it seems like NR has some pretty significant promise at reducing bitrate!

If it's using the motion vectors that x264's ME generates, it must have a lot of potential.

I wonder if something cooler like MVDegrain could be ported in??? Just a thought, I'm no coder :p

I might have to try this out on some uber low bitrate stuff (500kbps CBR high complexity 480p)

~MiSfit

[b66pak]

how can be properly decoded the new x264 lossless mode?

the problem is described here:

http://forum.doom9.org/showthread.php?p=1283664#post1283664
_

[Dark Shikari]

how can be properly decoded the new x264 lossless mode?

the problem is described here:

http://forum.doom9.org/showthread.php?p=1283664#post1283664
_By using a libavcodec or CoreAVC that isn't a year old.

[Sharktooth]

how can be properly decoded the new x264 lossless mode?

the problem is described here:

http://forum.doom9.org/showthread.php?p=1283664#post1283664
_
you already got your answer in the other thread.

[Clumpco]

Build 1153 crashing on decoder start under MeGui.
"Null exception"

[Trahald]

Megui is using an experimental patched 1153 which is causing the issue. use techhouses version for now

[Clumpco]

Megui is using an experimental patched 1153 which is causing the issue. use techhouses version for now

Ta mate! Rolled back to Skystrife 1148 and it worked fine again

[LoRd_MuldeR]

It appears that x264 r1157 crashes when using the ESA me-method:

$ gdb --args x264 --crf 22 --output c:\out.mkv --progress --me esa ../soccer.avs
GNU gdb 6.8
Copyright (C) 2008 Free Software Foundation, Inc.
License GPLv3+: GNU GPL version 3 or later <http://gnu.org/licenses/gpl.html>
This is free software: you are free to change and redistribute it.
There is NO WARRANTY, to the extent permitted by law. Type "show copying"
and "show warranty" for details.
This GDB was configured as "i686-pc-mingw32"...
(gdb) run
Starting program: c:\temp\x264\vanilla/x264.exe --crf 22 --output c:out.mkv --progress --me esa ../soccer.avs
[New thread 4712.0x13f8]
[...] // Skipping Avisynth debug messages
x264 [info]: using cpu capabilities: MMX2 SSE2Fast SSSE3 Cache64
x264 [info]: profile Main, level 3.0
[0.3%] 1/299 frames, 16.13 fps, 19572.20 kb/s, eta 0:00:18
Program received signal SIGSEGV, Segmentation fault.
0x00457287 in x264_pixel_ads4_ssse3 ()
(gdb) bt
#0 0x00457287 in x264_pixel_ads4_ssse3 ()
#1 0x0279de52 in ?? ()
#2 0x00030000 in ?? ()
#3 0x50000161 in ?? ()
#4 0x7d62bc35 in wcstoui64 () from C:\WINDOWS\SysWOW64\ntdll32.dll
#5 0x00000000 in ?? ()
(gdb)

This is with an unpatched debug build, using MinGW GCC 4.4.0. No crash when using "--me umh" instead.

[Dark Shikari]

Fixed in r1158.

[LoRd_MuldeR]

Fixed in r1158.

:thanks:

[simps]

I couldn't find the answer for this question, searching doom9.

Talking about 2passes encode here.

On old versions of x264, when a --vbv-maxrate was specified, but no --vbv-bufsize (leaving it at default "0" value), the encoder used to pop a warning like this:

"x264 [warning]: VBV maxrate specified, but no bufsize."

I was testing the current version, and when I enter a --vbv-maxrate, without any --vbv-bufsize, the encoder will proceed without any warnings.

Does this mean, x264 is now just ignoring --vbv-maxrate when no --vbv-bufsize is set and not showing any warning anymore, or is --vbv-maxrate actually working the right way without any --vbv-bufsize set? Or something different is happening?

Thanks,
Simps

[Dark Shikari]

I couldn't find the answer for this searching doom9.

Talking about 2passes encode here.

On old versions of x264, when a --vbv-maxrate was specified, but no --vbv-bufsize (leaving it at default "0" value), the encoder used to pop a warning like this:

"x264 [warning]: VBV maxrate specified, but no bufsize."

I was testing the current version, and when I enter a --vbv-maxrate, without any --vbv-bufsize, the encoder will proceed without any warnings.

Does this mean, x264 is now just ignoring --vbv-maxrate when no --vbv-bufsize is set, or is --vbv-maxrate actually working the right way without any --vbv-bufsize? Or something different is happening?

Thanks,
Simps if( h->param.rc.i_vbv_max_bitrate < h->param.rc.i_bitrate &&
h->param.rc.i_vbv_max_bitrate > 0)
x264_log(h, X264_LOG_WARNING, "max bitrate less than average bitrate, ignored.\n");
else if( h->param.rc.i_vbv_max_bitrate > 0 &&
h->param.rc.i_vbv_buffer_size > 0 )
{
if( h->param.rc.i_vbv_buffer_size < 3 * h->param.rc.i_vbv_max_bitrate / rc->fps )
{
h->param.rc.i_vbv_buffer_size = 3 * h->param.rc.i_vbv_max_bitrate / rc->fps;
x264_log( h, X264_LOG_WARNING, "VBV buffer size too small, using %d kbit\n",
h->param.rc.i_vbv_buffer_size );
}

...

}
else if( h->param.rc.i_vbv_max_bitrate )
{
x264_log(h, X264_LOG_WARNING, "VBV maxrate specified, but no bufsize.\n");
h->param.rc.i_vbv_max_bitrate = 0;
}

[simps]

DS, I am still confused. I set --bitrate 240 --vbv-maxrate 1200 and --vbv-bufsize 0 (I just left it default), and x264 proceded without warning.



From the code you posted:



1st part (not my case, I don't have --vbv-maxrate < --bitrate)

if( h->param.rc.i_vbv_max_bitrate < h->param.rc.i_bitrate &&
h->param.rc.i_vbv_max_bitrate > 0)
x264_log(h, X264_LOG_WARNING, "max bitrate less than average bitrate, ignored.\n");



2nd part (not my case, I don't have --vbv-bufsize > 0)

else if( h->param.rc.i_vbv_max_bitrate > 0 &&
h->param.rc.i_vbv_buffer_size > 0 )
{
if( h->param.rc.i_vbv_buffer_size < 3 * h->param.rc.i_vbv_max_bitrate / rc->fps )
{
h->param.rc.i_vbv_buffer_size = 3 * h->param.rc.i_vbv_max_bitrate / rc->fps;
x264_log( h, X264_LOG_WARNING, "VBV buffer size too small, using %d kbit\n",
h->param.rc.i_vbv_buffer_size );
}



3rd part (This should be my case, but why I don't see any warnings? If there is no warning, how do I know that routine is going on, and setting -vbv-maxrate to 0 in this case???)


else if( h->param.rc.i_vbv_max_bitrate )
{
x264_log(h, X264_LOG_WARNING, "VBV maxrate specified, but no bufsize.\n");
h->param.rc.i_vbv_max_bitrate = 0; }




I got no warning:
http://img33.imageshack.us/img33/515/vbv.jpg

Thanks,
Simps

[Dark Shikari]

DS, I am still confused. I set --bitrate 240 --vbv-maxrate 1500 and --vbv-bufsize 0 (I just left it default), and x264 proceded without warning.Bug introduced by aku's level autodetection code. Fix will be incoming soon. Thanks for the report.

[simps]

Bug introduced by aku's level autodetection code. Fix will be incoming soon. Thanks for the report.

No problem, btw, it is probably setting --vbv-maxrate to 0 or ignoring it, just not showing the warning.
I set --vbv-maxrate 1200, but I got up to 1700kb/s peak as per pic below:

http://img194.imageshack.us/img194/8930/vbv2.jpg

[b66pak]

i encoded a movie using "x264 core 67 r1159 3da3f95":


General
Complete name : movie.mp4
Format : MPEG-4
Format profile : Base Media
Codec ID : isom
File size : 3.98 GiB
Duration : 2h 9mn
Overall bit rate : 4 404 Kbps
Encoded date : UTC 2009-05-25 06:54:24
Tagged date : UTC 2009-05-25 06:54:24

Video
ID : 1
Format : AVC
Format/Info : Advanced Video Codec
Format profile : High@L4.1
Format settings, CABAC : Yes
Format settings, ReFrames : 3 frames
Codec ID : avc1
Codec ID/Info : Advanced Video Coding
Duration : 2h 9mn
Bit rate mode : Variable
Bit rate : 4 211 Kbps
Maximum bit rate : 11.2 Mbps
Width : 1 280 pixels
Height : 532 pixels
Display aspect ratio : 2.406
Frame rate mode : Constant
Frame rate : 23.976 fps
Resolution : 24 bits
Colorimetry : 4:2:0
Scan type : Progressive
Bits/(Pixel*Frame) : 0.258
Stream size : 3.80 GiB (96%)
Writing library : x264 core 67 r1159 3da3f95
Encoding settings : cabac=1 / ref=3 / deblock=1:0:0 / analyse=0x3:0x113 / me=hex / subme=5 / psy_rd=0.0:0.0 / mixed_ref=1 / me_range=16 / chroma_me=1 / trellis=0 / 8x8dct=1 / cqm=0 / deadzone=21,11 / chroma_qp_offset=0 / threads=1 / nr=0 / decimate=1 / mbaff=0 / bframes=3 / b_pyramid=0 / b_adapt=2 / b_bias=0 / direct=3 / wpredb=1 / keyint=24 / keyint_min=1 / scenecut=40 / rc=2pass / bitrate=4211 / ratetol=1.0 / qcomp=0.50 / qpmin=10 / qpmax=51 / qpstep=4 / cplxblur=20.0 / qblur=0.5 / vbv_maxrate=30000 / vbv_bufsize=30000 / ip_ratio=1.10 / pb_ratio=1.10 / aq=1:1.00
Encoded date : UTC 2009-05-25 06:54:24
Tagged date : UTC 2009-05-25 06:59:53

Audio
ID : 2
Format : AAC
Format/Info : Advanced Audio Codec
Format version : Version 4
Format profile : LC
Format settings, SBR : No
Codec ID : 40
Duration : 2h 9mn
Bit rate mode : Variable
Bit rate : 192 Kbps
Maximum bit rate : 233 Kbps
Channel(s) : 2 channels
Channel positions : L R
Sampling rate : 48.0 KHz
Resolution : 16 bits
Stream size : 175 MiB (4%)
Encoded date : UTC 2009-05-25 06:59:41
Tagged date : UTC 2009-05-25 06:59:53


here is the log:


[Information] Log
-[Information] Versions
--[NoImage] MeGUI Version : 0.3.1.1037
--[NoImage] OS : Windows XP Professional x86 SP3 (5.1.196608.2600)
--[NoImage] Framework used : 2.0 SP1 (2.0.50727.1433)
-[Information] Hardware
--[NoImage] CPU : AMD Athlon(tm) 64 Processor 3800+
-[Information] Log for job1 (video, movie.avs -> )
--[Information] [5/25/2009 9:09:14 PM] Started handling job
--[Information] [5/25/2009 9:09:14 PM] Preprocessing
---[Information] Bitrate calculation for video
----[NoImage] Desired size after subtracting audio: 3987101KBs
----[NoImage] Calculated desired bitrate: 4211kbit/s
--[NoImage] Job commandline: "C:\Program Files\megui\tools\x264\x264.exe" --pass 1 --bitrate 4211 --stats "movie.stats" --level 4.1 --keyint 24 --min-keyint 1 --bframes 3 --b-adapt 2 --weightb --direct auto --subme 2 --partitions none --ipratio 1.1 --pbratio 1.1 --vbv-bufsize 30000 --vbv-maxrate 30000 --qcomp 0.5 --me dia --threads auto --thread-input --sar 1:1 --progress --no-psnr --no-ssim --output NUL "movie.avs"
--[Information] [5/25/2009 9:09:19 PM] Encoding started
--[NoImage] Standard output stream
--[NoImage] Standard error stream
---[NoImage] avis [info]: 1280x532 @ 23.98 fps (185957 frames)
---[NoImage] x264 [warning]: width or height not divisible by 16 (1280x532), compression will suffer.
---[NoImage] x264 [info]: using SAR=1/1
---[NoImage] x264 [info]: using cpu capabilities: MMX2 SSE2Slow
---[NoImage] x264 [info]: profile Main, level 4.1
---[NoImage]
---[NoImage] x264 [info]: slice I:14188 Avg QP:19.24 size: 49411
---[NoImage] x264 [info]: slice P:81071 Avg QP:18.35 size: 27061
---[NoImage] x264 [info]: slice B:90698 Avg QP:19.37 size: 12875
---[NoImage] x264 [info]: consecutive B-frames: 12.4% 39.4% 36.3% 11.9%
---[NoImage] x264 [info]: mb I I16..4: 39.6% 0.0% 60.4%
---[NoImage] x264 [info]: mb P I16..4: 34.9% 0.0% 0.0% P16..4: 58.0% 0.0% 0.0% 0.0% 0.0% skip: 7.1%
---[NoImage] x264 [info]: mb B I16..4: 11.8% 0.0% 0.0% B16..8: 31.7% 0.0% 0.0% direct:22.6% skip:33.9% L0:31.7% L1:45.5% BI:22.8%
---[NoImage] x264 [info]: final ratefactor: 19.71
---[NoImage] x264 [info]: direct mvs spatial:99.7% temporal:0.3%
---[NoImage] x264 [info]: coded y,uvDC,uvAC intra:79.3% 64.2% 29.7% inter:33.2% 31.6% 1.8%
---[NoImage] x264 [info]: kb/s:4190.4
---[NoImage] encoded 185957 frames, 10.14 fps, 4190.94 kb/s
--[Information] [5/26/2009 2:14:50 AM] Postprocessing
--[Information] [5/26/2009 2:14:50 AM] Job completed
-[Information] Log for job2 (video, movie.avs -> movie.264)
--[Information] [5/26/2009 2:14:50 AM] Started handling job
--[Information] [5/26/2009 2:14:50 AM] Preprocessing
--[NoImage] Job commandline: "C:\Program Files\megui\tools\x264\x264.exe" --pass 2 --bitrate 4211 --stats "movie.stats" --level 4.1 --keyint 24 --min-keyint 1 --ref 3 --mixed-refs --no-fast-pskip --bframes 3 --b-adapt 2 --weightb --direct auto --subme 5 --partitions p8x8,b8x8,i4x4,i8x8 --8x8dct --ipratio 1.1 --pbratio 1.1 --vbv-bufsize 30000 --vbv-maxrate 30000 --qcomp 0.5 --threads auto --thread-input --sar 1:1 --aud --progress --no-psnr --no-ssim --output "movie.264" "movie.avs"
--[Information] [5/26/2009 2:14:51 AM] Encoding started
--[NoImage] Standard output stream
--[NoImage] Standard error stream
---[NoImage] avis [info]: 1280x532 @ 23.98 fps (185957 frames)
---[NoImage] x264 [warning]: width or height not divisible by 16 (1280x532), compression will suffer.
---[NoImage] x264 [info]: using SAR=1/1
---[NoImage] x264 [info]: using cpu capabilities: MMX2 SSE2Slow
---[NoImage] x264 [info]: profile High, level 4.1
---[NoImage]
---[NoImage] x264 [info]: slice I:14188 Avg QP:18.98 size: 49148
---[NoImage] x264 [info]: slice P:81071 Avg QP:18.11 size: 26511
---[NoImage] x264 [info]: slice B:90698 Avg QP:18.80 size: 13627
---[NoImage] x264 [info]: consecutive B-frames: 12.4% 39.4% 36.3% 11.9%
---[NoImage] x264 [info]: mb I I16..4: 14.6% 54.5% 31.0%
---[NoImage] x264 [info]: mb P I16..4: 8.8% 24.9% 6.4% P16..4: 31.8% 17.6% 5.3% 0.0% 0.0% skip: 5.2%
---[NoImage] x264 [info]: mb B I16..4: 10.4% 0.0% 0.0% B16..8: 32.1% 4.0% 2.9% direct:16.4% skip:34.1% L0:39.4% L1:42.7% BI:17.9%
---[NoImage] x264 [info]: 8x8 transform intra:49.7% inter:39.8%
---[NoImage] x264 [info]: direct mvs spatial:95.7% temporal:4.3%
---[NoImage] x264 [info]: coded y,uvDC,uvAC intra:83.5% 67.4% 30.4% inter:29.4% 27.8% 1.3%
---[NoImage] x264 [info]: ref P L0 78.5% 14.2% 7.3%
---[NoImage] x264 [info]: ref B L0 83.0% 17.0%
---[NoImage] x264 [info]: kb/s:4211.0
---[NoImage] encoded 185957 frames, 6.75 fps, 4212.74 kb/s
--[Information] Final statistics
---[NoImage] Video Bitrate Desired: 4211 kbit/s
---[NoImage] Video Bitrate Obtained (approximate): 4212 kbit/s
--[Information] [5/26/2009 9:54:22 AM] Postprocessing
--[Information] [5/26/2009 9:54:22 AM] Job completed
-[Information] Log for job3 (mux, movie.264 -> movie-muxed.mp4)
--[Information] [5/26/2009 9:54:22 AM] Started handling job
--[Information] [5/26/2009 9:54:22 AM] Preprocessing
--[NoImage] Job commandline: "C:\Program Files\megui\tools\mp4box\mp4box.exe" -add "movie.264:fps=23.976" -add "audio.m4a#trackID=1" -tmp "F:\Movie" -new "movie.mp4"
--[Information] [5/26/2009 9:54:23 AM] Encoding started
--[NoImage] Standard output stream
---[NoImage] AVC-H264 import - frame size 1280 x 532 at 23.976 FPS
---[NoImage] Import results: 185957 samples - Slices: 14188 I 81071 P 90698 B - 1 SEI - 14188 IDR
---[NoImage] IsoMedia import - track ID 1 - Audio (SR 48000 - 2 channels)
---[NoImage] Saving movie-muxed.mp4: 0.500 secs Interleaving
--[NoImage] Standard error stream
--[Information] [5/26/2009 10:04:34 AM] Postprocessing
--[Information] [5/26/2009 10:04:34 AM] Job completed
-[Information] [5/26/2009 10:05:07 AM] Shutdown initiated


since now i never noticed that the I frames have lover QP that P frames...is this normal?
_

[Audionut]

since now i never noticed that the I frames have lover QP that P frames...is this normal?
_

ip_ratio=1.10 / pb_ratio=1.10

It is with those silly settings. Use defaults

ip_ratio=1.40 / pb_ratio=1.30

[juGGaKNot]

It is with those silly settings. Use defaults

ip_ratio=1.40 / pb_ratio=1.30

1.1 is ps3/xbox profile default in megui.

[Audionut]

megui often uses stupid settings in it's profiles.

[plato79]

will there be any OpenCL support in the future?

[J_Darnley]

"Patches welcome"

[Fr4nz]

will there be any OpenCL support in the future?

Very very unlikely: it would require a LOT of work. On top of all, from what I understand, it would require a complete redesign of the approach, regarding mainly the parallelization, in writing various algorithms which x264 currently uses.

[Sharktooth]

megui often uses stupid settings in it's profiles.
that choice was already been discussed. and FYI it's normal I frames have lower QP than other frames... even more at default settings. so please dont make silly comments.

[Audionut]

and FYI it's normal I frames have lower QP than other frames.

Yes!! But read more closely and you'll see,

---[NoImage] x264 [info]: slice I:14188 Avg QP:19.24 size: 49411
---[NoImage] x264 [info]: slice P:81071 Avg QP:18.35 size: 27061


Caused by stupid ip/pb ratio settings.

so please dont make silly comments.

Thanks, but open your eyes.

[Sharktooth]

it's an average it doesnt mean ALL I-Frames have "higher" QP than the corresponding P-Frames. you arent considering there could be consecutive I-Frames too due to flashes or very high motion, I-frame only GOPs, etc... and btw, as i said, that choice was already discussed and still, i dont think it's bad for certain targets. the only adjustment needed it's probably the qcomp parameter since the presets were made well before some critical x264 changes (AQ, psy-rd, etc...). so id say to set qcomp to default and if there is still some visible i-frame pumping effect.

[akupenguin]

btw, as i said, that choice was already discussed and still, i dont think it's bad for certain targets.
Discuss it again, for me. What is it about certain targets that could make their optimal QP offsets different?

[Sharktooth]

to make it short, on some big TVs some users reported a visible difference between b/p frames and i frames with default i/p p/b ratio.
those TVs were maybe not properly calibrated too, or just fancy TV "features" were producing some strange behaviours, but the solution was easy since i assumed x264 was far more efficient than any other h.264 encoder, so i ended up reducing the quant. ratio between frametypes in "default" blu-ray and ps3/xbox360 presets (megui) and everything went good.

[hajj_3]

builds since v1148 dont seem to have the correct version number tag for mediainfo to display, they all say: "x264 core 67 r1148M 400740b" instead of v1173 etc. Hope you guys can fix this in the next version.

[LoRd_MuldeR]

builds since v1148 dont seem to have the correct version number tag for media info to display, they all say: "x264 core 67 r1148M 400740b" instead of v1173 etc. Hope you guys can fix this in the next version.

That most likely isn't a problem of x264. The x264 configure script detects the revision number via GIT. And that part didn't change!

Whoever made the build you are referring to didn't do it properly. The "M" also indicates unofficial patches ;)


Encoded with my own build of r1173, information was detected with Avinaptic:

[ About H.264 encoding ]

User data: x264
User data: core 67 r1173M f6d3166
User data: H.264/MPEG-4 AVC codec
User data: Copyleft 2003-2009
User data: http://www.videolan.org/x264.html
User data: cabac=1
User data: ref=8
User data: deblock=1:0:0
User data: analyse=0x3:0x133
[...]

[hajj_3]

ahh, i'm using the 1 megui had on it's update list (jeeb's patch).

[JEEB]

Hmm... I always use the information from 'x264 --version' to get the stuff to my relnotes.txt files and so far it has worked well - I can see the version number just fine. So I'd guess the problem is somewhere else...

Just in case, try running 'x264 --version' from command line on this (http://jeeb.fiveforty.jp/x264/1173/x264.exe) build. Of course you should also try the official builds from x264.nl that are compiled without any patches, but I think that my build shouldn't have problems with version numbers...

Also made a test encode with my 1173 build and got the following string in the raw H.264 stream as always - "x264 - core 67 r1173M f6d3166 - H.264/MPEG-4 AVC codec - Copyleft 2003-2009..."

[hajj_3]

can you try opening it with latest mediainfo please to see what version that says.

i emailed the author before posting earlier, he said his app just reads the version from the .mkv file, he doesnt manually detect which version was used so he said the problem was with x264.exe.

[nm]

can you try opening it with latest mediainfo please to see what version that says.
JEEB said that he made a test encode and checked the raw stream, which contains the same version string that MediaInfo would print out.

so he said the problem was with x264.exe.
and more specifically your/MeGUI's x264.exe, which apparently still is r1148.

[LoRd_MuldeR]

I tried to make a fprofiled build of x264 r1178, but it seems the fprofile commands are not compatible with the new arguments :eek:

So I tried to fix the problem like that:
http://pastie.org/537203

Unfortunately I get a new problem now, one that I did not see before:

common/macroblock.c:1417: error: corrupted profile info: profile data is not flow-consistent
common/macroblock.c:1417: error: corrupted profile info: number of executions for edge 3-4 thought to be 53856

Any ideas? :confused:

-- [EDIT] --

D'oh! I found the cause of the problem: fprofile doesn't like threads, but threads are on by default now :D

The proper path should look like this:
http://pastie.org/537224

BTW: I wonder whether presets should be used in the fprofile commands now?

[kemuri-_9]

the fprofile codepaths need a major overhaul...
first of all it doesn't cover b-adapt 0 or b-adapt 2, and now there's the whole incompatibility issue with the new defaults.

not to mention that something broke with the latest patch slew, since x264 exits on a majority of the fprofile codepaths once they're corrected.
seeing at how it exits with a -1 return value with no output, i would say it's from the preset/profile/tune patch

which i've also confirmed by reverting that specific commit.

[LoRd_MuldeR]

not to mention that something broke with the latest patch slew, since x264 exits on a majority of the fprofile codepaths.
seeing at how it exits with a -1 return value with no output, i would say it's from the preset/profile/tune patch

The fprofile commands simply use deprecated CLI arguments that no longer exists in r1178. This will cause x264 to exit.

My patch should fix that. I removes all invalid arguments. The meaning should stay the same, as these options are enabled by default now ;)

But we need to explicitly set "--threads 1" for fprofiled now. That's what I added in the second patch...

[komisar]

LoRd_MuldeR, for gcc 4.4.X add "-fprofile-correction" to CFLAGS for multithread compilation...

...and for "most of the important codepaths" in profiling process this patch (http://komisar.gin.by/x.patch/x265_profile_fix.1178.diff) make identical options for last version

[kemuri-_9]

The fprofile commands simply use deprecated CLI arguments that no longer exists in r1178. This will cause x264 to exit.

My patch should fix that. I removes all invalid arguments. The meaning should stay the same, as these options are enabled by default now ;)

But we need to explicitly set "--threads 1" for fprofiled now. That's what I added in the second patch...

no, i fixed the code paths to use --psnr --ssim for paths 1+ while removing --no-ssim and --no-psnr from 0
and removed --progress replacing it with --threads 1 within the double foreach.
paths 1,2,3,4,5 all exit with -1 with no other output after the above changes.

[LoRd_MuldeR]

paths 1,2,3,4,5 all exit with -1 with no other output after the above changes.

There is no "-8" and no "-w" any longer. These must be removed too. After that it works for me...

[J_Darnley]

For anyone else, try: http://pastebin.com/d2a2c226e It should revert all the tests back to the old settings.

[LoRd_MuldeR]

For anyone else, try: http://pastebin.com/d2a2c226e

Why do you put "--threads 1" to each single line?

Putting it once at the location where you removed the "--progress" is sufficient and avoids redundant code.

[J_Darnley]

Yeah. Fixed that and a couple of remaining default settings. http://pastebin.com/d2a8165aa

[Dark Shikari]

Forgot to update the fprofile settings. Fixed.

[LoRd_MuldeR]

Forgot to update the fprofile settings. Fixed.

You are still using "-w", but I thought that one was removed too :confused:

So I think the correct fix would be to remove "-w" and add "--no-weightb" to all lines that didn't have "-w" before...

[Dark Shikari]

You are still using "-w", but I thought that one was removed too :confused:

So I think the correct fix would be to remove "-w" and add "--no-weightb" to all lines that didn't have "-w" before...The profiles don't have to be exactly the same as before, they just have to exercise all the code paths in as short a commandline as is reasonably possible.

[LoRd_MuldeR]

The profiles don't have to be exactly the same as before, they just have to exercise all the code paths in as short a commandline as is reasonably possible.

That doesn't explain why you explicitly set an option that is enabled by default anyway (and actually doesn't exist as a parameter any more).

So if you don't want to set "--no-weightb", you can at least remove the "-w" parameters, right?

[Dark Shikari]

that doesn't explain why you explicitly set an option that is enabled by default anyway (and actually doesn't exists as a parameter any more).

So if you don't want to set "--no-weightb", you can at least remove the "-w" parameters, right?Ｌｏｏｋ　ａｇａｉｎ　；）

[LoRd_MuldeR]

Ｌｏｏｋ　ａｇａｉｎ　；）

Aaaaand it's gone :D

[Varies]

in last build "x264 modified rev. 1181 (jeeb's)" i can't set threads to 1 :( after encode it's always 6

[kemuri-_9]

in last build "x264 modified rev. 1181 (jeeb's)" i can't set threads to 1 :( after encode it's always 6

adding --threads 1 to the command line is working as it is supposed to.
you're doing something wrong if it isn't

[microchip8]

@wuemura

this thread is ONLY for x264 development discussions, NOT for normal user help!

Please post a new thread about your problems

[Kurtnoise]

@DS: a simple question for you...:D

MaxDPB = (pixel width)*(pixel height)*1.5*min(16, ref #)

does it look correct ?

[LoRd_MuldeR]

I think fprofile is borked in x264 r1189:

OPT5 = --frames 50 --crf 24 -b3 -m10 -r3 --me tesa -t1

It sets SubeME=10, but not Trellis=2. So actually SubME=9 will be used...

[Dark Shikari]

I think fprofile is borked in x264 r1189:

OPT5 = --frames 50 --crf 24 -b3 -m10 -r3 --me tesa -t1

It sets SubeME=10, but not Trellis=2. So actually SubME=9 will be used...Correct. Fixed locally.

[LoRd_MuldeR]

Correct. Fixed locally.

:thanks:

[juGGaKNot]

What is the max subme atm ? 10/11 ? if one would enter a higher value what would happen ?

ask because it does not crash with subme 11/12/666.

or it throws an error message for "invalid" parameters.

Thats why i asked.

[J_Darnley]

10 is the current maximum, and nothing crashes because the library clips the value to between 0 and 10 in x264_validate_parameters

[LoRd_MuldeR]

What is the max subme atm ? 10/11 ? if one would enter a higher value what would happen ?

If you don't want to look at the sources, x264 does have "--(long)help" option you can use. Maybe you look there, before you ask the obvious ;)

-m, --subme <integer> Subpixel motion estimation and mode decision [7]
- 0: fullpel only (not recommended)
- 1: SAD mode decision, one qpel iteration
- 2: SATD mode decision
- 3-5: Progressively more qpel
- 6: RD mode decision for I/P-frames
- 7: RD mode decision for all frames
- 8: RD refinement for I/P-frames
- 9: RD refinement for all frames
- 10: QP-RD - requires trellis=2, aq-mode>0

ask because it does not crash with subme 11/12/666.

Sure it doesn't crash. x264 either clips "wrong" parameter values to the valid range or it throws an error message for "invalid" parameters.

[Chengbin]

Just curious, what does 10L mean?

[Dark Shikari]

first result on google for "10l cola" (http://lists.mplayerhq.hu/pipermail/mplayer-dev-eng/2002-May/007585.html)

[J_Darnley]

The last time that was asked: http://forum.doom9.org/showthread.php?p=1218259&highlight=10L#post1218259
[EDIT] Snap.

[DarkZell666]

The last time that was asked: http://forum.doom9.org/showthread.php?p=1218259&highlight=10L#post1218259
[EDIT] Snap.
first result on google for "10l cola" (http://lists.mplayerhq.hu/pipermail/mplayer-dev-eng/2002-May/007585.html)

And me who thought it simply meant "lol", to indicate self-derision ... nice one ;)

[IgorC]

Is it possible that x264 has surpassed general expecatation for H.264 standard as "twice better than MPEG-2".
I encode a few videos at different by last x264 with new tree MB algo at half of bitrate and it was visually still better than Xvid. (bitrates around 500-1000 kbit/s for DVD resolution).

I know that at high bitrates Xvid can obtain better results but there is also a bitrate saturation effect. And last revision x264 has outstanding quality at much lower bitrates.
IMHO it makes sense to say in case of x264 that is "twice better than MPEG-4 ASP".

My hat is off badly. I've never seen before such brilliant development.

[popper]

i thought the "general expecatation for H.264 standard as "twice better than MPEG-2" was infact 2.5 times better compression for a given quality against mpeg2 (realtime DVB Transport Streams), given the charts given out by the tear one Pro Hardware Encoder AVC vendors (i pointed to way back, but cant find the direct URL again with a quick search), but perhaps you didnt mean it in that sense ?

[Gabriel_Bouvigne]

Is it possible that x264 has surpassed general expecatation for H.264 standard as "twice better than MPEG-2".
I encode a few videos at different by last x264 with new tree MB algo ...
MBTree rate control is totally independant on the fact that the encoder is an h.264 encoder. A similar algorithm could be implemented in an mpeg-2 or mpeg4-asp encoder. Thus I'd say that such an algorithm is irrelevant to the " twice better" claim.

[IgorC]

I wasn't clear.

I tried new treeMB algo and saw that I could go to lower bitrates and still got good quality. Then I decided to compare it to Xvid.
I know that it can be implemented to MPEG-2 and ASP as it's actually happened with VAQ(-like) in HCENC and Xvid implementations. It's not like native H.264 tools: CABAC, reference frames, inloop filter etc...

Now when I rethought my statements I see that I forgot that it isn't correct to compare particular implementations (x264) with format (ASP). It's more correct to compare formats or implementations separately. Xvid vs x264 or H.264 vs ASP.
IIRC H.264 should have ~1.5x quality of ASP. While at reasonable low/middle bitrates x264 has ~2x quality/reduction of bitrate comparing to Xvid.

Main reason of this is that x264 is intensively developing while last important quality (not speed, bugfixes etc...) improvements for Xvid were made during the end of 2004 (except of VAQ in 2008).

[Ku2_BiO_X]

is the new x264 r1203 support vista ultimate x64?

x264 --pass 1 --preset slower --tune animation --bitrate 1000 --output NUL test.avs
i get an error: "could not open test.avs"

the avs script:
"DirectShowSource("test.mkv").converttoyv12"

and when i use test.mkv directly, it only encode ~300 frames but the mkv has more than 300 frames.

another problem: unsupported input format (xvid) when i try to encode avi file.

i use the old revision of x264 (core 65) before and it's working fine.
tq

[nm]

is the new x264 r1203 support vista ultimate x64?

x264 --pass 1 --preset slower --tune animation --bitrate 1000 --output NUL test.avs
i get an error: "could not open test.avs"
Do you have a 32-bit x264 build? Otherwise it can't use AviSynth directly, unless you happen to have squid_80's 64-bit AviSynth build (http://members.optusnet.com.au/squid_80/).

and when i use test.mkv directly, it only encode ~300 frames but the mkv has more than 300 frames.

another problem: unsupported input format (xvid) when i try to encode avi file.
Besides AviSynth, x264 only supports uncompressed YUV 4:2:0 video in AVI, yuv4mpeg or raw files.

[Ku2_BiO_X]

thank you very much nm.
i try the 32-bit version and it works well in my 64bit system. tq

[hajj_3]

when will the x264 in megui update be updated? it is v1183 at the moment, 1210 is the latest version on x264.nl

[kemuri-_9]

when will the x264 in megui update be updated? it is v1183 at the moment, 1210 is the latest version on x264.nl

the x264 devs don't control meGUI, ask the meGUI people instead.

[LoRd_MuldeR]

when will the x264 in megui update be updated? it is v1183 at the moment, 1210 is the latest version on x264.nl

This was answered already in the sticky:
http://forum.doom9.org/showpost.php?p=1311316&postcount=13

Also: Why not replace x264.exe yourself? More than enough people post new builds whenever a new x264 revision becomes available.
You don't need to wait for the x264 developers to upload the new build to the update server for you...

[Astrophizz]

This was answered already in the sticky:
http://forum.doom9.org/showpost.php?p=1311316&postcount=13

Also: Why not replace x264.exe yourself? More than enough people post new builds whenever a new x264 revision becomes available.
You don't need to wait for the x264 developers to upload the new build to the update server for you...

Unless the build changes the cli. To hajj_3: Read some of the Megui threads and you'll see that Kurtnoise has some updates and you can swap over to his server to get a new x264 and core megui version (among other things).

[LoRd_MuldeR]

Unless the build changes the cli. To hajj_3: Read some of the Megui threads and you'll see that Kurtnoise has some updates and you can swap over to his server to get a new x264 and core megui version (among other things).

You can still pass custom CLI parameters that are not supported natively by MeGUI yet ;)

[Chengbin]

Can we get a "roadmap" of where x264 development is heading? The last 1 year(ish) had been almost all performance improvements. Right now it seems like it is adding features. Can we get a very general idea of what x264's development is going, like Intel's CPU roadmap?

[nakTT]

Can we get a "roadmap" of where x264 development is heading? The last 1 year(ish) had been almost all performance improvements. Right now it seems like it is adding features. Can we get a very general idea of what x264's development is going, like Intel's CPU roadmap?
I agree with you. Even a general idea would be helpful to end user like me.

:thanks:

[PlazzTT]

There was such a chart a few years (less?) ago. With info on which features are being developed, how difficult they are to program, etc. An updated version would be interesting.

[G_M_C]

Can we get a "roadmap" of where x264 development is heading? The last 1 year(ish) had been almost all performance improvements. Right now it seems like it is adding features. Can we get a very general idea of what x264's development is going, like Intel's CPU roadmap?

Performance imrovements ? Maybe, i've seen many quality-improvements too; --Subme x, --aqmode x, --mbtree etc. And now the lookahead VBV.

My feeling is that there are more quality improvements, and that the speedimprovements weren't the intended goal, but are to the credit of the patch writers :)

[Chengbin]

Performance imrovements ? Maybe, i've seen many quality-improvements too; --Subme x, --aqmode x, --mbtree etc. And now the lookahead VBV.

My feeling is that there are more quality improvements, and that the speedimprovements weren't the intended goal, but are to the credit of the patch writers :)

I meant "last year" as before subme, aqmode, mbtree.

[burfadel]

With 'Direct Mode' and CRF, I realise that for the most part x264 will use 'Spatial' when in CRF mode since typically information was not forward reaching. With the rc-lookahead, is it possible that 'Direct Mode' can make use of 'Temporal' in 'Auto' mode, since I believe the required information from future frames with the inclusion of rc-lookahead is now available?

[akupenguin]

Unrelated to rc-lookahead. The information needed to decide between spatial and temporal has always been available if you're doing any kind of ratecontrol or frametype decision, at least if lowres would be good enough. It's still not implemented.

[Revgen]

when will the x264 in megui update be updated? it is v1183 at the moment, 1210 is the latest version on x264.nl

Kurtnoise is now handling MeGUI updates. You can find a link to his server in the MeGUI thread.

[me7]

Any news on weightp, that is supposed to help fades with MBTree?

[microchip8]

Any news on weightp, that is supposed to help fades with MBTree?

weightp will be committed when it's GSOC author is done - within days or so. Stop asking

[Zachs]

weightp progress here - http://repo.or.cz/w/x264/x264-p-frames.git?a=shortlog;h=refs/heads/gsoc

[LoRd_MuldeR]

weightp progress here - http://repo.or.cz/w/x264/x264-p-frames.git?a=shortlog;h=refs/heads/gsoc

How can I check this out? It always fails via HTTP and doesn't work at all via GIT protocol.

http://pastie.org/593644

(I know that I can download the snapshot as ZIP file, but then the configure script will complain that it's not a valid GIT repository)

[ACoolie]

You're using the wrong urls. s/w/r/ in the HTTP url: http://repo.or.cz/r/x264/x264-p-frames.git
and Git: git://repo.or.cz/x264/x264-p-frames.git

[LoRd_MuldeR]

You're using the wrong urls. s/w/r/ in the HTTP url: http://repo.or.cz/r/x264/x264-p-frames.git
and Git: git://repo.or.cz/x264/x264-p-frames.git

Ah :thanks:

[LoRd_MuldeR]

I was getting a crash (reproducible!) with libx264 r1232 on a specific sample clip, using the following settings:

cabac=1 ref=8 deblock=1:-1:-1 analyse=0x3:0x133 me=umh subme=10 psy=1 psy_rd=1.00:0.15 mixed_ref=1 me_range=24 chroma_me=1 trellis=2 8x8dct=1 cqm=0 deadzone=21,11 chroma_qp_offset=-3 threads=6 nr=0 decimate=1 mbaff=0 bframes=5 b_pyramid=0 b_adapt=2 b_bias=0 direct=3 wpredb=1 keyint=500 keyint_min=25 scenecut=40 rc_lookahead=60 rc=crf mbtree=1 crf=21.0 qcomp=0.60 qpmin=10 qpmax=51 qpstep=4 ip_ratio=1.40 aq=1:1.00

The backtrace goes like this:
*********** EXCEPTION **************
Registers:
EAX: 00000012 EBX: 075F1750 ECX: 00000000 EDX: 00000001 ESI: 00000000
EDI: 0022A984 ESP: 002180B0 EBP: 00000002 EIP: 04C22F07 EFlags: 00010246

Exception Code: EXCEPTION_ACCESS_VIOLATION (C0000005)
Exception Flags: 00000000

Origin:
D:\Avidemux 2.5\libx264-72.dll(x264_quant_4x4_trellis+0x72F07) [0x04C22F07]
*********** EXCEPTION **************

*********** BACKTRACE **************
Frame 0: D:\Avidemux 2.5\libADM_core.dll(Z16exceptionHandlerP17_EXCEPTION_RECORDPvP8_CONTEXTS1_+0x530E) [0x6510530E]
Frame 1: C:\WINDOWS\system32\ntdll.dll(RtlRaiseStatus+0x1EC4A) [0x7D61EC4A]
Frame 2: C:\WINDOWS\system32\ntdll.dll(RtlRaiseStatus+0x1EC1B) [0x7D61EC1B]
Frame 3: C:\WINDOWS\system32\ntdll.dll(KiUserExceptionDispatcher+0x1EA56) [0x7D61EA56]
*********** BACKTRACE **************

However I am unable to re-produce the problem with a debug build of libx264! Also x264.exe did not crash on the sample. Any ideas ??? :eek:

(BTW: There's no such problem with my build of libx264 r1222, so it must be something between r1222 and r1232)

[IgorC]

x264 supports ARM now. :eek:

It's important news considering that now ARM is expanding from embedded/mobile systems to laptop sector. Also ARM is extremely
power efficient that makes its systems runs without power supply during many hours or even days while Intel can't.

I'm working with some basic application based on ARM7 (maybe Cortex in 1 or 2 years) as part of student project. In future I'm considering to move to some multimedia approach.
It's great to count with x264 and CoreAVC 2.0 supporting ARM.

[LoRd_MuldeR]

Okay, I was able to track down my crash (http://forum.doom9.org/showpost.php?p=1318401&postcount=765) to the "-march=core2" compiler option!

With that option set, the resulting libx264 will crash. Tried both builds of GCC 4.4.1 , Komisar's and TDM's. Both with the same result:

Crash with "-march=core2" and no crash without it. Also "-march=pentium2" does not trigger the crash. However my previous builds (r1222 and older) compiled with "-march=core2" are working fine :confused:

I suspect something between r1222 and r1232 did cause GCC/MinGW to break x264 with "-march=core2" for some reason. Any ideas ???

[juGGaKNot]

the 1222 march core2 also crashed for 2-3 people using my bat, some did not crash but the fps was double on some frames ( faster, music lasted longer )

and youtube converter did not work on them ( before worked )

-march=i686 all problems solved.

[LoRd_MuldeR]

Ha! It's getting more curious:

There's no crash with GCC 4.3.3 and there also is no crash with GCC 4.4.1 after I re-added the Graphite Loop Transformations (-floop-interchange -floop-strip-mine -floop-block).

Damn, I just removed the Graphite optimizations on suggestion (http://forum.doom9.org/showpost.php?p=1317722&postcount=36) of Komisar :D


Also this warning just appeared on the TDM web-site:

The 4.4.1-tdm-1 release is known to have a bug which causes drastically increased CPU usage in programs compiled with it. You are urged to use a previous release until this bug is fixed.

:eek:

[juGGaKNot]

i guess 4.3.3 for life haha

aa forgot about it, 1222 ( i think ) also crashed when i set bframes to 0 than to 3

try this out.

[lexor]

Seeing the new commits for the GSoC ARM patch makes me curious. Why does x264 need ARM optimizations? Why would you run encoder on an ARM device, I see why you would want to play back on it, but why encode? I just don't see a use case.

[Dark Shikari]

Seeing the new commits for the GSoC ARM patch makes me curious. Why does x264 need ARM optimizations? Why would you run encoder on an ARM device, I see why you would want to play back on it, but why encode? I just don't see a use case.Video recording? Video calls/conferencing? Personal video broadcast (a'la Justin.tv)?

See the blog post (http://x264dev.multimedia.cx/?p=142).

[LoRd_MuldeR]

Seeing the new commits for the GSoC ARM patch makes me curious. Why does x264 need ARM optimizations? Why would you run encoder on an ARM device, I see why you would want to play back on it, but why encode? I just don't see a use case.

As far as I know, we will see (Sub)Notebooks and/or Netbooks with ARM architecture on the market very soon (if they aren't available yet).

Given that you own such a device and want to encode a video with x264, but there's no "heavyweight" x86 machine available at your location, then you'll be very happy for ARM optimizations!

So I see no reason to not have those optimizations, as long as somebody is skilled and willing to do it. They certainly won't hurt :)

[CruNcher]

Think about being able to use x264 to save your tv programm with a itchy small device that fits in your pocket and you can cary it to wherever you want no big boxes standing anywhere anymore :) (a dream) and with a pico projector integrated you can watch it on every wall in reach (though that will become IP problematic because you not allowed to show the content in public most of the time hehe)

[rack04]

Ha! It's getting more curious:

There's no crash with GCC 4.3.3 and there also is no crash with GCC 4.4.1 after I re-added the Graphite Loop Transformations (-floop-interchange -floop-strip-mine -floop-block).

Damn, I just removed the Graphite optimizations on suggestion (http://forum.doom9.org/showpost.php?p=1317722&postcount=36) of Komisar :D


Also this warning just appeared on the TDM web-site:

The 4.4.1-tdm-1 release is known to have a bug which causes drastically increased CPU usage in programs compiled with it. You are urged to use a previous release until this bug is fixed.

:eek:

Does it crash with this build?

x264 core:72 r1232M x86

Download (http://www.megaupload.com/?d=RF89ZJL8)

Built by rack04 on August 25, 2009, 12:52:28 PM CST
$ gcc --version
gcc.exe (GCC) 4.3.4 20090526 (prerelease) (x86.core2.Komisar)
$ ./configure --extra-cflags="-march=core2"
Platform: X86
System: MINGW
asm: yes
avis input: yes
mp4 output: yes
pthread: yes
debug: no
gprof: no
PIC: no
shared: no
visualize: no
$ make fprofiled VIDS="bigbuckbunny.avs LosslessTouhou.avs riverbed.1920x1080.yuv"

Patched with:

x264_hrd_pd_interlace.16.fix.1232.diff (http://pastebin.org/11898)
x264_win_zone_parse_fix_06.diff (http://kemuri9.net/dev/x264/patches/x264_win_zone_parse_fix_06.diff)

[DeeGee]

Yes, quite a lot of new hardware is coming/has already came with ARM Cortex A8/A9 based processors. I myself have been eyeing the rumored Nokia N900 (RX-51/Rover) as my next tech toy/phone :)

[lexor]

I'm not convinced that's a use case. Why would you choose to encode on that as consumer. And we can't install x264 (or anything else for that matter) on video cameras, so again I don't see a point.

[Dark Shikari]

I'm not convinced that's a use case. Why would you choose to encode on that as consumer. And we can't install x264 (or anything else for that matter) on video cameras, so again I don't see a point.Whoever said that this patch was for the consumer?

[popper]

I'm not convinced that's a use case. Why would you choose to encode on that as consumer. And we can't install x264 (or anything else for that matter) on video cameras, so again I don't see a point.

actually You Can, (well this developer version doesnt have a cam right now,and the netbook when its ready may or not but they can take a usb cam) at a higher price right now, or if your a developer you may get a unit under consession if your doing something of real value to them in the OSS space perhaps.

http://bbrv.blogspot.com/2009/08/efikamx-on-sale-now.html
http://www.genesi-usa.com/products/openclient
http://www.genesi-usa.com/products/netbook not available yet.

theres no report of the Efika dev manager (neko) or anyone else with a board (markos, freevec etc),
compiling x264 Cortex A8 and running D1 H@L3.1 Encoder tests on that, and comparing to its internal Cortex A8 Encoder (can that even do High@? anything) quality so far.

but it may be tryed soon as their latest firmware catchs up with their HW, as 264 NEON is ready made for them ,and they DO know about the x264 NEON additions.

[LoRd_MuldeR]

Two questions regarding r1233 (http://git.videolan.org/gitweb.cgi?p=x264.git;a=commitdiff;h=ee95ca7970140e8b99af233b03828feb250f0af0):

In case of Core2 optimized builds: Would it be helpful to pass "-mssse3" rather than the "-msse" set by default? Or is that redundant to the "-march=core2" switch?
Is it sufficient to pass "-mfpmath=387" in --extra-cflags to make a build that will run on a CPU that only supports MMX, but no SSE? (like the good old Pentium2)

[J_Darnley]

Is it sufficient to pass "-mfpmath=387" in --extra-cflags to make a build that will run on a CPU that only supports MMX, but no SSE? (like the good old Pentium2)

No, because of the changes made to encoder/encoder.c

[LoRd_MuldeR]

No, because of the changes made to encoder/encoder.c

What "changes" are you referring to please?

I saw that there's a new error message that will pop up on machines without SSE support, if x264 was compiled with SSE enabled.

But what I'm trying to get is a build that doesn't have SSE enabled in a "hardcoded" way. So what do I need to do?

(I don't want to completely disable all ASM stuff, because at least MMX optimized code should stay enabled in the Pentium2 build)

[J_Darnley]

What "changes" are you referring to please?

You posted the link.

static int x264_validate_parameters( x264_t *h )
{
#ifdef HAVE_MMX
- if( !(x264_cpu_detect() & X264_CPU_MMXEXT) )
+ if( !(x264_cpu_detect() & X264_CPU_SSE) )
{
- x264_log( h, X264_LOG_ERROR, "your cpu does not support MMXEXT, but x264 was compiled with asm support\n");
+ x264_log( h, X264_LOG_ERROR, "your cpu does not support SSE1, but x264 was compiled with asm support\n");
x264_log( h, X264_LOG_ERROR, "to run x264, recompile without asm support (configure --disable-asm)\n");
return -1;
}


[EDIT] The message isn't completely new. To allow non-sse machines to run x264, just revert all the changes in 1233.
[EDIT2] If you are going to continue providing builds like this then watch out for actual SSE instructions being added.

[LoRd_MuldeR]

Sorry, please see my edit in the previous post...

[J_Darnley]

LOL, see my edits.

[LoRd_MuldeR]

[EDIT2] If you are going to continue providing builds like this then watch out for actual SSE instructions being added.

Well, from all that I understand, x264 does its own runtime CPU detection. So it won't call any SSE optimized ASM functions, unless SSE is explicitly supported by the CPU.

AFAIK only MMX instructions will be used in a "hardcoded" way, if MMX was supported by the machine x264 was compiled on. But that's not a problem for the Pentium2 processor.

However if the C compiler uses SSE instructions, this may result in code which won't run on Non-SSE processors. And we are not talking about the ASM code here anymore!

Starting with r1233 x264 does tell the C compiler to use SSE. All I wanted to know: Is passing "-mfpmath=387" in the --extra-cflags sufficient to avoid that? :confused:

(BTW: What about my second question: Is "-mssse2" redundant, if "-march=core" was set ???)

[J_Darnley]

When non-MMX support was dropped, this was because of an inlined function that (I think) was only disabled with --disable-asm. This hasn't happened for SSE, yet, because the commit is just plain C but it is faster when done with SSE instructions.

[EDIT] I have no idea about 'Is "-mssse2" redundant, if "-march=core" was set?' so I only responded to the bit I could answer.

[LoRd_MuldeR]

When non-MMX support was dropped, this was because of an inlined function that (I think) was only disabled with --disable-asm. This hasn't happened for SSE, yet, because the commit is just plain C but it is faster when done with SSE instructions.

Yes, I see. But this didn't answer my question. Or did I miss the point? ^^

Unless they start using SSE code in Inline-Assembly we should be on the safe side by disabling SSE code generation in the C compiler. But still I wonder whether "-mfpmath=387" is sufficient for that :confused:

I'd simply test it, if I only had a Pentium2 machine here. But I don't have one anymore...

[J_Darnley]

We need to stop editing our posts. :)

I meant: No, MMX is now used unless you configure with --disable-asm. See the (old) error message:
your cpu does not support MMXEXT, but x264 was compiled with asm support to run x264, recompile without asm support (configure --disable-asm)

That is a necessary restriction because MMX is explicitly used in one function. But with SSE, the restriction is only required because of the CFLAGS which tell the compiler to use use SSE. If you tell the compiler to use the "old" instructions then there is no need for the "no SSE" error.

[LoRd_MuldeR]

I meant: No, MMX is now used unless you configure with --disable-asm. See the (old) error message:
your cpu does not support MMXEXT, but x264 was compiled with asm support to run x264, recompile without asm support (configure --disable-asm)

That code is only enabled when HAVE_MMX is defined. And that is only defined when the machine where x264 was compiled on did support MMX.

If x264 was compiled on a Non-MMX machine, it will run on all Non-MMX machines just fine. No error message will appear...

That is a necessary restriction because MMX is explicitly used in one function. But with SSE, the restriction is only required because of the CFLAGS which tell the compiler to use use SSE. If you tell the compiler to use the "old" instructions then there is no need for the "no SSE" error.

But I suspect the error message will appear anyway on a Pentium2 CPU, even when I told the C compiler to not use SSE!

That's because the code in question is enabled (HAVE_MMX is defined), but the return value of x264_cpu_detect() won't have X264_CPU_SSE set for obvious reasons :eek:

[J_Darnley]

Configure doesn't check to see if MMX can be run, HAVE_MMX is defined if you don't disable asm and you are on x86 or x64.

If you disable asm, x264 will run on all x86 CPUs.

At present, the "no SEE" error will appear if x264 was build with asm and x264 detects your CPU lacks SSE. Previously it was "no MMX" if your CPU lacked MMX.

If for some masochistic reason you want to run x264 on a Pentium 2, you had better revert the changes in rev 1233 and compile. Then abort your encode, because your only encode is still going and start again.

[LoRd_MuldeR]

Maybe time to rename the HAVE_MMX symbol to ENABLE_ASM or something like that...

[kemuri-_9]

Maybe time to rename the HAVE_MMX symbol to ENABLE_ASM or something like that...

or not since the HAVE_(__ASM_FLAG__) is hardware specific:

PPC has no such flag,
x86/x86_64 uses HAVE_MMX
ARM uses HAVE_ARMV6, HAVE_ARMV6T2, and HAVE_NEON

you can't realistically use something as vague as 'ENABLE_ASM' to cover all these situations

[LoRd_MuldeR]

You are right. But "HAVE_MMX" just isn't meaningful any more. It's meaning now changed to "HAVE_SSE1" and probably may change again in the future...

[kemuri-_9]

make a patch to change all instances of HAVE_MMX to HAVE_SSE within the code base if you're that paranoid about it.
and maybe it'll get committed

[akupenguin]

HAVE_MMX hasn't meant MMX to the exclusion of SSE since r634 (http://git.videolan.org/?p=x264.git;a=commitdiff;h=8b37cc6aa5e23bf4a529b79745f73dedab1fa4ff), and it isn't going to change now.

[Dark Shikari]

We really don't care about Athlon K6-2s and Pentium 2s. Seriously.

[LoRd_MuldeR]

We really don't care about Athlon K6-2s and Pentium 2s. Seriously.

Okay, but with "--disable-asm" (lib)x264 will still run on those, right?

[Dark Shikari]

Okay, but with "--disable-asm" (lib)x264 will still run on those, right?Of course.

[LoRd_MuldeR]

i8x8/i4x4 never got analysed when fast_intra was toggled and RD was off; up to a 2-3% quality improvement in non-RD mode.
With this bug dating back to r369, this is probably the second-oldest bug ever fixed in x264.

So do I understand right that this bug did effect "--subme 5" and lower only?

[Dark Shikari]

i8x8/i4x4 never got analysed when fast_intra was toggled and RD was off; up to a 2-3% quality improvement in non-RD mode.
With this bug dating back to r369, this is probably the second-oldest bug ever fixed in x264.

So do I understand right that this bug did effect "--subme 5" and lower only?Subme 6 and lower, as 7 enables RD on B-frames.

[LoRd_MuldeR]

Subme 6 and lower, as 7 enables RD on B-frames.

Ah, okay :)

[Daemon404]

Since ARM support has been added recently, I was wondering, have any tests been done to compare speed and/or power draw on ARM procs, in comparison to similar (as in price/speed/mobile) x86 procs (e.g. atom)?

[aegisofrime]

Since ARM support has been added recently, I was wondering, have any tests been done to compare speed and/or power draw on ARM procs, in comparison to similar (as in price/speed/mobile) x86 procs (e.g. atom)?

I'm interested to know as well.

[Dark Shikari]

A Cortex A8 uses about 0.2 watts, while an Atom with chipset uses about 15 watts. They're hardly comparable.

From a test done a while back on H.264 decoding with all assembly code turned off, if a Core 2 has a speed of "1.0" per clock, while a Cortex A9 has a speed of about "0.4" per clock, a Pentium 4 has "0.38", and a Cortex A8 has about "0.28", if I recall the numbers correctly. I don't know if the gap closes or increases with the addition of NEON, but the Atom has much much worse performance than the Pentium 4, so I would expect the Cortex A9 to trash the Atom clock-for-clock; even the A8 should beat it.

[popper]

A Cortex A8 uses about 0.2 watts, while an Atom with chipset uses about 15 watts. They're hardly comparable.

From a test done a while back on H.264 decoding with all assembly code turned off, if a Core 2 has a speed of "1.0" per clock, while a Cortex A9 has a speed of about "0.4" per clock, a Pentium 4 has "0.38", and a Cortex A8 has about "0.28", if I recall the numbers correctly. I don't know if the gap closes or increases with the addition of NEON, but the Atom has much much worse performance than the Pentium 4, so I would expect the Cortex A9 to trash the Atom clock-for-clock; even the A8 should beat it.

as a more general baseline on a full real devs/consumer PCB, and evolving Firmware/SW
(remember "Efika" and 'Efika MX' Are two totally seperate PCB, one a PPC G3 ,discontinued, one the arm i.MX515 )

Matt Sealey,Developer Relations,Product Development Analyst
for Genesi said of their 'Efika MX' Open Client, Freescale i.MX515 (ARM Cortex-A8/NEON 800MHz) board

"BTW Open Client power consumption running Citrix is 5W maximum, tested using a Kill-A-Watt.

I've not been able to try out any video decoding yet, and again this is before any advanced power management has been enabled."

"the chip is running at top speed and the backlight is turned on full.. You can expect some very decent power savings across the board once the software is more mature."

http://www.powerdeveloper.org/forums/viewtopic.php?t=1557&postdays=0&postorder=asc&start=0

regarding their codecs for it, he also said this on the blog
"You can play video from anything, depending on the speed of the device. USB should be fine for DVD-quality video. You might hit limits on an SDHC card (15MB/s or thereabouts) for HD content.

We did not release accelerated codecs yet but we are working on it."

there are OC several tests on that thread and linked off it, so you can get a good feel for their full load averages etc,
the only real No's you should care about as its doing real work, its odd, but no one seems to have tryed using the
included wireless 11N streaming/down/uploading while collecting the power No's so far...

for instance Peter Czanik (czp) Jul 13, 2009 said

"I just measured the power consumption of my iMX515 based EFIKA system - display device is included. It also has 4x more RAM and 2x the MHz of the original EFIKA, but no HDD, just a larger SD card. Power consumption is 9W with the bundled LCD turned on. A 2.5" USB HDD add one more Watt and an USB Ethernet adapter another one. When the LCD turns off, it spares 2-3W.
"

[Daemon404]

Results are pretty much what I figured. Anyway, is there anything publicly available that uses A9 yet?

[nakTT]

A Cortex A8 uses about 0.2 watts, while an Atom with chipset uses about 15 watts. They're hardly comparable.

From a test done a while back on H.264 decoding with all assembly code turned off, if a Core 2 has a speed of "1.0" per clock, while a Cortex A9 has a speed of about "0.4" per clock, a Pentium 4 has "0.38", and a Cortex A8 has about "0.28", if I recall the numbers correctly. I don't know if the gap closes or increases with the addition of NEON, but the Atom has much much worse performance than the Pentium 4, so I would expect the Cortex A9 to trash the Atom clock-for-clock; even the A8 should beat it.
I also getting more exited with these ARMs thingy going on, especially the upcoming Cortex-A9 MPCore.

Just out of curiosity to get a more clear picture, may I know the clock speed of all the processors in question? (Core 2, ARM Cortex-A9 [from which company?], the Pentium 4).

[popper]

I also getting more exited with these ARMs thingy going on, especially the upcoming Cortex-A9 MPCore.

Just out of curiosity to get a more clear picture, may I know the clock speed of all the processors in question? (Core 2, ARM Cortex-A9 [from which company?], the Pentium 4).

in reply to a imagination/POWERVR VXE320 and VXE360 cores Encoder IP question apparently "Raquel and Bill owners of genesi said... the graphic cores found in the i.mx51 were licensed from AMD not Imagination. We do have a source license and the code from AMD so you can expect the performance to be very good." make of that what you will, until some real Numbers become available at least.

as for the Freescale Cortex-A9, that comes in both a single and dual packages, but theres NO sign what so ever of any retail PCBs on the horizon if thats what your hoping for and after, so no real life clock data for you , but just for interest you might find this http://arm.com/products/CPUs/ARMCortex-A9_MPCore.html a good starting point.

its interesting from a sometime in the future NEON MP x264 POV IF these initial freescale A8/NEON take off on mass...

[Blue_MiSfit]

Hmm... I wonder if a whole mess of tiny embedded systems would be more power / space efficient than my 2U 8 core encoders? :)

~MiSfit

[nakTT]

in reply to a imagination/POWERVR VXE320 and VXE360 cores Encoder IP question apparently "Raquel and Bill owners of genesi said... the graphic cores found in the i.mx51 were licensed from AMD not Imagination. We do have a source license and the code from AMD so you can expect the performance to be very good." make of that what you will, until some real Numbers become available at least.

as for the Freescale Cortex-A9, that comes in both a single and dual packages, but theres NO sign what so ever of any retail PCBs on the horizon if thats what your hoping for and after, so no real life clock data for you , but just for interest you might find this http://arm.com/products/CPUs/ARMCortex-A9_MPCore.html a good starting point.

its interesting from a sometime in the future NEON MP x264 POV IF these initial freescale A8/NEON take off on mass...
Many thanks for you reply. As of now I only have eyes for Cortex-A9 MPCore or at least A8. You seems to know quite a lot about ARM processor. This is kind of guy I'm looking for when comes to advice and real-life info.

As for the retail PCB unavailability, I think I can settle for the upcoming ARM SmartBook instead (rumour to be released sometime in the near future) though I still prefer it to be in the 'OpenRD's form factor' instead of a SmartBook.

Any idea (dispense with the manufacturer's marketing talk) what can we expect from Cortex-A9 MPCore from Qualcomm 1.5 GHz part, TI 1.0 GHz part (rumour to be more power efficient compared to Qualcomm's 1.5 GHz part clock-for-clock) and Freescale, Marvell, Samsung and the like? Please share your thought.


:thanks:

[nakTT]

Hmm... I wonder if a whole mess of tiny embedded systems would be more power / space efficient than my 2U 8 core encoders? :)

~MiSfit
I think they are, in term of power/performance/space. But I maybe wrong.

[popper]

Hmm... I wonder if a whole mess of tiny embedded systems would be more power / space efficient than my 2U 8 core encoders? :)

~MiSfit

if you can write a (message based ? Realtime Multitasking [Amigs OS style] data passing) patch to make x264 Effectively use these mass of clustered CPU/SOC/SIMD over a gigE+ tcp:IP/UDP links.

or start a well run community based 'Bounty program' Original AROS bounty style http://www.power2people.org/projects.html http://aros-exec.org/ to collect financing for missing and/or interesting quality based extensions such as adding Generic multi platform clustered Encoding to the x264 codebase and GUI support applications etc, and so kick start them to get more professional coders interested in contributing.

then you might stand a reasonable chance of better using the under used generic LAN/WAN connected device resources, both now, and in the future, OR not as the case may be, again make of this really Old idea what you will ;)

[Dark Shikari]

I also getting more exited with these ARMs thingy going on, especially the upcoming Cortex-A9 MPCore.

Just out of curiosity to get a more clear picture, may I know the clock speed of all the processors in question? (Core 2, ARM Cortex-A9 [from which company?], the Pentium 4).Clock speed doesn't matter for those numbers, and it's rather easy to look up what clock speeds the P4/Core 2 were/are available in, but as far as I recall the Cortex chips are currently available around ~600mhz and may scale up to about 1Ghz.

[nakTT]

Clock speed doesn't matter for those numbers, and it's rather easy to look up what clock speeds the P4/Core 2 were/are available in, but as far as I recall the Cortex chips are currently available around ~600mhz and may scale up to about 1Ghz.
Sorry, totally my mistake. I have missed your "per clock" word while reading your previous post. My bad.


From a test done a while back on H.264 decoding with all assembly code turned off
I'm no programmer, perhaps you could explain a little bit what is the significance of "assembly code turned off/on" so that people like me can better understand things.

:thanks:

[Dark Shikari]

I'm no programmer, perhaps you could explain a little bit what is the significance of "assembly code turned off/on" so that people like me can better understand things.

:thanks:It means we're measuring (counting in whatever inefficiencies of GCC and whatnot) the performance per clock in ordinary situations without using the SIMD unit.

If you don't know what SIMD is, look it up.

[Astrophizz]

Assembly code turned off: slow (relative to regular performance), relies more heavily on compiler for speed.
Assembly code turned on: blazing fast.

Edit: doh i got beat!

[nakTT]

It means we're measuring (counting in whatever inefficiencies of GCC and whatnot) the performance per clock in ordinary situations without using the SIMD unit.

If you don't know what SIMD is, look it up.
I see. In other word we can expect the "H.264 Decoding" number will somewhat get improved overtime (per clock). Am I understand you correctly?

How about encoding? Any info/figure that you can share from your previous testing?

:thanks:

[Dark Shikari]

I see. In other word we can expect the "H.264 Decoding" number will somewhat get improved overtime (per clock). Am I understand you correctly?Sure, if ARM improves their chip designs, the chips' performance will improve.

[nakTT]

Sure, if ARM improves their chip designs, the chips' performance will improve.
Actually what I meant is the improvement from better coding.

As of now we can even encode using x264 on ARM based system?

[popper]

i do wonder if theres any plans by anyone to make x264 FPGA ready one day,Dark Shikari, or if its even possible as the code stands today for effective translation, now that might be very interesting, nakTT that means a version of x264 routines translated to and built into hardware rather than software running on some cpu/simd hardware as today.

the old The H.264 Scalable Video Codec (SVC) is still a very popular read
http://www.dspdesignline.com/howto/206902266

[Dark Shikari]

Actually what I meant is the improvement from better coding.

As of now we can even encode using x264 on ARM based system?Did you even read the past two pages of this thread? ;)

[nakTT]

Did you even read the past two pages of this thread? ;)
No I'm not. :D

Sorry, will try to read it.

:thanks:

[Kurtnoise]

The most useful patch for today (http://pastebin.com/f737a88e8)...:D

[Dark Shikari]

The most useful patch for today (http://pastebin.com/f737a88e8)...:DWhy? It works fine as-is.

[Kurtnoise]

yes but I was just curious by the -Aall during compilation...so

[vucloutr]

Does Multi-Slice encoding support (http://git.videolan.org/gitweb.cgi?p=x264.git;a=commit;h=4d553edf178bf0ae01547731a48e1fb08c5cc1f4) affect multithreading in any way ? If yes, how ?

[Dark Shikari]

Does Multi-Slice encoding support (http://git.videolan.org/gitweb.cgi?p=x264.git;a=commit;h=4d553edf178bf0ae01547731a48e1fb08c5cc1f4) affect multithreading in any way ? no .

[kemuri-_9]

yes but I was just curious by the -Aall during compilation...so

the short options can either be followed immediately by the value or have a space between:

-m0 can also be done as -m 0, both are accepted.

[G_M_C]

Any news on weightp, that is supposed to help fades with MBTree?

weightp will be committed when it's GSOC author is done - within days or so. Stop asking


Since GSoC was/has finished on 17 august I wondered if there is any news on the status on weightp atm ?

[J_Darnley]

Lurk more.
http://mailman.videolan.org/pipermail/x264-devel/2009-September/006324.html

[G_M_C]

Lurk more.
http://mailman.videolan.org/pipermail/x264-devel/2009-September/006324.html

Thx :)

I'm no developer, so i'm not usually lurking on the development pages, so i missed that post. I'm "just a user", but an interested one, that why i hope to expiriment with this patch (new addition/new feature might be a better description) when it's commited.

[microchip8]

Thx :)

I'm no developer, so i'm not usually lurking on the development pages, so i missed that post. I'm "just a user", but an interested one, that why i hope to expiriment with this patch (new addition/new feature might be a better description) when it's commited.

If you don't want to lurk on these pages, then do as I do. Subscribe to x264 devel mailing list and get your notifications for free and on time ;)

[iwod]

A Cortex A8 uses about 0.2 watts, while an Atom with chipset uses about 15 watts. They're hardly comparable.

From a test done a while back on H.264 decoding with all assembly code turned off, if a Core 2 has a speed of "1.0" per clock, while a Cortex A9 has a speed of about "0.4" per clock, a Pentium 4 has "0.38", and a Cortex A8 has about "0.28", if I recall the numbers correctly. I don't know if the gap closes or increases with the addition of NEON, but the Atom has much much worse performance than the Pentium 4, so I would expect the Cortex A9 to trash the Atom clock-for-clock; even the A8 should beat it.

Arh... That is some very Sad numbers.
When i saw Intel do have a very clear path to 10nm. I thought that was the end of ARM. Since within 2 - 4 years time they could scale Atom into current ARM size with nearly the same power envelop. And since Hardware is Cheap, Software is expensive, no one will be border to port anything over to ARM.

However these numbers looks like there is still a huge gap.

Core2Duo have 4 times the Maximum Frequency then Cortex A9.
Core2Duo uses around 50 - 100 times the Max Power of Cortex A9's Max power.
Core2Duo has , i guess nearly 10 times the Die Size. ( Properly much less with out the Cache though )

Now i wonder if Core2Duo with MMX+SSE would bring higher performance increase then Cortex A9 + Neon.

Looks like ARM still have some fighting chance.

[Sagekilla]

Sure, a Core 2 can make the Cortex A8/A9 look like crap performance wise. Dark just said that.

But Atom is not the same chip as Core 2. It's based off that architecture, but they removed a lot from it to get it to be power efficient (IIRC, Out-of-order execution, fewer execution units, instruction emulation, among other things).

Sure, if they can shrink down a Core 2 and adjust the core just right so they don't gimp it too much, it could be a threat to Cortex.

IMO, they'd have better results trying to optimize their process for ultra low voltage processors and then doing some stripping of more power hungry features. Latest ULV processors dissipate 5.5 - 10 W, atom averages 2.5 - 8. Those ones are full fledged Pernyns, mind you.

[popper]

Arh... That is some very Sad numbers.
When i saw Intel do have a very clear path to 10nm. I thought that was the end of ARM. Since within 2 - 4 years time they could scale Atom into current ARM size with nearly the same power envelop. And since Hardware is Cheap, Software is expensive, no one will be border to port anything over to ARM.

However these numbers looks like there is still a huge gap.

Core2Duo have 4 times the Maximum Frequency then Cortex A9.
Core2Duo uses around 50 - 100 times the Max Power of Cortex A9's Max power.
Core2Duo has , i guess nearly 10 times the Die Size. ( Properly much less with out the Cache though )

Now i wonder if Core2Duo with MMX+SSE would bring higher performance increase then Cortex A9 + Neon.

Looks like ARM still have some fighting chance.

OC they have now commited to a 2gig speed for the A9, although no ones going to produce anything substantial until the last Quarter it seems, and we still dont know the generic configuration of the dual-NEON™ as yet...
and one of the reasons i asked about any existing or planed x264 IP for FPGA/ASIC and DSP related use sooner or later....

http://www.arm.com/news/25922.html
16 September 2009

ARM Announces 2GHz Capable Cortex-A9 Dual Core Processor Implementation
...
"The Cortex-A9 power-optimized hard macro implementation delivers its peak performance of 4000 DMIPS while consuming less than 250mW per CPU when selected from typical silicon.

The hard macro implementations include ARM AMBA®-compliant high performance system components to maximize data traffic speed and minimize power consumption and silicon area. Each Cortex-A9 hard macro implementation also includes the CoreSight™ Program Trace Macrocell (PTM) which provides full visibility into the processor’s instruction flow, enabling the software community to develop code for optimal performance.
..."

"...
Both ARM dual core Cortex-A9 hard macros will share a common seven-power domain, dual-NEON™ technology configuration supporting SMP (symmetrical multiprocessing) operating systems with up to 8MB of Level2 cache memory and will be delivered with all scripts, vectors and libraries required to integrate the macro directly within any SoC device.
..."

"implementations are available for license today with delivery in the fourth quarter of 2009. ARM’s 40G physical IP platform is also available today at designstart.arm.com."

http://www.arm.com/products/physicalip/designstart.html
"...
DesignStart
Welcome to DesignStart, the most comprehensive online IP access portal in the industry. DesignStart offers a full compliment of Free Physical IP libraries and extensive front end views or models of Fee based IP supporting industry leading EDA tools, providing a unique try before you buy design environment. In addition, we offer selected foundry sponsored Processor Design Kits.
..."


http://search.arm.com/search?q=high+profile+h.264&site=FullSiteSearch&client=ARM_Search&output=xml_no_dtd&sort=date%3AD%3AL%3Ad1&ie=UTF-8&oe=UTF-8&proxystylesheet=ARM_Search

[Fr4nz]

I saw that the upcoming weighted p-frame prediction uses k-means; my question to DS is: are you using the old Lloyd's algorithm or something better (in terms of speed and result quality), like k-means++ (http://theory.stanford.edu/~sergei/papers/kMeansPP-soda.pdf)? If not, consider that paper, it should also be easy to implement as the Lloyd's algo...

[Dark Shikari]

I saw that the upcoming weighted p-frame prediction uses k-means; my question to DS is: are you using the usual Lloyd's algorithm or something really better (in terms of speed and result quality), like k-means++ (http://theory.stanford.edu/~sergei/papers/kMeansPP-soda.pdf)? If not, consider that paper...Yes, we use k-means++.

[LeonLanford]

Hi, I want to ask a little, I just updated from "x264 core 75 r1251M e553a4c" to "x264 core 75 r1259M dd026f2".

I use crf for encoding and I noticed that with the same old crf the size became quite lot smaller(and the quality degrades also).

Is the crf settings got changed? I usually use crf 26, I want to know how much the crf changed so I can predict what crf I can use now.(is it 23?)

Hope someone can answer,
Thanks

[Dark Shikari]

There has been no significant change in CRF in the past 8 revisions.

[wyti]

And what about this (http://git.videolan.org/gitweb.cgi?p=x264.git;a=commitdiff;h=411ee507328890fb1fde96b37b6eef854d27101a) ? it should slightly reduce bitrate at the same CRF.

btw any news on planning release date of weighted P ? (weeks ? months ?) it's seems amazing and i really want to test it with Mb-tree (fades make me sick :P)

[LeonLanford]

There has been no significant change in CRF in the past 8 revisions.

Uh I'm using the same script and same file but I got about 30% size reduction and quality degrades also..

the old one

Video
ID : 1
Format : AVC
Format/Info : Advanced Video Codec
Format profile : High@L3.0
Format settings, CABAC : Yes
Format settings, ReFrames : 5 frames
Muxing mode : Container profile=Unknown@3.0
Codec ID : V_MPEG4/ISO/AVC
Duration : 2mn 59s
Bit rate : 289 Kbps
Width : 512 pixels
Height : 384 pixels
Display aspect ratio : 4:3
Frame rate : 29.970 fps
Resolution : 24 bits
Colorimetry : 4:2:0
Scan type : Progressive
Bits/(Pixel*Frame) : 0.049
Stream size : 6.19 MiB (85%)
Writing library : x264 core 75 r1251M e553a4c
Encoding settings : cabac=1 / ref=5 / deblock=1:0:0 / analyse=0x3:0x133 / me=umh / subme=6 / psy=1 / psy_rd=1.0:0.0 / mixed_ref=1 / me_range=16 / chroma_me=1 / trellis=1 / 8x8dct=1 / cqm=0 / deadzone=21,11 / chroma_qp_offset=-2 / threads=6 / nr=0 / decimate=1 / mbaff=0 / bframes=4 / b_pyramid=0 / b_adapt=2 / b_bias=0 / direct=3 / wpredb=1 / keyint=250 / keyint_min=25 / scenecut=40 / rc_lookahead=40 / rc=crf / mbtree=1 / crf=27.0 / qcomp=0.90 / qpmin=10 / qpmax=51 / qpstep=4 / ip_ratio=1.40 / aq=1:0.50


the new one

Video
ID : 1
Format : AVC
Format/Info : Advanced Video Codec
Format profile : High@L3.0
Format settings, CABAC : Yes
Format settings, ReFrames : 5 frames
Muxing mode : Container profile=Unknown@3.0
Codec ID : V_MPEG4/ISO/AVC
Duration : 2mn 59s
Bit rate : 209 Kbps
Width : 512 pixels
Height : 384 pixels
Display aspect ratio : 4:3
Frame rate : 29.970 fps
Resolution : 24 bits
Colorimetry : 4:2:0
Scan type : Progressive
Bits/(Pixel*Frame) : 0.036
Stream size : 4.49 MiB (81%)
Writing library : x264 core 75 r1259M dd026f2
Encoding settings : cabac=1 / ref=5 / deblock=1:0:0 / analyse=0x3:0x133 / me=umh / subme=6 / psy=1 / psy_rd=1.0:0.0 / mixed_ref=1 / me_range=16 / chroma_me=1 / trellis=1 / 8x8dct=1 / cqm=0 / deadzone=21,11 / chroma_qp_offset=-2 / threads=6 / nr=0 / decimate=1 / mbaff=0 / bframes=4 / b_pyramid=0 / b_adapt=2 / b_bias=0 / direct=3 / wpredb=1 / keyint=250 / keyint_min=25 / scenecut=40 / rc_lookahead=40 / rc=crf / mbtree=1 / crf=27.0 / qcomp=0.90 / qpmin=10 / qpmax=51 / qpstep=4 / ip_ratio=1.40 / aq=1:0.50

[Dark Shikari]

Source, exact commandline, and full x264 output (output stats) for both?

[LeonLanford]

Source, exact commandline, and full x264 output (output stats) for both?

I'm using avidemux, I don't know how to generate output stats or exact commandline. I only use custom job script to encode.
The source is chi's new address ep 68 (http://www.nyaatorrents.org/?page=download&tid=85708).
I also use other source and get the same reduction.

[Dark Shikari]

Does it occur if you use x264 itself natively instead of Avidemux?

[LeonLanford]

Does it occur if you use x264 itself natively instead of Avidemux?

I don't know how to use x264 without avidemux.

Anyway I just want to ask about if the crf scale got changed or not, but it seems it's avidemux problem? I'll just do some experiments to get my ideal crf again then.

Thanks for the reply..

[LoRd_MuldeR]

Anyway I just want to ask about if the crf scale got changed or not, but it seems it's avidemux problem?

Nope, the CRF scale didn't change noteworthy between r1251 and r1259. Also I see no such behavior with Avidemux :confused:

Just encoded the "Black Pearl" sample with Avidemux r5341 (Gruntster's build) with both, libx264 r1251 and libx264 r1259 (my own builds), using a CRF of 22.

As you can see the difference in total size is marginal:

22.09.2009 17:04 18.771.812 BlackPerl.x264-r1251.crf-22.avi
22.09.2009 16:57 18.679.516 BlackPerl.x264-r1259.crf-22.avi

That's a difference of less than 0.5% and therefore should be inside the expected fluctuations ;)


I don't know how to use x264 without avidemux.

http://forum.doom9.org/forumdisplay.php?f=78 :sly:

[Chengbin]

What does the upcoming weight p feature do?

Does it improve quality beside improving fades?

[burfadel]

I also take it weighted B frames will work with mb-tree once weight p comes in?

[Dark Shikari]

I also take it weighted B frames will work with mb-tree once weight p comes in?Weighted B-frames work just fine with MB-tree right now.

[martinfrombern]

ETA, for weighted p-frames?

[Chengbin]

ETA, for weighted p-frames?

Be patient.

It is probably already done, just at the stage of intense testing.

[burfadel]

Weighted B-frames work just fine with MB-tree right now.

Oops my mistake :)

[martinfrombern]

It is probably already done, just at the stage of intense testing.My intention is not to rush anybody, just wondering an estimate of when could it be released.
Of course it's better to test it before going public :)

[G_M_C]

Be patient.

It is probably already done, just at the stage of intense testing.

seeing all the post about it, its also at the state of intense expectations ;)

[kemuri-_9]

afaik, the GSoC student who did the weightp project has become very busy with other matters and as such there is no set time for it to get committed to the repository

[Sagekilla]

afaik, the GSoC student who did the weightp project has become very busy with other matters and as such there is no set time for it to get committed to the repository

That aside, isn't Dark more or less taking over all those students who dropped their projects anyway? IIRC, the student who was working on MBTree dropped it, so Dark took it over.

[Chengbin]

That aside, isn't Dark more or less taking over all those students who dropped their projects anyway? IIRC, the student who was working on MBTree dropped it, so Dark took it over.

Well, when they drop it, they don't get paid.

The student working on weight-p will still finish it, even though SoC is over. After all he IS getting $4500 to do this project, he should finish it.

[moviefan]

Is there at least any progress with the nal-hrd patch? It's ashame that Blu-ray compliance is not possible with the current revision... :(

[G_M_C]

It's now 7 days since the last commit on the GIT shortlog. I'm getting an anxious feeling Dark Shikari, Akupenguin and others are preparing for a big surprise and/or big bumb of the build number.

DS, can lift a bit of the veil without spoiling the surprise ... just a teaser of what's to come so to speak ?

;)

[Dark Shikari]

It's now 7 days since the last commit on the GIT shortlog. I'm getting an anxious feeling Dark Shikari, Akupenguin and others are preparing for a big surprise and/or big bumb of the build number.

DS, can lift a bit of the veil without spoiling the surprise ... just a teaser of what's to come so to speak ?

;)Well, at the moment, I'm getting a whole lot less crappy at Scarlet Weather Rhapsody (http://en.wikipedia.org/wiki/Scarlet_Weather_Rhapsody). I'm also finally getting around to watching Akagi (http://en.wikipedia.org/wiki/Akagi_(manga)). I played a pretty awesome game of Civilization 4: Beyond the Sword (http://en.wikipedia.org/wiki/Civilization_IV:_Beyond_the_Sword) where I got a Space Race victory in ~1850. I also upgraded my Touhou merchandise collection with one of these (http://i36.tinypic.com/v3288y.jpg). And I finally ordered a new laptop (http://www.dell.com/us/en/home/notebooks/laptop-studio-xps-16/pd.aspx?refid=laptop-studio-xps-16&s=dhs&cs=19) now that the Core i7 mobiles came out.

Wait, you mean x264? There were some intra-prediction-related optimizations that didn't pan out. And a media streaming company that wishes to remain anonymous is sponsoring some low-latency streaming optimizations for x264. But nothing important... :p

[juGGaKNot]

But nothing important.

cool, finaly working on 4:4:4!

[G_M_C]

Well, at the moment, I'm getting a whole lot less crappy at Scarlet Weather Rhapsody (http://en.wikipedia.org/wiki/Scarlet_Weather_Rhapsody). I'm also finally getting around to watching Akagi (http://en.wikipedia.org/wiki/Akagi_(manga)). I played a pretty awesome game of Civilization 4: Beyond the Sword (http://en.wikipedia.org/wiki/Civilization_IV:_Beyond_the_Sword) where I got a Space Race victory in ~1850. I also upgraded my Touhou merchandise collection with one of these (http://i36.tinypic.com/v3288y.jpg). And I finally ordered a new laptop (http://www.dell.com/us/en/home/notebooks/laptop-studio-xps-16/pd.aspx?refid=laptop-studio-xps-16&s=dhs&cs=19) now that the Core i7 mobiles came out.

Wait, you mean x264? There were some intra-prediction-related optimizations that didn't pan out. And a media streaming company that wishes to remain anonymous is sponsoring some low-latency streaming optimizations for x264. But nothing important... :p

Looks like a week well spent :cool:

[iwod]

@ DS
What do you think of the New Nvidia Chip? Any chance X264 getting something out of it?

[LigH]

The advantage of x264 (from the quality PoV) is just that it calculates the compressed content itself, instead of leaving that completely to an external engine it cannot control as detailed as necessary.

If a software will use a GPU internal compressor for AVC video, it won't be x264 anymore. And if it only uses marginal basic functions of video conversion, then it won't gain much speed -- the slowest part of the AVC compression is the intelligence to decide what kind and how much on which part of the video, and that can't be delegated to the outside easily.

At least it would be a challenge to port motion prediction and quantization adaption - in the complexity of x264 routines - to GPU shaders... :D

[benwaggoner]

If a software will use a GPU internal compressor for AVC video, it won't be x264 anymore. And if it only uses marginal basic functions of video conversion, then it won't gain much speed -- the slowest part of the AVC compression is the intelligence to decide what kind and how much on which part of the video, and that can't be delegated to the outside easily.
Yep, doing anything with refinement loops where the entire frame needs to be analyzed is pretty tricky to parallelize on a GPU. I imagine a trellis implementation would require slicing, for example, as would adaptive quantization.

Initially, I think GPUs will be most valuable for "good enough, faster." Particularly as we look at technologies like Smooth Streaming where 8-12 different versions of the same content need to be encoded live simultaneously, ideally in a single rack-mount box.

The biggest potential for quality gains in GPU, as I said before, is really in source decode and preprocessing, where things are much more determinant. ATSC/DVB stream reencoding is just exploding as a market, so a good deblocking MPEG-2 decoder would be very valuable, as would be high-quality resizing, noise reduction and deinterlacing/IVTC.

For those kinds of tasks, a GPU could do AVISynth like quality with real-time HD perf a lot easier than they could do x264 like quality, since the algorithmic mix is MUCH more tilted towards parallelizable DSP functions.

So, for those who want to see GPUs speed up x264 workflows, getting AVISynth and your favorite filters using the GPU is probably much lower hanging fruit. After all, all the CPU power saved from that goes straight to x264, and there's plenty of times where decode or preprocessing is the long pole anyway, particularly on 8-way machines.

[Dark Shikari]

Yep, doing anything with refinement loops where the entire frame needs to be analyzed is pretty tricky to parallelize on a GPU. I imagine a trellis implementation would require slicing, for example, as would adaptive quantization.Trellis is impossible on a GPU. You'd need to have each thread warp do a single trellis, which would be absurd.

AQ is trivial.

[Blue_MiSfit]

So, for those who want to see GPUs speed up x264 workflows, getting AVISynth and your favorite filters using the GPU is probably much lower hanging fruit. After all, all the CPU power saved from that goes straight to x264, and there's plenty of times where decode or preprocessing is the long pole anyway, particularly on 8-way machines.


fft3dgpu ;) - this sucker came out way back in '05 if memory serves. Now all we need is a good way to pass all this along via remote desktop in a way that doesn't break everything. Am I wrong in thinking that Windows 7 can do this?

If I could get MDegrain2 "for free" on the GPU, I'd be disgustingly happy.

But yes, hear hear - lots of the time on my 8 core machines, the decode / preprocess is the bottleneck - especially when using DGDecode or QuickTime. Single threaded decoding sucks :( The solution for now is to run as many encodes in parallel until I've saturated the CPUs or I/O.

~MiSfit

[benwaggoner]

fft3dgpu ;) - this sucker came out way back in '05 if memory serves. Now all we need is a good way to pass all this along via remote desktop in a way that doesn't break everything. Am I wrong in thinking that Windows 7 can do this?
Well, DirectX 11 certainly allows pixel shaders to pass frames back for further processing if thats what you mean. There's nothing architecturally challenging about doing this, but it's a whole lot of code that needs to get written.

If I could get MDegrain2 "for free" on the GPU, I'd be disgustingly happy.


But yes, hear hear - lots of the time on my 8 core machines, the decode / preprocess is the bottleneck - especially when using DGDecode or QuickTime. Single threaded decoding sucks :( The solution for now is to run as many encodes in parallel until I've saturated the CPUs or I/O.
Exactly. Only speeding up the bottlenecks actually speed up the end-to-end process. Windows 7 is doing some pretty amazing stuff with decode in both software and hardware, including a good bob deinterlace; maybe we need a MediaFoundationSource for AVISynth :).

The dream would be able to have bitstream to final YV12 all happen on GPU, leaving the CPU wide open for compression.

I could imagine some preanalysis stuff being done on GPU that could help x264. Delivering a good luma histogram for determining fade correction, for example. That should be trivally parallelizable and wouldn't need any feedback from the encoder.

There has been some interesting research on reusing source motion vectors as a hint for a transcode. It didn't help quality in the end, but was useful improving speed, as it basically served as an initial coarse motion search to be refined by the real encoer.

[Blue_MiSfit]

Well, DirectX 11 certainly allows pixel shaders to pass frames back for further processing if thats what you mean. There's nothing architecturally challenging about doing this, but it's a whole lot of code that needs to get written


I mean running anything that uses DirectX (or any hardware acceleration, actually) across remote desktop without totally breaking things. If you run fft3dgpu on a remote server, and RDP into it, things die :(

~MiSfit

[SeeManRun]

Hi Guys,
Trying to figure something out. I use the profile x264 HQ Slowest for my encodes, and I can barely tell the diff between the encode and a blu-ray movie, which is what I am going for. The problem is it takes a very long time to perform the encode, about 10 fps on first pass and 4 fps on second pass with quad core machine (q6600). I have just upgraded from x264 1183 to 1259, and now the second pass runs at nearly 12 fps, which is awesome. But I am curious as to what has been changed to give it such a speed boost and am wondering if the quality has been sacrificed.

Does someone know of the change that gave this crazy speed boost?

[RunningSkittle]

@ SeeManRun: http://git.videolan.org/?p=x264.git;a=shortlog;h=496d79dfcb90066f0254e07d593471f2c885a153

[me7]

What happened to www.x264.nl ?

[LoRd_MuldeR]

What happened to www.x264.nl ?

http://forum.doom9.org/showthread.php?t=149949

http://forum.doom9.org/showthread.php?t=149962

:search:

[SeeManRun]

@ SeeManRun: http://git.videolan.org/?p=x264.git;a=shortlog;h=496d79dfcb90066f0254e07d593471f2c885a153

Thanks, that is perfect. I see a couple CL's that could be this speed increase, but the performance has been dramatic, so whichever one, good work to the x264 devs!

[nm]

Thanks, that is perfect. I see a couple CL's that could be this speed increase, but the performance has been dramatic, so whichever one, good work to the x264 devs!
Your 3x speed increase is probably not caused by improvements in x264 but because something else changed at your end or your old x264 build was broken.

[dstln]

I'm not sure what x264-hq-slowest is, but judging things on your cpu and fps, it sounds to me like it was more something on the order of having tesa on previously and umh on now.

[DarkZell666]

I'm not sure what x264-hq-slowest is, but judging things on your cpu and fps, it sounds to me like it was more something on the order of having tesa on previously and umh on now.

It's an MeGUI preset, and AFAIK they've been converted for other GUI's lying around here, so it could be any of them ...
x264's cli options have been heavily modified on 07/07/09, so it could be the profile's option conflicts with x264's new defaults, if they weren't updated (or something along those lines, get the idea ?).

@SeeManRun, would you mind posting the x264 command line generated by MeGUI or <insert_your_GUI_here> with that preset so we can check the settings ?

[juGGaKNot]

Constrained intra prediction support, enable with --constrained-intra. Significantly reduces compression, but required for the base layer of SVC encodes and maybe some other use-cases.

Required for what ?

[LoRd_MuldeR]

Constrained intra prediction support, enable with --constrained-intra. Significantly reduces compression, but required for the base layer of SVC encodes and maybe some other use-cases.
Required for what ?

http://en.wikipedia.org/wiki/Scalable_Video_Coding

[VFR maniac]

Bug report.

Rev1283 crashes without B-frames.

[rack04]

Bug report.

Rev1283 crashes without B-frames.

That would probably be the reason for my crash. Reported here (http://forum.doom9.org/showthread.php?p=1333951#post1333951).

[pcordes]

I was looking over the git commits after updating my x264 repo (some people follow sports. I follow open source develpment for entertainment :).

I noticed something that could be improved on in commit 60c630c01dd4cb125a2bcc08da88873ce3a41dbc
(Optimize exp2fix8
Slightly faster and more accurate rounding.)

int i = foo + 512.5;
isn't as good as the C99 rounding functions which can be implemented more efficiently with one of the newer SSE float->int instructions.
int i = lrint(foo + 512);
may generate better code (with -ffast-math, so lrint is inlined). Although I haven't checked what the asm looks like in either case...

see the comments on
http://assemblyrequired.crashworks.org/2009/01/12/why-you-should-never-cast-floats-to-ints/

I tend to use lrint() for all my float->int conversions these days. On projects where I'm limited by MSVC compat requirements, I use some ugly code that prob. doesn't actually generate very good code with MSVC. But at least I can still write lrint(f) instead of (int)(f+0.5).


#ifdef _MSC_VER
// Windows math.h doesn't include C99 standard rounding functions :(
// workaround from https://svn.boost.org/trac/boost/ticket/2513
#if _MSC_VER < 1400
extern "C" { const double rint(double); }
#else
static inline double rint(double x) {
_asm FLD [x] ;
_asm FRNDINT ;
//_asm RET ;
}
#endif
static inline long lrint(double x){ return static_cast<long>(rint(x)); }
#endif


FYI, C99 has other useful rounding functions (although I don't know which of them are fast):
http://www.gnu.org/software/libc/manual/html_node/Rounding-Functions.html
http://linux.die.net/man/3/lrint

straying a little further off topic, see also the references at the bottom of
http://www.digitalmars.com/d/2.0/d-floating-point.html
for general floating-point goodness.

[akupenguin]

lrintf compiles to cvtss2si. (int) compiles to cvttss2si. No difference in speed or precision.

[Dark Shikari]

On GCC 3.4.5, lrintf compiles to:

29a: f3 0f 11 44 24 1c movss [esp+0x1c],xmm0
2a0: d9 44 24 1c fld dword[esp+0x1c]
2a4: db 5c 24 3c fistp dword[esp+0x3c]

Oh dear. :p

[me7]

What about weightp? Has it been put on hold or are you still waiting for the SoC student?
Don't mean to rush anyone, just asking for an update.

[LoRd_MuldeR]

What about weightp? Has it been put on hold or are you still waiting for the SoC student?
Don't mean to rush anyone, just asking for an update.

Status page:
http://wiki.videolan.org/SoC_2009/Weighted_P-frame_Prediction

Git Repository:
http://repo.or.cz/w/x264/x264-p-frames.git?a=shortlog

Looks like the project has already made some good progress. But there isn't much development going on lately, probably because SoC '09 has ended long ago (August 17th).

[me7]

Looks like the project has already made some good progress. But there isn't much development going on lately, probably because SoC '09 has ended long ago (August 17th).

Exactly, so does this mean that the feature has been put on hold or are the x264 devs waiting for the student to finish it?

[Chengbin]

Exactly, so does this mean that the feature has been put on hold or are the x264 devs waiting for the student to finish it?

The student is finishing it.

Unforunately university students are extremely busy. It will take a while.

I'm hoping it'll be done within a month. That's when I'll be re-encoding my huge collection of videos, DVDs, and some Blu-rays.

[CpT]

Possible bug report.

I have to disable B-Frame Pyramid even if mbtree is disabled or I get this error
Assertion failed: cost >= 0, file encoder/slicetype.c, line 1035

Tested with 1292 from here http://x264.nl/
and 1292 techouse > http://techouse.project357.com/

Running windows xp32
Gui's used, MeGui and sx264. Both give the same error.

I also tested all versions starting at 1286 though 1292. All give the same error. 1281 works.

[Dark Shikari]

Possible bug report.

I have to disable B-Frame Pyramid even if mbtree is disabled or I get this error
Assertion failed: cost >= 0, file encoder/slicetype.c, line 1035

Tested with 1292 from here http://x264.nl/
and 1292 techouse > http://techouse.project357.com/

Running windows xp32
Gui's used, MeGui and sx264. Both give the same error.

I also tested all versions starting at 1286 though 1292. All give the same error. 1281 works.Known issue, fixed locally already.

[CpT]

fast reply...

Thanks for the heads up ;)

[jpsdr]

Hello.
Actualy B-Frame Pyramid must be disabled if i use mbtree, for what i've understood.
Is something planed to make the two compatibles ?
If yes, is it scheduled for 2009 or not before 2010 ?
Just asking to know how much the wait could be, not to push anything.

[CpT]

Hello.
Actualy B-Frame Pyramid must be disabled if i use mbtree, for what i've understood.

Re-read my post ;)

[nurbs]

@jpsdr:
x264 | branch: master | Lamont Alston <wewk584 at gmail.com> | Mon Oct 12 23:32:16 2009 -0700| [e2659dbdc0aed2d2cd4f6538faddf370e7740ada] | committer: Jason Garrett-Glaser

Make B-pyramid spec-compliant
The rules of the specification with regard to picture buffering for pyramid coding are widely ignored.
x264's b-pyramid implementation, despite being practically identical to that proposed by the original paper, was technically not compliant.
Now it is.
Two modes are now available:
1) strict b-pyramid, while worse for compression, follows the rule mandated by Blu-ray (no P-frames can reference B-frames)
2) normal b-pyramid, which is like the old mode except fully compliant.
This patch also adds MMCO support (necessary for compliant pyramid in some cases).
MB-tree still doesn't support b-pyramid (but will soon).

[G_M_C]

The student is finishing it.

Unforunately university students are extremely busy. It will take a while.

I'm hoping it'll be done within a month. That's when I'll be re-encoding my huge collection of videos, DVDs, and some Blu-rays.

You shure ? Cause this answer is about the same answer the stundent posted himself, about a month ago.

I'm just hoping we get it as a christmas present :)

[Chengbin]

I couldn't help but notice x264 r1301 is a lot smaller, at 961KB, than revisions before, which were 1100KB.

[juGGaKNot]

--b-pyramid strict: Strictly heirarchical pyramid
--b-pyramid normal: Non-strict (not Blu-ray compatible)

1) strict b-pyramid, while worse for compression, follows the rule mandated by Blu-ray (no P-frames can reference B-frames)
2) normal b-pyramid, which is like the old mode except fully compliant.

So no more DXVA problems when using --b-pyramid normal + --b-pyramid strict for blu-ray right ?

What about hrd ? when will it be commited ?

When will the presets be updated to include --b-pyramid normal ? after mb-tree fix ?

[LoRd_MuldeR]

Improve CRF initial QP selection, fix get_qscale bug
If qcomp=1 (as in mb-tree), we don't need ABR_INIT_QP.
if( !h->param.rc.i_lookahead || h->param.i_keyint_max == 1 || h->param.rc.f_qcompress == 1 )
h->param.rc.b_mb_tree = 0;

The first implies MB-Tree uses qcomp=1 (or do I read it wrong?) while the second shows that qcomp=1 will disable MB-Tree. I'm confused :confused:

[MasterNobody]

LoRd_MuldeR
The real qcomp is "rc->qcompress" not the "h->param.rc.f_qcompress". So look at this:

if( h->param.rc.b_mb_tree )
{
h->param.rc.f_pb_factor = 1;
rc->qcompress = 1;
}
else
rc->qcompress = h->param.rc.f_qcompress;

[LoRd_MuldeR]

I see :)

[plonk420]

how hard would it be to make nehalem pass 1 use 4 cores (if hyperthreading is on)? ... or whatever is most optimal (like maybe "cores" 0, 2, 4, and 6)

[akupenguin]

taskset -c 0,2,4,6 x264 ...
(and x264's cpu detection takes that into account, so it'll default to 6 threads.)

[plonk420]

i'm assuming if you use 6 threads on pass 1, you *should* use 6 threads on future passes?

[Dark Shikari]

i'm assuming if you use 6 threads on pass 1, you *should* use 6 threads on future passes?Not particularly.

[prOnorama]

Hardly an "issue", but there's a small typo in the x264 --longhelp/--fullhelp commands:

--b-pyramid <string> Keep some B-frames as references [none]
- none: Disabled
- strict: Strictly heirarchical pyramid
- normal: Non-strict (not Blu-ray compatible)

heirarchical = hierarchical

[Dark Shikari]

Hardly an "issue", but there's a small typo in the x264 --longhelp/--fullhelp commands:



heirarchical = hierarchicalFixed locally.

[Groucho2004]

how hard would it be to make nehalem pass 1 use 4 cores (if hyperthreading is on)? ... or whatever is most optimal (like maybe "cores" 0, 2, 4, and 6)

x264 already chooses the most optimal number of threads because it is the most bestest h.264 encoder.
:):)

[Revgen]

So what's the plan on open GOP's? I saw a patch awhile ago, but haven't seen anything since.

Did the author go MIA?

[kemuri-_9]

So what's the plan on open GOP's? I saw a patch awhile ago, but haven't seen anything since.

Did the author go MIA?

no, from what i understand Trahald (wewk on IRC) is still working on it, but his MMCO patch was a higher priority so focus went to get it committed first.

the open gop patch also is going to need an overhaul with the latest commits afaik.

[easyfab]

Can I have a idea or better a list or link of the functions that are not implemented yet in x264 ( such as weightp ) and that are in H.264 specifications or compatible with H.264 speficications.

[seemees]

Dark Shikari
In a source code (for example analyse.c) we count many warnings and messages on compile with mingw environment.
Why do not clear code from this stuck? Example: array[t]+!!i; why using some weird constructions? What does it mean !!i?
And what can happen if we replace

const float x264_log2_lut[128] = {
0.00000, 0.01123, 0.02237, 0.03342, 0.04439, 0.05528, 0.06609, 0.07682,
0.08746, 0.09803, 0.10852, 0.11894, 0.12928, 0.13955, 0.14975, 0.15987,
.....
/* Avoid an int/float conversion. */
const float x264_log2_lz_lut[32] = {
31,30,29,28,27,26,25,24,23,22,21,20,19,18,17,16,15,14,13,12,11,10,9,8,7,6,5,4,3,2,1,0
};

with

const float x264_log2_lut[128] = {
0.00000f, 0.01123f, 0.02237f, 0.03342f, 0.04439f, 0.05528f, 0.06609f, 0.07682f,
0.08746f, 0.09803f, 0.10852f, 0.11894f, 0.12928f, 0.13955f, 0.14975f, 0.15987f,
.....
/* Avoid an int/float conversion. */
const float x264_log2_lz_lut[32] = {
31f,30f,29f,28f,27f,26f,25f,24f,23f,22f,21f,20f,19f,18f,17f,16f,15f,14f,13f,12f,11f,10f,9f,8f,7f,6f,5f,4f,3f,2f,1f,0f
};
I think that 0.3 is a double, and 0.3f is a float. Isn't it?
Please help me to understand politics of coding h264.
Why *.c but not *.cpp? :-)

Here is the example of warning messages:

gcc -O4 -ffast-math -Wall -I. -march=i686 -mfpmath=sse -msse -DHAVE_MMX -DARCH_X86 -DSYS_MINGW -DPTW32_STATIC_LIB -DHAVE_PTHREAD -s -fomit-frame-pointer -c -o encoder/macroblock.o encoder/macroblock.c
In file included from ./common/common.h:698,
from encoder/macroblock.c:25:
./common/macroblock.h: In function 'x264_mb_transform_8x8_allowed':
./common/macroblock.h:465: warning: dereferencing type-punned pointer will break strict-aliasing rules
encoder/macroblock.c: In function 'x264_macroblock_encode':
encoder/macroblock.c:893: warning: dereferencing type-punned pointer will break strict-aliasing rules
encoder/macroblock.c: In function 'x264_macroblock_probe_skip':
encoder/macroblock.c:918: warning: 'mvp[1]' may be used uninitialized in this function
encoder/macroblock.c:918: warning: 'mvp[0]' may be used uninitialized in this function



With best regards, seemees

[Dark Shikari]

Why *.c but not *.cpp? :-)To keep out people who don't understand basic coding constructs, like !!i ;)

[seemees]

Dark Shikari
Thank you. :-)

I understand.
!!0 = 1;
!!2 = 1;
!!3 = 1;
Does it +!!i means +1? :-)
Or it means +0? :-)
Or am I wrong?

But can we see clear and warning free code in the future? :-)
With best regards, seemees

[Guest]

If you think !!0 = 1 then you should take up knitting instead of coding, and move your queries to the Newbies forum.

[seemees]

Here is the full message list (from analyse.c and below):

encoder/analyse.c: In function 'x264_mb_analyse_inter_b8x16':
encoder/analyse.c:1999: warning: dereferencing pointer 'mvc.443' does break strict-aliasing rules
encoder/analyse.c:1999: note: initialized from here
encoder/analyse.c:2000: warning: dereferencing pointer '({anonymous})' does break strict-aliasing rules
encoder/analyse.c:2000: note: initialized from here
encoder/analyse.c:1999: warning: dereferencing pointer 'mvc.443' does break strict-aliasing rules
encoder/analyse.c:1999: note: initialized from here
encoder/analyse.c:2000: warning: dereferencing pointer '({anonymous})' does break strict-aliasing rules
encoder/analyse.c:2000: note: initialized from here
encoder/analyse.c: In function 'x264_mb_analyse_inter_b16x8':
encoder/analyse.c:1930: warning: dereferencing pointer 'mvc.436' does break strict-aliasing rules
encoder/analyse.c:1930: note: initialized from here
encoder/analyse.c:1931: warning: dereferencing pointer '({anonymous})' does break strict-aliasing rules
encoder/analyse.c:1931: note: initialized from here
encoder/analyse.c:1930: warning: dereferencing pointer 'mvc.436' does break strict-aliasing rules
encoder/analyse.c:1930: note: initialized from here
encoder/analyse.c:1931: warning: dereferencing pointer '({anonymous})' does break strict-aliasing rules
encoder/analyse.c:1931: note: initialized from here
encoder/analyse.c: In function 'x264_intra_rd_refine':
encoder/analyse.c:1104: warning: 'i_nnz[1]' may be used uninitialized in this function
encoder/analyse.c:1104: warning: 'i_nnz[0]' may be used uninitialized in this function
encoder/analyse.c:1103: warning: 'pels_v[6]' may be used uninitialized in this function
encoder/analyse.c:1103: warning: 'pels_v[5]' may be used uninitialized in this function
encoder/analyse.c:1103: warning: 'pels_v[4]' may be used uninitialized in this function
encoder/analyse.c:1103: warning: 'pels_v[3]' may be used uninitialized in this function
encoder/analyse.c:1103: warning: 'pels_v[2]' may be used uninitialized in this function
encoder/analyse.c:1103: warning: 'pels_v[1]' may be used uninitialized in this function
encoder/analyse.c:1103: warning: 'pels_v[0]' may be used uninitialized in this function
encoder/analyse.c: In function 'x264_macroblock_analyse':
encoder/analyse.c:1407: warning: dereferencing pointer 'mvc.380' does break strict-aliasing rules
encoder/analyse.c:1407: note: initialized from here
encoder/analyse.c:1408: warning: dereferencing pointer '({anonymous})' does break strict-aliasing rules
encoder/analyse.c:1408: note: initialized from here
encoder/analyse.c:1409: warning: dereferencing pointer '({anonymous})' does break strict-aliasing rules
encoder/analyse.c:1409: note: initialized from here
encoder/analyse.c:1456: warning: dereferencing pointer 'mvc.388' does break strict-aliasing rules
encoder/analyse.c:1456: note: initialized from here
encoder/analyse.c:1457: warning: dereferencing pointer '({anonymous})' does break strict-aliasing rules
encoder/analyse.c:1457: note: initialized from here
encoder/analyse.c:1458: warning: dereferencing pointer '({anonymous})' does break strict-aliasing rules
encoder/analyse.c:1458: note: initialized from here
In file included from ./common/common.h:698,
from encoder/me.c:25:
./common/macroblock.h: In function 'x264_mb_transform_8x8_allowed':
./common/macroblock.h:465: warning: dereferencing type-punned pointer will break strict-aliasing rules
In file included from ./common/common.h:698,
from encoder/ratecontrol.c:31:
./common/macroblock.h: In function 'x264_mb_transform_8x8_allowed':
./common/macroblock.h:465: warning: dereferencing type-punned pointer will break strict-aliasing rules
In file included from ./common/common.h:698,
from encoder/set.c:26:
./common/macroblock.h: In function 'x264_mb_transform_8x8_allowed':
./common/macroblock.h:465: warning: dereferencing type-punned pointer will break strict-aliasing rules
In file included from ./common/common.h:698,
from encoder/macroblock.c:25:
./common/macroblock.h: In function 'x264_mb_transform_8x8_allowed':
./common/macroblock.h:465: warning: dereferencing type-punned pointer will break strict-aliasing rules
encoder/macroblock.c: In function 'x264_macroblock_encode':
encoder/macroblock.c:893: warning: dereferencing type-punned pointer will break strict-aliasing rules
encoder/macroblock.c: In function 'x264_macroblock_probe_skip':
encoder/macroblock.c:918: warning: 'mvp[1]' may be used uninitialized in this function
encoder/macroblock.c:918: warning: 'mvp[0]' may be used uninitialized in this function
encoder/macroblock.c: In function 'x264_macroblock_encode':
./common/macroblock.h:413: warning: dereferencing pointer 'x' does break strict-aliasing rules
./common/macroblock.h:407: note: initialized from here
In file included from ./common/common.h:698,
from encoder/cabac.c:25:
./common/macroblock.h: In function 'x264_mb_transform_8x8_allowed':
./common/macroblock.h:465: warning: dereferencing type-punned pointer will break strict-aliasing rules
In file included from ./common/common.h:698,
from encoder/cavlc.c:25:
./common/macroblock.h: In function 'x264_mb_transform_8x8_allowed':
./common/macroblock.h:465: warning: dereferencing type-punned pointer will break strict-aliasing rules
In file included from ./common/common.h:698,
from encoder/encoder.c:27:
./common/macroblock.h: In function 'x264_mb_transform_8x8_allowed':
./common/macroblock.h:465: warning: dereferencing type-punned pointer will break strict-aliasing rules
encoder/encoder.c: In function 'x264_slice_write':
encoder/encoder.c:1303: warning: 'bs_bak.p_start' may be used uninitialized in this function
encoder/encoder.c:1303: warning: 'bs_bak.p' may be used uninitialized in this function
encoder/encoder.c:1303: warning: 'bs_bak.p_end' may be used uninitialized in this function
encoder/encoder.c:1303: warning: 'bs_bak.cur_bits' may be used uninitialized in this function
encoder/encoder.c:1303: warning: 'bs_bak.i_left' may be used uninitialized in this function
encoder/encoder.c:1303: warning: 'bs_bak.i_bits_encoded' may be used uninitialized in this function
In file included from ./common/common.h:698,
from encoder/lookahead.c:37:
./common/macroblock.h: In function 'x264_mb_transform_8x8_allowed':
./common/macroblock.h:465: warning: dereferencing type-punned pointer will break strict-aliasing rules
In file included from ./common/common.h:698,
from common/x86/mc-c.c:29:
./common/macroblock.h: In function 'x264_mb_transform_8x8_allowed':
./common/macroblock.h:465: warning: dereferencing type-punned pointer will break strict-aliasing rules
In file included from ./common/common.h:698,
from common/x86/predict-c.c:24:
./common/macroblock.h: In function 'x264_mb_transform_8x8_allowed':
./common/macroblock.h:465: warning: dereferencing type-punned pointer will break strict-aliasing rules


How can fix it? Please help. :-)
With best regards, seemees

[Dark Shikari]

Use a gcc that doesn't suck (i.e. not 4.4.x).

(We do intend to fix those warnings eventually, but the fact that the C specification is rather braindead with respect to aliasing rules doesn't help.)

[seemees]

Dark Shikari
Thank you.
What gcc version we can use to construct not only 32bit but 64bit x264.exe?
I see in SSE4.1 SAD instruction with 8 sad at one core cycle. Do you plan to use it to remake sad procedure in SSE4.1?
2.2.9 Improved Sums of Absolute Differences (SAD) for 4-Byte
Blocks
SSE4.1 adds an instruction (MPSADBW) that performs eight 4-byte wide SAD operations
per instruction. Compared to PSADBW, MPSADBW operates on smaller chunks
(4-byte instead of 8-byte chunks); this makes the instruction better suited to video
coding standards such as VC.1 and H.264. MPSADBW performs four times the
number of absolute difference operations than that of PSADBW (per instruction). This
can improve performance for dense motion searches.
MPSADBW uses a 4-byte wide field from a source operand; the offset of the 4-byte
field within the 128-bit source operand is specified by two immediate control bits.
MPSADBW produces eight 16-bit SAD results. Each 16-bit SAD result is formed from
overlapping pairs of 4 bytes in the destination with the 4-byte field from the source
operand. MPSADBW uses eleven consecutive bytes in the destination operand, its
offset is specified by a control bit in the immediate byte (i.e. the offset can be from
byte 0 or from byte 4). Figure 2-1 illustrates the operation of MPSADBW. MPSADBW
can simplify coding of dense motion estimation by providing source and destination
offset control, higher throughput of SAD operations, and the smaller chunk size.

With best regards, seemees

[Dark Shikari]

MPSADBW is completely useless. It's not actually much faster than regular SAD (all the benchmarks vs regular SAD were not done with cacheline-split-avoiding regular SAD), and it's only useful in an exhaustive search. And Sequential Elimination is over 5 times faster for an exhaustive search than MPSADBW is.

[seemees]

Dark Shikari
Understand.
Thanks again.
With best regards, seemees

[jakor]

BTW are there actually useful instructions for video encoding in SSSE3, SSE4.1, SSE 4.2 whatsoever ?
I happenned to use one instruction of SSE3 once though - it was a kind of useful packed multiply, but in overall it seemes that intel engineers ran out of ideas since SSE2 which operations can be useful for coding people....

[Dark Shikari]

BTW are there actually useful instructions for video encoding in SSSE3, SSE4.1, SSE 4.2 whatsoever ?
I happenned to use one instruction of SSE3 once though - it was a kind of useful packed multiply, but in overall it seemes that intel engineers ran out of ideas since SSE2 which operations can be useful for coding people....
SSSE3: pabsw, psignw, palignr, pmaddubsw
SSE4.1: pblendw, and the new multiply instructions are useful in various cases related to LPCs (but not in x264). the insert/extract ones are sometimes marginally useful.
SSE4.2: useless crap

[akupenguin]

How can you forget pshufb?

[_Woland_]

It seems to be a bug with --b-pyramid "strict". For example:
http://s44.radikal.ru/i105/0910/32/fc34bd152189t.jpg (http://radikal.ru/F/s44.radikal.ru/i105/0910/32/fc34bd152189.png.html)
Here is sample - http://narod.ru/disk/14401053000/Futurama_S01E01_Trim.mkv.html - problems starts from ~1:20.
I hope it will be useful. =)

[Dark Shikari]

Bug confirmed; JM agrees. Forwarded to Trahald.

There's also corruption around ~1:47.

[jpsdr]

Hello.
I don't know where to ask, so i'm asking this here. I've look at the options given in the full help, and unless i've missed
something, there is no such option i'm asking.
Is it possible to have in the future an option to add chapters (final purporse is of course for blu-ray authoring) on
the encoded stream (like it's possible on CineVision).
What could be done is something like :
--chapters file.txt
Chapters could be in format like : HH:MM:SS:CC (hours,minutes,seconds, 100/th seconds) => 01:20:56:89 => chapter
at 1h 20min 56.89s (Or anything like this).
In the files, the chapters will simply be like this :
00:20:56:05
00:35:10:68
01:00:05:54
etc...
If this already exist or is possible on another way, i'm willing to know it.

[detmek]

That is a job for muxer and all muxers (mkvmerge, tsmuxer, mp4box) have that option.

[shon3i]

No x264 have --qpfile option to force frame types. To make qpfile which is text file with framenubers and frametypes i recomend you to use ChapterGen or Chapter Grabber tools. Btw. Cinevision use HH:MM:SS:ff format

[jpsdr]

@detmek : If you want to have the chapters where you want, you have to do it during the encoding process, otherwise, you may not be able to put the chapter at the exact frame you want, you can't put chapter on any type of frame.
@shon3i (or anyone else who know) : I've look at the QP file option... For chapters, what kind of frametype it should be ?
I first think I, but is it the best ? Is there a better choice ? I think QP to -1 is the best to leave ?

[J_Darnley]

Adding the actual chapter metadata is the job the the muxer. If you want to seek instantly to a chapter point, it will have to be an IDR frame (I in the qpfile) at that time. Yes, if you don't want to force the quantiser, set it to -1 and let x264 decide as it normally would.

[jpsdr]

Well, in Scenarist at least, you can't put chapter on any kind of frame, so, you've to make sure during the encoding
process, that frames where you'll put chapter are compatible. Maybe there is not these restrictions on muxer
for pc files (mkvtoolnix or other).
Anyway, thanks for these informations.

[shon3i]

I first think I, but is it the best ? Is there a better choice ? I think QP to -1 is the best to leave ? That's why i recomend to use some of apps to generate qpfile from some chapter source, if you reencode.

[jpsdr]

@shon3i : The purpose is to purely encode, not re-encode, so, I put the chapters on a lagarith YV12 avi file which will be encoded.
The chapters on my file in the first place didn't exist, so chapter source didn't exist. Nevetheless, i'll take a look a the tools you're
talking, to see what they can do.
Last question : frame number in qpfile, is frame begin to 0 (like in VDub) or 1 ?
If i decide to put a chapter on frame number 999 given by VDub (wich so is in fact the 1000th frame). The frame number parameter
of the qpfile, should it be 999 or 1000 ?

[Dark Shikari]

@shon3i : The purpose is to purely encode, not re-encode, so, I put the chapters on a lagarith YV12 avi file which will be encoded.
The chapters on my file in the first place didn't exist, so chapter source didn't exist. Nevetheless, i'll take a look a the tools you're
talking, to see what they can do.
Last question : frame number in qpfile, is frame begin to 0 (like in VDub) or 1 ?
If i decide to put a chapter on frame number 999 given by VDub (wich so is in fact the 1000th frame). The frame number parameter
of the qpfile, should it be 999 or 1000 ?Frames in x264 begin at 0.

[jpsdr]

Thanks DS.

@shion3 : As i was afraid, the tools you're talking about are totaly useless for me, as they extract chapters from files wich already have them, and that's absolutely not my case, as i create them. So, notepad method !

But with all of this, i think i've got now all the informations i need, and thanks to those who have answered to me,
and i'll close this wich may be off topic of this thread finaly.

[bob0r]

I couldn't help but notice x264 r1301 is a lot smaller, at 961KB, than revisions before, which were 1100KB.

I accidentally deleted a wrong directory.
x264 did not fprofile, but did however compile.
This is fixed with revision 1309.
New fprofile line:
make fprofiled VIDS="../1280x720p50_parkrun_ter.yuv";

Thanks!

[thewebchat]

x264 r1310 HEAD
http://img294.imageshack.us/img294/4742/noweightp.png

x264 r1301 WEIGHTP
http://img294.imageshack.us/img294/560/weightp.png

2pass encode. ABR 2500 kb/s. Good work, guys. :devil:

[Chengbin]

Weight-p??????? It is not even committed yet.

[thewebchat]

Doesn't mean you can't use it.

[Chengbin]

Doesn't mean you can't use it.

Can't figure it out.

But I must say, that's very impressive!

[poisondeathray]

How were you able to use it?

What about other frames? Did the other ones that were initially improved with mb-tree get worse?

[thewebchat]

Compiled the GIT GSoC repository (well, JEEB did). For some reason, there doesn't seem to be a way to turn weightp off, so the reference was generated with x264 HEAD. I did not test against MBTree from HEAD, but MBTree still produces crappy fades with weightp on. Without MBTree, quality is all-around better (since this is a pure efficiency advance), as bits that would have been wasted on fades can be put on keyframes and other scenes. Test clip was an artificial composite of 3 dark fadeouts and 1 low-motion bright scene.

[burfadel]

The AQ mod patch works well with fades, maybe a combination of the AQ mod patch (with the setting 1.0:1.0 as standard) combined with weightp will give well balanced results?...

[Puncakes]

Compiled the GIT GSoC repository (well, JEEB did).

Where'd you get that build? Obviously the site is... uninformative, and I don't see any posts here about it...

[thewebchat]

http://repo.or.cz/w/x264/x264-p-frames.git?a=shortlog;h=refs/heads/gsoc

There seems to have been a few useful updates since I posted that screenshot, one of them being labeled "mbtree stuffs," so it seems that this is getting somewhere. I believe JEEB told me that compilation fails on Win64 though.

[Puncakes]

Thanks! Now to get the slightest clue on how to compile it. Google, here I come.

[Wishbringer]

@Puncakes:

Get Komisars MinGW http://komisar.gin.by/mingw/index.html / and Tools

Get MSys Phoenix http://www.cadforte.com/msys.html

Get YASM http://www.tortall.net/projects/yasm/wiki/Download
and you've done the first step.

[wyti]

You can compile it under win64 but you have to use --disable-asm

error log if someone is interested :
common/x86/mc-a.asm:395: error: cannot override register size
common/x86/mc-a.asm:396: error: cannot override register size
common/x86/mc-a.asm:397: error: cannot override register size
common/x86/mc-a.asm:398: error: cannot override register size
common/x86/mc-a.asm:399: error: cannot override register size
common/x86/mc-a.asm:400: error: cannot override register size
common/x86/mc-a.asm:402: error: cannot override register size
common/x86/mc-a.asm:403: error: cannot override register size
common/x86/mc-a.asm:404: error: cannot override register size
common/x86/mc-a.asm:405: error: cannot override register size
make[1]: *** [common/x86/mc-a.o] Error 1

so it isn't interesting to have an x64 build without asm :p

but here is the build for win32 fprofiled build if interested : http://www.mediafire.com/download.php?dtdxzk4d0nl (note : mp4 and mkv oupture not activated, it can only produce elementary streams (you will have to mux) but avisynth input enabled)

[easyfab]

to complete Wishbringer post : How to compile x264

http://forum.doom9.org/showthread.php?t=148615

[moviefan]

Concerning custom builds including a weightp patch: Where can I get the latest revision of the patch and where can I find the current development status of it to know if there are major issues or not?

[wyti]

http://repo.or.cz/w/x264/x264-p-frames.git?a=shortlog;h=refs/heads/gsoc

it's not a patch but a git you will find here all you need (a complete x264 with weighted p

[moviefan]

OK, sound good and easy to do. How about the quality of the modification? Is it reliable?

Edit: Sorry, I just noticed the website on videolan.org about the weightp patch.

[Snake91]

but here is the build for win32 fprofiled build if interested : http://www.mediafire.com/download.php?dtdxzk4d0nl (note : mp4 and mkv oupture not activated, it can only produce elementary streams (you will have to mux) but avisynth input enabled)
It uses only one core and if I forced with --threads 6 x264 turns with the warning:"not compiled with pthread support" . May you compile a test-build with pthread? :)

[XhmikosR]

./configure --extra-cflags="-march=core2"
Platform: X86
System: MINGW
asm: yes
avis input: yes
mp4 output: yes
pthread: yes
debug: no
gprof: no
PIC: no
shared: no
visualize: no

You can run 'make' or 'make fprofiled' now.

make fprofiled VIDS="SOCCER_352x288_30_orig_02.yuv"
GCC 4.4.1, unpatched (despite the "M")

Download here (http://www.mediafire.com/?zzqktjoowot). (Built on 11/01/2009)

[Snake91]

thx a lot

[CpT]

When using the above build is it automatically on or is there a command requirement to activate it?

And Thanks for the build btw :thanks:

[RainyDog]

So, am I right in thinking that if I download JEEB's 64bit patched 1318 build from here http://jeeb.fiveforty.jp/x264/1309_x64/x264.exe, weightp will be on as default? Thanks.

[thewebchat]

@CpT: When I tried, it printed something like [Weighted P frames: 40.9%] regardless of if I actually specified --weightp. Both modes seem to do the same thing (?).

@RainyDog: No.

[CpT]

-thewebchat
Ahh yep just saw it in the output log.

I take it the 2 pics from the previous page were done with mbtree on?

[RainyDog]

@CpT: When I tried, it printed something like [Weighted P frames: 40.9%] regardless of if I actually specified --weightp. Both modes seem to do the same thing (?).

@RainyDog: No.

Ok, thanks. Will it be on if I specify --weightp?

[RainyDog]

I take it the 2 pics from the previous page were done with mbtree on?

I think they were done with mbtree turned off if I'm interpreting his post near the top of the previous page correctly.

[XhmikosR]

Why don't you run x264M.x86.core2.pframes --fullhelp to see what options are available?

[thewebchat]

The message from --fullhelp doesn't indicate it, but the default value of --weightp is 2, and 0 can be used to disable it. It does not seem to have a significant impact on speed (maybe ~10%).


From x264.h:
98 #define X264_WEIGHTP_NONE 0
99 #define X264_WEIGHTP_BLIND 1
100 #define X264_WEIGHTP_SMART 2

From common.c defaults declaration:
139 param->analyse.i_weighted_pred = X264_WEIGHTP_SMART;

[Chengbin]

-- weightp 3 is available too.

[thewebchat]

No, it's not.

11 days ago Dylan Yudaken remove kmeans

[CpT]

Yea I can't tell a diff between weightp 1 and 2.

weightp 0 is slightly faster but its such a little difference its hardly noticeable. I can't wait for the full commit wewt!

[moviefan]

If I compile from this GIT: http://repo.or.cz/w/x264/x264-p-frames.git, is weightp enabled by default and in case which weightp option is enabled? (1 or 2?) There is no entry in the --fullhelp about the --weigthp option.

[XhmikosR]

The answer is 4 posts above. If you want an unpatched compiled build, you can try this (http://www.mediafire.com/?zzqktjoowot) (built 11/01/2009).

[shon3i]

Can somebody compile last with weightp, nalhrd r16, and bugmaster AQmod, to test what is best for fades?

[moviefan]

Do you want it fprofiled or is it fine without for testing purposes?

[XhmikosR]

If you give me the links to nalhdr16 and AQmod I could do it.

[moviefan]

http://komisar.gin.by/x.patch/BugMaster/20090926/x264_AQ2mod.diff (I guess)
http://h264enc.sourceforge.net/x264_hrd_pd_interlace.16_r1301.diff

[XhmikosR]

x264 patched built from http://repo.or.cz/w/x264/x264-p-frames.git
./configure --extra-cflags="-march=core2"
Platform: X86
System: MINGW
asm: yes
avis input: yes
mp4 output: yes
pthread: yes
debug: no
gprof: no
PIC: no
shared: no
visualize: no

You can run 'make' or 'make fprofiled' now.

make fprofiled VIDS="SOCCER_352x288_30_orig_02.yuv"
GCC 4.4.1, patched with:

AQ2Mod (http://komisar.gin.by/x.patch/BugMaster/20090926/x264_AQ2mod.diff)
HDR_PD_interlace.16_r1301 (http://komisar.gin.by/x.patch/last.used/x264_hrd_pd_interlace.16_r1301.diff)
x264_win_zone_parse_fix_06


Download patched build here (http://www.mediafire.com/?3yzg1jfmgmg). (Built on 11/01/2009)

Download unpatched build here (http://www.mediafire.com/?zzqktjoowot). (Built on 11/01/2009)

[moviefan]

@XhmikosR: When you look at the compile log in the terminal while compiling, do you get a lot of warnings about dereferencing pointers etc.? I get very similar information like in this post: http://forum.doom9.org/showthread.php?p=1310293#post1310293. I read through the thread, but I couldn't find a reply concerning the warnings...

[XhmikosR]

Yes, lots of warnings. But I always get them since I can remember. There was recently a post by Dark Shikari in which he explained it a little, but I can't find it.

[moviefan]

OK, good to know as I did my first own build of x264 yesterday and got confused by the warnings... However if this is a known "issue" and my build works fine despite warnings, I am relieved everything is as expected.

[LoRd_MuldeR]

Yes, GCC 4.x.x throws hundreds of warning when compiling x264. That's well known. As far as I can tell, you can safely ignore them...

[thewebchat]

There were just a few updates to mc-a.asm up at the GIT tree. Anyone want to try building on Win64 again? Also, I'm pretty sure that AQ2Mod 1) sucks and 2) is definitely useless with weightp (this was tested).

[shon3i]

Also, I'm pretty sure that AQ2Mod 1) sucks and 2) is definitely useless with weightp (this was tested). I am testing right now. Anyway AQmod show help in other cases too, like dark frames etc.

[Arshad07]

Why am i getting this error using latest x264?

http://i37.tinypic.com/r93hpk.jpg

[shon3i]

MeGUI can't handle x264 propertly. So don't use MeGUI for x264 until new version is out.

[J_Darnley]

Paste the command line in future. You have not used a correct command line, the argument for --b-pyramid is "--direct". I would say MeGUI needs updating to work with the latest B-pyramid commit.

[XhmikosR]

Why am i getting this error using latest x264?

http://i37.tinypic.com/r93hpk.jpg

Because you are using a non-latest MeGUI build. But anyway, latest MeGUI has lots of bugs. Try another GUI until MeGUI is fixed.

EDIT: lol I'm very slow:p

[moviefan]

Is there any explanation for the following: I encoded a 1080p clip with --vbv-maxrate 15000 --vbv-bufsize 30000 --keyint 48 --min-keyint 2 and when I analyze the output stream with Elecard Stream Eye, the info box shows

bitrate declared : 15 000 000
--------------------------------------------
real max : 17 844 411
real avg : 8 717 175
real min : 3 532 232


real max is greater than --vbv-maxrate so that Blu-ray compliancy is broken considering the large GOP I used, right? I encoded with the latest build.

[shon3i]

real max is greater than --vbv-maxrate so that Blu-ray compliancy is broken considering the large GOP I used, right? I encoded with the latest build.No that is because you use --vbv-bufsize 30000. Highest peak is 30mbps, everything is fine.

[moviefan]

So please help me understand: what does --vbv-maxrate do then? I thought it limits the max bitrate to 15000 kbps.

[LoRd_MuldeR]

So please help me understand: what does --vbv-maxrate do then? I thought it limits the max bitrate to 15000 kbps.

This has been explained a million times! The "--vbv-maxrate" option doesn't have a meaning without the "--vbv-bufsize" option specified as well.

And it doesn't directly limit the maximum bitrate of the video. It limits the maximum rate at which data can enter the buffer in the VBV model.

:search:

[juGGaKNot]

The answer is 4 posts above. If you want an unpatched compiled build, you can try this (http://www.mediafire.com/?zzqktjoowot) (built 11/01/2009).

Looks great, better than aq2mod at 1.0:1.0 and faster than without ( weird no ? ) tried mode 2.

Why is it not done yet ? what specific problems i want to know, i'm thinking of using it now and not waiting for the final release.

I am testing right now. Anyway AQmod show help in other cases too, like dark frames etc.

It helps a bit ( not as much as a weightp build ) but ratefactor is bigger by 2 on my source ( 20 compared to 18 )

[thewebchat]

Uh, maybe it's not done yet because it's not done? I'm pretty sure that it's not working 100% optimal, as I've found some fades to white are actually worse now.

Edit: Crossfades too.

[Japhsoncross]

i tested several revisions of weightp with several kinds of fades, it's really exciting.
i think crossfades are hard to distinguish between motion and fade.

[G_M_C]

Peeps, just pe patient; It'll get there :)

Pengvado review status = 1

:p

[kemuri-_9]

that was only pengvado/akupenguin's first review, there are more to come.
also weightp 2 is broken afaik.

but it is coming along.

[moviefan]

Is weightp only broken in terms of the visual output and/or in terms of spec compliancy?

[kemuri-_9]

the issues i recall being stated surrounding weightp being broken (haven't seen any confirmation of fixes) were

A) 2pass encoding does not work to a miss constructed check under some situations which i don't recall offhand (x264 errors out and doesn't start processing, so you'll know it if you come across it).
B) weightp 2 triggers an incredibly large number of scenecuts (when scenecut is active) which will cause a quality drop.

on a side note, this email (http://mailman.videolan.org/pipermail/x264-devel/2009-November/006502.html) is relevant.
though it was stated as 'few days' in above email, pengvado/akupenguin is now on vacation for a while
and weightp will not commit until it has his full approval.

[LoRd_MuldeR]

CoreAVC broken with "weight-p" in default mode. D'oh! Hope they get the fix out very soon :D

[Snake91]

x264+pframes doesn't give any error when compiling in 64bit mode now:)

[shon3i]

So that mean, can also have complication with standalone devices, similar like b-pyramid?I have no issues with Ateme's weightp

[G_M_C]

Woot:
Multiple additions / fixes per day on the weight-p/GSOC page. Looks like Dylan Yudaken is flying now. Amazing stuff, I wish him much success and good luck for getting it done :)

[juGGaKNot]

http://mirror01.x264.nl/x264/revision1330/x264.exe

x264.exe --preset veryslow --level 3.2 --ref 5 --min-keyint 30 --keyint 300 --bframes 3 --b-pyramid normal --merange 32 --sar 1:1 --aud

http://i286.photobucket.com/albums/ll105/juGGaKNot4cs/th_1330error.png (http://s286.photobucket.com/albums/ll105/juGGaKNot4cs/?action=view&current=1330error.png)

cabac=1 / ref=5 / deblock=1:0:0 / analyse=0x3:0x133 / me=umh / subme=10 / psy=1 / psy_rd=1.0:0.0 / mixed_ref=1 / me_range=32 / chroma_me=1 / trellis=2 / 8x8dct=1 / cqm=0 / deadzone=21,11 / chroma_qp_offset=-2 / threads=3 / nr=0 / decimate=1 / mbaff=0 / constrained_intra=0 / bframes=3 / b_pyramid=0 / b_adapt=2 / b_bias=0 / direct=3 / wpredb=1 / wpredp=2 / keyint=300 / keyint_min=30 / scenecut=40 / rc_lookahead=60 / rc=2pass / mbtree=1 / bitrate=4300 / ratetol=1.0 / qcomp=0.60 / qpmin=10 / qpmax=51 / qpstep=4 / cplxblur=20.0 / qblur=0.5 / ip_ratio=1.40 / aq=1:1.00

[Audionut]

With my unpatched build,

--tune grain --crf 18 --keyint 240 --min-keyint 24 -o

No problems here so far, 125000/158075 frames.

win7 64bit - q6600.