Current Patches, Where to get them, How they affect speed/output

[alexVS]

I have a question that may be very simple, may be not
I use VFW version of h264 encoder for using in TMpg Express 4.0 Encoder.
I use default settings, just set up ABR mode and bitrait.

What h264 codec settings can I change to get much better output quality WITHOUT big decrease of encoding speed?
I've got not a superb CPU. Only Athlon64 X2 4000+

[bmnot]

Why this patch was not accepted in main code?


HD HRD/Pulldown Patch:
Current: x264_hrd_pulldown.diff
Creator: Ian Caulfield/Trahald
Description: HRD and pulldown for HD compatibility.


Sorry, I didn't read 70 pages, but I am seeing it in many alternative builds.

[J_Darnley]

Why this patch was not accepted in main code?
Sorry, I didn't read 70 pages, but I am seeing it in many alternative builds.
:search: http://forum.doom9.org/showthread.php?p=1086281&highlight=hrd#post1086281

Granted it was not in this thread but it wasn't hard to find by searching for posts by akupenguin that contain hrd

[MasterNobody]

I have a question that may be very simple, may be not
I use VFW version of h264 encoder for using in TMpg Express 4.0 Encoder.
I use default settings, just set up ABR mode and bitrait.

What h264 codec settings can I change to get much better output quality WITHOUT big decrease of encoding speed?
I've got not a superb CPU. Only Athlon64 X2 4000+
This is bad thread for VfW discussion (better here (http://forum.doom9.org/showthread.php?t=98247)). If you mean x264vfw then I would recommend to change this options (they are not optimal by default but there is reason why they used):
Max consecutive B-frames: 0 -> 3
Threads: 1 -> 0
Max frame refs: 1 -> 3
BUT this changes would lead to drop of few last frames (and may be video vs audio desync). And you can't use "VirtualDub hack" to fix this problems because TMPGEnc doesn't support it.

[burfadel]

I thought www.x264.nl was self updating? I see there's rev 1058 up on git for 12 hours and www.x264.nl still shows build 1057...

I don't use ESA for which the patch pertains to, however I'm still curious why it hasn't self updated in case the same thing happens later! :)

[LoRd_MuldeR]

I thought www.x264.nl was self updating? I see there's rev 1058 up on git for 12 hours and www.x264.nl still shows build 1057...

I don't use ESA for which the patch pertains to, however I'm still curious why it hasn't self updated in case the same thing happens later! :)

Maybe the compile script failed? :p

[EDIT]

Yup, something seems to be wrong with r1058 :eek:

This happens with "make fprofiled" on my system:

http://img411.imageshack.us/img411/6885/x264crashja4.th.png (http://img411.imageshack.us/img411/6885/x264crashja4.png)

[EDIT²]

Seems like "-me esa" causes a crash, as soon as the encoding process reaches 100%.

Can't post backtrace, because GDB encounters an "internal error" right at the beginning with my r1058 debug build :confused:

[burfadel]

At a guess I'd presume the output file is fine? It may be related to the issue that I and someone else mentioned a few days ago under the x64 x264 thread, that rarely and unreproduceably the x86 x264 crashes right at the end, exactly the same fault as in your screenshot!

[LoRd_MuldeR]

At a guess I'd presume the output file is fine? It may be related to the issue that I and someone else mentioned a few days ago under the x64 x264 thread, that rarely and unreproduceably the x86 x264 crashes right at the end, exactly the same fault as in your screenshot!

Since it only happens with "--me esa" and didn't happen in previous revision, I suspect the latest commit...

[Audionut]

No problems here.


x264-r1058.rar (http://rapidshare.com/files/175769647/x264-r1058.rar)


Patched with,
x264_hrd_pulldown.09_interlace.diff
x264_mingw_aligned_03.diff
x264_win_zone_parse_fix_04.diff

[LoRd_MuldeR]

No problems here.


x264-r1058.rar (http://rapidshare.com/files/175769647/x264-r1058.rar)


Patched with,
x264_hrd_pulldown.09_interlace.diff
x264_mingw_aligned_03.diff
x264_win_zone_parse_fix_04.diff

Nope. Identical problem with that build:

http://img254.imageshack.us/img254/1669/x264crash2pnguj9.th.jpg (http://img254.imageshack.us/img254/1669/x264crash2pnguj9.jpg)

[Audionut]

Well I had no problems with a simple command line such as yours, with a 300 frame yuv or a 9000 frame avs.

But I am getting a crash with a more complex command line.
Still testing.

edit: ok, i've nailed it down to the input parameter.


edit2. No it's not. It's the resolution making it crash.

width of 960 and under works
962 = unsupported input format
964 = crash.

[LoRd_MuldeR]

Well, it crashed for me with a 352x288 sample, raw YUV data. And only with the "--me esa" parameter...

[Audionut]

http://img72.imageshack.us/img72/1654/x264qs4.png

[LoRd_MuldeR]

Any ideas what to do with that ???

C:\TEMP\x264\x264>gdb --args x264.exe --crf 22 --me esa --output NUL --progress c:\temp\x264\sample.avs
GNU gdb 6.8
Copyright (C) 2008 Free Software Foundation, Inc.
License GPLv3+: GNU GPL version 3 or later <http://gnu.org/licenses/gpl.html>
This is free software: you are free to change and redistribute it.
There is NO WARRANTY, to the extent permitted by law. Type "show copying"
and "show warranty" for details.
This GDB was configured as "i686-pc-mingw32"...
(gdb) run
Starting program: C:\TEMP\x264\x264/x264.exe --crf 22 --me esa --output NUL --pr
ogress c:\\temp\\x264\\sample.avs
[New thread 2036.0xce8]
Error: dll starting at 0x77d40000 not found.
Error while mapping shared library sections:
NOT_AN_IMAGE: No such file or directory.
Error while mapping shared library sections:
C:\WINDOWS\SysWOW64\ntdll32.dll: No such file or directory.
Error: dll starting at 0x77d40000 not found.
Error: dll starting at 0x77c20000 not found.
[New thread 2036.0x18c]
warning: DllMain: hModule=0x024a0000, ulReason=1, lpReserved=0x00000000, gRefCnt
= 0

warning: DllGetClassObject() CLSID: CAVIFileSynth

warning: 00038190->CAVIFileSynth::CAVIFileSynth()

warning: 00038190->CAVIFileSynth::AddRef() gRefCnt=1, m_refs=1

warning: 00038190->CAVIFileSynth::QueryInterface() {00000001-0000-0000-c000-0000
00000046} (IClassFactory)

warning: 00038190->CAVIFileSynth::AddRef() gRefCnt=2, m_refs=2

warning: 00038190->CAVIFileSynth::Release() gRefCnt=1, m_refs=1

warning: DllGetClassObject() result=0x0, object=00038198

warning: 00038190->CAVIFileSynth::CreateInstance()

warning: 000381E8->CAVIFileSynth::CAVIFileSynth()

warning: 000381E8->CAVIFileSynth::AddRef() gRefCnt=2, m_refs=1

warning: 000381E8->CAVIFileSynth::QueryInterface() {00000000-0000-0000-c000-0000
00000046} (IUnknown)

warning: 000381E8->CAVIFileSynth::AddRef() gRefCnt=3, m_refs=2

warning: 000381E8->CAVIFileSynth::Release() gRefCnt=2, m_refs=1

warning: 00038190->CAVIFileSynth::CreateInstance() result=0x0, object=000381E8

warning: 000381E8->CAVIFileSynth::AddRef() gRefCnt=3, m_refs=2

warning: 000381E8->CAVIFileSynth::Release() gRefCnt=2, m_refs=1

warning: 00038190->CAVIFileSynth::Release() gRefCnt=1, m_refs=0

warning: 00038190->CAVIFileSynth::~CAVIFileSynth(), gRefCnt = 1

warning: 000381E8->CAVIFileSynth::QueryInterface() {00000000-0000-0000-c000-0000
00000046} (IUnknown)

warning: 000381E8->CAVIFileSynth::AddRef() gRefCnt=2, m_refs=2

warning: 000381E8->CAVIFileSynth::Release() gRefCnt=1, m_refs=1

warning: 000381E8->CAVIFileSynth::QueryInterface() {00020025-0000-0000-c000-0000
00000046} (unsupported!)

warning: 000381E8->CAVIFileSynth::QueryInterface() {0000010b-0000-0000-c000-0000
00000046} (IPersistFile)

warning: 000381E8->CAVIFileSynth::AddRef() gRefCnt=2, m_refs=2

warning: 000381E8->CAVIFileSynth::QueryInterface() {00020020-0000-0000-c000-0000
00000046} (IAVIFile)

warning: 000381E8->CAVIFileSynth::AddRef() gRefCnt=3, m_refs=3

warning: 000381E8->CAVIFileSynth::Load("c:\\temp\\x264\\sample.avs", 0x0)

warning: 000381E8->CAVIFileSynth::Release() gRefCnt=2, m_refs=2

warning: 000381E8->CAVIFileSynth::Release() gRefCnt=1, m_refs=1

warning: 000381E8->CAVIFileSynth::GetStream(*, 73646976(vids), 0)

dbxread.c:1336: internal-error: Section index is uninitialized
A problem internal to GDB has been detected,
further debugging may prove unreliable.
Quit this debugging session? (y or n)

[Kurtnoise]

Error: dll starting at 0x77d40000 not found.
Error while mapping shared library sections:
NOT_AN_IMAGE: No such file or directory.
Error while mapping shared library sections:
C:\WINDOWS\SysWOW64\ntdll32.dll: No such file or directory.

[burfadel]

There is no ntdll32.dll file in Vista x64 or XP x64! - that explains this line in your output above:

C:\WINDOWS\SysWOW64\ntdll32.dll: No such file or directory.

[Kurtnoise]

edit2. No it's not. It's the resolution making it crash.

width of 960 and under works
962 = unsupported input format
964 = crash.
mmmh, width>=960 works for me...:D

c:\tmp\x264-r1058>x264.exe --crf 22 --me esa --progress -o "E:\HD\t1.h264" "E:\H
D\Vc1.avs"
avis [info]: 1920x800 @ 23.98 fps (2001 frames)
x264 [info]: using cpu capabilities: MMX2 SSE2Fast SSSE3 PHADD SSE4.1 Cache64
x264 [info]: profile Main, level 4.0
x264 [info]: slice I:16 Avg QP:16.30 size:114738 PSNR Mean Y:46.93 U:51.53
V:52.23 Avg:48.03 Global:47.84
x264 [info]: slice P:1985 Avg QP:19.59 size: 32387 PSNR Mean Y:44.16 U:49.33
V:50.11 Avg:45.35 Global:45.09
x264 [info]: mb I I16..4: 30.4% 0.0% 69.6%
x264 [info]: mb P I16..4: 1.2% 0.0% 1.3% P16..4: 56.7% 16.6% 10.2% 0.0% 0
.0% skip:13.9%
x264 [info]: SSIM Mean Y:0.9737218
x264 [info]: PSNR Mean Y:44.184 U:49.347 V:50.123 Avg:45.369 Global:45.105 kb/s:
6338.38

encoded 2001 frames, 1.94 fps, 6338.49 kb/s

http://pix.nofrag.com/6/b/c/176548141bfff13a98501d6a10438t.jpg (http://pix.nofrag.com/6/b/c/176548141bfff13a98501d6a10438.html)

using your build...

[bob0r]

--frames 50 --crf 24 --me esa (or --me tesa) = crash

From the MAKEFILE:
--crf 22 -b3 -m7 -r4 --me esa -8 -t2 -A all --mixed-refs = no crash
--frames 50 --crf 24 -b3 -m9 -r3 --me tesa -8 -t1 --mixed-refs = crash

We shall wait for the mighty penguin to solve this issue...

[kemuri-_9]

i only bothered to compile the x64 version with my usual patches and it fprofiled just fine <_<

also tossed it a 2120x1200 avs script and it ran esa and tesa just fine

if it's crashing at the end it's likely crashing while freeing memory on the encoder close/cleanup calls.
if it's indeed this case, then the resulting encode would be just fine.

Edit:
tried an x86 build and confirmed the crash

Program received signal SIGTRAP, Trace/breakpoint trap.
0x7d61002e in ?? ()
warning: HEAP[x264_debug.exe]:
warning: Heap block at 024EA610 modified at 02502C2C past requested size of 18614

[LoRd_MuldeR]

Error: dll starting at 0x77d40000 not found.
Error while mapping shared library sections:
NOT_AN_IMAGE: No such file or directory.
Error while mapping shared library sections:
C:\WINDOWS\SysWOW64\ntdll32.dll: No such file or directory.


But the file is there for sure:

http://img391.imageshack.us/img391/3773/ntdllqf1.th.png (http://img391.imageshack.us/img391/3773/ntdllqf1.png)

I doubt Windows would even boot up, if the "native" API (ntdll.dll) was missing. Nor would any application work :p

So there must be another problem. I got no idea what though...

[video_magic]

It says ntdll32 not just ntdll

[LoRd_MuldeR]

It says ntdll32 not just ntdll

Your are absolutely right :o

[Audionut]

x264-r1058-clean.rar (http://rapidshare.com/files/176381226/x264-r1058-clean.rar)

No patches.

Seems to be working fine.

[Sharktooth]

bobor's build (x264.nl) is "clean" too, but still crashes.

[LoRd_MuldeR]

Any comment on that crash from the x264 gurus yet? :confused:

[Dark Shikari]

Any comment on that crash from the x264 gurus yet? :confused:worksforme

[LoRd_MuldeR]

worksforme

Well, obviously something was broken between r1057 and r1058. Various people can reproduce the crash. Nobody is going the investigate this?

I could give a more detailed error report, if GDB wouldn't fail with "C:\WINDOWS\SysWOW64\ntdll32.dll: No such file or directory." when I try to debug the build :rolleyes:

[kemuri-_9]

it might be limited to x86 windows builds, as everyone here has reported problems with x86 mingw builds...
i've gotten x64 to work fine.


Program received signal SIGTRAP, Trace/breakpoint trap.
0x7d61002e in strchr () from C:\WINDOWS\SysWOW64\ntdll32.dll
warning: HEAP[x264.exe]:
warning: Heap block at 0243A318 modified at 02452937 past requested size of 18617

bt full
#0 0x7d61002e in strchr () from C:\WINDOWS\SysWOW64\ntdll32.dll
No symbol table info available.
#1 0x7d6820f9 in ntdll!LdrAccessOutOfProcessResource ()
from C:\WINDOWS\SysWOW64\ntdll32.dll
No symbol table info available.
#2 0x0022f618 in ?? ()
No symbol table info available.
#3 0x7d66e7ec in ntdll!RtlpNtEnumerateSubKey ()
from C:\WINDOWS\SysWOW64\ntdll32.dll
No symbol table info available.
#4 0x0243a318 in ?? ()
No symbol table info available.
#5 0x0243a318 in ?? ()
No symbol table info available.
#6 0x003f0000 in ?? ()
No symbol table info available.
#7 0x0243a318 in ?? ()
No symbol table info available.
#8 0x0022f62c in ?? ()
No symbol table info available.
#9 0x7d659d2e in ntdll!LdrGetDllHandleEx ()
from C:\WINDOWS\SysWOW64\ntdll32.dll
No symbol table info available.
#10 0x00000000 in ?? ()
No symbol table info available.


and msvc (to confirm that's it not just specific to mingw):
http://kemuri9.net/forumpics/x264_r1058.png
'retrying' it, pulls up

retval = HeapFree(_crtheap, 0, pBlock);

as the problem, from free()

[LoRd_MuldeR]

But why ntdll32.dll, when x264.exe is not even linked against that DLL ???

The debugger is right: There is no ntdll32.dll in my "SysWOW64" folder, nor in my "System32" folder. But x264.exe doesn't need/use that DLL, verified by Dependency Walker.

It indirectly requires ntdll.dll, because it's linked to MSVCRT.dll, which is linked against ntdll.dll. But there's certainly no ntdll32.dll in the dependency tree...

http://img187.imageshack.us/img187/6974/x264dependsgw8.th.png (http://img187.imageshack.us/img187/6974/x264dependsgw8.png)

Is ntdll32.dll an alias name for the 32-Bit version of ntdll.dll? And if so, why it doesn't work for GDB anymore all of a sudden? :confused:

[Dark Shikari]

How about you debug on a sane operating system then? :rolleyes:

[LoRd_MuldeR]

How about you debug on a sane operating system then? :rolleyes:

If you don't consider Windows XP sane, what shall I use? :p

Note that debugging with GDB always worked 100% fine up to build r1058, so the must be something "strange" with that one. It's not like I'm trying to debug for the first time.

And no, I didn't change the OS, the GCC version or the GDB version recently...


Anyways. Since kemuri-_9 tracked the problem down to "retval = HeapFree(_crtheap, 0, pBlock);", is there a chance for a fix or a workaround at least? :o

[Dark Shikari]

Anyways. Since kemuri-_9 tracked the problem down to "retval = HeapFree(_crtheap, 0, pBlock);", is there a chance for a fix or a workaround at least? :oHow can you "track down" a problem to code that doesn't even exist?

xeelee x264_commit_repo
$ grep -r "HeapFree" *

xeelee x264_commit_repo
$

[LoRd_MuldeR]

How can you "track down" a problem to code that doesn't even exist?

xeelee x264_commit_repo
$ grep -r "HeapFree" *

xeelee x264_commit_repo
$

I hope kemuri-_9 can answer where he got this code from :)

[squid_80]

A quick poke at google seems to indicate gdb has issues with XP64 (http://cygwin.com/ml/cygwin/2007-10/msg00179.html). But I don't think the ntdll32 problem is making it crash on you, since it continues to spit out all the avisynth debug info.

[kemuri-_9]

Anyways. Since kemuri-_9 tracked the problem down to "retval = HeapFree(_crtheap, 0, pBlock);", is there a chance for a fix or a workaround at least? :o

i stated that the code was from free.c, where the definition of free is.

free is crashing due to the heap violation that x264 is apparently performing.

so according to the error stated by both gdb and msvc, x264 is editing memory beyond its requested allocation in the latest revision.

are the linux builds not exhibiting the problem?
if only windows had a decent and relatively cheap/free equivalent of valgrind, nailing this down would be easier.

btw, i just symlinked C:\Windows\sysWOW64\ntdll32.dll to C:\Windows\sysWOW64\ntdll.dll (copying would suffice as well) to solve the gdb problem.

[Dark Shikari]

i stated that the code was from free.c, where the definition of free is.

free is crashing due to the heap violation that x264 is apparently performing.

so according to the error stated by both gdb and msvc, x264 is editing memory beyond its requested allocation in the latest revision.If x264 edited memory beyond what it allocated, there would be an instant segfault, not a free() crash.

[kemuri-_9]

i'm not an expert at using at msvc as a debugger, but i did spot this:
http://kemuri9.net/forumpics/x264_r1058_2.png

showing that free is being called and crashing due to a heap corruption
(well, this doesn't show the heap corruption part, but the code it's pointing to is throwing the actual heap corruption window box)

Edit:
If i'm reading this correctly,
it appears to be crashing on:

for( i = 0; i < 4; i++ )
x264_free( frame->buffer[i] );

[bob0r]

Why are we all, all of a sudden pretending Windows never needed fixing before and now it has made a mistake, we aren't allowed to even backtrace together with the developers help?

Aren't there like plenty of work-arounds for GCC and/or Windows issues?

I offered my help 3x, and 3x i was ignored, i know i am annoying at most times, but i also know i am way too dumb to know exactly what commands to time for the correct info.


I hear some "company whispering".....


Again: http://x264.nl/gdb.1058.txt what more shall/can i do to help?

[Dark Shikari]

Why are we all, all of a sudden pretending Windows never needed fixing before and now it has made a mistake, we aren't allowed to even backtrace together with the developers help?No, I'm saying that if you have the choice between a debugging environment that does work, and one that doesn't, why the hell would you insist on the one that doesn't when you could do the debugging far more easily on the one that does work?

I don't understand the reason why people spend hours and hours trying to get a Windows gdb to work when it clearly isn't--when they could just boot up Linux and send me a proper backtrace and disass from the crash and stop whining.

[bob0r]

If you just said to me: "jarod, Windows gdb is bugged, it can't be used for a proper debug backtrace."
Thats all i needed to know, in fact i didn't even know before i read: :stupid:

I would have said okay, then lets wait for someone with linux who might help, even though the problem might be Windows related.

If there is nothing I can do, then all i can do is wait (hmm that a weird sentence :D)

[Dark Shikari]

If you just said to me: "jarod, Windows gdb is bugged, it can't be used for a proper debug backtrace."No, I said to you on IRC half a dozen times that I wanted a disass, but I still don't have a disass from you, so I made the assumption that you are incapable of making one.

[kemuri-_9]

here's one from a debian server i rent out:
gdb bt (http://kemuri9.net/dev/x264/r1058_gdb.bt.txt)

[MasterNobody]

Dark Shikari
Something wrong with decrease of buffer[3] size in http://git.videolan.org/gitweb.cgi?p=x264.git;a=commitdiff;h=a4ec1020efb1a2a6757f8f891d78c2dd9344bb91 when there is no p4x4. I think somewhere in x264_frame_filter there is writes out of allocated memory block (and it writes out of range also without asm).

There is no crash if I return "*2" instead of "<< h->frames.b_have_sub8x8_esa"

[Likewise]

> If x264 edited memory beyond what it allocated, there would be an instant segfault, not a free() crash.

Only if it reaches into a next page boundary for dynamically allocated memory, with pages typically 4096 bytes for most Linux platforms.

Please correct me if I'm mistaken.

Leon.

[LoRd_MuldeR]

There is no crash if I return "*2" instead of "<< h->frames.b_have_sub8x8_esa"

Interesting. Could you kindly provide a patch? :thanks:

[bob0r]

> If x264 edited memory beyond what it allocated, there would be an instant segfault, not a free() crash.

Only if it reaches into a next page boundary for dynamically allocated memory, with pages typically 4096 bytes for most Linux platforms.

Please correct me if I'm mistaken.

Leon.


Likewise
Registered User


Join Date: Jun 2006
Posts: 1




It's ALIIIIIVE!! (MERRY X-MAS! :D)
Likewise must be pengvado's third half.

[MasterNobody]

Interesting. Could you kindly provide a patch? :thanks:

I think this would be more correct than increasing size of buffer[3]: http://stashbox.org/337696/x264_fix_esa_crash.diff

[LoRd_MuldeR]

I think this would be more correct than increasing size of buffer[3]: http://stashbox.org/337696/x264_fix_esa_crash.diff

That fixes the crash for me :)

:goodpost:

[Audionut]

x264-r1060.rar (http://rapidshare.com/files/176544392/x264-r1060.rar)

Patched with,
x264_custom_strtok_r.r1038.diff
x264_debug_defines.r1038.diff
x264_fix_stats_file_work.r1038.diff
x264_multithreading_bug_check.r1038.diff
x264_error_memoryleaks.03.r1038.diff
x264_thread_pool.r1038.diff
x264_thread_priority_with_pool.02.diff
x264_Cosmetic.02.diff
x264_log_file.03k.diff
x264_hrd_pulldown.09_interlace.diff
x264_single_frame_flash.diff
x264_mingw_aligned_03.diff
x264_fix_esa_crash.diff

[video_magic]

Thanks as always for the build audionut (I assume that it is still as before one of your P4 Prescott optimised builds right?) :)

Just a bit 'concerned' at the large patch list - I've read some of the latest previous comments, and wonder if I'm supposed to leave any Windows builds for a while until something is resolved (to make x264 work probably reliably under Windows?), I don't know the technicalities but because I'm archiving I have to wonder - what is the chances of this problem that the builders and developers are mentioning, affecting the output (on WinXP SP2 +some security updates), if at all?

PS. audionut: re: the PMs. My board seems to be such an unstable P.O.S. that I am so distraught after some moths wasted, no overclocking or anything will be sensible now, because having tracked down and resolved so many problems and settings related to encoding to X264 from VHS captues, (over the last several months) the usual (and final remaining) problem, is 'interference lines' in the captured video on any BIOS or other setting, with any capture codec. It seems to be because of having a VIA chipset and the effect on the PCI bus :( :( After all this time I am so upset that. I will get an Intel chipset based board for my Intel SL9KF P4.

When I do finally get every last remaining problem squished, I just want to know that the X264 build is going to be alright (as in generally safe) for H.264 output, and also fairly well optimised for the encoding speed. Thanks and praise as always to the developers and builders for what seems like a great codec. I just wondered regarding the last few posts and this quite long patch-list for this build.

Thanks for any advice as always guys, it's really appreciated.