x264 development

[kemuri-_9]

ah...
it turns out i was misreading some of the code.

it didn't help that what can be used within the zone parser isn't specifically documented anywhere from what i can remember
(longhelp doesn't even mention anything outside of force QP and bitrate multiplier).
so i guess this compounded the misread.

so the proper zones would be
start_frame,end_frame,b=X,options
start_frame,end_frame,q=X,options
start_frame,end_frame,options

'options' being supported ones that were listed above in a option=value format if they take values or the name of the parameter if they don't ... i.e.
direct=none
nr=100
me=umh
merange=32
no-deblock
scenecut=60
no-fast-pskip
etc...

like
0,100,psy-rd=0.0:0.0,trellis=1,no-fast-pskip,me=dia

trying it out a bit now...
i seem to be getting a crash if me=t/esa is used in a zone when --me is < esa
would need to make a debug build to nail it down further...
is this a known issue?

[Dark Shikari]

trying it out a bit now...
i seem to be getting a crash if me=t/esa is used in a zone when --me is < esa
would need to make a debug build to nail it down further...
is this a known issue?Yes, TESA/ESA cannot be enabled unless the frames have the "integral" data allocated, so I'll add in code to ensure that you cannot enable them if they were off to begin with.

[crypto]

zones?

also yes sane b-adapt 2 <= 5
even 5 is a bit much
I know that >3 b-frames have a performance hit with b-adapt 2. My idea is about automatically detecting fades. VC-1 encoders can do that and can even signal the fade to the decoder which results in smooth fades like butter.

[Dark Shikari]

I know that >3 b-frames have a performance hit with b-adapt 2. My idea is about automatically detecting fades. VC-1 encoders can do that and can even signal the fade to the decoder which results in smooth fades like butter.That's explicit weighted prediction, which x264 doesn't support [yet]. Patches welcome.

[crypto]

That's explicit weighted prediction, which x264 doesn't support [yet]. Patches welcome.
Ok, I see.

About the zones, is that what's called segmented encoding in other encoders?

After the 2 pass encode is ready, the result is displayed and you can mark several segments for reencoding.

[Dark Shikari]

After the 2 pass encode is ready, the result is displayed and you can mark several segments for reencoding.Yes, it can be used for that purpose. No, I do not endorse the concept that companies can compensate for awfully-designed encoders by simply telling the user to fix every mistake their encoder made.

[crypto]

Yes, it can be used for that purpose. No, I do not endorse the concept that companies can compensate for awfully-designed encoders by simply telling the user to fix every mistake their encoder made.
LOL, thanks for the explanation.

And even if it might sounded so, I am in no way criticizing the results of x264. I tried many encoders and x264 is my choice because of its superior results.

[Adub]

Wow, okay, so i just thought I would run some tests recently, and see if I should be one to jump on the psy trellis band wagon.

Okay, so I encoded the same source twice (Heroes Season 1 Episode 11, DVD NTSC R1) once with out psy trellis, once with. It was with MeGUI, and Sharktooth's unrestricted 1 pass const. quality Extra Q modded for psy rd = 0.7 and psy rd = 0.7, along with aq = 0.6 (I stay safe in my usage), and crf = 19.0. I compared the output quality, and yes psy trellis is just that much sharper, albeit with increased grain.

But then I looked at the file sizes:
WithOUT psy trellis = 445mb
With psy trellis = 581mb

Difference = 136mb or 30.561797752808988764044943820225% (edited) larger.

Is this right?! I didn't think that enabling psy trellis would suck up so much bitrate. Hell if I wanted a perfect encode, I would have used a higher crf, or even just done a two pass encode at 581 mb target file size.

DS, is this the expected behavior? Or is this because I used CRF?

[Dark Shikari]

But then I looked at the file sizes:
WithOUT psy trellis = 445mb
With psy trellis = 581mbYes, psy-trellis lowers the effective lambda, raising bitrate at the same QP.
Difference = 136mb or 76.592082616179001721170395869191% larger.Might I suggest you check your math? :p

[DarkZell666]

You can't keep more details/grain and expect the filesize to be the same in quality-based mode, it's just not compatible by design.

Here's how I interpret it : Psy-Whatever enhances grain, but seems to leave the rest of the picture untouched, so the filesize simply goes up by the same amount. It's the bitrate-based modes' job to balance the other frequencies out, unlike in the case of AQ that was designed to balance the details out by itself.

Of course, the bitrate only goes up by the same amount as grain retained, so if the source is already clean enough, the bitrate won't go up as much ... but since there's no grain to keep in that case, why use psy in the first place ? ;)

I suppose we could agree on a rule of thumb about psy+CRF, like "CRF 18 w/o psy = CRF 21 w/ psy" bitrate-wise ? (random numbers just to illustrate).

(Edit: lol, it took me half an hour to finish my post 'coz I kept being disturbed by my boss :p)

[ACoolie]

Yes, TESA/ESA cannot be enabled unless the frames have the "integral" data allocated, so I'll add in code to ensure that you cannot enable them if they were off to begin with.

IMO x264 should automagically transform
x264 --me umh --zones 400,500,me=tesa
to
x264 --me tesa --zones 0,400,me=umh --zones 500,END_FRAME,me=umh

[Adub]

Yes, psy-trellis lowers the effective lambda, raising bitrate at the same QP.
Might I suggest you check your math? :p


That's what I get for doing math at 3 am. But, okay, thanks for your reply. I will have to run further tests, with varying crf's and possibly 2 pass settings. Right now I am not convinced quite yet.

[kemuri-_9]

IMO x264 should automagically transform
x264 --me umh --zones 400,500,me=tesa
to
x264 --me tesa --zones 0,400,me=umh --zones 500,END_FRAME,me=umh

imo, that would be more work in the code base than it's worth, when you can just do it yourself easier.

[foxyshadis]

Can you just always allocate the data, if it's not too large, whether it's originally used or not?

[akupenguin]

Can you just always allocate the data, if it's not too large, whether it's originally used or not?
Assume 1080p, 4 refs, 3 B-frames, 6 threads.
edit, now with actual measurement
Then x264 with ESA takes 800MB vmem, and without ESA takes 500MB. If you're on linux with overcommitted memory, there's no difference: as long as the ESA arrays aren't used, they never really get allocated. But if your OS expects virtual memory to be backed up by physical memory, IMO that's too much waste.

[lexor]

Assume 1080p, 4 refs, 3 B-frames, 6 threads. Then the ESA arrays take 200MB. Is that not too large?

If there is only one of those at a time, and the rest of the x264 stuff in RAM doesn't increase the total by more than 2x that value, I don't think it's too large. Assuming that it has practical usefulness to begin with, of course.

[Sagekilla]

@akupenguin: How do you calculate how large an esa array would be? Just out of curiosity.

[akupenguin]

It's 4 bytes per pixel, and extends 32 pixels off the edge of the visible image in each direction.
The only tricky part is calculating how many frames x264 will allocate.

[Sagekilla]

So (width + 64) * (height + 64) * references * threads * 4 bytes is how big you would expect the array to be.

[akupenguin]

(width + 64) * (height + 64) * 4 * (references + threads + other frames that don't really need ESA but allocate it anyway)

[kemuri-_9]

hmm....
how viable would it be to have the zone parser trigger the T/ESA memory allocation if it sees a me=t/esa, if it is not already allocated (say the --me is <= UMH)?
that currently seems to be the easiest solution for the issue from what i know of the situation, compared to rewriting and swapping zones and allowing it to work...

[Dark Shikari]

hmm....
how viable would it be to have the zone parser trigger the T/ESA memory allocation if it sees a me=t/esa, if it is not already allocated (say the --me is <= UMH)?
that currently seems to be the easiest solution for the issue from what i know of the situation, compared to rewriting and swapping zones and allowing it to work...That would require a massive amount of code--going through all the frames, allocating new data for each frame, and then calling the various functions to create the data... not worth it.

[kemuri-_9]

no no, I'm not talking about at the time of encoding the frame/on demand, but at initializer time.

though this would be dependent on when the frame memory allocation and zone parsing events happen in the initializer.

the ideal situation the code could be in is that the zone parser comes before the frame memory allocation,
so then the zone parser could add to the switch of which allocation to do (esa based allocation vs <=umh allocation), if a zone has t/esa specified.
so when it comes time to do the zone's t/esa, the memory is already allocated and there would be no crashes.

horribly oversimplified (object-based) psuedocode of this would be similar to

zones.parse(char *); // parse the zone string
if (zones.contains(me,ESA | TESA) || (me == ESA || me == TESA)) // determine which memory allocation type we need
esa_memory_allocation();
else
non_esa_memory_allocation();


if the memory allocation comes before the zone parsing, the situation then becomes 'how struggling would it be to move the zone parsing before the memory allocation', so above situation could be realized.

and I'm at work, so i don't have any code in front of me when i speculated this up.

[Quark.Fusion]

Assume 1080p, 4 refs, 3 B-frames, 6 threads.
edit, now with actual measurement
Then x264 with ESA takes 800MB vmem, and without ESA takes 500MB. If you're on linux with overcommitted memory, there's no difference: as long as the ESA arrays aren't used, they never really get allocated. But if your OS expects virtual memory to be backed up by physical memory, IMO that's too much waste.

Isn't windows also don't physically allocate that memory? But you can hit 2GB limit on 32bit apps (already problem for avisynth plugins).

[pcordes]

there's no way possible with tee and/or grep to have what you want to write to file and have it display the progress to the screen for when you want to occasionally want to check up on it.


Never underestimate what you can do with the Unix shell, young padawan.


if you do
2>&1 | tee file
you get to see everything as normal, but then the file gets the progress bloat

if you do
2>&1 | grep -v " eta " | tee file
the file gets what you want (besides the I slice statistics, since they're removed with the last progress line), but you don't see jack.


So you need to send output to the terminal before you filter it. So try this:
x264 2>&1 | tee /dev/tty | tr '\r' '\n' | LANG=C grep -v '^\[' > log

Or do it all in one command with awk. (At first I though I had a simpler awk program, but it didn't get the output quite right).
x264 2>&1 | LANG=C gawk '{if(/^\[/){ print $0"\r";fflush() }else{print $0"\n"; print $0"\n" >"logfile"}}' ORS='' RS='[\r\n]'

I timed these (with time cat x264.log x264.log x264.log ... | tee /dev/null | ... > log), cat(1)ing a total of 108MB of x264 logs to make it take at least a second. This is probably way overkill because x264 cuts back its logging rate on longer files, so this only matters when it's going to be really fast anyway.

On my C2D E6600, DDR2-800, running AMD64 Linux 2.6.27 (Ubuntu Intrepid):
real 0m0.940s, user 0m0.672s, sys 0m0.288s for the tee /dev/null| tr | grep pipeline.
real 0m0.747s, user 0m0.420s, sys 0m0.424s for the tee /dev/null| tr | LANG=C grep pipeline.
real 0m3.547s, user 0m2.872s, sys 0m0.636s for LANG=C gawk
real 0m10.089s,user 0m9.345s, sys 0m0.700s for LANG=en_CA.utf8 gawk
real 0m1.753s, user 0m1.172s, sys 0m0.572s for mawk

mawk is fast, but it isn't usable for this, because it always waits until it has 4kB of input before breaking it into lines with RS. gawk considers each block of input to see if it's a whole line.

UTF-8 makes grep and gawk's regexes slow, so use LANG=C. The tee|tr|grep combination uses less CPU when that's the limiting factor, but probably the one-process gawk way is best, to minimize context switches and cache pollution. Being woken once for each line is totally different from being CPU-limited by grep or gawk. OTOH, gawk must be doing something to use all that CPU time... Again, x264 doesn't log that often on longer files, and I went nuts figuring out how to benchmark these things before I realized that.

I think I once had a program lying around that processed input containing carriage returns, etc. and output what would be left on the screen of a terminal emulator. I think it came from a CS prof who wrote it for people to run their assignments through before submitting the output. I can't find it now, though. That would be much less overhead than either of my suggestions.

the purpose of --progress and then the purpose of what you would use 2> for are counters to each other. they just don't work together well
with the log file patch you can get --progress without some drawback.

x264 doesn't normally use stdout for anything (right?), so why not put messages that people want to redirect on stdout, and progress on stderr?
Then it's
x264 | tee log

Perhaps a --log-interval option would be good (or make --progress take an optional argument). Some terminal emulators aren't that fast, and the log messages could use a few % CPU if they're actually getting rendered onto video memory. I guess --progress isn't the default, though, so just leave it off on fast encodes where logging is a significant overhead.

Ok, currently x264 logs 1000 times for each input file, or once per 10 frames if it doesn't know the input frame count. It already knows what time it is (for fps calc), so you could also have it log at most once per second. Or just do that instead of the aim for 1000 per file trick.

Actual error messages could go to both, to make sure they're not lost, except when stdout and stderr are the same file (e.g. if fstat(2) is available, use it and see if st_dev and st_ino are equal.)


To test overhead of different pipe methods, I did this to
get a giant log file: x264 &> output, then I copy/pasted the progress lines to inflate it.

[kemuri-_9]

Never underestimate what you can do with the Unix shell, young padawan.

this isn't unix shell, but windows command prompt.
a different ball game.

so just drop it.

[Shinigami-Sama]

this isn't unix shell, but windows command prompt.
a different ball game.

so just drop it.

cygwin

[LoRd_MuldeR]

cygwin

... or MSYS

[J_Darnley]

... or change x264 (http://users.telenet.be/darnley/x264/x264_double-print_03.diff).

[kemuri-_9]

try all those solutions with --quiet -o - and see how well they work.
(why? because i didn't want to continue this discussion, it's OT)

@devs
however, on the subject of --quiet...
why was the output in muxers.c not made to respect --quiet?
-
referring to the lack of p_param->i_log_level comparisons where able...
this (http://kemuri9.net/dev/x264/patches/x264.muxers.respect.log.level.diff) should explain it better

*figures there might be some angry/irritated ppl after this and is apologizing now*

[Dark Shikari]

referring to the lack of p_param->i_log_level comparisons where able...
this (http://kemuri9.net/dev/x264/patches/x264.muxers.respect.log.level.diff) should explain it better< pengvado> macro, or call x264_log_default

[Emp3r0r]

-[NoImage] Job commandline: "C:\Programs\x264\x264.exe" --pass 2 --bitrate 6059 --stats "H:\BD\RD\rd.stats" --level 4.1 --ref 4 --mixed-refs --bframes 3 --b-adapt 2 --weightb --direct auto --deblock -1:-1 --subme 7 --trellis 2 --partitions p8x8,b8x8,i4x4,i8x8 --8x8dct --vbv-bufsize 50000 --vbv-maxrate 50000 --me umh --threads auto --thread-input --zones 174400,180359,q=40 --sar 1:1 --progress --no-psnr --no-ssim --output "H:\BD\RD\rd.mkv" "H:\BD\RD\rd.avs"

1st pass worked but at beginning of 2nd pass it crashes x264.

Problem signature:
Problem Event Name: APPCRASH
Application Name: x264.exe
Application Version: 0.0.0.0
Application Timestamp: 491c385f
Fault Module Name: ntdll.dll
Fault Module Version: 6.0.6001.18000
Fault Module Timestamp: 4791a783
Exception Code: c0000005
Exception Offset: 0002f217
OS Version: 6.0.6001.2.1.0.256.6
Locale ID: 1033
Additional Information 1: fd00
Additional Information 2: ea6f5fe8924aaa756324d57f87834160
Additional Information 3: fd00
Additional Information 4: ea6f5fe8924aaa756324d57f87834160

I'm on a Q9450 x64 VistaSP1.

[Dark Shikari]

-Problem signature:
Problem Event Name: APPCRASH
Application Name: x264.exe
Application Version: 0.0.0.0
Application Timestamp: 491c385f
Fault Module Name: ntdll.dll
Fault Module Version: 6.0.6001.18000
Fault Module Timestamp: 4791a783
Exception Code: c0000005
Exception Offset: 0002f217
OS Version: 6.0.6001.2.1.0.256.6
Locale ID: 1033
Additional Information 1: fd00
Additional Information 2: ea6f5fe8924aaa756324d57f87834160
Additional Information 3: fd00
Additional Information 4: ea6f5fe8924aaa756324d57f87834160There's no useful information in there: we can't do any debugging unless you have a backtrace from gdb, at a minimum.

[Kurtnoise]

:script:

[Emp3r0r]

[4304] DllMain: hModule=0x06a30000, ulReason=1, lpReserved=0x00000000, gRefCnt = 0
[4304] DllMain: hModule=0x06a30000, ulReason=0, lpReserved=0x00000000, gRefCnt = 0
[4584] DllMain: hModule=0x10000000, ulReason=1, lpReserved=0x00000000, gRefCnt = 0
[4584] DllGetClassObject() CLSID: CAVIFileSynth
[4584] 002D9C90->CAVIFileSynth::CAVIFileSynth()
[4584] 002D9C90->CAVIFileSynth::AddRef() gRefCnt=1, m_refs=1
[4584] 002D9C90->CAVIFileSynth::QueryInterface() {00000001-0000-0000-c000-000000000046} (IClassFactory)
[4584] 002D9C90->CAVIFileSynth::AddRef() gRefCnt=2, m_refs=2
[4584] 002D9C90->CAVIFileSynth::Release() gRefCnt=1, m_refs=1
[4584] DllGetClassObject() result=0x0, object=002D9C98
[4584] 002D9C90->CAVIFileSynth::CreateInstance()
[4584] 002D9CD8->CAVIFileSynth::CAVIFileSynth()
[4584] 002D9CD8->CAVIFileSynth::AddRef() gRefCnt=2, m_refs=1
[4584] 002D9CD8->CAVIFileSynth::QueryInterface() {00000000-0000-0000-c000-000000000046} (IUnknown)
[4584] 002D9CD8->CAVIFileSynth::AddRef() gRefCnt=3, m_refs=2
[4584] 002D9CD8->CAVIFileSynth::Release() gRefCnt=2, m_refs=1
[4584] 002D9C90->CAVIFileSynth::CreateInstance() result=0x0, object=002D9CD8
[4584] 002D9CD8->CAVIFileSynth::AddRef() gRefCnt=3, m_refs=2
[4584] 002D9CD8->CAVIFileSynth::Release() gRefCnt=2, m_refs=1
[4584] 002D9C90->CAVIFileSynth::Release() gRefCnt=1, m_refs=0
[4584] 002D9C90->CAVIFileSynth::~CAVIFileSynth(), gRefCnt = 1
[4584] 002D9CD8->CAVIFileSynth::QueryInterface() {00000000-0000-0000-c000-000000000046} (IUnknown)
[4584] 002D9CD8->CAVIFileSynth::AddRef() gRefCnt=2, m_refs=2
[4584] 002D9CD8->CAVIFileSynth::Release() gRefCnt=1, m_refs=1
[4584] 002D9CD8->CAVIFileSynth::QueryInterface() {00020025-0000-0000-c000-000000000046} (unsupported!)
[4584] 002D9CD8->CAVIFileSynth::QueryInterface() {0000010b-0000-0000-c000-000000000046} (IPersistFile)
[4584] 002D9CD8->CAVIFileSynth::AddRef() gRefCnt=2, m_refs=2
[4584] 002D9CD8->CAVIFileSynth::QueryInterface() {00020020-0000-0000-c000-000000000046} (IAVIFile)
[4584] 002D9CD8->CAVIFileSynth::AddRef() gRefCnt=3, m_refs=3
[4584] 002D9CD8->CAVIFileSynth::Load("H:\BD\RD\rd.avs", 0x0)
[4584] 002D9CD8->CAVIFileSynth::Release() gRefCnt=2, m_refs=2
[4584] 002D9CD8->CAVIFileSynth::Release() gRefCnt=1, m_refs=1
[4584] 002D9CD8->CAVIFileSynth::GetStream(*, 73646976(vids), 0)
[4584] 02676428->CAVIStreamSynth(video)
[4584] 02676428->CAVIStreamSynth::AddRef() (video) gRefCnt=2, m_refs=1
[4584] 002D9CD8->CAVIFileSynth::AddRef() gRefCnt=3, m_refs=2
[4584] 002D9CD8->CAVIFileSynth::Release() gRefCnt=2, m_refs=1
[4584] 02676428->CAVIStreamSynth::Info(0027F6C8, 204) (video)
[4584] 02676428->CAVIStreamSynth::Info(0027F6D8, 204) (video)
[4584] DllMain: hModule=0x10000000, ulReason=0, lpReserved=0x00000001, gRefCnt = 2

LoadPlugin("C:\Program Files (x86)\AviSynth\plugins\DGAVCDecode.dll")
AVCSource("H:\BD\RD\00025.dga")
crop( 0, 16, 0, -24)
Lanczos4Resize(1280,960)

If someone else can confirm that it is working on their Yorkfield then it is probably just my machine. Also, I'll try rolling back to an earlier version to see if the problem persists.

[Dark Shikari]

[4304] DllMain: hModule=0x06a30000, ulReason=1, lpReserved=0x00000000, gRefCnt = 0
[4304] DllMain: hModule=0x06a30000, ulReason=0, lpReserved=0x00000000, gRefCnt = 0
[4584] DllMain: hModule=0x10000000, ulReason=1, lpReserved=0x00000000, gRefCnt = 0
[4584] DllGetClassObject() CLSID: CAVIFileSynth
[4584] 002D9C90->CAVIFileSynth::CAVIFileSynth()
[4584] 002D9C90->CAVIFileSynth::AddRef() gRefCnt=1, m_refs=1
[4584] 002D9C90->CAVIFileSynth::QueryInterface() {00000001-0000-0000-c000-000000000046} (IClassFactory)
[4584] 002D9C90->CAVIFileSynth::AddRef() gRefCnt=2, m_refs=2
[4584] 002D9C90->CAVIFileSynth::Release() gRefCnt=1, m_refs=1
[4584] DllGetClassObject() result=0x0, object=002D9C98
[4584] 002D9C90->CAVIFileSynth::CreateInstance()
[4584] 002D9CD8->CAVIFileSynth::CAVIFileSynth()
[4584] 002D9CD8->CAVIFileSynth::AddRef() gRefCnt=2, m_refs=1
[4584] 002D9CD8->CAVIFileSynth::QueryInterface() {00000000-0000-0000-c000-000000000046} (IUnknown)
[4584] 002D9CD8->CAVIFileSynth::AddRef() gRefCnt=3, m_refs=2
[4584] 002D9CD8->CAVIFileSynth::Release() gRefCnt=2, m_refs=1
[4584] 002D9C90->CAVIFileSynth::CreateInstance() result=0x0, object=002D9CD8
[4584] 002D9CD8->CAVIFileSynth::AddRef() gRefCnt=3, m_refs=2
[4584] 002D9CD8->CAVIFileSynth::Release() gRefCnt=2, m_refs=1
[4584] 002D9C90->CAVIFileSynth::Release() gRefCnt=1, m_refs=0
[4584] 002D9C90->CAVIFileSynth::~CAVIFileSynth(), gRefCnt = 1
[4584] 002D9CD8->CAVIFileSynth::QueryInterface() {00000000-0000-0000-c000-000000000046} (IUnknown)
[4584] 002D9CD8->CAVIFileSynth::AddRef() gRefCnt=2, m_refs=2
[4584] 002D9CD8->CAVIFileSynth::Release() gRefCnt=1, m_refs=1
[4584] 002D9CD8->CAVIFileSynth::QueryInterface() {00020025-0000-0000-c000-000000000046} (unsupported!)
[4584] 002D9CD8->CAVIFileSynth::QueryInterface() {0000010b-0000-0000-c000-000000000046} (IPersistFile)
[4584] 002D9CD8->CAVIFileSynth::AddRef() gRefCnt=2, m_refs=2
[4584] 002D9CD8->CAVIFileSynth::QueryInterface() {00020020-0000-0000-c000-000000000046} (IAVIFile)
[4584] 002D9CD8->CAVIFileSynth::AddRef() gRefCnt=3, m_refs=3
[4584] 002D9CD8->CAVIFileSynth::Load("H:\BD\RD\rd.avs", 0x0)
[4584] 002D9CD8->CAVIFileSynth::Release() gRefCnt=2, m_refs=2
[4584] 002D9CD8->CAVIFileSynth::Release() gRefCnt=1, m_refs=1
[4584] 002D9CD8->CAVIFileSynth::GetStream(*, 73646976(vids), 0)
[4584] 02676428->CAVIStreamSynth(video)
[4584] 02676428->CAVIStreamSynth::AddRef() (video) gRefCnt=2, m_refs=1
[4584] 002D9CD8->CAVIFileSynth::AddRef() gRefCnt=3, m_refs=2
[4584] 002D9CD8->CAVIFileSynth::Release() gRefCnt=2, m_refs=1
[4584] 02676428->CAVIStreamSynth::Info(0027F6C8, 204) (video)
[4584] 02676428->CAVIStreamSynth::Info(0027F6D8, 204) (video)
[4584] DllMain: hModule=0x10000000, ulReason=0, lpReserved=0x00000001, gRefCnt = 2

LoadPlugin("C:\Program Files (x86)\AviSynth\plugins\DGAVCDecode.dll")
AVCSource("H:\BD\RD\00025.dga")
crop( 0, 16, 0, -24)
Lanczos4Resize(1280,960)That's an Avisynth backtrace, not an x264 backtrace.

[Emp3r0r]

I rolled back to version x264.1016.modified.02.exe and the 2nd pass is now running without crash. I'll try 1024 next.

That's all DebugView is giving me.... downloading GDB.

[New thread 4552.0x1138]
(no debugging symbols found)
Error: dll starting at 0x76f90000 not found.
Error: dll starting at 0x75510000 not found.
Error: dll starting at 0x76f90000 not found.
Error: dll starting at 0x76ec0000 not found.
(no debugging symbols found)
(no debugging symbols found)
(no debugging symbols found)
(no debugging symbols found)
(no debugging symbols found)
(no debugging symbols found)
(no debugging symbols found)
(no debugging symbols found)
(no debugging symbols found)
(no debugging symbols found)
(no debugging symbols found)
(no debugging symbols found)
(no debugging symbols found)
(no debugging symbols found)
(no debugging symbols found)
(no debugging symbols found)
(no debugging symbols found)
(no debugging symbols found)
(no debugging symbols found)
(no debugging symbols found)
(no debugging symbols found)
(no debugging symbols found)
(no debugging symbols found)
(no debugging symbols found)
(no debugging symbols found)
(no debugging symbols found)
(no debugging symbols found)
(no debugging symbols found)
(no debugging symbols found)
(no debugging symbols found)
warning: DllMain: hModule=0x10000000, ulReason=1, lpReserved=0x00000000, gRefC
= 0

warning: DllGetClassObject() CLSID: CAVIFileSynth

warning: 00B43B28->CAVIFileSynth::CAVIFileSynth()

warning: 00B43B28->CAVIFileSynth::AddRef() gRefCnt=1, m_refs=1

warning: 00B43B28->CAVIFileSynth::QueryInterface() {00000001-0000-0000-c000-00
00000046} (IClassFactory)

warning: 00B43B28->CAVIFileSynth::AddRef() gRefCnt=2, m_refs=2

warning: 00B43B28->CAVIFileSynth::Release() gRefCnt=1, m_refs=1

warning: DllGetClassObject() result=0x0, object=00B43B30

warning: 00B43B28->CAVIFileSynth::CreateInstance()

warning: 00B43B80->CAVIFileSynth::CAVIFileSynth()

warning: 00B43B80->CAVIFileSynth::AddRef() gRefCnt=2, m_refs=1

warning: 00B43B80->CAVIFileSynth::QueryInterface() {00000000-0000-0000-c000-00
00000046} (IUnknown)

warning: 00B43B80->CAVIFileSynth::AddRef() gRefCnt=3, m_refs=2

warning: 00B43B80->CAVIFileSynth::Release() gRefCnt=2, m_refs=1

warning: 00B43B28->CAVIFileSynth::CreateInstance() result=0x0, object=00B43B80

warning: 00B43B80->CAVIFileSynth::AddRef() gRefCnt=3, m_refs=2

warning: 00B43B80->CAVIFileSynth::Release() gRefCnt=2, m_refs=1

warning: 00B43B28->CAVIFileSynth::Release() gRefCnt=1, m_refs=0

warning: 00B43B28->CAVIFileSynth::~CAVIFileSynth(), gRefCnt = 1

warning: 00B43B80->CAVIFileSynth::QueryInterface() {00000000-0000-0000-c000-00
00000046} (IUnknown)

warning: 00B43B80->CAVIFileSynth::AddRef() gRefCnt=2, m_refs=2

warning: 00B43B80->CAVIFileSynth::Release() gRefCnt=1, m_refs=1

warning: 00B43B80->CAVIFileSynth::QueryInterface() {00020025-0000-0000-c000-00
00000046} (unsupported!)

warning: 00B43B80->CAVIFileSynth::QueryInterface() {0000010b-0000-0000-c000-00
00000046} (IPersistFile)

warning: 00B43B80->CAVIFileSynth::AddRef() gRefCnt=2, m_refs=2

warning: 00B43B80->CAVIFileSynth::QueryInterface() {00020020-0000-0000-c000-00
00000046} (IAVIFile)

warning: 00B43B80->CAVIFileSynth::AddRef() gRefCnt=3, m_refs=3

warning: 00B43B80->CAVIFileSynth::Load("H:\BD\RD\rd.avs", 0x0)

warning: 00B43B80->CAVIFileSynth::Release() gRefCnt=2, m_refs=2

warning: 00B43B80->CAVIFileSynth::Release() gRefCnt=1, m_refs=1

warning: 00B43B80->CAVIFileSynth::GetStream(*, 73646976(vids), 0)

(no debugging symbols found)
(no debugging symbols found)
(no debugging symbols found)
(no debugging symbols found)
(no debugging symbols found)
(no debugging symbols found)
(no debugging symbols found)
(no debugging symbols found)
(no debugging symbols found)
(no debugging symbols found)
(no debugging symbols found)
(no debugging symbols found)
(no debugging symbols found)
(no debugging symbols found)
warning: 00B43B28->CAVIStreamSynth(video)

warning: 00B43B28->CAVIStreamSynth::AddRef() (video) gRefCnt=2, m_refs=1

warning: 00B43B80->CAVIFileSynth::AddRef() gRefCnt=3, m_refs=2

warning: 00B43B80->CAVIFileSynth::Release() gRefCnt=2, m_refs=1

warning: 00B43B28->CAVIStreamSynth::Info(0027F6C8, 204) (video)

avis [info]: 1280x960 @ 23.98 fps (180360 frames)
warning: 00B43B28->CAVIStreamSynth::Info(0027F6D8, 204) (video)

x264 [info]: using SAR=1/1
x264 [info]: using cpu capabilities: MMX2 SSE2Fast SSSE3 PHADD SSE4.1 Cache64

Program received signal SIGSEGV, Segmentation fault.
0x7728f217 in ntdll!RtlDecompressBuffer () from C:\Windows\system32\ntdll.dll
(gdb) info stack
#0 0x7728f217 in ntdll!RtlDecompressBuffer ()
from C:\Windows\system32\ntdll.dll
#1 0x90909090 in ?? ()
#2 0x00000012 in ?? ()
#3 0x0bb36230 in ?? ()
#4 0x0027f26c in ?? ()
#5 0x75593593 in KERNEL32!GetNumaAvailableMemoryNode ()
from C:\Windows\syswow64\kernel32.dll
#6 0x00b40000 in ?? ()
#7 0x00000000 in ?? ()
(gdb)

[kemuri-_9]

does r1028 have problems for you as well?

[cbarroso]

Here 1028 crashes in CRF mode and 1024 works fine on Vista, core2duo.

[Emp3r0r]

yea just tried 1028, it crashed:

x264 [info]: using cpu capabilities: MMX2 SSE2Fast SSSE3 PHADD SSE4.1 Cache64

Program received signal SIGSEGV, Segmentation fault.
0x7728f217 in ntdll!RtlDecompressBuffer () from C:\Windows\system32\ntdll.dll

#0 0x7728f217 in ntdll!RtlDecompressBuffer ()
from C:\Windows\system32\ntdll.dll
#1 0x90909090 in ?? ()
#2 0x00000012 in ?? ()
#3 0x0be16230 in ?? ()
#4 0x0027f26c in ?? ()
#5 0x75593593 in KERNEL32!GetNumaAvailableMemoryNode ()
from C:\Windows\syswow64\kernel32.dll
#6 0x00020000 in ?? ()
#7 0x00000000 in ?? ()

[kemuri-_9]

try gdb again with this debug build of x264, should get more information out of it.
x264_debug.exe (http://kemuri9.net/dev/x264/x264_debug.exe)

[Emp3r0r]

here it is from debug
x264 [info]: using SAR=1/1
x264 [info]: using cpu capabilities: MMX2 SSE2Fast SSSE3 PHADD SSE4.1 Cache64
x264 [error]: ratecontrol_init: can't open stats file
x264 [error]: x264_encoder_open failed
warning: 00C43B10->CAVIStreamSynth::Release() (video) gRefCnt=1, m_refs=0

warning: 00C43B10->~CAVIStreamSynth() (video), gRefCnt = 1

warning: 00C43B68->CAVIFileSynth::Release() gRefCnt=0, m_refs=0

warning: 00C43B68->CAVIFileSynth::~CAVIFileSynth(), gRefCnt = 0

warning: DllCanUnloadNow(): gRefCnt = 0

warning: DllMain: hModule=0x10000000, ulReason=0, lpReserved=0x00000000, gRefCnt
= 0


Program exited with code 037777777777.
(gdb) info stack
No stack.

[kemuri-_9]

calling bt full when it stops working in gdb prints lots of helpful information.

but the one you just posted says it doesn't have the .stats file for the 2nd pass, so what was the arguments you used?

[Emp3r0r]

woops, your right, the debug version does not crash. Yay! I'm not sure is what wrong with 1024 and 1028.
Reading symbols from c:\programs\x264/x264.1028.exe...
(no debugging symbols found)...done.
<o-ssim --output "H:\BD\RD\rd.mkv" "H:\BD\RD\rd.avs"
The program being debugged has been started already.
Start it from the beginning? (y or n) y

Starting program: c:\programs\x264/x264.1028.exe --pass 2 --bitrate 6059 --stats
"H:\BD\RD\rd.stats" --level 4.1 --ref 4 --mixed-refs --bframes 3 --b-adapt 2 --
weightb --direct auto --deblock -1:-1 --subme 7 --trellis 2 --partitions p8x8,b8
x8,i4x4,i8x8 --8x8dct --vbv-bufsize 50000 --vbv-maxrate 50000 --me umh --threads
auto --thread-input --zones 174400,180359,q=40 --sar 1:1 --progress --no-psnr -
-no-ssim --output "H:\BD\RD\rd.mkv" "H:\BD\RD\rd.avs"
[New thread 4540.0x724]
(no debugging symbols found)
Error: dll starting at 0x76f90000 not found.
Error: dll starting at 0x75510000 not found.
Error: dll starting at 0x76f90000 not found.
Error: dll starting at 0x76ec0000 not found.
(no debugging symbols found)
(no debugging symbols found)
(no debugging symbols found)
(no debugging symbols found)
(no debugging symbols found)
(no debugging symbols found)
(no debugging symbols found)
(no debugging symbols found)
(no debugging symbols found)
(no debugging symbols found)
(no debugging symbols found)
(no debugging symbols found)
(no debugging symbols found)
(no debugging symbols found)
(no debugging symbols found)
(no debugging symbols found)
(no debugging symbols found)
(no debugging symbols found)
(no debugging symbols found)
(no debugging symbols found)
(no debugging symbols found)
(no debugging symbols found)
(no debugging symbols found)
(no debugging symbols found)
(no debugging symbols found)
(no debugging symbols found)
(no debugging symbols found)
(no debugging symbols found)
(no debugging symbols found)
(no debugging symbols found)
warning: DllMain: hModule=0x10000000, ulReason=1, lpReserved=0x00000000, gRefCnt
= 0

warning: DllGetClassObject() CLSID: CAVIFileSynth

warning: 00B73AF0->CAVIFileSynth::CAVIFileSynth()

warning: 00B73AF0->CAVIFileSynth::AddRef() gRefCnt=1, m_refs=1

warning: 00B73AF0->CAVIFileSynth::QueryInterface() {00000001-0000-0000-c000-0000
00000046} (IClassFactory)

warning: 00B73AF0->CAVIFileSynth::AddRef() gRefCnt=2, m_refs=2

warning: 00B73AF0->CAVIFileSynth::Release() gRefCnt=1, m_refs=1

warning: DllGetClassObject() result=0x0, object=00B73AF8

warning: 00B73AF0->CAVIFileSynth::CreateInstance()

warning: 00B73B48->CAVIFileSynth::CAVIFileSynth()

warning: 00B73B48->CAVIFileSynth::AddRef() gRefCnt=2, m_refs=1

warning: 00B73B48->CAVIFileSynth::QueryInterface() {00000000-0000-0000-c000-0000
00000046} (IUnknown)

warning: 00B73B48->CAVIFileSynth::AddRef() gRefCnt=3, m_refs=2

warning: 00B73B48->CAVIFileSynth::Release() gRefCnt=2, m_refs=1

warning: 00B73AF0->CAVIFileSynth::CreateInstance() result=0x0, object=00B73B48

warning: 00B73B48->CAVIFileSynth::AddRef() gRefCnt=3, m_refs=2

warning: 00B73B48->CAVIFileSynth::Release() gRefCnt=2, m_refs=1

warning: 00B73AF0->CAVIFileSynth::Release() gRefCnt=1, m_refs=0

warning: 00B73AF0->CAVIFileSynth::~CAVIFileSynth(), gRefCnt = 1

warning: 00B73B48->CAVIFileSynth::QueryInterface() {00000000-0000-0000-c000-0000
00000046} (IUnknown)

warning: 00B73B48->CAVIFileSynth::AddRef() gRefCnt=2, m_refs=2

warning: 00B73B48->CAVIFileSynth::Release() gRefCnt=1, m_refs=1

warning: 00B73B48->CAVIFileSynth::QueryInterface() {00020025-0000-0000-c000-0000
00000046} (unsupported!)

warning: 00B73B48->CAVIFileSynth::QueryInterface() {0000010b-0000-0000-c000-0000
00000046} (IPersistFile)

warning: 00B73B48->CAVIFileSynth::AddRef() gRefCnt=2, m_refs=2

warning: 00B73B48->CAVIFileSynth::QueryInterface() {00020020-0000-0000-c000-0000
00000046} (IAVIFile)

warning: 00B73B48->CAVIFileSynth::AddRef() gRefCnt=3, m_refs=3

warning: 00B73B48->CAVIFileSynth::Load("H:\BD\RD\rd.avs", 0x0)

warning: 00B73B48->CAVIFileSynth::Release() gRefCnt=2, m_refs=2

warning: 00B73B48->CAVIFileSynth::Release() gRefCnt=1, m_refs=1

warning: 00B73B48->CAVIFileSynth::GetStream(*, 73646976(vids), 0)

(no debugging symbols found)
(no debugging symbols found)
(no debugging symbols found)
(no debugging symbols found)
(no debugging symbols found)
(no debugging symbols found)
(no debugging symbols found)
(no debugging symbols found)
(no debugging symbols found)
(no debugging symbols found)
(no debugging symbols found)
(no debugging symbols found)
(no debugging symbols found)
(no debugging symbols found)
warning: 02955308->CAVIStreamSynth(video)

warning: 02955308->CAVIStreamSynth::AddRef() (video) gRefCnt=2, m_refs=1

warning: 00B73B48->CAVIFileSynth::AddRef() gRefCnt=3, m_refs=2

warning: 00B73B48->CAVIFileSynth::Release() gRefCnt=2, m_refs=1

warning: 02955308->CAVIStreamSynth::Info(0027F6C8, 204) (video)

avis [info]: 1280x960 @ 23.98 fps (180360 frames)
warning: 02955308->CAVIStreamSynth::Info(0027F6D8, 204) (video)

x264 [info]: using SAR=1/1
x264 [info]: using cpu capabilities: MMX2 SSE2Fast SSSE3 PHADD SSE4.1 Cache64

Program received signal SIGSEGV, Segmentation fault.
0x7728f217 in ntdll!RtlDecompressBuffer () from C:\Windows\system32\ntdll.dll
(gdb) bt full
#0 0x7728f217 in ntdll!RtlDecompressBuffer ()
from C:\Windows\system32\ntdll.dll
No symbol table info available.
#1 0x90909090 in ?? ()
No symbol table info available.
#2 0x00000012 in ?? ()
No symbol table info available.
#3 0x0bad6230 in ?? ()
No symbol table info available.
#4 0x0027f26c in ?? ()
No symbol table info available.
#5 0x75593593 in KERNEL32!GetNumaAvailableMemoryNode ()
from C:\Windows\syswow64\kernel32.dll
No symbol table info available.
#6 0x00b70000 in ?? ()
No symbol table info available.
#7 0x00000000 in ?? ()
No symbol table info available.
(gdb) file x264_debug.exe
A program is being debugged already.
Are you sure you want to change the file? (y or n) y
Reading symbols from c:\programs\x264/x264_debug.exe...done.
(gdb) file x264.1028.exe
A program is being debugged already.
Are you sure you want to change the file? (y or n) y

Load new symbol table from "c:\programs\x264/x264.1028.exe"? (y or n) n
Not confirmed.
<o-ssim --output "H:\BD\RD\rd.mkv" "H:\BD\RD\rd.avs"
The program being debugged has been started already.
Start it from the beginning? (y or n) y

Starting program: c:\programs\x264/x264.1028.exe --pass 2 --bitrate 6059 --stats
"H:\BD\RD\rd.stats" --level 4.1 --ref 4 --mixed-refs --bframes 3 --b-adapt 2 --
weightb --direct auto --deblock -1:-1 --subme 7 --trellis 2 --partitions p8x8,b8
x8,i4x4,i8x8 --8x8dct --vbv-bufsize 50000 --vbv-maxrate 50000 --me umh --threads
auto --thread-input --zones 174400,180359,q=40 --sar 1:1 --progress --no-psnr -
-no-ssim --output "H:\BD\RD\rd.mkv" "H:\BD\RD\rd.avs"
[New thread 952.0x137c]
Error: dll starting at 0x76f90000 not found.
Error: dll starting at 0x75510000 not found.
Error: dll starting at 0x76f90000 not found.
Error: dll starting at 0x76ec0000 not found.
warning: DllMain: hModule=0x10000000, ulReason=1, lpReserved=0x00000000, gRefCnt
= 0

warning: DllGetClassObject() CLSID: CAVIFileSynth

warning: 003E3AF0->CAVIFileSynth::CAVIFileSynth()

warning: 003E3AF0->CAVIFileSynth::AddRef() gRefCnt=1, m_refs=1

warning: 003E3AF0->CAVIFileSynth::QueryInterface() {00000001-0000-0000-c000-0000
00000046} (IClassFactory)

warning: 003E3AF0->CAVIFileSynth::AddRef() gRefCnt=2, m_refs=2

warning: 003E3AF0->CAVIFileSynth::Release() gRefCnt=1, m_refs=1

warning: DllGetClassObject() result=0x0, object=003E3AF8

warning: 003E3AF0->CAVIFileSynth::CreateInstance()

warning: 003E3B48->CAVIFileSynth::CAVIFileSynth()

warning: 003E3B48->CAVIFileSynth::AddRef() gRefCnt=2, m_refs=1

warning: 003E3B48->CAVIFileSynth::QueryInterface() {00000000-0000-0000-c000-0000
00000046} (IUnknown)

warning: 003E3B48->CAVIFileSynth::AddRef() gRefCnt=3, m_refs=2

warning: 003E3B48->CAVIFileSynth::Release() gRefCnt=2, m_refs=1

warning: 003E3AF0->CAVIFileSynth::CreateInstance() result=0x0, object=003E3B48

warning: 003E3B48->CAVIFileSynth::AddRef() gRefCnt=3, m_refs=2

warning: 003E3B48->CAVIFileSynth::Release() gRefCnt=2, m_refs=1

warning: 003E3AF0->CAVIFileSynth::Release() gRefCnt=1, m_refs=0

warning: 003E3AF0->CAVIFileSynth::~CAVIFileSynth(), gRefCnt = 1

warning: 003E3B48->CAVIFileSynth::QueryInterface() {00000000-0000-0000-c000-0000
00000046} (IUnknown)

warning: 003E3B48->CAVIFileSynth::AddRef() gRefCnt=2, m_refs=2

warning: 003E3B48->CAVIFileSynth::Release() gRefCnt=1, m_refs=1

warning: 003E3B48->CAVIFileSynth::QueryInterface() {00020025-0000-0000-c000-0000
00000046} (unsupported!)

warning: 003E3B48->CAVIFileSynth::QueryInterface() {0000010b-0000-0000-c000-0000
00000046} (IPersistFile)

warning: 003E3B48->CAVIFileSynth::AddRef() gRefCnt=2, m_refs=2

warning: 003E3B48->CAVIFileSynth::QueryInterface() {00020020-0000-0000-c000-0000
00000046} (IAVIFile)

warning: 003E3B48->CAVIFileSynth::AddRef() gRefCnt=3, m_refs=3

warning: 003E3B48->CAVIFileSynth::Load("H:\BD\RD\rd.avs", 0x0)

warning: 003E3B48->CAVIFileSynth::Release() gRefCnt=2, m_refs=2

warning: 003E3B48->CAVIFileSynth::Release() gRefCnt=1, m_refs=1

warning: 003E3B48->CAVIFileSynth::GetStream(*, 73646976(vids), 0)

warning: 02A15308->CAVIStreamSynth(video)

warning: 02A15308->CAVIStreamSynth::AddRef() (video) gRefCnt=2, m_refs=1

warning: 003E3B48->CAVIFileSynth::AddRef() gRefCnt=3, m_refs=2

warning: 003E3B48->CAVIFileSynth::Release() gRefCnt=2, m_refs=1

warning: 02A15308->CAVIStreamSynth::Info(0027F6C8, 204) (video)

avis [info]: 1280x960 @ 23.98 fps (180360 frames)
warning: 02A15308->CAVIStreamSynth::Info(0027F6D8, 204) (video)

x264 [info]: using SAR=1/1
x264 [info]: using cpu capabilities: MMX2 SSE2Fast SSSE3 PHADD SSE4.1 Cache64

Program received signal SIGSEGV, Segmentation fault.
0x7728f217 in ntdll!RtlDecompressBuffer () from C:\Windows\system32\ntdll.dll
(gdb) bt full
#0 0x7728f217 in ntdll!RtlDecompressBuffer ()
from C:\Windows\system32\ntdll.dll
No symbol table info available.
#1 0x90909090 in ?? ()
No symbol table info available.
#2 0x00000012 in ?? ()
No symbol table info available.
#3 0x0bc36230 in ?? ()
No symbol table info available.
#4 0x0027f26c in ?? ()
No symbol table info available.
#5 0x75593593 in KERNEL32!GetNumaAvailableMemoryNode ()
from C:\Windows\syswow64\kernel32.dll
No symbol table info available.
#6 0x003e0000 in ?? ()
No symbol table info available.
#7 0x00000000 in ?? ()
No symbol table info available.
(gdb) See Above (http://forum.doom9.org/showthread.php?p=1213197#post1213197) for arguments

[kemuri-_9]

all right, well this is getting more and more confusing...
the debug i provided was just r1028 straight from the repository with debug information & unprofiled,
guess it's time to try one with the same patches that skystrife used at least.

x264_mod_debug.exe (http://kemuri9.net/dev/x264/x264_mod_debug.exe)

see how it goes with this.

[Sharktooth]

or try profiling it...

[kemuri-_9]

that would be the next step if the + patches one works fine.

[Audionut]

Here 1028 crashes in CRF mode and 1024 works fine on Vista, core2duo.

My 1028 build that has no patches and is profiled works fine for me. I'd say it is a problem with one of the patches skystrife is using.

I use gcc 3.4.5 aswell.

[Dark Shikari]

If it turns out CFLAGS cause the crash (e.g. -O3, etc) you can compile a working debug version with --extra-cflags="-ffast-math -O3" and --enable-debug as well.

[Emp3r0r]

tried mod_debug

x264 [info]: using SAR=1/1
x264 [info]: using cpu capabilities: MMX2 SSE2Fast SSSE3 PHADD SSE4.1 Cache64
≡¡║ε½½½½½½½½■ε■ε■ε■ = (null)aram: ¡║
x264 [error]: failed to parse zones
x264 [error]: x264_encoder_open failed
warning: 02A55308->CAVIStreamSynth::Release() (video) gRefCnt=1, m_refs=0

warning: 02A55308->~CAVIStreamSynth() (video), gRefCnt = 1

warning: 010D3B60->CAVIFileSynth::Release() gRefCnt=0, m_refs=0

warning: 010D3B60->CAVIFileSynth::~CAVIFileSynth(), gRefCnt = 0

warning: DllCanUnloadNow(): gRefCnt = 0

warning: DllMain: hModule=0x10000000, ulReason=0, lpReserved=0x00000000, gRefCnt
= 0


Program exited with code 037777777777.