x264: --no-coef-thresh

[Sirber]

Can someone explain what is --no-coef-thresh?

Thanks!

[ChronoCross]

it's now --no-dct-decimate. I can't explain it unfortunately.

[Sirber]

Should I use it or not?

[shon3i]

You try, you see. Anyway i think can only be a good.

@ChronnoCroos, SharkTooth when you back AQ patch becouse i need them, thanks

[DarkZell666]

You try, you see. Anyway i think can only be a good.

For sure that's the best solution, but what if the effect isn't visible right away ? ;)

Visually I don't have a clue to what it should change either lol ;)

*Might do some test encodes later*

[Sirber]

alright, gonna update realanime

[foxyshadis]

You try, you see. Anyway i think can only be a good.

@ChronnoCroos, SharkTooth when you back AQ patch becouse i need them, thanks
AQ has been in all of CC's builds, except for a few where it was broken a couple of months ago. ^^;; In fact he just posted that we have incarnation 3 of AQ freshly at our disposal, hopefully this one will be the Nero-killer. :p
alright, gonna update realanime
Haha, you put the option in RA without testing or even knowing what it does? xD

[ChronoCross]

Haali's AQ is no longer compatible. It will be updated once the Alternate Decimation is commited to cvs. However I recommend testing the latest version of AQ (in build 502). It might provide better results.

Also we discovered that the Haali patch was only affecting I-frames. So how you guys managed to see it doing anything is beyond me....I think it had a placebo effect on all of you =P.

[shon3i]

hopefully this one will be the Nero-killerThis new AQ patch seem to work fine, i now doing some test with very dark movies.

[Sirber]

Haha, you put the option in RA without testing or even knowing what it does? xDWell, I fellow the recommendations :p

[edit]

Alright, not updating :(

[foxyshadis]

If it was only affecting I-frames, it must have broken recently (I never tested it the last 50 revisions), because I used elecard streameye on one back when it was first updated after breaking, and it definitely affected every frame.

Actually, sirber, it looks like that might be a useful option to bundle with no-fast-pskip in RA, based on the explanation in svn. Sounds like it's mostly meant for higher bitrates, since that skip's an optimization to raise quality per bit.

[Sirber]

I mostly do lower bitrates, so I should not use it?

[ChronoCross]

I think it helps even in lower bitrates. from my testing anyway.

[ShAQ]

I think it helps even in lower bitrates. from my testing anyway.

Can you support us with a sample?! :D

[ChronoCross]

Unfortunately no. The clips I have are currently copyrighted.

[Sirber]

[17:32] Sirber: did you try --no-dct-decimate ?
[17:32] DeathTheSheep: yes
[17:32] DeathTheSheep: of course
[17:33] Sirber: so?
[17:33] DeathTheSheep: it takes the 3rd log coefficient, uses SAD to merge with other ratio...forgot wat its called
[17:33] DeathTheSheep: basically, makes files bigger per quant
[17:33] Sirber: is it good or not? :|
[17:33] DeathTheSheep: but better for crisp detail
[17:33] DeathTheSheep: not for anime, and especially not for low bitrate anime
[17:33] Sirber: k

[DeathTheSheep]

HAH looks like I lied (or "didn't evaluate my unbased claim worth a crap" ;))

decimate scores. yah.

and looky here.
if( !array_non_zero( (int*)dct2x2, sizeof(dct2x2)/sizeof(int) ) )
yep.... arrays, 2x2dct stuff goin' on

so now we take off the threshold measure by old school 1-dimention variable coeff, so's no more dct decimation or somethin (an option to do so no more).

Makes most o' my p frames bigger. better quality? that's debatable. no sad in the picture here, just different way of decimating p-frame quant coeffs n' stuff.

heh, can't believe Sir berube posted that LOL... sorry 'b0ut that guyz :)

[foxyshadis]

Daaaaaaaaaaaang, the latest version is fast, or maybe it's just my new computer; I get over 100 fps on vcd-size video on the first pass, and 16-20 fps on the second with every quality option enabled (except esa). @_@ Anyway, it looks decent at 180 kbps, much better than the last time I tried (acceptable was around 240). I'll post screens or clips later once I can more thoroughly test it.

Freaking useless streameye keeps crashing after a few frames while I try to check out the effect of the AQ. What a piece of uselessly expensive vaporware.

[Sirber]

I get 150FPS on iPod size on first pass, and about 30 on second one on my 1½ years old 3000+ :)

[DeathTheSheep]

Dear foxyshadis,

Daaaaaaaaaaaang, the latest version is fast.
Which version do you speak of, and who issued the build? I anticipate my encoding to be as fast as that which you outlined above; thus does arrive my need to know such select pieces of information. Thank you.

Regards,
YourDeadSheepDeathTheSheepyFellow

[Sharktooth]

My builds should be a bit faster than ChronoCross builds due to gcc optimizations on both gpac lib and x264.
However feel free to do your own benchmarks.

[Sirber]

I never notice speed difference when it's less than 2 FPS ;)

[emmel]

My builds should be a bit faster than ChronoCross builds due to gcc optimizations on both gpac lib and x264.
However feel free to do your own benchmarks.

With the megui def.params its about 1% faster than the home made gcc compile from "./confugure && make && make installl".

Whats the "worst case test" that you could test the optimizations against?

[Sirber]

so fprofiled only boost speed by 1%? sad...

[akupenguin]

You'd rather that gcc were less efficient without fprofile? Or that x264 didn't spend the majority of its time in hand-coded asm?

[DeathTheSheep]

My builds should be a bit faster than ChronoCross builds due to gcc optimizations on both gpac lib and x264.
However feel free to do your own benchmarks.

Yeah, 4.7 Sharktooth vs 4.2 CC (build 503(B) tested)... That 12% increase for my clip was absolutely amazing for me...and that's with 10 refs and all quality settings enabled on a complex source (fafner).
Of course, it wasn't a very professional test or accurate benchmark, it's just what that meGUI thing (darn meGUI thing ;)) reported to me as the fps when it was done. Identical settings, source, system load (as far as I could tell).

12% though... that's pretty steep. I can't believe my own results now that I actually calculated this. How can a different build give that much of a speedup? It's insane, that's what it is :eek: :p
Now I just can't wait till the VFW patches are added :devil:

[ChronoCross]

I ues the default safe optimizations that gcc chooses. if you force additional ones usiong custom compiler options like sharkthooth does then in all likelyhood you will get better results.

[DeathTheSheep]

Ok, sounds good! That word "safe" is bugging me though... does it mean that your builds will tend to be more stable, or that gcc standard optimizations are usually the fastest and improperly tweaking them could cause slowdown)?

Well, either way, I *did* notice the resulting filesize between you 2 guys' builds was different, even with same patches and options for both... similar, but different at q28. But that's natural, I s'pose, cuz different optimizations, rounding, compiler nitty-gritty functions, etc would be altered in these custom optimizations that wouldn't be present with the standard ones.

[emmel]

Please provide me with a simple test that reveals the max. gain for the gcc optimizations under consideration. thx.

[DeathTheSheep]

Max. gain, eh? Well, since all sources are different, the builds change constantly, optimizations are constantly being worked on, and x264 itself is being optimized daily via the revisions, each of which might throw off old comparison logic or gcc optimization effects, such a comparison might be useless for more than a proof-of-concept-type stint.:o

[emmel]

If its not too much asked, please upload a 10G sample of the content somewhere, and let me know the x264 options to use for verification. I'll then do a verification and report on the differences.

[DeathTheSheep]

please upload a 10G sample
Full DVDs aren't even 10 gigs... You mean 10mb right...? (I hope :O)

[emmel]

Any sample you wish will do fine.

[edit] The 10G would be nice, though... :)

[foxyshadis]

http://www.apple.com/quicktime/guide/hd/

Pick a few you like. Use the script:
DirectShowSource("lol.mov")
LanczosResize(720,480)

Test.
???
Profit.
(With apologies to mug funky.)

As for unsafe optimizations, that usually results in crashes if anything. Shark's been using everything that works in all the earlier builds, and you didn't see data corruption problems then.

[Sirber]

Test.
???
Profit.Not always that easy to make profit :(

[emmel]

http://www.apple.com/quicktime/guide/hd/

Pick a few you like. Use the script:
DirectShowSource("lol.mov")
LanczosResize(720,480)


Ok, I picked up this one:

http://movies.apple.com/movies/disney/roving_mars/roving_mars_h720p.mov

and converted into mp4.

Then, I used your avs-script and the following x264 command line:


x264.exe --crf 22 --threads 2 --thread-input --no-fast-pskip --direct auto --ref 3 --bframes 3 --b-pyramid --bime --weightb --subme 6 --filter -2,-2 --trellis 1 --analyse p8x8,b8x8,i4x4 --me umh --progress --output roving_mars.264 roving_mars.avs


I tested two executables:

1) I pickep up the newest x264.exe from x264.nl and run it:

encoded 2377 frames, 15.12 fps, 791.44 kb/s
PSNR Mean Avg 47.604 Global 44.455

2) I did a svn checkout and complied the source with gcc simply as "./configure && make && make install":

encoded 2377 frames, 15.09 fps, 796.13 kb/s
PSNR Mean Avg: 47.821 Global 44.456

I did the conversion several times with both executables. Sometimes the precompiled version was faster, sometimes it was slower. Really hard to find any differences at all, except that the home made compilation produces slightly bigger files. The test was performed with amd x2 4400+.

[edit] Actually it was vice versa: it was the precompiled executable that for some reason prioduced a slightly bigger file (9635k vs. 9579k).

[Sharktooth]

x264.nl ones are not my builds...
my builds are hosted here: http://files.x264.nl/?dir=./Sharktooth
also you cant compare them with x264.nl builds coz i added some patches and the results would be different in filesize, quality and encoding speed...
a correct comparison would be:
1) get the sources from SVN (same revision as my build... you may want to get gpac too)
2) apply all the patches i applied
3) configure and compile
4) compare the speed.

[Sirber]

That is to compare fprofiled and normal?

[Sharktooth]

that is to compare my standard SVN builds + patches (with or without profiling) vs my profiled and optimized builds.

[emmel]

x264.nl ones are not my builds...
my builds are hosted here: http://files.x264.nl/?dir=./Sharktooth
also you cant compare them with x264.nl builds coz i added some patches and the results would be different in filesize, quality and encoding speed...
a correct comparison would be:
1) get the sources from SVN (same revision as my build... you may want to get gpac too)
2) apply all the patches i applied
3) configure and compile
4) compare the speed.

The exe was exactly yours, from the above mentioned directory. I also have the latest gpac.

But you are right, the patches make a difference, and thats why e.g. I'm using your builds instead of my own.

A correct comparison would of course be based on the exactly same source with and without additional optimizations. A good test should also be clean, simple, reproducible, and everyone should be able to perform it without extra hazzle. Your proposed test setup is good, but unfortunately fails in this respect. Maybe you could make two simple test runs, if its not too much trouble, and report on the speedup of your additional optimizations?

Anyways, my post was not supposed to be opinionated. I simply asked for a simple test to reproduce the reported >10% speedup, to check if there is something to learn for future.

[GodofaGap]

Ok, I did a small test with Sharttooth's build, an own build + patches and an own build + patches + fprofiled. I used a small ~1000 frame source, with 3 different option sets.

option set 1 (normal): --crf 22 -b 2 -r 4 --8x8dct -t 1
option set 2 (slow): --crf 18 -b 2 -r 4 --8x8dct -t 1 --me umh --subme 6 --b-rdo --analyse "all"
option set 3 (fast): --crf 26 -b 2 --me dia --subme 2 --analyse "none"

Below is a table with normalized encoding times (ST's are 1.000).


speed | ST | own | own-fp
-------+--------+-------+---------
normal | 1.000 | 1.023 | 1.004
slow | 1.000 | 1.040 | 1.003
fast | 1.000 | 1.026 | 1.006


So it appears compiling with fprofiled is certainly benifical (as it appears here at least 2%), but the gain of Sharktooth's build over my own with fprofiled is a bit smaller.

[Sharktooth]

someone said 10% or 12%. it sounds a bit too much to me. I said "a bit" faster and the GodofaGap findings are what i would expect (more or less, depending on the system).
also, most of the critical parts in x264 are hand optimized in .asm, so gcc optimizations are not going to provide any huge speedup... just marginal but still noticeable expecially on long encodes.