x264 development

[nhakobian]

Is much happening with x264 development? The last checkin was three months ago.

Perhaps x265 is getting a lot of attention from the core devs :)?

http://git.videolan.org/gitweb.cgi?p=x264.git;a=shortlog

There appears to have been some things done since then that are listed on Dark Shikari's github:

https://github.com/DarkShikari/x264-devel/commits/master

[iwod]

The latest newsletter from 30th of Oct.

This is the fortieth x264 development newsletter. This is a regular
email containing updates on fixes and improvements in the most recent
x264 push, along with updates on what's coming next. Previous
versions can be found in the mailing list archives.

Fixes:

Work around an FFMS bug with empty index files.

Fix INSTALL in configure on Solaris systems.

Fix possible crashes in resize/crop filters on high bit depth input.

Fix shared library compilation on the original Windows MinGW toolchain.

Fix compilation in the case that the HAVE_LOG2F check fails spuriously.

Include dependency libraries in pkg-config generation.

Don't try to generate a .git version number on build if .git isn't
present (e.g. in snapshots)

Update code to the current libav/ffmpeg API.

Improvements:

Don't warn in non-verbose mode if VBV underflow occurs with CRF-max
(since the user likely wanted it to occur).

Slight speedup in chroma BI analysis.

Make encoder_reconfig more threadsafe: fixes some rare crash issues
with reconfiguring with frame threads.

Add a --filler option to generate streams with filler without using NAL HRD.

Add AVC Intra 1080p50/60 Class 100 parameters and some further AVC
Intra compatibility hacks/fixes.

Add L-SMASH support as a preferred alternative to GPAC for mp4 muxing.

Remove the --visualize option, as it is very old cruft and probably
does not work with many combinations of settings anymore (as it hasn't
been maintained).

Jason Garrett-Glaser

The x264 Team

Well at least we know DS is alive and well. I wonder if he had stop writing code for x264 and move on.

[LigH]

A newsletter or blog entry won't appear with each commit. Probably only when something "revolutionary" happens. Or when silent evolution progressed long enough. Following the commit log is certainly more reliable.

[ NO PANIC ]

And not all x264 developers are part of the x265 development too, I believe.

[MasterNobody]

And not all x264 developers are part of the x265 development too, I believe.
I would say even more. Almost zero x264 developers (I don't know anyone) are part of active x265 developers (which are mostly MulticoreWare workers). This doesn't mean that we don't support/help them to understand and adapt x264 algorithm but that is almost only this.

[Sharc]

No plans to support MVC with x264 in near future?

[LoRd_MuldeR]

No plans to support MVC with x264 in near future?

See here:
https://www.leffster.se/showthread.php?t=168010

Also, FWIW, the MVC patch can still be found at:
http://pastebin.com/qZ1xSmuc

[Sharc]

See here:
https://www.leffster.se/showthread.php?t=168010

Also, FWIW, the MVC patch can still be found at:
http://pastebin.com/qZ1xSmuc
These activities look a bit orphaned. Has anyone tried?
CoreCodec offerings apply to the MVC Decoder only it appears ....

[pistacho]

The latest newsletter from 30th of Oct.

Well at least we know DS is alive and well. I wonder if he had stop writing code for x264 and move on.

Bump dates to 2014…

https://github.com/DarkShikari/x264-devel/commits/master

:):):)

[Selur]

when using:
mencoder -lavdopts threads=8 -really-quiet -of rawvideo -o - -ovc raw -vf scale,format=i420,scale,format=444p -forcedsubsonly -nosub -nosound -mc 0 "H:\Temp\encodingTempAvisynthSkript_08_44_17_5010.avs" | x264 --preset ultrafast --tune fastdecode --qp 0 --profile high444 --no-psy --sar 32:27 --non-deterministic --colormatrix bt470bg --input-csp yv24 --fps 30000/1001 --input-res 720x480 --output "H:\Temp\lossless_08_44_17_5010_03.264" -
why does x264 convert to yuv420?
raw [info]: 720x480p 32:27 @ 30000/1001 fps (cfr)
resize [warning]: converting from yuv444p to yuv420p
x264 [info]: using SAR=32/27
x264 [info]: using cpu capabilities: MMX2 SSE2Fast SSSE3 SSE4.2 AVX AVX2 FMA3 LZCNT BMI2
x264 [info]: profile High 4:4:4 Predictive, level 3.0, 4:2:0 8-bit
x264 [info]: frame I:3 Avg QP: 0.00 size: 77126
x264 [info]: frame P:705 Avg QP: 0.00 size: 83652
x264 [info]: mb I I16..4: 100.0% 0.0% 0.0%
x264 [info]: mb P I16..4: 70.9% 0.0% 0.0% P16..4: 28.7% 0.0% 0.0% 0.0% 0.0% skip: 0.3%
x264 [info]: coded y,uvDC,uvAC intra: 89.6% 83.0% 82.6% inter: 93.6% 88.2% 87.9%
x264 [info]: i16 v,h,dc,p: 65% 35% 0% 0%
x264 [info]: i8c dc,h,v,p: 17% 34% 48% 1%
x264 [info]: kb/s:20049.75
encoded 708 frames, 14.21 fps, 20049.75 kb/s
mencoder converted to 4:2:0 to 4:4:4, the profile is High444 -> why does x264 revert the 4:2:0 to 4:4:4 conversion?

Cu Selur

Ps.: I used:
x264 0.142.2389 956c8d8
(libswscale 2.5.101)
(libavformat 55.25.101)
(ffmpegsource 2.17.4.0)
built by Komisar on Jan 22 2014, gcc: 4.8.2 (multilib.generic.Komisar)
configuration: --bit-depth=8 --chroma-format=all
x264 license: GPL version 2 or later
libswscale/libavformat/ffmpegsource license: GPL version 2 or later

[nevcairiel]

You need to set --output-csp, or it'll always encode 420.

[Selur]

Thanks for clearing that up. :)

[mastrboy]

Regarding qpfiles, is the 2 following "statements" threated as the same?
"50 K"
and
"50 K -1"

And does qpfiles support comments within the file? with for example a "#" or "//" pre-sign?

[MasterNobody]

Regarding qpfiles, is the 2 following "statements" threated as the same?
"50 K"
and
"50 K -1"

yes
And does qpfiles support comments within the file? with for example a "#" or "//" pre-sign?
no

[mastrboy]

Thanks for the fast and precise answer :)

[Selur]

Seems like the sample (http://www.multiupload.nl/CAS9LDCIVI) is too short for 10bit rate control to work properly:

Avisynth script used:
SetMemoryMax(768)
SetMTMode(6,0) # change MT mode
LoadPlugin("G:\Hybrid\avisynthPlugins\DGDecodeNV.dll")
LoadPlugin("G:\Hybrid\avisynthPlugins\TIVTC.dll")
LoadPlugin("G:\Hybrid\avisynthPlugins\mt_masktools-26.dll")
LoadPlugin("G:\Hybrid\avisynthPlugins\DctFilter.dll")
LoadPlugin("G:\Hybrid\avisynthPlugins\deblock.dll")
LoadPlugin("G:\Hybrid\avisynthPlugins\MosquitoNR.dll")
LoadPlugin("G:\Hybrid\avisynthPlugins\flash3kyuu_deband.dll")
Import("G:\Hybrid\avisynthPlugins\Deblock_QED.avs")
# loading source
DGSource(dgi="H:\Temp\1452664460mkv_deb1536f480475f7d593219aa1afd74c_18716.dgi",fieldop=0)
RequestLinear(rlim=50,clim=50)
# deblocking
SetMTMode(2) # change MT mode
Deblock_QED(quant1=24,quant2=26,aOff1=1,bOff1=1,aOff2=2,bOff2=2,uv=3)
# deringing
MosquitoNR(strength=16,restore=128,radius=2,threads=0)
# debanding
f3kdb(range=15,sample_mode=2,dither_algo=3,grainY=64,grainC=64,dynamic_grain=true,Y=64,Cb=64,Cr=64,blur_first=true,keep_tv_range=false)
return last

2pass target bit rate failing on me on short sample:
1st pass:
"G:\Hybrid\x264-10bit.exe" --pass 1 --bitrate 2889 --profile high10 --level 4.1 --direct auto --b-adapt 2 --rc-lookahead 60 --qpmax 81 --deblock -1:-1 --non-deterministic --stats "H:\Temp\DeblockQED, Flash 3k Deband, MosquitoNR_10bit_10_16_58_5710_03.stats" --output-csp i420 --input-res 1280x672 --output NUL -
output:
raw [info]: 1280x672p 0:0 @ 25/1 fps (cfr)
x264 [info]: using cpu capabilities: MMX2 SSE2Fast SSSE3 SSE4.2 AVX AVX2 FMA3 LZCNT BMI2
x264 [info]: profile High 10, level 4.1, 4:2:0 10-bit
x264 [info]: cabac=1 ref=1 deblock=1:-1:-1 analyse=0x1:0 me=dia subme=2 psy=1 fade_compensate=0.00 psy_rd=1.00:0.00 mixed_ref=0 me_range=16 chroma_me=1 trellis=0 8x8dct=0 cqm=0 deadzone=21,11 fast_pskip=1 chroma_qp_offset=0 threads=12 lookahead_threads=5 sliced_threads=0 nr=0 decimate=1 interlaced=0 bluray_compat=0 constrained_intra=0 fgo=0 bframes=3 b_pyramid=2 b_adapt=2 b_bias=0 direct=3 weightb=1 open_gop=0 weightp=2 keyint=250 keyint_min=25 scenecut=40 intra_refresh=0 rc_lookahead=60 rc=abr mbtree=1 bitrate=2889 ratetol=1.0 qcomp=0.60 qpmin=0 qpmax=81 qpstep=4 ip_ratio=1.40 aq=1:1.00
x264 [info]: frame I:1 Avg QP:17.00 size:211322
x264 [info]: frame P:41 Avg QP:35.77 size: 25045
x264 [info]: frame B:77 Avg QP:35.93 size: 25327
x264 [info]: consecutive B-frames: 5.9% 5.0% 55.5% 33.6%
x264 [info]: mb I I16..4: 99.0% 0.0% 1.0%
x264 [info]: mb P I16..4: 46.4% 0.0% 0.0% P16..4: 11.8% 0.0% 0.0% 0.0% 0.0% skip:41.8%
x264 [info]: mb B I16..4: 15.4% 0.0% 0.0% B16..8: 10.9% 0.0% 0.0% direct: 2.9% skip:70.8% L0:32.6% L1:56.8% BI:10.5%
x264 [info]: final ratefactor: 16.97
x264 [info]: direct mvs spatial:79.2% temporal:20.8%
x264 [info]: coded y,uvDC,uvAC intra: 47.9% 52.1% 44.3% inter: 2.8% 5.0% 0.2%
x264 [info]: i16 v,h,dc,p: 24% 15% 48% 13%
x264 [info]: i4 v,h,dc,ddl,ddr,vr,hd,vl,hu: 4% 2% 68% 4% 7% 3% 5% 4% 3%
x264 [info]: i8c dc,h,v,p: 81% 9% 5% 5%
x264 [info]: Weighted P-Frames: Y:7.3% UV:7.3%
x264 [info]: kb/s:5358.60
encoded 119 frames, 18.53 fps, 5358.61 kb/s, duration 0:00:06.42
finished after 00:00:06.498

2nd pass:
"G:\Hybrid\x264-10bit.exe" --preset slower --tune film --pass 2 --bitrate 2889 --profile high10 --level 4.1 --ref 3 --qpmax 81 --vbv-maxrate 150000 --vbv-bufsize 187500 --non-deterministic --colormatrix bt709 --stats "H:\Temp\DeblockQED, Flash 3k Deband, MosquitoNR_10bit_10_16_58_5710_03.stats" --output-csp i420 --input-res 1280x672 --output "H:\Temp\DeblockQED, Flash 3k Deband, MosquitoNR_10bit_10_16_58_5710_04.264" -
output:
raw [info]: 1280x672p 0:0 @ 25/1 fps (cfr)
x264 [info]: using cpu capabilities: MMX2 SSE2Fast SSSE3 SSE4.2 AVX AVX2 FMA3 LZCNT BMI2
x264 [info]: profile High 10, level 4.1, 4:2:0 10-bit
x264 [info]: cabac=1 ref=3 deblock=1:-1:-1 analyse=0x3:0x133 me=umh subme=9 psy=1 fade_compensate=0.00 psy_rd=1.00:0.15 mixed_ref=1 me_range=16 chroma_me=1 trellis=2 8x8dct=1 cqm=0 deadzone=21,11 fast_pskip=1 chroma_qp_offset=-3 threads=12 lookahead_threads=1 sliced_threads=0 nr=0 decimate=1 interlaced=0 bluray_compat=0 constrained_intra=0 fgo=0 bframes=3 b_pyramid=2 b_adapt=2 b_bias=0 direct=3 weightb=1 open_gop=0 weightp=2 keyint=250 keyint_min=25 scenecut=40 intra_refresh=0 rc_lookahead=60 rc=2pass mbtree=1 bitrate=2889 ratetol=1.0 qcomp=0.60 qpmin=0 qpmax=81 qpstep=4 cplxblur=20.0 qblur=0.5 vbv_maxrate=150000 vbv_bufsize=187500 nal_hrd=none filler=0 ip_ratio=1.40 aq=1:1.00
x264 [info]: frame I:1 Avg QP:26.00 size: 11415
x264 [info]: frame P:41 Avg QP:35.69 size: 73080
x264 [info]: frame B:77 Avg QP:38.49 size: 47455
x264 [info]: consecutive B-frames: 5.9% 5.0% 55.5% 33.6%
x264 [info]: mb I I16..4: 99.8% 0.2% 0.0%
x264 [info]: mb P I16..4: 12.1% 25.4% 1.5% P16..4: 17.8% 0.6% 5.7% 0.0% 0.0% skip:37.0%
x264 [info]: mb B I16..4: 3.8% 5.1% 1.6% B16..8: 21.5% 1.3% 0.5% direct: 1.6% skip:64.7% L0:38.8% L1:55.1% BI: 6.0%
x264 [info]: 8x8 transform intra:57.2% inter:55.7%
x264 [info]: direct mvs spatial:80.5% temporal:19.5%
x264 [info]: coded y,uvDC,uvAC intra: 60.4% 65.4% 61.0% inter: 2.3% 6.1% 1.8%
x264 [info]: i16 v,h,dc,p: 9% 15% 33% 43%
x264 [info]: i8 v,h,dc,ddl,ddr,vr,hd,vl,hu: 13% 5% 47% 6% 6% 6% 6% 6% 6%
x264 [info]: i4 v,h,dc,ddl,ddr,vr,hd,vl,hu: 5% 5% 23% 11% 12% 11% 11% 11% 12%
x264 [info]: i8c dc,h,v,p: 73% 12% 2% 13%
x264 [info]: Weighted P-Frames: Y:7.3% UV:7.3%
x264 [info]: ref P L0: 48.3% 27.4% 23.9% 0.3%
x264 [info]: ref B L0: 68.2% 21.8% 9.9%
x264 [info]: ref B L1: 85.5% 14.5%
x264 [info]: kb/s:11196.12
encoded 119 frames, 11.33 fps, 11196.12 kb/s, duration 0:00:10.50
finished after 00:00:10.566

Using 8bit everything seems fine.

I used:
x264 0.142.2389kMod 956c8d8
(libswscale 2.5.101)
(libavformat 55.25.101)
(ffmpegsource 2.17.4.0)
built by Komisar on Jan 22 2014, gcc: 4.8.2 (multilib.generic.Komisar)
configuration: --bit-depth=10 --chroma-format=all
x264 license: GPL version 2 or later
libswscale/libavformat/ffmpegsource license: GPL version 2 or later

Cu Selur

Ps.: I know the clip is really short and mainly contains of black frames, but I thought I should report it since 8bit encoding worked fine and only 10bit encoding fumbled.

[Selur]

got it, using a larger "--rc-lookahead" solved the problem with the clip

[MasterNobody]

Selur
1) if you going to make bug reports than use clean version of x264 (no patched one).
2) why do you use so different params for 1-st and 2-nd pass?
3) use source directly or provide already filtered source because developers not going to install all needed plugins and scripts only to find out that they can't reproduce your bug.

P.S. I can't reproduce it with clean build + your sample without any filtering + your command lines. 2-nd pass gives 2799.90 kb/s here (while 1-st pass gives 16670.55 kb/s).

[Selur]

regarding
1.: used clean build first
2. What parameter do you think has changed? All differences should be due to the normal 'fast 1st pass' adjustments
3. also happened with the filtered source

I can't reproduce it with clean build + your sample without any filtering + your command lines. 2-nd pass gives 2799.90 kb/s here
will look into it some more tomorrow (you did use a 10bit build right?)

[LigH]

You should probably use the same preset and tuning for both 1st and 2nd pass, because you may collect statistics for a different complexity than you intend to encode with. Except you really know that the "fast 1st pass" would reset just the same basic options as omitting the preset and tuning.

[Selur]

Except you really know that the "fast 1st pass" would reset just the same basic options as omitting the preset and tuning.
I hope I do. Otherwise I made a mistake I'm not aware of. :)

[benwaggoner]

I note a ton of commits have been coming in; 15 in the last two days!

github.com/DarkShikari/x264-devel/commits/master (https://github.com/DarkShikari/x264-devel/commits/master)

It's nice to see some new stuff coming in!

[Dark Shikari]

The dates are misleading; they're not dates of authorship, but dates of last edit, and amending a commit causes all commits above it to be edited.

It's really roughly one month of commits.

[turbojet]

Any chance x264 officially adds support to prevent standby/sleep?

Standby/sleep breaks encoding with x264 but it keeps encoding the last frame it recieved until the framecount is reached, so it's not very obvious.

http://forum.doom9.org/showthread.php?p=1659494#post1659494 is a start. It only works on windows vista/7/8 though, pre-vista can be added with one line and then mac and linux would be a few more lines.

[Dark Shikari]

Any chance x264 officially adds support to prevent standby/sleep?

Standby/sleep breaks encoding with x264Do you have a specific test case that fails? Because it has always worked fine on my machine; I confirmed it works just now in a test and many times in the past I've used sleep to put an encode on hold while I carried my laptop elsewhere.

[turbojet]

Start a cli encode, send it into deep sleep (S3), when it wakes up x264 appears to keep encoding but the bitrate will continually drop because it's encoding the same frame.

On a laptop, it might be going into S1, mine does if I close the lid, power light blinks but it's set to S3 after 1 hour, the light stops blinking.

[Dark Shikari]

It works fine for me; are you sure it's not a problem with your input method, e.g. an Avisynth script/plugin with problems, or a bad DirectShow filter? I have never had an issue with sleep mode breaking encodes on multiple computers and operating systems in my entire history using x264.

[turbojet]

I'm pretty sure I've reproduced it numerous times with direct x264 but that was months ago, will test again in a few hours.

Most gui's work around this program by forciing no sleep. Others are working around it in batch scripts in the patch thread but that comes with some issues mentioned in the thread.

[mandarinka]

Sleeping (S3, S4) always worked for me with ffvideo/avisynth input (and dgdecode and ffvideosource in there). Even feeding via avs2yuv over - is working as far as I can see (Windows XP, Windows 8).

Do you use some unreliable sources, like directshowsource? Or something that uses GPU acceleration? Or does your harddrive or other storage has issues waking up /maybe file handles are broken after sleep - I've seen external HDDs having issues there/?

[turbojet]

You guys are right, I must have only been testing dss2 which is what I normally use. It makes sense to not always prevent sleep. Maybe a switch would be more appropriate?

A common situation for me is starting an encode and leaving the computer alone for hours. Upon returning I'd expect the encode to be done and computer sleeping but with a vanilla x264 build the result is a partial encode with computer sleeping.

[LigH]

Just what we keep preaching for what feels like "decades": Try to prefer Avisynth-native source plugins as long as they work correctly (which is much easier than ever before now with a wide selection: DGDec{NV|IM}, FFMS2, L-SMASH Works, DSS2Mod in native LAV Filters mode).

Remember, DirectShow is "like a box of chocolates": Always a surprise if you didn't select its content carefully.

[turbojet]

DSS2Mod is what I normally use with preroll otherwise there's video hiccups with interlaced ts files, occasionally lsmash when I need frame accurate, ffms2 detects weird fps with interlaced ts files, I forgot it can interact with lavfilters and just recently tried to figure out how to use cuda when encoding but not playback. Thanks for reminder!

[schweinsz]

I build the x264 using the following configuration using the MinGW:
./configure --enable-static --disable-opencl --enable-win32thread --disable-interlaced

I get the libx264.a and x264.exe, then I link the libx264.a into a vc project and get the following errors:
1>x264-test.c(31): warning C4244: 'initializing' : conversion from 'int' to 'float', possible loss of data
1>libx264.a(analyse.o) : warning LNK4229: invalid directive '/aligncomm:_x264_cabac_size_unary,5' encountered; ignored
1>libx264.a(analyse.o) : warning LNK4229: invalid directive '/aligncomm:_x264_cabac_transition_unary,5' encountered; ignored
1>libx264.a(cabac.o) : warning LNK4229: invalid directive '/aligncomm:_x264_cabac_contexts,5' encountered; ignored
1>libx264.a(vlc.o) : warning LNK4229: invalid directive '/aligncomm:_x264_run_before,5' encountered; ignored
1>libx264.a(vlc.o) : warning LNK4229: invalid directive '/aligncomm:_x264_level_token,5' encountered; ignored
1>libx264.a(encoder.o) : error LNK2001: unresolved external symbol _snprintf
1>libx264.a(ratecontrol.o) : error LNK2001: unresolved external symbol ___mingw_vfprintf
1>libx264.a(ratecontrol.o) : error LNK2001: unresolved external symbol ___mingw_vsprintf
1>libx264.a(ratecontrol.o) : error LNK2001: unresolved external symbol ___fpclassify
1>E:\prog\x264test\Release\x264test.exe : fatal error LNK1120: 4 unresolved externals

How can I link all the dependent functions into the libx264.a static just the vc does by enabling the option /MT?

[MasterNobody]

It is very bad idea to mix MinGW libraries (.a) and MSVS projects.
It would be easier and less hacky to use shared library instead of static library if you want to use mingw compiled libx264 in MSVS:
How to build x264 as DLL (http://doom10.org/index.php?topic=1876.0)
or
How to build x264/libx264.dll in Windows (http://www.ayobamiadewole.com/Blog/Others/x264compilation.aspx)
Other way is to compile static library by ICL but this is require you to have ICL.

[schweinsz]

It is very bad idea to mix MinGW libraries (.a) and MSVS projects.
It would be easier and less hacky to use shared library instead of static library if you want to use mingw compiled libx264 in MSVS:
How to build x264 as DLL (http://doom10.org/index.php?topic=1876.0)
or
How to build x264/libx264.dll in Windows (http://www.ayobamiadewole.com/Blog/Others/x264compilation.aspx)
Other way is to compile static library by ICL but this is require you to have ICL.
When I run the follows:
./configure --disable-cli --enable-shared --enable-win32thread --extra-ldflags=-Wl,--output-def=libx264.def
I get:
C99 compiler is needed for compilation.

So I delete the --disable-cli and it succeeds.
What does the --disable-cli mean?
And what does the --extra-ldflags=-Wl mean?

[microchip8]

When I run the follows:
./configure --disable-cli --enable-shared --enable-win32thread --extra-ldflags=-Wl,--output-def=libx264.def
I get:
C99 compiler is needed for compilation.

So I delete the --disable-cli and it succeeds.
What does the --disable-cli mean?
And what does the --extra-ldflags=-Wl mean?

afaik, --disable-cli disables the building of the x264 executable. The other option seems to imply that you pass it additional option for when linking

[LoRd_MuldeR]

And what does the --extra-ldflags=-Wl mean?

The complete option is:
--extra-ldflags=-Wl,--output-def=libx264.def

Which means that the extra flag "-Wl,--output-def=libx264.def" will be passed to GCC in the linking phase. Here "-Wl" is used to pass commands directly to the linker (see here (http://gcc.gnu.org/onlinedocs/gcc/Link-Options.html#Link-Options) for details) and "--output-def=libx264.def" is the command being passed to the linker. It tells the linker to output a DEF for for the x264 library (DLL on Windows). The DEF file can be used to generate an import library (.lib) for the DLL using the "lib" tool of Visual Studio, as described here (http://www.fftw.org/install/windows.html) for FFTW.

I get:
C99 compiler is needed for compilation.

I would recommend you look at your "config.log" to see what exactly has failed and why.

[jpsdr]

Is there any dev who can answer this (http://forum.doom9.org/showthread.php?p=1691396#post1691396) ?

[dj_alek]

Hi!
Does anyone know the reason of hardcoded limit of RefPicList1 size?:

b_ref_reorder[2];
sps->vui.i_num_reorder_frames = param->i_bframe_pyramid ? 2 : param->i_bframe ? 1 : 0;
h->frames.i_max_ref1 = X264_MIN( h->sps->vui.i_num_reorder_frames, h->param.i_frame_reference );
etc

Is the increase in size will not improve the encoding quality? I mean, eg, redistribution of parameter i_frame_reference = 10 to num_ref_l0 = 5 & num_ref_l1 = 5 instead of the current num_ref_l0 = 8 & num_ref_l1 = 2 (to use 5 forward + 5 backward frames for reference rather than 2 forward + 8 backward)?

[kotuwa]

x264 supports raw-input of MKV files with x264/AVC video.
what is the difference using raw input vs. input via .AVS script that uses FFMS2 etc?
(Nothing else like resizing is used... Just input...)

[huhn]

nothing.
as far as i know x264 uses ffmpegsource2 to decode too or at least something based on the same decoder.

[LigH]

Rather the kernel of it: libav libraries. But only if the person who built a specific executable set it up to use it; it is not mandatory, some other builds may omit it.

By the way: "raw input" and "MKV input" are completely different things. MKV is a container, usually with compressed content. The term "raw" usually means uncompressed video without any container (means: pure video pixels or samples according to chroma subsampling, e.g. YUV 4:2:0 known by the VfW FourCCs I420 or YV12, depending on the order of chroma planes).

Files with raw YUV video data usually have the extension "*.yuv". And they have no information at all what kind of video format they contain, you have to know that, and you have to tell the encoder via parameters about "pixel format", sample bitdepth, width, height, and framerate. A compromise is the YUV4MPEG format (*.y4m) which has at least a header with the most important attributes, but the rest of this file is exactly the same as raw YUV without any header.

[kotuwa]

nothing.
as far as i know x264 uses ffmpegsource2 to decode too or at least something based on the same decoder.

Rather the kernel of it: libav libraries. But only if the person who built a specific executable set it up to use it; it is not mandatory, some other builds may omit it.

By the way: "raw input" and "MKV input" are completely different things. MKV is a container, usually with compressed content. The term "raw" usually means uncompressed video without any container (means: pure video pixels or samples according to chroma subsampling, e.g. YUV 4:2:0 known by the VfW FourCCs I420 or YV12, depending on the order of chroma planes).

Files with raw YUV video data usually have the extension "*.yuv". And they have no information at all what kind of video format they contain, you have to know that, and you have to tell the encoder via parameters about "pixel format", sample bitdepth, width, height, and framerate. A compromise is the YUV4MPEG format (*.y4m) which has at least a header with the most important attributes, but the rest of this file is exactly the same as raw YUV without any header.

My bad...
I used the term RAW, as x264 programs says MKV inputs as raw...
I mean when I tried without giving input resolution, it said raw input needs a resolution.. :)

And I am taking about encoders x264.nl gives in main page..
precisely http://mirror01.x264.nl/x264/32bit/8bit_depth/revision2334/x264.exe

How can we get the ones that uses lavf?

Thnak You Very Much All!...

[Selur]

How can we get the ones that uses lavf?
iirc the offical binaries from http://download.videolan.org/x264/binaries/ should be fine,..

[foxyshadis]

No newsletter yet, but a big series of updates committed today. Highlights include: Experimental AQ mode 3 now added to stable (affects dark areas more strongly); LOTS of ARM64 performance improvements; tiny x86 performance improvements; small bugfixes, included to HRD; some updates to AvxSynth wrapper and the included libx264 example.

[LoRd_MuldeR]

Does anybody know what happened to x264 development branch at:
https://github.com/DarkShikari/x264-devel

:confused:

[cyberbeing]

Does anybody know what happened to x264 development branch at:
https://github.com/DarkShikari/x264-devel

:confused:
It's been moved to:
http://git.videolan.org/?p=x264/x264-sandbox.git

[Selur]

Any new link for http://download.videolan.org/pub/videolan/x264/binaries/ (linked on http://www.videolan.org/developers/x264.html) in regard to Mac OS X binaries?
The old http://download.videolan.org/x264/binaries/ is still there, but doesn't have the new binaries (for any OS).

[LoRd_MuldeR]

It's been moved to:
http://git.videolan.org/?p=x264/x264-sandbox.git

:thanks:

[jpsdr]

Does anyone know with what option i have to build ffmpeg for generating libraries i can use to compile x264, and also avoiding message like "Programm cannot start becase avcodec-56.dll is not found" :confused:

[MasterNobody]

Any new link for http://download.videolan.org/pub/videolan/x264/binaries/ (linked on http://www.videolan.org/developers/x264.html) in regard to Mac OS X binaries?
The old http://download.videolan.org/x264/binaries/ is still there, but doesn't have the new binaries (for any OS).
I get info at IRC that server's hard disk died and all the URLs used by buildbot went 404 (more info (https://twitter.com/videolan/status/545620746505322496)). So no new builds there before it would be fixed (ETA for fix is 2 weeks or more). So for now you can use builds made by komisar (http://komisar.gin.by/) or others.

[Selur]

So for now you can use builds made by komisar or others.
I wrote:
in regard to Mac OS X binaries?
didn't know that you and/or komisar provide Mac OS X builds (+ can't find any )

[MasterNobody]

Sorry, I don't know anyone building Mac OS X builds. This info was for all people who can come to this topic with question where to get build (and most of those who have such questions are Windows users because Linux users usually don't have problem to build it themself) and why there none at videolan site.

[the_weirdo]

and also avoiding message like "Programm cannot start becase avcodec-56.dll is not found" :confused:

"--enable-static --disable-shared". However, I think they should be on by default. If your build configure has "--enable-shared" and/or "--disable-static", remove them.

[Selur]

... because Linux users usually don't have problem to build it themself) and why there none at videolan site.
Ehmm,.. there were static up-to-date builds for: freebsd10-x86_64, freebsd9-x86_64, linux-i386, linux-x86_64, macosx-x86-64, win32, win64
over at videolan, not just Windows builds.

[the_weirdo]

Ehmm,.. there were static up-to-date builds for: freebsd10-x86_64, freebsd9-x86_64, linux-i386, linux-x86_64, macosx-x86-64, win32, win64
over at videolan, not just Windows builds.

I think you've misunderstood what he meant. He meant if there's someone who comes to ask where to get the builds and why there's none at videolan site then his previous post has the answer for them. Those people are likely Windows users, because "Linux users usually don't have problem to build it themself" if they couldn't find the builds. He didn't mean there're only Windows builds over at videolan.

[Selur]

You are right, seems like I misunderstood that. :)

[Selur]

btw. win32 got updated over at http://download.videolan.org/x264/binaries/ (hoping win64 and mac will follow soon :))
-> they now went back to an older version of the folder :/

[MasterNobody]

A little bit late (sorry) but official buildbot again working and the latest version (r2538-121396c) is already uploaded to http://download.videolan.org/x264/binaries/

[Selur]

Thanks for the info!

[tfouto]

The anime preset for encoding h264, should be used for all CGI movies like pixar? Or it's more of flat 2d anime?

For movies of pixar, should i use anime or film preset?

Many thanks

Meanwhile i found the answer.

http://forum.doom9.org/showthread.php?p=1682913#post1682913

[jpsdr]

Here part of OreAQ code :


get_image_mb( h, mb_x, mb_y, frame, PARAM_INTERLACED, &luma, &bluePoint );
x264_emms();

_energy_y = X264_MAX( energy[0], 1 );
_energy_uv = X264_MAX( energy[1], 1 );

h->rc->aq3_threshold = logf( powf( h->param.rc.f_aq3_sensitivity, 4 ) / 2.0 );
// logf(energy) = 1.0397 * x264_log2( energy ) / 1.5
// energy_y = 1.2 * (logf(_energy_y ) - ((h->rc->aq3_threshold + 2*(BIT_DEPTH-8)) * .91) + 0.5);
// energy_uv = 0.8 * (logf(_energy_uv) - ((h->rc->aq3_threshold + 2*(BIT_DEPTH-8)) * .91) + 0.5);
energy_y = 0.83176f * x264_log2( _energy_y ) - ((h->rc->aq3_threshold + 2*(BIT_DEPTH-8)) * 1.092f) + 0.5f;
energy_uv = 0.55451f * x264_log2( _energy_uv ) - ((h->rc->aq3_threshold + 2*(BIT_DEPTH-8)) * 0.728f) + 0.5f;
f_qp_adj = 0.f;

if( luma > h->param.rc.i_aq3_boundary[0] )
{
// *** Bright ***
// Y & UV Flat -> qp up
// Y Flat / UV Bump -> qp y up
// Y Bump / UV Flat -> even
// Y & UV Bump -> qp down
mode = 0x00;

// qp up
if( !bluePoint && energy_y < 0 && energy_uv < 0 )
f_qp_adj = X264_MIN( energy_y, energy_uv );
// qp y up
else if( !bluePoint && energy_y < 0 && energy_uv >= 0 )
f_qp_adj = energy_y;
// qp down
else if( energy_y >= 0 && energy_uv >= 0 )
f_qp_adj = X264_MAX( energy_y, energy_uv ) * 0.5f;
}

energy is result of x264_ac_energy_mb.

I found the idea interesting, but i think it's missing a "middle" case, where nothing is done (not flat, but not bumpy enough).
Here it's using some formula with 0 threshold, i want to change it to 2 threshold.

So, my question for experienced dev. : What are standard/classical/average values limit for energy[0] and energy[1] can be used to limit "flat" and "bumpy" ?
Here they are limited, but, what are the value without limits ?

:thanks:

Edit :
Maybe i can, from "threshold" parameter, create 2 threshold values (high and low), for exemple +/-25%, and with these, calul and "high" and "low" value of energy_y and energy_uv.

[Dan203]

Does anyone here know if you can build x264 in Visual Studio 2015 without the Intel compiler? Apparently VS2015 adds C90 support and a little C99, and it's now possible to build ffmpeg with VS2015 so I was wondering if anyone had tried building x264 standalone using VS2015 yet? We have a commercial license and are currently build using VS2010 and the Intel compiler. But we're looking at upgrading to VS2015 and we're wondering if we need the Intel compiler any more?

[MasterNobody]

It should be possible to build x264 with MSVS 2013 Update 2 (http://git.videolan.org/gitweb.cgi?p=x264.git;a=commit;h=6fbbb5b0c05a1d95cbd6efa7f01808ea87a39dc9) or newer (but I don't tested Visual Studio 2015 yet if they broke something or not).

[jpsdr]

Thanks a lot! I think it's a very useful tutorial.
Here is a 7mod x264 version which is very similar to tmod and still update with offical git.
https://github.com/Freecom/x264
Thanks. The potentiel interesting things i've noticed and are not included in the t_mod are the k-means and the VBV patch of MasterNobody. This last one is from 2014, so i'm wondering why it's still not put in standard release.
I'll work on this to include them on my t_mod branch on github. Could be interesting, but i'm wondering if k-means is Blu-Ray compliant, not caring of buggy chipset...

[jpsdr]

Is there somewhere informations about the k-means weightp mode ? What's it supposed/intend to do ? Is it an improvement of weightp mode 2 ? Is it made with a specific purpose in mind ?
If anyone has any information, i'm interested in.

[kotuwa]

FULLRANGE
In 2597 build, when --fullrange used, it says unknown parameter fullrange.
Why is that? Is it removed?
!

[sneaker_ger]

It was replaced years ago:
--range <string> Specify color range ["auto"]
- auto, tv, pc
--input-range <string> Specify input color range ["auto"]
- auto, tv, pc
Use --fullhelp to see all options.

[Romario]

Halo guys, one question for x264 devs. Is is possible to optimize x264 specificcly for AMD Ryzen and to add optimisation for upcoming AVX-512 instruction set, which come now in new XEON processors ?

[sneaker_ger]

The first AVX-512 optimizations are already in the development git. But this is not something that's implemented once-and-for-all. It's being added in small increments, function for function. I don't know how much faster these optimizations are at this point.

[jpsdr]

With for now a little trouble for me to build new version (when commited in official release), the nasm required is 2.13rc2, and the nasm provided with msys2 is 2.12... :(
But maybe this will be solved during the time the new release arrives...:D

[Midzuki]

.....the nasm required is 2.13rc2, and the nasm provided with msys2 is 2.12... :(

Don't you know that you can update nasm.exe manually? :confused:

http://www.nasm.us/pub/nasm/releasebuilds/2.13rc22/

[Romario]

The first AVX-512 optimizations are already in the development git. But this is not something that's implemented once-and-for-all. It's being added in small increments, function for function. I don't know how much faster these optimizations are at this point.

From what I know, is that AVX-512 is speccificaly wroted for Video encoding, 3D/CAD/CAM. So, I deeply believe that x264 (and x265, of course), will have visible improvments, when AVX-512 correct applied.

[jpsdr]

I'm only using "pacman -needed" and "pacman -Syuu". What happens if i change the nasm version manualy and do a "pacman -Syuu" ? Won't the version check be screwed up ? (Or anything similar)

[benwaggoner]

From what I know, is that AVX-512 is speccificaly wroted for Video encoding, 3D/CAD/CAM. So, I deeply believe that x264 (and x265, of course), will have visible improvments, when AVX-512 correct applied.
After all, an 8x8 block with 8-bit samples is exactly 512 bits!

Although from talking to codec developers, they seem to believe that HEVC will get a bigger perf improvement than H.264 from AVX-512, since there are lots of bigger block sizes. One row of a 32x32 TU in >8-bit is also 512 bits. And you can get a 8x8 chroma plane in a 16x16 TU.

[NikosD]

Latest heavily optimized H.264 binaries since 2013 (first year of AVX2), have a speed-up of around 3%-5% using AVX2.

Expect AVX512 to add nothing to H.264 performance.

[dbart]

Latest heavily optimized H.264 binaries since 2013 (first year of AVX2), have a speed-up of around 3%-5% using AVX2.

Expect AVX512 to add nothing to H.264 performance.

which version ?

[LoRd_MuldeR]

which version ?

Initial support for AVX2 assembler code has been added to x264 in late 2012, around revision 2243/2242. But work to implement more AVX2-optimized functions was going on until mid 2013 (revision 2352).

commit ccda1ba4d8d902945c68aa25ec20867055d1b079 [revision 2243]
Author: Fiona Glaser <fiona@x264.com>
Date: Mon Nov 12 10:28:53 2012 -0800

AVX2/FMA3 version of mbtree_propagate
First AVX2 function for testing.
Bump yasm version to 1.2.0 for AVX2 support.

commit 8a9608bbbdf77ceb3ee537271549111468175a2b [revision 2242]
Author: Henrik Gramner <hengar-6@student.ltu.se>
Date: Tue Dec 11 16:05:34 2012 +0100

x86inc: Use VEX-encoded instructions in AVX functions
Automatically use VEX-encoding in AVX/AVX2/XOP/FMA3/FMA4 functions for all instructions that exists in a VEX-encoded version.
This change makes it easier to extend existing code to use AVX2.
Also add support for AVX emulation of a few instructions that were missing before.

[nevcairiel]

Initial support for AVX2 assembler code has been added to x264 in late 2012, around revision 2243/2242. But work to implement more AVX2-optimized functions was going on until mid 2013 (revision 2352).

There have been new AVX2 optimizations as recently as January 2017, fwiw.

[sneaker_ger]

Looks like lossless is gonna break again...

http://git.videolan.org/?p=x264/x264-sandbox.git;a=commit;h=9b6adb376b32ab68fa9bada32225652b0fce93a7
http://git.videolan.org/?p=x264/x264-sandbox.git;a=commit;h=45ee092bfa5e52391740a0cdcc3c2891bf02611d

[nevcairiel]

avcodec just needs a check to switch to spec-compliant mode with newer x264 versions, it already checks for x264 as the encoder to enable the x264-compatibility decoding mode.

[sneaker_ger]

Hmm, yes. So "only" "old" decoders gonna break. Personally, I don't think the compression gain is worth it (and it also comes at a speed cost).

Since the changelog mentions some mixed lossy/lossless mode: is that something new/yet to come?

[MasterNobody]

First of all it is not decided yet when "Remove compatibility workarounds" will be pushed (but it probably would be after avcodec will be able to decode them). And yes avcodec will need check for old x264 version to decode old streams (there could be problems if someone removed x264 SEI). Same for the 4:4:4 decoding.

P.S. Imho default (i.e. without SEI) decoding in avcodec should be according to specs. But that is debatable.

[sneaker_ger]

For default settings, what is the difference between --partitions p8x8,b8x8,i4x4 and --no-8x8dct? What should we enable to match current lossless (and 4:4:4 lossy?) behavior?

[MasterNobody]

For default settings, what is the difference between --partitions p8x8,b8x8,i4x4 and --no-8x8dct? What should we enable to match current lossless (and 4:4:4 lossy?) behavior?
1) --patritioins only influence inter-frames analyse
2) inter-macroblocks also can use 8x8dct transform.
To be compatible with current decoders you will need --no-8x8dct (but it wouldn't exactly match current behavior).

[sneaker_ger]

but it wouldn't exactly match current behavior
Why not?

[MasterNobody]

Why not?
1) Because currently 8x8dct is allowed in inter-macroblocks of lossless encoding and only disabled for intra-blocks (disabled i8x8 in intra/inter frames).
2) 4:4:4 encoding currently is out of specs with cabac+8x8dct and you wouldn't be able to return to out of specs behavior.

[sneaker_ger]

I see. Thx.

And yes avcodec will need check for old x264 version to decode old streams (there could be problems if someone removed x264 SEI).
On the other hand, 2014 ~ 2017 ffmpeg will break if you don't remove the SEI. Fun times. :D

[LoRd_MuldeR]

IMO it is much better to break old libavcodec/ffmpeg once (and disable the compatibility workarounds in new libavcodec/ffmpeg), instead of continuing to produce out-of-spec streams for ever and ever.

What x264 currently produces in "lossless" mode probably has never been working with any H.264 decoders, except for libavcodec/ffmpeg...

[MasterNobody]

What x264 currently produces in "lossless" mode probably has never been working with any H.264 decoders, except for libavcodec/ffmpeg...
Wrong. Currently lossless produce correct streams with out of spec feature disabled 3 years ago (commit (http://git.videolan.org/?p=x264.git;a=commitdiff;h=af8e768e2bd3b4398bca033998f83b0eb8874914))
4:4:4+cabac+8x8dct is out of spec now so I would recommend anyone encoding 4:4:4 with cabac to disable 8x8dct.

[LoRd_MuldeR]

Wrong. Currently lossless produce correct streams with out of spec feature disabled 3 years ago (commit (http://git.videolan.org/?p=x264.git;a=commitdiff;h=af8e768e2bd3b4398bca033998f83b0eb8874914))
4:4:4+cabac+8x8dct is out of spec now so I would recommend anyone encoding 4:4:4 with cabac to disable 8x8dct.

Thanks for clarification.

Anyway, I think it's safe to assume that disabling the "out of spec" features in lossless mode costs some compression efficiency. So it's preferable to finally have it fixed and re-enabled.

[sneaker_ger]

Efficiency loss in lossless mode is 1%, if that.

[asarian]

(Maybe I should post this here instead)

Hmm, just tried the latest x264, with 10-bit encoding:

x264 [warning]: OpenCL: not compiled with OpenCL support, disabling

That's disappointing. OpenCL works just fine for 8-bit encodings. Is there a particular reason OpenCL can't/doesn't work using 10bit encodings?

[LoRd_MuldeR]

That's disappointing. OpenCL works just fine for 8-bit encodings. Is there a particular reason OpenCL can't/doesn't work using 10bit encodings?

Let's have a look at x264 code (https://git.videolan.org/?p=x264.git;a=blob;f=encoder/encoder.c;h=074c4a5c2ba8f0a9df3778d2f1d593b22e09fa20#l593) (current master):
int validate_parameters( x264_t *h, int b_open )
{
[...]

#if !HAVE_OPENCL
x264_log( h, X264_LOG_WARNING, "OpenCL: not compiled with OpenCL support, disabling\n" );
h->param.b_opencl = 0;
#elif BIT_DEPTH > 8
x264_log( h, X264_LOG_WARNING, "OpenCL lookahead does not support high bit depth, disabling opencl\n" );
h->param.b_opencl = 0;
#else
if( h->param.i_width < 32 || h->param.i_height < 32 )
{
x264_log( h, X264_LOG_WARNING, "OpenCL: frame size is too small, disabling opencl\n" );
h->param.b_opencl = 0;
}
#endif

[...]
}

So, no OpenCL support for bit-depths greater than 8-Bit, or for very small frames.

I'd assume that's either because nobody bothered porting the OpenCL code to "high bit-depth". Or it's because GPUs tend to be orders of magnitude slower when doing calculations on data-types that the haven't been optimized for, and therefore OpenCL may not actually be worth it at "high bit-depth" (on most GPUs). For example, FP64 (double precision) math is 24 times to 32 times slower than FP32 (single precision) math on Kepler/Maxwell GPUs (details (https://arrayfire.com/explaining-fp64-performance-on-gpus/)).

[asarian]

^^ That code is pretty self-explanatory, I guess. Thx. Except I would then expect to get the error msg for '#elif BIT_DEPTH > 8', and not the one for not having OpenCL ('#if !HAVE_OPENCL'), which is the one I got, right?

[asarian]

So, no OpenCL support for bit-depths greater than 8-Bit, or for very small frames.

I'd assume that's either because nobody ever bothered porting the OpenCL code to "high bit-depth". Or it's because of the fact that GPUs tend to be orders of magnitude slower when doing calculations on data-types that the haven't been optimized for, and therefore OpenCL may not actually be worth it at "high bit-depth" (on most GPUs). Or a combination of both reasons.

Sorry, I had missed that part of your post. Good explanation. Makes sense. Thanks. :goodpost:

[LoRd_MuldeR]

^^ That code is pretty self-explanatory, I guess. Thx. Except I would then expect to get the error msg for '#elif BIT_DEPTH > 8', and not the one for not having OpenCL ('#if !HAVE_OPENCL'), which is the one I got, right?

Pre-processor macros like BIT_DEPTH or HAVE_OPENCL are set at compile-time, not run-time.

Also, since the "8/10 bits unification", the exactly same source code files will be compiled twice, once to generate the machine code for "8-Bit" encoding, and once to generate the machine code for "10-Bit" encoding.

Of course, pre-processor macros will be set differently for "8-Bit" and "10-Bit" compilation, so the generated machine code will actually be different for the "8-Bit" and "10-Bit" paths.

Now, it would seem that HAVE_OPENCL simply was not defined at the time when the "10-Bit" version has been compiled – which makes some sense considering that we know beforehand that OpenCL is for 8-Bit only.

(The "BIT_DEPTH > 8" check may seem a bit redundant then. But maybe it's not guaranteed that HAVE_OPENCL will always be unset for "BIT_DEPTH > 8" in every possible situation)

[UPDATE]

Indeed, HAVE_OPENCL is not simply defined as "0" or "1". It is actually defined as "(BIT_DEPTH == 8)", when building x264 with OpenCL support enabled; would probably be defined to "0" otherwise.

So, it may actually be preferable to change the code to:
#if !HAVE_OPENCL
#if BIT_DEPTH > 8
x264_log( h, X264_LOG_WARNING, "OpenCL lookahead does not support high bit depth, disabling opencl\n" );
#else
x264_log( h, X264_LOG_WARNING, "OpenCL: not compiled with OpenCL support, disabling\n" );
#endif
h->param.b_opencl = 0;
#else
[...]

(But that's nitpicking, I suppose)

[asarian]

Pre-processor macros like BIT_DEPTH or HAVE_OPENCL are set at compile-time, not run-time.

Doh on me! :o

Also, since the "8/10 bits unification", the exactly same source code files will be compiled twice, once to generate the machine code for "8-Bit" encoding, and once to generate the machine code for "10-Bit" encoding.

Of course, pre-processor macros will be set differently for "8-Bit" and "10-Bit" compilation, so the generated machine code will actually be different for "8-Bit" and "10-Bit" .

Now, it would seem that HAVE_OPENCL simply was not defined at the time when the "10-Bit" version has been compiled – which makes some sense, because we know beforehand that OpenCL is for 8-Bit only.

(The "BIT_DEPTH > 8" check may seem a bit redundant then. But maybe it's not guaranteed that HAVE_OPENCL will always be unset for "BIT_DEPTH > 8" in every possible situation)

As usual, thanks for the deep insight. :)

[hydra3333]

:goodpost: :thanks:

[FranceBB]

I'd assume that's either because nobody bothered porting the OpenCL code to "high bit-depth". Or it's because GPUs tend to be orders of magnitude slower when doing calculations on data-types that the haven't been optimized for, and therefore OpenCL may not actually be worth it at "high bit-depth" (on most GPUs). For example, FP64 (double precision) math is 24 times to 32 times slower than FP32 (single precision) math on Kepler/Maxwell GPUs (details (https://arrayfire.com/explaining-fp64-performance-on-gpus/)).

Very interesting article indeed.
NVIDIA performs better in Single-Precision Floating Point 16 and 32, but less for 64 on consumer-grade GPUs 'cause there aren't as many 64-capable units as the 32 ones, while AMD consumer-grade GPUs have better 64 performance due to more 64 capable units at the expense of the 32 and 16 ones.
However, on an enterprise level, NVIDIA has better performance on both Single-Precision Floating Point 32 and 64 then AMD has.
An interesting thing is that NVIDIA GPUs have 32-capable units that are also 16-bit capable, therefore not wasting space on 16-bit capable units.
The White Paper (https://images.nvidia.com/content/pdf/tesla/whitepaper/pascal-architecture-whitepaper.pdf) at page 12 says "One new capability that has been added [...] is the ability to process both 16-bit and 32-bit precision instructions and data, as described later in this paper. FP16 operation throughput is up to twice FP32 operation throughput". Page 14: " Using FP16 computation improves
performance up to 2x compared to FP32 arithmetic, and similarly FP16 data transfers take less time than FP32 or FP64 transfers."

[LigH]

Just a side note ... the mentioned precision may be convenient for video processing; but there are applications which would gain severe speed boost from GPGPU parallelization if it just had the required precision for their demands (like astronomical multi body simulations, see Universe Sandbox forums: PhysX had to be rejected, OpenCL is only partially used).

[FLX90]

I‘m currently 2-pass transcoding a BluRay and it takes about 26 hours on my 5th gen. i7.
CPU load: 100 %

Is it bad for the transcode if I do some web surfing during the process.
Sometimes my browser freezes for less than a second ...
Would that harm my transcodes?

[Groucho2004]

Is it bad for the transcode if I do some web surfing during the process.
Sometimes my browser freezes for less than a second ...
Would that harm my transcodes?
No.
If you are planning to use your computer while encoding you should even lower the priority of the process to low/idle. This way your browsing (or whatever else you're doing) will not 'freeze' and the encoder only gets the CPU cycles that are available. Of course that only works when you have sufficient memory installed.

[FLX90]

Sounds great.
Thank you for your answer.

I asked because more than 10 years ago, I transcoded my then library with iTunes to mp3. I was gaming and doing other CPU intense things during the transcoding process.

After that, I noticed for 10 % of my 10,000 mp3s an audible click noise for less than a millisecond.
Thought that was because of the high CPU load.

[Groucho2004]

I asked because more than 10 years ago, I transcoded my then library with iTunes to mp3. I was gaming and doing other CPU intense things during the transcoding process.

After that, I noticed for 10 % of my 10,000 mp3s an audible click noise for less than a millisecond.
Thought that was because of the high CPU load.
No idea, should not happen. Does itunes by any chance use the GPU for transcoding?

[FLX90]

I really don’t know ...

[LigH]

I would rather suspect misinterpreted ID3 tags as source of distortions.

Back to x264?

[FLX90]

Yes, that could possibly be.
Thanks for clarification.

[sneaker_ger]

qcomp 0.8 is part of --tune grain so it's not anything insane. We usually don't recommend users to fiddle with those parameters because they don't really understand them and are better off simply using preset+tune than choosing "wrong" values.

From what i understand of the lower values it distributes more bitrate to more visible areas and uses less bitrate in lower visible areas but when i did some video comparison the lower qcomp appears to take freely from a bunch of areas like parts of the fingers or hair or background were missing, i guess the bitrate for those areas are put elsewhere? I could see the benefit of this in constant rate factor maybe but it doesn't seem to be as needed in two-pass.
Yes, it's about redistributing bits. With higher qcomp the redistribution is less aggressive. This redistribution affects 2pass and crf in the same way.

[benwaggoner]

No idea, should not happen. Does itunes by any chance use the GPU for transcoding?
There definitely were not GPU accelerated MP3 encoders in iTunes >10 years ago!

[benwaggoner]

qcomp 0.8 is part of --tune grain so it's not anything insane. We usually don't recommend users to fiddle with those parameters because they don't really understand them and are better off simply using preset+tune than choosing "wrong" values.
Yeah, finding a better value for a particular type or piece of content is possible. But finding a significantly better generic default value is highly unlikely.

[Emulgator]

Maybe interesting:
https://forum.doom9.org/showthread.php?t=175723
Never mind, I typed into the wrong thread here, too many windows open...

[dev84]

Why there is no official binary r2935 10bits ?

[sneaker_ger]

8 bit and 10 bit are now combined into a single binary. Use --output-depth 10

[dev84]

:( i'm using StaxRIP 1.7 but it doesn't work with "--output-bitdepth 10" i get "failed with exit code: -1 (0xFFFFFFFF)"
:(

[sneaker_ger]

Sorry, --output-depth, not --output-bitdepth

[dev84]

Thanks

[FranceBB]

8 bit and 10 bit are now combined into a single binary.

Oh, just like x265. Nice.

[utack]

Does anyone have inisight on how x264 uses NEON instruction sets?
I just upgraded from a mobile device with 8x symmetric Cortex A53 to 4xA53 slower +4xA72 faster that has more or less the same multicore performance for a majority of tasks
However I noticed that x264 has had more than a 2x speedup on this device
Does Cortex A72 come with some special vector magic it uses?

[Blue_MiSfit]

IIRC there is a lot of hand tune NEON assembly in x264 :)

[Boulder]

I've been experimenting with going back to x264 for some of my BD re-encodes where the source is rather soft or noisy/grainy. I'm having a hard time figuring out when one would need to tweak psy-trellis. I understand that psy-rd is mainly responsible for keeping the grain, or at least fooling the viewing into believing that there is detail there, but I don't know what psy-trellis adjusts. Can anyone explain any use cases?

[jpsdr]

Unless i've missed it in the code, but it seems that there is no check value on --output-depth, you can put whatever you want ! Didn't see any check on i_bitdepth, and i also didn't see anything on validate_parameters in encoder.c file...
Don't you think you should add a check on allowed value ? And the help maybe more specific, instead of "just int value" :confused:

[MasterNobody]

Unless i've missed it in the code, but it seems that there is no check value on --output-depth, you can put whatever you want ! Didn't see any check on i_bitdepth, and i also didn't see anything on validate_parameters in encoder.c file...
Don't you think you should add a check on allowed value ? And the help maybe more specific, instead of "just int value" :confused:

check (https://code.videolan.org/videolan/x264/blob/master/encoder/api.c#L113)

[jpsdr]

Thanks, i had missed it, it was odd that i didn't find any check, it makes more sense indeed like this... :D

[LigH]

There is a (quite demanding) thread in the VideoHelp forum (https://forum.videohelp.com/threads/404454-Where-is-documentation-for-the-x264-encoder) asking for a comprehensive official documentation of x264, similar to ReadTheDocs for x265. Does any exist which I missed? The VideoLAN homepage of x264 seems not to mention any, and the docs directory in the repo is hardly worth mentioning. So I guess the best source of knowledge appears to be the fullhelp combined with searching back in the developer mailinglist for discussions about every parameter...

[benwaggoner]

There is a (quite demanding) thread in the VideoHelp forum (https://forum.videohelp.com/threads/404454-Where-is-documentation-for-the-x264-encoder) asking for a comprehensive official documentation of x264, similar to ReadTheDocs for x265. Does any exist which I missed? The VideoLAN homepage of x264 seems not to mention any, and the docs directory in the repo is hardly worth mentioning. So I guess the best source of knowledge appears to be the fullhelp combined with searching back in the developer mailinglist for discussions about every parameter...
I am pretty confident such a thing doesn't exist, although it should. x265.readthedocs.io is, by a huge margin, the best documentation for an encoder that's existed in the last 20 years. The old Terran Interactive manuals around 1995-2000 were the closest, although there were a lot fewer parameters to document.

[FranceBB]

Ok, so, I have a task for you, guys: optimizing the quality of a command line x264 encode, but this time is for professional use, therefore we have some constraints.

This is the current Command Line:

x264-10b.exe "\\mibcisilonsc\avisynth\Scambio\FILM\4CS00091.avs" --preset medium --profile high422 --level 5.2 --keyint 1 --no-cabac --slices 8 --bitrate 500000 --vbv-maxrate 500000 --vbv-bufsize 500000 --deblock -1:-1 --overscan show --colormatrix bt2020nc --range tv --log-level info --thread-input --transfer arib-std-b67 --colorprim bt2020 --videoformat component --nal-hrd cbr --aud --output-csp i422 --output-depth 10 --output "I:\Scambio\FILM\raw_video.h264"

what you see in the command line so far is mandatory, in fact the constraints are:

- Profile High 422 10bit
- Level 5.2
- keyint 1 (i.e All Intra)
- no-cabac
- slices 8
- 500 Mbit/s constant bitrate
- aud (access unit delimiter NAL at the start of each slice access unit)


Input files are UHD HDR PQ 12bit 4:4:4 files in Apple ProRes 23,976p at 1502 Mbit/s or 25p at 1600 Mbit/s or 29,970p at 1990 Mbit/s or 50p at 3318 Mbit/s or 60p at 3981 Mbit/s which are indexed, brought to 16bit, frame-rate converted to 50p in all cases, LUT converted to HLG and downscaled in chroma to 4:2:2 with Avisynth.
The resulting 16bit 4:2:2 UHD HDR HLG stream is then delivered to x264 and dithered down by x264 itself to 10bit planar with the Sierra-2-4A error diffusion and encoded as above.

A few questions for you:

1) Would it make sense to optimize x264 for quality and use like a slower preset like --preset veryslow at such an high bitrate?

2) Would it make sense to perform a two pass encode given that it's CBR and All Intra and at such an high bitrate?

3) Do you think it would make sense to mess up with --aq-mode and such at such an high bitrate?


Keep in mind that the end user will never ever see those files. Those files are sent to the playout in which an hardware playback port plays them, delivers the signal through an SDI cable and then such a signal is re-encoded live by an hardware H.265 encoder which resizes the chroma and encodes the final UHD 4:2:0 25Mbit/s 10bit satellite feed that the user receives live in .ts.

[rwill]

For the use case 500Mbit/s is not high, just average. If you care about the quality of your intermediate file use high quality settings.

The H.265 hardware encoder at the end of the chain will botch the quality anyway though.

I don't think the x264 RC uses multipass statistics effectively for CBR btw.

[Emulgator]

1. Although I am using it always, I guess veryslow is not needed here
2. No 2-pass, I don't see blu-ray restriction kind of distribution as the main problem here,
3. I guess no.
I guess you want to make sure that enough bits are spent in any case.
What I did in such cases was simply limiting qp decisions and test for that on the most complex scene.
Release any bitrate restrictions, decrease qpmax as much as you see it driving bitrate up.
Maybe you see going below --qpmax 30 will definitely force bitrate upwards.
Then step back, meaning increase qpmax again a few ticks --qpmax 36 ?
till you feel safe giving enough headroom.
Now you shouldn't get too low rate decisions anymore, so hopefully safe from harm.

[stax76]

There is this:

http://www.chaneru.com/Roku/HLS/X264_Settings.htm

Used in staxrip for the context (right-click) help.

[DTL]

Moved from 'getting latest' thread: About better performance of MPEG encoders (including all x26x projects) at 'big' and 'large' architectures like AVX2 and AVX512 using multi-blocks processing program redesign from single block processing.
About I-frames only example:
I got C-sources from https://github.com/ShiftMediaProject/x264/tree/master/encoder and it is built with VisualStudio 2017. Other versions (including jpsdr) looks like not compatible with MSVS.
After some profiling I see some significant time is in the intra_satd_x9_4x4() function and track its call stack to the all macroblocks walking through:
It is loop in the encoder.c file: https://github.com/ShiftMediaProject/x264/blob/be0cb5426d4b9fecaf2e4b058466322a43f17241/encoder/encoder.c#L2812 . It walks through all frame macroblocks one by one by rows and columns (using MB number advancing mb_xy = i_mb_x + i_mb_y * h->mb.i_mb_width; where i_mb_x is current x-pos in MBs array and i_mb_y is current y-pos).
So practical 'workunit' size for each 'macro loop' pass is 1 macroblock only. If macroblock is of 16x16 size it mean the total CPU core executing this thread have only workunit size of 16x16 (lets 10bit proc and 16bit values per sample) - 16x16x(2 bytes per sample) = 512 bytes. Too few for CPU capable of processing up to kilobytes workunits.

So to make this part of encoder faster we need to re-design this 'macro loop' and all downstream called functions to process several macroblocks in a single pass. But it not very easy task and also if not all macroblocks in a 'group pass' are processed equally it need some more branching (like fallback to single macroblock proc if its processing is not equal to others).

The final 'macroloop' advancing at https://github.com/ShiftMediaProject/x264/blob/be0cb5426d4b9fecaf2e4b058466322a43f17241/encoder/encoder.c#L3068 will be not simple
i_mb_x++; (for progressive encode mode)
+1 advancing but
i_mb_x+=num_macroblocks_per_pass;

But program re-design to this simple 'internal parallelling to use SIMD' may take lots of time.

More close to reality of fixing example: At the processing of 16x16 macroblock with partititions down to 4x4 it split macroblock to 4x4 blocks of 4x4 and check some predictors for each 4x4 block. So it is the much smaller loop of https://github.com/ShiftMediaProject/x264/blob/be0cb5426d4b9fecaf2e4b058466322a43f17241/encoder/analyse.c#L924

for( ; *predict_mode >= 0; predict_mode++ )
{
int i_satd;
int i_mode = *predict_mode;

if( h->mb.b_lossless )
x264_predict_lossless_4x4( h, p_dst_by, 0, idx, i_mode );
else
h->predict_4x4[i_mode]( p_dst_by );

i_satd = h->pixf.mbcmp[PIXEL_4x4]( p_src_by, FENC_STRIDE, p_dst_by, FDEC_STRIDE );
if( i_pred_mode == x264_mb_pred_mode4x4_fix(i_mode) )
{
i_satd -= lambda * 3;
if( i_satd <= 0 )
{
i_best = i_satd;
a->i_predict4x4[idx] = i_mode;
break;
}
}

COPY2_IF_LT( i_best, i_satd, a->i_predict4x4[idx], i_mode );
}


where h->pixf.mbcmp[PIXEL_4x4]( p_src_by, FENC_STRIDE, p_dst_by, FDEC_STRIDE ); is call to single-block of SATD(SAD depending on options ?) of 2 4x4 blocks (assembly function typically for each SIMD family). Count of loop spins is typically number of non-negative members in predict_mode pointed vector (about 3 or 4).

When running of the very old architectures like SSE(2) the 2 of 4x4 16bit blocks for SATD calculation takes 64 bytes to load and at x86 SSE2 with 8 only 128 bit (16 bytes) SIMD register file of 128 bytes total size it takes about half of register file and close to no space left for immediate values if try to load 2 pairs of blocks. So this implementation is 'internally limited' to SSE2 32bit build target architecture. It is optimal for speed at that architecture because at each iteration it can break by condition i_satd <= 0 and skip some predictors and save some time.

At the larger architectures it is possible to process more SATD computing of 4x4 16bit pairs blocks in single SIMD pass. So this program block may be rearranged to more SATD computing per single pass using new multi-block SATD computing SIMD function and the cycle may be changed to processing groups of predictors (typically to single pass when using up to 4 predictors) and after single SIMD function call analyse for minimal i_satd value from vector of SATD values and select minimal (also can be attempted to do with SIMD min member of vector instruction _mm_minpos_epu16() from SSE 4.1 set if SATD not great than 16bit unsigned value - unfortunately no 32bit copy of this nice instructon even at AVX512 set). But this new program block need to be guarded by 'architecture' if() block like only for AVX2 and x64 or larger and it make total program text bigger and harder to understand (and debug and support and so on).

[DTL]

Based on current state of development:
- The activity of MPEG coder and preprocessing temporal denoiser is enough collaborative in both motion vectors search and usage.
- Current mvtools can use motion vectors stream from both system hardware MPEG encoding accelerators via standard now DX12-ME API from some generation of Win10 or full onCPU search. The format of motion vectors stream of both hardware MPEG encoder and mvtools is about equal. Precision may be down to qpel (current the only supported by hardware API precision).
- The multu-generation motion search for natural nosied sources shows significant improuvement in quality of motion vectors already at second generation (examples of execution structure - https://forum.doom9.org/showthread.php?p=1984152#post1984152 )

It may be interesting to reuse refined motion vectors in pre-MPEG denoising in x264 encoder (also making some offloading of mvs search to system hardware accelerators if present). May be someone with good knowledge in x264 motion vectors usage can make some fork and quick tests for performance/quality ?

[Selur]

Why the resolution limitation to 16384 since https://code.videolan.org/videolan/x264/-/commit/7923c5818b50a3d8816eed222a7c43b418a73b36 ?
One can't encode something like, 24800x90 anymore, which should work fine with level 5.2 at 30fps and works fine with older versions (see: discussion over at videohelp (https://forum.videohelp.com/threads/409093-file-size-limit-MP4-or-MOV)).

Cu Selur

[nevcairiel]

One can't encode something like, 24800x90 anymore, which should work fine with level 5.2

This is not accurate. Such a dimension is not supported in any defined level.

From the specification:

f) PicWidthInMbs <= Sqrt( MaxFS * 8 )
g) FrameHeightInMbs <= Sqrt( MaxFS * 8 )


The highest MaxFS, on Level 6(.1/.2) is 139264 Macroblocks.
A macroblock is 16x16 pixel, just for reference.

This makes the maximum for any single dimension .. sqrt(139264 * 8) = 1055 Mbs, or 16880 pixel. Pretty close to the limit chosen, which is a neat power of 2 close to this (or 1024 Mbs)

[Selur]

Argh, totally forgot about PicWidthInMbs and PicWidthInMbs which makes my calculation unimportant since it only took MaxMBPS into account :(
Thanks for clearing that up.

[MasterNobody]

This change was made for security reasons, and because the limit has to be somewhere, it was made high enough to be reasonable and not overflow some intermediate SAD/SSD cost calculations for row of MBs in 32-bit integers (especially for 10-bit output). In special cases, you can always compile x264 without this limitation and use it at your own risk without any warranty.

[PoeBear]

Do any of the custom x264 builds have aq-bias-strength enabled? It's been useful for spreading crf bitrate out in x265, and I was curious if it existed in x264, and came across these patches when searching, but no binaries. Would it be as useful in x264?

https://gist.github.com/noizuy/83ba825d796e86cab1f67de333f90e0d
https://gist.github.com/noizuy/588446e288164fd4792c9406bc0b1416

[jpsdr]

Made a new build with the aq-bias-strength patch, check on my github.