Current Patches, Where to get them, How they affect speed/output [Archive] - Page 66

rack04

9th June 2010, 15:05

Toolchain:
GCC 4.4.4
GPAC 0.4.6
Pthreads 2.9.0.0
Yasm 1.0.1
LAVF/FFMS Builds:

ffmpeg SVN-r23548 and libswscale SVN-r31351 (http://www.multiupload.com/4TQNG2JECT)
ffms2 SVN-r312 (http://www.multiupload.com/VDKSOX6M2Y)
x264 Builds:

x264_x64_r1643M (http://www.multiupload.com/FYPUE1NSNC)
x264_x86_r1643M (http://www.multiupload.com/7XE5S8YUR3)
System: MINGW
asm: yes
avs input: yes
lavf input: yes
ffms input: yes
mp4 output: yes
pthread: yes
debug: no
gprof: no
PIC: no
shared: no
visualize: no
Patches:
x264_sws_typecast (http://pastebin.com/EVSqTqAi)
x264.open_gop.28 (http://pastebin.com/suAVft2F)
x264_fade_compensation (http://pastebin.com/WGEapT1e)
x264_thread_pool_v2.9 (http://pastebin.com/9MMaKDER)

mp3dom

9th June 2010, 15:06

Thanks rack04!

outlaw.78

9th June 2010, 23:19

x264 0.98.1643M x86 & x64 (http://www.mediafire.com/?mmufji2y2nj)

gcc 4.5.1 20100523 prerelease
ffmpeg svn 23555
swscale svn 31357
ffms2 svn 312
pthreads 2.9.0.0 static
x264 0.98.1643M x86,x64,amdfam10,fprofiled
patches used :
x264_thread_pool_v2.9 (http://pastebin.com/9MMaKDER)

RainyDog

10th June 2010, 08:06

tormento

11th June 2010, 06:46

komisar: waiting for your build... braaaaiiinnn...

outlaw.78

16th June 2010, 00:20

x264 0.98.1649M x86 & x64 (http://www.mediafire.com/?xkluikx3jiu)

gcc 4.5.1 20100614 prerelease
ffmpeg svn 23619
swscale svn 31428
ffms2 svn 312
pthreads 2.9.0.0 static
x264 0.98.1649M amdfam10,fprofiled,LTO
patches used :
x264_thread_pool_v2.9 (http://komisar.gin.by/old/1643/p/x264_thread_pool_v2.9.r1643.diff)

rack04

21st June 2010, 16:44

I'm trying to compile the latest x264 with x264_open_gop_40. Each time make fprofiled finishes with one of the encodes I get a crash with debug screen. If I click debug the next encode runs. I'm wondering if anyone else has experienced this.

BTW, I'm using cross-mingw.gcc450.generic.20100612 (http://komisar.gin.by/mingw/cross-mingw.gcc450.generic.20100612.7z) without LTO.

bnshrdr

21st June 2010, 17:06

I just used the same compiler as you posted and with x264 patched and it seems to be working ok (I'm only about 5 encodes through in fprofile). I'm targeting x86_64-mingw32.

Would it matter what clip you use to fprofile? If it means anything, I'm using 720p5994_stockholm_ter.yuv

komisar

21st June 2010, 17:07

rack04, gcc 450 and 451 broked... :( dont use it (i am remove link from my site)

EDIT: but wait... 450 is normal compile... (recheck again...)

rack04

21st June 2010, 17:10

rack04, gcc 450 and 451 broked... :( dont use it (i am remove link from my site)

EDIT: but wait... 450 is normal compile... (recheck again...)

make fprofile i686-pc-mingw32? cross-mingw.gcc444.generic.20100506 works fine.

bnshrdr

21st June 2010, 17:17

Ok, I can confirm your error when targeting i686-pc-mingw32, but the fprofile seems to be fine when targeting x86_64-pc-mingw32

komisar

21st June 2010, 17:19

rack04, no... for 32-bit gcc450 also broked...

As BugMaster investigate this problem: "found that gcc can produce code with missaligned SSE (MOVAPS) access when -fprofile-generate used"
Need rebuild toolchain with "-fno-tree-vectorize" (as DS say)

EDIT: or wait gcc451 release ;)

bnshrdr

25th June 2010, 02:03

DOWNLOAD: x264_r1649 (http://www.mediafire.com/download.php?finamqymnzw)

Builds Included:
x86 Generic
x86 Patched (patch listed below)
x86_64 Generic
x86_64 Patched (patch listed below)

Libraries:
gcc 4.4.4
zlib-1.2.3
bzip2-1.0.5
pthreads 2.9.0.0 GC-static
ffmpeg-svn: FFMPEG-r23763/SWSCALE-r31555
yasm-svn: YASM-r2334
ffmpegsource-svn: ffmpegsource-r312
gpac-cvs: gpac-0.4.6-DEV

Patches:
ffmpeg_patch (http://pastebin.com/WbVwYT0z)
applied with patch -p0 -i ffmpeg_patch.patch
x264_thread_pool_v2.9_updated (http://pastebin.com/gANSYv4i)
applied with patch -p0 -i x264_thread_pool_v2.9_updated.patch

lexor

25th June 2010, 14:13

@bnshrdr: is that x264_thread_pool patch something different from the thread pool incorporated into the official git repository? I know there was a thread pool patch before git had thread pool support, but now that git has it, why is there still a patch for it?

sneaker_ger

25th June 2010, 14:42

bnshrdr's build is revision 1649 which didn't have the commit. The newest version is 1659.

rack04

25th June 2010, 15:24

Toolchain:
GCC 4.4.4
GPAC 0.4.6
Pthreads 2.9.0.0
Yasm 1.0.1
x264 Builds:

x264_x64_r1659 (http://www.multiupload.com/H1IYWFR5WM)
x264_x86_r1659 (http://www.multiupload.com/4BYZMQV7UI)
System: MINGW
asm: yes
avs input: yes
lavf input: no
ffms input: no
mp4 output: yes
pthread: yes
debug: no
gprof: no
PIC: no
shared: no
visualize: no

lexor

25th June 2010, 16:41

bnshrdr's build is revision 1649 which didn't have the commit. The newest version is 1659.

Is there a way to get revision # from git's web ui? I usually check http://git.videolan.org/?p=x264.git;a=shortlog, but it's hard to tell who's builds have which submissions in them, atm.

Chikuzen

25th June 2010, 18:11

Is there a way to get revision # from git's web ui?

I always look x264.nl's changelog.
http://mirror01.x264.nl/x264/changelog.txt

LoRd_MuldeR

25th June 2010, 20:48

Is there a way to get revision # from git's web ui?

I wrote this simple tool for that purpose:
http://mulder.googlecode.com/svn/trunk/libx264/git2rev.exe

Usage:
git log > changelog.txt
git2rev changelog.txt

outlaw.78

25th June 2010, 22:38

x264 0.100.1659 x86 & x64 (http://www.mediafire.com/?yi3dmkmqniw)

gcc 4.5.1 20100623 prerelease
ffmpeg svn 23787
swscale svn 31559
ffms2 svn 312
pthreads 2.9.0.0 static
x264 0.100.1659 amdfam10,fprofiled,LTO

qyot27

25th June 2010, 22:39

I wrote this simple tool for that purpose:
http://mulder.googlecode.com/svn/trunk/libx264/git2rev.exe

Usage:
git log > changelog.txt
git2rev changelog.txt
Awesome, I've been needing a tool like this. How portable is it? It'd be nice to use it under Linux or OSX, too.

Testing on ffmpeg's changelog resulted in a ~200 revision disparity, though (SVN reports r23791, git2rev reports r23587). Is that on their end with the SVN/git syncing process, or do you think something else is going on?

JEEB

25th June 2010, 22:43

Awesome, I've been needing a tool like this. How portable is it? It'd be nice to use it under Linux or OSX, too.

git rev-list HEAD | wc -l | sed -e 's/^ *//' > revision.txt

Testing on ffmpeg's changelog resulted in a ~200 revision disparity, though (SVN reports r23791, git2rev reports r23587). Is that on their end with the SVN/git syncing process, or do you think something else is going on?

I would guess this is because of libswscale being in the same repo on the SVN side.

LoRd_MuldeR

25th June 2010, 22:47

Awesome, I've been needing a tool like this. How portable is it? It'd be nice to use it under Linux or OSX, too.?

It's written in Delphi, so no native Linux or Mac binary. But I guess the Win32 binary would work under Wine/Linux too.

Anyway, it's really nothing special! I hacked that together in three minutes. You could probably do the same with a few lines of Shell/Python/Perl script :p

mgb

25th June 2010, 22:50

Are there any sources of x264 built as a shared library for windows?

I need to write a video stream as x264, the vfw is too slow and IU haven't had any luck trying to build it.

LoRd_MuldeR

25th June 2010, 22:53

Are there any sources of x264 built as a shared library for windows?

I need to write a video stream as x264, the vfw is too slow and IU haven't had any luck trying to build it.

http://forum.doom9.org/showthread.php?p=1412089#post1412089 :p

mgb

25th June 2010, 23:07

http://forum.doom9.org/showthread.php?p=1412089#post1412089 :p
Excellent thanks, now I just have to go through the Avidemux source to work out how to use it!

I need to record 1080p @ 60fps so the alternative is a $1000 hardware H264 capture board or a very big raid system.

Thanks again
Martin

ps. Unless there is a way of using the x264 command line client to read raw YUV from a pipe?

LoRd_MuldeR

25th June 2010, 23:34

Excellent thanks, now I just have to go through the Avidemux source to work out how to use it!

No need for that. All you need is the interface defined in "x264.h".

Applications using x264 as a library don't need to know anything but the functions/types defined in that file ;)

But make sure you use a "x264.h" that matches the DLL version. Currently 97 for my DLL's.

Anyway, you can have a look here (mainly the "encoder.cpp" will be interesting) for an usage example:
http://svn.berlios.de/wsvn/avidemux/branches/avidemux_2.5_branch_gruntster/plugins/ADM_videoEncoder/ADM_vidEnc_x264/#_branches_avidemux_2.5_branch_gruntster_plugins_ADM_videoEncoder_ADM_vidEnc_x264_

ps. Unless there is a way of using the x264 command line client to read raw YUV from a pipe?

This works for sure ;)

your_app.exe | x264.exe --demuxer yuv [more options] - 720x576

I need to record 1080p @ 60fps so the alternative is a $1000 hardware H264 capture board or a very big raid system.

1080p @ 60 fps will be though! I guess you want to use "--preset superfast" or even "--preset ultrafast" for that purpose ;)

mgb

25th June 2010, 23:48

No need for that. All you need is the interface defined in "x264.h".
I understand that there are one or two options to set in H264 ;-)

1080p @ 60 fps will be though! I guess you want to use "--preset superfast" or even "--preset ultrafast" for that purpose ;)
That's what I was trying with the vfw build but I was only getting 6-7fps.
It may have been either a disk I/O bottleneck or a threading issue - it was only using 15% CPU on a 8 core machine

LoRd_MuldeR

25th June 2010, 23:48

Testing on ffmpeg's changelog resulted in a ~200 revision disparity, though (SVN reports r23791, git2rev reports r23587). Is that on their end with the SVN/git syncing process, or do you think something else is going on?

Well, I simply grab the log from git via "git log > file.txt" and then I walk through all lines (in reverse direction) counting the lines that start with a "commit" keyword.

This is a very simple approach and it works fine for the x264 log. No idea what is going on with the FFmpeg log. Maybe they have branches and other fancy stuff that I don't handle at all...

LoRd_MuldeR

25th June 2010, 23:53

That's what I was trying with the vfw build but I was only getting 6-7fps.
It may have been either a disk I/O bottleneck or a threading issue - it was only using 15% CPU on a 8 core machine

I get something like ~10 fps for 1080p footage (source is a HuffYUV AVI from HDD) with "--preset ultrafast --crf 22" on my Core2 Q6600.

CPU usage is around 33% for this scenario...

EDIT: Actually for this scenario the "veryfast" preset doesn't seem to be noticeably slower on my system, but causes ~55% CPU usage.

J_Darnley

25th June 2010, 23:55

LoRd_MuldeR

26th June 2010, 00:03

I get something like ~10 fps for 1080p footage (source is a HuffYUV AVI from HDD) with "--preset ultrafast --crf 22" on my Core2 Q6600.

CPU usage is around 33% for this scenario...

EDIT: Actually for this scenario the "veryfast" preset doesn't seem to be noticeably slower on my system, but causes ~55% CPU usage.

I need to correct myself: The source turned out to be FFV1, not HuffYUV :o

This time I used a real HuffYUV 1080p source and I got ~22 fps with "--preset ultrafast --crf 22" for 1080p :cool:

qyot27

26th June 2010, 00:06

It's written in Delphi, so no native Linux or Mac binary. But I guess the Win32 binary would work under Wine/Linux too.

Anyway, it's really nothing special! I hacked that together in three minutes. You could probably do the same with a few lines of Shell/Python/Perl script :p
Yeah, Wine is always an option, I simply prefer to use Wine as a last resort (I was wrangling with it yesterday night trying to get Sherpya's mencoder build to correctly interpret Kanji in ASS subtitles; I have RVM's native build from SMPlayer's PPA working fine for that, but getting Sherpya's build to work was for the benefit of another system where I don't have that luxury).

git rev-list HEAD | wc -l | sed -e 's/^ *//' > revision.txt
Thanks. I knew that there was probably a way to do it with git and some of the query tools *nix systems usually come with, but said tools' usage generally goes right over my head.

I would guess this is because of libswscale being in the same repo on the SVN side.
I suspected as much, although running the rev-list command above on just libswscale prints out a revision number in the 1000s. I guess it's not a case of simple addition.

Doesn't matter as much for ffmpeg, though - I would only need it for some of the forks that lack an SVN repo to pull from (specifically the Lagarith decoder and Ordered Chapters branches hosted on repo.or.cz (http://repo.or.cz/w/FFMpeg-mirror.git/forks)), and I don't mess with them all too often, or I just do a regular merge with the main project, so it wouldn't really matter, knowing what the main branch's revision number is at.

In any case, this means I can [easily] use the revision number for x264 when I pack up tarballs rather than needing to compile it first to find out what the number is, or my usual solution of just using the date I cloned it instead.

ffmpeg's git doesn't track this "release" business they're doing.
'Release'? Do you mean the 0.5.2 or 0.6 packages? I ignore those completely.

J_Darnley

26th June 2010, 00:19

'Release'? Do you mean the 0.5.2 or 0.6 packages? I ignore those completely.

Yes, they are branches in the svn tree. A commit to one of these will have a revision number which is not known by git. Try subversion rev 23747, you won't find it in the git log.

qyot27

26th June 2010, 01:22

Yes, they are branches in the svn tree. A commit to one of these will have a revision number which is not known by git. Try subversion rev 23747, you won't find it in the git log.
I fully admit I might be misunderstanding this, but it seems like it is in there, though:
http://git.ffmpeg.org/?p=ffmpeg;a=commit;h=ebc0bc8eba20291fe45b0916020a4e2062c13464

Committed 5 days ago, although the SVN commit was logged to yesterday.

Maccara

26th June 2010, 06:59

Yes, they are branches in the svn tree. A commit to one of these will have a revision number which is not known by git. Try subversion rev 23747, you won't find it in the git log.

Why don't they track the branches in the mirror? It's trivial. (or do they? didn't check, as I did my own mirror long ago when I had trouble with the "official" mirror)

Shows just fine in mine:

$ git show 5aa73351e66
commit 5aa73351e66c88a0b5c1d0baa9cf9e801bacd5bf
Author: siretart <siretart@9553f0bf-9b14-0410-a0b8-cfaf0461ba5b>
Date: Thu Jun 24 05:46:58 2010 +0000

10l: aacsbr: Fix f_master[2] calculation when k2diff == -1.

backport r23660 by alexc

git-svn-id: svn://svn.ffmpeg.org/ffmpeg/branches/0.6@23747 9553f0bf-9b14-0410-a0b8-cfaf0461ba5b

diff --git a/libavcodec/aacsbr.c b/libavcodec/aacsbr.c
index 0de81a5..0de89c2 100644
--- a/libavcodec/aacsbr.c
+++ b/libavcodec/aacsbr.c
@@ -402,7 +402,7 @@ static int sbr_make_f_master(AACContext *ac, SpectralBandReplication *sbr,
k2diff = sbr->k[2] - sbr->k[0] - sbr->n_master * dk;
if (k2diff < 0) {
sbr->f_master[1]--;
- sbr->f_master[2]-= (k2diff < 1);
+ sbr->f_master[2]-= (k2diff < -1);
} else if (k2diff) {
sbr->f_master[sbr->n_master]++;
}

J_Darnley

26th June 2010, 10:09

The branches are not tracked on git.ffmpeg.org. If they were everyone should be able to find this commit twice over

Maccara

26th June 2010, 11:29

The branches are not tracked on git.ffmpeg.org.

Apparently so. I just wonder, why not...

B.F.

28th June 2010, 03:24

Any rev. 1659 with fade compensation patch?

MatLz

28th June 2010, 07:10

Any rev. 1659 with fade compensation patch?

There is one here (http://vfrmaniac.fushizen.eu/x264/x264_DANGEROUS/1600-1699/)
Other good things in bonus like fgo, mixaq...

rack04

29th June 2010, 16:30

Is it true that mingw has to be patched to compile the latest ffmpeg? From what I've read stdio.h and string.h must be patched.

bnshrdr

29th June 2010, 18:45

Is it true that mingw has to be patched to compile the latest ffmpeg? From what I've read stdio.h and string.h must be patched.
What version of mingw are you using?

Currently I'm using komisar's 4.4.4 build (4.5.0 had some i686 problems) and I'm not having any problems building ffmpeg, despite the small patch needed in libswscale/swscale_template.c

What kinds of errors/warnings are you running into?

Underground78

29th June 2010, 18:56

There is one here (http://vfrmaniac.fushizen.eu/x264/x264_DANGEROUS/1600-1699/)
Other good things in bonus like fgo, mixaq...

Can any of the patches used in this build become part of main tree one day ?

rack04

29th June 2010, 19:49

What version of mingw are you using?

Currently I'm using komisar's 4.4.4 build (4.5.0 had some i686 problems) and I'm not having any problems building ffmpeg, despite the small patch needed in libswscale/swscale_template.c

What kinds of errors/warnings are you running into?

I am using komisar's 4.4.4 build and I get the following error:

libavfilter/parseutils.c: In function 'color_table_compare'"
libavfilter/parseutils.c:217: error: implicit declaration of function 'strcasecmp'
make: *** [libavfilter/parseutils.o] Error 1

komisar

29th June 2010, 20:35

rack04

workaround-1: from general.texi in ffmpeg:FFmpeg cannot be compiled because of broken system headers, add
@code{--extra-cflags=-U__STRICT_ANSI__} to the configure options as aworkaround.

workaround-2: use "--disable-filters" in configure options for ffmpeg

information about this bug: http://permalink.gmane.org/gmane.comp.gnu.mingw.announce/2815

bnshrdr

29th June 2010, 20:42

Thanks komisar. I kept seeing that #ifndef __STRICT_ANSI__ in the header files and knew there had to be a way to feed it to gcc.

kemuri-_9

29th June 2010, 23:29

__STRICT_ANSI__ is defined automatically by -std=c99
which is what ffmpeg uses for some reason unbeknownst to me.
it should realistically use -std=gnu99 instead

rack04

1st July 2010, 04:49

Toolchain:
GCC 4.4.4
GPAC 0.4.6
Pthreads 2.9.0.0
Yasm 1.0.1
LAVF/FFMS Builds:

ffmpeg SVN-r23925 and libswscale SVN-r31591 (http://www.multiupload.com/RHKW6F9CM1)
ffms2 SVN-r314 (http://www.multiupload.com/OXSZWS0CXD)
x264 Builds:

x264_x64_r1659M (http://www.multiupload.com/MEXR8EO7J4)
x264_x86_r1659M (http://www.multiupload.com/E1J731O9LH)
System: MINGW
asm: yes
avs input: yes
lavf input: yes
ffms input: yes
mp4 output: yes
pthread: yes
debug: no
gprof: no
PIC: no
shared: no
visualize: no
Patches:
x264_sws_typecast (http://pastebin.com/nzYGvT9S)
x264_demuxer_threads (http://pastebin.com/qWkMKqbg)
x264_fade_compensation (http://pastebin.com/BBwB72Az)
x264_open_gop_bluray (http://pastebin.com/TvhtWhTu)