Welcome to Doom9's Forum, THE in-place to be for everyone interested in DVD conversion.

Before you start posting please read the forum rules. By posting to this forum you agree to abide by the rules.

 

Go Back   Doom9's Forum > Video Encoding > MPEG-4 AVC / H.264

Reply
 
Thread Tools Search this Thread Display Modes
Old 20th December 2008, 21:37   #81  |  Link
Blue_MiSfit
Derek Prestegard IRL
 
Blue_MiSfit's Avatar
 
Join Date: Nov 2003
Location: Los Angeles
Posts: 5,416
I smell the need for some MeGUI updates soon to automate using 64 bit x264

~MiSfit
Blue_MiSfit is offline   Reply With Quote
Old 20th December 2008, 21:49   #82  |  Link
skystrife
Registered User
 
skystrife's Avatar
 
Join Date: Feb 2007
Posts: 176
Quote:
Originally Posted by LoRd_MuldeR View Post
LOL. Stupid my used wrong syntax to create the pipe and overwrote x264_x64.exe with random data

It does work now, like this:


However I get some continuous horizontal shift in the output file. I suspect the color format is wrong. Any ideas?


[EDIT]

Here is a sample of what I get:
http://www.mediafire.com/file/kcy4n2...x64-avs2yuv.7z
When piping using stdout x264 is expecting raw yuv input, and avs2yuv is outputting y4m. The detection for this is based on file extension, and stdout has none, so x264 wrongly assumes yuv input.

Try using avs2yuv -raw file.avs - | x264 --stuff.

/me curses the lack of named pipes in windows.
skystrife is offline   Reply With Quote
Old 20th December 2008, 21:49   #83  |  Link
burfadel
Registered User
 
Join Date: Aug 2006
Posts: 2,235
Quote:
Originally Posted by LoRd_MuldeR View Post
Yes, passing data via STDOUT from a 32-Bit process to a 64-Bit process (or vice versa) should be possible.
That's because in that case we have to two completely separate processes. No 32-Bit and 64-Bit code is mixed in the same process.
I suspect I did not call it properly, because I didn't work with x264 and input from STDOUT so far...
I guess passing data via stdout is kind of like sending the frames to a buffer?

I wasn't sure of the exact issues, but it does appear then that it is possible to use x64 x264 using 32 bit avisynth & Gui frontends???
burfadel is offline   Reply With Quote
Old 20th December 2008, 21:53   #84  |  Link
LoRd_MuldeR
Software Developer
 
LoRd_MuldeR's Avatar
 
Join Date: Jun 2005
Location: Last House on Slunk Street
Posts: 12,934
Quote:
Originally Posted by burfadel View Post
I guess passing data via stdout is kind of like sending the frames to a buffer?
I pipe is nothing but a FIFO buffer. Similar to a message queue.

Quote:
Originally Posted by burfadel View Post
I wasn't sure of the exact issues, but it does appear then that it is possible to use x64 x264 using 32 bit avisynth & Gui frontends???
Should be possible indeed! The front-end needs to call like this:

Code:
cmd.exe /c ""C:\Path to Tools\avs2yuv.exe" "C:\Path to Source\input.avs" -raw - | "C:\Path to Tools\x264_x64.exe" --crf 22 --progress --output "C:\Path to Output\output.mkv" -"
__________________
There was of course no way of knowing whether you were being watched at any given moment.
How often, or on what system, the Thought Police plugged in on any individual wire was guesswork.


LoRd_MuldeR is offline   Reply With Quote
Old 20th December 2008, 21:56   #85  |  Link
roozhou
Registered User
 
Join Date: Apr 2008
Posts: 1,181
Quote:
Originally Posted by LoRd_MuldeR View Post
I pipe is nothing but a FIFO buffer. Similar to a message queue.



Should be possible indeed! The front-end needs to call like this:

Code:
cmd.exe /c ""C:\Path to Tools\avs2yuv.exe" "C:\Path to Source\input.avs" -raw - | "C:\Path to Tools\x264_x64.exe" --crf 22 --progress --output "C:\Path to Output\output.mkv" -"
Are there mplayer/mencoder builds for win64? It should make things easier.
roozhou is offline   Reply With Quote
Old 20th December 2008, 21:59   #86  |  Link
LoRd_MuldeR
Software Developer
 
LoRd_MuldeR's Avatar
 
Join Date: Jun 2005
Location: Last House on Slunk Street
Posts: 12,934
Quote:
Originally Posted by skystrife View Post
When piping using stdout x264 is expecting raw yuv input, and avs2yuv is outputting y4m. The detection for this is based on file extension, and stdout has none, so x264 wrongly assumes yuv input.

Try using avs2yuv -raw file.avs - | x264 --stuff.
That works flawlessly! Thank you!

Quote:
Originally Posted by skystrife View Post
/me curses the lack of named pipes in windows.
Windows does have named pipes. No idea how to create them from in the shell, but:
http://msdn.microsoft.com/en-us/libr...50(VS.85).aspx
__________________
There was of course no way of knowing whether you were being watched at any given moment.
How often, or on what system, the Thought Police plugged in on any individual wire was guesswork.


LoRd_MuldeR is offline   Reply With Quote
Old 20th December 2008, 22:15   #87  |  Link
LoRd_MuldeR
Software Developer
 
LoRd_MuldeR's Avatar
 
Join Date: Jun 2005
Location: Last House on Slunk Street
Posts: 12,934
Quote:
Originally Posted by roozhou View Post
Are there mplayer/mencoder builds for win64? It should make things easier.
Not really. Even if you had a working MEncoder x64 build with x264 x64 enabled, you'd still need Avisynth x64
Unless you don't work with Avisynth and rely on MEncoder's decoders/filters instead, of course...

The good thing with the "pipe" method is that we can use good old 32-Bit Avisynth including all the famous filters
__________________
There was of course no way of knowing whether you were being watched at any given moment.
How often, or on what system, the Thought Police plugged in on any individual wire was guesswork.



Last edited by LoRd_MuldeR; 20th December 2008 at 22:37.
LoRd_MuldeR is offline   Reply With Quote
Old 21st December 2008, 01:50   #88  |  Link
Snowknight26
Registered User
 
Join Date: Aug 2007
Posts: 1,391
My preliminary results show that the x64 version is 15% faster.

1080p video, ~15000kbps, essentially max settings.
x86: 1000 frames @ 2.11fps
x64: 1000 frames @ 2.44fps

Code:
C:\temp\Encoding\verticallimit>cd ..

C:\temp\Encoding>avs2yuv.exe verticallimit\verticallimit.avs -raw -   | x264.x64
 --pass 2 --bitrate 15093 --stats "verticallimit/verticallimit.stats" --bframes
3 --level 4.1 --ref 4 --mixed-refs --no-fast-pskip --b-pyramid --weightb --direc
t auto --deblock -3:-3 --subme 9 --trellis 2 --psy-rd 1.0:0 --analyse all --8x8d
ct --no-dct-decimate --me umh --threads auto --thread-input --progress --no-psnr
 --output "verticallimit/verticallimit.mkv" - 1912x1040
x264 [warning]: width or height not divisible by 16 (1912x1040), compression wil
l suffer.
x264 [info]: using cpu capabilities: MMX2 SSE2Fast SSSE3 Cache64
verticallimit\verticallimit.avs: 1912x1040, 24000/1001 fps, 1000 frames
x264 [info]: profile High, level 4.1
x264 [info]: slice I:14    Avg QP:15.25  size:128229
x264 [info]: slice P:505   Avg QP:19.12  size: 89484
x264 [info]: slice B:481   Avg QP:21.82  size: 53901
x264 [info]: consecutive B-frames: 26.7% 20.1% 14.3% 38.9%
x264 [info]: mb I  I16..4: 29.0% 60.4% 10.5%
x264 [info]: mb P  I16..4:  1.8%  7.0%  0.5%  P16..4: 47.1% 24.7%  8.7%  0.1%  0
.1%    skip:10.0%
x264 [info]: mb B  I16..4:  0.0%  0.8%  0.0%  B16..8: 57.6%  1.1%  2.2%  direct:
11.2%  skip:27.1%  L0:44.2% L1:50.7% BI: 5.1%
x264 [info]: 8x8 transform  intra:73.0%  inter:45.9%
x264 [info]: direct mvs  spatial:87.3%  temporal:12.7%
x264 [info]: ref P L0  75.6% 13.4%  6.7%  4.4%
x264 [info]: ref B L0  68.2% 22.5%  9.3%
x264 [info]: ref B L1  92.2%  7.8%
x264 [info]: SSIM Mean Y:0.9486720
x264 [info]: kb/s:14582.2

encoded 1000 frames, 2.44 fps, 14582.61 kb/s

C:\temp\Encoding>ping 127.0.0.1

Pinging 127.0.0.1 with 32 bytes of data:
Reply from 127.0.0.1: bytes=32 time<1ms TTL=128
Reply from 127.0.0.1: bytes=32 time<1ms TTL=128
Reply from 127.0.0.1: bytes=32 time<1ms TTL=128
Reply from 127.0.0.1: bytes=32 time<1ms TTL=128

Ping statistics for 127.0.0.1:
    Packets: Sent = 4, Received = 4, Lost = 0 (0% loss),
Approximate round trip times in milli-seconds:
    Minimum = 0ms, Maximum = 0ms, Average = 0ms

C:\temp\Encoding>x264 --pass 2 --bitrate 15093 --stats "verticallimit/verticalli
mit.stats" --bframes 3 --level 4.1 --ref 4 --mixed-refs --no-fast-pskip --b-pyra
mid --weightb --direct auto --deblock -3:-3 --subme 9 --trellis 2 --psy-rd 1.0:0
 --analyse all --8x8dct --no-dct-decimate --me umh --threads auto --thread-input
 --progress --no-psnr --output "verticallimit/verticallimit.mkv" "verticallimit/
verticallimit.avs"
avis [info]: 1912x1040 @ 23.98 fps (1000 frames)
x264 [warning]: width or height not divisible by 16 (1912x1040), compression wil
l suffer.
x264 [info]: using cpu capabilities: MMX2 SSE2Fast SSSE3 Cache64
x264 [warning]: 2nd pass has fewer frames than 1st pass (1000 vs 179031)
x264 [info]: profile High, level 4.1
x264 [info]: slice I:14    Avg QP:15.11  size:131874
x264 [info]: slice P:505   Avg QP:18.97  size: 93347
x264 [info]: slice B:481   Avg QP:21.68  size: 56405
x264 [info]: consecutive B-frames: 26.7% 20.1% 14.3% 38.9%
x264 [info]: mb I  I16..4: 28.6% 60.5% 10.9%
x264 [info]: mb P  I16..4:  1.9%  7.7%  0.6%  P16..4: 46.0% 24.9%  8.8%  0.1%  0
.1%    skip: 9.9%
x264 [info]: mb B  I16..4:  0.0%  0.9%  0.0%  B16..8: 57.5%  1.1%  2.3%  direct:
11.5%  skip:26.6%  L0:44.0% L1:50.4% BI: 5.6%
x264 [info]: 8x8 transform  intra:74.3%  inter:45.0%
x264 [info]: direct mvs  spatial:87.3%  temporal:12.7%
x264 [info]: ref P L0  75.6% 13.3%  6.7%  4.4%
x264 [info]: ref B L0  68.5% 22.3%  9.3%
x264 [info]: ref B L1  92.2%  7.8%
x264 [info]: SSIM Mean Y:0.9500119
x264 [info]: kb/s:14599.9

encoded 1000 frames, 2.11 fps, 14600.31 kb/s

C:\temp\Encoding>pause
Press any key to continue . . .
Snowknight26 is offline   Reply With Quote
Old 21st December 2008, 04:48   #89  |  Link
qyqgpower
Registered User
 
Join Date: Jul 2005
Posts: 108
the output of mingw x64 version seems to be different from msvc and normal 32bit version. (of course, without --threads option)

mingw x64
x264 [info]: SSIM Mean Y:0.9804991
x264 [info]: PSNR Mean Y:44.020 U:47.821 V:48.225 Avg:44.906 Global:41.938 kb/s:1574.50
encoded 2184 frames, 19.59 fps, 1574.64 kb/s

msvc2008 x64
x264 [info]: SSIM Mean Y:0.9804984
x264 [info]: PSNR Mean Y:44.019 U:47.820 V:48.224 Avg:44.905 Global:41.938 kb/s:1574.53
encoded 2184 frames, 19.65 fps, 1574.68 kb/s

mingw x86
x264 [info]: SSIM Mean Y:0.9804984
x264 [info]: PSNR Mean Y:44.019 U:47.820 V:48.224 Avg:44.905 Global:41.938 kb/s:1574.53
encoded 2184 frames, 18.56 fps, 1574.68 kb/s
qyqgpower is offline   Reply With Quote
Old 21st December 2008, 05:10   #90  |  Link
roozhou
Registered User
 
Join Date: Apr 2008
Posts: 1,181
Quote:
Originally Posted by LoRd_MuldeR View Post
Windows does have named pipes. No idea how to create them from in the shell, but:
http://msdn.microsoft.com/en-us/libr...50(VS.85).aspx
Rename attached file to Namedpipe.exe

usage:

Namedpipe pipename exename args

1) pipename must start with \\.\pipe\
2) exename is the program that you want to redirect its stdin
3) args is extra command-line arguments for exename, remember to put at least 2 spaces between exename and args

e.g. pipe mplayer's to neroAacEnc

1) in 1st cmd window, type "Namedpipe \\.\pipe\1.wav neroAacEnc.exe -q 0.3 -ignorelength -if - -of xxx.mp4"
2) in 2nd cmd window, type "mplayer xxx.xxx -vc NULL -vo NULL -ao fast:file=\\.\pipe\1.wav"

Source code(compiled in VC2005)
Code:
#pragma comment(linker,"/INCREMENTAL:NO /SECTION:.text,EWR /ALIGN:0x1000")
#include <Windows.h>

#define BUFSIZE 0x400000
char processname[256];
char pipename[64];
char *cmd,*tmp;
HANDLE hPipe/*,hCP,hChildIn*/;
STARTUPINFOA stinfo={0};
PROCESS_INFORMATION pinfo={0};
SECURITY_ATTRIBUTES sa={sizeof(SECURITY_ATTRIBUTES),NULL,TRUE};
void WINAPI Main()
{
	DWORD quoteon,state,cmdend;
	cmd = GetCommandLineA();
	quoteon = 0;
	state = 0;
	cmdend=0;
	tmp = pipename;
	while (!cmdend)
	{
		switch(*cmd)
		{
		case 0:
			cmdend=1;
			*tmp=0;
			goto END;
		case '\"':
			quoteon = !quoteon;
			break;
		case ' ':
		case '\t':
			if (quoteon)
			{
				switch(state)
				{
				case 2:
				case 4:
					*tmp++ = *cmd;
				}
			}
			else
			{
				switch(state)
				{
				case 0:
					state++;
					break;
				case 4:
					cmdend = 1;
				case 2:
					*tmp = 0;
					state++;
				}
			}
			break;
		default:
			switch(state)
			{
			case 3:
				tmp = processname;
			case 1:
				state++;
			case 2:
			case 4:
				*tmp++ = *cmd;
			}
		}
		cmd ++;
	}
END:
	if (state<2 || quoteon)
		return;
	if (INVALID_HANDLE_VALUE == (hPipe = CreateNamedPipeA(pipename,PIPE_ACCESS_DUPLEX,PIPE_TYPE_BYTE | PIPE_READMODE_BYTE | PIPE_WAIT,PIPE_UNLIMITED_INSTANCES,BUFSIZE,BUFSIZE,0,&sa)))
		return;
	ConnectNamedPipe(hPipe,NULL);
	stinfo.cb = sizeof(STARTUPINFO);
	stinfo.dwFlags = STARTF_USESTDHANDLES;
	stinfo.hStdInput = hPipe;//hChildIn;
	stinfo.hStdOutput = GetStdHandle(STD_OUTPUT_HANDLE);
	stinfo.hStdError = GetStdHandle(STD_ERROR_HANDLE);
	if (CreateProcessA(processname,cmd,0,0,TRUE,0,NULL,NULL,&stinfo,&pinfo))
	{

		WaitForSingleObject(pinfo.hProcess,INFINITE);
	}
}
Attached Files
File Type: txt NamedPipe.txt (2.5 KB, 283 views)

Last edited by roozhou; 21st December 2008 at 05:13.
roozhou is offline   Reply With Quote
Old 21st December 2008, 05:23   #91  |  Link
kemuri-_9
Compiling Encoder
 
kemuri-_9's Avatar
 
Join Date: Jan 2007
Posts: 1,348
Quote:
Originally Posted by qyqgpower View Post
the output of mingw x64 version seems to be different from msvc and normal 32bit version. (of course, without --threads option)
which mingw64 build did you use?
...asking as there's quite a few roaming around now...
__________________
custom x264 builds & patches | F@H | My Specs
kemuri-_9 is offline   Reply With Quote
Old 21st December 2008, 10:28   #92  |  Link
qyqgpower
Registered User
 
Join Date: Jul 2005
Posts: 108
Quote:
Originally Posted by kemuri-_9 View Post
which mingw64 build did you use?
...asking as there's quite a few roaming around now...
x64 builds are from #47 and #48
x86 build is from MeGUI's auto update (Skystrife's patched build)

commandline:
avs2yuv -raw "NCOP.avs" - | "x264.exe" --crf 22 --progress --sar 32:27 --output "F:\temp.mkv" - 720x480

I did another test without piping with x86 build:

x264 [info]: SSIM Mean Y:0.9804984
x264 [info]: PSNR Mean Y:44.019 U:47.820 V:48.224 Avg:44.905 Global:41.938 kb/s:1510.04
encoded 2184 frames, 19.81 fps, 1510.18 kb/s
output is exactly at same size (17,226,071 bytes) so the bitrate calculation here is more precision, but this is not very important.
The key is that the speed penalty of piping is larger than the speed advantage of x264 win64 build.

Last edited by qyqgpower; 21st December 2008 at 10:47.
qyqgpower is offline   Reply With Quote
Old 21st December 2008, 12:04   #93  |  Link
roozhou
Registered User
 
Join Date: Apr 2008
Posts: 1,181
Quote:
Originally Posted by qyqgpower View Post
x64 builds are from #47 and #48
x86 build is from MeGUI's auto update (Skystrife's patched build)

commandline:
avs2yuv -raw "NCOP.avs" - | "x264.exe" --crf 22 --progress --sar 32:27 --output "F:\temp.mkv" - 720x480

I did another test without piping with x86 build:

x264 [info]: SSIM Mean Y:0.9804984
x264 [info]: PSNR Mean Y:44.019 U:47.820 V:48.224 Avg:44.905 Global:41.938 kb/s:1510.04
encoded 2184 frames, 19.81 fps, 1510.18 kb/s
output is exactly at same size (17,226,071 bytes) so the bitrate calculation here is more precision, but this is not very important.
The key is that the speed penalty of piping is larger than the speed advantage of x264 win64 build.
Piping of Windows cmd.exe sucks.
roozhou is offline   Reply With Quote
Old 21st December 2008, 12:05   #94  |  Link
Dark Shikari
x264 developer
 
Dark Shikari's Avatar
 
Join Date: Sep 2005
Posts: 8,689
Is the piping penalty mitigated by multithreading?
Dark Shikari is offline   Reply With Quote
Old 21st December 2008, 16:56   #95  |  Link
kemuri-_9
Compiling Encoder
 
kemuri-_9's Avatar
 
Join Date: Jan 2007
Posts: 1,348
piping is also not required if one has avisynth x64.

but now one may say 'but none of my filters work in x64!'

you can either
1. create a lossless of your filtering from the x86 version to plainly open in avisynth x64 with AVISource()
(as there are x64 versions of lags and ffdshow)

2. pipe from avisynth x86 to x64 using the TCPDeliver plugin
(compatible x86 and x64 versions are on squid_80's avs x64 filter collection page - url is floating around somewhere on doom9...)
(also not sure if this is going to be more efficient than stdout piping or not... - my guess is it'll be worse to some degree)

3. limit what you're doing to what there is x64 filters for
(i lol inside as i write this last choice)

the options are there... though with differing degrees of feasibility...
__________________
custom x264 builds & patches | F@H | My Specs

Last edited by kemuri-_9; 21st December 2008 at 16:58.
kemuri-_9 is offline   Reply With Quote
Old 21st December 2008, 17:13   #96  |  Link
burfadel
Registered User
 
Join Date: Aug 2006
Posts: 2,235
Nag Microsoft to allow high level mixed mode x86/64 instead of disallowing it altogether. Its purely a security reason why its disallowed, not a technical one By high level I mean non system level operations, such as antivirus & firewall (which require a 'driver' to allow access to the system, in which case the driver can be 64 bit and the programme 32 bit), device drivers etc that allow 'low level' access to the system. It is quite pointless disallowing mixed mode for instances of passing processed data in my opinion.
burfadel is offline   Reply With Quote
Old 21st December 2008, 17:44   #97  |  Link
LoRd_MuldeR
Software Developer
 
LoRd_MuldeR's Avatar
 
Join Date: Jun 2005
Location: Last House on Slunk Street
Posts: 12,934
Quote:
Originally Posted by burfadel View Post
Nag Microsoft to allow high level mixed mode x86/64 instead of disallowing it altogether. Its purely a security reason why its disallowed, not a technical one By high level I mean non system level operations, such as antivirus & firewall (which require a 'driver' to allow access to the system, in which case the driver can be 64 bit and the programme 32 bit), device drivers etc that allow 'low level' access to the system. It is quite pointless disallowing mixed mode for instances of passing processed data in my opinion.
Well, I'm not aware of any OS that allows 32-Bit and 64-Bit code to run in the context of the same process.
And it is possible to pass data between 32-Bit and 64-Bit processes, either trough pipes or through network sockets.
__________________
There was of course no way of knowing whether you were being watched at any given moment.
How often, or on what system, the Thought Police plugged in on any individual wire was guesswork.


LoRd_MuldeR is offline   Reply With Quote
Old 21st December 2008, 20:23   #98  |  Link
squid_80
Registered User
 
Join Date: Dec 2004
Location: Melbourne, AU
Posts: 1,963
Quote:
Originally Posted by burfadel View Post
Its purely a security reason why its disallowed, not a technical one
It's got nothing to do with security. How do you mix code that has different registers and pointer sizes?
If you want to do IPC between 32/64-bit programs the quickest method is to map the same chunk of memory into both processes and restrict access with an event/semaphore etc.
squid_80 is offline   Reply With Quote
Old 21st December 2008, 21:11   #99  |  Link
burfadel
Registered User
 
Join Date: Aug 2006
Posts: 2,235
Thats why I mentioned 'high level' access. It doesn't matter if there are different registers and pointer sizes for high level access, which this is. If it were a programme loading a DLL, then it would matter Even encoding using x264 and Avisynth is high level, as Avisynth merely serves the frames (that is, they're separate)!
burfadel is offline   Reply With Quote
Old 21st December 2008, 21:21   #100  |  Link
LoRd_MuldeR
Software Developer
 
LoRd_MuldeR's Avatar
 
Join Date: Jun 2005
Location: Last House on Slunk Street
Posts: 12,934
If an application (e.g. x264) uses Avisynth input, then it unavoidably needs to load the Avisynth DLL.
And mapping a 32-Bit DLL into the context of a 64-Bit process (or vice versa) isn't possible for the reasons mentioned by squid_80.

So you got exactly two options to use 64-Bit x264 with Avisynth input:
* Use 64-Bit Avisynth and accept the limitations it currently has.
* Call 32-Bit Avisynth from a separate 32-Bit process and transfer the "raw" data to the 64-Bit x264 process (may it be via pipe, via network socket or maybe via shared memory).

I don't know if it is possible to share memory between 32-Bit and 64-Bit processes on Windows, but pipes do work for sure.
__________________
There was of course no way of knowing whether you were being watched at any given moment.
How often, or on what system, the Thought Police plugged in on any individual wire was guesswork.



Last edited by LoRd_MuldeR; 21st December 2008 at 21:28.
LoRd_MuldeR is offline   Reply With Quote
Reply

Thread Tools Search this Thread
Search this Thread:

Advanced Search
Display Modes

Posting Rules
You may not post new threads
You may not post replies
You may not post attachments
You may not edit your posts

BB code is On
Smilies are On
[IMG] code is On
HTML code is Off

Forum Jump


All times are GMT +1. The time now is 11:52.


Powered by vBulletin® Version 3.8.11
Copyright ©2000 - 2018, vBulletin Solutions Inc.