Questions about HDMI X264 encoding

[wcwman18]

I currently use a HDMI source for my encodes. With version 1471 I am able to encode at about 36-40FPS. With any latest version I get about 15 - 16 FPS.

Here are my current scripts

x264

Load_STDCall_Plugin("C:\Program Files (x86)\AviSynth 2.5\plugins\yadif.dll")
AVISource("%AVIfile%")
ConvertToYV12()
Yadif(0,1)
Lanczos4Resize(704,400,6,6,-6,-6)

720p
Load_STDCall_Plugin("C:\Program Files (x86)\AviSynth 2.5\plugins\yadif.dll")
AVISource("%AVIfile%")
ConvertToYV12()
Yadif(0,1)
Lanczos4Resize(1280,720,6,6,-6,-6)

1080p
Load_STDCall_Plugin("C:\Program Files (x86)\AviSynth 2.5\plugins\yadif.dll")
AVISource("%AVIfile%")
ConvertToYV12()
Yadif(0,1)


Any reason things have slowed down so much for me? Any way to fix it?

[sneaker_ger]

A lot happened in over two years. I don't think anyone would want to list everything for you, especially since you didn't post your x264 settings in the first place.
But it does not matter anyways. Just assume that x264 has become better and pick new settings.

http://forum.doom9.org/showpost.php?p=1567791&postcount=2

[Asmodian]

I notice you use AVISource(). This uses windows vfw codecs to decode the video, if you changed these it could change your speed (but I doubt by that much). You could try ffms2() (http://code.google.com/p/ffmpegsource/)

Also what is your x264 command line?

edit: sneaker_ger was faster and more to the point.

[wcwman18]

I currently use a HDMI source for my encodes. With version 1471 I am able to encode at about 36-40FPS. With any latest version I get about 15 - 16 FPS.

Here are my current scripts

x264

Load_STDCall_Plugin("C:\Program Files (x86)\AviSynth 2.5\plugins\yadif.dll")
AVISource("%AVIfile%")
ConvertToYV12()
Yadif(0,1)
Lanczos4Resize(704,400,6,6,-6,-6)

720p
Load_STDCall_Plugin("C:\Program Files (x86)\AviSynth 2.5\plugins\yadif.dll")
AVISource("%AVIfile%")
ConvertToYV12()
Yadif(0,1)
Lanczos4Resize(1280,720,6,6,-6,-6)

1080p
Load_STDCall_Plugin("C:\Program Files (x86)\AviSynth 2.5\plugins\yadif.dll")
AVISource("%AVIfile%")
ConvertToYV12()
Yadif(0,1)


Any reason things have slowed down so much for me? Any way to fix it?

Here is a little more detail

HD I use --threads 12 --crf 26 --direct auto --level 4.1 --ref 4 --subme 9 --weightp 2 --b-adapt 2 --b-pyramid strict --me umh --keyint 300 --min-keyint 30 --aud --output %mkvfile% %avsfile%

SD I use --threads 12 --crf 19 --vbv-bufsize 1450 --vbv-maxrate 1500 --direct auto --level 3.1 --ref 5 --subme 9 --weightp 2 --b-adapt 2 --b-pyramid strict --me umh --keyint 300 --min-keyint 30 --aud --output %mkvfile% %avsfile%

My card captures uncompressed 1920x1080 AVI JPEG Motion.

I use a Black Magic Intensity Pro. Hope that helps a bit more.

[Atak_Snajpera]

crf 26 for hd??? most people don't go below 20. i doubt that your encodings look at least aceptable.

[Lyris]

My card captures uncompressed 1920x1080 AVI JPEG Motion.
Motion JPEG (from the Blackmagic Media Express program) is actually quite heavily compressed.

[wcwman18]

Motion JPEG (from the Blackmagic Media Express program) is actually quite heavily compressed.

Any capture program that works well for Black Magic to cap to MPEG2?

[wcwman18]

Any capture program that works well for Black Magic to cap to MPEG2?

No one has any suggestions?

[mariush]

Have you tried one of the latest Virtualdub versions?

It has some options to pass the output to a command line encoder, but I'm not sure if that applies to Capture mode. I know it worked to pass output to x264.exe in the normal mode.

[wcwman18]

I am on the latest build of x264 and I have noticed a slight speed increase but nothing like what I had before. Can someone look at my command line and script and see if I am missing something.

x264 command line I use
--threads 12 --thread-input --crf 19 --vbv-bufsize 1450 --vbv-maxrate 1500 --direct auto --level 3.1 --ref 5 --subme 9 --weightp 2 --b-adapt 2 --b-pyramid strict --me umh --keyint 300 --min-keyint 30 --aud --output %mkvfile% %avsfile%

I have a 6 core AMD processor.

[wcwman18]

This is my script I use Load_STDCall_Plugin("C:\Program Files (x86)\AviSynth 2.5\plugins\yadif.dll")
AVISource("%AVIfile%")
Yadif(0,1)
Lanczos4Resize(704,400,6,6,-6,-6)

[sneaker_ger]

1.) Check CPU utilization for bottleneck (and fix it)
2.) Pick new settings with the speed you want. (see --preset and --tune in --fullhelp)

[wcwman18]

1.) Check CPU utilization for bottleneck (and fix it)
2.) Pick new settings with the speed you want. (see --preset and --tune in --fullhelp)

I am not sure how to do any of that

[sneaker_ger]

1.) ctrl + shift + esc -> performance -> cpu usage. Is it constantly 90% or more?

[wcwman18]

1.) ctrl + shift + esc -> performance -> cpu usage. Is it constantly 90% or more?

During 1080p encoding it is yes

[sneaker_ger]

Then proceed with step 2.)
New settings:
x264 --preset medium --crf 26 --level 4.1 --output %mkvfile% %avsfile%

If "medium" is too slow or too fast, replace it with:
ultrafast, superfast, veryfast, faster, fast, medium, slow, slower, veryslow, placebo

You can add "--tune film" for "normal" films and CGI or "--tune animation" for cartoons.

[wcwman18]

Then proceed with step 2.)
New settings:
x264 --preset medium --crf 26 --level 4.1 --output %mkvfile% %avsfile%

If "medium" is too slow or too fast, replace it with:
ultrafast, superfast, veryfast, faster, fast, medium, slow, slower, veryslow, placebo

You can add "--tune film" for "normal" films and CGI or "--tune animation" for cartoons.

--threads 12 --thread-input --crf 26 --direct auto --level 4.1 --ref 4 --subme 9 --weightp 2 --b-adapt 2 --b-pyramid strict --me umh --keyint 300 --min-keyint 30 --aud --output %mkvfile% %avsfile%

This is what I was given to use by the people I encode for.

[wcwman18]

Anything that seems wrong?

[sneaker_ger]

Not really "wrong", no.
If these are the settings you are forced to use and there's no bottleneck, then everything is "fine". Nothing you can do.

[wcwman18]

Not really "wrong", no.
If these are the settings you are forced to use and there's no bottleneck, then everything is "fine". Nothing you can do.

I was told by my co-worker that has intel I7 8 cores PC that with
--thread-input I would get faster FPS due to it using all 6 cores I have. He gets about 50fps I get about 22fps

[sneaker_ger]

Does he have two Core-i7 quad cores(or rather Xeon?) or a single i7 quad core with Hyper-threading?
Otherwise: exactly same source, same AviSynth version, same Yadif, same x264 binary, same settings?

[sneaker_ger]

Are you using 64 bit x264? It can give you at least a few percent speedup, but you need to pipe (http://forum.doom9.org/showpost.php?p=1565165&postcount=116) if you want to use it with 32 bit AviSynth.

[wcwman18]

Does he have two Core-i7 quad cores(or rather Xeon?) or a single i7 quad core with Hyper-threading?
Otherwise: exactly same source, same AviSynth version, same Yadif, same x264 binary, same settings?

a single i7 quad core with Hyper-threading exactly same source, same AviSynth version, same Yadif, same x264 binary, same settings?

[wcwman18]

Are you using 64 bit x264? It can give you at least a few percent speedup, but you need to pipe (http://forum.doom9.org/showpost.php?p=1565165&postcount=116) if you want to use it with 32 bit AviSynth.

We use pipebuf.exe

[sneaker_ger]

a single i7 quad core with Hyper-threading exactly same source, same AviSynth version, same Yadif, same x264 binary, same settings?

No idea then. Core-i7 can be fast, but twice as fast sounds a bit excessive. I have no experience with Sandy Bridge, though. Or did he over-clock to insanity?

[wcwman18]

No idea then. Core-i7 can be fast, but twice as fast sounds a bit excessive. I have no experience with Sandy Bridge, though. Or did he over-clock to insanity?

I know his PC is overclocked yes

[wcwman18]

Is HDMI source processed different then Component HD source?

[Dark Shikari]

Many of your settings don't even do anything. --thread-input is already the default.

Please use the presets instead of setting options that you don't understand.

[wcwman18]

Many of your settings don't even do anything. --thread-input is already the default.

Please use the presets instead of setting options that you don't understand.

How would I set the defaults?

[Dark Shikari]

By definition, a default doesn't need to be set.

[Blue_MiSfit]

I'd suggest using a speed preset, instead of manually specifying settings. It's pretty simple - if you're getting high CPU usage (i.e. 80%-90% or more) and are unhappy with speed, then use a faster speed preset until you are happy with the speed.

Also.. an i7 doesn't have 8 cores. It has 4 cores, and hyperhtreading, which means it presents 8 logical CPUs to the Operating System. Not the same thing, though it does typically help with x264 performance.

Derek

[wcwman18]

I'd suggest using a speed preset, instead of manually specifying settings. It's pretty simple - if you're getting high CPU usage (i.e. 80%-90% or more) and are unhappy with speed, then use a faster speed preset until you are happy with the speed.

Also.. an i7 doesn't have 8 cores. It has 4 cores, and hyperhtreading, which means it presents 8 logical CPUs to the Operating System. Not the same thing, though it does typically help with x264 performance.

Derek

Thanks so much for all your help. Do you have a full command line I can test?

[wcwman18]

This is what they have me use currently --threads 12 --crf 19 --vbv-bufsize 1450 --vbv-maxrate 1500 --direct auto --level 3.1 --ref 5 --subme 9 --weightp 2 --b-adapt 2 --b-pyramid strict --me umh --keyint 300 --min-keyint 30 --aud --output %mkvfile% %avsfile%

I get about 17FPS when encoding

[Pat357]

If you have to use this settings, there is nothing you can do except start looking for a faster system.
The new IVB systems will or course be faster ; the 6 core (+HT) systems will be a lot faster.

If you are allowed to change the X264 settings, I'd recommend trying one of the following :


x264 --crf 19 --preset medium --tune film -o output input
x264 --crf 19 --preset fast --tune film -o output input
x264 --crf 19 --preset faster --tune film -o output input

For film use --tune film; for anime, use --tune animation

Besides the presets above, you also have :
ultrafast, superfast, veryfast, faster, fast, medium, slow, slower, veryslow and placebo. Choose the one that you like.
For the same --crf xx , all presets should give about the same output quality, but the faster presets give a larger output file. (faster usually means less compression)

Also using 64-bit version will often be faster, but you have to use "pipe" you video :
I use for example
avs2yuv -raw "my.avs" - | x264 --input-res 1920x1080 --crf 16 --preset slow --tune film -o "output.mkv" -

[wcwman18]

Based on this that I use can you give me a full command line with the full changes along with what I use that I need to keep.

--threads 12 --crf 19 --vbv-bufsize 1450 --vbv-maxrate 1500 --direct auto --level 3.1 --ref 5 --subme 9 --weightp 2 --b-adapt 2 --b-pyramid strict --me umh --keyint 300 --min-keyint 30 --aud --output %mkvfile% %avsfile%

If you have to use this settings, there is nothing you can do except start looking for a faster system.
The new IVB systems will or course be faster ; the 6 core (+HT) systems will be a lot faster.

If you are allowed to change the X264 settings, I'd recommend trying one of the following :


x264 --crf 19 --preset medium --tune film -o output input
x264 --crf 19 --preset fast --tune film -o output input
x264 --crf 19 --preset faster --tune film -o output input

For film use --tune film; for anime, use --tune animation

Besides the presets above, you also have :
ultrafast, superfast, veryfast, faster, fast, medium, slow, slower, veryslow and placebo. Choose the one that you like.
For the same --crf xx , all presets should give about the same output quality, but the faster presets give a larger output file. (faster usually means less compression)

Also using 64-bit version will often be faster, but you have to use "pipe" you video :
I use for example
avs2yuv -raw "my.avs" - | x264 --input-res 1920x1080 --crf 16 --preset slow --tune film -o "output.mkv" -

[wcwman18]

Anyone?

[Blue_MiSfit]

Start with this:

--crf 19 --vbv-bufsize 1450 --vbv-maxrate 1400 --preset superfast --tune film

[wcwman18]

Start with this:

--crf 19 --vbv-bufsize 1450 --vbv-maxrate 1400 --preset superfast --tune film

Is that the only code line I need? No output info or anything?

[Asmodian]

Is that the only code line I need? No output info or anything?

Well no, you do still need the output. This is the full line I would use given your original settings and Blue_MiSfit's suggestions.

x264 --preset superfast --crf 19 --tune film --vbv-bufsize 1450 --vbv-maxrate 1500 --output %mkvfile% %avsfile%

Then pick the slowest preset that is "fast enough".

[Blue_MiSfit]

Correct. Superfast is only a starting point. Do lots of tests, and determine which speed preset is actually appropriate for your case.

Also, are you absolutely certain you need VBV?

[tyee]

@Blue_Misfit
I just ran across an older thread about you discussing using VBV-maxrate with crf to set a maximum bitrate. Did you ever stick to that method or did you drop the vbv-maxrate?? I tried it last night and it was working fine but I was wondering what is actually happening when the bitrate is instantly limited when it is actually needed. I saw a large picture breakup in a certain scene that had a lot of action for about 3 seconds. If crf is saying I need high bitrate instantly but vbvmaxrate says no, then picture breakup seems to be normal. If I used 2pass then I assume the picture would be a lot cleaner at that point and probably not breakup. Do you agree? But I don't want to use 2pass, that's the problem.

Right now I'm stuck because I want 1pass with an average bitrate of about 3000kb/s. What I've been trying is changing crf until I get 3000kb/s, but sometimes crf has to be 24-26 which means losing quality. I guess what I want is 1pass with crf 20 with average bitrate of 3000kb/s. Using the above limitation of vbvmaxrate does this but with consequences of poor quality in certain scenes. If I try -nr maybe that will work. I think I'll give it a test.

The other funny thing I noticed recently was that Dean's uncropmkv program used -bitrate in the one pass mode, which is absolute least suggested mode, but no one complained about it at all as far as I can read. Probably because you couldn't see the command line, but using another app, you can see what command line was passed, which is what I did. Anyway there are so many combinations to use with x264, they are all pretty good.

I bet if we took a bunch of doom9 encoders here and showed them one clip, on two projectors, side by side, using different encoding options, very few could tell you which options were or were not used, especially in static scenes. In moving scenes it would be easier but very hard I think. It makes me wonder how much all this theory relates to practical viewing. Some will say a lot, some will say a little I suspect.

[Asmodian]

I saw a large picture breakup in a certain scene that had a lot of action for about 3 seconds. If crf is saying I need high bitrate instantly but vbvmaxrate says no, then picture breakup seems to be normal. If I used 2pass then I assume the picture would be a lot cleaner at that point and probably not breakup. Do you agree? But I don't want to use 2pass, that's the problem.

Two pass mode will not improve quality noticeably compared to crf at the same size and vbv settings. If you used two pass without VBV it would look like the 24-26 crf encode without VBV. Two pass just allows you to set average bit rate instead of quality. A low VBV bit rate is not the correct way to lower average bit rate, as you saw it just caps the peak bit rate. VBV is for streaming or slow media; it sets the minimum data rate needed to play the video without stalling.

Right now I'm stuck because I want 1pass with an average bitrate of about 3000kb/s. What I've been trying is changing crf until I get 3000kb/s, but sometimes crf has to be 24-26 which means losing quality. I guess what I want is 1pass with crf 20 with average bitrate of 3000kb/s. Using the above limitation of vbv max rate does this but with consequences of poor quality in certain scenes. If I try -nr maybe that will work. I think I'll give it a test.

Asking for a crf setting at a bit rate is an oxymoron. crf 20 sets quality independent of size while average bit rate sets size independent of quality. You cannot set both as quality = X * size and X is a constant for each video. If you need >3000kbps for decent quality you cannot get around it without using a slower preset (if you are using medium or slower going even slower probably will not change quality/size a lot). If you really need vbv max rate at 3000 you can increase the vbv buffer size to allow larger or longer bit rate spikes. This might help the bad scenes (if they are short enough) but the average bit rate will still be >3000kbps.

Your only option is to filter to improve compressability (change X); i.e. -nr or Avisynth.

I bet if we took a bunch of doom9 encoders here and showed them one clip, on two projectors, side by side, using different encoding options, very few could tell you which options were or were not used, especially in static scenes. In moving scenes it would be easier but very hard I think. It makes me wonder how much all this theory relates to practical viewing. Some will say a lot, some will say a little I suspect.

What are you trying to say here? As your post states a too low VBV is pretty obvious, using single pass --bitrate vs two pass is noticeable, the different --tune settings can be pretty different but 5 vs 8 B-frames is not obvious. It really depends on the option changed, the source, and etc. :confused:

[tyee]

Thanks Asmodian for that summary. I tried my clip using 2pass 3000kb/s and I was very happy with the results. I will try some more tests with crf also. Up until now I've always used "veryfast" preset because I only have old Q6600 processor. I do need to upgrade to the current models to decrease encoding time so I can use the slow presets. Personally my speed must be real time encoding, like 24fps or better, so that means new processor.