Low CPU usage on x64 bit system

[ben13]

I recently shifted to 64bit system- Intel quadcore, 4gb ram -

But whenever I'm using x264 or any other encoder, my CPU usage is around just 30-35%...

I use Megui to encode, so it automatically uses 64bit x264 version to encode, but still it's slow n uses less of my CPU!

So help me to solve this problem of "threading" I guess.
I used '0' first in # of threads,
then I set it to '6' (1.5*4( # of cores, they said))
Script I've been using:

LoadPlugin("C:\plugins\DGDecode.dll")
LoadPlugin("C:\plugins\ColorMatrix.dll")
LoadPlugin("C:\plugins\TIVTC.dll")
LoadPlugin("C:\plugins\UnDot.dll")
LoadPlugin("C:\plugins\Toon-v1.1.dll")
LoadPlugin("C:\plugins\aWarpSharp.dll")
LoadPlugin("C:\plugins\TIsophote.dll")
LoadPlugin("C:\plugins\EEDI2.dll")
Load_Stdcall_Plugin("C:\plugins\yadif.dll")
import("source.avs")
ColorMatrix(hints=true, threads=0)
LanczosResize(1280,720) # Lanczos (Sharp)
tfm(order=-1).tdecimate(mode=1,hybrid=3)
crop( 0, 0, 0, 0)
Yadif(order=-1)

separatefields().selecteven().EEDI2(field=1)
__film = last
__t0 = __film.trim(4620, 9110)
__t1 = __film.trim(11213, 34018)
__t2 = __film.trim(40612, 55866)
__t0 ++ __t1 ++ __t2
undot()



P.S- I'm a newbie in video encoding, mind that!:D

[LoRd_MuldeR]

What is your x264 command-line?

If you are running x264 in 2-Pass mode, then it's perfectly normal to see low CPU usage in the first pass.
That's because x264 automatically uses "fast" settings during the first pass of a 2-Pass encode. With such fast settings, the non-parallel parts of the encoder dominate.

Also you are using EEDI2(), which is a pretty slow interpolation filter. I'm not sure if it is multi-threaded, but NNED3 definitely is. So maybe try that one...

(In case it turns out that your are indeed bottle-necked by a slow single-threaded Avisynth script, you might be able to get some speed-up with the "MT" branch of Avisynth)

[ben13]

What is your x264 command-line?

If you are running x264 in 2-Pass mode, then it's perfectly normal to see low CPU usage in the first pass.
That's because x264 automatically uses "fast" settings during the first pass of a 2-Pass encode. With such fast settings, the non-parallel parts of the encoder dominate.

Also you are using EEDI2(), which is a pretty slow interpolation filter. I'm not sure if it is multi-threaded, but NNED3 definitely is. So maybe try that one...

hmm...

program --level 4.1 --preset slower --tune animation --pass 2 --bitrate 1749 --stats ".stats" --bframes 16 --b-adapt 1 --scenecut 43 --ref 5 --vbv-bufsize 50000 --vbv-maxrate 50000 --rc-lookahead 75 --aq-strength 0.8 --merange 24 --subme 10 --output "output" "input"

can't understand by command line thing! sorry!
but my first pass in x86, used to consume 90-95% of cpu with everything same!

[ben13]

using all the x64 plugins, x264, will solve the problem???

[Didée]

The Avisynth script runs single-threaded. Quite possible that the script is the bottleneck, speedwise. If Avisynth is using only one thread, then x264 might be sitting there, yawning, waiting for next frame, yawning, waiting for the next frame, yawning, ...

But apart from that: in any case,
LanczosResize(1280,720) # Lanczos (Sharp)
tfm(order=-1).tdecimate(mode=1,hybrid=3)
crop( 0, 0, 0, 0)
Yadif(order=-1)

separatefields().selecteven().EEDI2(field=1)
- this script is utter nonsense. (No offense intended, but it must be pointed out very clearly.)

You can't simply throw all filters in that are vaguely related to (de)interlacing, and hope that something good will emerge. Looks more like deliberate destruction. :)

[CruNcher]

I recently shifted to 64bit system- Intel quadcore, 4gb ram -

But whenever I'm using x264 or any other encoder, my CPU usage is around just 30-35%...

I use Megui to encode, so it automatically uses 64bit x264 version to encode, but still it's slow n uses less of my CPU!

So help me to solve this problem of "threading" I guess.
I used '0' first in # of threads,
then I set it to '6' (1.5*4( # of cores, they said))
Script I've been using:


P.S- I'm a newbie in video encoding, mind that!:D

LoadPlugin("C:\plugins\TIsophote.dll")

are you invoking that somewhere that one is slow highly efficient but not really optimized @ all (and the higher iterations the slower it becomes) you combining their a hard set of stuff :) like Didée said that in an very chaotic way which might go very of the road of your goal

[ben13]

But apart from that: in any case,
LanczosResize(1280,720) # Lanczos (Sharp)
tfm(order=-1).tdecimate(mode=1,hybrid=3)
crop( 0, 0, 0, 0)
Yadif(order=-1)

separatefields().selecteven().EEDI2(field=1)
- this script is utter nonsense. (No offense intended, but it must be pointed out very clearly.)

You can't simply throw all filters in that are vaguely related to (de)interlacing, and hope that something good will emerge. Looks more like deliberate destruction. :)

Then help me bro!!:confused:


ColorMatrix(hints=true, threads=0)
tfm(order=-1).tdecimate(mode=1,hybrid=3)
Yadif(order=-1)
separatefields().selecteven().EEDI2(field=1)
LanczosResize(1280,720) # Lanczos (Sharp)
crop( 0, 0, 0, 0)

any good??

p.s- I'm trying to encode a anime.ts!

[ben13]

LoadPlugin("C:\plugins\TIsophote.dll")

are you invoking that somewhere that one is slow highly efficient but not really optimized @ all

I'm sorry, I never intended that!

[ben13]

The Avisynth script runs single-threaded. Quite possible that the script is the bottleneck, speedwise. If Avisynth is using only one thread, then x264 might be sitting there, yawning, waiting for next frame, yawning, waiting for the next frame, yawning, ...



so should convert it to lossless(huffyv or something) using all those plugins? then encode that lossless in to my desired video quality??

[LoRd_MuldeR]

any good??

Any good for what?

Before you write an Avisynth script or ask for help/suggestions, you should be aware of what exactly you are trying to achieve.

Do not put arbitrary commands into your script, just because you saw somebody use them in some context...

so should convert it to lossless(huffyv or something) using all those plugins? then encode that lossless in to my desired video quality??

Nope. In most cases this is not needed/advisable.

It might make sense to create a lossless intermediate file, if you are using very slow pre-processing and you are going to encode it in 2-Pass mode.

[ben13]

well, with the code in the first post, say it code1,

n second re-modified code, say it code 2...

I encoded using both, 2nd one was a bit fast!
Snaps:http://www.mediafire.com/?vwxrp7ii1c3u5du


which one is better??

[LoRd_MuldeR]

The second one, I would say.

Still it doesn't make much sense to first apply a deinterlacing filter (Yadif in this case), just to throw away half of the lines afterwards and re-interpolate the missing lines once again with EEDI2.

If you want to combine Yadif with EEDI or NNEDI, you should probably use something like this instead:
yadifmod(mode=1, edeint=NNEDI3(field=-2))

Or you go for QTGMC directly:
http://forum.doom9.org/showthread.php?t=156028

[ben13]

The second one, I would say.
If you want to combine Yadif with EEDI or NNEDI, you should probably use something like this instead:
yadifmod(mode=1, edeint=NNEDI3(field=-2))


ColorMatrix(hints=true, threads=0)
tfm(order=-1).tdecimate(mode=1,hybrid=3)


LanczosResize(1280,720) # Lanczos (Sharp)
crop( 0, 0, 0, 0)
undot()

Thanks a ton,man!
Looks good to me that way, Even quite fast!
Snaps: http://www.mediafire.com/?2ugt553dw4c7z4x

but my x64 cpu usage is still low(35-45%), I tried the same settings on my dual core 32bit system, cpu usage was always above 90% during both passes.

is it because I'm using all 32bit version of plugins???

[Blue_MiSfit]

Your script is still nonsense.

You don't apply inverse telecine by using TFM and TDecimate, then follow that with YADIFMod+NNEDI! You do one or the other.

You're 100% certain you were using precisely the same source, script, and x264 settings on both machines?

Derek

[ben13]

Your script is still nonsense.

You don't apply inverse telecine by using TFM and TDecimate, then follow that with YADIFMod+NNEDI! You do one or the other.

Yeah, you're right, it's altering the framerate from 29.97 to 59.94!:(

[ben13]

You're 100% certain you were using precisely the same source, script, and x264 settings on both machines?


Yep, exact copy!

[LoRd_MuldeR]

Yeah, you're right, it's altering the framerate from 29.97 to 59.94!:(

That's the purpose of a bobbing deinterlacer, such as Yadif run in "bob" mode!

Still, if your source is Telecined (not "true" interlaced!) then TFM and TDecimate will restore the progressive frames and no deinterlacer will be needed afterwards ;)

But, if your source is "true" interlaced, then TFM/TDecimate does nothing (or nonsense). Only a deinterlacer can deal with "true" interlaced video...


So you would either use some variant of:
yadifmod(mode=1, edeint=NNEDI3(field=-2))

Or you would use some variant of:
tfm(order=-1).tdecimate(mode=1,hybrid=3)

But not both at the same time!

So again: first find out what exactly you need/want to do, then choose the appropriate filters.

Do not just throw in a bunch of random filters and hope that the resolute happens to be good by chance...

(Rule of thumb: The less filters you need to apply in order to fix your source, the better)

[ben13]

LanczosResize(1280,720) # Lanczos (Sharp)
tfm(order=-1).tdecimate(mode=1,hybrid=3)
crop( 0, 0, 0, 0)
Yadif(order=-1)

separatefields().selecteven().EEDI2(field=1)
- this script is utter nonsense. (No offense intended, but it must be pointed out very clearly.)

The Avisynth script runs single-threaded. Quite possible that the script is the bottleneck, speedwise. If Avisynth is using only one thread, then x264 might be sitting there, yawning, waiting for next frame, yawning, waiting for the next frame, yawning, ...



Any solution to keep x264 "from sitting there, yawning, waiting for next frame, yawning, waiting for the next frame, yawning,..."???


So again: first find out what exactly you need/want to do, then choose the appropriate filters.
I need the best quality!!

After I removed,
yadifmod(mode=1, edeint=NNEDI3(field=-2))
quality is a bit dull!

Maybe I need a strong deinterlacer capable to keep the frames progressive!

[LoRd_MuldeR]

Any solution to keep x264 "from sitting there, yawning, waiting for next frame, yawning, waiting for the next frame, yawning,..."???

Don't use an input script that delivers the frames slower than x264 encodes. It's as simple as that ;)

(This means: Either make your script faster, preferably by kicking out superfluous filters, or use a slower x264 preset)

I need the best quality!!

Then putting a bunch of deinterlace/IVTC filters in a chain certainly isn't what you want...

After I removed,
yadifmod(mode=1, edeint=NNEDI3(field=-2))
quality is a bit dull!

You still didn't tell us what the nature of your source is (interlaced or telecined) or what exactly you are trying to achieve with all these filters.

So it seems you are just throwing in (or removing) random filters, hoping that this will 'somehow' improve things. But I doubt you'll ever get a very good result this way.

Instead I would suggest you remove ALL filters and then add only those that you actually need. For example one deinterlace filter or one IVTC filter.

Maybe I need a strong deinterlacer capable to keep the frames progressive!

This doesn't make much sense.

Every deinterlacer takes interlaced frames (fields) as input and return progressive frames as output.

If the input already is progressive, there is no need to apply a deinterlacer. Doing so anyway, will only destroy quality for zero benefit!

Note that after applying an IVTC filter on telecined video (given the video really is telecined) you do have progressive frames...

[ben13]

nice! i got that!


LoadPlugin("C:\plugins\DGDecode.dll")
DGDecode_mpeg2source("D:\OP 498 ts with script\onepiece-498_tss.d2v", info=3)

LoadPlugin("C:\plugins\ColorMatrix.dll")
ColorMatrix(hints=true, threads=0)

LoadPlugin("C:\plugins\TIVTC.dll")
tfm(order=-1).tdecimate(mode=1,hybrid=3)


__film = last
__t0 = __film.trim(4620, 9110)
__t1 = __film.trim(11213, 34018)
__t2 = __film.trim(40612, 55866)
__t0 ++ __t1 ++ __t2

LanczosResize(1280,720) # Lanczos (Sharp)

LoadPlugin("C:\plugins\UnDot.dll")
undot()


snaps: http://www.mediafire.com/?az6s8o901owlps4

I use this thing, less filters than before, source is telecined!
This script is quite fast, but Now I'm getting a kind of blurred picture quality, n the edges are not strong/dark!
I tried using toon n awarpsharp for that,

Toon(0.40)


but their quality is horrible if I use more than (0.40), plus blurring effect still remains!

X264 preset, I'm using is:

cabac=1 / ref=5 / deblock=1:1:1 / analyse=0x3:0x133 / me=umh / subme=10 / psy=1 / psy_rd=0.40:0.00 / mixed_ref=1 / me_range=24 / chroma_me=1 / trellis=2 / 8x8dct=1 / cqm=0 / deadzone=21,11 / fast_pskip=1 / chroma_qp_offset=-2 / threads=3 / sliced_threads=0 / nr=0 / decimate=1 / interlaced=0 / constrained_intra=0 / bframes=16 / b_pyramid=2 / b_adapt=1 / b_bias=0 / direct=3 / weightb=1 / weightp=2 / keyint=250 / keyint_min=25 / scenecut=43 / intra_refresh=0 / rc_lookahead=75 / rc=crf / mbtree=1 / crf=20.2 / qcomp=0.60 / qpmin=10 / qpmax=51 / qpstep=4 / vbv_maxrate=50000 / vbv_bufsize=50000 / crf_max=0.0 / ip_ratio=1.40 / aq=1:0.80 / nal_hrd=none

[TheRyuu]

...source is telecined!

So why do you need this:
separatefields().selecteven().EEDI2(field=1)

And you're wondering why
Now I'm getting a kind of blurred picture quality, n the edges are not strong/dark!

Also is the source really hybrid?

[ben13]

So why do you need this:
separatefields().selecteven().EEDI2(field=1)


Sorry! Mistake on my part, I didn't used that!

Also is the source really hybrid?

Yeah, it is. It's NTSC 29.97fps vfr telecined hybrid video.
I want to make a true vfr out of this.

[LoRd_MuldeR]

Avisynth can't do VFR.

[sneaker_ger]

I don't know if it's still state-of-the-art, but you might want to take a look at AnimeIVTC (http://forum.doom9.org/showthread.php?t=138305) for creating VFR out of hybrid sources.

[ben13]

I don't know if it's still state-of-the-art, but you might want to take a look at AnimeIVTC (http://forum.doom9.org/showthread.php?t=138305) for creating VFR out of hybrid sources.

didn't worked quite well!
I mean I couldn't put it to work like the way I want!

[ben13]

Avisynth can't do VFR.

it can't? then how do others do it?

[LoRd_MuldeR]

it can't? then how do others do it?

I don't know. But not with Avisynth, as Avisynth assumes CBR ;)

Well, when editing a source that already is VFR, you might be able to get around this limitation of Avisynth by using a "timecode" file.
But this will limit the operations you can do in Avisynth!
Any operation that changes the number of frames (like IVTC/BobDeint) or "moves" frames around will invalidate the timecode file...

[sneaker_ger]

AnimeIVTC, YATTA etc. work around that limitation by creating a timecode file for the encoder/muxer.

[ben13]

So if I use AnimeIVTC, then I won't need "tfm(order=-1).tdecimate(mode=1,hybrid=3)", rite??