RipBot264 v1.18.3 - Simple and easy to use GUI -> IPOD . PSP . CONSOLES . BLURAY

[guest]

Hi all
is there a command line option to set the number of encoding threads for EncodingServer?

Do you mean something like these options :-

https://i.imgur.com/mLmikgC.jpg

Click on the "spanner" icon to get to this menu

[rlev11]

Hi all
is there a command line option to set the number of encoding threads for EncodingServer?

To directly answer your question, YES
encodingserver /restart-if-no-progress /priority normal /affinity 3FFC3FFC

The affinity uses a hex number to enable/disable specific cores. You can use a site like (bitsum.com/tools/cpu-affinity-calculator) to get the hex number you desire.

If you are asking this to knock down say a 16 core ryzen that falls off a cliff when doing 4k encoding, you can use the above affinity switch or I have found that the following command lines works actually a tiny bit better. This also doesn't change regular 1080 and 720 encoding speeds.

encodingserver /priority normal /restart-if-no-progress /avisynth-prefetch-threads 12 /x264-threads 16 /x265-threads 16

The avisynth-prefetch-threads is what actually kills the 16 core ryzens when doing 4k encoding for whatever reason if you do not knock them down to 12

This works best regardless of what you are encoding and is what I use on all my 16 core ryzens

[Ryushin]

Yes, that's true I could setup a dual boot system. But this did not solve the initial problem!
Now it is going slightly off-topic:

[off-topic] People complain: I can't use Linux! There is no e.g. Photoshop available. I still need to use Windows. If there was a Linux Photoshop version I could switch to Linux. Adobe says: There is no need of a Photoshop Linux version because only 1 or 2% of the users are using Linux, development for Linux is not worth it. It's like the hen and egg problem. I switched to Linux 3 years ago privately. First with dual boot option Windows/Linux and since last year to Linux only as main OS. I am happy with it and I found replacement apps for my used Windows applications. But there are a few apps where no Linux version is available. There is a little video toolset I want to use with Linux. Most binaries in this toolset are also available for Linux. The toolset uses DetectBorders.exe and some Windows Batch Scripts to call the Windows binaries. I need a native Linux binary so I can use DetectBorders from the RipBot project with some shell scripts on Linux. So I will take the chance and try to port the DetectBorders Windows Tool to Linux. It's a start and more Windows only applications will be available for additional platforms Some people here in the forums said earlier that they can't use the toolset on their Linux systems because it's Windows only at the moment. I think they would also be happy if there is a Linux version available later. :) [/off-topic]

Sorry, for the long explanation and off-topic but I had a reason to ask for a native Linux version... If not, I wouldn't have asked for it...

Kudos for attempting to port it. I've been driving Linux as my main desktop since '97. Been using the same Debian install from Potato until today on my Thinkpad P17 Gen2. My main RB install is running in a VM on my Linux Server. I use Linux for just about everything and for the most part there is *nix software for everything I want to do. I also have a soft spot for FreeBSD. I also know Solaris quite well, though I don't care for it I has to dump half of GNU on it to make it usable. LOL

All my physical servers run ZFS as a file system including my laptop (has three M.2 NVMEs). I even take care of large installs of ZFS over 2PB with 232 hard drives and 100TB of NVME Cache (L2ARC). ZFS is almost unbelievable in what it can do.

So yea, I belong to the cult of the Penguin. :D

[guest]

I stumbled across something today, and for those of you that are lucky enough to own one on the AWESOME CPU's.

Add this command to your x265 profile:- --asm avx512

Too bad for you 13th Gen Intel owners :angry:

[JThomasG]

Hi,

quick question, im using an intel i9-12900KF,

will i see a performance benefit if i use the prefetch option below.

looking at cpu performance all cores seems to works at mostly 100%

To directly answer your question, YES
encodingserver /restart-if-no-progress /priority normal /affinity 3FFC3FFC

The affinity uses a hex number to enable/disable specific cores. You can use a site like (bitsum.com/tools/cpu-affinity-calculator) to get the hex number you desire.

If you are asking this to knock down say a 16 core ryzen that falls off a cliff when doing 4k encoding, you can use the above affinity switch or I have found that the following command lines works actually a tiny bit better. This also doesn't change regular 1080 and 720 encoding speeds.

encodingserver /priority normal /restart-if-no-progress /avisynth-prefetch-threads 12 /x264-threads 16 /x265-threads 16

The avisynth-prefetch-threads is what actually kills the 16 core ryzens when doing 4k encoding for whatever reason if you do not knock them down to 12

This works best regardless of what you are encoding and is
what I use on all my 16 core ryzens

[Ryushin]

I stumbled across something today, and for those of you that are lucky enough to own one on the AWESOME CPU's.

Add this command to your x265 profile:- --asm avx512

Too bad for you 13th Gen Intel owners :angry:

Now I'll have to do more testing. :)
Found this:
https://www.hwcooling.net/en/intel-avx-512-tested-in-x265-how-to-enable-it-and-does-it-help/

Not sure if the power consumption is going to be worth slightly increased performance. AMDs implementation is double pumped compared to Intels 12th gen. I'm running a test now against a entire 4K movie (Puss in Boots) so I'll report back with the times to encode both with or without.

Unfortunately, I won't be able to use it for my set up as one of my two servers does not have AVX-512 and the X265 options is added to the main x265 settings. But perhaps x265 will just ignore the option if it's not available to use. I'll test that too.

[Ryushin]

Hi,

quick question, im using an intel i9-12900KF,

will i see a performance benefit if i use the prefetch option below.

looking at cpu performance all cores seems to works at mostly 100%

Those are the results of the tests I ran for the 7950x which has 16 cores. Even though you will see 100% usage, it does not mean it is encoding as it's best efficiency. You might have to make a sample and run some tests. What I found is that you will want to set your avisynth-prefetch-threads to about 3/4 of that your x265 threads would be. Otherwise what I noticed is you will have threads waiting for other threads to be finished before they can continue.

What I've discovered is that you will want to see is your CPU usage bouncing between 85-100%.

So leave off the "/x264-threads 16 /x265-threads 16" and see what your Encoding Server shows as the default threads. Then just set the avisynth-prefetch-threads to about 3/4 of what it shows.

[rlev11]

The only time in my experience that you have to worry about knocking down cpu cores or avisynth threads is when you start getting into doing 4k encoding and specifically 4k encodes that are full screen aspect ratio (1.78:1 or even 1.85:1). There is something with encoding all those extra bits that I have found that the 16 core Ryzens really have an issue with those. I can't speak to the latest Intel i-9's as I don't have any of those. Not sure how the whole P and E cores effect encoding 4k, you may have to just do some testing.

When doing those 4k encodes with everything set full out on a 3950/5950/7950, The encoding speed takes a nose-dive to point where they are about 1/2 the speed of a 12 core 3900x or 5900x. Something gets overloaded with all those threads and they just can't handle it. This is where using either affinity mask or avi-synth thread setting knocking them down to only use 12 cores or threads is necessary. Even with cores or threads turned off, they will still out-perform their 12 core brothers.

I posted some quick tests a while ago that showed me that once I set the avisynth prefetch to 12, that it did not slow down or speed up doing regular 1080 or 720 encoding, but made the 4k stuff run as they should which is why I just use that setting now for everything.

[guest]

Now I'll have to do more testing. :)
Found this:
https://www.hwcooling.net/en/intel-avx-512-tested-in-x265-how-to-enable-it-and-does-it-help/

I found that site, yesterday :

Unfortunately, this has old info, and it only shows 11th Gen Intel & 5000 Series Ryzen's....it clearly shows that the Ryzen's DO NOT support avx 512 but the the Intel chips do, with a small increase in speed.

However, 12th & 13th Gen Intel chips with their "E" & "P" cores do not. although it has been suggested that with some motherboards, there is an option to enable it (I haven't got a motherboard that has that), also it has been suggested that turning off the "E" cores with enable avx 512 support, again, I haven't got that option, either :(

Not sure if the power consumption is going to be worth slightly increased performance. AMDs implementation is double pumped compared to Intels 12th gen. I'm running a test now against a entire 4K movie (Puss in Boots) so I'll report back with the times to encode both with or without.

Unfortunately, I won't be able to use it for my set up as one of my two servers does not have AVX-512 and the X265 options is added to the main x265 settings. But perhaps x265 will just ignore the option if it's not available to use. I'll test that too.

On CPU's that don't support avx 512, the x265 encoder will simply show a message...[warning]: AVX512 is not supported, and then continue's.

So I did a lot of frustrating testing yesterday, and there doesn't appear to be that much benefit, but it's nice be able to enable it, and it's just another feather in AMD's cap.

TBBH, I'm not all that impressed with the 13900KF, it run's as hot as a furnace, and is slower than the 7950X !!!!! (which is also a "cooler" furnace)
I am running a "budget" B660 MSI motherboard, but the "grunt" of the 13900KF should still be fully enabled, I think spending another couple of $100's wouldn't make much difference, and overclocking would just create a LOT more unwanted heat !!!

I will be interested in your test results :)

[guest]

Hi,

quick question, im using an intel i9-12900KF,

will i see a performance benefit if i use the prefetch option below.

looking at cpu performance all cores seems to works at mostly 100%

Hi JThomasG,

In my limited experience with my new 13900KF, this prefetch option doesn't seem to be as "important" with the Intel chips, as it is with the 16 core Ryzen's.

I did some tests using 3 different command line's, and there was little difference,

encodingserver /priority normal /restart-if-no-progress /avisynth-prefetch-threads 12 /x264-threads 16 /x265-threads 16

encodingserver /priority normal /restart-if-no-progress /avisynth-prefetch-threads 14 /x264-threads 16 /x265-threads 16

encodingserver /priority normal /restart-if-no-progress /avisynth-prefetch-threads 16 /x264-threads 16 /x265-threads 16

However, I did notice that if these commands are not included, the speed drops noticeably :(

As rlev11 mentioned, it does also depend on the "size" of the video being encoded, and of course what filters are being used. specifically 4k encodes that are full screen aspect ratio (1.78:1 or even 1.85:1)

Good luck..

[LigH]

Add this command to your x265 profile:- --asm avx512

:o

Guys, there is a good reason why this option has to be enabled manually and is not enabled when its CPU support is autodetected:

There is no certain advantage in it. Not as much between AVX2 and AVX512 as there is between AVX and AVX2.

It may execute some calculations faster in net CPU time. But gross CPU time can still be worse, because switching between a more basic CPU mode and the AVX512 context requires clearing more CPU registers which must be loaded again, and the power consumption is so much higher that the temperature rises much faster so that the CPU must be cooled with more efforts, up to throttling down the CPU clock. This may also worsen the durability of the whole CPU.

I don't want to discourage you from the start. Yes, try it. But take all the side effects into account.

[guest]

:o

Guys, there is a good reason why this option has to be enabled manually and is not enabled when its CPU support is autodetected:

There is no certain advantage in it. Not as much between AVX2 and AVX512 as there is between AVX and AVX2.

It may execute some calculations faster in net CPU time. But gross CPU time can still be worse, because switching between a more basic CPU mode and the AVX512 context requires clearing more CPU registers which must be loaded again, and the power consumption is so much higher that the temperature rises much faster so that the CPU must be cooled with more efforts, up to throttling down the CPU clock. This may also worsen the durability of the whole CPU.

I don't want to discourage you from the start. Yes, try it. But take all the side effects into account.

https://forum.doom9.org/showthread.php?p=1981275#post1981275

[Ryushin]

Now I'll have to do more testing. :)

Not sure if the power consumption is going to be worth slightly increased performance. AMDs implementation is double pumped compared to Intels 12th gen. I'm running a test now against a entire 4K movie (Puss in Boots) so I'll report back with the times to encode both with or without.


So I finished testing the AVX-512 testing and I also tested against underclocking the 7950x.

What I found, as others have found, there was no improvement in enabling AVX-512. In fact it was ever so slightly slower when using AVX-512.

Underclocking the 7950x resulted in a nice discovery. I have my 7950x underlocked to all cores at 4900MHz using 1.08 volts.
So the 7950x at the default frequencies took 1:24m:28s to run the encode while averaging 210 watts of energy. The 7950x underclocked took 1h:26m:58s while averaging 140 watts of energy. So underclocked it was 2.3% slower than the default frequencies and the 7950x uses 50% more electricity at the default frequencies. Pretty amazing how well the 7950x performs when underclocked slightly. So yea, the 7950x is not so much a furnace if you underclock it.


Testing Detail:
AVX-512 X265 Instruction set test. AMD 7950x with default frequencies.
Source: Puss In Boots 4K - 1h:30m:35s
AIO 420mm Cooler
CQ16

Without AVX-512:
Time to encode: 1:24m:28s
Avg Temp: 85 Celsius
Avg Power Package Watts: 210
Avg Power Cores Watts: 145

With AVX-512:
Tiem to Encode: 1h:25m:44s
Avg Temp: 89 Celsius

Underclocked to 4.9GHz (1.08v)
Without AVX-512
Time to encode: 1h:26m:58s
Avg Temp: 60 Celsius
Avg Power Package Watts: 140
Avg Power Cores Watts: 85

[rlev11]

Those are great numbers on under-clocking the 7950x. 2.3% performance drop is basically nothing for the huge power savings. This is something I am going to have to seriously consider.

I will probably also look into and see if doing the same with the 5950x results in a similar small performance hit with a big power savings as well.

Just did a quick test with my 7950x. All I did was go into Ryzen Master and in the Basic View changed the profile from Default to Eco Mode. This knocks the 7950x down to 65watt maximum. My CPU temp went from 85c to 63c. CPU Power via Ryzen Master showed a drop from 140w down to appx 55watt. Clock Speeds dropped about a 1,000 Mhz. FPS showed in the Encoding Client in default mode was about 26-27 fps on a 1080p encode. After ECO Mode the fps only dropped 1.5 to 3fps. next up will put a Kill-a-watt meter on at the wall and see what real world power drop is.

Looks like on the 7950x you can change the mode on the fly in Ryzen Master. No reboot required, just click ECO Mode and hit apply.

[guest]

So I finished testing the AVX-512 testing and I also tested against underclocking the 7950x.

Well, first of all, maybe I should offer a bit of an apology for introducing the AVX512 option, I guess I was "sucked in" by the hype that AVX2 & AVX512 was a bit of a game changer, but it's clearly hyped BS !!

I have also tried supposed AVX2 optimised x265's, and there's no significant improvement.

It might be advantageous in some scenarios, but not this.

It's also a shame that some of us spend our hard-earned money on "top of the line" hardware (eg:- 7950X's, and 13900K's), only to find they use SO much power, and produce SO much heat, that "we" go to great lengths to "hobble" them, to make them more acceptable...it's almost pointless, really.

But having said that, is underclocking the way to go, or is undervolting & overclocking the new thing ??

I was watching a YT clip of JayzTwoCents where he undervolted an Intel CPU, and was also able to overclock it, so more speed, less volts, less power use, less heat.

https://www.youtube.com/watch?v=4jjopjkJzxA

And a couple of other clips on undervolting 7950X's..

https://www.youtube.com/watch?v=1pizvaYiVbk

https://www.youtube.com/watch?v=7JiYAwKIHRY

And now back to your tests:-

Those encoding time's for "Puss In Boot's", that's encoding the movie in less time that it takes to watch it !!!

Are you using any filters, or is it only CQ 16 ??

And as for power usage, that fortunately isn't that much of an issue for me, as I only encode when the Solar Panels are generating enough, and I don't do any "overnight" encoding, been there, done that, was very disappointing, most of the time :(

[guest]

Those are great numbers on under-clocking the 7950x. 2.3% performance drop is basically nothing for the huge power savings. This is something I am going to have to seriously consider.

I will probably also look into and see if doing the same with the 5950x results in a similar small performance hit with a big power savings as well.

Just did a quick test with my 7950x. All I did was go into Ryzen Master and in the Basic View changed the profile from Default to Eco Mode. This knocks the 7950x down to 65watt maximum. My CPU temp went from 85c to 63c. CPU Power via Ryzen Master showed a drop from 140w down to appx 55watt. Clock Speeds dropped about a 1,000 Mhz. FPS showed in the Encoding Client in default mode was about 26-27 fps on a 1080p encode. After ECO Mode the fps only dropped 1.5 to 3fps. next up will put a Kill-a-watt meter on at the wall and see what real world power drop is.

Looks like on the 7950x you can change the mode on the fly in Ryzen Master. No reboot required, just click ECO Mode and hit apply.

That's all very interesting as well, I think I've tried a few of these settings, but it's just killing the powerhouse that the 7950X is.

You'll probably see a much bigger decrease when encoding 4K, with strong filters :(

I've gone back to some pretty basic BIOS settings, a small negative curse optimiser, and left almost everything else as default, it's running at approx 95- 96°C, on a hot day (but it doesn't seem to matter what the ambient is), and it's around 5100 - 5400MHz, when encoding 4K.

[rlev11]

Yea I'm not completely sold on ECO Mode. It was just a "Let's see what happens" and I was really surprised that the performance was as good as it was. It something I might enable on the fly if I que up a bunch of stuff to do overnight and don't really care when it gets done.

[Ryushin]

It's also a shame that some of us spend our hard-earned money on "top of the line" hardware (eg:- 7950X's, and 13900K's), only to find they use SO much power, and produce SO much heat, that "we" go to great lengths to "hobble" them, to make them more acceptable...it's almost pointless, really.

Those encoding time's for "Puss In Boot's", that's encoding the movie in less time that it takes to watch it !!!

Are you using any filters, or is it only CQ 16 ??

And as for power usage, that fortunately isn't that much of an issue for me, as I only encode when the Solar Panels are generating enough, and I don't do any "overnight" encoding, been there, done that, was very disappointing, most of the time :(

Not using any filters for those tests, just CQ16. I can run the same tests but I think the results will be similar.

I think both AMD and Intel are chasing every last bit of performance they can to one up each other. The amount of heat/watts consumed by the new Intel 13900K is staggering (yea, that is a furnace):
https://www.youtube.com/watch?v=P40gp_DJk5E&t=364s
The 13900K seems very bursty in the way it was designed and it throttles back in only a few seconds. The 7950x seems to go to it's limit and hold there. I'm curious, since you have both CPUs, you should run a test and let us know which CPU wins (regardless of power used). I think you'll need to disable the GPU portion though so it runs all CPU.

So even though the chips can eek out more performance by pushing them to their max, I don't think for real world use it's necessary. Even for us that torture the CPUs for encoding, underlocking them ever so slightly produces great performance. And if I want to get 2.3% more performance for 50% more power I can. :)

[guest]

Not using any filters for those tests, just CQ16. I can run the same tests but I think the results will be similar.

I think both AMD and Intel are chasing every last bit of performance they can to one up each other. The amount of heat/watts consumed by the new Intel 13900K is staggering (yea, that is a furnace):
https://www.youtube.com/watch?v=P40gp_DJk5E&t=364s
The 13900K seems very bursty in the way it was designed and it throttles back in only a few seconds. The 7950x seems to go to it's limit and hold there. I'm curious, since you have both CPUs, you should run a test and let us know which CPU wins (regardless of power used). I think you'll need to disable the GPU portion though so it runs all CPU.

So even though the chips can eek out more performance by pushing them to their max, I don't think for real world use it's necessary. Even for us that torture the CPUs for encoding, underlocking them ever so slightly produces great performance. And if I want to get 2.3% more performance for 50% more power I can. :)

OK, well those times make more sense, now that you've confirmed your settings.

I wonder if when using a filter, you might get different results ??

The 13900KF, is the non GPU variant :cool:

I'm going to try an older x265 build that's supposed to be optimised for AMD & Intel chips...not sure if it's going to yield better results.

And you're correct, that the Ryzen seems to get to it's limit, and stay's there, and the 13900 does indeed throttle back and forth, sadly.

But the Ryzen has the advantage, most definitely.

[Atak_Snajpera]

The only time in my experience that you have to worry about knocking down cpu cores or avisynth threads is when you start getting into doing 4k encoding and specifically 4k encodes that are full screen aspect ratio (1.78:1 or even 1.85:1). There is something with encoding all those extra bits that I have found that the 16 core Ryzens really have an issue with those. I can't speak to the latest Intel i-9's as I don't have any of those. Not sure how the whole P and E cores effect encoding 4k, you may have to just do some testing.

When doing those 4k encodes with everything set full out on a 3950/5950/7950, The encoding speed takes a nose-dive to point where they are about 1/2 the speed of a 12 core 3900x or 5900x. Something gets overloaded with all those threads and they just can't handle it. This is where using either affinity mask or avi-synth thread setting knocking them down to only use 12 cores or threads is necessary. Even with cores or threads turned off, they will still out-perform their 12 core brothers.

I posted some quick tests a while ago that showed me that once I set the avisynth prefetch to 12, that it did not slow down or speed up doing regular 1080 or 720 encoding, but made the 4k stuff run as they should which is why I just use that setting now for everything.

I wonder what happens if you downgrade your 7950 to 7700 by disabling one CCD in BIOS. Will you still experience the same problem with 4k working terrible with 8 threads (or 16)?

[guest]

I wonder what happens if you downgrade your 7950 to 7700 by disabling one CCD in BIOS. Will you still experience the same problem with 4k working terrible with 8 threads (or 16)?

That is an unusual suggestion, it's cutting the CPU in half :scared:

If it turned out similar to do what disabling all the "E" Cores on a 13900KF, then it's just SLOW !!

We buy these CPU's for the power, not to "cut them in half" !!

As rlev11, Ryushin & myself have determined, the sweet spot for the 7950X, is this (with about 95% of encodes) :-

/avisynth-prefetch-threads 12 /x264-threads 16 /x265-threads 16

As for the 13900KF, still not too sure what it likes, I might need to try the "Affinity" command line.

I am going to do some tests with an older x265 build, that is supposed to be optimised for different CPU's, so I will post some results, soon

[guest]

These results are from Atak's x265 Benchmark app :-

With updated x265 builds.

with default included x265 - encoded 2500 frames in 20.75s (120.51 fps), 7025.74 kbps, Avg QP:37.21

x265 r74 - encoded 2500 frames in 19.89s (125.71 fps), 10890.80 kbps, Avg QP:35.84

x265 r83 - encoded 2500 frames in 19.83s (126.07 fps), 10890.80 kbps, Avg QP:35.84

x265 r85 - encoded 2500 frames in 19.72s (126.76 fps), 10890.80 kbps, Avg QP:35.84

x265 v3.5+67-aMod-gcc12.2.1 - encoded 2500 frames in 19.77s (126.46 fps), 10891.97 kbps, Avg QP:35.84

x265-x64-v3.5+67-aMod-gcc12.2.1 failed


x265-3.5+84-5d8f209_vs2022 - encoded 2500 frames in 19.78s (126.39 fps), 10890.80 kbps, Avg QP:35.84

x265-3.5+84-5d8f209_vs2022-AVX2 - encoded 2500 frames in 19.84s (126.01 fps), 10890.80 kbps, Avg QP:35.84

x265-3.5+84-5d8f209_gcc122-AVX2 - encoded 2500 frames in 19.67s (127.07 fps), 10891.97 kbps, Avg QP:35.84

x265-3.5+84-5d8f209_gcc122 - encoded 2500 frames in 19.62s (127.40 fps), 10890.80 kbps, Avg QP:35.84 fastest

Patman builds failed this test :(

------------------------------------------
I have removed most of this post, as it was no longer relevant, after today's testing (21-01-23) :(

[Ryushin]

OK, well those times make more sense, now that you've confirmed your settings.

I wonder if when using a filter, you might get different results ??


Ran the tests using SMDegrain Hard. The underclocked took three minutes longer, which was 1.4% slower. So I'll take that for 50% of the power savings.

Power Tests:
Source: Puss In Boots 4K - 1h:30m:35s
AIO 420mm Cooler
CQ16
SMDegrain Hard: SMDegrain(video,tr=8,thSAD=800,thSADC=400,contrasharp=true,prefilter=2,refinemotion=true)

7950x Default Frequencies: 3h:26m:08s
7950x Underclocked 4.9GHz at 1.08v: 3h:29m:50s

[rlev11]

I wonder what happens if you downgrade your 7950 to 7700 by disabling one CCD in BIOS. Will you still experience the same problem with 4k working terrible with 8 threads (or 16)?

I sure wouldn't want to do that as part of my regular workflow, but sometime this weekend I'll give it a try and see what happens for a test. Who knows, the results may be surprising.

It would be nice if we all could figure out exactly what is the hangup when doing 4k once it appears you start going over 12 cores. There must be some program buffer that just gets overloaded with all that extra information contained in each frame

[Atak_Snajpera]

I sure wouldn't want to do that as part of my regular workflow, but sometime this weekend I'll give it a try and see what happens for a test. Who knows, the results may be surprising.

It would be nice if we all could figure out exactly what is the hangup when doing 4k once it appears you start going over 12 cores. There must be some program buffer that just gets overloaded with all that extra information contained in each frame

Who knows maybe communication between CCDs is the problem in 7950? That's why I asked you if you could check if single CCD also shows this behavior. Intel despite having two different CPU architectures under one roof is immune to excess number of threads.

[guest]

Who knows maybe communication between CCDs is the problem in 7950? That's why I asked you if you could check if single CCD also shows this behavior. Intel despite having two different CPU architectures under one roof is immune to excess number of threads.

I thought I'd try Atak's suggestion, I am yet to run a "proper" encode, but in the mean time, I ran his x265 benchmark app, and comparing it to here :-

https://forum.doom9.org/showthread.php?p=1980881#post1980881

I think you will see a HUGE difference :(

AMD Ryzen 9 7950X 16-Core @ 4.5GHz ( 8C / 16T )

y4m [info]: 1920x1080 fps 50/1 i420p8 sar 1:1 unknown frame count
raw [info]: output file: NUL
x265 [info]: HEVC encoder version 3.5+85-c2e8e8d13
x265 [info]: build info [Windows][GCC 12.2.0][64 bit] 8bit+10bit+12bit
x265 [info]: using cpu capabilities: MMX2 SSE2Fast LZCNT SSSE3 SSE4.2 AVX FMA3 BMI2 AVX2
x265 [info]: Main Still Picture profile, Level-4.1 (Main tier)
x265 [info]: Thread pool created using 16 threads
x265 [info]: Slices : 1
x265 [info]: frame threads / pool features : 4 / wpp(17 rows)
x265 [info]: Coding QT: max CU size, min CU size : 64 / 8
x265 [info]: Residual QT: max TU size, max depth : 32 / 1 inter / 1 intra
x265 [info]: ME / range / subpel / merge : hex / 57 / 2 / 3
x265 [info]: Keyframe min / max / scenecut / bias : 50 / 500 / 40 / 5.00
x265 [info]: Lookahead / bframes / badapt : 20 / 4 / 2
x265 [info]: b-pyramid / weightp / weightb : 1 / 1 / 0
x265 [info]: References / ref-limit cu / depth : 3 / off / on
x265 [info]: AQ: mode / str / qg-size / cu-tree : 2 / 1.0 / 32 / 1
x265 [info]: Rate Control / qCompress : CRF-28.0 / 0.60
x265 [info]: tools: rd=3 psy-rd=2.00 early-skip rskip mode=1 signhide tmvp
x265 [info]: tools: b-intra strong-intra-smoothing lslices=6 deblock sao

encoded 2500 frames in 35.68s (70.07 fps), 10890.80 kbps, Avg QP:35.84

I will run a short encode, and post those results. Not looking good, tho :(

OK, have run a test (same as yesterday, above), and using this :-


Tested a 5 minute 4K clip, using 1 x 6 minute chunk size, CQ 16, no filters !!

Time taken FPS

x265 v3.5+85.1 Mod by Patman clang-AVX2 with AVX512 9:30 12.66 (yesterday, using this command-- /avisynth-prefetch-threads 12 /x264-threads 16 /x265-threads 16)

x265 v3.5+85.1 Mod by Patman clang-AVX2 with AVX512 13:14 9.08 (today, with ONLY half the 7950X enabled, using this command-- /avisynth-prefetch-threads 12 /x264-threads 16 /x265-threads 16)

and last but not least, with the latest x265 build, released yesterday:-

x265 v3.5+87 with AVX512 13:50 8.69 (today, with ONLY half the 7950X enabled, using this command-- /avisynth-prefetch-threads 12 /x264-threads 16 /x265-threads 16)

Oh, and btw, the CPU was running in the high 80°C's, even with half disabled.

So the next step is doing this on the 13900KF...

[rlev11]

So I ran a whole slew of tests to try to answer Atak's question about turning the 7950x into a 7900x in the bios and see what happens when doing 4k encoding. I also did some runs doing a 1080p encode for comparisons to try to figure out if there was one setting that worked best for both.
For the 4k encode, I did Animal House starting 30 minutes in and just used the ending fps of the first chunk for each encoding server. Ran the same smdegrain heavy grain script. This way for each run, the same encoding server did the same chunk each time for control purposes. As a reference, a 12 core 5900x averaged 5.28 fps for all the runs.

First up is running the 7950x and 5950x servers with no extra settings, so they were running full out max cores and threads.
7950x single server - 4.5 fps
5950x single server - 3.95 fps
5950x dual svr - 6.52 fps combined
7950x with CCD1 disabled in bios, so running as a 7700x - 4.97fps, so in essence a 7700x is 10 percent faster that a 7950x

Next up was running the 7950x and 5950x with: /avisynth-prefetch-threads 12 /x264-threads 16 /x265-threads 16
7950x Dual svr - 8.27 fps combined
7950x single server - 6.95 fps
5950x single server - 5.52 fps
5950x dual svr - 6.89 fps combined

Next up was running the 7950x and 5950x with: /affinity 3FFC3FFC for all encoding servers, essentially running them as 12 core cpus
7950x Dual svr - 8.41 fps combined
7950x single server - 7.06 fps
5950x single server - 5.39 fps
5950x dual svr - 6.99 fps combined

Then I thought lets run the 5950x and 7950x as 2 separate 8 core machines using an affinity mask of FFFF0000 on 1 server and 0000FFFF on the other. So a dual 7700x and a dual 5700x
7950x Dual svr - 9.73 fps combined
5950x dual svr - 7.14 fps combined

Finally I tried on the 7950x running as two 7900x like above, but running 2 server with each affinity mask, so 4 total servers running
7950x quad server - 9.16 fps combined

So this is great, running especially the 7950x as 2 separate 7900x showed a significant improvement, but how does that work when doing 1080p stuff. So I re-ran the tests using where eagles dare using the same heavy smdegrain script as above. the 5900x averaged out at 19.75 fps through all the runs.

First up is running the 7950x and 5950x servers with no extra settings, so they were running full out max cores and threads.
7950x dual svr - 35.71 fps
5950x single server - 19.43 fps
5950x dual svr - 29.24 fps combined

Then running the prefetch 12 setting
7950x dual svr - 35.48 fps
5950x dual svr - 29.11 fps combined

Then running the both servers with the /affinity 3FFC3FFC
7950x dual svr - 34.2 fps
5950x dual svr - 27.92 fps combined

Then using an affinity mask of FFFF0000 on 1 server and 0000FFFF on the other. So a dual 7700x and a dual 5700x. This was the fastest for 4k
7950x dual svr - 25.72 fps
5950x dual svr - 22.2 fps combined
Well this stinks, fastest for 4k , but slowest for lower res by a good bit

Finally running the 7950x with 4 servers as above
7950x quad server - 36.17 fps combined

So my take on this is basically what we already know. Unless you do something to knock down the core counts on the 16 core Ryzens, they are horrible at doing 4k encoding, but once you do, they scale back up quickly and fall in line with what you would expect as they are still faster than their 12 core brothers. The fact that a 7950x cut in half is faster than stock by 10% is really telling.

My best guess is that ffmpeg is the culprit here once the core count goes above 12 in 4k. When running the 16 cores at default max. when I watch the green and blue encoding graph at the bottom of the encoding window, it's the Decoder that is having the most issues and that cpu usage corresponds to ffmpeg in task manager. The green decoder is constantly bouncing up and down with many times there is no blue encoding going on. When encoding is going as it should with reduced threads, the decoding and encoding graph move across the graph in a very steady state, It's the difference between the Plains and the Swiss Alps. The only time I see it wildly bouncing is running all 16 cores maxed out on the 39,59,79 50x's.

[Atak_Snajpera]

How about setting ffmpeg.exe affinity to CCD0 and x265.exe to CCD1 manually in task manager?

[guest]

How about setting ffmpeg.exe affinity to CCD0 and x265.exe to CCD1 manually in task manager?

I just had a look at this, and it seems that RB would have to be doing something for ffmpeg & x265 to be present in Task Manager, to be able to set the Affinity, and then isn't there a good chance that once RB has finished, those settings will be lost ??

EDIT:- I just did a simple test with RipBot264.exe in Task Manager, and it indeed loses any settings applied, when app is closed.

Is there somewhere else that FFmpeg could be changed ??

Some option within RB, maybe ??

As x265 could be changed in its command line/instructions.

I did a bit of Googling earlier, and it definitely appears that this IS an issue with FFmpeg :(

[guest]

So I ran a whole slew of tests to try to answer Atak's question about turning the 7950x into a 7900x in the bios and see what happens when doing 4k encoding.

My best guess is that ffmpeg is the culprit here once the core count goes above 12 in 4k.

Wow...so we definitely have a far different approach to testing.

But in the big picture, "we're" only chasing small differences.

[Ryushin]

I should also point out that the threading issue is really related to high core counts and not just the 7950x. My Intel Xeon system passes 24 cores and 48 threads to my KVM VM. My tests in the past showed 14 avisynth threads with two encoding servers (24 x264/x265 threads) to be optimum for my setup on that system.

Atak: There is also a little bug in the Audio Description setting if I use the same audio track more than once. When using a 7.1 audio track, I've been setting the Audio1 Description to be something like "AAC 7.1" and using FFMPEG 7.1, then Audio2 I use xcopy stream along with a description like "Dolby TrueHD with Dolby Atmos 7.1". The bug is it will apply a single description to both Audio1 and Audio2.

Output from mediainfo:
Audio #1
ID : 2
Format : AAC LC
Format/Info : Advanced Audio Codec Low Complexity
Codec ID : A_AAC-2
Duration : 1 h 30 min
Bit rate : 641 kb/s
Channel(s) : 8 channels
Channel layout : C L R Ls Rs Lw Rw LFE
Sampling rate : 48.0 kHz
Frame rate : 46.875 FPS (1024 SPF)
Compression mode : Lossy
Stream size : 415 MiB (7%)
Title : DTS-HD (X11 X) 7.1
Language : English
Default : Yes
Forced : No

Audio #2
ID : 3
Format : DTS XLL X
Format/Info : Digital Theater Systems
Commercial name : DTS-HD Master Audio
Codec ID : A_DTS
Duration : 1 h 30 min
Bit rate mode : Variable
Bit rate : 4 445 kb/s
Channel(s) : 8 channels
Channel(s)_Original : Object Based
ChannelLayout_Original : Object Based
Sampling rate : 48.0 kHz
Frame rate : 93.750 FPS (512 SPF)
Bit depth : 24 bits
Stream size : 2.81 GiB (46%)
Title : DTS-HD (X11 X) 7.1
Language : English
Default : No
Forced : No

[Atak_Snajpera]

I should also point out that the threading issue is really related to high core counts and not just the 7950x. My Intel Xeon system passes 24 cores and 48 threads to my KVM VM. My tests in the past showed 14 avisynth threads with two encoding servers (24 x264/x265 threads) to be optimum for my setup on that system.

Atak: There is also a little bug in the Audio Description setting if I use the same audio track more than once. When using a 7.1 audio track, I've been setting the Audio1 Description to be something like "AAC 7.1" and using FFMPEG 7.1, then Audio2 I use xcopy stream along with a description like "Dolby TrueHD with Dolby Atmos 7.1". The bug is it will apply a single description to both Audio1 and Audio2.

Output from mediainfo:
Audio #1
ID : 2
Format : AAC LC
Format/Info : Advanced Audio Codec Low Complexity
Codec ID : A_AAC-2
Duration : 1 h 30 min
Bit rate : 641 kb/s
Channel(s) : 8 channels
Channel layout : C L R Ls Rs Lw Rw LFE
Sampling rate : 48.0 kHz
Frame rate : 46.875 FPS (1024 SPF)
Compression mode : Lossy
Stream size : 415 MiB (7%)
Title : DTS-HD (X11 X) 7.1
Language : English
Default : Yes
Forced : No

Audio #2
ID : 3
Format : DTS XLL X
Format/Info : Digital Theater Systems
Commercial name : DTS-HD Master Audio
Codec ID : A_DTS
Duration : 1 h 30 min
Bit rate mode : Variable
Bit rate : 4 445 kb/s
Channel(s) : 8 channels
Channel(s)_Original : Object Based
ChannelLayout_Original : Object Based
Sampling rate : 48.0 kHz
Frame rate : 93.750 FPS (512 SPF)
Bit depth : 24 bits
Stream size : 2.81 GiB (46%)
Title : DTS-HD (X11 X) 7.1
Language : English
Default : No
Forced : No


Thank you! Fixed via auto update.

[apol847]

@Atak_Snajpera: Any news regarding the source code for detectborders? Have you seen my private message?

[rlev11]

Wow...so we definitely have a far different approach to testing.

But in the big picture, "we're" only chasing small differences.

I agree, over an entire encode, are those few percentage points really making a big difference in total time.

I guess my goal is just to find an encoding server setting that seems to work best for ALL encoding jobs. There is still so many variables in play even with my testing to say that a particular server profile is best for a given resolution. Even saying a 7950x run as 2 7700x's is best for the Animal House (1:1.85 AR), doesn't mean that setting would be BEST for a 1:2.40 AR encode or a 1:1.78 AR encode. Add in maybe different degraining scripts might also make a difference. I just want to find a setting for the 16 core Ryzens (I have 5 of them)that runs well with whatever I throw at it even if it may not be the best setting for that particular encode. As of now it seems to me still that adding the avisynth prefetch set to 12 is a happy medium.

I mean right now I am able to encode 4k with 15 servers active with some aggressive smdegraining somewhere between 60 and 80 fps depending on the aspect ratio, so things aren't all that bad.

I haven't tried it, but i am wondering if we are hard limited to 16 encoding servers in distributed mode. I am 1 more upgrade away from pushing past that.

[guest]

I should also point out that the threading issue is really related to high core counts and not just the 7950x. My Intel Xeon system passes 24 cores and 48 threads to my KVM VM. My tests in the past showed 14 avisynth threads with two encoding servers (24 x264/x265 threads) to be optimum for my setup on that system.

So can I interpret this as being like this:-

/avisynth-prefetch-threads 14 /x264-threads 24 /x265-threads 24

It makes you wonder how Threadripper's & multi-core Xeon's would co-operate with this situation.

[guest]

I agree, over an entire encode, are those few percentage points really making a big difference in total time.

I guess my goal is just to find an encoding server setting that seems to work best for ALL encoding jobs. There is still so many variables in play even with my testing to say that a particular server profile is best for a given resolution. Even saying a 7950x run as 2 7700x's is best for the Animal House (1:1.85 AR), doesn't mean that setting would be BEST for a 1:2.40 AR encode or a 1:1.78 AR encode. Add in maybe different degraining scripts might also make a difference. I just want to find a setting for the 16 core Ryzens (I have 5 of them)that runs well with whatever I throw at it even if it may not be the best setting for that particular encode. As of now it seems to me still that adding the avisynth prefetch set to 12 is a happy medium.

I mean right now I am able to encode 4k with 15 servers active with some aggressive smdegraining somewhere between 60 and 80 fps depending on the aspect ratio, so things aren't all that bad.

I haven't tried it, but i am wondering if we are hard limited to 16 encoding servers in distributed mode. I am 1 more upgrade away from pushing past that.

Yeah, there are so many variables that can affect the encoding speeds, that a "happy medium" is probably the best result.

I would think that adding more DE servers would have to be up to Atak, as he is the ONLY one with the code.

I'm sure that if asked nicely, he might consider adding more DE servers, maybe up to 20, or 24, or maybe an option for the user to add or subtract how many are available, and displayed in a "resizable window".

I have more than enough machines to occupy more than 16 servers, but ttytt, most of them are too old & now underpowered to warrant their use, and power consumption, and if you have slow PC's "helping" they are just holding up the fast ones, and end up holding up the whole process. :(

Having said that, I would be interested to know what you've got going on there, with your 15 servers? (maybe a PM) sent, thanks.

[guest]

I have also removed ALL the info here, as it's also irrelevant, and besides, there's a "copy" of it on rlev11's post, below.

[Boulder]

When you figure out these slowdowns with high thread amounts, have you made sure that Avisynth is not running out of cache memory?

[Atak_Snajpera]

@Atak_Snajpera: Any news regarding the source code for detectborders? Have you seen my private message?

Sorry for delay. I completely forgot about that.
https://www.mediafire.com/file/r2hmt95asxynruz/DetectBorders_Delphi7_Source.7z/file

[Atak_Snajpera]

When you figure out these slowdowns with high thread amounts, have you made sure that Avisynth is not running out of cache memory?

Or changing SetCacheMode
http://avisynth.nl/index.php/Internal_functions#SetCacheMode

Guys add that at the top of your script and see if this fixes that performance drop on 7850 with 16 prefetch threads with active MDegrain/SMdegrain

SetCacheMode(1)

[guest]

Or changing SetCacheMode
http://avisynth.nl/index.php/Internal_functions#SetCacheMode

Guys add that at the top of your script and see if this fixes that performance drop on 7850 with 16 prefetch threads with active MDegrain/SMdegrain

SetCacheMode(1)

Be nice if this was the "magic" formula...will definitely test this tomorrow.

:goodpost:

EDIT:- What a shame it wasn't :( :(

[rlev11]

So using the exact same test file as the 7950X tests.

Tested a 5 minute 4K clip, using 1 x 6 minute chunk size, CQ 16, no filters !!
using this x265 Profile:-

--level 6.2 --profile main10 --hdr10 --hdr10-opt --output-depth 10 --ctu 64 --high-tier --vbv-bufsize 800000 --vbv-maxrate 800000 --asm avx512 (note:- the avx512 command is ignored by the 13900KF)

/avisynth-prefetch-threads 12 /x264-threads 16 /x265-threads 16 Time Taken 10:07 @ 11.88fps

/avisynth-prefetch-threads 14 /x264-threads 16 /x265-threads 16 Time Taken 10:04 @ 11.95fps

/avisynth-prefetch-threads 16 /x264-threads 16 /x265-threads 16 Time Taken 10:39 @ 11.29fps


but then I thought I'd try some Affinity tests (no prefetch) :-

/affinity 3FFC3FFC (which equates to 12 threads, but displays as 24 in the Encoder Server window) Time Taken 08:29 @ 14.16fps

/affinity 3FFFFFFC (which equates to 14 threads, but displays as 28 in the Encoder Server window) Time Taken 08:15 @ 14.57fps

/affinity FFFFFFFF (which equates to 16 threads, but displays as 32 in the Encoder Server window) Time Taken 08:04 @ 14.92fps (fastest)

then no special "instructions", (which equates to 16 threads, but displays as 32 in the Encoder Server window) Time Taken 08:06 @ 14.85fps

xxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx

So what I would now like to know is, if the 7950X was the Client machine, to the 13900KF being a Server, do the Encoder command line settings get transferred

from the Client to the Server, meaning that the above settings would revert back to :-

/avisynth-prefetch-threads 12 /x264-threads 16 /x265-threads 16.

OR

Does each PC use it's own particular Encoder settings ??

So it appears that Affinity has a very different influence on the 13900KF, over 16 core Ryzen's.

I guess the next "test" is a proper encoding job, with 2 or so DE Servers working.


so I set the client machine encoding servers in the ripbot264.ini by just using this:
Server1CommandLine=/port 1000 /priority normal /restart-if-no-progress /avisynth-prefetch-threads 12 /x264-threads 16 /x265-threads 16
Server2CommandLine=/port 2000 /priority normal /restart-if-no-progress /avisynth-prefetch-threads 12 /x264-threads 16 /x265-threads 16

All my encoding servers are set by a windows batch file that sits on the desktop and also a shortcut is added to the startup folder so when a server is booted, it loads automatically and is ready to go. I have this in the /bat file for the 16 core Ryzens:
encodingserver /port 1000 /priority normal /restart-if-no-progress /avisynth-prefetch-threads 12 /x264-threads 16 /x265-threads 16

and just this in the 12 and 10 core boxes:
encodingserver /restart-if-no-progress /priority normal

So as far as I know the client does not "push" its encoding server settings to the servers

[rlev11]

Or changing SetCacheMode
http://avisynth.nl/index.php/Internal_functions#SetCacheMode

Guys add that at the top of your script and see if this fixes that performance drop on 7850 with 16 prefetch threads with active MDegrain/SMdegrain

SetCacheMode(1)

I want to setup my somewhat controlled test that I did over this past weekend to really look at this which I will get to later in the week.

My initial "way too early" observation is that it does make a difference, but I am not sure it is the magic pill yet. On the 7950x and 5950x the utilization graph is a lot more stable across the board, not the constant wild swings in the decoder utilization. There are still some points during the middle of a chunk where the encoder drops out and the decoder takes a dive for 10-15 seconds. Then it goes back to a stabilized somewhat flat bar. Running at 12 core the bar is stable the whole way across the chunk. FPS are definitely better coming close to the 5900x numbers, but not as high as they should be.

One thing I did notice was I opened a second encoding window on the 7950x at full cores, and performance dropped a little bit combined vs single. When I did this on the 5950x, both encoding windows puked their guts out.

Will post back when I look more thoroughly at it later this week

[guest]

so I set the client machine encoding servers in the ripbot264.ini by just using this:
Server1CommandLine=/port 1000 /priority normal /restart-if-no-progress /avisynth-prefetch-threads 12 /x264-threads 16 /x265-threads 16
Server2CommandLine=/port 2000 /priority normal /restart-if-no-progress /avisynth-prefetch-threads 12 /x264-threads 16 /x265-threads 16

All my encoding servers are set by a windows batch file that sits on the desktop and also a shortcut is added to the startup folder so when a server is booted, it loads automatically and is ready to go. I have this in the /bat file for the 16 core Ryzens:
encodingserver /port 1000 /priority normal /restart-if-no-progress /avisynth-prefetch-threads 12 /x264-threads 16 /x265-threads 16

and just this in the 12 and 10 core boxes:
encodingserver /restart-if-no-progress /priority normal

So as far as I know the client does not "push" its encoding server settings to the servers

I know I should have known about this, having used RB long enough.

I guess I just wanted another user's confirmation.

Thanks

Now onto more testing..

[guest]

So I did a lot more testing earlier today, and posted a rather large page of info.

But then I started to do some "proper" encodes, and the whole process went out the window, it just wasn't relevant, so I deleted the post :(

You can test all you like, and maybe the "test" file isn't too complex, so it doesn't really reflect what you will encounter with any other encode.

I added SetCacheMode(1) to the script of the encode I was going to do, and tbh, I don't think it makes much, if any difference, but having said that, it might as well added to any "custom" script, just in case.

So all the info I have posted over the last couple of days if kinda moot. :angry:, and I might even delete it..we'll see.

Anyway, I did a 4K encode with some different settings, and the 13900KF was actually faster than the 7950X (for the first time), but then when I went onto the next job, the 13900KF "fell off a cliff", and the 7950X , "went to the moon".

So about all I can say is I am currently using these command line's :-

/avisynth-prefetch-threads 12 /x264-threads 16 /x265-threads 32

for the 7950X, & 3950X.

&

/avisynth-prefetch-threads 16 /x264-threads 16 /x265-threads 32

for the 13900KF.

I even tried this x265 command :-

--level 6.2 --profile main10 --hdr10 --hdr10-opt --output-depth 10 --ctu 64 --high-tier --vbv-bufsize 800000 --vbv-maxrate 800000 --asm avx512 --pools 32

Note:- as I haven't been doing any x264 stuff, that number hasn't changed, but it might.

[rlev11]

I ran through the same tests and setup as this past weekend except this time i added the "SetCacheMode(1)" to the top of the avisynth script.

What I can basically say is that it made little to no difference on any of the tests as before. If anything the 5950x's trended a little better and the 7950x trended a little worse, but all by very small percentages. This was for both the 4k and 1080p runs

It definitely did not take care of the issue with 4k encodes running full out on the 5950x and 7950x, but it was worth a shot trying it, thanks...

I did one last test on the 4k Animal House, this time not using any avisynth script or settings, just load it and go. This results in no decoding going on and just showing encoder cpu time. The 7950x running at all 16 cores was by far the fastest, followed by the 5950x at all 16 cores, and then I tried with the 7950x with 12 core using affinity mask and that was slower than the others which I expected.

This makes me convinced the issue has to be with the relationship between ffmpeg and it using the avisynth script as the input and with avisynth sending more information to ffmpeg than it can handle when the size of the stream gets to a certain point with 4k. The hose between them just isn't big enough and it may just mean that the hose can't get any larger.

[Boulder]

Be nice if this was the "magic" formula...will definitely test this tomorrow.

:goodpost:

EDIT:- What a shame it wasn't :( :(

Then check how much memory one Avisynth related process consumes. The thing is that by default, the amount of cache is restricted to 4GB for 64-bit Avisynth. When you set threads=x in Prefetch, Avisynth+ will then set frames=x*2 as prefetched frames by default. This can easily kill performance (by eating way too much memory + overhead is bigger) so it's better to set the amount of frames manually. For my 5950X, threads=32, frames=12 works best but with RB it's different if you launch multiple encodes on the same machine.

A 4K encode can easily take over 4GB of cache memory. Some scripts I've used, have taken almost 20GB :devil: You'll need SetMemoryMax to raise the amount if the default is not enough.

[Atak_Snajpera]

I ran through the same tests and setup as this past weekend except this time i added the "SetCacheMode(1)" to the top of the avisynth script.

What I can basically say is that it made little to no difference on any of the tests as before. If anything the 5950x's trended a little better and the 7950x trended a little worse, but all by very small percentages. This was for both the 4k and 1080p runs

It definitely did not take care of the issue with 4k encodes running full out on the 5950x and 7950x, but it was worth a shot trying it, thanks...

I did one last test on the 4k Animal House, this time not using any avisynth script or settings, just load it and go. This results in no decoding going on and just showing encoder cpu time. The 7950x running at all 16 cores was by far the fastest, followed by the 5950x at all 16 cores, and then I tried with the 7950x with 12 core using affinity mask and that was slower than the others which I expected.

This makes me convinced the issue has to be with the relationship between ffmpeg and it using the avisynth script as the input and with avisynth sending more information to ffmpeg than it can handle when the size of the stream gets to a certain point with 4k. The hose between them just isn't big enough and it may just mean that the hose can't get any larger.

Try now with
SetMemoryMax(16384)

[rlev11]

Tried earlier with SetMemoryMax(8192) along with the cache setting and saw no difference. My only question with that is where would I see the memory being used by avisynth. It does not show up in task manager anywhere that I can see. ffmpeg with and without the 8192 setting was at around 5.5 gig memory used in task Manager. I will try setting it to 16 tomorrow sometime. In the middle of something (work) at the moment. Only problem i could see if 16gig works is some of my servers only have 16 in them (the majority are 32) and that setting would to push out to all the servers since it is in the avisynth script (but we would be closer to solving the mystery).

Also Atak, can we and what would be the correct way to set the frames for avisynth referenced by Boulder above for some testing. Would it be "/avisynth-prefetch-threads 16 frames 16" or "/avisynth-prefetch-threads 16 /avisynth-prefetch-frames 16"

[guest]

Tried earlier with SetMemoryMax(8192) along with the cache setting and saw no difference. My only question with that is where would I see the memory being used by avisynth. It does not show up in task manager anywhere that I can see. ffmpeg with and without the 8192 setting was at around 5.5 gig memory used in task Manager. I will try setting it to 16 tomorrow sometime. In the middle of something (work) at the moment. Only problem i could see if 16gig works is some of my servers only have 16 in them (the majority are 32) and that setting would to push out to all the servers since it is in the avisynth script (but we would be closer to solving the mystery).

Also Atak, can we and what would be the correct way to set the frames for avisynth referenced by Boulder above for some testing. Would it be "/avisynth-prefetch-threads 16 frames 16" or "/avisynth-prefetch-threads 16 /avisynth-prefetch-frames 16"

I have tried a couple of attempts of the "frames" setting, with no success.

Am currently trying SetMemoryMax(16384), & SetMemoryMax(24576), and this, SetMemoryMax(32768), failed, I only have 32Gb of system memory.

It's starting to be a PITA :mad:

EDIT:- So how I'm "testing" today is actually doing a "proper" encode, and stopping after each completed chunk, for adjustments, and I have tried different SetMemoryMax(), and it doesn't make any significant difference, sadly.
I think that it might be best practice to just run one DE Server per PC (which sounds like a waste), but the single chunk encoding speeds a really up there :), I am using a 2 minute chunk, but it might be worth using 3 or 4 minute chunks, so the CPU has a chance to "get its teeth into it", and it would also "trick" us into thinking that it's not a big job, as there are way less chunks in the list.