RipBot264 v1.18.3 - Simple and easy to use GUI -> IPOD . PSP . CONSOLES . BLURAY

[rlev11]

Yea I'm not completely sold on ECO Mode. It was just a "Let's see what happens" and I was really surprised that the performance was as good as it was. It something I might enable on the fly if I que up a bunch of stuff to do overnight and don't really care when it gets done.

[Ryushin]

Originally Posted by TDS

It's also a shame that some of us spend our hard-earned money on "top of the line" hardware (eg:- 7950X's, and 13900K's), only to find they use SO much power, and produce SO much heat, that "we" go to great lengths to "hobble" them, to make them more acceptable...it's almost pointless, really.

Those encoding time's for "Puss In Boot's", that's encoding the movie in less time that it takes to watch it !!!

Are you using any filters, or is it only CQ 16 ??

And as for power usage, that fortunately isn't that much of an issue for me, as I only encode when the Solar Panels are generating enough, and I don't do any "overnight" encoding, been there, done that, was very disappointing, most of the time

Not using any filters for those tests, just CQ16. I can run the same tests but I think the results will be similar.

I think both AMD and Intel are chasing every last bit of performance they can to one up each other. The amount of heat/watts consumed by the new Intel 13900K is staggering (yea, that is a furnace):
https://www.youtube.com/watch?v=P40gp_DJk5E&t=364s
The 13900K seems very bursty in the way it was designed and it throttles back in only a few seconds. The 7950x seems to go to it's limit and hold there. I'm curious, since you have both CPUs, you should run a test and let us know which CPU wins (regardless of power used). I think you'll need to disable the GPU portion though so it runs all CPU.

So even though the chips can eek out more performance by pushing them to their max, I don't think for real world use it's necessary. Even for us that torture the CPUs for encoding, underlocking them ever so slightly produces great performance. And if I want to get 2.3% more performance for 50% more power I can.

[TDS]

Originally Posted by Ryushin

Not using any filters for those tests, just CQ16. I can run the same tests but I think the results will be similar.

I think both AMD and Intel are chasing every last bit of performance they can to one up each other. The amount of heat/watts consumed by the new Intel 13900K is staggering (yea, that is a furnace):
https://www.youtube.com/watch?v=P40gp_DJk5E&t=364s
The 13900K seems very bursty in the way it was designed and it throttles back in only a few seconds. The 7950x seems to go to it's limit and hold there. I'm curious, since you have both CPUs, you should run a test and let us know which CPU wins (regardless of power used). I think you'll need to disable the GPU portion though so it runs all CPU.

So even though the chips can eek out more performance by pushing them to their max, I don't think for real world use it's necessary. Even for us that torture the CPUs for encoding, underlocking them ever so slightly produces great performance. And if I want to get 2.3% more performance for 50% more power I can.

OK, well those times make more sense, now that you've confirmed your settings.

I wonder if when using a filter, you might get different results ??

The 13900KF, is the non GPU variant

I'm going to try an older x265 build that's supposed to be optimised for AMD & Intel chips...not sure if it's going to yield better results.

And you're correct, that the Ryzen seems to get to it's limit, and stay's there, and the 13900 does indeed throttle back and forth, sadly.

But the Ryzen has the advantage, most definitely.

[Atak_Snajpera]

Originally Posted by rlev11

The only time in my experience that you have to worry about knocking down cpu cores or avisynth threads is when you start getting into doing 4k encoding and specifically 4k encodes that are full screen aspect ratio (1.78:1 or even 1.85:1). There is something with encoding all those extra bits that I have found that the 16 core Ryzens really have an issue with those. I can't speak to the latest Intel i-9's as I don't have any of those. Not sure how the whole P and E cores effect encoding 4k, you may have to just do some testing.

When doing those 4k encodes with everything set full out on a 3950/5950/7950, The encoding speed takes a nose-dive to point where they are about 1/2 the speed of a 12 core 3900x or 5900x. Something gets overloaded with all those threads and they just can't handle it. This is where using either affinity mask or avi-synth thread setting knocking them down to only use 12 cores or threads is necessary. Even with cores or threads turned off, they will still out-perform their 12 core brothers.

I posted some quick tests a while ago that showed me that once I set the avisynth prefetch to 12, that it did not slow down or speed up doing regular 1080 or 720 encoding, but made the 4k stuff run as they should which is why I just use that setting now for everything.

I wonder what happens if you downgrade your 7950 to 7700 by disabling one CCD in BIOS. Will you still experience the same problem with 4k working terrible with 8 threads (or 16)?

[TDS]

Originally Posted by Atak_Snajpera

I wonder what happens if you downgrade your 7950 to 7700 by disabling one CCD in BIOS. Will you still experience the same problem with 4k working terrible with 8 threads (or 16)?

That is an unusual suggestion, it's cutting the CPU in half

If it turned out similar to do what disabling all the "E" Cores on a 13900KF, then it's just SLOW !!

We buy these CPU's for the power, not to "cut them in half" !!

As rlev11, Ryushin & myself have determined, the sweet spot for the 7950X, is this (with about 95% of encodes) :-

/avisynth-prefetch-threads 12 /x264-threads 16 /x265-threads 16

As for the 13900KF, still not too sure what it likes, I might need to try the "Affinity" command line.

I am going to do some tests with an older x265 build, that is supposed to be optimised for different CPU's, so I will post some results, soon

[TDS]

These results are from Atak's x265 Benchmark app :-

With updated x265 builds.

with default included x265 - encoded 2500 frames in 20.75s (120.51 fps), 7025.74 kbps, Avg QP:37.21

x265 r74 - encoded 2500 frames in 19.89s (125.71 fps), 10890.80 kbps, Avg QP:35.84

x265 r83 - encoded 2500 frames in 19.83s (126.07 fps), 10890.80 kbps, Avg QP:35.84

x265 r85 - encoded 2500 frames in 19.72s (126.76 fps), 10890.80 kbps, Avg QP:35.84

x265 v3.5+67-aMod-gcc12.2.1 - encoded 2500 frames in 19.77s (126.46 fps), 10891.97 kbps, Avg QP:35.84

x265-x64-v3.5+67-aMod-gcc12.2.1 failed


x265-3.5+84-5d8f209_vs2022 - encoded 2500 frames in 19.78s (126.39 fps), 10890.80 kbps, Avg QP:35.84

x265-3.5+84-5d8f209_vs2022-AVX2 - encoded 2500 frames in 19.84s (126.01 fps), 10890.80 kbps, Avg QP:35.84

x265-3.5+84-5d8f209_gcc122-AVX2 - encoded 2500 frames in 19.67s (127.07 fps), 10891.97 kbps, Avg QP:35.84

x265-3.5+84-5d8f209_gcc122 - encoded 2500 frames in 19.62s (127.40 fps), 10890.80 kbps, Avg QP:35.84 fastest

Patman builds failed this test

------------------------------------------
I have removed most of this post, as it was no longer relevant, after today's testing (21-01-23)

[Ryushin]

Originally Posted by TDS

OK, well those times make more sense, now that you've confirmed your settings.

I wonder if when using a filter, you might get different results ??

Ran the tests using SMDegrain Hard. The underclocked took three minutes longer, which was 1.4% slower. So I'll take that for 50% of the power savings.

Power Tests:
Source: Puss In Boots 4K - 1h:30m:35s
AIO 420mm Cooler
CQ16
SMDegrain Hard: SMDegrain(video,tr=8,thSAD=800,thSADC=400,contrasharp=true,prefilter=2,refinemotion=true)

7950x Default Frequencies: 3h:26m:08s
7950x Underclocked 4.9GHz at 1.08v: 3h:29m:50s

[rlev11]

Originally Posted by Atak_Snajpera

I wonder what happens if you downgrade your 7950 to 7700 by disabling one CCD in BIOS. Will you still experience the same problem with 4k working terrible with 8 threads (or 16)?

I sure wouldn't want to do that as part of my regular workflow, but sometime this weekend I'll give it a try and see what happens for a test. Who knows, the results may be surprising.

It would be nice if we all could figure out exactly what is the hangup when doing 4k once it appears you start going over 12 cores. There must be some program buffer that just gets overloaded with all that extra information contained in each frame

[Atak_Snajpera]

Originally Posted by rlev11

I sure wouldn't want to do that as part of my regular workflow, but sometime this weekend I'll give it a try and see what happens for a test. Who knows, the results may be surprising.

It would be nice if we all could figure out exactly what is the hangup when doing 4k once it appears you start going over 12 cores. There must be some program buffer that just gets overloaded with all that extra information contained in each frame

Who knows maybe communication between CCDs is the problem in 7950? That's why I asked you if you could check if single CCD also shows this behavior. Intel despite having two different CPU architectures under one roof is immune to excess number of threads.

[TDS]

Originally Posted by Atak_Snajpera

Who knows maybe communication between CCDs is the problem in 7950? That's why I asked you if you could check if single CCD also shows this behavior. Intel despite having two different CPU architectures under one roof is immune to excess number of threads.

I thought I'd try Atak's suggestion, I am yet to run a "proper" encode, but in the mean time, I ran his x265 benchmark app, and comparing it to here :-

https://forum.doom9.org/showthread.p...81#post1980881

I think you will see a HUGE difference

AMD Ryzen 9 7950X 16-Core  @ 4.5GHz ( 8C / 16T )
 
y4m  [info]: 1920x1080 fps 50/1 i420p8 sar 1:1 unknown frame count
raw  [info]: output file: NUL
x265 [info]: HEVC encoder version 3.5+85-c2e8e8d13
x265 [info]: build info [Windows][GCC 12.2.0][64 bit] 8bit+10bit+12bit
x265 [info]: using cpu capabilities: MMX2 SSE2Fast LZCNT SSSE3 SSE4.2 AVX FMA3 BMI2 AVX2
x265 [info]: Main Still Picture profile, Level-4.1 (Main tier)
x265 [info]: Thread pool created using 16 threads
x265 [info]: Slices                              : 1
x265 [info]: frame threads / pool features       : 4 / wpp(17 rows)
x265 [info]: Coding QT: max CU size, min CU size : 64 / 8
x265 [info]: Residual QT: max TU size, max depth : 32 / 1 inter / 1 intra
x265 [info]: ME / range / subpel / merge         : hex / 57 / 2 / 3
x265 [info]: Keyframe min / max / scenecut / bias  : 50 / 500 / 40 / 5.00 
x265 [info]: Lookahead / bframes / badapt        : 20 / 4 / 2
x265 [info]: b-pyramid / weightp / weightb       : 1 / 1 / 0
x265 [info]: References / ref-limit  cu / depth  : 3 / off / on
x265 [info]: AQ: mode / str / qg-size / cu-tree  : 2 / 1.0 / 32 / 1
x265 [info]: Rate Control / qCompress            : CRF-28.0 / 0.60
x265 [info]: tools: rd=3 psy-rd=2.00 early-skip rskip mode=1 signhide tmvp
x265 [info]: tools: b-intra strong-intra-smoothing lslices=6 deblock sao

encoded 2500 frames in 35.68s (70.07 fps), 10890.80 kbps, Avg QP:35.84

I will run a short encode, and post those results. Not looking good, tho

OK, have run a test (same as yesterday, above), and using this :-


Tested a 5 minute 4K clip, using 1 x 6 minute chunk size, CQ 16, no filters !!

Time taken FPS

x265 v3.5+85.1 Mod by Patman clang-AVX2 with AVX512 9:30 12.66 (yesterday, using this command-- /avisynth-prefetch-threads 12 /x264-threads 16 /x265-threads 16)

x265 v3.5+85.1 Mod by Patman clang-AVX2 with AVX512 13:14 9.08 (today, with ONLY half the 7950X enabled, using this command-- /avisynth-prefetch-threads 12 /x264-threads 16 /x265-threads 16)

and last but not least, with the latest x265 build, released yesterday:-

x265 v3.5+87 with AVX512 13:50 8.69 (today, with ONLY half the 7950X enabled, using this command-- /avisynth-prefetch-threads 12 /x264-threads 16 /x265-threads 16)

Oh, and btw, the CPU was running in the high 80°C's, even with half disabled.

So the next step is doing this on the 13900KF...

[rlev11]

So I ran a whole slew of tests to try to answer Atak's question about turning the 7950x into a 7900x in the bios and see what happens when doing 4k encoding. I also did some runs doing a 1080p encode for comparisons to try to figure out if there was one setting that worked best for both.
For the 4k encode, I did Animal House starting 30 minutes in and just used the ending fps of the first chunk for each encoding server. Ran the same smdegrain heavy grain script. This way for each run, the same encoding server did the same chunk each time for control purposes. As a reference, a 12 core 5900x averaged 5.28 fps for all the runs.

First up is running the 7950x and 5950x servers with no extra settings, so they were running full out max cores and threads.
7950x single server - 4.5 fps
5950x single server - 3.95 fps
5950x dual svr - 6.52 fps combined
7950x with CCD1 disabled in bios, so running as a 7700x - 4.97fps, so in essence a 7700x is 10 percent faster that a 7950x

Next up was running the 7950x and 5950x with: /avisynth-prefetch-threads 12 /x264-threads 16 /x265-threads 16
7950x Dual svr - 8.27 fps combined
7950x single server - 6.95 fps
5950x single server - 5.52 fps
5950x dual svr - 6.89 fps combined

Next up was running the 7950x and 5950x with: /affinity 3FFC3FFC for all encoding servers, essentially running them as 12 core cpus
7950x Dual svr - 8.41 fps combined
7950x single server - 7.06 fps
5950x single server - 5.39 fps
5950x dual svr - 6.99 fps combined

Then I thought lets run the 5950x and 7950x as 2 separate 8 core machines using an affinity mask of FFFF0000 on 1 server and 0000FFFF on the other. So a dual 7700x and a dual 5700x
7950x Dual svr - 9.73 fps combined
5950x dual svr - 7.14 fps combined

Finally I tried on the 7950x running as two 7900x like above, but running 2 server with each affinity mask, so 4 total servers running
7950x quad server - 9.16 fps combined

So this is great, running especially the 7950x as 2 separate 7900x showed a significant improvement, but how does that work when doing 1080p stuff. So I re-ran the tests using where eagles dare using the same heavy smdegrain script as above. the 5900x averaged out at 19.75 fps through all the runs.

First up is running the 7950x and 5950x servers with no extra settings, so they were running full out max cores and threads.
7950x dual svr - 35.71 fps
5950x single server - 19.43 fps
5950x dual svr - 29.24 fps combined

Then running the prefetch 12 setting
7950x dual svr - 35.48 fps
5950x dual svr - 29.11 fps combined

Then running the both servers with the /affinity 3FFC3FFC
7950x dual svr - 34.2 fps
5950x dual svr - 27.92 fps combined

Then using an affinity mask of FFFF0000 on 1 server and 0000FFFF on the other. So a dual 7700x and a dual 5700x. This was the fastest for 4k
7950x dual svr - 25.72 fps
5950x dual svr - 22.2 fps combined
Well this stinks, fastest for 4k , but slowest for lower res by a good bit

Finally running the 7950x with 4 servers as above
7950x quad server - 36.17 fps combined

So my take on this is basically what we already know. Unless you do something to knock down the core counts on the 16 core Ryzens, they are horrible at doing 4k encoding, but once you do, they scale back up quickly and fall in line with what you would expect as they are still faster than their 12 core brothers. The fact that a 7950x cut in half is faster than stock by 10% is really telling.

My best guess is that ffmpeg is the culprit here once the core count goes above 12 in 4k. When running the 16 cores at default max. when I watch the green and blue encoding graph at the bottom of the encoding window, it's the Decoder that is having the most issues and that cpu usage corresponds to ffmpeg in task manager. The green decoder is constantly bouncing up and down with many times there is no blue encoding going on. When encoding is going as it should with reduced threads, the decoding and encoding graph move across the graph in a very steady state, It's the difference between the Plains and the Swiss Alps. The only time I see it wildly bouncing is running all 16 cores maxed out on the 39,59,79 50x's.

[Atak_Snajpera]

How about setting ffmpeg.exe affinity to CCD0 and x265.exe to CCD1 manually in task manager?

[TDS]

Originally Posted by Atak_Snajpera

How about setting ffmpeg.exe affinity to CCD0 and x265.exe to CCD1 manually in task manager?

I just had a look at this, and it seems that RB would have to be doing something for ffmpeg & x265 to be present in Task Manager, to be able to set the Affinity, and then isn't there a good chance that once RB has finished, those settings will be lost ??

EDIT:- I just did a simple test with RipBot264.exe in Task Manager, and it indeed loses any settings applied, when app is closed.

Is there somewhere else that FFmpeg could be changed ??

Some option within RB, maybe ??

As x265 could be changed in its command line/instructions.

I did a bit of Googling earlier, and it definitely appears that this IS an issue with FFmpeg

[TDS]

Originally Posted by rlev11

So I ran a whole slew of tests to try to answer Atak's question about turning the 7950x into a 7900x in the bios and see what happens when doing 4k encoding.

My best guess is that ffmpeg is the culprit here once the core count goes above 12 in 4k.

Wow...so we definitely have a far different approach to testing.

But in the big picture, "we're" only chasing small differences.

[Ryushin]

I should also point out that the threading issue is really related to high core counts and not just the 7950x. My Intel Xeon system passes 24 cores and 48 threads to my KVM VM. My tests in the past showed 14 avisynth threads with two encoding servers (24 x264/x265 threads) to be optimum for my setup on that system.

Atak: There is also a little bug in the Audio Description setting if I use the same audio track more than once. When using a 7.1 audio track, I've been setting the Audio1 Description to be something like "AAC 7.1" and using FFMPEG 7.1, then Audio2 I use xcopy stream along with a description like "Dolby TrueHD with Dolby Atmos 7.1". The bug is it will apply a single description to both Audio1 and Audio2.

Output from mediainfo:

Audio #1
ID                                       : 2
Format                                   : AAC LC
Format/Info                              : Advanced Audio Codec Low Complexity
Codec ID                                 : A_AAC-2
Duration                                 : 1 h 30 min
Bit rate                                 : 641 kb/s
Channel(s)                               : 8 channels
Channel layout                           : C L R Ls Rs Lw Rw LFE
Sampling rate                            : 48.0 kHz
Frame rate                               : 46.875 FPS (1024 SPF)
Compression mode                         : Lossy
Stream size                              : 415 MiB (7%)
Title                                    : DTS-HD (X11 X) 7.1
Language                                 : English
Default                                  : Yes
Forced                                   : No

Audio #2
ID                                       : 3
Format                                   : DTS XLL X
Format/Info                              : Digital Theater Systems
Commercial name                          : DTS-HD Master Audio
Codec ID                                 : A_DTS
Duration                                 : 1 h 30 min
Bit rate mode                            : Variable
Bit rate                                 : 4 445 kb/s
Channel(s)                               : 8 channels
Channel(s)_Original                      : Object Based
ChannelLayout_Original                   : Object Based
Sampling rate                            : 48.0 kHz
Frame rate                               : 93.750 FPS (512 SPF)
Bit depth                                : 24 bits
Stream size                              : 2.81 GiB (46%)
Title                                    : DTS-HD (X11 X) 7.1
Language                                 : English
Default                                  : No
Forced                                   : No

[Atak_Snajpera]

Originally Posted by Ryushin

I should also point out that the threading issue is really related to high core counts and not just the 7950x. My Intel Xeon system passes 24 cores and 48 threads to my KVM VM. My tests in the past showed 14 avisynth threads with two encoding servers (24 x264/x265 threads) to be optimum for my setup on that system.

Atak: There is also a little bug in the Audio Description setting if I use the same audio track more than once. When using a 7.1 audio track, I've been setting the Audio1 Description to be something like "AAC 7.1" and using FFMPEG 7.1, then Audio2 I use xcopy stream along with a description like "Dolby TrueHD with Dolby Atmos 7.1". The bug is it will apply a single description to both Audio1 and Audio2.

Output from mediainfo:

Audio #1
ID                                       : 2
Format                                   : AAC LC
Format/Info                              : Advanced Audio Codec Low Complexity
Codec ID                                 : A_AAC-2
Duration                                 : 1 h 30 min
Bit rate                                 : 641 kb/s
Channel(s)                               : 8 channels
Channel layout                           : C L R Ls Rs Lw Rw LFE
Sampling rate                            : 48.0 kHz
Frame rate                               : 46.875 FPS (1024 SPF)
Compression mode                         : Lossy
Stream size                              : 415 MiB (7%)
Title                                    : DTS-HD (X11 X) 7.1
Language                                 : English
Default                                  : Yes
Forced                                   : No

Audio #2
ID                                       : 3
Format                                   : DTS XLL X
Format/Info                              : Digital Theater Systems
Commercial name                          : DTS-HD Master Audio
Codec ID                                 : A_DTS
Duration                                 : 1 h 30 min
Bit rate mode                            : Variable
Bit rate                                 : 4 445 kb/s
Channel(s)                               : 8 channels
Channel(s)_Original                      : Object Based
ChannelLayout_Original                   : Object Based
Sampling rate                            : 48.0 kHz
Frame rate                               : 93.750 FPS (512 SPF)
Bit depth                                : 24 bits
Stream size                              : 2.81 GiB (46%)
Title                                    : DTS-HD (X11 X) 7.1
Language                                 : English
Default                                  : No
Forced                                   : No

Thank you! Fixed via auto update.

[apol847]

@Atak_Snajpera: Any news regarding the source code for detectborders? Have you seen my private message?

[rlev11]

Originally Posted by TDS

Wow...so we definitely have a far different approach to testing.

But in the big picture, "we're" only chasing small differences.

I agree, over an entire encode, are those few percentage points really making a big difference in total time.

I guess my goal is just to find an encoding server setting that seems to work best for ALL encoding jobs. There is still so many variables in play even with my testing to say that a particular server profile is best for a given resolution. Even saying a 7950x run as 2 7700x's is best for the Animal House (1:1.85 AR), doesn't mean that setting would be BEST for a 1:2.40 AR encode or a 1:1.78 AR encode. Add in maybe different degraining scripts might also make a difference. I just want to find a setting for the 16 core Ryzens (I have 5 of them)that runs well with whatever I throw at it even if it may not be the best setting for that particular encode. As of now it seems to me still that adding the avisynth prefetch set to 12 is a happy medium.

I mean right now I am able to encode 4k with 15 servers active with some aggressive smdegraining somewhere between 60 and 80 fps depending on the aspect ratio, so things aren't all that bad.

I haven't tried it, but i am wondering if we are hard limited to 16 encoding servers in distributed mode. I am 1 more upgrade away from pushing past that.

[TDS]

Originally Posted by Ryushin

I should also point out that the threading issue is really related to high core counts and not just the 7950x. My Intel Xeon system passes 24 cores and 48 threads to my KVM VM. My tests in the past showed 14 avisynth threads with two encoding servers (24 x264/x265 threads) to be optimum for my setup on that system.

So can I interpret this as being like this:-

/avisynth-prefetch-threads 14 /x264-threads 24 /x265-threads 24

It makes you wonder how Threadripper's & multi-core Xeon's would co-operate with this situation.

[TDS]

Originally Posted by rlev11

I agree, over an entire encode, are those few percentage points really making a big difference in total time.

I guess my goal is just to find an encoding server setting that seems to work best for ALL encoding jobs. There is still so many variables in play even with my testing to say that a particular server profile is best for a given resolution. Even saying a 7950x run as 2 7700x's is best for the Animal House (1:1.85 AR), doesn't mean that setting would be BEST for a 1:2.40 AR encode or a 1:1.78 AR encode. Add in maybe different degraining scripts might also make a difference. I just want to find a setting for the 16 core Ryzens (I have 5 of them)that runs well with whatever I throw at it even if it may not be the best setting for that particular encode. As of now it seems to me still that adding the avisynth prefetch set to 12 is a happy medium.

I mean right now I am able to encode 4k with 15 servers active with some aggressive smdegraining somewhere between 60 and 80 fps depending on the aspect ratio, so things aren't all that bad.

I haven't tried it, but i am wondering if we are hard limited to 16 encoding servers in distributed mode. I am 1 more upgrade away from pushing past that.

Yeah, there are so many variables that can affect the encoding speeds, that a "happy medium" is probably the best result.

I would think that adding more DE servers would have to be up to Atak, as he is the ONLY one with the code.

I'm sure that if asked nicely, he might consider adding more DE servers, maybe up to 20, or 24, or maybe an option for the user to add or subtract how many are available, and displayed in a "resizable window".

I have more than enough machines to occupy more than 16 servers, but ttytt, most of them are too old & now underpowered to warrant their use, and power consumption, and if you have slow PC's "helping" they are just holding up the fast ones, and end up holding up the whole process.

Having said that, I would be interested to know what you've got going on there, with your 15 servers? (maybe a PM) sent, thanks.